nihilist/tor
1
0
Fork 0
mirror of https://gitlab.torproject.org/tpo/core/tor.git synced 2024-11-13 14:43:46 +01:00
Code Issues Packages Projects Releases Wiki Activity
24,874 Commits 53 Branches 630 Tags 125 MiB
2ca827104d
Commit Graph

40 Commits

Author SHA1 Message Date
Nick Mathewson
2ca827104d Report deleted consensuses as NOT_FOUND rather than AVAILABLE 
This bug happened because of a bogus pointer check in
consdiffmgr_find_consensus(), not in any released Tor.

Fixes CID 1409670.  Good catch, Coverity!
 2017-05-16 10:44:24 -04:00
Nick Mathewson
460b923026 fix a memory leak 2017-05-15 18:13:20 -04:00
Nick Mathewson
afef059795 Merge remote-tracking branch 'public/prop140_aftermath_cfg' 2017-05-15 17:26:47 -04:00
Nick Mathewson
7591518d16 Copy valid/fresh-until and signatories values into diffs. 2017-05-15 17:21:55 -04:00
Nick Mathewson
fd1190581d Store fresh/valid-until and signatories values on all consensus objects. 2017-05-15 17:21:55 -04:00
Nick Mathewson
dcc533fb13 Implement functions to expose valid/fresh-until and voters 
These still won't do anything till I get the values to be filled in.

Also, I changed the API a little (with corresponding changes in
directory.c) to match things that it's easier to store.
 2017-05-15 17:21:55 -04:00
Nick Mathewson
2f06345db3 Move stub accessor functions a level higher, to consdiffmgr 2017-05-15 17:21:55 -04:00
Alexander Færøy
64116ab97f
Fix tautological constant out-of-range comparison warnings. 2017-05-12 17:59:29 +02:00
Alexander Færøy
a1e8ef0076
Fix DoubleNL warning from make check-spaces. 2017-05-12 17:57:11 +02:00
Nick Mathewson
db370bb8a8
Test fix: expect old consensuses to be deleted if not deflate-compressed 2017-05-12 17:45:44 +02:00
Nick Mathewson
30dfb36148
consdiffmgr: Reload latest consensus entries on start. 2017-05-12 17:45:33 +02:00
Nick Mathewson
8100305e71
consdiffmgr: expose cached consensuses 2017-05-12 17:45:24 +02:00
Nick Mathewson
7b0dcf5c4a
Cleanup logic: only retain zlib-compressed consensuses 
Now that we're making a bunch of these with consdiffmgr, we should
throw out all but one when we get a newer consensus.
 2017-05-12 17:45:15 +02:00
Nick Mathewson
151cd121a2
consdiffmgr: compress incoming consensuses in the background 
Also, compress them in several ways.

This breaks the unit tests; subsequent commits will make them pass
again.
 2017-05-12 17:45:07 +02:00
Nick Mathewson
6da31ec484
consdiffmgr: Extract the code for compressing and storing 
We're going to use this for consensuses too.
 2017-05-12 17:44:55 +02:00
Nick Mathewson
c276c10d3b Turn consdiffmgr.cache_max_age_hours into a parameter 2017-05-05 09:11:06 -04:00
Nick Mathewson
6beb7028d8 Do not BUG on missing sha3-as-signed field 
This can happen if you've been running an earlier alpha on your
relay.  Instead, just ignore the entry.
 2017-05-04 08:50:21 -04:00
Nick Mathewson
3af9704e45 bug#22143/prop#140: in consdiffmgr, store and use digest-as-signed 
We need to index diffs by the digest-as-signed of their source
consensus, so that we can find them even from consensuses whose
signatures are encoded differently.
 2017-05-04 08:36:50 -04:00
Nick Mathewson
fd437f2a02 Remove special-casing for NO_METHOD in consdiffmgr.c 2017-05-02 08:37:57 -04:00
Nick Mathewson
b8f7488e94 Fix a brazen memleak in consdiffmgr_add_consensus() 2017-04-28 15:41:52 -04:00
Nick Mathewson
00a12337ff Merge branches 'consdiffmgr_orig_squashed' and 'actually_compute_diffs_squashed' 2017-04-27 21:43:06 -04:00
Nick Mathewson
1e1581a24e Pre-compress consensus diffs with supported consensus methods. 2017-04-27 21:40:46 -04:00
Nick Mathewson
a1172b6774 Store archived consensuses compressed on disk. 
I'm just using gzip compression now, for performance.
 2017-04-27 21:40:46 -04:00
Nick Mathewson
7a0964279f Functionality to ensure there is space to add files to cache. 2017-04-27 21:40:13 -04:00
Nick Mathewson
920475f293 New force-delete option on consensus_cache_delete_pending() 
If we're out of file space in the storage directory, we'll need to
get rid of old files fast.
 2017-04-27 21:40:13 -04:00
Nick Mathewson
466e914088 Lower the file limit in consdiffmgr, to support seccomp2 2017-04-27 21:40:13 -04:00
Nick Mathewson
fba8d7b222 Initialize consdiffmgr when running as (or becoming) a server. 2017-04-27 21:40:12 -04:00
Sebastian Hahn
ee478bdf38 Fix coverity cid 1405510 
This is a false positive, but let's appease coverity.
 2017-04-26 08:45:38 +02:00
Nick Mathewson
24f7059704 Configure sandbox using consdiffmgr; free cdm on exit. 2017-04-25 19:52:34 -04:00
Nick Mathewson
9e081a44a9 Teach consdiffmgr to remember two digests: one compressed, one not. 2017-04-25 16:49:47 -04:00
Nick Mathewson
6cc21aa89c consdiffmgr: add tests for cdm_entry_get_sha3_value 2017-04-24 11:01:40 -04:00
Nick Mathewson
43d683e0ad Fix reference leak & handle leak in consensus_diff_worker_replyfn 
Found by previous test.
 2017-04-24 11:01:40 -04:00
Nick Mathewson
7ca86b9cd6 Add a hashtable to consdiffmgr to keep track of diff status 
In several places in the old code, we had problems that only an
in-memory index of diff status could solve, including:
   * Remembering which diffs were in-progress, so that we didn't
     re-launch them.
   * Remembering which diffs had failed, so that we didn't try to
     recompute them over and over.
   * Having a fast way to look up the diff from a given consensus to
     the latest consensus of a given flavor.

This patch adds a hashtable mapping from (flavor, source diff), to
solve the problem.  It maps to a cache entry handle, rather than to
a cache entry directly, so that it doesn't affect the reference
counts of the cache entries, and so that we don't otherwise need to
worry about lifetime management.
 2017-04-24 11:01:40 -04:00
Nick Mathewson
69a212ff3d Consdiffmgr: extract "get a sha3 digest" function. 
I'll be using this a lot in the hashtable tweaks here.
 2017-04-24 11:01:40 -04:00
Nick Mathewson
655f1c8e01 consdiffmgr: function to re-validate stored sha3 digests at startup 2017-04-24 11:00:28 -04:00
Nick Mathewson
6c86e63029 Consdiffmgr: use aggressive-release flag on consensuses 
This conscache flag tells conscache that it should munmap the
document as soon as reasonably possible, since its usage pattern is
expected to not have a lot of time-locality.
 2017-04-24 10:59:25 -04:00
Nick Mathewson
35f6b678ab Test the easiest cases of consdiffmgr_cleanup. 
One more to go: deleting the old diffs.
 2017-04-24 10:59:25 -04:00
Nick Mathewson
1fade37287 consdiffmgr non-test: check for initialization failure 
Unfortunately, this test doesn't work, so I've left it
defined-out. There is currently no way in our unit tests to catch a
fatal assertion failure.
 2017-04-24 10:59:25 -04:00
Nick Mathewson
7fc37d41b4 Unit tests for consdiffmgr module 
Initial tests. These just try adding a few consensuses, looking
them up, and making sure that consensus diffs are generated in a
more or less reasonable-looking way.  It's enough for 87% coverage,
but it leaves out a lot of functionality.
 2017-04-24 10:59:25 -04:00
Nick Mathewson
bc91808c47 Add a "Consensus diff manager" module. 
This module's job is to remember old consensus documents, to
calculate their diffs on demand, and to .

There are some incomplete points in this code; I've marked them with
"XXXX". I intend to fix them in separate commits, since I believe
doing it in separate commits will make the branch easier to review.
 2017-04-24 10:59:24 -04:00