Commit Graph

13503 Commits

Author SHA1 Message Date
Alexander Færøy
64116ab97f
Fix tautological constant out-of-range comparison warnings. 2017-05-12 17:59:29 +02:00
Alexander Færøy
a1e8ef0076
Fix DoubleNL warning from make check-spaces. 2017-05-12 17:57:11 +02:00
Nick Mathewson
077d3085ec
actually enable background compresion for consensuses 2017-05-12 17:45:55 +02:00
Nick Mathewson
db370bb8a8
Test fix: expect old consensuses to be deleted if not deflate-compressed 2017-05-12 17:45:44 +02:00
Nick Mathewson
30dfb36148
consdiffmgr: Reload latest consensus entries on start. 2017-05-12 17:45:33 +02:00
Nick Mathewson
8100305e71
consdiffmgr: expose cached consensuses 2017-05-12 17:45:24 +02:00
Nick Mathewson
7b0dcf5c4a
Cleanup logic: only retain zlib-compressed consensuses
Now that we're making a bunch of these with consdiffmgr, we should
throw out all but one when we get a newer consensus.
2017-05-12 17:45:15 +02:00
Nick Mathewson
151cd121a2
consdiffmgr: compress incoming consensuses in the background
Also, compress them in several ways.

This breaks the unit tests; subsequent commits will make them pass
again.
2017-05-12 17:45:07 +02:00
Nick Mathewson
6da31ec484
consdiffmgr: Extract the code for compressing and storing
We're going to use this for consensuses too.
2017-05-12 17:44:55 +02:00
Alexander Færøy
7a3efe25d9
Use different preferences for compression methods when streaming.
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
141f6e3211
Add client_meth_pref array to define client compression preference.
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
59d17ca2bb
Fix indentation when using the ternary operator in handle_get_status_vote().
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
ef187bc280
Use compression_method_get_human_name() in connection_dir_client_reached_eof()
This patch refactors connection_dir_client_reached_eof() to use
compression_method_get_human_name() to set description1 and
description2 variables.

See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
f8218b5ada
Use compression_method_get_by_name() instead of explicit checks.
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
cf2f7a1bea
Decide compression method in the various handle_* functions().
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
a3a31fa120
Send "Accept-Encoding" to directory servers.
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
6305637197
Use tor_compress_supports_method() instead of constants.
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
61b6de5906
Handle Zstandard and LZMA in our check for correct guessed compression.
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
fbef257c43
Handle x-zstd and x-tor-lzma in parse_http_response().
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Alexander Færøy
ca632144e5
Use dir_compressed(_len) instead of dir_z(_len).
This patch renames `dir_z` to `dir_compressed` and `dir_z_len` to
`dir_compressed_len`.

See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Nick Mathewson
4410271446 Merge branch 'ticket21953_029' 2017-05-12 08:40:30 -04:00
Nick Mathewson
503f101d2b Enable some windows hardening features
One (HeapEnableTerminationOnCorruption) is on-by-default since win8;
the other (PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION) supposedly only
affects ATL, which (we think) we don't use.  Still, these are good
hygiene. Closes ticket 21953.
2017-05-11 16:39:02 -04:00
Nick Mathewson
9905659573 Merge remote-tracking branch 'arma/cleanup22213' 2017-05-11 09:20:12 -04:00
Nick Mathewson
6390a0c3b6 Merge branch 'ticket21871_031_03_squashed' 2017-05-11 08:33:32 -04:00
David Goulet
ae1d4cfdad prop224: Change encryption keys descriptor encoding
A descriptor only contains the curve25519 public key in the enc-key field so
the private key should not be in that data structure. The service data
structures will have access to the full keypair (#20657).

Furthermore, ticket #21871 has highlighted an issue in the proposal 224 about
the encryption key and legacy key being mutually exclusive. This is very wrong
and this commit fixes the code to follow the change to the proposal of that
ticket.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-11 08:33:26 -04:00
Roger Dingledine
8f1ddf0cd2 Remove unused "ROUTER_ADDED_NOTIFY_GENERATOR" internal value
We already have a way to return a 400 response code along with a
personalized message response for the uploader.

Resolves ticket 22213.
2017-05-10 20:03:07 -04:00
Roger Dingledine
466e27feae simplify functions now that they don't use options param 2017-05-10 17:57:35 -04:00
Roger Dingledine
2330a3713d Merge branch 'maint-0.3.0' 2017-05-10 17:36:34 -04:00
Roger Dingledine
716d48581d resolve now-unused parameter from #21642 fix 2017-05-10 17:35:36 -04:00
Nick Mathewson
1a497dcd1e Merge branch 'maint-0.3.0' 2017-05-10 16:28:07 -04:00
Nick Mathewson
a868b84599 Merge branch 'maint-0.2.9' into maint-0.3.0 2017-05-10 16:27:15 -04:00
Nick Mathewson
8f5da804da Merge branch 'prop275_minimal_029' into maint-0.2.9 2017-05-10 16:26:45 -04:00
Nick Mathewson
ee3ccd2fac #22211 Fix a comment in routerparse.c 2017-05-10 11:16:07 -04:00
Nick Mathewson
d76cffda60 Merge remote-tracking branch 'public/my-family-list-fix-4498' 2017-05-10 11:12:24 -04:00
Nick Mathewson
8266d193a6 Restore wget behavior when fetching compressed objects
We do this by treating the presence of .z as meaning ZLIB_METHOD,
even if Accept-Encoding does not include deflate.

This fixes bug 22206; bug not in any released tor.
2017-05-10 11:09:52 -04:00
Nick Mathewson
5dab99d6a8 Fix compilation on libevent2-only systems
Patch from rubiate; fixes bug 22219.  Remember, we don't support
libevent1 any more.
2017-05-10 11:08:49 -04:00
Roger Dingledine
0266c4ac81 add an XXX with a minor bug in dirserv_add_multiple_descriptors 2017-05-10 03:11:29 -04:00
Nick Mathewson
9f5b71a7ca Add a check and a cast in rephist.c to fix a warning 2017-05-09 11:13:22 -04:00
Nick Mathewson
b2cb3c33ac Tidy or_options_t by removing obsolete options.
Nothing was setting or inspecting these fields, and they were marked
as OBSOLETE() in config.c -- but somehow we still had them in the
or_options_t structure.  Ouch.
2017-05-09 10:40:24 -04:00
Nick Mathewson
2a1013948d Merge branch 'dgoulet_ticket22060_031_01_squashed' 2017-05-09 10:32:21 -04:00
David Goulet
b867295ffd config: Parse ports at the start of options_validate()
There was a bug that got exposed with the removal of ORListenAddress. Within
server_mode(), we now only check ORPort_set which is set in parse_ports().

However, options_validate() is using server_mode() at the start to check if we
need to look at the uname but then the ORPort_set is unset at that point
because the port parsing was done just after. This commit fixes that.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
7f95ef6e66 config: Remove {Control,DNS,Dir,Socks,Trans,NATD,OR}ListenAddress option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
039e2a24da config: Remove TLSECGroup option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
8aedc589ed config: Remove WarnUnsafeSocks option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
60cf5ac297 config: Remove CloseHSServiceRendCircuitsImmediatelyOnTimeout option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
87e9dc48d1 config: Remove CloseHSClientCircuitsImmediatelyOnTimeout option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
bc34654ba2 config: Remove FastFirstHopPK option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
09bc858dd5 config: Remove ExcludeSingleHopRelays option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
d52a1e2faa config: Remove AllowSingleHopExits option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
fea72571df config: Remove AllowSingleHopCircuits option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
2b9823b310 config: Remove AllowInvalidNodes option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:51 -04:00
Nick Mathewson
ff7e8531ec Report common reasons for rejecting a relay before uncommon ones
"You're running 0.2.2; upgrade!" is more sensible than "You have no
ntor key!"

Closes ticket 20270; bugfix on 0.2.9.3-alpha.
2017-05-09 10:28:51 -04:00
Nick Mathewson
b8abedfeee Split MyFamily into user-specified version and normalized version
This change prevents a no-longer-supported behavior where we change
options that would later be written back to torrc with a SAVECONF.

Also, use the "Pointer to final pointer" trick to build the
normalized list, to avoid special-casing the first element.
2017-05-09 08:19:26 -04:00
Nick Mathewson
7751df61ca Fix a warning about a shadowed global 2017-05-09 07:25:52 -04:00
Nick Mathewson
ef2248d09b Fix a warning about an extraneous semicolon 2017-05-09 07:25:34 -04:00
Nick Mathewson
2e4f3b36bd clang-i386: use house style for public-when-testing variables
This fixes a warning from jenkins.
2017-05-08 15:38:05 -04:00
Nick Mathewson
4d6c79d1de Fix some clang-i386 warnings in master. 2017-05-08 15:34:37 -04:00
Nick Mathewson
fb97f76e71 whitespace fixes 2017-05-08 13:57:08 -04:00
Nick Mathewson
4d30dde156 Merge branch 'netflow_padding-v6-rebased2-squashed' 2017-05-08 13:54:59 -04:00
Mike Perry
02a5835c27 Fix issues from dgoulet's code review.
https://gitlab.com/dgoulet/tor/merge_requests/24
2017-05-08 13:49:23 -04:00
Mike Perry
687a85950a Cache netflow-related consensus parameters.
Checking all of these parameter lists for every single connection every second
seems like it could be an expensive waste.

Updating globally cached versions when there is a new consensus will still
allow us to apply consensus parameter updates to all existing connections
immediately.
2017-05-08 13:49:23 -04:00
Mike Perry
76c9330f9d Bug 17604: Converge on only one long-lived TLS conn between relays.
Accomplished via the following:

1. Use NETINFO cells to determine if both peers will agree on canonical
   status. Prefer connections where they agree to those where they do not.
2. Alter channel_is_better() to prefer older orconns in the case of multiple
   canonical connections, and use the orconn with more circuits on it in case
   of age ties.

Also perform some hourly accounting on how many of these types of connections
there are and log it at info or notice level.
2017-05-08 13:49:22 -04:00
Mike Perry
d5a151a067 Bug 17592: Clean up connection timeout logic.
This unifies CircuitIdleTimeout and PredictedCircsRelevanceTime into a single
option, and randomizes it.

It also gives us control over the default value as well as relay-to-relay
connection lifespan through the consensus.

Conflicts:
	src/or/circuituse.c
	src/or/config.c
	src/or/main.c
	src/test/testing_common.c
2017-05-08 13:49:22 -04:00
Mike Perry
b0e92634d8 Netflow record collapsing defense.
This defense will cause Cisco, Juniper, Fortinet, and other routers operating
in the default configuration to collapse netflow records that would normally
be split due to the 15 second flow idle timeout.

Collapsing these records should greatly reduce the utility of default netflow
data for correlation attacks, since all client-side records should become 30
minute chunks of total bytes sent/received, rather than creating multiple
separate records for every webpage load/ssh command interaction/XMPP chat/whatever
else happens to be inactive for more than 15 seconds.

The defense adds consensus parameters to govern the range of timeout values
for sending padding packets, as well as for keeping connections open.

The defense only sends padding when connections are otherwise inactive, and it
does not pad connections used solely for directory traffic at all. By default
it also doesn't pad inter-relay connections.

Statistics on the total padding in the last 24 hours are exported to the
extra-info descriptors.
2017-05-08 13:49:21 -04:00
Nick Mathewson
9decf86711 Merge remote-tracking branch 'dgoulet/ticket21978_031_02' 2017-05-05 16:32:25 -04:00
Nick Mathewson
c276c10d3b Turn consdiffmgr.cache_max_age_hours into a parameter 2017-05-05 09:11:06 -04:00
Nick Mathewson
c985592874 prop140 clients now only try to get diffs from recent consensuses
Rationale: If it's a year old, the relay won't have a diff to it.

This is as specified in prop140
2017-05-05 09:11:06 -04:00
Alexander Færøy
60e97953ef
Fix memory leak found in CID #1405876. 2017-05-05 11:35:12 +02:00
Nick Mathewson
c6fe65fcaf Grammar fix in a log message 2017-05-04 08:58:06 -04:00
Nick Mathewson
baf489fc08 Fix: our directory.c code expects header constants to end with a : 2017-05-04 08:57:34 -04:00
Nick Mathewson
6beb7028d8 Do not BUG on missing sha3-as-signed field
This can happen if you've been running an earlier alpha on your
relay.  Instead, just ignore the entry.
2017-05-04 08:50:21 -04:00
Nick Mathewson
e1d31f2a2f Update the consdiff directory code based on #22143 fixes
These are mostly just identifier renames, except for one place in
routerparse.c where we switch to using a correct hash.
2017-05-04 08:49:02 -04:00
Nick Mathewson
a8eccb6363 Turn DEFAULT_IF_MODIFIED_SINCE_DELAY into a const 2017-05-04 08:37:41 -04:00
Nick Mathewson
c12d2cb2dc Request (and try to use) consensus diffs. 2017-05-04 08:37:41 -04:00
Nick Mathewson
912b0641e9 Generate X-Or-Diff-From-Consensus headers correctly. 2017-05-04 08:37:41 -04:00
Nick Mathewson
94ae99067f Remove excess indentation from previous commit.
Review this with 'diff -b' to confirm
2017-05-04 08:37:41 -04:00
Nick Mathewson
afa39cef6c Extract the consensus-only part of directory_get_from_dirserver
Right now it just sets an if-modified-since header, but it's about
to get even bigger.

This patch avoids changing indentation; the next patch will be
whitespace fixes.
2017-05-04 08:37:41 -04:00
Nick Mathewson
57710c1587 New function to add additional headers to a directory request 2017-05-04 08:37:41 -04:00
Nick Mathewson
112286338b Store the sha3 of a networkstatus as part of the networkstatus_t
Also store it in the cached_dir_t.
2017-05-04 08:37:41 -04:00
Nick Mathewson
0418357ffd Serve consensus diffs on request. 2017-05-04 08:37:41 -04:00
Nick Mathewson
e5f82969ca Support writing Content-Encoding headers other than deflate
Right now this only sends "deflate" or "identity", but there's more
to come.
2017-05-04 08:37:41 -04:00
Nick Mathewson
e051c47e98 Remove old unused indentation from handle_get_current_consensus
This commit removes a pair of meaningless braces, and changes
whitespace only.
2017-05-04 08:37:41 -04:00
Nick Mathewson
a32083bd03 Add consensus_cache_entry spooling support to spooled_resource_t 2017-05-04 08:37:41 -04:00
Nick Mathewson
3af9704e45 bug#22143/prop#140: in consdiffmgr, store and use digest-as-signed
We need to index diffs by the digest-as-signed of their source
consensus, so that we can find them even from consensuses whose
signatures are encoded differently.
2017-05-04 08:36:50 -04:00
Nick Mathewson
c8baa9b783 bug#22143/prop#140: Use <n>,$d commands in diffs to remove signatures
In this patch I add support for "delete through end of file" in our
ed diff handler, and generate our diffs so that they remove
everything after in the consensus after the signatures begin.
2017-05-04 08:36:50 -04:00
Nick Mathewson
5acddbbbf7 bug#22143/prop#140: identify input diffs by their digest-as-signed
See may 3 changes to prop140 for more background.
2017-05-03 13:09:08 -04:00
David Goulet
90b840af60 control: Fix NULL pointer access in HS desc event
This was introduced 90562fc23a adding a code
path where we pass a NULL pointer for the HSDir fingerprint to the control
event subsystem. The HS desc failed function wasn't handling properly that
pointer for a NULL value.

Two unit tests are also added in this commit to make sure we handle properly
the case of a NULL hsdir fingerprint and a NULL content as well.

Fixes #22138

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-03 09:26:17 -04:00
Nick Mathewson
e0b1fd4d3d Merge branch 'refactor_dir_client_handler' 2017-05-02 20:42:00 -04:00
Nick Mathewson
7bc9f93abb Merge branch 'teor_connection-with-client-v2_squashed' 2017-05-02 19:18:04 -04:00
teor
f9af7e8bd0 Accurately identify client connections by their lack of peer authentication
This means that we bail out earlier if asked to extend to a client.

Follow-up to 21407.
Fixes bug 21406; bugfix on 0.2.4.23.
2017-05-02 19:17:56 -04:00
Taylor Yu
5494087ed7 Delete useless checks in confparse.c
config_parse_interval() and config_parse_msec_interval() were checking
whether the variable "ok" (a pointer to an int) was null, rather than
derefencing it.  Both functions are static, and all existing callers
pass a valid pointer to those static functions.  The callers do check
the variables (also confusingly named "ok") whose addresses they pass
as the "ok" arguments, so even if the pointer check were corrected to
be a dereference, it would be redundant.

Fixes #22103.
2017-05-02 16:12:30 -04:00
Nick Mathewson
e0c937f316 Reindent the just-extracted directory response handler functions 2017-05-02 13:11:44 -04:00
Nick Mathewson
db86b9194d Break connection_dir_client_reached_eof() into smaller functions
This was a >630-line function, which doesn't make anybody happy.  It
was also mostly composed of a bunch of if-statements that handled
different directory responses differently depending on the original
purpose of the directory connection.  The logical refactoring here
is to move the body of each switch statement into a separate handler
function, and to invoke those functions from a separate switch
statement.

This commit leaves whitespace mostly untouched, for ease of review.
I'll reindent in the next commit.
2017-05-02 13:06:25 -04:00
Nick Mathewson
fd437f2a02 Remove special-casing for NO_METHOD in consdiffmgr.c 2017-05-02 08:37:57 -04:00
Nick Mathewson
465448e659 Merge branch 'dgoulet_bug22042_031_01_squashed' 2017-05-01 16:08:23 -04:00
David Goulet
aadb99e5f9 control: Fix comment of control_event_hs_descriptor_content
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-01 15:50:38 -04:00
David Goulet
90562fc23a hs: Trigger control event when client can't pick HSDir
Inform the control port with an HS_DESC failed event when the client is unable
to pick an HSDir. It's followed by an empty HS_DESC_CONTENT event. In order to
achieve that, some control port code had to be modified to accept a NULL HSDir
identity digest.

This commit also adds a trigger of a failed event when we are unable to
base64-decode the descriptor cookie.

Fixes #22042

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-01 15:50:38 -04:00
Sebastian Hahn
915fa39d0f Add --enable-rust configure switch
Introduce a way to optionally enable Rust integration for our builds. No
actual Rust code is added yet and specifying the flag has no effect
other than failing the build if rustc and cargo are unavailable.
2017-04-29 08:55:57 +02:00
Nick Mathewson
b8f7488e94 Fix a brazen memleak in consdiffmgr_add_consensus() 2017-04-28 15:41:52 -04:00
Nick Mathewson
00a12337ff Merge branches 'consdiffmgr_orig_squashed' and 'actually_compute_diffs_squashed' 2017-04-27 21:43:06 -04:00
Nick Mathewson
1e1581a24e Pre-compress consensus diffs with supported consensus methods. 2017-04-27 21:40:46 -04:00
Nick Mathewson
a1172b6774 Store archived consensuses compressed on disk.
I'm just using gzip compression now, for performance.
2017-04-27 21:40:46 -04:00
Nick Mathewson
7a0964279f Functionality to ensure there is space to add files to cache. 2017-04-27 21:40:13 -04:00
Nick Mathewson
920475f293 New force-delete option on consensus_cache_delete_pending()
If we're out of file space in the storage directory, we'll need to
get rid of old files fast.
2017-04-27 21:40:13 -04:00
Nick Mathewson
466e914088 Lower the file limit in consdiffmgr, to support seccomp2 2017-04-27 21:40:13 -04:00
Nick Mathewson
ab73bda060 Pass incoming consensus documents to the consdiffmgr code 2017-04-27 21:40:12 -04:00
Nick Mathewson
7b8d48a6cb Clean the consdiffmgr cache and launch new diffs as needed. 2017-04-27 21:40:12 -04:00
Nick Mathewson
fba8d7b222 Initialize consdiffmgr when running as (or becoming) a server. 2017-04-27 21:40:12 -04:00
Nick Mathewson
480dab4f2f Use a cast to try to avoid a tautalogical comparison warning 2017-04-27 11:58:26 -04:00
Nick Mathewson
10a4f9cd07 Merge branch 'parse_accept_encoding' 2017-04-27 11:31:31 -04:00
Nick Mathewson
2903c329aa Move the "supported compression bitmask" into compress.[ch] 2017-04-27 11:30:51 -04:00
Nick Mathewson
33a2fd065d Merge branch 'dirreq' 2017-04-27 10:08:32 -04:00
Nick Mathewson
b4fe0a6a03 Improve control flow in authority_certs_fetch_resource_impl 2017-04-27 09:30:08 -04:00
Nick Mathewson
52316f9969 Include UPLOAD_RENDDESC_V2 in PURPOSE_IS_UPLOAD
This was only used in one place before, and it's safe to update it.
2017-04-27 09:27:00 -04:00
Sebastian Hahn
ee478bdf38 Fix coverity cid 1405510
This is a false positive, but let's appease coverity.
2017-04-26 08:45:38 +02:00
Nick Mathewson
24f7059704 Configure sandbox using consdiffmgr; free cdm on exit. 2017-04-25 19:52:34 -04:00
Nick Mathewson
65ff0f8267 Bitmask out the compression methods that we do not support 2017-04-25 19:07:17 -04:00
Nick Mathewson
fec3050ea9 Tests for parse_accept_encoding 2017-04-25 19:01:05 -04:00
Nick Mathewson
fd48b757d3 Parse recognized entries from the Accept-Encoding header. 2017-04-25 19:01:05 -04:00
Nick Mathewson
9e081a44a9 Teach consdiffmgr to remember two digests: one compressed, one not. 2017-04-25 16:49:47 -04:00
Nick Mathewson
43db91bd87 Teach cov-exclude to detect runaway LCOV_EXCL_START lines
Also, fix two instances of runaway LCOV_EXCL_START lines.
2017-04-25 10:59:19 -04:00
Nick Mathewson
232c9e14a8 Merge branch 'atomic_counters' 2017-04-25 10:46:23 -04:00
Nick Mathewson
ec7c512d27 Merge remote-tracking branch 'dgoulet/bug21293_031_01' 2017-04-25 10:43:53 -04:00
David Goulet
cb8ac1f331 trace: Add a basic event-tracing infrastructure.
This commit adds the src/trace directory containing the basics for our tracing
subsystem. It is not used in the code base. The "src/trace/debug.h" file
contains an example on how we can map our tor trace events to log_debug().

The tracing subsystem can only be enabled by tracing framework at compile
time. This commit introduces the "--enable-tracing-debug" option that will
make all "tor_trace()" function be maped to "log_debug()".

Closes #13802

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-25 10:37:31 -04:00
Nick Mathewson
4266ec766a Use atomic counters for compressor allocation. 2017-04-25 10:29:07 -04:00
David Goulet
368b091329 relay: Change LD_BUG log to LD_PROTOCOL_WARN
That log statement can be triggered if somebody on the Internet behaves badly
which is possible with buggy implementation for instance.

Fixes #21293

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-25 09:27:34 -04:00
Nick Mathewson
ba405f86bf Merge branch 'ahf_prop278_21662_squashed' 2017-04-25 08:12:59 -04:00
Alexander Færøy
c2d1d949de Use tor_compress_supports_method() before printing library versions.
This patch ensures that Tor checks if a given compression method is
supported before printing the version string when calling `tor
--library-versions`.

Additionally, we use the `tor_compress_supports_method()` to check if a
given version is supported for Tor's start-up version string, but here
we print "N/A" if a given compression method is unavailable.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:10 -04:00
Alexander Færøy
04682d302a Add tor_compress_get_total_allocation() function.
This patch adds the `tor_compress_get_total_allocation()` which returns
an approximate number of bytes currently in use by all the different
compression backends.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
be4dc54634 Display LZMA and Zstandard versions when starting Tor.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
2fa7b722ce Show liblzma and libzstd versions in tor --library-versions.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
ce1feae9d9 Add --enable-zstd to our configure script.
This patch adds support for enabling support for Zstandard to our configure
script. By default, the --enable-zstd option is set to "auto" which means if
libzstd is available we'll build Tor with Zstandard support.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
157af1d26e Add --enable-lzma to our configure script.
This patch adds support for enabling support for LZMA to our configure
script. By default, the --enable-lzma option is set to "auto" which
means if liblzma is available we'll build Tor with LZMA support.

See: https://bugs.torproject.org/21662
2017-04-25 08:06:02 -04:00
Alexander Færøy
04583df452 Rename the torgzip module to compress.
See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Alexander Færøy
9d5bc1a935 Move zlib compression code into its own module.
This patch refactors the `torgzip` module to allow us to extend a common
compression API to support multiple compression backends.

Additionally we move the gzip/zlib code into its own module under the
name `compress_zlib`.

See https://bugs.torproject.org/21664
2017-04-25 08:06:01 -04:00
David Goulet
6f27843d57 hs: Refactor rend_add_service()
Remove duplicate code that validates a service object which is now in
rend_validate_service().

Add some comments on why we nullify a service in the code path of
rend_config_services().

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-24 14:51:34 -04:00
David Goulet
ed7c0170c4 hs: Add rend_validate_service() function
This new function validates a service object and is used everytime a service
is successfully loaded from the configuration file.

It is currently copying the validation that rend_add_service() also does which
means both functions validate. It will be decoupled in the next commit.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-24 14:39:54 -04:00
Nick Mathewson
4cefda85e4 Merge remote-tracking branch 'dgoulet/ticket21980_031_01' 2017-04-24 12:42:11 -04:00
Nick Mathewson
e55c1412c1 Fix a signed/unsigned comparison warning on 32-bit 2017-04-24 11:41:11 -04:00
Nick Mathewson
b7567a6282 Merge branch 'consdiffmgr_squashed' 2017-04-24 11:02:22 -04:00
Nick Mathewson
6cc21aa89c consdiffmgr: add tests for cdm_entry_get_sha3_value 2017-04-24 11:01:40 -04:00
Nick Mathewson
43d683e0ad Fix reference leak & handle leak in consensus_diff_worker_replyfn
Found by previous test.
2017-04-24 11:01:40 -04:00
Nick Mathewson
7ca86b9cd6 Add a hashtable to consdiffmgr to keep track of diff status
In several places in the old code, we had problems that only an
in-memory index of diff status could solve, including:
   * Remembering which diffs were in-progress, so that we didn't
     re-launch them.
   * Remembering which diffs had failed, so that we didn't try to
     recompute them over and over.
   * Having a fast way to look up the diff from a given consensus to
     the latest consensus of a given flavor.

This patch adds a hashtable mapping from (flavor, source diff), to
solve the problem.  It maps to a cache entry handle, rather than to
a cache entry directly, so that it doesn't affect the reference
counts of the cache entries, and so that we don't otherwise need to
worry about lifetime management.
2017-04-24 11:01:40 -04:00
Nick Mathewson
69a212ff3d Consdiffmgr: extract "get a sha3 digest" function.
I'll be using this a lot in the hashtable tweaks here.
2017-04-24 11:01:40 -04:00
Nick Mathewson
655f1c8e01 consdiffmgr: function to re-validate stored sha3 digests at startup 2017-04-24 11:00:28 -04:00
Nick Mathewson
6c86e63029 Consdiffmgr: use aggressive-release flag on consensuses
This conscache flag tells conscache that it should munmap the
document as soon as reasonably possible, since its usage pattern is
expected to not have a lot of time-locality.
2017-04-24 10:59:25 -04:00
Nick Mathewson
82bb8afb60 Add handle support to consensus_cache_entry_t
This will allow us to have weak references to cache entries.
2017-04-24 10:59:25 -04:00
Nick Mathewson
35f6b678ab Test the easiest cases of consdiffmgr_cleanup.
One more to go: deleting the old diffs.
2017-04-24 10:59:25 -04:00
Nick Mathewson
1fade37287 consdiffmgr non-test: check for initialization failure
Unfortunately, this test doesn't work, so I've left it
defined-out. There is currently no way in our unit tests to catch a
fatal assertion failure.
2017-04-24 10:59:25 -04:00
Nick Mathewson
7fc37d41b4 Unit tests for consdiffmgr module
Initial tests. These just try adding a few consensuses, looking
them up, and making sure that consensus diffs are generated in a
more or less reasonable-looking way.  It's enough for 87% coverage,
but it leaves out a lot of functionality.
2017-04-24 10:59:25 -04:00