Commit Graph

18133 Commits

Author SHA1 Message Date
Nick Mathewson
b7dd3518b9 missing smartlist_free in new consdiff test 2017-04-18 22:56:34 -04:00
Nick Mathewson
afe1af7de6 Merge remote-tracking branch 'asn/bug21971' 2017-04-18 21:38:28 -04:00
Nick Mathewson
eeb266c139 Merge remote-tracking branches 'sebastian/consdiff_add' and 'sebastian/consdiff_newline' 2017-04-18 21:31:53 -04:00
George Kadianakis
e1a59ade95 prop224: Add time period functions and unittests
This will be used by the build blinded key functions.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-18 11:03:15 -04:00
Alexander Færøy
e6c6606a17
Expose tor_compress_is_compression_bomb() as part of the public compression API.
This patch exposes the old `is_compression_bomb()` function as a public
API as part of the compression module.

See https://bugs.torproject.org/21663
2017-04-18 03:12:19 +02:00
Alexander Færøy
6b5172bcf2
Expose tor_compress_memory_level() as a public function in the compression module.
This patch makes the internal `get_memlevel()` a part of the public
compression API as `tor_compress_memory_level()`.

See https://bugs.torproject.org/21663
2017-04-18 02:23:25 +02:00
Alexander Færøy
b8c9f229d7
Rename write_to_buf_zlib() to write_to_buf_compress().
See https://bugs.torproject.org/21663
2017-04-18 01:23:39 +02:00
Alexander Færøy
40ed68290e
Rename zlib_state to compress_state in the test_buffers.
See https://bugs.torproject.org/21663
2017-04-18 01:12:01 +02:00
Alexander Færøy
4b9349192d
Rename zlib_state to compress_state in dir_connection_t.
This patch renames the `zlib_state` field in `dir_connection_t` to
`compress_state`.

See https://bugs.torproject.org/21663
2017-04-18 01:10:25 +02:00
George Kadianakis
0c39cefc52 prop224 tests: Remove useless NULL check before circuit_free().
Addresses coverity issue CID 1405130.
2017-04-17 16:36:55 +03:00
George Kadianakis
a3ce303432 prop224 tests: Don't use tt_size_op to compare ssize_t.
Addresses coverity CID 1405129.
2017-04-17 16:23:58 +03:00
Alexander Færøy
3c4459bcbf
Refactor the streaming compression code.
This patch refactors our streaming compression code to allow us to
extend it with non-zlib/non-gzip based compression schemas.

See https://bugs.torproject.org/21663
2017-04-17 14:57:37 +02:00
Alexander Færøy
44cb86adbe
Rename tor_gzip_{compress,uncompress} to tor_{compress,uncompress}.
To allow us to use the API name `tor_compress` and `tor_uncompress` as
the main entry-point for all compression/uncompression and not just gzip
and zlib.

See https://bugs.torproject.org/21663
2017-04-17 14:29:10 +02:00
Alexander Færøy
e8b025dfc3
Rename zlib_compression_level_t to compression_level_t.
See https://bugs.torproject.org/21663
2017-04-17 14:22:13 +02:00
Alexander Færøy
7460b9755a
Remove unused function is_gzip_supported().
This patch removes the unused `is_gzip_supported()` and changes the
documentation string around the `compress_method_t` enumeration to
explicitly state that both `ZLIB_METHOD` and `GZIP_METHOD` are both
always supported.

Zlib version 1.2.0 was released on the 9'th of March, 2003 according to
their ChangeLog.

See https://bugs.torproject.org/21663
2017-04-17 14:11:35 +02:00
Alexander Færøy
4b834e0d5e
Fix whitespace in test_util_gzip() around OP_*`.
See https://bugs.torproject.org/21663
2017-04-17 14:07:23 +02:00
Alexander Færøy
a8821d8366
Use tt_int_op() over tt_assert() and do explicit NULL checks in test_util_gzip().
This patch changes some of the tt_assert() usage in test_util_gzip() to
use tt_int_op() to get better error messages upon failure.

Additionally we move to use explicit NULL checks.

See https://bugs.torproject.org/21663
2017-04-17 14:02:16 +02:00
Sebastian Hahn
5a7e39c0cb Fix diff generation with line added at start
The consdiff generation logic would skip over lines added at the start of the
second file, and generate a diff that it would the immediately refuse because
it couldn't be used to reproduce the second file from the first. Fixes #21996.
2017-04-17 10:10:46 +02:00
Sebastian Hahn
459643502b Don't accept ranges for add commands in consdiff
Fixes ticket #21963
2017-04-17 06:49:27 +02:00
Nick Mathewson
91a6fbcf77 Begin work to turn "directory request" into an object
This patch adds the object type, and makes all of the functions
that used to feed into directory_initiate_command_rend() feed into
that type instead.
2017-04-16 18:45:48 -04:00
Nick Mathewson
b022ea32a6 Expand cpuworker API to allow other work types 2017-04-15 11:21:33 -04:00
Nick Mathewson
222122450c Add a config_line_prepend() function 2017-04-15 11:21:32 -04:00
Nick Mathewson
06ecb9432f conscache.c: do not match entries that are slated for removal. 2017-04-15 11:21:32 -04:00
Nick Mathewson
b081a7ed21 Merge branch 'ticket21891_031_01_squashed' 2017-04-13 16:43:13 -04:00
David Goulet
0565f5a3bb hs: Make the service list pruning function public
The reason for making the temporary list public is to keep it encapsulated in
the rendservice subsystem so the prop224 code does not have direct access to
it and can only affect it through the rendservice pruning function.

It also has been modified to not take list as arguments but rather use the
global lists (main and temporary ones) because prop224 code will call it to
actually prune the rendservice's lists. The function does the needed rotation
of pointers between those lists and then prune if needed.

In order to make the unit test work and not completely horrible, there is a
"impl_" version of the function that doesn't free memory, it simply moves
pointers around. It is directly used in the unit test and two setter functions
for those lists' pointer have been added only for unit test.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-13 16:25:49 -04:00
Nick Mathewson
ba89520593 fix wide lines 2017-04-13 14:36:57 -04:00
Nick Mathewson
902672eac6 Merge branch 'ticket21889_031_01_squashed' 2017-04-13 14:23:59 -04:00
George Kadianakis
f02868bb53 hs: Add service-side circuitmap API.
Now we have separate getters and setters for service-side and relay-side. I
took this approach over adding arguments to the already existing methods to
have more explicit type-checking, and also because some functions would grow
too large and dirty.

This commit also fixes every callsite to use the new function names which
modifies the legacy HS (v2) and the prop224 (v3) code.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-13 14:23:51 -04:00
Nick Mathewson
63e39b291e Merge remote-tracking branch 'dgoulet/bug21155_031_02' 2017-04-13 09:56:14 -04:00
Nick Mathewson
28ec2d9c2c Merge remote-tracking branch 'dgoulet/ticket21919_031_01' 2017-04-13 09:38:59 -04:00
Nick Mathewson
755c88a474 Merge branch 'asn/prop224-ntor-v2-squashed' 2017-04-13 09:22:34 -04:00
George Kadianakis
ea5901bf1c prop224: Add Python integration tests for HS ntor.
This test is identical to the ./src/test/test_ntor.sh integration test.
2017-04-13 09:22:19 -04:00
George Kadianakis
18ee145cda prop224: Add basic HS ntor unittest.
The test checks that introduce1/rendezvous1 key material is generated
correctly both for client-side and service-side.
2017-04-13 09:22:19 -04:00
George Kadianakis
50b0bc5bfe prop224: Add module that performs the HS ntor handshake.
and also does the key expansion.
2017-04-13 09:22:19 -04:00
David Goulet
bf68c78e94 hs: Make check-spaces happy after rename
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-12 07:53:05 -04:00
David Goulet
6bacc3c7a8 hs: Change trunnel prop224 cell's namespace
One of the goals of this change is to have trunnel API/ABI being more explicit
so we namespace them with "trn_*". Furthermore, we can now create
hs_cells.[ch] without having to confuse it with trunnel which used to be
"hs_cell_*" before that change.

Here are the perl line that were used for this rename:

  perl -i -pe 's/cell_extension/trn_cell_extension/g;' src/*/*.[ch]
  perl -i -pe 's/cell_extension/trn_cell_extension/g;' src/trunnel/hs/*.trunnel
  perl -i -pe 's/hs_cell_/trn_cell_/g;' src/*/*.[ch]
  perl -i -pe 's/hs_cell_/trn_cell_/g;' src/trunnel/hs/*.trunnel

  And then "./scripts/codegen/run_trunnel.sh" with trunnel commit id
  613fb1b98e58504e2b84ef56b1602b6380629043.

Fixes #21919

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-11 13:46:41 -04:00
David Goulet
e5fc02c81c config: Warn if EntryNodes and HiddenService are used together
Pinning EntryNodes along with hidden services can be possibly harmful (for
instance #14917 and #21155) so at the very least warn the operator if this is
the case.

Fixes #21155

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-10 13:22:36 -04:00
Nick Mathewson
01fc93ffef Merge remote-tracking branch 'dgoulet/bug16706_031_01' 2017-04-10 12:12:14 -04:00
Nick Mathewson
f5258045c4 Fix some no-longer-reasonable unit tests for base64_decode()
These tests tried to use ridiculously large buffer sizes to check
the sanity-checking in the code; but since the sanity-checking
changed, these need to change too.
2017-04-10 12:08:31 -04:00
Taylor Yu
26dd4d92cf Remove SR_COMMIT_LEN workaround
Now that base64_decode() checks the destination buffer length against
the actual number of bytes as they're produced, shared_random.c no
longer needs the "SR_COMMIT_LEN+2" workaround.
2017-04-07 18:19:15 -04:00
Taylor Yu
2b38bb28b4 Remove base64_decode_nopad()
Remove base64_decode_nopad() because it is redundant now that
base64_decode() correctly handles both padded and unpadded base64
encodings with "right-sized" output buffers.
2017-04-07 18:18:23 -04:00
Taylor Yu
00ffefb41b Test odd-sized base64 decodes
Test base64_decode() with odd sized decoded lengths, including
unpadded encodings and padded encodings with "right-sized" output
buffers.  Convert calls to base64_decode_nopad() to base64_decode()
because base64_decode_nopad() is redundant.
2017-04-07 18:16:45 -04:00
Taylor Yu
f15818f280 Make base64_decode() check actual decoded length
base64_decode() was applying an overly conservative check on the
output buffer length that could incorrectly produce an error if the
input encoding contained padding or newlines.  Fix this by checking
the output buffer length against the actual decoded length produced
during decoding.
2017-04-07 18:13:22 -04:00
Nick Mathewson
05ef3b959d Merge branch 'maint-0.3.0' 2017-04-07 14:03:40 -04:00
Nick Mathewson
f1613b53c5 Merge remote-tracking branch 'public/bug21894_029' into maint-0.3.0 2017-04-07 14:03:25 -04:00
Nick Mathewson
f0fa7dcdf0 Merge branch 'ticket21842_squashed' 2017-04-07 13:21:15 -04:00
Nick Mathewson
7b60f0129a Remove tor-checkkey as obsolete
CVE-2008-0166 is long gone, and we no longer need a helper tool to
dump out public key moduli so folks can detect it.

Closes ticket 21842.
2017-04-07 13:21:07 -04:00
Nick Mathewson
506b4bfaba Merge remote-tracking branch 'dgoulet/ticket21893_031_01' 2017-04-07 11:04:06 -04:00
Nick Mathewson
4812441d34 Never read off the end of a buffer in base32_encode()
When we "fixed" #18280 in 4e4a7d2b0c
in 0291 it appears that we introduced a bug: The base32_encode
function can read off the end of the input buffer, if the input
buffer size modulo 5 is not equal to 0 or 3.

This is not completely horrible, for two reasons:
   * The extra bits that are read are never actually used: so this
     is only a crash when asan is enabled, in the worst case.  Not a
     data leak.

   * The input sizes passed to base32_encode are only ever multiples
      of 5. They are all either DIGEST_LEN (20), REND_SERVICE_ID_LEN
      (10), sizeof(rand_bytes) in addressmap.c (10), or an input in
      crypto.c that is forced to a multiple of 5.

So this bug can't actually trigger in today's Tor.

Closes bug 21894; bugfix on 0.2.9.1-alpha.
2017-04-07 10:47:16 -04:00
Nick Mathewson
d33c3627f4 Comment fix. (Catalyst spotted this) 2017-04-07 10:00:08 -04:00
Nick Mathewson
61f1838cdc Merge branch 'isolate_openssl' 2017-04-07 09:58:21 -04:00
Nick Mathewson
f5cc8da7e0 Use DIGEST512_LEN macro in crypto_hash_sha512.h in ref10 2017-04-07 09:58:04 -04:00
Nick Mathewson
1e54bdd48a Make the warnings about terminating nuls a bit stronger
It looks like 32_encoded_size/64_encode_size APIs are inconsistent
not only in the number of "d"s they have, but also in whether they
count the terminating NUL.  Taylor noted this in 86477f4e3f,
but I think we should note the inconsistently more loudly in order
to avoid trouble.

(I ran into trouble with this when writing 30b13fd82e243713c6a0d.)
2017-04-07 09:51:08 -04:00
Nick Mathewson
2c86380bde Merge remote-tracking branch 'argonblue/baseXXlen' 2017-04-07 09:48:18 -04:00
Nick Mathewson
30b13fd82e Add test for expected output from encode{,d}_length functions 2017-04-07 09:47:29 -04:00
David Goulet
dff50001ec prop224: Flag router with HSIntro/HSDir using protover
Note down in the routerstatus_t of a node if the router supports the HSIntro=4
version for the ed25519 authentication key and HSDir=2 version for the v3
descriptor supports.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-07 09:44:52 -04:00
David Goulet
e7b7e99cc7 hs: Move common defines to hs_common.h
Some of those defines will be used by the v3 HS protocol so move them to a
common header out of rendservice.c. This is also ground work for prop224
service implementation.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-07 09:22:58 -04:00
David Goulet
c716702625 hs: Remove redundant define of ed25519 auth key type
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-07 09:22:55 -04:00
David Goulet
419c0c0788 hs: Move service check private dir to hs_common.c
Another building blocks for prop224 service work. This also makes the function
takes specific argument instead of the or_option_t object.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-07 09:22:52 -04:00
George Kadianakis
037ce360bd hs: Refactor circuitmap to use circuit_t instead of or_circuit_t. 2017-04-07 09:11:09 -04:00
Taylor Yu
681812ea40 Use macros for base64 lengths in shared_random.h
Fixes #19564.
2017-04-06 17:43:17 -04:00
Taylor Yu
86477f4e3f Use baseXX length macros in baseXX_encode()
Use the new baseXX length macros to clean up the length checks in the
baseXX_encode() functions.
2017-04-06 17:43:16 -04:00
Taylor Yu
e7f40baade Add macros for baseXX encoding lengths 2017-04-06 15:43:55 -04:00
Taylor Yu
c5adab0258 Make CEIL_DIV() slightly more overflow-safe 2017-04-06 13:54:46 -04:00
Nick Mathewson
9d34a1e052 Merge branch 'storage_labeled_squashed' 2017-04-06 11:49:00 -04:00
Nick Mathewson
5567a80caa Explain config_line_find() behavior on duplicates. 2017-04-06 11:48:41 -04:00
Nick Mathewson
89b673f24b Add an assertion to config_line_append(). 2017-04-06 11:48:41 -04:00
Nick Mathewson
95a4f2d525 Additional unit tests to improve conscache coverage 2017-04-06 11:48:41 -04:00
Nick Mathewson
2b5b6025bd Tests for cleanup and reference counting on conscache 2017-04-06 11:48:41 -04:00
Nick Mathewson
73e9bc914f Tests for simple cases of conscache code. 2017-04-06 11:48:41 -04:00
Nick Mathewson
4404dc5756 Add a 'consensus cache' type on top of storagedir.
Every file in the cache is labeled.  The labels are held in memory;
the bodies are mapped on demand.
2017-04-06 11:48:41 -04:00
Nick Mathewson
5f8860a16f Unit tests for labelled storagedir entries 2017-04-06 11:48:41 -04:00
Nick Mathewson
37bb3b31fa Add some "labeled storagedir" abstractions.
These add a tiny bit of structure on top of the regular storagedir
abstractions in order to store key-value lists at the head of each
document.
2017-04-06 11:48:40 -04:00
Nick Mathewson
4c2ad48568 Merge branch 'maint-0.3.0' 2017-04-06 08:32:12 -04:00
Nick Mathewson
a5b50ef25b Merge branch 'maint-0.2.9' into maint-0.3.0 2017-04-06 08:32:04 -04:00
Nick Mathewson
7d7770f735 Merge branch 'maint-0.2.8' into maint-0.2.9 2017-04-06 08:31:54 -04:00
Nick Mathewson
91c6b18ca0 Merge branch 'maint-0.2.7-redux' into maint-0.2.8 2017-04-06 08:31:32 -04:00
Nick Mathewson
44bc8821b1 Merge branch 'maint-0.2.6' into maint-0.2.7-redux 2017-04-06 08:31:23 -04:00
Nick Mathewson
21d776738c Merge branch 'maint-0.2.5' into maint-0.2.6 2017-04-06 08:31:18 -04:00
Nick Mathewson
a8a860e1da Merge branch 'maint-0.2.4' into maint-0.2.5 2017-04-06 08:31:12 -04:00
Karsten Loesing
9d7933296c Update geoip and geoip6 to the April 4 2017 database. 2017-04-06 10:52:39 +02:00
David Goulet
0fdad78fe1 cell: Do not warn on hidden service invalid port
When a client tried to connect to an invalid port of an hidden service, a
warning was printed:

  [warn] connection_edge_process_relay_cell (at origin) failed.

This is because the connection subsystem wants to close the circuit because
the port can't be found and then returns a negative reason to achieve that.
However, that specific situation triggered a warning. This commit prevents it
for the specific case of an invalid hidden service port.

Fixes #16706

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-05 16:28:37 -04:00
Nick Mathewson
06fe8a14c4 bump to 0.3.0.5-rc-dev 2017-04-05 10:03:09 -04:00
Nick Mathewson
b6f22524ee Move config line parsing function to confline.c 2017-04-04 11:11:01 -04:00
Nick Mathewson
c2947dbb86 Move config_line_t functions from confparse.c into common.
I'm doing this to storagedir to used config_line_t.
2017-04-04 11:11:01 -04:00
Nick Mathewson
321c1c453e Merge branch 'maint-0.3.0' 2017-04-04 09:43:23 -04:00
David Goulet
7519584347 hs: Expose hs_service.c functions for unit tests
In order to avoid src/or/hs_service.o to contain no symbols and thus making
clang throw a warning, the functions are now exposed not just to unit tests.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-04 09:41:14 -04:00
Taylor Yu
b1c7e5d8c0 Capture expected log in test_circuitbuild.c
Capture the warning for the unhandled circuit purpose test case, both
to clean up the test log and to confirm that it gets logged.
2017-04-03 15:24:59 -04:00
Nick Mathewson
67c88fd10d Merge branch 'bug13790_rebased' 2017-04-03 11:58:37 -04:00
Taylor Yu
f0a57df55c Test unhandled purpose in route_len_for_purpose()
Check that route_len_for_purpose() (helper for new_route_len())
correctly fails a non-fatal bug assertion if it encounters an
unhandled circuit purpose when it is called with exit node info.
2017-04-03 11:58:11 -04:00
Taylor Yu
ceedcfe9f2 Refactor and comment new_route_len()
Add a new helper function route_len_for_purpose(), which explicitly
lists all of the known circuit purposes for a circuit with a chosen
exit node (unlike previously, where the default route length for a
chosen exit was DEFAULT_ROUTE_LEN + 1 except for two purposes).  Add a
non-fatal assertion for unhandled purposes that conservatively returns
DEFAULT_ROUTE_LEN + 1.

Add copious comments documenting which circuits need an extra hop and
why.

Thanks to nickm and dgoulet for providing background information.
2017-04-03 11:58:11 -04:00
Taylor Yu
1e8e8a4e94 Add tests for new_route_len() 2017-04-03 11:58:11 -04:00
Nick Mathewson
6ec3228220 version bump 2017-04-03 09:36:42 -04:00
Nick Mathewson
a06c465333 Merge remote-tracking branch 'teor/bug21596_030' into maint-0.3.0 2017-04-03 09:28:33 -04:00
Nick Mathewson
5d8061cd01 Merge remote-tracking branch 'public/bug21415_testfix_030' into maint-0.3.0 2017-04-03 09:27:40 -04:00
Nick Mathewson
28f9b68e87 Move "change cert expiration and re-sign" fn into tortls.c
This lets test_link_handshake stop including openssl headers.
2017-03-31 10:04:45 -04:00
Nick Mathewson
5ca0d6daf0 Mark many private tortls.h APIs as openssl-only.
This change lets us remove the openssl/ssl.h include from
test_link_handshake.c.
2017-03-31 10:04:45 -04:00
Nick Mathewson
a8b9aba912 Change many tortls.h declarations of private APIs to use structs
This change makes it so those those APIs will not require prior
inclusion of openssl headers.  I've left some APIs alone-- those
will change to be extra-private.
2017-03-31 10:04:45 -04:00
Nick Mathewson
706c44a6ce Remove some now-needless openssl includes from src/test.
It would appear that these includes weren't actually used.
2017-03-31 10:04:44 -04:00
Nick Mathewson
a4964466a5 Remove openssl/evp.h dependency from test_crypto.c 2017-03-31 10:04:44 -04:00
Nick Mathewson
1a14e5be91 Remove crypto/rand include from test_crypto.c
Create a new test_crypto_openssl to test openssl-only crypto.c
functionality.
2017-03-31 10:04:44 -04:00
Nick Mathewson
38fb651f0d Make our ed25519 implementations no longer use openssl directly. 2017-03-31 10:04:44 -04:00
Nick Mathewson
3fc4f81de3 Make crypto_ed25519.c no longer depend on openssl
Now it calls through our own crypto API.
2017-03-31 10:04:44 -04:00
Nick Mathewson
e7506c03cf Isolate dmalloc/openssl bridge code to crypto.c
This makes it so main.c, and the rest of src/or, no longer need to
include any openssl headers.
2017-03-31 10:04:44 -04:00
Nick Mathewson
04f1ddaa2a Fix utimbuf initialization in storagedir/cleaning test 2017-03-29 09:10:35 +02:00
Nick Mathewson
fc02b8cf7a Fix another 32-bit warning in the spooling code 2017-03-29 09:07:51 +02:00
Nick Mathewson
be778a65df Fix i386 compilation from spooling patch. I seem to be good at breaking that. 2017-03-28 21:41:59 +02:00
Nick Mathewson
208c2a5c74 Merge branch 'spooling_squashed' 2017-03-28 19:55:18 +02:00
Nick Mathewson
003a7fe2a3 Add nonfatal asserts for return val of dirserv_flushed_some 2017-03-28 19:55:04 +02:00
Nick Mathewson
436b975e21 Improve documentation for connection_dirserv_flushed_some. 2017-03-28 19:55:03 +02:00
Nick Mathewson
8aa69a1b69 Refactor the directory spool implementation
The old implementation had duplicated code in a bunch of places, and
it interspersed spool-management with resource management.  The new
implementation should make it easier to add new resource types and
maintain the spooling code.

Closing ticket 21651.
2017-03-28 19:55:03 +02:00
Nick Mathewson
4d5b46aad9 Merge remote-tracking branch 'origin/maint-0.3.0' 2017-03-27 15:44:31 +02:00
George Kadianakis
ef4c10fb42 Fix max sampled size logic when in bridge mode.
When calculating max sampled size, Tor would only count the number of
bridges in torrc, without considering that our state file might already
have sampled bridges in it. This caused problems when people swap
bridges, since the following error would trigger:

         [warn] Not expanding the guard sample any further; just hit the
                maximum sample threshold of 1
2017-03-27 15:39:26 +02:00
Nick Mathewson
e79f90c7f0 Storagedir test fix: save strings in binary mode to preserve length 2017-03-27 15:27:52 +02:00
Nick Mathewson
10888dcac6 Merge remote-tracking branch 'ahf/bugs/21757' 2017-03-27 15:02:48 +02:00
Nick Mathewson
61e8ee1117 Merge remote-tracking branch 'jigsaw52/fix_leak_21788' 2017-03-27 11:01:01 +02:00
Nick Mathewson
1d617e3ed0 Check and cast st_size to size_t in storagedir code
This prevents an i386 compilation warning and fixes bug 21828. Bug not
in any released Tor.
2017-03-27 10:40:15 +02:00
Nick Mathewson
0895808023 Merge remote-tracking branch 'origin/maint-0.3.0' 2017-03-26 12:30:45 +02:00
Nick Mathewson
a5130de432 Tweak test_entrynodes comment 2017-03-26 12:30:00 +02:00
Nick Mathewson
4f6025498d Use update_approx_time() to run a test 100 days in the past.
Fixes bug21799.
2017-03-23 10:44:43 +01:00
Daniel Pinto
e843481bf5 Fix very small memory leak #21788
Leak caused by clean_up_backtrace_handler not being called
on shutdown.
2017-03-20 01:03:05 +00:00
Nick Mathewson
58680d0429 Merge branch 'ahf_bugs_21641_squashed' 2017-03-17 11:16:24 -04:00
Alexander Færøy
946ccf3e4d Check onion key consensus parameters every hour.
This patch changes the way we decide when to check for whether it's time
to rotate and/or expiry our onion keys. Due to proposal #274 we can now
have the keys rotate at different frequencies than before and we thus
do the check once an hour when our Tor daemon is running in server mode.

This should allow us to quickly notice if the network consensus
parameter have changed while we are running instead of having to wait
until the current parameters timeout value have passed.

See: See: https://bugs.torproject.org/21641
2017-03-17 11:15:43 -04:00
Alexander Færøy
853b54dea4 Add periodic timer for expiring old onion keys.
This patch adds a new timer that is executed when it is time to expire
our current set of old onion keys. Because of proposal #274 this can no
longer be assumed to be at the same time we rotate our onion keys since
they will be updated less frequently.

See: https://bugs.torproject.org/21641
2017-03-17 11:15:43 -04:00
Alexander Færøy
d88f10cdf2 Add API to query the current onion key grace period.
This patch adds an API to get the current grace period, in days, defined
as the consensus parameter "onion-key-grace-period-days".

As per proposal #274 the values for "onion-key-grace-period-days" is a
default value of 7 days, a minimum value of 1 day, and a maximum value
defined by other consensus parameter "onion-key-rotation-days" also
defined in days.

See: https://bugs.torproject.org/21641
2017-03-17 11:15:43 -04:00
Alexander Færøy
23ae5b655b Make MIN_ONION_KEY_LIFETIME a consensus parameter defined value.
This patch turns `MIN_ONION_KEY_LIFETIME` into a new function
`get_onion_key_lifetime()` which gets its value from a network consensus
parameter named "onion-key-rotation-days". This allows us to tune the
value at a later point in time with no code modifications.

We also bump the default onion key lifetime from 7 to 28 days as per
proposal #274.

See: https://bugs.torproject.org/21641
2017-03-17 11:15:43 -04:00
Alexander Færøy
a28be68cf2
Split strings at newline in tor_get_lines_from_handle().
This patch fixes a regression described in bug #21757 that first
appeared after commit 6e78ede73f which was an attempt to fix bug #21654.

When switching from buffered I/O to direct file descriptor I/O our
output strings from get_string_from_pipe() might contain newline
characters (\n). In this patch we modify tor_get_lines_from_handle() to
ensure that the function splits the newly read string at the newline
character and thus might return multiple lines from a single call to
get_string_from_pipe().

Additionally, we add a test case to test_util_string_from_pipe() to
ensure that get_string_from_pipe() correctly returns multiple lines in a
single call.

See: https://bugs.torproject.org/21757
See: https://bugs.torproject.org/21654
2017-03-17 04:27:12 +01:00
Nick Mathewson
6657fe1e54 Generate src/test/fuzz/include.am from a script
It was very error-prone to maintain this by hand.
2017-03-16 17:14:10 -04:00
Nick Mathewson
45d008c4aa Merge branch 'storagedir_squashed' 2017-03-16 16:03:13 -04:00
Nick Mathewson
ee253e392a Define a "storagedir" abstraction to hold numerous similar files
We could use one of these for holding "junk" descriptors and
unparseable things -- but we'll _need_ it for having cached
consensuses and diffs between them.
2017-03-16 16:01:59 -04:00
Nick Mathewson
877468c781 Now, resolve a new 64-bit warning in compat_threads.c 2017-03-16 15:34:57 -04:00
Nick Mathewson
eca9b3424d consdiff: Fix 32-bit compilation.
Thanks, jenkins!
2017-03-16 15:14:16 -04:00
Nick Mathewson
4a4f1e44af Merge branch 'prop140_21643_diff_only_squashed' 2017-03-16 14:58:43 -04:00
Nick Mathewson
643c9078c9 Switch ed-command parsing to use tor_parse_long. 2017-03-16 14:42:56 -04:00
Nick Mathewson
672e2a5461 Consdiff: extract router ID hash iteration functions
There was a frequent block of code that did "find the next router
line, see if we've hit the end of the list, get the ID hash from the
line, and enforce well-ordering."  Per Ahf's review, I'm extracting
it to its own function.
2017-03-16 14:42:56 -04:00
Nick Mathewson
83049cebc0 Make CONSENSUS_LINE_MAX_LEN a real define 2017-03-16 14:42:56 -04:00
Nick Mathewson
5e81436428 Remove a couple of {\n\n instances 2017-03-16 14:42:56 -04:00
Nick Mathewson
e84276124b Use a better for X outside of base64_compare_table. 2017-03-16 14:42:56 -04:00
Nick Mathewson
91e3250bf1 Reindent a few labels. 2017-03-16 14:42:56 -04:00
Nick Mathewson
d8c129a11a Avoid all needless memory copies when computing consensus diffs.
Previously, we operated on smartlists of NUL-terminated strings,
which required us to copy both inputs to produce the NUL-terminated
strings.  Then we copied parts of _those_ inputs to produce an
output smartlist of NUL-terminated strings.  And finally, we
concatenated everything into a final resulting string.

This implementation, instead, uses a pointer-and-extent pattern to
represent each line as a pointer into the original inputs and a
length.  These line objects are then added by reference into the
output. No actual bytes are copied from the original strings until
we finally concatenate the final result together.

Bookkeeping structures and newly allocated strings (like ed
commands) are allocated inside a memarea, to avoid needless mallocs
or complicated should-I-free-this-or-not bookkeeping.

In my measurements, this improves CPU performance by something like
18%.  The memory savings should be much, much higher.
2017-03-16 14:42:56 -04:00
Nick Mathewson
52fa6bb947 Swap memory allocation strategy for lists of lines for diffs
Now we use a single allocation block for all the lines, rather than
calling strdup on them one at a time.  This should help performance
a tiny bit.
2017-03-16 14:42:56 -04:00
Nick Mathewson
b36e311eab Fill in the missing documentation on the new functions 2017-03-16 14:42:56 -04:00
Nick Mathewson
12d428aaff Prop140: Fix a crash bug.
Found while fuzzing: this could occur if we tried to copy a
nonexistent "line 0" while applying a diff.
2017-03-16 14:42:56 -04:00
Nick Mathewson
dd92579b63 Add fuzzers for consensus diff backend code
This takes two fuzzers: one which generates a diff and makes sure it
works, and one which applies a diff.

So far, they won't crash, but there's a bug in my
string-manipulation code someplace that I'm having to work around,
related to the case where you have a blank line at the end of a
file, or where you diff a file with itself.
2017-03-16 14:42:55 -04:00
Nick Mathewson
653c6d129e Make consensus diff sha3 operations mockable.
(We'll want this for fuzzing)
2017-03-16 14:40:33 -04:00
Nick Mathewson
6a36e5ff3b String-based API for consensus diffs.
Also, add very strict split/join functions, and totally forbid
nonempty files that end with somethig besides a newline.  This
change is necessary to ensure that diff/apply are actually reliable
inverse operations.
2017-03-16 14:39:54 -04:00
Nick Mathewson
eff9fbd17d Fix an abstraction violation.
Don't alias the insides of smartlist_t; that way lies madness.
2017-03-16 14:38:29 -04:00
Nick Mathewson
69b3e11e59 Use "const" in consdiff.[ch] 2017-03-16 14:38:29 -04:00
Nick Mathewson
3647751c2a prop140: Use sha3-256, not sha2-256
This is a protocol update from recent prop140 changes.

Also, per #21673, we need to check the entire document, including
signatures.
2017-03-16 14:38:29 -04:00
Nick Mathewson
d6594bfa8c Add a couple more cases to reach 100% coverage. 2017-03-16 14:38:29 -04:00