Commit Graph

2445 Commits

Author SHA1 Message Date
Nick Mathewson
09b3cb0d72 Whoops; I broke check-spaces. 2017-05-19 16:02:58 -04:00
Nick Mathewson
13034e1574 Try another approach to fixing the bug in the #1922 test.
This isn't elegant, but it seems to be the best way around all of
the issues involved in escaping and quoting that we've gotten into
over the years.
2017-05-19 15:50:29 -04:00
Nick Mathewson
19615bce64 mingw fix: avoid "unused var" warning. 2017-05-19 15:50:12 -04:00
Nick Mathewson
8cb08f716d fix a coverity resource leak warning 2017-05-19 15:20:57 -04:00
Nick Mathewson
3628efe29c Try to fix windows config/include_path_syntax test
It was trying to do %include "foo\", which won't work.  It has to be
%include "foo\\".
2017-05-19 09:55:22 -04:00
Nick Mathewson
dcfed8c1c8 tt_* macros can "goto done;" so define any freeable things before them. 2017-05-19 08:55:14 -04:00
Nick Mathewson
69ef94820b Merge branch 'add_rust_squashed' 2017-05-19 08:47:18 -04:00
Sebastian Hahn
70c067102b Allow Rust build using locally supplied crates or crates.io
This adds a couple of configure commands to control whether we're
requiring all dependencies to be available locally (default) or not
(--enable-cargo-online-mode). When building from a tarball, we require
the RUST_DEPENDENCIES variable to point to the local repository of
crates. This also adds src/ext/rust as a git submodule that contains
such a local repository for easy setup.
2017-05-19 08:47:11 -04:00
Sebastian Hahn
d6f9a4f11a cargo-online-mode configure argument
Passing --enable-cargo-online-mode during configure allows cargo to make
network requests while building Tor or running tests. If this flag is
not supplied, the dependencies need to be available in the form of a
local mirror.
2017-05-19 08:47:11 -04:00
Sebastian Hahn
f0516ed348 Run cargo test during make check 2017-05-19 08:47:11 -04:00
Sebastian Hahn
f8ef7c65d1 Add some Rust utility functions and print support
This gives an indication in the log that Tor was built with Rust
support, as well as laying some groundwork for further string-returning
APIs to be converted to Rust
2017-05-19 08:47:10 -04:00
Nick Mathewson
92d335b3dc Merge remote-tracking branch 'jigsaw/torrc-dir-fix-1922_squashed2' 2017-05-19 08:46:13 -04:00
Daniel Pinto
ba3a5f82f1 Add support for %include funcionality on torrc #1922
config_get_lines is now split into two functions:
 - config_get_lines which is the same as before we had %include
 - config_get_lines_include which actually processes %include
2017-05-18 23:44:16 +01:00
Nick Mathewson
e1b3c1bbec Test fix: always set address in new_dir_conn() 2017-05-15 18:00:38 -04:00
Nick Mathewson
afef059795 Merge remote-tracking branch 'public/prop140_aftermath_cfg' 2017-05-15 17:26:47 -04:00
Nick Mathewson
dae1242532 Merge branch 'ahf_bugs_21667_2_squashed' 2017-05-15 17:22:12 -04:00
Alexander Færøy
9e3f304113 Fix dir_handle_get/... test-cases for prop#278 support.
See: https://bugs.torproject.org/21667
2017-05-15 17:21:55 -04:00
Nick Mathewson
65d9408448 dir_handle_get: repair two test cases, note the fixes for 3 others 2017-05-15 17:21:55 -04:00
Nick Mathewson
85f75d617c Add hs_test_helpers.h to noinst_HEADERS 2017-05-15 11:45:20 -04:00
Nick Mathewson
db370bb8a8
Test fix: expect old consensuses to be deleted if not deflate-compressed 2017-05-12 17:45:44 +02:00
Taylor Yu
61a367cadd Untangle cleanup logic in test_hs_intropoint.c
Cleanup logic in test_intro_point_registration() invoked tt_assert()
in a way that could cause it to jump backward into the cleanup code if
the assertion failed, causing Coverity to see a double free (CID
1397192).  Move the tt_assert() calls into a helper function having
the well-defined task of testing hs_circuitmap_free_all().
Fixes #22231.
2017-05-11 13:06:45 -04:00
Nick Mathewson
6390a0c3b6 Merge branch 'ticket21871_031_03_squashed' 2017-05-11 08:33:32 -04:00
David Goulet
ae1d4cfdad prop224: Change encryption keys descriptor encoding
A descriptor only contains the curve25519 public key in the enc-key field so
the private key should not be in that data structure. The service data
structures will have access to the full keypair (#20657).

Furthermore, ticket #21871 has highlighted an issue in the proposal 224 about
the encryption key and legacy key being mutually exclusive. This is very wrong
and this commit fixes the code to follow the change to the proposal of that
ticket.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-11 08:33:26 -04:00
Nick Mathewson
dc9274dec3 Merge remote-tracking branch 'argonblue/bug22177' 2017-05-11 07:41:21 -04:00
Taylor Yu
3156392e9b Test config line parsing errors
Add tests for lower-level syntax errors in config file lines.
Fixes #22177.
2017-05-10 16:51:40 -04:00
Taylor Yu
2bf4263800 Resurrect dead code in test_channelpadding.c
A for-loop in test_channelpadding_timers() would never run because it
was trying to increment a counter up to CHANNELS_TO_TEST/3 after an
earlier block already incremented it to CHANNELS_TO_TEST/2.

Fixes #22221, CID 1405983.
2017-05-10 13:57:18 -04:00
Nick Mathewson
d76cffda60 Merge remote-tracking branch 'public/my-family-list-fix-4498' 2017-05-10 11:12:24 -04:00
Nick Mathewson
5dab99d6a8 Fix compilation on libevent2-only systems
Patch from rubiate; fixes bug 22219.  Remember, we don't support
libevent1 any more.
2017-05-10 11:08:49 -04:00
Nick Mathewson
95fa7d1cf8 In channelpadding tests that touch libevent, call event_reinit().
This is necessary to avoid crashes and test failures on kevent-based
systems.

Fixes bug 22209; bug not in any released Tor.
2017-05-10 11:01:13 -04:00
Nick Mathewson
2a1013948d Merge branch 'dgoulet_ticket22060_031_01_squashed' 2017-05-09 10:32:21 -04:00
David Goulet
7f95ef6e66 config: Remove {Control,DNS,Dir,Socks,Trans,NATD,OR}ListenAddress option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
039e2a24da config: Remove TLSECGroup option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
d52a1e2faa config: Remove AllowSingleHopExits option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:52 -04:00
David Goulet
2b9823b310 config: Remove AllowInvalidNodes option
Deprecated in 0.2.9.2-alpha, this commits changes it as OBSOLETE() and cleans
up the code associated with it.

Partially fixes #22060

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:51 -04:00
David Goulet
0958e3b208 test: Remove HS decode valid intro point test
The descriptor fields can't be validated properly during encoding because they
are signed by a descriptor signing key that we don't have in the unit test.

Removing the test case for now but ultimately we need an independent
implementation that can encode descriptor and test our decoding functions with
that.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:37 -04:00
David Goulet
0cc18ef64c test: Move duplicate HS test code and unify it
Create the hs_test_helpers.{c|h} files that contains helper functions to
create introduction point, descriptor and compare descriptor.

Used by both the hs cache and hs descriptor tests. Unify them to avoid code
duplication.

Also, this commit fixes the usage of the signing key that was wrongly used
when creating a cross signed certificate.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-09 10:30:37 -04:00
Nick Mathewson
2f714d60a4 Minor cleanups in test_config.
Use a more standard indentation style for the end block; remove
redundant if() around tor_free.
2017-05-09 08:20:14 -04:00
Nick Mathewson
b8abedfeee Split MyFamily into user-specified version and normalized version
This change prevents a no-longer-supported behavior where we change
options that would later be written back to torrc with a SAVECONF.

Also, use the "Pointer to final pointer" trick to build the
normalized list, to avoid special-casing the first element.
2017-05-09 08:19:26 -04:00
Nick Mathewson
674c1d2594 Fix an implicit conversion warning 2017-05-09 07:26:14 -04:00
Nick Mathewson
2e4f3b36bd clang-i386: use house style for public-when-testing variables
This fixes a warning from jenkins.
2017-05-08 15:38:05 -04:00
Nick Mathewson
4d6c79d1de Fix some clang-i386 warnings in master. 2017-05-08 15:34:37 -04:00
Nick Mathewson
4d30dde156 Merge branch 'netflow_padding-v6-rebased2-squashed' 2017-05-08 13:54:59 -04:00
Mike Perry
9f8e462c89 Fix some channelpadding test issues.
asan was finding an alignment issue with a cast, so set the field in the
trunnel struct and then encode it instead. Also, enable log capture and
verification.
2017-05-08 13:49:23 -04:00
Mike Perry
687a85950a Cache netflow-related consensus parameters.
Checking all of these parameter lists for every single connection every second
seems like it could be an expensive waste.

Updating globally cached versions when there is a new consensus will still
allow us to apply consensus parameter updates to all existing connections
immediately.
2017-05-08 13:49:23 -04:00
Mike Perry
ae4d8c9c74 Fix a breakage in test_options.c.
IMO, these tests should be calling options_init() to properly set everything
to default values, but when that is done, about a dozen tests fail. Setting
the one default value that broke the tests for my branch. Sorry for being
lame.
2017-05-08 13:49:22 -04:00
Mike Perry
20a3d4efac Remove a PredictedPortsRelevantTime test.
The option was deprecated by bug #17592.
2017-05-08 13:49:22 -04:00
Mike Perry
d5a151a067 Bug 17592: Clean up connection timeout logic.
This unifies CircuitIdleTimeout and PredictedCircsRelevanceTime into a single
option, and randomizes it.

It also gives us control over the default value as well as relay-to-relay
connection lifespan through the consensus.

Conflicts:
	src/or/circuituse.c
	src/or/config.c
	src/or/main.c
	src/test/testing_common.c
2017-05-08 13:49:22 -04:00
Mike Perry
b0e92634d8 Netflow record collapsing defense.
This defense will cause Cisco, Juniper, Fortinet, and other routers operating
in the default configuration to collapse netflow records that would normally
be split due to the 15 second flow idle timeout.

Collapsing these records should greatly reduce the utility of default netflow
data for correlation attacks, since all client-side records should become 30
minute chunks of total bytes sent/received, rather than creating multiple
separate records for every webpage load/ssh command interaction/XMPP chat/whatever
else happens to be inactive for more than 15 seconds.

The defense adds consensus parameters to govern the range of timeout values
for sending padding packets, as well as for keeping connections open.

The defense only sends padding when connections are otherwise inactive, and it
does not pad connections used solely for directory traffic at all. By default
it also doesn't pad inter-relay connections.

Statistics on the total padding in the last 24 hours are exported to the
extra-info descriptors.
2017-05-08 13:49:21 -04:00
Alexander Færøy
7344a4fdc6
Remove dead code in test_options_validate_impl().
Dead branch found by Coverity in CID #1405875.
2017-05-05 17:07:18 +02:00
Nick Mathewson
c276c10d3b Turn consdiffmgr.cache_max_age_hours into a parameter 2017-05-05 09:11:06 -04:00
Nick Mathewson
112286338b Store the sha3 of a networkstatus as part of the networkstatus_t
Also store it in the cached_dir_t.
2017-05-04 08:37:41 -04:00
Nick Mathewson
24ba1864d8 Merge branch 'ticket22143_squashed' 2017-05-04 08:36:59 -04:00
Nick Mathewson
df2bcaeb4a Add a test for $ with non-delete commands. 2017-05-04 08:36:50 -04:00
Nick Mathewson
87f6979a03 Remove some unused digests from test_consdiffmgr.c
These were unused before I started working on #22143 -- I just found
them while I was lookinging for digests to update.
2017-05-04 08:36:50 -04:00
Nick Mathewson
3af9704e45 bug#22143/prop#140: in consdiffmgr, store and use digest-as-signed
We need to index diffs by the digest-as-signed of their source
consensus, so that we can find them even from consensuses whose
signatures are encoded differently.
2017-05-04 08:36:50 -04:00
Nick Mathewson
c8baa9b783 bug#22143/prop#140: Use <n>,$d commands in diffs to remove signatures
In this patch I add support for "delete through end of file" in our
ed diff handler, and generate our diffs so that they remove
everything after in the consensus after the signatures begin.
2017-05-04 08:36:50 -04:00
Taylor Yu
38a13b91a8 Fix additional leaks in #22103 tests
test_options_validate_impl() incorrectly executed subsequent phases of
config parsing and validation after an expected error.  This caused
msg to leak when those later phases (which would likely produce errors
as well) overwrote it.
2017-05-03 13:47:06 -04:00
Nick Mathewson
5acddbbbf7 bug#22143/prop#140: identify input diffs by their digest-as-signed
See may 3 changes to prop140 for more background.
2017-05-03 13:09:08 -04:00
Nick Mathewson
77d9dd39c3 Merge remote-tracking branch 'argonblue/bug22103_fixtest' 2017-05-03 09:33:37 -04:00
David Goulet
90b840af60 control: Fix NULL pointer access in HS desc event
This was introduced 90562fc23a adding a code
path where we pass a NULL pointer for the HSDir fingerprint to the control
event subsystem. The HS desc failed function wasn't handling properly that
pointer for a NULL value.

Two unit tests are also added in this commit to make sure we handle properly
the case of a NULL hsdir fingerprint and a NULL content as well.

Fixes #22138

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-05-03 09:26:17 -04:00
Taylor Yu
7b64f1773d Fix memory management for #22103 tests
Code movement in the commit introducings tests for #22103 uncovered a
latent memory management bug.

Refactor the log message checking from test_options_checkmsgs() into a
helper test_options_checklog().  This avoids a memory leak (and
possible double-free) in a test failure condition.

Don't reuse variables (especially pointers to allocated memory!) for
multiple unrelated purposes.

Fixes CID 1405778.
2017-05-03 08:59:29 -04:00
Taylor Yu
a91f948cca Add tests for #22103
Also factor out the error message comparisions from
test_options_validate_impl() into a separate function so it can check
for error messages in different phases of config parsing.
2017-05-02 16:13:14 -04:00
Nick Mathewson
584ab1f29b Merge branch 'compress_none_v2_squashed' 2017-05-02 08:33:35 -04:00
Nick Mathewson
3836d9481f Add unit tests for the NO_METHOD compressor
These required some special-casing, since some of the assumption
about real compression algorithms don't actually hold for the
identity transform.  Specifically, we had assumed:

  - compression functions typically change the lengths of their
    inputs.
  - decompression functions can detect truncated inputs
  - compression functions have detectable headers

None of those is true for the identity transformation.
2017-05-02 08:31:32 -04:00
Nick Mathewson
c486ef57a3 Rename x-lzma to x-tor-lzma
We shouldn't call it lzma, because we are imposing a limit on the
memory needed for decoding.
2017-05-01 15:31:28 -04:00
Sebastian Hahn
915fa39d0f Add --enable-rust configure switch
Introduce a way to optionally enable Rust integration for our builds. No
actual Rust code is added yet and specifying the flag has no effect
other than failing the build if rustc and cargo are unavailable.
2017-04-29 08:55:57 +02:00
Nick Mathewson
00a12337ff Merge branches 'consdiffmgr_orig_squashed' and 'actually_compute_diffs_squashed' 2017-04-27 21:43:06 -04:00
Nick Mathewson
1e1581a24e Pre-compress consensus diffs with supported consensus methods. 2017-04-27 21:40:46 -04:00
Nick Mathewson
a1172b6774 Store archived consensuses compressed on disk.
I'm just using gzip compression now, for performance.
2017-04-27 21:40:46 -04:00
Nick Mathewson
920475f293 New force-delete option on consensus_cache_delete_pending()
If we're out of file space in the storage directory, we'll need to
get rid of old files fast.
2017-04-27 21:40:13 -04:00
Nick Mathewson
10a4f9cd07 Merge branch 'parse_accept_encoding' 2017-04-27 11:31:31 -04:00
Nick Mathewson
49deb1e1b8 Document and test nul-terminating behavior of tor_uncompress()
We added this as a safety feature, but there are a few places in the
code that actually depend on it.
2017-04-27 10:59:48 -04:00
Nick Mathewson
199e61feb5 whitespace fix 2017-04-27 10:55:39 -04:00
Nick Mathewson
ba9f235e17 Re-enable the partial-input test for zstd.
There were two issues here: first, zstd didn't exhibit the right
behavior unless it got a very large input.  That's fine.

The second issue was a genuine bug, fixed by 39cfaba9e2.
2017-04-27 10:43:38 -04:00
Nick Mathewson
166aa8d741 Have a separate entry point for each compresion test 2017-04-27 10:25:52 -04:00
Nick Mathewson
36e62ae7e2 Merge remote-tracking branch 'ahf/bugs/22085' 2017-04-27 10:12:48 -04:00
Nick Mathewson
33a2fd065d Merge branch 'dirreq' 2017-04-27 10:08:32 -04:00
Alexander Færøy
e0a3819dcc
Test support for multiple compression backends in the buffer code.
This patch refactors the buffer compression tests to support multiple
comprssion backends.

See: https://bugs.torproject.org/22085
2017-04-27 15:51:14 +02:00
Alexander Færøy
22e6ad6f26
Clean up mentions of 'zlib' and rename the mentions to 'compressed'.
This patch cleans up in various places where 'zlib' is mentioned.
2017-04-27 15:51:14 +02:00
Alexander Færøy
7bececbd69
Refactor compression tests into a single test.
This patch refactors our compression tests such that deflate, gzip,
lzma, and zstd are all tested using the same code.

Additionally we use run-time checks to see if the given compression
method is supported instead of using HAVE_LZMA and HAVE_ZSTD.

See: https://bugs.torproject.org/22085
2017-04-27 15:51:14 +02:00
Nick Mathewson
fec3050ea9 Tests for parse_accept_encoding 2017-04-25 19:01:05 -04:00
Nick Mathewson
49868340f7 Merge branch 'unified_compress_squashed' 2017-04-25 10:51:13 -04:00
Nick Mathewson
880fb3e3a9 Combine all *compress/*uncompress backend function into one
Since we have a streaming API for each compression backend, we don't
need a non-streaming API for each: we can build a common
non-streaming API at the front-end.
2017-04-25 10:50:50 -04:00
Nick Mathewson
232c9e14a8 Merge branch 'atomic_counters' 2017-04-25 10:46:23 -04:00
David Goulet
cb8ac1f331 trace: Add a basic event-tracing infrastructure.
This commit adds the src/trace directory containing the basics for our tracing
subsystem. It is not used in the code base. The "src/trace/debug.h" file
contains an example on how we can map our tor trace events to log_debug().

The tracing subsystem can only be enabled by tracing framework at compile
time. This commit introduces the "--enable-tracing-debug" option that will
make all "tor_trace()" function be maped to "log_debug()".

Closes #13802

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-25 10:37:31 -04:00
Nick Mathewson
4266ec766a Use atomic counters for compressor allocation. 2017-04-25 10:29:07 -04:00
Nick Mathewson
ba405f86bf Merge branch 'ahf_prop278_21662_squashed' 2017-04-25 08:12:59 -04:00
Alexander Færøy
1c77d8690c Add function to check if a given compression method is supported.
This patch adds support for checking if a given `compress_method_t` is
supported by the currently running Tor instance using
`tor_compress_supports_method()`.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
380736d045 Add Zstandard support.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
ce1feae9d9 Add --enable-zstd to our configure script.
This patch adds support for enabling support for Zstandard to our configure
script. By default, the --enable-zstd option is set to "auto" which means if
libzstd is available we'll build Tor with Zstandard support.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
bf1c07cb07 Add LZMA support.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:06 -04:00
Alexander Færøy
157af1d26e Add --enable-lzma to our configure script.
This patch adds support for enabling support for LZMA to our configure
script. By default, the --enable-lzma option is set to "auto" which
means if liblzma is available we'll build Tor with LZMA support.

See: https://bugs.torproject.org/21662
2017-04-25 08:06:02 -04:00
Alexander Færøy
04583df452 Rename the torgzip module to compress.
See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Nick Mathewson
db95a6e171 Fix implicit conversion warnings in the period_num tests 2017-04-24 15:38:16 -04:00
Nick Mathewson
4cefda85e4 Merge remote-tracking branch 'dgoulet/ticket21980_031_01' 2017-04-24 12:42:11 -04:00
Nick Mathewson
d949589c09 Fix some leaks in the consdiffmgr tests 2017-04-24 11:45:13 -04:00
Nick Mathewson
b7567a6282 Merge branch 'consdiffmgr_squashed' 2017-04-24 11:02:22 -04:00
Nick Mathewson
eb14faa0c1 Remove a checklist item that was already tested
The item referred to the cdm_ht_set_status() case where the item was
not already in the hashtable.  But that already happens naturally
when we scan the directory on startup... and we already have a test
for that.
2017-04-24 11:01:40 -04:00
Nick Mathewson
af86895581 consdiffmgr test: do not launch a diff task that is already pending 2017-04-24 11:01:40 -04:00
Nick Mathewson
2e9e2d023b consdiffmgr: tests for consdiffmgr_validate() 2017-04-24 11:01:40 -04:00