Commit Graph

3899 Commits

Author SHA1 Message Date
Nick Mathewson
c292e505ff Merge remote-tracking branch 'tor-github/pr/539' 2018-11-26 17:22:37 -05:00
Nick Mathewson
7d8e0cc9ab Merge branch 'dormant_v2_squashed' 2018-11-26 16:33:31 -05:00
Nick Mathewson
02843c4a4e Test for check_network_participation_callback() 2018-11-26 16:32:40 -05:00
Nick Mathewson
55512ef022 Test netstatus.c tracking of user participation status 2018-11-26 16:32:40 -05:00
Nick Mathewson
3743f79695 Add options to control dormant-client feature.
The DormantClientTimeout option controls how long Tor will wait before
going dormant.  It also provides a way to disable the feature by setting
DormantClientTimeout to e.g. "50 years".

The DormantTimeoutDisabledByIdleStreams option controls whether open but
inactive streams count as "client activity".  To implement it, I had to
make it so that reading or writing on a client stream *always* counts as
activity.

Closes ticket 28429.
2018-11-26 16:32:40 -05:00
Nick Mathewson
05dee063c8 Emit router families in canonical form
This patch has routers use the same canonicalization logic as
authorities when encoding their family lists.  Additionally, they
now warn if any router in their list is given by nickname, since
that's error-prone.

This patch also adds some long-overdue tests for family formatting.
2018-11-24 16:35:58 -05:00
Nick Mathewson
0a0c612b79 Add a consensus method in which md families get canonicalized.
Implements prop298. Closes ticket 28266.
2018-11-24 12:01:09 -05:00
Nick Mathewson
d29e3a02d5 Add a function to canonicalize nodefamilies per prop298
This is the same as the regular canonical nodefamily format, except
that unrecognized elements are preserved.
2018-11-24 10:53:38 -05:00
Nick Mathewson
0e9a963b6b Revise nodefamily.c to match proposal 298
Prop298 says that family entries should be formatted with
$hexids in uppercase, nicknames in lower case, $hexid~names
truncated, and everything sorted lexically.  These changes implement
that ordering for nodefamily.c.

We don't _strictly speaking_ need to nodefamily.c formatting use
this for prop298 microdesc generation, but it seems silly to have
two separate canonicalization algorithms.
2018-11-24 10:30:15 -05:00
Taylor R Campbell
997a8b0ca7 Create a temporary directory for tor's DataDirectory in test_rebind.
Fixes #28562.

While here, put the argument count test and usage message _before_ we
attempt to read from sys.argv.
2018-11-23 12:51:39 -05:00
teor
805f75182a Entry Nodes: Test on reasonably live consensuses
As well as live consensuses.

Tests for 24661.
2018-11-22 17:47:12 +10:00
teor
390112d07e Entry Nodes: refactor tests to use macros
Part of 24661.
2018-11-22 17:46:52 +10:00
teor
3741f9e524
Fix a comment typo in test_hs_common.c 2018-11-22 16:56:06 +10:00
teor
cebc39bcd5
Test: make unit tests use a reasonably live consensus
Cleanup after 24661.
2018-11-22 16:54:46 +10:00
teor
ffc7b81b5d
Test: Fix memory leaks and missing unmocks in entry guard tests
test_entry_guard_outdated_dirserver_exclusion leaks memory, and is
missing some unmocks.

Fixes 28554; bugfix on 0.3.0.1-alpha.
2018-11-22 16:42:32 +10:00
Nick Mathewson
48b08f0592 Merge branch 'ticket27359_v2_squashed' 2018-11-19 08:26:49 -05:00
Nick Mathewson
0e762c0cf5 Test new functions in nodelist.c 2018-11-19 08:26:10 -05:00
Nick Mathewson
aa1d767e6b Aim for 100% test coverage on nodefamily.c 2018-11-19 08:26:10 -05:00
Nick Mathewson
426c9561c5 Use nodefamily_t in microdescriptors.
Closes ticket 27359.
2018-11-19 08:26:10 -05:00
Nick Mathewson
83be4d2bbd Backend for compact node-family representation.
This representation is meant to save memory in microdescriptors --
we can't use it in routerinfo_t yet, since those families need to be
encoded losslessly for directory voting to work.

This representation saves memory in three ways:
   1. It uses only one allocation per family.  (The old way used a
      smartlist (2 allocs) plus one strdup per entry.)
   2. It stores identity digests in binary, not hex.
   3. It keeps families in a canonical format, memoizes, and
      reference-counts them.

Part of #27359.
2018-11-19 08:26:10 -05:00
Nick Mathewson
35558c39dd Merge remote-tracking branch 'dgoulet/ticket27471_035_02' into maint-0.3.5 2018-11-16 08:57:56 -05:00
Nick Mathewson
80a6228aac Merge branch 'bug25573-034-typefix' into maint-0.3.4 2018-11-15 16:58:16 -05:00
Nick Mathewson
15e752e6b1 Merge branch 'maint-0.3.3' into maint-0.3.4 2018-11-15 16:54:56 -05:00
Nick Mathewson
cbe04d4550 Merge branch 'maint-0.2.9' into maint-0.3.3 2018-11-15 16:54:16 -05:00
Nick Mathewson
8569166c70 Merge remote-tracking branch 'public/bug24104_029_squashed' into maint-0.2.9
Resolved conflicts with the 26269 fix in 015fcd0e11.
2018-11-15 16:43:50 -05:00
Nick Mathewson
2c15b65381 Make the NET_PARTICIPANT role dependent on user activity
This patch implements all of 28337, except for the part where we
turn off the role if we've been idle for a long time.
2018-11-15 11:17:22 -05:00
Nick Mathewson
a0380b705d Move control_per_second_events() into a callback with its own role
Part of making extra-dormant mode work; closes ticket 28421.
2018-11-15 11:17:22 -05:00
Nick Mathewson
b9a88bd53a Add new "ALL" and "NET_PARTICIPANT" roles for periodic events
The previous "ALL" role was the OR of a bunch of other roles,
which is a mistake: it's better if "ALL" means "all".

The "NET_PARTICIPANT" role refers to the anything that is actively
building circuits, downloading directory information, and
participating in the Tor network.  For now, it is set to
!net_is_disabled(), but we're going to use it to implement a new
"extra dormant mode".

Closes ticket 28336.
2018-11-15 11:17:22 -05:00
Nick Mathewson
f6b8c7da66 Move buffers.c out of lib/containers to resolve a circularity. 2018-11-14 16:07:03 -05:00
Nick Mathewson
bf4f55a13d Merge branch 'subsystems' 2018-11-09 15:01:49 -05:00
Nick Mathewson
c7b6ed1d80 Merge remote-tracking branch 'tor-github/pr/271' 2018-11-07 11:21:13 -05:00
Nick Mathewson
8020d6fb05 Merge remote-tracking branch 'tor-github/pr/464' 2018-11-06 15:33:25 -05:00
Nick Mathewson
019a044e5e Turn "compress" into a subsystem. 2018-11-05 09:22:02 -05:00
Nick Mathewson
207253df8d Move monotonic time setup into a subsystem 2018-11-05 09:22:02 -05:00
Nick Mathewson
cad61f0f6d Move prefork, postfork, and thread-exit hooks into subsys
So far, crypto is the only module that uses them, but others are
likely to do so in the future.
2018-11-05 09:22:02 -05:00
Nick Mathewson
50436ccea4 Add crypto module as a subsystem. 2018-11-05 09:22:02 -05:00
Nick Mathewson
cfe5b35edb Move networking startup/cleanup logic into a subsystem. 2018-11-05 09:22:02 -05:00
Nick Mathewson
05b54f6a6a Use subsystems manager for subsystems used in tests. 2018-11-05 09:22:02 -05:00
Nick Mathewson
6e7ff8cba0 Move the code that knows our tor version into a lowest-level lib 2018-11-05 09:22:02 -05:00
Neel Chauhan
45b28167d7 In count_acceptable_nodes(), count direct and indirect nodes with node_has_preferred_descriptor() 2018-11-05 08:26:02 -05:00
Nick Mathewson
3e2423d19b Update address tests to avoid offending coverity. 2018-11-02 13:29:24 -04:00
Nick Mathewson
f0bd6c2c9d Merge branch 'maint-0.3.5' 2018-11-02 13:14:37 -04:00
Nick Mathewson
996f24fcd3 Merge branch 'bug27968_squashed' into maint-0.3.5 2018-11-02 13:14:32 -04:00
teor
e36e4a9671 Sort the imports in test_rebind.py
Cleanup after #27968.
2018-11-02 13:14:23 -04:00
teor
a02d6c560d Make test_rebind.py timeout when waiting for a log message
Closes #27968.
2018-11-02 13:14:23 -04:00
teor
cd674a10ad Refactor test_rebind.py to consistently print FAIL on failure
Part of #27968.
2018-11-02 13:14:23 -04:00
teor
8f43b8fb47 Avoid a race condition in test_rebind.py
If tor terminates due to SIGNAL HALT before test_rebind.py calls
tor_process.terminate(), an OSError 3 (no such process) is thrown.

Fixes part of bug 27968 on 0.3.5.1-alpha.
2018-11-02 13:14:23 -04:00
George Kadianakis
da716fdfbb Add tests for the string_is_utf8_no_bom() function. 2018-11-01 12:55:04 +02:00
Nick Mathewson
383b44553c Merge branch 'ticket24393_036_01_squashed' 2018-10-31 11:07:15 -04:00
Neel Chauhan
e9adc200aa Add test for nodes_in_same_family() 2018-10-31 11:07:03 -04:00
Nick Mathewson
0d124488f8 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-10-31 11:02:58 -04:00
Nick Mathewson
158db532eb Merge branch 'maint-0.3.3' into maint-0.3.4 2018-10-31 11:01:41 -04:00
Nick Mathewson
2a4506776e Merge branch 'maint-0.3.5' 2018-10-31 11:01:41 -04:00
Nick Mathewson
59043665f8 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-10-31 11:01:41 -04:00
Nick Mathewson
bcc1a71808 Merge branch 'maint-0.2.9' into maint-0.3.3 2018-10-31 11:01:40 -04:00
Nick Mathewson
988d4903a3 Merge branch 'networkstatus_mmap' into networkstatus_mmap_merge 2018-10-31 09:04:12 -04:00
David Goulet
aa1ae1343a Merge branch 'maint-0.3.5' 2018-10-30 11:44:14 -04:00
David Goulet
488969fe9c Merge branch 'tor-github/pr/438' into maint-0.3.5 2018-10-30 11:43:54 -04:00
David Goulet
cdb065d6b2 Merge branch 'maint-0.3.5' 2018-10-30 10:55:10 -04:00
Neel Chauhan
cd9914d9f9 Add test for HiddenServiceAuthorizeClient and v3 onion services 2018-10-30 10:49:03 -04:00
Nick Mathewson
30d853a906 Merge remote-tracking branch 'tor-github/pr/425' 2018-10-30 09:16:55 -04:00
Nick Mathewson
084924360a Merge branch 'ticket23082_squashed' 2018-10-30 09:05:42 -04:00
rl1987
742cd15649 Move a check for trailing colon to tor_inet_pton()
That way, string_is_valid_ipv6_address() can benefit from it
2018-10-30 09:05:11 -04:00
Nick Mathewson
c19bb4d62e Merge branch 'maint-0.3.5' 2018-10-30 08:45:49 -04:00
Nick Mathewson
262b0fe7a0 Merge remote-tracking branch 'tor-github/pr/381' into maint-0.3.5 2018-10-30 08:39:57 -04:00
Alex Crichton
8285784966 Only pass -C default-linker-libraries with sanitizers
This'll help retain test compatibility until 1.31.0 is released!
2018-10-29 10:00:23 -07:00
Nick Mathewson
52a82bc53c Add a couple more checks to test_parsecommon.c
These checks should make coverity stop giving us a "dereference
before null check" warning here.
2018-10-25 09:24:30 -04:00
Nick Mathewson
594140574e Fix remaining cases of using consensus without a len parameter.
(Thanks to cyberpunks for noting two of them!)
2018-10-24 11:06:34 -04:00
Nick Mathewson
c0bd800d26 Re-alphabetize the list of tests in tests.[ch] 2018-10-24 09:09:40 -04:00
Nick Mathewson
c6d3bebb1e Merge remote-tracking branch 'tor-github/pr/409' 2018-10-24 09:06:29 -04:00
Nick Mathewson
05d65cb067 Merge branch 'maint-0.3.5' 2018-10-23 19:22:05 -04:00
Nick Mathewson
289a7dbac3 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-10-23 19:21:44 -04:00
Nick Mathewson
7edc594ee7 Merge branch 'maint-0.3.3' into maint-0.3.4 2018-10-23 19:18:23 -04:00
Nick Mathewson
169bc670dd Merge remote-tracking branch 'onionk/rust-allsupportednull1' into maint-0.3.3 2018-10-23 19:17:48 -04:00
George Kadianakis
29c194e022 Func rename: Make it clear that update_all_descriptors() does intro points.
With the new refresh_service_descriptor() function we had both
refresh_service_descriptor() and update_service_descriptor() which is basically
the same thing.

This commit renames update_service_descriptor() to
update_service_descriptor_intro_points() to make it clear it's not a generic
refresh and it's only about intro points.

Commit changes no code.
2018-10-23 16:43:54 +03:00
teor
939657771c
Treat backtrace test failures as expected on most BSD-derived systems
Treat backtrace test failures as expected on NetBSD, OpenBSD, and
macOS/Darwin, until we solve bug 17808.

(FreeBSD failures have been treated as expected since 18204 in 0.2.8.)

Fixes bug 27948; bugfix on 0.2.5.2-alpha.
2018-10-23 11:21:10 +10:00
rl1987
98cef6807e Exclude test and a supporting function when evdns_base_get_nameserver_addr() is not available 2018-10-20 20:34:08 +03:00
rl1987
d827902cb1 Unit test for DNS fallback in configure_nameservers 2018-10-20 20:34:08 +03:00
David Goulet
8b2e72106a test: Add test for closing intro circuits when storing a new descriptor
This is client side and related to 27471 for previous commit that fixes this
issue.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-10-18 12:56:51 -04:00
Nick Mathewson
d1eac7830f Merge branch 'maint-0.3.3' into maint-0.3.4 2018-10-18 09:12:58 -04:00
Taylor Yu
7f6c0fce46 Merge branch 'bug27800-034' into bug27800-035 2018-10-17 16:00:11 -05:00
Taylor Yu
4e7f65ee5d Merge branch 'bug27800-033' into tor-034 2018-10-17 15:42:58 -05:00
Taylor Yu
93fd924bdb Log more info for duplicate ed25519 IDs
Occasionally, key pinning doesn't catch a relay that shares an ed25519
ID with another relay.  Log the identity fingerprints and the shared
ed25519 ID when this happens, instead of making a BUG() warning.

Fixes bug 27800; bugfix on 0.3.2.1-alpha.
2018-10-17 15:39:55 -05:00
rl1987
7fd82a4570 One testcase for annotation handling in tokenize_string() 2018-10-16 18:04:54 +03:00
rl1987
f10d664fd1 Test AT_END checking in tokenize_string() 2018-10-16 18:04:54 +03:00
rl1987
1a4edceee9 Add testcase for too many elements in tokenize_string() input 2018-10-16 18:04:54 +03:00
rl1987
7c8bf2f7c7 Add testcase for too few elements in tokenize_string() input 2018-10-16 18:04:54 +03:00
rl1987
78c446af7e Unit-test multiple line parsing with tokenize_string() 2018-10-16 18:04:54 +03:00
rl1987
81731a290d Unit-test some error conditions in get_next_token() 2018-10-16 18:04:54 +03:00
rl1987
6c5ba2662a Test object parsing in get_next_token() 2018-10-16 18:04:54 +03:00
rl1987
38a7033d33 Fix memory management in test_parsecommon_get_next_token_parse_keys 2018-10-16 18:04:54 +03:00
rl1987
7764d6dfc9 Test RSA private key parsing with get_next_token() 2018-10-16 18:04:54 +03:00
rl1987
569d8d8cd7 Test-case for public key parsing using get_next_token() 2018-10-16 18:04:52 +03:00
rl1987
5c891dba77 Test argument concatenation in get_next_token() 2018-10-16 18:04:20 +03:00
rl1987
7829e3a868 First testcase for get_next_token() 2018-10-16 18:04:20 +03:00
rl1987
55412c4f3d Add new source file to test target 2018-10-16 18:04:17 +03:00
Nick Mathewson
b7fbd1f329 Merge branch 'maint-0.3.4' 2018-10-16 08:04:34 -04:00
David Goulet
38599de2dd test: Update approx_time before the test starts
This way we have the same time source when the IP is created and tested later.

Fixes #27810

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-10-16 08:03:55 -04:00
Nick Mathewson
2dccef0eb4 Merge branch 'bug27772_squashed' 2018-10-14 15:31:52 -04:00
Nick Mathewson
6925b61cfd Fix various GCC LTO warnings in the unit tests. 2018-10-14 15:25:16 -04:00
Nick Mathewson
370d9922a4 Use the correct function signatures in test_relaycell.c
This is now officially an antipattern: please let's never copy a
function declaration in two places again.  That's what headers are
for.
2018-10-14 15:25:16 -04:00
Nick Mathewson
67351f6724 Merge remote-tracking branch 'tor-github/pr/380' 2018-10-12 11:39:37 -04:00
Alex Crichton
74c1e44746 Fix segfaults related to sanitizers+jemalloc
It looks to be the case that Rust's standard allocator, jemalloc, is
incompatible with sanitizers. The incompatibility, for whatever reason,
seems to cause segfaults at runtime when jemalloc is linked with
sanitizers.

Without actually trying to figure out what's going on here this commit
instead takes the hammer of "let's remove jemalloc when testing". The
`tor_allocate` crate now by default switches to the system allocator
(eventually this will want to be the tor allocator). Most crates then
link to `tor_allocate` ot pick this up, but the `smartlist` crate had to
manually switch to the system allocator in testing and the `external`
crate had to be sure to link to `tor_allocate`.

The final gotcha here is that this patch also switches to
unconditionally passing `--target` to Cargo. For weird and arcane
reasons passing `--target` with the host target of the compiler (which
Cargo otherwise uses as the default) is different than not passing
`--target` at all. This ensure that our custom `RUSTFLAGS` with
sanitizer options doesn't make its way into build scripts, just the
final testing artifacts.
2018-10-01 22:55:59 -07:00
Alex Crichton
6ebb2c46d5 Remove the link_rust.sh.in script
This is no longer necessary with upstream rust-lang/rust changes as well
as some local tweaks. Namely:

* The `-fsanitize=address`-style options are now passed via `-C
  link-args` through `RUSTFLAGS`. This obviates the need for the shell
  script.
* The `-C default-linker-libraries`, disabling `-nodefaultlibs`, is
  passed through `RUSTFLAGS`, which is necessary to ensure that
  `-fsanitize=address` links correctly.
* The `-C linker` option is passed to ensure we're using the same C
  compiler as normal C code, although it has a bit of hackery to only
  get the `gcc` out of `gcc -std=c99`
2018-10-01 22:50:08 -07:00
Nick Mathewson
58299b2927 Merge remote-tracking branch 'public/bug27728' 2018-10-01 12:06:39 -05:00
Nick Mathewson
886dc8b0a5 Remove routerparse include from files that dont use it 2018-10-01 11:35:11 -05:00
Nick Mathewson
d199348664 Remove versions.h include from routerparse.h 2018-10-01 11:22:47 -05:00
Nick Mathewson
35db3f8162 Extract addr-policy parsing code. 2018-10-01 11:17:19 -05:00
Nick Mathewson
4201203845 extract networkstatus parsing to its own file. 2018-10-01 10:46:00 -05:00
Nick Mathewson
cd23903427 Pull detached-signatures code into dirauth. 2018-10-01 10:46:00 -05:00
Nick Mathewson
2be35f4d61 Split microdescriptor parser into its own file. 2018-10-01 10:46:00 -05:00
Nick Mathewson
82f4d3ca75 Move v2 hs parsing into feature/rend 2018-10-01 00:09:00 -05:00
Nick Mathewson
430ca38f70 Split the authority-cert and signature/hash code from routerparse 2018-10-01 00:09:00 -05:00
Nick Mathewson
fec3b3bb93 Extract logic for dumping unparseable junk from routerparse.c 2018-10-01 00:04:58 -05:00
Nick Mathewson
6785aa4010 Move routerparse and parsecommon to their own module. 2018-10-01 00:04:06 -05:00
Nick Mathewson
9e65e7a36f Merge branch 'split_stats' 2018-09-27 16:26:06 -04:00
Nick Mathewson
d0b2b5a202 Always initialize the periodic events list.
Various places in our code try to activate these events or check
their status, so we should make sure they're initialized as early as
possible.  Fixes bug 27861; bugfix on 0.3.5.1-alpha.
2018-09-27 13:24:36 -04:00
Nick Mathewson
c8f2a6d2fe Extract the non-stats part of geoip into a new src/lib/geoip. 2018-09-27 10:26:01 -04:00
Nick Mathewson
fa32574bdb Remove excess dependencies from geoip.c 2018-09-27 10:15:39 -04:00
Nick Mathewson
f403af2207 Split geoip from geoip-related stats.
This commit just moves the code to two separate files. The geoip
code still has a few needless dependencies on core/* and features/*.
2018-09-27 09:36:52 -04:00
Nick Mathewson
241c1505cc Move the predicted ports code out of rephist.c
It differs from the rest of the rephist code in that it's actually
necessary for Tor to operate, so it should probably go somewhere
else.  I'm not sure where yet, so I'll leave it in the same
directory, but give it its own file.
2018-09-27 08:21:17 -04:00
Nick Mathewson
4f0bc0c8f5 Revise things that had included router.h before
Make them only include the headers that they needed, and sort their
headers while we're at it.
2018-09-25 17:57:58 -04:00
Nick Mathewson
fcd0f76134 Extract all the "am I a server" functions from router.c 2018-09-25 16:00:50 -04:00
Nick Mathewson
934859cf80 Move key-loading and crosscert-checking out of feature/relay
This is also used by onion services, so it needs to go in another
module.
2018-09-25 15:14:57 -04:00
Nick Mathewson
78295904f7 Merge branch 'ticket26744' 2018-09-24 10:56:50 -04:00
Nick Mathewson
63e08f56b8 Merge remote-tracking branch 'tor-github/pr/365' 2018-09-21 16:13:59 -04:00
Nick Mathewson
194acfb51d Split directory.c code into several modules
Parts of this C file naturally belong in dircache, dirclient, and
dircommon: so, move them there.
2018-09-21 12:57:22 -04:00
Nick Mathewson
0e4c42a912 Merge remote-tracking branch 'ahf-github/asn/bugs4700_2' 2018-09-21 09:40:16 -04:00
Nick Mathewson
9399c579e5 Merge branch 'split_mainloop_onion' 2018-09-21 09:37:23 -04:00
Nick Mathewson
9b0a17a74f Merge branch 'maint-0.3.4' 2018-09-21 09:36:31 -04:00
Nick Mathewson
e818a71905 Merge branch 'maint-0.3.3' into maint-0.3.4 2018-09-21 09:35:54 -04:00
Nick Mathewson
2cadd93cea Merge branch 'maint-0.3.2' into maint-0.3.3 2018-09-21 09:35:51 -04:00
Nick Mathewson
5b04392c15 Merge branch 'maint-0.2.9' into maint-0.3.2 2018-09-21 09:35:23 -04:00
Nick Mathewson
c7ce6b9821 Split main.c into main.c and mainloop.c
The main.c code is responsible for initialization and shutdown;
the mainloop.c code is responsible for running the main loop of Tor.

Splitting the "generic event loop" part of mainloop.c from the
event-loop-specific part is not done as part of this patch.
2018-09-21 09:14:06 -04:00
Nick Mathewson
98ef3e82e4 Move the non-crypto parts of onion.c out of src/core/crypto
The parts for handling cell formats should be in src/core/or.

The parts for handling onionskin queues should be in src/core/or.

Only the crypto wrapper belongs in src/core/crypto.
2018-09-21 09:14:05 -04:00
rl1987
d40b0e73c2 Check Python version in test_rebind.py 2018-09-21 10:57:41 +03:00
rl1987
81199eaf0e Remove misleading first line in test_rebind.py
test_rebind.py actually does not require Python 3.

Bugfix on 5a11670fcaad0a58de48425ba80510effbe35628; bug not in any
Tor release.
2018-09-21 10:57:41 +03:00
cypherpunks
db89b4b152 rust/protover: fix null deref in protover_all_supported()
Fortunately with the current callers it couldn't happen in practice.

Fix on d1820c1516.
2018-09-21 04:57:26 +00:00
Nick Mathewson
e7ac8fabcc Merge remote-tracking branch 'dgoulet/ticket27410_035_01' 2018-09-20 16:22:16 -04:00
Nick Mathewson
813019cc57 Merge branch 'bug23512-v4-029-fixes' 2018-09-20 14:41:44 -04:00
Nick Mathewson
a406255cf3 Merge branch 'bug23512-v4-029-fixes' 2018-09-20 14:19:47 -04:00
Nick Mathewson
72e30f26ec Fix a pair of errors in bug23512 2018-09-20 14:18:09 -04:00
Nick Mathewson
bd6007d898 Merge branch 'split_routerlist_dirserv_v2' 2018-09-20 11:07:50 -04:00
Nick Mathewson
b54a5e704f Split most of dirserv.c into several new modules
In dirauth:
  * bwauth.c reads and uses bandwidth files
  * guardfraction.c reads and uses the guardfraction file
  * reachability.c tests relay reachability
  * recommend_pkg.c handles the recommended-packages lines.
  * recv_descs.c handles fingerprint files and processing incoming
    routerinfos that relays upload to us
  * voteflag.c computes flag thresholds and sets those thresholds on
    routerstatuses when computing votes

In control:
  * fmt_serverstatus.c generates the ancient "v1 server status"
    format that controllers expect.

In nodelist:
  * routerstatus_fmt.c formats routerstatus entries for a consensus,
    a vote, or for the controller.
2018-09-20 11:07:42 -04:00
Nick Mathewson
1f377e910f Merge branch 'maint-0.3.4' 2018-09-20 10:43:08 -04:00
Nick Mathewson
08e3b88f07 Split routerlist.c into 4 separate modules
There are now separate modules for:
    * the list of router descriptors
    * the list of authorities and fallbacks
    * managing authority certificates
    * selecting random nodes
2018-09-19 17:08:57 -04:00
David Goulet
cb81a69f90 test: hs-v3 desc has arrived unit test
That unit test makes sure we don't have pending SOCK request if the descriptor
turns out to be unusable.

Part of #27410.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-19 11:11:57 -04:00
Nick Mathewson
6d33f65638 Use the correct function signatures in test_relaycell.c
This is now officially an antipattern: please let's never copy a
function declaration in two places again.  That's what headers are
for.
2018-09-18 15:51:11 -04:00
Nick Mathewson
ea5792f333 Make crypto_strongest_rand() non-mockable
Instead, have it call a mockable function.  We don't want
crypto_strongest_rand() to be mockable, since doing so creates a
type error when we call it from ed25519-donna, which we do not build
in a test mode.

Fixes bug 27728; bugfix on 0.3.5.1-alpha
2018-09-18 12:40:18 -04:00
Nick Mathewson
50367d06f2 Merge remote-tracking branch 'tor-github/pr/352' 2018-09-18 08:44:58 -04:00
Nick Mathewson
4199c8b2d0 Merge remote-tracking branch 'github/bug27073_029' 2018-09-18 08:16:42 -04:00
Mike Perry
8a83c4b613 Merge branch 'bug23512-v4-033' into bug23512-v4-master 2018-09-18 00:17:14 +00:00
Mike Perry
ad10cafd9f Bug 23512: Test fix: cmux is now allocated by new_fake_channel() 2018-09-17 23:31:48 +00:00
Mike Perry
72cef61028 Merge branch 'bug23512-v4-032' into bug23512-v4-033 2018-09-17 23:31:34 +00:00
Mike Perry
dfd3823047 Bug 23512: Mock assert_circuit_ok in tests.
This mocking was not available in 0.2.9.
2018-09-17 23:12:53 +00:00
Mike Perry
36e81e1f59 Merge branch 'bug23512-v4-029' into bug23512-v4-032 2018-09-17 23:12:45 +00:00
Mike Perry
6af352172d Bug 23512: Test recording bytes in circ queues. 2018-09-17 22:24:25 +00:00
Nick Mathewson
b729bc202c Add tortls.h includes to expose critical macro. Fix #27734. 2018-09-16 22:08:02 -04:00
Nick Mathewson
a8ac21fbb5 Don't try to link C from rust doctests for nss detection
This is really annoying, since we can't use cfg(test) for doctests.
2018-09-16 14:34:31 -04:00
Nick Mathewson
078debb0de Merge branch 'bug25573-034-typefix' 2018-09-16 13:46:12 -04:00
Nick Mathewson
7fd61cf536 Fix duplicate declaration of pathbias_count_valid_cells. 2018-09-16 13:45:43 -04:00
Nick Mathewson
991bec67ee When Tor is compiled with NSS, don't claim support for LinkAuth=1
Closes ticket 27288
2018-09-16 13:28:29 -04:00
Alexander Færøy
3477a73af9 Add proxy headers as early as possible.
This patch moves the logic that adds the proxy headers to an earlier
point in the exit connection lifetime, which ensures that the
application data cannot be written to the outbuf before the proxy header
is added.

See: https://bugs.torproject.org/4700
2018-09-15 22:17:57 +02:00
Alexander Færøy
9b511dc5d6 Change HiddenServiceExportCircuitID to take a string parameter: the protocol.
This patch changes HiddenServiceExportCircuitID so instead of being a
boolean it takes a string, which is the protocol. Currently only the
'haproxy' protocol is defined.

See: https://bugs.torproject.org/4700
2018-09-15 16:52:36 +03:00
Alexander Færøy
8f085841ef Encode the 32-bit Global Identifier as 2 x 16-bit in the IPv6 address.
Without this patch we would encode the IPv6 address' last part as
::ffffffff instead of ::ffff:ffff when the GID is UINT32_MAX.

See: https://bugs.torproject.org/4700
2018-09-15 16:52:36 +03:00
George Kadianakis
b2092f1ced Add unittest for HiddenServiceExportCircuitID.
Had to move a function to test helpers.
2018-09-15 16:52:32 +03:00
Nick Mathewson
6e5e1be737 Make circuitmux ewma timing test more tolerant on 32bit osx
Since we use a 32-bit approximation for millisecond conversion here,
we can't expect so much precision.

Fixes part of bug 27139; bugfix on 0.3.4.1-alpha.
2018-09-14 08:40:12 -04:00
cypherpunks
e24195c7c1 protover: reject invalid protocol names
The spec only allows the characters [A-Za-z0-9-].

Fix on b2b2e1c7f2.
Fixes #27316; bugfix on 0.2.9.4-alpha.
2018-09-14 02:18:04 +00:00
Nick Mathewson
7a0ff5beb2 In conditionvar_timeout test, wait for threads to timeout
Previously we just waited 200msec, which was not enough on slow/busy
systems.

Fixes bug 27073; bugfix on 0.2.6.3-alpha when the test was introduced.
2018-09-13 20:47:41 -04:00
Nick Mathewson
b943721b2a Merge branch 'bug27224_take2_squashed' 2018-09-13 16:43:06 -04:00
Nick Mathewson
874eca6a8c Add a test case with a matching ip but mismatched identity. 2018-09-13 16:38:33 -04:00
rl1987
9741921094 Unit tests for ticket 27224.
Since this is a refactoring ticket, these tests should pass before
and after the changes are made.
2018-09-13 16:25:14 -04:00
Nick Mathewson
e8b81d7dc5 Merge branches 'bug27684' and 'bug27685' 2018-09-13 13:09:57 -04:00
Nick Mathewson
15596f6c0c Fix a memory leak in tortls/openssl/try_to_extract_certs_from_tls
Since this is an "intrusive" test, it only shows up for openssl <1.1

This is a bugfix on 0.3.5.x; bug not in any released Tor.
2018-09-13 12:47:42 -04:00
Nick Mathewson
d28018ea1e Fix a memory leak in tortls/openssl/context_new test.
Bugfix on 0.3.5.x; bugfix not on any released Tor.
2018-09-13 12:43:37 -04:00
Nick Mathewson
f308e81fa7 Merge branch 'maint-0.3.4' 2018-09-12 17:25:40 -04:00
Nick Mathewson
f8d5fb42a3 Merge branch 'maint-0.3.3' into maint-0.3.4 2018-09-12 17:24:41 -04:00
Nick Mathewson
19dbc385d5 Merge remote-tracking branch 'tor-github/pr/298' 2018-09-12 09:38:52 -04:00
Nick Mathewson
62743912bc Merge branch 'pr278_squashed' 2018-09-12 09:06:35 -04:00
rl1987
7b27d98eae Actually, just disable test_rebind.sh on Appveyor 2018-09-12 09:06:16 -04:00
rl1987
d30e47fd4e Disable test_rebind.sh on Windows 2018-09-12 09:06:16 -04:00
rl1987
5a11670fca Update/fix CI build
Update integration test to Python 3
2018-09-12 09:06:16 -04:00
rl1987
4811869d7a Pick random ports in test_rebind.py 2018-09-12 09:06:16 -04:00
rl1987
3f34fc921c Tweak test_rebind.py for future-proofness 2018-09-12 09:06:16 -04:00
rl1987
762c27b907 Integration test for socket rebinding
squash! Integration test for socket rebinding
2018-09-12 09:06:16 -04:00
Nick Mathewson
8294c40c96 Merge remote-tracking branch 'tor-github/pr/318' 2018-09-12 08:12:19 -04:00
cypherpunks
e9ef7d5ab4 test/protover: remove version zero from tests
This isn't legal according to dir-spec.txt.

We can write separate tests for it if the spec
is changed to make it legal.
2018-09-12 02:47:29 +00:00
Nick Mathewson
5a2374b074 Merge remote-tracking branch 'tor-github/pr/315' 2018-09-11 15:55:30 -04:00
Nick Mathewson
75ad1a1f2f Merge remote-tracking branch 'onionk/doublevote1' 2018-09-11 13:16:49 -04:00
Nick Mathewson
04bb70199b Followup: Make authority_cert_parse_from_string() take length too 2018-09-11 11:43:26 -04:00
Nick Mathewson
7e3005af30 Replace "read consensus from disk" with "map consensus from disk".
Implements 27244, and should save a bunch of RAM on clients.
2018-09-11 11:43:26 -04:00
Nick Mathewson
abaca3fc8c Revise networkstatus parsing code to use lengths
This way the networkstatus can be parsed without being
NUL-terminated, so we can implement 27244 and mmap our consensus objects.
2018-09-11 11:43:26 -04:00
Nick Mathewson
e014b72b73 Stop memcpy'ing uncompressed consensuses when making diffs 2018-09-11 11:16:50 -04:00
Nick Mathewson
5595b21227 Consdiff: use lengths on inputs so they don't need NUL at the end
This is part of #27244, so that we can safely mmap consensus
documents.
2018-09-11 11:16:50 -04:00
Taylor Yu
617160895c Defer reporting directory bootstrap progress
Existing cached directory information can cause misleadingly high
bootstrap percentages.  To improve user experience, defer reporting of
directory information progress until at least one connection has
succeeded to a relay or bridge.

Closes ticket 27169.
2018-09-10 15:20:50 -05:00
David Goulet
7ff67d0e90 test: Fix coverity CID 1439129
One HSv3 unit test used "tor_memeq()" without checking the return value. This
commit changes that to use "tt_mem_op()" to actually make the test validate
something :).

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-10 10:28:35 -04:00
David Goulet
064d3e7497 test: Fix coverity CID 1439130
Trivial fix of removing an uneeded NULL check in an HS v3 unit test.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-10 10:22:31 -04:00
David Goulet
58d74ad943 test: Fix coverity CID 1439131
Simple uninitialized object that we could free in an HS v3 unit test.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-10 10:20:58 -04:00
Nick Mathewson
9ca1af9a87 Merge remote-tracking branch 'dgoulet/ticket20700_035_03' 2018-09-07 15:03:32 -04:00
George Kadianakis
3695ef6343 HSv3: Don't assert when reading bad client-side privkeys. 2018-09-07 14:05:07 -04:00
George Kadianakis
1e9428dc61 HSv3: Add subcredential in client auth KDF on the service-side.
Also update some client auth test vectors that broke...
2018-09-07 14:05:07 -04:00
David Goulet
c76d00abfa hs-v3: Make hs_desc_build_fake_authorized_client() return an object
Return a newly allocated fake client authorization object instead of taking
the object as a parameter.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 14:03:55 -04:00
Suphanat Chunhapanya
5e1d36c7db bug: Use PATH_SEPARATOR instead of slash
In function get_fname_suffix, previously it uses /, but in fact it
should use PATH_SEPARATOR.
2018-09-07 14:03:55 -04:00
Suphanat Chunhapanya
83c8419e73 hs-v3: Rename client_pk to client_auth_pk
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 14:03:08 -04:00
Suphanat Chunhapanya
9f975e9995 hs-v3: Rename client_sk to client_auth_sk
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 14:03:07 -04:00
Suphanat Chunhapanya
b61403c787 test: HS v3 client auth is config equal function
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 14:02:43 -04:00
Suphanat Chunhapanya
3b08b23997 hs-v3: Make all descriptor content free functions public
Series of functions that we now need in hs_service.c.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 14:02:43 -04:00
Suphanat Chunhapanya
69fb25b0f6 test: HS v3 descriptor decoding with client authorization
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 14:02:43 -04:00
Suphanat Chunhapanya
63576b0166 hs-v3: Refactor the descriptor decryption/decoding
This commit refactors the existing decryption code to make it compatible with
a new logic for when the client authorization is enabled.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Suphanat Chunhapanya
9c36219236 test: HS v3 client authorization loading secret key
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Suphanat Chunhapanya
fd6bec923c test: HS v3 descriptor encoding with client authorization
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Suphanat Chunhapanya
fa50aee366 hs-v3: Encrypt the descriptor using a cookie
Previously, we encrypted the descriptor without the descriptor cookie. This
commit, when the client auth is enabled, the descriptor cookie is always used.

I also removed the code that is used to generate fake auth clients because it
will not be used anymore.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Suphanat Chunhapanya
10f4c46e50 test: Build an HSv3 descriptor with authorized client
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Suphanat Chunhapanya
0dab4ac2dd test: HS v3 building a descriptor with client auth
This commit tests that the descriptor building result, when the client
authorization is enabled, includes everything that is needed.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Suphanat Chunhapanya
15af47ede0 test: HS v3 loading client auth keys service side
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-09-07 13:59:22 -04:00
Nick Mathewson
8815960c46 Merge remote-tracking branch 'tor-github/pr/294' 2018-09-06 09:47:32 -04:00
Nick Mathewson
3b61bdb5ae Try to fix new coverity warnings in unit tests. 2018-09-05 08:30:35 -04:00
Nick Mathewson
03efb67b42 Debug one last reference-counting issue that only appeared on openssl master 2018-09-04 20:46:46 -04:00
Nick Mathewson
eeba944ee0 Fix an easy refcounting bug in a unit test 2018-09-04 20:25:25 -04:00
Nick Mathewson
0db5c54957 Merge branch 'nss_squashed' into nss_merge 2018-09-04 20:21:07 -04:00
Nick Mathewson
d644c93ae9 Resolve openssl-only memory leaks 2018-09-04 19:45:28 -04:00
Nick Mathewson
c50537fd94 Fix a pair of remaining leaks in tortls_nss.c
Fun fact: PR_Close leaks memory if its socket is not valid.
2018-09-04 19:45:21 -04:00
Nick Mathewson
274efb1263 Use FREE_AND_NULL for impl types 2018-09-04 14:52:35 -04:00
Nick Mathewson
ad94d43fc5 Port test_tortls_verify to not depend on openssl internals 2018-09-04 14:52:35 -04:00
Nick Mathewson
59c1b34b72 Remove tor_tls_check_lifetime as unused.
Everything that might have used it, uses tor_tls_cert_is_valid() instead.
2018-09-04 14:52:35 -04:00
Nick Mathewson
3cdf0497f9 Add unit test for bridge-style TLS initialization. 2018-09-04 14:52:35 -04:00
Nick Mathewson
600e046ed3 Rename crypto_pk_check_key(), use it more reasonably, add tests
This function was a wrapper around RSA_check_key() in openssl, which
checks for invalid RSA private keys (like those where p or q are
composite, or where d is not the inverse of e, or where n != p*q).
We don't need a function like this in NSS, since unlike OpenSSL, NSS
won't let you import a bogus private key.

I've renamed the function and changed its return type to make it
more reasonable, and added a unit test for trying to read a key
where n != p*q.
2018-09-04 14:52:35 -04:00
Nick Mathewson
b892133fb9 Do not leave a certificate allocated after testing dirvote_add() 2018-09-04 14:52:35 -04:00
Nick Mathewson
36f3bdac03 Update prefork and postfork NSS code for unit tests. 2018-09-04 14:52:35 -04:00
Nick Mathewson
52ac539b99 Test a few more tortls.c functions 2018-09-04 14:52:35 -04:00
Nick Mathewson
7163389b55 Several unit tests to improve test coverage of x509*.c 2018-09-04 14:52:35 -04:00
Nick Mathewson
dd04fc35c6 Remove tor_tls_shutdown()
This function was supposed to implement a half-duplex mode for our
TLS connections.  However, nothing in Tor actually uses it (besides
some unit tests), and the implementation looks really questionable
to me.  It's probably best to remove it.  We can add a tested one
later if we need one in the future.
2018-09-04 14:52:35 -04:00
Nick Mathewson
5205c7fd90 Initial NSS support for TLS.
This is enough to get a chutney network to bootstrap, though a bunch
of work remains.
2018-09-04 14:52:35 -04:00
Nick Mathewson
3507fead10 Merge branch 'tor_api_owning_control' 2018-09-04 11:04:21 -04:00
Nick Mathewson
94b04d6c64 Merge branch 'bug24104_029_squashed' 2018-09-04 10:44:36 -04:00
cypherpunks
d32b08af6f string: add string_is_utf8() helper
Ticket #27373.
2018-09-03 13:54:43 +00:00
juga0
81f4223329 Test for descriptor does not change when hibernating 2018-09-01 18:47:20 -04:00
juga0
d4e51a2eeb Add missing router_tests to test.h 2018-09-01 18:47:20 -04:00
juga0
1066fdd8d1 Add test for check_descriptor_bandwidth_changed 2018-09-01 18:47:20 -04:00
juga0
842b18ab26 Add test log helpers for msgs not containing str 2018-09-01 18:47:20 -04:00
David Fifield
feae813e1b Add tests for tor_addr_parse, separate from tor_addr_port_parse. 2018-08-31 18:57:42 +03:00
Nick Mathewson
94605f08fb Merge branch 'ticket27246_035_01_squashed' 2018-08-29 15:05:05 -04:00
David Goulet
2f6bc74914 router: Keep RSA onion public key in ASN.1 format
The OpenSSL "RSA" object is currently 408 bytes compares to the ASN.1 encoding
which is 140 for a 1024 RSA key.

We save 268 bytes per descriptor (routerinfo_t) *and* microdescriptor
(microdesc_t). Scaling this to 6000 relays, and considering client usually
only have microdescriptors, we save 1.608 MB of RAM which is considerable for
mobile client.

This commit makes it that we keep the RSA onion public key (used for TAP
handshake) in ASN.1 format instead of an OpenSSL RSA object.

Changes is done in both routerinfo_t and microdesc_t.

Closes #27246

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-08-29 15:01:38 -04:00
Mike Perry
93ff8b411a Merge branch 'ticket25573-034' into ticket25573-master 2018-08-29 17:10:06 +00:00
Mike Perry
ce894e20b5 Ticket #25573: Count TRUNCATED cells.
TRUNCATED cells were ignored while in path bias. Now they are obeyed, and
cause us to tear down the circuit. The actual impact is minimal, since we
would just wait around for a probe that would never arrive before.

This commit changes client behavior.
2018-08-29 04:12:15 +00:00
Mike Perry
144647031a Ticket #25573: Check half-opened stream ids when choosing a new one
Avoid data corrupton by avoiding mixing up old stream ids with new ones.

This commit changes client behavior.
2018-08-29 04:12:15 +00:00
Mike Perry
c56f63eadb Ticket #25573: Track half-closed stream ids
We allow their CONNECTEDs, RESOLVEDs, ENDs, SENDMEs, and DATA cells to not
count as dropped until the windows are empty, or we get an END.

This commit does not change behavior. It only changes CIRC_BW event field
values.
2018-08-29 04:12:09 +00:00
Nick Mathewson
3d7a705d3a Merge remote-tracking branch 'onionk/prototest1' 2018-08-28 21:32:46 -04:00
Nick Mathewson
48632455a5 Merge branch 'bug26367_035_01' 2018-08-28 16:02:04 -04:00
David Goulet
4976eca826 hs: Render obsolete Tor2web
Remove support for Tor2web in the code and build system. At this commit, tor
doesn't have Tor2web support anymore.

Ref: https://lists.torproject.org/pipermail/tor-dev/2018-July/013295.html

Close #26367

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-08-28 16:01:57 -04:00