Commit Graph

23810 Commits

Author SHA1 Message Date
Nick Mathewson
64f594499a Document crypto_fast_rng_one_in_n. 2019-03-01 08:20:54 -05:00
teor
7786198eef
Merge remote-tracking branch 'tor-github/pr/749' into maint-0.2.9 2019-03-01 14:15:00 +10:00
teor
c1ab538479
Merge remote-tracking branch 'tor-github/pr/748' into maint-0.2.9 2019-03-01 14:14:51 +10:00
teor
ecbc2e80a0
Merge remote-tracking branch 'tor-github/pr/747' into maint-0.2.9 2019-03-01 14:14:26 +10:00
Mike Perry
e8d84b18aa
Bug 25733: Avoid assert failure if all circuits time out.
Prior to #23100, we were not counting HS circuit build times in our
calculation of the timeout. This could lead to a condition where our timeout
was set too low, based on non HS circuit build times, and then we would
abandon all HS circuits, storing no valid timeouts in the histogram.

This commit avoids the assert.
2019-03-01 13:48:01 +10:00
Kris Katterjohn
389ee834b6
Log the correct "auto" port number for listening sockets
When "auto" was used for the port number for a listening socket, the
message logged after opening the socket would incorrectly say port 0
instead of the actual port used.

Fixes bug 29144; bugfix on 0.3.5.1-alpha

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-03-01 12:34:21 +10:00
Nick Mathewson
d5f6137547 Merge branch 'maint-0.4.0' 2019-02-28 11:22:06 -05:00
Nick Mathewson
dc19d65c3b Merge remote-tracking branch 'tor-github/pr/728' into maint-0.4.0 2019-02-28 11:20:26 -05:00
teor
9b7cdd23de
doc: update the man page and sample torrc for ExitRelay
We changed the default of ExitRelay in #21530 in 0.3.5.1-alpha, but
forgot to update the documentation.

Closes 29612.
2019-02-28 11:22:20 +10:00
teor
de0969ef78
Merge branch 'maint-0.3.4' into maint-0.3.5 2019-02-28 10:49:33 +10:00
teor
6a61a020ee
Merge branch 'maint-0.3.3' into maint-0.3.4 2019-02-28 10:49:19 +10:00
teor
be29dfedd9
Merge branch 'maint-0.3.4' into maint-0.3.5 2019-02-28 10:45:59 +10:00
teor
524731503e
Merge branch 'maint-0.3.3' into maint-0.3.4 2019-02-28 10:45:44 +10:00
teor
3313444ef0
Merge branch 'maint-0.2.9' into maint-0.3.3 2019-02-28 10:45:30 +10:00
Neel Chauhan
c142e3d1e6 Set CIRCLAUNCH_NEED_UPTIME in rend_service_relaunch_rendezvous() on a hs_service_requires_uptime_circ() 2019-02-27 17:37:10 -05:00
Nick Mathewson
761eb4cf79 Merge branch 'maint-0.4.0' 2019-02-27 09:59:42 -05:00
Nick Mathewson
56f01e58b4 Merge remote-tracking branch 'tor-github/pr/731' into maint-0.4.0 2019-02-27 09:59:33 -05:00
teor
d4b7975ce7 test/shared-random: use sr_state_free_all() rather than sr_state_free()
sr_state_free() was renamed to sr_state_free_all() between 0.2.9 and 0.3.3.

Part of 29599.
2019-02-27 15:06:53 +10:00
teor
64e082e892 Merge branch 'bug29599_029' into bug29599_033 2019-02-27 15:06:39 +10:00
teor
4d9eb4dd0e test/shared-random: Stop leaking shared random state in the unit tests
Stop leaking parts of the shared random state in the shared-random unit
tests.

Fixes bug 29599; bugfix on 0.2.9.1-alpha.
2019-02-27 15:04:40 +10:00
teor
15dc33849e
Merge branch 'maint-0.3.4' into maint-0.3.5 2019-02-27 09:38:04 +10:00
teor
1a194beb2c
Merge branch 'maint-0.3.3' into maint-0.3.4 2019-02-27 09:37:50 +10:00
teor
6c966b894c
Merge branch 'maint-0.2.9' into maint-0.3.3 2019-02-27 09:37:36 +10:00
David Goulet
b402a0887f Merge branch 'tor-github/pr/655' 2019-02-26 11:30:44 -05:00
David Goulet
2a44ee9b8c Merge branch 'maint-0.4.0' 2019-02-26 11:25:16 -05:00
David Goulet
a5dd41b9af Merge branch 'tor-github/pr/638' into maint-0.4.0 2019-02-26 11:24:43 -05:00
Alexander Færøy
aa360b255b Fix crash bug in PT subsystem.
This patch fixes a crash bug (assertion failure) in the PT subsystem
that could get triggered if the user cancels bootstrap via the UI in
TorBrowser. This would cause Tor to call `managed_proxy_destroy()` which
called `process_free()` after it had called `process_terminate()`. This
leads to a crash when the various process callbacks returns with data
after the `process_t` have been freed using `process_free()`.

We solve this issue by ensuring that everywhere we call
`process_terminate()` we make sure to detach the `managed_proxy_t` from
the `process_t` (by calling `process_set_data(process, NULL)`) and avoid
calling `process_free()` at all in the transports code. Instead we just
call `process_terminate()` and let the process exit callback in
`managed_proxy_exit_callback()` handle the `process_free()` call by
returning true to the process subsystem.

See: https://bugs.torproject.org/29562
2019-02-26 15:43:09 +01:00
Nick Mathewson
732855c2e5 Merge branch 'maint-0.4.0' 2019-02-26 07:27:50 -05:00
Nick Mathewson
35257a1c69 Merge remote-tracking branch 'tor-github/pr/726' into maint-0.4.0 2019-02-26 07:27:42 -05:00
George Kadianakis
57d33b5786 Merge branch 'tor-github/pr/698' 2019-02-26 12:35:14 +02:00
George Kadianakis
7fbfdf2af7 Merge branch 'tor-github/pr/611' 2019-02-26 12:33:23 +02:00
Nick Mathewson
e138bb8ffc
Downgrade some LOG_ERR messages in the address/* tests to warnings
Fixes bug 29530, where the LOG_ERR messages were occurring when
we had no configured network, and so we were failing the unit tests
because of the recently-merged #28668.

Commit message edited by teor:

We backported 28668 and released it in 0.3.5.8.
This commit backports 29530 to 0.3.5.

Fixes bug 29530 in Tor 0.3.5.8.
2019-02-26 09:53:59 +10:00
Kris Katterjohn
1b9e77349f Fix some error-checking logic and a misleading error message
When IPv4Only (IPv6Only) was used but the address could not be
interpreted as a IPv4 (IPv6) address, the error message referred
to the wrong IP version.

This also fixes up the error-checking logic so it's more precise
about what's being checked.

Fixes bug 13221; bugfix on 0.2.3.9-alpha

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-02-25 16:03:42 -06:00
Nick Mathewson
626e6d2c39 Merge remote-tracking branch 'tor-github/pr/684' 2019-02-25 13:33:39 -05:00
rl1987
d731ab4583 Check that all valid values of int and unsigned int can be put into void pointer 2019-02-25 20:04:02 +02:00
Nick Mathewson
065e7da8e6 Re-enable and fix unit test for nofork mappings
This test was previously written to use the contents of the system
headers to decide whether INHERIT_NONE or INHERIT_ZERO was going to
work.  But that won't work across different environments, such as
(for example) when the kernel doesn't match the headers.  Instead,
we add a testing-only feature to the code to track which of these
options actually worked, and verify that it behaved as we expected.

Closes ticket 29541; bugfix not on any released version of Tor.
2019-02-25 08:55:25 -05:00
George Kadianakis
18de065cbb Switch an int32_t bin to a circpad_hist_index_t. 2019-02-25 14:01:55 +02:00
George Kadianakis
71c11d7306 document picking infinity bin 2019-02-25 13:59:18 +02:00
George Kadianakis
331a067ae3 Fix dist_min_usec documentation and naming. 2019-02-25 13:46:58 +02:00
George Kadianakis
341cd6ea66 histogram_edges is histogram_len long 2019-02-25 13:28:35 +02:00
Nick Mathewson
69238ca2da Merge remote-tracking branch 'tor-github/pr/646' 2019-02-24 17:17:16 -05:00
Nick Mathewson
34601105a3 Merge branch 'ticket29065_squashed' 2019-02-24 17:02:35 -05:00
rl1987
b7dced893a Fix shellcheck SC2006 warnings in test_switch_id.sh 2019-02-24 17:02:06 -05:00
Neel Chauhan
df8ad64735 When a DirAuth checks reachability on itself and has IPv6, mark it as reachable 2019-02-22 13:36:02 -05:00
George Kadianakis
f229a33685 Merge remote-tracking branch 'nickm/thread_rng' 2019-02-22 16:07:57 +02:00
rl1987
c346eff223 Walk back from requiring bash
Refrain from using bash array to remember $@.
2019-02-21 21:09:40 +02:00
Nick Mathewson
1bff5646e6 Bump to 0.4.0.2-alpha-dev 2019-02-21 13:25:33 -05:00
Nick Mathewson
955ca72f95 Bump to 0.3.5.8-dev 2019-02-21 13:24:42 -05:00
Nick Mathewson
a56b9501f1 Bump to 0.3.4.11-dev 2019-02-21 13:23:46 -05:00
Nick Mathewson
fb309f6eba Bump to 0.3.3.12-dev 2019-02-21 13:22:56 -05:00
Nick Mathewson
00a93b19cf Merge branch 'maint-0.3.5' into maint-0.4.0 2019-02-21 10:08:14 -05:00
Nick Mathewson
4a8a1f76ea Merge branch 'maint-0.3.4' into maint-0.3.5 2019-02-21 10:08:14 -05:00
Nick Mathewson
5062647918 Merge branch 'maint-0.3.3' into maint-0.3.4 2019-02-21 10:08:14 -05:00
Nick Mathewson
54e42fe364 Merge branch 'maint-0.4.0' 2019-02-21 10:08:14 -05:00
David Goulet
be84ed1a64 kist: Don't write above the highwater outbuf mark
KIST works by computing how much should be allowed to write to the kernel for
a given socket, and then it writes that amount to the outbuf.

The problem is that it could be possible that the outbuf already has lots of
data in it from a previous scheduling round (because the kernel is full/busy
and Tor was not able to flush the outbuf yet). KIST ignores that the outbuf
has been filling (is above its "highwater") and writes more anyway. The end
result is that the outbuf length would exceed INT_MAX, hence causing an
assertion error and a corresponding "Bug()" message to get printed to the
logs.

This commit makes it for KIST to take into account the outbuf length when
computing the available space.

Bug found and patch by Rob Jansen.

Closes #29168. TROVE-2019-001.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-02-21 10:07:34 -05:00
Nick Mathewson
df3484b2b0 Bump version to 0.4.0.2-alpha 2019-02-21 09:52:27 -05:00
Nick Mathewson
c663716b67 Bump to 0.3.5.8 2019-02-21 09:51:29 -05:00
Nick Mathewson
508002a4c2 Bump to 0.3.4.11 2019-02-21 09:50:22 -05:00
Nick Mathewson
41c2bf590b Update to 0.3.3.12 2019-02-21 09:49:48 -05:00
Nick Mathewson
aa79196ee0 Merge branch 'maint-0.4.0' 2019-02-21 09:10:34 -05:00
Nick Mathewson
34183f0d71 Fix a goofy change from abcde10fce that broke test-slow linking
boklm tracked this down, and it doesn't make sense.  It caused

This change goes back to the previous LDFLAGS line.
2019-02-21 09:10:15 -05:00
Nick Mathewson
b3416476b4 Remove all uses of weak_rng.
I'm not removing the weak_rng code itself yet, since it is possible
that we will want to revert one of these.
2019-02-20 12:24:34 -05:00
Nick Mathewson
97b9dfe305 Add a convenience macro to get a fast one-in-n calculation 2019-02-20 12:21:05 -05:00
David Goulet
da179088ee Merge branch 'tor-github/pr/687' 2019-02-20 12:09:02 -05:00
David Goulet
7037fe1163 Merge branch 'tor-github/pr/700' 2019-02-20 12:06:39 -05:00
rl1987
b7ad8bcaad Use setrlimit instead of ulimit -c in backtrace tests 2019-02-20 12:06:26 -05:00
David Goulet
8f5a3c0460 Merge branch 'tor-github/pr/675' 2019-02-20 11:45:58 -05:00
David Goulet
a30f17f72f Merge branch 'tor-github/pr/696' 2019-02-20 11:29:55 -05:00
Roger Dingledine
c22446e305 Merge branch 'maint-0.4.0' 2019-02-20 10:40:20 -05:00
Roger Dingledine
5ec65be8bf Merge remote-tracking branch 'nickm/ticket29530_040' into maint-0.4.0 2019-02-20 10:39:28 -05:00
Roger Dingledine
4d4eda89a1 Merge branch 'maint-0.4.0' 2019-02-20 10:36:54 -05:00
Roger Dingledine
ccab4347e5 Merge remote-tracking branch 'nickm/ticket29534_040' into maint-0.4.0 2019-02-20 10:35:56 -05:00
Roger Dingledine
249319ec5d fix typos from #28614 2019-02-20 10:32:47 -05:00
Nick Mathewson
208f04e9b8 Add a quick test for get_thread_fast_rng() 2019-02-19 15:36:11 -05:00
Nick Mathewson
b25cd5cfe1 Implement code to manage a per-thread instance of crypto_fast_rng()
The subsystems API makes this really simple, fortunately.

Closes ticket 29536
2019-02-19 15:36:08 -05:00
Nick Mathewson
d32e407976 Downgrade some LOG_ERR messages in the address/* tests to warnings
Fixes bug 29530, where the LOG_ERR messages were occurring when
we had no configured network, and so we were failing the unit tests
because of the recently-merged #28668.

Bug not in any released Tor.
2019-02-19 14:02:32 -05:00
Nick Mathewson
c9ff6a7f83 Mark map_anon_nofork test as skipped in 0.4.0
This test fails in some environments; since the code isn't used in
0.4.0, let's disable it for now.

Band-aid solution for #29534; bug not in any released Tor.
2019-02-19 13:14:26 -05:00
Nick Mathewson
6927e9a60c Merge remote-tracking branch 'tor-github/pr/665' 2019-02-19 11:56:48 -05:00
Nick Mathewson
c3fca4e4b3 Merge branch 'maint-0.4.0' 2019-02-19 11:51:24 -05:00
Nick Mathewson
4df31adef7 Merge remote-tracking branch 'tor-github/pr/704' into maint-0.4.0 2019-02-19 11:51:19 -05:00
Nick Mathewson
f70929347f Merge branch 'maint-0.4.0' 2019-02-19 11:49:24 -05:00
Nick Mathewson
4bc55ed5ee Merge branch 'bug29145_029' into maint-0.4.0 2019-02-19 11:49:20 -05:00
Kris Katterjohn
4417ac880a Fix a compiler warning on OpenBSD
malloc_options needs to be declared extern (and declaring it extern
means we need to initialize it separately)

Fixes bug 29145; bugfix on 0.2.9.3-alpha

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-02-19 11:38:32 -05:00
Nick Mathewson
17e29bda5d Merge branch 'maint-0.4.0' 2019-02-19 11:34:25 -05:00
Nick Mathewson
b5f3a3d6a7 Merge remote-tracking branch 'tor-github/pr/707' into maint-0.4.0 2019-02-19 11:34:21 -05:00
Nick Mathewson
1239e411a7 Merge remote-tracking branch 'tor-github/pr/694' 2019-02-19 11:31:22 -05:00
Nick Mathewson
0db0b4769c Merge remote-tracking branch 'tor-github/pr/678' 2019-02-19 11:30:27 -05:00
Nick Mathewson
8a5c66cbb0 Merge branch 'maint-0.4.0' 2019-02-19 11:27:21 -05:00
Nick Mathewson
26873bc4ed Merge branch 'bug28698_035' into maint-0.4.0 2019-02-19 11:27:18 -05:00
José M. Guisado
78220aae1e Add circuit time check before logging about relaxing circuit time
Signed-off-by: José M. Guisado <guigom@riseup.net>
2019-02-19 11:24:51 -05:00
Nick Mathewson
485803dffe Merge branch 'maint-0.4.0' 2019-02-19 11:21:54 -05:00
Nick Mathewson
5c87add923 Merge remote-tracking branch 'tor-github/pr/701' into maint-0.4.0 2019-02-19 11:21:51 -05:00
teor
6c652eae0a fixup! test_dir: Refactor common code out of the dir_format unit tests 2019-02-20 00:40:18 +10:00
Nick Mathewson
9a158a45b1 Bump to 0.4.1.0-alpha-dev 2019-02-19 09:29:43 -05:00
teor
51f59f213e router: Add some missing #endif comments 2019-02-19 21:54:30 +10:00
teor
0c0f215822 routerkeys: Log failures at info-level in make_tap_onion_key_crosscert() 2019-02-19 21:54:13 +10:00
teor
39ab6c9f73 test_dir: Test descriptor variants
Including:
* relays and bridges,
* no stats, basic stats, and all stats

Part of 29017 and 29018.
2019-02-19 21:54:13 +10:00
teor
38fc52a50e test_dir: Refactor common code out of the dir_format unit tests
Also:
* delete some obsolete code that was #if 0
* improve cleanup on failure
* make the dir_format tests more consistent with each other
* construct the descriptors using smartlist chunks

This refactor is incomplete, because removing the remaining duplicate
code would be time-consuming.

Part of 29017 and 29018.
2019-02-19 21:54:13 +10:00
teor
8e5df40018 test_dir: Test rsa + ed25519 extrainfo creation and parsing
Also fix a missing mock in rsa-only parsing.
2019-02-19 21:44:41 +10:00
teor
7a2c8daded test_dir: Split test_dir_formats into separate rsa and rsa_ed25519 tests 2019-02-19 21:44:40 +10:00
teor
53b49d1a35 test_dir: Unit tests for RSA-only router and extrainfo descriptor creation
Tests 29017 and 29018.
2019-02-19 21:44:40 +10:00
teor
7c9450fb07 test_router: Add comment to explain mocking
Add comment in
test_router_dump_router_to_string_no_bridge_distribution_method to explain
the effect of a mocked function.
2019-02-19 21:43:12 +10:00
teor
a9f852a0f6 router: Document the additional config and state used to dump descriptors
Also, explicitly state when routerinfos and extra-infos are signed.
And tidy up some other comments.

Preparation for testing 29017 and 20918.
2019-02-19 21:43:07 +10:00
teor
a1f8558628 router: Move extrainfo signing into its own function
This refactoring improves the structure of router_build_fresh_descriptor().

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
9cab988696 router: eliminate router_update_info_send_unencrypted()
Remove router_update_info_send_unencrypted(), and move its code into the
relevant functions.

Then, re-use an options pointer.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
af0a43be2c router: eliminate tiny router_build_fresh_descriptor() static functions
Remove some tiny static functions called by router_build_fresh_descriptor(),
and move their code into more relevant functions.

Then, give router_update_{router,extra}info_descriptor_body identical layouts.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
a65c101973 router: check for NULL in router_build_fresh_descriptor() static functions
Make sure that these static functions aren't passed NULL.
If they are, log a BUG() warning, and return an error.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
f19b64dce9 router: refactor router_build_fresh_descriptor() static function interfaces
Tidy the arguments and return values of these functions, and clean up their
memory management.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:36 +10:00
teor
6c5a506cdb router: split router_build_fresh_descriptor() into static functions
Split the body of router_build_fresh_descriptor() into static functions,
by inserting function prologues and epilogues between existing sections.

Write a new body for router_build_fresh_descriptor() that calls the new
static functions.

Initial refactor with no changes to the body of the old
router_build_fresh_descriptor(), except for the split.

Preparation for testing 29017 and 20918.
2019-02-19 19:05:43 +10:00
teor
a798bd40fb stats: Stop reporting statistics when ExtraInfoStatistics is 0
When ExtraInfoStatistics is 0, stop including bandwidth usage statistics,
GeoIPFile hashes, ServerTransportPlugin lines, and bridge statistics
by country in extra-info documents.

Fixes bug 29018; bugfix on 0.2.4.1-alpha (and earlier versions).
2019-02-19 19:01:44 +10:00
teor
361738c964 Merge branch 'bug29017-033' into bug29017-master-merge 2019-02-19 18:59:16 +10:00
Roger Dingledine
94f7e53d04 fix a bootstrapping string typo
introduced in 85542ee5

next step is to fix it in torspec too
2019-02-17 16:56:13 -05:00
Neel Chauhan
384c5c6188 Make test-slow compile with libevent 2019-02-16 16:03:17 -05:00
George Kadianakis
3093d8afbe Fix tests to use the new design.
- All histogram tests were using start_usec/range_usec. We now manually specify
  the edges.
- Also add a test for histogram_get_bin_upper_bound().
2019-02-15 17:43:41 +02:00
George Kadianakis
80abe4170d Update all the histogram functions to use the new design. 2019-02-15 17:43:23 +02:00
George Kadianakis
98af25e013 Remove start_usec/range_usec and make equivalent fields for distributions. 2019-02-15 17:43:01 +02:00
George Kadianakis
f07c6ae57c Add histogram fields in header file that allow specifying edges. 2019-02-15 17:42:17 +02:00
Matt Traudt
b054a6c6b9 kist: When readding chans, check correct chan's sched_heap_idx
Closes #29508

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-02-14 16:19:31 -05:00
David Goulet
6c173d00f5 Merge branch 'tor-github/pr/702' 2019-02-14 11:43:10 -05:00
George Kadianakis
00b073b1bc Merge branch 'maint-0.3.5' 2019-02-14 18:01:07 +02:00
George Kadianakis
d83c299eba Merge branch 'tor-github/pr/689' into maint-0.3.5 2019-02-14 18:00:05 +02:00
George Kadianakis
9bfe4ed6dd Merge branch 'tor-github/pr/536' into maint-0.3.5 2019-02-14 17:39:34 +02:00
Nick Mathewson
6a29aa7b8c Add whitebox test for the long-output optimization of fast_rng 2019-02-14 09:26:40 -05:00
Nick Mathewson
3f28b98220 Add test for crypto_fast_rng_get_double(). 2019-02-14 09:26:40 -05:00
Nick Mathewson
acbde10fce Add a test-rng program so we can pipe to dieharder. 2019-02-14 09:26:40 -05:00
Nick Mathewson
490e187056 Add a benchmark for our several PRNGs. 2019-02-14 09:26:40 -05:00
Nick Mathewson
f3cbd6426c Implement a fast aes-ctr prng
This module is currently implemented to use the same technique as
libottery (later used by the bsds' arc4random replacement), using
AES-CTR-256 as its underlying stream cipher.  It's backtracking-
resistant immediately after each call, and prediction-resistant
after a while.

Here's how it works:

We generate psuedorandom bytes using AES-CTR-256.  We generate BUFLEN bytes
at a time.  When we do this, we keep the first SEED_LEN bytes as the key
and the IV for our next invocation of AES_CTR, and yield the remaining
BUFLEN - SEED_LEN bytes to the user as they invoke the PRNG.  As we yield
bytes to the user, we clear them from the buffer.

Every RESEED_AFTER times we refill the buffer, we mix in an additional
SEED_LEN bytes from our strong PRNG into the seed.

If the user ever asks for a huge number of bytes at once, we pull SEED_LEN
bytes from the PRNG and use them with our stream cipher to fill the user's
request.
2019-02-14 09:26:40 -05:00
Nick Mathewson
3d3578ab41 Extract RNG tests into a new test module
test_crypto.c is pretty big; it wouldn't hurt to split it up some
more before I start adding stuff to the PRNG tests.
2019-02-14 09:26:40 -05:00
Nick Mathewson
622a9a8a36 Extract the common body of our random-int functions into a macro
This is the second part of refactoring the random-int-in-range code.
2019-02-14 09:26:40 -05:00
George Kadianakis
f5a6d4c6ea Disable unstable circuit padding unittest.
until #29298 is implemented.
2019-02-14 12:09:41 +02:00
Nick Mathewson
5f42bc0f48 Merge remote-tracking branch 'tor-github/pr/699' 2019-02-13 15:19:59 -05:00
David Goulet
d5de1a0a55 Merge branch 'tor-github/pr/651' 2019-02-13 11:02:02 -05:00
David Goulet
6efc2a0e1f Merge branch 'tor-github/pr/650' into maint-0.3.5 2019-02-13 10:56:24 -05:00
rl1987
ad48aab056 Let's not double-quote EXTRA_CARGO_OPTIONS after all 2019-02-13 15:04:12 +02:00
rl1987
4f9061868b Use env to find bash 2019-02-13 14:51:42 +02:00
juga0
ec7da50ab4 dirvote: Add the bandwidth file digest in the vote 2019-02-13 12:26:00 +00:00
juga0
28490fa23e test: Add test to get the digest of a bw file 2019-02-13 12:26:00 +00:00
juga0
fc3e90a7b6 bwauth: Add function to get the digest of a bw file 2019-02-13 12:26:00 +00:00
David Goulet
c320c52e89 Merge branch 'tor-github/pr/690' 2019-02-12 13:04:25 -05:00
David Goulet
95e5f8fe03 Merge branch 'tor-github/pr/671' 2019-02-12 13:02:30 -05:00
Nick Mathewson
72b978c3a5 On windows, if we fail to load a consensus and it has a CRLF, retry.
Fixes bug 28614; bugfix on 0.4.0.1-alpha when we started mmapping
the consensus.
2019-02-12 12:57:33 -05:00
Nick Mathewson
a797a69679 Merge branch 'maint-0.3.5' 2019-02-12 09:56:50 -05:00
Nick Mathewson
db209d4dba Merge branch 'maint-0.3.3' into maint-0.3.4 2019-02-12 09:56:49 -05:00
Nick Mathewson
7ead2af622 Merge branch 'maint-0.2.9' into maint-0.3.3 2019-02-12 09:56:49 -05:00
Nick Mathewson
2e770216c4 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-02-12 09:56:49 -05:00
David Goulet
46b6df7122 test: Fix a warning underflow in rend_cache/clean
Because the test is adding entries to the "rend_cache" directly, the
rend_cache_increment_allocation() was never called which made the
rend_cache_clean() call trigger that underflow warning:

rend_cache/clean: [forking] Nov 29 09:55:04.024 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.0.0-alpha-dev 2240fe63feb9a8cf)

The test is still good and valid.

Fixes #28660

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-02-12 09:15:41 -05:00
Karsten Loesing
a3f9ddcf03 Update geoip and geoip6 to the February 5 2019 database. 2019-02-12 14:40:42 +01:00
rl1987
4c10221332 Use compress_dir_buf_add() function in a few places 2019-02-09 16:46:31 +02:00
rl1987
09a3c949f8 Add connection_dir_buf_add() helper function 2019-02-09 16:06:32 +02:00
rl1987
b53fee4622 Fix SC2086 warning in test_workqueue_*.sh shell scripts 2019-02-08 16:51:49 +02:00
Nick Mathewson
49ec29044d Add more openssl includes to fix no-deprecated compilation
Closes ticket 29026; patch from Mangix.
2019-02-08 08:51:23 -05:00
Nick Mathewson
b9abdcd6bc Fix compilation when openssl is compiled without engine support.
Patch from Mangix. Closes part of ticket 29026.
2019-02-08 08:50:43 -05:00
Nick Mathewson
7f59b9fb1f Merge branch 'maint-0.3.5' 2019-02-08 08:37:46 -05:00
Nick Mathewson
ab65347819 Merge branch 'ticket29040_1_changes' into maint-0.3.5 2019-02-08 08:37:43 -05:00
Nick Mathewson
b1ae2fd65b Merge branch 'maint-0.3.5' 2019-02-08 08:19:34 -05:00
Nick Mathewson
4b36f9676d Merge remote-tracking branch 'tor-github/pr/670' into maint-0.3.5 2019-02-08 08:19:31 -05:00
rl1987
8df6a65e6b Fix shellcheck warning in zero_length_keys.sh 2019-02-07 17:23:23 +02:00
Nick Mathewson
a49149fc13 Extract numeric CSPRNG functions into a new module.
Some of the code for getting a random value within a range wants to
be shared between crypto_rand() and the new crypto_fast_rng() code.
2019-02-06 22:06:05 -05:00
Nick Mathewson
21d184a184 Remove extraneous #if/#endif wrapper in crypto_rand.c
I don't know how this got here, but this kind of a wrapper only
belongs in a header file.
2019-02-06 22:05:39 -05:00
Nick Mathewson
8ca808f81d Code for anonymous mappings via mmap() or CreateFileMapping().
Using an anonymous mmap() is a good way to get pages that we can set
kernel-level flags on, like minherit() or madvise() or mlock().
We're going to use that so that we can make uninheritable locked
pages to store PRNG data.
2019-02-06 22:03:30 -05:00
rl1987
daff9e1ba1 Fix shellcheck warning in fuzz_static_testcases.sh 2019-02-06 11:01:22 +02:00
Roger Dingledine
bfd1d70243 Merge branch 'maint-0.3.5' 2019-02-05 08:10:43 -05:00
Roger Dingledine
bca25eebfb Merge branch 'maint-0.3.4' into maint-0.3.5 2019-02-05 08:10:14 -05:00
Roger Dingledine
8ea98c0f4c Merge branch 'maint-0.3.3' into maint-0.3.4 2019-02-05 08:09:47 -05:00
rl1987
d7e5086694 Fix one last SC2086 2019-02-02 16:49:19 +02:00
rl1987
f888b3e2ee Update test-network.sh to bash script to use array
This lets us to save original script argument to array (POSIX shell does not
support that). Fixes shellcheck warnings SC2124 and SC2086.
2019-02-02 16:46:30 +02:00
rl1987
7341d9acdc Fix all instances of SC2166 in test-network.sh 2019-02-02 16:32:46 +02:00
rl1987
3f5459cb95 Fix all instances of SC2015 in test_keygen.sh 2019-02-01 21:41:14 +02:00
rl1987
3ca1d58561 Fix SC2064 2019-02-01 21:26:29 +02:00
rl1987
82813315ed Fix instances of SC2086 in test_keygen.sh 2019-02-01 21:24:06 +02:00
rl1987
a84dc1973b Fix instances of SC2006 in test_keygen.sh 2019-02-01 21:15:10 +02:00
rl1987
10455aeff2 Fix shellcheck warnings in test_rust.sh 2019-02-01 20:53:40 +02:00
Nick Mathewson
4d7a0a1310 Update Cargo.lock with new comment; suppress 29244. 2019-02-01 16:38:23 +01:00
José M. Guisado
cb1072790f Warn about missing ContactInfo when MyFamily set
Operators should be warned when setting MyFamily in
addition to missing ContactInfo

Signed-off-by: José M. Guisado <guigom@riseup.net>
2019-01-31 13:27:42 +01:00
teor
6170d3fcf1 hs: Onion services put IPv6 addresses in service descriptors
Rewrite service_intro_point_new() to take a node_t. Since
node_get_link_specifier_smartlist() supports IPv6 link specifiers,
this refactor adds IPv6 addresses to onion service descriptors.

Part of 23576, implements 26992.
2019-01-31 07:53:22 +01:00
teor
cdda3dc484 hs: Move get_lspecs_from_node to nodelist.c
Also:
* rename to node_get_link_specifier_smartlist
* rewrite to return a smartlist
* add link_specifier_smartlist_free

Part of 23576.
2019-01-30 15:15:41 +01:00
Nick Mathewson
bbd893d6bd Write consensus files in binary mode
This will help us out on windows now that we mmap files.  Fixes part
of ticket 28614.
2019-01-29 16:18:41 +01:00
rl1987
790150e57a Allow empty username/password in SOCKS5 username/password auth message 2019-01-26 11:06:33 +02:00
Nick Mathewson
d9010c5b67 One more 32-bit clang warning 2019-01-24 15:23:06 -05:00
Nick Mathewson
e19222a0da Use tt_u64_op() in test_circuitpadding.c to fix compilation warnings
Fixes bug 29169.
2019-01-24 13:20:21 -05:00
Neel Chauhan
c985940de9 Add version 3 onion service support to HSFETCH 2019-01-24 10:22:41 -05:00
Nick Mathewson
bbe417ae8f Merge branch 'ticket28668_035' into ticket28668_040 2019-01-23 17:08:36 -05:00
Nick Mathewson
d71ca39682 Another case of possible gmtime angst. 2019-01-23 17:08:23 -05:00
Suphanat Chunhapanya
238a9080c6 hs-v3: add an option param to safe log functions
We add an option param to safe_str and safe_str_client because in
some case we need to use those functions before global_options is set.
2019-01-24 04:31:18 +07:00
Suphanat Chunhapanya
8de735f068 hs-v3: fix use after free in client auth config
We accidentally use `auth` after freeing it in
client_service_authorization_free. The way to solve it is to
free after using it.
2019-01-24 04:31:07 +07:00
Nick Mathewson
d23704bf26 Merge branch 'ticket28668_035' into ticket28668_040 2019-01-23 14:50:22 -05:00
David Goulet
2d74da3d0e test: Fix a warning underflow in rend_cache/clean
Because the test is adding entries to the "rend_cache" directly, the
rend_cache_increment_allocation() was never called which made the
rend_cache_clean() call trigger that underflow warning:

rend_cache/clean: [forking] Nov 29 09:55:04.024 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.0.0-alpha-dev 2240fe63feb9a8cf)

The test is still good and valid.

Fixes #28660

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-01-23 13:11:49 -05:00
Nick Mathewson
6144cf99ad Capture more BUG warnings in util/time test
These are ones that happen on windows only.

Fixes bug 29161.
2019-01-23 12:37:12 -05:00
Nick Mathewson
54c9c8b04f If address/get_if_addrs6 can't findipv6, log WARN, not ERR
Fixes 29160, and allows 28668 (treating ERR logs as test failures)
to procede.
2019-01-23 12:04:57 -05:00
Nick Mathewson
adeeb8841e Merge branch 'maint-0.3.5' 2019-01-23 11:18:14 -05:00
rl1987
712a622fce Log an HSDesc we failed to parse at Debug loglevel 2019-01-23 10:37:10 -05:00
Peter Gerber
db3ee1d862
Allow getsockopt(…, SOL_SOCKET, SO_ACCEPTCONN, …) in sandbox
SO_ACCEPTCONN checks whether socket listening is enabled and is
used ever since 9369152aae has been merged.

Closes ticket #29150
2019-01-22 21:51:25 +00:00
Nick Mathewson
8e1e71cc0d Merge branch 'bug29122' 2019-01-22 14:31:01 -05:00
George Kadianakis
6243133a71 Fix intermittent failures of test_circuitpadding_wronghop().
We fix it by disabling the scheduling of actual padding.
Fixes #29122.
2019-01-22 14:30:43 -05:00
Nick Mathewson
21dd3ece62 Merge branch 'maint-0.3.5' 2019-01-22 11:55:56 -05:00
Nick Mathewson
4159d103d6 Merge branch 'bug29042_035' into maint-0.3.5 2019-01-22 11:55:53 -05:00
rl1987
00fff96e48 Fix shellcheck warning in test_rebind.sh 2019-01-22 15:14:16 +02:00
Nick Mathewson
d1af4d65df Merge branch 'maint-0.3.5' 2019-01-18 12:25:08 -05:00
Nick Mathewson
761f1bf71e Merge branch 'maint-0.3.3' into maint-0.3.4 2019-01-18 12:25:08 -05:00
Nick Mathewson
4cca7fb99b Merge branch 'maint-0.2.9' into maint-0.3.3 2019-01-18 12:25:08 -05:00
Nick Mathewson
a8580a6836 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-01-18 12:25:08 -05:00
Nick Mathewson
67f275f8da Bump to 0.4.0.1-alpha-dev 2019-01-18 10:08:57 -05:00
Nick Mathewson
81f1b89efc Better failure message on stochastic test failure 2019-01-17 17:27:13 -05:00
Nick Mathewson
c08fc2e19e Speed up the deterministic prng in test_prob_distr, by a lot.
Using a single xof object and squeezing it repeatedly should make
everything MUCH faster here.
2019-01-17 16:10:02 -05:00
Nick Mathewson
f632335feb Fix users of base32_decode to check for expected length in return.
Also, when we log about a failure from base32_decode(), we now
say that the length is wrong or that the characters were invalid:
previously we would just say that there were invalid characters.

Follow-up on 28913 work.
2019-01-17 13:32:19 -05:00
Nick Mathewson
b770adbd03 Use crypto_xof() in hs_ntor.c. 2019-01-17 12:43:20 -05:00
Nick Mathewson
9b0dd1ae04 Add a function to compute an XOF in one shot.
Motivation:
  1. It's convenient.
  2. It's all that openssl supports.

Part of 28837.
2019-01-17 12:43:20 -05:00
Nick Mathewson
c393171403 Use openssl's version of sha3 when available.
Part of 28837.
2019-01-17 12:43:20 -05:00
Nick Mathewson
77712a5fa2 Merge remote-tracking branch 'tor-github/pr/645' 2019-01-17 12:04:50 -05:00
Nick Mathewson
3da3aca08f Fix a LOG_ERR message from test_pt.c
Fixes an instance of ticket28668.
2019-01-16 15:54:35 -05:00
Nick Mathewson
3baf99e81b Merge branch 'ticket28668_035' into ticket28668_040 2019-01-16 15:54:31 -05:00
Nick Mathewson
f684cd8005 Fail any unit test that causes an unhandled LD_BUG or LOG_ERR
Closes ticket 28668.
2019-01-16 15:47:33 -05:00
Nick Mathewson
16430fdaf5 Merge branch 'maint-0.3.5' 2019-01-16 14:43:53 -05:00
Kris Katterjohn
cbcf75d728 Update and fix a couple of comments in meminfo.c
This fixes a typo and also notes that HW_PHYSMEM64 is defined on
NetBSD (not just OpenBSD).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-16 14:40:15 -05:00
Kris Katterjohn
0f3d88a159 Use HW_PHYSMEM instead of HW_USERMEM in get_total_system_memory_impl
The code checked for sysctl being available and HW_PHYSMEM being
defined, but HW_USERMEM was actually being used with sysctl instead
of HW_PHYSMEM.

The case for OpenBSD, etc. use HW_PHYSMEM64 (which is obviously a
64-bit variant of HW_PHYSMEM) and the case for OSX uses HW_MEMSIZE
(which appears to be a 64-bit variant of HW_PHYSMEM).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-16 14:40:15 -05:00
Nick Mathewson
0981ac4c59 Log more messages when failing to decode RSA keys
We log these messages at INFO level, except when we are reading a
private key from a file, in which case we log at WARN.

This fixes a regression from when we re-wrote our PEM code to be
generic between nss and openssl.

Fixes bug 29042, bugfix on 0.3.5.1-alpha.
2019-01-16 14:35:30 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
efe55b8898 Bump copyright date to 2019. 2019-01-16 12:32:32 -05:00
Nick Mathewson
dd524c6335 Bump to 0.4.0.1-alpha 2019-01-16 08:32:09 -05:00
Nick Mathewson
b169c8c14f Merge remote-tracking branch 'asn-github/adaptive_padding-final' 2019-01-14 14:48:00 -05:00
Nick Mathewson
691dec5d46 Merge branch 'maint-0.3.5' 2019-01-14 14:02:42 -05:00
rl1987
9d9e71824c Rework rep_hist_log_link_protocol_counts() 2019-01-14 14:01:55 -05:00
Nick Mathewson
99947c3ce0 Merge remote-tracking branch 'teor/bug29015' 2019-01-14 13:59:07 -05:00
Alexander Færøy
59a88b3c3a Rename TYPE to TRANSPORT in PT STATUS messages.
See: https://bugs.torproject.org/28181
2019-01-14 18:35:13 +01:00
Nick Mathewson
cef910642e Merge branch 'bug28989_squashed' 2019-01-11 18:56:57 -05:00
rl1987
b69d28ae83 Fix a typo 2019-01-11 18:56:48 -05:00
rl1987
1bcececdd8 Fix CID 1442277 2019-01-11 18:56:48 -05:00
Kris Katterjohn
df1a8a657e Actually close the stdout pipe on error in process_unix_exec
When cleaning up after an error in process_unix_exec, the stdin
pipe was being double closed instead of closing both the stdin
and stdout pipes.  This occurred in two places.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-11 18:55:09 -05:00
Nick Mathewson
d21fa48cac Merge branch 'maint-0.3.5' 2019-01-11 18:53:24 -05:00
Nick Mathewson
efd765a948 Merge remote-tracking branch 'tor-github/pr/563' into maint-0.3.5 2019-01-11 18:53:18 -05:00
Nick Mathewson
5dd926caa6 Include the beginnings of a FAQ about which timer to use. 2019-01-11 09:08:51 -05:00
George Kadianakis
4dc189a904 Clarify immutability of global padding machine specs. 2019-01-11 14:41:28 +02:00
Taylor R Campbell
9728d3f8ac Fix wrong bases. 2019-01-11 14:41:28 +02:00
Taylor R Campbell
d82a8a7f9d Add some more type checking.
NOTE: This commit breaks the build, because there was a mistake in an
earlier change of exactly the sort that this is meant to detect!  I'm
leaving it broken for illustration.
2019-01-11 14:41:28 +02:00
Taylor R Campbell
0f8253bddb Use the distribution abstraction as an abstraction. 2019-01-11 14:40:50 +02:00
Taylor R Campbell
531df9590d Move ceil call back into the geometric sampler.
Test exactly what the geometric sampler returns, because that's what
the downstream callers of it are going to use.

While here, also assert that the geometric sampler returns a positive
integer.  (Our geometric distribution is the one suported on {1, 2,
3, ...} that returns the number of trials before the first success,
not the one supported on {0, 1, 2, ...} that returns the number of
failures before the first success.)
2019-01-11 14:40:50 +02:00
Taylor R Campbell
948856c03e Fix more type redefinition errors.
In file included from ./src/core/or/or_circuit_st.h:12:0,
                 from src/core/or/circuitlist.c:112:
./src/core/or/circuit_st.h:15:39: error: redefinition of typedef ‘circpad_machine_spec_t’
./src/core/or/circuitpadding.h:572:3: note: previous declaration of ‘circpad_machine_spec_t’ was here
./src/core/or/circuit_st.h:16:40: error: redefinition of typedef ‘circpad_machine_state_t’
./src/core/or/circuitpadding.h:517:3: note: previous declaration of ‘circpad_machine_state_t’ was here
2019-01-11 14:40:50 +02:00
Taylor R Campbell
ca54424602 Fix type redefinition errors.
In file included from src/core/or/connection_edge.c:70:0:
./src/core/or/circuitpadding.h:16:26: error: redefinition of typedef ‘circuit_t’
./src/core/or/or.h:930:26: note: previous declaration of ‘circuit_t’ was here
./src/core/or/circuitpadding.h:17:33: error: redefinition of typedef ‘origin_circuit_t’
./src/core/or/or.h:931:33: note: previous declaration of ‘origin_circuit_t’ was here
./src/core/or/circuitpadding.h:18:23: error: redefinition of typedef ‘cell_t’
./src/core/or/or.h:628:23: note: previous declaration of ‘cell_t’ was here

typedef doesn't work for forward declarations, but plain struct
without a typedef wrapper does (and unlike the _t type aliases makes
it clearer for everyone whether you're talking about the struct or
the pointer).
2019-01-11 14:40:50 +02:00
George Kadianakis
2b29bccb85 Misc trivial improvements around circpadding code. 2019-01-10 13:06:08 +02:00
George Kadianakis
4db9c3d63e Unittest for tor_isinf(). 2019-01-10 13:06:08 +02:00
George Kadianakis
e0e0338dc4 Rename crypto_rand_uint32() -> crypto_rand_u32()
See https://github.com/torproject/tor/pull/624#discussion_r246453777
2019-01-10 13:06:08 +02:00
George Kadianakis
f4938179c5 Rename circpad_machineinfo_t -> circpad_machine_state_t 2019-01-10 13:06:08 +02:00
George Kadianakis
5738a0ab6c Rename circpad_machine_t -> circpad_machine_spec_t 2019-01-10 13:06:08 +02:00
George Kadianakis
cdbd5c0af8 Add top-level file documentation for circuitpadding.c 2019-01-10 13:05:57 +02:00
teor
89a0b77740 rend: stop warning when clients send multiple rend establish cells
Stop logging "Tried to establish rendezvous on non-OR circuit..." as
a warning. Instead, log it as a protocol warning, because there is
nothing that relay operators can do to fix it.

Fixes bug 29029; bugfix on 0.2.5.7-rc.
2019-01-10 07:59:59 +10:00
Nick Mathewson
2f0e187191 Merge remote-tracking branch 'tor-github/pr/627' 2019-01-09 15:46:14 -05:00
Nick Mathewson
578a93bbbc Merge branch 'ticket28843' 2019-01-09 09:49:25 -05:00
Nick Mathewson
0ed966fde1 Merge branch 'maint-0.3.5' 2019-01-09 09:38:27 -05:00
Nick Mathewson
694e3c57fd Merge remote-tracking branch 'tor-github/pr/636' into maint-0.3.5 2019-01-09 09:38:22 -05:00
Nick Mathewson
d890ec29aa Merge branch 'maint-0.3.5' 2019-01-09 09:35:40 -05:00
rl1987
380ae2b74b Stop logging 'Your Guard' in circpathbias.c 2019-01-09 09:35:36 -05:00
Nick Mathewson
3df1d29d9b Merge branch 'bug28591_035_squashed' 2019-01-09 09:33:31 -05:00
teor
44db455cc8 Dir: allow directory mirrors to serve future consensuses
When Tor's clock is behind the clocks on the authorities, allow Tor to
serve future consensuses.

Fixes bug 28654; bugfix on 0.3.0.1-alpha.
2019-01-09 09:32:28 -05:00
Nick Mathewson
e6a4401200 Merge branch 'maint-0.3.5' 2019-01-09 08:52:10 -05:00
Kris Katterjohn
c13a81f7f9 Fix (and make consistent) the use of OpenBSD preprocessor macro tests
Prior to this commit, the testsuite was failing on OpenBSD.  After
this commit the testsuite runs fine on OpenBSD.

It was previously decided to test for the OpenBSD macro (rather than
__OpenBSD__, etc.) because OpenBSD forks seem to have the former
macro defined.  sys/param.h must be included for the OpenBSD macro
definition; however, many files tested for the OpenBSD macro without
having this header included.

This commit includes sys/param.h in the files where the OpenBSD macro
is used (and sys/param.h is not already included), and it also
changes some instances of the __OpenBSD__ macro to OpenBSD.

See commit 27df23abb6 which changed
everything to use OpenBSD instead of __OpenBSD__ or OPENBSD.  See
also tickets #6982 and #20980 (the latter ticket is where it was
decided to use the OpenBSD macro).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-09 08:51:57 -05:00
Nick Mathewson
f9a7701ff1 Merge branch 'ticket28856_v2' 2019-01-09 08:46:40 -05:00
teor
85c598cbc2 stats: Make PaddingStatistics depend on ExtraInfoStatistics
When ExtraInfoStatistics is 0, stop including PaddingStatistics in
relay and bridge extra-info documents.

Fixes bug 29017; bugfix on 0.3.1.1-alpha.
2019-01-08 18:21:10 +10:00
teor
671c34d9b4
lib/net: improve the docs for tor_{ersatz_,}socketpair()
Add some details about IP family support, and point to
tor_socketpair() from tor_ersatz_socketpair().

Closes ticket 29015.
2019-01-08 14:45:17 +10:00
Nick Mathewson
44edfa7adc Update to 0.3.5.7-dev 2019-01-07 21:18:25 -05:00
Nick Mathewson
41bb778227 Update to 0.3.4.10-dev 2019-01-07 21:17:34 -05:00
Nick Mathewson
b5fdb91ada Update to 0.3.3.3.11-dev 2019-01-07 21:16:40 -05:00
Kris Katterjohn
c59b572e93 Fix tor_ersatz_socketpair on IPv6-only systems
In get_local_listener used by tor_ersatz_socketpair, the address
family used when binding the IPv6 socket was AF_INET instead of
AF_INET6.

Fixes bug 28995; bugfix on 0.3.5.1-alpha.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-07 17:28:21 -06:00
Nick Mathewson
52b508735f Bump version to 0.3.5.7 2019-01-07 10:23:21 -05:00
Nick Mathewson
ea47563dd7 Bump to 0.3.4.10 2019-01-07 10:22:42 -05:00
Nick Mathewson
63eed6edff Bump to 0.3.3.11 2019-01-07 10:21:59 -05:00
Nick Mathewson
7a7685955e Merge remote-tracking branch 'tlyu-github/bug28929' 2019-01-07 08:46:27 -05:00
Nick Mathewson
85c5f58509 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-01-07 08:43:46 -05:00
Nick Mathewson
fe26e1f1c8 Merge branch 'maint-0.3.5' 2019-01-07 08:43:46 -05:00
Nick Mathewson
3d18903f53 Merge branch 'maint-0.3.3' into maint-0.3.4 2019-01-07 08:43:45 -05:00
Nick Mathewson
ba71b17865 Merge branch 'maint-0.2.9' into maint-0.3.3 2019-01-07 08:43:45 -05:00
Karsten Loesing
445d6750f7 Update geoip and geoip6 to the January 3 2019 database. 2019-01-07 09:00:29 +01:00
Nick Mathewson
41e3c760a4 Fix a warning in test_process.c on 32-bit platforms with clang.
Bug not in any released Tor.
2019-01-04 18:39:22 -05:00
Taylor Yu
7e75311714 Fix typo in bootstrap message
The message for the "ap_conn_proxy" bootstrap status tag was missing
some text.  Fixes bug 28929.  Bug not in any released Tor.
2019-01-04 17:27:28 -06:00
Nick Mathewson
73de7b4bcb Merge branch 'bug21394_029_redux' into maint-0.2.9 2019-01-04 15:04:54 -05:00
Nick Mathewson
fe3331b845 Merge branch 'maint-0.3.3' into maint-0.3.4 2019-01-04 14:57:06 -05:00
Nick Mathewson
bf8bb9c526 Merge remote-tracking branch 'dgoulet/ticket28619_033_01' into maint-0.3.3 2019-01-04 14:49:16 -05:00
Nick Mathewson
d47c9276de Merge branch 'ticket27750_034_01_squashed' into maint-0.3.4 2019-01-04 14:34:38 -05:00
Nick Mathewson
98a1b8770c Merge branch 'maint-0.3.5' 2019-01-03 21:34:43 -05:00
Nick Mathewson
c75cf802d3 Merge branch 'ticket28879' into maint-0.3.5 2019-01-03 21:34:35 -05:00
Nick Mathewson
7232f04939 Merge branch 'ticket28880' 2019-01-03 21:33:40 -05:00
Nick Mathewson
a7cbbf279e Merge branch 'ticket28852' 2019-01-03 21:31:08 -05:00
Nick Mathewson
4e4f93d364 Add a #ifdef HAVE_UNISTD_H check to buffers.c
Reported on tor-dev by Gisle Vanem.  Bug not in any released Tor

(The suggested patch used _MSC_VER, but that's not how we do stuff
with autoconf.  With autoconf, you detect the feature you want,
rather than trying to list all the systems that do or do not have
it.)
2019-01-03 09:52:19 -05:00
Nick Mathewson
9ba690e33f Merge branch 'maint-0.3.5' 2019-01-03 09:45:56 -05:00
Nick Mathewson
abdc6aede2 Merge branch 'ticket28851_035_squashed' into maint-0.3.5 2019-01-03 09:45:53 -05:00
Nick Mathewson
b82717b273 Stop re-checking our hardcoded dh parameters on every startup
Closes ticket 28851.
2019-01-03 09:45:43 -05:00
Nick Mathewson
47176eb678 Merge branch 'maint-0.3.5' 2019-01-03 09:44:15 -05:00
Nick Mathewson
98736cf36a Merge remote-tracking branch 'public/ticket28838_035' into maint-0.3.5 2019-01-03 09:44:10 -05:00
Nick Mathewson
27853938a1 Merge branch 'maint-0.3.5' 2019-01-03 09:02:40 -05:00
Nick Mathewson
3e64553f76 Merge branch 'maint-0.3.3' into maint-0.3.4 2019-01-03 09:02:39 -05:00
Nick Mathewson
ed62f0fa15 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-01-03 09:02:39 -05:00
Nick Mathewson
bf413829cb Detect openssl bug 7712 and work around it.
In theory it would be better to detect this bug in advance, but this
approach is much simpler, and therefore safer to backport.

This closes tor issue 28973.
2019-01-03 09:00:47 -05:00
rl1987
dbf1725a13 Completely remove 'GETINFO status/version/num-{concurring,versioning}' 2019-01-03 10:36:38 +02:00
Nick Mathewson
3e7f13a4ef Merge branch 'maint-0.3.5' 2019-01-02 16:15:39 -05:00
Nick Mathewson
968235ce6f Explicitly ignore check_result() result in test_voting_flags_minimal
Otherwise, coverity complains at is.
2019-01-02 15:45:52 -05:00
Nick Mathewson
b33bcb1e63 Add an errno.h include to freespace.c to fix bug 28974. 2019-01-02 15:19:52 -05:00
George Kadianakis
56a45eb409 Disable current padding machines.
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
George Kadianakis
926fc93be5 Concentrate all TOR_USEC_PER_SEC definitions in a single header file.
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
George Kadianakis
dd04917851 Use the new probability distribution code in WTF-PAD.
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
Co-authored-by: Taylor R Campbell <campbell+tor@mumble.net>
2019-01-02 15:25:55 +02:00
George Kadianakis
2ccf326837 Implement and test probability distributions used by WTF-PAD.
This project introduces the prob_distr.c subsystem which implements all the
probability distributions that WTF-PAD needs. It also adds unittests for all of
them.

Code and tests courtesy of Riastradh.

Co-authored-by: Taylor R Campbell <campbell+tor@mumble.net>
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
Mike Perry
8ad497bb57 Config option to specify specific MiddleNodes.
Hope is this will make it easier to test on the live tor network.

Does not need to be merged if we don't want to, but will come in handy
for researchers.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:25:55 +02:00
George Kadianakis
a336d816a6 Circuit padding tests.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:25:55 +02:00
Mike Perry
d62340018c Add relay crypto mock points for tests.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:35 +02:00
Mike Perry
9aaf72ea58 Circuit padding implementation.
This implements all of the event handling, state machines, and padding
decisions for circuit padding.

I recommend reviewing this after you look at the call-in points into it from
the rest of Tor.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:31 +02:00
Mike Perry
7be71903da Circuit padding cell event callbacks.
These callbacks allow the padding state machines to react to various types of
sent and received relay cells.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:27 +02:00
Mike Perry
43701e1ebe Circuit padding machine creation events.
These event callbacks allow circuit padding to decide when to attempt to
launch and negotiate new padding machines, and when to tear old ones down.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:23 +02:00
Mike Perry
4ca1df6b32 Add padding negotiation trunnel output.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:20 +02:00
Mike Perry
659a4f06d4 Circuit padding ProtoVer plumbing.
This helps us to determine if a middle node can pad to us or not.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:16 +02:00
Mike Perry
70e9245f6f Initialize circuit padding machines and global state.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:13 +02:00
Mike Perry
2f7b5a2d44 Hook up circuit padding to circuit_t.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:09 +02:00
Mike Perry
2a24e21fb0 Circuit padding header.
This is a good code review start point, to get an overview of the interfaces
and types used in circuit padding.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:09:37 +02:00
Mike Perry
3ba7581129 Provide a smartlist reverse-order traversal.
We need this for padding negotiation so that we can have later machine
revisions supercede earlier ones.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:09:13 +02:00
Kris Katterjohn
c11247e957 Fix a buffer overflow in setup_cfg() in src/test/test_voting_flags.c
signed_descriptor_digest has a length of DIGEST_LEN but the memset
used to fill it used DIGEST256_LEN.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2018-12-23 17:46:08 -06:00
Nick Mathewson
99713b176b Merge branch 'maint-0.3.5' 2018-12-21 15:42:58 -05:00
Nick Mathewson
a9eec33649 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-12-21 15:42:57 -05:00
Nick Mathewson
70dd6d07bb Merge branch 'orconn-tracker_squashed' 2018-12-21 14:22:11 -05:00
Taylor Yu
f0f971409a Add tests for bootstrap tracker
Part of ticket 27617.
2018-12-21 14:15:35 -05:00
Taylor Yu
85542ee5a0 The big bootstrap phase redefinition
Redefine the set of bootstrap phases to allow display of finer-grained
progress in the early connection stages of connecting to a relay.

This includes adding intermediate phases for proxy and PT connections.

Also add a separate new phase to indicate obtaining enough directory
info to build circuits so we can report that independently of actually
initiating an ORCONN to build the first application circuit.
Previously, we would claim to be connecting to a relay when we had
merely finished obtaining directory info.

Part of ticket 27167.
2018-12-21 14:15:35 -05:00
Taylor Yu
936c93e562 Hook up control_event_bootstrap() to btrack_orconn
Replace a few invocations of control_event_bootstrap() with calls from
the bootstrap tracker subsystem.  This mostly leaves behavior
unchanged.  The actual behavior changes come in the next commit.

Part of ticket 27167.
2018-12-21 14:15:35 -05:00
Taylor Yu
9d29abb34e Add a comment about bto_update_best. 2018-12-21 14:15:21 -05:00
Nick Mathewson
fd58e5e498 Fix priority on process subsystem level: it uses "net" 2018-12-21 14:12:20 -05:00
Nick Mathewson
ab4395d082 Merge branch 'ticket28847' 2018-12-21 13:26:47 -05:00
Alexander Færøy
bc836d559d Don't initialize the process module manually in tests.
It's not longer needed for us to initialize the process module in tests.

See: https://bugs.torproject.org/28847
2018-12-21 13:26:38 -05:00
Alexander Færøy
2322b56389 Fix typo in time_sys.h. 2018-12-21 13:26:38 -05:00
Alexander Færøy
cf4b3dbd44 Use the subsystem list to initialize and shutdown process module.
This patch makes the process module use the subsystem list for
initializing and shutting down.

See: https://bugs.torproject.org/28847
2018-12-21 13:26:38 -05:00
David Goulet
2420e84ba4 mainloop: Reactivate the linked connection event with a non empty list
Linked connections aren't woken up by libevent due to I/O but rather
artificially so we can, by chunks, empty the spooled object(s).

Commit 5719dfb48f (in 0.3.4.1-alpha) made it
that the schedule_active_linked_connections_event would be only called once at
startup but this is wrong because then we would never go through again the
active linked connections.

Fortunately, everytime a new linked connection is created, the event is
activated and thus we would go through the active list again. On a busy relay,
this issue is mitigated by that but on a slower relays or bridge, a connection
could get stuck for a while until a new directory information request would
show up.

Fixes #28717, #28912
2018-12-21 11:25:23 -05:00
Taylor Yu
b0ae6a332a Add bootstrap tracker subsystem
Add a tracker for bootstrap progress, tracking events related to
origin circuit and ORCONN states.  This uses the ocirc_event and
orconn_event publish-subscribe subsystems.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
b0f974633a Add LD_BTRACK log domain for bootstrap tracker
Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
a0b4fa1f16 Add origin circuit event pubsub system
Add a publish-subscribe subsystem to publish messages about changes to
origin circuits.

Functions in circuitbuild.c and circuitlist.c publish messages to this
subsystem.

Move circuit event constants out of control.h so that subscribers
don't have to include all of control.h to take actions based on
messages they receive.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
271b50f54a Add ORCONN event pubsub system
Add a publish-subscribe subsystem to publish messages about changes to
OR connections.

connection_or_change_state() in connection_or.c and
control_event_or_conn_event() in control.c publish messages to this
subsystem via helper functions.

Move state constants from connection_or.h to orconn_state.h so that
subscribers don't have to include all of connection_or.h to take
actions based on changes in OR connection state.  Move event constants
from control.h for similar reasons.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
308dde0c38 Remove unused old_state var in connection_or.c
connection_or_change_state() saved an old_state to pass to
channel_tls_handle_state_change_on_orconn(), which promptly cast it to
void.  Remove this unused variable and parameter.
2018-12-20 17:54:49 -06:00
Nick Mathewson
e4109020e9 Merge remote-tracking branch 'tor-github/pr/609' 2018-12-20 16:42:35 -05:00
Alexander Færøy
ab0d7d2dd4 Escape the PT K/V data before sending it to the logger.
See: https://bugs.torproject.org/28846
2018-12-20 19:05:50 +01:00
Nick Mathewson
5c85ba3077 Merge remote-tracking branch 'tor-github/pr/608' 2018-12-20 11:42:26 -05:00
Nick Mathewson
22c5ad682c Add base32 to the round-trip fuzzer 2018-12-20 08:37:04 -05:00
Alexander Færøy
01819faaba Remove Process initializer/shutdown function from main.c.
See: https://bugs.torproject.org/28847
2018-12-20 14:36:59 +01:00
Nick Mathewson
a517daa56f base32_decode(): Return number of bytes written on success.
This makes it consistent with base64_decode().

Closes ticket 28913.
2018-12-20 08:36:25 -05:00
Alexander Færøy
f7e175db57 Forward declare smartlist_t in process.h
This allows other libraries to include process.h without including
the smartlist_t headers first.

See: https://bugs.torproject.org/28847
2018-12-20 14:36:04 +01:00
Nick Mathewson
973a5db808 Merge remote-tracking branch 'tor-github/pr/445' 2018-12-20 07:53:57 -05:00
Alexander Færøy
7762088967 No need to log ordinary EOF conditions as LOG_WARN.
Let's not use log_warn() when a pipe is closed under what should be
considered normal conditions.

See: https://bugs.torproject.org/28179
2018-12-20 13:12:53 +01:00
Alexander Færøy
412fbe9f17 Make example CancelIoEx() code use CancelIo().
This patch changes the CancelIoEx() example code to use CancelIo(),
which is available for older versions of Windows too. I still think the
kernel handles this nicely by sending broken pipes if either side
closes the pipe while I/O operations are pending.

See: https://bugs.torproject.org/28179
2018-12-20 13:11:24 +01:00
Alexander Færøy
f58e597d42 Handle ERROR_BROKEN_PIPE in completion routines.
Handle `ERROR_BROKEN_PIPE` from ReadFileEx() and WriteFileEx() in
process_win32_stdin_write_done() and
process_win32_handle_read_completion() instead of in the early handler.
This most importantmly makes sure that `reached_eof` is set to true when
these errors appears.

See: https://bugs.torproject.org/28179
2018-12-20 13:04:49 +01:00
Alexander Færøy
36e24782f8 Remember to set reached_eof when our handles are reporting errors.
This patch adds some missing calls to set `reached_eof` of our handles
when various error conditions happens or when we close our handle (which
happens at `process_terminate()`.

See: https://bugs.torproject.org/28179
2018-12-20 13:02:22 +01:00
Alexander Færøy
c6e041e3d8 Handle errors even after success from ReadFileEx() and WriteFileEx().
This patch adds some additional error checking after calls to
ReadFileEx() and WriteFileEx(). I have not managed to get this code to
reach the branch where `error_code` is NOT `ERROR_SUCCESS`, but MSDN
says one should check for this condition so we do so just to be safe.

See: https://bugs.torproject.org/28179
2018-12-20 12:57:20 +01:00
Alexander Færøy
44586a89ef Delay checking process for termination until both stdout and stderr are closed.
This patch makes us delay checking for whether we have an exit code
value (via GetExitCodeProcess()) until both stdout and stderr have been
closed by the operating system either by the process itself or by
process cleanup after termination.

See: https://bugs.torproject.org/28179
2018-12-20 12:53:28 +01:00
Alexander Færøy
1d8dcb416c Remember to close the child process' ends of the pipes.
This prevents us from leaking the HANDLE for stdout, stderr, and stdin.

See: https://bugs.torproject.org/28179
2018-12-20 12:47:04 +01:00
Alexander Færøy
fe2f4f3ec5 Remember to check for whether we actually did exit in tests.
See: https://bugs.torproject.org/28179
2018-12-20 12:45:52 +01:00
Alexander Færøy
4efe4cc2f9 Add support for STATUS messages from Pluggable Transports.
This patch adds support for the new STATUS message that PT's can emit
from their standard out. The STATUS message uses the `config_line_t` K/V
format that was recently added in Tor.

See: https://bugs.torproject.org/28846
2018-12-20 03:55:02 +01:00
Alexander Færøy
426c52b377 Use K/V parser to handle LOG messages for pluggable transports.
This patch changes the LOG pluggable transport message to use the recent
K/V parser that landed in Tor. This allows PT's to specify the log
severity level as well as the message. A mapping between the PT log
severity levels and Tor's log serverity level is provided.

See: https://bugs.torproject.org/28846
2018-12-20 03:41:28 +01:00
Nick Mathewson
1c47459e5a Merge branch 'maint-0.3.5' 2018-12-19 15:36:08 -05:00
Nick Mathewson
b7018b1a24 Merge branch 'ticket28883_035' into maint-0.3.5 2018-12-19 15:36:03 -05:00
Nick Mathewson
ed0bc85ed0 Merge branch 'ticket28853' 2018-12-18 18:59:56 -05:00
Nick Mathewson
bb091da1e7 Merge branch 'ticket28839_v2_squashed' 2018-12-18 18:59:05 -05:00
Nick Mathewson
7113a339dc Avoid a needless decode/re-encode step in assigning onion keys
Previously we had decoded the asn.1 to get a public key, and then
discarded the asn.1 so that we had to re-encode the key to store it
in the onion_pkey field of a microdesc_t or routerinfo_t.

Now we can just do a tor_memdup() instead, which should be loads
faster.
2018-12-18 18:58:08 -05:00
Nick Mathewson
0556942284 Use a single path for all PEM-like objects in get_next_token()
Previously, we would decode the PEM wrapper for keys twice: once in
get_next_token, and once later in PEM decode.  Now we just do all of
the wrapper and base64 stuff in get_next_token, and store the
base64-decoded part in the token object for keys and non-keys alike.

This change should speed up parsing slightly by letting us skip a
bunch of stuff in crypto_pk_read_*from_string(), including the tag
detection parts of pem_decode(), and an extra key allocation and
deallocation pair.

Retaining the base64-decoded part in the token object will allow us
to speed up our microdesc parsing, since it is the asn1 portion that
we actually want to retain.
2018-12-18 18:58:08 -05:00
Nick Mathewson
372df7a630 Merge branch 'maint-0.3.5' 2018-12-18 13:56:22 -05:00
Nick Mathewson
26bbeb298d Merge branch 'bug28612_squashed' into maint-0.3.5 2018-12-18 13:55:57 -05:00
Nick Mathewson
1c2abea30a Call run_tor_main_loop() in ntmain.c, rather than do_main_loop().
Fixes bug 28612; bugfix on 0.3.5.3-alpha.
2018-12-18 13:55:08 -05:00
Nick Mathewson
0af0f78dff Merge branch 'maint-0.3.5' 2018-12-18 13:52:39 -05:00
Nick Mathewson
702fd6f0f2 Merge branch 'ticket28881_035' into maint-0.3.5 2018-12-18 13:52:36 -05:00
Nick Mathewson
4894d44ab8 Always initialize addr in parse_port_config()
It was always analyzed before use, but scan-build wasn't able to
persuade itself of that.

Closes ticket 28881.
2018-12-18 13:52:25 -05:00
Nick Mathewson
a3e6f2467b Merge remote-tracking branch 'tor-github/pr/595' 2018-12-18 13:51:21 -05:00
Alexander Færøy
ca7a2ecc51 Avoid breaking the event loop prematurely.
This patch makes sure that we terminate the event loop from the event
loop timer instead of directly in the process' exit handler. This allows
us to run the event loop an additional time to ensure that the SleepEx()
call on Windows is called and the data from stdout/stderr is delivered
to us.

Additionally we ensure that we don't try to read or write data from a
Unix process that have been terminated in the main loop, since its file
descriptors are closed at that time.

See: https://bugs.torproject.org/28179
2018-12-18 13:35:29 -05:00
Nick Mathewson
bf71dce01a Bump version to 0.3.5.6-rc-dev 2018-12-18 13:33:49 -05:00
Nick Mathewson
c61cd5775c Revert "Log bootstrap tag names"
This reverts commit 1b855af5e3.
2018-12-18 08:09:43 -05:00
Nick Mathewson
d8f41c2870 Bump to 0.3.5.6-rc 2018-12-18 08:04:04 -05:00
rl1987
c659603ac5 Unit test to check that we can parse NETINFO cell with unsupported address type 2018-12-18 12:11:33 +02:00
rl1987
c92c0cbc9f Actually allow unrecognized address types in NETINFO cell
Ignore the address value instead of failing with error condition in case
unrecognized address type is found.
2018-12-18 12:10:08 +02:00
Nick Mathewson
8a01f0eaab lib/process may include lib/buf. 2018-12-17 17:58:49 -05:00
Nick Mathewson
4ad59bfbc2 Update location of buffers.h 2018-12-17 17:01:50 -05:00
Nick Mathewson
e969d9c6b4 Merge branch 'ticket28179_squashed' into ticket28179_squashed_merged 2018-12-17 16:41:01 -05:00
Alexander Færøy
c8b8b15f0e Ensure that line_size >= 1 before trying to trim input string.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
651cdd05b7 Add an additional space when we check for the PROTO_LOG handler.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
fab22509d7 Make Windows process event timer API available for dormant interface.
This patch changes the API of the Windows backend of the Process
subsystem to allow the dormant interface to disable the Process event
timer.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
a33a77d9cd Document the format of process_t::arguments.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
0d796cce17 Use errno directly if we are not reading/writing from/to a socket.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
cacdd29087 Use const char * instead of char * for line parameter for process callbacks.
This patch changes the type definition of the process callbacks to use
`const char *` instead of `char *`.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ec2ae3ed8b Change EVENT_TRANSPORT_LOG to EVENT_PT_LOG.
This patch changes our EVENT_TRANSPORT_LOG event to be EVENT_PT_LOG. The
new message includes the path to the PT executable instead of the
transport name, since one PT binary can include multiple transport they
sometimes might need to log messages that are not specific to a given
transport.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
5585cbd08f Change the Process exit_callback to return bool.
This patch changes our process_t's exit_callback to return a boolean
value.  If the returned value is true, the process subsystem will call
process_free() on the given process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
22cb3c6ce9 Call close() on stdin/stdout/stderr in process_terminate().
Call close() on all process handles after we have called kill(pid,
SIGTERM).

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
bc6983afed Use run_main_loop_until_done() for process_t tests.
This patch changes the slow process_t tests to use
run_main_loop_until_done() instead of do_main_loop() since
do_main_loop() initializes a lot of subsystem callbacks that we don't
need to run in our tests.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
6e508e9eb4 Fix tests on kqueue() based platforms.
This patch disables fork()'ing of the slow process tests. This fixes the
tests on the MacOS and other kqueue() based platforms.

Without this patch the main loop exits eearly with EBADF as error.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ccc1963890 Move remaining code from subprocess.{h,c} to more appropriate places.
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.

We also delete the now empty subprocess module files.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
f7d13425fc Delete old process_handle_t code.
This patch removes the old process_handle_t code. Everything should by
now be using the process_t interface.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
289ed0849d Add tests for process environment functionality of process_t.
This patch adds tests for the process_environment_t interaction in
process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
9b6a10a26f Add slow test for process_t for main loop interaction.
This patch adds test cases for process_t which uses Tor's main loop.
This allows us to test that the callbacks are actually invoked by the
main loop when we expect them.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e3ceaebba2 Add support for logging messages from pluggable transports.
This patch adds support for the "LOG" protocol message from a pluggable
transport. This allows pluggable transport developers to relay log
messages from their binary to Tor, which will both emit them as log
messages from the Tor process itself, but also pass them on via the
control port.

See: https://bugs.torproject.org/28180
See: https://bugs.torproject.org/28181
See: https://bugs.torproject.org/28182
2018-12-17 16:39:28 -05:00
Alexander Færøy
bfb94dd2ca Use process_t for managed proxies.
This patch makes the managed proxy subsystem use the process_t data
structure such that we can get events from the PT process while Tor is
running and not just when the PT process is being configured.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ad4cc89c5d Add "PT" log domain.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
b0d268a822 Add process_reset_environment() to the Process subsystem.
This patch adds a new function that allows us to reset the environment
of a given process_t with a list of key/value pairs.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
4f611a1df7 Add process_terminate().
This patch adds support for process termination to the Process
subsystem.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
338137221c Make sure we call process_notify_event_exit() as the last thing in different callbacks.
This patch makes sure that we call process_notify_event_exit() after we
have done any modifications we need to do to the state of a process_t.
This allows application developers to call process_free() in the
exit_callback of the process.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e982fb1dae Add documentation for the is_socket and error argument of read_to_chunk().
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
89393a77e5 Add process_get_pid() to the Process subsystem.
This patch adds support for getting the unique process identifier from a
given process_t. This patch implements both support for both the Unix
and Microsoft Windows backend.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
bb784cf4f3 Add Windows backend for the Process subsystem.
This patch adds support for Microsoft Windows in the Process subsystem.

Libevent does not support mixing different types of handles (sockets,
named pipes, etc.) on Windows in its core event loop code. This have
historically meant that Tor have avoided attaching any non-networking
handles to the event loop. This patch uses a slightly different approach
to roughly support the same features for the Process subsystem as we do
with the Unix backend.

In this patch we use Windows Extended I/O functions (ReadFileEx() and
WriteFileEx()) which executes asynchronously in the background and
executes a completion routine when the scheduled read or write operation
have completed. This is much different from the Unix backend where the
operating system signals to us whenever a file descriptor is "ready" to
either being read from or written to.

To make the Windows operating system execute the completion routines of
ReadFileEx() and WriteFileEx() we must get the Tor process into what
Microsoft calls an "alertable" state. To do this we execute SleepEx()
with a zero millisecond sleep time from a main loop timer that ticks
once a second.  This moves the process into the "alertable" state and
when we return from the zero millisecond timeout all the outstanding I/O
completion routines will be called and we can schedule the next reads
and writes.

The timer loop is also responsible for detecting whether our child
processes have terminated since the last timer tick.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
2e957027e2 Add Unix backend for the Process subsystem.
This patch adds the Unix backend for the Process subsystem. The Unix
backend attaches file descriptors from the child process's standard in,
out and error to Tor's libevent based main loop using traditional Unix
pipes. We use the already available `waitpid` module to get events
whenever the child process terminates.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
35509978dd Add new Process subsystem.
This patch adds a new Process subsystem for running external programs in
the background of Tor. The design is focused around a new type named
`process_t` which have an API that allows the developer to easily write
code that interacts with the given child process. These interactions
includes:

- Easy API for writing output to the child process's standard input
  handle.
- Receive callbacks whenever the child has output on either its standard
  output or standard error handles.
- Receive callback when the child process terminates.

We also support two different "protocols" for handling output from the
child process. The default protocol is the "line" protocol where the
process output callbacks will be invoked only when there is complete
lines (either "\r\n" or "\n" terminated). We also support the "raw"
protocol where the read callbacks will get whatever the operating system
delivered to us in a single read operation.

This patch does not include any operating system backends, but the Unix
and Windows backends will be included in separate commits.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Taylor R Campbell
ed71e1e89c Create a temporary directory for tor's DataDirectory in test_rebind.
Fixes #28562.

While here, put the argument count test and usage message _before_ we
attempt to read from sys.argv.
2018-12-17 10:32:28 -05:00
Nick Mathewson
315c21d2e2 test_rebind: wait for tor to timeout, even if it is logging a lot
Fixes bug 28883; bugfix on 0.3.5.4-alpha.
2018-12-17 09:53:17 -05:00
Nick Mathewson
ce3d501040 Fix null-pointer-deref warning from scan-build in test_hs_service.c 2018-12-17 09:28:33 -05:00
Nick Mathewson
d58a597a55 Fix dead assignment warning in test_hs_service.c 2018-12-17 09:28:08 -05:00
Nick Mathewson
f50558ce8c Fix dead-assignment warning in test_shared_random.c 2018-12-17 09:27:40 -05:00
Nick Mathewson
82fb40c8dc Fix dead-assignment warnings in test_config.c
Found by scan-build.
2018-12-17 09:26:57 -05:00
Nick Mathewson
16199a54a2 Check hostname before using it in send_resolved_hostname_cell()
Also, turn an absent hostname into a BUG(), not a crash.

Found by scan-build.

Closes ticket 28879; bugfix on 0.1.2.7-alpha
2018-12-17 09:15:37 -05:00
Nick Mathewson
29254812a3 Remove strcmp_len(): it is now unused
(See 28856.)
2018-12-17 09:04:25 -05:00
Nick Mathewson
a0fad3981e Replace use of strcmp_len() with new mem_eq_token().
The strcmp_len() function was somewhat misconceived, since we're
only using it to test whether a length+extent string is equal to a
NUL-terminated string or not.  By simplifying it and making it
inlined, we should be able to make it a little faster.

(It *does* show up in profiles.)

Closes ticket 28856.
2018-12-17 09:03:04 -05:00
rl1987
5b2acbec0e Refrain from closing connection if found one unrecognized address in NETINFO cell 2018-12-16 10:19:37 +02:00
rl1987
3bec371d04 Refrain from hardcoding address length and type in netinfo.trunnel 2018-12-16 10:05:35 +02:00
Nick Mathewson
3dd1f064a7 Rewrite the core of parse_short_policy() to be faster.
The old implementation did some funky out-of-order lexing, and
tended to parse every port twice if the %d-%d pattern didn't match.

Closes ticket 28853.
2018-12-14 16:07:10 -05:00
Nick Mathewson
9dc53bc68f Remove a needless memset() in get_token_arguments()
I believe we originally added this for "just in case" safety, but it
isn't actually needed -- we never copy uninitialized stack here.
What's more, this one memset is showing up on our startup profiles,
so we ought to remove it.

Closes ticket 28852.
2018-12-14 14:48:12 -05:00
Nick Mathewson
6dc90d290d Use 25% less RAM for base64-encoded directory objects
We were allocating N bytes to decode an N-byte base64 encoding,
when 3N/4 would have been enough.
2018-12-14 13:51:51 -05:00
Nick Mathewson
3c35c0d441 Add a function to provide an upper bound on base64 decoded length 2018-12-14 13:51:51 -05:00
Nick Mathewson
cf7342ab6f Add a benchmark for parsing a microdescriptor 2018-12-14 13:51:51 -05:00
Rob Jansen
325348b360 allow any value for HearbeatPeriod in testing Tor networks 2018-12-14 09:22:23 -05:00
Nick Mathewson
4bc3983f64 Add a DROPOWNERSHIP controller command to undo TAKEOWNERSHIP.
Closes ticket 28843.
2018-12-13 19:35:02 -05:00
Nick Mathewson
f8dac5c900 Merge branch 'maint-0.3.5' 2018-12-13 19:01:29 -05:00
Nick Mathewson
94a7998158 Merge remote-tracking branch 'tlyu-github/ticket28731-035' into maint-0.3.5 2018-12-13 18:57:00 -05:00
Nick Mathewson
041e9235c1 Lower the loop_max constant in curve25519_basepoint_spot_check()
The point of this function is to make sure that the ed25519-based
implementation of curve25519_basepoint() actually works when we
start tor, and use the regular fallback implementation if it
doesn't.  But it accounts for 9% of our startup time in the case
when we have directory information, and I think it's safe to make
the test shorter.  After all, it has yet to find any actual bugs in
curved25519_scalarmult_basepoint_donna() on any platforms.

Closes ticket 28838.
2018-12-13 11:26:09 -05:00
Nick Mathewson
69264f96f3 Merge branch 'dormant_persist_squashed' 2018-12-13 08:26:10 -05:00
Nick Mathewson
e3b7fd2a81 Unit tests for back-end functions for persistent dormant state 2018-12-13 08:25:54 -05:00
Nick Mathewson
b5c04173c8 Change interaction between dormant mode and clock jumps.
When the clock jumps, and we have a record of last user activity,
adjust that record.  This way if I'm inactive for 10 minutes and
then the laptop is sleeping for an hour, I'll still count as having
been inactive for 10 minutes.

Previously, we treat every jump as if it were activity, which is
ridiculous, and would prevent a Tor instance with a jumpy clock from
ever going dormant.
2018-12-13 08:25:54 -05:00
Nick Mathewson
4afc6b172a Merge branch 'ticket28755_v2_squashed' 2018-12-12 11:07:38 -05:00
Nick Mathewson
845e8dbe59 Fuzzing module for various string operations, currently focusing on
encoding and decoding.

There are bunches of places where we don't want to invest in a full
fuzzer, but we would like to make sure that some string operation
can handle all its possible inputs.  This fuzzer uses the first byte
of its input to decide what to do with the rest of the input.  Right
now, all the possibilities are decoding a string, and seeing whether
it is decodeable.  If it is, we try to re-encode it and do the whole
thing again, to make sure we get the same result.

This turned up a lot of bugs in the key-value parser, and I think it
will help in other cases too.

Closes ticket 28808.
2018-12-12 11:07:08 -05:00
Nick Mathewson
f0a8664677 Add code to parse K=V lines into config_line_t format.
Closes ticket 28755
2018-12-12 11:07:08 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
6506b1ee9f Merge branch 'maint-0.3.3' into maint-0.3.4 2018-12-11 09:41:05 -05:00
Nick Mathewson
e1273d7d1b Merge branch 'maint-0.3.4' into maint-0.3.5 2018-12-11 09:41:05 -05:00
Nick Mathewson
27e4269929 Merge branch 'maint-0.3.5' 2018-12-11 09:41:05 -05:00
Nick Mathewson
c1f9191581 Merge branch 'maint-0.2.9' into maint-0.3.3 2018-12-11 09:41:04 -05:00
Nick Mathewson
ce501a529f Merge remote-tracking branch 'catalyst-github/ticket27402' 2018-12-11 09:37:41 -05:00
Taylor Yu
1b855af5e3 Log bootstrap tag names
Add the bootstrap tag name to the log messages, so people
troubleshooting connection problems can look up a symbol instead of a
number.  Closes ticket 28731.
2018-12-10 17:22:28 -06:00
teor
4991b29311 Fallbacks: Update the hard-coded fallback list in December 2018
Merge Phoul's two lists into teor's list.

Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in
January 2018 (of which ~115 were still functional), with a list of
157 fallbacks (92 new, 65 existing, 85 removed) generated in
December 2018.

Closes ticket 24803.
2018-12-10 17:02:19 +10:00
teor
78e177d622 Fallbacks: Update the hard-coded fallback list in December 2018
Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in
January 2018 (of which ~115 were still functional), with a list of
148 fallbacks (89 new, 59 existing, 91 removed) generated in
December 2018.

Closes ticket 24803.
2018-12-07 16:43:10 +10:00
Nick Mathewson
2ccb9e9444 Merge branch 'maint-0.3.5' 2018-12-06 09:26:34 -05:00
Nick Mathewson
c1f86f7492 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-12-06 09:26:33 -05:00
Nick Mathewson
c4f7953d8b Merge branch 'maint-0.3.3' into maint-0.3.4 2018-12-06 09:26:32 -05:00
Nick Mathewson
00341d97f3 Merge branch 'maint-0.2.9' into maint-0.3.3 2018-12-06 09:26:32 -05:00
Karsten Loesing
57798eb1cb Update geoip and geoip6 to the December 5 2018 database. 2018-12-05 21:02:39 +01:00
Nick Mathewson
46a321fbdd Merge branch 'maint-0.3.5' 2018-12-05 10:25:12 -05:00
Nick Mathewson
967efc0d28 Merge remote-tracking branch 'tor-github/pr/546' into maint-0.3.5 2018-12-05 10:23:28 -05:00
Nick Mathewson
1eb3719a62 Merge remote-tracking branch 'public/prop298' 2018-12-05 09:43:03 -05:00
Nick Mathewson
1f95e80351 Merge branch 'prop293_squashed' 2018-12-05 09:24:51 -05:00
Nick Mathewson
a2f81b644b Write tests for mark_my_descriptor_dirty_if_too_old() 2018-12-05 09:24:45 -05:00
Nick Mathewson
00509aaafa Merge remote-tracking branch 'tor-github/pr/553' 2018-12-05 08:28:54 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
Nick Mathewson
c01507a5fe remember why we are doing getsockopt() 2018-12-05 08:14:21 -05:00
David Goulet
cec616a0c8 hs-v3: Don't BUG() if descriptor is found on SOCKS connection retry
When retrying all SOCKS connection because new directory information just
arrived, do not BUG() if a connection in state AP_CONN_STATE_RENDDESC_WAIT is
found to have a usable descriptor.

There is a rare case when this can happen as detailed in #28669 so the right
thing to do is put that connection back in circuit wait state so the
descriptor can be retried.

Fixes #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:34:04 -05:00
David Goulet
43bd4d7509 hs-v3: Add the helper function mark_conn_as_waiting_for_circuit
This helper function marks an entry connection as pending for a circuit and
changes its state to AP_CONN_STATE_CIRCUIT_WAIT. The timestamps are set to
now() so it can be considered as new.

No behaviour change, this helper function will be used in next commit.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:34:04 -05:00
David Goulet
00b59d9281 conn: Use connection_ap_mark_as_waiting_for_renddesc()
Use the helper function connection_ap_mark_as_waiting_for_renddesc()
introduced in previous commit everywhere in the code where an AP connection
state is transitionned to AP_CONN_STATE_RENDDESC_WAIT.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:10:00 -05:00
David Goulet
d0682fe0f1 conn: Add an helper to mark a connection as waiting for an HS descriptor
The transition for a connection to either become or go back in
AP_CONN_STATE_RENDDESC_WAIT state must make sure that the entry connection is
_not_ in the waiting for circuit list.

This commit implements the helper function
connection_ap_mark_as_waiting_for_renddesc() that removes the entry connection
from the pending list and then change its state. This code pattern is used in
many places in the code where next commit will remove this code duplication to
use this new helper function.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:00:08 -05:00
Nick Mathewson
4f55884315 Add an option to start tor in dormant mode for the first time. 2018-12-04 12:08:24 -05:00