Commit Graph

24066 Commits

Author SHA1 Message Date
George Kadianakis
130eb227ac Merge branch 'tor-github/pr/1043' 2019-05-27 14:20:51 +03:00
Nick Mathewson
fcd51fd49f Tests for deciding how full our relay cells should be 2019-05-27 14:20:36 +03:00
Nick Mathewson
0bc1241494 Make sure that we send at least some random data in RELAY_DATA cells
Proposal 289 prevents SENDME-flooding by requiring the other side to
authenticate the data it has received.  But this data won't actually
be random if they are downloading a known resource.  "No problem",
we said, "let's fell the empty parts of our cells with some
randomness!" and we did that in #26871.

Unfortunately, if the relay data payloads are all completely full,
there won't be any empty parts for us to randomize.

Therefore, we now pick random "randomness windows" between
CIRCWINDOW_INCREMENT/2 and CIRCWINDOW_INCREMENT. We remember whether we have
sent a cell containing at least 16 bytes of randomness in that window.  If we
haven't, then when the window is exhausted, we send one.  (This window approach
is designed to lower the number of rng checks we have to do.  The number 16 is
pulled out of a hat to change the attacker's guessing difficulty to
"impossible".)

Implements 28646.
2019-05-27 14:20:07 +03:00
Roger Dingledine
94914e2a4d trivial whitespace fixes 2019-05-26 17:32:42 -04:00
Nick Mathewson
07ccffa989 Coverage: do not include test-rebind in coverage builds.
Because it invokes the Tor mainloop, it does unpredictable things to
test coverage of a lot of code that it doesn't actually test at
all.  (It is more an integration test than anything else.)
2019-05-23 12:48:51 -04:00
Nick Mathewson
b882810245 In coverage builds, use branch-free timeradd() and timersub()
The ordinary definitions of timeradd() and timersub() contain a
branch. However, in coverage builds, this means that we get spurious
complaints about partially covered basic blocks, in a way that makes
our coverage determinism harder to check.
2019-05-23 12:48:51 -04:00
Nick Mathewson
2bb5d8148b In coverage builds, avoid basic-block complexity in log_debug
Ordinarily we skip calling log_fn(LOG_DEBUG,...) if debug logging is
completely disabled.  However, in coverage builds, this means that
we get spurious complaints about partially covered basic blocks, in
a way that makes our coverage determinism harder to check.
2019-05-23 12:48:51 -04:00
David Goulet
29955f13e5 Merge branch 'tor-github/pr/1022' 2019-05-23 09:50:28 -04:00
David Goulet
e13e2012b9 Merge branch 'tor-github/pr/1034' 2019-05-23 09:40:07 -04:00
David Goulet
327bb0e2ca Merge branch 'tor-github/pr/988' 2019-05-23 09:30:36 -04:00
Nick Mathewson
530d1179ff Extract length-deciding function from package_raw_inbuf. 2019-05-23 08:28:46 -04:00
Roger Dingledine
e4d1187584 refactor logic to decide how much to package from inbuf
no actual changes in behavior
2019-05-23 08:28:46 -04:00
Nick Mathewson
57ee0e3af9 Only reject POSTDESCRIPTOR purpose= when the purpose is unrecognized
Fixes bug 30580; bugfix on 0.4.1.1-alpha.
2019-05-23 08:24:29 -04:00
Nick Mathewson
ebe39dcb92 Now this repository is full of 0.4.1.1-alpha-dev 2019-05-22 18:07:29 -04:00
Nick Mathewson
fa410162a3 circuitpadding tests: Use tt_i64_op() to compare int64_t values
Bug not in any released Tor.
2019-05-22 15:19:24 -04:00
Nick Mathewson
3a7ed8bc5f Bump to 0.4.1.1-alpha 2019-05-22 11:56:02 -04:00
Nick Mathewson
24c2502070 Merge remote-tracking branch 'dgoulet/ticket30454_035_01' 2019-05-22 11:50:46 -04:00
Nick Mathewson
e6b862e6a8 Merge branch 'ticket30428_041_02_squashed' 2019-05-22 11:48:43 -04:00
David Goulet
0cad83bea4 sendme: Add non fatal asserts for extra safety
Two non fatal asserts are added in this commit. First one is to see if the
SENDME digest list kept on the circuit for validation ever grows bigger than
the maximum number of expected SENDME on a circuit (currently 10).

The second one is to know if we ever send more than one SENDME at a time on a
circuit. In theory, we shouldn't but if we ever do, the v1 implementation
wouldn't work because we only keep one single cell digest (the previous cell
to the SENDME) on the circuit/cpath. Thus, sending two SENDME consecutively
will lead to a mismatch on the other side because the same cell digest would
be use and thus the circuit would collapse.

Finally, add an extra debug log in case we emit a v0 which also includes the
consensus emit version in that case.

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
5479ffabf8 sendme: Always pop last SENDME digest from circuit
We must not accumulate digests on the circuit if the other end point is using
another SENDME version that is not using those digests like v0.

This commit makes it that we always pop the digest regardless of the version.

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
482c4972b9 sendme: Clarify how sendme_circuit_cell_is_next() works
Commit 4ef8470fa5480d3b was actually reverted before because in the end we
needed to do this minus 1 check on the window.

This commit clarifies that in the code, takes the useful comment changes from
4ef8470fa5480d3b and makes sendme_circuit_cell_is_next() private since it
behaves in a very specific way that one external caller might expect.

Part of #30428.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
3835a3acf5 sendme: Properly record SENDMEs on both edges
Turns out that we were only recording the "b_digest" but to have
bidirectionnal authenticated SENDMEs, we need to use the "f_digest" in the
forward cell situation.

Because of the cpath refactoring, this commit plays with the crypt_path_ and
relay_crypto_t API a little bit in order to respect the abstractions.

Previously, we would record the cell digest as the SENDME digest in the
decrypt cell function but to avoid code duplication (both directions needs to
record), we now do that right after iff the cell is recognized (at the edge).
It is now done in circuit_receive_relay_cell() instead.

We now also record the cell digest as the SENDME digest in both relay cell
encryption functions since they are split depending on the direction.
relay_encrypt_cell_outbound() and relay_encrypt_cell_inbound() need to
consider recording the cell digest depending on their direction (f vs b
digest).

Fixes #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
44265dd671 sendme: Never fallback to v0 if unknown version
There was a missing cell version check against our max supported version. In
other words, we do not fallback to v0 anymore in case we do know the SENDME
version.

We can either handle it or not, never fallback to the unauthenticated version
in order to avoid gaming the authenticated logic.

Add a unit tests making sure we properly test that and also test that we can
always handle the default emit and accepted versions.

Fixes #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
69e0d5bfc7 sendme: Validate v1 SENDMEs on both client and exit side
The validation of the SENDME cell is now done as the very first thing when
receiving it for both client and exit. On failure to validate, the circuit is
closed as detailed in the specification.

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
59b9eecc19 sendme: Record cell digest on both client and exit
It turns out that only the exit side is validating the authenticated SENDME v1
logic and never the client side. Which means that if a client ever uploaded
data towards an exit, the authenticated SENDME logic wouldn't apply.

For this to work, we have to record the cell digest client side as well which
introduced a new function that supports both type of edges.

This also removes a test that is not valid anymore which was that we didn't
allow cell recording on an origin circuit (client).

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
245dccb77d Merge remote-tracking branch 'nickm/ticket30454_034_01_squashed' into ticket30454_035_01 2019-05-22 11:43:55 -04:00
rl1987
2845607f97 In microdesc_cache_reload(), set journal length to length of string we read
Hopefully this will fix CID 1444769.
2019-05-20 09:08:10 -04:00
David Goulet
56908c6f1c hs: Remove hs_cell_onion_key_type_t enum
Unify this with the trunnel ABI so we don't duplicate.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
7d3e904a27 trunnel: Remove INTRODUCE1 status code IN statement
We want to support parsing a cell with unknown status code so we are forward
compatible.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
79cfe2ddd7 hs: Remove hs_intro_auth_key_type_t enum
Like the previous commit about the INTRODUCE_ACK status code, change all auth
key type to use the one defined in the trunnel file.

Standardize the use of these auth type to a common ABI.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
dcc1d8d15b hs: Get rid of duplicate hs_cell_introd_ack_status_t
This enum was the exact same as hs_intro_ack_status_t that was removed at the
previous commit. It was used client side when parsing the INTRODUCE_ACK cell.

Now, the entire code dealing with the INTRODUCE_ACK cell (both sending and
receiving) have been modified to all use the same ABI defined in the trunnel
introduce1 file.

Finally, the client will default to the normal behavior when receiving an
unknown NACK status code which is to note down that we've failed and re-extend
to the next intro point. This way, unknown status code won't trigger a
different behavior client side.

Part of #30454.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
590d97bc10 hs: Define INTRODUCE_ACK status code in trunnel
Remove the hs_intro_ack_status_t enum and move the value into trunnel. Only
use these values from now on in the intro point code.

Interestingly enough, the client side also re-define these values in hs_cell.h
with the hs_cell_introd_ack_status_t enum. Next commit will fix that and force
to use the trunnel ABI.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
rl1987
49acbfad23 Don't forget to use the mutex in testing_enable_prefilled_rng() 2019-05-17 19:30:09 +03:00
Nick Mathewson
37aae23945 OOM-purge the DNS cache one part at a time
Previously we purged it in 1-hour increments -- but one-hour is the
maximum TTL for the cache!  Now we do it in 25%-TTL increments.

Fixes bug 29617; bugfix on 0.3.5.1-alpha.
2019-05-17 10:03:41 -04:00
Nick Mathewson
b2b779228d Merge remote-tracking branch 'tor-github/pr/1033' 2019-05-17 08:18:20 -04:00
Nick Mathewson
9cec7a7b5c Merge branch 'maint-0.4.0' 2019-05-17 08:10:17 -04:00
Nick Mathewson
e5deb2bbc7 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-05-17 08:10:16 -04:00
Nick Mathewson
c7f9f7e542 Merge branch 'maint-0.3.5' into maint-0.4.0 2019-05-17 08:10:16 -04:00
Nick Mathewson
a521c42788 Merge branch 'maint-0.2.9' into maint-0.3.4 2019-05-17 08:10:15 -04:00
Karsten Loesing
4e262196a8 Update geoip and geoip6 to the May 13 2019 database. 2019-05-17 08:52:13 +02:00
Mike Perry
857c54ca03 Refactor rend machines, stage 2/2: Move histogram code.
Comment clarifications now that the code is seperated. It's the same code, but
its doing this for different reasons on each side.
2019-05-16 20:17:14 +00:00
Mike Perry
0cba53c6ed Refactor rend machines, stage 1/2: Move state transition code. 2019-05-16 20:17:11 +00:00
Nick Mathewson
d5db40a014 test_channel_listener: free 'chan' explicitly
This should fix CID 1437442, where coverity can't tell that
channel_free_all() frees the fake channel we allocated.
2019-05-16 15:35:21 -04:00
Mike Perry
bbb974234c Refactor intro machines, stage 2/2: Move histogram code.
The client side had garbage histograms and deadcode here, too. That code has
been removed.

The tests have also been updated to properly test the intro circ by sending
padding from the relay side to the client, and verifying that both shut down
when padding was up. (The tests previously erroneously tested only the client
side of intro circs, which actually were supposed to be doing nothing).
2019-05-16 19:21:19 +00:00
Nick Mathewson
0a9685b3a7 hs tests: explicitly free 'service' variable.
This should fix about 15 CID issues, where coverity can't tell that
hs_free_all() frees the service we allocated.
2019-05-16 15:21:18 -04:00
Mike Perry
f237fed746 Refactor intro machines, stage 1/2: Move state transition code.
This just moves the state transition directives into the proper client/relay
side functions. It also allows us to remove some dead-code from the client
side (since the client doesn't send padding).
2019-05-16 19:21:14 +00:00
Nick Mathewson
1bf451cffb rng_test_helpers: add a needless lock/unlock pair to please coverity
Fix for CID 1444908
2019-05-16 15:04:40 -04:00
George Kadianakis
42ea3a416e Improve logging around the circpad module..
- Add some more useful logs for future debugging.

- Stop usage of circpad_state_to_string(). It's innacurate.

- Reduce severity and fix up log domain of some logging messages.
2019-05-16 14:23:32 +03:00
George Kadianakis
953dc601d9 Add unittests for the new machines. 2019-05-16 14:23:22 +03:00
George Kadianakis
ac895fa405 Add client-side onion service circuit hiding machines. 2019-05-16 14:23:17 +03:00
George Kadianakis
9b582edddb Correctly handle machines out of tokens that have not closed yet.
Perhaps the machine on the other side is still not done.
2019-05-16 14:07:32 +03:00
George Kadianakis
69a277f635 Introduce circpad free_all() function. 2019-05-16 14:07:25 +03:00
George Kadianakis
5791bc9d76 Generate non-padding circpad events for PADDING_NEGOTIATE(D).
As part of our machines, we need to know when a PADDING_NEGOATIATE(D) cell gets
sent out, so we add an event for this.
2019-05-16 14:06:27 +03:00
George Kadianakis
39c52d14a6 Make register_padding_machine part of the public API.
We are gonna use this function to register our new machine.
2019-05-16 14:05:58 +03:00
George Kadianakis
a014e01b68 Behave correctly when state->max_length is zero. 2019-05-16 14:05:27 +03:00
Roger Dingledine
d86896b29c fix typos, whitespace, comments 2019-05-15 23:20:03 -04:00
George Kadianakis
338cfb3179 Merge branch 'tor-github/pr/1002' 2019-05-15 23:23:18 +03:00
David Goulet
39a14421b1 Merge branch 'tor-github/pr/1021'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-15 15:35:22 -04:00
George Kadianakis
d71fa707dd Merge branch 'bug28780-squashed3-rebased' into bug28780_rebase 2019-05-15 16:46:51 +03:00
Mike Perry
56738ff8c6 Add control port circuit ID to all pathbias bug messages.
To ease debugging of miscount issues, attach vanguards with --loglevel DEBUG
and obtain control port logs (or use any other control port CIRC and
CIRC_MINOR event logging mechanism).
2019-05-15 16:44:59 +03:00
Mike Perry
b98bcd789e Pathbias should continue to ignore previously ignored circs.
If circuit padding wants to keep a circuit open and pathbias used to ignore
it, pathbias should continue to ignore it.

This may catch other purpose-change related miscounts (such as timeout
measurement, cannibalization, onion service circuit transitions, and
vanguards).
2019-05-15 16:44:59 +03:00
Mike Perry
e253a117c0 Bug 28780: Add tests
Also test circpad expiry safeguard.
2019-05-15 16:44:59 +03:00
Mike Perry
662825474c Bug 28780: Make use of purpose to keep padding circuits open.
When a circuit is marked for close, check to see if any of our padding
machines want to take ownership of it and continue padding until the machine
hits the END state.

For safety, we also ensure that machines that do not terminate are still
closed as follows: Because padding machine timers are UINT32_MAX in size, if
some sort of network event doesn't happen on a padding-only circuit within
that time, we can conclude it is deadlocked and allow
circuit_expire_old_circuits_clientside() to close it.

If too much network activity happens, then per-machine padding limits can be
used to cease padding, which will cause network cell events to cease, on the
circuit, which will cause circpad to abandon the circuit as per the above time
limit.
2019-05-15 16:44:59 +03:00
Mike Perry
d44e3e57b0 Bug 28780: Add purpose for keeping padding circuits open 2019-05-15 16:44:59 +03:00
George Kadianakis
a7779df84c Merge branch 'bug29085_rebase' 2019-05-15 15:13:08 +03:00
Mike Perry
5638d65f79 Check the token supply when we received a padding event, too.
We need to check here because otherwise we can try to schedule padding with no
tokens left upon the receipt of a padding event when our bins just became
empty.
2019-05-15 15:10:48 +03:00
Mike Perry
148c2d5bab Fix two typo bugs found by new state length test. 2019-05-15 15:10:48 +03:00
Mike Perry
e8a1f24178 Add test to explicitly check state lengths and token counts.
Our other tests tested state lengths against padding packets, and token counts
against non-padding packets. This test checks state lengths against
non-padding packets (and also padding packets too), and checks token counts
against padding packets (and also non-padding packets too).

The next three commits are needed to make this test pass (it found 3 bugs).
Yay?
2019-05-15 15:10:48 +03:00
Mike Perry
aef9be6ace Eliminate unneeded casts to circuit_t in circpad tests. 2019-05-15 15:10:48 +03:00
Mike Perry
14ec8b89f8 Bug 29085: Avoid monotime usage for rtt estimates if it is not in use. 2019-05-15 15:10:31 +03:00
Mike Perry
5c2d2b5d11 Make the relationship between mutable histograms and token removal explicit. 2019-05-15 15:09:49 +03:00
Nick Mathewson
536ba09ad1 Use reproducible RNG logic in circuitpadding tests. 2019-05-15 07:50:56 -04:00
Nick Mathewson
72e9c427b8 Testing: allow the user to pass a seed in for reproducible-RNG tests
The environment variable TOR_TEST_RNG_SEED, if provided, is a hex
value for the RNG seed.
2019-05-15 07:50:56 -04:00
Nick Mathewson
2d467544fe Use new RNG and tinytest code to simplify prob_distr tests
Since the reproducible RNG dumps its own seed, we don't need to do
it for it. Since tinytest can tell us if the test failed, we don't
need our own test_failed booleans.
2019-05-15 07:50:56 -04:00
Nick Mathewson
261d43cdd5 Make testing_disable_reproducible_rng() log seed on test failure
This should let us simplify test_prob_distr.c and other stuff in the
future.
2019-05-15 07:50:56 -04:00
Nick Mathewson
e4feb4ad01 Give tinytest a function to say whether the current test has failed 2019-05-15 07:50:56 -04:00
George Kadianakis
1de11dc47a Merge branch 'maint-0.4.0' 2019-05-15 13:46:20 +03:00
George Kadianakis
cd264e145b Merge branch 'tor-github/pr/1013' into maint-0.4.0 2019-05-15 13:45:39 +03:00
Mike Perry
57e5e940d3 Bug 29085: Minor unit test updates for refactoring.
Deliver nonpadding events instead of calling token removal functions.
2019-05-15 04:57:11 +00:00
Mike Perry
1c46790e0d Bug 29085: Refactor padding sent accounting out of callback.
This commit moves code that updates the state length and padding limit counts
out from the callback to its own function, for clarity.

It does not change functionality.
2019-05-15 04:57:11 +00:00
Mike Perry
010779176b Bug 29085: Refactor non-padding accounting out of token removal.
This commit moves the padding state limit checks and the padding rate limit
checks out of the token removal codepath, and causes all three functions to
get called from a single circpad_machine_count_nonpadding_sent() function.

It does not change functionality.
2019-05-15 04:57:11 +00:00
Nick Mathewson
370ea8d23b Merge branch 'ticket30452_035_v3' into ticket30452_041_v3 2019-05-14 19:55:51 -04:00
Nick Mathewson
0c451b31d2 Make --list-modules imply --hush 2019-05-14 19:55:35 -04:00
Nick Mathewson
1c95bdb83b Merge branch 'ticket30452_035_v3' into ticket30452_041_v3 2019-05-14 19:20:53 -04:00
Nick Mathewson
1b16fcb70c Add a --list-modules command
Closes ticket 30452.
2019-05-14 19:19:53 -04:00
Nick Mathewson
43d4119454 Merge remote-tracking branch 'tor-github/pr/1004' 2019-05-14 11:43:10 -04:00
George Kadianakis
0f4f4fdcf5 Merge branch 'tor-github/pr/1006' 2019-05-14 15:15:09 +03:00
Nick Mathewson
5d950f3edd Fix a compilation warning: function does not have to be STATIC. 2019-05-13 14:34:16 -04:00
Nick Mathewson
9ad2eb8f73 Merge branch 'bug28683_30173_29203_squashed' 2019-05-13 14:33:31 -04:00
Mike Perry
42eb02a327 Tests for bugs 28683, 30173, and 29203. 2019-05-13 14:30:35 -04:00
Mike Perry
621ea2315b Bug 29203: Provide ReducedCircuitPadding torrc and consensus params 2019-05-13 14:30:35 -04:00
Mike Perry
f4064d6ce2 Bug 28693: Provide Torrc option to disable circuit padding. 2019-05-13 14:30:35 -04:00
Nick Mathewson
c6523a6398 Merge remote-tracking branch 'tor-github/pr/998' 2019-05-13 14:25:54 -04:00
David Goulet
def96ce838 sendme: Fix coverity CID 1444999
The code flow in theory can end up with a layer_hint to be NULL but in
practice it should never happen because with an origin circuit, we must have
the layer_hint.

Just in case, BUG() on it if we ever end up in this situation and recover by
closing the circuit.

Fixes #30467.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-13 10:00:39 -04:00
David Goulet
2f44786e30 Merge branch 'tor-github/pr/976' 2019-05-13 07:34:00 -04:00
Nick Mathewson
ff55840343 Don't pass a NULL into a %s when logging client auth file load failure
Fortunately, in 0.3.5.1-alpha we improved logging for various
failure cases involved with onion service client auth.

Unfortunately, for this one, we freed the file right before logging
its name.

Fortunately, tor_free() sets its pointer to NULL, so we didn't have
a use-after-free bug.

Unfortunately, passing NULL to %s is not defined.

Fortunately, GCC 9.1.1 caught the issue!

Unfortunately, nobody has actually tried building Tor with GCC 9.1.1
before. Or if they had, they didn't report the warning.

Fixes bug 30475; bugfix on 0.3.5.1-alpha.
2019-05-10 17:47:43 -04:00
George Kadianakis
501d1ae0bd Merge branch 'tor-github/pr/973' 2019-05-10 12:49:01 +03:00
David Goulet
cbcc570ff4 hs: Remove usage of HS_INTRO_ACK_STATUS_CANT_RELAY
The INTRODUCE1 trunnel definition file doesn't support that value so it can
not be used else it leads to an assert on the intro point side if ever tried.

Fortunately, it was impossible to reach that code path.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-09 13:28:37 -04:00
David Goulet
3885e7b44b Merge branch 'tor-github/pr/1000'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-08 08:02:28 -04:00
Neel Chauhan
3cafdeb8c0 Only call tor_addr_parse() in circuit_is_acceptable() when needed 2019-05-07 11:52:56 -04:00