Commit Graph

24223 Commits

Author SHA1 Message Date
David Goulet
ff9aa32143 Merge branch 'tor-github/pr/1047' 2019-05-28 14:59:07 -04:00
George Kadianakis
0a86f14add Merge branch 'tor-github/pr/1042' 2019-05-27 14:22:01 +03:00
George Kadianakis
130eb227ac Merge branch 'tor-github/pr/1043' 2019-05-27 14:20:51 +03:00
Nick Mathewson
fcd51fd49f Tests for deciding how full our relay cells should be 2019-05-27 14:20:36 +03:00
Nick Mathewson
0bc1241494 Make sure that we send at least some random data in RELAY_DATA cells
Proposal 289 prevents SENDME-flooding by requiring the other side to
authenticate the data it has received.  But this data won't actually
be random if they are downloading a known resource.  "No problem",
we said, "let's fell the empty parts of our cells with some
randomness!" and we did that in #26871.

Unfortunately, if the relay data payloads are all completely full,
there won't be any empty parts for us to randomize.

Therefore, we now pick random "randomness windows" between
CIRCWINDOW_INCREMENT/2 and CIRCWINDOW_INCREMENT. We remember whether we have
sent a cell containing at least 16 bytes of randomness in that window.  If we
haven't, then when the window is exhausted, we send one.  (This window approach
is designed to lower the number of rng checks we have to do.  The number 16 is
pulled out of a hat to change the attacker's guessing difficulty to
"impossible".)

Implements 28646.
2019-05-27 14:20:07 +03:00
Roger Dingledine
94914e2a4d trivial whitespace fixes 2019-05-26 17:32:42 -04:00
Nick Mathewson
07ccffa989 Coverage: do not include test-rebind in coverage builds.
Because it invokes the Tor mainloop, it does unpredictable things to
test coverage of a lot of code that it doesn't actually test at
all.  (It is more an integration test than anything else.)
2019-05-23 12:48:51 -04:00
Nick Mathewson
b882810245 In coverage builds, use branch-free timeradd() and timersub()
The ordinary definitions of timeradd() and timersub() contain a
branch. However, in coverage builds, this means that we get spurious
complaints about partially covered basic blocks, in a way that makes
our coverage determinism harder to check.
2019-05-23 12:48:51 -04:00
Nick Mathewson
2bb5d8148b In coverage builds, avoid basic-block complexity in log_debug
Ordinarily we skip calling log_fn(LOG_DEBUG,...) if debug logging is
completely disabled.  However, in coverage builds, this means that
we get spurious complaints about partially covered basic blocks, in
a way that makes our coverage determinism harder to check.
2019-05-23 12:48:51 -04:00
David Goulet
29955f13e5 Merge branch 'tor-github/pr/1022' 2019-05-23 09:50:28 -04:00
David Goulet
e13e2012b9 Merge branch 'tor-github/pr/1034' 2019-05-23 09:40:07 -04:00
David Goulet
327bb0e2ca Merge branch 'tor-github/pr/988' 2019-05-23 09:30:36 -04:00
Nick Mathewson
530d1179ff Extract length-deciding function from package_raw_inbuf. 2019-05-23 08:28:46 -04:00
Roger Dingledine
e4d1187584 refactor logic to decide how much to package from inbuf
no actual changes in behavior
2019-05-23 08:28:46 -04:00
Nick Mathewson
57ee0e3af9 Only reject POSTDESCRIPTOR purpose= when the purpose is unrecognized
Fixes bug 30580; bugfix on 0.4.1.1-alpha.
2019-05-23 08:24:29 -04:00
Nick Mathewson
ebe39dcb92 Now this repository is full of 0.4.1.1-alpha-dev 2019-05-22 18:07:29 -04:00
Nick Mathewson
fa410162a3 circuitpadding tests: Use tt_i64_op() to compare int64_t values
Bug not in any released Tor.
2019-05-22 15:19:24 -04:00
Nick Mathewson
3a7ed8bc5f Bump to 0.4.1.1-alpha 2019-05-22 11:56:02 -04:00
Nick Mathewson
24c2502070 Merge remote-tracking branch 'dgoulet/ticket30454_035_01' 2019-05-22 11:50:46 -04:00
Nick Mathewson
e6b862e6a8 Merge branch 'ticket30428_041_02_squashed' 2019-05-22 11:48:43 -04:00
David Goulet
0cad83bea4 sendme: Add non fatal asserts for extra safety
Two non fatal asserts are added in this commit. First one is to see if the
SENDME digest list kept on the circuit for validation ever grows bigger than
the maximum number of expected SENDME on a circuit (currently 10).

The second one is to know if we ever send more than one SENDME at a time on a
circuit. In theory, we shouldn't but if we ever do, the v1 implementation
wouldn't work because we only keep one single cell digest (the previous cell
to the SENDME) on the circuit/cpath. Thus, sending two SENDME consecutively
will lead to a mismatch on the other side because the same cell digest would
be use and thus the circuit would collapse.

Finally, add an extra debug log in case we emit a v0 which also includes the
consensus emit version in that case.

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
5479ffabf8 sendme: Always pop last SENDME digest from circuit
We must not accumulate digests on the circuit if the other end point is using
another SENDME version that is not using those digests like v0.

This commit makes it that we always pop the digest regardless of the version.

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
482c4972b9 sendme: Clarify how sendme_circuit_cell_is_next() works
Commit 4ef8470fa5480d3b was actually reverted before because in the end we
needed to do this minus 1 check on the window.

This commit clarifies that in the code, takes the useful comment changes from
4ef8470fa5480d3b and makes sendme_circuit_cell_is_next() private since it
behaves in a very specific way that one external caller might expect.

Part of #30428.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
3835a3acf5 sendme: Properly record SENDMEs on both edges
Turns out that we were only recording the "b_digest" but to have
bidirectionnal authenticated SENDMEs, we need to use the "f_digest" in the
forward cell situation.

Because of the cpath refactoring, this commit plays with the crypt_path_ and
relay_crypto_t API a little bit in order to respect the abstractions.

Previously, we would record the cell digest as the SENDME digest in the
decrypt cell function but to avoid code duplication (both directions needs to
record), we now do that right after iff the cell is recognized (at the edge).
It is now done in circuit_receive_relay_cell() instead.

We now also record the cell digest as the SENDME digest in both relay cell
encryption functions since they are split depending on the direction.
relay_encrypt_cell_outbound() and relay_encrypt_cell_inbound() need to
consider recording the cell digest depending on their direction (f vs b
digest).

Fixes #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
44265dd671 sendme: Never fallback to v0 if unknown version
There was a missing cell version check against our max supported version. In
other words, we do not fallback to v0 anymore in case we do know the SENDME
version.

We can either handle it or not, never fallback to the unauthenticated version
in order to avoid gaming the authenticated logic.

Add a unit tests making sure we properly test that and also test that we can
always handle the default emit and accepted versions.

Fixes #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
69e0d5bfc7 sendme: Validate v1 SENDMEs on both client and exit side
The validation of the SENDME cell is now done as the very first thing when
receiving it for both client and exit. On failure to validate, the circuit is
closed as detailed in the specification.

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
59b9eecc19 sendme: Record cell digest on both client and exit
It turns out that only the exit side is validating the authenticated SENDME v1
logic and never the client side. Which means that if a client ever uploaded
data towards an exit, the authenticated SENDME logic wouldn't apply.

For this to work, we have to record the cell digest client side as well which
introduced a new function that supports both type of edges.

This also removes a test that is not valid anymore which was that we didn't
allow cell recording on an origin circuit (client).

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-22 11:47:20 -04:00
David Goulet
245dccb77d Merge remote-tracking branch 'nickm/ticket30454_034_01_squashed' into ticket30454_035_01 2019-05-22 11:43:55 -04:00
rl1987
2845607f97 In microdesc_cache_reload(), set journal length to length of string we read
Hopefully this will fix CID 1444769.
2019-05-20 09:08:10 -04:00
David Goulet
56908c6f1c hs: Remove hs_cell_onion_key_type_t enum
Unify this with the trunnel ABI so we don't duplicate.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
7d3e904a27 trunnel: Remove INTRODUCE1 status code IN statement
We want to support parsing a cell with unknown status code so we are forward
compatible.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
79cfe2ddd7 hs: Remove hs_intro_auth_key_type_t enum
Like the previous commit about the INTRODUCE_ACK status code, change all auth
key type to use the one defined in the trunnel file.

Standardize the use of these auth type to a common ABI.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
dcc1d8d15b hs: Get rid of duplicate hs_cell_introd_ack_status_t
This enum was the exact same as hs_intro_ack_status_t that was removed at the
previous commit. It was used client side when parsing the INTRODUCE_ACK cell.

Now, the entire code dealing with the INTRODUCE_ACK cell (both sending and
receiving) have been modified to all use the same ABI defined in the trunnel
introduce1 file.

Finally, the client will default to the normal behavior when receiving an
unknown NACK status code which is to note down that we've failed and re-extend
to the next intro point. This way, unknown status code won't trigger a
different behavior client side.

Part of #30454.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
David Goulet
590d97bc10 hs: Define INTRODUCE_ACK status code in trunnel
Remove the hs_intro_ack_status_t enum and move the value into trunnel. Only
use these values from now on in the intro point code.

Interestingly enough, the client side also re-define these values in hs_cell.h
with the hs_cell_introd_ack_status_t enum. Next commit will fix that and force
to use the trunnel ABI.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-20 08:56:34 -04:00
rl1987
49acbfad23 Don't forget to use the mutex in testing_enable_prefilled_rng() 2019-05-17 19:30:09 +03:00
Nick Mathewson
37aae23945 OOM-purge the DNS cache one part at a time
Previously we purged it in 1-hour increments -- but one-hour is the
maximum TTL for the cache!  Now we do it in 25%-TTL increments.

Fixes bug 29617; bugfix on 0.3.5.1-alpha.
2019-05-17 10:03:41 -04:00
Nick Mathewson
b2b779228d Merge remote-tracking branch 'tor-github/pr/1033' 2019-05-17 08:18:20 -04:00
Nick Mathewson
9cec7a7b5c Merge branch 'maint-0.4.0' 2019-05-17 08:10:17 -04:00
Nick Mathewson
e5deb2bbc7 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-05-17 08:10:16 -04:00
Nick Mathewson
c7f9f7e542 Merge branch 'maint-0.3.5' into maint-0.4.0 2019-05-17 08:10:16 -04:00
Nick Mathewson
a521c42788 Merge branch 'maint-0.2.9' into maint-0.3.4 2019-05-17 08:10:15 -04:00
Karsten Loesing
4e262196a8 Update geoip and geoip6 to the May 13 2019 database. 2019-05-17 08:52:13 +02:00
Mike Perry
857c54ca03 Refactor rend machines, stage 2/2: Move histogram code.
Comment clarifications now that the code is seperated. It's the same code, but
its doing this for different reasons on each side.
2019-05-16 20:17:14 +00:00
Mike Perry
0cba53c6ed Refactor rend machines, stage 1/2: Move state transition code. 2019-05-16 20:17:11 +00:00
Nick Mathewson
d5db40a014 test_channel_listener: free 'chan' explicitly
This should fix CID 1437442, where coverity can't tell that
channel_free_all() frees the fake channel we allocated.
2019-05-16 15:35:21 -04:00
Mike Perry
bbb974234c Refactor intro machines, stage 2/2: Move histogram code.
The client side had garbage histograms and deadcode here, too. That code has
been removed.

The tests have also been updated to properly test the intro circ by sending
padding from the relay side to the client, and verifying that both shut down
when padding was up. (The tests previously erroneously tested only the client
side of intro circs, which actually were supposed to be doing nothing).
2019-05-16 19:21:19 +00:00
Nick Mathewson
0a9685b3a7 hs tests: explicitly free 'service' variable.
This should fix about 15 CID issues, where coverity can't tell that
hs_free_all() frees the service we allocated.
2019-05-16 15:21:18 -04:00
Mike Perry
f237fed746 Refactor intro machines, stage 1/2: Move state transition code.
This just moves the state transition directives into the proper client/relay
side functions. It also allows us to remove some dead-code from the client
side (since the client doesn't send padding).
2019-05-16 19:21:14 +00:00
Nick Mathewson
1bf451cffb rng_test_helpers: add a needless lock/unlock pair to please coverity
Fix for CID 1444908
2019-05-16 15:04:40 -04:00
George Kadianakis
42ea3a416e Improve logging around the circpad module..
- Add some more useful logs for future debugging.

- Stop usage of circpad_state_to_string(). It's innacurate.

- Reduce severity and fix up log domain of some logging messages.
2019-05-16 14:23:32 +03:00
George Kadianakis
953dc601d9 Add unittests for the new machines. 2019-05-16 14:23:22 +03:00
George Kadianakis
ac895fa405 Add client-side onion service circuit hiding machines. 2019-05-16 14:23:17 +03:00
George Kadianakis
9b582edddb Correctly handle machines out of tokens that have not closed yet.
Perhaps the machine on the other side is still not done.
2019-05-16 14:07:32 +03:00
George Kadianakis
69a277f635 Introduce circpad free_all() function. 2019-05-16 14:07:25 +03:00
George Kadianakis
5791bc9d76 Generate non-padding circpad events for PADDING_NEGOTIATE(D).
As part of our machines, we need to know when a PADDING_NEGOATIATE(D) cell gets
sent out, so we add an event for this.
2019-05-16 14:06:27 +03:00
George Kadianakis
39c52d14a6 Make register_padding_machine part of the public API.
We are gonna use this function to register our new machine.
2019-05-16 14:05:58 +03:00
George Kadianakis
a014e01b68 Behave correctly when state->max_length is zero. 2019-05-16 14:05:27 +03:00
Roger Dingledine
d86896b29c fix typos, whitespace, comments 2019-05-15 23:20:03 -04:00
George Kadianakis
338cfb3179 Merge branch 'tor-github/pr/1002' 2019-05-15 23:23:18 +03:00
David Goulet
39a14421b1 Merge branch 'tor-github/pr/1021'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-15 15:35:22 -04:00
George Kadianakis
d71fa707dd Merge branch 'bug28780-squashed3-rebased' into bug28780_rebase 2019-05-15 16:46:51 +03:00
Mike Perry
56738ff8c6 Add control port circuit ID to all pathbias bug messages.
To ease debugging of miscount issues, attach vanguards with --loglevel DEBUG
and obtain control port logs (or use any other control port CIRC and
CIRC_MINOR event logging mechanism).
2019-05-15 16:44:59 +03:00
Mike Perry
b98bcd789e Pathbias should continue to ignore previously ignored circs.
If circuit padding wants to keep a circuit open and pathbias used to ignore
it, pathbias should continue to ignore it.

This may catch other purpose-change related miscounts (such as timeout
measurement, cannibalization, onion service circuit transitions, and
vanguards).
2019-05-15 16:44:59 +03:00
Mike Perry
e253a117c0 Bug 28780: Add tests
Also test circpad expiry safeguard.
2019-05-15 16:44:59 +03:00
Mike Perry
662825474c Bug 28780: Make use of purpose to keep padding circuits open.
When a circuit is marked for close, check to see if any of our padding
machines want to take ownership of it and continue padding until the machine
hits the END state.

For safety, we also ensure that machines that do not terminate are still
closed as follows: Because padding machine timers are UINT32_MAX in size, if
some sort of network event doesn't happen on a padding-only circuit within
that time, we can conclude it is deadlocked and allow
circuit_expire_old_circuits_clientside() to close it.

If too much network activity happens, then per-machine padding limits can be
used to cease padding, which will cause network cell events to cease, on the
circuit, which will cause circpad to abandon the circuit as per the above time
limit.
2019-05-15 16:44:59 +03:00
Mike Perry
d44e3e57b0 Bug 28780: Add purpose for keeping padding circuits open 2019-05-15 16:44:59 +03:00
George Kadianakis
a7779df84c Merge branch 'bug29085_rebase' 2019-05-15 15:13:08 +03:00
Mike Perry
5638d65f79 Check the token supply when we received a padding event, too.
We need to check here because otherwise we can try to schedule padding with no
tokens left upon the receipt of a padding event when our bins just became
empty.
2019-05-15 15:10:48 +03:00
Mike Perry
148c2d5bab Fix two typo bugs found by new state length test. 2019-05-15 15:10:48 +03:00
Mike Perry
e8a1f24178 Add test to explicitly check state lengths and token counts.
Our other tests tested state lengths against padding packets, and token counts
against non-padding packets. This test checks state lengths against
non-padding packets (and also padding packets too), and checks token counts
against padding packets (and also non-padding packets too).

The next three commits are needed to make this test pass (it found 3 bugs).
Yay?
2019-05-15 15:10:48 +03:00
Mike Perry
aef9be6ace Eliminate unneeded casts to circuit_t in circpad tests. 2019-05-15 15:10:48 +03:00
Mike Perry
14ec8b89f8 Bug 29085: Avoid monotime usage for rtt estimates if it is not in use. 2019-05-15 15:10:31 +03:00
Mike Perry
5c2d2b5d11 Make the relationship between mutable histograms and token removal explicit. 2019-05-15 15:09:49 +03:00
Nick Mathewson
536ba09ad1 Use reproducible RNG logic in circuitpadding tests. 2019-05-15 07:50:56 -04:00
Nick Mathewson
72e9c427b8 Testing: allow the user to pass a seed in for reproducible-RNG tests
The environment variable TOR_TEST_RNG_SEED, if provided, is a hex
value for the RNG seed.
2019-05-15 07:50:56 -04:00
Nick Mathewson
2d467544fe Use new RNG and tinytest code to simplify prob_distr tests
Since the reproducible RNG dumps its own seed, we don't need to do
it for it. Since tinytest can tell us if the test failed, we don't
need our own test_failed booleans.
2019-05-15 07:50:56 -04:00
Nick Mathewson
261d43cdd5 Make testing_disable_reproducible_rng() log seed on test failure
This should let us simplify test_prob_distr.c and other stuff in the
future.
2019-05-15 07:50:56 -04:00
Nick Mathewson
e4feb4ad01 Give tinytest a function to say whether the current test has failed 2019-05-15 07:50:56 -04:00
George Kadianakis
1de11dc47a Merge branch 'maint-0.4.0' 2019-05-15 13:46:20 +03:00
George Kadianakis
cd264e145b Merge branch 'tor-github/pr/1013' into maint-0.4.0 2019-05-15 13:45:39 +03:00
Mike Perry
57e5e940d3 Bug 29085: Minor unit test updates for refactoring.
Deliver nonpadding events instead of calling token removal functions.
2019-05-15 04:57:11 +00:00
Mike Perry
1c46790e0d Bug 29085: Refactor padding sent accounting out of callback.
This commit moves code that updates the state length and padding limit counts
out from the callback to its own function, for clarity.

It does not change functionality.
2019-05-15 04:57:11 +00:00
Mike Perry
010779176b Bug 29085: Refactor non-padding accounting out of token removal.
This commit moves the padding state limit checks and the padding rate limit
checks out of the token removal codepath, and causes all three functions to
get called from a single circpad_machine_count_nonpadding_sent() function.

It does not change functionality.
2019-05-15 04:57:11 +00:00
Nick Mathewson
370ea8d23b Merge branch 'ticket30452_035_v3' into ticket30452_041_v3 2019-05-14 19:55:51 -04:00
Nick Mathewson
0c451b31d2 Make --list-modules imply --hush 2019-05-14 19:55:35 -04:00
Nick Mathewson
1c95bdb83b Merge branch 'ticket30452_035_v3' into ticket30452_041_v3 2019-05-14 19:20:53 -04:00
Nick Mathewson
1b16fcb70c Add a --list-modules command
Closes ticket 30452.
2019-05-14 19:19:53 -04:00
Nick Mathewson
43d4119454 Merge remote-tracking branch 'tor-github/pr/1004' 2019-05-14 11:43:10 -04:00
George Kadianakis
0f4f4fdcf5 Merge branch 'tor-github/pr/1006' 2019-05-14 15:15:09 +03:00
Nick Mathewson
5d950f3edd Fix a compilation warning: function does not have to be STATIC. 2019-05-13 14:34:16 -04:00
Nick Mathewson
9ad2eb8f73 Merge branch 'bug28683_30173_29203_squashed' 2019-05-13 14:33:31 -04:00
Mike Perry
42eb02a327 Tests for bugs 28683, 30173, and 29203. 2019-05-13 14:30:35 -04:00
Mike Perry
621ea2315b Bug 29203: Provide ReducedCircuitPadding torrc and consensus params 2019-05-13 14:30:35 -04:00
Mike Perry
f4064d6ce2 Bug 28693: Provide Torrc option to disable circuit padding. 2019-05-13 14:30:35 -04:00
Nick Mathewson
c6523a6398 Merge remote-tracking branch 'tor-github/pr/998' 2019-05-13 14:25:54 -04:00
David Goulet
def96ce838 sendme: Fix coverity CID 1444999
The code flow in theory can end up with a layer_hint to be NULL but in
practice it should never happen because with an origin circuit, we must have
the layer_hint.

Just in case, BUG() on it if we ever end up in this situation and recover by
closing the circuit.

Fixes #30467.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-13 10:00:39 -04:00
David Goulet
2f44786e30 Merge branch 'tor-github/pr/976' 2019-05-13 07:34:00 -04:00
Nick Mathewson
ff55840343 Don't pass a NULL into a %s when logging client auth file load failure
Fortunately, in 0.3.5.1-alpha we improved logging for various
failure cases involved with onion service client auth.

Unfortunately, for this one, we freed the file right before logging
its name.

Fortunately, tor_free() sets its pointer to NULL, so we didn't have
a use-after-free bug.

Unfortunately, passing NULL to %s is not defined.

Fortunately, GCC 9.1.1 caught the issue!

Unfortunately, nobody has actually tried building Tor with GCC 9.1.1
before. Or if they had, they didn't report the warning.

Fixes bug 30475; bugfix on 0.3.5.1-alpha.
2019-05-10 17:47:43 -04:00
George Kadianakis
501d1ae0bd Merge branch 'tor-github/pr/973' 2019-05-10 12:49:01 +03:00
David Goulet
cbcc570ff4 hs: Remove usage of HS_INTRO_ACK_STATUS_CANT_RELAY
The INTRODUCE1 trunnel definition file doesn't support that value so it can
not be used else it leads to an assert on the intro point side if ever tried.

Fortunately, it was impossible to reach that code path.

Part of #30454

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-09 13:28:37 -04:00
David Goulet
3885e7b44b Merge branch 'tor-github/pr/1000'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-08 08:02:28 -04:00
Neel Chauhan
3cafdeb8c0 Only call tor_addr_parse() in circuit_is_acceptable() when needed 2019-05-07 11:52:56 -04:00
David Goulet
b72f5da03d Merge branch 'tor-github/pr/994'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-07 08:02:39 -04:00
Nick Mathewson
3c2648bbda Move "relay" and "router" periodic callbacks out of mainloop.c
(Some of these callbacks are specific to the OR module, so now it's
time to have an or_sys and or_periodic.)
2019-05-06 16:35:39 -04:00
Nick Mathewson
b394b5b2af Create a relay subsystem and move the shutdown functions there 2019-05-06 16:25:09 -04:00
David Goulet
07e4b09b5f sendme: Add FlowCtrl protover value
See proposal 289 section 4.3 for more details.

It describes the flow control protocol at the circuit and stream level. If
there is no FlowCtrl protocol version, tor supports the unauthenticated flow
control features from its supported Relay protocols.

At this commit, relay will start advertising FlowCtrl=1 meaning they support
authenticated SENDMEs v1.

Closes #30363

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-03 13:56:36 -04:00
David Goulet
2aa441b88e Merge branch 'tor-github/pr/954' 2019-05-03 13:11:03 -04:00
George Kadianakis
7f2cd6545c Hiding crypt_path_t: Hide 'crypto' usage in sendme.c 2019-05-03 18:29:51 +03:00
George Kadianakis
ea5f355fc9 Hiding crypt_path_t: Change code to use the privatization macro. 2019-05-03 18:15:26 +03:00
George Kadianakis
55d35c0caa Hiding crypt_path_t: Hiding 'crypto' using a macro. 2019-05-03 18:15:26 +03:00
George Kadianakis
2ef0324639 Revert "Hiding crypt_path_t: Ensure that ->private is initialized."
This reverts commit 7497c9193a0f2c891a0802bf5fbe73cf7ec1ca99.
2019-05-03 18:15:26 +03:00
George Kadianakis
4060b7623d Revert "Hiding crypt_path_t: Create a constructor for crypt_path_t."
This reverts commit ab8b80944967ee5a6a0c45dbf61839cf257bfe44.
2019-05-03 18:15:26 +03:00
George Kadianakis
2e9e3e7d41 Hiding crypt_path_t: Some TODO notes for future directions. 2019-05-03 18:15:26 +03:00
George Kadianakis
0ed5c6edf9 Hiding crypt_path_t: Move some more crypt_path-specific functions.
- Move test-only cpath_get_n_hops() to crypt_path.c.
- Move onion_next_hop_in_cpath() and rename to cpath_get_next_non_open_hop().

The latter function was directly accessing cpath->state, and it's a first step
at hiding ->state.
2019-05-03 18:15:26 +03:00
George Kadianakis
58fbbc1409 Hiding crypt_path_t: Rename some functions to fit the crypt_path API.
Some of these functions are now public and cpath-specific so their name should
signify the fact they are part of the cpath module:

assert_cpath_layer_ok -> cpath_assert_layer_ok
assert_cpath_ok -> cpath_assert_ok
onion_append_hop -> cpath_append_hop
circuit_init_cpath_crypto -> cpath_init_circuit_crypto
circuit_free_cpath_node -> cpath_free
onion_append_to_cpath -> cpath_extend_linked_list
2019-05-03 18:15:26 +03:00
George Kadianakis
593b7726e9 Hiding crypt_path_t: Trivial changes to satisfy check-local. 2019-05-03 18:15:26 +03:00
George Kadianakis
cd38e41620 Hiding crypt_path_t: Ensure that ->private is initialized.
Now that we are using a constructor we should be more careful that we are
always using the constructor to initialize crypt_path_t, so make sure that
->private is initialized.
2019-05-03 18:15:11 +03:00
George Kadianakis
f5635989b0 Hiding crypt_path_t: Create a constructor for crypt_path_t.
We are using an opaque pointer so the structure needs to be allocated on the
heap. This means we now need a constructor for crypt_path_t.

Also modify all places initializing a crypt_path_t to use the constructor.
2019-05-03 18:15:11 +03:00
George Kadianakis
18d61c0e6e Hiding crypt_path_t: Fixup broken unittests. 2019-05-03 18:15:11 +03:00
George Kadianakis
4bd0c4852a Hiding crypt_path_t: Move the free func in crypt_path.c.
Again everything is moved, apart from a free line using ->private.
2019-05-03 18:15:11 +03:00
George Kadianakis
5f96b7abcc Hiding crypt_path_t: Move some more init funcs in crypt_path.c.
Everything is moved, but the argument of the function is edited to access
->private->crypto.
2019-05-03 18:15:11 +03:00
George Kadianakis
0c5176d00c Hiding crypt_path_t: Start with crypt_path.crypto .
Create some functions to eventually be able to hide crypt_path_t.crypto.
2019-05-03 18:15:11 +03:00
George Kadianakis
f74a80dc3b Hiding crypt_path_t: Move init functions to crypt_path.c.
This commit only moves code.
2019-05-03 18:15:00 +03:00
George Kadianakis
9584798e57 Hiding crypt_path_t: Move assert functions in crypt_path.c.
This commit only moves code, and makes one function public.
2019-05-03 18:15:00 +03:00
George Kadianakis
c3a5e6b436 Hiding crypt_path_t: Introduce opaque crypt_path_private_t .
This will be our base for incrementally hiding crypt_path_t.
2019-05-03 18:15:00 +03:00
David Goulet
b3492d53c3 Merge branch 'tor-github/pr/984'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-03 10:56:12 -04:00
Nick Mathewson
2b54733dc0 Bump to 0.4.0.5-dev 2019-05-02 14:32:55 -04:00
George Kadianakis
b2c2cb9287 Merge branch 'tor-github/pr/986' 2019-05-02 18:12:52 +03:00
George Kadianakis
4d461e20f7 Merge branch 'tor-github/pr/999' 2019-05-02 17:48:09 +03:00
Nick Mathewson
721e65a1d5 Add comments to include.am files to note where new sources go
This mechanism isn't perfect, and sometimes it will guess wrong,
but it will help our automation.
2019-05-02 09:37:18 -04:00
Nick Mathewson
9c3aa22740 Remove some now-needless dirauth includes 2019-05-02 09:22:13 -04:00
Nick Mathewson
a45413e7d5 Make keypin.c dirauth-only 2019-05-02 09:22:13 -04:00
Nick Mathewson
31fb4a7845 Make the bwauth.c module dirauth-only. 2019-05-02 09:22:13 -04:00
Nick Mathewson
6f42efaa59 Move voteflags.[ch] to become dirauth only.
For various reasons, this was a nontrivial movement.  There are
several places in the code where we do something like "update the
flags on this routerstatus or node if we're an authority", and at
least one where we pretended to be an authority when we weren't.
2019-05-02 09:22:13 -04:00
David Goulet
77bd219808 sendme: Improve logging messages
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-02 08:58:58 -04:00
Mike Perry
d0fb74c902 Bug 29231: Report correct padding write totals and enabled totals. 2019-05-01 21:00:26 +00:00
Nick Mathewson
4973ceb46b Merge remote-tracking branch 'tor-github/pr/950' 2019-05-01 09:11:20 -04:00
Nick Mathewson
cafb999810 bump to 0.4.0.5 2019-04-30 15:53:53 -04:00
Nick Mathewson
996f7c75ba Make the reachability.c module dirauth-only. 2019-04-30 15:00:08 -04:00
Nick Mathewson
339ac4dc67 Make the guardfraction.c module dirauth-only. 2019-04-30 15:00:08 -04:00
Nick Mathewson
857bfc7033 Make the process_descs.c module dirauth-only. 2019-04-30 15:00:07 -04:00
Nick Mathewson
aab0245924 Make the dsigs_parse.c module dirauth-only. 2019-04-30 15:00:07 -04:00
Nick Mathewson
853942b71e Make the recommend_pkg file dirauth-only. 2019-04-30 15:00:07 -04:00
Nick Mathewson
295feeb093 Replace all remaining tor_mem_is_zero() with fast_mem_is_zero() 2019-04-30 14:49:05 -04:00
Nick Mathewson
0034f10956 Use safe_mem_is_zero in a few more places.
I don't believe any of these represent a real timing vulnerability
(remote timing against memcmp() on a modern CPU is not easy), but
these are the ones where I believe we should be more careful.
2019-04-30 14:45:58 -04:00
Nick Mathewson
309467c64e Rename tor_mem_is_zero to fast_mem_is_zero()
For memeq and friends, "tor_" indicates constant-time and "fast_"
indicates optimized.  I'm fine with leaving the constant-time
"safe_mem_is_zero" with its current name, but the "tor_" prefix on
the current optimized version is misleading.

Also, make the tor_digest*_is_zero() uniformly constant-time, and
add a fast_digest*_is_zero() version to use as needed.

A later commit in this branch will fix all the users of
tor_mem_is_zero().

Closes ticket 30309.
2019-04-30 14:45:51 -04:00
Taylor Yu
68caca58a8 Clean up formatting after Coccinelle
Clean up some minor formatting quirks after the Coccinelle run.
2019-04-30 13:18:46 -05:00
Taylor Yu
983452e221 Run Coccinelle for control.c refactor 2019-04-30 13:18:46 -05:00
Taylor Yu
769eb07a7a Manually fix some control replies
Manually fix up some reply-generating code that the Coccinelle scripts
won't match.  Some more complicated ones remain -- these are mostly
ones that accumulate data to send, and then call connection_buf_add()
or connection_write_str_to_buf() directly.
2019-04-30 13:18:46 -05:00
Taylor Yu
61976a4b1c Factor out control reply output
Create a set of abstractions for controller commands and events to
output replies to the control channel.  The control protocol has a
relatively consistent SMTP-like structure, so it's helpful when code
that implements control commands and events doesn't explicitly format
everything on its own.
2019-04-30 13:18:46 -05:00
Taylor Yu
482437754a Add clarifying comments to control_proto.c
Refer to control-spec.txt grammar productions in comments in
control_proto.c for clarity.
2019-04-30 13:18:46 -05:00
Taylor Yu
8e7316bae4 Split reply formatting out of control_fmt.c
Split the core reply formatting code out of control_fmt.c into
control_proto.c.  The remaining code in control_format.c deals with
specific subsystems and will eventually move to join those subsystems.
2019-04-30 13:18:46 -05:00
Taylor Yu
965c2064da Correct file name in doxygen comment 2019-04-30 13:18:46 -05:00
George Kadianakis
847fc3280d Merge branch 'maint-0.4.0' 2019-04-30 19:26:30 +03:00
George Kadianakis
e1d4e2badb Merge branch 'tor-github/pr/978' into maint-0.4.0 2019-04-30 19:26:14 +03:00
George Kadianakis
d885ed867f Merge branch 'tor-github/pr/937' 2019-04-30 19:21:46 +03:00
George Kadianakis
9084a90b00 Merge branch 'tor-github/pr/936' 2019-04-30 19:21:15 +03:00
George Kadianakis
a44aca5453 Merge branch 'tor-github/pr/993' 2019-04-30 19:13:57 +03:00
George Kadianakis
86f8dfe419 Merge branch 'tor-github/pr/983' 2019-04-30 19:13:30 +03:00
David Goulet
43c119fedb Merge branch 'tor-github/pr/980'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-30 11:50:36 -04:00
David Goulet
e543c4e20c Merge branch 'tor-github/pr/909'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-30 11:17:45 -04:00
Nick Mathewson
b7cc631d23 Rename and clarify some functions for periodic events
When we tell the periodic event manager about an event, we are
"registering" that event.  The event sits around without being
usable, however, until we "connect" the event to libevent.  In the
end, we "disconnect" the event and remove its libevent parts.

Previously, we called these operations "add", "setup", and
"destroy", which led to confusion.
2019-04-30 11:14:59 -04:00
Nick Mathewson
9a62a820fb Remove now-extraneous calls to initialize_periodic_events().
This is now the responsibility of the mainloop's subsystem initializer.
2019-04-30 11:14:59 -04:00
Nick Mathewson
b5a62b1ef5 Move dirauth periodic events into dirauth module.
Closes ticket 30294.
2019-04-30 11:14:59 -04:00
Nick Mathewson
6eb1b8da0a Turn 'mainloop' into a subsystem.
We need a little refactoring for this to work, since the
initialization code for the periodic events assumes that libevent is
already initialized, which it can't be until it's configured.

This change, combined with the previous ones, lets other subsystems
declare their own periodic events, without mainloop.c having to know
about them.  Implements ticket 30293.
2019-04-30 11:14:59 -04:00
Nick Mathewson
730dddc380 Make sure that the rng is not replaced if it is already replaced. 2019-04-30 11:11:39 -04:00
Nick Mathewson
604e849d36 Make the deterministic and reproducible rng test code handle fast_rng 2019-04-30 11:11:39 -04:00
Nick Mathewson
587a525cc5 Add improved debugging support to crypto_rand_fast code. 2019-04-30 11:11:39 -04:00
Nick Mathewson
e66b5153bd Extract add-entropy code from crypto_fast_rng to a new function 2019-04-30 11:11:39 -04:00
Nick Mathewson
c6a93beed8 Use preloaded-rng code in test_hs_descriptor.c 2019-04-30 11:11:39 -04:00
Nick Mathewson
7086a9f90e Make rng mock code also cover strongest_rand. 2019-04-30 11:11:39 -04:00
Nick Mathewson
0a9fb6938d Use prefilled PRNG replacement in test_extorport
This is the last remaining place where our tests had mocked
crypto_rand.
2019-04-30 11:11:39 -04:00
Nick Mathewson
7bd34698af Use prefilled_rng in test_addr.c in place of existing code. 2019-04-30 11:11:39 -04:00
Nick Mathewson
fe173ce0bc Add a testing PRNG replacement that returns canned data. 2019-04-30 11:11:39 -04:00
Nick Mathewson
64d5ed0415 Update circuit_timeout test to use deterministic prng 2019-04-30 11:11:39 -04:00
Nick Mathewson
d3526d3f2c Update test_prob_distr to use new reproducible RNG override code 2019-04-30 11:11:39 -04:00
Nick Mathewson
11eaed66bb Add support for deterministic override of crypto_rand() in tests
We had this previously, but we did it differently in different
places. This implementation is pulled from test_prob_distr.c
2019-04-30 11:11:39 -04:00
Nick Mathewson
821dd54586 Merge branch 'bug30316_035' into bug30316_040
Fixes conflicts and also moves bandwidth-file-digest.
2019-04-29 14:34:03 -04:00
Nick Mathewson
0ab4dc7ef7 Move bandwidth-file-headers line to appear in the correct vote section
Fixes bug 30316; bugfix on 0.3.5.1-alpha.
2019-04-29 14:31:09 -04:00
David Goulet
0f2ff267c5 sendme: Do not poke at crypto.sendme_digest directly
As per review from nickm, keep as much as we can the relay_crypto_t object
opaque.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
0d8b9b56c5 sendme: Better function names
From nickm's review, improve the names of some functions.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
67c2254183 sendme: Move note_cell_digest() to relay_crypto module
Because this function is poking within the relay_crypto_t object, move the
function to the module so we can keep it opaque as much as possible.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
d084f9115d sendme: Better handle the random padding
We add random padding to every cell if there is room. This commit not only
fixes how we compute that random padding length/offset but also improves its
safety with helper functions and a unit test.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
c7385b5b14 sendme: Keep cell digest only if a SENDME is next
This way, we reduce the load by only hashing when we absolutely must.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
805c81efed sendme: Add helper to note the cell digest
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
7c8e519b34 sendme: Helper to know if next cell is a SENDME
We'll use it this in order to know when to hash the cell for the SENDME
instead of doing it at every cell.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
aef7095c3e prop289: Add documentation for the circuit FIFO list
Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
44750b0de6 prop289: Skip the first 4 unused bytes in a cell
When adding random to a cell, skip the first 4 bytes and leave them zeroed. It
has been very useful in the past for us to keep bytes like this.

Some code trickery was added to make sure we have enough room for this 4 bytes
offset when adding random.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
77d560af64 prop289: Keep the digest bytes, not the object
The digest object is as large as the entire internal digest object's state,
which is often much larger than the actual set of bytes you're transmitting.

This commit makes it that we keep the digest itself which is 20 bytes.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
4efe9d653a prop289: Move digest matching in its own function
No behavior change but code had to be refactored a bit. Also, the tor_memcmp()
was changed to tor_memneq().

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
217b553193 prop289: Rename packaged functions with better name
The circuit and stream level functions that update the package window have
been renamed to have a "_note_" in them to make their purpose more clear.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
2ec25e847e prop289: Move SENDME cell processing in a separate function
No behavior change. Only moving code and fixing part of it in order to use the
parameters passed as pointers.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
504e05b029 prop289: Use a 20 bytes digest instead of 4
To achieve such, this commit also changes the trunnel declaration to use a
union instead of a seperate object for the v1 data.

A constant is added for the digest length so we can use it within the SENDME
code giving us a single reference.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
cede93b2d8 tests: Implement unit tests for SENDME v1
Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
a6e012508e prop289: Add random bytes to the unused portion of the cell
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
402f0a4f5d prop289: Remember the last cell digest for v1 SENDMEs
In order to do so, depending on where the cell is going, we'll keep the last
cell digest that is either received inbound or sent outbound.

Then it can be used for validation.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
bb473a807a prop289: Match the SENDME digest
Now that we keep the last seen cell digests on the Exit side on the circuit
object, use that to match the SENDME v1 transforming this whole process into a
real authenticated SENDME mechanism.

Part of #26841

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
93f9fbbd34 prop289: Keep track of the last seen cell digests
This makes tor remember the last seen digest of a cell if that cell is the
last one before a SENDME on the Exit side.

Closes #26839

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
81706d8427 prop289: Support SENDME v1 cell parsing
This commit makes tor able to parse and handle a SENDME version 1. It will
look at the consensus parameter "sendme_accept_min_version" to know what is
the minimum version it should look at.

IMPORTANT: At this commit, the validation of the cell is not fully
implemented. For this, we need #26839 to be completed that is to match the
SENDME digest with the last cell digest.

Closes #26841

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
023a70da84 prop289: Support sending SENDME version 1
This code will obey the consensus parameter "sendme_emit_min_version" to know
which SENDME version it should send. For now, the default is 0 and the
parameter is not yet used in the consensus.

This commit adds the support to send version 1 SENDMEs but aren't sent on the
wire at this commit.

Closes #26840

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
eef78ac0b0 prop289: Add SENDME trunnel declaration
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
c38d46bf4a prop289: Add two consensus parameters
In order to be able to deploy the authenticated SENDMEs, these two consensus
parameters are needed to control the minimum version that we can emit and
accept.

See section 4 in prop289 for more details.

Note that at this commit, the functions that return the values aren't used so
compilation fails if warnings are set to errors.

Closes #26842

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
0e6e800c89 sendme: Always close stream if deliver window is negative
Previously, we would only close the stream when our deliver window was
negative at the circuit-level but _not_ at the stream-level when receiving a
DATA cell.

This commit adds an helper function connection_edge_end_close() which
sends an END and then mark the stream for close for a given reason.

That function is now used both in case the deliver window goes below zero for
both circuit and stream level.

Part of #26840

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
8e38791baf sendme: Add helper functions for DATA cell packaging
When we are about to send a DATA cell, we have to decrement the package window
for both the circuit and stream level.

This commit adds helper functions to handle the package window decrement.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
2d3c600915 sendme: Add helper functions for DATA cell delivery
When we get a relay DATA cell delivered, we have to decrement the deliver
window on both the circuit and stream level.

This commit adds helper functions to handle the deliver window decrement.

Part of #26840

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
9c42cc1eb2 sendme: Refactor SENDME cell processing
This is a bit of a complicated commit. It moves code but also refactors part
of it. No behavior change, the idea is to split things up so we can better
handle and understand how SENDME cells are processed where ultimately it will
be easier to handle authenticated SENDMEs (prop289) using the intermediate
functions added in this commit.

The entry point for the cell arriving at the edge (Client or Exit), is
connection_edge_process_relay_cell() for which we look if it is a circuit or
stream level SENDME. This commit refactors that part where two new functions
are introduced to process each of the SENDME types.

The sendme_process_circuit_level() has basically two code paths. If we are a
Client (the circuit is origin) or we are an Exit. Depending on which, the
package window is updated accordingly. Then finally, we resume the reading on
every edge streams on the circuit.

The sendme_process_stream_level() applies on the edge connection which will
update the package window if needed and then will try to empty the inbuf if
need be because we can now deliver more cells.

Again, no behavior change but in order to split that code properly into their
own functions and outside the relay.c file, code modification was needed.

Part of #26840.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
ed8593b9e0 sendme: Modernize and cleanup old moved code
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
e5806dcea8 sendme: Move code to the new files sendme.{c|h}
Take apart the SENDME cell specific code and put it in sendme.{c|h}. This is
part of prop289 that implements authenticated SENDMEs.

Creating those new files allow for the already huge relay.c to not grow in LOC
and makes it easier to handle and test the SENDME cells in an isolated way.

This commit only moves code. No behavior change.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:33 -04:00
Nick Mathewson
806539b40a Use fast check for missing id in node_is_a_configured_bridge()
Fixes bug 30308; bugfix on 0.3.5.1-alpha.
2019-04-26 11:19:46 -04:00
Nick Mathewson
650b94ebc1 Use a linear algorithm to subtract two nodelists.
The nodelist_idx for each node_t serves as a unique identifier for
the node, so we can use a bitarray to hold all the excluded
nodes, and then remove them from the smartlist.

Previously use used smartlist_subtract(sl, excluded), which is
O(len(sl)*len(excluded)).

We can use this function in other places too, but this is the one
that showed up on the profiles of 30291.

Closes ticket 30307.
2019-04-26 11:04:44 -04:00
Nick Mathewson
1d44ac9acd Make nodelist_get_list() return a const pointer. 2019-04-26 10:36:49 -04:00
Nick Mathewson
efeb101b96 Merge remote-tracking branch 'tor-github/pr/889' 2019-04-25 20:25:34 -04:00
Nick Mathewson
36b4fc7437 Merge remote-tracking branch 'tor-github/pr/922' 2019-04-25 20:08:39 -04:00
Nick Mathewson
a5cced2b7a Extract keyword argument checking from argument parsing. 2019-04-25 14:13:03 -04:00
Nick Mathewson
a0299cd240 In control command api, rename "object" to "cmddata"
This makes it match control-spec.txt.
2019-04-25 14:13:03 -04:00
Nick Mathewson
ff9ba7d6c4 expand CMD_FL_WIPE to wipe the parsed arguments too 2019-04-25 14:13:03 -04:00
Nick Mathewson
88d22b898e Simplify handler logic in control_cmd.c
Now that the legacy handlers are gone, we can simplify the
structures and macros here.
2019-04-25 14:13:03 -04:00
Nick Mathewson
ddd33d39c7 Port the authenticate and authchallenge commands to the new parser
These two presented their own challenge, because of their use of
QString, and their distinguished handling of quoted versus
non-quoted values.
2019-04-25 14:13:03 -04:00
Nick Mathewson
8799b4e805 Add rudimentary qstring support to kvline.c 2019-04-25 14:13:03 -04:00
Nick Mathewson
ba05324242 Move and rename decode_escaped_string()
This function decodes something different from the usual c-escaped
format.

It is only used in controller authorization.
2019-04-25 14:13:03 -04:00
Nick Mathewson
0c0b869ba4 Use the new controller command parser for EXTENDCIRCUIT.
This command does not fit perfectly with the others, since its
second argument is optional and may contain equal signs.  Still,
it's probably better to squeeze it into the new metaformat, since
doing so allows us to remove several pieces of the old
command-parsing machinery.
2019-04-25 14:13:03 -04:00
Nick Mathewson
95afdb005c Use new parser logic for SETCONF/RESETCONF code.
Here we get to throw away a LOT of unused code, since most of the
old parsing was redundant with kvline.
2019-04-25 14:13:03 -04:00
Nick Mathewson
d8b3ec865d Update more controller commands, now that we have kvline support 2019-04-25 14:13:03 -04:00
Nick Mathewson
dab35386ca Add a case-insensitive variant to config_line_find() 2019-04-25 14:13:03 -04:00
Nick Mathewson
9471391694 Add kvline support to controller command parser.
This should let us handle all (or nearly all) of the remaining
commands.
2019-04-25 14:13:03 -04:00
Nick Mathewson
0841a69357 Allow kvlines in control commands. 2019-04-25 14:13:03 -04:00
Nick Mathewson
bb37ad6957 Add fuzzing support for several more groups of kvlines flags 2019-04-25 14:13:03 -04:00
Nick Mathewson
73df91bbb5 kvline: handle empty alues as well as empty keys
The two options are mutually exclusive, since otherwise an entry
like "Foo" would be ambiguous.  We want to have the ability to treat
entries like this as keys, though, since some controller commands
interpret them as flags.
2019-04-25 14:13:03 -04:00
Nick Mathewson
01b07c548b Use parsing code for the simpler controller commands.
(This should be all of the command that work nicely with positional
arguments only.)

Some of these commands should probably treat extra arguments as
incorrect, but for now I'm trying to be careful not to break
any existing users.
2019-04-25 14:13:03 -04:00
Nick Mathewson
cbd1a7e053 Unit tests for current control-command parser logic 2019-04-25 14:13:03 -04:00
Nick Mathewson
dbfe1a14e4 When parsing a multiline controller command, be careful with linebreaks
The first line break in particular was mishandled: it was discarded
if no arguments came before it, which made it impossible to
distinguish arguments from the first line of the body.

To solve this, we need to allocate a copy of the command rather than
using NUL to separate it, since we might have "COMMAND\n" as our input.

Fixes ticket 29984.
2019-04-25 14:13:03 -04:00
Nick Mathewson
f18b7dc473 Extract the argument-splitting part of control.c's parser
This is preliminary work for fixing 29984; no behavior has changed.
2019-04-25 14:13:03 -04:00
Nick Mathewson
de70eebc65 Start on a command-parsing tool for controller commands.
There _is_ an underlying logic to these commands, but it isn't
wholly uniform, given years of tweaks and changes.  Fortunately I
think there is a superset that will work.

This commit adds a parser for some of the most basic cases -- the
ones currently handled by getargs_helper() and some of the
object-taking ones.  Soon will come initial tests; then I'll start using
the parser.

After that, I'll expand the parser to handle the other cases that come
up in the controller protocol.
2019-04-25 14:13:03 -04:00
Nick Mathewson
0d650e7958 Move responsibility for checking if events are setup into periodic.c
We have checks in various places in mainlook.c to make sure that
events are initialized before we invoke any periodic_foo() functions
on them.  But now that each subsystem will own its own periodic
events, it will be cleaner if we don't assume that they are all
setup or not.
2019-04-25 13:23:18 -04:00
Nick Mathewson
233835e14f Move the responsibility for listing periodic events to periodic.c
The end goal here is to move the periodic callback to their
respective modules, so that mainloop.c doesn't have to include so
many other things.

This patch doesn't actually move any of the callbacks out of
mainloop.c yet.
2019-04-25 10:09:36 -04:00
George Kadianakis
974c2674eb Merge branch 'maint-0.4.0' 2019-04-25 15:47:07 +03:00
George Kadianakis
a39789a02c Merge branch 'tor-github/pr/960' into maint-0.4.0 2019-04-25 15:46:45 +03:00
George Kadianakis
811a93f803 Merge branch 'tor-github/pr/953' 2019-04-25 15:43:47 +03:00
Alexander Færøy
0429072495 Lower log level of unlink() errors in networkstatus_set_current_consensus().
In this patch we lower the log level of the failures for the three calls
to unlink() in networkstatus_set_current_consensus(). These errors might
trigger on Windows because the memory mapped consensus file keeps the
file in open state even after we have close()'d it. Windows will then
error on the unlink() call with a "Permission denied" error.

The consequences of ignoring these errors is that we leave an unused
file around on the file-system, which is an easier way to fix this
problem right now than refactoring networkstatus_set_current_consensus().

See: https://bugs.torproject.org/29930
2019-04-25 01:59:37 +02:00
Nick Mathewson
6320b2988c Merge remote-tracking branch 'tor-github/pr/942' 2019-04-24 17:06:56 -04:00
David Goulet
f39dd0a700 Merge branch 'tor-github/pr/951' 2019-04-24 09:53:15 -04:00
David Goulet
608cf2ba8d Merge branch 'tor-github/pr/943' 2019-04-24 09:51:14 -04:00
David Goulet
5bcf87c224 Merge branch 'tor-github/pr/955' 2019-04-24 09:45:32 -04:00
David Goulet
4b599aaae4 Merge branch 'tor-github/pr/939' 2019-04-24 09:43:28 -04:00
teor
f35bd36814
test-network-all: Test IPv6-only v3 single onion services
In "make test-network-all", test IPv6-only v3 single onion services,
using the chutney network single-onion-v23-ipv6-md. This test will
not pass until 23588 has been merged.

Closes ticket 27251.
2019-04-24 17:42:59 +10:00
teor
3d89f0374a
hs_config: Allow Tor to be configured as an IPv6-only v3 single onion service
Part of #23588.
2019-04-24 17:29:18 +10:00
teor
f05e3f3c9a
test/hs: Correctly convert an IPv6 intro point to an extend_info
Part of #23588.
2019-04-24 17:29:14 +10:00
teor
abe086dd97
test/hs: Re-enable the IPv6 intro point in the HS tests
Tests 23588, partially reverts 23576.

Implements 29237.
2019-04-24 17:29:10 +10:00
Neel Chauhan
b65f8c419a
Add firewall_choose_address_ls() and hs_get_extend_info_from_lspecs() tests 2019-04-24 17:28:38 +10:00
Neel Chauhan
2618347657
Use fascist_firewall_choose_address_ls() in hs_get_extend_info_from_lspecs() 2019-04-24 17:28:34 +10:00