Commit Graph

24223 Commits

Author SHA1 Message Date
David Goulet
3885e7b44b Merge branch 'tor-github/pr/1000'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-08 08:02:28 -04:00
Neel Chauhan
3cafdeb8c0 Only call tor_addr_parse() in circuit_is_acceptable() when needed 2019-05-07 11:52:56 -04:00
David Goulet
b72f5da03d Merge branch 'tor-github/pr/994'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-07 08:02:39 -04:00
Nick Mathewson
3c2648bbda Move "relay" and "router" periodic callbacks out of mainloop.c
(Some of these callbacks are specific to the OR module, so now it's
time to have an or_sys and or_periodic.)
2019-05-06 16:35:39 -04:00
Nick Mathewson
b394b5b2af Create a relay subsystem and move the shutdown functions there 2019-05-06 16:25:09 -04:00
David Goulet
07e4b09b5f sendme: Add FlowCtrl protover value
See proposal 289 section 4.3 for more details.

It describes the flow control protocol at the circuit and stream level. If
there is no FlowCtrl protocol version, tor supports the unauthenticated flow
control features from its supported Relay protocols.

At this commit, relay will start advertising FlowCtrl=1 meaning they support
authenticated SENDMEs v1.

Closes #30363

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-03 13:56:36 -04:00
David Goulet
2aa441b88e Merge branch 'tor-github/pr/954' 2019-05-03 13:11:03 -04:00
George Kadianakis
7f2cd6545c Hiding crypt_path_t: Hide 'crypto' usage in sendme.c 2019-05-03 18:29:51 +03:00
George Kadianakis
ea5f355fc9 Hiding crypt_path_t: Change code to use the privatization macro. 2019-05-03 18:15:26 +03:00
George Kadianakis
55d35c0caa Hiding crypt_path_t: Hiding 'crypto' using a macro. 2019-05-03 18:15:26 +03:00
George Kadianakis
2ef0324639 Revert "Hiding crypt_path_t: Ensure that ->private is initialized."
This reverts commit 7497c9193a0f2c891a0802bf5fbe73cf7ec1ca99.
2019-05-03 18:15:26 +03:00
George Kadianakis
4060b7623d Revert "Hiding crypt_path_t: Create a constructor for crypt_path_t."
This reverts commit ab8b80944967ee5a6a0c45dbf61839cf257bfe44.
2019-05-03 18:15:26 +03:00
George Kadianakis
2e9e3e7d41 Hiding crypt_path_t: Some TODO notes for future directions. 2019-05-03 18:15:26 +03:00
George Kadianakis
0ed5c6edf9 Hiding crypt_path_t: Move some more crypt_path-specific functions.
- Move test-only cpath_get_n_hops() to crypt_path.c.
- Move onion_next_hop_in_cpath() and rename to cpath_get_next_non_open_hop().

The latter function was directly accessing cpath->state, and it's a first step
at hiding ->state.
2019-05-03 18:15:26 +03:00
George Kadianakis
58fbbc1409 Hiding crypt_path_t: Rename some functions to fit the crypt_path API.
Some of these functions are now public and cpath-specific so their name should
signify the fact they are part of the cpath module:

assert_cpath_layer_ok -> cpath_assert_layer_ok
assert_cpath_ok -> cpath_assert_ok
onion_append_hop -> cpath_append_hop
circuit_init_cpath_crypto -> cpath_init_circuit_crypto
circuit_free_cpath_node -> cpath_free
onion_append_to_cpath -> cpath_extend_linked_list
2019-05-03 18:15:26 +03:00
George Kadianakis
593b7726e9 Hiding crypt_path_t: Trivial changes to satisfy check-local. 2019-05-03 18:15:26 +03:00
George Kadianakis
cd38e41620 Hiding crypt_path_t: Ensure that ->private is initialized.
Now that we are using a constructor we should be more careful that we are
always using the constructor to initialize crypt_path_t, so make sure that
->private is initialized.
2019-05-03 18:15:11 +03:00
George Kadianakis
f5635989b0 Hiding crypt_path_t: Create a constructor for crypt_path_t.
We are using an opaque pointer so the structure needs to be allocated on the
heap. This means we now need a constructor for crypt_path_t.

Also modify all places initializing a crypt_path_t to use the constructor.
2019-05-03 18:15:11 +03:00
George Kadianakis
18d61c0e6e Hiding crypt_path_t: Fixup broken unittests. 2019-05-03 18:15:11 +03:00
George Kadianakis
4bd0c4852a Hiding crypt_path_t: Move the free func in crypt_path.c.
Again everything is moved, apart from a free line using ->private.
2019-05-03 18:15:11 +03:00
George Kadianakis
5f96b7abcc Hiding crypt_path_t: Move some more init funcs in crypt_path.c.
Everything is moved, but the argument of the function is edited to access
->private->crypto.
2019-05-03 18:15:11 +03:00
George Kadianakis
0c5176d00c Hiding crypt_path_t: Start with crypt_path.crypto .
Create some functions to eventually be able to hide crypt_path_t.crypto.
2019-05-03 18:15:11 +03:00
George Kadianakis
f74a80dc3b Hiding crypt_path_t: Move init functions to crypt_path.c.
This commit only moves code.
2019-05-03 18:15:00 +03:00
George Kadianakis
9584798e57 Hiding crypt_path_t: Move assert functions in crypt_path.c.
This commit only moves code, and makes one function public.
2019-05-03 18:15:00 +03:00
George Kadianakis
c3a5e6b436 Hiding crypt_path_t: Introduce opaque crypt_path_private_t .
This will be our base for incrementally hiding crypt_path_t.
2019-05-03 18:15:00 +03:00
David Goulet
b3492d53c3 Merge branch 'tor-github/pr/984'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-03 10:56:12 -04:00
Nick Mathewson
2b54733dc0 Bump to 0.4.0.5-dev 2019-05-02 14:32:55 -04:00
George Kadianakis
b2c2cb9287 Merge branch 'tor-github/pr/986' 2019-05-02 18:12:52 +03:00
George Kadianakis
4d461e20f7 Merge branch 'tor-github/pr/999' 2019-05-02 17:48:09 +03:00
Nick Mathewson
721e65a1d5 Add comments to include.am files to note where new sources go
This mechanism isn't perfect, and sometimes it will guess wrong,
but it will help our automation.
2019-05-02 09:37:18 -04:00
Nick Mathewson
9c3aa22740 Remove some now-needless dirauth includes 2019-05-02 09:22:13 -04:00
Nick Mathewson
a45413e7d5 Make keypin.c dirauth-only 2019-05-02 09:22:13 -04:00
Nick Mathewson
31fb4a7845 Make the bwauth.c module dirauth-only. 2019-05-02 09:22:13 -04:00
Nick Mathewson
6f42efaa59 Move voteflags.[ch] to become dirauth only.
For various reasons, this was a nontrivial movement.  There are
several places in the code where we do something like "update the
flags on this routerstatus or node if we're an authority", and at
least one where we pretended to be an authority when we weren't.
2019-05-02 09:22:13 -04:00
David Goulet
77bd219808 sendme: Improve logging messages
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-05-02 08:58:58 -04:00
Mike Perry
d0fb74c902 Bug 29231: Report correct padding write totals and enabled totals. 2019-05-01 21:00:26 +00:00
Nick Mathewson
4973ceb46b Merge remote-tracking branch 'tor-github/pr/950' 2019-05-01 09:11:20 -04:00
Nick Mathewson
cafb999810 bump to 0.4.0.5 2019-04-30 15:53:53 -04:00
Nick Mathewson
996f7c75ba Make the reachability.c module dirauth-only. 2019-04-30 15:00:08 -04:00
Nick Mathewson
339ac4dc67 Make the guardfraction.c module dirauth-only. 2019-04-30 15:00:08 -04:00
Nick Mathewson
857bfc7033 Make the process_descs.c module dirauth-only. 2019-04-30 15:00:07 -04:00
Nick Mathewson
aab0245924 Make the dsigs_parse.c module dirauth-only. 2019-04-30 15:00:07 -04:00
Nick Mathewson
853942b71e Make the recommend_pkg file dirauth-only. 2019-04-30 15:00:07 -04:00
Nick Mathewson
295feeb093 Replace all remaining tor_mem_is_zero() with fast_mem_is_zero() 2019-04-30 14:49:05 -04:00
Nick Mathewson
0034f10956 Use safe_mem_is_zero in a few more places.
I don't believe any of these represent a real timing vulnerability
(remote timing against memcmp() on a modern CPU is not easy), but
these are the ones where I believe we should be more careful.
2019-04-30 14:45:58 -04:00
Nick Mathewson
309467c64e Rename tor_mem_is_zero to fast_mem_is_zero()
For memeq and friends, "tor_" indicates constant-time and "fast_"
indicates optimized.  I'm fine with leaving the constant-time
"safe_mem_is_zero" with its current name, but the "tor_" prefix on
the current optimized version is misleading.

Also, make the tor_digest*_is_zero() uniformly constant-time, and
add a fast_digest*_is_zero() version to use as needed.

A later commit in this branch will fix all the users of
tor_mem_is_zero().

Closes ticket 30309.
2019-04-30 14:45:51 -04:00
Taylor Yu
68caca58a8 Clean up formatting after Coccinelle
Clean up some minor formatting quirks after the Coccinelle run.
2019-04-30 13:18:46 -05:00
Taylor Yu
983452e221 Run Coccinelle for control.c refactor 2019-04-30 13:18:46 -05:00
Taylor Yu
769eb07a7a Manually fix some control replies
Manually fix up some reply-generating code that the Coccinelle scripts
won't match.  Some more complicated ones remain -- these are mostly
ones that accumulate data to send, and then call connection_buf_add()
or connection_write_str_to_buf() directly.
2019-04-30 13:18:46 -05:00
Taylor Yu
61976a4b1c Factor out control reply output
Create a set of abstractions for controller commands and events to
output replies to the control channel.  The control protocol has a
relatively consistent SMTP-like structure, so it's helpful when code
that implements control commands and events doesn't explicitly format
everything on its own.
2019-04-30 13:18:46 -05:00
Taylor Yu
482437754a Add clarifying comments to control_proto.c
Refer to control-spec.txt grammar productions in comments in
control_proto.c for clarity.
2019-04-30 13:18:46 -05:00
Taylor Yu
8e7316bae4 Split reply formatting out of control_fmt.c
Split the core reply formatting code out of control_fmt.c into
control_proto.c.  The remaining code in control_format.c deals with
specific subsystems and will eventually move to join those subsystems.
2019-04-30 13:18:46 -05:00
Taylor Yu
965c2064da Correct file name in doxygen comment 2019-04-30 13:18:46 -05:00
George Kadianakis
847fc3280d Merge branch 'maint-0.4.0' 2019-04-30 19:26:30 +03:00
George Kadianakis
e1d4e2badb Merge branch 'tor-github/pr/978' into maint-0.4.0 2019-04-30 19:26:14 +03:00
George Kadianakis
d885ed867f Merge branch 'tor-github/pr/937' 2019-04-30 19:21:46 +03:00
George Kadianakis
9084a90b00 Merge branch 'tor-github/pr/936' 2019-04-30 19:21:15 +03:00
George Kadianakis
a44aca5453 Merge branch 'tor-github/pr/993' 2019-04-30 19:13:57 +03:00
George Kadianakis
86f8dfe419 Merge branch 'tor-github/pr/983' 2019-04-30 19:13:30 +03:00
David Goulet
43c119fedb Merge branch 'tor-github/pr/980'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-30 11:50:36 -04:00
David Goulet
e543c4e20c Merge branch 'tor-github/pr/909'
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-30 11:17:45 -04:00
Nick Mathewson
b7cc631d23 Rename and clarify some functions for periodic events
When we tell the periodic event manager about an event, we are
"registering" that event.  The event sits around without being
usable, however, until we "connect" the event to libevent.  In the
end, we "disconnect" the event and remove its libevent parts.

Previously, we called these operations "add", "setup", and
"destroy", which led to confusion.
2019-04-30 11:14:59 -04:00
Nick Mathewson
9a62a820fb Remove now-extraneous calls to initialize_periodic_events().
This is now the responsibility of the mainloop's subsystem initializer.
2019-04-30 11:14:59 -04:00
Nick Mathewson
b5a62b1ef5 Move dirauth periodic events into dirauth module.
Closes ticket 30294.
2019-04-30 11:14:59 -04:00
Nick Mathewson
6eb1b8da0a Turn 'mainloop' into a subsystem.
We need a little refactoring for this to work, since the
initialization code for the periodic events assumes that libevent is
already initialized, which it can't be until it's configured.

This change, combined with the previous ones, lets other subsystems
declare their own periodic events, without mainloop.c having to know
about them.  Implements ticket 30293.
2019-04-30 11:14:59 -04:00
Nick Mathewson
730dddc380 Make sure that the rng is not replaced if it is already replaced. 2019-04-30 11:11:39 -04:00
Nick Mathewson
604e849d36 Make the deterministic and reproducible rng test code handle fast_rng 2019-04-30 11:11:39 -04:00
Nick Mathewson
587a525cc5 Add improved debugging support to crypto_rand_fast code. 2019-04-30 11:11:39 -04:00
Nick Mathewson
e66b5153bd Extract add-entropy code from crypto_fast_rng to a new function 2019-04-30 11:11:39 -04:00
Nick Mathewson
c6a93beed8 Use preloaded-rng code in test_hs_descriptor.c 2019-04-30 11:11:39 -04:00
Nick Mathewson
7086a9f90e Make rng mock code also cover strongest_rand. 2019-04-30 11:11:39 -04:00
Nick Mathewson
0a9fb6938d Use prefilled PRNG replacement in test_extorport
This is the last remaining place where our tests had mocked
crypto_rand.
2019-04-30 11:11:39 -04:00
Nick Mathewson
7bd34698af Use prefilled_rng in test_addr.c in place of existing code. 2019-04-30 11:11:39 -04:00
Nick Mathewson
fe173ce0bc Add a testing PRNG replacement that returns canned data. 2019-04-30 11:11:39 -04:00
Nick Mathewson
64d5ed0415 Update circuit_timeout test to use deterministic prng 2019-04-30 11:11:39 -04:00
Nick Mathewson
d3526d3f2c Update test_prob_distr to use new reproducible RNG override code 2019-04-30 11:11:39 -04:00
Nick Mathewson
11eaed66bb Add support for deterministic override of crypto_rand() in tests
We had this previously, but we did it differently in different
places. This implementation is pulled from test_prob_distr.c
2019-04-30 11:11:39 -04:00
Nick Mathewson
821dd54586 Merge branch 'bug30316_035' into bug30316_040
Fixes conflicts and also moves bandwidth-file-digest.
2019-04-29 14:34:03 -04:00
Nick Mathewson
0ab4dc7ef7 Move bandwidth-file-headers line to appear in the correct vote section
Fixes bug 30316; bugfix on 0.3.5.1-alpha.
2019-04-29 14:31:09 -04:00
David Goulet
0f2ff267c5 sendme: Do not poke at crypto.sendme_digest directly
As per review from nickm, keep as much as we can the relay_crypto_t object
opaque.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
0d8b9b56c5 sendme: Better function names
From nickm's review, improve the names of some functions.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
67c2254183 sendme: Move note_cell_digest() to relay_crypto module
Because this function is poking within the relay_crypto_t object, move the
function to the module so we can keep it opaque as much as possible.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
d084f9115d sendme: Better handle the random padding
We add random padding to every cell if there is room. This commit not only
fixes how we compute that random padding length/offset but also improves its
safety with helper functions and a unit test.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
c7385b5b14 sendme: Keep cell digest only if a SENDME is next
This way, we reduce the load by only hashing when we absolutely must.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
805c81efed sendme: Add helper to note the cell digest
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
7c8e519b34 sendme: Helper to know if next cell is a SENDME
We'll use it this in order to know when to hash the cell for the SENDME
instead of doing it at every cell.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
aef7095c3e prop289: Add documentation for the circuit FIFO list
Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
44750b0de6 prop289: Skip the first 4 unused bytes in a cell
When adding random to a cell, skip the first 4 bytes and leave them zeroed. It
has been very useful in the past for us to keep bytes like this.

Some code trickery was added to make sure we have enough room for this 4 bytes
offset when adding random.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
77d560af64 prop289: Keep the digest bytes, not the object
The digest object is as large as the entire internal digest object's state,
which is often much larger than the actual set of bytes you're transmitting.

This commit makes it that we keep the digest itself which is 20 bytes.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
4efe9d653a prop289: Move digest matching in its own function
No behavior change but code had to be refactored a bit. Also, the tor_memcmp()
was changed to tor_memneq().

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
217b553193 prop289: Rename packaged functions with better name
The circuit and stream level functions that update the package window have
been renamed to have a "_note_" in them to make their purpose more clear.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
2ec25e847e prop289: Move SENDME cell processing in a separate function
No behavior change. Only moving code and fixing part of it in order to use the
parameters passed as pointers.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
504e05b029 prop289: Use a 20 bytes digest instead of 4
To achieve such, this commit also changes the trunnel declaration to use a
union instead of a seperate object for the v1 data.

A constant is added for the digest length so we can use it within the SENDME
code giving us a single reference.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
cede93b2d8 tests: Implement unit tests for SENDME v1
Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
a6e012508e prop289: Add random bytes to the unused portion of the cell
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
402f0a4f5d prop289: Remember the last cell digest for v1 SENDMEs
In order to do so, depending on where the cell is going, we'll keep the last
cell digest that is either received inbound or sent outbound.

Then it can be used for validation.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
bb473a807a prop289: Match the SENDME digest
Now that we keep the last seen cell digests on the Exit side on the circuit
object, use that to match the SENDME v1 transforming this whole process into a
real authenticated SENDME mechanism.

Part of #26841

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
93f9fbbd34 prop289: Keep track of the last seen cell digests
This makes tor remember the last seen digest of a cell if that cell is the
last one before a SENDME on the Exit side.

Closes #26839

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
81706d8427 prop289: Support SENDME v1 cell parsing
This commit makes tor able to parse and handle a SENDME version 1. It will
look at the consensus parameter "sendme_accept_min_version" to know what is
the minimum version it should look at.

IMPORTANT: At this commit, the validation of the cell is not fully
implemented. For this, we need #26839 to be completed that is to match the
SENDME digest with the last cell digest.

Closes #26841

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
023a70da84 prop289: Support sending SENDME version 1
This code will obey the consensus parameter "sendme_emit_min_version" to know
which SENDME version it should send. For now, the default is 0 and the
parameter is not yet used in the consensus.

This commit adds the support to send version 1 SENDMEs but aren't sent on the
wire at this commit.

Closes #26840

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
eef78ac0b0 prop289: Add SENDME trunnel declaration
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
c38d46bf4a prop289: Add two consensus parameters
In order to be able to deploy the authenticated SENDMEs, these two consensus
parameters are needed to control the minimum version that we can emit and
accept.

See section 4 in prop289 for more details.

Note that at this commit, the functions that return the values aren't used so
compilation fails if warnings are set to errors.

Closes #26842

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
0e6e800c89 sendme: Always close stream if deliver window is negative
Previously, we would only close the stream when our deliver window was
negative at the circuit-level but _not_ at the stream-level when receiving a
DATA cell.

This commit adds an helper function connection_edge_end_close() which
sends an END and then mark the stream for close for a given reason.

That function is now used both in case the deliver window goes below zero for
both circuit and stream level.

Part of #26840

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
8e38791baf sendme: Add helper functions for DATA cell packaging
When we are about to send a DATA cell, we have to decrement the package window
for both the circuit and stream level.

This commit adds helper functions to handle the package window decrement.

Part of #26288

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
2d3c600915 sendme: Add helper functions for DATA cell delivery
When we get a relay DATA cell delivered, we have to decrement the deliver
window on both the circuit and stream level.

This commit adds helper functions to handle the deliver window decrement.

Part of #26840

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
9c42cc1eb2 sendme: Refactor SENDME cell processing
This is a bit of a complicated commit. It moves code but also refactors part
of it. No behavior change, the idea is to split things up so we can better
handle and understand how SENDME cells are processed where ultimately it will
be easier to handle authenticated SENDMEs (prop289) using the intermediate
functions added in this commit.

The entry point for the cell arriving at the edge (Client or Exit), is
connection_edge_process_relay_cell() for which we look if it is a circuit or
stream level SENDME. This commit refactors that part where two new functions
are introduced to process each of the SENDME types.

The sendme_process_circuit_level() has basically two code paths. If we are a
Client (the circuit is origin) or we are an Exit. Depending on which, the
package window is updated accordingly. Then finally, we resume the reading on
every edge streams on the circuit.

The sendme_process_stream_level() applies on the edge connection which will
update the package window if needed and then will try to empty the inbuf if
need be because we can now deliver more cells.

Again, no behavior change but in order to split that code properly into their
own functions and outside the relay.c file, code modification was needed.

Part of #26840.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
ed8593b9e0 sendme: Modernize and cleanup old moved code
Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:57 -04:00
David Goulet
e5806dcea8 sendme: Move code to the new files sendme.{c|h}
Take apart the SENDME cell specific code and put it in sendme.{c|h}. This is
part of prop289 that implements authenticated SENDMEs.

Creating those new files allow for the already huge relay.c to not grow in LOC
and makes it easier to handle and test the SENDME cells in an isolated way.

This commit only moves code. No behavior change.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-04-29 12:17:33 -04:00
Nick Mathewson
806539b40a Use fast check for missing id in node_is_a_configured_bridge()
Fixes bug 30308; bugfix on 0.3.5.1-alpha.
2019-04-26 11:19:46 -04:00
Nick Mathewson
650b94ebc1 Use a linear algorithm to subtract two nodelists.
The nodelist_idx for each node_t serves as a unique identifier for
the node, so we can use a bitarray to hold all the excluded
nodes, and then remove them from the smartlist.

Previously use used smartlist_subtract(sl, excluded), which is
O(len(sl)*len(excluded)).

We can use this function in other places too, but this is the one
that showed up on the profiles of 30291.

Closes ticket 30307.
2019-04-26 11:04:44 -04:00
Nick Mathewson
1d44ac9acd Make nodelist_get_list() return a const pointer. 2019-04-26 10:36:49 -04:00
Nick Mathewson
efeb101b96 Merge remote-tracking branch 'tor-github/pr/889' 2019-04-25 20:25:34 -04:00
Nick Mathewson
36b4fc7437 Merge remote-tracking branch 'tor-github/pr/922' 2019-04-25 20:08:39 -04:00
Nick Mathewson
a5cced2b7a Extract keyword argument checking from argument parsing. 2019-04-25 14:13:03 -04:00
Nick Mathewson
a0299cd240 In control command api, rename "object" to "cmddata"
This makes it match control-spec.txt.
2019-04-25 14:13:03 -04:00
Nick Mathewson
ff9ba7d6c4 expand CMD_FL_WIPE to wipe the parsed arguments too 2019-04-25 14:13:03 -04:00
Nick Mathewson
88d22b898e Simplify handler logic in control_cmd.c
Now that the legacy handlers are gone, we can simplify the
structures and macros here.
2019-04-25 14:13:03 -04:00
Nick Mathewson
ddd33d39c7 Port the authenticate and authchallenge commands to the new parser
These two presented their own challenge, because of their use of
QString, and their distinguished handling of quoted versus
non-quoted values.
2019-04-25 14:13:03 -04:00
Nick Mathewson
8799b4e805 Add rudimentary qstring support to kvline.c 2019-04-25 14:13:03 -04:00
Nick Mathewson
ba05324242 Move and rename decode_escaped_string()
This function decodes something different from the usual c-escaped
format.

It is only used in controller authorization.
2019-04-25 14:13:03 -04:00
Nick Mathewson
0c0b869ba4 Use the new controller command parser for EXTENDCIRCUIT.
This command does not fit perfectly with the others, since its
second argument is optional and may contain equal signs.  Still,
it's probably better to squeeze it into the new metaformat, since
doing so allows us to remove several pieces of the old
command-parsing machinery.
2019-04-25 14:13:03 -04:00
Nick Mathewson
95afdb005c Use new parser logic for SETCONF/RESETCONF code.
Here we get to throw away a LOT of unused code, since most of the
old parsing was redundant with kvline.
2019-04-25 14:13:03 -04:00
Nick Mathewson
d8b3ec865d Update more controller commands, now that we have kvline support 2019-04-25 14:13:03 -04:00
Nick Mathewson
dab35386ca Add a case-insensitive variant to config_line_find() 2019-04-25 14:13:03 -04:00
Nick Mathewson
9471391694 Add kvline support to controller command parser.
This should let us handle all (or nearly all) of the remaining
commands.
2019-04-25 14:13:03 -04:00
Nick Mathewson
0841a69357 Allow kvlines in control commands. 2019-04-25 14:13:03 -04:00
Nick Mathewson
bb37ad6957 Add fuzzing support for several more groups of kvlines flags 2019-04-25 14:13:03 -04:00
Nick Mathewson
73df91bbb5 kvline: handle empty alues as well as empty keys
The two options are mutually exclusive, since otherwise an entry
like "Foo" would be ambiguous.  We want to have the ability to treat
entries like this as keys, though, since some controller commands
interpret them as flags.
2019-04-25 14:13:03 -04:00
Nick Mathewson
01b07c548b Use parsing code for the simpler controller commands.
(This should be all of the command that work nicely with positional
arguments only.)

Some of these commands should probably treat extra arguments as
incorrect, but for now I'm trying to be careful not to break
any existing users.
2019-04-25 14:13:03 -04:00
Nick Mathewson
cbd1a7e053 Unit tests for current control-command parser logic 2019-04-25 14:13:03 -04:00
Nick Mathewson
dbfe1a14e4 When parsing a multiline controller command, be careful with linebreaks
The first line break in particular was mishandled: it was discarded
if no arguments came before it, which made it impossible to
distinguish arguments from the first line of the body.

To solve this, we need to allocate a copy of the command rather than
using NUL to separate it, since we might have "COMMAND\n" as our input.

Fixes ticket 29984.
2019-04-25 14:13:03 -04:00
Nick Mathewson
f18b7dc473 Extract the argument-splitting part of control.c's parser
This is preliminary work for fixing 29984; no behavior has changed.
2019-04-25 14:13:03 -04:00
Nick Mathewson
de70eebc65 Start on a command-parsing tool for controller commands.
There _is_ an underlying logic to these commands, but it isn't
wholly uniform, given years of tweaks and changes.  Fortunately I
think there is a superset that will work.

This commit adds a parser for some of the most basic cases -- the
ones currently handled by getargs_helper() and some of the
object-taking ones.  Soon will come initial tests; then I'll start using
the parser.

After that, I'll expand the parser to handle the other cases that come
up in the controller protocol.
2019-04-25 14:13:03 -04:00
Nick Mathewson
0d650e7958 Move responsibility for checking if events are setup into periodic.c
We have checks in various places in mainlook.c to make sure that
events are initialized before we invoke any periodic_foo() functions
on them.  But now that each subsystem will own its own periodic
events, it will be cleaner if we don't assume that they are all
setup or not.
2019-04-25 13:23:18 -04:00
Nick Mathewson
233835e14f Move the responsibility for listing periodic events to periodic.c
The end goal here is to move the periodic callback to their
respective modules, so that mainloop.c doesn't have to include so
many other things.

This patch doesn't actually move any of the callbacks out of
mainloop.c yet.
2019-04-25 10:09:36 -04:00
George Kadianakis
974c2674eb Merge branch 'maint-0.4.0' 2019-04-25 15:47:07 +03:00
George Kadianakis
a39789a02c Merge branch 'tor-github/pr/960' into maint-0.4.0 2019-04-25 15:46:45 +03:00
George Kadianakis
811a93f803 Merge branch 'tor-github/pr/953' 2019-04-25 15:43:47 +03:00
Alexander Færøy
0429072495 Lower log level of unlink() errors in networkstatus_set_current_consensus().
In this patch we lower the log level of the failures for the three calls
to unlink() in networkstatus_set_current_consensus(). These errors might
trigger on Windows because the memory mapped consensus file keeps the
file in open state even after we have close()'d it. Windows will then
error on the unlink() call with a "Permission denied" error.

The consequences of ignoring these errors is that we leave an unused
file around on the file-system, which is an easier way to fix this
problem right now than refactoring networkstatus_set_current_consensus().

See: https://bugs.torproject.org/29930
2019-04-25 01:59:37 +02:00
Nick Mathewson
6320b2988c Merge remote-tracking branch 'tor-github/pr/942' 2019-04-24 17:06:56 -04:00
David Goulet
f39dd0a700 Merge branch 'tor-github/pr/951' 2019-04-24 09:53:15 -04:00
David Goulet
608cf2ba8d Merge branch 'tor-github/pr/943' 2019-04-24 09:51:14 -04:00
David Goulet
5bcf87c224 Merge branch 'tor-github/pr/955' 2019-04-24 09:45:32 -04:00
David Goulet
4b599aaae4 Merge branch 'tor-github/pr/939' 2019-04-24 09:43:28 -04:00
teor
f35bd36814
test-network-all: Test IPv6-only v3 single onion services
In "make test-network-all", test IPv6-only v3 single onion services,
using the chutney network single-onion-v23-ipv6-md. This test will
not pass until 23588 has been merged.

Closes ticket 27251.
2019-04-24 17:42:59 +10:00
teor
3d89f0374a
hs_config: Allow Tor to be configured as an IPv6-only v3 single onion service
Part of #23588.
2019-04-24 17:29:18 +10:00
teor
f05e3f3c9a
test/hs: Correctly convert an IPv6 intro point to an extend_info
Part of #23588.
2019-04-24 17:29:14 +10:00
teor
abe086dd97
test/hs: Re-enable the IPv6 intro point in the HS tests
Tests 23588, partially reverts 23576.

Implements 29237.
2019-04-24 17:29:10 +10:00
Neel Chauhan
b65f8c419a
Add firewall_choose_address_ls() and hs_get_extend_info_from_lspecs() tests 2019-04-24 17:28:38 +10:00
Neel Chauhan
2618347657
Use fascist_firewall_choose_address_ls() in hs_get_extend_info_from_lspecs() 2019-04-24 17:28:34 +10:00
Neel Chauhan
da268e3b50
Add function fascist_firewall_choose_address_ls() 2019-04-24 17:28:30 +10:00
Nick Mathewson
15d4238383 Merge remote-tracking branch 'tor-github/pr/944' 2019-04-23 15:39:23 -04:00
Nick Mathewson
8bea0c2fa3 Rename outvar to follow _out convention. 2019-04-23 14:14:17 -04:00
Nick Mathewson
475ac11bc1 Merge remote-tracking branch 'tor-github/pr/935' 2019-04-23 14:11:04 -04:00
Nick Mathewson
a7599c5be2 Merge remote-tracking branch 'tor-github/pr/962' 2019-04-23 12:48:37 -04:00
Neel Chauhan
2ab19a48c2 Initialize rate_limited in hs_pick_hsdir() to false 2019-04-19 09:50:54 -04:00
Neel Chauhan
efde686aa5 Only set rate_limited in hs_pick_hsdir() if rate_limited_count or responsible_dirs_count is greater than 0 2019-04-19 09:21:20 -04:00
Neel Chauhan
144bc5026e Initialize rate_limited to false in directory_get_from_hs_dir() 2019-04-19 09:17:29 -04:00
Neel Chauhan
943559b180 Make rate_limited and is_rate_limited a bool 2019-04-19 08:33:00 -04:00
teor
231036a110
Merge branch 'maint-0.3.4' into maint-0.3.5 2019-04-19 12:00:41 +10:00
teor
742b5b32d5
Merge remote-tracking branch 'tor-github/pr/710' into maint-0.3.5 2019-04-19 11:52:48 +10:00
teor
cb084de5e5
Merge remote-tracking branch 'tor-github/pr/726' into maint-0.3.5 2019-04-19 11:51:05 +10:00
teor
2ae67fee42
Merge remote-tracking branch 'tor-github/pr/745' into maint-0.3.5 2019-04-19 11:48:41 +10:00
teor
116970dda7
Merge remote-tracking branch 'tor-github/pr/946' into maint-0.3.4 2019-04-19 11:47:10 +10:00
teor
b1762f8d12
Merge remote-tracking branch 'tor-github/pr/638' into maint-0.3.4 2019-04-19 11:44:38 +10:00
teor
8154b33f9c
Merge remote-tracking branch 'tor-github/pr/791' into maint-0.3.4 2019-04-19 11:43:46 +10:00
teor
2460b4461f
Merge remote-tracking branch 'tor-github/pr/792' into maint-0.2.9 2019-04-19 11:42:09 +10:00
teor
ec213ae8a0
Merge remote-tracking branch 'tor-github/pr/772' into maint-0.2.9 2019-04-19 11:38:52 +10:00
George Kadianakis
78223ab0fc Merge branch 'tor-github/pr/938' 2019-04-18 13:23:32 +03:00
George Kadianakis
d867b7ae1d Merge branch 'maint-0.4.0' 2019-04-18 13:22:23 +03:00
George Kadianakis
6a179b1072 Merge branch 'tor-github/pr/891' into maint-0.4.0 2019-04-18 13:21:59 +03:00
teor
c483ccf1c9
connection_edge: remove an extra ; 2019-04-17 18:43:20 +10:00
teor
728d20ed08
connection_edge: Return a web page when HTTPTunnelPort is misconfigured
Return an informative web page when the HTTPTunnelPort is used as an
HTTP proxy.

Closes ticket 27821, patch by "eighthave".
2019-04-17 17:58:40 +10:00
Mike Perry
17a164a827 Bug 30173: Rate limit padding rate limit log message.
Gotta limit to the limit when the limit is reached.
2019-04-17 02:51:48 +00:00
Mike Perry
21a4438c58 Bug 30173: Add consensus param to disable padding.
Disable padding via limit check and machine condition. Limits cause us to stop
sending padding. Machine conditions cause the machines to be shut down, and
not restarted.
2019-04-17 02:44:42 +00:00
teor
031ed59dba
test/relay: add a missing typedef
In 0.3.4 and later, these functions are declared in rephist.h:
STATIC uint64_t find_largest_max(bw_array_t *b);
STATIC void commit_max(bw_array_t *b);
STATIC void advance_obs(bw_array_t *b);

But in 0.2.9, they are declared in rephist.c and test_relay.c.

So compilers fail with a "must use 'struct' tag" error.

We add the missing struct typedef in test_relay.c, to match the
declarations in rephist.c.

(Merge commit 813019cc57 moves these functions into rephist.h instead.)

Fixes bug 30184; not in any released version of Tor.
2019-04-17 11:14:05 +10:00
Bernhard M. Wiedemann
8c4e68438d Do not warn about compatible OpenSSL upgrades
When releasing OpenSSL patch-level maintenance updates,
we do not want to rebuild binaries using it.
And since they guarantee ABI stability, we do not have to.

Without this patch, warning messages were produced
that confused users:
https://bugzilla.opensuse.org/show_bug.cgi?id=1129411

Fixes bug 30190; bugfix on 0.2.4.2-alpha commit 7607ad2bec

Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
2019-04-17 01:55:04 +02:00
Neel Chauhan
ada6732914 Clarify comment about IPv6Exit in policies_parse_exit_policy_from_options() 2019-04-16 08:22:17 -04:00
Neel Chauhan
06c76e79aa Clarify torrc comment for IPv6Exit 2019-04-16 08:20:48 -04:00
Neel Chauhan
e7c22e6e48 Add policy_using_default_exit_options() to determine if we're using the default exit options 2019-04-16 08:19:44 -04:00
Nick Mathewson
7b2357a37a Merge remote-tracking branch 'tor-github/pr/884' 2019-04-16 08:08:06 -04:00
teor
05d25d06b6
rephist: fix an undeclared type compilation error
In 0.3.4 and later, we declare write_array as:
extern struct bw_array_t *write_array;
...
typedef struct bw_array_t bw_array_t;

But in 0.2.9, we declare write_array as:
typedef struct bw_array_t bw_array_t;
extern bw_array_t *write_array;

And then again in rephist.c:
typedef struct bw_array_t bw_array_t;

So some compilers fail with a duplicate declaration error.

We backport 684b396ce5, which removes the duplicate declaration.
And this commit deals with the undeclared type error.

Backports a single line from merge commit 813019cc57.

Fixes bug 30184; not in any released version of Tor.
2019-04-16 15:39:45 +10:00
Nick Mathewson
684b396ce5
Remove another needless typedef 2019-04-16 15:33:14 +10:00
Nick Mathewson
950d890f77 In warn_if_nul_found, log surrounding context.
We need to encode here instead of doing escaped(), since fwict
escaped() does not currently handle NUL bytes.

Also, use warn_if_nul_found in more cases to avoid duplication.
2019-04-15 15:33:09 -04:00
Nick Mathewson
0c42ddf28c fixup! Even more diagnostic messages for bug 28223.
Use TOR_PRIuSZ in place of %zu.
2019-04-15 15:21:18 -04:00
Nick Mathewson
82a3161c41 Document check for 30176, since it's a bit subtle. 2019-04-15 14:52:08 -04:00
Tobias Stoeckmann
670d0f9f5b Clear memory in smartlist_remove_keeporder.
The smartlist functions take great care to reset unused pointers inside
the smartlist memory to NULL.

The function smartlist_remove_keeporder does not clear memory in such
way when elements have been removed. Therefore call memset after the
for-loop that removes elements. If no element is removed, it is
effectively a no-op.

Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
2019-04-15 14:51:36 -04:00
Tobias Stoeckmann
5a0c857996 Add test to verify that unused pointers are NULL.
The smartlist code takes great care to set all unused pointers inside
the smartlist memory to NULL. Check if this is also the case after
modifying the smartlist multiple times.

Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
2019-04-15 14:46:26 -04:00
Nick Mathewson
3105081c2f Fix assertf() issues when ALL_BUGS_ARE_FATAL is defined.
Fix from Gisle Vanem; fixes bug 30179.  Bug not in any released
version of Tor.
2019-04-15 14:37:46 -04:00
Nick Mathewson
358df18064 Merge branch 'bug30189_035' into bug30189_041 2019-04-15 14:27:32 -04:00
Nick Mathewson
802ac8ad61 Use a tor_abort_() wrapper in our util_bug.h macros
Previously, our use of abort() would break anywhere that we didn't
include stdlib.h.  This was especially troublesome in case where
tor_assert_nonfatal() was used with ALL_BUGS_ARE_FATAL, since that
one seldom gets tested.

As an alternative, we could have just made this header include
stdlib.h.  But that seems bloaty.

Fixes bug 30189; bugfix on 0.3.4.1-alpha.
2019-04-15 14:16:23 -04:00
George Kadianakis
3f9efc5e88 Merge branch 'maint-0.4.0' 2019-04-15 13:56:48 +03:00
George Kadianakis
15591e1bbd Merge branch 'tor-github/pr/948' into maint-0.4.0 2019-04-15 13:56:37 +03:00
rl1987
55b4f02ba6 Fix shellcheck warnings in fixup_filenames.sh 2019-04-15 12:13:35 +03:00
teor
1177818c32
Merge branch 'rust-panic1-035' into rust-panic1-040 2019-04-15 10:18:59 +10:00
teor
a6d0420f7c
Merge branch 'rust-panic1-034' into rust-panic1-035
Trivial merge: a blank line was removed between 0.3.4 and 0.3.5.
2019-04-15 10:18:01 +10:00
teor
5ab5c8ec15
Merge branch 'rust-panic1' into rust-panic1-034 2019-04-15 10:15:02 +10:00
Neel Chauhan
cc87acf29b Remove unused get_options() 2019-04-14 14:51:42 -04:00
Neel Chauhan
03464a9165 Update torrc.sample.in to IPv6Exit 1 being an exit by default 2019-04-14 12:48:07 -04:00
Neel Chauhan
5cad9fb477 Become an exit relay if IPv6Exit is 1 2019-04-14 12:48:07 -04:00
Nick Mathewson
88dc7bc171 Add an assertion to test_hs_cache.c to appease coverity.
Coverity doesn't like to see a path where we test a pointer for
NULL if we have already ready dereferenced the pointer on that
path.  While in this case, the check is not needed, it's best not to
remove checks from the unit tests IMO.  Instead, I'm adding an
earlier check, so that coverity, when analyzing this function, will
think that we have always checked the pointer before dereferencing
it.

Closes ticket 30180; CID 1444641.
2019-04-14 11:31:50 -04:00
rl1987
4fa4fe0945 Fix remaining shellcheck warnings in fuzz_multi.sh 2019-04-12 22:41:39 +03:00
rl1987
1ee991ed4b Add shebang line to fuzz_multi.sh (fixes SC2148) 2019-04-12 22:38:06 +03:00
Nick Mathewson
e9ca904dbf Define two more commands as wipe-after-parse. 2019-04-12 08:33:27 -04:00
Nick Mathewson
f3bd0240a6 Add assertions for correct input to handle_control_command. 2019-04-12 08:33:27 -04:00
Nick Mathewson
d1f5957c4e Improve handling of controller commands
Use a table-based lookup to find the right command handler.  This
will serve as the basement for several future improvements, as we
improve the API for parsing commands.
2019-04-12 08:33:27 -04:00
George Kadianakis
7b386f2356 Merge branch 'tor-github/pr/908' 2019-04-12 13:47:08 +03:00
George Kadianakis
86aa141572 Merge branch 'tor-github/pr/754' 2019-04-12 13:45:53 +03:00
teor
ebbc2c3d8f
crypt_ops: Stop using a separate buffer in ed25519_signature_from_base64()
Part of 29960.
2019-04-12 13:00:02 +10:00
Neel Chauhan
398c736230 Remove unused variable in fmt_serverstatus.c 2019-04-11 22:11:27 -04:00
Neel Chauhan
994b8ba424 Update networkstatus_getinfo_by_purpose() comment 2019-04-11 21:36:38 -04:00
Neel Chauhan
14d7008045 Stop setting routers as running in list_server_status_v1() 2019-04-11 21:30:48 -04:00
Neel Chauhan
c07d854772 Remove callback for setting bridges as running 2019-04-11 21:28:35 -04:00
Neel Chauhan
4172dcaa62 Move code for setting bridges as running to voteflags.c 2019-04-11 20:44:30 -04:00
Neel Chauhan
30279a7c57 Use authdir_mode_bridge() in set_bridge_running_callback() 2019-04-11 20:28:11 -04:00
Neel Chauhan
aa9940ed21 Make SET_BRIDGES_RUNNING_INTERVAL 5 minutes 2019-04-11 20:24:08 -04:00
Nick Mathewson
cdafcc49bc Fix a memory leak in tor-resolve.c
Closes bug 30151/coverity CID 1441830. Bugfix on 0.4.0.1-alpha when
we started doing trunnel parsing in tor-resolve.c.
2019-04-11 19:10:05 -04:00
Nick Mathewson
48a574604b Remove an extraneous _ from __COVERITY__
We had a typo in this check, so that coverity wasn't taking the
right path.

Bug not in any released Tor.
2019-04-11 18:56:02 -04:00
Nick Mathewson
66b07e7ec1 Add an assertion to num_ntors_per_tap().
This should please coverity, and fix CID 1415721.  It didn't
understand that networkstatus_get_param() always returns a value
between its minimum and maximum values.
2019-04-11 18:44:10 -04:00
Nick Mathewson
96e310911f Add an assertion to compute_weighted_bandwidths()
This should please coverity, and fix CID 1415722.  It didn't
understand that networkstatus_get_param() always returns a value
between its minimum and maximum values.
2019-04-11 18:41:38 -04:00
Nick Mathewson
55690d05bd Add an assertion to pathbias_get_scale_ratio()
This should please coverity, and fix CID 1415723.  It didn't understand
that networkstatus_get_param() always returns a value between its
minimum and maximum values.
2019-04-11 18:38:59 -04:00
Nick Mathewson
85ff6f9114 Fix a memory leak on failure to create keys directory.
Fixes bug 30148, which is also CID 1437429 and CID 1437454. Bugfix
on 0.3.3.1-alpha, when separate key directories became a thing.
2019-04-11 18:18:14 -04:00
Nick Mathewson
781d69f3a7 Make it clear to coverity we aren't leaking in protover_all_supported()
The logic here should be "use versions or free it".  The "free it"
part was previously in a kind of obfuscated place, so coverity
wasn't sure it was invoked as appropriate.  CID 1437436.
2019-04-11 17:51:11 -04:00
Nick Mathewson
4e3d144fb0 Don't leak on logic error in string_is_valid_nonrfc_hostname()
This is CID 1437438.  No backport needed: this is unreachable, and
guarded with a BUG() check.
2019-04-11 17:40:05 -04:00
Nick Mathewson
7c98105d56 On failure to create extend info for an introduction point, don't leak.
This is CID 1438152.  No backport needed: this path is already
inside a BUG() guard.
2019-04-11 17:35:19 -04:00
Nick Mathewson
40471d73e5 bump to 0.4.0.4-rc-dev 2019-04-11 17:05:38 -04:00
Neel Chauhan
011307dd5f Make repeated/rate limited HSFETCH queries fail with QUERY_RATE_LIMITED 2019-04-11 15:21:17 -04:00
Nick Mathewson
9fabf104ed Merge remote-tracking branch 'tor-github/pr/913' 2019-04-11 14:30:05 -04:00
Nick Mathewson
d549440124 Merge remote-tracking branch 'tor-github/pr/887' 2019-04-11 14:29:16 -04:00
Nick Mathewson
f8dc935fb7 Merge remote-tracking branch 'tor-github/pr/741' 2019-04-11 14:27:06 -04:00
Nick Mathewson
b2fc57426c Bump version to 0.4.0.4-rc 2019-04-10 11:51:49 -04:00
Nick Mathewson
3be1e26b8d Merge branch 'maint-0.4.0' 2019-04-10 11:31:44 -04:00
Nick Mathewson
412bcc5b2a Merge remote-tracking branch 'tor-github/pr/926' into maint-0.4.0 2019-04-10 11:31:33 -04:00
Tobias Stoeckmann
9ce0bdd226 Prevent double free on huge files with 32 bit.
The function compat_getdelim_ is used for tor_getline if tor is compiled
on a system that lacks getline and getdelim. These systems should be
very rare, considering that getdelim is POSIX.

If this system is further a 32 bit architecture, it is possible to
trigger a double free with huge files.

If bufsiz has been already increased to 2 GB, the next chunk would
be 4 GB in size, which wraps around to 0 due to 32 bit limitations.

A realloc(*buf, 0) could be imagined as "free(*buf); return malloc(0);"
which therefore could return NULL. The code in question considers
that an error, but will keep the value of *buf pointing to already
freed memory.

The caller of tor_getline() would free the pointer again, therefore
leading to a double free.

This code can only be triggered in dirserv_read_measured_bandwidths
with a huge measured bandwith list file on a system that actually
allows to reach 2 GB of space through realloc.

It is not possible to trigger this on Linux with glibc or other major
*BSD systems even on unit tests, because these systems cannot reach
so much memory due to memory fragmentation.

This patch is effectively based on the penetration test report of
cure53 for curl available at https://cure53.de/pentest-report_curl.pdf
and explained under section "CRL-01-007 Double-free in aprintf() via
unsafe size_t multiplication (Medium)".
2019-04-10 12:46:27 +03:00
teor
12b9bfc05f
test: Also avoid reading the system default torrc in integration tests
Part of 29702.
2019-04-10 19:03:43 +10:00
rl1987
93dcfc6593
Use empty torrc file when launching tor in test_rebind.py 2019-04-10 18:45:21 +10:00
teor
a1d9f44971
Merge branch 'maint-0.4.0' 2019-04-10 18:27:11 +10:00
teor
454bdb22ee
Merge remote-tracking branch 'tor-github/pr/920' into maint-0.4.0 2019-04-10 18:26:49 +10:00
teor
ce9b101574
bwauth: update measured bandwidth file comments
We forgot to update function header comments and code comments when we
made changes in 0.3.5.1-alpha and later.

Closes 30112.
2019-04-10 15:57:54 +10:00
Nick Mathewson
c28cdcc9bf Merge branch 'maint-0.4.0' 2019-04-09 13:51:44 -04:00
Nick Mathewson
ffdca3dd14 Merge branch 'bug29922_035' into maint-0.4.0 2019-04-09 13:49:58 -04:00
Nick Mathewson
9f3f99938e Actually I believe this should be an EINVAL. 2019-04-09 13:49:10 -04:00
Tobias Stoeckmann
a628e36024 Check return value of buf_move_to_buf for error.
If the concatenation of connection buffer and the buffer of linked
connection exceeds INT_MAX bytes, then buf_move_to_buf returns -1 as an
error value.

This value is currently casted to size_t (variable n_read) and will
erroneously lead to an increasement of variable "max_to_read".

This in turn can be used to call connection_buf_read_from_socket to
store more data inside the buffer than expected and clogging the
connection buffer.

If the linked connection buffer was able to overflow INT_MAX, the call
of buf_move_to_buf would have previously internally triggered an integer
overflow, corrupting the state of the connection buffer.

Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
2019-04-09 12:05:22 -04:00
Tobias Stoeckmann
5a6ab3e7db Protect buffers against INT_MAX datalen overflows.
Many buffer functions have a hard limit of INT_MAX for datalen, but
this limitation is not enforced in all functions:

- buf_move_all may exceed that limit with too many chunks
- buf_move_to_buf exceeds that limit with invalid buf_flushlen argument
- buf_new_with_data may exceed that limit (unit tests only)

This patch adds some annotations in some buf_pos_t functions to
guarantee that no out of boundary access could occur even if another
function lacks safe guards against datalen overflows.

Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
2019-04-09 12:05:14 -04:00
Tobias Stoeckmann
0fa95308fe Check return value of buf_move_to_buf for error.
If the concatenation of connection buffer and the buffer of linked
connection exceeds INT_MAX bytes, then buf_move_to_buf returns -1 as an
error value.

This value is currently casted to size_t (variable n_read) and will
erroneously lead to an increasement of variable "max_to_read".

This in turn can be used to call connection_buf_read_from_socket to
store more data inside the buffer than expected and clogging the
connection buffer.

If the linked connection buffer was able to overflow INT_MAX, the call
of buf_move_to_buf would have previously internally triggered an integer
overflow, corrupting the state of the connection buffer.

Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
2019-04-09 12:00:14 -04:00
Tobias Stoeckmann
74b2bc43fb Protect buffers against INT_MAX datalen overflows.
Many buffer functions have a hard limit of INT_MAX for datalen, but
this limitation is not enforced in all functions:

- buf_move_all may exceed that limit with too many chunks
- buf_move_to_buf exceeds that limit with invalid buf_flushlen argument
- buf_new_with_data may exceed that limit (unit tests only)

This patch adds some annotations in some buf_pos_t functions to
guarantee that no out of boundary access could occur even if another
function lacks safe guards against datalen overflows.

  [This is a backport of the submitted patch to 0.2.9, where the
  buf_move_to_buf and buf_new_with_data functions did not exist.]
2019-04-09 11:59:20 -04:00
George Kadianakis
16041d7918 Merge branch 'maint-0.4.0' 2019-04-09 12:25:18 +03:00
George Kadianakis
c7cf49dc3c Merge branch 'tor-github/pr/915' into maint-0.4.0 2019-04-09 12:24:52 +03:00
teor
3d45079c27
Merge branch 'maint-0.4.0' 2019-04-09 11:36:59 +10:00
teor
e4692fad96
Merge remote-tracking branch 'tor-github/pr/866' 2019-04-09 11:36:38 +10:00
teor
92e8bdf296
Merge remote-tracking branch 'tor-github/pr/892' into maint-0.4.0 2019-04-09 11:35:41 +10:00
Nick Mathewson
a63bd87760 Detect and suppress an additional gmtime() warning in test_util.c
Fixes bug 29922; bugfix on 0.2.9.3-alpha when we tried to capture
all these warnings.  No need to backport any farther than 0.3.5,
though -- these warnings don't cause test failures before then.

This one was tricky to find because apparently it only happened on
_some_ windows builds.
2019-04-08 17:02:14 -04:00
rl1987
0e0a0b9802 Fix SC2006 in minimize.sh 2019-04-08 11:16:45 +03:00
teor
4cffc7fe9c
Merge branch 'maint-0.4.0' 2019-04-06 12:23:25 +10:00
teor
b100c9e980
Merge remote-tracking branch 'tor-github/pr/911' into maint-0.4.0 2019-04-06 12:15:41 +10:00
Nick Mathewson
5cb94cbf9d
NSS: disable TLS1.2 SHA-384 ciphersuites.
In current NSS versions, these ciphersuites don't work with
SSL_ExportKeyingMaterial(), which was causing relays to fail when
they tried to negotiate the v3 link protocol authentication.

Fixes bug 29241; bugfix on 0.4.0.1-alpha.
2019-04-06 11:06:34 +10:00
Nick Mathewson
680fd3f8fb
NSS: Log an error message when SSL_ExportKeyingMaterial() fails
Diagnostic for 29241.
2019-04-06 11:06:22 +10:00
teor
48e990359b
Merge branch 'maint-0.2.9' into maint-0.3.4 2019-04-06 09:33:11 +10:00
George Kadianakis
df4e2c42a8 Merge branch 'tor-github/pr/907' 2019-04-05 16:45:55 +03:00
George Kadianakis
071a000d67 Merge branch 'tor-github/pr/906' into maint-0.4.0 2019-04-05 16:44:59 +03:00
George Kadianakis
3393c8d510 Merge branch 'maint-0.4.0' 2019-04-05 14:53:39 +03:00
George Kadianakis
217db9efe1 Merge branch 'tor-github/pr/902' into maint-0.4.0 2019-04-05 14:53:33 +03:00
George Kadianakis
b371ea5b0e Merge branch 'tor-github/pr/761' 2019-04-05 14:52:36 +03:00
George Kadianakis
574c207670 Merge branch 'maint-0.4.0' 2019-04-05 14:51:33 +03:00
George Kadianakis
747b74c182 Merge branch 'tor-github/pr/800' into maint-0.4.0 2019-04-05 14:51:21 +03:00
teor
c616f45776 binascii: Fix the base64_encode_nopad() buffer length requirement
Comment-only change.

Part of 29660.
2019-04-05 15:17:19 +10:00
teor
5e2cba8eb4 crypto_format: Stop adding padding in ed25519_signature_from_base64()
base64_decode() does not require padding.

Part of 29660.
2019-04-05 15:17:19 +10:00
teor
ce5e38642d crypto_format: Remove the return value from ed25519_signature_to_base64()
Also remove all checks for the return value, which were redundant anyway,
because the function never failed.

Part of 29660.
2019-04-05 15:17:19 +10:00
teor
e3124fef54 crypto_format: Remove the return value from curve25519_public_to_base64()
And fix the documentation on the function: it does produce trailing
"="s as padding.

Also remove all checks for the return value, which were redundant anyway,
because the function never failed.

Part of 29660.
2019-04-05 15:17:19 +10:00
teor
7d513a5d55 crypto_format: Remove the return values from digest256_to_base64()
... and ed25519_public_to_base64(). Also remove all checks for the return
values, which were redundant anyway, because the functions never failed.

Part of 29960.
2019-04-05 15:17:19 +10:00
teor
abaed046a6 crypto_format: Remove unused return value from digest_to_base64()
Part of 29660.
2019-04-05 15:17:19 +10:00
teor
0d136a12bb crypto_format: Remove outdated comments
(These functions look pretty unified to me.)

Part of 29660.
2019-04-05 15:17:08 +10:00
teor
e992c5e4bc
Merge branch 'bug29500_040_monoinit_revert' into bug29500_master_monoinit
Apply data structure changes between 0.4.0 and 0.4.1.
2019-04-05 12:24:20 +10:00
teor
593b33608d
Revert "test/circuitpadding: Delete circuitpadding_circuitsetup_machine()"
This reverts commit 387d9448de.
2019-04-05 12:22:18 +10:00
teor
da678213e0
circuitpadding: comment fixes 2019-04-05 12:17:21 +10:00
teor
387d9448de
test/circuitpadding: Delete circuitpadding_circuitsetup_machine()
This test was disabled in 0.4.0 and later, but the fix in #29298 was only
merged to 0.4.1. So this test will never be re-enabled in 0.4.0.

Part of 29500.
2019-04-05 12:17:10 +10:00
Mike Perry
b733044f7a
Bug #29500: Fix monotime mocking in circpad unittests.
Our monotime mocking forces us to call monotime_init() *before* we set the
mocked time value. monotime_init() thus stores the first ratchet value at
whatever the platform is at, and then we set fake mocked time to some later
value.

If monotime_init() gets a value from the host that is greater than what we
choose to mock time at for our unittests, all subsequent monotime_abosolute()
calls return zero, which breaks all unittests that depend on time moving
forward by updating mocked monotime values.

So, we need to adjust our mocked time to take the weird monotime_init() time
into account, when we set fake time.
2019-04-05 12:15:55 +10:00
Mike Perry
b027b06dbb
Bug 29500: Start monotime at 1000 nsec.
Hopefully this will stop monotime_absolute_usec() from returning 0 on some
platforms in the tests.
2019-04-05 12:14:11 +10:00
Mike Perry
1f48c6cd83
Bug 29500: Attempt to fix the tokens test.
Cancel the padding timer by changing order of sent vs recv (sent cancels).
2019-04-05 12:13:19 +10:00
Nick Mathewson
1779878f9e Merge remote-tracking branch 'tor-github/pr/752' 2019-04-04 20:39:36 -04:00
Nick Mathewson
0e7b34354a Merge branch 'maint-0.4.0' 2019-04-04 20:27:04 -04:00
Nick Mathewson
d016bbaa7d Merge branch 'bug29959_040_squashed' into maint-0.4.0 2019-04-04 20:26:47 -04:00
teor
8e961b2174 bwauth: Actually include the bandwidth-file-digest in authority votes
Fixes bug 29959; bugfix on 0.4.0.2-alpha.
2019-04-04 20:26:09 -04:00
Nick Mathewson
76912bf140 Use an enum for inherit_result_out. 2019-04-04 12:56:52 -04:00
Nick Mathewson
8c06f02c94 Syntax fix in test. 2019-04-04 12:56:52 -04:00
Nick Mathewson
027c536598 rename inherit values to avoid conflict with system defines 2019-04-04 12:56:52 -04:00
Nick Mathewson
785c3f84de fast_rng: if noinherit has failed, then check getpid() for bad forks
getpid() can be really expensive sometimes, and it can fail to
detect some kind of fork+prng mistakes, so we need to avoid it if
it's safe to do so.

This patch might slow down fast_prng a lot on any old operating
system that lacks a way to prevent ram from being inherited, AND
requires a syscall for any getpid() calls.  But it should make sure
that we either crash or continue safely on incorrect fork+prng usage
elsewhere in the future.
2019-04-04 12:56:52 -04:00
Nick Mathewson
361e955cf3 map_anon: define a macro if it is possible for noinherit to fail. 2019-04-04 12:56:52 -04:00
Nick Mathewson
12205c3cbe Make map_anon expose the result of a noinherit attempt
Previously we did this for tests only, but it's valuable for getting
proper fork behavior in rand_fast.
2019-04-04 12:56:52 -04:00
Nick Mathewson
ab6ad3c040 Drop thread-local fast_rng on fork.
This will cause the child process to construct a new one in a nice
safe way.

Closes ticket 29668; bug not in any released Tor.
2019-04-04 12:56:52 -04:00
Nick Mathewson
d194f6bedf Implement an DormantCanceledByStartup option
Closes ticket 29357, and comes with appropriate notions of caution.
2019-04-04 11:48:36 -04:00
Nick Mathewson
db1c1dba34 Merge branch 'bug30021_029' into bug30021_035 2019-04-04 11:26:33 -04:00
Nick Mathewson
1710f4bbd6 Do not cache bogus results from classifying client ciphers
When classifying a client's selection of TLS ciphers, if the client
ciphers are not yet available, do not cache the result. Previously,
we had cached the unavailability of the cipher list and never looked
again, which in turn led us to assume that the client only supported
the ancient V1 link protocol.  This, in turn, was causing Stem
integration tests to stall in some cases.  Fixes bug 30021; bugfix
on 0.2.4.8-alpha.
2019-04-04 11:24:55 -04:00
Neel Chauhan
d4d77b277e Stop setting bridges running in networkstatus_getinfo_by_purpose() 2019-04-03 15:27:33 -04:00
Nick Mathewson
5613968d57 Improve logging for 28614.
When we fixed 28614, our answer was "if we failed to load the
consensus on windows and it had a CRLF, retry it."  But we logged
the failure at "warn", and we only logged the retry at "info".

Now we log the retry at "notice", with more useful information.

Fixes bug 30004.
2019-04-03 14:30:56 -04:00
Nick Mathewson
821d29e420 fdio.c: add more includes.
This is just in case there is some rogue platform that uses a
nonstandard value for SEEK_*, and does not define that macro in
unistd.h.  I think that's unlikely, but it's conceivable.
2019-04-03 13:53:36 -04:00
Nick Mathewson
99b87d7ca4 Even more diagnostic messages for bug 28223.
Try to figure out _where exactly_ we are first encountering NULs in
microdescriptors, and what we are doing when that happens.
2019-04-03 13:53:06 -04:00
rl1987
4172b638b8 Fix SC2015 warning 2019-04-03 18:03:34 +03:00
George Kadianakis
0b6769a99e Merge branch 'maint-0.4.0' 2019-04-03 17:59:46 +03:00
George Kadianakis
42aae0e693 Merge branch 'tor-github/pr/867' into maint-0.4.0 2019-04-03 17:59:02 +03:00
rl1987
9e04a87220 Fix SC2064 warning 2019-04-03 17:58:05 +03:00
rl1987
700310df61 Fix SC2006 warnings 2019-04-03 17:56:52 +03:00
rl1987
9e0f0a5656 Fix SC2086 warnings in test_key_expiration.sh 2019-04-03 17:52:31 +03:00
Nick Mathewson
367dd9cf02 30001: Fix a race condition in test_dir_handle_get.c
Previously we used time(NULL) to set the Expires: header in our HTTP
responses.  This made the actual contents of that header untestable,
since the unit tests have no good way to override time(), or to see
what time() was at the exact moment of the call to time() in
dircache.c.

This gave us a race in dir_handle_get/status_vote_next_bandwidth,
where the time() call in dircache.c got one value, and the call in
the tests got another value.

I'm applying our regular solution here: using approx_time() so that
the value stays the same between the code and the test.  Since
approx_time() is updated on every event callback, we shouldn't be
losing any accuracy here.

Fixes bug 30001. Bug introduced in fb4a40c32c4a7e5; not in any
released Tor.
2019-04-03 10:16:18 -04:00
Nick Mathewson
4efbb5d2c1 Merge branch 'maint-0.4.0' 2019-04-03 09:27:44 -04:00
Nick Mathewson
de76862fd6 Merge branch 'maint-0.3.5' into maint-0.4.0 2019-04-03 09:27:44 -04:00
Nick Mathewson
4aa02d3c7a Merge branch 'maint-0.3.4' into maint-0.3.5 2019-04-03 09:27:43 -04:00
Nick Mathewson
3cfcfbac46 Merge branch 'maint-0.2.9' into maint-0.3.4 2019-04-03 09:27:42 -04:00
Nick Mathewson
ee6f54ff3f Merge remote-tracking branch 'tor-github/pr/860' 2019-04-03 08:33:40 -04:00
Karsten Loesing
54e249e269 Update geoip and geoip6 to the April 2 2019 database. 2019-04-03 09:26:28 +02:00
teor
fdee4dd501
Merge remote-tracking branch 'tor-github/pr/863' 2019-04-03 13:12:17 +10:00
rl1987
a549e4f7a3 Remove no-longer needed logging statements 2019-04-02 12:59:37 +03:00
rl1987
eaf071d7da Stop requiring bash in test-network.sh. Make it POSIX compliant 2019-04-02 12:31:57 +03:00
teor
965f0d8912
Merge remote-tracking branch 'tor-github/pr/875' 2019-04-02 11:47:05 +10:00
teor
583ed7c6ec
Merge remote-tracking branch 'tor-github/pr/864' 2019-04-02 11:46:42 +10:00
Nick Mathewson
809a3a748d bug_occurred: a place where we assumed that "buf" was still a buffer
In 9c132a5f9e we replaced "buf" with a pointer and replaced
one instance of snprintf with asprintf -- but there was still one
snprintf left over, being crashy.

Fixes bug 29967; bug not in any released Tor. This is CID 1444262.
2019-03-30 21:07:15 -04:00
Nick Mathewson
9ed02ec282 Merge branch 'ticket29662_squashed' into ticket29662_squashed_merged 2019-03-29 14:28:48 -04:00
rl1987
537ad0bca3 Check for NULL in tor_assertf_nonfatal() 2019-03-29 14:26:13 -04:00
rl1987
9c132a5f9e Refrain from using static buffer for assert failure message; call tor_asprintf() instead 2019-03-29 14:26:03 -04:00
rl1987
a959d7cb98 Use tor_assertf{_nonfatal} in code 2019-03-29 14:25:59 -04:00
rl1987
f66a17444e Silence compiler warnings 2019-03-29 14:25:59 -04:00
teor
194b25f0c7
dircache: Refactor handle_get_next_bandwidth() to use connection_dir_buf_add()
Implements ticket 29897.
2019-03-29 17:26:30 +10:00
rl1987
6ab1929f00
Add connection_dir_buf_add() helper function 2019-03-29 17:25:43 +10:00
Nick Mathewson
c66df27c90 Fix checkIncludes warning about "unusual pattern in src/ext/timeouts/"
Closes ticket 28806.
2019-03-28 16:35:24 -04:00
Nick Mathewson
a7bc47532b test_routerkeys.c: Always check mkdir() return value
After this fix, we have no more unchecked mkdir() calls.

Bug 29939; CID 144254. Bugfix on 0.2.7.2-alpha.
2019-03-28 09:31:13 -04:00
Nick Mathewson
f58587a68d Don't unconditionally deref pub and sub in lint_message_consistency
This can't actually result in a null pointer dereference, since
pub_excl and sub_excl are only set when the corresponding smartlists
are nonempty.  But coverity isn't smart enough to figure that out,
and we shouldn't really be depending on it.

Bug 29938; CID 1444257.  Bug not in any released Tor.
2019-03-28 09:19:23 -04:00
George Kadianakis
00ca3d04cf Merge branch 'tor-github/pr/859' 2019-03-27 14:30:53 +02:00
teor
8991280f89
Merge branch 'maint-0.4.0' 2019-03-27 12:31:37 +10:00
teor
6d188fb4cc
Merge remote-tracking branch 'tor-github/pr/835' into maint-0.4.0 2019-03-27 12:31:07 +10:00
teor
a10d4adc25
Stop assuming that /usr/bin/python3 exists
For scripts that work with python2, use /usr/bin/python.
Otherwise, use /usr/bin/env python3.

Fixes bug 29913; bugfix on 0.2.5.3-alpha.
2019-03-27 11:07:55 +10:00
Nick Mathewson
a47b61f329 Merge branch 'messaging_v3' into messaging_v3_merged 2019-03-26 20:13:49 -04:00
teor
6d057c56f1
Merge remote-tracking branch 'tor-github/pr/820' into maint-0.3.4 2019-03-27 10:01:45 +10:00
Nick Mathewson
203e9138d1 Remove message/subsystem numbers from log messages
Having the numbers in those messages makes some of the unit test
unstable, by causing them to depend on the initialization order of
the naming objects.
2019-03-26 19:56:39 -04:00
teor
c72526f168
Merge branch 'maint-0.4.0' 2019-03-27 09:36:55 +10:00
teor
9ae8d663ea
Merge remote-tracking branch 'tor-github/pr/836' into maint-0.4.0 2019-03-27 09:36:26 +10:00
George Kadianakis
989b6325d6 Merge branch 'tor-github/pr/842' 2019-03-26 16:41:07 +02:00
Nick Mathewson
f32d890531 Merge branch 'bug29805_rebased_squashed' 2019-03-26 09:39:46 -04:00
George Kadianakis
27f24484d4 prob-distr: Some more comments about the initializers.
Based on patches and review comments by Riastradh and Catalyst.

Co-authored-by: Taylor R Campbell <campbell+tor@mumble.net>
Co-authored-by: Taylor Yu <catalyst@torproject.org>
2019-03-26 09:39:40 -04:00
George Kadianakis
08176c2396 prob-distr: Silence some coverity warnings. 2019-03-26 09:39:37 -04:00
George Kadianakis
d11976b8bd Merge branch 'tor-github/pr/709' 2019-03-26 15:34:54 +02:00
George Kadianakis
2790ee3685 Merge branch 'maint-0.4.0' 2019-03-26 15:16:37 +02:00
George Kadianakis
06951cb3fc Merge branch 'tor-github/pr/847' into maint-0.4.0 2019-03-26 15:16:21 +02:00
teor
613c5ff357
Merge remote-tracking branch 'tor-github/pr/724' 2019-03-26 19:31:17 +10:00
teor
0642650865
Merge branch 'maint-0.4.0' 2019-03-26 19:16:06 +10:00
teor
4258728d56
Merge remote-tracking branch 'tor-github/pr/852' into maint-0.4.0 2019-03-26 19:15:46 +10:00
rl1987
4e6ba575a6 Add header guards to ptr_helpers.h 2019-03-26 09:49:32 +02:00
juga0
da7a8d7624
dircache: fix identation and remove unneded goto 2019-03-26 17:41:17 +10:00
juga0
892b918b66
bwauth: remove declaring args, they are now in use 2019-03-26 17:41:13 +10:00
juga0
a4bf3be8bc
test: check that .../bandwidth.z is compressed 2019-03-26 17:41:09 +10:00
juga0
fb4a40c32c
test: Check bw file cache lifetime 2019-03-26 17:41:06 +10:00