Commit Graph

20996 Commits

Author SHA1 Message Date
Alexander Færøy
1295044dc8 Lift the list of default directory servers into their own file.
This patch lifts the list of default directory authorities from config.c
into their own auth_dirs.inc file, which is then included in config.c
using the C preprocessor.

Patch by beastr0.

See: https://bugs.torproject.org/24854
2018-04-09 16:00:26 +02:00
Nick Mathewson
2fac948158 Include tor_log rust files in source distribution.
Fixes another case of #25732; bug not in any released Tor.
2018-04-06 16:19:14 -04:00
Nick Mathewson
fb2fe41f6f Merge branch 'maint-0.3.3' 2018-04-06 16:18:47 -04:00
Nick Mathewson
306563ac68 Ship all files needed to build Tor with rust
Fixes bug 25732; bugfix on 0.3.3.2-alpha when strings.rs was
introduced.
2018-04-06 16:18:11 -04:00
Nick Mathewson
98b694bfd5 Merge branch 'isolate_libevent_2_squashed' 2018-04-06 08:50:35 -04:00
Roger Dingledine
0b0e4886cf fix confusing comment
presumably introduced by copy-and-paste mistake
2018-04-05 15:59:37 -04:00
Nick Mathewson
245fdf8ca0 Remove needless event2/thread.h include from test_compat_libevent.c 2018-04-05 12:36:28 -04:00
Nick Mathewson
4225300648 Remove redundant event2/event.h usage from test_scheduler.c
This module doesn't actually need to mock the libevent mainloop at
all: it can just use the regular mainloop that the test environment
sets up.

Part of ticket 23750.
2018-04-05 12:36:28 -04:00
Nick Mathewson
6a5f62f68f Move responsibility for threadpool reply-handler events to workqueue
This change makes cpuworker and test_workqueue no longer need to
include event2/event.h.  Now workqueue.c needs to include it, but
that is at least somewhat logical here.
2018-04-05 12:36:28 -04:00
Nick Mathewson
b3586629c9 Wrap the function we use to run the event loop.
Doing this lets us remove the event2/event.h header from a few more
modules, particularly in the tests.

Part of work on 23750.
2018-04-05 12:36:27 -04:00
Nick Mathewson
39cb04335f Add wrappers for event_base_loopexit and event_base_loopbreak. 2018-04-05 12:36:05 -04:00
Nick Mathewson
f0d2733b46 Revise procmon.c to use periodic_timer_t
This removes its need to use event2/event.h, and thereby fixes
another instance of 23750.
2018-04-05 12:35:11 -04:00
Nick Mathewson
871ff0006d Add an API for a scheduled/manually activated event in the mainloop
Using this API lets us remove event2/event.h usage from half a dozen
modules, to better isolate libevent.  Implements part of ticket
23750.
2018-04-05 12:35:11 -04:00
Nick Mathewson
c6d7e0becf Merge remote-tracking branch 'public/split_relay_crypto' 2018-04-05 12:12:18 -04:00
Neel Chauhan
f5f9c25546 Switch to use should_record_bridge_info()
Both in geoip_note_client_seen() and options_need_geoip_info(), switch from
accessing the options directly to using the should_record_bridge_info() helper
function.

Fixes #25290

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-05 08:37:39 -04:00
Nick Mathewson
ad8347418f Merge branch 'maint-0.3.3' 2018-04-05 08:22:35 -04:00
Nick Mathewson
78bf564168 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-05 08:22:34 -04:00
Nick Mathewson
9b10eb2d7a Merge branch 'maint-0.3.1' into maint-0.3.2 2018-04-05 08:22:34 -04:00
Nick Mathewson
834eef2452 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-04-05 08:22:34 -04:00
Nick Mathewson
b68e636b33 Merge branch 'maint-0.2.5' into maint-0.2.9 2018-04-05 08:22:33 -04:00
Karsten Loesing
1fa396b0a4 Update geoip and geoip6 to the April 3 2018 database. 2018-04-05 10:42:25 +02:00
David Goulet
3d5bf12ac2 relay: Remove max middle cells dead code
Next commit is addressing the circuit queue cell limit so cleanup before doing
anything else.

Part of #25226

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-04 11:03:50 -04:00
Nick Mathewson
52846f728d Merge branch 'maint-0.3.3' 2018-04-04 08:57:24 -04:00
Nick Mathewson
ec8ee54129 Merge branch 'bug21394_029_redux' into maint-0.3.3 2018-04-04 08:55:37 -04:00
Dhalgren
06484eb5e1 Bug 21394 touchup: Increase DNS attempts to 3
Also don't give up on a resolver as quickly if multiple are configured.
2018-04-04 08:54:25 -04:00
Neel Chauhan
9df110cd72
Remove PortForwarding options
Signed-off-by: Isis Lovecruft <isis@torproject.org>
2018-04-04 00:19:33 +00:00
Nick Mathewson
218b1746ba Merge remote-tracking branch 'fristonio/ticket-25645' 2018-04-03 19:19:02 -04:00
Deepesh Pathak
2680a8b5b1
ticket(25645): remove unused variable n_possible from channel_get_for_extend() 2018-04-03 09:17:23 +05:30
Isis Lovecruft
c65088cb19
rust: Fix ProtoSet and ProtoEntry to use the same DoS limits as C.
Previously, the limit for MAX_PROTOCOLS_TO_EXPAND was actually being applied
in Rust to the maximum number of version (total, for all subprotocols).
Whereas in C, it was being applied to the number of subprotocols that were
allowed.  This changes the Rust to match C's behaviour.
2018-04-02 19:59:16 +00:00
Isis Lovecruft
4b4e36a413
rust: Port all C protover_all_supported tests to Rust.
The behaviours still do not match, unsurprisingly, but now we know where a
primary difference is: the Rust is validating version ranges more than the C,
so in the C it's possible to call protover_all_supported on a ridiculous
version range like "Sleen=0-4294967294" because the C uses
MAX_PROTOCOLS_TO_EXPAND to count the number of *subprotocols* whereas the Rust
uses it to count the total number of *versions* of all subprotocols.
2018-04-02 19:59:15 +00:00
Isis Lovecruft
6739a69c59
tests: Run all existing protover tests in both languages.
There's now no difference in these tests w.r.t. the C or Rust: both
fail miserably (well, Rust fails with nice descriptive errors, and C
gives you a traceback, because, well, C).
2018-04-02 19:59:14 +00:00
Isis Lovecruft
f769edd148
tests: Make inline comments in test_protover.c more accurate.
The DoS potential is slightly higher in C now due to some differences to the
Rust code, see the C_RUST_DIFFERS tags in src/rust/protover/tests/protover.rs.

Also, the comment about "failing at the splitting stage" in Rust wasn't true,
since when we split, we ignore empty chunks (e.g. "1--1" parses into
"(1,None),(None,1)" and "None" can't be parsed into an integer).

Finally, the comment about "Rust seems to experience an internal error" is only
true in debug mode, where u32s are bounds-checked at runtime.  In release mode,
code expressing the equivalent of this test will error with
`Err(ProtoverError::Unparseable)` because 4294967295 is too large.
2018-04-02 19:59:13 +00:00
Isis Lovecruft
ad369313f8
protover: Change protover_all_supported() to return only unsupported.
Previously, if "Link=1-5" was supported, and you asked protover_all_supported()
(or protover::all_supported() in Rust) if it supported "Link=3-999", the C
version would return "Link=3-999" and the Rust would return "Link=6-999".  These
both behave the same now, i.e. both return "Link=6-999".
2018-04-02 19:59:12 +00:00
Isis Lovecruft
cd28b4c7f5
rust: Refactor protover::compute_for_old_tor().
During code review and discussion with Chelsea Komlo, she pointed out
that protover::compute_for_old_tor() was a public function whose
return type was `&'static CStr`.  We both agree that C-like parts of
APIs should:

1. not be exposed publicly (to other Rust crates),
2. only be called in the appropriate FFI code,
3. not expose types which are meant for FFI code (e.g. `*mut char`,
   `CString`, `*const c_int`, etc.) to the pure-Rust code of other
   crates.
4. FFI code (e.g. things in `ffi.rs` modules) should _never_ be called
   from pure-Rust, not even from other modules in its own crate
   (i.e. do not call `protover::ffi::*` from anywhere in
   `protover::protoset::*`, etc).

With that in mind, this commit makes the following changes:

 * CHANGE `protover::compute_for_old_tor()` to be
   visible only at the `pub(crate)` level.
 * RENAME `protover::compute_for_old_tor()` to
   `protover::compute_for_old_tor_cstr()` to reflect the last change.
 * ADD a new `protover::compute_for_old_tor()` function wrapper which
   is public and intended for other Rust code to use, which returns a
   `&str`.
2018-04-02 19:59:12 +00:00
Isis Lovecruft
fd127bfbfa
rust: Refactor Rust implementation of protover_is_supported_here().
It was changed to take borrows instead of taking ownership.

 * REFACTOR `protover::ffi::protover_is_supported_here()` to use changed method
   signature on `protover::is_supported_here()`.
2018-04-02 19:36:26 +00:00
Isis Lovecruft
32638ed4a6
rust: Refactor Rust impl of protover_compute_vote().
This includes a subtle difference in behaviour to the previous Rust
implementation, where, for each vote that we're computing over, if a single one
fails to parse, we skip it.  This now matches the current behaviour in the C
implementation.

 * REFACTOR `protover::ffi::protover_compute_vote()` to use
   new types and methods.
2018-04-02 19:36:25 +00:00
Isis Lovecruft
269053a380
rust: Refactor Rust impl of protover_list_supports_protocol_or_later().
This includes a subtle difference in behaviour, as in 4258f1e18, where we return
(matching the C impl's return behaviour) earlier than before if parsing failed,
saving us computation in parsing the versions into a
protover::protoset::ProtoSet.

 * REFACTOR `protover::ffi::protover_list_supports_protocol_or_later()` to use
   new types and methods.
2018-04-02 19:36:25 +00:00
Isis Lovecruft
63eeda89ea
rust: Refactor Rust impl of protover_list_supports_protocol().
This includes a subtle difference in behaviour, as in 4258f1e18, where we return
(matching the C impl's return behaviour) earlier than before if parsing failed,
saving us computation in parsing the versions into a
protover::protoset::ProtoSet.

 * REFACTOR `protover::ffi::protover_list_supports_protocol()` to use new types
   and methods.
2018-04-02 19:34:26 +00:00
Isis Lovecruft
c7bcca0233
rust: Refactor Rust impl of protover_all_supported().
This includes differences in behaviour to before, which should now more closely
match the C version:

 - If parsing a protover `char*` from C, and the string is not parseable, this
   function will return 1 early, which matches the C behaviour when protocols
   are unparseable.  Previously, we would parse it and its version numbers
   simultaneously, i.e. there was no fail early option, causing us to spend more
   time unnecessarily parsing versions.

 * REFACTOR `protover::ffi::protover_all_supported()` to use new types and
   methods.
2018-04-02 19:34:26 +00:00
Isis Lovecruft
493e565226
rust: Refactor protover tests with new methods; note altered behaviours.
Previously, the rust implementation of protover considered an empty string to be
a valid ProtoEntry, while the C version did not (it must have a "=" character).
Other differences include that unknown protocols must now be parsed as
`protover::UnknownProtocol`s, and hence their entries as
`protover::UnvalidatedProtoEntry`s, whereas before (nearly) all protoentries
could be parsed regardless of how erroneous they might be considered by the C
version.

My apologies for this somewhat messy and difficult to read commit, if any part
is frustrating to the reviewer, please feel free to ask me to split this into
smaller changes (possibly hard to do, since so much changed), or ask me to
comment on a specific line/change and clarify how/when the behaviours differ.

The tests here should more closely match the behaviours exhibited by the C
implementation, but I do not yet personally guarantee they match precisely.

 * REFACTOR unittests in protover::protover.
 * ADD new integration tests for previously untested behaviour.
 * FIXES part of #24031: https://bugs.torproject.org/24031.
2018-04-02 19:34:25 +00:00
Isis Lovecruft
35b86a12e6
rust: Refactor protover::is_supported_here().
This changes `protover::is_supported_here()` to be aware of new datatypes
(e.g. don't call `.0` on things which are no longer tuple structs) and also
changes the method signature to take borrows, making it faster, threadable, and
easier to read (i.e. the caller can know from reading the function signature
that the function won't mutate values passed into it).

 * CHANGE the `protover::is_supported_here()` function to take borrows.
 * REFACTOR the `protover::is_supported_here()` function to be aware of new
   datatypes.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:34:25 +00:00
Isis Lovecruft
2eb1b7f2fd
rust: Add new ProtoverVote type and refactor functions to methods.
This adds a new type for votes upon `protover::ProtoEntry`s (technically, on
`protover::UnvalidatedProtoEntry`s, because the C code does not validate based
upon currently known protocols when voting, in order to maintain
future-compatibility), and converts several functions which would have operated
on this datatype into methods for ease-of-use and readability.

This also fixes a behavioural differentce to the C version of
protover_compute_vote().  The C version of protover_compute_vote() calls
expand_protocol_list() which checks if there would be too many subprotocols *or*
expanded individual version numbers, i.e. more than MAX_PROTOCOLS_TO_EXPAND, and
does this *per vote* (but only in compute_vote(), everywhere else in the C seems
to only care about the number of subprotocols, not the number of individual
versions).  We need to match its behaviour in Rust and ensure we're not allowing
more than it would to get the votes to match.

 * ADD new `protover::ProtoverVote` datatype.
 * REMOVE the `protover::compute_vote()` function and refactor it into an
   equivalent-in-behaviour albeit more memory-efficient voting algorithm based
   on the new underlying `protover::protoset::ProtoSet` datatype, as
   `ProtoverVote::compute()`.
 * REMOVE the `protover::write_vote_to_string()` function, since this
   functionality is now generated by the impl_to_string_for_proto_entry!() macro
   for both `ProtoEntry` and `UnvalidatedProtoEntry` (the latter of which is the
   correct type to return from a voting protocol instance, since the entity
   voting may not know of all protocols being voted upon or known about by other
   voting parties).
 * FIXES part of #24031: https://bugs.torproject.org/24031

rust: Fix a difference in compute_vote() behaviour to C version.
2018-04-02 19:34:24 +00:00
Isis Lovecruft
fa15ea104d
rust: Add macro for impl ToString for {Unvalidated}ProtoEntry.
This implements conversions from either a ProtoEntry or an UnvalidatedProtoEntry
into a String, for use in replacing such functions as
`protover::write_vote_to_string()`.

 * ADD macro for implementing ToString trait for ProtoEntry and
   UnvalidatedProtoEntry.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:34:24 +00:00
Isis Lovecruft
3c47d31e1f
rust: Add new protover::UnvalidatedProtoEntry type.
This adds a new protover::UnvalidatedProtoEntry type, which is the
UnknownProtocol variant of a ProtoEntry, and refactors several functions which
should operate on this type into methods.

This also fixes what was previously another difference to the C implementation:
if you asked the C version of protovet_compute_vote() to compute a single vote
containing "Fribble=", it would return NULL.  However, the Rust version would
return "Fribble=" since it didn't check if the versions were empty before
constructing the string of differences.  ("Fribble=" is technically a valid
protover string.)  This is now fixed, and the Rust version in that case will,
analogous to (although safer than) C returning a NULL, return None.

 * REMOVE internal `contains_only_supported_protocols()` function.
 * REMOVE `all_supported()` function and refactor it into
   `UnvalidatedProtoEntry::all_supported()`.
 * REMOVE `parse_protocols_from_string_with_no_validation()` and
   refactor it into the more rusty implementation of
   `impl FromStr for UnvalidatedProtoEntry`.
 * REMOVE `protover_string_supports_protocol()` and refactor it into
   `UnvalidatedProtoEntry::supports_protocol()`.
 * REMOVE `protover_string_supports_protocol_or_later()` and refactor
   it into `UnvalidatedProtoEntry::supports_protocol_or_later()`.
 * FIXES part of #24031: https://bugs.torproject.org/24031

rust: Fix another C/Rust different in compute_vote().

This fixes the unittest from the prior commit by checking if the versions are
empty before adding a protocol to a vote.
2018-04-02 19:34:23 +00:00
Isis Lovecruft
54c964332b
rust: Add new protover::ProtoEntry type which uses new datatypes.
This replaces the `protover::SupportedProtocols` (why would you have a type just
for things which are supported?) with a new, more generic type,
`protover::ProtoEntry`, which utilises the new, more memory-efficient datatype
in protover::protoset.

 * REMOVE `get_supported_protocols()` and `SupportedProtocols::tor_supported()`
   (since they were never used separately) and collapse their functionality into
   a single `ProtoEntry::supported()` method.
 * REMOVE `SupportedProtocols::from_proto_entries()` and reimplement its
   functionality as the more rusty `impl FromStr for ProtoEntry`.
 * REMOVE `get_proto_and_vers()` function and refactor it into the more rusty
   `impl FromStr for ProtoEntry`.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:32:36 +00:00
Isis Lovecruft
60daaa68b1
rust: Add new protover::UnknownProtocol type.
* ADD new type, protover::UnknownProtocol, so that we have greater type safety
   and our protover functionality which works with unsanitised protocol names is
   more clearly demarcated.
 * REFACTOR protover::Proto, renaming it protover::Protocol to mirror the new
   protover::UnknownProtocol type name.
 * ADD a utility conversion of `impl From<Protocol> for UnknownProtocol` so that
   we can easily with known protocols and unknown protocols simultaneously
   (e.g. doing comparisons, checking their version numbers), while not allowing
   UnknownProtocols to be accidentally used in functions which should only take
   Protocols.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:27:51 +00:00
Isis Lovecruft
e6625113c9
rust: Implement more memory-efficient protover datatype.
* ADD new protover::protoset module.
 * ADD new protover::protoset::ProtoSet class for holding protover versions.
 * REMOVE protover::Versions type implementation and its method
   `from_version_string()`, and instead implement this behaviour in a more
   rust-like manner as `impl FromStr for ProtoSet`.
 * MOVE the `find_range()` utility function from protover::protover to
   protover::protoset since it's only used internally in the
   implementation of ProtoSet.
 * REMOVE the `contract_protocol_list()` function from protover::protover and
   instead refactor it (reusing nearly the entire thing, with minor superficial,
   i.e. non-behavioural, changes) into a more rusty
   `impl ToString for ProtoSet`.
 * REMOVE the `expand_version_range()` function from protover::protover and
   instead refactor it into a more rusty implementation of
   `impl Into<Vec<Version>> for ProtoSet` using the new error types in
   protover::errors.
 * FIXES part of #24031: https://bugs.torproject.org/24031.
2018-04-02 19:26:26 +00:00
Isis Lovecruft
88204f91df
rust: Implement error types for Rust protover implementation.
This will allow us to do actual error handling intra-crate in a more
rusty manner, e.g. propogating errors in match statements, conversion
between error types, logging messages, etc.

 * FIXES part of #24031: https://bugs.torproject.org/24031.
2018-04-02 19:24:32 +00:00
Isis Lovecruft
f2daf82794
rust: Fix ProtoSet and ProtoEntry to use the same DoS limits as C.
Previously, the limit for MAX_PROTOCOLS_TO_EXPAND was actually being applied
in Rust to the maximum number of version (total, for all subprotocols).
Whereas in C, it was being applied to the number of subprotocols that were
allowed.  This changes the Rust to match C's behaviour.
2018-04-02 19:20:40 +00:00
Isis Lovecruft
6eea0dc5f1
rust: Port all C protover_all_supported tests to Rust.
The behaviours still do not match, unsurprisingly, but now we know where a
primary difference is: the Rust is validating version ranges more than the C,
so in the C it's possible to call protover_all_supported on a ridiculous
version range like "Sleen=0-4294967294" because the C uses
MAX_PROTOCOLS_TO_EXPAND to count the number of *subprotocols* whereas the Rust
uses it to count the total number of *versions* of all subprotocols.
2018-04-02 19:20:39 +00:00
Isis Lovecruft
527a239863
tests: Run all existing protover tests in both languages.
There's now no difference in these tests w.r.t. the C or Rust: both
fail miserably (well, Rust fails with nice descriptive errors, and C
gives you a traceback, because, well, C).
2018-04-02 19:20:38 +00:00
Isis Lovecruft
22c65a0e4b
tests: Make inline comments in test_protover.c more accurate.
The DoS potential is slightly higher in C now due to some differences to the
Rust code, see the C_RUST_DIFFERS tags in src/rust/protover/tests/protover.rs.

Also, the comment about "failing at the splitting stage" in Rust wasn't true,
since when we split, we ignore empty chunks (e.g. "1--1" parses into
"(1,None),(None,1)" and "None" can't be parsed into an integer).

Finally, the comment about "Rust seems to experience an internal error" is only
true in debug mode, where u32s are bounds-checked at runtime.  In release mode,
code expressing the equivalent of this test will error with
`Err(ProtoverError::Unparseable)` because 4294967295 is too large.
2018-04-02 19:20:37 +00:00
Isis Lovecruft
6e353664dd
protover: Change protover_all_supported() to return only unsupported.
Previously, if "Link=1-5" was supported, and you asked protover_all_supported()
(or protover::all_supported() in Rust) if it supported "Link=3-999", the C
version would return "Link=3-999" and the Rust would return "Link=6-999".  These
both behave the same now, i.e. both return "Link=6-999".
2018-04-02 19:20:36 +00:00
Isis Lovecruft
fc2a42cc49
rust: Refactor protover::compute_for_old_tor().
During code review and discussion with Chelsea Komlo, she pointed out
that protover::compute_for_old_tor() was a public function whose
return type was `&'static CStr`.  We both agree that C-like parts of
APIs should:

1. not be exposed publicly (to other Rust crates),
2. only be called in the appropriate FFI code,
3. not expose types which are meant for FFI code (e.g. `*mut char`,
   `CString`, `*const c_int`, etc.) to the pure-Rust code of other
   crates.
4. FFI code (e.g. things in `ffi.rs` modules) should _never_ be called
   from pure-Rust, not even from other modules in its own crate
   (i.e. do not call `protover::ffi::*` from anywhere in
   `protover::protoset::*`, etc).

With that in mind, this commit makes the following changes:

 * CHANGE `protover::compute_for_old_tor()` to be
   visible only at the `pub(crate)` level.
 * RENAME `protover::compute_for_old_tor()` to
   `protover::compute_for_old_tor_cstr()` to reflect the last change.
 * ADD a new `protover::compute_for_old_tor()` function wrapper which
   is public and intended for other Rust code to use, which returns a
   `&str`.
2018-04-02 19:20:36 +00:00
Isis Lovecruft
9766d53cf9
rust: Refactor Rust implementation of protover_is_supported_here().
It was changed to take borrows instead of taking ownership.

 * REFACTOR `protover::ffi::protover_is_supported_here()` to use changed method
   signature on `protover::is_supported_here()`.
2018-04-02 19:20:35 +00:00
Isis Lovecruft
6f252e0986
rust: Refactor Rust impl of protover_compute_vote().
This includes a subtle difference in behaviour to the previous Rust
implementation, where, for each vote that we're computing over, if a single one
fails to parse, we skip it.  This now matches the current behaviour in the C
implementation.

 * REFACTOR `protover::ffi::protover_compute_vote()` to use
   new types and methods.
2018-04-02 19:20:34 +00:00
Isis Lovecruft
0a5494b81d
rust: Refactor Rust impl of protover_list_supports_protocol_or_later().
This includes a subtle difference in behaviour, as in 4258f1e18, where we return
(matching the C impl's return behaviour) earlier than before if parsing failed,
saving us computation in parsing the versions into a
protover::protoset::ProtoSet.

 * REFACTOR `protover::ffi::protover_list_supports_protocol_or_later()` to use
   new types and methods.
2018-04-02 19:20:33 +00:00
Isis Lovecruft
52c3ea5045
rust: Refactor Rust impl of protover_list_supports_protocol().
This includes a subtle difference in behaviour, as in 4258f1e18, where we return
(matching the C impl's return behaviour) earlier than before if parsing failed,
saving us computation in parsing the versions into a
protover::protoset::ProtoSet.

 * REFACTOR `protover::ffi::protover_list_supports_protocol()` to use new types
   and methods.
2018-04-02 19:20:32 +00:00
Isis Lovecruft
2f3a7376c0
rust: Refactor Rust impl of protover_all_supported().
This includes differences in behaviour to before, which should now more closely
match the C version:

 - If parsing a protover `char*` from C, and the string is not parseable, this
   function will return 1 early, which matches the C behaviour when protocols
   are unparseable.  Previously, we would parse it and its version numbers
   simultaneously, i.e. there was no fail early option, causing us to spend more
   time unnecessarily parsing versions.

 * REFACTOR `protover::ffi::protover_all_supported()` to use new types and
   methods.
2018-04-02 19:20:31 +00:00
Isis Lovecruft
15e59a1fed
rust: Refactor protover tests with new methods; note altered behaviours.
Previously, the rust implementation of protover considered an empty string to be
a valid ProtoEntry, while the C version did not (it must have a "=" character).
Other differences include that unknown protocols must now be parsed as
`protover::UnknownProtocol`s, and hence their entries as
`protover::UnvalidatedProtoEntry`s, whereas before (nearly) all protoentries
could be parsed regardless of how erroneous they might be considered by the C
version.

My apologies for this somewhat messy and difficult to read commit, if any part
is frustrating to the reviewer, please feel free to ask me to split this into
smaller changes (possibly hard to do, since so much changed), or ask me to
comment on a specific line/change and clarify how/when the behaviours differ.

The tests here should more closely match the behaviours exhibited by the C
implementation, but I do not yet personally guarantee they match precisely.

 * REFACTOR unittests in protover::protover.
 * ADD new integration tests for previously untested behaviour.
 * FIXES part of #24031: https://bugs.torproject.org/24031.
2018-04-02 19:20:31 +00:00
Isis Lovecruft
aa241e99de
rust: Refactor protover::is_supported_here().
This changes `protover::is_supported_here()` to be aware of new datatypes
(e.g. don't call `.0` on things which are no longer tuple structs) and also
changes the method signature to take borrows, making it faster, threadable, and
easier to read (i.e. the caller can know from reading the function signature
that the function won't mutate values passed into it).

 * CHANGE the `protover::is_supported_here()` function to take borrows.
 * REFACTOR the `protover::is_supported_here()` function to be aware of new
   datatypes.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:20:30 +00:00
Isis Lovecruft
9abbd23df7
rust: Add new ProtoverVote type and refactor functions to methods.
This adds a new type for votes upon `protover::ProtoEntry`s (technically, on
`protover::UnvalidatedProtoEntry`s, because the C code does not validate based
upon currently known protocols when voting, in order to maintain
future-compatibility), and converts several functions which would have operated
on this datatype into methods for ease-of-use and readability.

This also fixes a behavioural differentce to the C version of
protover_compute_vote().  The C version of protover_compute_vote() calls
expand_protocol_list() which checks if there would be too many subprotocols *or*
expanded individual version numbers, i.e. more than MAX_PROTOCOLS_TO_EXPAND, and
does this *per vote* (but only in compute_vote(), everywhere else in the C seems
to only care about the number of subprotocols, not the number of individual
versions).  We need to match its behaviour in Rust and ensure we're not allowing
more than it would to get the votes to match.

 * ADD new `protover::ProtoverVote` datatype.
 * REMOVE the `protover::compute_vote()` function and refactor it into an
   equivalent-in-behaviour albeit more memory-efficient voting algorithm based
   on the new underlying `protover::protoset::ProtoSet` datatype, as
   `ProtoverVote::compute()`.
 * REMOVE the `protover::write_vote_to_string()` function, since this
   functionality is now generated by the impl_to_string_for_proto_entry!() macro
   for both `ProtoEntry` and `UnvalidatedProtoEntry` (the latter of which is the
   correct type to return from a voting protocol instance, since the entity
   voting may not know of all protocols being voted upon or known about by other
   voting parties).
 * FIXES part of #24031: https://bugs.torproject.org/24031

rust: Fix a difference in compute_vote() behaviour to C version.
2018-04-02 19:20:29 +00:00
Isis Lovecruft
26bafb3c33
rust: Add macro for impl ToString for {Unvalidated}ProtoEntry.
This implements conversions from either a ProtoEntry or an UnvalidatedProtoEntry
into a String, for use in replacing such functions as
`protover::write_vote_to_string()`.

 * ADD macro for implementing ToString trait for ProtoEntry and
   UnvalidatedProtoEntry.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:20:28 +00:00
Isis Lovecruft
b786b146ed
rust: Add new protover::UnvalidatedProtoEntry type.
This adds a new protover::UnvalidatedProtoEntry type, which is the
UnknownProtocol variant of a ProtoEntry, and refactors several functions which
should operate on this type into methods.

This also fixes what was previously another difference to the C implementation:
if you asked the C version of protovet_compute_vote() to compute a single vote
containing "Fribble=", it would return NULL.  However, the Rust version would
return "Fribble=" since it didn't check if the versions were empty before
constructing the string of differences.  ("Fribble=" is technically a valid
protover string.)  This is now fixed, and the Rust version in that case will,
analogous to (although safer than) C returning a NULL, return None.

 * REMOVE internal `contains_only_supported_protocols()` function.
 * REMOVE `all_supported()` function and refactor it into
   `UnvalidatedProtoEntry::all_supported()`.
 * REMOVE `parse_protocols_from_string_with_no_validation()` and
   refactor it into the more rusty implementation of
   `impl FromStr for UnvalidatedProtoEntry`.
 * REMOVE `protover_string_supports_protocol()` and refactor it into
   `UnvalidatedProtoEntry::supports_protocol()`.
 * REMOVE `protover_string_supports_protocol_or_later()` and refactor
   it into `UnvalidatedProtoEntry::supports_protocol_or_later()`.
 * FIXES part of #24031: https://bugs.torproject.org/24031

rust: Fix another C/Rust different in compute_vote().

This fixes the unittest from the prior commit by checking if the versions are
empty before adding a protocol to a vote.
2018-04-02 19:20:27 +00:00
Isis Lovecruft
88b2f170e4
rust: Add new protover::ProtoEntry type which uses new datatypes.
This replaces the `protover::SupportedProtocols` (why would you have a type just
for things which are supported?) with a new, more generic type,
`protover::ProtoEntry`, which utilises the new, more memory-efficient datatype
in protover::protoset.

 * REMOVE `get_supported_protocols()` and `SupportedProtocols::tor_supported()`
   (since they were never used separately) and collapse their functionality into
   a single `ProtoEntry::supported()` method.
 * REMOVE `SupportedProtocols::from_proto_entries()` and reimplement its
   functionality as the more rusty `impl FromStr for ProtoEntry`.
 * REMOVE `get_proto_and_vers()` function and refactor it into the more rusty
   `impl FromStr for ProtoEntry`.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:20:26 +00:00
Isis Lovecruft
811178434e
rust: Add new protover::UnknownProtocol type.
* ADD new type, protover::UnknownProtocol, so that we have greater type safety
   and our protover functionality which works with unsanitised protocol names is
   more clearly demarcated.
 * REFACTOR protover::Proto, renaming it protover::Protocol to mirror the new
   protover::UnknownProtocol type name.
 * ADD a utility conversion of `impl From<Protocol> for UnknownProtocol` so that
   we can easily with known protocols and unknown protocols simultaneously
   (e.g. doing comparisons, checking their version numbers), while not allowing
   UnknownProtocols to be accidentally used in functions which should only take
   Protocols.
 * FIXES part of #24031: https://bugs.torproject.org/24031
2018-04-02 19:20:12 +00:00
Isis Lovecruft
9925d2e687
rust: Implement more memory-efficient protover datatype.
* ADD new protover::protoset module.
 * ADD new protover::protoset::ProtoSet class for holding protover versions.
 * REMOVE protover::Versions type implementation and its method
   `from_version_string()`, and instead implement this behaviour in a more
   rust-like manner as `impl FromStr for ProtoSet`.
 * MOVE the `find_range()` utility function from protover::protover to
   protover::protoset since it's only used internally in the
   implementation of ProtoSet.
 * REMOVE the `contract_protocol_list()` function from protover::protover and
   instead refactor it (reusing nearly the entire thing, with minor superficial,
   i.e. non-behavioural, changes) into a more rusty
   `impl ToString for ProtoSet`.
 * REMOVE the `expand_version_range()` function from protover::protover and
   instead refactor it into a more rusty implementation of
   `impl Into<Vec<Version>> for ProtoSet` using the new error types in
   protover::errors.
 * FIXES part of #24031: https://bugs.torproject.org/24031.
2018-04-02 19:04:19 +00:00
Isis Lovecruft
b6059297d7
rust: Implement error types for Rust protover implementation.
This will allow us to do actual error handling intra-crate in a more
rusty manner, e.g. propogating errors in match statements, conversion
between error types, logging messages, etc.

 * FIXES part of #24031: https://bugs.torproject.org/24031.
2018-04-02 18:27:39 +00:00
Nick Mathewson
f9e32a2084 Remove an unnecessary event2 include.
The rest, are, unfortunately, necessary for now.
2018-04-02 11:11:34 -04:00
Nick Mathewson
f4bcf3f34c Remove event2/event.h include from compat_libevent.h
Only one module was depending on this include (test_helpers.c), and
it was doing so incorrectly.
2018-04-02 11:11:31 -04:00
Nick Mathewson
3df9545492 Merge branch 'maint-0.3.3' 2018-04-02 08:51:47 -04:00
Roger Dingledine
6190593256 use channel_is_client for create cell counts
When a relay is collecting internal statistics about how many
create cell requests it has seen of each type, accurately count the
requests from relays that temporarily fall out of the consensus.

(To be extra conservative, we were already ignoring requests from clients
in our counts, and we continue ignoring them here.)

Fixes bug 24910; bugfix on 0.2.4.17-rc.
2018-04-02 01:00:31 -04:00
Roger Dingledine
961d2ad597 dir auths no longer vote Guard if they're not voting V2Dir
Directory authorities no longer vote in favor of the Guard flag
for relays that don't advertise directory support.

Starting in Tor 0.3.0.1-alpha, Tor clients have been avoiding using
such relays in the Guard position, leading to increasingly broken load
balancing for the 5%-or-so of Guards that don't advertise directory
support.

Fixes bug 22310; bugfix on 0.3.0.6.
2018-04-02 00:20:01 -04:00
Roger Dingledine
0983c203e5 misc tiny fixes 2018-04-01 23:47:44 -04:00
Taylor Yu
596eed3715 Fix CID 1433643
Add a missing lock acquisition around access to queued_control_events
in control_free_all().  Use the reassign-and-unlock strategy as in
queued_events_flush_all().  Fixes bug 25675.  Coverity found this bug,
but only after we recently added an access to
flush_queued_event_pending.
2018-03-29 17:21:33 -05:00
Nick Mathewson
2c1afc2def Merge branch 'maint-0.3.3' 2018-03-29 14:59:01 -04:00
Nick Mathewson
4f473fadbd Merge branch 'bug25617_029' into maint-0.3.3 2018-03-29 14:58:58 -04:00
Nick Mathewson
2e9e91ebbf bump version to 0.3.3.4-alpha-dev 2018-03-29 11:24:02 -04:00
Nick Mathewson
e35eb9baaa Mark controller-initiated DNS lookups as permitted to do DNS.
Fixes bug 25617; bugfix on 0.2.9.3-alpha.
2018-03-29 09:27:28 -04:00
Nick Mathewson
3fa66f9799 Bump version to 0.3.3.4-alpha 2018-03-29 08:01:37 -04:00
Nick Mathewson
2f872f9762 Merge remote-tracking branch 'hello71/bug25398' 2018-03-28 14:47:05 -04:00
Nick Mathewson
4c0e434f33 Merge remote-tracking branch 'public/bug25512' 2018-03-28 14:45:47 -04:00
Nick Mathewson
cb083b5d3e Merge remote-tracking branch 'asn-github/t-25432' 2018-03-28 14:43:33 -04:00
Nick Mathewson
794a25f8c1 Merge branch 'maint-0.3.3' 2018-03-28 14:23:06 -04:00
Nick Mathewson
e0bbef48bf Merge remote-tracking branch 'dgoulet/bug24767_033_03' into maint-0.3.3 2018-03-28 14:21:04 -04:00
Nick Mathewson
6317aa2cc0 Merge branch 'maint-0.3.3' 2018-03-28 07:50:47 -04:00
Nick Mathewson
d4bf1f6c8e Add a paranoia check in string_is_valid_nonrfc_hostname()
The earlier checks in this function should ensure that components is
always nonempty.  But in case somebody messes with them in the
future, let's add an extra check to make sure we aren't crashing.
2018-03-28 07:48:18 -04:00
Nick Mathewson
b504c854d3 Rename string_is_valid_hostname -> string_is_valid_nonrfc_hostname
Per discussion on 25055.
2018-03-28 07:42:27 -04:00
rl1987
09351c34e9 Don't strlen before checking for NULL 2018-03-28 07:39:03 -04:00
rl1987
a28e350cff Tweak loop condition 2018-03-28 07:39:03 -04:00
rl1987
6b6d003f43 Don't explode on NULL or empty string 2018-03-28 07:39:03 -04:00
rl1987
d891010fdd Allow alphanumeric TLDs in test for now 2018-03-28 07:39:03 -04:00
rl1987
ee1fca727c Simplify hostname validation code 2018-03-28 07:39:03 -04:00
rl1987
dbb7c8e6fd Validate hostnames with punycode TLDs correctly 2018-03-28 07:39:03 -04:00
rl1987
4413e52f9e Improve handling of trailing dot 2018-03-28 07:39:03 -04:00
rl1987
db850fec3a Test TLD validation 2018-03-28 07:39:03 -04:00
rl1987
6335db9fce Refrain from including <ctype.h> 2018-03-28 07:39:03 -04:00
rl1987
12afd8bfed Also test bracket-less IPv6 string validation 2018-03-28 07:39:03 -04:00
rl1987
5986589b48 Call strlen() once 2018-03-28 07:39:03 -04:00
rl1987
b0ba4aa7e9 Fix bracketed IPv6 string validation 2018-03-28 07:39:03 -04:00
rl1987
1af016e96e Do not consider IP strings valid DNS names. Fixes #25055 2018-03-28 07:39:03 -04:00
rl1987
0e453929d2 Allow IPv6 address strings to be used as hostnames in SOCKS5 requests 2018-03-28 07:39:03 -04:00
Nick Mathewson
fa6eaab83e Merge branch 'maint-0.3.3' 2018-03-27 18:25:52 -04:00
Nick Mathewson
5acfc30876 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-03-27 18:23:53 -04:00
Taylor Yu
471f28a2a8 Fix CID 1430932
Coverity found a null pointer reference in nodelist_add_microdesc().
This is almost certainly impossible assuming that the routerstatus_t
returned by router_get_consensus_status_by_descriptor_digest() always
corresponds to an entry in the nodelist.  Fixes bug 25629.
2018-03-27 16:11:29 -05:00
Taylor Yu
0c13a84c0d Fix CID 1430932
Coverity found a null pointer reference in nodelist_add_microdesc().
This is almost certainly impossible assuming that the routerstatus_t
returned by router_get_consensus_status_by_descriptor_digest() always
corresponds to an entry in the nodelist.  Fixes bug 25629.
2018-03-27 16:08:39 -05:00
Taylor Yu
4bb7d9fd12 Fix CID 1430932
Coverity found a null pointer reference in nodelist_add_microdesc().
This is almost certainly impossible assuming that the routerstatus_t
returned by router_get_consensus_status_by_descriptor_digest() always
corresponds to an entry in the nodelist.  Fixes bug 25629.
2018-03-27 15:29:00 -05:00
George Kadianakis
ab16f1e2a1 test: Add unittest for the OR connection failure cache
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-03-27 12:43:39 -04:00
David Goulet
f29d158330 relay: Avoid connecting to down relays
If we failed to connect at the TCP level to a relay, note it down and refuse
to connect again for another 60 seconds.

Fixes #24767

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-03-27 12:42:31 -04:00
Nick Mathewson
979c7e5c83 Merge branch 'maint-0.3.3' 2018-03-27 07:05:15 -04:00
Nick Mathewson
46c2b0ca22 Merge branch 'bug25213_033' into maint-0.3.3 2018-03-27 07:04:33 -04:00
Nick Mathewson
0eed0899cd Merge branch 'bug24658-rm-curve25519-header' into bug24658-merge 2018-03-26 20:12:59 -04:00
Nick Mathewson
d96dc2060a Merge branch 'maint-0.3.3' 2018-03-26 19:47:48 -04:00
Nick Mathewson
841ed9dbb9 Merge remote-tracking branch 'dgoulet/bug24904_033_01' into maint-0.3.3 2018-03-26 19:47:44 -04:00
Nick Mathewson
5278d72f97 Merge branch 'maint-0.3.3' 2018-03-26 10:32:53 -04:00
Nick Mathewson
ca2d9cbb93 Merge branch 'bug24903_029' into maint-0.3.3 2018-03-26 10:32:49 -04:00
Nick Mathewson
068d092749 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-03-26 10:29:29 -04:00
Nick Mathewson
b5a6c03998 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-03-26 10:29:29 -04:00
Nick Mathewson
c68bfc556c Merge branch 'maint-0.3.3' 2018-03-26 10:29:29 -04:00
Nick Mathewson
33606405e3 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-03-26 10:29:29 -04:00
Nick Mathewson
969a38a375 Fix a unit test which was broken by the previous commit
This test was expecting Tor to find and use routerinfos, but hadn't
cleared the UseMicrodescriptors flag.  Part of the fix for 25213.
2018-03-26 09:57:39 -04:00
Nick Mathewson
d1874b4339 Make extend_info_from_node() more picky about node contents
This update is needed to make it consistent with the behavior of
node_awaiting_ipv6(), which doesn't believe in the addresses from
routerinfos unless it actually plans to use those routerinfos.

Fixes bug 25213; bugfix on b66b62fb75 in 0.3.3.1-alpha,
which tightened up the definition of node_awaiting_ipv6().
2018-03-26 09:56:12 -04:00
Nick Mathewson
33a80921a2 When extending a circuit's path length, clear onehop_tunnel.
There was a nonfatal assertion in pathbias_should_count that would
trigger if onehop_tunnel was set, but the desired_path_length was
greater than 1.  This patch fixes that.  Fixes bug 24903; bugfix on
0.2.5.2-alpha.
2018-03-26 09:17:50 -04:00
Nick Mathewson
a9fa483004 Document a requirement for cells to be encrypted. 2018-03-24 13:49:44 -04:00
Nick Mathewson
7db4d0c55f Basic unit tests for relay_crypto.c
These tests handle incoming and outgoing cells on a three-hop
circuit, and make sure that the crypto works end-to-end.  They don't
yet test spec conformance, leaky-pipe, or various error cases.
2018-03-24 13:49:08 -04:00
Nick Mathewson
d749f6b5f6 Merge branch 'maint-0.3.3' 2018-03-23 17:49:29 -04:00
Nick Mathewson
eacfd29112 Fix windows compilation warnings in hs_service.c
These were breaking jenkins builds.  Bugfix on 5804ccc9070dc54;
bug not in any released Tor.
2018-03-23 17:47:56 -04:00
Nick Mathewson
bb9012c818 test: more data on geoip load failure. 2018-03-23 11:48:15 -04:00
Nick Mathewson
3519d0c808 Clear all control.c flags on control_free_all()
Fixes bug 25512.
2018-03-23 11:31:56 -04:00
Nick Mathewson
e263317e07 Merge remote-tracking branch 'fristonio/ticket-24740' 2018-03-23 11:22:58 -04:00
Nick Mathewson
11114c7e83 Merge branch 'maint-0.3.3' 2018-03-23 11:19:19 -04:00
Isis Lovecruft
657d5cbbbc tests: Automatically detect Rust crates to test and also pass --verbose.
* FIXES #25560: https://bugs.torproject.org/25560.
2018-03-23 11:13:04 -04:00
David Goulet
236c92a0a7 chan: Use channel_is_client() in channel_do_open_actions()
Make sure we actually only report client channel to the geoip cache instead of
looking if it is a known relay. Looking if it is a known relay can be
unreliable because they come and go from the consensus.

Fixes #24904

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-03-23 10:11:37 -04:00
Isis Lovecruft
fae5254783
hs: Fix two typos in an inline comment.
* FIXES #25602: https://bugs.torproject.org/25602
2018-03-22 22:33:34 +00:00
Nick Mathewson
24abcf9771 Merge branch 'bug25399_squashed' 2018-03-22 08:49:43 -04:00
Alex Xu (Hello71)
946ed24ca5 Do not page-align mmap length. #25399 2018-03-22 08:47:37 -04:00
Nick Mathewson
d9ba7db38b Merge remote-tracking branch 'public/geoip_testing' 2018-03-22 08:43:28 -04:00
Nick Mathewson
f8e53a545a Update tor_log to libc 0.2.39 as well. 2018-03-21 17:14:15 -04:00
Nick Mathewson
4e82441e4c Merge branch 'maint-0.3.3' 2018-03-21 17:10:10 -04:00
Nick Mathewson
2b31387410 Update src/ext/rust to latest master for libc update. 2018-03-21 17:09:59 -04:00
Nick Mathewson
03e787e220 Merge branch 'maint-0.3.3' 2018-03-21 17:05:42 -04:00
Isis Lovecruft
00a473733d maint: Update Rust libc dependency from 0.2.22 to 0.2.39.
Requires the update/libc-0.2.39 branch from
https://github.com/isislovecruft/tor-rust-dependencies to be merged
first.
2018-03-21 17:04:28 -04:00
Nick Mathewson
2c36a02bb1 Merge branch 'maint-0.3.3' 2018-03-20 12:55:46 -04:00
Nick Mathewson
b069979142 Merge branch 'bug25306_032_01_squashed_v2' into maint-0.3.3 2018-03-20 12:54:51 -04:00
David Goulet
5804ccc907 hs-v3: BUG() on missing descriptors during rotation
Because of #25306 for which we are unable to reproduce nor understand how it
is possible, this commit removes the asserts() and BUG() on the missing
descriptors instead when rotating them.

This allows us to log more data on error but also to let tor recover
gracefully instead of dying.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-03-20 12:54:05 -04:00
Nick Mathewson
9f93bcd16d Remove sb_poll check: all poll() calls are ok. 2018-03-20 08:30:21 -04:00
Nick Mathewson
070eda5a21 Add the poll() syscall as permitted by the sandbox
Apparently, sometimes getpwnam will call this.

Fixes bug 25513.
2018-03-20 08:23:44 -04:00
Nick Mathewson
56ae6d8766 Merge branch 'maint-0.3.3' 2018-03-20 07:50:46 -04:00
Alexander Færøy
fd36bd8971 Log information on specific compression backends in the OOM handler.
This patch adds some additional logging to circuits_handle_oom() to give
us more information about which specific compression backend that is
using a certain amount of memory.

See: https://bugs.torproject.org/25372
2018-03-20 07:47:19 -04:00
Nick Mathewson
228b655935 Move rust-specific declarations outside of #else block
These declarations need to exist unconditionally, but they were
trapped inside an "#else /* !(defined(HAVE_SYSLOG_H)) */" block.

Fixes a travis regression caused by 23881; bug not in any released tor.
2018-03-19 19:18:23 -04:00
Nick Mathewson
d8893bc93c Merge remote-tracking branch 'isis/bug23881_r1' 2018-03-19 17:20:37 -04:00
Nick Mathewson
910422e8fa Merge branch 'maint-0.3.2' into maint-0.3.3 2018-03-19 16:59:49 -04:00
Nick Mathewson
3716611fea Merge branch 'maint-0.3.3' 2018-03-19 16:59:49 -04:00
Isis Lovecruft
66d3120634 tests: Fix HS test against max IP lifetime.
* FIXES part of #25450: https://bugs.torproject.org/25450
2018-03-19 16:59:07 -04:00
Isis Lovecruft
1f8bd93ecb
rust: Fix typo in name of logged function. 2018-03-19 19:23:35 +00:00
Isis Lovecruft
547c62840e
rust: Remove #[no_mangle]s on two constants.
These won't/shouldn't ever be called from C, so there's no reason to
preserve naming.
2018-03-19 19:23:34 +00:00
Neel Chauhan
bc5f79b95c Use tor_asprintf for in have_enough_mem_for_dircache()
(Fixes bug 20887)
2018-03-19 12:38:28 -04:00
Nick Mathewson
b0f0c0f550 Merge remote-tracking branch 'fristonio/ticket-6236' 2018-03-19 06:42:10 -04:00
Nick Mathewson
a324cd9020 Merge branch 'ticket25268_034_01' 2018-03-19 06:01:02 -04:00
Nick Mathewson
92c60b572c Merge branch 'maint-0.3.3' 2018-03-19 05:39:56 -04:00
Gisle Vanem
53914f7dae tests: Fix crash on win32 due to uninitialised mutex in bench.c.
Signed-off-by: Isis Lovecruft <isis@torproject.org>
2018-03-19 05:38:19 -04:00
Nick Mathewson
bcea98a4b4 Merge branch 'maint-0.3.3' 2018-03-19 05:36:06 -04:00
Nick Mathewson
296e429ebc Merge branch 'maint-0.3.2' into maint-0.3.3 2018-03-19 05:36:06 -04:00
Nick Mathewson
a0cc7e9492 Merge remote-tracking branch 'isis/bug25450_032' into maint-0.3.2 2018-03-19 05:35:39 -04:00
Nick Mathewson
5ecad6c95d Extract the cryptographic parts of crypt_path_t and or_circuit_t.
Additionally, this change extracts the functions that created and
freed these elements.

These structures had common "forward&reverse stream&digest"
elements, but they were initialized and freed through cpath objects,
and different parts of the code depended on them.  Now all that code
is extacted, and kept in relay_crypto.c
2018-03-17 10:59:15 -04:00
Nick Mathewson
80955be6ec Move relay-crypto functions from relay.[ch] to relay_crypto.[ch]
This should help us improve modularity, and should also make it
easier for people to experiment with other relay crypto strategies
down the road.

This commit is pure function movement.
2018-03-17 10:23:44 -04:00
Nick Mathewson
320dcf65b7 Extract the crypto parts of circuit_package_relay_cell. 2018-03-17 10:16:41 -04:00
Nick Mathewson
2989326054 Rename 'relay_crypt' to 'relay_decrypt_cell'
This function is used upon receiving a cell, and only handles the
decrypting part.  The encryption part is currently handled inside
circuit_package_relay_cell.
2018-03-17 10:05:25 -04:00
Nick Mathewson
becae4c943 Add a test for geoip_load_file(). 2018-03-15 15:21:34 +01:00
Nick Mathewson
1debe57563 On geoip_free_all, reset geoip[6]_digest. 2018-03-15 15:21:23 +01:00
Nick Mathewson
ffb00404b1 Split geoip tests into a separate module. 2018-03-15 15:12:54 +01:00
Nick Mathewson
4e5e973421 Merge remote-tracking branch 'public/restart_reset_bootstrap' 2018-03-14 12:12:37 +01:00
Nick Mathewson
c6d364e8ae Merge branch 'maint-0.3.3' 2018-03-13 10:59:56 -04:00
Nick Mathewson
d60dc27555 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-03-13 10:59:30 -04:00
Nick Mathewson
950606dcc9 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-03-13 10:58:03 -04:00
Nick Mathewson
38b7885c90 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-03-13 10:58:03 -04:00
Nick Mathewson
0e7f15fdb6 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-03-13 10:58:02 -04:00
Nick Mathewson
67a313f0ec Merge branch 'maint-0.2.5' into maint-0.2.9 2018-03-13 10:58:02 -04:00
Karsten Loesing
3418a3a7f0 Update geoip and geoip6 to the March 8 2018 database. 2018-03-13 10:57:49 -04:00
Nick Mathewson
40154c1f9e Merge branch 'maint-0.3.3' 2018-03-13 10:00:58 +01:00
Nick Mathewson
e9dbd6dd8f Update the documentation in tor_api.h 2018-03-13 10:00:41 +01:00
Isis Lovecruft
0545f64d24
test: Increase time limit for IP creation in an HS test.
This should avoid most intermittent test failures on developer and CI machines,
but there could (and probably should) be a more elegant solution.

Also, this test was testing that the IP was created and its expiration time was
set to a time greater than or equal to `now+INTRO_POINT_LIFETIME_MIN_SECONDS+5`:

    /* Time to expire MUST also be in that range. We add 5 seconds because
     * there could be a gap between setting now and the time taken in
     * service_intro_point_new. On ARM, it can be surprisingly slow... */
    tt_u64_op(ip->time_to_expire, OP_GE,
              now + INTRO_POINT_LIFETIME_MIN_SECONDS + 5);

However, this appears to be a typo, since, according to the comment above it,
adding five seconds was done because the IP creation can be slow on some
systems.  But the five seconds is added to the *minimum* time we're comparing
against, and so it actually functions to make this test *more* likely to fail on
slower systems.  (It should either subtract five seconds, or instead add it to
time_to_expire.)

 * FIXES #25450: https://bugs.torproject.org/25450
2018-03-08 20:50:50 +00:00
Caio Valente
8775c93a99 Refactor: suppress duplicated functions from router.c and encapsulate NODE_DESC_BUF_LEN constant.
Also encapsulates format_node_description().

Closes ticket 25432.
2018-03-06 20:42:32 +01:00
Deepesh Pathak
930b985581
Fix redundant authority certificate fetch
- Fixes #24740
- Fetch certificates only in those cases when consensus are waiting for certs.
2018-03-04 21:13:58 +05:30
Nick Mathewson
699bb803ba Fix a crash bug when testing reachability
Fixes bug 25415; bugfix on 0.3.3.2-alpha.
2018-03-04 10:31:17 -05:00
Nick Mathewson
df9d2de441 Merge remote-tracking branch 'fristonio/ticket4187' 2018-03-03 12:02:30 -05:00
Nick Mathewson
338dbdab93 Merge branch 'maint-0.3.3' 2018-03-03 11:59:27 -05:00
Alexander Færøy
59a7b00384 Update tor.1.txt with the currently available log domains.
See: https://bugs.torproject.org/25378
2018-03-03 11:58:14 -05:00
Nick Mathewson
62482ea279 Merge branch 'maint-0.3.3' 2018-03-03 11:53:05 -05:00
Nick Mathewson
cc7de9ce1d Merge branch 'ticket23814' into maint-0.3.3 2018-03-03 11:53:01 -05:00
Nick Mathewson
aec505a310 bump to 0.3.3.3-alpha-dev 2018-03-03 11:33:56 -05:00
Nick Mathewson
0026d1a673 bump version to 0.3.2.10-dev 2018-03-03 11:33:27 -05:00
Nick Mathewson
0aa794d309 version bump to 0.3.1.10-dev 2018-03-03 11:32:51 -05:00
Nick Mathewson
9eb6f9d3c8 Bump version to 0.2.9.15-dev 2018-03-03 11:32:16 -05:00
Alex Xu (Hello71)
45d3b5fa4c Remove uncompilable tor_mmap_file fallback. #25398 2018-03-02 09:51:53 -05:00
Nick Mathewson
15f6201a5b increment to 0.3.3.3-alpha 2018-03-01 16:44:07 -05:00
Nick Mathewson
1ec386561e version bump to 0.3.2.10 2018-03-01 16:43:35 -05:00
Nick Mathewson
c527a8a9c9 Update to 0.3.1.10 2018-03-01 16:43:01 -05:00
Nick Mathewson
35753c0774 version bump to 0.2.9.15 2018-03-01 16:42:17 -05:00
Nick Mathewson
d22963938f Merge branch 'maint-0.3.3' 2018-03-01 16:10:47 -05:00
Nick Mathewson
f7eff2f8c5 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-03-01 16:10:43 -05:00
Nick Mathewson
d01abb9346 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-03-01 16:07:59 -05:00
Nick Mathewson
d4a758e083 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-03-01 16:07:59 -05:00
Nick Mathewson
c1bb8836ff Protover tests: disable some obsoleted tests
These were meant to demonstrate old behavior, or old rust behavior.

One of them _should_ work in Rust, but won't because of
implementation details.  We'll fix that up later.
2018-03-01 16:05:17 -05:00
Nick Mathewson
c5295cc1be Spec conformance on protover: always reject ranges where lo>hi 2018-03-01 16:05:17 -05:00
Nick Mathewson
1fe0bae508 Forbid UINT32_MAX as a protocol version
The C code and the rust code had different separate integer overflow
bugs here.  That suggests that we're better off just forbidding this
pathological case.

Also, add tests for expected behavior on receiving a bad protocol
list in a consensus.

Fixes another part of 25249.
2018-03-01 16:05:17 -05:00
Nick Mathewson
8b405c609e Forbid "-0" as a protocol version.
Fixes part of 24249; bugfix on 0.2.9.4-alpha.
2018-03-01 16:05:17 -05:00
Nick Mathewson
0953c43c95 Add more of Teor's protover tests.
These are as Teor wrote them; I've disabled the ones that don't pass
yet, with XXXX comments.
2018-03-01 16:05:17 -05:00
Nick Mathewson
d3a1bdbf56 Add some protover vote round-trip tests from Teor.
I've refactored these to be a separate function, to avoid tricky
merge conflicts.

Some of these are disabled with "XXXX" comments; they should get
fixed moving forward.
2018-03-01 16:05:17 -05:00
Nick Mathewson
a83650852d Add another NULL-pointer fix for protover.c.
This one can only be exploited if you can generate a correctly
signed consensus, so it's not as bad as 25074.

Fixes bug 25251; also tracked as TROVE-2018-004.
2018-03-01 16:05:17 -05:00
Nick Mathewson
65f2eec694 Correctly handle NULL returns from parse_protocol_list when voting.
In some cases we had checked for it, but in others we had not.  One
of these cases could have been used to remotely cause
denial-of-service against directory authorities while they attempted
to vote.

Fixes TROVE-2018-001.
2018-03-01 16:05:17 -05:00
Isis Lovecruft
167da4bc81
rust: Remove extra whitespace from a static log/error message. 2018-02-27 20:43:54 +00:00
Deepesh Pathak
130e2ffad7
Remove duplicate code between parse_{c,s}method in transport.c
- Merged common code in function parse_{c,s}method to a single function
- Removed duplicate code in transport.c
- Fixes #6236
2018-02-24 20:27:08 +05:30
Nick Mathewson
fc22bcadb5 Revert 4438ef32's changes to test_address.c
Apparently some versions of the mac sdk care about the ordering of
net/if.h wrt other headers.

Fixes bug 25319; bug not in any released tor.
2018-02-21 09:36:37 -05:00
Isis Lovecruft
7759ac8df2
crypto: Remove crypto_rsa.h from crypto_digest.c.
* ADD include for "crypto_openssl_mgt.h" so that we have OpenSSL
   defined SHA* types and functions.
 * FIXES part of #24658: https://bugs.torproject.org/24658#comment:30
2018-02-20 20:29:54 +00:00
Isis Lovecruft
3e9140e79a
crypto: Remove unnecessary curve25519 header from crypto_digest.h.
* ADD includes for "torint.h" and "container.h" to crypto_digest.h.
 * ADD includes for "crypto_digest.h" to a couple places in which
   crypto_digest_t was then missing.
 * FIXES part of #24658: https://bugs.torproject.org/24658#comment:30
2018-02-20 20:29:54 +00:00
Nick Mathewson
4438ef3288 Remove a bunch of other redundant #includes
Folks have found two in the past week or so; we may as well fix the
others.

Found with:

\#!/usr/bin/python3
import re

def findMulti(fname):
    includes = set()
    with open(fname) as f:
        for line in f:
            m = re.match(r'^\s*#\s*include\s+["<](\S+)[>"]', line)
            if m:
                inc = m.group(1)
                if inc in includes:
                    print("{}: {}".format(fname, inc))
                includes.add(m.group(1))

import sys
for fname in sys.argv[1:]:
    findMulti(fname)
2018-02-20 10:14:15 -05:00
Nick Mathewson
a4ab273a0d Merge remote-tracking branch 'fristonio/ticket-25261' 2018-02-20 10:03:52 -05:00
Nick Mathewson
5199b9b337 Use autoconf to check for optional zstd functionality.
Fixes a bug in our zstd-static code.  Bug not in any released
version of Tor.
2018-02-18 16:19:43 -05:00
Fernando Fernandez Mancera
0fad49e1c4 Move crypto_pk_obsolete_* functions into RSA module.
We moved the crypto_pk_obselete_* functions into crypto_rsa.[ch] because they fit
better with the RSA module.

Follows #24658.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
2018-02-16 18:23:21 +01:00
Fernando Fernandez Mancera
541b6b2433 Remove useless included files in crypto_rsa.[ch].
Follows #24658.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
2018-02-16 17:49:58 +01:00
Nick Mathewson
bd71e0a0c8 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-02-16 09:54:13 -05:00
Nick Mathewson
2bcd264a28 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-02-16 09:48:11 -05:00
Nick Mathewson
cb92d47dec Merge remote-tracking branch 'dgoulet/ticket24902_029_05' into maint-0.2.9 2018-02-16 09:41:06 -05:00
Nick Mathewson
bbc73c5d1c Whoops. 256 was not big enough. 2018-02-16 09:40:29 -05:00
Nick Mathewson
a34fc1dad2 Allow checkpointing of non-sha1 digests.
This is necessary because apparently v3 rendezvous cpath hops use
sha3, which I had forgotten.

Bugfix on master; bug not in any released Tor.
2018-02-16 09:25:50 -05:00
Roger Dingledine
d21e5cfc24 stop calling channel_mark_client in response to a create_fast
since all it does is produce false positives

this commit should get merged into 0.2.9 and 0.3.0 *and* 0.3.1, even
though the code in the previous commit is already present in 0.3.1. sorry
for the mess.

[Cherry-picked]
2018-02-16 08:46:57 -05:00
Roger Dingledine
2b99350ca4 stop calling channel_mark_client in response to a create_fast
since all it does is produce false positives

this commit should get merged into 0.2.9 and 0.3.0 *and* 0.3.1, even
though the code in the previous commit is already present in 0.3.1. sorry
for the mess.
2018-02-16 08:46:31 -05:00
Roger Dingledine
8d5dcdbda2 backport to make channel_is_client() accurate
This commit takes a piece of commit af8cadf3a9 and a piece of commit
46fe353f25, with the goal of making channel_is_client() be based on what
sort of connection handshake the other side used, rather than seeing
whether the other side ever sent a create_fast cell to us.
2018-02-16 08:39:10 -05:00
Neel Chauhan
c2fa743806 Remove the return value of node_get_prim_orport() and node_get_prim_dirport() 2018-02-16 08:20:33 -05:00
Nick Mathewson
200fc8c640 Compilation workaround for windows, which lacks O_SYNC
Bug not in any released Tor.
2018-02-16 08:16:12 -05:00
Nick Mathewson
5a9ada342f tor_zstd_format_version shouldn't be built when !HAVE_ZSTD
Fixes bug 25276; bugfix not in any released Tor.
2018-02-16 08:06:01 -05:00
Fernando Fernandez Mancera
f9f0dd5b9a Move the pk-digest functions into crypto_rsa.[ch].
We moved the crypto_pk_* digest functions into crypto_rsa.[ch] because they fit
better with the RSA module.

Follows #24658.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
2018-02-16 12:04:22 +01:00
Nick Mathewson
b56fd17d00 Merge branch 'maint-0.3.3' 2018-02-15 21:05:12 -05:00
Nick Mathewson
d662d4470a Merge remote-tracking branch 'dgoulet/ticket24343_033_01' into maint-0.3.3 2018-02-15 21:05:08 -05:00
Nick Mathewson
d9804691df Merge remote-tracking branch 'ffmancera-1/bug18918' 2018-02-15 21:00:10 -05:00
Nick Mathewson
92a42f795c Merge branch 'bug23909' 2018-02-15 20:56:23 -05:00
Nick Mathewson
8da6bfa5de Merge branch 'bug24914' 2018-02-15 20:53:50 -05:00
Nick Mathewson
4d994e7a9c Fix a stack-protector warning: don't use a variable-length buffer
Instead, define a maximum size, and enforce it with an assertion.
2018-02-15 20:52:01 -05:00
Nick Mathewson
ed1d630f0e Merge branch 'onion_ntor_malloc_less' 2018-02-15 20:40:03 -05:00
Nick Mathewson
28c3f538e5 Documentation fixes suggested by catalyst. 2018-02-15 20:38:08 -05:00
Nick Mathewson
bda1dfb9e0 Merge remote-tracking branch 'isis/bug25185' 2018-02-15 20:35:30 -05:00
Nick Mathewson
acb7a536c2 Merge branch 'maint-0.3.3' 2018-02-15 20:33:00 -05:00
Nick Mathewson
799c82be70 Merge remote-tracking branch 'isis/bug25171' into maint-0.3.3 2018-02-15 20:32:57 -05:00
Nick Mathewson
a1dd8afc16 Merge branch '25162_zstd_static' 2018-02-15 20:28:07 -05:00
Nick Mathewson
066a15af63 This stats_n_seconds_working variable needs to be static now.
(When a variable isn't going to be declared extern in the header, we
require that it be static.)
2018-02-15 20:26:09 -05:00
Nick Mathewson
3ca04aada2 Merge remote-tracking branch 'valentecaio/t-25081' 2018-02-15 20:23:23 -05:00
Nick Mathewson
3d7bf98d13 Merge remote-tracking branch 'valentecaio/t-24714' 2018-02-15 20:19:53 -05:00
Isis Lovecruft
5f7d78ce2a
tests: Remove duplicate included header file in src/test/test.c.
* FIXES #25271: https://bugs.torproject.org/25271
2018-02-16 01:19:12 +00:00
Nick Mathewson
3c8a481599 Merge branch 'bug18105' 2018-02-15 20:17:31 -05:00