Commit Graph

20993 Commits

Author SHA1 Message Date
Nick Mathewson
6182f60f75 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-24 08:51:55 -04:00
Nick Mathewson
d2951b381b Merge branch 'maint-0.3.1' into maint-0.3.2 2018-04-24 08:49:24 -04:00
Nick Mathewson
e888634076 Merge remote-tracking branch 'public/bug23693_031_redux' into maint-0.3.1 2018-04-24 08:49:20 -04:00
David Goulet
b259008c56 hs: Fix memleak in v3 on SIGHUP
Fixes #25901

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 11:09:57 -04:00
Nick Mathewson
192c7c8bf9 Merge remote-tracking branch 'dgoulet/ticket25762_034_05' 2018-04-23 11:02:05 -04:00
David Goulet
665e23c59a test: Add periodic events unit tests
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:57:28 -04:00
David Goulet
87cb9ce900 main: Update periodic events comment based on latest code
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:57:28 -04:00
Nick Mathewson
f70fa67da6 main: Use rescan_periodic_events in initialize_periodic_events_cb 2018-04-23 10:57:28 -04:00
David Goulet
4e85f17eec periodic: Add an enable and disable function
Two helper functions to enable an event and disable an event which wraps the
launch and destroy of an event but takes care of the enabled flag.

They are also idempotent that is can be called multiple time on the same event
without effect if the event was already enabled or disabled.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:57:28 -04:00
David Goulet
1d864987cb config: Set up periodic events when options changes
In case we transitionned to a new role in Tor, we need to launch and/or
destroy some periodic events.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:57:28 -04:00
David Goulet
a4fcdc5dec main: Launch periodic events by roles
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:57:28 -04:00
David Goulet
ed89bb3253 main: Specialize the periodic events on a per-role basis
In tor, we have a series of possible "roles" that the tor daemon can be
enabled for. They are:

  Client, Bridge, Relay, Authority (directory or bridge) and Onion service.

They can be combined sometimes. For instance, a Directory Authority is also a
Relay. This adds a "roles" field to a periodic event item object which is used
to know for which roles the event is for.

The next step is to enable the event only if the roles apply. No behavior
change at this commit.

Pars of #25762

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:21:59 -04:00
David Goulet
269cd5dba7 main: Sort alphabetically periodic event callbacks
No behavior change, just to make it easier to find callbacks and for the sake
of our human brain to parse the list properly.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-23 10:21:59 -04:00
Nick Mathewson
3527f4b8a4 Merge remote-tracking branch 'github/lazy_bucket_refill' 2018-04-23 09:47:05 -04:00
Nick Mathewson
e8683bcbb1 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-23 09:24:33 -04:00
Nick Mathewson
1438c6c713 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-04-23 09:23:31 -04:00
Nick Mathewson
5c3639923f Merge branch 'maint-0.2.9' into maint-0.3.1 2018-04-23 09:23:31 -04:00
Nick Mathewson
c5ffcbb43f Merge branch 'maint-0.3.3' 2018-04-23 09:23:31 -04:00
Nick Mathewson
beb321d8cd Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-23 09:23:31 -04:00
Nick Mathewson
c4be6dfeab Permit the nanosleep system call in the seccomp2 callbox
Fixes bug 24969; bugfix on 0.2.5.1-alpha when the sandbox was introduced.
2018-04-23 09:15:40 -04:00
Nick Mathewson
cd3fc2aa48 Merge remote-tracking branch 'neel/b25511-r4' 2018-04-23 09:13:24 -04:00
Nick Mathewson
915791bc48 Merge branch 'ticket25024_squashed' 2018-04-22 20:53:04 -04:00
Deepesh Pathak
72bfcb37f1 add changes file and update check-typos in makefile 2018-04-22 20:48:48 -04:00
Nick Mathewson
1ba9b7e013 Merge remote-tracking branch 'mikeperry/bug25400_squashed' 2018-04-22 20:39:35 -04:00
Nick Mathewson
0e8ae82a87 Merge branch 'remove_old_consensus_methods_2018_squashed' 2018-04-22 20:01:09 -04:00
Nick Mathewson
2609a8be81 Require MIN_METHOD_FOR_RECOMMENDED_PROTOCOLS
(Remove support for running without this method.)
2018-04-22 20:00:47 -04:00
Nick Mathewson
5f90d28c01 Remove MIN_METHOD_FOR_{SHARED_RANDOM,EXCLUDING_INVALID_NODES}
Also remove client detection for pre-EXCLUDING_INVALID_NODES
consensuses, and a test for that detection.
2018-04-22 20:00:47 -04:00
Nick Mathewson
73c9c16faa Remove MIN_METHOD_FOR_ED25519_ID_VOTING
This also lets us remove the old rsa-based routerstatus collator.
2018-04-22 20:00:47 -04:00
Nick Mathewson
4d6f21bb6b Remove MIN_METHOD_FOR_{PACKAGE_LINES,GUARDFRACTION,ED25519_ID_IN_MD}
Also remove a rest for pre-19 microdesc versions.
2018-04-22 20:00:47 -04:00
Nick Mathewson
caf766991d Remove MIN_METHOD_FOR_ID_HASH_IN_MD and a test for running without it. 2018-04-22 20:00:47 -04:00
Nick Mathewson
93380db833 Remove MIN_METHOD_{FOR_P6_LINES,FOR_NTOR_KEY,TO_CLIP_UNMEASURED_BW}
Also remove a unit test for pre-MIN_METHOD_FOR_NTOR_KEY consensuses.
2018-04-22 20:00:47 -04:00
Nick Mathewson
08373467b1 Remove MIN_METHOD_FOR and MIN_METHOD_FOR_A_LINES
Also, in networkstatus.c, remove client code for recognizing pre-
MIN_METHOD_FOR_A_LINES consensuses, and corresponding unit tests in
test_dir.c.
2018-04-22 20:00:47 -04:00
Nick Mathewson
69347f48e0 Disable consensus methods before 25.
Consensus method 25 is the oldest one supported by any stable
version of 0.2.9, which is our current most-recent LTS.  Thus, by
proposal 290, they should be removed.

This commit does not actually remove the code to implement these
methods: it only makes it so authorities will no longer support
them.  I'll remove the backend code for them in later commits.
2018-04-22 20:00:47 -04:00
Nick Mathewson
6773102c92 Merge branch 'bug25691_033_again_squashed' 2018-04-22 19:44:27 -04:00
Nick Mathewson
0941c8bfe6 control EXTENDCIRCUIT: check node_has_preferred_descriptor().
Suggested by teor during code review for 25691.
2018-04-22 19:43:27 -04:00
Nick Mathewson
f1c1328f85 Repair the legacy_hs/pick_tor2web_rendezvous_node unit test
It tried to pick nodes for which only routerinfo_t items are set,
but without setting UseMicroDescriptors to 0.  This won't work any
more, now that we're strict about using the right descriptor types
due to 25691/25692/25213.
2018-04-22 19:43:07 -04:00
Nick Mathewson
47163780c3 Rename node_has_descriptor() to node_has_any_descriptor()
Changing the name of this function should help keep us from misusing
it when node_has_preferred_descriptor() would be more appropriate.
2018-04-22 19:43:07 -04:00
Nick Mathewson
7915efd1b8 Use router_crn_flags in more places, to pass direct-connect flag
In order to fix 25691 and 25692, we need to pass the "direct_conn"
flag to more places -- particularly when choosing single-hop
tunnels.  The right way to do this involves having a couple more
functions accept router_crn_flags_t, rather than a big list of
boolean arguments.

This commit also makes sure that choose_good_exit_server_general()
honors the direct_conn flag, to fix 25691 and 25692.
2018-04-22 19:42:28 -04:00
Nick Mathewson
388d217c40 Use node_has_preferred_descriptor() in another case
In router_add_running_nodes_to_smartlist(), we had an inline
implementation of the logic from node_has_descriptor(), which should
be changed to node_has_preferred_descriptor().
2018-04-22 19:42:28 -04:00
Nick Mathewson
948dd2c79e Check for "the right descriptor", not just "any descriptor".
This patch adds a new node_has_preferred_descriptor() function, and
replaces most users of node_has_descriptor() with it.  That's an
important change, since as of d1874b4339 (our fix for #25213),
we are willing to say that a node has _some_ descriptor, but not the
_right_ descriptor for a particular use case.

Part of a fix for 25691 and 25692.
2018-04-22 19:42:26 -04:00
Nick Mathewson
c3e40a8361 Allow cpuworkers to exist without onion keys
Now that we allow cpuworkers for dirport-only hosts (to fix 23693),
we need to allow dup_onion_keys() to succeed for them.

The change to construct_ntor_key_map() is for correctness,
but is not strictly necessary.
2018-04-22 17:17:07 -04:00
Nick Mathewson
5102208447 Improve documentation for CONFIG_TYPE_CSV_INTERVAL. 2018-04-22 15:55:09 -04:00
Nick Mathewson
8b6fc47cc3 Fix an absurdly wide line. 2018-04-22 15:55:09 -04:00
Nick Mathewson
ff6f49f033 Rename find_dl_schedule to find_dl_min_delay.
(We no longer need two separate functions here.)
2018-04-22 15:55:09 -04:00
Nick Mathewson
82d1d8b071 Remove extra values from InitialDelay defaults
These options are now ignored.
2018-04-22 15:55:09 -04:00
Nick Mathewson
2f792c041a Add aliases for the old DownloadSchedule options
These will produce a warning, but still work fine.
2018-04-22 15:55:09 -04:00
Nick Mathewson
2d7b5c6fe5 Change the type of "download schedule" from smartlist to int.
This is done as follows:
  * Only one function (find_dl_schedule()) actually returned a
    smartlist. Now it returns an int.

  * The CSV_INTERVAL type has been altered to ignore everything
    after the first comma, and to store the value before the first
    comma in an int.
2018-04-22 15:55:09 -04:00
Nick Mathewson
9aaed729c1 Rename *DownloadSchedule to *DownloadInitialDelay; make them ints
This commit won't compile. It was made with the following perl
scripts:

s/smartlist_t \*(.*)DownloadSchedule;/int $1DownloadInitialDelay;/;
s/\b(\w*)DownloadSchedule\b/$1DownloadInitialDelay/;
2018-04-22 15:55:09 -04:00
Nick Mathewson
9d27e3f014 Make test_tortls.c build with openssl no_deprecated.
Also for 19981.
2018-04-18 12:32:39 -04:00
Nick Mathewson
a15b2c57e1 Add support for openssl built with "no-deprecated".
Patch from Andrew John Hughes; partial fix for 19981.
2018-04-18 12:31:24 -04:00
Nick Mathewson
bd3f8260a3 Rename some functions to start with a uniform prefix 2018-04-18 11:45:44 -04:00
Nick Mathewson
3bf9974b6c Fix a pointer size error in test_bridges.c
sizeof(ret) is the size of the pointer, not the size of what it
points to.  Fortunately, we already have a function to compare
tor_addr_port_t values for equality.

Bugfix on c2c5b13e5d8a77e; bug not in any released Tor. Found by
clang's scan-build.
2018-04-17 19:45:59 -04:00
Nick Mathewson
1abe0a5769 Add an initialization case to node_get_prim_dirport
Fixes a bug found by scan-build; bugfix on c2fa743806. Bug not in
any released Tor.
2018-04-17 19:43:14 -04:00
Nick Mathewson
31a450a5b6 Add a redundant memset to node_get_pref_ipv6_orport()
For whatever reason, clang's scan-build isn't sure that this
function actually initializes its output.
2018-04-17 19:41:10 -04:00
Nick Mathewson
d67d3dd145 Fix a copy-and-paste error from 6be994fa71
Found by clang's scan-build too.  Bug not in any released Tor.
2018-04-17 19:39:50 -04:00
Nick Mathewson
087ace7009 Fix a compilation warning on clang 2018-04-17 18:41:39 -04:00
Nick Mathewson
47df912f1c Remove the periodic refill event entirely.
Now that we update our buckets on demand before reading or writing,
we no longer need to update them all every TokenBucketRefillInterval
msec.

When a connection runs out of bandwidth, we do need a way to
reenable it, however.  We do this by scheduling a timer to reenable
all blocked connections for TokenBucketRefillInterval msec after a
connection becomes blocked.

(If we were using PerConnBWRate more, it might make sense to have a
per-connection timer, rather than a single timeout. But since
PerConnBWRate is currently (mostly) unused, I'm going to go for the
simpler approach here, since usually whenever one connection has
become blocked on bandwidth, most connections are blocked on
bandwidth.)

Implements ticket 25373.
2018-04-17 18:20:03 -04:00
Nick Mathewson
780d1b44cf Move responsibility for recording read/written bytes
Previously this was done as part of the refill callback, but there's
no real reason to do it like that.  Since we're trying to remove the
refill callback completely, we can do this work as part of
record_num_bytes_transferred_impl(), which already does quite a lot
of this.
2018-04-17 18:06:46 -04:00
Nick Mathewson
31fbbf2377 Fixup timing wheel warnings related to recent WHEEL_BIT change. 2018-04-17 12:45:53 -04:00
Nick Mathewson
a2acb9b9e9 Refill each token bucket at the last instant before reading/writing.
(This patch does not yet eliminate the global refill callback;
fortunately, bucket refilling is idempotent.)
2018-04-17 12:20:06 -04:00
Nick Mathewson
9af4cd6f31 Refactor responsibility for checking global write bucket emptiness
We used to do this 10x per second in connection_buckets_refill();
instead, we now do it when the bucket becomes empty. This change is
part of the work of making connection_buckets_refill() obsolete.

Closes ticket 25828; bugfix on 0.2.3.5-alpha.
2018-04-17 12:12:07 -04:00
Nick Mathewson
b36c450b57 Amend token_bucket_rw_dec to indicate which buckets became empty. 2018-04-17 12:02:49 -04:00
Nick Mathewson
1356d51af6 Rename connection_bucket_refill to connection_bucket_refill_all
Also document its actual behavior
2018-04-17 11:47:31 -04:00
Nick Mathewson
993f5d284d Rename connection_bucket_round_robin -> get_share
There was nothing round_robinish about this function.
2018-04-17 11:42:14 -04:00
Nick Mathewson
488e2b00bf Refactor the "block the connection on bandwidth" logic
Right now, this patch just introduces and exposes some new
functions. Later, these functions will get a little more complexity.
2018-04-17 11:39:16 -04:00
Nick Mathewson
2bf6f1cd39 token bucket: Add parens to rate_per_sec_to_rate_per_step()
Typecasts bind more tightly than division, so we need to do the
division first.
2018-04-17 11:09:55 -04:00
Nick Mathewson
c5bbf72fb8 Merge branch 'maint-0.3.3' 2018-04-17 10:45:58 -04:00
David Goulet
93ff1870ba heartbeat: Log the number of circuits killed because too many cells
We recently merged a circuit cell queue size safeguard. This commit adds the
number of killed circuits that have reached the limit to the DoS heartbeat. It
now looks like this:

  [notice] DoS mitigation since startup: 0 circuits killed with too many
  cells. 0 circuits rejected, 0 marked addresses. 0 connections closed. 0
  single hop clients refused.

Second thing that this patch does. It makes tor always print the DoS
mitigation heartbeat line (for a relay) even though no DoS mitigation have
been enabled. The reason is because we now kill circuits that have too many
cells regardless on if it is enabled or not but also it will give the operator
a chance to learn what is enabled with the heartbeat instead of suddenly
appearing when it is enabled by let say the consensus.

Fixes #25824

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-17 10:44:43 -04:00
Nick Mathewson
c32108ee0f Merge remote-tracking branch 'public/bug24688' 2018-04-17 09:14:44 -04:00
Neel Chauhan
3a6e37f57f Add GETINFO current-time/{local,utc} regression test 2018-04-16 20:37:50 -04:00
Neel Chauhan
ce84de39ef Make tor_gettimeofday() mockable 2018-04-16 20:37:50 -04:00
Neel Chauhan
e72742d693 Add GETINFO current-time/{local,utc} command to ControlPort 2018-04-16 20:37:50 -04:00
Neel Chauhan
9e3e1b8bfb Add format_local_iso_time_nospace() 2018-04-16 20:37:50 -04:00
Mike Perry
f921fd771a Use u32 add helper for CIRC_BW accounting.
There are quite a few other places this could be used, but keeping it simple
for now.
2018-04-16 21:46:31 +00:00
Mike Perry
0e06a9c3e7 Helper function to add u32 without overflow. 2018-04-16 21:46:31 +00:00
Mike Perry
dfa6808f57 Bug 25400: Make CIRC_BW event properly total everything on a circ. 2018-04-16 21:46:12 +00:00
David Goulet
ae4e5b9824 token: Fix uint32_t to uint64_t conversion
Unfortunately, the units passed to
monotime_coarse_stamp_units_to_approx_msec() was always 0 due to a type
conversion.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-16 15:05:41 -04:00
juga0
8be1ac8abe
Add test to check that loading a 2nd file replaces the 1st
Signed-off-by: Isis Lovecruft <isis@torproject.org>
2018-04-16 19:02:57 +00:00
juga0
071236e3e2
Add a test for geoip_load_file() using geoip6
Signed-off-by: Isis Lovecruft <isis@torproject.org>
2018-04-16 19:01:30 +00:00
Nick Mathewson
c7d3de216c Merge branch 'maint-0.3.3' 2018-04-16 13:48:39 -04:00
Nick Mathewson
c5899d5cf3 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-04-16 13:48:23 -04:00
Nick Mathewson
5e0fbd7006 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-04-16 13:48:23 -04:00
Nick Mathewson
9ef4c05df8 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-16 13:48:23 -04:00
Nick Mathewson
0e13ff4815 Fix an LCOV exclusion pattern in address.c 2018-04-16 13:48:21 -04:00
Nick Mathewson
3463b4e065 Merge branch 'maint-0.3.3' 2018-04-16 10:06:21 -04:00
Nick Mathewson
22845df2a7 Merge remote-tracking branch 'dgoulet/bug25226_033_02' into maint-0.3.3 2018-04-16 10:04:36 -04:00
David Goulet
d064122e70 relay: Implement a circuit cell queue maximum size
This commit introduces the consensus parameter "circ_max_cell_queue_size"
which controls the maximum number of cells a circuit queue should have.

The default value is currently 50000 cells which is above what should be
expected but keeps us a margin of error for padding cells.

Related to this is #9072. Back in 0.2.4.14-alpha, we've removed that limit due
to a Guard discovery attack. Ticket #25226 details why we are putting back the
limit due to the memory pressure issue on relays.

Fixes #25226

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-16 09:59:12 -04:00
Nick Mathewson
197d1992db Remove old tor-fw-helper README from EXTRA_DIST
We removed this file, but didn't take it out of EXTRA_DIST -- thus
breaking "make dist".
2018-04-16 09:52:15 -04:00
Nick Mathewson
3ee4c9b1fa bump to 0.3.3.5-rc-dev 2018-04-15 15:41:03 -04:00
Nick Mathewson
b65024f57d bump to 0.3.3.5-rc 2018-04-14 12:21:36 -04:00
Nick Mathewson
4b58b97c68 32-bit compilation warnings 2018-04-13 17:01:03 -04:00
David Goulet
c2f83746f4 token_bucket: Fix indentation
Both header and code file had some indentation issues after mass renaming.

No code behavior change.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-13 16:58:49 -04:00
Nick Mathewson
34c2574aa9 Merge branch 'token_bucket_once_again_squashed' 2018-04-13 16:31:58 -04:00
Nick Mathewson
1b31195b4f Fix "make check-spaces" 2018-04-13 16:31:47 -04:00
Nick Mathewson
003e6595bf Refactor "timestamp" not to be its own type coupled to token buffers
Really, the uint32_t is only an optimization; any kind of unit
should work fine.  Some users might want to use time_t or
monotime_coarse_t or something like that.
2018-04-13 16:31:47 -04:00
Nick Mathewson
2307bef7a2 Move token_bucket_raw_* functions to the start of the module.
(These functions were previously helper functions for
token_bucket_rw_t).
2018-04-13 16:31:47 -04:00
Nick Mathewson
9c405ba595 Never pick a rate of 0.
(The tests caught this one.)
2018-04-13 16:31:47 -04:00
Nick Mathewson
0b40ed5e70 Start re-refactoring the token bucket interface.
Begin by creating a lowest-level triple of the types needed to
implement a token bucket: a configuration, a timestamp, and the raw
bucket itself.

Note that for low-level buckets, the units of the timestamp and the
bucket itself are unspecified: each user can use a different type.

(This patch breaks check-spaces; a later patch will fix it)
2018-04-13 16:31:47 -04:00
Nick Mathewson
c9de30c590 Merge branch 'maint-0.3.3' 2018-04-13 13:05:53 -04:00
Nick Mathewson
61d87dfa15 Merge branch 'postloop_callbacks_2' 2018-04-13 12:12:46 -04:00