Commit Graph

15121 Commits

Author SHA1 Message Date
Nick Mathewson
780d1b44cf Move responsibility for recording read/written bytes
Previously this was done as part of the refill callback, but there's
no real reason to do it like that.  Since we're trying to remove the
refill callback completely, we can do this work as part of
record_num_bytes_transferred_impl(), which already does quite a lot
of this.
2018-04-17 18:06:46 -04:00
Nick Mathewson
a2acb9b9e9 Refill each token bucket at the last instant before reading/writing.
(This patch does not yet eliminate the global refill callback;
fortunately, bucket refilling is idempotent.)
2018-04-17 12:20:06 -04:00
Nick Mathewson
9af4cd6f31 Refactor responsibility for checking global write bucket emptiness
We used to do this 10x per second in connection_buckets_refill();
instead, we now do it when the bucket becomes empty. This change is
part of the work of making connection_buckets_refill() obsolete.

Closes ticket 25828; bugfix on 0.2.3.5-alpha.
2018-04-17 12:12:07 -04:00
Nick Mathewson
1356d51af6 Rename connection_bucket_refill to connection_bucket_refill_all
Also document its actual behavior
2018-04-17 11:47:31 -04:00
Nick Mathewson
993f5d284d Rename connection_bucket_round_robin -> get_share
There was nothing round_robinish about this function.
2018-04-17 11:42:14 -04:00
Nick Mathewson
488e2b00bf Refactor the "block the connection on bandwidth" logic
Right now, this patch just introduces and exposes some new
functions. Later, these functions will get a little more complexity.
2018-04-17 11:39:16 -04:00
Nick Mathewson
c5bbf72fb8 Merge branch 'maint-0.3.3' 2018-04-17 10:45:58 -04:00
David Goulet
93ff1870ba heartbeat: Log the number of circuits killed because too many cells
We recently merged a circuit cell queue size safeguard. This commit adds the
number of killed circuits that have reached the limit to the DoS heartbeat. It
now looks like this:

  [notice] DoS mitigation since startup: 0 circuits killed with too many
  cells. 0 circuits rejected, 0 marked addresses. 0 connections closed. 0
  single hop clients refused.

Second thing that this patch does. It makes tor always print the DoS
mitigation heartbeat line (for a relay) even though no DoS mitigation have
been enabled. The reason is because we now kill circuits that have too many
cells regardless on if it is enabled or not but also it will give the operator
a chance to learn what is enabled with the heartbeat instead of suddenly
appearing when it is enabled by let say the consensus.

Fixes #25824

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-17 10:44:43 -04:00
Neel Chauhan
e72742d693 Add GETINFO current-time/{local,utc} command to ControlPort 2018-04-16 20:37:50 -04:00
Mike Perry
f921fd771a Use u32 add helper for CIRC_BW accounting.
There are quite a few other places this could be used, but keeping it simple
for now.
2018-04-16 21:46:31 +00:00
Mike Perry
dfa6808f57 Bug 25400: Make CIRC_BW event properly total everything on a circ. 2018-04-16 21:46:12 +00:00
Nick Mathewson
3463b4e065 Merge branch 'maint-0.3.3' 2018-04-16 10:06:21 -04:00
Nick Mathewson
22845df2a7 Merge remote-tracking branch 'dgoulet/bug25226_033_02' into maint-0.3.3 2018-04-16 10:04:36 -04:00
David Goulet
d064122e70 relay: Implement a circuit cell queue maximum size
This commit introduces the consensus parameter "circ_max_cell_queue_size"
which controls the maximum number of cells a circuit queue should have.

The default value is currently 50000 cells which is above what should be
expected but keeps us a margin of error for padding cells.

Related to this is #9072. Back in 0.2.4.14-alpha, we've removed that limit due
to a Guard discovery attack. Ticket #25226 details why we are putting back the
limit due to the memory pressure issue on relays.

Fixes #25226

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-16 09:59:12 -04:00
Nick Mathewson
4b58b97c68 32-bit compilation warnings 2018-04-13 17:01:03 -04:00
Nick Mathewson
34c2574aa9 Merge branch 'token_bucket_once_again_squashed' 2018-04-13 16:31:58 -04:00
Nick Mathewson
c9de30c590 Merge branch 'maint-0.3.3' 2018-04-13 13:05:53 -04:00
Nick Mathewson
61d87dfa15 Merge branch 'postloop_callbacks_2' 2018-04-13 12:12:46 -04:00
Nick Mathewson
4c03af4880 Remove tell_event_loop_to_run_external_code() per review
(This function is no longer used.)
2018-04-13 12:11:22 -04:00
Nick Mathewson
03b96882de Rename token_bucket_t to token_bucket_rw_t.
This is a simple search-and-replace to rename the token bucket type
to indicate that it contains both a read and a write bucket, bundled
with their configuration.  It's preliminary to refactoring the
bucket type.
2018-04-13 10:54:26 -04:00
Nick Mathewson
b152d62cee Merge branch 'token_bucket_refactor_squashed' 2018-04-13 10:47:24 -04:00
Nick Mathewson
320bd2b3a5 Move connection_ap_attach_pending(0) into a postloop event
This is a second motivating case for our postloop event logic.
2018-04-13 10:44:15 -04:00
Nick Mathewson
5719dfb48f Move the "activate linked connections" logic to a postloop event.
A linked connection_t is one that gets its I/O, not from the
network, but from another connection_t.  When such a connection has
something to write, we want the corresponding connection to run its
read callback ... but not immediately, to avoid infinite recursion
and/or event loop starvation.

Previously we handled this case by activating the read events
outside the event loop.  Now we use the "postloop event" logic.
This lets us simplify do_main_loop_once() a little.
2018-04-13 10:44:15 -04:00
Nick Mathewson
12f58f2f87 Remove a bunch of int casts; make clang happier. 2018-04-13 10:41:14 -04:00
Nick Mathewson
6be994fa71 Ensure that global buckets are updated on configuration change 2018-04-13 10:41:14 -04:00
Nick Mathewson
a38fd9bc5b Replace the global buckets with token_bucket_t 2018-04-13 10:41:14 -04:00
Nick Mathewson
9fced56ef1 Refactor or_connection token buckets to use token_bucket_t 2018-04-13 10:41:14 -04:00
Nick Mathewson
2d6914e391 Refine extend_info_for_node's "enough info" check once again.
In d1874b4339, we adjusted this check so that we insist on
using routerinfos for bridges.  That's almost correct... but if we
have a bridge that is also a regular relay, then we should use
insist on its routerinfo when connecting to it as a bridge
(directly), and be willing to use its microdescriptor when
connecting to it elsewhere in our circuits.

This bug is a likely cause of some (all?) of the (exit_ei == NULL)
failures we've been seeing.

Fixes bug 25691; bugfix on 0.3.3.4-alpha
2018-04-12 16:56:29 -04:00
Nick Mathewson
467c882baa Merge branch 'maint-0.3.3' 2018-04-12 12:25:51 -04:00
Nick Mathewson
4aaa4215e7 Attempt to fix 32-bit builds, which broke with 31508a0abc
When size_t is 32 bits, doing "size_t ram; if (ram > 8GB) { ... }"
produces a compile-time warning.

Bug caused by #24782 fix; not in any released Tor.
2018-04-12 12:25:09 -04:00
Nick Mathewson
037fb0c804 Merge branch 'maint-0.3.3' 2018-04-12 11:14:42 -04:00
Alexander Færøy
31508a0abc Use less memory for MaxMemInQueues for machines with more than 8 GB of RAM.
This patch changes the algorithm of compute_real_max_mem_in_queues() to
use 0.4 * RAM iff the system has more than or equal to 8 GB of RAM, but
will continue to use the old value of 0.75 * RAM if the system have less
than * GB of RAM available.

This patch also adds tests for compute_real_max_mem_in_queues().

See: https://bugs.torproject.org/24782
2018-04-12 11:14:16 -04:00
Alexander Færøy
5633a63379 Use STATIC for compute_real_max_mem_in_queues
This patch makes compute_real_max_mem_in_queues use the STATIC macro,
which allows us to test the function.

See: https://bugs.torproject.org/24782
2018-04-12 10:51:48 -04:00
Nick Mathewson
a51630cc9a Merge branch 'maint-0.3.3' 2018-04-11 15:38:00 -04:00
Nick Mathewson
0803d79f55 Merge branch 'bug25581_033_v2_asn_squashed' into maint-0.3.3 2018-04-11 15:37:56 -04:00
Nick Mathewson
8b8630a501 Rename HSLayer{2,3}Nodes to start without an underscore.
The old single-underscore names remain as a deprecated synonym.

Fixes bug 25581; bugfix on 0.3.3.1-alpha.
2018-04-11 15:37:49 -04:00
Nick Mathewson
0c8f901ee7 Merge branch 'maint-0.3.3' 2018-04-11 10:48:46 -04:00
Mike Perry
f9ba0c6546 Bug 24989: Count client hsdir gets towards MaxClientCircuitsPending.
We removed this by breaking them out from general in #13837.
2018-04-11 10:47:06 -04:00
Nick Mathewson
6bdfaa8b24 Merge remote-tracking branch 'isis-github/bug25425_squashed2' 2018-04-10 15:32:26 -04:00
Nick Mathewson
6e467a7a34 Merge remote-tracking branch 'isis-github/bug25409' 2018-04-10 15:27:09 -04:00
Isis Lovecruft
65d6b66e99
config: Obsolete PortForwarding and PortForwardingHelper options.
* FIXES part of #25409: https://bugs.torproject.org/25409
2018-04-10 19:08:59 +00:00
Nick Mathewson
386f8016b7 Fix another crash-on-no-threadpool bug.
This one happens if for some reason you start with DirPort enabled
but server mode turned off entirely.

Fixes a case of bug 23693; bugfix on 0.3.1.1-alpha.
2018-04-10 14:44:38 -04:00
Nick Mathewson
d3ac47b415 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-04-10 14:26:32 -04:00
Nick Mathewson
0b1a054d68 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-10 14:26:32 -04:00
Nick Mathewson
ef16a11b90 Merge branch 'maint-0.3.3' 2018-04-10 14:26:32 -04:00
Nick Mathewson
db6902c235 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-04-10 14:26:29 -04:00
Nick Mathewson
10a1969ca3 Merge remote-tracking branch 'ahf-github/bugs/24854_029_2' into maint-0.2.9 2018-04-10 14:25:57 -04:00
Nick Mathewson
16f08de0fd Remove TestingEnableTbEmptyEvent
This option was used for shadow testing previously, but is no longer
used for anything.  It interferes with refactoring our token buckets.
2018-04-10 12:16:21 -04:00
Isis Lovecruft
c2c5b13e5d
test: Add testing module and some unittests for bridges.c.
This roughly doubles our test coverage of the bridges.c module.

 * ADD new testing module, .../src/test/test_bridges.c.
 * CHANGE a few function declarations from `static` to `STATIC`.
 * CHANGE one function in transports.c, transport_get_by_name(), to be
   mockable.
 * CLOSES #25425: https://bugs.torproject.org/25425
2018-04-09 19:32:46 +00:00
Nick Mathewson
e58555135a Add a comment explaining why we do a certain redundant check
Closes ticket 25291.
2018-04-09 12:58:17 -04:00
Alexander Færøy
1295044dc8 Lift the list of default directory servers into their own file.
This patch lifts the list of default directory authorities from config.c
into their own auth_dirs.inc file, which is then included in config.c
using the C preprocessor.

Patch by beastr0.

See: https://bugs.torproject.org/24854
2018-04-09 16:00:26 +02:00
Nick Mathewson
98b694bfd5 Merge branch 'isolate_libevent_2_squashed' 2018-04-06 08:50:35 -04:00
Roger Dingledine
0b0e4886cf fix confusing comment
presumably introduced by copy-and-paste mistake
2018-04-05 15:59:37 -04:00
Nick Mathewson
6a5f62f68f Move responsibility for threadpool reply-handler events to workqueue
This change makes cpuworker and test_workqueue no longer need to
include event2/event.h.  Now workqueue.c needs to include it, but
that is at least somewhat logical here.
2018-04-05 12:36:28 -04:00
Nick Mathewson
b3586629c9 Wrap the function we use to run the event loop.
Doing this lets us remove the event2/event.h header from a few more
modules, particularly in the tests.

Part of work on 23750.
2018-04-05 12:36:27 -04:00
Nick Mathewson
39cb04335f Add wrappers for event_base_loopexit and event_base_loopbreak. 2018-04-05 12:36:05 -04:00
Nick Mathewson
871ff0006d Add an API for a scheduled/manually activated event in the mainloop
Using this API lets us remove event2/event.h usage from half a dozen
modules, to better isolate libevent.  Implements part of ticket
23750.
2018-04-05 12:35:11 -04:00
Nick Mathewson
c6d7e0becf Merge remote-tracking branch 'public/split_relay_crypto' 2018-04-05 12:12:18 -04:00
Neel Chauhan
f5f9c25546 Switch to use should_record_bridge_info()
Both in geoip_note_client_seen() and options_need_geoip_info(), switch from
accessing the options directly to using the should_record_bridge_info() helper
function.

Fixes #25290

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-05 08:37:39 -04:00
David Goulet
3d5bf12ac2 relay: Remove max middle cells dead code
Next commit is addressing the circuit queue cell limit so cleanup before doing
anything else.

Part of #25226

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-04 11:03:50 -04:00
Nick Mathewson
52846f728d Merge branch 'maint-0.3.3' 2018-04-04 08:57:24 -04:00
Nick Mathewson
ec8ee54129 Merge branch 'bug21394_029_redux' into maint-0.3.3 2018-04-04 08:55:37 -04:00
Dhalgren
06484eb5e1 Bug 21394 touchup: Increase DNS attempts to 3
Also don't give up on a resolver as quickly if multiple are configured.
2018-04-04 08:54:25 -04:00
Neel Chauhan
9df110cd72
Remove PortForwarding options
Signed-off-by: Isis Lovecruft <isis@torproject.org>
2018-04-04 00:19:33 +00:00
Nick Mathewson
218b1746ba Merge remote-tracking branch 'fristonio/ticket-25645' 2018-04-03 19:19:02 -04:00
Deepesh Pathak
2680a8b5b1
ticket(25645): remove unused variable n_possible from channel_get_for_extend() 2018-04-03 09:17:23 +05:30
Isis Lovecruft
ad369313f8
protover: Change protover_all_supported() to return only unsupported.
Previously, if "Link=1-5" was supported, and you asked protover_all_supported()
(or protover::all_supported() in Rust) if it supported "Link=3-999", the C
version would return "Link=3-999" and the Rust would return "Link=6-999".  These
both behave the same now, i.e. both return "Link=6-999".
2018-04-02 19:59:12 +00:00
Isis Lovecruft
6e353664dd
protover: Change protover_all_supported() to return only unsupported.
Previously, if "Link=1-5" was supported, and you asked protover_all_supported()
(or protover::all_supported() in Rust) if it supported "Link=3-999", the C
version would return "Link=3-999" and the Rust would return "Link=6-999".  These
both behave the same now, i.e. both return "Link=6-999".
2018-04-02 19:20:36 +00:00
Nick Mathewson
f9e32a2084 Remove an unnecessary event2 include.
The rest, are, unfortunately, necessary for now.
2018-04-02 11:11:34 -04:00
Nick Mathewson
3df9545492 Merge branch 'maint-0.3.3' 2018-04-02 08:51:47 -04:00
Roger Dingledine
6190593256 use channel_is_client for create cell counts
When a relay is collecting internal statistics about how many
create cell requests it has seen of each type, accurately count the
requests from relays that temporarily fall out of the consensus.

(To be extra conservative, we were already ignoring requests from clients
in our counts, and we continue ignoring them here.)

Fixes bug 24910; bugfix on 0.2.4.17-rc.
2018-04-02 01:00:31 -04:00
Roger Dingledine
961d2ad597 dir auths no longer vote Guard if they're not voting V2Dir
Directory authorities no longer vote in favor of the Guard flag
for relays that don't advertise directory support.

Starting in Tor 0.3.0.1-alpha, Tor clients have been avoiding using
such relays in the Guard position, leading to increasingly broken load
balancing for the 5%-or-so of Guards that don't advertise directory
support.

Fixes bug 22310; bugfix on 0.3.0.6.
2018-04-02 00:20:01 -04:00
Roger Dingledine
0983c203e5 misc tiny fixes 2018-04-01 23:47:44 -04:00
Taylor Yu
596eed3715 Fix CID 1433643
Add a missing lock acquisition around access to queued_control_events
in control_free_all().  Use the reassign-and-unlock strategy as in
queued_events_flush_all().  Fixes bug 25675.  Coverity found this bug,
but only after we recently added an access to
flush_queued_event_pending.
2018-03-29 17:21:33 -05:00
Nick Mathewson
2c1afc2def Merge branch 'maint-0.3.3' 2018-03-29 14:59:01 -04:00
Nick Mathewson
4f473fadbd Merge branch 'bug25617_029' into maint-0.3.3 2018-03-29 14:58:58 -04:00
Nick Mathewson
e35eb9baaa Mark controller-initiated DNS lookups as permitted to do DNS.
Fixes bug 25617; bugfix on 0.2.9.3-alpha.
2018-03-29 09:27:28 -04:00
Nick Mathewson
4c0e434f33 Merge remote-tracking branch 'public/bug25512' 2018-03-28 14:45:47 -04:00
Nick Mathewson
cb083b5d3e Merge remote-tracking branch 'asn-github/t-25432' 2018-03-28 14:43:33 -04:00
Nick Mathewson
794a25f8c1 Merge branch 'maint-0.3.3' 2018-03-28 14:23:06 -04:00
Nick Mathewson
e0bbef48bf Merge remote-tracking branch 'dgoulet/bug24767_033_03' into maint-0.3.3 2018-03-28 14:21:04 -04:00
Nick Mathewson
6317aa2cc0 Merge branch 'maint-0.3.3' 2018-03-28 07:50:47 -04:00
rl1987
0e453929d2 Allow IPv6 address strings to be used as hostnames in SOCKS5 requests 2018-03-28 07:39:03 -04:00
Nick Mathewson
fa6eaab83e Merge branch 'maint-0.3.3' 2018-03-27 18:25:52 -04:00
Nick Mathewson
5acfc30876 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-03-27 18:23:53 -04:00
Taylor Yu
471f28a2a8 Fix CID 1430932
Coverity found a null pointer reference in nodelist_add_microdesc().
This is almost certainly impossible assuming that the routerstatus_t
returned by router_get_consensus_status_by_descriptor_digest() always
corresponds to an entry in the nodelist.  Fixes bug 25629.
2018-03-27 16:11:29 -05:00
Taylor Yu
0c13a84c0d Fix CID 1430932
Coverity found a null pointer reference in nodelist_add_microdesc().
This is almost certainly impossible assuming that the routerstatus_t
returned by router_get_consensus_status_by_descriptor_digest() always
corresponds to an entry in the nodelist.  Fixes bug 25629.
2018-03-27 16:08:39 -05:00
Taylor Yu
4bb7d9fd12 Fix CID 1430932
Coverity found a null pointer reference in nodelist_add_microdesc().
This is almost certainly impossible assuming that the routerstatus_t
returned by router_get_consensus_status_by_descriptor_digest() always
corresponds to an entry in the nodelist.  Fixes bug 25629.
2018-03-27 15:29:00 -05:00
George Kadianakis
ab16f1e2a1 test: Add unittest for the OR connection failure cache
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-03-27 12:43:39 -04:00
David Goulet
f29d158330 relay: Avoid connecting to down relays
If we failed to connect at the TCP level to a relay, note it down and refuse
to connect again for another 60 seconds.

Fixes #24767

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-03-27 12:42:31 -04:00
Nick Mathewson
979c7e5c83 Merge branch 'maint-0.3.3' 2018-03-27 07:05:15 -04:00
Nick Mathewson
46c2b0ca22 Merge branch 'bug25213_033' into maint-0.3.3 2018-03-27 07:04:33 -04:00
Nick Mathewson
0eed0899cd Merge branch 'bug24658-rm-curve25519-header' into bug24658-merge 2018-03-26 20:12:59 -04:00
Nick Mathewson
d96dc2060a Merge branch 'maint-0.3.3' 2018-03-26 19:47:48 -04:00
Nick Mathewson
841ed9dbb9 Merge remote-tracking branch 'dgoulet/bug24904_033_01' into maint-0.3.3 2018-03-26 19:47:44 -04:00
Nick Mathewson
5278d72f97 Merge branch 'maint-0.3.3' 2018-03-26 10:32:53 -04:00
Nick Mathewson
ca2d9cbb93 Merge branch 'bug24903_029' into maint-0.3.3 2018-03-26 10:32:49 -04:00
Nick Mathewson
d1874b4339 Make extend_info_from_node() more picky about node contents
This update is needed to make it consistent with the behavior of
node_awaiting_ipv6(), which doesn't believe in the addresses from
routerinfos unless it actually plans to use those routerinfos.

Fixes bug 25213; bugfix on b66b62fb75 in 0.3.3.1-alpha,
which tightened up the definition of node_awaiting_ipv6().
2018-03-26 09:56:12 -04:00
Nick Mathewson
33a80921a2 When extending a circuit's path length, clear onehop_tunnel.
There was a nonfatal assertion in pathbias_should_count that would
trigger if onehop_tunnel was set, but the desired_path_length was
greater than 1.  This patch fixes that.  Fixes bug 24903; bugfix on
0.2.5.2-alpha.
2018-03-26 09:17:50 -04:00
Nick Mathewson
a9fa483004 Document a requirement for cells to be encrypted. 2018-03-24 13:49:44 -04:00