Commit Graph

14922 Commits

Author SHA1 Message Date
Nick Mathewson
6892d32921 Add an address_set to the nodelist.
This set is rebuilt whenever a consensus arrives.  In between
consensuses, it is add-only.
2018-02-08 14:40:05 -05:00
David Goulet
211fe44e07 dirserv: Improve returned message when relay is rejected
Explicitly inform the operator of the rejected relay to set a valid email
address in the ContactInfo field and contact bad-relays@ mailing list.

Fixes #25170

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-08 09:42:26 -05:00
David Goulet
652d3a5b66 Remove anything related to the old SocksSockets option
At this commit, the SocksSocketsGroupWritable option is renamed to
UnixSocksGroupWritable. A deprecated warning is triggered if the old option is
used and tor will use it properly.

Fixes #24343

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-07 14:05:33 -05:00
Roger Dingledine
bf91da75ab remove blank line between function-comment and function
also be more consistent about punctuation in doxygen comments
2018-02-07 12:51:05 -05:00
Nick Mathewson
12b58ba551 Merge remote-tracking branch 'dgoulet/ticket25163_033_01' 2018-02-07 12:46:27 -05:00
Roger Dingledine
a7440d9c9d more fixes for typos, grammar, whitespace, etc
some of these ought to have been noticed by the "misspell" tool,
so if anybody is debugging it, here are some bug reports :)
2018-02-07 12:22:29 -05:00
Nick Mathewson
86498e5aa5 Fix wide lines from typo-fix patch. 2018-02-07 10:46:05 -05:00
Deepesh Pathak
ca6682f3f8 Fix spelling mistakes corresponding to ticket #23650 2018-02-07 10:41:57 -05:00
George Kadianakis
13f5adc86c Improve doc of primary_guards_up_to_date. 2018-02-07 11:46:30 +02:00
David Goulet
93ebcc2b8f rephist: Stop tracking relay connection status
Remove a series of connection counters that were only used when dumping the
rephist statistics with SIGUSR1 signal.

This reduces the or_history_t structure size.

Closes #25163

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-06 12:56:36 -05:00
David Goulet
199bc37290 rephist: Stop tracking EXTEND attempts
This removes the code that tracks the extend attemps a client makes. We don't
use it and it was only used to provide statistics on a SIGUSR1 from the
rephist dump stats function.

Part of #25163

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-06 12:43:55 -05:00
Nick Mathewson
6961609a1c Merge remote-tracking branch 'dgoulet/bug25116_029_01' 2018-02-06 12:13:41 -05:00
Nick Mathewson
87db5a6b75 Merge remote-tracking branch 'arma/bug22212' 2018-02-06 11:36:13 -05:00
Nick Mathewson
b5a8fd1566 Merge remote-tracking branch 'dgoulet/ticket24902_029_05' 2018-02-05 14:00:50 -05:00
David Goulet
7ce8d5513b Make circuit_log_ancient_one_hop_circuits() ignore established service rendezvous
Services can keep rendezvous circuits for a while so don't log them if tor is
a single onion service.

Fixes #25116

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-05 13:46:43 -05:00
David Goulet
f08fa97460 geoip: Make geoip_client_cache_total_allocation() return the counter
The HT_FOREACH() is insanely heavy on the CPU and this is part of the fast
path so make it return the nice memory size counter we added in
4d812e29b9.

Fixes #25148

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-05 13:38:55 -05:00
Nick Mathewson
f0d7905bc9 Merge remote-tracking branch 'dgoulet/ticket24902_029_05' 2018-02-05 11:38:58 -05:00
David Goulet
78d6cb5870 dos: We can put less token than the current amount
Becasue the circuit creation burst and rate can change at runtime it is
possible that between two refill of a bucket, we end up setting the bucket
value to less than there currently is.

Fixes #25128

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-02 17:04:12 -05:00
Nick Mathewson
eafa252b26 Merge remote-tracking branch 'dgoulet/ticket24902_029_05' 2018-02-02 15:00:35 -05:00
David Goulet
475218c108 Merge branch 'ticket25122_029_02' into ticket24902_029_05 2018-02-02 14:55:01 -05:00
David Goulet
e758d659a0 geoip: Add clientmap_entry_new() function
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-02 14:48:41 -05:00
David Goulet
4d812e29b9 geoip: Increment and decrement functions for the geoip client cache
These functions protect againts over and underflow. They BUG() in case we
overflow the counter.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-02 14:48:41 -05:00
David Goulet
51839f4765 geoip: Hook the client history cache into the OOM handler
If the cache is using 20% of our maximum allowed memory, clean 10% of it. Same
behavior as the HS descriptor cache.

Closes #25122

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-02 14:48:41 -05:00
Nick Mathewson
9e48338a12 Merge branch 'maint-0.3.2' 2018-02-02 12:03:54 -05:00
David Goulet
005e228f80 sched: When releasing a channel, do not BUG() if absent from the pending list
The current code flow makes it that we can release a channel in a PENDING
state but not in the pending list. This happens while the channel is being
processed in the scheduler loop.

Fixes #25125

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-02 12:03:27 -05:00
Nick Mathewson
77634795b0 Merge remote-tracking branch 'dgoulet/bug24700_032_01' into maint-0.3.2 2018-02-01 16:57:57 -05:00
David Goulet
e1a40535ea Merge branch 'bug24700_032_01' into bug24700_033_01 2018-02-01 16:39:04 -05:00
Nick Mathewson
cb5654f300 sched: Use the sched_heap_idx field to double-check our fix for 24700.
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-02-01 16:00:59 -05:00
Nick Mathewson
31542cc306 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-02-01 15:28:17 -05:00
Nick Mathewson
9773cd0f94 Merge branch 'maint-0.3.2' 2018-02-01 15:28:17 -05:00
Nick Mathewson
9cbc40e376 Merge remote-tracking branch 'teor/bug25070_031' into maint-0.3.1 2018-02-01 15:28:11 -05:00
Nick Mathewson
c2757c3774 Remove nodelist_recompute_all_hsdir_indices() as unused.
Closes 25108.
2018-02-01 08:44:47 -05:00
Nick Mathewson
88b146cda5 Merge remote-tracking branch 'dgoulet/bug24469_033_01' 2018-02-01 08:22:44 -05:00
Nick Mathewson
ca85d66217 Merge branch 'maint-0.3.2' 2018-02-01 08:15:09 -05:00
Nick Mathewson
61cb2993dd Merge remote-tracking branch 'dgoulet/bug24975_032_01' into maint-0.3.2 2018-02-01 08:10:34 -05:00
Nick Mathewson
d1c2597096 Merge remote-tracking branch 'dgoulet/ticket24902_029_05' 2018-01-31 16:17:04 -05:00
Fernando Fernandez Mancera
33d9889a2b channel_tls_get_remote_addr_method now returns real_addr.
The accurate address of a connection is real_addr, not the addr member.
channel_tls_get_remote_addr_method() now returns real_addr instead.

Fixes #24952; bugfix on 707c1e2 in 0.2.4.11-alpha.

Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
2018-01-31 16:12:13 -05:00
David Goulet
fb93c6fc51 circ: Don't cannibalize a circuit if the guard state is unusable
Tor preemptiely builds circuits and they can be cannibalized later in their
lifetime. A Guard node can become unusable (from our guard state) but we can
still have circuits using that node opened. It is important to not pick those
circuits for any usage through the cannibalization process.

Fixes #24469

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-31 16:10:48 -05:00
Nick Mathewson
946ebd8419 Obsolete the now-unused MaxTries options. 2018-01-31 15:11:47 -05:00
Nick Mathewson
c0024edd26 Remove two vestigial MaxDownloadTries checks from directory.c
These are no longer meaningful, since there's no longer an upper
limit to how many times (in the exponential-backoff world) one can
retry a download.  download_status_is_ready() didn't check these any
more, and neither do we.
2018-01-31 15:08:46 -05:00
Nick Mathewson
b8ff7407a7 remove the max_failures argument from download_status_is_ready. 2018-01-31 15:03:47 -05:00
Nick Mathewson
a846fd267e Merge branch 'bug23954_squashed' 2018-01-31 14:37:48 -05:00
Nick Mathewson
da778f2921 Use thread-safe types to store the LOG_PROTOCOL_WARN severity
Fixes a race condition; resolves 23954.
2018-01-31 14:37:09 -05:00
Nick Mathewson
98dd3757bf Merge branch 'bug25008' 2018-01-31 14:32:24 -05:00
David Goulet
fbc455cbd2 ns: Add a before and after consensus has changed notification
In 0.3.2.1-alpha, we've added notify_networkstatus_changed() in order to have
a way to notify other subsystems that the consensus just changed. The old and
new consensus are passed to it.

Before this patch, this was done _before_ the new consensus was set globally
(thus NOT accessible by getting the latest consensus). The scheduler
notification was assuming that it was set and select_scheduler() is looking at
the latest consensus to get the parameters it might needs. This was very wrong
because at that point it is still the old consensus set globally.

This commit changes the notify_networkstatus_changed() to be the "before"
function and adds an "after" notification from which the scheduler subsystem
is notified.

Fixes #24975
2018-01-31 14:15:02 -05:00
Nick Mathewson
31f2a8771c Look at the correct protocol for supports_v3_rendezvous_point
Fixes bug 25105; bugfix on 0.3.2.1-alpha.

(This is a backport of bbf2d9cf6b for 0.3.2.)
2018-01-31 14:09:47 -05:00
Nick Mathewson
3d937043c2 Fix a failing unit test.
When we stopped looking at the "protocols" variable directly, we
broke the hs_service/build_update_descriptors test, since it didn't
actually update any of the flags.

The fix here is to call summarize_protover_flags() from that test,
and to expose summarize_protover_flags() as "STATIC" from
routerparse.c.
2018-01-31 14:06:37 -05:00
Nick Mathewson
bbf2d9cf6b Look at the correct protocol for supports_v3_rendezvous_point
Fixes bug 25105; bugfix on 0.3.2.1-alpha.
2018-01-31 14:01:49 -05:00
David Goulet
c85f78e74c Revert "ns: Call notify_networkstatus_changed() after the new consensus is set globally"
This reverts commit 3a247ca92a.
2018-01-31 13:59:05 -05:00
Nick Mathewson
144bf015f8 Document remaining cases for protocol support
For each support flag, document which subprotocol version it requires.
2018-01-31 13:50:04 -05:00
Nick Mathewson
0dc1595d03 Merge branch 'maint-0.3.2' 2018-01-31 13:47:01 -05:00
Nick Mathewson
1c39d969b9 Merge remote-tracking branch 'dgoulet/bug24975_032_01' into maint-0.3.2 2018-01-31 13:46:58 -05:00
David Goulet
adaf3e9b89 sched: Avoid adding the same channel twice to the KIST pending list
This is the quick fix that is keeping the channel in PENDING state so if we
ever try to reschedule the same channel, it won't happened.

Fixes #24700

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-31 13:46:31 -05:00
Nick Mathewson
2294e330bd Merge branch 'maint-0.3.2' 2018-01-31 12:51:45 -05:00
David Goulet
df312b3cf6 hs-v3: Remove a BUG() when storing a descriptor in the client cache
It is possible in normal circumstances that  a client fetches a descriptor
that has a lower revision counter than the one in its cache. This can happen
due to HSDir desync.

Fixes #24976

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-31 12:51:42 -05:00
Mike Perry
ac1a78b977 Bug 24769: Reduce and parameterize the max number of cbt circs.
Setting the default for this at 10 and the learning timeout to 3 minutes means
we will complete our cbt learning in 30 minutes, which is under the reduced
padding connection timeout window.
2018-01-31 17:21:53 +00:00
Mike Perry
148c2410af Bug 24769: Reduce and parameterize the cbt learning idle timeout.
This is only half of the changes needed. We should also parameterize the
number of concurrent cbt learning circuits in needs_circuits_for_build().
2018-01-31 17:21:53 +00:00
David Goulet
3a247ca92a ns: Call notify_networkstatus_changed() after the new consensus is set globally
In 0.3.2.1-alpha, we've added this function in order to have a way to notify
other subsystems that the consensus just changed. The old consensus and the
new one are passed to it.

Before this patch, this was done _before_ the new consensus was set globally
(thus NOT accessible by getting the latest consensus). The scheduler
notification was assuming that it was set and select_scheduler() is looking at
the latest consensus to get the parameters it might needs. This was very wrong
because at that point it is still the old consensus set globally.

With this commit, notify_networkstatus_changed() has been moved _after_ the
new consensus is set globally. The main obvious reasons is to fix the bug
described above and in #24975. The other reason is that this notify function
doesn't return anything which could be allowing the possibility of refusing to
set the new consensus on error. In other words, the new consensus is set right
after the notification whatever happens.

It does no harm or change in behavior to set the new consensus first and then
notify the subsystems. The two functions currently used are for the control
port using the old and new consensus and sending the diff. The second is the
scheduler that needs the new consensus to be set globally before being called.

Of course, the function has been documented accordinly to clearly state it is
done _after_ the new consensus is set.

Fixes #24975

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-31 11:21:18 -05:00
Nick Mathewson
1e81aaa62f Merge branch 'maint-0.3.2' 2018-01-31 10:06:49 -05:00
Nick Mathewson
9bfb6fe395 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-01-31 10:06:49 -05:00
Nick Mathewson
5fc0437e74 But in most Earth cultures, there are 60s in a minute. 2018-01-31 10:06:43 -05:00
Nick Mathewson
8b162443b9 Merge branch 'maint-0.3.2' 2018-01-31 10:01:13 -05:00
Nick Mathewson
cb90defba6 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-01-31 10:01:13 -05:00
Nick Mathewson
86e6cb6409 add a rate-limit. 2018-01-31 10:01:10 -05:00
Nick Mathewson
69e242f845 Merge branch 'maint-0.3.2' 2018-01-31 09:50:24 -05:00
Nick Mathewson
e81896adda Merge branch 'maint-0.3.1' into maint-0.3.2 2018-01-31 09:50:24 -05:00
Nick Mathewson
914ec372a9 Merge branch 'bug24927' 2018-01-31 09:47:36 -05:00
Nick Mathewson
80c8689be1 Merge remote-tracking branch 'public/ticket24849_032' 2018-01-31 09:38:24 -05:00
Nick Mathewson
94878cf1ea Merge remote-tracking branch 'dgoulet/ticket24902_029_05' 2018-01-31 09:35:07 -05:00
teor
a09d5f5735 dos: Make sure cc_stats_refill_bucket can't overflow while calculating
Debug log the elapsed time in cc_stats_refill_bucket

Part of #25094.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-31 09:27:59 -05:00
Nick Mathewson
c0447033f5 Merge branch 'maint-0.3.2' 2018-01-31 09:19:55 -05:00
Roger Dingledine
3d9dcb49eb count flushing as channel activity
Stop adding unneeded channel padding right after we finish flushing
to a connection that has been trying to flush for many seconds.
Instead, treat all partial or complete flushes as activity on the
channel, which will defer the time until we need to add padding.

This fix should resolve confusing and scary log messages like
"Channel padding timeout scheduled 221453ms in the past."

Fixes bug 22212; bugfix on 0.3.1.1-alpha.

I think technically we could resolve bug 22212 by adding a call to
channel_timestamp_active() only in the finished_flushing case. But I added
a call in the flushed_some case too since that seems to more accurately
reflect the notion of "active".
2018-01-31 05:26:06 -05:00
David Goulet
cd81403cc0 Merge branch 'ticket24902_029_05' into ticket24902_033_02 2018-01-30 09:33:12 -05:00
David Goulet
e58a4fc6cf dos: Make circuit rate limit per second, not tenths anymore
Because this touches too many commits at once, it is made into one single
commit.

Remove the use of "tenths" for the circuit rate to simplify things. We can
only refill the buckets at best once every second because of the use of
approx_time() and our token system is set to be 1 token = 1 circuit so make
the rate a flat integer of circuit per second.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:16 -05:00
George Kadianakis
c3c2b55dec test: Add unit tests for the DoS subsystem
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:16 -05:00
David Goulet
82de4ea900 dos: Clear connection tracked flag if geoip entry is removed
Imagine this scenario. We had 10 connections over the 24h lifetime of a geoip
cache entry. The lifetime of the entry has been reached so it is about to get
freed but 2 connections remain for it. After the free, a third connection
comes in thus making us create a new geoip entry for that address matching the
2 previous ones that are still alive. If they end up being closed, we'll have
a concurrent count desynch from what the reality is.

To mitigate this probably very rare scenario in practice, when we free a geoip
entry and it has a concurrent count above 0, we'll go over all connections
matching the address and clear out the tracked flag. So once they are closed,
we don't try to decrement the count.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:16 -05:00
David Goulet
14a8b87852 dos: Add a heartbeat log
Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:16 -05:00
David Goulet
36a0ae151f dos: Add the DoSRefuseSingleHopClientRendezvous option
This option refuses any ESTABLISH_RENDEZVOUS cell arriving from a client
connection. Its default value is "auto" for which we can turn it on or off
with a consensus parameter. Default value is 0.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:16 -05:00
David Goulet
acf7ea77d8 dos: Add the connection DoS mitigation subsystem
Defend against an address that has reached the concurrent connection count
threshold.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:15 -05:00
David Goulet
1bfc91a029 dos: Apply defense for circuit creation DoS
If the client address was detected as malicious, apply a defense which is at
this commit to return a DESTROY cell.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:15 -05:00
David Goulet
97abb3543b dos: Detect circuit creation denial of service
Add a function that notifies the DoS subsystem that a new CREATE cell has
arrived. The statistics are updated accordingly and the IP address can also be
marked as malicious if it is above threshold.

At this commit, no defense is applied, just detection with a circuit creation
token bucket system.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:15 -05:00
David Goulet
c05272783d dos: Track new and closed OR client connections
Implement a basic connection tracking that counts the number of concurrent
connections when they open and close.

This commit also adds the circuit creation mitigation data structure that will
be needed at later commit to keep track of the circuit rate.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:15 -05:00
David Goulet
51fda85c23 geoip: Remember client stats if DoS mitigation is enabled
Make the geoip cache track client address if the DoS subsystem is enabled.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:15 -05:00
David Goulet
64149353dd dos: Initial code of Denial of Service mitigation
This commit introduces the src/or/dos.{c|h} files that contains the code for
the Denial of Service mitigation subsystem. It currently contains basic
functions to initialize and free the subsystem. They are used at this commit.

The torrc options and consensus parameters are defined at this commit and
getters are implemented.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-30 09:18:15 -05:00
teor
a8e5e3a492
Add Link protocol version 5 to the supported protocols list in protover.c
Part of #25070, bugfix on 0.3.1.1-alpha.
2018-01-30 01:56:50 +11:00
Nick Mathewson
75d4bd3497 Improve log when unable to add sigs to pending consensus
Closes ticket 24849.
2018-01-26 14:19:59 -05:00
Nick Mathewson
ee5c624beb When a tor_cert_T check fails, log the reason why.
Diagnostic attempt for 24972.
2018-01-26 13:55:25 -05:00
Nick Mathewson
0755bcc36a Remove a needless (always-true) check.
Also add an assertion and rename a variable.

Closes ticekt 24927.
2018-01-26 13:35:00 -05:00
Nick Mathewson
9c2bc441f8 If out-of-disk when saving a consensus cache entry, don't BUG.
Just warn instead.

Fixes bug 24859.
2018-01-26 13:14:14 -05:00
Nick Mathewson
5b55e15707 Remove all the old max_delay logic.
We had tests for it, but it was always INT_MAX.
2018-01-25 16:05:20 -05:00
Nick Mathewson
bf74194f57 fixup! Remove the old ("deterministic") download schedule.
Un-indent a block.

I'm doing this as a separate fixup commit to make review simpler.
2018-01-25 15:52:33 -05:00
Nick Mathewson
e0049ef022 Remove the old ("deterministic") download schedule.
We haven't meant to use it since we introduced the random
exponential schedule.

Closes ticket 23814.
2018-01-25 15:51:13 -05:00
David Goulet
93b826faaa geoip: Add a lookup function for client map entry
The upcoming DoS mitigation subsytem needs to keep information on a per-IP
basis which is also what the geoip clientmap does.

For another subsystem to access that clientmap, this commit adds a lookup
function that returns the entry. For this, the clientmap_entry_t had to be
moved to the header file.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-01-25 15:44:48 -05:00
Nick Mathewson
7e504515b3 Always look at the subprotocol versions summary flags
Previously, we wouldn't do this when running with a routerinfo_t in
some cases, leading to many needless calls to the protover module.

This change also cleans up the code in nodelist.c a bit.

Fixes bug 25008; bugfix on 0.2.9.4-alpha.
2018-01-24 13:53:56 -05:00
Nick Mathewson
92496a739a Also cache the protover summary in the routerinfo_t, if we're using that 2018-01-24 13:53:56 -05:00
Nick Mathewson
7792be2d44 Extract code to summarize protocol versions into new function
This will let us put this summary into routerinfo_t too.

No behavior change.
2018-01-24 13:53:55 -05:00
Nick Mathewson
d9fbd34f42 Extract protover summary flags into a new structure
This will let us use them on routerinfo_t as well as on
routerstatus_t, and save some time on relays.

No behavioral changes here.
2018-01-24 13:53:55 -05:00
Nick Mathewson
6f4ee6e5e7 Merge remote-tracking branch 'mikeperry/bug24946' 2018-01-23 14:08:47 -05:00
George Kadianakis
17daab76b8 Add onion service activity information to our heartbeat logs. 2018-01-23 12:31:06 +02:00
Mike Perry
db5b670d85 Bug 24946: Fix a warning message caused by a missed purpose check.
Also fix three other checks (found by inspection of all
CIRCUIT_PURPOSE_C_GENERAL occurrences).
2018-01-20 03:18:31 +00:00