Proposal 289 prevents SENDME-flooding by requiring the other side to
authenticate the data it has received. But this data won't actually
be random if they are downloading a known resource. "No problem",
we said, "let's fell the empty parts of our cells with some
randomness!" and we did that in #26871.
Unfortunately, if the relay data payloads are all completely full,
there won't be any empty parts for us to randomize.
Therefore, we now pick random "randomness windows" between
CIRCWINDOW_INCREMENT/2 and CIRCWINDOW_INCREMENT. We remember whether we have
sent a cell containing at least 16 bytes of randomness in that window. If we
haven't, then when the window is exhausted, we send one. (This window approach
is designed to lower the number of rng checks we have to do. The number 16 is
pulled out of a hat to change the attacker's guessing difficulty to
"impossible".)
Implements 28646.
Turns out that we were only recording the "b_digest" but to have
bidirectionnal authenticated SENDMEs, we need to use the "f_digest" in the
forward cell situation.
Because of the cpath refactoring, this commit plays with the crypt_path_ and
relay_crypto_t API a little bit in order to respect the abstractions.
Previously, we would record the cell digest as the SENDME digest in the
decrypt cell function but to avoid code duplication (both directions needs to
record), we now do that right after iff the cell is recognized (at the edge).
It is now done in circuit_receive_relay_cell() instead.
We now also record the cell digest as the SENDME digest in both relay cell
encryption functions since they are split depending on the direction.
relay_encrypt_cell_outbound() and relay_encrypt_cell_inbound() need to
consider recording the cell digest depending on their direction (f vs b
digest).
Fixes#30428
Signed-off-by: David Goulet <dgoulet@torproject.org>
It turns out that only the exit side is validating the authenticated SENDME v1
logic and never the client side. Which means that if a client ever uploaded
data towards an exit, the authenticated SENDME logic wouldn't apply.
For this to work, we have to record the cell digest client side as well which
introduced a new function that supports both type of edges.
This also removes a test that is not valid anymore which was that we didn't
allow cell recording on an origin circuit (client).
Part of #30428
Signed-off-by: David Goulet <dgoulet@torproject.org>
Because github PRs choose the most recent origin/master at the time of the PR
(and for any fixups pushed to a PR later to send to CI), there are tons of
conflicts and unexpected practracker issues.
This means CI can suddenly fail after fixups to a branch that pass locally.
Then CI fails and we have to close and re-open the PR.
Split the core reply formatting code out of control_fmt.c into
control_proto.c. The remaining code in control_format.c deals with
specific subsystems and will eventually move to join those subsystems.
(This should be all of the command that work nicely with positional
arguments only.)
Some of these commands should probably treat extra arguments as
incorrect, but for now I'm trying to be careful not to break
any existing users.
The first line break in particular was mishandled: it was discarded
if no arguments came before it, which made it impossible to
distinguish arguments from the first line of the body.
To solve this, we need to allocate a copy of the command rather than
using NUL to separate it, since we might have "COMMAND\n" as our input.
Fixes ticket 29984.
Previously we excluded any line containing one of these terms from
consideration as the start or end of a function. Now we're more
careful, and we only ignore these terms when they appear to be
starting a function definition.
This was causing issues because the exceptions file is written using Posix
paths, whereas practracker in Windows was trying to match Windows paths ("\"
instead of "/").
- Introduce 'make check-best-practices'.
- Fix up Tor topdir etc to work with the way 'make check-local' gets called.
- Make practracker less likely to print useless stuff.
- every .h file needs an #ifndef/#define pair.
- They must refer to the same macro.
- The guard macros that they refer to must be unique across all headers.
Replace the "git pull" by one single "git fetch origin" and then "git merge"
into each defined branches.
This speeds up the process considerably.
Closes#29616
Signed-off-by: David Goulet <dgoulet@torproject.org>
No longer messes up with MOCK_IMPL. Also update the tests to show that. We are
still being innacurate on the line count in some cases, but that doesnt matter
so much.
If a relay matches at least one fingerprint, IPv4 address, or IPv6
address in the fallback whitelist, it can become a fallback. This
reduces the work required to keep the list up to date.
Closes ticket 28768.
Tor clients on 0.3.5.6-rc? and later will use a consensus that will become
valid up to 24 hours in the future.
Clients on 0.3.5.5-alpha? and earlier won't accept future consensuses.
Update the fallback expiry tolerance to match tor's checks.
Part of 28768, follow-up on 28591.
Tor clients will use a consensus that expired up to 24 hours ago.
Clients on 0.3.5.5-alpha? and earlier won't select guards from an expired
consensus, but they can still bootstrap if they have existing guards.
Update the fallback expiry tolerance to match tor's checks.
Part of 28768, follow-up on 24661.
This updateVersions.pl script was one of the only essential perl
scripts left in out maint system, and was the only one that used
autoconf to fill in the script.
This script adds a feature to define an APPROX_RELEASE_DATE macro
that is updated when the version changes. We'll use this to
implement prop297, so that we have an accurate view of when a
release date happens.
This is a very gentle commit that just lays the groundwork in the
build system: it puts the include files to build libtor-app.a into
src/core, and to build the tor executable into src/app. The
executable is now "src/app/tor".
Includes configuration files to enforce these rules on lib and
common. Of course, "common" *is* a modularity violation right now,
so these rules aren't as strict as I would like them to be.
Avoid selecting fallbacks that change their IP addresses too often.
Select more fallbacks by ignoring the Guard flag, and allowing lower
cutoffs for the Running and V2Dir flags. Also allow a lower bandwidth,
and a higher number of fallbacks per operator (5% of the list).
Implements ticket 24785.
Add the generateFallbackDirLine.py script for automatically generating
fallback directory mirror lines from relay fingerprints. No more typos!
Add the lookupFallbackDirContact.py script for automatically looking up
operator contact info from relay fingerprints.
Implements ticket 24706.
This removes some redundant repeated lines.
Ticket 24681 will maintain the current fallback weights by changing
Tor's default fallback weight to 10.
Implements ticket 24679.
We only occasionally checked for fallbacks on the same machine.
And I'm not convinced it makes much of a difference with ~150 fallbacks.
Part of #22321.
To do this, we had to make sure it passes when the changes directory
is empty. I also tried to improve the quality of the output, and
fix some false-positive cases. Let's see how this goes!
Closes ticket 23564.
This is a large and important unit test for the hidden service version
3! It tests the service reachability for a client using different
consensus timings and makes sure that the computed hashring is the same
on both side so it is actually reachable.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Increase the fallback stability requirement to 30 days.
When this was at 7 days, we chose far too many unstable fallbacks.
Decrease the guard flag requirement to 0.8.
When this was at 0.9, we lost too many fallbacks due to version upgrades.
(The running and v2dir flags ensure DirPorts are available to clients.)
Partial fixes to #20913.
Found using flake8 which gave the following output;
lintChanges.py:10:13: E225 missing whitespace around operator
lintChanges.py:25:1: E302 expected 2 blank lines, found 1
lintChanges.py:46:5: F841 local variable 'lines' is assigned to but never used
lintChanges.py:52:32: E228 missing whitespace around modulo operator
lintChanges.py:53:11: E201 whitespace after '('
lintChanges.py:56:41: E228 missing whitespace around modulo operator
Part of ticket 21096.
Sometimes, the fallback generation script doesn't add attributes to the
fallbacks in the list. If this happens, log an error, and avoid selecting
that fallback.
This is a rare issue: it should not change selection behaviour.
Fixes issue #20945.
Exclude relays that have been down for 1 or more days from the fallback
candidate list.
When a relay operator has multiple relays, this prioritises relays that are
up over relays that are down.
Fixes issue #20926.
7 days is a tradeoff between the expected time between major Tor releases,
which is 6 months, and the number of relays with enough stability.
Relays whose OnionOO stability timer is reset on restart by bug #18050
should upgrade to Tor 0.2.8.7 or later, which has a fix for this issue.
Closes ticket #20880; maintains short-term fix in e220214 in tor-0.2.8.2-alpha.
The functions it warns about are:
assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
strdup, strndup, calloc.
Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
If we manually remove fallbacks in C by adding '/*' and '*/' on separate
lines, stem still parses them as being present, because it only looks at
the start of a line.
Add a comment to this effect in the generated source code.
Remove a fallback that changed its fingerprint after it was listed
This happened after to a software update:
https://lists.torproject.org/pipermail/tor-relays/2016-June/009473.html
Remove a fallback that changed IPv4 address
Remove two fallbacks that were slow to deliver consensuses,
we can't guarantee they'll be fast in future.
Blacklist all these fallbacks until operators confirm they're stable.
Operators have sent emails asking to have their relays added or
removed from the fallback list. Since none of the blacklisted
relays are in the hard-coded falback list, it does not need to
be changed.
Log a notice just before the script is about to perform a
potentially time-consuming operation
Clarify the warning when py2-ipaddress isn't found
Make log levels more consistent
No behavioural change (just logging)
As well as the existing reports of IPv6 address additions or removals,
the script now warns when keys change but IPv4:ORPort or
IPv6:IPv6ORPort remain the same.
Existing checks for other whitelist detail changes have also
been re-worded and upgraded to warnings.
This makes it easier for changes to be identified so operators can
be contacted to confirm whether the change is stable.
Also add misbehaving relays to updateFallbackDirs.py blacklist,
but leave them commented out in case it's a transient issue,
or it's been resolved by the download check fixes. (These
relays hang stem's downloader. It's unlikely they'll ever help
clients.)
Use IP address, effective family, and contact info to
discover and limit fallbacks to one per operator.
Also analyse netblock, ports, IP version, and Exit flag,
and print the results. Don't exclude any fallbacks from
the list because of netblocks, ports, IP version, or
Exit flag.
But as advertised bandwidth is controlled by relays,
use consensus weight and median weight to bandwidth ratio
to approximate measured bandwidth.
Includes minor comment changes and parameter reordering.
Previously, we would cut the list down to 100 fallbacks,
then check if they could serve a consensus, and comment
them out if they couldn't.
This would leave us with fewer than 100 active fallbacks.
Now, we stop when there are 100 active fallbacks.
Also count fallbacks with identical contact info.
Also fix minor logging issues.
Give each fallback a set weight of 10.0 for client selection.
Fallbacks must have at least 3000 consensus weight.
This is (nominally) 100 times the expected extra load of
20 kilobytes per second (50 GB per month).
Fixes issue #17905.
Improve the download test:
* Allow IPv4 DirPort checks to be turned off.
* Add a timeout to stem's consensus download.
* Actually check for download errors, rather than ignoring them.
* Simplify the timeout and download error checking logic.
Tweak whitelist/blacklist checks to be more robust.
Improve logging, make it warn by default.
Cleanse fallback comments more thoroughly:
* non-printables (yes, ContactInfo can have these)
* // comments (don't rely on newlines to prevent // */ escapes)
