First I began with a set of candidates:
The client's _required_ list starts with all the protocols
supported by every release in 0.2.9-stable through current
master.
The client's _required_ list starts with all the protocols
supported by every release in 0.3.5-stable through current
master.
Everybody's _recommended_ list starts with all the protocols
supported by every release in 0.3.5-stable through current master.
Then I removed the protocol versions that we do not actually want to
require or recommend:
DirCache=1 (deprecated)
HSDir=1, HSIntro=1-3, HSRend=1 (deprecated)
(All HS* protocol requirements for clients)
Link=1-3 (deprecated)
LinkAuth=1 (obsolete)
Relay=1 (obsolete)
Make it possible to specify multiple ConsensusParams torrc
lines.
Now directory authority operators can for example put the
main ConsensusParams config in one torrc file and then add to it
from a different torrc file.
Closes ticket 40164.
When calling `circpad_send_padding_cell_for_callback()`, the flag
`is_padding_timer_scheduled` was not resetted to 0 which caused an issue
in the circpad simulator.
This commit fixes this problem.
Fixes#32671.
Cons=1 is the old format of consensuses, without ed25519 keys. It
is no longer required or recommended.
Cons=2 is the new format of consensuses, with ed25519 keys. It
is now required.
(Similarly for Desc=1,2 and Microdesc=1,2)
No supported client or relay versions should be affected by this
change, since these versions are supported by clients and relays
running 0.2.9 and later. It will only take effect once enough
authorities vote for it.
Closes ticket 40162.
In brief: we go through a lot of gymnastics to handle huge protover
numbers, but after years of development we're not even close to 10
for any of our current versions. We also have a convenient
workaround available in case we ever run out of protocols: if (for
example) we someday need Link=64, we can just add Link2=0 or
something.
This patch is a minimal patch to change tor's behavior; it doesn't
take advantage of the new restrictions.
Implements #40133 and proposal 318.
Tor has a feature to preserve unrecognized state file entries in
order to maintain forward compatibility. But this feature, along
with some unused code that we never actually removed, led to us
keeping items that were of no use to the user, other than at worst
to preserve ancient information about them.
This commit adds a feature to remove obsolete entries when we load
the file.
Closes ticket 40137.
If at least one service is configured as a version 2, a log warning is emitted
once and only once.
Closes#40003
Signed-off-by: David Goulet <dgoulet@torproject.org>
First, we introduce a flag to teach src/test/test to split its work
into chunks. Then we replace our invocation of src/test/test in our
"make check" target with a set of 8 scripts that invoke the first
8th of the tests, the second 8th, and so on.
This change makes our "make -kj4 check" target in our hardened
gitlab build more than twice as fast, since src/test/test was taking
the longest to finish.
Closes 40098.
This patch adds support for exposing the environment variables
`TOR_PT_OUTBOUND_BIND_ADDRESS_V4` and `TOR_PT_OUTBOUND_BIND_ADDRESS_V6` to
Pluggable Transport proccesses. These two values will contain the IPv4
and IPv6 address that the user have specified in torrc that they wish
the PT to use for all outgoing IP packets.
It is important to note here that it is up to the indvidual Pluggable
Transport if they are willing to honor these values or ignore them
completely.
One can test this feature using the following dummy PT written in POSIX
shell script:
#!/bin/sh
echo "LOG SEVERITY=warning MESSAGE=\"Value for IPv4: ${TOR_PT_OUTBOUND_BIND_ADDRESS_V4}\""
echo "LOG SEVERITY=warning MESSAGE=\"Value for IPv6: ${TOR_PT_OUTBOUND_BIND_ADDRESS_V6}\""
while true ; do
sleep 1
done
with the following entries in your torrc:
OutboundBindAddressPT 203.0.113.4
OutboundBindAddress 203.0.113.5
OutboundBindAddressPT 2001:db8::4
OutboundBindAddress 2001:db8::5
See: https://bugs.torproject.org/5304
[This is a squashed patch for ticket 7193, based on taking a "git
diff" for the original branch, then applying it with "git apply
-3". I earlier attempted to squash the branch with "git rebase",
but there were too many conflicts. --nickm]
The option `--enable-all-bugs-are-fatal` when used with
`--disable-module-relay` caused GCC to warn about functions that
it thought should be labeled noreturn.
I've tried a couple of approaches, but disabling the warning on
these functions seems to be the best approach.
Fixed#40129; bugfix on 0.4.4.1-alpha.
We set this flag if we've launched the connection in order to
satisfy an origin circuit, or when we decide the connection _would_
satisfy an origin circuit. These are the only or_connections we
want to consider for bootstrapping: other or_connections are opened
because of client EXTEND requests, and they may succeed or fail
because of the clients' confusion or misconfiguration.
Closes#25061.
The rend_cache/entry_free was missing the rend cache allocation increment
before freeing the object.
Without it, it had an underflow bug:
Sep 17 08:40:13.845 [warn] rend_cache_decrement_allocation(): Bug: Underflow
in rend_cache_decrement_allocation (on Tor 0.4.5.0-alpha-dev
7eef9ced61)
Fixes#40125
Signed-off-by: David Goulet <dgoulet@torproject.org>
Our code was using [01] as for the key type of signed->link certs,
which was incorrect. The value should be [03], to indicate that the
value as the SHA256 of an x.509 cert.
Fortunately, nothing cares about this value, so there shouldn't be
compatibility issues.
Fixes bug 40124; bugfix on 0.2.7.2-alpha.
tl;dr We were not counting cells flying from the client to the service, but we
were counting cells flying from the service to the client.
When a rendezvous cell arrives from the client to the RP, the RP forwards it to
the service.
For this to happen, the cell first passes through command_process_relay_cell()
which normally does the statistics counting. However because the `rend_circ`
circuit was not flagged with `circuit_carries_hs_traffic_stats` in
rend_mid_rendezvous(), the cell is not counted there.
Then the cell goes to circuit_receive_relay_cell() which has a special code
block based on `rend_splice` specifically for rendezvous cells, and the cell
gets directly passed to `rend_circ` via a direct call to
circuit_receive_relay_cell(). The cell never passes through
command_process_relay_cell() ever again and hence is never counted by our
rephist module.
The fix here is to flag the `rend_circ` circuit with
`circuit_carries_hs_traffic_stats` so that the cell is counted as soon as it
hits command_process_relay_cell().
Furthermore we avoid double-counting cells since the special code block of
circuit_receive_relay_cell() makes us count rendezvous cells only as they enter
the RP and not as they exit it.
Fixes#40117.
tl;dr We were not counting cells flying from the client to the service, but we
were counting cells flying from the service to the client.
When a rendezvous cell arrives from the client to the RP, the RP forwards it to
the service.
For this to happen, the cell first passes through command_process_relay_cell()
which normally does the statistics counting. However because the `rend_circ`
circuit was not flagged with `circuit_carries_hs_traffic_stats` in
rend_mid_rendezvous(), the cell is not counted there.
Then the cell goes to circuit_receive_relay_cell() which has a special code
block based on `rend_splice` specifically for rendezvous cells, and the cell
gets directly passed to `rend_circ` via a direct call to
circuit_receive_relay_cell(). The cell never passes through
command_process_relay_cell() ever again and hence is never counted by our
rephist module.
The fix here is to flag the `rend_circ` circuit with
`circuit_carries_hs_traffic_stats` so that the cell is counted as soon as it
hits command_process_relay_cell().
Furthermore we avoid double-counting cells since the special code block of
circuit_receive_relay_cell() makes us count rendezvous cells only as they enter
the RP and not as they exit it.
Fixes#40117.
Opening a new listener connection can fail in many ways like a bind()
permission denied on a low port for instance.
And thus, we should expect to handle an error when creating a new one instead
of assert() on it.
To hit the removed assert:
ORPort 80
KeepBindCapabilities 0
Start tor. Then edit torrc:
ORPort <some-IP>:80
HUP tor and the assert is hit.
Fixes#40073
Signed-off-by: David Goulet <dgoulet@torproject.org>
Previously, we would treat *any* incoming circuit on a non-local
channel as meaning that our ORPort was reachable. With this patch,
we make sure that the address that the peer _says_ we have is the
same as the one we're trying to advertise right now.
Closes 20165. Bugfix on 4f5192b280 in 0.1.0.1-rc, when
reachability self-tests were first introduced.
Turns out that the HS DoS defenses parameters were overwritten by the
consensus parameters everytime a new consensus would arrive.
This means that a service operator can still enable the defenses but as soon
as the intro point relay would get a new consensus, they would be overwritten.
And at this commit, the network is entirely disabling DoS defenses.
Fix this by introducing an "explicit" flag that indicate if the
ESTABLISH_INTRO cell DoS extension set those parameters or not. If set, avoid
using the consenus at once.
We are not bumping the protover HSIntro value for this because 0.4.2.x series
is EOL in 1 month and thus 0.4.3.x would be the only series with this bug. We
are confident that a backport and then upgrade path to the latest 0.4.4.x
stable coming up soon is enough to mitigate this problem in the coming months.
It avoids the upgrade path on the service side by keeping the requirement for
protover HSIntro=5.
Fixes#40109
Signed-off-by: David Goulet <dgoulet@torproject.org>
First, we introduce a flag to teach src/test/test to split its work
into chunks. Then we replace our invocation of src/test/test in our
"make check" target with a set of 8 scripts that invoke the first
8th of the tests, the second 8th, and so on.
This change makes our "make -kj4 check" target in our hardened
gitlab build more than twice as fast, since src/test/test was taking
the longest to finish.
Closes 40098.
Previous message would say "N messages in the last T seconds", but
would give an inaccurate number for N.
We now give an accurate number, rounded up to the nearest 60 seconds.
Closes#19431.
This function once served to let circuits continue to be built over
version-1 link connections. But such connections are long-obsolete,
and it's time to remove this check.
Closes#40081.
We found this in #40076, after we started using buf_move_all() in
more places. Fixes bug #40076; bugfix on 0.3.3.1-alpha. As far as
I know, the crash only affects master, but I think this warrants a
backport, "just in case".
Without this fix, if an PT forgets to send a USERADDR command, that
results in a connection getting treated as local for the purposes of
rate-limiting.
If the PT _does_ use USERADDR, we still believe it.
Closes ticket 33747.
Previously we tolerated up to 1.5 connections for every relay we
were connected to, and didn't warn if we had fewer than 5
connections total.
Now we tolerate up to 1.5 connections per relay, and up to 4
connections per authority, and we don't warn at all when we have
fewer than 25 connections total.
Fixes bug 33880, which seems to have been provoked by our #17592
change in 0.3.5.
This commit makes it that if the ORPort is set with a single port, it will
bind to both global listen IPv4 and IPv6 addresses.
To pin an "ORPort <PORT>" to be IPv4 or IPv6, the IPv4Only/IPv6Only flags are
honored thus this will _only_ bind on IPv6 for that port value:
ORPort 9050 IPv6Only
Results in: [::]:9050
ORPort 9051 IPv4Only
Results in: [0.0.0.0]:9051
Attempting to configure an explicit IPv4 address with IPv6Only flag is an
error and vice versa.
Closes#33246
Signed-off-by: David Goulet <dgoulet@torproject.org>
The need for casting negative syscall arguments depends on the
glibc version. This affects the rules for the openat syscall which
uses the constant AT_FDCWD that is defined as a negative number.
This commit adds logic to only apply the cast when necessary, on
glibc versions from 2.27 onwards.
Different versions of glibc use either open or openat for the
opendir function. This commit adds logic to use the correct rule
for each glibc version, namely:
- Until 2.14 open is used
- From 2.15 to to 2.21 openat is used
- From 2.22 to 2.26 open is used
- From 2.27 onwards openat is used
The need for casting negative syscall arguments depends on the
glibc version. This affects the rules for the openat syscall which
uses the constant AT_FDCWD that is defined as a negative number.
This commit adds logic to only apply the cast when necessary, on
glibc versions from 2.27 onwards.
Instead of replacing connection_t.{addr,port} with a canonical
orport, and tracking the truth in real_addr, we now leave
connection_t.addr alone, and put the canonical address in
canonical_orport.
Closes#40042Closes#33898
(This is not fully general yet: we only pick randomly among
_supported_ addresses, and each extendinfo contains at most one IPv4
address and at most one IPv6 address, no matter what the extend cell
had.)
This change will help dual-stack relays do IPv6 reachability tests,
in theory, by having them sometimes do IPv4 connections and
sometimes do ipv6 connections.
Closes ticket 33220.
Create a doc/tracing/ directory to contain a top level README.md which is the
previously named Tracing.md and add the EventsCircuit.md which describes the
circuit subsystem tracing events in depth.
Closes#40036
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch ensures that we strip "\r" characters on both Windows as well
as Unix when we read text files. This should prevent the issue where
some Tor state files have been moved from a Windows machine, and thus
contains CRLF line ending, to a Unix machine where only \n is needed.
We add a test-case to ensure that we handle this properly on all our
platforms.
See: https://bugs.torproject.org/tpo/core/tor/33781
This changes a LOT of code but in the end, behavior is the same.
Unfortunately, many functions had to be changed to accomodate but in majority
of cases, to become simpler.
Functions are also removed specifically those that were there to convert an
IPv4 as a host format to a tor_addr_t. Those are not needed anymore.
The IPv4 address field has been standardized to "ipv4_addr", the ORPort to
"ipv4_orport" (currently IPv6 uses ipv6_orport) and DirPort to "ipv4_dirport".
This is related to Sponsor 55 work that adds IPv6 support for relays and this
work is needed in order to have a common interface between IPv4 and IPv6.
Closes#40043.
Signed-off-by: David Goulet <dgoulet@torproject.org>
check-cocci is still a good idea -- perhaps as a cron job? But
doing it as part of our regular tests has just been confusing,
especially to volunteers who shouldn't have to become coccinelle
experts in order to get their patches through our CI.
Closes#40030.
When receiving an introduction NACK, the client either decides to close or
re-extend the circuit to another intro point.
In order to do this, the service descriptor needs to exists but it is possible
that it gets removed from the cache between the establishement of the
introduction circuit and the reception of the (N)ACK.
For that reason, the BUG(desc == NULL) is removed because it is a possible
normal use case. Tor recovers gracefully already.
Fixes#34087
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit modifies the behavior of `parse_extended_address` in such a way
that if it fails, it will always return a `BAD_HOSTNAME` value, which is then
used to return the 0xF6 extended error code. This way, in any case that is
not a valid v2 address, we return the 0xF6 error code, which is the expected
behavior.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This reverts commit bf2a399fc0.
Don't set by default the prefer IPv6 feature on client ports because it breaks
the torsocks use case. The SOCKS resolve command is lacking a mechanism to ask
for a specific address family (v4 or v6) thus prioritizing IPv6 when an IPv4
address is asked on the resolve SOCKS interface resulting in a failure.
Tor Browser explicitly set PreferIPv6 so this should not affect the majority
of our users.
Closes#33796
Signed-off-by: David Goulet <dgoulet@torproject.org>
If no Address statement are found in the configuration file, attempt to learn
our address by looking at the ORPort address if any. Specifying an address is
optional so if we can't find one, it is fine, we move on to the next discovery
mechanism.
Note that specifying a hostname on the ORPort is not yet supported at this
commit.
Closes#33236
Signed-off-by: David Goulet <dgoulet@torproject.org>
For some reasons, Appveyor started to use the stdio printf format for 64 bit
values (PRIu64, ...). Mingw doesn't like that so force it to use the Windows
specific macros by setting D__USE_MINGW_ANSI_STDIO=0.
Fixes#40026
This patch fixes an out-of-bound memory read in
`tor_tls_cert_matches_key()` when Tor is compiled to use Mozilla's NSS
instead of OpenSSL.
The NSS library stores some length fields in bits instead of bytes, but
the comparison function found in `SECITEM_ItemsAreEqual()` needs the
length to be encoded in bytes. This means that for a 140-byte,
DER-encoded, SubjectPublicKeyInfo struct (with a 1024-bit RSA public key
in it), we would ask `SECITEM_ItemsAreEqual()` to compare the first 1120
bytes instead of 140 (140bytes * 8bits = 1120bits).
This patch fixes the issue by converting from bits to bytes before
calling `SECITEM_ItemsAreEqual()` and convert the `len`-fields back to
bits before we leave the function.
This patch is part of the fix for TROVE-2020-001.
See: https://bugs.torproject.org/33119
If at least one service is configured as a version 2, a log warning is emitted
once and only once.
Closes#40003
Signed-off-by: David Goulet <dgoulet@torproject.org>
The find_my_address() function now prioritize the local interface over the
local hostname when guessing the IP address.
See proposal 312, section 3.2.1, general case:
https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv6-addr.txt#n359
The entire unit tests had to be refactored to make this possible. Instead of
hot patching it, it has been rewritten to cover all possible cases and the
test interface has been changed to accomodate both IPv4 and IPv6 in order for
them to be tested identically.
Closes#33238
Signed-off-by: David Goulet <dgoulet@torproject.org>
Authorities currently add themselves to the trusted dir servers list,
but if they have an IPv6 ORPort, they leave it out.
This commit makes authorities add their own IPv6 ORPort to the trusted
dir servers list.
Closes ticket 32822.
This is not actually a bug! It can happen for a bunch of reasons,
which all boil down to "trying to add an extrainfo for which we no
longer have the corresponding routerinfo".
Fixes#16016; bugfix on 0.2.6.3-alpha.
Previously we had two chains of logic for reachability tests: one
for launching them, and one for telling the user that we had
launched them. Now, we simply have the launch code inform the user:
this way, we can't get out of sync.
Closes ticket 34137.
AssumeReachable should only be about whether a relay thinks that it
is reachable itself. But previously, we've had it also turn off
reachability checking of _other_ relays on authorities.
(Technically, reachability tests were still run, but we would ignore
the results.)
With this patch, there is a new AuthDirTestReachability option
(default 1) that controls whether authorities run reachability
tests.
Making this change allows us to have test networks where authorities
set AssumeReachable without disabling their reachability testing
entirely.
Closes ticket #34445.
These parameters do not suppress checks, but they tell relays that
it's okay to publish even when those checks fail.
I have chosen lowercase hyphenated names, since these seem to be
more common in networkstatus params.
Closes#33224 and part of #34064.
The buf_read_from_tls() function was designed to read up to a certain number
of bytes a TLS socket using read_to_chunk_tls() which boils down to SSL_read()
(with OpenSSL, common case).
However, at the end of the loop, the returned number of bytes from
read_to_chunk_tls() was treated like the syscall read() for which if less
bytes than the total asked are returned, it signals EOF.
But, with SSL_read(), it returns up to a TLS record which can be less than
what was asked. The assumption that it was EOF was wrong which made the while
loop exiting before it was able to consume all requested bytes (at_most
parameter).
The general use case that Tor sees is that it will ask the network layer to
give it at most 16KB (that is roughly 32 cells) but because of KIST scheduler,
the highest possible TLS record we currently observe is 4096 bytes (4KB or 8
cells). Thus the loop would at best always return 8 cells even though much
more could be on the TLS socket. See ticket #40006 for more details.
Fixes#40006
Signed-off-by: David Goulet <dgoulet@torproject.org>
I've managed to keep this change mainly contained to our
self-testing module. The changes here are:
* There are two different variables for tracking "is our orport
reachable".
* We have a new function that says whether we can skip a single
family's orport reachability test; the old function for this now
tells whether we can skip _all_ orport reachability testing.
(The name, router_should_skip_orport_reachability_test, is not
so good. I will rename it later if I can think of a good
replacement.)
* The function that launches orport reachability tests now only
launches the ones that haven't completed.
* The function that notes that we're reachable on an ORPort now
takes a family.
* Various log messages are cleaned up.
Since Tor 0.2.9 has been unsupported since January, 0.3.5 is the
oldest supported version, and its features constitute the oldest
recommended feature-set.
This patch updates these recommendations:
DirCache=2
Support for consensus diffs.
New in 0.3.1.1-alpha.
HSDir=2
Support for v3 onion service descriptors.
New in 0.3.0.4-alpha.
HSIntro=4
Support for Ed25519 intropoint authentication keys.
New in 0.3.0-4-alpha.
HSRend=2
Support for rendezvous cells longer than 20 bytes.
New in 0.2.9.4-alpha.
Link=5
Link padding and link padding negotiation.
New in 0.3.3.2-alpha.
LinkAuth=3
Ed25519 link authentication.
New in 0.3.0.1-alpha.
M_SYSCALL is used to report information about a sandbox violation,
but when we don't have a definition for it, it still makes sense to
compile.
Closes ticket 34382.
Resume being willing to use preemptively-built circuits when
UseEntryGuards is set to 0. We accidentally disabled this feature with
that config setting (in our fix for #24469), leading to slower load times.
Fixes bug 34303; bugfix on 0.3.3.2-alpha.
