Handle the EOF situation for a metrics connection. Furthermore, if we failed
to fetch the data from the inbuf properly, mark the socket as closed because
the caller, connection_process_inbuf(), assumes that we did so on error.
Fixes#40257
Signed-off-by: David Goulet <dgoulet@torproject.org>
Previously we would warn in this case... but there's really no
justification for doing so, and it can only cause confusion.
Fixes bug #40281; bugfix on 0.4.0.1-alpha.
In two instances we must look at this flag:
1. When we build the descriptor so the IPv6 is NOT added to the descriptor in
case we judge that we need to omit the address but still publish.
2. When we are deciding if the descriptor is publishable. This flags tells us
that the IPv6 was not found reachable but we should still publish.
Fixes#40279
Signed-off-by: David Goulet <dgoulet@torproject.org>
While trying to resolve our CI issues, the Windows build broke with an
unused function error:
src/test/test_switch_id.c:37:1: error: ‘unprivileged_port_range_start’
defined but not used [-Werror=unused-function]
We solve this by moving the `#if !defined(_WIN32)` test above the
`unprivileged_port_range_start()` function defintion such that it is
included in its body.
This is an unreviewed commit.
See: tor#40275
We currently assume that the only way for Tor to listen on ports in the
privileged port range (1 to 1023), on Linux, is if we are granted the
NET_BIND_SERVICE capability. Today on Linux, it's possible to specify
the beginning of the unprivileged port range using a sysctl
configuration option. Docker (and thus the CI service Tor uses) recently
changed this sysctl value to 0, which causes our tests to fail as they
assume that we should NOT be able to bind to a privileged port *without*
the NET_BIND_SERVICE capability.
In this patch, we read the value of the sysctl value via the /proc/sys/
filesystem iff it's present, otherwise we assume the default
unprivileged port range begins at port 1024.
See: tor#40275
The TORPROTOCOL reason causes the client to close the circuit which is not
what we want because other valid streams might be on it.
Instead, CONNECTION_REFUSED will leave it open but will not allow more streams
to be attached to it. The client then open a new circuit to the destination.
Closes#40270
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is to minimize false positive and thus deny reentry to Exit connections
that were in reality not re-entering. Helps with overall UX.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Obey the "allow-network-reentry" consensus parameters in order to decide to
allow it or not at the Exit.
Closes#40268
Signed-off-by: David Goulet <dgoulet@torproject.org>
The TORPROTOCOL reason causes the client to close the circuit which is not
what we want because other valid streams might be on it.
Instead, CONNECTION_REFUSED will leave it open but will not allow more streams
to be attached to it. The client then open a new circuit to the destination.
Closes#40270
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is to minimize false positive and thus deny reentry to Exit connections
that were in reality not re-entering. Helps with overall UX.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Obey the "allow-network-reentry" consensus parameters in order to decide to
allow it or not at the Exit.
Closes#40268
Signed-off-by: David Goulet <dgoulet@torproject.org>
Exit relays now reject exit attempts to known relay addresses + ORPort and
also to authorities on the ORPort and DirPort.
Closes#2667
Signed-off-by: David Goulet <dgoulet@torproject.org>
In order to deny re-entry in the network, we now keep a bloomfilter of relay
ORPort + address and authorities ORPort + address and DirPort + address
combinations.
So when an Exit stream is handled, we deny anything connecting back into the
network on the ORPorts for relays and on the ORPort+DirPort for the
authorities.
Related to #2667
Signed-off-by: David Goulet <dgoulet@torproject.org>
Exit relays now reject exit attempts to known relay addresses + ORPort and
also to authorities on the ORPort and DirPort.
Closes#2667
Signed-off-by: David Goulet <dgoulet@torproject.org>
In order to deny re-entry in the network, we now keep a bloomfilter of relay
ORPort + address and authorities ORPort + address and DirPort + address
combinations.
So when an Exit stream is handled, we deny anything connecting back into the
network on the ORPorts for relays and on the ORPort+DirPort for the
authorities.
Related to #2667
Signed-off-by: David Goulet <dgoulet@torproject.org>
Everytime we try to discover an address we want to publish, emit a log notice
if we are unable to find it even though an ORPort was configured for it.
Because the function can be called quite often, we rate limit that notice to
every hour so it gets annoying just enough so the operator fixes that.
Related to #40254
Signed-off-by: David Goulet <dgoulet@torproject.org>
We would before do an address discovery and then a lookup in the cache if not
found which is now simplified by calling relay_find_addr_to_publish() directly
which does all those combined.
Furthermore, by doing so, we won't trigger an address discovery every minute
if we have no ORPort configured for the family.
Fixes#40254
Signed-off-by: David Goulet <dgoulet@torproject.org>
Now that relay_find_addr_to_publish() checks if we actually have an ORPort, we
can simplify the descriptor building phase for IPv6.
This also avoid triggering an IPv6 discovery if the IPv4 can't be found in the
first place.
Related to #40254
Signed-off-by: David Goulet <dgoulet@torproject.org>
In other words, if we don't have an ORPort configured for a specific family
(IPv4/v6), we don't bother doing address discovery.
Related to #40254
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is a _subtle_ bug introduced by d1494d14, which resolved
connections that was allocated in the extorport/handshake test. So
how did the connection get freed? Our test was set up so that every
extorport connection would get the same ext_or_id. Two connections
couldn't have the same ext_or_id, and if they did, one would get
freed. This meant that the _next_ connection to be constructed in
the test would cause the previous connection to become closeable,
even if it hadn't been closeable before.
But when we applied d149d14, we stopped making it so our code
enforced this uniqueness, and thereby make it so we _weren't_
freeing this connection in the tests.
Closes#40260; bug not in any released version of Tor.
This step happens after we make each consensus flavor, and before we
worry about sigs or anything. That way if Tor crashes, or if we fail to
get enough sigs, we still have a chance to know what consensus we wanted
to make.
This validation was only done if DisableNetwork was off because we would use
the global list of transports/bridges and DisableNetwork would not populate
it.
This was a problem for any user using DisableNetwork which includes Tor
Browser and thus leading to the Bug() warning.
Without a more in depth refactoring, we can't do this validation without the
global list.
The previous commit makes it that any connection to a bridge without a
transport won't happen thus we keep the security feature of not connecting to
a bridge without its corresponding transport.
Related to #40106
Signed-off-by: David Goulet <dgoulet@torproject.org>
Don't pick the bridge as the guard or launch descriptor fetch if no transport
is found.
Fixes#40106
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch limits the number of items in the consensus diff cache to 64
on the Windows platform. Hopefully, this will allow us to investigate a
smarter fix while avoiding the situation reported in tor#24857 where
Windows relay operators report Tor using 100% CPU.
See: tor#24857
When selecting the first advertised port, we always prefer the one with an
explicit address.
Closes#40246
Signed-off-by: David Goulet <dgoulet@torproject.org>
We used to actually discard ORPorts that were the same port and same family
but they could have different address.
Instead, we need to keep all different ORPorts so we can bind a listener on
each of them. We will publish only one of these in our descriptor though.
Related to #40246
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch removes a call to `tor_assert_nonfatal()` if
`extend_info_from_node()` returns NULL. This is unnecessary as we
already handle the case where `info` is NULL in the next `if (!info) {
... }` block in the code.
See: tor#32666.
This reverts commit d07f17f676.
We don't want to consider an entire routable IPv6 network as sybil if more
than 2 relays happen to be on it. For path selection it is very important but
not for selecting relays in the consensus.
Fixes#40243
We can end up trying to find our address from an authority while we don't have
yet its descriptor.
In this case, don't BUG() and just come back later.
Closes#40231
Signed-off-by: David Goulet <dgoulet@torproject.org>
In case building the descriptor would fail, we could still flag that we did in
fact publish the descriptors leading to no more attempt at publishing it which
in turn makes the relay silent for some hours and not try to rebuild the
descriptor later.
This has been spotted with #40231 because the operator used a localhost
address for the ORPort and "AssumeReachable 1" leading to this code path where
the descriptor failed to build but all conditions to "can I publish" were met.
Related to #40231
Signed-off-by: David Goulet <dgoulet@torproject.org>
This one should work on GCC _and_ on Clang. The previous version
made Clang happier by not having unreachable "fallthrough"
statements, but made GCC sad because GCC didn't think that the
unconditional failures were really unconditional, and therefore
_wanted_ a FALLTHROUGH.
This patch adds a FALLTHROUGH_UNLESS_ALL_BUGS_ARE_FATAL macro that
seems to please both GCC and Clang in this case: ordinarily it is a
FALLTHROUGH, but when ALL_BUGS_ARE_FATAL is defined, it's an
abort().
Fixes bug 40241 again. Bugfix on earlier fix for 40241, which was
merged into maint-0.3.5 and forward, and released in 0.4.5.3-rc.
Our original code for parsing these parameters out of our list of
parameters pre-dated us having the
dirvote_get_intermediate_param_value() function... and it was buggy.
Specifically, it would reject any " ... K=V ..." value
where there were additional unconverted characters after the V, and
use the default value instead,
We haven't run into this yet because we've never voted for
bwweightscale to be anything besides the default 10000, or
maxunmeasuredbw to be anything besides the default 20.
This requires a new consensus method because it is a change in how
consensuses are computed.
Fixes bug 19011; bugfix on 0.2.2.10-alpha.
Some days before this commit, the network experienced a DDoS on the directory
authorities that prevented them to generate a consensus for more than 5 hours
straight.
That in turn entirely disabled onion service v3, client and service side, due
to the subsystem requiring a live consensus to function properly.
We know require a reasonably live consensus which means that the HSv3
subsystem will to its job for using the best consensus tor can find. If the
entire network is using an old consensus, than this should be alright.
If the service happens to use a live consensus while a client is not, it
should still work because the client will use the current SRV it sees which
might be the previous SRV for the service for which it still publish
descriptors for.
If the service is using an old one and somehow can't get a new one while
clients are on a new one, then reachability issues might arise. However, this
is a situation we already have at the moment since the service will simply not
work if it doesn't have a live consensus while a client has one.
Fixes#40237
Signed-off-by: David Goulet <dgoulet@torproject.org>
Thanks to proposal 315 / ticket #30132, more fields are now
required in these documents. But ancient Tors that try to upload
obsolete documents were causing the authorities to log warnings
about missing fields, and to do so very spammily.
We now detect the missing fields before tokenizing, and log at
debug. This is a bit of ugliness, but it's probably a safer choice
than making _all_ unparseable-desc warnings into debug-level logs.
I'm looking at identity-ed25519 in extrainfos and proto in
routerdescs because they were (I believe) the latest-added fields in
Tor's history: any Tor that lacks them will also lack the other
newly required fields.
Fixes bug #40238; bugfix on 0.4.5.1-alpha.
We're getting "fallback annotation annotation in unreachable code"
warnings when we build with ALL_BUGS_ARE_FATAL. This patch fixes
that.
Fixes bug 40241. Bugfix on 0.3.5.4-alpha.
If we get an address suggestion from a directory authority and we have no
address configured or discovered, log it at notice level so the operator can
learn what address will be used by Tor.
Fixes#40201
Signed-off-by: David Goulet <dgoulet@torproject.org>
The previous parser only considered stats files _starting_ with the
timestamp tag, not stats files having the timestamp tag in a later
position. While this applies to all current stats files, a future
stats file might look differently. Better to fix the function now than
be surprised in another 9 years from now.
This commit also adds a test case for such future stats, and it fixes
stats file paths in newly added unit tests.
It turns out that 9 years ago, we stopped appending data into stats file and
rather overwrite everytime we have new stats (see commit
a6a127c833)
The load_stats_file() function was still thinking that we could have the same
line many times in the file which turns out to be false since 9 years ago.
However, that did not cause problem until IPv6 connection stats came along
which introduced a new line in conn-stats: "ipv6-conn-bi-direct ...".
Before, that file contained a single line starting with the tag
"conn-bi-direct". That very tag appears also in the IPv6 tag (see above) so
the load_stats_file() function would consider that the IPv6 line as the last
tag to be appeneded to the file and fail to report the line above (for IPv4).
It would actually truncate the IPv6 line and report it (removing the "ipv6-"
part).
In other words, "conn-bi-direct" was not reported and instead
"ipv6-conn-bi-direct" was used without the "ipv6-" part.
This commit refactors the entire function so that now it looks for a
"timestamp tag" to validate and then if everything is fine, returns the entire
content of the file. The refactor simplifies the function, adds logging in
case of failures and modernize it in terms of coding standard.
Unit tests are also added that makes sure the loaded content matches the
entire file if timestamp validation passes.
Fixes#40226
Signed-off-by: David Goulet <dgoulet@torproject.org>
When making sure we have a Bridge line with a ClientTransportPlugin, we
now check in the managed proxy list and so we can catch any missing
ClientTransportPlugin for a Bridge line.
Fixes#40106
Signed-off-by: David Goulet <dgoulet@torproject.org>
Previously, our code would send back an error if the socks5 request
parser said anything but DONE. But there are other non-error cases,
like TRUNCATED: we shouldn't send back errors for them.
This patch lowers the responsibility for setting the error message
into the parsing code, since the actual type of the error message
will depend on what problem was encountered.
Fixes bug 40190; bugfix on 0.3.5.1-alpha.
Adds a more user-friendly error message when the configuration is
reloaded and a new %include is added that makes its unglobbing
access files/folders not allowed by the seccomp sandbox.
Previously, "--list-fingerprint --quiet" was an error. Now, the
handler for optional arguments to "--list-fingerprint" can tell that
"--quiet" is a flag, not an argument.
This only affects flags that take an _optional_ argument, so you can
still put your torrc file in a location starting with "-".
Closes#40223.
The "-static" compile flag was set globally which means that all autoconf test
were attempting to be built statically and lead to failures of detecting
OpenSSL libraries and others.
This commit adds this flag only to the "tor" binary build.
There is also a fix on where to find libevent.a since it is using libtool, it
is in .libs/.
At this commit, there are still warnings being emitted that informs the user
that the built binary must still be linked dynamically with glibc.
Fixes#40111
Signed-off-by: David Goulet <dgoulet@torproject.org>
The Python code is such a nice addition to the documentation and the C
code for better understanding of onion v3 address generation. Straight
to the point and easy to understand.
Unfortunately it did not work with my distribution's Python version. I
have adjusted the code to support Python 3.8 (tested with 3.8.6) and
to still be compatible with Python 2.
Currently Tor fails with the following error:
src/test/test_stats.c: In function ‘test_rephist_v3_onions’:
src/test/test_stats.c:527:22: error: overflow in implicit constant conversion [-Werror=overflow]
update_approx_time(10101010101);
This patch changes the constant passed to update_approx_time() to avoid
the overflow in the implicit conversion.
See: tor#40199
The rest of rephist.c is doing the same kind of unsigned casting. For example
see rep_hist_format_buffer_stats() and rep_hist_format_exit_stats().
The previous switch to %ld made Appveyor fail:
https://ci.appveyor.com/project/torproject/tor/builds/36118502
When sending the stream level SENDME, it is possible the cirucit was marked
for close or any other failures that can occur. These events can occur
naturally.
Fixes#40142
Signed-off-by: David Goulet <dgoulet@torproject.org>
The function in charge of removing duplicate ORPorts from our configured ports
was skipping all non ORPorts port but only for the outer loop thus resulting
in comparing an ORPort with a non-ORPort which lead to problems.
For example, tor configured with the following would fail:
ORPort auto
DirPort auto
Both end up being the same configuration except that one is a OR listener and
one is a Dir listener. Thus because of the missing check in the inner loop,
they looked exactly the same and thus one is removed.
Fixes#40195
Signed-off-by: David Goulet <dgoulet@torproject.org>
First, this commit moves the launch_dummy_circuit_as_needed() function into
relay_find_addr.c and renames it to relay_addr_learn_from_dirauth(). This is
an attempt to centralize anything relate with address discovery in the right
module.
Second, when building a descriptor and we fail to discover our address,
immediately launch a dummy circuit to an authority in an attempt to learn our
descriptor.
It is still only done every 20 minutes even though the descriptor build is
done every minute. We ought to avoid load on the authority and if we can't
learn in the first place our address from them, chances are more things are
wrong.
Related to #40071
Signed-off-by: David Goulet <dgoulet@torproject.org>
Regularly, tor looks if its IP has changed. It does the entire auto discovery
process again. However, it is possible that it does not find anything.
Instead of thinking the IP changed to an unknown address, look at our cache
and see if that value has changed.
The reason for this is because if tor gets its address as a suggestion from a
directory authority, it is because the auto discovery failed and thus that
address should be consider for the IP change check.
Related to #40071
Signed-off-by: David Goulet <dgoulet@torproject.org>
Tor now can learn its address from a NETINFO cell coming from an authority.
Thus, instead from launching a dummy descriptor fetch to learn the address
from the directory response (unauthenticated), we simply now launch a one-hop
testing circuit.
Related to #40071
Signed-off-by: David Goulet <dgoulet@torproject.org>
This changes the behaviour of `tor --version` in such a way.
```console
src/app/tor --version
Tor version 0.4.5.1-alpha-dev (git-46ccde66a97d7985).
Tor is running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1h, Zlib 1.2.11, Liblzma 5.2.4, Libzstd 1.4.5 and Glibc 2.31 as libc.
Tor compiled with GCC version 10.2.0
```
Fixes#32102
Commit c3a0f75796 added this feature for ORPort
that we ignore any port that is not the family of our default address when
parsing the port. So if port_parse_config() was called with an IPv4 default
address, all IPv6 address would be ignored.
That makes sense for ORPort since we call twice port_parse_config() for
0.0.0.0 and [::] but for the rest of the ports, it is not good since a
perfectly valid configuration can be:
SocksPort 9050
SocksPort [::1]:9050
Any non-ORPort only binds by default to an IPv4 except the ORPort that binds
to both IPv4 and IPv6 by default.
The fix here is to always parse all ports within port_parse_config() and then,
specifically for ORPort, remove the duplicates or superseding ones. The
warning is only emitted when a port supersedes another.
A unit tests is added to make sure SocksPort of different family always exists
together.
Fixes#40183
Signed-off-by: David Goulet <dgoulet@torproject.org>
There were three separate places where we were hitting a sandbox Bug
warning before we actually exited.
Fixes#40094; bugfix on 0.3.1.1-alpha when %includes were introduced.
