When a service can not upload its descriptor(s), we have no logs on why. This
adds logging for each possible reason for each descriptors.
That logging is emitted every second so it is rate limited for each reason and
per descriptor.
Closes#33400
Signed-off-by: David Goulet <dgoulet@torproject.org>
For a bridge configured with a pluggable transport, the transport name is
used, with the IP address, for the GeoIP client cache entry.
However, the DoS subsystem was not aware of it and always passing NULL when
doing a lookup into the GeoIP cache.
This resulted in bridges with a PT are never able to apply DoS defenses for
newly created connections.
Fixes#33491
Signed-off-by: David Goulet <dgoulet@torproject.org>
Allow the "practracker" coding best practices checking script to read
unicode files, when using Python 2.
We made the script use unicode literals in 0.4.3.1-alpha, but didn't
change the codec for opening files.
Fixes bug 33374; bugfix on 0.4.3.1-alpha.
Use bridges+hs-v23 for "make test-network",
rather than using chutney's old default.
This change requires a recent version of chutney,
because the old bridges+hs-v23 did not work.
(See chutney's 33302 for details.)
Closes 28208.
The IPv6 test only runs if IPv6 is available.
Also, explicitly use the bridges+hs-v2 network for the IPv4-only test.
This network was chutney's default as of January 2020.
Closes 33300.
Add test-network-ipv4 and test-network-ipv6 tests to the Makefile.
These tests run the IPv4-only and dual-stack chutney flavours from
test-network-all.
Closes ticket 33280.
Closing these file descriptors can hide sanitiser logs.
Instead, flush the logs before tor exits, using fsync().
Some Windows environments don't have fsync(), so we check
for it at compile time.
Fixes bug 33087; bugfix on 0.4.1.6.
Stop executing the checked-out pre-commit hook from the pre-push hook.
Instead, execute the copy in the user's git dir.
Fixes bug 33284; bugfix on 0.4.1.1-alpha.
This patch ensures that we always lowercase the BridgeDistribution from
torrc in descriptors before submitting it.
See: https://bugs.torproject.org/32753
Conflicts:
doc/tor.1.txt
src/app/config/config.c
src/app/config/or_options_st.h
src/core/mainloop/connection.h
Between 042 and 043, the dirauth options were modularized so this merge commit
address this by moving the AuthDirRejectUncompressedRequests to the module
along with a series of accessors.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Authorities were never sending back 503 error code because by design they
should be able to always answer directory requests regardless of bandwidth
capacity.
However, that recently backfired because of a large number of requests from
unknown source using the DirPort that are _not_ getting their 503 code which
overloaded the DirPort leading to the authority to be unable to answer to its
fellow authorities.
This is not a complete solution to the problem but it will help ease off the
load on the authority side by sending back 503 codes *unless* the connection
is from a known relay or an authority.
Fixes#33029
Signed-off-by: David Goulet <dgoulet@torproject.org>
When the ONION_CLIENT_AUTH_REMOVE command is given to tor, now also remove the
descriptor associated with the client authorization credentials.
Fixes#33148
Signed-off-by: David Goulet <dgoulet@torproject.org>
The "once" flag makes tor_bug_occurred_() say that future instances
of the warning will be suppressed -- but that's not something that
BUG() does. Fixes bug 33095; bugfix on 0.4.1.1-alpha.
When we added the ACTIVE and DORMANT virtual signals, we taught the
signal command to handle them, but we didn't teach SIGNAL event to
report them.
To solve this problem and prevent it from recurring, this patch
revises the implementation of control_event_signal() to use the same
signal_table that handle_control_signal() uses. This way, the two
controller commands can't become out of sync.
Fixes bug 33104; bugfix on 0.4.0.1-alpha.
Fixes bug 32673; bugfix on 0.3.0.4-alpha. We introduced these
checks in ee5471f9aa to help diagnose 21369, but we used "-1"
when "TOR_TLS_ERROR_MISC" would have been correct. Found by opara.
I don't think that this is actually getting triggered in the wild,
but if it were, it could cause nasty behavior: spurious
WANTREAD/WANTWRITE returns have a way of turning into CPU-eating
busy-loops.
We introduced these BUG() checks in b0ddaac074 to prevent a
recurrence of bug 23690. But there's a report of the BUG() message
getting triggered and filling up the disk. Let's change it to
IF_BUG_ONCE().
Fixes bug 33093; bugfix on 0.3.2.2-alpha.
Stop allowing failures on the Travis CI stem tests job. It looks like
all the stem hangs we were seeing are now fixed, but let's make sure we
see them if they happen again.
Closes ticket 33075.
Make Tor clients tell dual-stack exits that they prefer IPv6
connections.
This change is equivalent to setting the PreferIPv6 flag on
SOCKSPorts (and most other listener ports). Tor Browser has
been setting this flag for some time, and we want to remove a
client distinguisher at exits.
Also update the man page, putting all the IP flags in their
non-default forms.
Closes ticket 32637.
* ticket 32695 removed networkstatus_consensus_has_ipv6(),
keep that change in master.
* ticket 20218 modifies the function name and comment for
routerstatus_has_visibly_changed(), keep that change
in ticket20218_rebased_squashed.
We already check if there are invalid values in
check_bridge_distribution_setting() and reject the value if that is the
case. We can therefore only have strings of [A-Z] | [a-z] | [0-9] | '-'
| '_' here which is according to the directory specification.
See: https://bugs.torproject.org/32753
The variable lenght cells are handled directly by
channel_tls_handle_var_cell() from an OR connection reading its inbuf. The
channel var cell handler (agnostic) was never used.
Closes#32892
Signed-off-by: David Goulet <dgoulet@torproject.org>
We hit this assertion with bug 32868, but I'm stymied figuring out
how we wound up with a routerstatus like this. This patch is a
diagnostic to attempt to figure out what is going on, and to avoid a
crash in the meantime.
When practracker is disabled, its output will be empty. We don't
want that happening during our tests.
Fixes bug 32705; bugfix on 0.4.2.1-alpha, when test_practracker.sh
was introduced.
When we updated main.c to use the same entry points as tor_api.h, we
didn't update ntmain.c. This led to bugs as the two got out of
sync.
There are two calls that we drop in this patch: first, I drop the
call to set_main_thread(), since that's redundant with the init
function in compat_winthreads.c (but see #32884). Second, I drop
the check for CMD_RUN_TOR: I'll add that back with a subsequent
commit.
Closes ticket 32883.
Allow calls to dup() which was introduced in commit a22fbab986.
From a security perspective, I don't think this should impact the
security of the sandbox significantly. As far as I can tell, there
is nothing an adversary can do with a duplicated FD that can't be
done with the original.
We stopped looking at this option in 85cf6dcba3, back when we
implemented the minimal pieces of prop275. Since then, we've had
code to validate and adjust this option, and to give it a different
value in testing networks, but the option hasn't actually done
anything.
We can safely mark it as OBSOLETE, since doing so does not make any
old configuration get rejected.
Closes ticket 32807.
With v3, the "pending_final_cpath" of a circuit is always NULL which means
that for v3, established client rendezvous circuit waiting for the intro point
to ACK, will always end up timing out quickly.
This can increase the delays to which you connect to a service since in order
to succeed, the rendezvous circuit needs to fully established
(CIRCUIT_PURPOSE_C_REND_JOINED) within the cutoff of the introduction circuit
as well which is these days around 2-3 seconds.
Fixes#32021
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add a check-practracker-unit-test target, and run it as part of
"make check".
Add the practracker unit tests to scripts/git/pre-commit.git-hook.
Part of 32609.
Require C99 standards-conforming code in Travis CI, but allow GNU gcc
extensions. Also activates clang's -Wtypedef-redefinition warnings.
Builds some jobs with -std=gnu99, and some jobs without.
Closes ticket 32500.
Python 2 will be end-of-life as of 1 Jan 2020, so we can finally
stop supporting it. As a first step, we should make our configure
script stop accepting python 2 as something acceptable to run our
tests with.
Closes ticket 32608.
In #26913 we solved a bug where CacheDirectoryGroupReadable would
override DataDirectoryGroupReadable when the two directories are the
same. We never did the same for KeyDirectory, though, because
that's a rare setting.
Now that I'm testing this code, though, fixing this issue seems
fine. Fixes bug #27992; bugfix on 0.3.3.1-alpha.
When we added the $FILTER for Windows newlines, we made
the pipeline always exit successfully, even if tor failed.
Fixes bug 32468; bugfix on 0.4.2.1-alpha.
When a SOCKS5 client sends a RESOLVE_PTR request, it must include
either an IPv4 or IPv6 address. In the past this was required to be a
binary address (address types 1 or 4), but since the refactoring of
SOCKS5 support in Tor 0.3.5.1-alpha, strings (address type 3) are also
allowed if they represent an IPv4 or IPv6 literal.
However, when a binary IPv6 address is provided,
parse_socks5_client_request converts it into a string enclosed in
brackets. This doesn't match what string_is_valid_ipv6_address
expects, so this would fail with the error "socks5 received
RESOLVE_PTR command with hostname type. Rejecting."
By replacing string_is_valid_ipv4_address/string_is_valid_ipv6_address
with tor_addr_parse, we accept strings both with and without brackets.
This fixes the handling of binary addresses, and also improves
symmetry with CONNECT and RESOLVE requests.
Fixes bug 32315.
Install the mingw OpenSSL package in Appveyor. This makes sure that
the OpenSSL headers and libraries match in Tor's Appveyor builds.
(This bug was triggered by an Appveyor image update.)
Fixes bug 32449; bugfix on 0.3.5.6-rc.
It no longer warns, and is now defined in terms of an "IGNORE" type.
(The "IGNORE" type is the same as "OBSOLETE", except that it is not
reported as obsolete. It should be useful for disabled modules.)
Closes ticket 32404.
We still interpret "AccelName" as turning on the "HardwareAccel"
feature, but we no longer modify the user's options here.
Fixes bug 32382; bugfix on 0.2.2.1-alpha when we added openssl
engine support.
When the relay module is disabled, make "ClientOnly 1" and
"DirCache 0" by default. (But keep "ClientOnly 0" and
"DirCache 1" as the defaults for the unit tests.)
And run "make autostyle".
Part of ticket 32410.
Doing so caused us to crash in some unusual circumstances, such as
using --verify-config to verify a configuration that failed during
the options_act() stage.
Fixes bug 32407; bugfix on 0.3.3.1-alpha.
test_parseconf.sh now supports:
* {error,expected}{,_lzma,_nss,_zstd}{,_no_dirauth,_no_relay_dirauth}
Or any combination of two or more optional libraries.
Closes ticket 32397.
This check was accidentally disabled by a bad find command.
Fixes bug 32402; bugfix on 0.4.2.1-alpha.
Obviously correct changes to already reviewed code.
Up till now, we have warned about all missing documentation, which
meant that we could never make doxygen warnings fatal. This has led
to our doxygen output getting full of errors several times in the
past.
This commit changes our approach to doxygen warnings: missing
documentation warnings are only on when the user asks for them with
--enable-missing-doc-warnings. When that option is not present,
doxygen respects the --enable-fatal-warnings flag.
Closes ticket 32385.
The current pkg-config setup has no sense of whether it is cross-compiling,
so it will detect things on the build system that are not present or are
wrong for the host system. This forces the cross-compiling build to only
look for pkg-config .pc files in --prefix.
A version of this has been the setup for many years with the Android builds.
Fixes#32191
Signed-off-by: Hans-Christoph Steiner <hans@eds.org>
When picking an intro point from the service descriptor, the client failed to
lookup the failure cache.
It made an HS v2 client re-pick bad intro points for which we already know it
won't work in the first place.
Based on Neel Chauhan original patch.
Fixes#25568
Signed-off-by: David Goulet <dgoulet@torproject.org>
Before, when parsing memunits, if overflow occured it failed silently.
Use nowrap u64 math to detect overflow, compare to INT64_MAX and if
greater tell user and fail accordingly.
15000000.5 TB fails double check as it a greater floating number than
(double)INT64_MAX
8388608.1 TB passes double check because it falls in the same value as
(double)INT64_MAX (which is 2^63), but will fail the int check because
(uint64_t)d, which is 2^63, is strictly greater than 2^63-1 (INT64_MAX).
Fixes#30920
Signed-off-by: José M. Guisado <guigom@riseup.net>
Before inspecting the p_chan, we must check if the circuit is marked for close
because if it is the case, the channels are nullified from the circuit.
Several valid cases can mark the circuit for close of the directory
connection.
Fixes#31958
Signed-off-by: David Goulet <dgoulet@torproject.org>
hs_client_purge_state() and hs_cache_clean_as_client() can remove a descriptor
from the client cache with a NEWNYM or simply when the descriptor expires.
Which means that for an INTRO circuit being established during that time, once
it opens, we lookup the descriptor to get the IP object but hey surprised, no
more descriptor.
The approach here is minimalist that is accept the race and close the circuit
since we can not continue. Before that, the circuit would stay opened and the
client wait the SockTimeout.
Fixers #28970.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Only use the HS circuit map to know if an introduction circuit is established
or not. No need for a flag to keep state of something we already have in the
circuit map. Furthermore, the circuit map gets cleaned up properly so it will
always have the "latest truth".
This commit also removes a unit test that was testing specifically that flag
but now we rely solely on the HS circuit map which is also tested few lines
below the removed test.
Fixes#32094
Signed-off-by: David Goulet <dgoulet@torproject.org>
Our minimum version is now 0.2.9.5-alpha. Series 0.3.0, 0.3.1,
0.3.2, 0.3.3, and 0.3.4 are now rejected.
Also, extract this version-checking code into a new function, so we
can test it.
Closes ticket 31549.
Also reject 0.3.5.0 through 0.3.5.6-rc as unstable.
Make clear in the man page, in both the bandwidth section and the
accountingmax section, that Tor counts in powers of two, not
powers of ten: 1 GByte is 1024*1024*1024 bytes, not one billion
bytes.
Resolves ticket 32106.
Conflicts:
src/feature/dirparse/authcert_parse.c
src/feature/dirparse/ns_parse.c
src/feature/hs/hs_service.c
src/lib/conf/conftesting.h
src/lib/log/log.h
src/lib/thread/threads.h
src/test/test_options.c
These conflicts were mostly related to autostyle improvements, with
one or two due to doxygen fixes.
Relays now respect their AccountingMax bandwidth again. When relays
entered "soft" hibernation (which typically starts when we've hit
90% of our AccountingMax), we had stopped checking whether we should
enter hard hibernation. Soft hibernation refuses new connections and
new circuits, but the existing circuits can continue, meaning that
relays could have exceeded their configured AccountingMax.
This commit rolls back some of the cpu-saving fixes, where we tried
to avoid calling so many of our events while we're off the network.
That's because PERIODIC_EVENT_FLAG_NEED_NET checks net_is_disabled(),
which returns true even if we're only in soft hibernation.
Fixes bug 32108; bugfix on 0.4.0.1-alpha.
Create an optional relay module, which can be disabled using the
--disable-module-relay configure option. When it is set, also disable
the dirauth module.
Add a minimal implemention, which disables the relay and dircache modes
in tor.
Closes ticket 32123.
* actually sleep when tor has not logged anything
* log at debug level when waiting for tor to log something
* backslash-replace bad UTF-8 characters in logs
* format control messages as ASCII: tor does not accept UTF-8 control commands
Fixes bug 31837; bugfix on 0.3.5.1-alpha.
This patch removes an overly strict tor_assert() and an ignorable BUG()
expression. Both of these would trigger if a PT was unable to configure
itself during startup. The easy way to trigger this is to configure an
obfs4 bridge where you make the obfs4 process try to bind on a port
number under 1024.
See: https://bugs.torproject.org/31091
This patch removes a call to tor_assert_unreached() after execve()
failed. This assertion leads to the child process emitting a stack trace
on its standard output, which makes the error harder for the user to
demystify, since they think it is an internal error in Tor instead of
"just" being a "no such file or directory" error.
The process will now instead output "Error from child process: X" where
X is the stringified version of the errno value.
See: https://bugs.torproject.org/31810
This patch removes a call to tor_assert_unreached() after execve()
failed. This assertion leads to the child process emitting a stack trace
on its standard output, which makes the error harder for the user to
demystify, since they think it is an internal error in Tor instead of
"just" being a "no such file or directory" error.
The process will now instead output "Error from child process: X" where
X is the stringified version of the errno value.
See: https://bugs.torproject.org/31810
When tearing down all periodic events during shutdown, disable them first so
their enable flag is updated.
This allows the tor_api.h to relaunch tor properly after a clean shutdown.
Fixes#32058
Signed-off-by: David Goulet <dgoulet@torproject.org>
Running doxygen with latex gave us all manner of unicode issues,
slowed down the "make doxygen" target by a lot, and added several
latex dependencies... all to produce a 4000-page reference manual
which is probably not what anybody wanted.
Closes ticket 32099.
Found by coverity CID 1454769.
There were a second possible leak that is also fixed in this commit.
Fixes#32063
Signed-off-by: David Goulet <dgoulet@torproject.org>
Code adapted from Rob's proposed patch in #30344.
Also add a comment in connection_mark_for_close_internal_() on why we should
not be adding extra code there without a very good reason.
When considering introduction point of a service's descriptor, do not remove
an intro point that has an established or pending circuit.
Fixes#31652
Signed-off-by: David Goulet <dgoulet@torproject.org>
When encoding introduction points, we were not checking if that intro points
had an established circuit.
When botting up, the service will pick, by default, 3 + 2 intro points and the
first 3 that establish, we use them and upload the descriptor.
However, the intro point is removed from the service descriptor list only when
the circuit has opened and we see that we have already enough intro points, it
is then removed.
But it is possible that the service establishes 3 intro points successfully
before the other(s) have even opened yet.
This lead to the service encoding extra intro points in the descriptor even
though the circuit is not opened or might never establish (#31561).
Fixes#31548
Signed-off-by: David Goulet <dgoulet@torproject.org>
These tests all invoke the hostname resolver in one way or another,
and therefore potentially block if our DNS server is missing,
absent, or extremely slow. Closes ticket 31841.
Our minimum version is now 0.2.9.5-alpha. Series 0.3.0, 0.3.1,
0.3.2, 0.3.3, and 0.3.4 are now rejected.
Also, extract this version-checking code into a new function, so we
can test it.
Closes ticket 31549.
Also reject 0.3.5.0 through 0.3.5.6-rc as unstable.
When pkg-config is not installed, or a library that depends on
pkg-config is not found, tell the user what to do to fix the
problem.
Fixes bug 31922; bugfix on 0.3.1.1-alpha.
The documentation for this function says that the smartlist can
contain NULLs, but the code only handled NULLs if they were at the
start of the list.
We didn't notice this for a long time, because when Tor is run
normally, the sequence of msg_id_t is densely packed, and so this
list (mapping msg_id_t to channel_id_t) contains no NULL elements.
We could only run into this bug:
* when Tor was running in embedded mode, and starting more than once.
* when Tor ran first with more pubsub messages enabled, and then
later with fewer.
* When the second run (the one with fewer enabled pubsub messages)
had at least some messages enabled, and those messages were not
the ones with numerically highest msg_id_t values.
Fixes bug 31898; bugfix on 47de9c7b0a
in 0.4.1.1-alpha.
Merge master's RUST_VERSION="nightly" into the macOS Rust job
from bug31859_041.
Drop master's RUST_VERSION="beta", because we only have two
Rust jobs now.
Since Travis macOS has IPv6 support (and Travis Linux does not), chutney
will now run its IPv6 networks as part of Travis CI.
But since chutney is slow, don't wait for the macOS chutney to finish.
(Travis have fixed the duplicate notification bug in fast_finish. So we
can use fast_finish and allow_failure to finish early. Unfortunately,
allow_failure also means we ignore failures in macOS chutney.)
Also make sure that we have:
* a compile on each platform, with each compiler,
* a check on each platform, and
* a check on each compiler.
Finally, sort builds: allow fail last, macOS first, slowest first.
Closes ticket 30860.
Closes ticket 31859 for 0.2.9.
This test failure happened due to a signed/unsigned integer
comparison.
This bug occurred on SunOS, it may also occur on other systems that
use signed char as the default. (And cast 1-byte integer constants
to an unsigned integer.)
Fixes bug 31897; bugfix on 0.4.1.1-alpha.
Frequently, when a patch fails, it has failures in several files.
Using the "-k" flag will let us learn all the compilation errors,
not just the first one that the compiler hits.
Based on a patch by rl1987.
Closes ticket 31372.
Frequently, when a patch fails, it has failures in several files.
Using the "-k" flag will let us learn all the compilation errors,
not just the first one that the compiler hits.
Based on a patch by rl1987.
- The function `decrypt_desc_layer` has a cleaner interface.
- `is_superencrypted_layer` changed from `int` -> `bool`
[ticket details](https://trac.torproject.org/projects/tor/ticket/31589)
add(changes/*): changes file
fix(src/features/hs): is_superencrypted changed from `int` -> `bool`
fix(changes/ticket31589): header
add(changes/ticket31589): subsystem(onion services) to change
When processing a %included folder, a bug caused the pointer to
the last element of the options list to be set to NULL when
processing a file with only comments or whitepace. This could
cause options from other files on the same folder to be
discarded depending on the lines after the affected %include.
This warning would previously be given every time we tried to open a
connection to a foo.exit address, which could potentially be used to
flood the logs. Now, we don't allow this warning to appear more
than once every 15 minutes.
Fixes bug 31466; bugfix on 0.2.2.1-alpha, when .exit was first
deprecated.
We want to forbid this pattern since, unlike the other log_*()
macros, log_debug() conditionally evaluates its arguments only if
debug-level logging is enabled. Thus, a call to
log_debug("%d", x++);
will only increment x if debugging logs are enabled, which is
probably not what the programmer intended.
One bug caused by this pattern was #30628.
This script detects log_debug( ) calls with any of E++, E--, ++E,
or --E in their arguments, where E is an arbitrary expression.
Closes ticket 30743.
Our dimap code asserts if you try to add the same key twice; this
can't happen if everything is running smoothly, but it's possible if
you try to start a relay where secret_onion_key_ntor is the same as
secret_onion_key_ntor.old.
Fixes bug 30916; bugfix on 0.2.4.8-alpha when ntor keys were
introduced.
We previously used tor_fragile_assert() to declare that this case
could not happen: VERSIONS cells are always supposed to be
variable-sized, right?
This is incorrect, though. On a v1 link protocol connection, all
cells are fixed-sized. There aren't supposed to be any VERSIONS
cells with this version of the protocol, but apparently, somebody
was messing up. (The v1 link protocol is obsolete, so probably the
implementer responsible didn't mean to be using it.)
Fixes bug 31107. Bugfix on 0.2.4.4-alpha, when we introduced a
tor_fragile_assert() for this case.
These errors can occur if we are built on a system with support for
madvise(MADV_NOFORK) but then we are run on a system whose kernel
does not support that flag.
If the error is something that we don't tolerate at all, we now log
it before crashing.
Fixes bug 31696. I am calling this a bugfix on 0.4.1.1-alpha, where
we actually started using the map_anon code.
This is similar to, but not the same as, the fix for #31570.
Our code assumes that when we're configured to get IPv6 addresses
out of a TRANS_PF transparent proxy connection, we actually will.
But we didn't check that, and so FreeBSD started warning us about a
potential NULL pointer dereference.
Fixes part of bug 31687; bugfix on 0.2.3.4-alpha when this code was
added.
We used to do this on Windows only, but it appears to affect
multiple platforms when building with certain versions of GCC, and a
common pattern for defining the floating-point classifier functions.
Fixes part of 31687. I'm calling this a bugfux on 31687, when we
started suppressing these warnings on Windows.
