Add a check-practracker-unit-test target, and run it as part of
"make check".
Add the practracker unit tests to scripts/git/pre-commit.git-hook.
Part of 32609.
Require C99 standards-conforming code in Travis CI, but allow GNU gcc
extensions. Also activates clang's -Wtypedef-redefinition warnings.
Builds some jobs with -std=gnu99, and some jobs without.
Closes ticket 32500.
Python 2 will be end-of-life as of 1 Jan 2020, so we can finally
stop supporting it. As a first step, we should make our configure
script stop accepting python 2 as something acceptable to run our
tests with.
Closes ticket 32608.
In #26913 we solved a bug where CacheDirectoryGroupReadable would
override DataDirectoryGroupReadable when the two directories are the
same. We never did the same for KeyDirectory, though, because
that's a rare setting.
Now that I'm testing this code, though, fixing this issue seems
fine. Fixes bug #27992; bugfix on 0.3.3.1-alpha.
When we added the $FILTER for Windows newlines, we made
the pipeline always exit successfully, even if tor failed.
Fixes bug 32468; bugfix on 0.4.2.1-alpha.
When a SOCKS5 client sends a RESOLVE_PTR request, it must include
either an IPv4 or IPv6 address. In the past this was required to be a
binary address (address types 1 or 4), but since the refactoring of
SOCKS5 support in Tor 0.3.5.1-alpha, strings (address type 3) are also
allowed if they represent an IPv4 or IPv6 literal.
However, when a binary IPv6 address is provided,
parse_socks5_client_request converts it into a string enclosed in
brackets. This doesn't match what string_is_valid_ipv6_address
expects, so this would fail with the error "socks5 received
RESOLVE_PTR command with hostname type. Rejecting."
By replacing string_is_valid_ipv4_address/string_is_valid_ipv6_address
with tor_addr_parse, we accept strings both with and without brackets.
This fixes the handling of binary addresses, and also improves
symmetry with CONNECT and RESOLVE requests.
Fixes bug 32315.
Install the mingw OpenSSL package in Appveyor. This makes sure that
the OpenSSL headers and libraries match in Tor's Appveyor builds.
(This bug was triggered by an Appveyor image update.)
Fixes bug 32449; bugfix on 0.3.5.6-rc.
It no longer warns, and is now defined in terms of an "IGNORE" type.
(The "IGNORE" type is the same as "OBSOLETE", except that it is not
reported as obsolete. It should be useful for disabled modules.)
Closes ticket 32404.
We still interpret "AccelName" as turning on the "HardwareAccel"
feature, but we no longer modify the user's options here.
Fixes bug 32382; bugfix on 0.2.2.1-alpha when we added openssl
engine support.
When the relay module is disabled, make "ClientOnly 1" and
"DirCache 0" by default. (But keep "ClientOnly 0" and
"DirCache 1" as the defaults for the unit tests.)
And run "make autostyle".
Part of ticket 32410.
Doing so caused us to crash in some unusual circumstances, such as
using --verify-config to verify a configuration that failed during
the options_act() stage.
Fixes bug 32407; bugfix on 0.3.3.1-alpha.
test_parseconf.sh now supports:
* {error,expected}{,_lzma,_nss,_zstd}{,_no_dirauth,_no_relay_dirauth}
Or any combination of two or more optional libraries.
Closes ticket 32397.
This check was accidentally disabled by a bad find command.
Fixes bug 32402; bugfix on 0.4.2.1-alpha.
Obviously correct changes to already reviewed code.
Up till now, we have warned about all missing documentation, which
meant that we could never make doxygen warnings fatal. This has led
to our doxygen output getting full of errors several times in the
past.
This commit changes our approach to doxygen warnings: missing
documentation warnings are only on when the user asks for them with
--enable-missing-doc-warnings. When that option is not present,
doxygen respects the --enable-fatal-warnings flag.
Closes ticket 32385.
The current pkg-config setup has no sense of whether it is cross-compiling,
so it will detect things on the build system that are not present or are
wrong for the host system. This forces the cross-compiling build to only
look for pkg-config .pc files in --prefix.
A version of this has been the setup for many years with the Android builds.
Fixes#32191
Signed-off-by: Hans-Christoph Steiner <hans@eds.org>
When picking an intro point from the service descriptor, the client failed to
lookup the failure cache.
It made an HS v2 client re-pick bad intro points for which we already know it
won't work in the first place.
Based on Neel Chauhan original patch.
Fixes#25568
Signed-off-by: David Goulet <dgoulet@torproject.org>
Before, when parsing memunits, if overflow occured it failed silently.
Use nowrap u64 math to detect overflow, compare to INT64_MAX and if
greater tell user and fail accordingly.
15000000.5 TB fails double check as it a greater floating number than
(double)INT64_MAX
8388608.1 TB passes double check because it falls in the same value as
(double)INT64_MAX (which is 2^63), but will fail the int check because
(uint64_t)d, which is 2^63, is strictly greater than 2^63-1 (INT64_MAX).
Fixes#30920
Signed-off-by: José M. Guisado <guigom@riseup.net>
Before inspecting the p_chan, we must check if the circuit is marked for close
because if it is the case, the channels are nullified from the circuit.
Several valid cases can mark the circuit for close of the directory
connection.
Fixes#31958
Signed-off-by: David Goulet <dgoulet@torproject.org>
hs_client_purge_state() and hs_cache_clean_as_client() can remove a descriptor
from the client cache with a NEWNYM or simply when the descriptor expires.
Which means that for an INTRO circuit being established during that time, once
it opens, we lookup the descriptor to get the IP object but hey surprised, no
more descriptor.
The approach here is minimalist that is accept the race and close the circuit
since we can not continue. Before that, the circuit would stay opened and the
client wait the SockTimeout.
Fixers #28970.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Only use the HS circuit map to know if an introduction circuit is established
or not. No need for a flag to keep state of something we already have in the
circuit map. Furthermore, the circuit map gets cleaned up properly so it will
always have the "latest truth".
This commit also removes a unit test that was testing specifically that flag
but now we rely solely on the HS circuit map which is also tested few lines
below the removed test.
Fixes#32094
Signed-off-by: David Goulet <dgoulet@torproject.org>
Our minimum version is now 0.2.9.5-alpha. Series 0.3.0, 0.3.1,
0.3.2, 0.3.3, and 0.3.4 are now rejected.
Also, extract this version-checking code into a new function, so we
can test it.
Closes ticket 31549.
Also reject 0.3.5.0 through 0.3.5.6-rc as unstable.
Make clear in the man page, in both the bandwidth section and the
accountingmax section, that Tor counts in powers of two, not
powers of ten: 1 GByte is 1024*1024*1024 bytes, not one billion
bytes.
Resolves ticket 32106.
Conflicts:
src/feature/dirparse/authcert_parse.c
src/feature/dirparse/ns_parse.c
src/feature/hs/hs_service.c
src/lib/conf/conftesting.h
src/lib/log/log.h
src/lib/thread/threads.h
src/test/test_options.c
These conflicts were mostly related to autostyle improvements, with
one or two due to doxygen fixes.
Relays now respect their AccountingMax bandwidth again. When relays
entered "soft" hibernation (which typically starts when we've hit
90% of our AccountingMax), we had stopped checking whether we should
enter hard hibernation. Soft hibernation refuses new connections and
new circuits, but the existing circuits can continue, meaning that
relays could have exceeded their configured AccountingMax.
This commit rolls back some of the cpu-saving fixes, where we tried
to avoid calling so many of our events while we're off the network.
That's because PERIODIC_EVENT_FLAG_NEED_NET checks net_is_disabled(),
which returns true even if we're only in soft hibernation.
Fixes bug 32108; bugfix on 0.4.0.1-alpha.
Create an optional relay module, which can be disabled using the
--disable-module-relay configure option. When it is set, also disable
the dirauth module.
Add a minimal implemention, which disables the relay and dircache modes
in tor.
Closes ticket 32123.
* actually sleep when tor has not logged anything
* log at debug level when waiting for tor to log something
* backslash-replace bad UTF-8 characters in logs
* format control messages as ASCII: tor does not accept UTF-8 control commands
Fixes bug 31837; bugfix on 0.3.5.1-alpha.
This patch removes an overly strict tor_assert() and an ignorable BUG()
expression. Both of these would trigger if a PT was unable to configure
itself during startup. The easy way to trigger this is to configure an
obfs4 bridge where you make the obfs4 process try to bind on a port
number under 1024.
See: https://bugs.torproject.org/31091
This patch removes a call to tor_assert_unreached() after execve()
failed. This assertion leads to the child process emitting a stack trace
on its standard output, which makes the error harder for the user to
demystify, since they think it is an internal error in Tor instead of
"just" being a "no such file or directory" error.
The process will now instead output "Error from child process: X" where
X is the stringified version of the errno value.
See: https://bugs.torproject.org/31810
This patch removes a call to tor_assert_unreached() after execve()
failed. This assertion leads to the child process emitting a stack trace
on its standard output, which makes the error harder for the user to
demystify, since they think it is an internal error in Tor instead of
"just" being a "no such file or directory" error.
The process will now instead output "Error from child process: X" where
X is the stringified version of the errno value.
See: https://bugs.torproject.org/31810
When tearing down all periodic events during shutdown, disable them first so
their enable flag is updated.
This allows the tor_api.h to relaunch tor properly after a clean shutdown.
Fixes#32058
Signed-off-by: David Goulet <dgoulet@torproject.org>
Running doxygen with latex gave us all manner of unicode issues,
slowed down the "make doxygen" target by a lot, and added several
latex dependencies... all to produce a 4000-page reference manual
which is probably not what anybody wanted.
Closes ticket 32099.
Found by coverity CID 1454769.
There were a second possible leak that is also fixed in this commit.
Fixes#32063
Signed-off-by: David Goulet <dgoulet@torproject.org>
Code adapted from Rob's proposed patch in #30344.
Also add a comment in connection_mark_for_close_internal_() on why we should
not be adding extra code there without a very good reason.
When considering introduction point of a service's descriptor, do not remove
an intro point that has an established or pending circuit.
Fixes#31652
Signed-off-by: David Goulet <dgoulet@torproject.org>
When encoding introduction points, we were not checking if that intro points
had an established circuit.
When botting up, the service will pick, by default, 3 + 2 intro points and the
first 3 that establish, we use them and upload the descriptor.
However, the intro point is removed from the service descriptor list only when
the circuit has opened and we see that we have already enough intro points, it
is then removed.
But it is possible that the service establishes 3 intro points successfully
before the other(s) have even opened yet.
This lead to the service encoding extra intro points in the descriptor even
though the circuit is not opened or might never establish (#31561).
Fixes#31548
Signed-off-by: David Goulet <dgoulet@torproject.org>
These tests all invoke the hostname resolver in one way or another,
and therefore potentially block if our DNS server is missing,
absent, or extremely slow. Closes ticket 31841.
Our minimum version is now 0.2.9.5-alpha. Series 0.3.0, 0.3.1,
0.3.2, 0.3.3, and 0.3.4 are now rejected.
Also, extract this version-checking code into a new function, so we
can test it.
Closes ticket 31549.
Also reject 0.3.5.0 through 0.3.5.6-rc as unstable.
When pkg-config is not installed, or a library that depends on
pkg-config is not found, tell the user what to do to fix the
problem.
Fixes bug 31922; bugfix on 0.3.1.1-alpha.
The documentation for this function says that the smartlist can
contain NULLs, but the code only handled NULLs if they were at the
start of the list.
We didn't notice this for a long time, because when Tor is run
normally, the sequence of msg_id_t is densely packed, and so this
list (mapping msg_id_t to channel_id_t) contains no NULL elements.
We could only run into this bug:
* when Tor was running in embedded mode, and starting more than once.
* when Tor ran first with more pubsub messages enabled, and then
later with fewer.
* When the second run (the one with fewer enabled pubsub messages)
had at least some messages enabled, and those messages were not
the ones with numerically highest msg_id_t values.
Fixes bug 31898; bugfix on 47de9c7b0a
in 0.4.1.1-alpha.
Merge master's RUST_VERSION="nightly" into the macOS Rust job
from bug31859_041.
Drop master's RUST_VERSION="beta", because we only have two
Rust jobs now.
Since Travis macOS has IPv6 support (and Travis Linux does not), chutney
will now run its IPv6 networks as part of Travis CI.
But since chutney is slow, don't wait for the macOS chutney to finish.
(Travis have fixed the duplicate notification bug in fast_finish. So we
can use fast_finish and allow_failure to finish early. Unfortunately,
allow_failure also means we ignore failures in macOS chutney.)
Also make sure that we have:
* a compile on each platform, with each compiler,
* a check on each platform, and
* a check on each compiler.
Finally, sort builds: allow fail last, macOS first, slowest first.
Closes ticket 30860.
Closes ticket 31859 for 0.2.9.
This test failure happened due to a signed/unsigned integer
comparison.
This bug occurred on SunOS, it may also occur on other systems that
use signed char as the default. (And cast 1-byte integer constants
to an unsigned integer.)
Fixes bug 31897; bugfix on 0.4.1.1-alpha.
Frequently, when a patch fails, it has failures in several files.
Using the "-k" flag will let us learn all the compilation errors,
not just the first one that the compiler hits.
Based on a patch by rl1987.
Closes ticket 31372.
Frequently, when a patch fails, it has failures in several files.
Using the "-k" flag will let us learn all the compilation errors,
not just the first one that the compiler hits.
Based on a patch by rl1987.
- The function `decrypt_desc_layer` has a cleaner interface.
- `is_superencrypted_layer` changed from `int` -> `bool`
[ticket details](https://trac.torproject.org/projects/tor/ticket/31589)
add(changes/*): changes file
fix(src/features/hs): is_superencrypted changed from `int` -> `bool`
fix(changes/ticket31589): header
add(changes/ticket31589): subsystem(onion services) to change
When processing a %included folder, a bug caused the pointer to
the last element of the options list to be set to NULL when
processing a file with only comments or whitepace. This could
cause options from other files on the same folder to be
discarded depending on the lines after the affected %include.
This warning would previously be given every time we tried to open a
connection to a foo.exit address, which could potentially be used to
flood the logs. Now, we don't allow this warning to appear more
than once every 15 minutes.
Fixes bug 31466; bugfix on 0.2.2.1-alpha, when .exit was first
deprecated.
We want to forbid this pattern since, unlike the other log_*()
macros, log_debug() conditionally evaluates its arguments only if
debug-level logging is enabled. Thus, a call to
log_debug("%d", x++);
will only increment x if debugging logs are enabled, which is
probably not what the programmer intended.
One bug caused by this pattern was #30628.
This script detects log_debug( ) calls with any of E++, E--, ++E,
or --E in their arguments, where E is an arbitrary expression.
Closes ticket 30743.
Our dimap code asserts if you try to add the same key twice; this
can't happen if everything is running smoothly, but it's possible if
you try to start a relay where secret_onion_key_ntor is the same as
secret_onion_key_ntor.old.
Fixes bug 30916; bugfix on 0.2.4.8-alpha when ntor keys were
introduced.
We previously used tor_fragile_assert() to declare that this case
could not happen: VERSIONS cells are always supposed to be
variable-sized, right?
This is incorrect, though. On a v1 link protocol connection, all
cells are fixed-sized. There aren't supposed to be any VERSIONS
cells with this version of the protocol, but apparently, somebody
was messing up. (The v1 link protocol is obsolete, so probably the
implementer responsible didn't mean to be using it.)
Fixes bug 31107. Bugfix on 0.2.4.4-alpha, when we introduced a
tor_fragile_assert() for this case.
These errors can occur if we are built on a system with support for
madvise(MADV_NOFORK) but then we are run on a system whose kernel
does not support that flag.
If the error is something that we don't tolerate at all, we now log
it before crashing.
Fixes bug 31696. I am calling this a bugfix on 0.4.1.1-alpha, where
we actually started using the map_anon code.
This is similar to, but not the same as, the fix for #31570.
Our code assumes that when we're configured to get IPv6 addresses
out of a TRANS_PF transparent proxy connection, we actually will.
But we didn't check that, and so FreeBSD started warning us about a
potential NULL pointer dereference.
Fixes part of bug 31687; bugfix on 0.2.3.4-alpha when this code was
added.
We used to do this on Windows only, but it appears to affect
multiple platforms when building with certain versions of GCC, and a
common pattern for defining the floating-point classifier functions.
Fixes part of 31687. I'm calling this a bugfux on 31687, when we
started suppressing these warnings on Windows.
Fix levels for subsystems that depend on log/err
* winprocess (security) doesn't use err:
* call windows process security APIs as early as possible
* init err after winprocess
* move wallclock so it's still after err
* network and time depend on log:
* make sure that network and time can use logging.
* init network and time after log
Add comments explaining the module init order.
Fixes bug 31615; bugfix on 0.4.0.1-alpha.
When tor is missing descriptors for some primary entry guards, make the
log message less alarming. It's normal for descriptors to expire, as long
as tor fetches new ones soon after.
Fixes bug 31657; bugfix on 0.3.3.1-alpha.
tor-github/pr/1283, with the following changes:
* cherry-pick the merge commit in 1283 on top of...
* tor-github/pr/1174, but with the last commit re-worded to remove
the fixup, because fixups break our push rules.
This is an "ours" merge, except for the bugfix version change in
changes/bug30649.
When we parse a CLEAR line (e.g., "/OrPort" or /OrPort blah blah"),
we always suppress the value, even if one exists. That means that
the block of code was meant to handle CLEAR lines didn't actually do
anything, since we previously handled them the same way as with
other empty values.
Closes ticket 31529.
These errors can occur if we are built on a system with support for
madvise(MADV_NOFORK) but then we are run on a system whose kernel
does not support that flag.
If the error is something that we don't tolerate at all, we now log
it before crashing.
Fixes bug 31570. I am calling this a bugfix on 0.4.1.1-alpha, where
we actually started using the map_anon code.
Some platforms (macOS, maybe others?) can swallow the last write before an
abort. This issue is probably caused by a race condition between write
buffer cache flushing, and process termination. So we write an extra
newline, to make sure that the message always gets through.
Fixes bug 31571; bugfix on 0.3.5.1-alpha.
This fixes LTO compilation for Android and -O0 compilation in
general, when --disable-module-dirauth is provided.
Fixes bug 31552; bugfix on 0.4.1.1-alpha.
Skip test branches that are the same as remote maint/release/master
branches.
Add a TOR_PUSH_SAME and -s argument to git-push-all.sh to change this
default.
Part of 31314.
TOR_GIT_PUSH provides the git push command and default arguments.
Also fix handling of git-push-all.sh script arguments and arguments that
are passed through to $TOR_GIT_PUSH, using a "--" argument as a separator.
Fix on 29879.
Add a -t <test-branch-prefix> argument to git-merge-forward.sh and
git-push-all.sh, which makes these scripts create, merge forward, and
push test branches.
Add a -r <remote-name> argument to git-push-all.sh, so the script can
push test branches to a personal remote.
Closes ticket 31314.
