Only use the HS circuit map to know if an introduction circuit is established
or not. No need for a flag to keep state of something we already have in the
circuit map. Furthermore, the circuit map gets cleaned up properly so it will
always have the "latest truth".
This commit also removes a unit test that was testing specifically that flag
but now we rely solely on the HS circuit map which is also tested few lines
below the removed test.
Fixes#32094
Signed-off-by: David Goulet <dgoulet@torproject.org>
Our minimum version is now 0.2.9.5-alpha. Series 0.3.0, 0.3.1,
0.3.2, 0.3.3, and 0.3.4 are now rejected.
Also, extract this version-checking code into a new function, so we
can test it.
Closes ticket 31549.
Also reject 0.3.5.0 through 0.3.5.6-rc as unstable.
There was one that it could not find because it was in a macro definition.
I used the following semantic patch:
@@
expression e1, e2, e3, e4, e5;
@@
options_validate(e1,
e2,
- e3,
- e4,
e5)
Conflicts:
src/feature/dirparse/authcert_parse.c
src/feature/dirparse/ns_parse.c
src/feature/hs/hs_service.c
src/lib/conf/conftesting.h
src/lib/log/log.h
src/lib/thread/threads.h
src/test/test_options.c
These conflicts were mostly related to autostyle improvements, with
one or two due to doxygen fixes.
Since each of these tests only applies to testing networks, put them
all into a single block that checks for testing networks.
(I recommend reviewing with the "diff -b" option, since the change
is mostly indentation.)
Two things needed to be changed. First, we used to set quiet_level
to the default (QUIET_NONE) when running tests, since we would not
call anything that acted based upon it. But since we sometimes call
options_init_logs(), we need to pre-set quiet_level to QUIET_SILENT
in the logs so that we don't add the default logs. This did not
cause test failure: just unwanted logs.
Second, we had a test that checked whether options_validate was
messing with options->Logs correctly. Since options_validate no
longer messes with the logs, we no longer want a test for this.
Formerly, we would use quiet_level as an excuse to rewrite the log
configuration, adding a default log line if none existed, and if
RunAsDaemon was not set, and if we were not being invoked via
setconf (!).
This is against our best practices for several reasons:
* We should not be changing configured options except when the
user tells us to do so.
* We should especially not be changing options in the options_validate
function.
* Distinguishing whether we are being called from setconf adds a
risky special-case.
Instead, this patch take a simpler approach: it changes the
interpretation of having no logging lines set to mean: If there is a
stdout, add a default log based on quiet_level.
Solves ticket 31999.
Since this code passes the same options to options_validate() more
than once, options_validate() needs to be prepared for that. (This
previously worked by accident, since the smartlist of schedulers
wasn't initialized.)
This finally became the empty string, since we no longer have to do
anything in individual test_options.c tests to make "" be a valid
set of options. Now we can remove it at last.
Now that ConnLimit is set to the default value in the
testing helper functions, the individual tests don't all need to
make sure it is set to something valid.
Several of our tests assumed that ConnLimit would be set to 0 by
default, causing the default options not to be parseable. These
tests had nothing to do with ConnLimit.
Now that MaxClientCircuitsPending is set to the default value in the
testing helper functions, the individual tests don't all need to
make sure it is set to something valid.
Several of our tests assumed that MaxClientCircuitsPending would be
set to 0 by default, causing the default options not to be
parseable. These tests had nothing to do with
MaxClientCircuitsPending.
Now that KeepalivePeriod is set to the default value in the testing
helper functions, the individual tests don't all need to make sure
it is set to something valid.
Several of our tests assumed that KeepalivePeriod would be set to 0
by default, causing the default options not to be parseable. These
tests had nothing to do with KeepalivePeriod.
This macro used to have a big list of "default" values that we
needed to set in the test_options tests in order to have reasonable
behavior. But now that we initialize options objects to the default
settings in these tests, we no longer need such a long list of
things to replace.
Previously, we just used options set to all 0s, but this causes some
pretty severe workarounds throughout the code, as our options fail
to validate, or count as "default" for the wrong reasons.
Note that in some places, we stop getting spurious warnings or
failures which the tests previously demanded. In these cases, I've
changed the test behavior.
Fixes 32175.
Relays now respect their AccountingMax bandwidth again. When relays
entered "soft" hibernation (which typically starts when we've hit
90% of our AccountingMax), we had stopped checking whether we should
enter hard hibernation. Soft hibernation refuses new connections and
new circuits, but the existing circuits can continue, meaning that
relays could have exceeded their configured AccountingMax.
This commit rolls back some of the cpu-saving fixes, where we tried
to avoid calling so many of our events while we're off the network.
That's because PERIODIC_EVENT_FLAG_NEED_NET checks net_is_disabled(),
which returns true even if we're only in soft hibernation.
Fixes bug 32108; bugfix on 0.4.0.1-alpha.
* actually sleep when tor has not logged anything
* log at debug level when waiting for tor to log something
* backslash-replace bad UTF-8 characters in logs
* format control messages as ASCII: tor does not accept UTF-8 control commands
Fixes bug 31837; bugfix on 0.3.5.1-alpha.
This patch removes an overly strict tor_assert() and an ignorable BUG()
expression. Both of these would trigger if a PT was unable to configure
itself during startup. The easy way to trigger this is to configure an
obfs4 bridge where you make the obfs4 process try to bind on a port
number under 1024.
See: https://bugs.torproject.org/31091
This patch adds a test to check for whether the exit callback is always
called when process_exec() fails, both on Windows and Unix.
See: https://bugs.torproject.org/31810
This patch fixes an issue where the exit handler is not called for the
given process_t in case CreateProcessA() fails. This could, for example,
happen if the user tries to execute a binary that does not exist.
See: https://bugs.torproject.org/31810
This patch removes a call to tor_assert_unreached() after execve()
failed. This assertion leads to the child process emitting a stack trace
on its standard output, which makes the error harder for the user to
demystify, since they think it is an internal error in Tor instead of
"just" being a "no such file or directory" error.
The process will now instead output "Error from child process: X" where
X is the stringified version of the errno value.
See: https://bugs.torproject.org/31810
We used to have this function so that we could mark our initial
log-to-stdout as specifically temporary so that we would delete it
once regular logs were configured. But it's no longer necessary to
mark these logs as temporary, since we now use a mark-and-sweep
process to ensure that _all_ not-configured logs are closed when we
change our configuration.
Instead, this function will be the basis of a refactoring in how we
handle default logs.
Previously this was done with a big list of options in main.c which
implied "hush" or "quiet". One of these options ("--digests") no
longer existed, but we still checked for it.
Now we use the table of command-line-only arguments to set this
value.
Previously these were implemented with a search in
options_init_from_torrc(), but that led to each option being
declared more than needed: once to say that it was a valid option,
and once to say what it meant.
This commit introduces the hs_desc_decode_status_t enum which aims at having
more fine grained error code when decoding a descriptor.
This will be useful in later commits when we support keeping a descriptor that
can't be decrypted due to missing or bad client authorization creds.
No behavior change.
Part of #30382.
Signed-off-by: David Goulet <dgoulet@torproject.org>
No code behavior change. This removes duplicate code that was finding all
entry connections for a specific onion service identity key.
The find_entry_conns() helper function is introduced for that.
Part of #30382
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch adds a test to check for whether the exit callback is always
called when process_exec() fails, both on Windows and Unix.
See: https://bugs.torproject.org/31810
This patch fixes an issue where the exit handler is not called for the
given process_t in case CreateProcessA() fails. This could, for example,
happen if the user tries to execute a binary that does not exist.
See: https://bugs.torproject.org/31810
This patch removes a call to tor_assert_unreached() after execve()
failed. This assertion leads to the child process emitting a stack trace
on its standard output, which makes the error harder for the user to
demystify, since they think it is an internal error in Tor instead of
"just" being a "no such file or directory" error.
The process will now instead output "Error from child process: X" where
X is the stringified version of the errno value.
See: https://bugs.torproject.org/31810
When tearing down all periodic events during shutdown, disable them first so
their enable flag is updated.
This allows the tor_api.h to relaunch tor properly after a clean shutdown.
Fixes#32058
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit defines the new extended error codes. It also flags the socks
request object that it can use them.
Part of #30382
Signed-off-by: David Goulet <dgoulet@torproject.org>
This new flag tells tor that it can send back the SOCKS5 extended error code
detailed in prop304.
Part of #30382
Signed-off-by: David Goulet <dgoulet@torproject.org>
In case of error, a negative value will be returned or NULL written into
first supplied argument.
This patch uses both cases to comply with style in the specific files.
A tor_vasprintf error in process_vprintf would lead to a NULL dereference
later on in buf_add, because the return value -1 casted to size_t would
pass an assertion check inside of buf_add.
On the other hand, common systems will fail on such an operation, so it
is not a huge difference to a simple assertion. Yet it is better to
properly fail instead of relying on such behaviour on all systems.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Found by coverity CID 1454769.
There were a second possible leak that is also fixed in this commit.
Fixes#32063
Signed-off-by: David Goulet <dgoulet@torproject.org>
Code adapted from Rob's proposed patch in #30344.
Also add a comment in connection_mark_for_close_internal_() on why we should
not be adding extra code there without a very good reason.
When considering introduction point of a service's descriptor, do not remove
an intro point that has an established or pending circuit.
Fixes#31652
Signed-off-by: David Goulet <dgoulet@torproject.org>
I'm doing this for consistency, so that all the values for this enum
have the same prefix.
This is an automated commit, generated by the following shell commands:
for fn in $(git ls-tree --name-only -r HEAD src |grep '\.[ch]$'); do \
perl -i -pe 's!\bTAKES_NO_ARGUMENT\b!ARGUMENT_NONE!g;' "$fn"; \
done
When encoding introduction points, we were not checking if that intro points
had an established circuit.
When botting up, the service will pick, by default, 3 + 2 intro points and the
first 3 that establish, we use them and upload the descriptor.
However, the intro point is removed from the service descriptor list only when
the circuit has opened and we see that we have already enough intro points, it
is then removed.
But it is possible that the service establishes 3 intro points successfully
before the other(s) have even opened yet.
This lead to the service encoding extra intro points in the descriptor even
though the circuit is not opened or might never establish (#31561).
Fixes#31548
Signed-off-by: David Goulet <dgoulet@torproject.org>
These tests all invoke the hostname resolver in one way or another,
and therefore potentially block if our DNS server is missing,
absent, or extremely slow. Closes ticket 31841.
Our minimum version is now 0.2.9.5-alpha. Series 0.3.0, 0.3.1,
0.3.2, 0.3.3, and 0.3.4 are now rejected.
Also, extract this version-checking code into a new function, so we
can test it.
Closes ticket 31549.
Also reject 0.3.5.0 through 0.3.5.6-rc as unstable.
We have a getaddrinfo() implementation that we prefer, and a
gethostbyname*() implementation that we fall back on. Give them
both the same interface, and let them be called by the same name.
This is a preparatory step for making them both mockable.
The documentation for this function says that the smartlist can
contain NULLs, but the code only handled NULLs if they were at the
start of the list.
We didn't notice this for a long time, because when Tor is run
normally, the sequence of msg_id_t is densely packed, and so this
list (mapping msg_id_t to channel_id_t) contains no NULL elements.
We could only run into this bug:
* when Tor was running in embedded mode, and starting more than once.
* when Tor ran first with more pubsub messages enabled, and then
later with fewer.
* When the second run (the one with fewer enabled pubsub messages)
had at least some messages enabled, and those messages were not
the ones with numerically highest msg_id_t values.
Fixes bug 31898; bugfix on 47de9c7b0a
in 0.4.1.1-alpha.
There is a bad design choice in two of our configuration types,
where the empty string encodes a value that is not the same as the
default value. This design choice, plus an implementation mistake,
meant that config_dup() did not preserve the value of routerset_t,
and thereby caused bug #31495.
This comment-only patch documents the two types with the problem,
and suggests that implementors try to avoid it in the future.
Closes ticket 31907.
This test failure happened due to a signed/unsigned integer
comparison.
This bug occurred on SunOS, it may also occur on other systems that
use signed char as the default. (And cast 1-byte integer constants
to an unsigned integer.)
Fixes bug 31897; bugfix on 0.4.1.1-alpha.
The log mutex is dynamically initialized, guarded by log_mutex_initialized.
We don't want to destroy it, because after it is destroyed, we won't see
any more logs.
If tor is re-initialized, log_mutex_initialized will still be 1. So we
won't trigger any undefined behaviour by trying to re-initialize the
log mutex.
Part of 31736, but committed in this branch to avoid merge conflicts.
cb_buf_mutex is statically initialised, so we can not destroy it when
we are shutting down the err subsystem. If we destroy it, and then
re-initialise tor, all our backtraces will fail.
Part of 31736, but committed in this branch to avoid merge conflicts.
Move SEVERITY_MASK_IDX() to log.h private/unit tests section, so that
we can use it in log.c, the unit tests, and the fuzzers.
(The test and fuzzer code changes are in a subsequent commit.)
Preparation for bug 31334.
- The function `decrypt_desc_layer` has a cleaner interface.
- `is_superencrypted_layer` changed from `int` -> `bool`
[ticket details](https://trac.torproject.org/projects/tor/ticket/31589)
add(changes/*): changes file
fix(src/features/hs): is_superencrypted changed from `int` -> `bool`
fix(changes/ticket31589): header
add(changes/ticket31589): subsystem(onion services) to change
When processing a %included folder, a bug caused the pointer to
the last element of the options list to be set to NULL when
processing a file with only comments or whitepace. This could
cause options from other files on the same folder to be
discarded depending on the lines after the affected %include.
The code here parses the fields from the microdescriptor, including
possible annotations, and stores them into a microdesc_t object.
This commit is almost pure code movement; I recommend using
--color-moved to review it.
This code is logically independent of the rest of the function, and
goes better in its own function.
This is almost purely code movement; I suggest reviewing with
--color-moved.
This warning would previously be given every time we tried to open a
connection to a foo.exit address, which could potentially be used to
flood the logs. Now, we don't allow this warning to appear more
than once every 15 minutes.
Fixes bug 31466; bugfix on 0.2.2.1-alpha, when .exit was first
deprecated.
Our dimap code asserts if you try to add the same key twice; this
can't happen if everything is running smoothly, but it's possible if
you try to start a relay where secret_onion_key_ntor is the same as
secret_onion_key_ntor.old.
Fixes bug 30916; bugfix on 0.2.4.8-alpha when ntor keys were
introduced.
These macros are used in multiple functions, and as such really
don't belong within a single function.
Also #undef them once we are done with them.
This change makes practracker pass again.
We previously used tor_fragile_assert() to declare that this case
could not happen: VERSIONS cells are always supposed to be
variable-sized, right?
This is incorrect, though. On a v1 link protocol connection, all
cells are fixed-sized. There aren't supposed to be any VERSIONS
cells with this version of the protocol, but apparently, somebody
was messing up. (The v1 link protocol is obsolete, so probably the
implementer responsible didn't mean to be using it.)
Fixes bug 31107. Bugfix on 0.2.4.4-alpha, when we introduced a
tor_fragile_assert() for this case.
There seems to be some unreliability issue with this test on
appveyor.
Addresses ticket 31757; This isn't a final fix for this issue, but
it should make CI pass.
This script takes a set of example torrcs and command-lines from
src/test/conf_examples. If a success is expected, it runs "tor
--dump-config" and compares the result with the one we expect. If a
failure is expected, it runs "tor --verify-config" and greps for the
error we expect.
GCC complains that we are using too many variables here, probably
because of the sheer number of locals used for our tinytest macros.
Eventually we should fix that (see 30968), but this commit just
makes the "note" go away by splitting the test function into two.
Coccinelle doesn't understand it when we use "==" and "!=" and so on as
arguments to macros. To solve this, we prefer OP_EQ, OP_NE, and so
on.
This commit is automatically generated by running
./scripts/coccinelle/test_operator_cleanup over all of the source
code in src.
Here we make it clear we're only looking at listable variable names,
not at whether the variables themselves are gettable.
Also, remove an extraneous h.
(This commit is not a fixup, because of rebase conflicts.)
Since the flags are now stored with compatible numbering, we can
just OR them together and see whether the flag we want is in the
result.
(Net code removal!)
Using a bitfield here will enable us to unify the var_type_def_t flags
with the config_var_t flags.
(This commit does not yet do that unification, and does not yet
rename or refactor any flags. It only changes booleans into bits.)
Previously they checked the individual flags inside var_type_def_t;
now they call the appropriate var_type_is_*() functions.
(These functions will be removed entirely by the end of this branch.)
We had though to make all obsolete and invisible variables
ungettable, so that GETCONF would reject them. But it turns out
that this isn't the current behavior of GETCONF with those
variables. So for now, I'm leaving the current behavior unchanged.
(See ticket 31647 for a proposal to change the behavior.)
These errors can occur if we are built on a system with support for
madvise(MADV_NOFORK) but then we are run on a system whose kernel
does not support that flag.
If the error is something that we don't tolerate at all, we now log
it before crashing.
Fixes bug 31696. I am calling this a bugfix on 0.4.1.1-alpha, where
we actually started using the map_anon code.
This is similar to, but not the same as, the fix for #31570.
Our code assumes that when we're configured to get IPv6 addresses
out of a TRANS_PF transparent proxy connection, we actually will.
But we didn't check that, and so FreeBSD started warning us about a
potential NULL pointer dereference.
Fixes part of bug 31687; bugfix on 0.2.3.4-alpha when this code was
added.
We used to do this on Windows only, but it appears to affect
multiple platforms when building with certain versions of GCC, and a
common pattern for defining the floating-point classifier functions.
Fixes part of 31687. I'm calling this a bugfux on 31687, when we
started suppressing these warnings on Windows.
We have code in microdescs_parse_from_string() to record the digests
of microdescriptors that we could not parse. But right now, that
code looks at the md->digest field, which is a bit inelegant, and
will stand in the way of sensible refactoring.
Instead, use a local variable to hold the digest.
This test makes sure that we parse ed25519 identities to get the
correct data from them. It also tests:
* That a microdescriptor may not have two ed25519 identities.
* That a microdescriptor may not have an ed25519 identity that is
not a valid base64-encoded ed25519 key.
* That a microdescriptor may have an unrecognized identity type.
It will help test the refactoring of ticket31675.
These levels get out of date really easily: we'll implement a level
dump command in tor in 31614.
They also cause conflicts and inconsistencies when merging forward
level changes.
Part of 31615.
Fix levels for subsystems that depend on log/err
* winprocess (security) doesn't use err:
* call windows process security APIs as early as possible
* init err after winprocess
* move wallclock so it's still after err
* network and time depend on log:
* make sure that network and time can use logging.
* init network and time after log
Add comments explaining the module init order.
Fixes bug 31615; bugfix on 0.4.0.1-alpha.
When tor is missing descriptors for some primary entry guards, make the
log message less alarming. It's normal for descriptors to expire, as long
as tor fetches new ones soon after.
Fixes bug 31657; bugfix on 0.3.3.1-alpha.
Now that the variants of these functions that took config_line_t are
gone, there is no longer any reason for the remaining variants to
have "ex" at the end of their names.
This commit was made by running this perl script over all the files
in src/:
#!/usr/bin/perl -w -i -p
s{typed_var_(assign|free|encode|copy|eq|ok|kvassign|kvencode|mark_fragile)_ex}
{typed_var_$1}g;
Previously we used int in some places and off_t for others. Neither
is correct: ptrdiff_t is right for differences between pointers.
(off_t is only for offsets and sizes on the filesystem.)
Also add an explanation of a possible future refactoring where we
might remove the config_type_t enumeration entierly.
Fixes ticket 31624.
No changes file, since this is a comment-only change.
When we parse a CLEAR line (e.g., "/OrPort" or /OrPort blah blah"),
we always suppress the value, even if one exists. That means that
the block of code was meant to handle CLEAR lines didn't actually do
anything, since we previously handled them the same way as with
other empty values.
Closes ticket 31529.
Previously we used int here, but it is more correct to use
ptrdiff_t. (This never actually matters for our code in practice,
since the structure we are managing here never exceed INT_MAX in
size.)
We can't use strlcat() or strlcpy() in torerr, because they are defined
in string/compat_string.h on some platforms, and string uses torerr.
Part of 31571.
These errors can occur if we are built on a system with support for
madvise(MADV_NOFORK) but then we are run on a system whose kernel
does not support that flag.
If the error is something that we don't tolerate at all, we now log
it before crashing.
Fixes bug 31570. I am calling this a bugfix on 0.4.1.1-alpha, where
we actually started using the map_anon code.
Some platforms (macOS, maybe others?) can swallow the last write before an
abort. This issue is probably caused by a race condition between write
buffer cache flushing, and process termination. So we write an extra
newline, to make sure that the message always gets through.
Fixes bug 31571; bugfix on 0.3.5.1-alpha.
We want to report the tor version, even on platforms that don't have
backtrace support (like Android).
This commit stores the backtrace Tor version, regardless of USE_BACKTRACE.
Preparation for 31571.
This fixes LTO compilation for Android and -O0 compilation in
general, when --disable-module-dirauth is provided.
Fixes bug 31552; bugfix on 0.4.1.1-alpha.
Rewrite format_node_description() and router_get_verbose_nickname() to
use strlcpy() and strlcat(). The previous implementation used memcpy()
and pointer arithmetic, which was error-prone.
Closes ticket 31545. This is CID 1452819.
These functions are all used to implement the ROUTERSET_type_defn
object, which maps strings to and from routerset_t configuration
variables for the configuration module.
routerset_t has two representations of an empty routerset: NULL, and
a set containing no elements. But some of our config code assumes
that empty routersets are represented as NULL. So let's give it
what it assumes.
Fixes bug 31495. Bugfix on e16b90b88a76; but not in any released
Tor.
A configuration manager, in addition to a top-level format object,
may now also know about a suite of sub-formats. Top-level
configuration objects, in turn, may now have a suite of
sub-objects.
