This patch refactors our compression tests such that deflate, gzip,
lzma, and zstd are all tested using the same code.
Additionally we use run-time checks to see if the given compression
method is supported instead of using HAVE_LZMA and HAVE_ZSTD.
See: https://bugs.torproject.org/22085
Since we have a streaming API for each compression backend, we don't
need a non-streaming API for each: we can build a common
non-streaming API at the front-end.
This commit adds the src/trace directory containing the basics for our tracing
subsystem. It is not used in the code base. The "src/trace/debug.h" file
contains an example on how we can map our tor trace events to log_debug().
The tracing subsystem can only be enabled by tracing framework at compile
time. This commit introduces the "--enable-tracing-debug" option that will
make all "tor_trace()" function be maped to "log_debug()".
Closes#13802
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch adds support for checking if a given `compress_method_t` is
supported by the currently running Tor instance using
`tor_compress_supports_method()`.
See: https://bugs.torproject.org/21662
This patch adds support for enabling support for Zstandard to our configure
script. By default, the --enable-zstd option is set to "auto" which means if
libzstd is available we'll build Tor with Zstandard support.
See: https://bugs.torproject.org/21662
This patch adds support for enabling support for LZMA to our configure
script. By default, the --enable-lzma option is set to "auto" which
means if liblzma is available we'll build Tor with LZMA support.
See: https://bugs.torproject.org/21662
The item referred to the cdm_ht_set_status() case where the item was
not already in the hashtable. But that already happens naturally
when we scan the directory on startup... and we already have a test
for that.
This commit adds some helper functions to look up the diff from one
consensus and to make sure that applying it leads to another. Then
we add them throughout the existing test cases. Doing this turned
up a reference-leaking bug in consensus_diff_worker_replyfn.
Initial tests. These just try adding a few consensuses, looking
them up, and making sure that consensus diffs are generated in a
more or less reasonable-looking way. It's enough for 87% coverage,
but it leaves out a lot of functionality.
This patch refactors our streaming compression code to allow us to
extend it with non-zlib/non-gzip based compression schemas.
See https://bugs.torproject.org/21663
To allow us to use the API name `tor_compress` and `tor_uncompress` as
the main entry-point for all compression/uncompression and not just gzip
and zlib.
See https://bugs.torproject.org/21663
This patch changes some of the tt_assert() usage in test_util_gzip() to
use tt_int_op() to get better error messages upon failure.
Additionally we move to use explicit NULL checks.
See https://bugs.torproject.org/21663
The consdiff generation logic would skip over lines added at the start of the
second file, and generate a diff that it would the immediately refuse because
it couldn't be used to reproduce the second file from the first. Fixes#21996.
The reason for making the temporary list public is to keep it encapsulated in
the rendservice subsystem so the prop224 code does not have direct access to
it and can only affect it through the rendservice pruning function.
It also has been modified to not take list as arguments but rather use the
global lists (main and temporary ones) because prop224 code will call it to
actually prune the rendservice's lists. The function does the needed rotation
of pointers between those lists and then prune if needed.
In order to make the unit test work and not completely horrible, there is a
"impl_" version of the function that doesn't free memory, it simply moves
pointers around. It is directly used in the unit test and two setter functions
for those lists' pointer have been added only for unit test.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Now we have separate getters and setters for service-side and relay-side. I
took this approach over adding arguments to the already existing methods to
have more explicit type-checking, and also because some functions would grow
too large and dirty.
This commit also fixes every callsite to use the new function names which
modifies the legacy HS (v2) and the prop224 (v3) code.
Signed-off-by: David Goulet <dgoulet@torproject.org>
One of the goals of this change is to have trunnel API/ABI being more explicit
so we namespace them with "trn_*". Furthermore, we can now create
hs_cells.[ch] without having to confuse it with trunnel which used to be
"hs_cell_*" before that change.
Here are the perl line that were used for this rename:
perl -i -pe 's/cell_extension/trn_cell_extension/g;' src/*/*.[ch]
perl -i -pe 's/cell_extension/trn_cell_extension/g;' src/trunnel/hs/*.trunnel
perl -i -pe 's/hs_cell_/trn_cell_/g;' src/*/*.[ch]
perl -i -pe 's/hs_cell_/trn_cell_/g;' src/trunnel/hs/*.trunnel
And then "./scripts/codegen/run_trunnel.sh" with trunnel commit id
613fb1b98e58504e2b84ef56b1602b6380629043.
Fixes#21919
Signed-off-by: David Goulet <dgoulet@torproject.org>
These tests tried to use ridiculously large buffer sizes to check
the sanity-checking in the code; but since the sanity-checking
changed, these need to change too.
Test base64_decode() with odd sized decoded lengths, including
unpadded encodings and padded encodings with "right-sized" output
buffers. Convert calls to base64_decode_nopad() to base64_decode()
because base64_decode_nopad() is redundant.
Check that route_len_for_purpose() (helper for new_route_len())
correctly fails a non-fatal bug assertion if it encounters an
unhandled circuit purpose when it is called with exit node info.
The old implementation had duplicated code in a bunch of places, and
it interspersed spool-management with resource management. The new
implementation should make it easier to add new resource types and
maintain the spooling code.
Closing ticket 21651.
This patch turns `MIN_ONION_KEY_LIFETIME` into a new function
`get_onion_key_lifetime()` which gets its value from a network consensus
parameter named "onion-key-rotation-days". This allows us to tune the
value at a later point in time with no code modifications.
We also bump the default onion key lifetime from 7 to 28 days as per
proposal #274.
See: https://bugs.torproject.org/21641
This patch fixes a regression described in bug #21757 that first
appeared after commit 6e78ede73f which was an attempt to fix bug #21654.
When switching from buffered I/O to direct file descriptor I/O our
output strings from get_string_from_pipe() might contain newline
characters (\n). In this patch we modify tor_get_lines_from_handle() to
ensure that the function splits the newly read string at the newline
character and thus might return multiple lines from a single call to
get_string_from_pipe().
Additionally, we add a test case to test_util_string_from_pipe() to
ensure that get_string_from_pipe() correctly returns multiple lines in a
single call.
See: https://bugs.torproject.org/21757
See: https://bugs.torproject.org/21654
We could use one of these for holding "junk" descriptors and
unparseable things -- but we'll _need_ it for having cached
consensuses and diffs between them.
Previously, we operated on smartlists of NUL-terminated strings,
which required us to copy both inputs to produce the NUL-terminated
strings. Then we copied parts of _those_ inputs to produce an
output smartlist of NUL-terminated strings. And finally, we
concatenated everything into a final resulting string.
This implementation, instead, uses a pointer-and-extent pattern to
represent each line as a pointer into the original inputs and a
length. These line objects are then added by reference into the
output. No actual bytes are copied from the original strings until
we finally concatenate the final result together.
Bookkeeping structures and newly allocated strings (like ed
commands) are allocated inside a memarea, to avoid needless mallocs
or complicated should-I-free-this-or-not bookkeeping.
In my measurements, this improves CPU performance by something like
18%. The memory savings should be much, much higher.
This takes two fuzzers: one which generates a diff and makes sure it
works, and one which applies a diff.
So far, they won't crash, but there's a bug in my
string-manipulation code someplace that I'm having to work around,
related to the case where you have a blank line at the end of a
file, or where you diff a file with itself.
(This commit was extracted by nickm based on the final outcome of
the project, taking only the changes in the files touched by this
commit from the consdiff_rebased branch. The directory-system
changes are going to get worked on separately.)
This patch removes the `tor_fgets()` wrapper around `fgets(3)` since it
is no longer needed. The function was created due to inconsistency
between the returned values of `fgets(3)` on different versions of Unix
when using `fgets(3)` on non-blocking file descriptors, but with the
recent changes in bug #21654 we switch from unbuffered to direct I/O on
non-blocking file descriptors in our utility module.
We continue to use `fgets(3)` directly in the geoip and dirserv module
since this usage is considered safe.
This patch also removes the test-case that was created to detect
differences in the implementation of `fgets(3)` as well as the changes
file since these changes was not included in any releases yet.
See: https://bugs.torproject.org/21654
This patch removes the buffered I/O stream usage in process_handle_t and
its related utility functions. This simplifies the code and avoids racy
code where we used buffered I/O on non-blocking file descriptors.
See: https://bugs.torproject.org/21654
This patch resets `buf` in test_util_fgets_eagain() after each succesful
ivocation to avoid stray artifacts left in the buffer by erroneous
tor_fgets() calls.
(But use bash if it's available.)
This is a workaround until we remove bash-specific code in 19699.
Fixes bug 21581; bugfix on 21562, not in any released version of tor.
This feature makes it possible to turn off memory sentinels (like
those used for safety in buffers.c and memarea.c) when fuzzing, so
that we can catch bugs that they would otherwise prevent.
When encoding a legacy ESTABLISH_INTRO cell, we were using the sizeof() on a
pointer instead of using the real size of the destination buffer leading to an
overflow passing an enormous value to the signing digest function.
Fortunately, that value was only used to make sure the destination buffer
length was big enough for the key size and in this case it always was because
of the overflow.
Fixes#21553
Signed-off-by: David Goulet <dgoulet@torproject.org>
Instead of returning 404 error code, this led to a NULL pointer being used and
thus a crash of tor.
Fixes#21471
Signed-off-by: David Goulet <dgoulet@torproject.org>
Fixes bug 20894; bugfix on 0.2.0.16-alpha.
We already applied a workaround for this as 20834, so no need to
freak out (unless you didn't apply 20384 yet).
I think this one probably can't underflow, since the input ranges
are small. But let's not tempt fate.
This patch also replaces the "cmp" functions here with just "eq"
functions, since nothing actually checked for anything besides 0 and
nonzero.
Related to 21278.
According to 21116, it seems to be needed for Wheezy Raspbian build. Also,
manpage of socket(2) does confirm that this errno value should be catched as
well in case of no support from the OS of IPv4 or/and IPv6.
Fixes#21116
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch adds checks for expected log messages for failure cases of
different ill-formed ESTABLISH_INTRO cell's.
See: https://bugs.torproject.org/21266
Determining if OpenSSL structures are opaque now uses an autoconf check
instead of comparing the version number. Some definitions have been
moved to their own check as assumptions which were true for OpenSSL
with opaque structures did not hold for LibreSSL. Closes ticket 21359.
This disregards anything smaller than an IPv6 /64, and rejects ports that
are rejected on an IPv6 /16 or larger.
Adjust existing unit tests, and add more to cover exceptional cases.
No IPv4 behaviour changes.
Fixes bug 21357
If there are no ephemeral or detached onion services, then
"GETINFO onions/current" or "GETINFO onions/detached" should
return an empty list instead of an error
The server-side clipping now clamps to one of two values, both
for what to report, and how long to cache.
Additionally, we move some defines to dns.h, and give them better
names.
It is no longer possible for the IPv6 preference options to differ from the
IPv6 usage: preferring IPv6 implies possibly using IPv6.
Also remove the corresponding unit test warning message checks.
(But keep the unit tests themselves - they now run without warnings.)
This commit adds 3 unit tests which validates a wrong signature length, a
wrong authentication key length and a wrong MAC in the cell.
Closes#20992
Signed-off-by: David Goulet <dgoulet@torproject.org>
Some DNS NXDOMAIN hijackers hijack truly ridiculous domains, like
"invalid-stuff!!" or "1.2.3.4.5". This would provoke unit test
failures where we used addresses like that to force
tor_addr_lookup() to fail. The fix, for testing, is to mock
tor_addr_lookup() with a variant that always fails when it gets
a name with a !.
Fixes bugs 20862 and 20863.
The abort handler masks the exit status of the backtrace generator by
capturing the abort signal from the backtrace handler and exiting with
zero. Because the output of the backtrace generator is meant to be piped
to `bt_test.py`, its exit status is unimportant and is currently
ignored.
The abort handler calls `exit(3)` which is not asynchronous-signal-safe
and calling it in this context is undefined behavior [0].
Closes ticket 21026.
[0] https://www.securecoding.cert.org/confluence/x/34At
Since both the client and service will use that data structure to store the
descriptor decoded data, only the public keys are common to both.
Fixes#20572.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is an important thing I hadn't considered when writing prop271:
sometimes you have to restrict what guard you use for a particular
circuit. Most frequently, that would be because you plan to use a
certain node as your exit, and so you can't choose that for your
guard.
This change means that the upgrade-waiting-circuits algorithm needs
a slight tweak too: circuit A cannot block circuit B from upgrading
if circuit B needs to follow a restriction that circuit A does not
follow.
George pointed out that (-1,0,1) for (never usable, maybe usable
later, usable right now) was a pretty rotten convention that made
the code harder to read.
This includes:
* making bridge_info_t exposed but opaque
* allowing guards where we don't know an identity
* making it possible to learn the identity of a guard
* creating a guard that lacks a node_t
* remembering a guard's address and port.
* Looking up a guard by address and port.
* Only enforcing the rule that we need a live consensus to update
the "listed" status for guards when we are not using bridges.
Currently, this code doesn't actually have the contexts behave
differently, (except for the legacy context), but it does switch
back and forth between them nicely.
Some of these will get torrc options to override them too; this
is just the mechanical conversion.
Also, add documentation for a couple of undocumented (but now used)
parameters.
Act I.
" But that I am forbid
To tell the secrets of my prison-house,
I could a tale unfold..."
Here's the bug: sometimes, rend_cache/store_v2_desc_as_client would
say:
"Dec 15 08:31:26.147 [warn] rend_cache_store_v2_desc_as_client():
Bug: Couldn't decode base32 [scrubbed] for descriptor id. (on Tor
0.3.0.0-alpha-dev 4098bfa260)"
When we merged ade5005853 back in 0.2.8.1-alpha, we added that
test: it mangles the hidden service ID for a hidden service, and
ensures that when the descriptor ID doesn't match the descriptor's
key, we don't store the descriptor.
How did it mangle the descriptor ID? By doing
desc_id_base32[0]++;
So, if the hidden service ID started with z or 7, we'd wind up with an
invalid base32 string, and get the warning. And if it started with
any other character, we wouldn't.
That there is part 1 of the bug: in 2/32 cases, we'd get a BUG
warning. But we wouldn't display it, since warnings weren't shown
from the unit tests.
Act II.
"Our indiscretion sometime serves us well,
When our deep plots do pall"
Part two: in 0.2.9.3-alpha, for part of #19999, we turned on BUG
warnings in the unit tests, so that we'd actually start seeing them.
At this point we also began to consider each BUG warning that made
it through the unit tests to be an actual bug. So before this
point, we wouldn't actually notice anything happening in those 2/32
cases.
So, at this point it was a nice random _visible_ bug.
Act III.
"Our thoughts are ours, their ends none of our own"
In acbb60cd63, which was part of my prop220 work, I
changed how RSA key generation worked in the unit tests. While
previously we'd use pre-made RSA keys in some cases, this change
made us use a set of pregenerated RSA keys for _all_ 1024 or 2048
keys, and to return them in a rotation when Tor tried to generate a
key.
And now we had the heisenbug: anything that affected the number of
pregenerated keys that we had yielded before reaching
rend_cache/store_v2_desc_as_client would make us return a different
key, which would give us a different base32 ID, which would make the
bug occur, or not. So as we added or removed test cases, the bug
might or might not happen.
So yeah. Don't mangle a base32 ID like that. Do it this way instead.
The new HS circuitmap API replaces old public functions as follows:
circuit_clear_rend_token -> hs_circuitmap_remove_circuit
circuit_get_rendezvous -> hs_circuitmap_get_rend_circ
circuit_get_intro_point -> hs_circuitmap_get_intro_circ_v2
circuit_set_rendezvous_cookie -> hs_circuitmap_register_rend_circ
circuit_set_intro_point_digest -> hs_circuitmap_register_intro_circ_v2
This commit also removes the old rendinfo code that is now unused.
It also fixes the broken rendinfo unittests.
Circuit object wasn't freed correctly. Also, the cpath build state object
needed to be zeroed else we were freeing garbage pointers.
Closes#20936
Signed-off-by: David Goulet <dgoulet@torproject.org>
This was breaking the build on debian precise, since it thought that
using a 'const int' to dimension an array made that array
variable-size, and made us not get protection.
Bug not in any released version of Tor.
I will insist that this one wasn't my fault.
"Variables won't. Constants aren't." -- Osborn's Law
If a node can prove its Ed25519 identity, don't consider connections
to it canonical unless they match both identities.
Includes link handshake changes needed to avoid crashing with bug
warnings, since the tests now reach more parts of the code.
Closes ticket 20355
Coverity doesn't like it when there are paths to the end of the
function where something doesn't get freed, even when those paths
are only reachable on unit test failure.
Fixes CID 1372899 and CID 1372900. Bug not in any released Tor.
Instead, refuse to start tor if any hidden service key has been used in
a different hidden service anonymity mode.
Fixes bug 20638; bugfix on 17178 in 0.2.9.3-alpha; reported by ahf.
The original single onion service poisoning code checked poisoning state
in options_validate, and poisoned in options_act. This was problematic,
because the global array of hidden services had not been populated in
options_validate (and there were ordrering issues with hidden service
directory creation).
This patch fixes this issue in rend_service_check_dir_and_add, which:
* creates the directory, or checks permissions on an existing directory, then
* checks the poisoning state of the directory, then
* poisons the directory.
When validating, only the permissions checks and the poisoning state checks
are perfomed (the directory is not modified).
To do this, it makes sense to treat legacy guards as a separate
guard_selection_t *, and handle them separately. This also means we
add support here for having multiple guard selections.
Note that we don't persist pathbias information yet; that will take
some refactoring.
This patch doesn't cover every case; omitted cases are marked with
"XXXX prop271", as usual. It leaves both the old interface and the
new interface for guard status notification, since they don't
actually work in the same way: the new API wants to be told when a
circuit has failed or succeeded, whereas the old API wants to know
when a channel has failed or succeeded.
I ran into some trouble with directory guard stuff, since when we
pick the directory guard, we don't actually have a circuit to
associate it with. I solved that by allowing guard states to be
associated with directory connections, not just circuits.
I expect we'll be ripping this out somewhere in 0.3.0, but let's
keep it around for a little while in case it turns out to be the
only way to avert disaster?
This code handles:
* Maintaining the sampled set, the filtered set, and the
usable_filtered set.
* Maintaining the confirmed and primary guard lists.
* Picking guards for circuits, and updating guard state when
circuit state changes.
Additionally, I've done code structure movement: even more constants
and structures from entrynodes.c have become ENTRYNODES_PRIVATE
fields of entrynodes.h.
I've also included a bunch of documentation and a bunch of unit
tests. Coverage on the new code is pretty high.
I've noted important things to resolve before this branch is done
with the /XXXX.*prop271/ regex.
This patch is just:
* Code movement
* Adding headers here and there as needed
* Adding a bridges_free_all() with a call to it.
It breaks compilation, since the bridge code needed to make exactly
2 calls into entrynodes.c internals. I'll fix those in the next
commit.
The encoding code is very straightforward. The decoding code is a
bit tricky, but clean-ish. The sampling code is untested and
probably needs more work.
This field indicates if the service is a Single Onion Service if present in
the descriptor.
Closes#19642
Signed-off-by: David Goulet <dgoulet@torproject.org>
The test was broken and skipped because the hardcoded cross certificate didn't
include the dynamically generated signing key generated by the test. The only
way we could have fixed that is extracting the signing key from the hardcoded
string and put it in the descriptor object or dynamically generate the cross
certificate.
In the end, all this was kind of pointless as we already test the decoding of
multiple introduction points elsewhere and we don't gain anything with that
specific test thus the removal.
Fixes#20570
Signed-off-by: David Goulet <dgoulet@torproject.org>
This implements the proposal 224 directory descriptor cache store and lookup
functionalities. Furthermore, it merges the OOM call for the HSDir cache with
current protocol v2 and the new upcoming v3.
Add hs_cache.{c|h} with store/lookup API.
Closes#18572
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
In order to implement proposal 224, we need the data structure rend_data_t to
be able to accomodate versionning that is the current version of hidden
service (2) and the new version (3) and future version.
For that, we implement a series of accessors and a downcast function to get
the v2 data structure. rend_data_t becomes a top level generic place holder.
The entire rend_data_t API has been moved to hs_common.{c|h} in order to
seperate code that is shared from between HS versions and unshared code (in
rendcommon.c).
Closes#19024
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
One is fixed by disabling the -Wredundant-decls warnings around
openssl headers here, because of the old double-declaration of
SSL_get_selected_srtp_profile().
One is fixed by including compat.h before or.h so that we get the
winsock2.h include before the windows.h include.
This was a stopgap method, designed on the theory that some routers
might support it before they could support Ed25519. But it looks
like everybody who supports RFC5705 will also have an Ed25519 key,
so there's not a lot of reason to have this even supported.
This patch moves the pregenerated RSA key logic into a new
testing_rsakeys.c.
Also, it adds support for RSA2048, since the link handshake tests
want that.
Also, it includes pregenerated keys, rather than trying to actually
generate the keys at startup, since generating even a small handful
of RSA2048 keys makes for an annoying delay.
In particular, these functions are the ones that set the identity of
a given connection or channel, and/or confirm that we have learned
said IDs.
There's a lot of stub code here: we don't actually need to use the
new keys till we start looking up connections/channels by Ed25519
IDs. Still, we want to start passing the Ed25519 IDs in now, so it
makes sense to add these stubs as part of 15055.
Also, adjust signing approach to more closely match the signing
scheme in the proposal.
(The format doesn't quite match the format in the proposal, since
RSA signatures aren't fixed-length.)
Closes 19020.
passthrough_test_setup doesn't pass through arguments if the argument
is equal to 0 or TT_SKIP. Instead, it fails or skips the test.
Assert on this, so we don't accidentally fail or skip tests.
ome policies are default-reject, some default-accept. But
policy_is_reject_star() assumed they were all default_reject. Fix
that!
Also, document that policy_is_reject_star() treats a NULL policy as
empty. This allows us to simplify the checks in
parse_reachable_addresses() by quite a bit.
Fxes bug 20306; bugfix on 0.2.8.2-alpha.
Use the following coccinelle script to change uses of
smartlist_add(sl, tor_strdup(str)) to
smartlist_add_strdup(sl, string) (coccinelle script from nickm
via bug 20048):
@@
expression a;
expression b;
@@
- smartlist_add
+ smartlist_add_strdup
(a,
- tor_strdup(
b
- )
)
When we refactored purpose_needs_anonymity(), we made it so _all_
bridge requests required anonymity. But that missed the case
that we are allowed to ask a bridge for its own descriptor.
With this patch, we consider the resource, and allow "authority.z"
("your own descriptor, compressed") for a bridge's server descriptor
to be non-anonymous.
Fix for bug 20410; bug not in any released Tor.
I believe that this should never trigger, but if it does, it
suggests that there was a gap between is_sensitive_dir_purpose and
purpose_needs_anonymity that we need to fill. Related to 20077.
(Specifically, carriage return after a quoted value in a config
line. Fixes bug 19167; bugfix on 0.2.0.16-alpha when we introduced
support for quoted values. Unit tests, changes file, and this
parenthetical by nickm.)
[This is a brute-force method that potentially uses way too much
RAM. Need to rethink this a little. Right now you can DOS an
authority by saying "Foo=1-4294967295".]
Our use of the (mockable) tor_close_socket() in the util/socket_..
tests confused coverity, which could no longer tell that we were
actually closing the sockets.
Previously, the IV and key were stored in the structure, even though
they mostly weren't needed. The only purpose they had was to
support a seldom-used API where you could pass NULL when creating
a cipher in order to get a random key/IV, and then pull that key/IV
back out.
This saves 32 bytes per AES instance, and makes it easier to support
different key lengths.
* Check consistency between the two single onion torrc options
* Use the more relevant option each time we check for single onion mode
* Clarify log messages
* Clarify comments
* Otherwise, no behaviour change
Add experimental OnionServiceSingleHopMode and
OnionServiceNonAnonymousMode options. When both are set to 1, every
hidden service on a tor instance becomes a non-anonymous Single Onion
Service. Single Onions make one-hop (direct) connections to their
introduction and renzedvous points. One-hop circuits make Single Onion
servers easily locatable, but clients remain location-anonymous.
This is compatible with the existing hidden service implementation, and
works on the current tor network without any changes to older relays or
clients.
Implements proposal #260, completes ticket #17178. Patch by teor & asn.
squash! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! Implement Prop #260: Single Onion Services
Redesign single onion service poisoning.
When in OnionServiceSingleHopMode, each hidden service key is poisoned
(marked as non-anonymous) on creation by creating a poison file in the
hidden service directory.
Existing keys are considered non-anonymous if this file exists, and
anonymous if it does not.
Tor refuses to launch in OnionServiceSingleHopMode if any existing keys
are anonymous. Similarly, it refuses to launch in anonymous client mode
if any existing keys are non-anonymous.
Rewrite the unit tests to match and be more comprehensive.
Adds a bonus unit test for rend_service_load_all_keys().
The other test vectors are pretty complete, and get full coverage, I
believe.
This one test vector accounted for half the time spent in
test-slow. "Now that's slow!"
We have a mock for our RSA key generation function, so we now wire
it to pk_generate(). This covers all the cases that were not using
pk_generate() before -- all ~93 of them.
Previously, you needed to store the previous log severity in a local
variable, and it wasn't clear if you were allowed to call these
functions more than once.
The functions it warns about are:
assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
strdup, strndup, calloc.
Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
We should consider them bugs. If they are happening intentionally,
we should use the log_test_helpers code to capture and suppress
them. But having them off-by-default has potential to cause
programming errors.
Previously setup_capture_of_logs would prevent log messages from
going to the console entirely. That's a problem, since sometimes
log messages are bugs! Now setup_capture_of_logs() acts sensibly.
If you really do need to keep a message from going to the console
entirely, there is setup_full_capture_of_logs(). But only use that
if you're prepared to make sure that there are no extraneous
messages generated at all.
Copying the integer 42 in a char buffer has a different representation
depending on the endianess of the system thus that unit test was failing on
big endian system.
This commit introduces a python script, like the one we have for SRV, that
computes a COMMIT/REVEAL from scratch so we can use it as a test vector for
our encoding unit tests.
With this, we use a random value of bytes instead of a number fixing the
endianess issue and making the whole test case more solid with an external
tool that builds the COMMIT and REVEAL according to the spec.
Fixes#19977
Signed-off-by: David Goulet <dgoulet@torproject.org>
OnionTrafficOnly is equivalent to NoDNSRequest, NoIPv4Traffic,
and NoIPv6Traffic.
Add unit tests for parsing and checking option validity.
Add documentation for each flag to the man page.
Add changes file for all of #18693.
Parsing only: the flags do not change client behaviour (yet!)
These functions were there so that we could abstract the differences
between evbuffer and buf_t. But with the bufferevent removal, this
no longer serves a purpose.
The test was checking for EISDIR which is a Linux-ism making other OSes
unhappy. Instead of checking for a negative specific errno value, just make
sure it's negative indicating an error. We don't need more for this test.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Keep the base16 representation of the RSA identity digest in the commit object
so we can use it without using hex_str() or dynamically encoding it everytime
we need it. It's used extensively in the logs for instance.
Fixes#19561
Signed-off-by: David Goulet <dgoulet@torproject.org>
Encoded commit has an extra byte at the end for the NUL terminated byte and
the test was overrunning the payload buffer by one byte.
Found by Coverity issue 1362984.
Fixes#19567
Signed-off-by: David Goulet <dgoulet@torproject.org>
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
The test_state_update() test would fail if you run it between 23:30 and
00:00UTC in the following line because n_protocol_runs was 2:
tt_u64_op(state->n_protocol_runs, ==, 1);
The problem is that when you launch the test at 23:30UTC (reveal phase),
sr_state_update() gets called from sr_state_init() and it will prepare
the state for the voting round at 00:00UTC (commit phase). Since we
transition from reveal to commit phase, this would trigger a phase
transition and increment the n_protocol_runs counter.
The solution is to initialize the n_protocol_runs to 0 explicitly in the
beginning of the test, as we do for n_reveal_rounds, n_commit_rounds etc.
The *get* state query functions for the SRVs now only return const pointers
and the DEL action needs to be used to delete the SRVs from the state.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch makes us retain the intermediate list of K=V entries for
the duration of computing our vote, and lets us use that list with
a new function in order to look up parameters before the consensus
is published.
We can't actually use this function yet because of #19011: our
existing code to do this doesn't actually work, and we'll need a new
consensus method to start using it.
Closes ticket #19012.
Code has been changed so every RSA fingerprint for a commit in our state is
validated before being used. This fixes the unit tests by mocking one of the
key function and updating the hardcoded state string.
Also, fix a time parsing overflow on platforms with 32bit time_t
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
The prop250 code used the RSA identity key fingerprint to index commit in a
digestmap instead of using the digest.
To behavior change except the fact that we are actually using digestmap
correctly.
Signed-off-by: David Goulet <dgoulet@torproject.org>
From 0.2.7.2-alpha onwards, Exits would reject all the IP addresses
they knew about in their exit policy. But this may have disclosed
addresses that were otherwise unlisted.
Now, only advertised addresses are rejected by default by
ExitPolicyRejectPrivate. All known addresses are only rejected when
ExitPolicyRejectLocalInterfaces is explicitly set to 1.
This hack provides a way to make sure we can see coverage from
test-switch-id. If you set OVERRIDE_GCDA_PERMISSIONS_HACK, we
temporarily make the .gcda files mode 0666 before we run the
test scripts, and then we set them to 0644 again afterwards.
That's necessary because the test_switch_id.sh script does a
setuid() to 'nobody' part way through, and drops the ability to
change its mind back.
Slow system can sometime take more than 10 seconds to reach the test
callsite resulting in the unit test failing when using time in the future or
in the past.
Fixes#19465
Signed-off-by: David Goulet <dgoulet@torproject.org>
base16_decodes() now returns the number of decoded bytes. It's interface
changes from returning a "int" to a "ssize_t". Every callsite now checks the
returned value.
Fixes#14013
Signed-off-by: David Goulet <dgoulet@torproject.org>
zlib 1.2 came out in 2003; earlier versions should be dead by now.
Our workaround code was only preventing us from using the gzip
encoding (if we decide to do so), and having some dead code linger
around in torgzip.c
This is a big-ish patch, but it's very straightforward. Under this
clang warning, we're not actually allowed to have a global variable
without a previous extern declaration for it. The cases where we
violated this rule fall into three roughly equal groups:
* Stuff that should have been static.
* Stuff that was global but where the extern was local to some
other C file.
* Stuff that was only global when built for the unit tests, that
needed a conditional extern in the headers.
The first two were IMO genuine problems; the last is a wart of how
we build tests.
This warning triggers on silently promoting a float to a double. In
our code, it's just a sign that somebody used a float by mistake,
since we always prefer double.
This warning, IIUC, means that the compiler doesn't like it when it
sees a NULL check _after_ we've already dereferenced the
variable. In such cases, it considers itself free to eliminate the
NULL check.
There are a couple of tricky cases:
One was the case related to the fact that tor_addr_to_in6() can
return NULL if it gets a non-AF_INET6 address. The fix was to
create a variant which asserts on the address type, and never
returns NULL.
This is a fairly easy way for us to get our test coverage up on
compat_threads.c and workqueue.c -- I already implemented these
tests, so we might as well enable them.
So, back long ago, XXX012 meant, "before Tor 0.1.2 is released, we
had better revisit this comment and fix it!"
But we have a huge pile of such comments accumulated for a large
number of released versions! Not cool.
So, here's what I tried to do:
* 0.2.9 and 0.2.8 are retained, since those are not yet released.
* XXX+ or XXX++ or XXX++++ or whatever means, "This one looks
quite important!"
* The others, after one-by-one examination, are downgraded to
plain old XXX. Which doesn't mean they aren't a problem -- just
that they cannot possibly be a release-blocking problem.
Remove support for "GET /tor/bytes.txt" DirPort request, and
"GETINFO dir-usage" controller request, which were only available
via a compile-time option in Tor anyway.
Feature was added in 0.2.2.1-alpha. Resolves ticket 19035.
Previously, if the header was present, we'd proceed even if the
function wasn't there.
Easy fix for bug 19161. A better fix would involve trying harder to
find libscrypt_scrypt.
AddressSanitizer's (ASAN) SIGSEGV handler overrides the backtrace
handler and prevents it from printing its backtrace. The output of ASAN
is different from what 'bt_test.py' expects and causes backtrace test
failures.
The 'allow_user_segv_handler' option allows applications to set their
own SIGSEGV handler but is not supported by older GCC versions. These
older GCC versions do support the 'handle_segv' which prevents ASAN from
setting its SIGSEGV handler.
With the fix for #17150, I added a duplicate certificate here. Here
I remove the original location in 0.2.8. (I wouldn't want to do
that in 027, due to the amount of authority-voting-related code
drift.)
Closes 19073.
We know there are overflows in curve25519-donna-c32, so we'll have
to have that one be fwrapv.
Only apply the asan, ubsan, and trapv options to the code that does
not need to run in constant time. Those options introduce branches
to the code they instrument.
(These introduced branches should never actually be taken, so it
might _still_ be constant time after all, but branch predictors are
complicated enough that I'm not really confident here. Let's aim for
safety.)
Closes 17983.
The goal here is to provide a way to decouple pieces of the code
that want to learn "when something happens" from those that realize
that it has happened.
The implementation here consists of a generic backend, plus a set of
macros to define and implement a set of type-safe frontends.
Tor stores client authorization cookies in two slightly different forms.
The service's client_keys file has the standard base64-encoded cookie,
including two chars of padding. The hostname file and the client remove
the two padding chars, and store an auth type flag in the unused bits.
The distinction makes no sense. Refactor all decoding to use the same
function, which will accept either form, and use a helper function for
encoding the truncated format.
