With the latest change on how we use the HSDir index, the client and service
need to pick their responsible HSDir differently that is depending on if they
are before or after a new time period.
The overlap mode is active function has been renamed for this and test added.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Because of #23387, we've realized that there is one scenario that makes
the client unable to reach the service because of a desynch in the time
period used. The scenario is as follows:
+------------------------------------------------------------------+
| |
| 00:00 12:00 00:00 12:00 00:00 12:00 |
| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 |
| |
| $==========|-----------$===========|-----------$===========| |
| ^ ^ |
| C S |
+------------------------------------------------------------------+
In this scenario the HS has a newer consensus than the client, and the
HS just moved to the next TP but the client is still stuck on the old
one. However, the service is not in any sort of overlap mode so it
doesn't cover the old TP anymore, so the client is unable to fetch a
descriptor.
We've decided to solve this by extending the concept of overlap period
to be permanent so that the service always publishes two descriptors and
aims to cover clients with both older and newer consensuses. See the
spec patch in #23387 for more details.
Based on our #23387 findings, it seems like to maintain 24/7
reachability we need to employ different logic when computing hsdir
indices for fetching vs storing. That's to guarantee that the client
will always fetch the current descriptor, while the service will always
publish two descriptors aiming to cover all possible edge cases.
For more details see the next commit and the spec branch.
Signed-off-by: David Goulet <dgoulet@torproject.org>
The pruning process and the deleting ephemeral service function iterates over
all circuits and were asserting on rend_data for a matching circuit. This is
not good because now we have v3 circuits without a rend_data.
Fixes#23429
Signed-off-by: David Goulet <dgoulet@torproject.org>
Use the valid_after time from the consensus to get the time period number else
we might get out of sync with the overlap period that uses valid_after.
Make it an optional feature since some functions require passing a
specific time (like hs_get_start_time_of_next_time_period()).
Signed-off-by: David Goulet <dgoulet@torproject.org>
Undeprecate it;
rename it to TestingClientDNSRejectInternalAddresses;
add the old name as an alias;
reject configurations where it is set but TestingTorNetwork is not;
change the documentation accordingly.
Closes tickets 21031 and 21522.
Version 3 hidden service needs rendezvous point that have the protocol version
HSRend >= 2 else the rendezvous cells are rejected.
Fixes#23361
Signed-off-by: David Goulet <dgoulet@torproject.org>
There are two reasons this is likeliest to happen -- no kernel
support, and some bug in Tor. We'll ask people to check the former
before they report. Closes 23090.
The chdir() call in RunAsDaemon makes the behavior here surprising,
and either way of trying to resolve the surprise seems sure to
startle a significant fraction of users. Instead, let's refuse to
guess, and refuse these configurations.
Closes ticket 22731.
I'm doing this using the Proxy-Authorization: header to support
clients that understand it, and with a new tor-specific header that
makes more sense for our use.
By convention, a function that frobs a foo_t should be called
foo_frob, and it should have a foo_t * as its first argument. But
for many of the buf_t functions, the buf_t was the final argument,
which is silly.
Our convention is that functions which manipulate a type T should be
named T_foo. But the buffer functions were super old, and followed
all kinds of conventions. Now they're uniform.
Here's the perl I used to do this:
\#!/usr/bin/perl -w -i -p
s/read_to_buf\(/buf_read_from_socket\(/;
s/flush_buf\(/buf_flush_to_socket\(/;
s/read_to_buf_tls\(/buf_read_from_tls\(/;
s/flush_buf_tls\(/buf_flush_to_tls\(/;
s/write_to_buf\(/buf_add\(/;
s/write_to_buf_compress\(/buf_add_compress\(/;
s/move_buf_to_buf\(/buf_move_to_buf\(/;
s/peek_from_buf\(/buf_peek\(/;
s/fetch_from_buf\(/buf_get_bytes\(/;
s/fetch_from_buf_line\(/buf_get_line\(/;
s/fetch_from_buf_line\(/buf_get_line\(/;
s/buf_remove_from_front\(/buf_drain\(/;
s/peek_buf_startswith\(/buf_peek_startswith\(/;
s/assert_buf_ok\(/buf_assert_ok\(/;
This lets us drop the testing-only function buf_get_first_chunk_data(),
and lets us implement proto_http and proto_socks without looking at
buf_t internals.
The service needs the latest SRV and set of relays for the best accurate
hashring to upload its descriptor to so it needs a live consensus thus don't
do anything until we have it.
Fixes#23331
Signed-off-by: David Goulet <dgoulet@torproject.org>
When merging #20657, somehow hs_service_dir_info_changed() became unused
leading to not use the re-upload to HSDir when we were missing information
feature.
Turns out that it is not possible to pick an HSDir with a missing descriptor
because in order to compute the HSDir index, the descriptor is mandatory to
have so we can know its position on the hashring.
This commit removes that dead feature and fix the
hs_service_dir_info_changed() not being used.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Based on questions and comments from dgoulet, I've tried to fill
in the reasoning about why these functions work in the way that they
do, so that it will be easier for future programmers to understand
why this code exists and works the way it does.
We used to check if it was set to 0 which is what unused circuit have but when
the rendezvous circuit was cannibalized, the timestamp_dirty is not 0 but we
still need to reset it so we can actually use it without having the chance of
expiring the next second (or very soon).
Fixes#23123
Signed-off-by: David Goulet <dgoulet@torproject.org>
The function was never returning an error code on failure to parse the
OutboundAddress* options.
In the process, it was making our test_options_validate__outbound_addresses()
not test the right thing.
Fixes#23366
Signed-off-by: David Goulet <dgoulet@torproject.org>
This fixes a serious bug in our hsdir set change logic:
We used to add nodes in the list of previous hsdirs everytime we
uploaded to a new hsdir and we only cleared the list when we built a new
descriptor. This means that our prev_hsdirs list could end up with 7
hsdirs, if for some reason we ended up uploading our desc to 7 hsdirs
before rebuilding our descriptor (e.g. this can happen if the set of
hsdirs changed).
After our previous hdsir set had 7 nodes, then our old algorithm would
always think that the set has changed since it was comparing a smartlist
with 7 elements against a smartlist with 6 elements.
This commit fixes this bug, by clearning the prev_hsdirs list before we
upload to all hsdirs. This makes sure that our prev_hsdirs list always
contains the latest hsdirs!
Our logic for detecting hsdir set changes was needlessly compicated: we
had to sort smartlists and compare them.
Instead, we can simplify things by employing the following logic:
"We should reupload our descriptor if the latest HSDir set contains
nodes that were not previously there"
Since we can't be sure that we can unlink enough files on windows
here, let's let the number of permitted entries grow huge if it
really must.
We do this by letting the storagedir hold lots of entries, but still
trying to keep the number of entries under the configured limit. We
also have to tell consdiffmgr not to freak out if it can't actually
remove enough entries.
Part of a fix for bug 22752
Some parentheses were missing making the rend_max_intro_circs_per_period()
return a lower value than it was suppose to.
The calculation is that a service at most will open a number of intro points
that it wants which is 3 by default or HiddenServiceNumIntroductionPoints. Two
extra are launched for performance reason. Finally, this can happen twice for
two descriptors for the current and next time period.
From:
2 * n_intro_wanted + 2
...which resulted in 8 for 3 intro points, this commit fixes it to:
(n_intro_wanted + 2) * 2
... resulting in 12 possible intro point circuit which is the correct maximum
intro circuit allowed per period.
Last, this commit rate limits the the log message if we ever go above that
limit else over a INTRO_CIRC_RETRY_PERIOD, we can print it often!
Fixes#22159
Signed-off-by: David Goulet <dgoulet@torproject.org>
Since ssize_t is signed and might be 64 bits, we should use
tt_i64_op to make sure it's positive. Otherwise, if it is negative,
and we use tt_u64_op, we'll be treating it as a uint64_t, and we
won't detect negative values.
This fixes CID 1416338 and 1416339. Bug not in any released Tor.
That check was wrong:
a) We should be making sure that the size of `key` is big enough before
proceeding, since that's the buffer that we would overread with the
tor_memeq() below.
The old check used to check that `req_key_str` is big enough which is
not right, since we won't read deep into that buffer.
The new check makes sure that `key` has enough size to survive the
tor_memeq(), and if not it moves to the next element of the strmap.
b) That check shouldn't be a BUG since that strmap contains
variable-sized elements and we should not be bugging out if we happen
to compare a small sized element (v2) to a bigger one (v3).
This way, we can clear off the directory requests from our cache and thus
allow the next client to query those HSDir again at the next SOCKS connection.
Signed-off-by: David Goulet <dgoulet@torproject.org>
v3 client now cleans up the HSDir request cache when a connection to a service
was successful.
Closes#23308
Signed-off-by: David Goulet <dgoulet@torproject.org>
We used to not copy the state which means that after HUP we would forget
if we are in overlap mode or not. That caused bugs where the service
would enter overlap mode twice, and rotate its descs twice, causing all
sorts of bugs.
Apart from the fact that a newly allocated service doesn't have descriptors
thus the move condition can never be true, the service needs the descriptor
signing key to cross-certify the authentication key of each intro point so we
need to move the descriptors between services and not only the intro points.
Fixes#23056
Signed-off-by: David Goulet <dgoulet@torproject.org>
We refactor the descriptor reupload logic to be similar to the v2 logic
where we update a global 'consider_republishing_rend_descriptors' flag
and then we use that to check for hash ring changes during the global
hidden service callbacks.
This fixes bugs where we would inspect the hash ring immediately as we
receive new dirinfo (e.g. consensus) but before running the hidden
service housekeeping events. That was leaving us in an inconsistent
state wrt hsdir indices and causing bugs all around.
The problem was that when we went from overlap mode to non-overlap mode,
we were not wiping the 'desc_next' descriptor and instead we left it on
the service. This meant that all functions that iterated service
descriptors were also inspecting the useless 'desc_next' descriptor that
should have been deleted.
This commit refactors rotate_all_descriptors() so that it rotates
descriptor both when entering overlap mode and also when leaving it.
The `test-operator-cleanup` patch, and related coccinelle patches,
don't do any checks for line length. This patch fixes the line
length issues caused by the previous commits.
This patch fixes the operator usage in src/test/*.c to use the symbolic
operators instead of the normal C comparison operators.
This patch was generated using:
./scripts/coccinelle/test-operator-cleanup src/test/*.[ch]
A client can re-extend up to 3 intro points on the same circuit. This happens
when we get NACKed by the intro point for which we choose a new intro and
re-extend the circuit to it.
That process can be arbitrarly long so reset the dirty timestamp of the
circuit everytime we choose to re-extend so we get a bit more time to actually
do our introduction.
This is a client circuit so it is short live once opened thus giving us a bit
more time to complete the introduction is ok.
Signed-off-by: David Goulet <dgoulet@torproject.org>
When looking for an introduction circuit in circuit_get_best(), we log an info
message if we are about to launch a new intro circuit in parallel. However,
the condition was considering marked for close circuit leading to the function
triggering the log info even though there is actually no valid intro circuit.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Only register the RP circuit when it opens and not when we send the INTRODUCE1
cell else, when re-extending to a new IP, we would register the same RP
circuit with the same cookie twice leading to the circuit being closed.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Changed the assert_intro_circ_ok() to an almost non fatal function so tor can
recover properly. We keep the anonymity assert because if that is not right,
we have much deeper problems and client should stop sending bytes to the
network immediately.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This function has been replaced by hs_client_receive_rendezvous_acked(() doing
the same exact thing for both v2 and v3 service.
Signed-off-by: David Goulet <dgoulet@torproject.org>
The client needs to find the right intro point object from the circuit
identity digest it is opened to. This new function does that.
Signed-off-by: David Goulet <dgoulet@torproject.org>
New function named hs_cell_introduce1_data_clear() is introduced to clear off
an hs_cell_introduce1_data_t object.
Signed-off-by: David Goulet <dgoulet@torproject.org>
When a client decodes a descriptor, make sure it matches the expected blinded
key which is derived from the hidden service identity key.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We can't trigger a valid upload because it would require us to MOCK a long
list of functions ultimately not really testing the upload because we aren't
on a running network.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Conflicts:
src/test/test_hs_service.c
Needed by the client when fetching a descriptor. This function checks the
directory purpose and hard assert if it is not for fetching.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit makes the client use the intro point state cache. It notes down
when we get a NACK from the intro point and then uses that cache to decide if
it should either close the circuits or re-extend to a new intro point.
This also introduces a very useful function that checks if an intro point is
usable that is query the state cache and checks a series of requirement.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This cache keeps track of the state of intro points which is needed when we
have failures when using them. It is similar to the failure cache of the
legacy system.
At this commit, it is unused but initialized, cleanup and freed.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This moves it to hs_client.c so it can be used by both system (legacy and
prop224). For now, only the legacy system uses it.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Don't assert() on rend_data when closing circuits to report an IP failure. The
prop224 code doesn't have yet the support for this.
Signed-off-by: David Goulet <dgoulet@torproject.org>
For now, prop224 doesn't have a mechanism to note down connection attempts so
we only do it for legacy system using rend_data.
Signed-off-by: David Goulet <dgoulet@torproject.org>
