We will need to edit this function, and it's already pretty huge. Let's make
it a bit smaller.
This commit moves code, fixes a 80 char line and add two lines at the start to
make it compile. Trivial change.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We need this func so that we recognize SOCKS conns to v3 addresses.
- Also rename rend_valid_service_id() to rend_valid_v2_service_id()
- Also move parse_extended_hostname() tests to their own unittest, and
add a v3 address to the test as well.
Signed-off-by: David Goulet <dgoulet@torproject.org>
When we enter overlap mode we start using the next hsdir index of
relays. However, we only compute the next hsdir index of relays when we
receive a consensus or their descriptor. This means that there is a
window of time between entering the overlap period and fetching the
consensus where relays have their next hsdir index uninitialized. This
patch fixes this by recomputing all hsdir indices when we first enter
the overlap period.
We want to reupload our descriptor if its set of responsible HSDirs
changed to minimize reachability issues.
This patch adds a callback everytime we get new dirinfo which checks if
the hash ring changed and reuploads descriptor if needed.
Because the HS subsystem calls it every second, change the log level to debug
so it doesn't spam the info log.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Our Windows compiler treats "time_t" as long long int but Linux likes it
long int so cast those to make Windows happy.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This change should improve overhead for downloading small numbers of
descriptors and microdescriptors by improving compression
performance and lowering directory request overhead.
Closes ticket 23220.
The old implementation would fail with super-long inputs. We never
gave it any, but still, it's nicer to dtrt here.
Reported by Guido Vranken. Fixes bug 19281.
- Fix various ssize_t/size_t confusions in the tests.
- Fix a weird memset argument:
"bad_memset: Argument -16 in memset loses precision in
memset(&desc_two->blinded_kp.pubkey.pubkey, -16, 32UL)."
- Fix check_after_deref instance in check_state_line_for_service_rev_counter():
"check_after_deref: Null-checking items suggests that it may be null,
but it has already been dereferenced on all paths leading to the
check."
Add a common function for both legacy and prop224 hidden service to increment
and decrement the rendezvous stream counter on an origin circuit.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We used to have a small HS desc cert lifetime but those certs can stick
around for 36 hours if they get initialized in the beginning of overlap
period.
[warn] Bug: Non-fatal assertion !(hs_desc_encode_descriptor(desc->desc, &desc->signing_kp, &encoded_desc) < 0) failed in
upload_descriptor_to_hsdir at src/or/hs_service.c:1886. Stack trace: (on Tor 0.3.2.0-alpha-dev b4a14555597fb9b3)
I repurposed the old directory_request_set_hs_ident() into a new
directory_request_upload_set_hs_ident() which is only used for the
upload purpose and so it can assert on the dir_purpose.
When coding the client-side we can make a second function for fetch.
We used to do:
h = H(BLIND_STRING | H(A | s | B | N )
when we should be doing:
h = H(BLIND_STRING | A | s | B | N)
Change the logic so that hs_common.c does the hashing, and our ed25519
libraries just receive the hashed parameter ready-made. That's easier
than doing the hashing on the ed25519 libraries, since that means we
would have to pass them a variable-length param (depending on whether
's' is set or not).
Also fix the ed25519 test vectors since they were also double hashing.
We also had to alter the SRV functions to take a consensus as optional
input, since we might be setting our HSDir index using a consensus that
is currently being processed and won't be returned by the
networkstatus_get_live_consensus() function.
This change has two results:
a) It makes sure we are using a fresh consensus with the right SRV value
when we are calculating the HSDir hash ring.
b) It ensures that we will not use the sr_get_current/previous()
functions when we don't have a consensus which would have falsely
triggered the disaster SRV logic.
In Nick's words:
"We want to always return false if the platform is a Tor version, and it
is not as new as 0.3.0.8 -- but if the platform is not a Tor version, or
if the version is as new as 0.3.0.8, then we want to obey the protocol
list.
That way, other implementations of our protocol won't have to claim any
particular Tor version, and future versions of Tor will have the freedom
to drop this protocol in the distant future."
- Fix log message format string.
- Do extra circuit purpose check.
- wipe memory in a clear function
- Make sure we don't double add intro points in our list
- Make sure we don't double close intro circuits.
- s/tt_u64_op/tt_i64_op/
Turns out that introduction points don't care about the INTRODUCE2 cell
format as long as the top field is LEGACY_KEY_ID as expected. So let's
use a single INTRODUCE format regardless of the introduction point being
legacy or not.
This also removes the polymorphic void* situation.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We consider to be in overlap mode when we are in the period of time between a
fresh SRV and the beginning of the new time period (in the normal network this
is between 00:00 and 12:00 UTC). This commit edits that function to use the
above semantic logic instead of absolute times.
Signed-off-by: David Goulet <dgoulet@torproject.org>
It used to be that time periods were 24 hours long even on chutney,
which made testing harder. With this commit, time periods have the same
length as a full SRV protocol run, which means that they will change
every 4 minutes in a 10-second voting interval chutney network!
Make this function public so we can use it both in hs_circuit.c and
hs_service.c to avoid code duplication.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This introduces a callback to relaunch a service rendezvous circuit when a
previous one failed to build or expired.
It unifies the legacy function rend_service_relaunch_rendezvous() with one for
specific to prop224. There is now only one entry point for that which is
hs_circ_retry_service_rendezvous_point() supporting both legacy and prop224
circuits.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Change the timing for intro point's lifetime and maximum amount of circuit we
are allowed to launch in a TestingNetwork. This is particurlarly useful for
chutney testing to test intro point rotation.
Signed-off-by: David Goulet <dgoulet@torproject.org>
When the circuit is about to be freed which has been marked close before, for
introduction circuit we now call this has_closed() callback so we can cleanup
any introduction point that have retried to many times or at least flag them
that their circuit is not established anymore.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We used to use NULL subcredential which is a terrible terrible idea. Refactor
HS unittests to use subcredentials.
Also add some non-fatal asserts to make sure that we always use subcredentials
when decoding/encoding descs.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit refactors the handle_hs_exit_conn() function introduced at a prior
commit that connects the rendezvous circuit to the edge connection used to
connect to the service virtual port requested in a BEGIN cell.
The refactor adds the support for prop224 adding the
hs_service_set_conn_addr_port() function that has the same purpose has
rend_service_set_connection_addr_port() from the legacy code.
The rend_service_set_connection_addr_port() has also been a bit refactored so
the common code can be shared between the two HS subsystems (legacy and
prop224).
In terms of functionallity, nothing has changed, we still close the circuits
in case of failure for the same reasons as the legacy system currently does.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit simply moves the code from the if condition of a rendezvous
circuit to a function to handle such a connection. No code was modified
_except_ the use or rh.stream_id changed to n_stream->stream_id so we don't
have to pass the cell header to the function.
This is groundwork for prop224 support which will break down the
handle_hs_exit_conn() depending on the version of hidden service the circuit
and edge connection is for.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Introduction point are rotated either if we get X amounts of INTRODUCE2 cells
on it or a time based expiration. This commit adds two consensus parameters
which are the min and max value bounding the random value X.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Imagine a Tor network where you have only 8 nodes available due to some
reasons. And your hidden service wants 8 introduction points. Everything is
fine but then a node goes down bringing the network to 7. The service will
retry 3 times that node and then give up but keep it in a failure cache for 5
minutes (INTRO_CIRC_RETRY_PERIOD) so it doesn't retry it non stop and exhaust
the maximum number of circuit retry.
In the real public network today, this is unlikely to happen unless the
ExcludeNodes list is extremely restrictive.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit adds a directory command function to make an upload directory
request for a service descriptor.
It is not used yet, just the groundwork.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This hsdir index value is used to give an index value to all node_t (relays)
that supports HSDir v3. An index value is then computed using the blinded key
to know where to fetch/upload the service descriptor from/to.
To avoid computing that index value everytime the client/service needs it, we
do that everytime we get a new consensus which then doesn't change until the
next one. The downside is that we need to sort them once we need to compute
the set of responsible HSDir.
Finally, the "hs_index" function is also added but not used. It will be used
in later commits to compute which node_t is a responsible HSDir for the
service we want to fetch/upload the descriptor.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add this helper function that can lookup and return all the needed object from
a circuit identifier. It is a pattern we do often so make it nicer and avoid
duplicating it everywhere.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add the entry point from the circuit subsystem of "circuit has opened" which
is for all type of hidden service circuits. For the introduction point, this
commit actually adds the support for handling those circuits when opened and
sending ESTABLISH_INTRO on a circuit.
Rendevzou point circuit aren't supported yet at this commit.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit adds the functionality for a service to build its descriptor.
Also, a global call to build all descriptors for all services is added to the
service scheduled events.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add the main loop entry point to the HS service subsystem. It is run every
second and make sure that all services are in their quiescent state after that
which means valid descriptors, all needed circuits opened and latest
descriptors have been uploaded.
For now, only v2 is supported and placeholders for v3 actions for that main
loop callback.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add a function for both the client and service side that is building a blinded
key from a keypair (service) and from a public key (client). Those two
functions uses the current time period information to build the key.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add a new and free function for hs_desc_intro_point_t so the service can use
them to setup those objects properly.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Fixes bug 23106; bugfix on 0.2.4.8-alpha.
Fortunately, we only support big-endian and little-endian platforms,
and on both of those, hton*() and ntoh*() behave the same. And if
we did start to support middle endian systems (haha, no), most of
_those_ have hton*(x) == ntoh*(x) too.
Since we start with desc_clean_since = 0, we should have been
starting with non-null desc_dirty_reason.
Fixes bug 22884; bugfix on 0.2.3.4-alpha when X-Desc-Gen-Reason was
added.
Patch from Vort; fixes bug 23081; bugfix on fd992deeea in
0.2.1.16-rc when set_main_thread() was introduced.
See the changes file for a list of all the symptoms this bug has
been causing when running Tor as a Windows Service.
The condition was always true meaning that we would reconsider updating our
directory information every 2 minutes.
If valid_until is 6am today, then now - 24h == 1pm yesterday which means that
"valid_until < (now - 24h)" is false. But at 6:01am tomorrow, "valid_until <
(now - 24h)" becomes true which is that point that we shouldn't trust the
consensus anymore.
Fixes#23091
Signed-off-by: David Goulet <dgoulet@torproject.org>
One log statement was a warning and has been forgotten. It is triggered for a
successful attempt at introducting from a client.
It has been reported here:
https://lists.torproject.org/pipermail/tor-relays/2017-August/012689.html
Three other log_warn() statement changed to protocol warning because they are
errors that basically can come from the network and thus triggered by anyone.
Fixes#23078.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Now that half the threads are permissive and half are strict, we
need to make sure we have at least two threads, so that we'll
have at least one of each kind.
A prop224 descriptor was missing the onion key for an introduction point which
is needed to extend to it by the client.
Closes#22979
Signed-off-by: David Goulet <dgoulet@torproject.org>
Remove the legacy intro point key because both service and client only uses
the ed25519 key even though the intro point chosen is a legacy one.
This also adds the CLIENT_PK key that is needed for the ntor handshake.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We need to keep these around for TAP and old-style hidden services,
but they're obsolete, and we shouldn't encourage anyone to use them.
So I've added "obsolete" to their names, and a comment explaining
what the problem is.
Closes ticket 23026.
It makes more sense to have the version in the configuration object of the
service because it is afterall a torrc option (HiddenServiceVersion).
Signed-off-by: David Goulet <dgoulet@torproject.org>
The added function frees any allocated pointers in a service configuration
object and reset all values to 0.
Signed-off-by: David Goulet <dgoulet@torproject.org>
As per nickm suggestion, an array of config handlers will not play well with
our callgraph tool.
Instead, we'll go with a switch case on the version which has a good side
effect of allowing us to control what we pass to the function intead of a fix
set of parameters.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add a helper function to parse uint64_t and also does logging so we can reduce
the amount of duplicate code.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This tests our hs_config.c API to properly load v3 services and register them
to the global map. It does NOT test the service object validity, that will be
the hs service unit test later on.
At this commit, we have 100% code coverage of hs_config.c.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Every hidden service option don't apply to every version so this new function
makes sure we don't have for instance an option that is only for v2 in a v3
configured service.
This works using an exclude lists for a specific version. Right now, there is
only one option that is not allowed in v3. The rest is common.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Try to load or/and generate service keys for v3. This write both the public
and private key file to disk along with the hostname file containing the onion
address.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This also adds unit test and a small python script generating a deterministic
test vector that a unit test tries to match.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit adds the support in the HS subsystem for loading a service from a
set of or_options_t and put them in a staging list.
To achieve this, service accessors have been created and a global hash map
containing service object indexed by master public key. However, this is not
used for now. It's ground work for registration process.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Introduces hs_init() located in hs_common.c which initialize the entire HS v3
subsystem. This is done _prior_ to the options being loaded because we need to
allocate global data structure before we load the configuration.
The hs_free_all() is added to release everything from tor_free_all().
Note that both functions do NOT handle v2 service subsystem but does handle
the common interface that both v2 and v3 needs such as the cache and
circuitmap.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add the hs_config.{c|h} files contains everything that the HS subsystem needs
to load and configure services. Ultimately, it should also contain client
functions such as client authorization.
This comes with a big refactoring of rend_config_services() which has now
changed to only configure a single service and it is stripped down of the
common directives which are now part of the generic handler.
This is ground work for prop224 of course but only touches version 2 services
and add XXX note for version 3.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This object is the foundation of proposal 224 service work. It will change
and be adapted as it's being used more and more in the codebase. So, this
version is just a basic skeleton one that *will* change.
Signed-off-by: David Goulet <dgoulet@torproject.org>
These statistics were largely ununsed, and kept track of statistical information
on things like how many time we had done TLS or how many signatures we had
verified. This information is largely not useful, and would only be logged
after receiving a SIGUSR1 signal (but only if the logging severity level was
less than LOG_INFO).
* FIXES#19871.
* REMOVES note_crypto_pk_op(), dump_pk_op(), and pk_op_counts from
src/or/rephist.c.
* REMOVES every external call to these functions.
Relay operators (especially bridge operators) can use this to lower
or raise the number of consensuses that they're willing to hold for
diff generation purposes.
This enables a workaround for bug 22883.
Groundwork for more prop224 service and client code. This object contains
common data that both client and service uses.
Signed-off-by: David Goulet <dgoulet@torproject.org>
- Move some crypto structures so that they are visible by tests.
- Introduce a func to count number of hops in cpath which will be used
by the tests.
- Mark a function as mockable.
This commit paves the way for the e2e circuit unittests.
Add a stub for the prop224 equivalent of rend_client_note_connection_attempt_ended().
That function was needed for tests, since the legacy function would get
called when we attach streams and our client-side tests would crash with
assert failures on rend_data.
This also introduces hs_client.[ch] to the codebase.
This commit adds most of the work of #21859. It introduces hs_circuit.c
functions that can handle the setup of e2e circuits for prop224 hidden
services, and also for legacy hidden service clients. Entry points are:
prop224 circuits: hs_circuit_setup_e2e_rend_circ()
legacy client-side circuits: hs_circuit_setup_e2e_rend_circ_legacy_client()
This commit swaps the old rendclient code to use the new API.
I didn't try to accomodate the legacy service-side code in this API, since
that's too tangled up and it would mess up the new API considerably IMO (all
this service_pending_final_cpath_ref stuff is complicated and I didn't want to
change it).
Signed-off-by: David Goulet <dgoulet@torproject.org>
The legacy HS circuit code uses rend_data to match between circuits and
streams. We refactor some of that code so that it understands hs_ident
as well which is used for prop224.
circuit_init_cpath_crypto() is responsible for creating the cpath of legacy
SHA1/AES128 circuits currently. We want to use it for prop224 circuits, so we
refactor it to create circuits with SHA3-256 and AES256 as well.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We want to use the circuit_init_cpath_crypto() function to setup our
cpath, and that function accepts a key array as input. So let's make our
HS ntor key expansion function also return a key array as output,
instead of a struct.
Also, we actually don't need KH from the key expansion, so the key
expansion output can be one DIGEST256_LEN shorter. See here for more
info: https://trac.torproject.org/projects/tor/ticket/22052#comment:3
