Commit Graph

28377 Commits

Author SHA1 Message Date
David Goulet
cdb270d55e Change git.tpo URLs to gitlab.tpo
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-06-15 13:00:11 -04:00
Mike Perry
44cd704636 Bug 40811: Count conflux leg launch attempts early.
Also, double check that the consensus has enough overall exits before
attempting conflux set launch.
2023-06-15 16:13:34 +00:00
David Goulet
44368a727a Merge branch 'tor-gitlab/mr/721' 2023-06-14 09:45:27 -04:00
Mike Perry
5d63842e86 Bug 40810: Avoid using 0 RTT legs 2023-06-13 18:18:46 +00:00
Mike Perry
dbd37c0e7b Bug 40810: Improve validation checks to ignore 0-RTT legs
Also add calls to dump the legs of a conflux set if we have too many
2023-06-13 18:18:07 +00:00
David Goulet
d5306e107f Merge branch 'tor-gitlab/mr/715' 2023-06-13 13:03:11 -04:00
Mike Perry
6a513e2ff5 Bug 40801: Do not change read state of marked conns 2023-06-09 16:29:10 +00:00
Mike Perry
da50d21c42 Bug 40801: Send LINKED_ACK before attaching streams
Otherwise, the BEGIN cell arrives at the exit before it has an RTT,
and then it does not know which circuit to prefer in response.
2023-06-09 16:29:10 +00:00
Mike Perry
ff59e2f490 Add BUG() macro to marked edge reads
This will give us a full stacktrace.
2023-06-09 16:24:03 +00:00
Mike Perry
176f0929bb Add conflux logs to diagnose cases where RTTs are absent/zero. 2023-06-09 16:24:03 +00:00
Neel Chauhan
a91315f931 Fix the spacing in the 'Your Tor identity key fingerprint is' log line' 2023-06-07 10:02:33 -07:00
Mike Perry
03d63bc7bd Add a conflux helper to log conflux sets. 2023-06-06 15:15:20 +00:00
Micah Elizabeth Scott
cfbf74352f More fixes for compile-time warnings in equix and hashx
This addresses issue #40800 and a couple other problems I noticed while
trying to reproduce that one.

The original issue is just a missing cast to void* on the args of
__builtin___clear_cache(), and clang is picky about the implicit cast
between what it considers to be char of different signedness. Original
report is from MacOS but it's also reproducible on other clang targets.

The cmake-based original build system for equix and hashx was a handy
way to run tests, but it suffered from some warnings due to incorrect
application of include_directories().

And lastly, there were some return codes from hashx_exec() that get
ignored on equix when asserts are disabled. It bugged me too much to
just silence this with a (void) cast, since even though this is in the
realm of low-likelyhood programming errors and not true runtime errors, I
don't want to make it easy for the hashx_exec() wrappers to return
values that are dangerously wrong if an error is ignored. I made sure
that even if asserts are disabled, we return values that will cause the
solver and verifier to both fail to validate a potential solution.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-06-05 11:45:33 -07:00
Gabriela Moldovan
45ee8a10e2
Fix compilation error on older gcc versions and MSVC.
This fixes an "initializer is not a constant" compilation error that manifests
itself on gcc versions < 8.1 and MSVC (see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69960#c18).

Fixes bug #40773

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-06-05 15:03:39 +01:00
Tor CI Release
5e2f6d5433 fallbackdir: Update list generated on June 01, 2023 2023-06-01 09:47:36 -04:00
Tor CI Release
c2c6c7a5e6 Update geoip files to match ipfire location db, 2023/06/01. 2023-06-01 09:47:22 -04:00
David Goulet
7f5355826b test: Really fix the mem leak from prior commit
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-06-01 09:07:43 -04:00
David Goulet
faff592c3b test: Fix a mem leak reported by Coverity
Here is the report:

  *** CID 1531835:  Resource leaks  (RESOURCE_LEAK)
  /src/test/test_crypto_slow.c: 683 in test_crypto_equix()
  677
  678           /* Solve phase: Make sure the test vector matches */
  679           memset(&output, 0xa5, sizeof output);
  680           equix_result result;
  681           result = equix_solve(solve_ctx, challenge_literal,
  682                                challenge_len, &output);
  >>>     CID 1531835:  Resource leaks  (RESOURCE_LEAK)
  >>>     Variable "solve_ctx" going out of scope leaks the storage it points to.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-06-01 08:35:08 -04:00
David Goulet
97008526db Merge branch 'maint-0.4.7' 2023-05-31 14:32:07 -04:00
David Goulet
d77f1e7aea Merge branch 'tor-gitlab/mr/714' into maint-0.4.7 2023-05-31 14:28:44 -04:00
Micah Elizabeth Scott
3036bedf30 Update CI builds to Debian Bullseye, fix associated compatibility bugs
This is a change intended for 0.4.7 maintenance as well as main.

The CI builds use Debian Buster which is now end of life, and I was
experiencing inconsistent CI failures with accessing its security update
server. I wanted to update CI to a distro that isn't EOL, and Bullseye
is the current stable release of Debian.

This opened up a small can of worms that this commit also deals with.
In particular there's a docker engine bug that we work around by
removing the docker-specific apt cleanup script if it exists, and
there's a new incompatibility between tracing and sandbox support.

The tracing/sandbox incompatibility itself had two parts:

  - The membarrier() syscall is used to deliver inter-processor
    synchronization events, and the external "userspace-rcu"
    data structure library would make assumptions that if membarrier
    is available at initialization it always will be. This caused
    segfaults in some cases when running trace + sandbox. Resolved this
    by allowing membarrier entirely, in the sandbox.

  - userspace-rcu also assumes it can block signals, and fails
    hard if this can't be done. We already include a similar carveout
    to allow this in the sandbox for fragile-hardening, so I extended
    that to cover tracing as well.

Addresses issue #40799

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-31 11:08:27 -07:00
David Goulet
925201c946 Merge branch 'tor-gitlab/mr/713' 2023-05-31 09:07:45 -04:00
orbea
9850dc59c0 tls: Disable a warning with LibreSSL >= 3.8.0
Skip a warning using EC_GFp_nist_method() which was removed in LibreSSL
3.8.

Based on a patch from OpenBSD.

33fe251a08

These functions are deprecated since OpenSSL 3.0.

https://www.openssl.org/docs/man3.1/man3/EC_GFp_nist_method.html
2023-05-29 13:00:32 -07:00
Micah Elizabeth Scott
415c0354b2 hs_pow: Add CompiledProofOfWorkHash torrc option
This exposes the new fallback behavior in hashx via a new AUTOBOOL
configuration option, available to both clients and services. The
default should be fine for nearly everyone, but it might be necessary
to enable or disable the compiler manually for diagnostic purposes.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 20:02:03 -07:00
Micah Elizabeth Scott
a397a92be2 hs_pow: Update for equix API to fix issue 40794
This change adapts the hs_pow layer and unit tests to API changes
in hashx and equix which modify the fault recovery responsibilities
and reporting behaivor.

This and the corresponding implementation changes in hashx and equix
form the fix for #40794, both solving the segfault and giving hashx a
way to report those failures up the call chain without them being
mistaken for a different error (unusable seed) that would warrant a
retry.

To handle these new late compiler failures with a minimum of fuss or
inefficiency, the failover is delegated to the internals of hashx and
tor needs only pass in a EQUIX_CTX_TRY_COMPILE flag to get the behavior
that tor was previously responsible for implementing.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 20:02:02 -07:00
Micah Elizabeth Scott
a3513dea54 equix: API changes for new result codes and hashx compatibility
This change adapts Equi-X to the corresponding HashX API changes that
added HASHX_TRY_COMPILE. The new regularized HashX return codes are
reflected by revised corresponding Equi-X return codes.

Both solve and verify operations now return an error/success code, and a
new equix_solutions_buffer struct includes both the solution buffer
and information about the solution count and hashx implementation.

With this change, it's possible to discern between hash construction
failures (invalid seed) and some external error like an mprotect()
failure.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 20:02:02 -07:00
Micah Elizabeth Scott
5a4f92ea7b hashx: API changes to allow recovery from late compile failures
This is an API breaking change to hashx, which modifies the error
handling strategy. The main goal here is to allow unproblematic
recovery from hashx_compile failures.

hashx_alloc can no longer fail for reasons other than memory
allocation. All platform-specific compile failures are now reported via
hashx_make(), in order to both allow later failure and avoid requiring
users of the API to maintain and test multiple failure paths.

Note that late failures may be more common in actual use than early
failures. Early failures represent architectures other than x86_64 and
aarch64. Late failures could represent a number of system configurations
where syscalls are restricted.

The definition of a hashx context no longer tries to overlay storage for
the different types of program, and instead allows one context to always
contain an interpretable description of the program as well as an optional
buffer for compiled code.

The hashx_type enum is now used to mean either a specific type of hash
function or a type of hashx context. You can allocate a context for use
only with interpreted or compiled functions, or you can use
HASHX_TRY_COMPILE to prefer the compiler with an automatic fallback on
the interpreter. After calling hashx_make(), the new hashx_query_type()
can be used if needed to determine which implementation was actually
chosen.

The error return types have been overhauled so that everyone uses the
hashx_result enum, and seed failures vs compile failures are always
clearly distinguishable.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 20:02:02 -07:00
Micah Elizabeth Scott
6fd5ca4914 hashx: allow hashx_compile to fail, avoid segfault without changing API
This is a minimal portion of the fix for tor issue #40794, in which
hashx segfaults due to denial of mprotect() syscalls at runtime.

Prior to this fix, hashx makes the assumption that if the JIT is
supported on the current architecture, it will also be usable at
runtime. This isn't true if mprotect fails on linux, which it may for
various reasons: the tor built-in sandbox, the shadow simulator, or
external security software that implements a syscall filter.

The necessary error propagation was missing internally in hashx,
causing us to obliviously call into code which was never made
executable. With this fix, hashx_make() will instead fail by returning
zero.

A proper fix will require API changes so that callers can discern
between different types of failures. Zero already means that a program
couldn't be constructed, which requires a different response: choosing a
different seed, vs switching implementations. Callers would also benefit
from a way to use one context (with its already-built program) to
run in either compiled or interpreted mode.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 19:54:50 -07:00
Micah Elizabeth Scott
941613c663 hashx: minor, another logical operator change
The code style in equix and hashx sometimes uses bitwise operators
in place of logical ones in cases where it doesn't really matter
either way. This sometimes annoys our static analyzer tools.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 19:54:50 -07:00
Micah Elizabeth Scott
c40c5adec2 test_sandbox: equix crypto test case for issue 40794
This is an additional test case for test_sandbox that runs a small
subset of test_crypto_equix() inside the syscall sandbox, where
mprotect() is filtered.

It's reasonable for the sandbox to disallow JIT. We could revise this
policy if we want, but it seems a good default for now. The problem
in issue 40794 is that both equix and hashx need improvements in their
API to handle failures after allocation time, and this failure occurs
while the hash function is being compiled.

With this commit only, the segfault from issue 40794 is reproduced.
Subsequent commits will fix the segfault and revise the API.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-28 19:54:50 -07:00
friendly73
273227afe1 Forgot about the stub names 2023-05-25 11:03:35 -04:00
friendly73
3d5d8d59c1 Added relay prefix to new metrics functions 2023-05-25 11:03:35 -04:00
friendly73
7e57b9dbbf Fixed enum type not found in relay_stub 2023-05-25 11:03:35 -04:00
friendly73
5b76ce1843 Added void stubs for the relay metrics functions to fix building without relay module 2023-05-25 11:03:35 -04:00
friendly73
8cbfc90686 Fixed new arguments for metrics_store_add 2023-05-25 11:03:35 -04:00
friendly73
1899b6230d Removed getter abstraction and moved from rephist to relay_metrics. 2023-05-25 11:03:35 -04:00
friendly73
2f8a88448d Fixed est intro getter using wrong array 2023-05-25 11:03:35 -04:00
friendly73
24bc66f663 Fixed REND1 metric label value 2023-05-25 11:03:35 -04:00
friendly73
36076d3c46 Added INTRO and REND metrics for relay. 2023-05-25 11:03:35 -04:00
David Goulet
a1042f4873 Merge branch 'tor-gitlab/mr/443' 2023-05-25 10:50:15 -04:00
Alexander Færøy
506781d41e Restart PT processes when they die on us.
This patch forces a PT reconfigure of infant PT processes as part of the
PT process' exit handler.

See: tpo/core/tor#33669
2023-05-25 10:50:11 -04:00
Alexander Færøy
58f0e548ff Log at LD_PT instead of LD_GENERAL for PT process stdout lines.
See: tpo/core/tor#33669
2023-05-25 10:50:11 -04:00
Alexander Færøy
3338b34ec9 Only terminate PT processes that are running.
See: tpo/core/tor#33669
2023-05-25 10:50:11 -04:00
Alexander Færøy
0d51dfa605 Log name of managed proxy in exit handler.
This patch ensures that we can figure out which PT that terminated in
the PT exit handler.

See: tpo/core/tor#33669
2023-05-25 10:50:11 -04:00
Alexander Færøy
5118a8003b Log state transitions for Pluggable Transports
This patch makes Tor log state transitions within the PT layer at the
info log-level. This should make it easier to figure out if Tor ends up
in a strange state.

See: tpo/core/tor#33669
2023-05-25 10:50:11 -04:00
David Goulet
970a534f03 test: Fix parseconf to account for ClientUseIPv6 change for dirauth disabled
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-25 10:20:12 -04:00
David Goulet
86bc3cc452 test: Fix parseconf to account for ClientUseIPv6 change
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-25 09:21:23 -04:00
David Goulet
a2ec9a1199 Merge branch 'tor-gitlab/mr/711' 2023-05-24 11:45:40 -04:00
Micah Elizabeth Scott
23f4a28f97 token_bucket_ctr: replace 32-bit wallclock time with monotime
This started as a response to ticket #40792 where Coverity is
complaining about a potential year 2038 bug where we cast time_t from
approx_time() to uint32_t for use in token_bucket_ctr.

There was a larger can of worms though, since token_bucket really
doesn't want to be using wallclock time here. I audited the call sites
for approx_time() and changed any that used a 32-bit cast or made
inappropriate use of wallclock time. Things like certificate lifetime,
consensus intervals, etc. need wallclock time. Measurements of rates
over time, however, are better served with a monotonic timer that does
not try and sync with wallclock ever.

Looking closer at token_bucket, its design is a bit odd because it was
initially intended for use with tick units but later forked into
token_bucket_rw which uses ticks to count bytes per second, and
token_bucket_ctr which uses seconds to count slower events. The rates
represented by either token bucket can't be lower than 1 per second, so
the slower timer in 'ctr' is necessary to represent the slower rates of
things like connections or introduction packets or rendezvous attempts.

I considered modifying token_bucket to use 64-bit timestamps overall
instead of 32-bit, but that seemed like an unnecessarily invasive change
that would grant some peace of mind but probably not help much. I was
more interested in removing the dependency on wallclock time. The
token_bucket_rw timer already uses monotonic time. This patch converts
token_bucket_ctr to use monotonic time as well. It introduces a new
monotime_coarse_absolute_sec(), which is currently the same as nsec
divided by a billion but could be optimized easily if we ever need to.

This patch also might fix a rollover bug.. I haven't tested this
extensively but I don't think the previous version of the rollover code
on either token bucket was correct, and I would expect it to get stuck
after the first rollover.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-24 11:43:11 -04:00
David Goulet
9976da9367 Merge branch 'tor-gitlab/mr/709' 2023-05-24 11:37:05 -04:00
David Goulet
0781c2968d Merge branch 'tor-gitlab/mr/710' 2023-05-24 11:12:22 -04:00
Micah Elizabeth Scott
71b2958a62 test_hs_descriptor: Add a test case that fails without the fix for 40793
This adds a bit more to hs_descriptor/test_decode_descriptor, mostly
testing pow-params and triggering the tor_assert() in issue #40793.

There was no mechanism for adding arbitrary test strings to the
encrypted portion of the desc without duplicating encode logic. One
option might be to publicize get_inner_encrypted_layer_plaintext enough
to add a mock implementation. In this patch I opt for what seems like
the simplest solution, at the cost of a small amount of #ifdef noise.
The unpacked descriptor grows a new test-only member that's used for
dropping arbitrary data in at encode time.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-24 11:12:15 -04:00
David Goulet
6afe03aa51 Merge branch 'tor-gitlab/mr/708' 2023-05-24 11:03:47 -04:00
agowa338
ffb764949e ipv6: Flip ClientUseIPv6 to 1
Fixes #40785

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-24 11:03:15 -04:00
David Goulet
8eae9f17ae metrics: Add ticket 40546 changes file and code fix
The MR was using an old function definition so the code fix is for that.

Closes #40546

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-24 10:45:21 -04:00
David Goulet
21ec9017f6 Merge branch 'tor-gitlab/mr/698' 2023-05-24 10:40:25 -04:00
David Goulet
6bf56ac301 Merge branch 'tor-gitlab/mr/703' 2023-05-24 10:38:58 -04:00
Micah Elizabeth Scott
459b775a7e hs_pow: fix insufficient length check in pow-params
The descriptor validation table had an out of date minimum length
for pow-params (3) whereas the spec and the current code expect at
least 4 parameters. This was an opportunity for a malicious service
to cause an assert failure in clients which attempted to parse its
descriptor.

Addresses issue #40793

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-15 12:11:00 -07:00
Micah Elizabeth Scott
a3ff3155c2 test_crypto: avoid memory leak in some hashx test failures
This should fix one of the warnings in issue #40792.

I was sloppy with freeing memory in the failure cases for
test_crypto_hashx. ASAN didn't notice but coverity did. Okay, I'll eat
my vegetables and put hashx_ctx's deinit in an upper scope and use
'goto done' correctly like a properly diligent C programmer.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-11 11:17:43 -07:00
Micah Elizabeth Scott
c71b6a14a3 equix: avoid a coverity warning in hashx_alloc()
This addresses one of the warnings in issue #40792. As far as I can tell
this is a false positive, since the use of "ctx->type" in hashx_free()
can only be hit after the unioned code/program pointer is non-NULL. It's
no big deal to zero this value explicitly to silence the warning though.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-11 11:10:15 -07:00
Mike Perry
79dab29a05 Add torrc option for conflux client UX. 2023-05-11 18:05:28 +00:00
Mike Perry
0c11577987 Fix unit tests. 2023-05-11 18:05:28 +00:00
Mike Perry
a340acb492 Clean up UX decision logic; hardcode for browser UX case. 2023-05-11 18:02:51 +00:00
Roger Dingledine
34da50718a fix minor typos in conflux and pow areas 2023-05-11 13:09:34 -04:00
Mike Perry
c8341abf82 Clean up and disable switch rate limiting.
Switch rate limiting will likely be helpful for limiting OOQ, but according to
shadow it was the cause of slower performance in Hong Kong endpoints.

So let's disable it, and then optimize for OOQ later.
2023-05-10 17:49:51 +00:00
Mike Perry
a98b1d3ab6 Remove two conflux algs: maxrate and cwndrate.
Maxrate had slower throughput than lowrtt in Shadow, which is not too
surprising. We just wanted to test it.
2023-05-10 17:49:51 +00:00
Micah Elizabeth Scott
e643a70879 hs_pow: Modify challenge format, include blinded HS id
This is a protocol breaking change that implements nickm's
changes to prop 327 to add an algorithm personalization string
and blinded HS id to the EquiX challenge string for our onion
service client puzzle.

This corresponds with the spec changes in torspec!130,
and it fixes a proposed vulnerability documented in
ticket tor#40789.

Clients and services prior to this patch will no longer
be compatible with the proposed "v1" proof-of-work protocol.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
138fd57072 hs_pow: add per-circuit effort information to control port
This lets controller apps see the outgoing PoW effort on client
circuits, and the validated effort received on an incoming service
circuit.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
971de27c07 hs_pow: fix error path with outdated assumption
This error path with the "PoW cpuworker returned with no solution.
Will retry soon." message was usually lying. It's concerning
now because we expect to always find a solution no matter how
long it takes, rather than re-enter the solver repeatedly, so any
exit without a solution is a sign of a problem.

In fact when this error path gets hit, we are usually missing a
circuit instead because the request is quite old and the circuits
have been destroyed. This is not an emergency, it's just a sign
of client-side overload.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
cba1ffb43a hs_pow: swap out some comments
i think we're done with these?
and swap in a nonfatal assert to replace one of the comments.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
a13d7bd5e9 hs_pow: always give other events a chance to run between rend requests
This dequeue path has been through a few revisions by now, first
limiting us to a fixed number per event loop callback, then an
additional limit based on a token bucket, then the current version
which has only the token bucket.

The thinking behing processing multiple requests per callback was to
optimize our usage of libevent, but in effect this creates a
prioritization problem. I think even a small fixed limit would be less
reliable than just backing out this optimization and always allowing
other callbacks to interrupt us in-between dequeues.

With this patch I'm seeing much smoother queueing behavior when I add
artificial delays to the main thread in testing.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
6023153631 hs_pow: modified approach to pqueue level thresholds
This centralizes the logic for deciding on these magic thresholds,
and tries to reduce them to just two: a min and max. The min should be a
"nearly empty" threshold, indicating that the queue only contains work
we expect to be able to complete very soon. The max level triggers a
bulk culling process that reduces the queue to half that amount.

This patch calculates both thresholds based on the torrc pqueue rate
settings if they're present, and uses generic defaults if the user asked
for an unlimited dequeue rate in torrc.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
50313d114f hs_pow: faster hs_circuitmap lookup for rend in pow_worker_job_t
The worker job queue for hs_pow needs what's effectively a weak pointer
to two circuits, but there's not a generic mechanism for this in c-tor.
The previous approach of circuit_get_by_global_id() is straightforward
but not efficient. These global IDs are normally only used by the
control port protocol. To reduce the number of O(N) lookups we have over
the whole circuit list, we can use hs_circuitmap to look up the rend
circuit by its auth cookie.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
a6138486f7 hs_pow: review feedback, use MAX for max_trimmed_effort
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
ee63863dca hs_pow: Lower several logs from notice to info
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:41:37 -07:00
Micah Elizabeth Scott
ff678d0fb5 hs_pow: update_suggested_effort fix and cleanup
This is trying to be an AIMD event-driven algorithm, but we ended up with
two different add paths with diverging behavior. This fix makes the AIMD
events more explicit, and it fixes an earlier behavior where the effort
could be decreased (by the add/recalculate branch) even when the pqueue
was not emptying at all. With this patch we shouldn't drop down to an
effort of zero as long as even low-effort attacks are flooding the
pqueue.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:40:46 -07:00
Micah Elizabeth Scott
903c6cf1ab hs_pow: client side effort adjustment
The goal of this patch is to add an additional mechanism for adjusting
PoW effort upwards, where clients rather than services can choose to
solve their puzzles at a higher effort than what was suggested in the
descriptor.

I wanted to use hs_cache's existing unreachability stats to drive this
effort bump, but this revealed some cases where a circuit (intro or
rend) closed early on can end up in hs_cache with an all zero intro
point key, where nobody will find it. This moves intro_auth_pk
initialization earlier in a couple places and adds nonfatal asserts to
catch the problem if it shows up elsewhere.

The actual effort adjustment method I chose is to multiply the suggested
effort by (1 + unresponsive_count), then ensure the result is at least
1. If a service has suggested effort of 0 but we fail to connect,
retries will all use an effort of 1. If the suggestion was 50, we'll try
50, 100, 150, 200, etc. This is bounded both by our client effort limit
and by the limit on unresponsive_count (currently 5).

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:40:46 -07:00
Micah Elizabeth Scott
ac466a2219 hs_pow: leak fix, free the contents of pqueue entries in hs_pow_free_service_state
Asan catches this pretty readily when ending a service gracefully while
a DoS is in progress and the queue is full of items that haven't yet
timed out.

The module boundaries in hs_circuit are quite fuzzy here, but I'm trying
to follow the vibe of the existing hs_pow code.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:29 -07:00
Micah Elizabeth Scott
ac29c7209d hs_pow: bump client-side effort limit from 500 to 10000
500 was quite low, but this limit was helpful when the suggested-effort
estimation algorithm was likely to give us large abrupt increases. Now
that this should be fixed, let's allow spending a bit more time on the
client puzzles if it's actually necessary.

Solving a puzzle with effort=10000 usually completes within a minute
on my old x86_64 machine. We may want to fine tune this further, and it
should probably be made into a config option.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:29 -07:00
Micah Elizabeth Scott
6a0809c4e3 hs_pow: stop having a "minimum effort", and let PoW effort start low
I don't think the concept of "minimum effort" is really useful to us,
so this patch removes it entirely and consequentially changes the way
that "total" effort is calculated so that we don't rely on any minimum
and we instead ramp up effort no faster than necessary.

If at least some portion of the attack is conducted by clients that
avoid PoW or provide incorrect solutions, those (potentially very
cheap) attacks will end up keeping the pqueue full. Prior to this patch,
that would cause suggested efforts to be unnecessarily high, because
rounding these very cheap requests up to even a minimum of 1 will
overestimate how much actual attack effort is being spent.

The result is that this patch is a simplification and it also allows a
slower start, where PoW effort jumps up either by a single unit or by an
amount calculated from actual effort in the queue.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:29 -07:00
Micah Elizabeth Scott
2de98a7f4e hs_pow: Represent equix_solution as a byte array
This patch is intended to clarify the points at which we convert
between the internal representation of an equix_solution and a portable
but opaque byte array representation.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
287c78c5a8 sandbox: allow stack mmap with prot_none
This fixes a failure that was showing up on i386 Debian hosts
with sandboxing enabled, now that cpuworker is enabled on clients.
We already had allowances for creating threads and creating stacks
in the sandbox, but prot_none (probably used for a stack guard)
was not allowed so thread creation failed.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
700814a3a1 hs_pow: Fix nonce cache entry leak
This leak was showing up in address sanitizer runs of test_hs_pow,
but it will also happen during normal operation as seeds are rotated.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
00d9e0d252 hs_pow: Define seed_head as uint8_t[4] instead of uint32_t
This is more consistent with the specification, and it's much
less confusing with endianness. This resolves the underlying
cause of the earlier byte-swap. This patch itself does not
change the wire protocol at all, it's just tidying up the
types we use at the trunnel layer.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
209a59face hs_pow: Don't require uint128_t
We were using a native uint128_t to represent the hs_pow nonce,
but as the comments note it's more portable and more flexible to
use a byte array. Indeed the uint128_t was a problem for 32-bit
platforms. This swaps in a new implementation that uses multiple
machine words to implement the nonce incrementation.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
1a3afeb387 hs_pow: unswap byte order of seed_head field
In proposal 327, "POW_SEED is the first 4 bytes of the seed used".

The proposal doesn't specifically mention the data type of this field,
and the code in hs_pow so far treats it as an integer but semantically
it's more like the first four bytes of an already-encoded little endian
blob. This leads to a byte swap, since the type confusion takes place
in a little-endian subsystem but the wire encoding of seed_head uses
tor's default of big endian.

This patch does not address the underlying type confusion, it's a
minimal change that only swaps the byte order and updates unit tests
accordingly. Further changes will clean up the data types.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
037dea2252 hs_pow: fix assert in services that receive unsolicited proof of work
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
3129910b11 hs_pow: use the compiled HashX implementation
Much faster per-hash, affects both verify and solve.
Only implemented on x86_64 and aarch64, other platforms
always use the interpreted version of hashx.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
c6b168e141 test_hs_pow: add test vectors for our hs_pow client puzzle
This adds test vectors for the overall client puzzle at the
hs_pow and hs_cell layers.

These are similar to the crypto/equix tests, but they also cover
particulars of our hs_pow format like the conversion to byte arrays,
the replay cache, the effort test, and the formatting of the equix
challenge string.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
0c11411f35 hashx: trim trailing whitespace
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
ae86d98815 equix: Portability fixes for big endian platforms
Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
daa08557ad equix: Build cleanly with -Wall -Werror
Fixes some type nitpicks that show up in Tor development builds,
which usually run with -Wall -Werror. Tested on x86_64 and aarch64
for clean build and passing equix-tests + hashx-tests.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
246ced3a8c ext: build equix and hashx using automake
This replaces the sketchy cmake invocation we had inside configure

The libs are always built and always used in unit tests, but only
included in libtor and tor when --enable-gpl is set.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
bfa2102c95 hs_pow: Replace libb2 dependency with hashx's internal blake2
This forgoes another external library dependency, and instead
introduces a compatibility header so that interested parties
(who already depend on equix, like hs_pow and unit tests) can
use the implementation of blake2b included in hashx.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
ffa8531fe0 test_crypto: add equix and hashx tests
This adds test vectors for the Equi-X proof of work algorithm and the
Hash-X function it's based on. The overall Equi-X test takes about
10 seconds to run on my machine, so it's in test_crypto_slow. The hashx
test still covers both the compiled and interpreted versions of the
hash function.

There aren't any official test vectors for Equi-X or for its particular
configuration of Hash-X, so I made some up based on the current
implementation.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
92f83347f7 test_crypto: add blake2b test vectors
I'm planning on swapping blake2b implementations, and this test
is intended to prevent regressions. Right now blake2b is only used by
hs_pow.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
dcb9c4df67 hs_pow: Make proof-of-work support optional in configure
This adds a new "pow" module for the user-visible proof
of work support in ./configure, and this disables
src/feature/hs/hs_pow at compile-time.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
9d1a573977 configure: Add --enable-gpl option
This change on its own doesn't use the option for anything, but
it includes support for configure and a message in 'tor --version'

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
557eb81486 hs_pow_solve: use equix_solve more efficiently
This was apparently misinterpreting "zero solutions" as an error
instead of just moving on to the next nonce. Additionally, equix
could have been returning up to 8 solutions and we would only
give one of those a chance to succeed.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
0e271dda77 hs_pow: reduce min_effort default to 1
We may want to choose something larger eventually, but 20 seemed
much too large. Very low nonzero efforts are still useful against
a script kiddie level DoS attack.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
f3b98116b6 hs_pow: Rate limited dequeue
This adds a token bucket ratelimiter on the dequeue side
of hs_pow's priority queue. It adds config options and docs
for those options. (HiddenServicePoWQueueRate/Burst)

I'm testing this as a way to limit the overhead of circuit
creation when we're experiencing a flood of rendezvous requests.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
20d7c8ce14 fix typo in HiddenServiceExportCircuitID
Really inconsequential, since the string was only used for logging a
warning.
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
a0b9f3546e hs_pow: check for expired params in can_client_refetch_desc
Without this check, we never actually refetch the hs descriptor
when PoW parameters expire, because can_client_refetch_desc
deems the descriptor to be still good.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
48c67263d9 hs_metrics: Proof of Work pqueue depth, suggested effort
Adds two new metrics for hs_pow, and an internal parameter within
hs_metrics for implementing gauge parameters that reset before
every update.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-05-10 07:38:28 -07:00
Micah Elizabeth Scott
09afc5eacf update_suggested_effort: avoid assert if the pqueue has emptied
top_of_rend_pqueue_is_worthwhile requires a nonempty queue.
2023-05-10 07:37:11 -07:00
Roger Dingledine
eba9190933 compute the client-side pow in a cpuworker thread
We mark the intro circuit with a new flag saying that the pow is
in the cpuworker queue. When the cpuworker comes back, it either
has a solution, in which case we proceed with sending the intro1
cell, or it has no solution, in which case we unmark the intro
circuit and let the whole process restart on the next iteration of
connection_ap_handshake_attach_circuit().
2023-05-10 07:37:11 -07:00
Roger Dingledine
aa41d4b939 refactor send_introduce1()
into two parts:

* a "consider whether to send an intro2 cell" part (now called
consider_sending_introduce1()), and

* an "actually send it" (now called send_introduce1()).
2023-05-10 07:37:11 -07:00
Roger Dingledine
a5b0c7b404 start the cpuworkers always, even for clients
prepares the way for client-side pow cpuworkers

also happens to resolve bug https://bugs.torproject.org/tpo/core/tor/40617
(which went into 0.4.7.4-alpha) because now we survive initing the
cpuworker subsystem when we're not a relay.
2023-05-10 07:37:11 -07:00
Roger Dingledine
0716cd7cb2 allow suggested effort to be 0
First (both client and service), make descriptor parsing not fail when
suggested_effort is 0.

Second (client side), if we get a descriptor with a pow_params section
but with suggested_effort of 0, treat it as not requiring a pow.

Third (service side), when deciding whether the suggested effort has
changed, don't treat "previous suggested effort 0, new suggested effort 0"
as a change.

An alternative design to resolve 'first' and 'second' above would be
to omit the pow_params from the descriptor when suggested_effort is 0,
so clients never see the pow_params so they don't compute a pow. But
I decided to include a pow_params with an explicit suggested_effort
of 0, since this way the client knows the seed etc so they can solve
a higher-effort pow if they want. The tradeoff is that the descriptor
reveals whether HiddenServicePoWDefensesEnabled is set to 1 for this onion
service, even if the AIMD calculation is currently requiring effort 0.
2023-05-10 07:37:11 -07:00
Mike Perry
d36144ba31 Initialize startup effort at 0.
If it works correctly, auto-tuning should set a non-zero effort once
an attack begins.
2023-05-10 07:37:11 -07:00
Mike Perry
ec9e95cf1e Implement AIMD effort estimation.
Now, pow should auto-enable and auto-disable itself.
2023-05-10 07:37:11 -07:00
Mike Perry
5b3a067fe3 Replace the constant bottom-half rate with handled count.
This allows us to more accurately estimate effort, based on real bottom-half
throughput over the duration of a descriptor update.
2023-05-10 07:37:11 -07:00
Mike Perry
121766e6b8 Make the thing compile. 2023-05-10 07:37:11 -07:00
Roger Dingledine
e605620744 clients defend themselves from absurd pow requests
if asked for higher than a cap, we just solve it at the cap

i picked 500 for now but maybe we'll pick a better number in the future.
2023-05-10 07:37:11 -07:00
Roger Dingledine
ec7495d35a log_err is reserved for fatal failures 2023-05-10 07:37:11 -07:00
Roger Dingledine
e436ce2a3c drop the default min effort to 20
effort 100 is really quite expensive
2023-05-10 07:37:11 -07:00
Roger Dingledine
a575e35c17 sort pqueue ties by time-added
our pqueue implementation does bizarre unspecified things with
ordering of elements that are equal. it certainly doesn't do any
sort of "first in first out" property that i was expecting.

now make it explicit by saying that "equal-effort, added-earlier" is
higher priority.
2023-05-10 07:37:11 -07:00
Roger Dingledine
13f6258245 rate-limit low-effort rendezvous responses
specifically, if we have 16 in-flight rend circs, and the next
one at the top of the pqueue is lower than our suggested effort,
then don't launch it yet.

this way we always launch adequate-effort requests immediately, and
we always handle *some* low-effort requests, but we are ready at any
moment to handle a few new adequate-effort requests.
2023-05-10 07:37:11 -07:00
Roger Dingledine
dec3a0af7a make the rend_pqueue_cb event be postloop
this change makes us reach the callback *after* each mainloop
run, rather than as the next event to run immediately after
activation.

with the old behavior, we were starving everything else to drain the
pqueue entirely, each time we got a new intro2 cell.

now we at least will get to other activities as well.
2023-05-10 07:37:11 -07:00
Roger Dingledine
b95bd5017f track how many in-flight hs-side rend circs
not used in decision-making yet, but it's all ready to use in a
"don't dequeue any more if we have too many in-flight" kind of way
2023-05-10 07:37:11 -07:00
Roger Dingledine
5e768d5cb9 we were sorting our pqueue the wrong way
i.e. we were putting higher effort intro2 cells at the *end*
2023-05-10 07:37:11 -07:00
Roger Dingledine
d0c2d4cb43 add a log line for when client succeeds 2023-05-10 07:37:11 -07:00
Roger Dingledine
4e55f28220 bump up some log messages for easier debugging 2023-05-10 07:37:11 -07:00
Roger Dingledine
8042379c44 new design for handling too many pending rend reqs
now we let ourselves queue up to twice as many as we expect, and when
we get to the limit we make a new pqueue and move over the first n
elements that we like most.

(the old approach, of calling SMARTLIST_DEL_CURRENT_KEEPORDER() on
elements in a pqueue, will destroy its heapify property.)

we also discard elements that are too old, either during the trimming
process or if they come up as the next request to respond to.

lastly, fix a fencepost error on how many rend reqs we would handle
per iteration.
2023-05-10 07:37:11 -07:00
Roger Dingledine
85cba057e7 make a log message clearer about our actual intent 2023-05-10 07:37:11 -07:00
Roger Dingledine
4571faf0c3 pass time around as a parameter
should help with unit testing
2023-05-10 07:37:11 -07:00
David Goulet
047f8c63ee hs: Maximum rend request and trimming of the queue
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
bc9fe5a6f8 hs: Handle multiple rend request per mainloop run
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
c2f6b057b8 hs: Don't expire RP circuits to HS with PoW
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
35227a7a15 trunnel: Centralize the INTRO1 extension type
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
4eb783e97b hs: Priority queue for rendezvous requests
If PoW are enabled, use a priority queue by effort for the rendezvous
requests hooked into the mainloop.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
f0b63ca242 hs: Move rendezvous circuit data structure
When parsing an INTRODUCE2 cell, we extract data in order to launch the
rendezvous circuit. This commit creates a data structure just for that
data so it can be used by future commits for prop327 in order to copy
that data over a priority queue instead of the whole intro data data
structure which contains pointers that could dissapear.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
ca74530b40 hs: Setup service side PoW defenses
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
8b41e09a77 hs: Client now solve PoW if present
At this commit, the tor main loop solves it. We might consider moving
this to the CPU pool at some point.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
26957b47ac hs: Descriptor support for PoW
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:11 -07:00
David Goulet
51ce0bb6ef hs: Add solve and verify PoW functions
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:10 -07:00
David Goulet
c611e328de hs: Add data structure needed for PoW 2023-05-10 07:37:10 -07:00
David Goulet
d79814f1b1 hs: PoW extension encoding
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:10 -07:00
David Goulet
5ef811b7d0 trunnel: INTRODUCE1 PoW cell extension
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:10 -07:00
David Goulet
95445f49f1 ext: Add Equi-X library
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-05-10 07:37:10 -07:00
Mike Perry
9ee71eaf5a CID 1524707: Quiet coverity noise 2023-05-04 16:31:08 +00:00
Mike Perry
bdf4fef2db CID 1524706: Remove dead assignment 2023-05-04 16:31:08 +00:00
Mike Perry
33c3059c82 Handle infinite loop with only one bridge (or snowflake). 2023-05-04 16:31:08 +00:00
Mike Perry
61aa4c3657 Actually count exits with conflux support, rather than relays. 2023-04-18 16:51:07 +00:00
David Goulet
2bb8988629 Fix cases where edge connections can stall.
We discovered two cases where edge connections can stall during testing:
  1. Due to final data sitting in the edge inbuf when it was resumed
  2. Due to flag synchronization between the token bucket and XON/XOFF

The first issue has always existed in C-Tor, but we were able to tickle it
in scp testing. If the last data from the protocol is able to fit in the
inbuf, but not large enough to send, if an XOFF or connection block comes in
at exactly that point, when the edge connection resumes, there will be no
data to read from the socket, but the inbuf can just sit there, never
draining.

We noticed the second issue along the way to finding the first. It seems
wrong, but it didn't seem to affect anything in practice.

These are extremely rare in normal operation, but with conflux, XON/XOFF
activity is more common, so we hit these.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:11 +00:00
Mike Perry
7c70f713c3 Avoid closing dirty circs with active half-edges
In https://gitlab.torproject.org/tpo/core/tor/-/issues/40623, we changed the
DESTROY propogation to ensure memory was freed quickly at relays. This was a
good move, but it exacerbates the condition where a stream is closed on a
circuit, and then it is immediately closed because it is dirty. This creates a
race between the DESTROY and the last data sent on the stream. This race is
visible in shadow, and does happen.

This could be backported. A better solution to these kinds of problems is to
create an ENDED cell, and not close any circuits until the ENDED comes back.
But this will also require thinking, since this ENDED cell can also get lost,
so some kind of timeout may be needed either way. The ENDED cell could just
allow us to have much longer timeouts for this case.
2023-04-06 15:57:11 +00:00
David Goulet
731a50c8c4 Prop#329: Add conflux to build
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:11 +00:00
Mike Perry
8d4781e730 Prop#329 Tests: Add tests for the conflux pool 2023-04-06 15:57:11 +00:00
David Goulet
39c2927d6f Prop#329 Pool: Handle pre-building and using conflux sets.
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:11 +00:00
Mike Perry
46e473f43e Prop#329 Pool: Avoid sharing Guards and Middles between circuits.
Conflux must not use the same Guard for each leg; nor the same middle for each
leg.
2023-04-06 15:57:11 +00:00
David Goulet
336a24754d Prop#329 Pool: Handle linking, unlinking, and relaunching conflux circuit legs.
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:11 +00:00
Mike Perry
2f865b4bba Prop#329 streams: Handle stream usage with conflux
This adds utility functions to help stream block decisions, as well as cpath
layer_hint checks for stream cell acceptance, and syncing stream lists
for conflux circuits.

These functions are then called throughout the codebase to properly manage
conflux streams.
2023-04-06 15:57:11 +00:00
Mike Perry
21c861bfa3 Refactor stream blocking due to channel cell queues
Streams can get blocked on a circuit in two ways:
  1. When the circuit package window is full
  2. When the channel's cell queue is too high

Conflux needs to decouple stream blocking from both of these conditions,
because streams can continue on another circuit, even if the primary circuit
is blocked for either of these cases.

However, both conflux and congestion control need to know if the channel's
cell queue hit the highwatermark and is still draining, because this condition
is used by those components, independent of stream state.

Therefore, this commit renames the 'streams_blocked_on_chan' variable to
signify that it refers to the cell queue state, and also refactors the actual
stream blocking bits out, so they can be handled separately if conflux is
present.
2023-04-06 15:57:10 +00:00
Mike Perry
a4ee0c29ee Prop#329: Add purposes for conflux circuits
Because UNLINKED circuits must never be used for streams, but LINKED circuits
can be, we want these separate.
2023-04-06 15:57:10 +00:00
Mike Perry
cf715a56f1 Prop#329 sendme: Adjust sendme sending and tracking for conflux
Because circuit-level sendmes are sent before relay data cells are processed,
we can safely move this to before the conflux decision. In this way,
regardless of conflux being negotiated, we still send sendmes as soon as data
cells are recieved. This avoids introducing conflux queue delay into RTT
measurement, which is important for measuring actual circuit capacity.

The circuit-level tracking must happen inside the call to send a data cell,
since that call now chooses a circuit to send on. Turns out, we were already
doing this kind of here, but only for the digest. Now we do both things here.
2023-04-06 15:57:10 +00:00
David Goulet
b999051e44 Prop#329 OOM: Handle freeing conflux queues on OOM
We use the oldest-circ-first method here, since that seems good for conflux:
queues could briefly spike, but the bad case is if they are maliciously
bloated to stick around for a long time.

The tradeoff here is that it is possible to kill old circuits on a relay
quickly, but that has always been the case with this algorithm choice.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:10 +00:00
Mike Perry
e0881a669a Prop#329 Algs: Conflux multiplexed cell sending decision algs 2023-04-06 15:57:10 +00:00
Mike Perry
2bd1eca78c Prop#329 Algs: Conflux multiplexed cell receive handling 2023-04-06 15:57:10 +00:00
Mike Perry
a1794ef687 Prop#329 Headers: Header files for conflux 2023-04-06 15:57:10 +00:00
Mike Perry
eac2bad86b Prop#329 params: Consensus parameter and torrc handling
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:10 +00:00
David Goulet
0d14d6b44a Prop#329 Tests: Add tests for conflux cells.
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:10 +00:00
David Goulet
8b185b2ac3 Prop#329 Cells: Building and parsing parsing conflux commands
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:10 +00:00
David Goulet
45432175fe trunnel: Add Conflux related cell definition
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:10 +00:00
David Goulet
a9fc6c937c protover: Support Relay=5 for Conflux (prop329)
Closes #40721

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-04-06 15:57:10 +00:00
Saksham Mittal
738785ead8
Test both \r and \r\n with the expected failure conditions added 2023-03-31 11:04:18 +05:30
Saksham Mittal
c5889d3f15
Test \r more effectively using 2 lines 2023-03-27 20:22:24 +05:30
trinity-1686a
0c634999c1
don't emit key expiration metric instead of setting it to zero 2023-03-19 10:31:28 +01:00
Saksham Mittal
bbb3396d79
Add test for \r in directory parsing 2023-03-19 08:28:29 +05:30
David Goulet
59456cb3cf Merge branch 'tor-gitlab/mr/700' 2023-03-13 11:22:31 -04:00
Gabriela Moldovan
1a60fa547f metrics: Add HS service side circuit build time metrics.
This adds 2 histogram metrics for hidden services:
* `tor_hs_rend_circ_build_time` - the rendezvous circuit build time in milliseconds
* `tor_hs_intro_circ_build_time` - the introduction circuit build time in milliseconds

The text representation representation of the new metrics looks like this:
```
# HELP tor_hs_rend_circ_build_time The rendezvous circuit build time in milliseconds
# TYPE tor_hs_rend_circ_build_time histogram
tor_hs_rend_circ_build_time_bucket{onion="<elided>",le="1000.00"} 2
tor_hs_rend_circ_build_time_bucket{onion="<elided>",le="5000.00"} 10
tor_hs_rend_circ_build_time_bucket{onion="<elided>",le="10000.00"} 10
tor_hs_rend_circ_build_time_bucket{onion="<elided>",le="30000.00"} 10
tor_hs_rend_circ_build_time_bucket{onion="<elided>",le="60000.00"} 10
tor_hs_rend_circ_build_time_bucket{onion="<elided>",le="+Inf"} 10
tor_hs_rend_circ_build_time_sum{onion="<elided>"} 10824
tor_hs_rend_circ_build_time_count{onion="<elided>"} 10
# HELP tor_hs_intro_circ_build_time The introduction circuit build time in milliseconds
# TYPE tor_hs_intro_circ_build_time histogram
tor_hs_intro_circ_build_time_bucket{onion="<elided>",le="1000.00"} 0
tor_hs_intro_circ_build_time_bucket{onion="<elided>",le="5000.00"} 6
tor_hs_intro_circ_build_time_bucket{onion="<elided>",le="10000.00"} 6
tor_hs_intro_circ_build_time_bucket{onion="<elided>",le="30000.00"} 6
tor_hs_intro_circ_build_time_bucket{onion="<elided>",le="60000.00"} 6
tor_hs_intro_circ_build_time_bucket{onion="<elided>",le="+Inf"} 6
tor_hs_intro_circ_build_time_sum{onion="<elided>"} 9843
tor_hs_intro_circ_build_time_count{onion="<elided>"} 6
```

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-13 11:18:40 -04:00
Gabriela Moldovan
d1264d11c3 metrics: Add support for histograms.
This will enable us to add e.g. circuit build metrics (#40717).

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-13 11:18:40 -04:00
nonameformee
8f50f490a6 Update 3 files
- /src/feature/dirauth/process_descs.c
- /src/test/test_process_descs.c
- /changes/ticket40760
2023-03-12 11:40:52 +00:00
David Goulet
3fa08dc9a7 Merge branch 'tor-gitlab/mr/697' 2023-03-07 09:49:53 -05:00
Gabriela Moldovan
16c6788fbc metrics: Add a reason label to the HS error metrics.
This adds a `reason` label to the `hs_intro_rejected_intro_req_count` and
`hs_rdv_error_count` metrics introduced in #40755.

Metric look up and intialization is now more a bit more involved. This may be
fine for now, but it will become unwieldy if/when we add more labels (and as
such will need to be refactored).

Also, in the future, we may want to introduce finer grained `reason` labels.
For example, the `invalid_introduce2` label actually covers multiple types of
errors that can happen during the processing of an INTRODUCE2 cell (such as
cell parse errors, replays, decryption errors).

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-03-07 09:46:05 -05:00
David Goulet
85f5318f7f Merge branch 'tor-gitlab/mr/696' 2023-03-07 08:52:34 -05:00
David Goulet
73eab76e6d Merge branch 'maint-0.4.7' 2023-03-07 08:39:48 -05:00
trinity-1686a
0222fc4d71
add new metrics entry for cert expiration 2023-03-06 13:56:39 +01:00
Gabriela Moldovan
db4c4d656a
metrics: Add metrics for rendezvous and introduction request failures.
This introduces a couple of new service side metrics:
* `hs_intro_rejected_intro_req_count`, which counts the number of introduction
  requests rejected by the hidden service
* `hs_rdv_error_count`, which counts the number of rendezvous errors as seen by
  the hidden service (this number includes the number of circuit establishment
  failures, failed retries, end-to-end circuit setup failures)

Closes #40755. This partially addresses #40717.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-02-16 18:54:30 +00:00
Richard Pospesel
c71f31dccc Reworded OpenSSL bug 7712 detection warning to avoid OpenSSL 1.1.1b detection false positive. 2023-02-16 13:27:30 +00:00
Roger Dingledine
36612b9bf8 vote AuthDirMaxServersPerAddr in consensus params
Directory authorities now include their AuthDirMaxServersPerAddr
config option in the consensus parameter section of their vote. Now
external tools can better predict how they will behave.

In particular, the value should make its way to the
https://consensus-health.torproject.org/#consensusparams page.

Once enough dir auths vote this param, they should also compute a
consensus value for it in the consensus document. Nothing uses this
consensus value yet, but we could imagine having dir auths consult it
in the future.

Implements ticket 40753.
2023-02-13 13:54:29 -05:00
David Goulet
482ce87a8d Merge branch 'maint-0.4.7' 2023-02-13 10:16:13 -05:00
Gabriela Moldovan
21b3397f9b
metrics: Decrement hs_intro_established_count on intro circuit close.
Closes #40751.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-02-10 20:31:36 +00:00
Gabriela Moldovan
c98d78c95c
Update find_service documentation.
This updates the docs to stop suggesting `pk` can be NULL, as that doesn't seem
to be the case anymore (`tor_assert(pk)`).

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-02-10 15:27:43 +00:00
Nick Mathewson
43d3a41157 Extend blinding testvec with timeperiod test.
When I copied this to arti, I messed up and thought that the default
time period was 1440 seconds for some weird testing reason. That led
to confusion.

This commit adds a test case that time period 1440 is May 20, 1973:
now arti and c tor match!
2023-02-10 08:11:39 -05:00
Gabriela Moldovan
a9c7cd6b2c
Fix small typo in mainloop.c docs.
The docs should reference `tor_event_new()` rather than `tor_libevent_new()`.

Signed-off-by: Gabriela Moldovan <gabi@torproject.org>
2023-02-07 12:15:12 +00:00
Dimitris Apostolou
c1b940cebf
Fix typos 2023-02-05 00:36:37 +02:00
Roger Dingledine
a4d61c84e3 fix trivial typos 2023-01-30 16:23:24 -05:00
David Goulet
7770c5a2ae Merge branch 'tor-gitlab/mr/686' 2023-01-27 08:47:46 -05:00
David Goulet
fc11b38f2a Merge branch 'maint-0.4.7' 2023-01-26 13:11:22 -05:00
David Goulet
e390a7cdee Merge branch 'tor-gitlab/mr/687' into maint-0.4.7 2023-01-26 13:11:18 -05:00
David Goulet
7b87ecf7e2 Merge branch 'maint-0.4.5' into maint-0.4.7 2023-01-25 14:13:35 -05:00
Micah Elizabeth Scott
a1d3d201ae compress_lzma: New enum values from liblzma 5.3.x
Add new liblzma enums (LZMA_SEEK_NEEDED and LZMA_RET_INTERNAL*)
conditional to the API version they arrived in. The first stable
version of liblzma this affects is 5.4.0

Fixes #40741

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-01-25 14:13:31 -05:00
David Goulet
bff6d7a944 relay: Use the right max queue size value in log
Fixes #40745

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-25 14:10:20 -05:00
Nick Mathewson
a5d8c9fef7 Add a test vector for disaster SRV calculation. 2023-01-25 09:04:07 -05:00
Micah Elizabeth Scott
3d7e3af91e compress_lzma: New enum values from liblzma 5.3.x
Add new liblzma enums (LZMA_SEEK_NEEDED and LZMA_RET_INTERNAL*)
conditional to the API version they arrived in. The first stable
version of liblzma this affects is 5.4.0

Fixes #40741

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>
2023-01-19 13:59:08 -05:00
Nick Mathewson
ee153db5a1 Add more test-vectors for key blinding.
These are verified-as-correct against the current C implementation;
adding them here gives us something to copy into Arti.
2023-01-19 12:54:33 -05:00
David Goulet
d02d2a4338 Fix compiler warnings about unused variables
Fixes #40743

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-19 11:18:02 -05:00
David Goulet
49f10c5130 Merge branch 'tor-gitlab/mr/683' 2023-01-19 10:10:14 -05:00
Nick Mathewson
5629a391c2 Add a test vector for crypto_mac_sha3 2023-01-19 09:09:09 -05:00
Tor CI Release
fa2face3b5 version: Bump version to 0.4.7.13-dev 2023-01-12 12:09:34 -05:00
Tor CI Release
40e39b1e65 version: Bump version to 0.4.5.16-dev 2023-01-12 12:09:10 -05:00
Tor CI Release
89d1db6ad9 version: Bump version to 0.4.7.13 2023-01-12 11:15:53 -05:00
Tor CI Release
5a085a8f0f version: Bump version to 0.4.5.16 2023-01-12 11:15:23 -05:00
David Goulet
d9821bdea5 Merge branch 'maint-0.4.7' 2023-01-12 11:07:40 -05:00
David Goulet
64cebf4e1b Merge branch 'maint-0.4.5' into maint-0.4.7 2023-01-12 11:07:39 -05:00
Tor CI Release
64c0a9fa74 fallbackdir: Update list generated on January 12, 2023 2023-01-12 11:07:33 -05:00
Tor CI Release
049d0818c6 Update geoip files to match ipfire location db, 2023/01/12. 2023-01-12 11:07:23 -05:00
David Goulet
610b791aa6 Merge branch 'maint-0.4.7' 2023-01-12 10:52:35 -05:00
David Goulet
0df4083299 Merge branch 'maint-0.4.5' into maint-0.4.7 2023-01-12 10:52:35 -05:00
David Goulet
7b83e336ec Merge branch 'ticket40730_045_01' into maint-0.4.5 2023-01-12 10:52:31 -05:00
David Goulet
637213fce3 Merge branch 'maint-0.4.7' 2023-01-12 10:49:57 -05:00
David Goulet
f2e9ce72d6 dirauth: Reject 0.4.6.x series at the authority level
Closes #40664

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-12 09:07:52 -05:00
David Goulet
754dbcd6d3 Merge branch 'maint-0.4.7' 2023-01-11 13:14:45 -05:00
Mike Perry
c6ef04e0d0 Coverity CID 1518991: Tighter bounds on consensus param value.
This prevents sign extension overflow in cwnd_became_full().
2023-01-11 17:32:20 +00:00
David Goulet
b38630ca56 Merge branch 'maint-0.4.7' 2023-01-11 09:03:29 -05:00
David Goulet
b9c7825f0e Merge branch 'maint-0.4.5' into maint-0.4.7 2023-01-11 09:03:29 -05:00
David Goulet
85547a9b5b Merge branch 'tor-gitlab/mr/538' into maint-0.4.5 2023-01-11 09:03:26 -05:00
David Goulet
5d6e0b8e13 Merge branch 'maint-0.4.7' 2023-01-10 15:58:28 -05:00
Mike Perry
482cde5931 Do not reset our RTT in slow start.
If a circuit only sends a tiny amount of data such that its cwnd is not
full, it won't increase its cwnd above the minimum. Since slow start circuits
should never hit the minimum otherwise, we can just ignore them for RTT reset
to handle this.
2023-01-10 20:47:11 +00:00
David Goulet
d6cf3ca5c1 Merge branch 'tor-gitlab/mr/678' 2023-01-10 11:57:07 -05:00
David Goulet
c50496036b cc: Rename function to avoid confusion
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-10 11:56:21 -05:00
Mike Perry
5ddd3a9069 Safety fixes to RFC3742 2023-01-10 11:56:21 -05:00
Mike Perry
a9a27ffa3a Reduce size of congestion control next_*_event fields.
Since these are derived from the number of SENDMEs in a cwnd/cc update,
and a cwnd should not exceed ~10k, there's plenty of room in uint16_t
for them, even if the network gets significantly faster.
2023-01-10 11:56:21 -05:00
Mike Perry
f4499bb5e2 Clean up next_cc_event handling. 2023-01-10 11:56:21 -05:00
Mike Perry
48de1a392e Avoid increasing the congestion window if it is not full.
Also provides some stickiness, so that once full, the congestion window is
considered still full for the rest of an update cycle, or the entire
congestion window.

In this way, we avoid increasing the congestion window if it is not fully
utilized, but we can still back off in this case. This substantially reduces
queue use in Shadow.
2023-01-10 11:56:21 -05:00
David Goulet
c420667a2e Merge branch 'tor-gitlab/mr/676' 2023-01-10 11:53:10 -05:00
Mike Perry
8c017e9cff Merge branch 'mr-674-fixup' into main+mr-674-fixup 2023-01-10 16:18:41 +00:00
David Goulet
ce7476cb59 Merge branch 'maint-0.4.7' 2023-01-10 11:15:36 -05:00
David Goulet
35e221688b Merge branch 'tor-gitlab/mr/675' into maint-0.4.7 2023-01-10 11:15:32 -05:00
David Goulet
fd86420d96 cc: Rename function to avoid confusion
Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-10 15:46:03 +00:00
Mike Perry
90e67f34a7 Safety fixes to RFC3742 2023-01-10 15:46:03 +00:00
Mike Perry
b7759403bf Reduce size of congestion control next_*_event fields.
Since these are derived from the number of SENDMEs in a cwnd/cc update,
and a cwnd should not exceed ~10k, there's plenty of room in uint16_t
for them, even if the network gets significantly faster.
2023-01-10 15:46:03 +00:00
Mike Perry
967ae3ab0e Clean up next_cc_event handling. 2023-01-10 15:46:03 +00:00
Mike Perry
7a06763b22 Avoid increasing the congestion window if it is not full.
Also provides some stickiness, so that once full, the congestion window is
considered still full for the rest of an update cycle, or the entire
congestion window.

In this way, we avoid increasing the congestion window if it is not fully
utilized, but we can still back off in this case. This substantially reduces
queue use in Shadow.
2023-01-10 15:46:03 +00:00
David Goulet
4db610d6d9 state: Fix segfault on malformed file
Having no TotalBuildTimes along a positive CircuitBuildAbandonedCount
count lead to a segfault. We check for that condition and then BUG + log
warn if that is the case.

It should never happened in theory but if someone modified their state
file, it can lead to this problem so instead of segfaulting, warn.

Fixes #40437

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-10 09:25:50 -05:00
David Goulet
7e055c383c Merge branch 'maint-0.4.7' 2023-01-10 09:12:20 -05:00
David Goulet
d456885dac shellcheck: Fix new warnings
Nothing important, mostly false positive except one case.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-10 09:11:58 -05:00
David Goulet
726e9ec0a3 sandbox: Allow my-consensus-* files for an authority
Fixes #40729

Signed-off-by: David Goulet <dgoulet@torproject.org>
2023-01-10 09:02:13 -05:00
David Goulet
1e6d839ce9 Merge branch 'tor-gitlab/mr/668' 2022-12-22 14:03:40 -05:00
Georg Koppen
5ba2bf2551
Add back comments about Stable/familiar decay
Closes: #40734.
2022-12-21 10:43:26 +00:00
David Goulet
923463a1e6 Fix duplicate code after tor-gitlab/mr/671 forward merge
Signed-off-by: David Goulet <dgoulet@torproject.org>
2022-12-20 09:11:30 -05:00
David Goulet
713efae94b Merge branch 'maint-0.4.7' 2022-12-20 09:09:47 -05:00
Alex Xu (Hello71)
1d9166c8c9 Enable IP_BIND_ADDRESS_NO_PORT if supported
Signed-off-by: David Goulet <dgoulet@torproject.org>
2022-12-20 09:09:33 -05:00
Dimitris Apostolou
1da9dec348
Fix typos 2022-12-16 08:09:04 +02:00
David Goulet
cfdc9f9d29 circ: Add function to learn if queue is full
Related to #40731

Signed-off-by: David Goulet <dgoulet@torproject.org>
2022-12-14 10:19:14 -05:00
David Goulet
03ddeb9539 Merge branch 'tor-gitlab/mr/660' 2022-12-12 15:10:06 -05:00
David Goulet
da48104c99 Merge branch 'tor-gitlab/mr/667' into maint-0.4.7 2022-12-12 15:07:00 -05:00
David Goulet
58f41c379d Merge branch 'maint-0.4.7' 2022-12-12 13:15:08 -05:00
David Goulet
a282145b36 socks: Make SafeSocks refuse SOCKS4 and accept SOCKS4a
The logic was inverted. Introduced in commit
9155e08450.

This was reported through our bug bounty program on H1. It fixes the
TROVE-2022-002.

Fixes #40730

Signed-off-by: David Goulet <dgoulet@torproject.org>
2022-12-12 10:02:07 -05:00
qontinuum
7dd55c29f9
Replace socket_failed_from_resource_exhaustion() by socket_failed_from_fd_exhaustion() 2022-12-11 10:14:23 +01:00