KIST works by computing how much should be allowed to write to the kernel for
a given socket, and then it writes that amount to the outbuf.
The problem is that it could be possible that the outbuf already has lots of
data in it from a previous scheduling round (because the kernel is full/busy
and Tor was not able to flush the outbuf yet). KIST ignores that the outbuf
has been filling (is above its "highwater") and writes more anyway. The end
result is that the outbuf length would exceed INT_MAX, hence causing an
assertion error and a corresponding "Bug()" message to get printed to the
logs.
This commit makes it for KIST to take into account the outbuf length when
computing the available space.
Bug found and patch by Rob Jansen.
Closes#29168. TROVE-2019-001.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Fixes bug 29530, where the LOG_ERR messages were occurring when
we had no configured network, and so we were failing the unit tests
because of the recently-merged #28668.
Bug not in any released Tor.
This test fails in some environments; since the code isn't used in
0.4.0, let's disable it for now.
Band-aid solution for #29534; bug not in any released Tor.
malloc_options needs to be declared extern (and declaring it extern
means we need to initialize it separately)
Fixes bug 29145; bugfix on 0.2.9.3-alpha
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
Also:
* delete some obsolete code that was #if 0
* improve cleanup on failure
* make the dir_format tests more consistent with each other
* construct the descriptors using smartlist chunks
This refactor is incomplete, because removing the remaining duplicate
code would be time-consuming.
Part of 29017 and 29018.
Remove router_update_info_send_unencrypted(), and move its code into the
relevant functions.
Then, re-use an options pointer.
Preparation for testing 29017 and 20918.
Remove some tiny static functions called by router_build_fresh_descriptor(),
and move their code into more relevant functions.
Then, give router_update_{router,extra}info_descriptor_body identical layouts.
Preparation for testing 29017 and 20918.
Make sure that these static functions aren't passed NULL.
If they are, log a BUG() warning, and return an error.
Preparation for testing 29017 and 20918.
Split the body of router_build_fresh_descriptor() into static functions,
by inserting function prologues and epilogues between existing sections.
Write a new body for router_build_fresh_descriptor() that calls the new
static functions.
Initial refactor with no changes to the body of the old
router_build_fresh_descriptor(), except for the split.
Preparation for testing 29017 and 20918.
When ExtraInfoStatistics is 0, stop including bandwidth usage statistics,
GeoIPFile hashes, ServerTransportPlugin lines, and bridge statistics
by country in extra-info documents.
Fixes bug 29018; bugfix on 0.2.4.1-alpha (and earlier versions).
This module is currently implemented to use the same technique as
libottery (later used by the bsds' arc4random replacement), using
AES-CTR-256 as its underlying stream cipher. It's backtracking-
resistant immediately after each call, and prediction-resistant
after a while.
Here's how it works:
We generate psuedorandom bytes using AES-CTR-256. We generate BUFLEN bytes
at a time. When we do this, we keep the first SEED_LEN bytes as the key
and the IV for our next invocation of AES_CTR, and yield the remaining
BUFLEN - SEED_LEN bytes to the user as they invoke the PRNG. As we yield
bytes to the user, we clear them from the buffer.
Every RESEED_AFTER times we refill the buffer, we mix in an additional
SEED_LEN bytes from our strong PRNG into the seed.
If the user ever asks for a huge number of bytes at once, we pull SEED_LEN
bytes from the PRNG and use them with our stream cipher to fill the user's
request.
Because the test is adding entries to the "rend_cache" directly, the
rend_cache_increment_allocation() was never called which made the
rend_cache_clean() call trigger that underflow warning:
rend_cache/clean: [forking] Nov 29 09:55:04.024 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.0.0-alpha-dev 2240fe63feb9a8cf)
The test is still good and valid.
Fixes#28660
Signed-off-by: David Goulet <dgoulet@torproject.org>
Using an anonymous mmap() is a good way to get pages that we can set
kernel-level flags on, like minherit() or madvise() or mlock().
We're going to use that so that we can make uninheritable locked
pages to store PRNG data.
Rewrite service_intro_point_new() to take a node_t. Since
node_get_link_specifier_smartlist() supports IPv6 link specifiers,
this refactor adds IPv6 addresses to onion service descriptors.
Part of 23576, implements 26992.
Because the test is adding entries to the "rend_cache" directly, the
rend_cache_increment_allocation() was never called which made the
rend_cache_clean() call trigger that underflow warning:
rend_cache/clean: [forking] Nov 29 09:55:04.024 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.0.0-alpha-dev 2240fe63feb9a8cf)
The test is still good and valid.
Fixes#28660
Signed-off-by: David Goulet <dgoulet@torproject.org>
Also, when we log about a failure from base32_decode(), we now
say that the length is wrong or that the characters were invalid:
previously we would just say that there were invalid characters.
Follow-up on 28913 work.
The code checked for sysctl being available and HW_PHYSMEM being
defined, but HW_USERMEM was actually being used with sysctl instead
of HW_PHYSMEM.
The case for OpenBSD, etc. use HW_PHYSMEM64 (which is obviously a
64-bit variant of HW_PHYSMEM) and the case for OSX uses HW_MEMSIZE
(which appears to be a 64-bit variant of HW_PHYSMEM).
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
We log these messages at INFO level, except when we are reading a
private key from a file, in which case we log at WARN.
This fixes a regression from when we re-wrote our PEM code to be
generic between nss and openssl.
Fixes bug 29042, bugfix on 0.3.5.1-alpha.
When cleaning up after an error in process_unix_exec, the stdin
pipe was being double closed instead of closing both the stdin
and stdout pipes. This occurred in two places.
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
NOTE: This commit breaks the build, because there was a mistake in an
earlier change of exactly the sort that this is meant to detect! I'm
leaving it broken for illustration.
Test exactly what the geometric sampler returns, because that's what
the downstream callers of it are going to use.
While here, also assert that the geometric sampler returns a positive
integer. (Our geometric distribution is the one suported on {1, 2,
3, ...} that returns the number of trials before the first success,
not the one supported on {0, 1, 2, ...} that returns the number of
failures before the first success.)
In file included from ./src/core/or/or_circuit_st.h:12:0,
from src/core/or/circuitlist.c:112:
./src/core/or/circuit_st.h:15:39: error: redefinition of typedef ‘circpad_machine_spec_t’
./src/core/or/circuitpadding.h:572:3: note: previous declaration of ‘circpad_machine_spec_t’ was here
./src/core/or/circuit_st.h:16:40: error: redefinition of typedef ‘circpad_machine_state_t’
./src/core/or/circuitpadding.h:517:3: note: previous declaration of ‘circpad_machine_state_t’ was here
In file included from src/core/or/connection_edge.c:70:0:
./src/core/or/circuitpadding.h:16:26: error: redefinition of typedef ‘circuit_t’
./src/core/or/or.h:930:26: note: previous declaration of ‘circuit_t’ was here
./src/core/or/circuitpadding.h:17:33: error: redefinition of typedef ‘origin_circuit_t’
./src/core/or/or.h:931:33: note: previous declaration of ‘origin_circuit_t’ was here
./src/core/or/circuitpadding.h:18:23: error: redefinition of typedef ‘cell_t’
./src/core/or/or.h:628:23: note: previous declaration of ‘cell_t’ was here
typedef doesn't work for forward declarations, but plain struct
without a typedef wrapper does (and unlike the _t type aliases makes
it clearer for everyone whether you're talking about the struct or
the pointer).
Stop logging "Tried to establish rendezvous on non-OR circuit..." as
a warning. Instead, log it as a protocol warning, because there is
nothing that relay operators can do to fix it.
Fixes bug 29029; bugfix on 0.2.5.7-rc.
Prior to this commit, the testsuite was failing on OpenBSD. After
this commit the testsuite runs fine on OpenBSD.
It was previously decided to test for the OpenBSD macro (rather than
__OpenBSD__, etc.) because OpenBSD forks seem to have the former
macro defined. sys/param.h must be included for the OpenBSD macro
definition; however, many files tested for the OpenBSD macro without
having this header included.
This commit includes sys/param.h in the files where the OpenBSD macro
is used (and sys/param.h is not already included), and it also
changes some instances of the __OpenBSD__ macro to OpenBSD.
See commit 27df23abb6 which changed
everything to use OpenBSD instead of __OpenBSD__ or OPENBSD. See
also tickets #6982 and #20980 (the latter ticket is where it was
decided to use the OpenBSD macro).
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
In get_local_listener used by tor_ersatz_socketpair, the address
family used when binding the IPv6 socket was AF_INET instead of
AF_INET6.
Fixes bug 28995; bugfix on 0.3.5.1-alpha.
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
Reported on tor-dev by Gisle Vanem. Bug not in any released Tor
(The suggested patch used _MSC_VER, but that's not how we do stuff
with autoconf. With autoconf, you detect the feature you want,
rather than trying to list all the systems that do or do not have
it.)
In theory it would be better to detect this bug in advance, but this
approach is much simpler, and therefore safer to backport.
This closes tor issue 28973.
This project introduces the prob_distr.c subsystem which implements all the
probability distributions that WTF-PAD needs. It also adds unittests for all of
them.
Code and tests courtesy of Riastradh.
Co-authored-by: Taylor R Campbell <campbell+tor@mumble.net>
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
Hope is this will make it easier to test on the live tor network.
Does not need to be merged if we don't want to, but will come in handy
for researchers.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
This implements all of the event handling, state machines, and padding
decisions for circuit padding.
I recommend reviewing this after you look at the call-in points into it from
the rest of Tor.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
These callbacks allow the padding state machines to react to various types of
sent and received relay cells.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
These event callbacks allow circuit padding to decide when to attempt to
launch and negotiate new padding machines, and when to tear old ones down.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
This is a good code review start point, to get an overview of the interfaces
and types used in circuit padding.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
We need this for padding negotiation so that we can have later machine
revisions supercede earlier ones.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
signed_descriptor_digest has a length of DIGEST_LEN but the memset
used to fill it used DIGEST256_LEN.
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
Redefine the set of bootstrap phases to allow display of finer-grained
progress in the early connection stages of connecting to a relay.
This includes adding intermediate phases for proxy and PT connections.
Also add a separate new phase to indicate obtaining enough directory
info to build circuits so we can report that independently of actually
initiating an ORCONN to build the first application circuit.
Previously, we would claim to be connecting to a relay when we had
merely finished obtaining directory info.
Part of ticket 27167.
Replace a few invocations of control_event_bootstrap() with calls from
the bootstrap tracker subsystem. This mostly leaves behavior
unchanged. The actual behavior changes come in the next commit.
Part of ticket 27167.
Linked connections aren't woken up by libevent due to I/O but rather
artificially so we can, by chunks, empty the spooled object(s).
Commit 5719dfb48f (in 0.3.4.1-alpha) made it
that the schedule_active_linked_connections_event would be only called once at
startup but this is wrong because then we would never go through again the
active linked connections.
Fortunately, everytime a new linked connection is created, the event is
activated and thus we would go through the active list again. On a busy relay,
this issue is mitigated by that but on a slower relays or bridge, a connection
could get stuck for a while until a new directory information request would
show up.
Fixes#28717, #28912
Add a tracker for bootstrap progress, tracking events related to
origin circuit and ORCONN states. This uses the ocirc_event and
orconn_event publish-subscribe subsystems.
Part of ticket 27167.
Add a publish-subscribe subsystem to publish messages about changes to
origin circuits.
Functions in circuitbuild.c and circuitlist.c publish messages to this
subsystem.
Move circuit event constants out of control.h so that subscribers
don't have to include all of control.h to take actions based on
messages they receive.
Part of ticket 27167.
Add a publish-subscribe subsystem to publish messages about changes to
OR connections.
connection_or_change_state() in connection_or.c and
control_event_or_conn_event() in control.c publish messages to this
subsystem via helper functions.
Move state constants from connection_or.h to orconn_state.h so that
subscribers don't have to include all of connection_or.h to take
actions based on changes in OR connection state. Move event constants
from control.h for similar reasons.
Part of ticket 27167.
connection_or_change_state() saved an old_state to pass to
channel_tls_handle_state_change_on_orconn(), which promptly cast it to
void. Remove this unused variable and parameter.
This patch changes the CancelIoEx() example code to use CancelIo(),
which is available for older versions of Windows too. I still think the
kernel handles this nicely by sending broken pipes if either side
closes the pipe while I/O operations are pending.
See: https://bugs.torproject.org/28179
Handle `ERROR_BROKEN_PIPE` from ReadFileEx() and WriteFileEx() in
process_win32_stdin_write_done() and
process_win32_handle_read_completion() instead of in the early handler.
This most importantmly makes sure that `reached_eof` is set to true when
these errors appears.
See: https://bugs.torproject.org/28179
This patch adds some missing calls to set `reached_eof` of our handles
when various error conditions happens or when we close our handle (which
happens at `process_terminate()`.
See: https://bugs.torproject.org/28179
This patch adds some additional error checking after calls to
ReadFileEx() and WriteFileEx(). I have not managed to get this code to
reach the branch where `error_code` is NOT `ERROR_SUCCESS`, but MSDN
says one should check for this condition so we do so just to be safe.
See: https://bugs.torproject.org/28179
This patch makes us delay checking for whether we have an exit code
value (via GetExitCodeProcess()) until both stdout and stderr have been
closed by the operating system either by the process itself or by
process cleanup after termination.
See: https://bugs.torproject.org/28179
This patch adds support for the new STATUS message that PT's can emit
from their standard out. The STATUS message uses the `config_line_t` K/V
format that was recently added in Tor.
See: https://bugs.torproject.org/28846
This patch changes the LOG pluggable transport message to use the recent
K/V parser that landed in Tor. This allows PT's to specify the log
severity level as well as the message. A mapping between the PT log
severity levels and Tor's log serverity level is provided.
See: https://bugs.torproject.org/28846
Previously we had decoded the asn.1 to get a public key, and then
discarded the asn.1 so that we had to re-encode the key to store it
in the onion_pkey field of a microdesc_t or routerinfo_t.
Now we can just do a tor_memdup() instead, which should be loads
faster.
Previously, we would decode the PEM wrapper for keys twice: once in
get_next_token, and once later in PEM decode. Now we just do all of
the wrapper and base64 stuff in get_next_token, and store the
base64-decoded part in the token object for keys and non-keys alike.
This change should speed up parsing slightly by letting us skip a
bunch of stuff in crypto_pk_read_*from_string(), including the tag
detection parts of pem_decode(), and an extra key allocation and
deallocation pair.
Retaining the base64-decoded part in the token object will allow us
to speed up our microdesc parsing, since it is the asn1 portion that
we actually want to retain.
This patch makes sure that we terminate the event loop from the event
loop timer instead of directly in the process' exit handler. This allows
us to run the event loop an additional time to ensure that the SleepEx()
call on Windows is called and the data from stdout/stderr is delivered
to us.
Additionally we ensure that we don't try to read or write data from a
Unix process that have been terminated in the main loop, since its file
descriptors are closed at that time.
See: https://bugs.torproject.org/28179
This patch changes the API of the Windows backend of the Process
subsystem to allow the dormant interface to disable the Process event
timer.
See: https://bugs.torproject.org/28179
This patch changes our EVENT_TRANSPORT_LOG event to be EVENT_PT_LOG. The
new message includes the path to the PT executable instead of the
transport name, since one PT binary can include multiple transport they
sometimes might need to log messages that are not specific to a given
transport.
See: https://bugs.torproject.org/28179
This patch changes our process_t's exit_callback to return a boolean
value. If the returned value is true, the process subsystem will call
process_free() on the given process_t.
See: https://bugs.torproject.org/28179
This patch changes the slow process_t tests to use
run_main_loop_until_done() instead of do_main_loop() since
do_main_loop() initializes a lot of subsystem callbacks that we don't
need to run in our tests.
See: https://bugs.torproject.org/28179
This patch disables fork()'ing of the slow process tests. This fixes the
tests on the MacOS and other kqueue() based platforms.
Without this patch the main loop exits eearly with EBADF as error.
See: https://bugs.torproject.org/28179
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.
We also delete the now empty subprocess module files.
See: https://bugs.torproject.org/28179
This patch adds test cases for process_t which uses Tor's main loop.
This allows us to test that the callbacks are actually invoked by the
main loop when we expect them.
See: https://bugs.torproject.org/28179
This patch adds support for the "LOG" protocol message from a pluggable
transport. This allows pluggable transport developers to relay log
messages from their binary to Tor, which will both emit them as log
messages from the Tor process itself, but also pass them on via the
control port.
See: https://bugs.torproject.org/28180
See: https://bugs.torproject.org/28181
See: https://bugs.torproject.org/28182
This patch makes the managed proxy subsystem use the process_t data
structure such that we can get events from the PT process while Tor is
running and not just when the PT process is being configured.
See: https://bugs.torproject.org/28179
This patch adds a new function that allows us to reset the environment
of a given process_t with a list of key/value pairs.
See: https://bugs.torproject.org/28179
This patch makes sure that we call process_notify_event_exit() after we
have done any modifications we need to do to the state of a process_t.
This allows application developers to call process_free() in the
exit_callback of the process.
See: https://bugs.torproject.org/28179
This patch adds support for getting the unique process identifier from a
given process_t. This patch implements both support for both the Unix
and Microsoft Windows backend.
See: https://bugs.torproject.org/28179
This patch adds support for Microsoft Windows in the Process subsystem.
Libevent does not support mixing different types of handles (sockets,
named pipes, etc.) on Windows in its core event loop code. This have
historically meant that Tor have avoided attaching any non-networking
handles to the event loop. This patch uses a slightly different approach
to roughly support the same features for the Process subsystem as we do
with the Unix backend.
In this patch we use Windows Extended I/O functions (ReadFileEx() and
WriteFileEx()) which executes asynchronously in the background and
executes a completion routine when the scheduled read or write operation
have completed. This is much different from the Unix backend where the
operating system signals to us whenever a file descriptor is "ready" to
either being read from or written to.
To make the Windows operating system execute the completion routines of
ReadFileEx() and WriteFileEx() we must get the Tor process into what
Microsoft calls an "alertable" state. To do this we execute SleepEx()
with a zero millisecond sleep time from a main loop timer that ticks
once a second. This moves the process into the "alertable" state and
when we return from the zero millisecond timeout all the outstanding I/O
completion routines will be called and we can schedule the next reads
and writes.
The timer loop is also responsible for detecting whether our child
processes have terminated since the last timer tick.
See: https://bugs.torproject.org/28179
This patch adds the Unix backend for the Process subsystem. The Unix
backend attaches file descriptors from the child process's standard in,
out and error to Tor's libevent based main loop using traditional Unix
pipes. We use the already available `waitpid` module to get events
whenever the child process terminates.
See: https://bugs.torproject.org/28179
This patch adds a new Process subsystem for running external programs in
the background of Tor. The design is focused around a new type named
`process_t` which have an API that allows the developer to easily write
code that interacts with the given child process. These interactions
includes:
- Easy API for writing output to the child process's standard input
handle.
- Receive callbacks whenever the child has output on either its standard
output or standard error handles.
- Receive callback when the child process terminates.
We also support two different "protocols" for handling output from the
child process. The default protocol is the "line" protocol where the
process output callbacks will be invoked only when there is complete
lines (either "\r\n" or "\n" terminated). We also support the "raw"
protocol where the read callbacks will get whatever the operating system
delivered to us in a single read operation.
This patch does not include any operating system backends, but the Unix
and Windows backends will be included in separate commits.
See: https://bugs.torproject.org/28179
The strcmp_len() function was somewhat misconceived, since we're
only using it to test whether a length+extent string is equal to a
NUL-terminated string or not. By simplifying it and making it
inlined, we should be able to make it a little faster.
(It *does* show up in profiles.)
Closes ticket 28856.
The old implementation did some funky out-of-order lexing, and
tended to parse every port twice if the %d-%d pattern didn't match.
Closes ticket 28853.
I believe we originally added this for "just in case" safety, but it
isn't actually needed -- we never copy uninitialized stack here.
What's more, this one memset is showing up on our startup profiles,
so we ought to remove it.
Closes ticket 28852.
The point of this function is to make sure that the ed25519-based
implementation of curve25519_basepoint() actually works when we
start tor, and use the regular fallback implementation if it
doesn't. But it accounts for 9% of our startup time in the case
when we have directory information, and I think it's safe to make
the test shorter. After all, it has yet to find any actual bugs in
curved25519_scalarmult_basepoint_donna() on any platforms.
Closes ticket 28838.
When the clock jumps, and we have a record of last user activity,
adjust that record. This way if I'm inactive for 10 minutes and
then the laptop is sleeping for an hour, I'll still count as having
been inactive for 10 minutes.
Previously, we treat every jump as if it were activity, which is
ridiculous, and would prevent a Tor instance with a jumpy clock from
ever going dormant.
encoding and decoding.
There are bunches of places where we don't want to invest in a full
fuzzer, but we would like to make sure that some string operation
can handle all its possible inputs. This fuzzer uses the first byte
of its input to decide what to do with the rest of the input. Right
now, all the possibilities are decoding a string, and seeing whether
it is decodeable. If it is, we try to re-encode it and do the whole
thing again, to make sure we get the same result.
This turned up a lot of bugs in the key-value parser, and I think it
will help in other cases too.
Closes ticket 28808.
Add the bootstrap tag name to the log messages, so people
troubleshooting connection problems can look up a symbol instead of a
number. Closes ticket 28731.
Merge Phoul's two lists into teor's list.
Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in
January 2018 (of which ~115 were still functional), with a list of
157 fallbacks (92 new, 65 existing, 85 removed) generated in
December 2018.
Closes ticket 24803.
Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in
January 2018 (of which ~115 were still functional), with a list of
148 fallbacks (89 new, 59 existing, 91 removed) generated in
December 2018.
Closes ticket 24803.
When retrying all SOCKS connection because new directory information just
arrived, do not BUG() if a connection in state AP_CONN_STATE_RENDDESC_WAIT is
found to have a usable descriptor.
There is a rare case when this can happen as detailed in #28669 so the right
thing to do is put that connection back in circuit wait state so the
descriptor can be retried.
Fixes#28669
Signed-off-by: David Goulet <dgoulet@torproject.org>
This helper function marks an entry connection as pending for a circuit and
changes its state to AP_CONN_STATE_CIRCUIT_WAIT. The timestamps are set to
now() so it can be considered as new.
No behaviour change, this helper function will be used in next commit.
Part of #28669
Signed-off-by: David Goulet <dgoulet@torproject.org>
Use the helper function connection_ap_mark_as_waiting_for_renddesc()
introduced in previous commit everywhere in the code where an AP connection
state is transitionned to AP_CONN_STATE_RENDDESC_WAIT.
Part of #28669
Signed-off-by: David Goulet <dgoulet@torproject.org>
The transition for a connection to either become or go back in
AP_CONN_STATE_RENDDESC_WAIT state must make sure that the entry connection is
_not_ in the waiting for circuit list.
This commit implements the helper function
connection_ap_mark_as_waiting_for_renddesc() that removes the entry connection
from the pending list and then change its state. This code pattern is used in
many places in the code where next commit will remove this code duplication to
use this new helper function.
Part of #28669
Signed-off-by: David Goulet <dgoulet@torproject.org>
Rather than initializing the "Dormant" status to "off" and the "last
activity" count to "now", initialize them based on our state file:
stay dormant if we were dormant, or remember the amount of time
we've spent inactive.
Additionally, use it to test that is_staledesc is set correctly.
Eventually we'll want to test all the other flags, but I'm aiming
for only adding coverage on the changed code here.
Because the test is adding entries to the "rend_cache" directly, the
rend_cache_increment_allocation() was never called which made the
rend_cache_clean() call trigger that underflow warning:
rend_cache/clean: [forking] Nov 29 09:55:04.024 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.0.0-alpha-dev 2240fe63feb9a8cf)
The test is still good and valid.
Fixes#28660
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch adds two new functions: buf_flush_to_pipe() and
buf_read_from_pipe(), which makes use of our new buf_flush_to_fd() and
buf_read_from_fd() functions.
See: https://bugs.torproject.org/28179
This patch refactors buf_read_from_socket() into buf_read_from_fd(), and
creates a specialized function for buf_read_from_socket(), which uses
buf_read_from_fd().
See: https://bugs.torproject.org/28179
This patch refactors buf_flush_to_socket() into buf_flush_to_fd() and
creates a specialization function for buf_flush_to_socket() that makes
use of buf_flush_to_fd().
See: https://bugs.torproject.org/28179
The DormantClientTimeout option controls how long Tor will wait before
going dormant. It also provides a way to disable the feature by setting
DormantClientTimeout to e.g. "50 years".
The DormantTimeoutDisabledByIdleStreams option controls whether open but
inactive streams count as "client activity". To implement it, I had to
make it so that reading or writing on a client stream *always* counts as
activity.
Closes ticket 28429.
Specifically, if the consensus is older than the (estimted or
measured) release date for this version of tor, we assume that the
required versions may have changed in between that consensus and
this release.
Implements ticket 27735 and proposal 297.
This patch has routers use the same canonicalization logic as
authorities when encoding their family lists. Additionally, they
now warn if any router in their list is given by nickname, since
that's error-prone.
This patch also adds some long-overdue tests for family formatting.
Prop298 says that family entries should be formatted with
$hexids in uppercase, nicknames in lower case, $hexid~names
truncated, and everything sorted lexically. These changes implement
that ordering for nodefamily.c.
We don't _strictly speaking_ need to nodefamily.c formatting use
this for prop298 microdesc generation, but it seems silly to have
two separate canonicalization algorithms.
After we clear the protover map for getting full, we need to
re-create it, since we are about to use it.
This is a bugfix for bug 28558. It is a bugfix for the code from
ticket 27225, which is not in any released Tor. Found by Google
OSS-Fuzz, as issue 11475.
To succesful compile tor-print-ed-signing-cert.exe on Windows we
sometimes need to include the @TOR_LIB_GDI@ library.
See: https://bugs.torproject.org/28485
This representation is meant to save memory in microdescriptors --
we can't use it in routerinfo_t yet, since those families need to be
encoded losslessly for directory voting to work.
This representation saves memory in three ways:
1. It uses only one allocation per family. (The old way used a
smartlist (2 allocs) plus one strdup per entry.)
2. It stores identity digests in binary, not hex.
3. It keeps families in a canonical format, memoizes, and
reference-counts them.
Part of #27359.
This event makes us become dormant if we have seen no activity in a
long time.
Note that being any kind of a server, or running an onion service,
always counts as being active.
Note that right now, just having an open stream that Tor
did not open on its own (for a directory request) counts as "being
active", so if you have an idle ssh connection, that will keep Tor
from becoming dormant.
Many of the features here should become configurable; I'd like
feedback on which.
This is part of 28422, so we don't have to call
consider_hibernation() once per second when we're dormant.
This commit does not remove delayed shutdown from hibernate.c: it
uses it as a backup shutdown mechanism, in case the regular shutdown
timer mechanism fails for some reason.
The previous "ALL" role was the OR of a bunch of other roles,
which is a mistake: it's better if "ALL" means "all".
The "NET_PARTICIPANT" role refers to the anything that is actively
building circuits, downloading directory information, and
participating in the Tor network. For now, it is set to
!net_is_disabled(), but we're going to use it to implement a new
"extra dormant mode".
Closes ticket 28336.
Correctly identify Windows 8.1, Windows 10, and Windows Server 2008
and later from their NT versions.
On recent Windows versions, the GetVersionEx() function may report
an earlier Windows version than the running OS. To avoid user
confusion, add "[or later]" to Tor's version string on affected
versions of Windows.
Remove Windows versions that were never supported by the
GetVersionEx() function.
Stop duplicating the latest Windows version in get_uname().
Fixes bug 28096; bugfix on 0.2.2.34; reported by Keifer Bly.
In conn_close_if_marked(), we can decide to keep a connection open that still
has data to flush on the wire if it is being rate limited on the write side.
However, in this process, we were also looking at the read() side which can
still have token in its bucket and thus not stop the reading. This lead to a
BUG() introduced in 0.3.4.1-alpha that was expecting the read side to be
closed due to the rate limit but which only applies on the write side.
This commit removes any bandwidth check on the read side and simply stop the
read side on the connection regardless of the bucket state. If we keep the
connection open to flush it out before close, we should not read anything.
Fixes#27750
Signed-off-by: David Goulet <dgoulet@torproject.org>
Apparently some freebsd compilers can't tell that 'c' will never
be used uninitialized.
Fixes bug 28413; bugfix on 0.2.9.3-alpha when we added support for
longer AES keys to this function.
We don't use this syscall, but openssl apparently does.
(This syscall puts a socket into a half-closed state. Don't worry:
It doesn't shut down the system or anything.)
Fixes bug 28183; bugfix on 0.2.5.1-alpha where the sandbox was
introduced.
Apparently, even though the manpage says it returns an int, it
can return a long instead and cause a warning.
Bug not in any released Tor. Part of #28399
Our tests showed that this function is responsible for a huge number
of our malloc/free() calls. It's a prime candidate for being
memoized.
Closes ticket 27225.
Resume refusing to start with relative file paths and RunAsDaemon
set (regression from the fix for bug 22731).
Fixes bug 28298; bugfix on 0.3.3.1-alpha.
If tor terminates due to SIGNAL HALT before test_rebind.py calls
tor_process.terminate(), an OSError 3 (no such process) is thrown.
Fixes part of bug 27968 on 0.3.5.1-alpha.
Remember, you can't check to see if there are N bytes left in a
buffer by doing (buf + N < end), since the buf + N computation might
take you off the end of the buffer and result in undefined behavior.
Fixes 28202; bugfix on 0.2.0.3-alpha.
It is not enough to look at protover for v3 rendezvous support but also we
need to make sure that the curve25519 onion key is present or in other words
that the descriptor has been fetched and does contain it.
Fixes#27797.
Signed-off-by: David Goulet <dgoulet@torproject.org>
With the new refresh_service_descriptor() function we had both
refresh_service_descriptor() and update_service_descriptor() which is basically
the same thing.
This commit renames update_service_descriptor() to
update_service_descriptor_intro_points() to make it clear it's not a generic
refresh and it's only about intro points.
Commit changes no code.
Treat backtrace test failures as expected on NetBSD, OpenBSD, and
macOS/Darwin, until we solve bug 17808.
(FreeBSD failures have been treated as expected since 18204 in 0.2.8.)
Fixes bug 27948; bugfix on 0.2.5.2-alpha.
Before this commit, we would create the descriptor signing key certificate
when first building the descriptor.
In some extreme cases, it lead to the expiry of the certificate which triggers
a BUG() when encoding the descriptor before uploading.
Ticket #27838 details a possible scenario in which this can happen. It is an
edge case where tor losts internet connectivity, notices it and closes all
circuits. When it came back up, the HS subsystem noticed that it had no
introduction circuits, created them and tried to upload the descriptor.
However, in the meantime, if tor did lack a live consensus because it is
currently seeking to download one, we would consider that we don't need to
rotate the descriptors leading to using the expired signing key certificate.
That being said, this commit does a bit more to make this process cleaner.
There are a series of things that we need to "refresh" before uploading a
descriptor: signing key cert, intro points and revision counter.
A refresh function is added to deal with all mutable descriptor fields. It in
turn simplified a bit the code surrounding the creation of the plaintext data.
We keep creating the cert when building the descriptor in order to accomodate
the unit tests. However, it is replaced every single time the descriptor is
uploaded.
Fixes#27838
Signed-off-by: David Goulet <dgoulet@torproject.org>
When storing a descriptor in the client cache, if we are about to replace an
existing descriptor, make sure to close every introduction circuits of the old
descriptor so we don't have leftovers lying around.
Ticket 27471 describes a situation where tor is sending an INTRODUCE1 cell on
an introduction circuit for which it doesn't have a matching intro point
object (taken from the descriptor).
The main theory is that, after a new descriptor showed up, the introduction
points changed which led to selecting an introduction circuit not used by the
service anymore thus for which we are unable to find the corresponding
introduction point within the descriptor we just fetched.
Closes#27471.
Signed-off-by: David Goulet <dgoulet@torproject.org>
It won't be used if there are no authorized client configured. We do that so
we can easily support the addition of a client with a HUP signal which allow
us to avoid more complex code path to generate that cookie if we have at least
one client auth and we had none before.
Fixes#27995
Signed-off-by: David Goulet <dgoulet@torproject.org>
Both client and service had their own code for this. Consolidate into one
place so we avoid duplication.
Closes#27549
Signed-off-by: David Goulet <dgoulet@torproject.org>
Occasionally, key pinning doesn't catch a relay that shares an ed25519
ID with another relay. Log the identity fingerprints and the shared
ed25519 ID when this happens, instead of making a BUG() warning.
Fixes bug 27800; bugfix on 0.3.2.1-alpha.
Commit 488e2b00bf introduced an issue, most
likely introduced by a bad copy paste, that made us stop reading on the
connection if our write bandwidth limit was reached.
The problem is that because "read_blocked_on_bw" was never set, the connection
was never reenabled for reading.
This is most likely the cause of #27813 where bytes were accumulating in the
kernel TCP bufers because tor was not doing reads. Only relays with
RelayBandwidthRate would suffer from this but affecting all relays connecting
to them. And using that tor option is recommended and best practice so many
many relays have it enabled.
Fixes#28089.
