Two new values in each direction. DELIVERED counts valid end-to-end circuit
data that is accepted by our end and OVERHEAD counts the slack unused data in
each of the relay command cells for those accepted cells.
Control port changes are in the next commit.
We still do this time update here, since we do it from all
callbacks, but it is no longer a reason to keep the once-per-second
callback enabled.
Closes ticket 26009.
Since we're going to be disabling the second-elapsed callback, we're
going to sometimes have long periods when no events file, and so the
current second is not updated. Handle that by having a better means
to detect "clock jumps" as opposed to "being idle for a while".
Tolerate far more of the latter.
Part of #26009.
The any_client_port_set() returns true if the ControlPort is set which is
wrong because we can have that port open but still not behave as a tor client
(like many relays for instance).
Fixes#26062
Signed-off-by: David Goulet <dgoulet@torproject.org>
The options_any_client_port_set() returns true if the ControlPort is set which
is wrong because we can have that port open but still not behave as a tor
client (like many relays for instance).
Fixes#26062
Signed-off-by: David Goulet <dgoulet@torproject.org>
This functions is now used outside of networkstatus.c and makes more sense to
be in config.c.
It is also renamed to options_any_client_port_set() for the config.c
namespace.
No code behavior change.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Now it has a function that can tell the rest of Tor whether any
once-a-second controller item should fire, and a function to fire
all the once-a-second events.
Remove v3 optimization which made Tor not detect disabling services.
This optimization is not so needed because we only call that function after HUP
anyway.
Fixes bug #25761.
During service configuration, rend_service_prune_list_impl_() sets
rend_service_staging_list to NULL, which blocked pruning after a HUP.
This patch initializes rend_service_staging_list when needed, so that HUP can
detect disabled onion services.
Fixes bug #25761.
Previously, an authority with a clock more than 60 seconds ahead could
cause a client with a correct clock to warn that the client's clock
was behind. Now the clocks of a majority of directory authorities
have to be ahead of the client before this warning will occur.
Relax the early-consensus check so that a client's clock must be 60
seconds behind the earliest time that a given sufficiently-signed
consensus could possibly be available.
Add a new unit test that calls warn_early_consensus() directly.
Fixes bug 25756; bugfix on 0.2.2.25-alpha.
Included crypto_dh.h in some files in order to solve DH module dependency
issues.
Follows #24658.
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
This functionality was covered only accidentally by our voting-test
code, and as such wasn't actually tested at all. The tests that
called it made its coverage nondeterministic, depending on what time
of day you ran the tests.
Closes ticket 26014.
We cleared this value in second_elapsed_callback. But what were we
using it for? For detecting if Libevent returned EINVAL too often!
We already have a way to detect too-frequent events, and that's with
a ratelim_t. Refactor the code to use that instead. Closes ticket
26016.
From Neel's latest patch on optimizing the hs_circ_service_get_intro_circ()
digest calculation, remove an extra white-space and clarify a comment of the
legacy key digest to inform when to use it.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This function must return false if the module is not compiled in. In order to
do that, we move the authdir_mode_v3() function out of router.c and into the
dirauth module new header file named mode.h.
It is always returning false if we don't have the module.
Closes#25990
Signed-off-by: David Goulet <dgoulet@torproject.org>
When directory authorities read a zero-byte bandwidth file, they log
a warning with the contents of an uninitialised buffer. Log a warning
about the empty file instead.
Fixes bug 26007; bugfix on 0.2.2.1-alpha.
Arguably, the conditions under which these events happen should be a
bit different, but the rules are complex enough here that I've tried
to have this commit be pure refactoring.
Closes ticket 25952.
Finally, before this code goes away, take a moment to look at the
amazing way that we used to try to have an event happen
every N seconds:
get_uptime() / N != (get_uptime()+seconds_elapsed) / N
Truly, it is a thing of wonder. I'm glad we didn't start using this
pattern everywhere else.
By doing so, it is renamed to voting_schedule_recalculate_timing(). This
required a lot of changes to include voting_schedule.h everywhere that this
function was used.
This effectively now makes voting_schedule.{c|h} not include dirauth/dirvote.h
for that symbol and thus no dependency on the dirauth module anymore.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This function doesn't need to be public from the dirvote common file (which
will get renamed in future commit) so move it to dirauth/dirvote.c and make it
static.
Part of #25988
Signed-off-by: David Goulet <dgoulet@torproject.org>
It makes more sense to be in networkstatus.c so move it there and rename it
with the "networkstatus_" prefix.
Part of #25988
Signed-off-by: David Goulet <dgoulet@torproject.org>
Apparently, we can decide our state is dirty before we create the
event to tell the mainloop that we should save it. That's not a
problem, except for the assertion failure.
This requires that when a log cb happens, the event for flushing
queued events is scheduled, so we also add the necessary machinery
to have that happen.
Note that this doesn't actually help with logs from outside the main
thread, but those were already suppressed: see #25987 for a ticket
tracking that issue.
Commit 0f3b765b3c added
tor_assert_nonfatal_unreached() to dirvote_add_vote() and
dirvote_add_signatures() when the dirauth module is disabled.
However, they need to return a value. Furthermore, the dirvote_add_vote()
needs to set the msg_out and status_out so it can be sent back. Else,
uninitialized values would be used.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Originally, it was made public outside of the dirauth module but it is no
longer needed. In doing so, we put it back in dirvote.c and reverted its name
to the original one:
dirvote_authority_cert_dup() --> authority_cert_dup()
Signed-off-by: David Goulet <dgoulet@torproject.org>
The --disable-module-* configure option removes code from the final binary but
we still build the unit tests with the disable module(s) so we can actually
test that code path all the time and not forget about it.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This code is only for dirauth so this commit moves it into the module in
dirvote.c.
No code behavior change.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Both functions are used for directory request but they can only be used if the
running tor instance is a directory authority.
For this reason, make those symbols visible but hard assert() if they are
called when the module is disabled. This would mean we failed to safeguard the
entry point into the module.
Signed-off-by: David Goulet <dgoulet@torproject.org>
In order to further isolate the dirauth code into its module, this moves the
handling of the directory request GET /tor/status-vote/* into the module.
Signed-off-by: David Goulet <dgoulet@torproject.org>
In order to make sr_commit_free() only used by the dirauth module, this
commits moves the commits free from a vote object into the dirvote.c file
which is now only for the module.
The function does nothing if the module is disabled.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Adds two unittests:
- First checks the path selection of basic Tor circs.
- Second checks the path selection of vanguard circs.
There is a TODO on the second unittest that we might want to test sooner than
later, but it's not trivial to do it right now.
To do these unittests we needed the following mods:
- Make some functions STATIC.
- Add some more fields to the big fake network nodes of test_entrynodes.c
- Switch fake node nicknames to base32 (because base64 does not produce valid nicknames).
This prevents a malicious RP/IP from learning the guard node in the case that
we are using only one (because we aren't using two guards, or because one of
those two guards is temporarily down).
This ensures the "strong" version of Property #6 from
https://lists.torproject.org/pipermail/tor-dev/2018-April/013098.html
(Information about the guard(s) does not leak to the website/RP at all).
The last hop in vanguard circuits can be an RP/IP/HSDir.
Since vanguard circuits are at least 3 hops (sometimes 4) before this node,
this change will not cause A - B - A paths.
When parsing a vote in routerparse.c, only dirauth extract the commits from
the vote so move all this code into dirvote.c so we can make it specific to
the dirauth module.
If the dirauth module is disabled, the commit parsing does nothing.
Signed-off-by: David Goulet <dgoulet@torproject.org>
From dirvote.c to networkstatus.c where it makes more sense both in terms of
namespace and subsystem responsability.
This removes one less dependency on the dirauth module.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Add static inline dirauth public functions used outside of the dirauth module
so they can be seen by the tor code but simply do nothing.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Move most of the shared random functions that are needed outside of the
dirauth module.
At this commit, because dirvote.c hasn't been refactor, it doesn't compile
because some SR functions need a dirvote function.
Furthermore, 5 functions haven't been touched yet because they are dirauth
only but are in used in other C files than the dirauth module ones.
No code behavior change. Only moving code around.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is a pretty big commit but it only moves these files to src/or/dirauth:
dircollate.c dirvote.c shared_random.c shared_random_state.c
dircollate.h dirvote.h shared_random.h shared_random_state.h
Then many files are modified to change the include line for those header files
that have moved into a new directory.
Without using --disable-module-dirauth, everything builds fine. When using the
flag to disable the module, tor doesn't build due to linking errors. This will
be addressed in the next commit(s).
No code behavior change.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Remove useless include.
Clearly identify functions that are used by other part of Tor, functions that
are only used by the dirauth subsystem and functions that are exposed for unit
tests.
This will help us in the dirauth modularization effort.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Many functions become static to the C file or exposed to the tests within the
PRIVATE define of dirvote.h.
This commit moves a function to the top. No code behavior change.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Don't access the AuthoritativeDir options directly. We do this so we can move
authdir_mode() to the dirauth module.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Make our build system support a disable dirauth module option. It can only be
disabled explicitly with:
$ ./configure --disable-module-dirauth
If *not* specified that is enabled, an automake conditional variable is set to
true and a defined value for the C code:
AM_CONDITIONAL: BUILD_MODULE_DIRAUTH
AC_DEFINE: HAVE_MODULE_DIRAUTH=1
This introduces the dirauth/ module directory in src/or/ for which .c files
are only compiled if the BUILD_MODULE_DIRAUTH is set.
All the header files are compiled in regardless of the support so we can use
the alternative entry point functions of the dirauth subsystem.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Because ADD_ONION/DEL_ONION can modify the global service map (both for v2 and
v3), we need to rescan the event list so we either enable or disable the HS
service main loop event.
Fixees #25939
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is done because it makes our life easier with unit tests. Also, a rescan
on an uninitialized event list will result in a stacktrace.
Signed-off-by: David Goulet <dgoulet@torproject.org>
When we change the hibernation state, rescan the main loop event list because
the new state might affect the events.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Implement the ability to set flags per events which influences the set up of
the event.
This commit only adds one flag which is "need network" meaning that the event
is not enabled if tor has disabled the network or if hibernation mode.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Prior to #23100, we were not counting HS circuit build times in our
calculation of the timeout. This could lead to a condition where our timeout
was set too low, based on non HS circuit build times, and then we would
abandon all HS circuits, storing no valid timeouts in the histogram.
This commit avoids the assert.
In 25374, we created the necessary post-loop event for scheduling
connection_ap_attach_pending as needed. Before that, we were
already running this event once per mainloop. There's no reason to
also run it once per second.
Closes ticket 25933. No changes file, since the relevant change is
already in 25374. Or possibly in 17590, depending on how you look
at it.
Our main function, though accurate on all platforms, can be very
slow on 32-bit hosts. This one is faster on all 32-bit hosts, and
accurate everywhere except apple, where it will typically be off by
1%. But since 32-bit apple is a relic anyway, I think we should be
fine.
Previously were using this value to have a cheap highish-resolution
timer. But we were only using it in one place, and current dogma is
to use monotime_coarse_t for this kind of thing.
This part of the code was the only part that used "cached
getttimeofday" feature, which wasn't monotonic, which we updated at
slight expense, and which I'd rather not maintain.
The clean_consdiffmgr() callback is only for relays acting as a directory
server, not all relays.
This commit adds a role for only directory server and sets the
clean_consdiffmgr() callback to use it.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We can't end up in the removed else {} condition since we first validate the
flavor we get and then we validate the flavor we parse from the given
consensus which means we can only handle the two flavors of the if/elseif
conditions.
Fixes#25914
Signed-off-by: David Goulet <dgoulet@torproject.org>
Previously, we were ignoring values _over_ EPSILON. This bug was
also causing a warning at startup because the default value is set
to -1.0.
Fixes bug 25577; bugfix on 6b1dba214d. Bug not in any released tor.
Two helper functions to enable an event and disable an event which wraps the
launch and destroy of an event but takes care of the enabled flag.
They are also idempotent that is can be called multiple time on the same event
without effect if the event was already enabled or disabled.
Signed-off-by: David Goulet <dgoulet@torproject.org>
In case we transitionned to a new role in Tor, we need to launch and/or
destroy some periodic events.
Signed-off-by: David Goulet <dgoulet@torproject.org>
In tor, we have a series of possible "roles" that the tor daemon can be
enabled for. They are:
Client, Bridge, Relay, Authority (directory or bridge) and Onion service.
They can be combined sometimes. For instance, a Directory Authority is also a
Relay. This adds a "roles" field to a periodic event item object which is used
to know for which roles the event is for.
The next step is to enable the event only if the roles apply. No behavior
change at this commit.
Pars of #25762
Signed-off-by: David Goulet <dgoulet@torproject.org>
No behavior change, just to make it easier to find callbacks and for the sake
of our human brain to parse the list properly.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Consensus method 25 is the oldest one supported by any stable
version of 0.2.9, which is our current most-recent LTS. Thus, by
proposal 290, they should be removed.
This commit does not actually remove the code to implement these
methods: it only makes it so authorities will no longer support
them. I'll remove the backend code for them in later commits.
In order to fix 25691 and 25692, we need to pass the "direct_conn"
flag to more places -- particularly when choosing single-hop
tunnels. The right way to do this involves having a couple more
functions accept router_crn_flags_t, rather than a big list of
boolean arguments.
This commit also makes sure that choose_good_exit_server_general()
honors the direct_conn flag, to fix 25691 and 25692.
In router_add_running_nodes_to_smartlist(), we had an inline
implementation of the logic from node_has_descriptor(), which should
be changed to node_has_preferred_descriptor().
This patch adds a new node_has_preferred_descriptor() function, and
replaces most users of node_has_descriptor() with it. That's an
important change, since as of d1874b4339 (our fix for #25213),
we are willing to say that a node has _some_ descriptor, but not the
_right_ descriptor for a particular use case.
Part of a fix for 25691 and 25692.
Now that we allow cpuworkers for dirport-only hosts (to fix 23693),
we need to allow dup_onion_keys() to succeed for them.
The change to construct_ntor_key_map() is for correctness,
but is not strictly necessary.
This is done as follows:
* Only one function (find_dl_schedule()) actually returned a
smartlist. Now it returns an int.
* The CSV_INTERVAL type has been altered to ignore everything
after the first comma, and to store the value before the first
comma in an int.
This commit won't compile. It was made with the following perl
scripts:
s/smartlist_t \*(.*)DownloadSchedule;/int $1DownloadInitialDelay;/;
s/\b(\w*)DownloadSchedule\b/$1DownloadInitialDelay/;
Now that we update our buckets on demand before reading or writing,
we no longer need to update them all every TokenBucketRefillInterval
msec.
When a connection runs out of bandwidth, we do need a way to
reenable it, however. We do this by scheduling a timer to reenable
all blocked connections for TokenBucketRefillInterval msec after a
connection becomes blocked.
(If we were using PerConnBWRate more, it might make sense to have a
per-connection timer, rather than a single timeout. But since
PerConnBWRate is currently (mostly) unused, I'm going to go for the
simpler approach here, since usually whenever one connection has
become blocked on bandwidth, most connections are blocked on
bandwidth.)
Implements ticket 25373.
Previously this was done as part of the refill callback, but there's
no real reason to do it like that. Since we're trying to remove the
refill callback completely, we can do this work as part of
record_num_bytes_transferred_impl(), which already does quite a lot
of this.
We used to do this 10x per second in connection_buckets_refill();
instead, we now do it when the bucket becomes empty. This change is
part of the work of making connection_buckets_refill() obsolete.
Closes ticket 25828; bugfix on 0.2.3.5-alpha.
We recently merged a circuit cell queue size safeguard. This commit adds the
number of killed circuits that have reached the limit to the DoS heartbeat. It
now looks like this:
[notice] DoS mitigation since startup: 0 circuits killed with too many
cells. 0 circuits rejected, 0 marked addresses. 0 connections closed. 0
single hop clients refused.
Second thing that this patch does. It makes tor always print the DoS
mitigation heartbeat line (for a relay) even though no DoS mitigation have
been enabled. The reason is because we now kill circuits that have too many
cells regardless on if it is enabled or not but also it will give the operator
a chance to learn what is enabled with the heartbeat instead of suddenly
appearing when it is enabled by let say the consensus.
Fixes#25824
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit introduces the consensus parameter "circ_max_cell_queue_size"
which controls the maximum number of cells a circuit queue should have.
The default value is currently 50000 cells which is above what should be
expected but keeps us a margin of error for padding cells.
Related to this is #9072. Back in 0.2.4.14-alpha, we've removed that limit due
to a Guard discovery attack. Ticket #25226 details why we are putting back the
limit due to the memory pressure issue on relays.
Fixes#25226
Signed-off-by: David Goulet <dgoulet@torproject.org>
This is a simple search-and-replace to rename the token bucket type
to indicate that it contains both a read and a write bucket, bundled
with their configuration. It's preliminary to refactoring the
bucket type.
A linked connection_t is one that gets its I/O, not from the
network, but from another connection_t. When such a connection has
something to write, we want the corresponding connection to run its
read callback ... but not immediately, to avoid infinite recursion
and/or event loop starvation.
Previously we handled this case by activating the read events
outside the event loop. Now we use the "postloop event" logic.
This lets us simplify do_main_loop_once() a little.
In d1874b4339, we adjusted this check so that we insist on
using routerinfos for bridges. That's almost correct... but if we
have a bridge that is also a regular relay, then we should use
insist on its routerinfo when connecting to it as a bridge
(directly), and be willing to use its microdescriptor when
connecting to it elsewhere in our circuits.
This bug is a likely cause of some (all?) of the (exit_ei == NULL)
failures we've been seeing.
Fixes bug 25691; bugfix on 0.3.3.4-alpha
This patch changes the algorithm of compute_real_max_mem_in_queues() to
use 0.4 * RAM iff the system has more than or equal to 8 GB of RAM, but
will continue to use the old value of 0.75 * RAM if the system have less
than * GB of RAM available.
This patch also adds tests for compute_real_max_mem_in_queues().
See: https://bugs.torproject.org/24782
This one happens if for some reason you start with DirPort enabled
but server mode turned off entirely.
Fixes a case of bug 23693; bugfix on 0.3.1.1-alpha.
This roughly doubles our test coverage of the bridges.c module.
* ADD new testing module, .../src/test/test_bridges.c.
* CHANGE a few function declarations from `static` to `STATIC`.
* CHANGE one function in transports.c, transport_get_by_name(), to be
mockable.
* CLOSES#25425: https://bugs.torproject.org/25425
