The new HS circuitmap API replaces old public functions as follows:
circuit_clear_rend_token -> hs_circuitmap_remove_circuit
circuit_get_rendezvous -> hs_circuitmap_get_rend_circ
circuit_get_intro_point -> hs_circuitmap_get_intro_circ_v2
circuit_set_rendezvous_cookie -> hs_circuitmap_register_rend_circ
circuit_set_intro_point_digest -> hs_circuitmap_register_intro_circ_v2
This commit also removes the old rendinfo code that is now unused.
It also fixes the broken rendinfo unittests.
This patch doesn't cover every case; omitted cases are marked with
"XXXX prop271", as usual. It leaves both the old interface and the
new interface for guard status notification, since they don't
actually work in the same way: the new API wants to be told when a
circuit has failed or succeeded, whereas the old API wants to know
when a channel has failed or succeeded.
I ran into some trouble with directory guard stuff, since when we
pick the directory guard, we don't actually have a circuit to
associate it with. I solved that by allowing guard states to be
associated with directory connections, not just circuits.
In order to implement proposal 224, we need the data structure rend_data_t to
be able to accomodate versionning that is the current version of hidden
service (2) and the new version (3) and future version.
For that, we implement a series of accessors and a downcast function to get
the v2 data structure. rend_data_t becomes a top level generic place holder.
The entire rend_data_t API has been moved to hs_common.{c|h} in order to
seperate code that is shared from between HS versions and unshared code (in
rendcommon.c).
Closes#19024
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
Only when we were actually flushing the cell stats to a controller
would we free them. Thus, they could stay in RAM even after the
circuit was freed (eg if we didn't have any controllers).
Fixes bug 18673; bugfix on 0.2.5.1-alpha.
Check for consistency between the queued destroy cells and the marked
circuit IDs. Check for consistency in the count of queued destroy
cells in several ways. Check to see whether any of the marked circuit
IDs have somehow been marked longer than the channel has existed.
A new set of unit test cases are provided, as well as introducing
an alternative paradigm and macros to support it. Primarily, each test
case is given its own namespace, in order to isolate tests from each
other. We do this by in the usual fashion, by appending module and
submodule names to our symbols. New macros assist by reducing friction
for this and other tasks, like overriding a function in the global
namespace with one in the current namespace, or declaring integer
variables to assist tracking how many times a mock has been called.
A set of tests for a small-scale module has been included in this
commit, in order to highlight how the paradigm can be used. This
suite gives 100% coverage to status.c in test execution.
On busy servers, this function takes up something like 3-7% in
different profiles, and gets invoked every time we need to participate
as the midpoint in a hidden service.
So maybe walking through a linked list of all the circuits here wasn't
a good idea.
This patch splits out some of the functions in OOM handling so that
it's easier to check them without involving the rest of Tor or
requiring that the circuits be "wired up".
Right now this accounts for about 1% of circuits over all, but if you
pick a guard that's running 0.2.3, it will be about 6% of the circuits
running through that guard.
Making sure that every circuit has at least one ntor link means that
we're getting plausibly good forward secrecy on every circuit.
This implements ticket 9777,
This implements "algorithm 1" from my discussion of bug #9072: on OOM,
find the circuits with the longest queues, and kill them. It's also a
fix for #9063 -- without the side-effects of bug #9072.
The memory bounds aren't perfect here, and you need to be sure to
allow some slack for the rest of Tor's usage.
This isn't a perfect fix; the rest of the solutions I describe on
codeable.
In C, we technically aren't supposed to define our own things that
start with an underscore.
This is a purely machine-generated commit. First, I ran this script
on all the headers in src/{common,or,test,tools/*}/*.h :
==============================
use strict;
my %macros = ();
my %skipped = ();
FILE: for my $fn (@ARGV) {
my $f = $fn;
if ($fn !~ /^\.\//) {
$f = "./$fn";
}
$skipped{$fn} = 0;
open(F, $fn);
while (<F>) {
if (/^#ifndef ([A-Za-z0-9_]+)/) {
$macros{$fn} = $1;
next FILE;
}
}
}
print "#!/usr/bin/perl -w -i -p\n\n";
for my $fn (@ARGV) {
if (! exists $macros{$fn}) {
print "# No macro known for $fn!\n" if (!$skipped{$fn});
next;
}
if ($macros{$fn} !~ /_H_?$/) {
print "# Weird macro for $fn...\n";
}
my $goodmacro = uc $fn;
$goodmacro =~ s#.*/##;
$goodmacro =~ s#[\/\-\.]#_#g;
print "s/(?<![A-Za-z0-9_])$macros{$fn}(?![A-Za-z0-9_])/TOR_${goodmacro}/g;\n"
}
==============================
It produced the following output, which I then re-ran on those same files:
==============================
s/(?<![A-Za-z0-9_])_TOR_ADDRESS_H(?![A-Za-z0-9_])/TOR_ADDRESS_H/g;
s/(?<![A-Za-z0-9_])_TOR_AES_H(?![A-Za-z0-9_])/TOR_AES_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_H(?![A-Za-z0-9_])/TOR_COMPAT_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_LIBEVENT_H(?![A-Za-z0-9_])/TOR_COMPAT_LIBEVENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTAINER_H(?![A-Za-z0-9_])/TOR_CONTAINER_H/g;
s/(?<![A-Za-z0-9_])_TOR_CRYPTO_H(?![A-Za-z0-9_])/TOR_CRYPTO_H/g;
s/(?<![A-Za-z0-9_])TOR_DI_OPS_H(?![A-Za-z0-9_])/TOR_DI_OPS_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMAREA_H(?![A-Za-z0-9_])/TOR_MEMAREA_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMPOOL_H(?![A-Za-z0-9_])/TOR_MEMPOOL_H/g;
s/(?<![A-Za-z0-9_])TOR_PROCMON_H(?![A-Za-z0-9_])/TOR_PROCMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORGZIP_H(?![A-Za-z0-9_])/TOR_TORGZIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORINT_H(?![A-Za-z0-9_])/TOR_TORINT_H/g;
s/(?<![A-Za-z0-9_])_TOR_LOG_H(?![A-Za-z0-9_])/TOR_TORLOG_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORTLS_H(?![A-Za-z0-9_])/TOR_TORTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_UTIL_H(?![A-Za-z0-9_])/TOR_UTIL_H/g;
s/(?<![A-Za-z0-9_])_TOR_BUFFERS_H(?![A-Za-z0-9_])/TOR_BUFFERS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_H(?![A-Za-z0-9_])/TOR_CHANNEL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_TLS_H(?![A-Za-z0-9_])/TOR_CHANNELTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITBUILD_H(?![A-Za-z0-9_])/TOR_CIRCUITBUILD_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITLIST_H(?![A-Za-z0-9_])/TOR_CIRCUITLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_EWMA_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_EWMA_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITUSE_H(?![A-Za-z0-9_])/TOR_CIRCUITUSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMMAND_H(?![A-Za-z0-9_])/TOR_COMMAND_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONFIG_H(?![A-Za-z0-9_])/TOR_CONFIG_H/g;
s/(?<![A-Za-z0-9_])TOR_CONFPARSE_H(?![A-Za-z0-9_])/TOR_CONFPARSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_EDGE_H(?![A-Za-z0-9_])/TOR_CONNECTION_EDGE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_H(?![A-Za-z0-9_])/TOR_CONNECTION_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_OR_H(?![A-Za-z0-9_])/TOR_CONNECTION_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTROL_H(?![A-Za-z0-9_])/TOR_CONTROL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CPUWORKER_H(?![A-Za-z0-9_])/TOR_CPUWORKER_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRECTORY_H(?![A-Za-z0-9_])/TOR_DIRECTORY_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRSERV_H(?![A-Za-z0-9_])/TOR_DIRSERV_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRVOTE_H(?![A-Za-z0-9_])/TOR_DIRVOTE_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNS_H(?![A-Za-z0-9_])/TOR_DNS_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNSSERV_H(?![A-Za-z0-9_])/TOR_DNSSERV_H/g;
s/(?<![A-Za-z0-9_])TOR_EVENTDNS_TOR_H(?![A-Za-z0-9_])/TOR_EVENTDNS_TOR_H/g;
s/(?<![A-Za-z0-9_])_TOR_GEOIP_H(?![A-Za-z0-9_])/TOR_GEOIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_HIBERNATE_H(?![A-Za-z0-9_])/TOR_HIBERNATE_H/g;
s/(?<![A-Za-z0-9_])_TOR_MAIN_H(?![A-Za-z0-9_])/TOR_MAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_MICRODESC_H(?![A-Za-z0-9_])/TOR_MICRODESC_H/g;
s/(?<![A-Za-z0-9_])_TOR_NETWORKSTATUS_H(?![A-Za-z0-9_])/TOR_NETWORKSTATUS_H/g;
s/(?<![A-Za-z0-9_])_TOR_NODELIST_H(?![A-Za-z0-9_])/TOR_NODELIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_NTMAIN_H(?![A-Za-z0-9_])/TOR_NTMAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_ONION_H(?![A-Za-z0-9_])/TOR_ONION_H/g;
s/(?<![A-Za-z0-9_])_TOR_OR_H(?![A-Za-z0-9_])/TOR_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_POLICIES_H(?![A-Za-z0-9_])/TOR_POLICIES_H/g;
s/(?<![A-Za-z0-9_])_TOR_REASONS_H(?![A-Za-z0-9_])/TOR_REASONS_H/g;
s/(?<![A-Za-z0-9_])_TOR_RELAY_H(?![A-Za-z0-9_])/TOR_RELAY_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCLIENT_H(?![A-Za-z0-9_])/TOR_RENDCLIENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCOMMON_H(?![A-Za-z0-9_])/TOR_RENDCOMMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDMID_H(?![A-Za-z0-9_])/TOR_RENDMID_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDSERVICE_H(?![A-Za-z0-9_])/TOR_RENDSERVICE_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPHIST_H(?![A-Za-z0-9_])/TOR_REPHIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPLAYCACHE_H(?![A-Za-z0-9_])/TOR_REPLAYCACHE_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTER_H(?![A-Za-z0-9_])/TOR_ROUTER_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERLIST_H(?![A-Za-z0-9_])/TOR_ROUTERLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERPARSE_H(?![A-Za-z0-9_])/TOR_ROUTERPARSE_H/g;
s/(?<![A-Za-z0-9_])TOR_ROUTERSET_H(?![A-Za-z0-9_])/TOR_ROUTERSET_H/g;
s/(?<![A-Za-z0-9_])TOR_STATEFILE_H(?![A-Za-z0-9_])/TOR_STATEFILE_H/g;
s/(?<![A-Za-z0-9_])_TOR_STATUS_H(?![A-Za-z0-9_])/TOR_STATUS_H/g;
s/(?<![A-Za-z0-9_])TOR_TRANSPORTS_H(?![A-Za-z0-9_])/TOR_TRANSPORTS_H/g;
s/(?<![A-Za-z0-9_])_TOR_TEST_H(?![A-Za-z0-9_])/TOR_TEST_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_NATPMP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_NATPMP_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_UPNP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_UPNP_H/g;
==============================
