Also add a trunnel definition for link_specifier_list
Signed-off-by: John Brooks <special@torproject.org>
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
In order to implement proposal 224, we need the data structure rend_data_t to
be able to accomodate versionning that is the current version of hidden
service (2) and the new version (3) and future version.
For that, we implement a series of accessors and a downcast function to get
the v2 data structure. rend_data_t becomes a top level generic place holder.
The entire rend_data_t API has been moved to hs_common.{c|h} in order to
seperate code that is shared from between HS versions and unshared code (in
rendcommon.c).
Closes#19024
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
One is fixed by disabling the -Wredundant-decls warnings around
openssl headers here, because of the old double-declaration of
SSL_get_selected_srtp_profile().
One is fixed by including compat.h before or.h so that we get the
winsock2.h include before the windows.h include.
In our code to write public keys to a string, for some unfathomable
reason since 253f0f160e, we would allocate a memory BIO, then
set the NOCLOSE flag on it, extract its memory buffer, and free it.
Then a little while later we'd free the memory buffer with
BUF_MEM_free().
As of openssl 1.1 this doesn't work any more, since there is now a
BIO_BUF_MEM structure that wraps the BUF_MEM structure. This
BIO_BUF_MEM doesn't get freed in our code.
So, we had a memory leak!
Is this an openssl bug? Maybe. But our code was already pretty
silly. Why mess around with the NOCLOSE flag here when we can just
keep the BIO object around until we don't need the buffer any more?
Fixes bug 20553; bugfix on 0.0.2pre8
This function is allowed to return NULL if the certified key isn't
RSA. But in a couple of places we were treating this as a bug or
internal error, and in one other place we weren't checking for it at
all!
Caught by Isis during code review for #15055. The serious bug was
only on the 15055 branch, thank goodness.
All supported Tors (0.2.4+) require versions of openssl that can
handle this.
Now that our link certificates are RSA2048, this might actually help
vs fingerprinting a little.
This was a stopgap method, designed on the theory that some routers
might support it before they could support Ed25519. But it looks
like everybody who supports RFC5705 will also have an Ed25519 key,
so there's not a lot of reason to have this even supported.
This patch moves the pregenerated RSA key logic into a new
testing_rsakeys.c.
Also, it adds support for RSA2048, since the link handshake tests
want that.
Also, it includes pregenerated keys, rather than trying to actually
generate the keys at startup, since generating even a small handful
of RSA2048 keys makes for an annoying delay.
This code stores the ed certs as appropriate, and tries to check
them. The Ed25519 result is not yet used, and (because of its
behavior) this will break RSA authenticate cells. That will get
fixed as we go, however.
This should implement 19157, but it needs tests, and it needs
to get wired in.
In particular, these functions are the ones that set the identity of
a given connection or channel, and/or confirm that we have learned
said IDs.
There's a lot of stub code here: we don't actually need to use the
new keys till we start looking up connections/channels by Ed25519
IDs. Still, we want to start passing the Ed25519 IDs in now, so it
makes sense to add these stubs as part of 15055.
The impact here isn't too bad. First, the only affected certs that
expire after 32-bit signed time overflows in Y2038. Second, it could
only make it seem that a non-expired cert is expired: it could never
make it seem that an expired cert was still live.
Fixes bug 20027; bugfix on 0.2.7.2-alpha.
Also, adjust signing approach to more closely match the signing
scheme in the proposal.
(The format doesn't quite match the format in the proposal, since
RSA signatures aren't fixed-length.)
Closes 19020.
See proposal 244. This feature lets us stop looking at the internals
of SSL objects, *and* should let us port better to more SSL libraries,
if they have RFC5705 support.
Preparatory for #19156
We no longer generate certs cells by pasting the certs together one
by one. Instead we use trunnel to generate them.
Preliminary work for 19155 (send CERTS cell with ed certs)
passthrough_test_setup doesn't pass through arguments if the argument
is equal to 0 or TT_SKIP. Instead, it fails or skips the test.
Assert on this, so we don't accidentally fail or skip tests.
Fixes bug 19969; bugfix on b1d56fc58. We can fix this some more in
later Tors, but for now, this is probably the simplest fix possible.
This is a belt-and-suspenders fix, where the earlier fix ("Ask
event_base_loop to finish when we add a pending stream") aims to respond
to new streams as soon as they arrive, and this one aims to make sure
that we definitely respond to all of the streams.
ome policies are default-reject, some default-accept. But
policy_is_reject_star() assumed they were all default_reject. Fix
that!
Also, document that policy_is_reject_star() treats a NULL policy as
empty. This allows us to simplify the checks in
parse_reachable_addresses() by quite a bit.
Fxes bug 20306; bugfix on 0.2.8.2-alpha.
(Also, refactor the code to create a hidden service directory into a
separate funcion, so we don't have to duplicate it.)
Fixes bug 20484; bugfix on 0.2.9.3-alpha.
This simplifies the function: if we have an ntor key, use ntor/EXTEND2,
otherwise, use TAP/EXTEND.
Bugfix on commit 10aa913 from 19163 in 0.2.9.3-alpha.
I had replaced a comment implying that a set of ifs was meant to be
exhaustive with an actual check for exhaustiveness. It turns out,
they were exhaustive, but not in the way I had assumed. :(
Bug introduced in f3e158edf7, not in any released Tor.
Use the following coccinelle script to change uses of
smartlist_add(sl, tor_strdup(str)) to
smartlist_add_strdup(sl, string) (coccinelle script from nickm
via bug 20048):
@@
expression a;
expression b;
@@
- smartlist_add
+ smartlist_add_strdup
(a,
- tor_strdup(
b
- )
)
The tor_fragile_assert() bug has existed here since c8a5e2d588
in tor-0.2.1.7-alpha forever, but tor_fragile_assert() was mostly a
no-op until 0.2.9.1-alpha.
Fixes bug 19869.
When we refactored purpose_needs_anonymity(), we made it so _all_
bridge requests required anonymity. But that missed the case
that we are allowed to ask a bridge for its own descriptor.
With this patch, we consider the resource, and allow "authority.z"
("your own descriptor, compressed") for a bridge's server descriptor
to be non-anonymous.
Fix for bug 20410; bug not in any released Tor.
I believe that this should never trigger, but if it does, it
suggests that there was a gap between is_sensitive_dir_purpose and
purpose_needs_anonymity that we need to fill. Related to 20077.
This helps protect against bugs where any part of a buf_t's memory
is passed to a function that expects a NUL-terminated input.
It also closes TROVE-2016-10-001 (aka bug 20384).
This commit adds or improves the module-level documenation for:
buffers.c circuitstats.c command.c connection_edge.c control.c
cpuworker.c crypto_curve25519.c crypto_curve25519.h
crypto_ed25519.c crypto_format.c dircollate.c dirserv.c dns.c
dns_structs.h fp_pair.c geoip.c hibernate.c keypin.c ntmain.c
onion.c onion_fast.c onion_ntor.c onion_tap.c periodic.c
protover.c protover.h reasons.c rephist.c replaycache.c
routerlist.c routerparse.c routerset.c statefile.c status.c
tor_main.c workqueue.c
In particular, I've tried to explain (for each documented module)
what each module does, what's in it, what the big idea is, why it
belongs in Tor, and who calls it. In a few cases, I've added TODO
notes about refactoring opportunities.
I've also renamed an argument, and fixed a few DOCDOC comments.
(I've done this instead of changing the semantics of
router_compare_to_my_exit_policy, because dns.c uses
router_compare_to_my_exit_policy too, in a slightly weird way.)
Some compilers apparently noticed that p2len was allowed to be equal
to msg, and so maybe we would be doing memset(prompt2, ' ', 0), and
decided that we probably meant to do memset(prompt2, 0, 0x20);
instead.
Stupid compilers, doing optimization before this kind of warning!
My fix is to just fill the entire prompt2 buffer with spaces,
because it's harmless.
Bugfix on e59f0d4cb9, not in any released Tor.
(Specifically, carriage return after a quoted value in a config
line. Fixes bug 19167; bugfix on 0.2.0.16-alpha when we introduced
support for quoted values. Unit tests, changes file, and this
parenthetical by nickm.)
This is a kludge to deal with the fact that `tor_addr_t` doesn't contain
`sun_path`. This currently ONLY happens when circuit isolation is being
checked, for an isolation mode that is force disabled anyway, so the
kludge is "ugly but adequate", but realistically, making `tor_addr_t`
and the AF_UNIX SocksPort code do the right thing is probably the better
option.
Closes ticket 20303.
The LIBRESSL_VERSION_NUMBER check is needed because if our openssl
is really libressl, it will have an openssl version number we can't
really believe.
Previously, we would reject even rendezvous connections to IPv6
addresses when IPv6Exit was false. But that doesn't make sense; we
don't count that as "exit"ing. I've corrected the logic and tried
to make it a lottle more clear.
Fixes bug 18357; this code has been wrong since 9016d9e829 in
0.2.4.7-alpha.
When deleting unsuitable addresses in get_interface_address6_list(), to
avoid reordering IPv6 interface addresses and keep the order returned by
the OS, use SMARTLIST_DEL_CURRENT_KEEPORDER() instead of
SMARTLIST_DEL_CURRENT().
This issue was reported by René Mayrhofer.
[Closes ticket 20163; changes file written by teor. This paragraph
added by nickm]
We removed that feature in 0.2.4.2-alpha, but some comments seem to
have lingered.
I didn't add a changes/ file since this is just internal code cleanup.
(like clients do) rather than old-style router descriptors. Now bridges
will blend in with clients in terms of the circuits they build.
Fixes bug 6769; bugfix on 0.2.3.2-alpha.
Clients that use bridges were ignoring their cached microdesc-flavor
consensus files, because they only thought they should use the microdesc
flavor once they had a known-working bridge that could offer microdescs,
and at first boot no bridges are known-working.
This bug caused bridge-using clients to download a new microdesc consensus
on each startup.
Fixes bug 20269; bugfix on 0.2.3.12-alpha.
The client addr is essentially meaningless in this context (yes, it is
possible to explicitly `bind()` AF_LOCAL client side sockets to a path,
but no one does it, and there are better ways to grant that sort of
feature if people want it like using `SO_PASSCRED`).
As before, we check server protocols whenever server_mode(options)
is true and we check client protocols whenever server_mode(options)
is false.
Additionally, we now _also_ check client protocols whenever any
client port is set.
(Technically, we could just remove extend2 cell checking entirely,
since all Tor versions on our network are required to have it, but
let's keep this around as an example of How To Do It.)
(Despite the increased size of the consensus, this should have
approximately zero effect on the compressed consensus size, since
the "proto" line should be completely implied by the "v" line.)
[This is a brute-force method that potentially uses way too much
RAM. Need to rethink this a little. Right now you can DOS an
authority by saying "Foo=1-4294967295".]
Sierra provides clock_gettime(), but not pthread_condattr_setclock.
So we had better lot try to use CLOCK_MONOTONIC as our source for
time when waiting, since we ccan never actually tell the condition
that we mean CLOCK_MONOTONIC.
This isn't a tor bug yet, since we never actually pass a timeout to
tor_cond_wait() outside of the unit tests.
Not telling the cmux would sometimes cause an assertion failure in
relay.c when we tried to get an active circuit and found an "active"
circuit with no cells.
Additionally, replace that assert with a test and a log message.
Fix for bug 20203. This is actually probably a bugfix on
0.2.8.1-alpha, specifically my code in 8b4e5b7ee9 where I
made circuit_mark_for_close_() do less in order to simplify our call
graph. Thanks to "cypherpunks" for help diagnosing.
Our use of the (mockable) tor_close_socket() in the util/socket_..
tests confused coverity, which could no longer tell that we were
actually closing the sockets.
It's a macro that calls down to a function whose behavior has been
getting progresively more complicated.... but we named it as if it
were a variable. That's not so smart. So, replace it with a
function call to a function that was just doing "return
current_consensus".
Fixes bug 20176.
Commit 41cc1f612b introduced a "dns_request"
configuration value which wasn't set to 1 for an entry connection on the
DNSPort leading to a refusal to resolve the given hostname.
This commit set the dns_request flag by default for every entry connection
made to the DNSPort.
Fixes#20109
Signed-off-by: David Goulet <dgoulet@torproject.org>
For a brief moment in networkstatus_set_current_consensus(), the old
consensus has been freed, but the node_t objects still have dead
pointers to the routerstatus_t objects within it. During that
interval, we absolutely must not do anything that would cause Tor to
look at those dangling pointers.
Unfortunately, calling the (badly labeled!) current_consensus macro
or anything else that calls into we_use_microdescriptors_for_circuits(),
can make us look at the nodelist.
The fix is to make sure we identify the main consensus flavor
_outside_ the danger zone, and to make the danger zone much much
smaller.
Fixes bug 20103. This bug has been implicitly present for AGES; we
just got lucky for a very long time. It became a crash bug in
0.2.8.2-alpha when we merged 35bbf2e4a4 to make
find_dl_schedule start looking at the consensus, and 4460feaf28
which made node_get_all_orports less (accidentally) tolerant of
nodes with a valid ri pointer but dangling rs pointer.
Previously, the IV and key were stored in the structure, even though
they mostly weren't needed. The only purpose they had was to
support a seldom-used API where you could pass NULL when creating
a cipher in order to get a random key/IV, and then pull that key/IV
back out.
This saves 32 bytes per AES instance, and makes it easier to support
different key lengths.
* Check consistency between the two single onion torrc options
* Use the more relevant option each time we check for single onion mode
* Clarify log messages
* Clarify comments
* Otherwise, no behaviour change
Tor checks that the flag matches the configured onion service anonymity.
Tor refuses to create unflagged onion service using ADD_ONION, if they
would be non-anonymous. The error is:
512 Tor is in non-anonymous onion mode
Similarly, if the NonAnonymous flag is present, and Tor has the default
anonymous onion config:
512 Tor is in anonymous onion mode
Parse the value to UseEntryNodes_option, then set UseEntryNodes before
validating options.
This way, Authorities, Tor2web, and Single Onion Services don't write
spurious "UseEntryNodes 0" lines to their configs. Document the fact that
these tor configurations ignore UseEntryNodes in the manual page.
Also reorder options validation so we modify UseEntryNodes first, then
check its value against EntryNodes.
And silence a warning about disabled UseEntryNodes for hidden services
when we're actually in non-anonymous single onion service mode.
We log this message every time we validate tor's options.
There's no need to log a duplicate in main() as well.
(It is impossible to run main() without validating our options.)
The changes in #19973 fixed ReachableAddresses being applied
too broadly, but they also broke Tor2web (somewhat unintentional)
compatibility with ReachableAddresses.
This patch restores that functionality, which makes intro and
rend point selection is consistent between Tor2web and Single Onion
Services.
Add experimental OnionServiceSingleHopMode and
OnionServiceNonAnonymousMode options. When both are set to 1, every
hidden service on a tor instance becomes a non-anonymous Single Onion
Service. Single Onions make one-hop (direct) connections to their
introduction and renzedvous points. One-hop circuits make Single Onion
servers easily locatable, but clients remain location-anonymous.
This is compatible with the existing hidden service implementation, and
works on the current tor network without any changes to older relays or
clients.
Implements proposal #260, completes ticket #17178. Patch by teor & asn.
squash! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! Implement Prop #260: Single Onion Services
Redesign single onion service poisoning.
When in OnionServiceSingleHopMode, each hidden service key is poisoned
(marked as non-anonymous) on creation by creating a poison file in the
hidden service directory.
Existing keys are considered non-anonymous if this file exists, and
anonymous if it does not.
Tor refuses to launch in OnionServiceSingleHopMode if any existing keys
are anonymous. Similarly, it refuses to launch in anonymous client mode
if any existing keys are non-anonymous.
Rewrite the unit tests to match and be more comprehensive.
Adds a bonus unit test for rend_service_load_all_keys().
(We check consensus method when deciding whether to assume a node is
valid. No need to check the consensus method for Running, since
we will never see a method before 13.)
Closes ticket 20001
g
The other test vectors are pretty complete, and get full coverage, I
believe.
This one test vector accounted for half the time spent in
test-slow. "Now that's slow!"
We have a mock for our RSA key generation function, so we now wire
it to pk_generate(). This covers all the cases that were not using
pk_generate() before -- all ~93 of them.
Previously, you needed to store the previous log severity in a local
variable, and it wasn't clear if you were allowed to call these
functions more than once.
The functions it warns about are:
assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
strdup, strndup, calloc.
Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
We should consider them bugs. If they are happening intentionally,
we should use the log_test_helpers code to capture and suppress
them. But having them off-by-default has potential to cause
programming errors.
Previously setup_capture_of_logs would prevent log messages from
going to the console entirely. That's a problem, since sometimes
log messages are bugs! Now setup_capture_of_logs() acts sensibly.
If you really do need to keep a message from going to the console
entirely, there is setup_full_capture_of_logs(). But only use that
if you're prepared to make sure that there are no extraneous
messages generated at all.
Users can't run an anonymous client and non-anonymous single
onion service at the same time. We need to know whether we have
any client ports or sockets open to do this check.
When determining whether a client port (SOCKS, Trans, NATD, DNS)
is set, count unix sockets when counting client listeners. This
has no user-visible behaviour change, because these options are
set once and never read in the current tor codebase.
Don't count sockets when setting ControlPort_set, that's what
ControlSocket is for. (This will be reviewed in #19665.)
Don't count sockets when counting server listeners, because the code
that uses these options expects to count externally-visible ports.
(And it would change the behaviour of Tor.)
Copying the integer 42 in a char buffer has a different representation
depending on the endianess of the system thus that unit test was failing on
big endian system.
This commit introduces a python script, like the one we have for SRV, that
computes a COMMIT/REVEAL from scratch so we can use it as a test vector for
our encoding unit tests.
With this, we use a random value of bytes instead of a number fixing the
endianess issue and making the whole test case more solid with an external
tool that builds the COMMIT and REVEAL according to the spec.
Fixes#19977
Signed-off-by: David Goulet <dgoulet@torproject.org>
Check NoOnionTraffic before attaching a stream.
NoOnionTraffic refuses connections to all onion hostnames,
but permits non-onion hostnames and IP addresses.
Check NoDNSRequest, NoIPv4Traffic, and NoIPv6Traffic before
attaching a stream.
NoDNSRequest refuses connections to all non-onion hostnames,
but permits IP addresses.
NoIPv4Traffic refuses connections to IPv4 addresses, but resolves
hostnames.
NoIPv6Traffic refuses connections to IPv6 addresses, but resolves
hostnames.
Combined, they refuse all non-onion hostnames and IP addresses.
OnionTrafficOnly is equivalent to NoDNSRequest, NoIPv4Traffic,
and NoIPv6Traffic.
Add unit tests for parsing and checking option validity.
Add documentation for each flag to the man page.
Add changes file for all of #18693.
Parsing only: the flags do not change client behaviour (yet!)
Rely on onion_populate_cpath to check that we're only using
TAP for the rare hidden service cases.
Check and log if handshakes only support TAP when they should support
ntor.
When a client connects to an intro point not in the client's consensus,
or a hidden service connects to a rend point not in the hidden service's
consensus, we are stuck with using TAP, because there is no ntor link
specifier.
This bug had existed since 0.2.4.7-alpha, but now that we have
FallbackDirs by default, it actually matters.
Fixes bug 19947; bugfix on 0.2.4.7-alpha or maybe 0.2.8.1-alpha.
Rubiate wrote the patch; teor wrote the changes file.
Longer and more explicit log message so we don't confuse users with behind NAT with working configurations and state that public IP addresses only should be provided with "Address", won't work with internal addresses.
OpenBSD removes this function, and now that Tor requires Libevent 2,
we should also support the OpenBSD Libevent 2.
Fixes bug 19904; bugfix on 0.2.5.4-alpha.
These functions were there so that we could abstract the differences
between evbuffer and buf_t. But with the bufferevent removal, this
no longer serves a purpose.
To maintain precision, to get nanoseconds, we were multiplying our
tick count by a billion, then dividing by ticks-per-second. But
that apparently isn't such a great idea, since ticks-per-second is
sometimes a billion on its own, so our intermediate result was
giving us attoseconds.
When you're counting in attoseconds, you can only fit about 9
seconds into an int64_t, which is not so great for our purposes.
Instead, we now simplify the 1000000000/1000000000 fraction before
we start messing with nanoseconds. This has potential to mess us
up if some future MS version declares that performance counters will
use 1,000,000,007 units per second, but let's burn that bridge when
we come to it.
* Raise limit: 16k isn't all that high.
* Don't log when limit exceded; log later on.
* Say "over" when we log more than we say we log.
* Add target version to changes file
This code uses QueryPerformanceCounter() [**] on Windows,
mach_absolute_time() on OSX, clock_gettime() where available, and
gettimeofday() [*] elsewhere.
Timer types are stored in an opaque OS-specific format; the only
supported operation is to compute the difference between two timers.
[*] As you know, gettimeofday() isn't monotonic, so we include
a simple ratchet function to ensure that it only moves forward.
[**] As you may not know, QueryPerformanceCounter() isn't actually
always as monotonic as you might like it to be, so we ratchet that
one too.
We also include a "coarse monotonic timer" for cases where we don't
actually need high-resolution time. This is GetTickCount{,64}() on
Windows, clock_gettime(CLOCK_MONOTONIC_COARSE) on Linux, and falls
back to regular monotonic time elsewhere.
If we know a node's version, and it can't do ntor, consider it not running.
If we have a node's descriptor, and it doesn't have a valid ntor key,
consider it not running.
Refactor these checks so they're consistent between authorities and clients.
Before, they checked for version 0.2.4.18-rc or later, but this
would not catch relays without version lines, or buggy or malicious
relays missing an ntor key.
If we did not find a non-private IPaddress by iterating over interfaces,
we would try to get one via
get_interface_address6_via_udp_socket_hack(). This opens a datagram
socket with IPPROTO_UDP. Previously all our datagram sockets (via
libevent) used IPPROTO_IP, so we did not have that in the sandboxing
whitelist. Add (SOCK_DGRAM, IPPROTO_UDP) sockets to the sandboxing
whitelist. Fixes bug 19660.
This fixes#19608, allowing IPv6-only clients to use
microdescriptors, while preserving the ability of bridge clients
to have some IPv4 bridges and some IPv6 bridges.
Fix on c281c036 in 0.2.8.2-alpha.
The test was checking for EISDIR which is a Linux-ism making other OSes
unhappy. Instead of checking for a negative specific errno value, just make
sure it's negative indicating an error. We don't need more for this test.
Signed-off-by: David Goulet <dgoulet@torproject.org>
We introduded a shadowed variable, thereby causing a log message to
be wrong. Fixes 19578. I believe the bug was introduced by
54d7d31cba in 0.2.2.29-beta.
I grepped and hand-inspected the "it's" instances, to see if any
were supposed to be possessive. While doing that, I found a
"the the", so I grepped to see if there were any more.
Keep the base16 representation of the RSA identity digest in the commit object
so we can use it without using hex_str() or dynamically encoding it everytime
we need it. It's used extensively in the logs for instance.
Fixes#19561
Signed-off-by: David Goulet <dgoulet@torproject.org>
Encoded commit has an extra byte at the end for the NUL terminated byte and
the test was overrunning the payload buffer by one byte.
Found by Coverity issue 1362984.
Fixes#19567
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch also updates a comment in the same function for accuracy.
Found by Coverity issue 1362985. Partily fixes#19567.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
Our sandboxing code would not allow us to write to stats/hidserv-stats,
causing tor to abort while trying to write stats. This was previously
masked by bug#19556.
When sandboxing is enabled, we could not write any stats to disk.
check_or_create_data_subdir("stats"), which prepares the private stats
directory, calls check_private_dir(), which also opens and not just stats() the
directory. Therefore, we need to also allow open() for the stats dir in our
sandboxing setup.
The test_state_update() test would fail if you run it between 23:30 and
00:00UTC in the following line because n_protocol_runs was 2:
tt_u64_op(state->n_protocol_runs, ==, 1);
The problem is that when you launch the test at 23:30UTC (reveal phase),
sr_state_update() gets called from sr_state_init() and it will prepare
the state for the voting round at 00:00UTC (commit phase). Since we
transition from reveal to commit phase, this would trigger a phase
transition and increment the n_protocol_runs counter.
The solution is to initialize the n_protocol_runs to 0 explicitly in the
beginning of the test, as we do for n_reveal_rounds, n_commit_rounds etc.
The *get* state query functions for the SRVs now only return const pointers
and the DEL action needs to be used to delete the SRVs from the state.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch makes us retain the intermediate list of K=V entries for
the duration of computing our vote, and lets us use that list with
a new function in order to look up parameters before the consensus
is published.
We can't actually use this function yet because of #19011: our
existing code to do this doesn't actually work, and we'll need a new
consensus method to start using it.
Closes ticket #19012.
Commit and reveal length macro changed from int to unsigned long int
(size_t) because of the sizeof().
Signed-off-by: David Goulet <dgoulet@torproject.org>
See ticket #19132 for the clang/llvm warning.
Since voting_schedule is a global static struct, it will be initialized
to zero even without explicitly initializing it with {0}.
This is what the C spec says:
If an object that has automatic storage duration is not initialized
explicitly, its value is indeterminate. If an object that has static
storage duration is not initialized explicitly, then:
— if it has pointer type, it is initialized to a null pointer;
— if it has arithmetic type, it is initialized to (positive or unsigned) zero;
— if it is an aggregate, every member is initialized (recursively) according to these rules;
— if it is a union, the first named member is initialized (recursively) according to these rules.
Code has been changed so every RSA fingerprint for a commit in our state is
validated before being used. This fixes the unit tests by mocking one of the
key function and updating the hardcoded state string.
Also, fix a time parsing overflow on platforms with 32bit time_t
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
We assert on it using the ASSERT_COMMIT_VALID() macro in critical places
where we use them expecting a commit to be valid.
Signed-off-by: David Goulet <dgoulet@torproject.org>
The prop250 code used the RSA identity key fingerprint to index commit in a
digestmap instead of using the digest.
To behavior change except the fact that we are actually using digestmap
correctly.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit makes it that tor now uses the shared random protocol by
initializing the subsystem.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
One of the last piece that parses the votes and consensus in order to update
our state and make decision for the SR values.
We need to inform the SR subsystem when we set the current consensus because
this can be called when loaded from file or downloaded from other authorities
or computed.
The voting schedule is used for the SR timings since we are bound to the
voting system.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
This commit adds the commit(s) line in the vote as well as the SR values. It
also has the mechanism to add the majority SRVs in the consensus.
Signed-off-by: George Kadianakis <desnacked@riseup.net>
Signed-off-by: David Goulet <dgoulet@torproject.org>
This adds the logic of commit and SR values generation. Furthermore, the
concept of a protocol run is added that is commit is generated at the right
time as well as SR values which are also rotated before a new protocol run.
Signed-off-by: George Kadianakis <desnacked@riseup.net>
Signed-off-by: David Goulet <dgoulet@torproject.org>
From 0.2.7.2-alpha onwards, Exits would reject all the IP addresses
they knew about in their exit policy. But this may have disclosed
addresses that were otherwise unlisted.
Now, only advertised addresses are rejected by default by
ExitPolicyRejectPrivate. All known addresses are only rejected when
ExitPolicyRejectLocalInterfaces is explicitly set to 1.
Validate that tv_usec inputs to tv_udiff and tv_mdiff are in range.
Do internal calculations in tv_udiff and tv_mdiff in 64-bit,
which makes the function less prone to integer overflow,
particularly on platforms where long and time_t are 32-bit,
but tv_sec is 64-bit, like some BSD configurations.
Check every addition and subtraction that could overflow.
If we manually remove fallbacks in C by adding '/*' and '*/' on separate
lines, stem still parses them as being present, because it only looks at
the start of a line.
Add a comment to this effect in the generated source code.
Remove a fallback that changed its fingerprint after it was listed
This happened after to a software update:
https://lists.torproject.org/pipermail/tor-relays/2016-June/009473.html
Remove a fallback that changed IPv4 address
Remove two fallbacks that were slow to deliver consensuses,
we can't guarantee they'll be fast in future.
Blacklist all these fallbacks until operators confirm they're stable.
This commit introduces two new files with their header.
"shared_random.c" contains basic functions to initialize the state and allow
commit decoding for the disk state to be able to parse them from disk.
"shared_random_state.c" contains everything that has to do with the state
for both our memory and disk. Lots of helper functions as well as a
mechanism to query the state in a synchronized way.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
Make sure to memset(0) the destination buffer so we don't leave any
uninitialized data.
Fixes#19462
Signed-off-by: David Goulet <dgoulet@torproject.org>
This hack provides a way to make sure we can see coverage from
test-switch-id. If you set OVERRIDE_GCDA_PERMISSIONS_HACK, we
temporarily make the .gcda files mode 0666 before we run the
test scripts, and then we set them to 0644 again afterwards.
That's necessary because the test_switch_id.sh script does a
setuid() to 'nobody' part way through, and drops the ability to
change its mind back.
Slow system can sometime take more than 10 seconds to reach the test
callsite resulting in the unit test failing when using time in the future or
in the past.
Fixes#19465
Signed-off-by: David Goulet <dgoulet@torproject.org>
base16_decodes() now returns the number of decoded bytes. It's interface
changes from returning a "int" to a "ssize_t". Every callsite now checks the
returned value.
Fixes#14013
Signed-off-by: David Goulet <dgoulet@torproject.org>
realloc()ing a thing in order to try to save memory on it just
doesn't make sense with today's allocators. Instead, let's use the
fact that whenever we decompress something, either it isn't too big,
or we chop it up, or we reallocate it.
zlib 1.2 came out in 2003; earlier versions should be dead by now.
Our workaround code was only preventing us from using the gzip
encoding (if we decide to do so), and having some dead code linger
around in torgzip.c
The Autoconf macro AC_USE_SYSTEM_EXTENSIONS defines preprocessor macros
which turn on extensions to C and POSIX. The macro also makes it easier
for developers to use the extensions without needing (or forgetting) to
define them manually.
The macro can be safely used because it was introduced in Autoconf 2.60
and Tor requires Autoconf 2.63 and above.
When deleting an ephemeral HS, we were only iterating on circuit with an
OPEN state. However, it could be possible that an intro point circuit didn't
reached the open state yet.
This commit makes it that we close the circuit regardless of its state
except if it was already marked for close.
Fixes#18604
Signed-off-by: David Goulet <dgoulet@torproject.org>
The FetchHidServDescriptors check was placed before the descriptor cache
lookup which made the option not working because it was never using the
cache in the first place.
Fixes#18704
Patched-by: twim
Signef-off-by: David Goulet <dgoulet@torproject.org>
There's accessors to get at things, but it ends up being rather
cumbersome. The only place where behavior should change is that the
code will fail instead of attempting to generate a new DH key if our
internal sanity check fails.
Like the previous commit, this probably breaks snapshots prior to pre5.
Instead of `ERR_remove_thread_state()` having a modified prototype, it
now has the old prototype and a deprecation annotation. Since it's
pointless to add extra complexity just to remain compatible with an old
OpenSSL development snapshot, update the code to work with 1.1.0pre5
and later.
When you divide an int by an int and get a fraction and _then_ cast
to double, coverity assumes that you meant to cast to a double
first.
In my fix for -Wfloat-conversion in 493499a339, I
did something like this that coverity didn't like.
Instead, I'm taking another approach here.
Fixes CID 1232089, I hope.
This is a big-ish patch, but it's very straightforward. Under this
clang warning, we're not actually allowed to have a global variable
without a previous extern declaration for it. The cases where we
violated this rule fall into three roughly equal groups:
* Stuff that should have been static.
* Stuff that was global but where the extern was local to some
other C file.
* Stuff that was only global when built for the unit tests, that
needed a conditional extern in the headers.
The first two were IMO genuine problems; the last is a wart of how
we build tests.
This warning triggers on silently promoting a float to a double. In
our code, it's just a sign that somebody used a float by mistake,
since we always prefer double.
This warning, IIUC, means that the compiler doesn't like it when it
sees a NULL check _after_ we've already dereferenced the
variable. In such cases, it considers itself free to eliminate the
NULL check.
There are a couple of tricky cases:
One was the case related to the fact that tor_addr_to_in6() can
return NULL if it gets a non-AF_INET6 address. The fix was to
create a variant which asserts on the address type, and never
returns NULL.
This is a fairly easy way for us to get our test coverage up on
compat_threads.c and workqueue.c -- I already implemented these
tests, so we might as well enable them.
Previously, we used !directory_fetches_from_authorities() to predict
that we would tunnel connections. But the rules have changed
somewhat over the course of 0.2.8
So, back long ago, XXX012 meant, "before Tor 0.1.2 is released, we
had better revisit this comment and fix it!"
But we have a huge pile of such comments accumulated for a large
number of released versions! Not cool.
So, here's what I tried to do:
* 0.2.9 and 0.2.8 are retained, since those are not yet released.
* XXX+ or XXX++ or XXX++++ or whatever means, "This one looks
quite important!"
* The others, after one-by-one examination, are downgraded to
plain old XXX. Which doesn't mean they aren't a problem -- just
that they cannot possibly be a release-blocking problem.
There was a > that should have been an ==, and a missing !. These
together prevented us from issuing a warning in the case that a
nickname matched an Unnamed node only.
Fixes bug 19203; bugfix on 0.2.3.1-alpha.
There are a few places where we want to disable a warning: for
example, when it's impossible to call a legacy API without
triggering it, or when it's impossible to include an external header
without triggering it.
This pile of macros uses GCC's c99 _Pragma support, plus the usual
macro trickery, to enable and disable warnings.
Remove support for "GET /tor/bytes.txt" DirPort request, and
"GETINFO dir-usage" controller request, which were only available
via a compile-time option in Tor anyway.
Feature was added in 0.2.2.1-alpha. Resolves ticket 19035.
I introduced this bug when I moved signing_key_cert into
signed_descriptor_t. Bug not in any released Tor. Fixes bug 19175, and
another case of 19128.
Just like signed_descriptor_from_routerinfo(), routerlist_reparse_old()
copies the fields from one signed_descriptor_t to another, and then
clears the fields from the original that would have been double-freed by
freeing the original. But when I fixed the s_d_f_r() bug [#19128] in
50cbf22099, I missed the fact that the code was duplicated in
r_p_o().
Duplicated code strikes again!
For a longer-term solution here, I am not only adding the missing fix to
r_p_o(): I am also extracting the duplicated code into a new function.
Many thanks to toralf for patiently sending me stack traces until
one made sense.
If OpenSSL fails to generate an RSA key, do not retain a dangling
pointer to the previous (uninitialized) key value. The impact here
should be limited to a difficult-to-trigger crash, if OpenSSL is
running an engine that makes key generation failures possible, or if
OpenSSL runs out of memory. Fixes bug 19152; bugfix on
0.2.1.10-alpha. Found by Yuan Jochen Kang, Suman Jana, and Baishakhi
Ray.
This is potentially scary stuff, so let me walk through my analysis.
I think this is a bug, and a backport candidate, but not remotely
triggerable in any useful way.
Observation 1a:
Looking over the OpenSSL code here, the only way we can really fail in
the non-engine case is if malloc() fails. But if malloc() is failing,
then tor_malloc() calls should be tor_asserting -- the only way that an
attacker could do an exploit here would be to figure out some way to
make malloc() fail when openssl does it, but work whenever Tor does it.
(Also ordinary malloc() doesn't fail on platforms like Linux that
overcommit.)
Observation 1b:
Although engines are _allowed_ to fail in extra ways, I can't find much
evidence online that they actually _do_ fail in practice. More evidence
would be nice, though.
Observation 2:
We don't call crypto_pk_generate*() all that often, and we don't do it
in response to external inputs. The only way to get it to happen
remotely would be by causing a hidden service to build new introduction
points.
Observation 3a:
So, let's assume that both of the above observations are wrong, and the
attacker can make us generate a crypto_pk_env_t with a dangling pointer
in its 'key' field, and not immediately crash.
This dangling pointer will point to what used to be an RSA structure,
with the fields all set to NULL. Actually using this RSA structure,
before the memory is reused for anything else, will cause a crash.
In nearly every function where we call crypto_pk_generate*(), we quickly
use the RSA key pointer -- either to sign something, or to encode the
key, or to free the key. The only exception is when we generate an
intro key in rend_consider_services_intro_points(). In that case, we
don't actually use the key until the intro circuit is opened -- at which
point we encode it, and use it to sign an introduction request.
So in order to exploit this bug to do anything besides crash Tor, the
attacker needs to make sure that by the time the introduction circuit
completes, either:
* the e, d, and n BNs look valid, and at least one of the other BNs is
still NULL.
OR
* all 8 of the BNs must look valid.
To look like a valid BN, *they* all need to have their 'top' index plus
their 'd' pointer indicate an addressable region in memory.
So actually getting useful data of of this, rather than a crash, is
going to be pretty damn hard. You'd have to force an introduction point
to be created (or wait for one to be created), and force that particular
crypto_pk_generate*() to fail, and then arrange for the memory that the
RSA points to to in turn point to 3...8 valid BNs, all by the time the
introduction circuit completes.
Naturally, the signature won't check as valid [*], so the intro point
will reject the ESTABLISH_INTRO cell. So you need to _be_ the
introduction point, or you don't actually see this information.
[*] Okay, so if you could somehow make the 'rsa' pointer point to a
different valid RSA key, then you'd get a valid signature of an
ESTABLISH_INTRO cell using a key that was supposed to be used for
something else ... but nothing else looks like that, so you can't use
that signature elsewhere.
Observation 3b:
Your best bet as an attacker would be to make the dangling RSA pointer
actually contain a fake method, with a fake RSA_private_encrypt
function that actually pointed to code you wanted to execute. You'd
still need to transit 3 or 4 pointers deep though in order to make that
work.
Conclusion:
By 1, you probably can't trigger this without Tor crashing from OOM.
By 2, you probably can't trigger this reliably.
By 3, even if I'm wrong about 1 and 2, you have to jump through a pretty
big array of hoops in order to get any kind of data leak or code
execution.
So I'm calling it a bug, but not a security hole. Still worth
patching.
Fortunately, the arithmetic cannot actually overflow, so long as we
*always* check for the size of potentially hostile input before
copying it. I think we do, though. We do check each line against
MAX_LINE_LENGTH, and each object name or object against
MAX_UNPARSED_OBJECT_SIZE, both of which are 128k. So to get this
overflow, we need to have our memarea allocated way way too high up
in RAM, which most allocators won't actually do.
Bugfix on 0.2.1.1-alpha, where memarea was introduced.
Found by Guido Vranken.
Previously, if the header was present, we'd proceed even if the
function wasn't there.
Easy fix for bug 19161. A better fix would involve trying harder to
find libscrypt_scrypt.
AddressSanitizer's (ASAN) SIGSEGV handler overrides the backtrace
handler and prevents it from printing its backtrace. The output of ASAN
is different from what 'bt_test.py' expects and causes backtrace test
failures.
The 'allow_user_segv_handler' option allows applications to set their
own SIGSEGV handler but is not supported by older GCC versions. These
older GCC versions do support the 'handle_segv' which prevents ASAN from
setting its SIGSEGV handler.
Now that the field exists in signed_descriptor_t, we need to make
sure we free it when we free a signed_descriptor_t, and we need to
make sure that we don't free it when we convert a routerinfo_t to a
signed_descriptor_t.
But not in any released Tor. I found this while working on #19128.
One problem: I don't see how this could cause 19128.
We use a pretty specific pair of autoconf tests here to make sure
that we only add this code when:
a) a 64-bit signed multiply fails to link,
AND
b) the same 64-bit signed multiply DOES link correctly when
__mulodi4 is defined.
Closes ticket 19079.
We need to define this function when compiling with clang -m32 -ftrapv,
since otherwise we get link errors, since apparently some versions
of libclang_rt.builtins don't define a version of it that works? Or
clang doesn't know to look for it?
This definition is taken from the LLVM source at
https://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/builtins/mulodi4.c
I've also included the license (dual BSD-ish/MIT-ish).
With the fix for #17150, I added a duplicate certificate here. Here
I remove the original location in 0.2.8. (I wouldn't want to do
that in 027, due to the amount of authority-voting-related code
drift.)
Closes 19073.
This API change makes it so that routerinfo_incompatible...() no
longer takes a routerinfo_t, so that it's obvious that it should
only look at fields from the signed_descriptor_t.
This change should prevent a recurrence of #17150.
We need this field to be in signed_descriptor_t so that
routerinfo_incompatible_with_extrainfo can work correctly (#17150).
But I don't want to move it completely in this patch, since a great
deal of the code that messes with it has been in flux since 0.2.7,
when this ticket was opened. I should open another ticket about
removing the field from routerinfo_t and extrainfo_t later on.
This patch fixes no actual behavior.
The routerinfo we pass to routerinfo_incompatible_with_extrainfo is
the latest routerinfo for the relay. The signed_descriptor_t, on
the other hand, is the signed_descriptor_t that corresponds to the
extrainfo. That means we should be checking the digest256 match
with that signed_descriptor_t, not with the routerinfo.
Fixes bug 17150 (and 19017); bugfix on 0.2.7.2-alpha.
When parsing detached signature, we make sure that we use the length of the
digest algorithm instead of an hardcoded DIGEST256_LEN in order to avoid
comparing bytes out of bound with a smaller digest length such as SHA1.
Fixes#19066
Signed-off-by: David Goulet <dgoulet@torproject.org>
We know there are overflows in curve25519-donna-c32, so we'll have
to have that one be fwrapv.
Only apply the asan, ubsan, and trapv options to the code that does
not need to run in constant time. Those options introduce branches
to the code they instrument.
(These introduced branches should never actually be taken, so it
might _still_ be constant time after all, but branch predictors are
complicated enough that I'm not really confident here. Let's aim for
safety.)
Closes 17983.
Make directory authorities write the v3-status-votes file out
to disk earlier in the consensus process, so we have the votes
even if we abort the consensus process later on.
Resolves ticket 19036.
The goal here is to provide a way to decouple pieces of the code
that want to learn "when something happens" from those that realize
that it has happened.
The implementation here consists of a generic backend, plus a set of
macros to define and implement a set of type-safe frontends.
In dirserv_compute_performance_thresholds, we allocate arrays based
on the length of 'routers', a list of routerinfo_t, but loop over
the nodelist. The 'routers' list may be shorter when relays were
filtered by routers_make_ed_keys_unique, leading to an out-of-bounds
write on directory authorities.
This bug was originally introduced in 26e89742, but it doesn't look
possible to trigger until routers_make_ed_keys_unique was introduced
in 13a31e72.
Fixes bug 19032; bugfix on tor 0.2.8.2-alpha.
This was one of our longest functions, at 600 lines. It makes a nice
table-driven URL-based function instead.
The code is a bit ugly, it leave the indentation as it is in hopes of
making pending directory.c changes easier to merge. Later we can
clean up the indentation.
Also, remove unused mallinfo export code from directory.c
Closes ticket 16698
Previously, we were using the generic schedule for some downloads,
and the consensus schedule for others.
Resolves ticket 18816; fix on fddb814fe in 0.2.4.13-alpha.
We used to be locked in to the "tap" handshake length, and now we can
handle better handshakes like "ntor".
Resolves ticket 18998.
I checked that relay_send_command_from_edge() behaves fine when you
hand it a payload with length 0. Clients behave fine too, since current
clients remain strict about the required length in the rendezvous2 cells.
(Clients will want to become less strict once they have an alternate
format that they're willing to receive.)
we should avoid launching a consensus fetch if we don't want one,
but if we do end up with an extra one, we should let the other checks
take care of it.
We'll back off from the request in connection_ap_handshake_attach_circuit,
or cancel it in connection_dir_close_consensus_fetches, and those are the
only places we need to check.
Tor stores client authorization cookies in two slightly different forms.
The service's client_keys file has the standard base64-encoded cookie,
including two chars of padding. The hostname file and the client remove
the two padding chars, and store an auth type flag in the unused bits.
The distinction makes no sense. Refactor all decoding to use the same
function, which will accept either form, and use a helper function for
encoding the truncated format.
This improves client anonymity and avoids directory header tampering.
The extra load on the authorities should be offset by the fallback
directories feature.
This also simplifies the fixes to #18809.
Delete an unnecessary check for non-preferred IP versions.
Allows clients which can't reach any directories of their
preferred IP address version to get directory documents.
Patch on #17840 in 0.2.8.1-alpha.
