Rely on onion_populate_cpath to check that we're only using
TAP for the rare hidden service cases.
Check and log if handshakes only support TAP when they should support
ntor.
When a client connects to an intro point not in the client's consensus,
or a hidden service connects to a rend point not in the hidden service's
consensus, we are stuck with using TAP, because there is no ntor link
specifier.
This bug had existed since 0.2.4.7-alpha, but now that we have
FallbackDirs by default, it actually matters.
Fixes bug 19947; bugfix on 0.2.4.7-alpha or maybe 0.2.8.1-alpha.
Rubiate wrote the patch; teor wrote the changes file.
Longer and more explicit log message so we don't confuse users with behind NAT with working configurations and state that public IP addresses only should be provided with "Address", won't work with internal addresses.
These functions were there so that we could abstract the differences
between evbuffer and buf_t. But with the bufferevent removal, this
no longer serves a purpose.
If we know a node's version, and it can't do ntor, consider it not running.
If we have a node's descriptor, and it doesn't have a valid ntor key,
consider it not running.
Refactor these checks so they're consistent between authorities and clients.
Before, they checked for version 0.2.4.18-rc or later, but this
would not catch relays without version lines, or buggy or malicious
relays missing an ntor key.
This fixes#19608, allowing IPv6-only clients to use
microdescriptors, while preserving the ability of bridge clients
to have some IPv4 bridges and some IPv6 bridges.
Fix on c281c036 in 0.2.8.2-alpha.
I grepped and hand-inspected the "it's" instances, to see if any
were supposed to be possessive. While doing that, I found a
"the the", so I grepped to see if there were any more.
Keep the base16 representation of the RSA identity digest in the commit object
so we can use it without using hex_str() or dynamically encoding it everytime
we need it. It's used extensively in the logs for instance.
Fixes#19561
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch also updates a comment in the same function for accuracy.
Found by Coverity issue 1362985. Partily fixes#19567.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
Our sandboxing code would not allow us to write to stats/hidserv-stats,
causing tor to abort while trying to write stats. This was previously
masked by bug#19556.
When sandboxing is enabled, we could not write any stats to disk.
check_or_create_data_subdir("stats"), which prepares the private stats
directory, calls check_private_dir(), which also opens and not just stats() the
directory. Therefore, we need to also allow open() for the stats dir in our
sandboxing setup.
The *get* state query functions for the SRVs now only return const pointers
and the DEL action needs to be used to delete the SRVs from the state.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This patch makes us retain the intermediate list of K=V entries for
the duration of computing our vote, and lets us use that list with
a new function in order to look up parameters before the consensus
is published.
We can't actually use this function yet because of #19011: our
existing code to do this doesn't actually work, and we'll need a new
consensus method to start using it.
Closes ticket #19012.
Commit and reveal length macro changed from int to unsigned long int
(size_t) because of the sizeof().
Signed-off-by: David Goulet <dgoulet@torproject.org>
See ticket #19132 for the clang/llvm warning.
Since voting_schedule is a global static struct, it will be initialized
to zero even without explicitly initializing it with {0}.
This is what the C spec says:
If an object that has automatic storage duration is not initialized
explicitly, its value is indeterminate. If an object that has static
storage duration is not initialized explicitly, then:
— if it has pointer type, it is initialized to a null pointer;
— if it has arithmetic type, it is initialized to (positive or unsigned) zero;
— if it is an aggregate, every member is initialized (recursively) according to these rules;
— if it is a union, the first named member is initialized (recursively) according to these rules.
Code has been changed so every RSA fingerprint for a commit in our state is
validated before being used. This fixes the unit tests by mocking one of the
key function and updating the hardcoded state string.
Also, fix a time parsing overflow on platforms with 32bit time_t
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
We assert on it using the ASSERT_COMMIT_VALID() macro in critical places
where we use them expecting a commit to be valid.
Signed-off-by: David Goulet <dgoulet@torproject.org>
The prop250 code used the RSA identity key fingerprint to index commit in a
digestmap instead of using the digest.
To behavior change except the fact that we are actually using digestmap
correctly.
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit makes it that tor now uses the shared random protocol by
initializing the subsystem.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
One of the last piece that parses the votes and consensus in order to update
our state and make decision for the SR values.
We need to inform the SR subsystem when we set the current consensus because
this can be called when loaded from file or downloaded from other authorities
or computed.
The voting schedule is used for the SR timings since we are bound to the
voting system.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
This commit adds the commit(s) line in the vote as well as the SR values. It
also has the mechanism to add the majority SRVs in the consensus.
Signed-off-by: George Kadianakis <desnacked@riseup.net>
Signed-off-by: David Goulet <dgoulet@torproject.org>
This adds the logic of commit and SR values generation. Furthermore, the
concept of a protocol run is added that is commit is generated at the right
time as well as SR values which are also rotated before a new protocol run.
Signed-off-by: George Kadianakis <desnacked@riseup.net>
Signed-off-by: David Goulet <dgoulet@torproject.org>
From 0.2.7.2-alpha onwards, Exits would reject all the IP addresses
they knew about in their exit policy. But this may have disclosed
addresses that were otherwise unlisted.
Now, only advertised addresses are rejected by default by
ExitPolicyRejectPrivate. All known addresses are only rejected when
ExitPolicyRejectLocalInterfaces is explicitly set to 1.
If we manually remove fallbacks in C by adding '/*' and '*/' on separate
lines, stem still parses them as being present, because it only looks at
the start of a line.
Add a comment to this effect in the generated source code.
Remove a fallback that changed its fingerprint after it was listed
This happened after to a software update:
https://lists.torproject.org/pipermail/tor-relays/2016-June/009473.html
Remove a fallback that changed IPv4 address
Remove two fallbacks that were slow to deliver consensuses,
we can't guarantee they'll be fast in future.
Blacklist all these fallbacks until operators confirm they're stable.
This commit introduces two new files with their header.
"shared_random.c" contains basic functions to initialize the state and allow
commit decoding for the disk state to be able to parse them from disk.
"shared_random_state.c" contains everything that has to do with the state
for both our memory and disk. Lots of helper functions as well as a
mechanism to query the state in a synchronized way.
Signed-off-by: David Goulet <dgoulet@torproject.org>
Signed-off-by: George Kadianakis <desnacked@riseup.net>
base16_decodes() now returns the number of decoded bytes. It's interface
changes from returning a "int" to a "ssize_t". Every callsite now checks the
returned value.
Fixes#14013
Signed-off-by: David Goulet <dgoulet@torproject.org>
When deleting an ephemeral HS, we were only iterating on circuit with an
OPEN state. However, it could be possible that an intro point circuit didn't
reached the open state yet.
This commit makes it that we close the circuit regardless of its state
except if it was already marked for close.
Fixes#18604
Signed-off-by: David Goulet <dgoulet@torproject.org>
The FetchHidServDescriptors check was placed before the descriptor cache
lookup which made the option not working because it was never using the
cache in the first place.
Fixes#18704
Patched-by: twim
Signef-off-by: David Goulet <dgoulet@torproject.org>
When you divide an int by an int and get a fraction and _then_ cast
to double, coverity assumes that you meant to cast to a double
first.
In my fix for -Wfloat-conversion in 493499a339, I
did something like this that coverity didn't like.
Instead, I'm taking another approach here.
Fixes CID 1232089, I hope.
This is a big-ish patch, but it's very straightforward. Under this
clang warning, we're not actually allowed to have a global variable
without a previous extern declaration for it. The cases where we
violated this rule fall into three roughly equal groups:
* Stuff that should have been static.
* Stuff that was global but where the extern was local to some
other C file.
* Stuff that was only global when built for the unit tests, that
needed a conditional extern in the headers.
The first two were IMO genuine problems; the last is a wart of how
we build tests.
This warning triggers on silently promoting a float to a double. In
our code, it's just a sign that somebody used a float by mistake,
since we always prefer double.
This warning, IIUC, means that the compiler doesn't like it when it
sees a NULL check _after_ we've already dereferenced the
variable. In such cases, it considers itself free to eliminate the
NULL check.
There are a couple of tricky cases:
One was the case related to the fact that tor_addr_to_in6() can
return NULL if it gets a non-AF_INET6 address. The fix was to
create a variant which asserts on the address type, and never
returns NULL.
Previously, we used !directory_fetches_from_authorities() to predict
that we would tunnel connections. But the rules have changed
somewhat over the course of 0.2.8
So, back long ago, XXX012 meant, "before Tor 0.1.2 is released, we
had better revisit this comment and fix it!"
But we have a huge pile of such comments accumulated for a large
number of released versions! Not cool.
So, here's what I tried to do:
* 0.2.9 and 0.2.8 are retained, since those are not yet released.
* XXX+ or XXX++ or XXX++++ or whatever means, "This one looks
quite important!"
* The others, after one-by-one examination, are downgraded to
plain old XXX. Which doesn't mean they aren't a problem -- just
that they cannot possibly be a release-blocking problem.
There was a > that should have been an ==, and a missing !. These
together prevented us from issuing a warning in the case that a
nickname matched an Unnamed node only.
Fixes bug 19203; bugfix on 0.2.3.1-alpha.
Remove support for "GET /tor/bytes.txt" DirPort request, and
"GETINFO dir-usage" controller request, which were only available
via a compile-time option in Tor anyway.
Feature was added in 0.2.2.1-alpha. Resolves ticket 19035.
I introduced this bug when I moved signing_key_cert into
signed_descriptor_t. Bug not in any released Tor. Fixes bug 19175, and
another case of 19128.
Just like signed_descriptor_from_routerinfo(), routerlist_reparse_old()
copies the fields from one signed_descriptor_t to another, and then
clears the fields from the original that would have been double-freed by
freeing the original. But when I fixed the s_d_f_r() bug [#19128] in
50cbf22099, I missed the fact that the code was duplicated in
r_p_o().
Duplicated code strikes again!
For a longer-term solution here, I am not only adding the missing fix to
r_p_o(): I am also extracting the duplicated code into a new function.
Many thanks to toralf for patiently sending me stack traces until
one made sense.
Now that the field exists in signed_descriptor_t, we need to make
sure we free it when we free a signed_descriptor_t, and we need to
make sure that we don't free it when we convert a routerinfo_t to a
signed_descriptor_t.
But not in any released Tor. I found this while working on #19128.
One problem: I don't see how this could cause 19128.
With the fix for #17150, I added a duplicate certificate here. Here
I remove the original location in 0.2.8. (I wouldn't want to do
that in 027, due to the amount of authority-voting-related code
drift.)
Closes 19073.
This API change makes it so that routerinfo_incompatible...() no
longer takes a routerinfo_t, so that it's obvious that it should
only look at fields from the signed_descriptor_t.
This change should prevent a recurrence of #17150.
We need this field to be in signed_descriptor_t so that
routerinfo_incompatible_with_extrainfo can work correctly (#17150).
But I don't want to move it completely in this patch, since a great
deal of the code that messes with it has been in flux since 0.2.7,
when this ticket was opened. I should open another ticket about
removing the field from routerinfo_t and extrainfo_t later on.
This patch fixes no actual behavior.
The routerinfo we pass to routerinfo_incompatible_with_extrainfo is
the latest routerinfo for the relay. The signed_descriptor_t, on
the other hand, is the signed_descriptor_t that corresponds to the
extrainfo. That means we should be checking the digest256 match
with that signed_descriptor_t, not with the routerinfo.
Fixes bug 17150 (and 19017); bugfix on 0.2.7.2-alpha.
When parsing detached signature, we make sure that we use the length of the
digest algorithm instead of an hardcoded DIGEST256_LEN in order to avoid
comparing bytes out of bound with a smaller digest length such as SHA1.
Fixes#19066
Signed-off-by: David Goulet <dgoulet@torproject.org>
We know there are overflows in curve25519-donna-c32, so we'll have
to have that one be fwrapv.
Only apply the asan, ubsan, and trapv options to the code that does
not need to run in constant time. Those options introduce branches
to the code they instrument.
(These introduced branches should never actually be taken, so it
might _still_ be constant time after all, but branch predictors are
complicated enough that I'm not really confident here. Let's aim for
safety.)
Closes 17983.
Make directory authorities write the v3-status-votes file out
to disk earlier in the consensus process, so we have the votes
even if we abort the consensus process later on.
Resolves ticket 19036.
In dirserv_compute_performance_thresholds, we allocate arrays based
on the length of 'routers', a list of routerinfo_t, but loop over
the nodelist. The 'routers' list may be shorter when relays were
filtered by routers_make_ed_keys_unique, leading to an out-of-bounds
write on directory authorities.
This bug was originally introduced in 26e89742, but it doesn't look
possible to trigger until routers_make_ed_keys_unique was introduced
in 13a31e72.
Fixes bug 19032; bugfix on tor 0.2.8.2-alpha.
This was one of our longest functions, at 600 lines. It makes a nice
table-driven URL-based function instead.
The code is a bit ugly, it leave the indentation as it is in hopes of
making pending directory.c changes easier to merge. Later we can
clean up the indentation.
Also, remove unused mallinfo export code from directory.c
Closes ticket 16698
Previously, we were using the generic schedule for some downloads,
and the consensus schedule for others.
Resolves ticket 18816; fix on fddb814fe in 0.2.4.13-alpha.
We used to be locked in to the "tap" handshake length, and now we can
handle better handshakes like "ntor".
Resolves ticket 18998.
I checked that relay_send_command_from_edge() behaves fine when you
hand it a payload with length 0. Clients behave fine too, since current
clients remain strict about the required length in the rendezvous2 cells.
(Clients will want to become less strict once they have an alternate
format that they're willing to receive.)
we should avoid launching a consensus fetch if we don't want one,
but if we do end up with an extra one, we should let the other checks
take care of it.
We'll back off from the request in connection_ap_handshake_attach_circuit,
or cancel it in connection_dir_close_consensus_fetches, and those are the
only places we need to check.
Tor stores client authorization cookies in two slightly different forms.
The service's client_keys file has the standard base64-encoded cookie,
including two chars of padding. The hostname file and the client remove
the two padding chars, and store an auth type flag in the unused bits.
The distinction makes no sense. Refactor all decoding to use the same
function, which will accept either form, and use a helper function for
encoding the truncated format.
This improves client anonymity and avoids directory header tampering.
The extra load on the authorities should be offset by the fallback
directories feature.
This also simplifies the fixes to #18809.
Delete an unnecessary check for non-preferred IP versions.
Allows clients which can't reach any directories of their
preferred IP address version to get directory documents.
Patch on #17840 in 0.2.8.1-alpha.
After #17840 in 0.2.8.1-alpha, we incorrectly chose an IPv4
address for all DIRIND_ONEHOP directory connections,
even if the routerstatus didn't have an IPv4 address.
This likely affected bridge clients with IPv6 bridges.
Resolves#18921.
The problem is that "q" is always set on the first iteration even
if the question is not a supported question. This set of "q" is
not necessary, and will be handled after exiting the loop if there
if a supported q->type was found.
[Changes file by nickm]
lease enter the commit message for your changes. Lines starting
Decide to advertise begindir support in a similar way to how
we decide to advertise DirPort.
Fix up the associated descriptor-building unit tests.
Resolves#18616, bugfix on 0c8e042c30 in #12538 in 0.2.8.1-alpha.
Apparently somewhere along the line we decided that MIN might be
missing.
But we already defined it (if it was missing) in compat.h, which
everybody includes.
Closes ticket 18889.
When we connect to a hidden service as a client we may need three internal
circuits, one for the descriptor retrieval, introduction, and rendezvous.
Let's try to make sure we have them. Closes#13239.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Unlike tor_assert(), these macros don't abort the process. They're
good for checking conditions we want to warn about, but which don't
warrant a full crash.
This commit also changes the default implementation for
tor_fragile_assert() to tor_assert_nonfatal_unreached_once().
Closes ticket 18613.
When the directory authorities refuse a bad relay's descriptor,
encourage the relay operator to contact us. Many relay operators
won't notice this line in their logs, but it's a win if even a
few learn why we don't like what their relay was doing.
Resolves ticket 18760.
I didn't specify a contact mechanism (e.g. an email address), because
every time we've done that in the past, a few years later we noticed
that the code was pointing people to an obsolete contact address.
This changes simply renames them by removing "Testing" in front of them and
they do not require TestingTorNetwork to be enabled anymore.
Fixes#18481
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Only when we were actually flushing the cell stats to a controller
would we free them. Thus, they could stay in RAM even after the
circuit was freed (eg if we didn't have any controllers).
Fixes bug 18673; bugfix on 0.2.5.1-alpha.
They incorrectly summarized what the function was planning to do,
leading to wrong behavior like making an http request to an orport,
or making a begindir request to a dirport.
This change backs out some of the changes made in commit e72cbf7a, and
most of the changes made in commit ba6509e9.
This patch resolves bug 18625. There more changes I want to make
after this one, for code clarity.
This change allows us to simplify path selection for clients, and it
should have minimal effect in practice since >99% of Guards already have
the Stable flag. Implements ticket 18624.
Commit e72cbf7a4 introduced a change to directory_initiate_command_rend()
that made tor use the ORPort when making a directory request to the DirPort.
The primary consequence was that a relay couldn't selftest its DirPort thus
failing to work and join the network properly.
The main issue was we were always considering an anonymized connection to be
an OR connection which is not true.
Fixes#18623
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Regardless of the setting of ExtendAllowPrivateAddresses.
This fixes a bug with pluggable transports that ignore the
(potentially private) address in their bridge line.
Fixes bug 18517; bugfix on 23b088907f in tor-0.2.8.1-alpha.
When requesting extrainfo descriptors from a trusted directory
server, check whether it is an authority or a fallback directory
which supports extrainfo descriptors.
Fixes bug 18489; bugfix on 90f6071d8d in tor-0.2.4.7-alpha.
Reported by "atagar", patch by "teor".
Make it clearer that they are about outgoing connection attempts.
Specify the options involved where they were missing from one log
message.
Clarify a comment.
Downgrade logs and backtraces about IP versions to
info-level. Only log backtraces once each time tor runs.
Assists in diagnosing bug 18351; bugfix on c3cc8e16e in
tor-0.2.8.1-alpha.
Reported by "sysrqb" and "Christian", patch by "teor".
commit edeba3d4 removed a switch, but left the "break" lines in
from that switch. fortunately the resulting behavior was not wrong,
since there was an outer switch that it was ok to break from.
no actual changes here -- but the new indenting makes it clear
that the fixes in #18332 were not as good as they should have been.
the next commit will deal with that.
We've got to make sure that every single subsequent calculation in
dirserv_generate_networkstatus_vote_obj() are based on the list of
routerinfo_t *after* we've removed possible duplicates, not before.
Fortunately, none of the functions that were taking a routerlist_t
as an argument were actually using any fields other than this list
of routers.
Resolves issue 18318.DG3.
I had a half-built mechanism to track, during the voting process,
whether the Ed25519 value (or lack thereof) reflected a true
consensus among the authorities. But we never actually inserted this
field in the consensus.
The key idea here is that we first attempt to match up votes by pairs
of <Ed,RSA>, where <Ed> can be NULL if we're told that there is no
Ed key. If this succeeds, then we can treat all those votes as 'a
consensus for Ed'. And we can include all other votes with a
matching RSA key and no statement about Ed keys as being "also about
the same relay."
After that, we look for RSA keys we haven't actually found an entry
for yet, and see if there are enough votes for them, NOT considering
Ed keys. If there are, we match them as before, but we treat them
as "not a consensus about ed".
When we include an entry in a consensus, if it does not reflect a
consensus about ed keys, then we include a new NoEdConsensus flag on
it.
This is all only for consensus method 22 or later.
Also see corresponding dir-spec patch.
When generating a vote, and we have two routerinfos with the same ed
key, omit the one published earlier.
This was supposed to have been solved by key pinning, but when I
made key pinning optional, I didn't realize that this would jump up
and bite us. It is part of bug 18318, and the root cause of 17668.
If we're a server with no address configured, resolve_my_hostname
will need this. But not otherwise. And the preseeding itself can
consume a few seconds if like tails we have no resolvers.
Fixes bug 18548.
I didn't want to grant blanket permissions for chmod() and chown(),
so here's what I had to do:
* Grant open() on all parent directories of a unix socket
* Write code to allow chmod() and chown() on a given file only.
* Grant chmod() and chown() on the unix socket.
This is a part of a fix for 18253; bugfix on 0.2.8.1-alpha.
Alternatively, we could permit chmod/chown in the sandbox, but I
really don't like giving the sandbox permission to alter
permissions.
Launching 7 descriptor fetches makes a connection to each HSDir that is 6
and the seventh one fails to pick an HSDir because they are all being used
already so it was killing all pending connections at once.
Fixes#15937
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
When we made HidServDirectoryV2 always 1, we removed the situation
where a relay could choose not to be an HSDir. Now simplify the
rest of the code to reflect this decision.
(We have to remove two apparently unrelated free() calls in the unit
tests, since they used to free stuff that we created as a side effect
of calling router_get_my_routerinfo(), and now we no longer call that.)
This simplifies relay behavior, because the relay offers the hsdir
functionality independent of whether the directory authorities have
decided this relay is suitable for clients to use yet.
Implements ticket 18332.
The transproxy feature is only enabled when __FreeBSD__ is defined, and
only regular FreeBSD does that. Change this to __FreeBSD_kernel__ which
is defined on derivatives as well.
This enables the relevant options/validate__transproxy test on FreeBSD
derivatives.
Allow fallback directories which have been stable for 7 days
to work around #18050, which causes relays to submit descriptors
with 0 DirPorts when restarted. (Particularly during Tor version
upgrades.)
Ignore low fallback directory count in alpha builds.
Set the target count to 50.
This is in accordance with our usual policy against freelists,
now that working allocators are everywhere.
It should also make memarea.c's coverage higher.
I also doubt that this code ever helped performance.
Previously, I had left in some debugging code with /*XXX*/ after it,
which nobody noticed. Live and learn! Next time I will use /*XXX
DO NOT COMMIT*/ or something.
We need to define a new consensus method for this; consensus method
21 shouldn't actually be used.
Fixes bug 17702; bugfix on 0.2.7.2-alpha.
We already write out bootstrapping progress (see bug 9927) per new
microdesc batch. There's no need to do a full "I learned some more
directory information, but not enough to..." line each time too.
Now, when a user who has set EntryNodes finishes bootstrapping, Tor
automatically repopulates the guard set based on this new directory
information. Fixes bug 16825; bugfix on 0.2.3.1-alpha.
Originally it can overflow in some weird cases. Now it should no longer
be able to do so.
Additionally, limit main's timers to 30 days rather than to 38 years;
we don't actually want any 38-year timers.
Closes bug 17682.
They are no longer "all" digests, but only the "common" digests.
Part of 17795.
This is an automated patch I made with a couple of perl one-liners:
perl -i -pe 's/crypto_digest_all/crypto_common_digests/g;' src/*/*.[ch]
perl -i -pe 's/\bdigests_t\b/common_digests_t/g;' src/*/*.[ch]
If unix socket was configured as listener (such as a ControlSocket or a
SocksPort unix socket), and tor was started as root but not configured
to switch to another user, tor would segfault while trying to string
compare a NULL value. Fixes bug 18261; bugfix on 0.2.8.1-alpha. Patch
by weasel.
When ClientPreferIPv6ORPort is auto, bridges prefer the configured
bridge ORPort address. Otherwise, they use the value of the option.
Other clients prefer IPv4 ORPorts if ClientPreferIPv6ORPort is auto.
When ClientPreferIPv6DirPort is auto, all clients prefer IPv4 DirPorts.
Modify callers to correctly handle these new NULL returns:
* fix assert in onion_extend_cpath
* warn and discard circuit in circuit_get_open_circ_or_launch
* warn, discard circuit, and tell controller in handle_control_extendcircuit
Bridge clients ignore ClientUseIPv6, acting as if it is always 1.
This preserves existing behaviour.
Make ClientPreferIPv6OR/DirPort auto by default:
* Bridge clients prefer IPv6 by default.
* Other clients prefer IPv4 by default.
This preserves existing behaviour.
ClientUseIPv4 0 tells tor to avoid IPv4 client connections.
ClientPreferIPv6DirPort 1 tells tor to prefer IPv6 directory connections.
Refactor policy for IPv4/IPv6 preferences.
Fix a bug where node->ipv6_preferred could become stale if
ClientPreferIPv6ORPort was changed after the consensus was loaded.
Update documentation, existing code, add unit tests.
node_get_all_orports and router_get_all_orports incorrectly used or_port
with IPv6 addresses. They now use ipv6_orport.
Also refactor and remove duplicated code.
Avoid using a pronoun where it makes comments unclear.
Avoid using gender for things that don't have it.
Avoid assigning gender to people unnecessarily.
Allow fallback directories which have been stable for 30 days
to work around #18050, which causes relays to submit descriptors
with 0 DirPorts when restarted. (Particularly during Tor version
upgrades.)
Ignore low fallback directory count in alpha builds.
Otherwise, relays publish a descriptor with DirPort 0 when the DirPort
reachability test takes longer than the ORPort reachability test.
Closes bug #18050. Reported by "starlight", patch by "teor".
Bugfix on 0.1.0.1-rc, commit a1f1fa6ab on 27 Feb 2005.
* DIGEST_SHA3_[256,512] added as supported algorithms, which do
exactly what is said on the tin.
* test/bench now benchmarks all of the supported digest algorithms,
so it's possible to see just how slow SHA-3 is, though the message
sizes could probably use tweaking since this is very dependent on
the message size vs the SHA-3 rate.
This will give relay operators the ability of disabling the caching of
directory data. In general, this should not be necessary, but on some
lower-resource systems it may beneficial.
I believe that the final SMARTLIST_DEL_CURRENT was sometimes
double-removing items that had already been removed by
connection_mark_unattached_ap or
connection_ap_handshake_attach_circuit().
The fix here is to prevent iteration over the list that other
functions might be modifying.
According to the POSIX standard the option value is a pointer to void
and the option length a socklen_t. The Windows implementation makes the
option value be a pointer to character and the option length an int.
Casting the option value to a pointer to void conforms to the POSIX
standard while the implicit cast to a pointer to character conforms to
the Windows implementation.
The casts of the option length to the socklen_t data type conforms to
the POSIX standard. The socklen_t data type is actually an alias of an
int so it also conforms to the Windows implementation.
It is AP-specific, so that's where it belongs. This shouldn't have
caused a bug, but due to #17876, we were never actually calling
connection_edge_about_to_close from connection_ap_about_to_close,
causing bug #17874 (aka bug #17752).
When a relay does not have an open directory port but it has an
orport configured and is accepting client connections then it can
now service tunnelled directory requests, too. This was already true
of relays with an dirport configured.
We also conditionally stop advertising this functionality if the
relay is nearing its bandwidth usage limit - same as how dirport
advertisement is determined.
Partial implementation of prop 237, ticket 12538
Applies the 6c443e987d fix to router_pick_directory_server_impl.
6c443e987d applied to directory servers chosen from the consensus,
and was:
"Tweak the 9969 fix a little
If we have busy nodes and excluded nodes, then don't retry with the
excluded ones enabled. Instead, wait for the busy ones to be nonbusy."
These IPv6 addresses must be quoted, because : is the port separator,
and "acce" is a valid hex block.
Add unit tests for assumed actions in IPv6 policies.
"Tor has included a feature to fetch the initial consensus from nodes
other than the authorities for a while now. We just haven't shipped a
list of alternate locations for clients to go to yet.
Reasons why we might want to ship tor with a list of additional places
where clients can find the consensus is that it makes authority
reachability and BW less important.
We want them to have been around and using their current key, address,
and port for a while now (120 days), and have been running, a guard,
and a v2 directory mirror for most of that time."
Features:
* whitelist and blacklist for an opt-in/opt-out trial.
* excludes BadExits, tor versions that aren't recommended, and low
consensus weight directory mirrors.
* reduces the weighting of Exits to avoid overloading them.
* places limits on the weight of any one fallback.
* includes an IPv6 address and orport for each FallbackDir, as
implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks
until #17840 is merged.)
* generated output includes timestamps & Onionoo URL for traceability.
* unit test ensures that we successfully load all included default
fallback directories.
Closes ticket #15775. Patch by "teor".
OnionOO script by "weasel", "teor", "gsathya", and "karsten".
* The option is now KeepBindCapabilities
* We now warn if the user specifically asked for KeepBindCapabilities
and we can't deliver.
* The unit tests are willing to start.
* Fewer unused-variable warnings.
* More documentation, fewer misspellings.
router_digest_is_fallback_dir returns 1 if the digest is in the
currently loaded list of fallback directories, and 0 otherwise.
This function is for future use.
Once tor is downloading a usable consensus, any other connection
attempts are not needed.
Choose a connection to keep, favouring:
* fallback directories over authorities,
* connections initiated earlier over later connections
Close all other connections downloading a consensus.
Prop210: Add attempt-based connection schedules
Existing tor schedules increment the schedule position on failure,
then retry the connection after the scheduled time.
To make multiple simultaneous connections, we need to increment the
schedule position when making each attempt, then retry a (potentially
simultaneous) connection after the scheduled time.
(Also change find_dl_schedule_and_len to find_dl_schedule, as it no
longer takes or returns len.)
Prop210: Add multiple simultaneous consensus downloads for clients
Make connections on TestingClientBootstrapConsensus*DownloadSchedule,
incrementing the schedule each time the client attempts to connect.
Check if the number of downloads is less than
TestingClientBootstrapConsensusMaxInProgressTries before trying any
more connections.
UseDefaultFallbackDirs enables any hard-coded fallback
directory mirrors. Default is 1, set it to 0 to disable fallbacks.
Implements ticket 17576.
Patch by "teor".
Update the code for IPv6 authorities and fallbacks for function
argument changes.
Update unit tests affected by the function argument changes in
the patch.
Add unit tests for authority and fallback:
* adding via a function
* line parsing
* adding default authorities
(Adding default fallbacks is unit tested in #15775.)
The internal memory allocation and history object counters of the
reputation code can be used to verify the correctness of (part of) the
code. Using these counters revealed an issue where the memory allocation
counter is not decreased when the bandwidth arrays are freed.
A new function ensures the memory allocation counter is decreased when a
bandwidth array is freed.
This commit also removes an unnecessary cast which was found while
working on the code.
* Since the variable is no longer modified, it should be called
'policy' instead of 'dest'. ("Dest" is short for
"destination".)
* Fixed the space issue that dgoulet found on the ticket.
* Fixed the comment a little. (We use the imperative for function
documentation.)
Some functions that use digest maps did not mention that the digests are
expected to have DIGEST_LEN bytes. This lead to buffer over-reads in the
past.
When an HS process an INTRODUCE2 cell, we didn't validate if the IP address
of the rendezvous point was a local address. If it's the case, we end up
wasting resources by trying to extend to a local address which fails since
we do not allow that in circuit_extend().
This commit now rejects a rendezvous point that has a local address once
seen at the hidden service side unless ExtendAllowPrivateAddresses is set.
Fixes#8976
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
There was a dead check when we made sure that an array member of a
struct was non-NULL. Tor has been doing this check since at least
0.2.3, maybe earlier.
Fixes bug 17781.
Previously we'd suppressed the mask-bits field in the output when
formatting a policy if it was >=32. But that should be a >=128 if
we're talking about IPv6.
Since we didn't put these in descriptors, this bug affects only log
messages and controller outputs.
Fix for bug 16056. The code in question was new in 0.2.0, but the
bug was introduced in 0.2.4 when we started supporting IPv6 exits.
port is in host order (addr is tor_addr_t, endianness is abstracted).
addr and port can be different to conn->addr and conn->port if
connecting via a proxy.
Consistently ignore multicast addresses when automatically
generating reject private exit policies.
Closes ticket 17763. Bug fix on 10a6390deb,
not in any released version of Tor. Patch by "teor".
The tor_cert_get_checkable_sig function uses the signing key included in
the certificate (if available) when a separate public key is not given.
When the signature is valid, the tor_cert_checksig function copies the
public key from the checkable structure to the public key field of the
certificate signing key.
In situations where the separate public key is not given but the
certificate includes a signing key, the source and destination pointers
in the copy operation are equal and invoke undefined behavior.
Undefined behaviour is avoided by ensuring both pointers are different.
These functions must really never fail; so have crypto_rand() assert
that it's working okay, and have crypto_seed_rng() demand that
callers check its return value. Also have crypto_seed_rng() check
RAND_status() before returning.
Stop ignoring ExitPolicyRejectPrivate in getinfo
exit-policy/reject-private. Fix a memory leak.
Set ExitPolicyRejectPrivate in the unit tests, and make a mock
function declaration static.
(If we take the branch above this assertion, than we *didn't* have a
v1 handshake. So if we don't take the branch, we did. So if we
reach this assertion, we must be running as a server, since clients
no longer attempt v1 handshakes.)
Fix for bug 17654; bugfix on 9d019a7db7.
Bug not in any released Tor.
Refuse connection requests to private OR addresses unless
ExtendAllowPrivateAddresses is set. Previously, tor would
connect, then refuse to send any cells to a private address.
Fixes bugs 17674 and 8976; bugfix on b7c172c9ec (28 Aug 2012)
Original bug 6710, released in 0.2.3.21-rc and an 0.2.2 maint
release.
Patch by "teor".
This migrates away from SHA1, and provides further hash flooding
protection on top of the randomised siphash implementation.
Add unit tests to make sure that different inputs don't have the
same hash.
The wrong list was used when looking up expired intro points in a rend
service object causing what we think could be reachability issues and
triggering a BUG log.
Fixes#16702
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
exit-policy/reject-private lists the reject rules added by
ExitPolicyRejectPrivate. This makes it easier for stem to
display exit policies.
Add unit tests for getinfo exit-policy/*.
Completes ticket #17183. Patch by "teor".
Modify policies_parse_exit_policy_reject_private so it also blocks
the addresses configured for OutboundBindAddressIPv4_ and
OutboundBindAddressIPv6_, and any publicly routable port addresses
on exit relays.
Add and update unit tests for these functions.
Move the code that rejects publicly routable exit relay addresses
to policies_parse_exit_policy_reject_private. Add
addr_policy_append_reject_addr_list and use it to reject interface
addresses.
This removes the duplicate reject checks on local_address and
ipv6_local_address, but duplicates will be removed by
exit_policy_remove_redundancies at the end of the function.
This also removes the info-level logging on rejected interface
addresses. Instead, log a debug-level message in
addr_policy_append_reject_addr.
This simplifies policies_parse_exit_policy_internal and prepares for
reporting these addresses over the control port in #17183.
Loading a on disk bridge descriptor causes a directory download to be
scheduled, which asserts due to the periodic events not being
initialized yet.
Fixes bug #17635, not in any released version of tor.
Now we only re-scan the list in the cases we did before: when we
have a new circuit that we should try attaching to, or when we have
added a new stream that we haven't tried to attach yet.
This is part of 17590.
Long ago we used to call connection_ap_handshake_attach_circuit()
only in a few places, since connection_ap_attach_pending() attaches
all the pending connections, and does so regularly. But this turned
out to have a performance problem: it would introduce a delay to
launching or connecting a stream.
We couldn't just call connection_ap_attach_pending() every time we
make a new connection, since it walks the whole connection list. So
we started calling connection_ap_attach_pending all over, instead!
But that's kind of ugly and messes up our callgraph.
So instead, we now have connection_ap_attach_pending() use a list
only of the pending connections, so we can call it much more
frequently. We have a separate function to scan the whole
connection array to see if we missed adding anything, and log a
warning if so.
Closes ticket #17590
Mark fallback directory mirrors as "too busy" when they return
a 503 response. Previously, the code just marked authorities as busy.
Unless clients set their own fallback directories, they will never see
this bug. (There are no default fallbacks yet.)
Fixes bug 17572; bugfix on 5c51b3f1f0 released in 0.2.4.7-alpha.
Patch by "teor".
When logging to syslog, allow a tag to be added to the syslog identity
("Tor"), i.e. the string prepended to every log message. The tag can be
configured by setting SyslogIdentityTag and defaults to none. Setting
it to "foo" will cause logs to be tagged as "Tor-foo". Closes: #17194.
Warn when the state file was last written in the future.
Tor doesn't know that consensuses have expired if the clock is in the past.
Patch by "teor". Implements ticket #17188.
BSD make takes spaces around = literally
and produces a "TESTING_TOR_BINARY "
variable with a trailing space, which leads
to test_keygen.sh failure.
Fixes 17154
When we find a conflict in the keypinning journal, treat the new
entry as superseding all old entries that overlap either of its
keys.
Also add a (not-yet-used) configuration option to disable keypinning
enforcement.
src/test/test_policy.c:
Merged calls to policies_parse_exit_policy by adding additional arguments.
fixup to remaining instance of ~EXIT_POLICY_IPV6_ENABLED.
Compacting logic test now produces previous list length of 4, corrected this.
src/config/torrc.sample.in:
src/config/torrc.minimal.in-staging:
Merged torrc modification dates in favour of latest.
Log an info-level message containing the reject line added to the
exit policy for each local IP address blocked by ExitPolicyRejectPrivate:
- Published IPv4 and IPv6 addresses
- Publicly routable IPv4 and IPv6 interface addresses
ExitPolicyRejectPrivate now rejects more local addresses by default:
* the relay's published IPv6 address (if any), and
* any publicly routable IPv4 or IPv6 addresses on any local interfaces.
This resolves a security issue for IPv6 Exits and multihomed Exits that
trust connections originating from localhost.
Resolves ticket 17027. Patch by "teor".
Patch on 42b8fb5a15 (11 Nov 2007), released in 0.2.0.11-alpha.
In previous versions of Tor, ExitPolicy accept6/reject6 * produced
policy entries for IPv4 and IPv6 wildcard addresses.
To reduce operator confusion, change accept6/reject6 * to only produce
an IPv6 wildcard address.
Resolves bug #16069.
Patch on 2eb7eafc9d and a96c0affcb (25 Oct 2012),
released in 0.2.4.7-alpha.
Tor now warns when ExitPolicy lines occur after accept/reject *:*
or variants. These lines are redundant, and were always ignored.
Partial fix for ticket 16069. Patch by "teor".
Patch on 2eb7eafc9d and a96c0affcb (25 Oct 2012),
released in 0.2.4.7-alpha.
When parsing torrc ExitPolicies, we now warn if:
* an IPv4 address is used on an accept6 or reject6 line. The line is
ignored, but the rest of the policy items in the list are used.
(accept/reject continue to allow both IPv4 and IPv6 addresses in torrcs.)
* a "private" address alias is used on an accept6 or reject6 line.
The line filters both IPv4 and IPv6 private addresses, disregarding
the 6 in accept6/reject6.
When parsing torrc ExitPolicies, we now issue an info-level message:
* when expanding an accept/reject * line to include both IPv4 and IPv6
wildcard addresses.
In each instance, usage advice is provided to avoid the message.
Partial fix for ticket 16069. Patch by "teor".
Patch on 2eb7eafc9d and a96c0affcb (25 Oct 2012),
released in 0.2.4.7-alpha.
routerset_parse now accepts IPv6 literal addresses.
Fix for ticket 17060. Patch by "teor".
Patch on 3ce6e2fba2 (24 Jul 2008), and related commits,
released in 0.2.1.3-alpha.
When validating a new descriptor against our rend cache failure, we were
added the failure entry to the new cache entry without duplicating. It was
then freed just after the validation ending up in a very bad memory state
that was making tor abort(). To fix this, a dup function has been added and
used just before adding the failure entry.
Fixes#17041
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
No functional changes, but since NoKeepAliveIsolateSOCKSAuth isn't
part of isoflag, it should be checked where all other similar options
are, and bypass the (no-op) masking at the end.
This controls the circuit dirtyness reset behavior added for Tor
Browser's user experience fix (#15482). Unlike previous iterations
of this patch, the tunable actually works, and is documented.
Performing lookups in both the client and service side descriptor
caches from the same rend_cache_lookup_entry() function increases the
risk of accidental API misuse.
I'm separating the lookup functions to keep the caches distinct.
Parameterize the rend_cache_clean() function to allow it clean
old rendezvous descriptors from the service-side cache as well as
the client descriptor cache.
Including the replica number in the HS_DESC CREATED event provides
more context to a control port client. The replica allows clients
to more easily identify each replicated descriptor from the
independantly output control events.
Entries in the service-side descriptor cache are now cleaned when
rend_cache_free_all() is called. The call to tor_free(intro_content)
in rend_cache_store_v2_desc_as_service() is moved to prevent a
potential double-free when a service has a descriptor with a newer
timestamp already in it's service-side descriptor cache.
Adds an Enum which represents the different types of rendezvous
descriptor caches. This argument is passed in each call to
rend_cache_lookup_entry() to specify lookup in the client-side or
service-side descriptor caches.
Adds a control command to fetch a local service descriptor from the
service descriptor cache. The local service descriptor cache is
referenced by the onion address of the service.
This control command is documented in the control spec.
When this is set, and Tor is running as a relay, it will not
generate or load its secret identity key. You can manage the secret
identity key with --keygen. Implements ticket 16944.
In a nutshell, since a circuit can not exit at its entry point, it's very
easy for an attacker to find the hidden service guard if only one EntryNodes
is specified since for that guard, the HS will refuse to build a rendezvous
circuit to it.
For now, the best solution is to stop tor to allow a single EntryNodes for
an hidden service.
Fixes#14917
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Only applies to connections with SOCKS auth set, so that non-web Tor
activity is not affected.
Simpler version of Nick's patch because the randomness worried me, and I'm not
otherwise sure why we want a max here.
In validate_recommended_package_line, at this point in the function,
n_entries is always >= 1. Coverity doesn't like us checking it for
0.
CID 1268063.
Removes a check to PublishHidServDescriptors in
rend_consider_services_upload(). This allows descriptors to be
generated and stored in the local cache when PublishHidServDescriptor = 0.
Keep the PublishHidServDescriptors option check in
rend_consider_descriptor_republication(). We will never need to republish
a descriptor if we are not publishing descriptors to the HSDirs.
Service descriptors are now generated regardless of the the
PublishHidServDescriptors option. The generated descriptors are stored
in the service descriptor cache.
The PublishHidServDescriptors = 1 option now prevents descriptor
publication to the HSDirs rather than descriptor generation.
Deindent a block of code inside the PublishHidServDescriptors option
check in upload_service_descriptor(). Stylistic commit to make the
subsequent reworking of this code cleaner.
The HS_DESC CREATED event should be emmited when a new service descriptor
is generated for a local rendevous service. This event is documented
in the control spec.
This commit resolves ticket #16291.
Adds a service descriptor cache which is indexed by service ID. This
descriptor cache is used to store service descriptors generated by a
local rendevous service.
The service-side cach can be queried by calling rend_cache_lookup_entry()
with the 'service' argument set to 1.
We used to use this when we had some controllers that would accept
long names and some that wouldn't. But it's been obsolete for a
while, and it's time to strip it out of the code.
Previously we'd put these strings right on the controllers'
outbufs. But this could cause some trouble, for these reasons:
1) Calling the network stack directly here would make a huge portion
of our networking code (from which so much of the rest of Tor is
reachable) reachable from everything that potentially generated
controller events.
2) Since _some_ events (EVENT_ERR for instance) would cause us to
call connection_flush(), every control_event_* function would
appear to be able to reach even _more_ of the network stack in
our cllgraph.
3) Every time we generated an event, we'd have to walk the whole
connection list, which isn't exactly fast.
This is an attempt to break down the "blob" described in
http://archives.seul.org/tor/dev/Mar-2015/msg00197.html -- the set of
functions from which nearly all the other functions in Tor are
reachable.
Closes ticket 16695.
Make it easier to unit test TestingDirAuthVote{Exit,Guard,HSDir}
by refactoring the code which sets flags based on them into a
new function dirserv_set_routerstatus_testing.
"option to prevent guard,exit,hsdir flag assignment"
"A node will never receive the corresponding flag unless
that node is specified in the
TestingDirAuthVote{Exit,Guard,HSDir} list, regardless of
its uptime, bandwidth, exit policy, or DirPort".
Patch modified by "teor": VoteOnHidServDirectoriesV2
is now obsolete, so TestingDirAuthVoteHSDir always
votes on HSDirs.
Closes ticket 14882. Patch by "robgjansen".
Commit message and changes file by "teor"
with quotes from "robgjansen".
Fix an error in the manual page and comments for
TestingDirAuthVoteHSDir, which suggested that a
HSDir required "ORPort connectivity". While this is true,
it is in no way unique to the HSDir flag. Of all the flags,
only HSDirs need a DirPort configured in order for the
authorities to assign that particular flag.
Fixed as part of 14882. Patch by "teor".
Bugfix on 0.2.6.3 (f9d57473e1 on 10 January 2015).
* FIXES#16823: https://bugs.torproject.org/16823
If an OP were to send a CREATE_FAST cell to an OR, and that
CREATE_FAST cell had unparseable key material, then tor_free() would
be called on the create cell twice. This fix removes the second
(conditional on the key material being bad) call to tor_free(), so
that now the create cell is always freed once, regardless of the status of
the key material.
(This isn't actually a double-free bug, since tor_free() sets its
input to NULL, and has no effect when called with input NULL.)
Instead of having it call update_all_descriptor_downloads and
update_networkstatus_downloads directly, we can have it cause them to
get rescheduled and called from run_scheduled_events.
Closes ticket 16789.
When fetching a descriptor, we know test every introduction points in it
against our rend failure cache to know if we keep it or not. For this to
work, now everytime an introduction points is discareded (ex: receiving a
NACK), we note it down in our introduction cache.
See rendcache.c for a detailed explanation of the cache's behavior.
Fixes#16389
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
When we removed Running/Valid checks from Fast and Stable in 8712, I
removed them from HSDir too, which apparently wasn't a good idea.
Reverts part of a65e835800. Fixes bug 16524. Bugfix
on 0.2.7.2-alpha.
microdesc_free_() called get_microdesc_cache(), which had the fun
side-effect of potentially reloading the whole cache from disk.
Replace it with a variant that doesn't.
The base64 and base32 functions used to be in crypto.c;
crypto_format.h had no header; some general-purpose functions were in
crypto_curve25519.c.
This patch makes a {crypto,util}_format.[ch], and puts more functions
there. Small modules are beautiful!
The control port was using set_max_file_descriptors() with a limit set to 0
to query the number of maximum socket Tor can use. With the recent changes
to that function, a check was introduced to make sure a user can not set a
value below the amount we reserved for non socket.
This commit adds get_max_sockets() that returns the value of max_sockets so
we can stop using that "setter" function to get the current value.
Finally, the dead code is removed that is the code that checked for limit
equal to 0. From now on, set_max_file_descriptors() should never be used
with a limit set to 0 for a valid use case.
Fixes#16697
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
1) We already require C99.
2) This allows us to support MSVC again (thanks to Gisle Vanem for
this part)
3) This change allows us to dump some rotten old compatibility code
from log.c
Make sure that signing certs are signed by the right identity key,
to prevent a recurrence of #16530. Also make sure that the master
identity key we find on disk matches the one we have in RAM, if we
have one.
This is for #16581.
When there is a signing key and the certificate lists a key, make
sure that the certificate lists the same signing key.
When there are public key and secret key stored in separate files,
make sure they match.
Use the right file name when we load an encrypted secret key and
then find a problem with it.
This is part of 16581.
When cleaning up extra circuits that we've opened for performance reason, we
need to count all the introduction circuit and not only the established ones
else we can end up with too many introduction points.
This also adds the check for expiring nodes when serving an INTRODUCE cell
since it's possible old clients are still using them before we have time to
close them.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
To upload a HS descriptor, this commits makes it that we wait for all
introduction point to be fully established.
Else, the HS ends up uploading a descriptor that may contain intro points
that are not yet "valid" meaning not yet established or proven to work. It
could also trigger three uploads for the *same* descriptor if every intro
points takes more than 30 seconds to establish because of desc_is_dirty
being set at each intro established.
To achieve that, n_intro_points_established varialbe is added to the
rend_service_t object that is incremented when we established introduction
point and decremented when we remove a valid intro point from our list.
The condition to upload a descriptor also changes to test if all intro
points are ready by making sure we have equal or more wanted intro points
that are ready.
The desc_id_dirty flag is kept to be able to still use the
RendInitialPostPeriod option.
This partially fixes#13483.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
There is a case where if the introduction circuit fails but the node is
still in the consensus, we clean up the intro point and choose an other one.
This commit fixes that by trying to reuse the existing intro point with a
maximum value of retry.
A retry_nodes list is added to rend_services_introduce() and when we remove
an invalid intro points that fits the use case mentionned before, we add the
node to the retry list instead of removing it. Then, we retry on them before
creating new ones.
This means that the requirement to remove an intro point changes from "if no
intro circuit" to "if no intro circuit then if no node OR we've reached our
maximum circuit creation count".
For now, the maximum retries is set to 3 which it completely arbitrary. It
should also at some point be tied to the work done on detecting if our
network is down or not.
Fixes#8239
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The reasoning for refactoring this function is that removing the
introduction point adaptative algorithm (#4862) ended up changing quite a
bit rend_services_introduce(). Also, to fix some open issues (#8239, #8864
and #13483), this work had to be done.
First, this removes time_expiring variable in an intro point object and
INTRO_POINT_EXPIRATION_GRACE_PERIOD trickery and use an expiring_nodes list
where intro nodes that should expire are moved to that list and cleaned up
only once the new descriptor is successfully uploaded. The previous scheme
was adding complexity and arbitrary timing to when we expire an intro point.
We keep the intro points until we are sure that the new descriptor is
uploaded and thus ready to be used by clients. For this,
rend_service_desc_has_uploaded() is added to notify the HS subsystem that
the descriptor has been successfully uploaded. The purpose of this function
is to cleanup the expiring nodes and circuits if any.
Secondly, this adds the remove_invalid_intro_points() function in order to
split up rend_services_introduce() a bit with an extra modification to it
that fixes#8864. We do NOT close the circuit nor delete the intro point if
the circuit is still alive but the node was removed from the consensus. Due
to possible information leak, we let the circuit and intro point object
expire instead.
Finally, the whole code flow is simplified and large amount of documentation
has been added to mostly explain the why of things in there.
Fixes#8864
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
This is a way to specify the amount of introduction points an hidden service
can have. Maximum value is 10 and the default is 3.
Fixes#4862
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
When we ran out of intro points for a hidden service (which could
happen on a newnym), we would change the connection's state back to
"waiting for hidden service descriptor." But this would make an
assertion fail if we went on to call circuit_get_open_circ_or_launch
again.
This fixes bug 16013; I believe the bug was introduced in
38be533c69, where we made it possible for
circuit_get_open_circ_or_launch() to change the connection's state.
When I fixed#11243, I made it so we would take the digest of a
descriptor before tokenizing it, so we could desist from download
attempts if parsing failed. But when I did that, I didn't remove an
assertion that the descriptor began with "onion-key". Usually, this
was enforced by "find_start_of_next_microdescriptor", but when
find_start_of_next_microdescriptor returned NULL, the assertion was
triggered.
Fixes bug 16400. Thanks to torkeln for reporting and
cypherpunks_backup for diagnosing and writing the first fix here.
Every functions and objects that are used for hidden service descriptor
caches are moved to rendcache.{c|h}.
This commit does NOT change anything, just moving code around.
Fixes#16399
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
For now, rend_cache_entry_t has been moved from or.h to rendcache.h and
those files have been added to the build system.
In the next commit, these will contain hidden service descriptor cache ABI
and API for both client and directory side. The goal is to consolidate the
descriptor caches in one location to ease development, maintenance, review
and improve documentation for each cache behavior and algorithm.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
When --keygen is provided, we prompt for a passphrase when we make a
new master key; if it is nonempty, we store the secret key in a new
crypto_pwbox.
Also, if --keygen is provided and there *is* an encrypted master key,
we load it and prompt for a passphrase unconditionally.
We make a new signing key unconditionally when --keygen is provided.
We never overwrite a master key.
If crypto_early_init fails, a typo in a return value from tor_init
means that tor_main continues running, rather than returning
an error value.
Fixes bug 16360; bugfix on d3fb846d8c in 0.2.5.2-alpha,
introduced when implementing #4900.
Patch by "teor".
This reverts commit 9407040c59.
Small fix, "e->received" had to be removed since that variable doesn't exist
anymore.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
signing_key can be NULL in ed_key_init_from_file in routerkeys.c.
Discovered by clang 3.7 address sanitizer.
Fix on c03694938e, not in any released version of Tor.
Rend_add_service() frees its argument on failure; no need to free again.
Fixes bug 16228, bugfix on 0.2.7.1-alpha
Found by coverity; this is CID 1301387.
Also, have testing-level options to set the lifetimes and
expiration-tolerances of all key types, plus a non-testing-level
option to set the lifetime of any auto-generated signing key.
# The first commit's message is:
Regenerate ed25519 keys when they will expire soon.
Also, have testing-level options to set the lifetimes and
expiration-tolerances of all key types, plus a non-testing-level
option to set the lifetime of any auto-generated signing key.
# The 2nd commit message will be skipped:
# fixup! Regenerate ed25519 keys when they will expire soon.
This is a new collator type that follows proposal 220 for deciding
which identities to include. The rule is (approximately):
If a <ed,rsa> identity is listed by more than half of authorities,
include it. And include all <rsa> votes about that node as
matching.
Otherwise, if an <*,rsa> or <rsa> identity is listed by more than
half of the authorities, and no <ed,rsa> has been listed, include
it.
* Include ed25519 identities in votes
* Include "no ed25519 identity" in votes
* Include some commented-out code about identity voting. (This
will disappear.)
* Include some functions for identity voting (These will disappear.)
* Enforce uniqueness in ed25519 keys within a vote
Extrainfo documents are now ed-signed just as are router
descriptors, according to proposal 220. This patch also includes
some more tests for successful/failing parsing, and fixes a crash
bug in ed25519 descriptor parsing.
An earlier version of these tests was broken; now they're a nicer,
more robust, more black-box set of tests. The key is to have each
test check a handshake message that is wrong in _one_ way.
When there are annotations on a router descriptor, the
ed25519-identity element won't be at position 0 or 1; it will be at
router+1 or router-1.
This patch also adds a missing smartlist function to search a list for
an item with a particular pointer.
With this patch:
* Authorities load the key-pinning log at startup.
* Authorities open a key-pinning log for writing at startup.
* Authorities reject any router with an ed25519 key where they have
previously seen that ed25519 key with a different RSA key, or vice
versa.
* Authorities warn about, but *do not* reject, RSA-only descriptors
when the RSA key has previously gone along with an Ed25519 key.
(We should make this a 'reject' too, but we can't do that until we're
sure there's no legit reason to downgrade to 0.2.5.)
This module implements a key-pinning mechanism to ensure that it's
safe to use RSA keys as identitifers even as we migrate to Ed25519
keys. It remembers, for every Ed25519 key we've seen, what the
associated Ed25519 key is. This way, if we see a different Ed25519
key with that RSA key, we'll know that there's a mismatch.
We persist these entries to disk using a simple format, where each
line has a base64-encoded RSA SHA1 hash, then a base64-endoded
Ed25519 key. Empty lines, misformed lines, and lines beginning with
a # are ignored. Lines beginning with @ are reserved for future
extensions.
Routers now use TAP and ntor onion keys to sign their identity keys,
and put these signatures in their descriptors. That allows other
parties to be confident that the onion keys are indeed controlled by
the router that generated the descriptor.
Routers now use TAP and ntor onion keys to sign their identity keys,
and put these signatures in their descriptors. That allows other
parties to be confident that the onion keys are indeed controlled by
the router that generated the descriptor.
For prop220, we have a new ed25519 certificate type. This patch
implements the code to create, parse, and validate those, along with
code for routers to maintain their own sets of certificates and
keys. (Some parts of master identity key encryption are done, but
the implementation of that isn't finished)
As OpenSSL >= 1.0.0 is now required, ECDHE is now mandatory. The group
has to be validated at runtime, because of RedHat lawyers (P224 support
is entirely missing in the OpenSSL RPM, but P256 is present and is the
default).
Resolves ticket #16140.
When set, this limits the maximum number of simultaneous streams per
rendezvous circuit on the server side of a HS, with further RELAY_BEGIN
cells being silently ignored.
This can be modified via "HiddenServiceMaxStreamsCloseCircuit", which
if set will cause offending rendezvous circuits to be torn down instead.
Addresses part of #16052.
Ephemeral services will be listed in rend_services_list at the end of
rend_config_services, so it must check whether directory is non-NULL
before comparing.
This crash happens when reloading config on a tor with mixed configured
and ephemeral services.
Fixes bug #16060. Bugfix on 0.2.7.1-alpha.
For FAILED and RECEIVED action of the HS_DESC event, we now sends back the
descriptor ID at the end like specified in the control-spec section 4.1.25.
Fixes#15881
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The length of auth_data from an INTRODUCE2 cell is checked when the
auth_type is recognized (1 or 2), but not for any other non-zero
auth_type. Later, auth_data is assumed to have at least
REND_DESC_COOKIE_LEN bytes, leading to a client-triggered out of bounds
read.
Fixed by checking auth_len before comparing the descriptor cookie
against known clients.
Fixes#15823; bugfix on 0.2.1.6-alpha.
"+HSPOST" and the related event changes allow the uploading of HS
descriptors via the control port, and more comprehensive event
monitoring of HS descriptor upload status.
When we have a new descriptor ID for an onion address request, change it in
the rend_data_t object and purge the old one from the last hid serv request
cache.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Stop using an onion address since it's not indexed with that anymore in the
last hid serv request cache. Instead use a base32 encoded descriptor ID
contained in the rend_data_t object.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Every callsite that use to allocate a rend_data_t object now use the
rend_data_client/service_create() function.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Ground works for fixing #15816. This adds the rend_data_create() function in
order to have a single place where we initialize that data structure.
Furthermore, an array of descriptor IDs is added (one per replica) so we can
keep a copy of the current id in the object. It will be used to purge the
last hid serv request cache using those descriptor IDs. When they change,
they will be replaced and the old ones will be purged from the cache.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Fixes#15850, part of #15801. Change file is added by this commit. The
original comment in the reverted commit is removed because right now we
*need* a DirPort until #15849 is implemented so no doubt nor confusion there
anymore.
This reverts commit 80bed1ac96.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
These commands allow for the creation and management of ephemeral
Onion ("Hidden") services that are either bound to the lifetime of
the originating control connection, or optionally the lifetime of
the tor instance.
Implements #6411.
The "longest possible policy" comment in
router_parse_addr_policy_item_from_string() used an example policy
that was actually shorter than the maximum length.
This comment was amended, and expanded to count the maximum number of
characters.
Comment change only.
Unit tests for the 10 valid combinations of set/NULL config options
DirAuthorities, AlternateBridgeAuthority, AlternateDirAuthority,
and FallbackDir.
Add assertion in consider_adding_dir_servers() for checks in
validate_dir_servers():
"You cannot set both DirAuthority and Alternate*Authority."
Only add the default fallback directories when the DirAuthorities,
AlternateDirAuthority, and FallbackDir directory config options
are set to their defaults.
The default fallback directory list is currently empty, this fix will
only change tor's behaviour when it has default fallback directories.
Fixes bug 15642; bugfix on 90f6071d8d in 0.2.4.7-alpha. Patch by "teor".
When self-testing reachability, use ExtendAllowPrivateAddresses
to determine if local/private addresses imply reachability.
The previous fix used TestingTorNetwork, which implies
ExtendAllowPrivateAddresses, but this excluded rare configs where
ExtendAllowPrivateAddresses is set but TestingTorNetwork is not.
Fixes bug 15771; bugfix on 0.2.6.1-alpha, bug #13924.
Patch by "teor", issue discovered by CJ Ess.
The HS_DESC_CONTENT event results in multiple line thus must be prefixed
with a "650+" and ending with "650 OK".
Reported-by: Damian Johnson <atagar@torproject.org>
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The HS_DESC event was using rend_data_t from the dir connection to reply the
onion address and authentication type. With the new HSFETCH command, it's
now possible to fetch a descriptor only using the descriptor id thus
resulting in not having an onion address in any HS_DESC event.
This patch removes rend_query from the hs desc control functions and replace
it by an onion address string and an auth type.
On a successful fetch, the service id is taken from the fetched descriptor.
For that, an extra parameter is added to "store as a client" function that
contains the cache entry stored.
This will make the control event functions scale more easily over time if
other values not present in rend_data_t are needed since the rend_data from
the dir connection might not contained everything we need.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Big refactor of the HS client descriptor fetch functionnality. This allows
to fetch an HS descriptor using only a descriptor ID. Furthermore, it's also
possible to provide a list of HSDir(s) now that are used instead of the
automatically choosen one.
The approach taken was to add a descriptor_id field to the rend_data_t
structure so it can be used, if available, by the HS client. The onion
address field however has priority over it that is if both are set, the
onion address is used to fetch the descriptor.
A new public function is introduced called rend_client_fetch_v2_desc(...)
that does NOT lookup the client cache before fetching and can take a list of
HSDirs as a parameter.
The HSFETCH control command now uses this new function thus making it work
and final.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
As defined in section 4.1.26 in the control-spec.txt, this new event replies
the content of a successfully fetched HS descriptor. This also adds a unit
test for the controller event.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
This adds the command on the controller side that parses and validate
arguments but does nothing for now. The HS desriptor fetch must be
modularized a bit more before we can use the command.
See control-spec.txt section 3.26 for more information on this command.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Incidently, this fixes a bug where the maximum value was never used when
only using crypto_rand_int(). For instance this example below in
rendservice.c never gets to INTRO_POINT_LIFETIME_MAX_SECONDS.
int intro_point_lifetime_seconds =
INTRO_POINT_LIFETIME_MIN_SECONDS +
crypto_rand_int(INTRO_POINT_LIFETIME_MAX_SECONDS -
INTRO_POINT_LIFETIME_MIN_SECONDS);
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
An introduction point is currently rotated when the amount of INTRODUCE2
cells reached a fixed value of 16384. This makes it pretty easy for an
attacker to inflate that number and observe when the IP rotates which leaks
the popularity of the HS (amount of client that passed through the IP).
This commit makes it a random count between the current value of 16384 and
two times that.
Fixes#15745
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Background processes spawned by Tor now will have a valid stdin.
Pluggable transports can detect this behavior with the aformentioned
enviornment variable, and exit if stdin ever gets closed.
The rend-spec.txt document doesn't specify this extra newline. Furthermore,
this is the only descryptor type that contains one. Client and HSDir without
this patch still work perfectly since the HS descriptor parsing doesn't
expect a newline at the end.
Fixes#15296
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Remove src/or/or_sha1.i and src/common/common_sha1.i on `make clean` and remove
the temporary micro-revision file when its no longer needed.
Additional changes;
- show a message when generating the micro-revision file.
- add the temporary micro revision file to the list of files to be removed on
`make clean` just in case.
- fix indentation of the make rule to improve readability.
PTs expect the auth cookie to be available immedieately after launch,
leading to a race condition when PTs opt to cache the extorport cookie
once immediately after startup.
Fixes#15240.
Report errors if the notification fails; report success only if it
succeeds; and if we are not notifying systemd because we aren't
running with systemd, don't log at notice.
We already log whenever our state changes, e.g. whenever new directory
information arrives. This additional log_warn() will at best just add more
log messages, or worse, make the user wonder what she needs to fix.
(Changed after consultation with Yawning.)
Nothing ever uses the string when we're in "have minimum dir info"
state. The flow of the function is "check for problems, if you see a
problem write an explanation to dir_info_status and set res to 0". If
you get to the end of the function without any problems, then res =
1 and we're all ready to start making circuits.
(Changed after consultation with Yawning.)
Parse the file just before voting and apply its information to the
provided vote_routerstatus_t. This follows the same logic as when
dirauths parse bwauth files.
`dir_info_status` is used from main.c:directory_info_has_arrived() to
provide useful (INFO/NOTICE) level logging to users, and should always
be updated regardless of the rate limiting.
This adds the key "hs/client/desc/id/<ADDR>" to the GETINFO command used to
lookup the given onion address in the client hs descriptor cache.
If found, prints it formatted as specified in section 1.3 of rend-spec.txt.
Fixes#14845
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The idea here is that a controller should be able to make Tor produce a
new relay descriptor on demand, without that descriptor actually being
uploaded to the dirauths (they would likely reject it anyway due to
freshness concerns).
Implements #14784.
Allow building a router descriptor without storing it to global state.
This is in preparation of a patch to export the created descriptors via
the control port.
The issue is that we use the cpuworker system with relays only, so if we
start up as a client and transition to being a relay later, we'll be
sad.
This fixes bug 14901; not in any released version of Tor.
They have been off-by-default since 0.2.5 and nobody has complained. :)
Also remove the buf_shrink() function, which hasn't done anything
since we first stopped using contiguous memory to store buffers.
Closes ticket 14848.
This fixes a bug where we decide to free the circuit because it isn't on
any workqueue anymore, and then the job finishes and the circuit gets
freed again.
Fixes bug #14815, not in any released version of Tor.
like might happen for Tails or Whonix users who start with a very wrong
hardware clock, use Tor to discover a more accurate time, and then
fix their clock.
Resolves part of ticket 8766.
(There are still some timers in various places that aren't addressed yet.)
In #14803, Damian noticed that his Tor sometimes segfaults. Roger noted
that his valgrind gave an invalid write of size one here. Whenever we
use FLEXIBLE_ARRAY_MEMBER, we have to make sure to actually malloc a
thing that's large enough.
Fixes bug #14803, not in any released version of Tor.
If the guard unreachable_since variable was set, the status "up" was
reported which is wrong. This adds the "down" status followed by the
unreachable_since time value.
Fixes#14184
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
David Goulet finds that when he runs a busy relay for a while with the
latest version of the git code, the number of onionskins handled
slowly dwindles to zero, with total_pending_tasks wedged at its
maximum value.
I conjecture this is because the total_pending_tasks variable isn't
decremented when we successfully cancel a job. Fixed that.
Fixes bug 14741; bugfix not on any released version of tor.
