Add a new and slow unit test that checks if libscrypt_scrypt() and
EBP_PBE_scrypt() yield the same keys from test vectors.
squash! Assert interoperability betweeen libscrypt and OpenSSL EBP_PBE_scrypt().
squash! Assert interoperability betweeen libscrypt and OpenSSL EBP_PBE_scrypt().
squash! Assert interoperability betweeen libscrypt and OpenSSL EBP_PBE_scrypt().
When cleaning up extra circuits that we've opened for performance reason, we
need to count all the introduction circuit and not only the established ones
else we can end up with too many introduction points.
This also adds the check for expiring nodes when serving an INTRODUCE cell
since it's possible old clients are still using them before we have time to
close them.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
To upload a HS descriptor, this commits makes it that we wait for all
introduction point to be fully established.
Else, the HS ends up uploading a descriptor that may contain intro points
that are not yet "valid" meaning not yet established or proven to work. It
could also trigger three uploads for the *same* descriptor if every intro
points takes more than 30 seconds to establish because of desc_is_dirty
being set at each intro established.
To achieve that, n_intro_points_established varialbe is added to the
rend_service_t object that is incremented when we established introduction
point and decremented when we remove a valid intro point from our list.
The condition to upload a descriptor also changes to test if all intro
points are ready by making sure we have equal or more wanted intro points
that are ready.
The desc_id_dirty flag is kept to be able to still use the
RendInitialPostPeriod option.
This partially fixes#13483.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
There is a case where if the introduction circuit fails but the node is
still in the consensus, we clean up the intro point and choose an other one.
This commit fixes that by trying to reuse the existing intro point with a
maximum value of retry.
A retry_nodes list is added to rend_services_introduce() and when we remove
an invalid intro points that fits the use case mentionned before, we add the
node to the retry list instead of removing it. Then, we retry on them before
creating new ones.
This means that the requirement to remove an intro point changes from "if no
intro circuit" to "if no intro circuit then if no node OR we've reached our
maximum circuit creation count".
For now, the maximum retries is set to 3 which it completely arbitrary. It
should also at some point be tied to the work done on detecting if our
network is down or not.
Fixes#8239
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The reasoning for refactoring this function is that removing the
introduction point adaptative algorithm (#4862) ended up changing quite a
bit rend_services_introduce(). Also, to fix some open issues (#8239, #8864
and #13483), this work had to be done.
First, this removes time_expiring variable in an intro point object and
INTRO_POINT_EXPIRATION_GRACE_PERIOD trickery and use an expiring_nodes list
where intro nodes that should expire are moved to that list and cleaned up
only once the new descriptor is successfully uploaded. The previous scheme
was adding complexity and arbitrary timing to when we expire an intro point.
We keep the intro points until we are sure that the new descriptor is
uploaded and thus ready to be used by clients. For this,
rend_service_desc_has_uploaded() is added to notify the HS subsystem that
the descriptor has been successfully uploaded. The purpose of this function
is to cleanup the expiring nodes and circuits if any.
Secondly, this adds the remove_invalid_intro_points() function in order to
split up rend_services_introduce() a bit with an extra modification to it
that fixes#8864. We do NOT close the circuit nor delete the intro point if
the circuit is still alive but the node was removed from the consensus. Due
to possible information leak, we let the circuit and intro point object
expire instead.
Finally, the whole code flow is simplified and large amount of documentation
has been added to mostly explain the why of things in there.
Fixes#8864
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The runtime sanity checking is slightly different from the optimized
basepoint stuff in that it uses a given implementation's self tests if
available, and checks if signing/verification works with a test vector
from the IETF EdDSA draft.
The unit tests include a new testcase that will fuzz donna against ref0,
including the blinding and curve25519 key conversion routines. If this
is something that should be done at runtime (No?), the code can be
stolen from there.
Note: Integrating batch verification is not done yet.
Integration work scavanged from nickm's `ticket8897_9663_v2` branch,
with minor modifications. Tor will still sanity check the output but
now also attempts to catch extreme breakage by spot checking the
optimized implementation vs known values from the NaCl documentation.
Implements feature 9663.
This needs to be done to allow for the possibility of removing the
ref10 code at a later date, though it is not performance critical.
When integrated by kludging it into tor, it passes unit tests, and is
twice as fast.
Integrating it the "wrong" way into common/crypto_ed25519.c passes
`make check`, and there appear to be some known answer tests for this,
so I assume I got it right.
Blinding a public key goes from 139.10 usec to 70.78 usec using
ed25519-donna (NB: Turboboost/phase of moon), though the code isn't
critical path, so supporting it is mostly done for completeness.
Integrate ed25519-donna into the build process, and provide an
interface that matches the `ref10` code. Apart from the blinding and
Curve25519 key conversion, this functions as a drop-in replacement for
ref10 (verified by modifying crypto_ed25519.c).
Tests pass, and the benchmarks claim it is quite a bit faster, however
actually using the code requires additional integration work.
The following arguments change how chutney verifies the network:
--bytes n sends n bytes per test connection (10 KBytes)
--connections n makes n test connections per client (1)
--hs-multi-client 1 makes each client connect to each HS (0)
Requires the corresponding chutney performance testing changes.
Note: using --connections 7 or greater on a HS will trigger #15937.
Patch by "teor".
This is a way to specify the amount of introduction points an hidden service
can have. Maximum value is 10 and the default is 3.
Fixes#4862
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
When we ran out of intro points for a hidden service (which could
happen on a newnym), we would change the connection's state back to
"waiting for hidden service descriptor." But this would make an
assertion fail if we went on to call circuit_get_open_circ_or_launch
again.
This fixes bug 16013; I believe the bug was introduced in
38be533c69, where we made it possible for
circuit_get_open_circ_or_launch() to change the connection's state.
RFC 952 is approximately 30 years old, and people are failing to comply,
by serving A records with '_' as part of the hostname. Since relaxing
the check is a QOL improvement for our userbase, relax the check to
allow such abominations as destinations, especially since there are
likely to be other similarly misconfigured domains out there.
When I fixed#11243, I made it so we would take the digest of a
descriptor before tokenizing it, so we could desist from download
attempts if parsing failed. But when I did that, I didn't remove an
assertion that the descriptor began with "onion-key". Usually, this
was enforced by "find_start_of_next_microdescriptor", but when
find_start_of_next_microdescriptor returned NULL, the assertion was
triggered.
Fixes bug 16400. Thanks to torkeln for reporting and
cypherpunks_backup for diagnosing and writing the first fix here.
Every functions and objects that are used for hidden service descriptor
caches are moved to rendcache.{c|h}.
This commit does NOT change anything, just moving code around.
Fixes#16399
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
For now, rend_cache_entry_t has been moved from or.h to rendcache.h and
those files have been added to the build system.
In the next commit, these will contain hidden service descriptor cache ABI
and API for both client and directory side. The goal is to consolidate the
descriptor caches in one location to ease development, maintenance, review
and improve documentation for each cache behavior and algorithm.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
When --keygen is provided, we prompt for a passphrase when we make a
new master key; if it is nonempty, we store the secret key in a new
crypto_pwbox.
Also, if --keygen is provided and there *is* an encrypted master key,
we load it and prompt for a passphrase unconditionally.
We make a new signing key unconditionally when --keygen is provided.
We never overwrite a master key.
If crypto_early_init fails, a typo in a return value from tor_init
means that tor_main continues running, rather than returning
an error value.
Fixes bug 16360; bugfix on d3fb846d8c in 0.2.5.2-alpha,
introduced when implementing #4900.
Patch by "teor".
This reverts commit 9407040c59.
Small fix, "e->received" had to be removed since that variable doesn't exist
anymore.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
clang complains that the address of struct member in an assert in
SSL_SESSION_get_master_key is always non-NULL.
Instead, check each pointer argument is non-NULL before using it.
Fix on f90a704f12 from 27 May 2015, not in any released version of tor.
signing_key can be NULL in ed_key_init_from_file in routerkeys.c.
Discovered by clang 3.7 address sanitizer.
Fix on c03694938e, not in any released version of Tor.
clang 3.7 complains that using a preprocessor directive inside
a macro invocation in test_util_writepid in test_util.c is undefined.
Fix on 79e85313aa on 0.2.7.1-alpha.
Unused variable warnings were still generated under some versions of OpenSSL.
Instead, make sure all variables are used under all versions.
Fix on 496df21c89, not in any released version of tor.
Rend_add_service() frees its argument on failure; no need to free again.
Fixes bug 16228, bugfix on 0.2.7.1-alpha
Found by coverity; this is CID 1301387.
Also, have testing-level options to set the lifetimes and
expiration-tolerances of all key types, plus a non-testing-level
option to set the lifetime of any auto-generated signing key.
# The first commit's message is:
Regenerate ed25519 keys when they will expire soon.
Also, have testing-level options to set the lifetimes and
expiration-tolerances of all key types, plus a non-testing-level
option to set the lifetime of any auto-generated signing key.
# The 2nd commit message will be skipped:
# fixup! Regenerate ed25519 keys when they will expire soon.
This is a new collator type that follows proposal 220 for deciding
which identities to include. The rule is (approximately):
If a <ed,rsa> identity is listed by more than half of authorities,
include it. And include all <rsa> votes about that node as
matching.
Otherwise, if an <*,rsa> or <rsa> identity is listed by more than
half of the authorities, and no <ed,rsa> has been listed, include
it.
* Include ed25519 identities in votes
* Include "no ed25519 identity" in votes
* Include some commented-out code about identity voting. (This
will disappear.)
* Include some functions for identity voting (These will disappear.)
* Enforce uniqueness in ed25519 keys within a vote
Extrainfo documents are now ed-signed just as are router
descriptors, according to proposal 220. This patch also includes
some more tests for successful/failing parsing, and fixes a crash
bug in ed25519 descriptor parsing.
An earlier version of these tests was broken; now they're a nicer,
more robust, more black-box set of tests. The key is to have each
test check a handshake message that is wrong in _one_ way.
This includes the link handshake variations for proposal220.
We'll use this for testing first, and then use it to extend our
current code to support prop220.
When there are annotations on a router descriptor, the
ed25519-identity element won't be at position 0 or 1; it will be at
router+1 or router-1.
This patch also adds a missing smartlist function to search a list for
an item with a particular pointer.
With this patch:
* Authorities load the key-pinning log at startup.
* Authorities open a key-pinning log for writing at startup.
* Authorities reject any router with an ed25519 key where they have
previously seen that ed25519 key with a different RSA key, or vice
versa.
* Authorities warn about, but *do not* reject, RSA-only descriptors
when the RSA key has previously gone along with an Ed25519 key.
(We should make this a 'reject' too, but we can't do that until we're
sure there's no legit reason to downgrade to 0.2.5.)
This module implements a key-pinning mechanism to ensure that it's
safe to use RSA keys as identitifers even as we migrate to Ed25519
keys. It remembers, for every Ed25519 key we've seen, what the
associated Ed25519 key is. This way, if we see a different Ed25519
key with that RSA key, we'll know that there's a mismatch.
We persist these entries to disk using a simple format, where each
line has a base64-encoded RSA SHA1 hash, then a base64-endoded
Ed25519 key. Empty lines, misformed lines, and lines beginning with
a # are ignored. Lines beginning with @ are reserved for future
extensions.
Routers now use TAP and ntor onion keys to sign their identity keys,
and put these signatures in their descriptors. That allows other
parties to be confident that the onion keys are indeed controlled by
the router that generated the descriptor.
Routers now use TAP and ntor onion keys to sign their identity keys,
and put these signatures in their descriptors. That allows other
parties to be confident that the onion keys are indeed controlled by
the router that generated the descriptor.
For prop220, we have a new ed25519 certificate type. This patch
implements the code to create, parse, and validate those, along with
code for routers to maintain their own sets of certificates and
keys. (Some parts of master identity key encryption are done, but
the implementation of that isn't finished)
If the OpenSSL team accepts my patch to add an
SSL_get_client_ciphers function, this patch will make Tor use it
when available, thereby working better with openssl 1.1.
We previously used this function instead of SSL_set_cipher_list() to
set up a stack of client SSL_CIPHERs for these reasons:
A) In order to force a particular order of the results.
B) In order to be able to include dummy entries for ciphers that
this build of openssl did not support, so we could impersonate
Firefox harder.
But we no longer do B, since we merged proposal 198 and stopped
lying about what ciphers we know.
And A was actually pointless, since I had misread the implementation
of SSL_set_cipher_list(). It _does_ do some internal sorting, but
that is pre-sorting on the master list of ciphers, not sorting on
the user's preferred order.
As OpenSSL >= 1.0.0 is now required, ECDHE is now mandatory. The group
has to be validated at runtime, because of RedHat lawyers (P224 support
is entirely missing in the OpenSSL RPM, but P256 is present and is the
default).
Resolves ticket #16140.
The key here is to never touch ssl->cipher_list directly, but only
via SSL_get_ciphers(). But it's not so simple.
See, if there is no specialized cipher_list on the SSL object,
SSL_get_ciphers returns the cipher_list on the SSL_CTX. But we sure
don't want to modify that one! So we need to use
SSL_set_cipher_list first to make sure that we really have a cipher
list on the SSL object.
When set, this limits the maximum number of simultaneous streams per
rendezvous circuit on the server side of a HS, with further RELAY_BEGIN
cells being silently ignored.
This can be modified via "HiddenServiceMaxStreamsCloseCircuit", which
if set will cause offending rendezvous circuits to be torn down instead.
Addresses part of #16052.
Ephemeral services will be listed in rend_services_list at the end of
rend_config_services, so it must check whether directory is non-NULL
before comparing.
This crash happens when reloading config on a tor with mixed configured
and ephemeral services.
Fixes bug #16060. Bugfix on 0.2.7.1-alpha.
With #15881 implemented, this adds the missing descriptor ID at the end of
the expected control message.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
For FAILED and RECEIVED action of the HS_DESC event, we now sends back the
descriptor ID at the end like specified in the control-spec section 4.1.25.
Fixes#15881
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
This field was only needed to work with the now-long-gone (I hope,
except for some horrible apples) openssl 0.9.8l; if your headers say
you have openssl 1.1, you won't even need it.
OpenSSL 1.1.0 must be built with "enable-deprecated", and compiled with
`OPENSSL_USE_DEPRECATED` for this to work, so instead, use the newer
routine as appropriate.
The length of auth_data from an INTRODUCE2 cell is checked when the
auth_type is recognized (1 or 2), but not for any other non-zero
auth_type. Later, auth_data is assumed to have at least
REND_DESC_COOKIE_LEN bytes, leading to a client-triggered out of bounds
read.
Fixed by checking auth_len before comparing the descriptor cookie
against known clients.
Fixes#15823; bugfix on 0.2.1.6-alpha.
"+HSPOST" and the related event changes allow the uploading of HS
descriptors via the control port, and more comprehensive event
monitoring of HS descriptor upload status.
When we have a new descriptor ID for an onion address request, change it in
the rend_data_t object and purge the old one from the last hid serv request
cache.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Stop using an onion address since it's not indexed with that anymore in the
last hid serv request cache. Instead use a base32 encoded descriptor ID
contained in the rend_data_t object.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Every callsite that use to allocate a rend_data_t object now use the
rend_data_client/service_create() function.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Ground works for fixing #15816. This adds the rend_data_create() function in
order to have a single place where we initialize that data structure.
Furthermore, an array of descriptor IDs is added (one per replica) so we can
keep a copy of the current id in the object. It will be used to purge the
last hid serv request cache using those descriptor IDs. When they change,
they will be replaced and the old ones will be purged from the cache.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Fixes#15850, part of #15801. Change file is added by this commit. The
original comment in the reverted commit is removed because right now we
*need* a DirPort until #15849 is implemented so no doubt nor confusion there
anymore.
This reverts commit 80bed1ac96.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
These commands allow for the creation and management of ephemeral
Onion ("Hidden") services that are either bound to the lifetime of
the originating control connection, or optionally the lifetime of
the tor instance.
Implements #6411.
The SH_LOG_COMPILER feature doesn't work with older automakes, and
those are still in use in many environments we want to support
development on, like Debian Stable.
Instead, use autoconf substitution to fill out the shebang lines on
the shell scripts, and an intermediate make target to make them
executable.
This is a bugfix on the patches for #15344. Bug not in any released
tor.
When we made assertions not get compiled in for the coverage case, we
missed one case where, for our tests, we really DO want to have an
assertion fail: the backtrace test.
Bugfix on 1228dd293b60a8eaab03472fa29428c5e2752c44; bug not in any
released tor
The "longest possible policy" comment in
router_parse_addr_policy_item_from_string() used an example policy
that was actually shorter than the maximum length.
This comment was amended, and expanded to count the maximum number of
characters.
Comment change only.
For this to work bt_test.py now returns an exit code indicating success or
failure. Additionally, check-local and its specific dependencies are now
obsolete so they are removed.
The zero length keys test now requires the path to the Tor binary as the first
parameter to ensure the correct Tor binary is used without hard coding a path.
The wrapper script calls the zero length keys test for each test separately to
ensure the correct shell is used (as configured by autoconf). Another solution
would have been to place the tests into separate functions so multiple tests
could be run internally. This would have made a diff of considerable size and
frankly it is outside the scope of this fix.
Unit tests for the 10 valid combinations of set/NULL config options
DirAuthorities, AlternateBridgeAuthority, AlternateDirAuthority,
and FallbackDir.
Add assertion in consider_adding_dir_servers() for checks in
validate_dir_servers():
"You cannot set both DirAuthority and Alternate*Authority."
Only add the default fallback directories when the DirAuthorities,
AlternateDirAuthority, and FallbackDir directory config options
are set to their defaults.
The default fallback directory list is currently empty, this fix will
only change tor's behaviour when it has default fallback directories.
Fixes bug 15642; bugfix on 90f6071d8d in 0.2.4.7-alpha. Patch by "teor".
When self-testing reachability, use ExtendAllowPrivateAddresses
to determine if local/private addresses imply reachability.
The previous fix used TestingTorNetwork, which implies
ExtendAllowPrivateAddresses, but this excluded rare configs where
ExtendAllowPrivateAddresses is set but TestingTorNetwork is not.
Fixes bug 15771; bugfix on 0.2.6.1-alpha, bug #13924.
Patch by "teor", issue discovered by CJ Ess.
The HS_DESC_CONTENT event results in multiple line thus must be prefixed
with a "650+" and ending with "650 OK".
Reported-by: Damian Johnson <atagar@torproject.org>
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
The HS_DESC event was using rend_data_t from the dir connection to reply the
onion address and authentication type. With the new HSFETCH command, it's
now possible to fetch a descriptor only using the descriptor id thus
resulting in not having an onion address in any HS_DESC event.
This patch removes rend_query from the hs desc control functions and replace
it by an onion address string and an auth type.
On a successful fetch, the service id is taken from the fetched descriptor.
For that, an extra parameter is added to "store as a client" function that
contains the cache entry stored.
This will make the control event functions scale more easily over time if
other values not present in rend_data_t are needed since the rend_data from
the dir connection might not contained everything we need.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Big refactor of the HS client descriptor fetch functionnality. This allows
to fetch an HS descriptor using only a descriptor ID. Furthermore, it's also
possible to provide a list of HSDir(s) now that are used instead of the
automatically choosen one.
The approach taken was to add a descriptor_id field to the rend_data_t
structure so it can be used, if available, by the HS client. The onion
address field however has priority over it that is if both are set, the
onion address is used to fetch the descriptor.
A new public function is introduced called rend_client_fetch_v2_desc(...)
that does NOT lookup the client cache before fetching and can take a list of
HSDirs as a parameter.
The HSFETCH control command now uses this new function thus making it work
and final.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
As defined in section 4.1.26 in the control-spec.txt, this new event replies
the content of a successfully fetched HS descriptor. This also adds a unit
test for the controller event.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
This adds the command on the controller side that parses and validate
arguments but does nothing for now. The HS desriptor fetch must be
modularized a bit more before we can use the command.
See control-spec.txt section 3.26 for more information on this command.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Incidently, this fixes a bug where the maximum value was never used when
only using crypto_rand_int(). For instance this example below in
rendservice.c never gets to INTRO_POINT_LIFETIME_MAX_SECONDS.
int intro_point_lifetime_seconds =
INTRO_POINT_LIFETIME_MIN_SECONDS +
crypto_rand_int(INTRO_POINT_LIFETIME_MAX_SECONDS -
INTRO_POINT_LIFETIME_MIN_SECONDS);
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
An introduction point is currently rotated when the amount of INTRODUCE2
cells reached a fixed value of 16384. This makes it pretty easy for an
attacker to inflate that number and observe when the IP rotates which leaks
the popularity of the HS (amount of client that passed through the IP).
This commit makes it a random count between the current value of 16384 and
two times that.
Fixes#15745
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Till someone writes get_interface_address6 interface enumeration that is
routing table aware, these tests will continue to fail on certain
systems because the get_interface_address6() code is broken.
We no longer base our opinion on whether someone is a directory solely
on the routerstatus we might have for that relay, but also on a
routerinfo. Remove logic in test checking that. This broke unit tests in
05f7336624.
Reported by toralf on #tor-dev, thanks!
This uses a Linux-ism to attempt to always clean up background processes
if possible. Note that it is not a catch-all, in that executables with
suid/sgid or elevated capabilities will have the prctl() attribute
stripped as part of the execve().
Resolves ticket 15471.
Background processes spawned by Tor now will have a valid stdin.
Pluggable transports can detect this behavior with the aformentioned
enviornment variable, and exit if stdin ever gets closed.
The compiler is allowed to assume that a "uint64_t *" is aligned
correctly, and will inline a version of memcpy that acts as such.
Use "uint8_t *", so the compiler does the right thing.
It invokes undefined behavior, I'm afraid, since there's no other
c-legal way to test whether memwipe() works when we're not allowed to
look at it.
Closes ticket 15377.
The rend-spec.txt document doesn't specify this extra newline. Furthermore,
this is the only descryptor type that contains one. Client and HSDir without
this patch still work perfectly since the HS descriptor parsing doesn't
expect a newline at the end.
Fixes#15296
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
On clang (and elsewhere?) __PRETTY_FUNCTION__ includes parenthesized
argument lists. This is clever, but it makes our old "%s(): " format
look funny.
This is a fix on 0957ffeb, aka svn:r288. Fixes bug 15269.
Remove src/or/or_sha1.i and src/common/common_sha1.i on `make clean` and remove
the temporary micro-revision file when its no longer needed.
Additional changes;
- show a message when generating the micro-revision file.
- add the temporary micro revision file to the list of files to be removed on
`make clean` just in case.
- fix indentation of the make rule to improve readability.
PTs expect the auth cookie to be available immedieately after launch,
leading to a race condition when PTs opt to cache the extorport cookie
once immediately after startup.
Fixes#15240.
When calling pthread_attr_setdetachstate, we were using 1 as the
argument. But the pthreads documentation says that you have to say
PTHREAD_CREATE_DETACH, which on Solaris is apparently 0x40. Calling
pthread_attr_setdetachstate with 1 crashes on Solaris with FLTBOUNDS.
(Because we're so late in the release cycle, I made the code define
PTHREAD_CREATE_DETACHED if it doesn't exist, so we aren't likely to
break any other platforms.)
This bug was introduced when we made threading mandatory in
0.2.6.1-alpha; previously, we had force-disabled threading on
Solaris. See #9495 discussion.
