This will give relay operators the ability of disabling the caching of
directory data. In general, this should not be necessary, but on some
lower-resource systems it may beneficial.
According to the POSIX standard the option value is a pointer to void
and the option length a socklen_t. The Windows implementation makes the
option value be a pointer to character and the option length an int.
Casting the option value to a pointer to void conforms to the POSIX
standard while the implicit cast to a pointer to character conforms to
the Windows implementation.
The casts of the option length to the socklen_t data type conforms to
the POSIX standard. The socklen_t data type is actually an alias of an
int so it also conforms to the Windows implementation.
When a relay does not have an open directory port but it has an
orport configured and is accepting client connections then it can
now service tunnelled directory requests, too. This was already true
of relays with an dirport configured.
We also conditionally stop advertising this functionality if the
relay is nearing its bandwidth usage limit - same as how dirport
advertisement is determined.
Partial implementation of prop 237, ticket 12538
These IPv6 addresses must be quoted, because : is the port separator,
and "acce" is a valid hex block.
Add unit tests for assumed actions in IPv6 policies.
"Tor has included a feature to fetch the initial consensus from nodes
other than the authorities for a while now. We just haven't shipped a
list of alternate locations for clients to go to yet.
Reasons why we might want to ship tor with a list of additional places
where clients can find the consensus is that it makes authority
reachability and BW less important.
We want them to have been around and using their current key, address,
and port for a while now (120 days), and have been running, a guard,
and a v2 directory mirror for most of that time."
Features:
* whitelist and blacklist for an opt-in/opt-out trial.
* excludes BadExits, tor versions that aren't recommended, and low
consensus weight directory mirrors.
* reduces the weighting of Exits to avoid overloading them.
* places limits on the weight of any one fallback.
* includes an IPv6 address and orport for each FallbackDir, as
implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks
until #17840 is merged.)
* generated output includes timestamps & Onionoo URL for traceability.
* unit test ensures that we successfully load all included default
fallback directories.
Closes ticket #15775. Patch by "teor".
OnionOO script by "weasel", "teor", "gsathya", and "karsten".
Once tor is downloading a usable consensus, any other connection
attempts are not needed.
Choose a connection to keep, favouring:
* fallback directories over authorities,
* connections initiated earlier over later connections
Close all other connections downloading a consensus.
Prop210: Add attempt-based connection schedules
Existing tor schedules increment the schedule position on failure,
then retry the connection after the scheduled time.
To make multiple simultaneous connections, we need to increment the
schedule position when making each attempt, then retry a (potentially
simultaneous) connection after the scheduled time.
(Also change find_dl_schedule_and_len to find_dl_schedule, as it no
longer takes or returns len.)
Prop210: Add multiple simultaneous consensus downloads for clients
Make connections on TestingClientBootstrapConsensus*DownloadSchedule,
incrementing the schedule each time the client attempts to connect.
Check if the number of downloads is less than
TestingClientBootstrapConsensusMaxInProgressTries before trying any
more connections.
UseDefaultFallbackDirs enables any hard-coded fallback
directory mirrors. Default is 1, set it to 0 to disable fallbacks.
Implements ticket 17576.
Patch by "teor".
Using variables removes the ambiguity about when to use variables and
when to use substitutions. Variables always work. Substitutions only
work when Autoconf knows about them which is not always the case.
The variables are also placed between quotes to ensures spaces in the
variables are handled properly.
Update the code for IPv6 authorities and fallbacks for function
argument changes.
Update unit tests affected by the function argument changes in
the patch.
Add unit tests for authority and fallback:
* adding via a function
* line parsing
* adding default authorities
(Adding default fallbacks is unit tested in #15775.)
The hidden service descriptor cache (rendcache) tests use digest maps
which expect keys to have a length of DIGEST_LEN.
Because the tests use key strings with a length lower than DIGEST_LEN,
the internal copy operation reads outside the key strings which leads to
buffer over-reads.
The issue is resolved by using character arrays with a size of
DIGEST_LEN.
Patch on ade5005853.
The tests pass empty digest strings to the dir_server_new function which
copies it into a directory server structure. The copy operation expects
the digest strings to be DIGEST_LEN characters long.
Because the length of the empty digest strings are lower than
DIGEST_LEN, the copy operation reads outside the digest strings which
leads to buffer over-reads.
The issue is resolved by using character arrays with a size of
DIGEST_LEN.
Patch on 4ff08bb581.
These functions must really never fail; so have crypto_rand() assert
that it's working okay, and have crypto_seed_rng() demand that
callers check its return value. Also have crypto_seed_rng() check
RAND_status() before returning.
Stop ignoring ExitPolicyRejectPrivate in getinfo
exit-policy/reject-private. Fix a memory leak.
Set ExitPolicyRejectPrivate in the unit tests, and make a mock
function declaration static.
Fix unit tests for get_interface_address6_list to assume less
about the interface addresses on the system.
Instead, mock get_interface_address6_list and use the mocked
function to provide a range of address combinations.
This migrates away from SHA1, and provides further hash flooding
protection on top of the randomised siphash implementation.
Add unit tests to make sure that different inputs don't have the
same hash.
exit-policy/reject-private lists the reject rules added by
ExitPolicyRejectPrivate. This makes it easier for stem to
display exit policies.
Add unit tests for getinfo exit-policy/*.
Completes ticket #17183. Patch by "teor".
Modify policies_parse_exit_policy_reject_private so it also blocks
the addresses configured for OutboundBindAddressIPv4_ and
OutboundBindAddressIPv6_, and any publicly routable port addresses
on exit relays.
Add and update unit tests for these functions.
In my testing, an IPv6-only FreeBSD jail without ::1 returned EINVAL
from tor_ersatz_socketpair. Let's not fail the unit test because of
this - it would only ever use tor_socketpair() anyway.
(But it won't work on some systems without IPv4/IPv6 localhost
(some BSD jails) by design, to avoid creating sockets on routable
IP addresses. However, those systems likely have the AF_UNIX socketpair,
which tor prefers.)
Fixes bug #17638; bugfix on a very early tor version,
earlier than 22dba27d8d (23 Nov 2004) / svn:r2943.
Patch by "teor".
Make unit tests pass on IPv6-only systems, and systems without
localhost addresses (like some FreeBSD jails).
Fixes:
* get_if_addrs_ifaddrs: systems without localhost
* get_if_addrs_ioctl: only works on IPv4 systems
* socket: check IPv4 and IPv6, skip on EPROTONOSUPPORT
* socketpair_ersatz: uses IPv4, skip on EPROTONOSUPPORT
Fixes bug #17632; bugfix on unit tests in 0.2.7.3-rc.
c464a36772 was a partial fix for this issue in #17255;
it was released in unit tests in 0.2.7.4-rc.
Patch by "teor".
Make unit tests pass on IPv6-only systems, and systems without
localhost addresses (like some FreeBSD jails).
Fixes:
* get_if_addrs_ifaddrs: systems without localhost
* get_if_addrs_ioctl: only works on IPv4 systems
* socket: check IPv4 and IPv6, skip on EPROTONOSUPPORT
* socketpair_ersatz: uses IPv4, skip on EPROTONOSUPPORT
Fixes bug #17632; bugfix on unit tests in 0.2.7.3-rc.
c464a36772 was a partial fix for this issue in #17255;
it was released in unit tests in 0.2.7.4-rc.
Patch by "teor".
* Don't assume that every test box has an IPv4 address
* Don't assume that every test box has a non-local address
Resolves issue #17255 released in unit tests in 0.2.7.3-rc.
Ensure that either a valid address is returned in address pointers,
or that the address data is zeroed on error.
Ensure that free_interface_address6_list handles NULL lists.
Add unit tests for get_interface_address* failure cases.
Fixes bug #17173.
Patch by fk/teor, not in any released version of tor.
Use environment variables instead. This repairs 'make distcheck',
which was running into trouble when it tried to chmod the generated
scripts.
Fixes 17148.
When we find a conflict in the keypinning journal, treat the new
entry as superseding all old entries that overlap either of its
keys.
Also add a (not-yet-used) configuration option to disable keypinning
enforcement.
src/test/test_policy.c:
Merged calls to policies_parse_exit_policy by adding additional arguments.
fixup to remaining instance of ~EXIT_POLICY_IPV6_ENABLED.
Compacting logic test now produces previous list length of 4, corrected this.
src/config/torrc.sample.in:
src/config/torrc.minimal.in-staging:
Merged torrc modification dates in favour of latest.
ExitPolicyRejectPrivate now rejects more local addresses by default:
* the relay's published IPv6 address (if any), and
* any publicly routable IPv4 or IPv6 addresses on any local interfaces.
This resolves a security issue for IPv6 Exits and multihomed Exits that
trust connections originating from localhost.
Resolves ticket 17027. Patch by "teor".
Patch on 42b8fb5a15 (11 Nov 2007), released in 0.2.0.11-alpha.
The unit tests added in e033d5e90b got malformed_list added to
router_parse_addr_policy_item_from_string calls, but unit tests from
subsequent commits didn't get the extra argument until now.
In previous versions of Tor, ExitPolicy accept6/reject6 * produced
policy entries for IPv4 and IPv6 wildcard addresses.
To reduce operator confusion, change accept6/reject6 * to only produce
an IPv6 wildcard address.
Resolves bug #16069.
Patch on 2eb7eafc9d and a96c0affcb (25 Oct 2012),
released in 0.2.4.7-alpha.
When parsing torrc ExitPolicies, we now warn if:
* an IPv4 address is used on an accept6 or reject6 line. The line is
ignored, but the rest of the policy items in the list are used.
(accept/reject continue to allow both IPv4 and IPv6 addresses in torrcs.)
* a "private" address alias is used on an accept6 or reject6 line.
The line filters both IPv4 and IPv6 private addresses, disregarding
the 6 in accept6/reject6.
When parsing torrc ExitPolicies, we now issue an info-level message:
* when expanding an accept/reject * line to include both IPv4 and IPv6
wildcard addresses.
In each instance, usage advice is provided to avoid the message.
Partial fix for ticket 16069. Patch by "teor".
Patch on 2eb7eafc9d and a96c0affcb (25 Oct 2012),
released in 0.2.4.7-alpha.
Add get_interface_address[6]_list by refactoring
get_interface_address6. Add unit tests for new and existing functions.
Preparation for ticket 17027. Patch by "teor".
Patch on 42b8fb5a15 (11 Nov 2007), released in 0.2.0.11-alpha.
Increase default boostrap time in test-network.sh to 30 seconds,
for larger networks like bridges+ipv6+hs.
This avoids the failure-hiding issues inherent in the retry approach
in #16952.
make test-network-all is Makefile target which verifies a series
of test networks generated using test-network.sh and chutney.
It runs IPv6 and mixed version test networks if the prerequisites are
available.
Each test network reports PASS, FAIL, or SKIP.
Closes ticket 16953. Patch by "teor".
Also adds "--hs-multi-client 1" option to TEST_NETWORK_FLAGS.
This resolves#17012.
Larger networks, such as bridges+hs, may fail until #16952 is merged.
Make "bridges+hs" the default test network. This tests almost all
tor functionality during make test-network, while allowing tests
to succeed on non-IPv6 systems.
Requires chutney commit 396da92 in test-network-bridges-hs.
Closes tickets 16945 (tor), 16946 (chutney) . Patches by "teor".
Service descriptors are now generated regardless of the the
PublishHidServDescriptors option. The generated descriptors are stored
in the service descriptor cache.
The PublishHidServDescriptors = 1 option now prevents descriptor
publication to the HSDirs rather than descriptor generation.
We don't want to accept any work after one of our worker functions has
returned WQ_RPL_SHUTDOWN. This testcase currently fails, because we do
not actually stop any of the worker threads.
Previously we'd put these strings right on the controllers'
outbufs. But this could cause some trouble, for these reasons:
1) Calling the network stack directly here would make a huge portion
of our networking code (from which so much of the rest of Tor is
reachable) reachable from everything that potentially generated
controller events.
2) Since _some_ events (EVENT_ERR for instance) would cause us to
call connection_flush(), every control_event_* function would
appear to be able to reach even _more_ of the network stack in
our cllgraph.
3) Every time we generated an event, we'd have to walk the whole
connection list, which isn't exactly fast.
This is an attempt to break down the "blob" described in
http://archives.seul.org/tor/dev/Mar-2015/msg00197.html -- the set of
functions from which nearly all the other functions in Tor are
reachable.
Closes ticket 16695.
Test that TestingDirAuthVote{Exit,Guard,HSDir}[Strict] work on
routersets matching all routers, one router, and no routers.
TestingDirAuthVote{Exit,Guard,HSDir} set the corresponding flag
on routerstatuses which match the routerset, but leave other flags
unmodified.
TestingDirAuthVote{Exit,Guard,HSDir}Strict clear the corresponding flag
on routerstatuses which don't match the routerset.
URI syntax (and DNS syntax) allows for a single trailing `.` to
explicitly distinguish between a relative and absolute
(fully-qualified) domain name. While this is redundant in that RFC 1928
DOMAINNAME addresses are *always* fully-qualified, certain clients
blindly pass the trailing `.` along in the request.
Fixes bug 16674; bugfix on 0.2.6.2-alpha.
The workqueue test help message has two issues. First, the message uses 4 space
indentation when 2 space indentation seems more common. Second, the help
message misses some options.
This commit fixes both issues.
Add a new and slow unit test that checks if libscrypt_scrypt() and
EBP_PBE_scrypt() yield the same keys from test vectors.
squash! Assert interoperability betweeen libscrypt and OpenSSL EBP_PBE_scrypt().
squash! Assert interoperability betweeen libscrypt and OpenSSL EBP_PBE_scrypt().
squash! Assert interoperability betweeen libscrypt and OpenSSL EBP_PBE_scrypt().
The runtime sanity checking is slightly different from the optimized
basepoint stuff in that it uses a given implementation's self tests if
available, and checks if signing/verification works with a test vector
from the IETF EdDSA draft.
The unit tests include a new testcase that will fuzz donna against ref0,
including the blinding and curve25519 key conversion routines. If this
is something that should be done at runtime (No?), the code can be
stolen from there.
Note: Integrating batch verification is not done yet.
Integration work scavanged from nickm's `ticket8897_9663_v2` branch,
with minor modifications. Tor will still sanity check the output but
now also attempts to catch extreme breakage by spot checking the
optimized implementation vs known values from the NaCl documentation.
Implements feature 9663.
The following arguments change how chutney verifies the network:
--bytes n sends n bytes per test connection (10 KBytes)
--connections n makes n test connections per client (1)
--hs-multi-client 1 makes each client connect to each HS (0)
Requires the corresponding chutney performance testing changes.
Note: using --connections 7 or greater on a HS will trigger #15937.
Patch by "teor".
RFC 952 is approximately 30 years old, and people are failing to comply,
by serving A records with '_' as part of the hostname. Since relaxing
the check is a QOL improvement for our userbase, relax the check to
allow such abominations as destinations, especially since there are
likely to be other similarly misconfigured domains out there.
When I fixed#11243, I made it so we would take the digest of a
descriptor before tokenizing it, so we could desist from download
attempts if parsing failed. But when I did that, I didn't remove an
assertion that the descriptor began with "onion-key". Usually, this
was enforced by "find_start_of_next_microdescriptor", but when
find_start_of_next_microdescriptor returned NULL, the assertion was
triggered.
Fixes bug 16400. Thanks to torkeln for reporting and
cypherpunks_backup for diagnosing and writing the first fix here.
clang 3.7 complains that using a preprocessor directive inside
a macro invocation in test_util_writepid in test_util.c is undefined.
Fix on 79e85313aa on 0.2.7.1-alpha.
# The first commit's message is:
Regenerate ed25519 keys when they will expire soon.
Also, have testing-level options to set the lifetimes and
expiration-tolerances of all key types, plus a non-testing-level
option to set the lifetime of any auto-generated signing key.
# The 2nd commit message will be skipped:
# fixup! Regenerate ed25519 keys when they will expire soon.
Extrainfo documents are now ed-signed just as are router
descriptors, according to proposal 220. This patch also includes
some more tests for successful/failing parsing, and fixes a crash
bug in ed25519 descriptor parsing.
An earlier version of these tests was broken; now they're a nicer,
more robust, more black-box set of tests. The key is to have each
test check a handshake message that is wrong in _one_ way.
When there are annotations on a router descriptor, the
ed25519-identity element won't be at position 0 or 1; it will be at
router+1 or router-1.
This patch also adds a missing smartlist function to search a list for
an item with a particular pointer.
This module implements a key-pinning mechanism to ensure that it's
safe to use RSA keys as identitifers even as we migrate to Ed25519
keys. It remembers, for every Ed25519 key we've seen, what the
associated Ed25519 key is. This way, if we see a different Ed25519
key with that RSA key, we'll know that there's a mismatch.
We persist these entries to disk using a simple format, where each
line has a base64-encoded RSA SHA1 hash, then a base64-endoded
Ed25519 key. Empty lines, misformed lines, and lines beginning with
a # are ignored. Lines beginning with @ are reserved for future
extensions.
Routers now use TAP and ntor onion keys to sign their identity keys,
and put these signatures in their descriptors. That allows other
parties to be confident that the onion keys are indeed controlled by
the router that generated the descriptor.
For prop220, we have a new ed25519 certificate type. This patch
implements the code to create, parse, and validate those, along with
code for routers to maintain their own sets of certificates and
keys. (Some parts of master identity key encryption are done, but
the implementation of that isn't finished)
As OpenSSL >= 1.0.0 is now required, ECDHE is now mandatory. The group
has to be validated at runtime, because of RedHat lawyers (P224 support
is entirely missing in the OpenSSL RPM, but P256 is present and is the
default).
Resolves ticket #16140.
With #15881 implemented, this adds the missing descriptor ID at the end of
the expected control message.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
For FAILED and RECEIVED action of the HS_DESC event, we now sends back the
descriptor ID at the end like specified in the control-spec section 4.1.25.
Fixes#15881
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
- Rewrite changes file.
- Avoid float comparison with == and use <= instead.
- Add teor's tor_llround(trunc(...)) back to silence clang warnings.
- Replace tt_assert() with tt_i64_op() and friends.
- Fix whitespace and a comment.
Consistently check for overflow in round_*_to_next_multiple_of.
Check all round_*_to_next_multiple_of functions with expected values.
Check all round_*_to_next_multiple_of functions with maximal values.
Related to HS stats in #13192.
Avoid division by zero.
Avoid taking the log of zero.
Silence clang type conversion warnings using round and trunc.
The existing values returned by the laplace functions do not change.
Add tests for laplace edge cases.
These changes pass the existing unit tests without modification.
Related to HS stats in #13192.
These commands allow for the creation and management of ephemeral
Onion ("Hidden") services that are either bound to the lifetime of
the originating control connection, or optionally the lifetime of
the tor instance.
Implements #6411.
The SH_LOG_COMPILER feature doesn't work with older automakes, and
those are still in use in many environments we want to support
development on, like Debian Stable.
Instead, use autoconf substitution to fill out the shebang lines on
the shell scripts, and an intermediate make target to make them
executable.
This is a bugfix on the patches for #15344. Bug not in any released
tor.
When we made assertions not get compiled in for the coverage case, we
missed one case where, for our tests, we really DO want to have an
assertion fail: the backtrace test.
Bugfix on 1228dd293b60a8eaab03472fa29428c5e2752c44; bug not in any
released tor
For this to work bt_test.py now returns an exit code indicating success or
failure. Additionally, check-local and its specific dependencies are now
obsolete so they are removed.
The zero length keys test now requires the path to the Tor binary as the first
parameter to ensure the correct Tor binary is used without hard coding a path.
The wrapper script calls the zero length keys test for each test separately to
ensure the correct shell is used (as configured by autoconf). Another solution
would have been to place the tests into separate functions so multiple tests
could be run internally. This would have made a diff of considerable size and
frankly it is outside the scope of this fix.
Unit tests for the 10 valid combinations of set/NULL config options
DirAuthorities, AlternateBridgeAuthority, AlternateDirAuthority,
and FallbackDir.
Add assertion in consider_adding_dir_servers() for checks in
validate_dir_servers():
"You cannot set both DirAuthority and Alternate*Authority."
The HS_DESC event was using rend_data_t from the dir connection to reply the
onion address and authentication type. With the new HSFETCH command, it's
now possible to fetch a descriptor only using the descriptor id thus
resulting in not having an onion address in any HS_DESC event.
This patch removes rend_query from the hs desc control functions and replace
it by an onion address string and an auth type.
On a successful fetch, the service id is taken from the fetched descriptor.
For that, an extra parameter is added to "store as a client" function that
contains the cache entry stored.
This will make the control event functions scale more easily over time if
other values not present in rend_data_t are needed since the rend_data from
the dir connection might not contained everything we need.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
As defined in section 4.1.26 in the control-spec.txt, this new event replies
the content of a successfully fetched HS descriptor. This also adds a unit
test for the controller event.
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Till someone writes get_interface_address6 interface enumeration that is
routing table aware, these tests will continue to fail on certain
systems because the get_interface_address6() code is broken.
We no longer base our opinion on whether someone is a directory solely
on the routerstatus we might have for that relay, but also on a
routerinfo. Remove logic in test checking that. This broke unit tests in
05f7336624.
Reported by toralf on #tor-dev, thanks!
Background processes spawned by Tor now will have a valid stdin.
Pluggable transports can detect this behavior with the aformentioned
enviornment variable, and exit if stdin ever gets closed.
It invokes undefined behavior, I'm afraid, since there's no other
c-legal way to test whether memwipe() works when we're not allowed to
look at it.
Closes ticket 15377.
