If we can't detect the physical memory, the new default is 8 GB on
64-bit architectures, and 1 GB on 32-bit architectures.
If we *can* detect the physical memory, the new default is
CLAMP(256 MB, phys_mem * 0.75, MAX_DFLT)
where MAX_DFLT is 8 GB on 64-bit architectures and 2 GB on 32-bit
architectures.
You can still override the default by hand. The logic here is simply
trying to choose a lower default value on systems with less than 12 GB
of physical RAM.
Use a per-channel ratelim_t to control the rate at which we report
failures for each channel.
Explain why I picked N=32.
Never return a zero circID.
Thanks to Andrea and to cypherpunks.
The memarea_strndup() function would have hit undefined behavior by
creating an 'end' pointer off the end of a string if it had ever been
given an 'n' argument bigger than the length of the memory ares that
it's scanning. Fortunately, we never did that except in the unit
tests. But it's not a safe behavior to leave lying around.
If we had an address of the form "1.2.3.4" and we tried to pass it to
tor_inet_pton with AF_INET6, it was possible for our 'eow' pointer to
briefly move backwards to the point before the start of the string,
before we moved it right back to the start of the string. C doesn't
allow that, and though we haven't yet hit a compiler that decided to
nuke us in response, it's best to fix.
So, be more explicit about requiring there to be a : before any IPv4
address part of the IPv6 address. We would have rejected addresses
without a : for not being IPv6 later on anyway.
Instead of taking the length of a buffer, we were taking the length of
a pointer, so that our debugging log would cover only the first
sizeof(void*) bytes of the client nonce.
scan-build recognizes that in theory there could be a numeric overflow
here.
This can't numeric overflow can't trigger IRL, since in order to fill a
hash table with more than P=402653189 buckets with a reasonable load
factor of 0.5, we'd first have P/2 malloced objects to put in it--- and
each of those would have to take take at least sizeof(void*) worth of
malloc overhead plus sizeof(void*) content, which would run you out of
address space anyway on a 32-bit system.
If 'intro' is NULL in these functions, I'm pretty sure that the
error message must be set before we hit the end. But scan-build
doesn't notice that, and is worried that we'll do a null-pointer
dereference in the last-chance errormsg generation.
As it stands, it relies on the fact that onion_queue_entry_remove
will magically remove each onionskin from the right list. This
patch changes the logic to be more resilient to possible bugs in
onion_queue_entry_remove, and less confusing to static analysis tools.
scan-build doesn't realize that a request can't be timed at the end
unless it's timed at the start, and so it's not possible for us to
be subtracting start from end without start being set.
Nevertheless, let's not confuse it.
When get_proxy_addrport returned PROXY_NONE, it would leave
addr/port unset. This is inconsistent, and could (if we used the
function in a stupid way) lead to undefined behavior. Bugfix on
5b050a9b0, though I don't think it affects tor-as-it-is.
Throughout circuituse, when we log about a circuit, we log its
desired path length from build_state. scan-build is irrationally
concerned that build_state might be NULL.
In circuitmux_detach_all_circuits, we check whether an HT iterator
gives us NULL. That should be impossible for an HT iterator. But
our checking it has confused scan-build (justly) into thinking that
our later use of HT_NEXT_RMV might not be kosher. I'm taking the
coward's route here and strengthening the check. Bugfix on
fd31dd44. (Not a real bug though)
If we fail in circuit_get_by_rend_token_and_purpose because the
circuit has no rend_info, don't try to reference fiends from its
rend_info when logging an error. Bugfix on 8b9a2cb68, which is
going into Tor 0.2.5.4-alpha.
Previously we said "Sandbox is not implemented on this platform" on
Linux boxes without libseccomp. Now we say that you need to build
Tor built with libseccomp. Fixes bug 11543; bugfix on 0.2.5.1-alpha.
Fixes a possible root cause of 11553 by only making 64 attempts at
most to pick a circuitID. Previously, we would test every possible
circuit ID until we found one or ran out.
This algorithm succeeds probabilistically. As the comment says:
This potentially causes us to give up early if our circuit ID
space is nearly full. If we have N circuit IDs in use, then we
will reject a new circuit with probability (N / max_range) ^
MAX_CIRCID_ATTEMPTS. This means that in practice, a few percent
of our circuit ID capacity will go unused.
The alternative here, though, is to do a linear search over the
whole circuit ID space every time we extend a circuit, which is
not so great either.
This makes new vs old clients distinguishable, so we should try to
batch it with other patches that do that, like 11438.
The server cipher list is (thanks to #11513) chosen systematically to
put the best choices for Tor first. The client cipher list is chosen
to resemble a browser. So let's set SSL_OP_CIPHER_SERVER_PREFERENCE
to have the servers pick according to their own preference order.
This means that tor can run without needing to communicate with ioctls
to the firewall, and therefore doesn't need to run with privileges to
open the /dev/pf device node.
A new TransProxyType is added for this purpose, "pf-divert"; if the user
specifies this TransProxyType in their torrc, then the pf device node is
never opened and the connection destination is determined with getsockname
(as per pf(4)). The default behaviour (ie., when TransProxyType is "default"
when using the pf firewall) is still to assume that pf is configured with
rdr-to rules.
This isn't on by default; to get it, you need to set "TransProxyType
ipfw". (The original patch had automatic detection for whether
/dev/pf is present and openable, but that seems marginally fragile.)
Older versions of Libevent are happy to open SOCK_DGRAM sockets
non-cloexec and non-nonblocking, and then set those flags
afterwards. It's nice to be able to allow a flag to be on or off in
the sandbox without having to enumerate all its values.
Also, permit PF_INET6 sockets. (D'oh!)
Libevent uses an arc4random implementation (I know, I know) to
generate DNS transaction IDs and capitalization. But it liked to
initialize it either with opening /dev/urandom (which won't work
under the sandbox if it doesn't use the right pointer), or with
sysctl({CTL_KERN,KERN_RANDOM,RANDOM_UUIC}). To make _that_ work, we
were permitting sysctl unconditionally. That's not such a great
idea.
Instead, we try to initialize the libevent PRNG _before_ installing
the sandbox, and make sysctl always fail with EPERM under the
sandbox.
The compiler doesn't warn about this code:
rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(openat), 1,
SCMP_CMP(0, SCMP_CMP_EQ, AT_FDCWD),
SCMP_CMP(1, SCMP_CMP_EQ, param->value),
SCMP_CMP(2, SCMP_CMP_EQ, O_RDONLY|...));
but note that the arg_cnt argument above is only 1. This means that
only the first filter (argument 0 == AT_FDCWD) is actually checked!
This patch also fixes the above error in the openat() filter.
Earlier I fixed corresponding errors in filters for rename() and
mprotect().
Appearently, the majority of the filenames we pass to
sandbox_cfg_allow() functions are "freeable right after". So, consider
_all_ of them safe-to-steal, and add a tor_strdup() in the few cases
that aren't.
(Maybe buggy; revise when I can test.)
(If we don't restrict rename, there's not much point in restricting
open, since an attacker could always use rename to make us open
whatever they want.)
A new set of unit test cases are provided, as well as introducing
an alternative paradigm and macros to support it. Primarily, each test
case is given its own namespace, in order to isolate tests from each
other. We do this by in the usual fashion, by appending module and
submodule names to our symbols. New macros assist by reducing friction
for this and other tasks, like overriding a function in the global
namespace with one in the current namespace, or declaring integer
variables to assist tracking how many times a mock has been called.
A set of tests for a small-scale module has been included in this
commit, in order to highlight how the paradigm can be used. This
suite gives 100% coverage to status.c in test execution.
Back in 175b2678, we allowed servers to recognize clients who are
telling them the truth about their ciphersuites, and select the best
cipher from on that list. This implemented the server side of proposal
198.
In bugs 11492, 11498, and 11499, cypherpunks found a bunch of mistakes
and omissions and typos in the UNRESTRICTED_SERVER_CIPHER_LIST we had.
In #11513, I found a couple more.
Rather than try to hand-edit this list, I wrote a short python script
to generate our ciphersuite preferences from the openssl headers.
The new rules are:
* Require forward secrecy.
* Require RSA (since our servers only configure RSA keys)
* Require AES or 3DES. (This means, reject RC4, DES, SEED, CAMELLIA,
and NULL.)
* No export ciphersuites.
Then:
* Prefer AES to 3DES.
* If both suites have the same cipher, prefer ECDHE to DHE.
* If both suites have the same DHE group type, prefer GCM to CBC.
* If both suites have the same cipher mode, prefer SHA384 to SHA256
to SHA1.
* If both suites have the same digest, prefer AES256 to AES128.
This involves some duplicate code between backtrace.c and sandbox.c,
but I don't see a way around it: calling more functions would mean
adding more steps to our call stack, and running clean_backtrace()
against the wrong point on the stack.
My first implementation was broken, since it returned "whether there
is one bridge" rather than "how many bridges."
Also, the implementation for the n_options_out feature in
choose_random_entry_impl was completely broken due to a missing *.
When we successfully create a usable circuit after it previously
timed out for a certain amount of time, we should make sure that
our public IP address hasn't changed and update our descriptor.
The major changes are to re-order some ciphers, to drop the ECDH suites
(note: *not* ECDHE: ECDHE is still there), to kill off some made-up
stuff (like the SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA suite), to drop
some of the DSS suites... *and* to enable the ECDHE+GCM ciphersuites.
This change is autogenerated by get_mozilla_ciphers.py from
Firefox 28 and OpenSSL 1.0.1g.
Resolves ticket 11438.
In C, it's a bad idea to do this:
char *cp = array;
char *end = array + array_len;
/* .... */
if (cp + 3 >= end) { /* out of bounds */ }
because cp+3 might be more than one off the end of the array, and
you are only allowed to construct pointers to the array elements,
and to an element one past the end. Instead you have to say
if (cp - array + 3 >= array_len) { /* ... */ }
or something like that.
This patch fixes two of these: one in process_versions_cell
introduced in 0.2.0.10-alpha, and one in process_certs_cell
introduced in 0.2.3.6-alpha. These are both tracked under bug
10363. "bobnomnom" found and reported both. See also 10313.
In our code, this is likely to be a problem as we used it only if we
get a nasty allocator that makes allocations end close to (void*)-1.
But it's best not to have to worry about such things at all, so
let's just fix all of these we can find.
According to reports, most programs degrade somewhat gracefully on
getting no answer for an MX or a CERT for www.example.com, but many
flip out completely on a NOTIMPL error.
Also, treat a QTYPE_ALL query as just asking for an A record.
The real fix here is to implement proposal 219 or something like it.
Fixes bug 10268; bugfix on 0.2.0.1-alpha.
Based on a patch from "epoch".
Found by testing with chutney. The old behavior was "fail an
assertion", which obviously isn't optimal.
Bugfix on 8b9a2cb68b290e550695124d7ef0511225b451d5; bug not in any
released version.
Also, stop accepting the old kind of RESOLVED cells with no TTL
fields; they haven't been sent since 0.1.1.6-alpha.
This patch won't work without the fix to #10468 -- it will break
DNSPorts unless they set the proper ipv4/6 flags on entry_connection_t.
Otherwise, it could mung the thing that came over the net on windows,
which would defeat the purpose of recording the unparseable thing.
Fixes bug 11342; bugfix on 0.2.2.1-alpha.
This is a fix for 9963. I say this is a feature, but if it's a
bugfix, it's a bugfix on 0.2.4.18-rc.
Old behavior:
Mar 27 11:02:19.000 [notice] Bootstrapped 50%: Loading relay descriptors.
Mar 27 11:02:20.000 [notice] Bootstrapped 51%: Loading relay descriptors.
Mar 27 11:02:20.000 [notice] Bootstrapped 52%: Loading relay descriptors.
... [Many lines omitted] ...
Mar 27 11:02:29.000 [notice] Bootstrapped 78%: Loading relay descriptors.
Mar 27 11:02:33.000 [notice] We now have enough directory information to build circuits.
New behavior:
Mar 27 11:16:17.000 [notice] Bootstrapped 50%: Loading relay descriptors
Mar 27 11:16:19.000 [notice] Bootstrapped 55%: Loading relay descriptors
Mar 27 11:16:21.000 [notice] Bootstrapped 60%: Loading relay descriptors
Mar 27 11:16:21.000 [notice] Bootstrapped 65%: Loading relay descriptors
Mar 27 11:16:21.000 [notice] Bootstrapped 70%: Loading relay descriptors
Mar 27 11:16:21.000 [notice] Bootstrapped 75%: Loading relay descriptors
Mar 27 11:16:21.000 [notice] We now have enough directory information to build circuits.
Most of these are simple. The only nontrivial part is that our
pattern for using ENUM_BF was confusing doxygen by making declarations
that didn't look like declarations.
In circuitlist_free_all, we free all the circuits, removing them from
the map as we go, but we weren't actually freeing the placeholder
entries that we use to indicate pending DESTROY cells.
Fix for bug 11278; bugfix on the 7912 code that was merged in
0.2.5.1-alpha
There are still quite a few 0.2.3.2x relays running for x<5, and while I
agree they should upgrade, I don't think cutting them out of the network
is a net win on either side.
ubsan doesn't like us to do (1u<<32) when 32 is wider than
unsigned. Fortunately, we already special-case
addr_mask_get_bits(0), so we can just change the loop bounds.
In digestmap_set/get benchmarks, doing unaligned access on x86
doesn't save more than a percent or so in the fast case. In the
slow case (where we cross a cache line), it could be pretty
expensive. It also makes ubsan unhappy.
This make clang's memory sanitizer happier that we aren't reading
off the end of a char[1]. We hadn't replaced the char[1] with a
char[FLEXIBLE_ARRAY_MEMBER] before because we were doing a union
trick to force alignment. Now we use __attribute__(aligned) where
available, and we do the union trick elsewhere.
Most of this patch is just replacing accesses to (x)->u.mem with
(x)->U_MEM, where U_MEM is defined as "u.mem" or "mem" depending on
our implementation.
This contains the obvious implementation using the circuitmux data
structure. It also runs the old (slow) algorithm and compares
the results of the two to make sure that they're the same.
Needs review and testing.
This change prevents LD_BUG warnings and bootstrap failure messages
when we try to do directory fetches when starting with
DisableNetwork == 1, a consensus present, but no descriptors (or
insufficient descriptors) yet.
Fixes bug 11200 and bug 10405. It's a bugfix on 0.2.3.9-alpha.
Thanks to mcs for walking me through the repro instructions!
This is meant to be a better bug 9229 fix -- or at least, one more
in tune with the intent of the original code, which calls
router_retry_directory_downloads() only on the first bridge descriptor.
This prevents long stalls when we're starting with a state file but
with no bridge descriptors. Fixes bug 9229. I believe this bug has
been present since 0.2.0.3-alpha.
By default, after you've made a connection to port XYZ, we assume
you might still want to have an exit ready to connect to XYZ for one
hour. This patch lets you lower that interval.
Implements ticket 91
This should fixes some "hey, that function could have
__attribute__((noreturn))" warnings introduced by f96400d9.
Bug not in any released version of Tor.
We have ignored any ports listed here since 80365b989 (0.0.7rc1),
but we didn't warn the user that we were ignoring them. This patch
adds a warning if you put explicit ports in any of the options
{Socks,Dir}Policy or AuthDir{Reject,Invalid,BadDir,BadExit}. It
also adjusts the manpage to say that ports are ignored.
Fixes ticket 11108.
There was one "missing prototype" warning because the test function
wasn't static, and one "unused parameter" warning about the "data"
parameter.
Also, I added a couple of tests to make sure that the "make_null"
addresses really were the addresses we expected, by formatting them
as strings.
In a couple of places, to implement the OOM-circuit-killer defense
against sniper attacks, we have counters to remember the age of
cells or data chunks. These timers were based on wall clock time,
which can move backwards, thus giving roll-over results for our age
calculation. This commit creates a low-budget monotonic time, based
on ratcheting gettimeofday(), so that even in the event of a time
rollback, we don't do anything _really_ stupid.
A future version of Tor should update this function to do something
even less stupid here, like employ clock_gettime() or its kin.
See 1d2179bc90 in master for details.
"""
Fall back to registered country if necessary.
When extracting geoip and geoip6 files from MaxMind's GeoLite2 Country
database, we only look at country->iso_code which is the two-character ISO
3166-1 country code of the country where MaxMind believes the end user is
located.
But if MaxMind thinks a range belongs to anonymous proxies, they don't put
anything there. Hence, we omit those ranges and resolve them all to '??'.
That's not what we want.
What we should do is first try country->iso_code, and if there's no such
key, try registered_country->iso_code which is the country in which the
ISP has registered the IP address.
In short: let's fill all A1 entries with what ARIN et. al think.
"""
When extracting geoip and geoip6 files from MaxMind's GeoLite2 Country
database, we only look at country->iso_code which is the two-character ISO
3166-1 country code of the country where MaxMind believes the end user is
located.
But if MaxMind thinks a range belongs to anonymous proxies, they don't put
anything there. Hence, we omit those ranges and resolve them all to '??'.
That's not what we want.
What we should do is first try country->iso_code, and if there's no such
key, try registered_country->iso_code which is the country in which the
ISP has registered the IP address.
In short: let's fill all A1 entries with what ARIN et. al think.
If the cert turns out to be invalid or if wget is otherwise unable to
verify it, it's going to return an error and not download the file for us.
Spotted by nickm.
It's possible for two threads to hit assertion failures at the same
time. If that happens, let's keep them from stomping on the same
cb_buf field.
Fixes bug 11048; bugfix on 0.2.5.2-alpha. Reported by "cypherpunks".
Back in 5e762e6a5c, non-exit servers
stopped launching DNS requests for users. So there's no need for them
to see if their DNS answers are hijacked.
Patch from Matt Pagan. I think this is a 965 fix.
For a client using a SocksPort connection and IPv6, the connect reply
from tor daemon did not handle AF_INET6 thus sending back the wrong
payload to the client.
A changes file is provided and this fixes#10987
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
Since the first stat call is made for it to deliberately fail, and we
reference st.st_mode without st having valid data, st.st_mode can contain
garbage and cause chmod to fail with EINVAL. We rerun stat and ensure it
succeeded.
Also make use of tt_abort_perror, to properly convey failure reasons to
the user.
clang 3.4 introduced a new by-default warning about unused static
functions, which we triggered heavily for the hashtable and map function
generating macros. We can use __attribute__ ((unused)) (thanks nickm for
the suggestion :-) ) to silence these warnings.
When we have more than two return values, we should really be using
an enum rather than "-2 means this, -1 means that, 0 means this, and
1 or more means a number."
On busy servers, this function takes up something like 3-7% in
different profiles, and gets invoked every time we need to participate
as the midpoint in a hidden service.
So maybe walking through a linked list of all the circuits here wasn't
a good idea.
The latest GeoLite2 database includes a pointer from 2001::/32 to the root
node of the IPv4 address space in the tree. We need to exclude this whole
address space from geoip6, similar to how we exclude IPv4-mapped IPv6
addresses and the 6to4 mapping subnet.
We log only one message, containing a complete list of what's
wrong. We log the complete list whenever any of the possible things
that could have gotten wrong gets worse.
Fix for #9870. Bugfix on 10480dff01, which we merged in
0.2.5.1-alpha.
If you had a resolv.conf file with a nameserver line containing no
nameserver IP, we would crash. That's not terrible, but it's not
desirable.
Fixes bug 8788; bugfix on 0.1.1.23. Libevent already has this fix.
This patch splits out some of the functions in OOM handling so that
it's easier to check them without involving the rest of Tor or
requiring that the circuits be "wired up".
It's increasingly apparent that we want to make sure we initialize our
PRNG nice and early, or else OpenSSL will do it for us. (OpenSSL
doesn't do _too_ bad a job, but it's nice to do it ourselves.)
We'll also need this for making sure we initialize the siphash key
before we do any hashes.
I've made an exception for cases where I'm sure that users can't
influence the inputs. This is likely to cause a slowdown somewhere,
but it's safer to siphash everything and *then* look for cases to
optimize.
This patch doesn't actually get us any _benefit_ from siphash yet,
since we don't really randomize the key at any point.
siphash is a hash function designed for producing hard-to-predict
64-bit outputs from short inputs and a 128-bit key. It's chosen for
security and speed.
See https://131002.net/siphash/ for more information on siphash.
Source: https://github.com/majek/csiphash/
These options were added back in 0.1.2.5-alpha, but no longer make any
sense now that all directories support tunneled connections and
BEGIN_DIR cells. These options were on by default; now they are
always-on.
This is a fix for 10849, where TunnelDirConns 0 would break hidden
services -- and that bug arrived, I think, in 0.2.0.10-alpha.
(There is no longer meaningfully any such thing as a HS authority,
since we stopped uploading or downloading v0 hs descriptors in
0.2.2.1-alpha.)
Implements #10881, and part of #10841.
Apparently fedora currently has ECDH but not P224. This isn't a huge
deal, since we no longer use OpenSSL's P224 ever (see #9780 and
72c1e5acfe). But we shouldn't have segfaulting benchmarks really.
Fixes bug 10835; bugfix on 0.2.4.8-alpha.
This time, we use a pthread_attr to make sure that if pthread_create
succeeds, the thread is successfully detached.
This probably isn't the big thing going on with 4345, since it'd be
a bit weird for pthread_detach to be failing. But it's worth
getting it right.
This patch removes an "if (chan)" that occurred at a place where
chan was definitely non-NULL. Having it there made some static
analysis tools conclude that we were up to shenanigans.
This resolves#9979.
Right now this accounts for about 1% of circuits over all, but if you
pick a guard that's running 0.2.3, it will be about 6% of the circuits
running through that guard.
Making sure that every circuit has at least one ntor link means that
we're getting plausibly good forward secrecy on every circuit.
This implements ticket 9777,
It's possible to set your ExitNodes to contains only exits that don't
have the Exit flag. If you do that, we'll decide that 0 of your exits
are working. Instead, in that case we should look at nodes which have
(or which might have) exit policies that don't reject everything.
Fix for bug 10543; bugfix on 0.2.4.10-alpha.
According to control spec, longname should not contain any spaces and is
consists only of identy_digest + nickname
added two functions:
* node_get_verbose_nickname_by_id()
* node_describe_longname_by_id()
My OSX laptop rightly gave a warning because of sticking strlen() into
an int, but once I took a closer look... it appears that the strlen()
was part of a needlessly verbose implementation for tor_strdup().
While I was there, I fixed the usage of tor_free() in test_hs.c: It
checks for NULL, and it zeros its argument. So instead of
if (foo) {
tor_free(foo);
foo = NULL;
}
we should just say
tor_free(foo);
If you want a slow shutdown, send SIGNAL SHUTDOWN.
(Why not just have the default be SIGNAL QUIT? Because this case
should only happen when an owning controller has crashed, and a
crashed controller won't be able to give the user any "tor is
shutting down" feedback, and so the user gets confused for a while.
See bug 10449 for more info)
Improvement on f308adf838, where we made the ntor
unit tests run everywhere... so long as a python curve25519 module
was installed. Now the unit tests don't require that module.
