The next series of commits begins addressing the issue that we're
currently including the complete or.h file in all of our source files.
To change that, we're splitting function definitions into new header
files (one header file per source file).
Right now it says "552 internal error" because there's no way for
getinfo_helper_*() countries to specify an error message. This
patch changes the getinfo_helper_*() interface, and makes most of the
getinfo helpers give useful error messages in response to failures.
This should prevent recurrences of bug 1699, where a missing GeoIPFile
line in the torrc made GETINFO ip-to-county/* fail in a "not obvious
how to fix" way.
V3 authorities no longer decide not to vote on Guard+Exit. The bandwidth
weights should take care of this now.
Also, lower the max threshold for WFU to 0.98, to allow more nodes to become
guards.
This should make us conflict less with system files named "log.h".
Yes, we shouldn't have been conflicting with those anyway, but some
people's compilers act very oddly.
The actual change was done with one "git mv", by editing
Makefile.am, and running
find . -name '*.[ch]' | xargs perl -i -pe 'if (/^#include.*\Wlog.h/) {s/log.h/torlog.h/; }'
We now record large times as abandoned, to prevent a filter step from
happening and skewing our results.
Also, issue a warn for a rare case that can happen for funky values of Xm or
too many abandoned circuits. Can happen (very rarely) during unit tests, but
should not be possble during live operation, due to network liveness filters
and discard logic.
Many friendly operating systems have 64-bit times, and it's not nice
to pass them to an %ld format.
It's also extremely not-nice to write a time to the log as an
integer. Most people think it's 2010 June 29 23:57 UTC+epsilon, not
1277855805+epsilon.
These timers behave better with non-monotonic clocks than our old
ones, and also try harder to make once-per-second events get called
one second apart, rather than one-plus-epsilon seconds apart.
This fixes bug 943 for everybody using Libevent 2.0 or later.
From the code:
zlib 1.2.4 and 1.2.5 do some "clever" things with macros. Instead of
saying "(defined(FOO) ? FOO : 0)" they like to say "FOO-0", on the theory
that nobody will care if the compile outputs a no-such-identifier warning.
Sorry, but we like -Werror over here, so I guess we need to define these.
I hope that zlib 1.2.6 doesn't break these too.
Possible fix for bug 1526.
This should never happen unless openssl is buggy or some of our
assumptions are deeply wrong, but one of those might have been the
cause of the not-yet-reproducible bug 1209. If it ever happens again,
let's get some info we can use.
We need to ensure that we close timeout measurement circuits. While
we're at it, we should close really old circuits of certain types that
aren't in use, and log really old circuits of other types.
We need to record different statistics at point of timeout, vs the point
of forcible closing.
Also, give some better names to constants and state file variables
to indicate they are not dealing with timeouts, but abandoned circuits.
Having ~/.tor expand into /.tor is, after all, almost certainly not
what the user wanted, and it deserves a warning message.
Also, convert a guess-and-malloc-and-sprintf triple into an asprintf.
In rare cases, we could cannibalize a one-hop circuit, ending up
with a two-hop circuit. This circuit would not be actually used,
but we should prevent its creation in the first place.
Thanks to outofwords and swissknife for helping to analyse this.
Most of the changes here are switches to use APIs available on Windows
CE. The most pervasive change is that Windows CE only provides the
wide-character ("FooW") variants of most of the windows function, and
doesn't support the older ASCII verions at all.
This patch will require use of the wcecompat library to get working
versions of the posix-style fd-based file IO functions.
[commit message by nickm]
Back when we changed the idea of a connection being "too old" for new
circuits into the connection being "bad" for new circuits, we didn't
actually change the info messages. This led to telling the user that
we were labelling connections as "too old" for being worse than
connections that were actually older than them.
Found by Scott on or-talk.
There are now four ways that CBT can be disabled:
1. Network-wide, with the cbtdisabled consensus param.
2. Via config, with "LearnCircuitBuildTimeout 0"
3. Via config, with "AuthoritativeDirectory 1"
4. Via a state file write failure.
This should prevent some asserts and storage of incorrect build times
for the cases where Tor is suspended during a circuit construction, or
just after completing a circuit. The idea is that if the circuit
build time is much greater than we would have cut it off at, we probably
had a suspend event along this codepath, and we should discard the
value.
In case we decide that the timeout rate is now too high due to our
change of the max synthetic quantile value, this consensus parameter
will allow us to restore it to the previous value.
This is for the other issue we saw in Bug 1335. A large number of high
timeouts were causing the timeout calculation to slowly drift upwards,
especially in conditions of load. This fix repeatedly regenerates all of
our synthetic timeouts whenever the timeout changes, to try to prevent
drift.
It also lowers the timeout cap to help for some cases of Bug 1245, where
some timeout values were so large that we ended up allocating a ton of
scratch memory to count the histogram bins.
The downside is that lowering this cap is affecting our timeout rate.
Unfortunately, the buildtimeout quantile is now higher than the actual
completion rate by what appears to be about 7-10%, which probably
represents the skew in the distribution due to lowering this synthetic
cap.
In my state files, I was seeing several peaks, probably due to different
guards having different latency. This change is meant to better capture
this behavior and generate more reasonable timeouts when it happens. It
is improving the timeout values for my collection of state files.
what's happening here is that we're fetching certs for obsolete
authorities -- probably legacy signers in this case. but try to
remain general in the log message.
It's natural for the definition of bandwidth_rule_t to be with the functions
that actually care about its values. Unfortunately, this means declaring
bandwidth_rate_rule_to_string() out of sequence. Someday we'll just rename
reasons.c to strings.c, and put it at the end of or.h, and this will all be
better.
1) mingw doesn't have _vscprintf(); mingw instead has a working snprintf.
2) windows compilers that _do_ have a working _vscprintf spell it so; they do
not spell it _vcsprintf().
Works like the --enable-static-openssl/libevent options. Requires
--with-zlib-dir to be set. Note that other dependencies might still
pull in a dynamicly linked zlib, if you don't link them in statically
too.
Our code assumed that any version of OpenSSL before 0.9.8l could not
possibly require SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION. This is
so... except that many vendors have backported the flag from later
versions of openssl when they backported the RFC5476 renegotiation
feature.
The new behavior is particularly annoying to detect. Previously,
leaving SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION unset meant that
clients would fail to renegotiate. People noticed that one fast!
Now, OpenSSL's RFC5476 support means that clients will happily talk to
any servers there are, but servers won't accept renegotiation requests
from unpatched clients unless SSL_OP_ALLOW_etc is set. More fun:
servers send back a "no renegotiation for you!" error, which unpatched
clients respond to by stalling, and generally producing no useful
error message.
This might not be _the_ cause of bug 1346, but it is quite likely _a_
cause for bug 1346.
Everything that accepted the 'Circ' name handled it wrong, so even now
that we fixed the handling of the parameter, we wouldn't be able to
set it without making all the 0.2.2.7..0.2.2.10 relays act wonky.
This patch makes Tors accept the 'Circuit' name instead, so we can
turn on circuit priorities without confusing the versions that treated
the 'Circ' name as occasion to act weird.
I'm adding this because I can never remember what stuff like 'rule 3'
means. That's the one where if somebody goes limp or taps out, the
fight is over, right?
When you mean (a=b(c,d)) >= 0, you had better not say (a=b(c,d)>=0).
We did the latter, and so whenever CircPriorityHalflife was in the
consensus, it was treated as having a value of 1 msec (that is,
boolean true).
We need to make sure we have an event_base in dns.c before we call
anything that wants one. Make sure we always have one in dns_reset()
when we're a client. Fixes bug 1341.
Now if you're a published relay and you set RefuseUnknownExits, even
if your dirport is off, you'll fetch dir info from the authorities,
fetch it early, and cache it.
In the future, RefuseUnknownExits (or something like it) will be on
by default.
From http://archives.seul.org/tor/relays/Mar-2010/msg00006.html :
As I understand it, the bug should show up on relays that don't set
Address to an IP address (so they need to resolve their Address
line or their hostname to guess their IP address), and their
hostname or Address line fails to resolve -- at that point they'll
pick a random 4 bytes out of memory and call that their address. At
the same time, relays that *do* successfully resolve their address
will ignore the result, and only come up with a useful address if
their interface address happens to be a public IP address.
When the bandwidth-weights branch added the "directory-footer"
token, and began parsing the directory footer at the first
occurrence of "directory-footer", it made it possible to fool the
parsing algorithm into accepting unsigned data at the end of a
consensus or vote. This patch fixes that bug by treating the footer
as starting with the first "directory-footer" or the first
"directory-signature", whichever comes first.
Treat strings returned from signed_descriptor_get_body_impl() as not
NUL-terminated. Since the length of the strings is available, this is
not a big problem.
Discovered by rieo.
Don't allow anything but directory-signature tokens in a consensus after
the first directory-signature token. Fixes bug in bandwidth-weights branch.
Found by "outofwords."
Another dereference-then-NULL-check sequence. No reports of this bug
triggered in the wild. Fixes bugreport 1256.
Thanks to ekir for discovering and reporting this bug.
Fix a dereference-then-NULL-check sequence. This bug wasn't triggered
in the wild, but we should fix it anyways in case it ever happens.
Also make sure users get a note about this being a bug when they
see it in their log.
Thanks to ekir for discovering and reporting this bug.
We used to only zero the first ptrsize bytes of the cipher. Since
cipher is large enough, we didn't zero too many bytes. Discovered
and fixed by ekir. Fixes bug 1254.
This means that "if (E<G) {abc} else if (E>=G) {def}" can be replaced with
"if (E<G) {abc} else {def}"
Doing the second test explicitly made my mingw gcc nervous that we might
never be initializing casename.
For my 64-bit Linux system running with GCC 4.4.3-fc12-whatever, you
can't do 'printf("%lld", (int64_t)x);' Instead you need to tell the
compiler 'printf("%lld", (long long int)x);' or else it doesn't
believe the types match. This is why we added U64_PRINTF_ARG; it
looks like we needed an I64_PRINTF_ARG too.
asprintf() is a GNU extension that some BSDs have picked up: it does a printf
into a newly allocated chunk of RAM.
Our tor_asprintf() differs from standard asprintf() in that:
- Like our other malloc functions, it asserts on OOM.
- It works on windows.
- It always sets its return-field.
All other bandwidthrate settings are restricted to INT32_MAX, but
this check was forgotten for PerConnBWRate and PerConnBWBurst. Also
update the manpage to reflect the fact that specifying a bandwidth
in terabytes does not make sense, because that value will be too
large.
Fix a dereference-then-NULL-check sequence. This bug wasn't triggered
in the wild, but we should fix it anyways in case it ever happens.
Also make sure users get a note about this being a bug when they
see it in their log.
Thanks to ekir for discovering and reporting this bug.
On Windows, we don't have a notion of ~ meaning "our homedir", so we
were deliberately using an #ifdef to avoid calling expand_filename()
in multiple places. This is silly: The right place to turn a function
into a no-op on a single platform is in the function itself, not in
every single call-site.
We used to only zero the first ptrsize bytes of the cipher. Since
cipher is large enough, we didn't zero too many bytes. Discovered
and fixed by ekir. Fixes bug 1254.
Spec conformance issue: The code didn't force the network-status-version
token to be the first token in a v3 vote or consensus.
Problem discovered by Parakeep.
We need to use evdns_add_server_port_with_base() when configuring
our DNS listener, because libevent segfaults otherwise. Add a macro
in compat_libevent.h to pick the correct implementation depending
on the libevent version.
Fixes bug 1143, found by SwissTorExit
This time, set the SSL3_FLAGS_ALLOW_UNSAFE_RENEGOTIATION flag on every
version before OpenSSL 0.9.8l. I can confirm that the option value (0x0010)
wasn't reused until OpenSSL 1.0.0beta3.
The src and dest of a memcpy() call aren't supposed to overlap,
but we were sometimes calling tor_addr_copy() as a no-op.
Also, tor_addr_assign was a redundant copy of tor_addr_copy(); this patch
removes it.
We implemented ratelimiting for warnings going into the logfile, but didn't
rate-limit controller events. Now both log warnings and controller events
are rate-limited.
Tor has tor_lookup_hostname(), which prefers ipv4 addresses automatically.
Bug 1244 occured because gethostbyname() returned an ipv6 address, which
Tor cannot handle currently. Fixes bug 1244; bugfix on 0.0.2pre25.
Reported by Mike Mestnik.
The problem was that we didn't allocate enough memory on 32-bit
platforms with 64-bit time_t. The memory leak occured every time
we fetched a hidden service descriptor we've fetched before.
When calculating the is_exit flag for a routerinfo_t, we don't need
to call exit_policy_is_general_exit() if router_exit_policy_rejects_all()
tells us it definitely is an exit. This check is much cheaper than
running exit_policy_is_general_exit().
exit_policy_is_general_exit() assumed that there are no redundancies
in the passed policy, in the sense that we actively combine entries
in the policy to really get rid of any redundancy. Since we cannot
do that without massively rewriting the policy lines the relay
operators set, fix exit_policy_is_general_exit().
Fixes bug 1238, discovered by Martin Kowalczyk.
In brief: you mustn't use the SSL3_FLAG solution with anything but 0.9.8l,
and you mustn't use the SSL_OP solution with anything before 0.9.8m, and
you get in _real_ trouble if you try to set the flag in 1.0.0beta, since
they use it for something different.
For the ugly version, see my long comment in tortls.c
We need to do this because Apple doesn't update its dev-tools headers
when it updates its libraries in a security patch. On the bright
side, this might get us out of shipping a statically linked OpenSSL on
OSX.
May fix bug 1225.
[backported]
We need to do this because Apple doesn't update its dev-tools headers
when it updates its libraries in a security patch. On the bright
side, this might get us out of shipping a statically linked OpenSSL on
OSX.
May fix bug 1225.
We accidentally freed the internal buffer for bridge stats when we
were writing the bridge stats file or honoring a control port
request for said data. Change the interfaces for
geoip_get_bridge_stats* to prevent these problems, and remove the
offending free/add a tor_strdup.
Fixes bug 1208.
I believe that since we were allocating *cp while holding a mutex,
coverity deduced that *cp must be protected by that mutex, and later
flipped out when we didn't use it that way. If this is so, we can
solve our problems by moving the *cp = tor_strdup(buf) part outside of
the mutex-protected code.
It's a bit confusing to have a loop where another function,
confusingly named "*_free", is responsible for advancing the loop
variable (or rather, for altering a structure so that the next time
the loop variable's initializer is evaluated it evaluates to something
different.)
Not only has this confused people: it's also confused coverity scan.
Let's fix that.
This was freaking out some relay operators without good reason, as
it is nothing the relay operator can do anything about anyways.
Quieting this warning suggested by rieo.
We were checking for msg==NULL, but not lib or proc. This case can
only occur if we have an error whose string we somehow haven't loaded,
but it's worth coding defensively here.
Spotted by rieo on IRC.
The OutboundBindAddress option is useful for making sure that all of
your outbond connections use a given interface. But when connecting
to 127.0.0.1 (or ::1 even) it's important to actually have the
connection come _from_ localhost, since lots of programs running on
localhost use the source address to authenticate that the connection
is really coming from the same host.
Our old code always bound to OutboundBindAddress, whether connecting
to localhost or not. This would potentially break DNS servers on
localhost, and socks proxies on localhost. This patch changes the
behavior so that we only look at OutboundBindAddress when connecting
to a non-loopback address.
this case can now legitimately happen, if you have a cached v2 status
from moria1, and you run with the new list of dirservers that's missing
the old moria1. it's nothing to worry about; the file will die off in
a month or two.
The following commit:
commit e56747f9cf
Author: Nick Mathewson <nickm@torproject.org>
Date: Tue Dec 15 14:32:55 2009 -0500
Refactor a bit so that it is safe to include math.h, and mostly not needed.
introduced this line:
tor_resolve_LDADD = -lm ../common/libor.a @TOR_LIB_WS32@
which caused the build to fail, because only ../common/libor.a
(via the embedded ../common/util.o via ../common/util.c)
referenced libm's `lround' and `log' symbols, so that the
linker (GNU ld) didn't bother to import those symbols before
reading ../common/libor.a, thus leaving those symbols undefined.
The solution was to swap the order, producing the line:
tor_resolve_LDADD = ../common/libor.a -lm @TOR_LIB_WS32@
Signed-off-by: Michael Witten <mfwitten@gmail.com>
...to let us
rate-limit client connections as they enter the network. It's
controlled in the consensus so we can turn it on and off for
experiments. It's starting out off. Based on proposal 163.
Specifically, there are two cases: a) are we willing to start a new
circuit at a node not in your ExitNodes config option, and b) are we
willing to make use of a circuit that's already established but has an
unsuitable exit.
Now we discard all your circuits when you set ExitNodes, so the only
way you could end up with an exit circuit that ends at an unsuitable
place is if we explicitly ran out of exit nodes, StrictNodes was 0,
and we built this circuit to solve a stream that needs solving.
Fixes bug in dc322931, which would ignore the just-built circuit because
it has an unsuitable exit.
Before it would prepend your requested entrynodes to your list of guard
nodes, but feel free to use others after that. Now it chooses only
from your EntryNodes if any of those are available, and only falls back
to others if a) they're all down and b) StrictNodes is not set.
Also, now we refresh your entry guards from EntryNode at each consensus
fetch (rather than just at startup and then they slowly rot as the
network changes).
The goal here is to make users less likely to set StrictNodes, since
it's doing closer to what they expect it should be doing.
We do this in too many places throughout the code; it's time to start
clamping down.
Also, refactor Karsten's patch to use strchr-then-strndup, rather than
malloc-then-strlcpy-then-strchr-then-clear.
Fix statistics on client numbers by country as seen by bridges that were
broken in 0.2.2.1-alpha. Also switch to reporting full 24-hour intervals
instead of variable 12-to-48-hour intervals.
The HSAuthorityRecordStats option was used to track statistics of overall
hidden service usage on the version 0 hidden service authorities. With the
version 2 hidden service directories being deployed and version 0
descriptors being phased out, these statistics are not as useful anymore.
Goodbye, you fine piece of software; my first major code contribution to
Tor.
The new rule is: safe_str_X() means "this string is a piece of X
information; make it safe to log." safe_str() on its own means
"this string is a piece of who-knows-what; make it safe to log".
The rule is now: take the value from the CircuitPriorityHalflife
config option if it is set. If it zero, disable the cell_ewma
algorithm. If it is set, use it to calculate the scaling factor.
If it is not set, look for a CircPriorityHalflifeMsec parameter in the
consensus networkstatus. If *that* is zero, then disable the cell_ewma
algorithm; if it is set, use it to calculate the scaling factor.
If it is not set at all, disable the algorithm.
In connection_dir_client_reached_eof, we make sure that we either
return when we get an http status code of 503 or handle the problem
and set it to 200. Later we check if the status code is 503. Remove
that check.
There are two big changes here:
- We store active circuits in a priority queue for each or_conn,
rather than doing a linear search over all the active circuits
before we send each cell.
- Rather than multiplying every circuit's cell-ewma by a decay
factor every time we send a cell (thus normalizing the value of a
current cell to 1.0 and a past cell to alpha^t), we instead
only scale down the cell-ewma every tick (ten seconds atm),
normalizing so that a cell sent at the start of the tick has
value 1.0).
Each circuit is ranked in terms of how many cells from it have been
relayed recently, using a time-weighted average.
This patch has been tested this on a private Tor network on PlanetLab,
and gotten improvements of 12-35% in time it takes to fetch a small
web page while there's a simultaneous large data transfer going on
simultaneously.
[Commit msg by nickm based on mail from Ian Goldberg.]
This changes the pqueue API by requiring an additional int in every
structure that we store in a pqueue to hold the index of that structure
within the heap.
Some *_free functions threw asserts when passed NULL. Now all of them
accept NULL as input and perform no action when called that way.
This gains us consistence for our free functions, and allows some
code simplifications where an explicit null check is no longer necessary.
It turns out that OpenSSL 0.9.8m is likely to take a completely
different approach for reenabling renegotiation than OpenSSL 0.9.8l
did, so we need to work with both. :p Fixes bug 1158.
(patch by coderman; commit message by nickm)
Do not segfault when writing buffer stats when we haven't observed a
single circuit to report about. This is a minor bug that would only show
up in testing environments with no traffic and with reduced stats
intervals.
Avoid crashing if the client is trying to upload many bytes and the
circuit gets torn down at the same time, or if the flip side
happens on the exit relay. Bugfix on 0.2.0.1-alpha; fixes bug 1150.
New config option "CircuitStreamTimeout" to override our internal
timeout schedule for how many seconds until we detach a stream from
a circuit and try a new circuit. If your network is particularly
slow, you might want to set this to a number like 60.
On this OSX version, there is a stub mlockall() function
that doesn't work, *and* the declaration for it is hidden by
an '#ifdef _P1003_1B_VISIBLE'. This would make autoconf
successfully find the function, but our code fail to build
when no declaration was found.
This patch adds an additional test for the declaration.
This fixes bug 1147:
bionic doesn't have an actual implementation of mlockall();
mlockall() is merely in the headers but not actually in the library.
This prevents Tor compilation with the bionic libc for Android handsets.
To fix a major security problem related to incorrect use of
SSL/TLS renegotiation, OpenSSL has turned off renegotiation by
default. We are not affected by this security problem, however,
since we do renegotiation right. (Specifically, we never treat a
renegotiated credential as authenticating previous communication.)
Nevertheless, OpenSSL's new behavior requires us to explicitly
turn renegotiation back on in order to get our protocol working
again.
Amusingly, this is not so simple as "set the flag when you create
the SSL object" , since calling connect or accept seems to clear
the flags.
For belt-and-suspenders purposes, we clear the flag once the Tor
handshake is done. There's no way to exploit a second handshake
either, but we might as well not allow it.
This commit implements a new config option: 'DisableAllSwap'
This option probably only works properly when Tor is started as root.
We added two new functions: tor_mlockall() and tor_set_max_memlock().
tor_mlockall() attempts to mlock() all current and all future memory pages.
For tor_mlockall() to work properly we set the process rlimits for memory to
RLIM_INFINITY (and beyond) inside of tor_set_max_memlock().
We behave differently from mlockall() by only allowing tor_mlockall() to be
called one single time. All other calls will result in a return code of 1.
It is not possible to change DisableAllSwap while running.
A sample configuration item was added to the torrc.complete.in config file.
A new item in the man page for DisableAllSwap was added.
Thanks to Moxie Marlinspike and Chris Palmer for their feedback on this patch.
Please note that we make no guarantees about the quality of your OS and its
mlock/mlockall implementation. It is possible that this will do nothing at all.
It is also possible that you can ulimit the mlock properties of a given user
such that root is not required. This has not been extensively tested and is
unsupported. I have included some comments for possible ways we can handle
this on win32.
If your relay can't keep up with the number of incoming create cells, it
would log one warning per failure into your logs. Limit warnings to 1 per
minute.
In 5e4d53d535 we made it so that
crypto_cipher_set_key cannot fail. The call will now
always succeed, to returning a boolean for success/failure makes
no sense.
This was left over from an early draft of the microdescriptor code; it
began to populate the signatures array of a networkstatus vote, even
though there's no actual need to do that for a vote.
In its zeal to keep me from saying memset(x, '0', sizeof(x)), Coverity
disallows memset(x, 48, sizeof(x)). Fine. I'll choose a different
magic number, see if I care!
In C, the code "char x[10]; if (x) {...}" always takes the true branch of
the if statement. Coverity notices this now.
In some cases, we were testing arrays to make sure that an operation
we wanted to do would suceed. Those cases are now always-true.
In some cases, we were testing arrays to see if something was _set_.
Those caes are now tests for strlen(s), or tests for
!tor_mem_is_zero(d,len).
If all authorities restart at once right before a consensus vote, nobody
will vote about "Running", and clients will get a consensus with no usable
relays. Instead, authorities refuse to build a consensus if this happens.
The first happens on an error case when a controller wants an
impossible directory object. The second happens when we can't write
our fingerprint file.
The code for these was super-wrong, but will only break things when we
reset an option on a platform where sizeof(time_t) is different from
sizeof(int).
See task 1114. The most plausible explanation for someone sending us weak
DH keys is that they experiment with their Tor code or implement a new Tor
client. Usually, we don't care about such events, especially not on warn
level. If we really care about someone not following the Tor protocol, we
can set ProtocolWarnings to 1.
This patch introduces a new type called document_signature_t to represent the
signature of a consensus document. Now, each consensus document can have up
to one document signature per voter per digest algorithm. Also, each
detached-signatures document can have up to one signature per <voter,
algorithm, flavor>.
Previously, we insisted that a valid signature must be a signature of
the expected digest. Now we accept anything that starts with the
expected digest. This lets us include another digest later.
When we tried to use the deprecated non-threadsafe evdns
interfaces in Libevent 2 without using the also-deprecated
event_init() interface, Libevent 2 would sensibly crash, since it
has no guess where to find the Libevent library.
Here we use the evdns_base_*() functions instead if they're
present, and fake them if they aren't.
This is a possible fix for bug 1023, where if we vote (or make a v2
consensus networkstatus) right after we come online, we can call
rep_hist_note_router_unreachable() on every router we haven't connected
to yet, and thereby make all their uptime values reset.
This seems to be happening to me a lot on a garbage DSL line.
We may need to come up with 2 threshholds: a high short onehop
count and a lower longer count.
Don't count one-hop circuits when we're estimating how long it
takes circuits to build on average. Otherwise we'll set our circuit
build timeout lower than we should. Bugfix on 0.2.2.2-alpha.
Directory authorities now reject Tor relays with versions less than
0.1.2.14. This step cuts out four relays from the current network,
none of which are very big.
Previously, tor-gencert would call RSA_generate_key() directly.
This won't work on Android, which removes the (deprecated since
OpenSSL 0.9.8) function. We can't call RSA_generate_key_ex()
unconditionally either, since that didn't exist before 0.9.8.
Instead, we must call our own crypto_pk_generate_key_with_bits,
which knows how to call RSA_generate_key or RSA_generate_key_ex as
appropriate.
[Based on patch by Nathan Freitas]
Apparently the Android developers dumped OpenSSL's support for hardware
acceleration in order to save some memory, so you can't build programs using
engines on Android.
[Patch revised by nickm]
This shouldn't be necessary, but apparently the Android cross-compiler
doesn't respect -I as well as it should. (-I is supposed to add to the
*front* of the search path. Android's gcc wrapper apparently likes to add to
the end. This is broken, but we need to work around it.)
Found by coverity
test_mem_op_hex was leaking memory, which showed up in a few
tests.
Also, the dir_param test had a memleak of its own.
Found by Coverity
- Avoid memmoving 0 bytes which might lead to compiler warnings.
- Don't require relays to be entry node AND bridge at the same to time to
record clients.
- Fix a memory leak when writing dirreq-stats.
- Don't say in the stats files that measurement intervals are twice as long
as they really are.
- Reduce minimum observation time for requests to 12 hours, or we might
never record usage.
- Clear exit stats correctly after writing them, or we accumulate old stats
over time.
- Reset interval start for buffer stats, too.
The big change is to add a function to display the current SSL handshake
state, and to log it everywhere reasonable. (A failure in
SSL23_ST_CR_SRVR_HELLO_A is different from one in
SSL3_ST_CR_SESSION_TICKET_A.)
This patch also adds a new log domain for OR handshaking, so you can pull out
all the handshake log messages without having to run at debug for everything.
For example, you'd just say "log notice-err [handshake]debug-err file
tor.log".
This was the only log notice that happened during other
tor invocations, like --verify-config and --list-fingerprint.
Plus, now we think it works, so no need to hear about it.
"Tinytest" is a minimalist C unit testing framework I wrote for
Libevent. It supports some generally useful features, like being able
to run separate unit tests in their own processes.
I tried to do the refactoring to change test.c as little as possible.
Thus, we mostly don't call the tinytest macros directly. Instead, the
test.h header is now a wrapper on tinytest.h to make our existing
test_foo() macros work.
The next step(s) here will be:
- To break test.c into separate files, each with its own test group.
- To look into which things we can test
- To refactor the more fiddly tests to use the tinytest macros
directly and/or run forked.
- To see about writing unit tests for things we couldn't previously
test without forking.
If the networkstatus consensus tells us that we should use a
negative circuit package window, ignore it. Otherwise we'll
believe it and then trigger an assert.
Also, change the interface for networkstatus_get_param() so we
don't have to lookup the consensus beforehand.
A) We were considering a circuit had timed out in the special cases
where we close rendezvous circuits because the final rendezvous
circuit couldn't be built in time.
B) We were looking at the wrong timestamp_created when considering
a timeout.
Don't discard all circuits every MaxCircuitDirtiness, because the
user might legitimately have set that to a very lower number.
Also don't use up all of our idle circuits with testing circuits,
since that defeats the point of preemptive circuits.
We want it to be under our control so it doesn't mess
up initialization. This is likely the cause for
the bug the previous assert-adding commit (09a75ad) was
trying to address.
Using CircuitBuildTimeout is prone to issues with SIGHUP, etc.
Also, shuffle the circuit build times array after loading it
in so that newer measurements don't replace chunks of
similarly timed measurements.
To further attempt to fix bug 1090, make sure connection_ap_can_use_exit
always returns 0 when the chosen exit router is excluded. This should fix
bug1090.
When we excluded some Exits, we were sometimes warning the user that we
were going to use the node regardless. Many of those warnings were in
fact bogus, because the relay in question was not used to connect to
the outside world.
Based on patch by Rotor, thanks!
Tor now reads the "circwindow" parameter out of the consensus,
and uses that value for its circuit package window rather than the
default of 1000 cells. Begins the implementation of proposal 168.
This code adds a new field to vote on: "params". It consists of a list of
sorted key=int pairs. The output is computed as the median of all the
integers for any key on which anybody voted.
Improved with input from Roger.
Adding the same vote to a networkstatus consensus leads to a memory leak
on the client side. Fix that by only using the first vote from any given
voter, and ignoring the others.
Problem found by Rotor, who also helped writing the patch. Thanks!
A vote may only contain exactly one signature. Make sure we reject
votes that violate this.
Problem found by Rotor, who also helped writing the patch. Thanks!
Fix an obscure bug where hidden services on 64-bit big-endian
systems might mis-read the timestamp in v3 introduce cells, and
refuse to connect back to the client. Discovered by "rotor".
Bugfix on 0.2.1.6-alpha.
(Given that we're pretty much assuming that int is 32 bits, and given that
hex values are always unsigned, taking out the "ul" from 0xff000000 should
be fine.)
Add a "getinfo status/accepted-server-descriptor" controller
command, which is the recommended way for controllers to learn
whether our server descriptor has been successfully received by at
least on directory authority. Un-recommend good-server-descriptor
getinfo and status events until we have a better design for them.
We were telling the controller about CHECKING_REACHABILITY and
REACHABILITY_FAILED status events whenever we launch a testing
circuit or notice that one has failed. Instead, only tell the
controller when we want to inform the user of overall success or
overall failure. Bugfix on 0.1.2.6-alpha. Fixes bug 1075. Reported
by SwissTorExit.
When we added support for fractional units (like 1.5 MB) I broke
support for giving units with no space (like 2MB). This patch should
fix that. It also adds a propoer tor_parse_double().
Fix for bug 1076. Bugfix on 0.2.2.1-alpha.
We were triggering a CLOCK_SKEW controller status event whenever
we connect via the v2 connection protocol to any relay that has
a wrong clock. Instead, we should only inform the controller when
it's a trusted authority that claims our clock is wrong. Bugfix
on 0.2.0.20-rc; starts to fix bug 1074. Reported by SwissTorExit.
- Refactor geoip.c by moving duplicate code into rotate_request_period().
- Don't leak memory when cleaning up cell queues.
- Make sure that exit_(streams|bytes_(read|written)) are initialized in all
places accessing these arrays.
- Read only the last block from *stats files and ensure that its timestamp
is not more than 25 hours in the past and not more than 1 hour in the
future.
- Stop truncating the last character when reading *stats files.
The only thing that's left now is to avoid reading whole *stats files into
memory.
Note that unlike subversion revision numbers, it isn't meaningful to
compare these for anything but equality. We define a sort-order anyway,
in case one of these accidentally slips into a recommended-versions
list.
If any the v3 certs we download are unparseable, we should actually
notice the failure so we don't retry indefinitely. Bugfix on 0.2.0.x;
reported by "rotator".
Once we had called log_free_all(), anything that tried to log a
message (like a failed tor_assert()) would fail like this:
1. The logging call eventually invokes the _log() function.
2. _log() calls tor_mutex_lock(log_mutex).
3. tor_mutex_lock(m) calls tor_assert(m).
4. Since we freed the log_mutex, tor_assert() fails, and tries to
log its failure.
5. GOTO 1.
Now we allocate the mutex statically, and never destroy it on
shutdown.
Bugfix on 0.2.0.16-alpha, which introduced the log mutex.
This bug was found by Matt Edman.
(This would be everywhere running OpenSSL 0.9.7x and earlier, including
all current Macintosh users.)
The code is based on Tom St Denis's LibTomCrypt implementation,
modified to be way less general and use Tor's existing facilities. I
picked this one because it was pretty fast and pretty free, and
because Python uses it too.
The more verbose logs that were added in ee58153 also include a string
that might not have been initialized. This can lead to segfaults, e.g.,
when setting up private Tor networks. Initialize this string with NULL.
Send circuit or stream sendme cells when our window has decreased
by 100 cells, not when it has decreased by 101 cells. Bug uncovered
by Karsten when testing the "reduce circuit window" performance
patch. Bugfix on the 54th commit on Tor -- from July 2002,
before the release of Tor 0.0.0. This is the new winner of the
oldest-bug prize.
I don't think we actually use (or plan to use) strtok_r in a reentrant
way anywhere in our code, but would be nice not to have to think about
whether we're doing it.
Relays no longer publish a new server descriptor if they change
their MaxAdvertisedBandwidth config option but it doesn't end up
changing their advertised bandwidth numbers. Bugfix on 0.2.0.28-rc;
fixes bug 1026. Patch from Sebastian.
Specifically, every time we get a create cell but we have so many already
queued that we refuse it.
Bugfix on 0.2.0.19-alpha; fixes bug 1034. Reported by BarkerJr.
The problem is that clients and hidden services are receiving
relay_early cells, and they tear down the circuit.
Hack #1 is for rendezvous points to rewrite relay_early cells to
relay cells. That way there are never any incoming relay_early cells.
Hack #2 is for clients and hidden services to never send a relay_early
cell on an established rendezvous circuit. That works around rendezvous
points that haven't upgraded yet.
Hack #3 is for clients and hidden services to not tear down the circuit
when they receive an inbound relay_early cell. We already refuse extend
cells at clients.
When determining how long directory requests take or how long cells spend
in queues, we were comparing timestamps on microsecond detail only to
convert results to second or millisecond detail later on. But on 32-bit
architectures this means that 2^31 microseconds only cover time
differences of up to 36 minutes. Instead, compare timestamps on
millisecond detail.
Now that we require EntryStatistics to be 1 for counting connecting
clients, unit tests need to set that config option, too.
Reported by Sebastian Hahn.
Changes to directory request statistics:
- Rename GEOIP statistics to DIRREQ statistics, because they now include
more than only GeoIP-based statistics, whereas other statistics are
GeoIP-dependent, too.
- Rename output file from geoip-stats to dirreq-stats.
- Add new config option DirReqStatistics that is required to measure
directory request statistics.
- Clean up ChangeLog.
Also ensure that entry guards statistics have access to a local GeoIP
database.
This new option will allow clients to download the newest fresh consensus
much sooner than they normally would do so, even if they previously set
FetchDirInfoEarly. This includes a proper ChangeLog entry and an updated man
page.
Introduce a threshold of 0.01% of bytes that must be read and written per
port in order to be included in the statistics. Otherwise we cannot include
these statistics in extra-info documents, because they are too big.
Change the labels "-written" and "-read" so that the meanings are as
intended.
The internal error "could not find intro key" occurs when we want to send
an INTRODUCE1 cell over a recently finished introduction circuit and think
we built the introduction circuit with a v2 hidden service descriptor, but
cannot find the introduction key in our descriptor.
My first guess how we can end up in this situation is that we are wrong in
thinking that we built the introduction circuit based on a v2 hidden
service descriptor. This patch checks if we have a v0 descriptor, too, and
uses that instead.
o Minor features:
- If we're a relay and we change our IP address, be more verbose
about the reason that made us change. Should help track down
further bugs for relays on dynamic IP addresses.
when we write out our stability info, detect relays that have slipped
through the cracks. log about them and correct the problem.
if we continue to see a lot of these over time, it means there's another
spot where relays fall out of the routerlist without being marked as
unreachable.
Marks the control port connection for flushing before closing when
the QUIT command is issued. This allows a QUIT to be issued during
a long reply over the control port, flushing the reply and then
closing the connection. Fixes bug 1015.
rather than the bandwidth values in each relay descriptor. This approach
opens the door to more accurate bandwidth estimates once the directory
authorities start doing active measurements. Implements more of proposal
141.
arma's rationale: "I think this is a bug, since people intentionally
set DirPortFrontPage, so they really do want their relay to serve that
page when it's asked for. Having it appear only sometimes (or roughly
never in Sebastian's case) makes it way less useful."
Fixes bug 1013; bugfix on 0.2.1.8-alpha.
Added a sanity check in config.c and a check in directory.c
directory_initiate_command_rend() to catch any direct connection attempts
when a socks proxy is configured.
If the Tor is running with AutomapHostsOnResolve set, it _is_
reasonable to do a DNS lookup on a .onion address. So instead we make
tor-resolve willing to try to resolve anything. Only if Tor refuses
to resolve it do we suggest to the user that resolving a .onion
address may not work.
Fix for bug 1005.
The rest of the code was only including event.h so that it could see
EV_READ and EV_WRITE, which we were using as part of the
connection_watch_events interface for no very good reason.
This patch adds a new compat_libevent.[ch] set of files, and moves our
Libevent compatibility and utilitity functions there. We build them
into a separate .a so that nothing else in src/commmon depends on
Libevent (partially fixing bug 507).
Also, do not use our own built-in evdns copy when we have Libevent
2.0, whose evdns is finally good enough (thus fixing Bug 920).
Fix an edge case where a malicious exit relay could convince a
controller that the client's DNS question resolves to an internal IP
address. Bug found and fixed by "optimist"; bugfix on 0.1.2.8-beta.
Fix an edge case where a malicious exit relay could convince a
controller that the client's DNS question resolves to an internal IP
address. Bug found and fixed by "optimist"; bugfix on 0.1.2.8-beta.
Hidden service clients didn't use a cached service descriptor that
was older than 15 minutes, but wouldn't fetch a new one either. Now,
use a cached descriptor no matter how old it is and only fetch a new
one when all introduction points fail. Fix for bug 997. Patch from
Marcus Griep.
Apparently all the stuff that does a linear scan over all the DNS
cache entries can get really expensive when your DNS cache is very
large. It's hard to say how much this will help performance, since
gprof doesn't count time spent in OpenSSL or zlib, but I'd guess 10%.
Also, this patch removes calls to assert_connection_ok() from inside
the read and write callbacks, which are similarly unneeded, and a
little costlier than I'm happy with.
This is probably worth backporting to 0.2.0.
Provide a useful warning when launch_circuit tries to make us use a
node we don't want to use. Just give an info message when this is a
normal and okay situation. Fix for logging issues in bug 984.
This patch adds a function to determine whether we're in the main
thread, and changes control_event_logmsg() to return immediately if
we're in a subthread. This is necessary because otherwise we will
call connection_write_to_buf, which modifies non-locked data
structures.
Bugfix on 0.2.0.x; fix for at least one of the things currently
called "bug 977".
Tas (thanks!) noticed that when *ListenAddress is set, Tor would
still warn on startup when *Port is low and hibernation is active.
The patch parses all the *ListenAddress lines, and checks the
ports. Bugfix on 0.2.1.15-rc
With the last fix of task 932 (5f03d6c), client requests are only added to
the history when they happen after the start of the current history. This
conflicts with the unit tests that insert current requests first (defining
the start of the client request history) followed by requests in the past.
The fix is to insert requests in chronological order in the unit tests.
- Write geoip stats to disk every 24 hours, not every hour.
- Remove configuration options and define reasonable defaults.
- Clear history of client requests every 24 hours (which wasn't done at
all before).
Specifically if you send SIGUSR1, it will add two lines to the log file:
May 22 07:41:59.576 [notice] Our DNS cache has 3364 entries.
May 22 07:41:59.576 [notice] Our DNS cache size is approximately 1022656
bytes.
[tweaked a bit by nickm]
Really, our idiocy was that we were calling event_set() on the same
event more than once, which sometimes led to us calling event_set() on
an event that was already inserted, thus making it look uninserted.
With this patch, we just initialize the timeout events when we create
the requests and nameservers, and we don't need to worry about
double-add and double-del cases at all.
If we ever add an event, then set it, then add it again, there will be
now two pointers to the event in the event base. If we delete one and
free it, the first pointer will still be there, and possibly cause a
crash later.
This patch adds detection for this case to the code paths in
eventdns.c, and works around it. If the warning message ever
displays, then a cleverer fix is in order.
{I am not too confident that this *is* the fix, since bug 957 is very
tricky. If it is, it is a bugfix on 0.2.0.}
When we got a descriptor that we (as an authority) rejected as totally
bad, we were freeing it, then using the digest in its RAM to look up its
download status. Caught by arma with valgrind. Bugfix on 0.2.1.9-alpha.
The trick is that we should assert that our next_mem pointer has not
run off the end of the array _before_ we realign the pointer, since
doing that could take us over the end... but only if we're on a system
where malloc() gives us ram in increments smaller than sizeof(void*).
Bridges are not supposed to publish router descriptors to the directory
authorities. It defeats the point of bridges when they are included in the
public relay directory.
This patch puts out a warning and exits when the node is configured as
a bridge and to publish v1, v2, or v3 descriptors at the same time.
Also fixes part of bug 932.
This matters because a cpuworker can close its socket when it
finishes. Cpuworker typically runs in another thread, so without a
lock here, we can have a race condition and get confused about how
many sockets are open. Possible fix for bug 939.
This might detect some possible causes of bug 930, and will at least
make sure we aren't doing some dumb memory-corruption stuff with the heap
and router-parsing.
This addresses the first part of bug 918. Users are now warned when
they try to use hibernation in combination with a port below 1024
when they're not on Windows. We don't want to die here, because
people might run Tor as root, use a capabilities system or some
other platform that will allow them to re-attach low ports.
Wording suggested by Marian
(Don't crash immediately if we have leftover chunks to free after
freeing chunks in a buffer freelist; instead log a debugging message
that might help.)
Now, when you call tor --digests, it dumps the SHA1 digest of each
source file that Tor was built with. We support both 'sha1sum' and
'openssl sha1'. If the user is building from a tarball and they
haven't edited anything, they don't need any program that calculates
SHA1. If they _have_ modified a file but they don't have a program to
calculate SHA1, we try to build so we do not output digests.
bytes (aka 20KB/s), to match our documentation. Also update
directory authorities so they always assign the Fast flag to relays
with 20KB/s of capacity. Now people running relays won't suddenly
find themselves not seeing any use, if the network gets faster
on average.
svn:r19305
IP address changes: directory mirrors were mistakenly telling them
their old address if they asked via begin_dir, so they never got
an accurate answer about their new address, so they just vanished
after a day. Should fix bugs 827, 883, and 900 -- but alas, only
after every directory mirror has upgraded.
svn:r19291
ago. This change should significantly improve client performance,
especially once more people upgrade, since relays that have been
a guard for a long time are currently overloaded.
svn:r19287
