On win64, sockets are of type UINT_PTR; on win32 they're u_int;
elsewhere they're int. The correct windows way to check a socket for
being set is to compare it with INVALID_SOCKET; elsewhere you see if
it is negative.
On Libevent 2, all callbacks take sockets as evutil_socket_t; we've
been passing them int.
This patch should fix compilation and correctness when built for
64-bit windows. Fixes bug 3270.
Rename crypto_pk_check_key_public_exponent to crypto_pk_public_exponent_ok:
it's nice to name predicates s.t. you can tell how to interpret true
and false.
This was harmless, since we only used this for checking for lists of
port values, but it's the principle of the thing.
Fixes 3175; bugfix on 0.1.0.1-rc
We'll need this for checking permissions on the directories that hold
control sockets: if somebody says "ControlSocket ~/foo", it would be
pretty rude to do a chmod 700 on their homedir.
Conflicts in various places, mainly node-related. Resolved them in
favor of HEAD, with copying of tor_mem* operations from bug3122_memcmp_022.
src/common/Makefile.am
src/or/circuitlist.c
src/or/connection_edge.c
src/or/directory.c
src/or/microdesc.c
src/or/networkstatus.c
src/or/router.c
src/or/routerlist.c
src/test/test_util.c
Conflicts throughout. All resolved in favor of taking HEAD and
adding tor_mem* or fast_mem* ops as appropriate.
src/common/Makefile.am
src/or/circuitbuild.c
src/or/directory.c
src/or/dirserv.c
src/or/dirvote.c
src/or/networkstatus.c
src/or/rendclient.c
src/or/rendservice.c
src/or/router.c
src/or/routerlist.c
src/or/routerparse.c
src/or/test.c
Here I looked at the results of the automated conversion and cleaned
them up as follows:
If there was a tor_memcmp or tor_memeq that was in fact "safe"[*] I
changed it to a fast_memcmp or fast_memeq.
Otherwise if there was a tor_memcmp that could turn into a
tor_memneq or tor_memeq, I converted it.
This wants close attention.
[*] I'm erring on the side of caution here, and leaving some things
as tor_memcmp that could in my opinion use the data-dependent
fast_memcmp variant.
These functions can return NULL for otherwise-valid values of
time_t. Notably, the glibc gmtime manpage says it can return NULL
if the year if greater than INT_MAX, and the windows MSDN gmtime
page says it can return NULL for negative time_t values.
Also, our formatting code is not guaranteed to correctly handle
years after 9999 CE.
This patch tries to correct this by detecting NULL values from
gmtime/localtime_r, and trying to clip them to a reasonable end of
the scale. If they are in the middle of the scale, we call it a
downright error.
Arguably, it's a bug to get out-of-bounds dates like this to begin
with. But we've had bugs of this kind in the past, and warning when
we see a bug is much kinder than doing a NULL-pointer dereference.
Boboper found this one too.
We want to use the discard port correctly, so a htons() was missing.
Also we need to set it correctly depending on address family.
Review provided by danieldg
SSL_*_app_data uses ex_data index 0, which will be the first one allocated
by SSL_get_ex_new_index. Thus, if we ever started using the ex_data feature
for some other purpose, or a library linked to Tor ever started using
OpenSSL's ex_data feature, Tor would break in spectacular and mysterious
ways. Using the SSL_*_ex_data functions directly now may save us from
that particular form of breakage in the future.
But I would not be surprised if using OpenSSL's ex_data functions at all
(directly or not) comes back to bite us on our backends quite hard. The
specified behaviour of dup_func in the man page is stupid, and
crypto/ex_data.c is a horrific mess.
Sets:
* Documentation
* Logging domain
* Configuration option
* Scheduled event
* Makefile
It also creates status.c and the log_heartbeat() function.
All code was written by Sebastian Hahn. Commit message was
written by me (George Kadianakis).
Our regular DH parameters that we use for circuit and rendezvous
crypto are unchanged. This is yet another small step on the path of
protocol fingerprinting resistance.
(Backport from 0.2.2's 5ed73e3807)
Patch our implementation of tor_lockfile_lock() to handle this case
correctly. Also add a note that blocking behaviour differs from windows
to *nix. Fixes bug 2504, issue pointed out by mobmix.
Previously if you wanted to say "All messages except network
messages", you needed to say "[*,~net]" and if you said "[~net]" by
mistake, you would get no messages at all. Now, if you say "[~net]",
you get everything except networking messages.
Our regular DH parameters that we use for circuit and rendezvous
crypto are unchanged. This is yet another small step on the path of
protocol fingerprinting resistance.
Our public key functions assumed that they were always writing into a
large enough buffer. In one case, they weren't.
(Incorporates fixes from sebastian)
The C standard says that INT32_MAX is supposed to be a signed
integer. On platforms that have it, we get the correct
platform-defined value. Our own replacement, however, was
unsigned. That's going to cause a bug somewhere eventually.
C99 allows a syntax for structures whose last element is of
unspecified length:
struct s {
int elt1;
...
char last_element[];
};
Recent (last-5-years) autoconf versions provide an
AC_C_FLEXIBLE_ARRAY_MEMBER test that defines FLEXIBLE_ARRAY_MEMBER
to either no tokens (if you have c99 flexible array support) or to 1
(if you don't). At that point you just use offsetof
[STRUCT_OFFSET() for us] to see where last_element begins, and
allocate your structures like:
struct s {
int elt1;
...
char last_element[FLEXIBLE_ARRAY_MEMBER];
};
tor_malloc(STRUCT_OFFSET(struct s, last_element) +
n_elements*sizeof(char));
The advantages are:
1) It's easier to see which structures and elements are of
unspecified length.
2) The compiler and related checking tools can also see which
structures and elements are of unspecified length, in case they
wants to try weird bounds-checking tricks or something.
3) The compiler can warn us if we do something dumb, like try
to stack-allocate a flexible-length structure.
On Windows, we never use pthreads, since it doesn't usually exist,
and when it does it tends to be a little weirdly-behaved. But some
mingw installations have a pthreads installed, so autoconf detects
pthread.h and tells us about it. This would make us include
pthread.h, which could make for trouble when the iffy pthread.h
tried to include config.h.
This patch changes compat.h so that we never include pthread.h on
Windows. Fixes bug 2313; bugfix on 0.1.0.1-rc.
Libevent 2.0 has a "changelist" feature that avoids making redundant
syscalls if we wind up doing a lot of event_add/event_del operations
on the same fd in a row. Unfortunately, due to a weird design
choice in Linux, it doesn't work right with epoll when multiple fds
refer to the same socket (e.g., one is a dup() of the other). We
don't dup() anything we give to Libevent, though, so it is safe for
us to explicitly turn this feature on.
It's all too easy in C to convert an unsigned value to a signed one,
which will (on all modern computers) give you a huge signed value. If
you have a size_t value of size greater than SSIZE_T_MAX, that is way
likelier to be an underflow than it is to be an actual request for
more than 2gb of memory in one go. (There's nothing in Tor that
should be trying to allocate >2gb chunks.)
Doing so could make Libevent call Libevent from inside a Libevent
logging call, which is a recipe for reentrant confusion and
hard-to-debug crashes. This would especially hurt if Libevent
debug-level logging is enabled AND the user has a controller
watching for low-severity log messages.
Fix bug 2190; fix on 0.1.0.2-rc.
Doing so could make Libevent call Libevent from inside a Libevent
logging call, which is a recipe for reentrant confusion and
hard-to-debug crashes. This would especially hurt if Libevent
debug-level logging is enabled AND the user has a controller
watching for low-severity log messages.
Fix bug 2190; fix on 0.1.0.2-rc.
Having very long single lines with lots and lots of things in them
tends to make files hard to diff and hard to merge. Since our tools
are one-line-at-a-time, we should try to construct lists that way too,
within reason.
This incidentally turned up a few headers in configure.in that we were
for some reason searching for twice.
We need filtering bufferevent_openssl so that we can wrap around
IOCP bufferevents on Windows. This patch adds a temporary option to
turn on filtering mode, so that we can test it out on non-IOCP
systems to make sure it hasn't got any surprising bugs.
It also fixes some allocation/teardown errors in using
bufferevent_openssl as a filter.
In a2bb0bf we started using a separate client identity key. When we are
in "public server mode" (that means not a bridge) we will use the same
key. Reusing the key without doing the proper refcounting leads to a
segfault on cleanup during shutdown. Fix that.
Also introduce an assert that triggers if our refcount falls below 0.
That should never happen.
The old comment was from before I tried a huge pile of crazy stuff to
make the inner loop faster. Short answer: GCC already knows how to
unroll loops pretty well. Other short answer: we should have made the
relay payload size an even multiple of 4, 8, or ideally 16.
The short version is, "where we want to do it, we have nothing real to
chose from and we can't do it easily. Where it's easy to do, we have
no reason to do it yet."
First start of a fix for bug2001, but my test network still isn't
working: the client and the server send each other VERSIONS cells,
but never notice that they got them.
* Make tor_tls_context_new internal to tortls.c, and return the new
tor_tls_context_t from it.
* Add a public tor_tls_context_init wrapper function to replace it.
- Responsibility of clearing hex_errno is no longer with caller
- More conservative bounds checking
- Length requirement of hex_errno documented
- Output format documented
Some of these functions only work for routerinfo-based nodes, and as
such are only usable for advisory purposes. Fortunately, our uses
of them are compatible with this limitation.
The node_t type is meant to serve two key functions:
1) Abstracting difference between routerinfo_t and microdesc_t
so that clients can use microdesc_t instead of routerinfo_t.
2) Being a central place to hold mutable state about nodes
formerly held in routerstatus_t and routerinfo_t.
This patch implements a nodelist type that holds a node for every
router that we would consider using.
This is needed for IOCP, since telling the IOCP backend about all
your CPUs is a good idea. It'll also come in handy with asn's
multithreaded crypto stuff, and for people who run servers without
reading the manual.
This requires the latest Git version of Libevent as of 24 March 2010.
In the future, we'll just say it requires Libevent 2.0.5-alpha or
later.
Since Libevent doesn't yet support hierarchical rate limit groups,
there isn't yet support for tracking relayed-bytes separately when
using the bufferevent system. If a future version does add support
for hierarchical buckets, we can add that back in.
This function uses GetSystemDirectory() to make sure we load the version
of the library from c:\windows\system32 (or local equivalent) rather than
whatever version lives in the cwd.
This should keep WinCE working (unicode always-on) and get Win98
working again (unicode never-on).
There are two places where we explicitly use ASCII-only APIs, still:
in ntmain.c and in the unit tests.
This patch also fixes a bug in windoes tor_listdir that would cause
the first file to be listed an arbitrary number of times that was
also introduced with WinCE support.
Should fix bug 1797.
This should make us conflict less with system files named "log.h".
Yes, we shouldn't have been conflicting with those anyway, but some
people's compilers act very oddly.
The actual change was done with one "git mv", by editing
Makefile.am, and running
find . -name '*.[ch]' | xargs perl -i -pe 'if (/^#include.*\Wlog.h/) {s/log.h/torlog.h/; }'
These timers behave better with non-monotonic clocks than our old
ones, and also try harder to make once-per-second events get called
one second apart, rather than one-plus-epsilon seconds apart.
This fixes bug 943 for everybody using Libevent 2.0 or later.
From the code:
zlib 1.2.4 and 1.2.5 do some "clever" things with macros. Instead of
saying "(defined(FOO) ? FOO : 0)" they like to say "FOO-0", on the theory
that nobody will care if the compile outputs a no-such-identifier warning.
Sorry, but we like -Werror over here, so I guess we need to define these.
I hope that zlib 1.2.6 doesn't break these too.
Possible fix for bug 1526.
This should never happen unless openssl is buggy or some of our
assumptions are deeply wrong, but one of those might have been the
cause of the not-yet-reproducible bug 1209. If it ever happens again,
let's get some info we can use.
Having ~/.tor expand into /.tor is, after all, almost certainly not
what the user wanted, and it deserves a warning message.
Also, convert a guess-and-malloc-and-sprintf triple into an asprintf.
Most of the changes here are switches to use APIs available on Windows
CE. The most pervasive change is that Windows CE only provides the
wide-character ("FooW") variants of most of the windows function, and
doesn't support the older ASCII verions at all.
This patch will require use of the wcecompat library to get working
versions of the posix-style fd-based file IO functions.
[commit message by nickm]
1) mingw doesn't have _vscprintf(); mingw instead has a working snprintf.
2) windows compilers that _do_ have a working _vscprintf spell it so; they do
not spell it _vcsprintf().
Our code assumed that any version of OpenSSL before 0.9.8l could not
possibly require SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION. This is
so... except that many vendors have backported the flag from later
versions of openssl when they backported the RFC5476 renegotiation
feature.
The new behavior is particularly annoying to detect. Previously,
leaving SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION unset meant that
clients would fail to renegotiate. People noticed that one fast!
Now, OpenSSL's RFC5476 support means that clients will happily talk to
any servers there are, but servers won't accept renegotiation requests
from unpatched clients unless SSL_OP_ALLOW_etc is set. More fun:
servers send back a "no renegotiation for you!" error, which unpatched
clients respond to by stalling, and generally producing no useful
error message.
This might not be _the_ cause of bug 1346, but it is quite likely _a_
cause for bug 1346.
We used to only zero the first ptrsize bytes of the cipher. Since
cipher is large enough, we didn't zero too many bytes. Discovered
and fixed by ekir. Fixes bug 1254.
For my 64-bit Linux system running with GCC 4.4.3-fc12-whatever, you
can't do 'printf("%lld", (int64_t)x);' Instead you need to tell the
compiler 'printf("%lld", (long long int)x);' or else it doesn't
believe the types match. This is why we added U64_PRINTF_ARG; it
looks like we needed an I64_PRINTF_ARG too.
asprintf() is a GNU extension that some BSDs have picked up: it does a printf
into a newly allocated chunk of RAM.
Our tor_asprintf() differs from standard asprintf() in that:
- Like our other malloc functions, it asserts on OOM.
- It works on windows.
- It always sets its return-field.
On Windows, we don't have a notion of ~ meaning "our homedir", so we
were deliberately using an #ifdef to avoid calling expand_filename()
in multiple places. This is silly: The right place to turn a function
into a no-op on a single platform is in the function itself, not in
every single call-site.
We used to only zero the first ptrsize bytes of the cipher. Since
cipher is large enough, we didn't zero too many bytes. Discovered
and fixed by ekir. Fixes bug 1254.
We need to use evdns_add_server_port_with_base() when configuring
our DNS listener, because libevent segfaults otherwise. Add a macro
in compat_libevent.h to pick the correct implementation depending
on the libevent version.
Fixes bug 1143, found by SwissTorExit
This time, set the SSL3_FLAGS_ALLOW_UNSAFE_RENEGOTIATION flag on every
version before OpenSSL 0.9.8l. I can confirm that the option value (0x0010)
wasn't reused until OpenSSL 1.0.0beta3.
The src and dest of a memcpy() call aren't supposed to overlap,
but we were sometimes calling tor_addr_copy() as a no-op.
Also, tor_addr_assign was a redundant copy of tor_addr_copy(); this patch
removes it.
In brief: you mustn't use the SSL3_FLAG solution with anything but 0.9.8l,
and you mustn't use the SSL_OP solution with anything before 0.9.8m, and
you get in _real_ trouble if you try to set the flag in 1.0.0beta, since
they use it for something different.
For the ugly version, see my long comment in tortls.c
We need to do this because Apple doesn't update its dev-tools headers
when it updates its libraries in a security patch. On the bright
side, this might get us out of shipping a statically linked OpenSSL on
OSX.
May fix bug 1225.
[backported]
We need to do this because Apple doesn't update its dev-tools headers
when it updates its libraries in a security patch. On the bright
side, this might get us out of shipping a statically linked OpenSSL on
OSX.
May fix bug 1225.
We were checking for msg==NULL, but not lib or proc. This case can
only occur if we have an error whose string we somehow haven't loaded,
but it's worth coding defensively here.
Spotted by rieo on IRC.
We do this in too many places throughout the code; it's time to start
clamping down.
Also, refactor Karsten's patch to use strchr-then-strndup, rather than
malloc-then-strlcpy-then-strchr-then-clear.
This changes the pqueue API by requiring an additional int in every
structure that we store in a pqueue to hold the index of that structure
within the heap.
Some *_free functions threw asserts when passed NULL. Now all of them
accept NULL as input and perform no action when called that way.
This gains us consistence for our free functions, and allows some
code simplifications where an explicit null check is no longer necessary.
It turns out that OpenSSL 0.9.8m is likely to take a completely
different approach for reenabling renegotiation than OpenSSL 0.9.8l
did, so we need to work with both. :p Fixes bug 1158.
(patch by coderman; commit message by nickm)
On this OSX version, there is a stub mlockall() function
that doesn't work, *and* the declaration for it is hidden by
an '#ifdef _P1003_1B_VISIBLE'. This would make autoconf
successfully find the function, but our code fail to build
when no declaration was found.
This patch adds an additional test for the declaration.
This fixes bug 1147:
bionic doesn't have an actual implementation of mlockall();
mlockall() is merely in the headers but not actually in the library.
This prevents Tor compilation with the bionic libc for Android handsets.
To fix a major security problem related to incorrect use of
SSL/TLS renegotiation, OpenSSL has turned off renegotiation by
default. We are not affected by this security problem, however,
since we do renegotiation right. (Specifically, we never treat a
renegotiated credential as authenticating previous communication.)
Nevertheless, OpenSSL's new behavior requires us to explicitly
turn renegotiation back on in order to get our protocol working
again.
Amusingly, this is not so simple as "set the flag when you create
the SSL object" , since calling connect or accept seems to clear
the flags.
For belt-and-suspenders purposes, we clear the flag once the Tor
handshake is done. There's no way to exploit a second handshake
either, but we might as well not allow it.
This commit implements a new config option: 'DisableAllSwap'
This option probably only works properly when Tor is started as root.
We added two new functions: tor_mlockall() and tor_set_max_memlock().
tor_mlockall() attempts to mlock() all current and all future memory pages.
For tor_mlockall() to work properly we set the process rlimits for memory to
RLIM_INFINITY (and beyond) inside of tor_set_max_memlock().
We behave differently from mlockall() by only allowing tor_mlockall() to be
called one single time. All other calls will result in a return code of 1.
It is not possible to change DisableAllSwap while running.
A sample configuration item was added to the torrc.complete.in config file.
A new item in the man page for DisableAllSwap was added.
Thanks to Moxie Marlinspike and Chris Palmer for their feedback on this patch.
Please note that we make no guarantees about the quality of your OS and its
mlock/mlockall implementation. It is possible that this will do nothing at all.
It is also possible that you can ulimit the mlock properties of a given user
such that root is not required. This has not been extensively tested and is
unsupported. I have included some comments for possible ways we can handle
this on win32.
In 5e4d53d535 we made it so that
crypto_cipher_set_key cannot fail. The call will now
always succeed, to returning a boolean for success/failure makes
no sense.
In C, the code "char x[10]; if (x) {...}" always takes the true branch of
the if statement. Coverity notices this now.
In some cases, we were testing arrays to make sure that an operation
we wanted to do would suceed. Those cases are now always-true.
In some cases, we were testing arrays to see if something was _set_.
Those caes are now tests for strlen(s), or tests for
!tor_mem_is_zero(d,len).
See task 1114. The most plausible explanation for someone sending us weak
DH keys is that they experiment with their Tor code or implement a new Tor
client. Usually, we don't care about such events, especially not on warn
level. If we really care about someone not following the Tor protocol, we
can set ProtocolWarnings to 1.
This patch introduces a new type called document_signature_t to represent the
signature of a consensus document. Now, each consensus document can have up
to one document signature per voter per digest algorithm. Also, each
detached-signatures document can have up to one signature per <voter,
algorithm, flavor>.
Apparently the Android developers dumped OpenSSL's support for hardware
acceleration in order to save some memory, so you can't build programs using
engines on Android.
[Patch revised by nickm]
This shouldn't be necessary, but apparently the Android cross-compiler
doesn't respect -I as well as it should. (-I is supposed to add to the
*front* of the search path. Android's gcc wrapper apparently likes to add to
the end. This is broken, but we need to work around it.)
The big change is to add a function to display the current SSL handshake
state, and to log it everywhere reasonable. (A failure in
SSL23_ST_CR_SRVR_HELLO_A is different from one in
SSL3_ST_CR_SESSION_TICKET_A.)
This patch also adds a new log domain for OR handshaking, so you can pull out
all the handshake log messages without having to run at debug for everything.
For example, you'd just say "log notice-err [handshake]debug-err file
tor.log".
"Tinytest" is a minimalist C unit testing framework I wrote for
Libevent. It supports some generally useful features, like being able
to run separate unit tests in their own processes.
I tried to do the refactoring to change test.c as little as possible.
Thus, we mostly don't call the tinytest macros directly. Instead, the
test.h header is now a wrapper on tinytest.h to make our existing
test_foo() macros work.
The next step(s) here will be:
- To break test.c into separate files, each with its own test group.
- To look into which things we can test
- To refactor the more fiddly tests to use the tinytest macros
directly and/or run forked.
- To see about writing unit tests for things we couldn't previously
test without forking.
This code adds a new field to vote on: "params". It consists of a list of
sorted key=int pairs. The output is computed as the median of all the
integers for any key on which anybody voted.
Improved with input from Roger.
(Given that we're pretty much assuming that int is 32 bits, and given that
hex values are always unsigned, taking out the "ul" from 0xff000000 should
be fine.)
When we added support for fractional units (like 1.5 MB) I broke
support for giving units with no space (like 2MB). This patch should
fix that. It also adds a propoer tor_parse_double().
Fix for bug 1076. Bugfix on 0.2.2.1-alpha.
Once we had called log_free_all(), anything that tried to log a
message (like a failed tor_assert()) would fail like this:
1. The logging call eventually invokes the _log() function.
2. _log() calls tor_mutex_lock(log_mutex).
3. tor_mutex_lock(m) calls tor_assert(m).
4. Since we freed the log_mutex, tor_assert() fails, and tries to
log its failure.
5. GOTO 1.
Now we allocate the mutex statically, and never destroy it on
shutdown.
Bugfix on 0.2.0.16-alpha, which introduced the log mutex.
This bug was found by Matt Edman.
(This would be everywhere running OpenSSL 0.9.7x and earlier, including
all current Macintosh users.)
The code is based on Tom St Denis's LibTomCrypt implementation,
modified to be way less general and use Tor's existing facilities. I
picked this one because it was pretty fast and pretty free, and
because Python uses it too.
I don't think we actually use (or plan to use) strtok_r in a reentrant
way anywhere in our code, but would be nice not to have to think about
whether we're doing it.
When determining how long directory requests take or how long cells spend
in queues, we were comparing timestamps on microsecond detail only to
convert results to second or millisecond detail later on. But on 32-bit
architectures this means that 2^31 microseconds only cover time
differences of up to 36 minutes. Instead, compare timestamps on
millisecond detail.
This patch adds a new compat_libevent.[ch] set of files, and moves our
Libevent compatibility and utilitity functions there. We build them
into a separate .a so that nothing else in src/commmon depends on
Libevent (partially fixing bug 507).
Also, do not use our own built-in evdns copy when we have Libevent
2.0, whose evdns is finally good enough (thus fixing Bug 920).
This patch adds a function to determine whether we're in the main
thread, and changes control_event_logmsg() to return immediately if
we're in a subthread. This is necessary because otherwise we will
call connection_write_to_buf, which modifies non-locked data
structures.
Bugfix on 0.2.0.x; fix for at least one of the things currently
called "bug 977".
The trick is that we should assert that our next_mem pointer has not
run off the end of the array _before_ we realign the pointer, since
doing that could take us over the end... but only if we're on a system
where malloc() gives us ram in increments smaller than sizeof(void*).
This matters because a cpuworker can close its socket when it
finishes. Cpuworker typically runs in another thread, so without a
lock here, we can have a race condition and get confused about how
many sockets are open. Possible fix for bug 939.
This might detect some possible causes of bug 930, and will at least
make sure we aren't doing some dumb memory-corruption stuff with the heap
and router-parsing.
Now, when you call tor --digests, it dumps the SHA1 digest of each
source file that Tor was built with. We support both 'sha1sum' and
'openssl sha1'. If the user is building from a tarball and they
haven't edited anything, they don't need any program that calculates
SHA1. If they _have_ modified a file but they don't have a program to
calculate SHA1, we try to build so we do not output digests.
tor_sscanf() only handles %u and %s for now, which will make it
adequate to replace sscanf() for date/time/IP parsing. We want this
to prevent attackers from constructing weirdly formed descriptors,
cells, addresses, HTTP responses, etc, that validate under some
locales but not others.
svn:r18760
It seems that 64-bit Sparc Solaris demands 64-bit-aligned access to
uint64_t, but does not 64-bit-align the stack-allocated char array we
use for cpuworker tags. So this patch adds a set/get_uint64 pair, and
uses them to access the conn_id field in the tag.
svn:r18743
crypto_global_init gets called. Also have it be crypto_global_init
that calls crypto_seed_rng, so we are not dependent on OpenSSL's
RAND_poll in these fiddly cases.
Should fix bug 907. Bugfix on 0.0.9pre6. Backport candidate.
svn:r18210
There was a field that _HT_FOI_INSERT was never setting. Everything that calls _HT_FOI_INSERT was setting it via tor_malloc_zero, but that's fragile.
svn:r18064
The subversion $Id$ fields made every commit force a rebuild of
whatever file got committed. They were not actually useful for
telling the version of Tor files in the wild.
svn:r17867
dmalloc_malloc, dmalloc_realloc and dmalloc_strdup. It only calls those
functions if we're using the magic USE_DMALLOC macro. If we're not doing
that, we call the normal malloc, realloc and strdup. This is my first
night at malloc disambiguation club, so I had to disambiguate. Also, first commit, I have my commit bit now. Huzzzah!!!
svn:r17157
Make definition of tor_mutex_t go into compat.h, so that it is possible to inline mutexes in critical objects. Add init/uninit functions for mutexes allocated inside other structs.
svn:r16623
Initial conversion of uint32_t addr to tor_addr_t addr in connection_t and related types. Most of the Tor wire formats using these new types are in, but the code to generate and use it is not. This is a big patch. Let me know what it breaks for you.
svn:r16435
Allow alternate form of SMARTLIST_FOREACH with paired BEGIN and END macros. This lets the compiler tell us which line an error has occurred on.
svn:r16256
Refactor tor_addr_from_string: it didnt need most of parse_addr_mask_port_range, and its dependence on that latter function made it less flexible.
svn:r16255
Tor_addr_compare did a semantic comparison, such that ::1.2.3.4 and 1.2.3.4 were "equal". we sometimes need an exact comparison. Add a feature to do that.
svn:r16210
Make generic address manipulation functions work better. Switch address policy code to use tor_addr_t, so it can handle IPv6. That is a good place to start.
svn:r16178
Fix for bug 742: do not use O_CREAT on 2-option version of open(). Especially do not use it on /dev/null. Fix from Michael Scherer. Bugfix on 0.0.2pre19 (wow).
svn:r15626
Implement code to manually force the OpenSSL client cipher list to match the one recommended in proposal 124, *even if* we do not know all those ciphers. This is a bit of a kludge, but it is at least decently well commented.
svn:r15173
Forward-port: I had apparently broken OSX and Freebsd by not initializing threading before we initialize the logging system. This patch should do so, and fix bug 671.
svn:r14430
On platforms using pthreads, allow a thread to acquire a lock it already holds. This is crucial for logging: otherwise any log message thrown from inside the logging process (especially from control.c) will deadlock. Win32 CriticalSections are already recursive. Bug spotted by nwf. Bugfix on 0.2.0.16-alpha. Backport candidate. I hope this is portable.
svn:r14406
Make dumpstats() log the size and fullness of openssl-internal buffers, so I can test my hypothesis that many of them are empty, and my alternative hypothesis that many of them are mostly empty, against the null hypothesis that we really need to be burning 32K per open OR connection on this.
svn:r14350
The optimist calls the glass half full. The pessimist calls it half empty. The engineer says it is twice as large as it needs to be. In this case, the engineer says that the default smartlist size is twice as large as it needs to be and wouldn't it be nice to save half a megabyte with a one-line patch?
svn:r14341
When we remove old routers, use Bloom filters rather than a digestmap-based set in order to tell which ones we absolutely need to keep. This will save us roughly a kazillion little short-lived allocations for hash table entries.
svn:r14318
Add a new SMARTLIST_FOREACH_JOIN macro to iterate through two sorted lists in lockstep. This happens at least 3 times in the code so far, and is likely to happen more in the future. Previous attempts to do so proved touchy, tricky, and error-prone: now, we only need to get it right in one place.
svn:r14309
New --hush command-line option similar to --quiet. While --quiet disables all
logging to the console on startup, --hush limits the output to messages of
warning and error severity.
svn:r14222
Add new stacklike, free-all-at-once memory allocation strategy. Use it when parsing directory information. This helps parsing speed, and may well help fragmentation some too. hidden-service-related stuff still uses the old tokenizing strategies.
svn:r14194
Combine common code in set_max_file_descriptors(): all that varies from platform to platform in the no-getrlimit() case is the connection limit and the platform name.
svn:r14101
Use 8k pages in openbsd malloc code on alpha. Bug and solution found by weasel. Also, when initializing openbsd malloc code, check that compiled page size matches output of getpagesize().
svn:r14056
Add a malloc_good_size() implementation to OpenBSD_malloc_Linux.c. Also, make configure.in not use support functions for the platform malloc when we are not using the platform mallocs.
svn:r14010
Have OpenBSD_malloc_Linux.c use SIZE_T_MAX from torint.h, instead of checking cpu macros. There is always one more cpu you havent checked for.
svn:r14000
Fix for bug 614: always look at the network BIO for the SSL object, not at the buffering BIO (if one exists because we are renegotiating or something). Bugfix on 0.1.2.x, oddly enough, though it should be impossible to trigger the problem there. Backport candidate. See comments in tortls.c for detailed implementation note.
svn:r13975
Implement domain-selection for logging. Source is documented; needs documentation in manpage (maybe). For now, see doxygen comment on parse_log_severity_config in log.c
svn:r13875
<weasel> tortls.c:634: warning: passing arg 1 of `SSL_get_session' discards
qualifiers from pointer target type
Nick, see if you like this patch.
svn:r13690
Fix all remaining shorten-64-to-32 errors in src/common. Some were genuine problems. Many were compatibility errors with libraries (openssl, zlib) that like predate size_t. Partial backport candidate.
svn:r13665
Do the last part of arma's fix for bug 437: Track the origin of every addrmap, and use this info so we can remove all the trackhostexits-originated mappings for a given exit.
svn:r13660
Answer one xxx020 item; move 7 other ones to a new "XXX020rc" category: they should get fixed before we cut a release candidate. arma: please review these to see whether you have fixes/answers for any. Please check out the other 14 XXX020s to see if any look critical for the release candidate.
svn:r13640
Apply patch from Sebastian Hahn: stop imposing an arbitrary maximum on the number of file descriptors used for busy servers. Bug reported by Olaf Selke.
svn:r13626
Add some checks in torgzip.c to make sure we never overflow size_t there. Also make sure we do not realloc(list,0) in container.c. Backport candidate.
svn:r13587
Add openbsd memory allocator discussed in bug 468, to make it easier for linux users to get non-awful allocation patterns. Use --enable-openbsd-malloc to turn it on. Needs more testing.
svn:r13544
Fix all but 2 DOCDOC items; defer many XXX020s (particularly those where fixing them would fix no bugs at the risk of introducing some bugs).
svn:r13529
Re-tune mempool parametes based on testing on peacetime: use smaller chuncks, free them a little more aggressively, and try very hard to concentrate allocations on fuller chunks. Also, lots of new documentation.
svn:r13484
Add a couple of (currently disabled) strategies for trying to avoid using too much ram in memory pools: prefer putting new cells in almost-full chunks, and be willing to free the last empty chunk if we have not needed it for a while. Also add better output to mp_pool_log_status to track how many mallocs a given memory pool strategy is saving us, so we can tune the mempool parameters.
svn:r13428
Fix some warnings identified by building with -D_FORTIFY_SOURCE=2. Remove a redundant (and nuts) definition of _FORTIFY_SOURCE from eventdns.c.
svn:r13424
Be more thorough about memory poisoning and clearing. Add an in-place version of aes_crypt in order to remove a memcpy from relay_crypt_one_payload.
svn:r13414
The SSL portion of the revised handshake now seems to work: I just finally got a client and a server to negotiate versions. Now to make sure certificate verification is really happening, connections are getting opened, etc.
svn:r13409
Add more documentation; change the behavior of read_to_buf_tls to be more consistent. Note a longstanding problem with current read/write interfaces.
svn:r13407
Write a new autoconf macro to test whether a function is declared. It is suboptimal and possibly buggy in some way, but it seems to work for me. use it to test for a declaration of malloc_good_size, so we can workaround operating systems (like older OSX) that have the function in their libc but do not deign to declare it in their headers. Should resolve bug 587.
svn:r13339
Add a reverse mapping from SSL to tor_tls_t*: we need this in order to do a couple of things the sensible way from inside callbacks. Also, add a couple of missing cases in connection_or.c
svn:r13040
Another test for the increasingly bad check-spaces style checker to check: #else\n#if is almost a sure sign of a failure to use #elif. Fortunately, we only did that 3 times.
svn:r13039
Push the strdups used for parsing configuration lines into parse_line_from_string(). This will make it easier to parse more complex value formats, which in turn will help fix bug 557
svn:r13020
Fix bug 575: protect the list of logs with a mutex. I couldn't find any appreciable change in logging performance on osx, but ymmv. You can undef USE_LOG_MUTEX to see if stuff gets faster for you.
svn:r13019
Use reference-counting to avoid allocating a zillion little addr_policy_t objects. (This is an old patch that had been sitting on my hard drive for a while.)
svn:r13017
New, slightly esoteric function, tor_malloc_roundup(). While tor_malloc(x) allocates x bytes, tor_malloc_roundup(&x) allocates the same size of chunk it would use to store x bytes, and sets x to the usable size of that chunk.
svn:r12981
Aaand, do the code to enable the client side of the new TLS handshake. There are some loose ends that need tying up in connection_or, and a lot of half-baked code to remove, and some special cases to test for, and lots and lots of testing to do, but that is what weekends are for.
svn:r12721
Change tor_addr_t to be a tagged union of in_addr and in6_addr, not of sockaddr_in and sockaddr_in6. It's hardly used in the main code as it is, but let's get it right before it gets popular.
svn:r12660
Add support to get a callback invoked when the client renegotiate a connection. Also, make clients renegotiate. (not enabled yet, until they detect that the server acted like a v2 server)
svn:r12623
Start getting freaky with openssl callbacks in tortls.c: detect client ciphers, and if the list doesn't look like the list current Tors use, present only a single cert do not ask for a client cert. Also, support for client-side renegotiation. None of this is enabled unless you define V2_HANDSHAKE_SERVER.
svn:r12622
Mess with the formula for the Guard flag again. Now it requires that you be in the most familiar 7/8 of nodes, and have above median wfu for that 7/8th. See spec for details. Also, log thresholds better.
svn:r12440
Make TLS contexts reference-counted, and add a reference from TLS objects to their corresponding context. This lets us reliably get the certificates for a given TLS connection, even if we have rotated TLS contexts.
svn:r12383
If setting our rlimit to rlim_max or cap fails, fall back to OPEN_FILES if defiled. This makes Tor run on OSX 10.5, while allowing OSX to mend its ways in the future.
svn:r12341
Improved skew reporting: "You are 365 days in the duture" is more useful than "You are 525600 minutes in the future". Also, when we get something that proves we are at least an hour in the past, tell the controller "CLOCK_SKEW MIN_SKEW=-3600" rather than just "CLOCK_SKEW"
svn:r12283
New code (disabled for now) to use the SSL context's cert store instead of using its "extra chain cert" list to get our identity certificate sent. This is a little close to what OpenSSL expects people to do, and it has the advantage that we should be able to keep the id cert from being sent by setting the NO_CHAIN_CERT bit. I have tried turning new code on, and it seemed to work fine.
svn:r12086
oprofile was telling me that a fair bit of our time in openssl was spent in base64_decode, so replace base64_decode with an all-at-once fairly optimized implementation. For decoding keys and digests, it seems 3-3.5x faster than calling out to openssl. (Yes, I wrote it from scratch.)
svn:r12002
Add a bunch of function documentation; clean up a little code; fix some XXXXs; tag the nonsensical EXTRAINFO_PURPOSE_GENERAL as nonsesnse; note another bit of "do not cache special routers" code to nuke.
svn:r11761
Re-optimize counter-mode: save about 15% on my core2 by (1) not regenerating the entire counter buffer every time we encrypt a block of keystream (2) using the platform-optimized htonl to convert to big-endian (It's a single instruction on 486 and later ) and (3) not even keeping a separate "counter" and "buffer" when the platform is big-endian. The third still needs testing.
svn:r11536
New (untested) code to implement AES-with-IV. Currently, IVs are generated randomly. Once tested, should be (almost) a drop-in replacement for the CBC functions.
svn:r11519
Switch our AES implementation from "128 bit counter with to 64 bits set to 0" to a proper implementation of counter mode. Also, add an aes_set_iv function to initialize the counter to a nonzero value.
svn:r11518
Refactor write_chunks_to_file_impl: break out the "pick a temporary name if it makes sense, and open the right filename" logic and the "close the file and unlink or rename if necessary" logic. This will let us write big files in a smarter way than "Build a big string" or "make a list of chunks", once we get around to using it.
svn:r11300
Add a new ClientDNSRejectInternalAddresses option (default: on) to refuse to believe that any address can map to or from an internal address. This blocks some kinds of potential browser-based attacks, especially on hosts using DNSPort. Also clarify behavior in some comments. Backport candiate?
svn:r11287
Glibc (and maybe others) define a mallinfo() that can be used to see how the platform malloc is acting inside. When we have it, dump its output on dumpmemusage().
svn:r10996
Cheesy attempt to break some censorware. Not a long-term fix, but it will be intersting to watch the epidemiology of the workarounds as the censors apply them.
svn:r10975
Another patch from croup: drop support for address masks that do not correspond to bit prefixes. Nobody has used this for a while, and we have given warnings for a long time.
svn:r10881
Patch from croup: rewrite the logic of get_next_token() to do the right thing with input that ends at weird places, or aligns with block boundaries after mmap. should fix bug 455. Needs fuzzing.
svn:r10847
Implement conditions in compat.c; switch windows to use "critical sections" instead of mutexes. Apparently, mutexes are for IPC and critical sections are for multithreaded.
svn:r10716
Sun CC likes to give warnings for the do { } while(0) construction for making statement-like macros. Define STMT_BEGIN/STMT_END macros that do the right thing, and use them everywhere.
svn:r10645
Build with fewer compiler warnings on Sun. (This and previous Sun patches are thanks to the tremendously handy services of unix-center.net.)
svn:r10644
Fix the fix for bug 445: set umask properly. Also use open+fdopen rather than just umask+fopen, and create authority identity key with mode 400.
svn:r10485
First bare stubs of ipv6 work: commit some (untested, hence doublessly broken) implementations of inet_ntop/pton for systems that lack them.
svn:r10326
Add math functions to round values to the nearest power of 2. Make mempools more careful about making sure that the size of their chunks is a little less than a power of 2, not a little more.
svn:r10304
More v3 directory code: have authorities load certificates; have everybody store certificates to disk and load them; provide a way to configure v3 authorities.
svn:r10293
Partial backport candidate: do not rely on finding a \0 after an mmaped() router/extrainfo file. Also, set journal length correctly when starting up.
svn:r10248
Add a "swap" function to smartlist, add a "shuffle" function for smartlist to crypto.c, and make appropriate hashtable functions be more const.
svn:r10208
Fix warnings from -Wunsafe-loop-optimizations, which incidentally turned up a logic bug in connection_or_flush_from_first_active_circuit that would overcount the number of cells flushed.
svn:r10199
[Backport candidate] On windows, open cached-routers with the sharing mode "FILE_SHARE_READ so that other processes can read it while Tor is running. (Reported by Janbar).
svn:r10148
Track the number of connection_t separately from the number of open sockets. It is already possible to have connections that do not count: resolving conns, for one. Once we move from socketpairs to linked conns, and once we do dns proxying, there will be lots of such connections.
svn:r9994
Fix a bug in displaying memory pool usage. Also dump cell allocation, and track padded_cell_ts as they are allocated and freed, to make sure we are not leaking cells.
svn:r9992
Add code to shrink the cell memory pool by discarding empty chunks that have been empty for the last 60 seconds. Also, instead of having test.c duplicate declarations for exposed functions, put them inside #ifdef foo_PRIVATE blocks in the headers. This prevents bugs where test.c gets out of sync.
svn:r9944
Make all LD_BUG log messsages get prefixed with "Bug: ". Remove manually-generated "Bug: "s from log-messages. (Apparently, we remembered to add them about 40% of the time.)
svn:r9733
On mingw, use "%I64u" to printf/scanf 64-bit integers, instead of the usual GCC "%llu". This prevents a bug when saving 64-bit int configuration values on mingw; the high-order 32 bits would get truncated. If the value was then reloaded, disaster would occur. (Fixes bug 400 and maybe also bug 397.) Backport candidate.
svn:r9691
try to make aes encryption of cells about 30-40% faster where applicable. offer not available for all architectures or all versions of openssl.
svn:r9663
Another optimization suggested by Shark output: shave off >90% of uses of logv by cutting down on calls to log_debug when log actually debugging. This is showing up in some profiles bug not others, and might be as much as 2.5%.
svn:r9612
Apply patch from coderman: have posix subthreads mask out signals. This could prevent some kinds of crashes when subthreads try to handle SIGPIPEs and die in the attempt. Backport candidate.
svn:r9603
Move all struct-offset-manipulation macros into util.h, and use them consistently. Because there are days when "SUBTYPE_P(handle, subtype, _base)" is just easier to read and write than "(basetp*)(((handle) - STRUCT_OFFSET(subtype, _base))".
svn:r9592
Node-picking fixes: Never warn when a down node is listed in a config option (bug 348); always warn when a node in a config option is unnamed. Also, when entrynodes is configured, then build the guard list as: (parts of EntryNodes that were guards before), (the rest of EntryNodes), (the rest of the old guards). This last point ensures that EntryNode users will get the EntryNodes they want with the minimum change to their guard list.
svn:r9574
Handle errors on opening cached-routers* more uniformly and sanely: log not-found errors at level INFO, and all other errors at level WARN. Needs testing on win32.
svn:r9569
Implement proposal 106: stop requiring clients to have certificates, and stop checking for nicknames in certificates. [See proposal 106 for rationale.] Also improve messages when checking TLS handshake, to re-resolve bug 382.
svn:r9568
Add documentation to src/common/*.h; improve documentation for SMARTLIST_FOREACH; remove never-used options and corresponding tests from tor_strpartition.
svn:r9483
Removing the last DOCDOC comment hurt so much that I had to use Doxygen to identify undocumented macros and comments, and add 150 more DOCDOCs to point out where they were. Oops. Hey, kids! Fixing some of these could be your first Tor patch!
svn:r9477
Note a few questionable bits pointed out by RATS. (And no, RATS, I'm not about to stop having fixed-sized local buffers for you. I know how to use them, thankyouverymuch.)
svn:r9474
Implement SOCKS_BAD_HOSTNAME status event. Defer remaining status events. Clean up control-spec.txt a little, and fill in recommendations for events.
svn:r9374
Yes, apparently saying strcpy in front of openbsd is like saying "intellectual property" in front of RMS. They both have a point, I guess, even though they extend it to contexts where it is completely irrelevant.
svn:r9370
Apparently, the OpenBSD linker thinks it knows C better than I do, and gets to call me names for having strcat and strcpy and sprintf in my code--whether I use them safely or not. All right, OpenBSD. You win... this round.
svn:r9360
Tidy up ORCONN reason patch from Mike Perry. Changes: make some of the handling of TLS error codes less error prone. Enforce house style wrt spaces. Make it compile with --enable-gcc-warnings. Only set or_conn->tls_error in the case of an actual error. Add a changelog entry.
svn:r9355
"read this if you don't understand the code and want some help."
which is not the same as "hey, you think you understand this code,
but you don't."
svn:r9307
Count TLS bytes accurately: previously, we counted only the number of bytes read or transmitted via tls, not the number of extra bytes used to do so. This has been a lonstanding wart. The fix "Works for me".
svn:r9207
Fix bug found by Keith Skinner: Treat malformed max-ports in address ranges as an error, and dont ignore errors with min-ports even if a max-port is present.
svn:r9168
Fix a couple of obvious bugs in tor_mmap_file on Windows: first, fix a boolean error when checking the return value of CreateFileMapping. Second, CreateFileMapping is documented to return NULL on failure.
svn:r9035
Try to compile with fewer warnings on irix64's MIPSpro compiler /
environment, which apparently believes that:
- off_t can be bigger than size_t.
- only mean kids assign things they do not subsequently inspect.
I don't try to fix the "error" that makes it say:
cc-3970 cc: WARNING File = main.c, Line = 1277
conversion from pointer to same-sized integral type (potential portability
problem)
uintptr_t sig = (uintptr_t)arg;
Because really, what can you do about a compiler that claims to be c99
but doesn't understand that void* x = NULL; uintptr_t y = (uintptr_t) x;
is safe?
svn:r8948
Add string.h include to compat.h so that strlcpy() and strlcat() will always be defined after including compat.h. This should resolve warnings on centos.
svn:r8824
Change HT_INIT from a special-case to a macro-calling-a-function like the rest of ht.h. This might prevent errors if somebody did, I dunno, "HT_INIT(tp, hashtable++)". Based on a patch from Watson Ladd.
svn:r8776
Add unit tests for tor_mmap_file(); make tor_mmap_t.size always be the size of the file (not the size of the mapping); add an extra argument to read_file_to_str() so it can return the size of the result string.
svn:r8762
The otherwise regrettable MIPSpro C compiler warns about values set but never used, and about mixing enums and ints; these are good warnings, and so should be fixed. This removes some dead code and some potential bugs. Thanks to pnx.
svn:r8664
Remove/clarify some XXXs for no longer being accurate; for begin things we do not indend to fix; for already being parts of big todo issues (like "/* XXX ipv6 */"); etc. Also fix some spaces.
svn:r8580
Move is_local_IP to config.c; have it check for same-/24; make it used only for reachability (not for banwidth, because that is probably not what we want). Fixes an XXX.
svn:r8578
Improvement to last entry guards patch: track when we last attempted to connect to a node in our state file along with how long it has been unreachable. Also clarify behavior of parse_iso_time() when it gets extra characters.
svn:r8520
Do not graciously increase the size to be mmaped if the current size already is
at a page_size boundary. This is important since if a file has a size of zero
and we mmap() it with length > 0, then accessing the mmaped memory area causes
a bus error. However, if we pass a length of 0 to mmap() it will return with -1
and things work from there.
svn:r8387
spawn_func fixes: have cpuworker_main and dnsworker_main confirm to the right interfaces [casting func to void* is icky]. Also, make pthread_create() build without warnings.
svn:r8327
- made configure link to required system dll's if building for win32
- added diffs for libevent 1.1b
- forced user to turn off eventdns if win32 is set
- cleaned up tor_mmap_file()_win32 (not sure if it's stable)
- cleaned up some warnings and typos
svn:r8322
Try to appease some warnings with newer gccs that believe that ignoring a return value is okay, but casting a return value and then ignoring it is a sign of madness.
svn:r8312
Fix bug 327 (part 2): Cast char to unsigned char before passing to toupper/tolower. (Follow the same idiom as with isupper and friends, in case we run into the same problem on SGI or whereever it was.)
svn:r8310
And another GCC change: predict that tor_frees() are usually real frees, and tor_asserts() usually wont happen. Other test should wait till -fprofile-arcs
svn:r7022
possible. Cache at the server side independently from the TTL, to
prevent attackers from probing the server to see who has been asking
for what hostnames. (Hi, Dan Kaminski!)
Also, clean some whitespace.
svn:r6526
we were refusing names that had $ in them, which people who specify
$key.exit will be sad about. There are likely other examples.
If people can think of reasons why we should be picky, let me know.
svn:r6496
This is needed because Windows already has an err() that we
can't clobber. And we need to be able to make the log functions
a macro so we can print the function's name in the log entry.
svn:r6000
constraining ourselves to a max of 1024 connections. Now if there is no
getrlimit, just assume that we can handle the whole 15000 connections.
The better answer is to find a getrlimit equivalent on Windows, but hey,
one step at a time.
svn:r5854
