This patch moves curve25519_keypair_t from src/or/onion_ntor.h to
src/common/crypto_curve25519.h, and adds new functions to generate,
load, and store keypairs.
Previously, we only used the strong OS entropy source as part of
seeding OpenSSL's RNG. But with curve25519, we'll have occasion to
want to generate some keys using extremely-good entopy, as well as the
means to do so. So let's!
This patch refactors the OS-entropy wrapper into its own
crypto_strongest_rand() function, and makes our new
curve25519_secret_key_generate function try it as appropriate.
We want to use donna-c64 when we have a GCC with support for
64x64->uint128_t multiplying. If not, we want to use libnacl if we
can, unless it's giving us the unsafe "ref" implementation. And if
that isn't going to work, we'd like to use the
portable-and-safe-but-slow 32-bit "donna" implementation.
We might need more library searching for the correct libnacl,
especially once the next libnacl release is out -- it's likely to have
bunches of better curve25519 implementations.
I also define a set of curve25519 wrapper functions, though it really
shouldn't be necessary.
We should eventually make the -donna*.c files get build with
-fomit-frame-pointer, since that can make a difference.
This implements the server-side of proposal 198 by detecting when
clients lack the magic list of ciphersuites that indicates that
they're lying faking some ciphers they don't really have. When
clients lack this list, we can choose any cipher that we'd actually
like. The newly allowed ciphersuites are, currently, "All ECDHE-RSA
ciphers that openssl supports, except for ECDHE-RSA-RC4".
The code to detect the cipher list relies on on (ab)use of
SSL_set_session_secret_cb.
We already use this classification for deciding whether (as a server)
to do a v2/v3 handshake, and we're about to start using it for
deciding whether we can use good ciphersuites too.
This is less easy than you might think; we can't just look at the
client ciphers list, since openssl doesn't remember client ciphers if
it doesn't know about them. So we have to keep a list of the "v2"
ciphers, with the ones we don't know about removed.
We want to be saying fast_mem{cmp,eq,neq} when we're doing a
comparison that's allowed to exit early, or tor_mem{cmp,eq,neq} when
we need a data-invariant timing. Direct use of memcmp tends to imply
that we haven't thought about the issue.
This is a customizable extract-and-expand HMAC-KDF for deriving keys.
It derives from RFC5869, which derives its rationale from Krawczyk,
H., "Cryptographic Extraction and Key Derivation: The HKDF Scheme",
Proceedings of CRYPTO 2010, 2010, <http://eprint.iacr.org/2010/264>.
I'm also renaming the existing KDF, now that Tor has two of them.
This is the key derivation scheme specified in ntor.
There are also unit tests.
By default, "*" means "All IPv4 addresses" with
tor_addr_parse_mask_ports, so I won't break anything. But if the new
EXTENDED_STAR flag is provided, then * means "any address", *4 means
"any IPv4 address" (that is, 0.0.0.0/0), and "*6" means "any IPv6
address" (that is, [::]/0).
This is going to let us have a syntax for specifying exit policies in
torrc that won't drive people mad.
Also, add a bunch of unit tests for tor_addr_parse_mask_ports to test
these new features, and to increase coverage.
Apparently some compilers like to eliminate memset() operations on
data that's about to go out-of-scope. I've gone with the safest
possible replacement, which might be a bit slow. I don't think this
is critical path in any way that will affect performance, but if it
is, we can work on that in 0.2.4.
Fixes bug 7352.
This is based on code by yayooo for 7260, but:
- It allows for SIZEOF_PID_T == SIZEOF_SHORT
- It addresses some additional cases where we weren't getting any
warnings only because we were casting pid_t to int.
The implementation we added has a tendency to crash with lists of 0 or
one element. That can happen if we get a consensus vote, v2
consensus, consensus, or geoip file with 0 or 1 element. There's a
DOS opportunity there that authorities could exploit against one
another, and which an evil v2 authority could exploit against anything
downloading v2 directory information..
This fix is minimalistic: It just adds a special-case for 0- and
1-element lists. For 0.2.4 (the current alpha series) we'll want a
better patch.
This is bug 7191; it's a fix on 0.2.0.10-alpha.
OpenSSL 1.0.0 added an implementation of TLS session tickets, a
"feature" that let session resumption occur without server-side state
by giving clients an encrypted "ticket" that the client could present
later to get the session going again with the same keys as before.
OpenSSL was giving the keys to decrypt these tickets the lifetime of
the SSL contexts, which would have been terrible for PFS if we had
long-lived SSL contexts. Fortunately, we don't. Still, it's pretty
bad. We should also drop these, since our use of the extension stands
out with our non-use of session cacheing.
Found by nextgens. Bugfix on all versions of Tor when built with
openssl 1.0.0 or later. Fixes bug 7139.
In C, we technically aren't supposed to define our own things that
start with an underscore.
This is a purely machine-generated commit. First, I ran this script
on all the headers in src/{common,or,test,tools/*}/*.h :
==============================
use strict;
my %macros = ();
my %skipped = ();
FILE: for my $fn (@ARGV) {
my $f = $fn;
if ($fn !~ /^\.\//) {
$f = "./$fn";
}
$skipped{$fn} = 0;
open(F, $fn);
while (<F>) {
if (/^#ifndef ([A-Za-z0-9_]+)/) {
$macros{$fn} = $1;
next FILE;
}
}
}
print "#!/usr/bin/perl -w -i -p\n\n";
for my $fn (@ARGV) {
if (! exists $macros{$fn}) {
print "# No macro known for $fn!\n" if (!$skipped{$fn});
next;
}
if ($macros{$fn} !~ /_H_?$/) {
print "# Weird macro for $fn...\n";
}
my $goodmacro = uc $fn;
$goodmacro =~ s#.*/##;
$goodmacro =~ s#[\/\-\.]#_#g;
print "s/(?<![A-Za-z0-9_])$macros{$fn}(?![A-Za-z0-9_])/TOR_${goodmacro}/g;\n"
}
==============================
It produced the following output, which I then re-ran on those same files:
==============================
s/(?<![A-Za-z0-9_])_TOR_ADDRESS_H(?![A-Za-z0-9_])/TOR_ADDRESS_H/g;
s/(?<![A-Za-z0-9_])_TOR_AES_H(?![A-Za-z0-9_])/TOR_AES_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_H(?![A-Za-z0-9_])/TOR_COMPAT_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_LIBEVENT_H(?![A-Za-z0-9_])/TOR_COMPAT_LIBEVENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTAINER_H(?![A-Za-z0-9_])/TOR_CONTAINER_H/g;
s/(?<![A-Za-z0-9_])_TOR_CRYPTO_H(?![A-Za-z0-9_])/TOR_CRYPTO_H/g;
s/(?<![A-Za-z0-9_])TOR_DI_OPS_H(?![A-Za-z0-9_])/TOR_DI_OPS_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMAREA_H(?![A-Za-z0-9_])/TOR_MEMAREA_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMPOOL_H(?![A-Za-z0-9_])/TOR_MEMPOOL_H/g;
s/(?<![A-Za-z0-9_])TOR_PROCMON_H(?![A-Za-z0-9_])/TOR_PROCMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORGZIP_H(?![A-Za-z0-9_])/TOR_TORGZIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORINT_H(?![A-Za-z0-9_])/TOR_TORINT_H/g;
s/(?<![A-Za-z0-9_])_TOR_LOG_H(?![A-Za-z0-9_])/TOR_TORLOG_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORTLS_H(?![A-Za-z0-9_])/TOR_TORTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_UTIL_H(?![A-Za-z0-9_])/TOR_UTIL_H/g;
s/(?<![A-Za-z0-9_])_TOR_BUFFERS_H(?![A-Za-z0-9_])/TOR_BUFFERS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_H(?![A-Za-z0-9_])/TOR_CHANNEL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_TLS_H(?![A-Za-z0-9_])/TOR_CHANNELTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITBUILD_H(?![A-Za-z0-9_])/TOR_CIRCUITBUILD_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITLIST_H(?![A-Za-z0-9_])/TOR_CIRCUITLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_EWMA_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_EWMA_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITUSE_H(?![A-Za-z0-9_])/TOR_CIRCUITUSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMMAND_H(?![A-Za-z0-9_])/TOR_COMMAND_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONFIG_H(?![A-Za-z0-9_])/TOR_CONFIG_H/g;
s/(?<![A-Za-z0-9_])TOR_CONFPARSE_H(?![A-Za-z0-9_])/TOR_CONFPARSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_EDGE_H(?![A-Za-z0-9_])/TOR_CONNECTION_EDGE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_H(?![A-Za-z0-9_])/TOR_CONNECTION_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_OR_H(?![A-Za-z0-9_])/TOR_CONNECTION_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTROL_H(?![A-Za-z0-9_])/TOR_CONTROL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CPUWORKER_H(?![A-Za-z0-9_])/TOR_CPUWORKER_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRECTORY_H(?![A-Za-z0-9_])/TOR_DIRECTORY_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRSERV_H(?![A-Za-z0-9_])/TOR_DIRSERV_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRVOTE_H(?![A-Za-z0-9_])/TOR_DIRVOTE_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNS_H(?![A-Za-z0-9_])/TOR_DNS_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNSSERV_H(?![A-Za-z0-9_])/TOR_DNSSERV_H/g;
s/(?<![A-Za-z0-9_])TOR_EVENTDNS_TOR_H(?![A-Za-z0-9_])/TOR_EVENTDNS_TOR_H/g;
s/(?<![A-Za-z0-9_])_TOR_GEOIP_H(?![A-Za-z0-9_])/TOR_GEOIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_HIBERNATE_H(?![A-Za-z0-9_])/TOR_HIBERNATE_H/g;
s/(?<![A-Za-z0-9_])_TOR_MAIN_H(?![A-Za-z0-9_])/TOR_MAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_MICRODESC_H(?![A-Za-z0-9_])/TOR_MICRODESC_H/g;
s/(?<![A-Za-z0-9_])_TOR_NETWORKSTATUS_H(?![A-Za-z0-9_])/TOR_NETWORKSTATUS_H/g;
s/(?<![A-Za-z0-9_])_TOR_NODELIST_H(?![A-Za-z0-9_])/TOR_NODELIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_NTMAIN_H(?![A-Za-z0-9_])/TOR_NTMAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_ONION_H(?![A-Za-z0-9_])/TOR_ONION_H/g;
s/(?<![A-Za-z0-9_])_TOR_OR_H(?![A-Za-z0-9_])/TOR_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_POLICIES_H(?![A-Za-z0-9_])/TOR_POLICIES_H/g;
s/(?<![A-Za-z0-9_])_TOR_REASONS_H(?![A-Za-z0-9_])/TOR_REASONS_H/g;
s/(?<![A-Za-z0-9_])_TOR_RELAY_H(?![A-Za-z0-9_])/TOR_RELAY_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCLIENT_H(?![A-Za-z0-9_])/TOR_RENDCLIENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCOMMON_H(?![A-Za-z0-9_])/TOR_RENDCOMMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDMID_H(?![A-Za-z0-9_])/TOR_RENDMID_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDSERVICE_H(?![A-Za-z0-9_])/TOR_RENDSERVICE_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPHIST_H(?![A-Za-z0-9_])/TOR_REPHIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPLAYCACHE_H(?![A-Za-z0-9_])/TOR_REPLAYCACHE_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTER_H(?![A-Za-z0-9_])/TOR_ROUTER_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERLIST_H(?![A-Za-z0-9_])/TOR_ROUTERLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERPARSE_H(?![A-Za-z0-9_])/TOR_ROUTERPARSE_H/g;
s/(?<![A-Za-z0-9_])TOR_ROUTERSET_H(?![A-Za-z0-9_])/TOR_ROUTERSET_H/g;
s/(?<![A-Za-z0-9_])TOR_STATEFILE_H(?![A-Za-z0-9_])/TOR_STATEFILE_H/g;
s/(?<![A-Za-z0-9_])_TOR_STATUS_H(?![A-Za-z0-9_])/TOR_STATUS_H/g;
s/(?<![A-Za-z0-9_])TOR_TRANSPORTS_H(?![A-Za-z0-9_])/TOR_TRANSPORTS_H/g;
s/(?<![A-Za-z0-9_])_TOR_TEST_H(?![A-Za-z0-9_])/TOR_TEST_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_NATPMP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_NATPMP_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_UPNP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_UPNP_H/g;
==============================
The rationale for treating these files differently is that we should
be checking upstream for changes as applicable, and merging changes
upstream as warranted.
Conflicts:
src/or/circuitbuild.c
The conflict was trivial, since no line of code actually changed in
both branches: There was a fmt_addr() that turned into fmt_addrport()
in bug7011, and a "if (!n_conn)" that turned into "if (!n_chan)" in
master.
Note: this is a squashed commit; see branch bug6465_rebased_v2 of user/andrea/tor.git for full history of the following 90 commits:
Add channel.c/channel.h for bug 6465
Fix make check-spaces in new channel.c/channel.h
Make sure new channel.h is in nodist_HEADERS and Makefile.nmake is up to date too
Add channel_state_t and state utility functions
Add channel_change_state()
Better comments in channel.h
Add CHANNEL_STATE_LISTENING for channel_t
Fix wide line in channel.c
Add structures/prototypes for incoming cell handling
Implement channel_queue_cell() and channel_queue_var_cell()
Implement channel_process_cells()
Fix asserts in channel_queue_cell() and channel_queue_var_cell()
Add descriptive comments for channel_queue_cell() and channel_queue_var_cell()
Implement channel cell handler getters/setters
Queue outgoing writes when not in writeable state
Drain queues and test assertions when changing channel_t state
Add log_debug() messages for channel_t stuff
Add log_debug() messages for channel_t stuff
Add some channel_t metadata
Add time_t client_used to channel_t
Add channel_touched_by_client()
Declare a few channel_t metadata queries we'll have to implement later for use by circuitbuild.c
Add next_circ_id/circ_id_type to channel_t for use by circuitbuild.c
Count n_circuits in channel_t
Channel timestamp calls
Add create timestamp for channel.h
Declare some new metadata queries on channel_t
Add get_real_remote_descr() prototype
Move active_circuits stuff to channel_t, some other or.h and channel.h changes
Make channel_t refcounted and use global lists of active channels
Update channel_request_close() and channel_change_state() for channel_t registration mechanism
Handle closing channels sensibly
Add global_identifier for channels, channel_init() internal use function
Add timestamp_last_added_nonpadding to channel_t
Better comments in channel_init()
Correctly handle next_circ_id in channel_init()
Correctly handle next_circ_id in channel_init() and even compile this time
Appease make check-spaces
Update timestamps when writing cells to channel_t
Add channel_flush_some_cells() to call channel_flush_from_first_active_circuit()
Add registered channel lookup functions
Get rid of client_used in or_connection_t; it's in channel_t now
Get rid of circ_id_type in or_connection_t; implement channel_set_circ_id_type()
Eliminate is_bad_for_new_circs in or_connection_t; implement getter/setter for it in channel_t
Eliminate next_circ_id in or_connection_t in favor of channel_t
Handle packed cells in channel_t for relay.c
Add channel_identity_map and related functions
Handle add/remove from channel identity map on state transitions
Implement channel_is_local() and channel_mark_local()
Implement channel_is_client() and channel_mark_client()
Implement channel_is_outgoing() and channel_mark_outgoing()
Eliminate declaration for redundant channel_nonopen_was_started_here()
Add channel timestamps
Add channel timestamps, fix some make-check-spaces complaints
Remove redundant channel_was_started_here() function and initiated_remotely bit
Rename channel_get_remote_descr()/channel_get_real_remote_descr() to something clearer in channel.h
Replace channel_get_write_queue_len() with sufficient and easier to implement channel_has_queued_writes() in channel.h
Change return type of channel_is_bad_for_new_circs() to int for consistency
Implement channel_has_queued_writes()
Rename channel_touched_by_client() and client_used field for consistency with other timestamps in channel.{c,h}
Implement channel_get_actual_remote_descr() and channel_get_canonical_remote_descr() in channel.{c,h}
Implement channel_matches_extend_info() in channel.{c,h}
Implement channel_get_for_extend() and channel_is_better() in channel.{c,h}
Make channel_is_better() public in channel.{c,h}
Implement channel_matches_target_addr_for_extend() in channel.{c,h}
Implement channel_is_canonical_is_reliable() in channel.{c,h}
Demoronize get_remote_descr() method prototype - what the hell was I thinking there?
Timestamp channels in the right places in channel.c
Add missing tor_assert() in channel.c
Check if the lower layer accepted a cell in channel_write_cell() et al. of channel.c
Implement channel_flush_cells() in channel.c (w00t, it builds at last)
Call channel_timestamp_drained() at the right places in channel.c
Implement channel_run_cleanup()
Support optional channel_get_remote_addr() method and use it for GeoIP in channel_do_open_actions()
Get rid of channel refcounting; it'll be too complicated to handle it properly with all the pointers from circuits to channels, and closing from channel_run_cleanup() will work okay just like with connections
Doxygenate channel.c
Appease make check-spaces in channel.c
Fix superfluous semicolons in channel.c
Add/remove channels from identity digest map in all the right places in channel.c
The cell queues on channel_t must be empty when going to a CLOSED or ERROR state
Appease make check-spaces in channel.c
Add channel_clear/set_identity_digest() and some better logging to channel.{c,h}
Fix better logging to channel.c
Avoid SIGSEGV testing for queue emptiness in channel_flush_some_cells_from_outgoing_queue()
Remove TODO about checking cell queue in channel_free(); no need for it
Appease make check-spaces in channel.c
Add channel_free_all() and support functions
Check nullness of active_circuit_pqueue in channel_free()
Fix SMARTLIST_FOREACH_END usage in channel_process_cells()
Rearrange channel_t struct to use a union distinguishing listener from cell-bearing channels in channel.{c,h}
Now that crypto_pk_cmp_keys might return the result of tor_memcmp, there
is no guarantee that it will only return -1, 0, or 1. (It currently does
only return -1, 0, or 1, but that's a lucky accident due to details of the
current implementation of tor_memcmp and the particular input given to it.)
Fortunately, none of crypto_pk_cmp_keys's callers rely on this behaviour,
so changing its documentation is sufficient.
add read_file_to_str_until_eof which is used by read_file_to_str
if the file happens to be a FIFO.
change file_status() to return FN_FILE if st_mode matches S_IFIFO
(on not-windows) so that init_key_from_file() will read from a FIFO.
We already had code on windows to fix our file sizes when we're
reading a file in text mode and its size doesn't match the size from
fstat. But that code was only enabled when _WIN32 was defined, and
Cygwin defines __CYGWIN__ instead.
Fixes bug 6844; bugfix on 0.1.2.7-alpha.
This would be undefined behavior if it happened. (It can't actually
happen as we're using round_to_power_of_2, since we would have to
be trying to allocate exabytes of data.)
While we're at it, fix the behavior of round_to_power_of_2(0),
and document the function better.
Fix for bug 6831.
097 hasn't seen a new version since 2007; we can drop support too.
This lets us remove our built-in sha256 implementation, and some
checks for old bugs.
We already do this for libevent; let's do it for openssl too.
For now, I'm making it always a warn, since this has caused some
problems in the past. Later, we can see about making it less severe.
Add handle_fw_helper_output(), a function responsible for parsing the
output of tor-fw-helper. Refactor tor_check_port_forwarding() and
run_scheduled_events() accordingly too.
We now issue warnings when we get control output from tor-fw-helper,
and we log the verbose output of tor-fw-helper in LOG_INFO.
Conflicts:
src/common/util.c
get_lines_from_handle() is a multiplatform function which drains lines
from a stream and stuffs it into a smartlist. It's useful for
line-based protocols, like the one managed proxy and the tor-fw-helper
protocols.
Apparently, (void)writev is not enough to suppress the "you are
ignoring the return value!" warnings on Linux. Instead, remove the
whole warning/error logic when compiling openbsd_malloc for Tor: we
can't use it.
The warning fixes are:
- Only define issetugid if it's missing.
- Explicitly ignore the return value of writev.
- Explicitly cast the retval of readlink() to int.
The 64-bit problems are related to just storing a size_t in an int. Not cool! Use a size_t instead.
Fix for bug 6379. Bugfix on 0.2.0.20-rc, which introduced openbsd-malloc.
This should make our preferred solution to #6538 easier to
implement, avoid a bunch of potential nastiness with excessive
int-vs-double math, and generally make the code there a little less
scary.
"But wait!" you say. "Is it really safe to do this? Won't the
results come out differently?"
Yes, but not much. We now round every weighted bandwidth to the
nearest byte before computing on it. This will make every node that
had a fractional part of its weighted bandwidth before either
slighty more likely or slightly less likely. Further, the rand_bw
value was only ever set with integer precision, so it can't
accurately sample routers with tiny fractional bandwidth values
anyway. Finally, doing repeated double-vs-uint64 comparisons is
just plain sad; it will involve an implicit cast to double, which is
never a fun thing.
This gives us a few benefits:
1) make -j clean all
this will start working, as it should. It currently doesn't.
2) increased parallel build
recursive make will max out at number of files in a directory,
non-recursive make doesn't have such a limitation
3) Removal of duplicate information in make files,
less error prone
I've also slightly updated how we call AM_INIT_AUTOMAKE, as the way
that was used was not only deprecated but will be *removed* in the next
major automake release (1.13).... so probably best that we can continue
to bulid tor without requiring old automake.
(see http://www.gnu.org/software/automake/manual/html_node/Public-Macros.html )
For more reasons why, see resources such as:
http://miller.emu.id.au/pmiller/books/rmch/
The SMARTLIST_FOREACH macro is more convenient than BEGIN/END when
you have a nice short loop body, but using it for long bodies makes
your preprocessor tell the compiler that all the code is on the same
line. That causes grief, since compiler warnings and debugger lines
will all refer to that one line.
So, here's a new style rule: SMARTLIST_FOREACH blocks need to be
short.
With glibc 2.15 and clang 3.0, I get warnings from where we use the
strcpsn implementation in the header as strcspn(string, "="). This
is apparently because clang sees that part of the strcspn macro
expands to "="[2], and doesn't realize that that part of the macro
is only evaluated when "="[1] != 0.
We can treat this case as an EAGAIN (probably because of an
unexpected internal NUL) rather than a crash-worthy problem.
Fixes bug 6225, again. Bug not in any released version of Tor.
Because the string output was no longer equal in length to
HEX_ERRNO_SIZE, the write() call would add some extra spaces and
maybe a NUL, and the NUL would trigger an assert in
get_string_from_pipe.
Fixes bug 6225; bug not in any released version of Tor.
This is a feature removal: we no longer fake any ciphersuite other
than the not-really-standard SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA
(0xfeff). This change will let servers rely on our actually
supporting what we claim to support, and thereby let Tor migrate to
better TLS ciphersuites.
As a drawback, Tor instances that use old openssl versions and
openssl builds with ciphers disabled will no longer give the
"firefox" cipher list.
The function is not guaranteed to NUL-terminate its output. It
*is*, however, guaranteed not to generate more than two bytes per
multibyte character (plus terminating nul), so the general approach
I'm taking is to try to allocate enough space, AND to manually add a
NUL at the end of each buffer just in case I screwed up the "enough
space" thing.
Fixes bug 5909.
These include:
- Having a weird in_addr that can't be initialized with {0}
- Needing INVALID_HANDLE_VALUE instead of -1 for file handles.
- Having a weird dependent definition for struct stat.
- pid is signed, not unsigned.
Also, try to resolve some doxygen issues. First, define a magic
"This is doxygen!" macro so that we take the correct branch in
various #if/#else/#endifs in order to get the right documentation.
Second, add in a few grouping @{ and @} entries in order to get some
variables and fields to get grouped together.
It appears that when OpenSSL negotiates a 1.1 or 1.2 connection, and it
decides to renegotiate, the client will send a record with version "1.0"
rather than with the current TLS version. This would cause the
connection to fail whenever both sides had OpenSSL 1.0.1, and the v2 Tor
handshake was in use.
As a workaround, disable TLS 1.1 and TLS 1.2. When a later version of
OpenSSL is released, we can make this conditional on running a fixed
version of OpenSSL.
Alternatively, we could disable TLS 1.1 and TLS 1.2 only on the client
side. But doing it this way for now means that we not only fix TLS with
patched clients; we also fix TLS when the server has this patch and the
client does not. That could be important to keep the network running
well.
Fixes bug 6033.
Conflicts:
src/test/test_util.c
Merge the unit tests; I added some when I did this branch against
0.2.2, and then the test format changed and master added more tests.
Conflicts:
src/common/compat.h
Conflict was between replacement of MS_WINDOWS with _WIN32 in
master, and with removal of file_handle from tor_mmap_t struct in
close_file_mapping branch (for bug 5951 fix).
It turns out that if you set the third argument of
__attribute__(format) to 0, GCC and Clang will check the format
argument without expecting to find variadic arguments. This is the
correct behavior for vsnprintf, vasprintf, and vscanf.
I'm hoping this will fix bug 5969 (a clang warning) by telling clang that
the format argument to tor_vasprintf is indeed a format string.
The parent of "/foo" is "/"; and "/" is its own parent.
This would cause Tor to fail if you tried to have a PF_UNIX control
socket in the root directory. That would be a stupid thing to do
for other reasons, but there's no reason to fail like _this_.
Bug found by Esteban Manchado Velázquez. Fix for bug 5089; bugfix on
Tor 0.2.2.26-beta. Unit test included.
* It seems parse_http_time wasn't parsing correctly any date with commas (RFCs
1123 and 850). Fix that.
* It seems parse_http_time was reporting the wrong month (they start at 0, not
1). Fix that.
* Add some tests for parse_http_time, covering all three formats.
We've been only treating SW_SERVER_HELLO_A as meaning that an SSL
handshake was happening. But that's not right: if the initial
attempt to write a ServerHello fails, we would get a callback in
state SW_SERVER_HELLO_B instead.
(That's "instead" and not "in addition": any failed attempt to write
the hello will fail and cause the info callback not to get written.)
Fix for bug 4592; bugfix on 0.2.0.13-alpha.
This tells the windows headers to give us definitions that didn't
exist before XP -- like the ones that we need for IPv6 support.
See bug #5861. We didn't run into this issue with mingw, since
mingw doesn't respect _WIN32_WINNT as well as it should for some of
its definitions.
For uname-based detection, we now give only the OS name (e.g.,
"Darwin", "Linux".) For Windows, we give only the Operating System
name as inferred from dw(Major|Minor)version, (e.g., "Windows XP",
"Windows 7"), and whether the VER_NT_SERVER flag is set.
For ticket 2988.
The underlying strtoX functions handle overflow by saturating and
setting errno to ERANGE. If the min/max arguments to the
tor_parse_* functions are equal to the minimum/maximum of the
underlying type, then with the old approach, we wouldn't treat a
too-large value as genuinely broken.
Found this while looking at bug 5786; bugfix on 19da1f36 (in Tor
0.0.9), which introduced these functions.
We had been checking for EINVAL, but that means that SOCK_* isn't
supported, not that the syscall itself is missing.
Bugfix on 0.2.3.1-alpha, which started to use accept4.
I think that the trailing __ got added in false analogy to
HAVE_MACRO__func__, HAVE_MACRO__FUNC__, and HAVE_MACRO__FUNCTION__.
But those macros actually indicate the presence of __func__,
__FUNC__, and __FUNCTION__ respectively. The __ at the end of
HAVE_EXTERN_ENVIRON_DECLARED would only be appropriate if the
environ were declared__, whatever that means.
(As a side-note, HAVE_MACRO__func__ and so on should probably be
renamed HAVE_MACRO___func__ and so on. But that can wait.)
This is an identifier renaming only.
If the client uses a v2 cipherlist on the renegotiation handshake,
it looks as if they could fail to get a good cert chain from the
server, since they server would re-disable certificate chaining.
This patch makes it so the code that make the server side of the
first v2 handshake special can get called only once.
Fix for 4591; bugfix on 0.2.0.20-rc.
They boil down to:
- MS_WINDOWS is dead and replaced with _WIN32, but we let a few
instances creep in when we merged Esteban's tests.
- Capitalizing windows header names confuses mingw.
- #ifdef 0 ain't C.
- One unit test wasn't compiled on windows, but was being listed
anyway.
- One unit test was checking for the wrong value.
Gisle Vanem found and fixed the latter 3 issues.
Fixes coverity CID 508: coverity scan doesn't like checking a
variable for non-NULL after it has been definitely dereferenced.
This should take us back down to zero coverity issues.
* Document fmt_addr_impl() and friends.
* Parenthesize macro arguments.
* Rename get_first_listener_addrport_for_pt() to
get_first_listener_addrport_string().
* Handle port_cfg_t with no_listen.
* Handle failure of router_get_active_listener_port_by_type().
* Add an XXX to router_get_active_listener_port_by_type().
==
Nick here. I tweaked this patch a little to make it apply cleanly to
master, to extract some common code into a function, and to replace
snprintf with tor_snprintf.
-- nickm
One of our unit tests checks that they behave correctly (giving an
error) when the base is negative. But there isn't a guarantee that
strtol and friends actually handle negative bases correctly.
Found by Coverity Scan; fix for CID 504.
Calling crypto_cipher_free(NULL) is always safe, since (by
convention) all of our xyz_free() functions treat xyz_free(NULL) as
a no-op.
Flagged by coverity scan; fixes CID 508 and 509.
Since 0.2.3.1-alpha, we've supported the Linux extensions to socket(),
open(), socketpair(), and accept() that enable us to create an fd and
make it close-on-exec with a single syscall. This not only saves us a
syscall (big deal), but makes us less vulnerable to race conditions
where we open a socket and then exec before we can make it
close-on-exec.
But these extensions are not supported on all Linuxes: They were added
between 2.6.23 or so and 2.6.28 or so. If you were to build your Tor
against a recent Linux's kernel headers, and then run it with a older
kernel, you would find yourselve unable to open sockets. Ouch!
The solution here is that, when one of these syscalls fails with
EINVAL, we should try again in the portable way. This adds an extra
syscall in the case where we built with new headers and are running
with old ones, but it will at least allow Tor to work.
Fixes bug 5112; bugfix on 0.2.3.1-alpha.
There is a facility (not used now in Tor) to avoid storing the hash
of a given type if it is a fast-to-calculate hash.
There are also a few ancient-openbsd compilation issues fixed here.
The fact that Tor says INLINE while Libevent says inline remains
unaddressed.
in Makefile.am, we used it without quoting it, causing build failure if
your openssl/sed/sha1sum happened to live in a directory with a space in
it (very common on windows)
There was one MS_WINDOWS that remained because it wasn't on a macro
line; a few remaining uses (and the definition!) in configure.in;
and a now-nonsensical stanza of eventdns_tor.h that previously
defined 'WIN32' if it didn't exist.
This commit is completely mechanical; I used this perl script to make it:
#!/usr/bin/perl -w -i.bak -p
if (/^\s*\#/) {
s/MS_WINDOWS/_WIN32/g;
s/\bWIN32\b/_WIN32/g;
}
These were found by looking for tor_snprintf() instances that were
preceeded closely by tor_malloc(), though I probably converted some
more snprintfs as well.
(In every case, make sure that the length variable (if any) is
removed, renamed, or lowered, so that anything else that might have
assumed a longer buffer doesn't exist.)
To solve bug 4779, we want to avoid OpenSSL 1.0.0's counter mode.
But Fedora (and maybe others) lie about the actual OpenSSL version,
so we can't trust the header to tell us if it's safe.
Instead, let's do a run-time test to see whether it's safe, and if
not, use our built-in version.
fermenthor contributed a pretty essential fixup to this patch. Thanks!
MAX_DNS_LABEL_SIZE was only defined for old versions of openssl, which
broke the build. Spotted by xiando. Fixes bug 4413; not in any released
version.
The thing that's limited to 63 bytes is a "label", not a hostname.
Docment input constraints and behavior on bogus inputs.
Generally it's better to check for overflow-like conditions before
than after. In this case, it's not a true overflow, so we're okay,
but let's be consistent.
pedantic less->fewer in the documentation
Fixes bug 4413; bugfix on xxxx.
Hostname components cannot be larger than 63 characters.
This simple check makes certain randlen cannot overflow rand_bytes_len.
Previously we required 1.0.0, but there was a bug in the 1.0.0 counter
mode. Found by Pascal. Fixes bug 4779.
A more elegant solution would be good here if somebody has time to code
one.
If a relay is dormant at startup, it will call init_keys before
crypto_set_tls_dh_prime. This is bad. Let's make it not so bad, because
someday it *will* happen again.
Comments below focus on changes, see diff for added code.
New type tor_addr_port_t holding an IP address and a TCP/UDP port.
New flag in routerinfo_t, ipv6_preferred. This should go in the
node_t instead but not now.
Replace node_get_addr() with
- node_get_prim_addr() for primary address, i.e. IPv4 for now
- node_get_pref_addr() for preferred address, IPv4 or IPv6.
Rename node_get_addr_ipv4h() node_get_prim_addr_ipv4h() for
consistency. The primary address will not allways be an IPv4 address.
Same for node_get_orport() -> node_get_prim_orport().
Rewrite node_is_a_configured_bridge() to take all OR ports into account.
Extend argument list to extend_info_from_node and
extend_info_from_router with a flag indicating if we want to use the
routers primary address or the preferred address. Use the preferred
address in as few situtations as possible for allowing clients to
connect to bridges over IPv6.
Instead of only writing the dynamic DH prime modulus to a file, write
the whole DH parameters set for forward compatibility. At the moment
we only accept '2' as the group generator.
The DH parameters gets stored in base64-ed DER format to the
'dynamic_dh_params' file.
Let's *not* expose more cross-platform-compatibility structures, or
expect code to use them right.
Also, don't fclose() stdout_handle and stdin_handle until we do
tor_process_handle_destroy, or we risk a double-fclose.
This shaves about 7% off our per-cell AES crypto time for me; the
effect for accelerated AES crypto should be even more, since the AES
calculation itself will make an even smaller portion of the
counter-mode performance.
(We don't want to do this for pre-1.0.0 OpenSSL, since our AES_CTR
implementation was actually faster than OpenSSL's there, by about
10%.)
Fixes issue #4526.
This is a fancier bug4457 workaround for 0.2.3. In 0.2.2, we could
just tell Libevent "Don't enable locking!" so it wouldn't try to make
the event_base notifiable. But for IOCP, we need a notifiable base.
(Eventually, we'll want a notifiable base for other stuff, like
multithreaded crypto.) So the solution is to try a full-featured
initialization, and then retry with all the options turned off if that
fails.
Conflicts:
src/common/compat_libevent.c
Resolving conflict by not taking 7363eae13c ("Use the
EVENT_BASE_FLAG_NOLOCK flag to prevent socketpair() invocation"): in
Tor 0.2.3.x, we _do_ sometimes use notifiable event bases.
In Tor 0.2.2, we never need the event base to be notifiable, since we
don't call it from other threads. This is a workaround for bug 4457,
which is not actually a Tor bug IMO.
This thing was pretty pointless on versions of OpenSSL 0.9.8 and later,
and almost totally pointless on OpenSSL 1.0.0.
Also, favor EVP by default, since it lets us get hardware acceleration
where present. (See issue 4442)
The old behavior was susceptible to the compiler optimizing out our
assertion check, *and* could still overflow size_t on 32-bit systems
even when it did work.
- Rename tor_tls_got_server_hello() to tor_tls_got_client_hello().
- Replaced some aggressive asserts with LD_BUG logging.
They were the innocent "I believe I understand how these callbacks
work, and this assert proves it" type of callbacks, and not the "If
this statement is not true, computer is exploding." type of
callbacks.
- Added a changes file.
In a2bb0bf we started using a separate client identity key. When we are
in "public server mode" (that means not a bridge) we will use the same
key. Reusing the key without doing the proper refcounting leads to a
segfault on cleanup during shutdown. Fix that.
Also introduce an assert that triggers if our refcount falls below 0.
That should never happen.
* Make tor_tls_context_new internal to tortls.c, and return the new
tor_tls_context_t from it.
* Add a public tor_tls_context_init wrapper function to replace it.
Conflicts:
src/or/main.c
src/or/router.c
SSL_read(), SSL_write() and SSL_do_handshake() can always progress the
SSL protocol instead of their normal operation, this means that we
must be checking for needless renegotiations after they return.
Introduce tor_tls_got_excess_renegotiations() which makes the
tls->server_handshake_count > 2
check for us, and use it in tor_tls_read() and tor_tls_write().
Cases that should not be handled:
* SSL_do_handshake() is only called by tor_tls_renegotiate() which is a
client-only function.
* The SSL_read() in tor_tls_shutdown() does not need to be handled,
since SSL_shutdown() will be called if SSL_read() returns an error.
From the code:
zlib 1.2.4 and 1.2.5 do some "clever" things with macros. Instead of
saying "(defined(FOO) ? FOO : 0)" they like to say "FOO-0", on the theory
that nobody will care if the compile outputs a no-such-identifier warning.
Sorry, but we like -Werror over here, so I guess we need to define these.
I hope that zlib 1.2.6 doesn't break these too.
Possible fix for bug 1526.
Since we check for naughty renegotiations using
tor_tls_t.server_handshake_count we don't need that semi-broken
function (at least till there is a way to disable rfc5746
renegotiations too).
Switch 'server_handshake_count' from a uint8_t to 2 unsigned int bits.
Since we won't ever be doing more than 3 handshakes, we don't need the
extra space.
Toggle tor_tls_t.got_renegotiate based on the server_handshake_count.
Also assert that when we've done two handshakes as a server (the initial
SSL handshake, and the renegotiation handshake) we've just
renegotiated.
Finally, in tor_tls_read() return an error if we see more than 2
handshakes.
The renegotiation callback was called only when the first Application
Data arrived, instead of when the renegotiation took place.
This happened because SSL_read() returns -1 and sets the error to
SSL_ERROR_WANT_READ when a renegotiation happens instead of reading
data [0].
I also added a commented out aggressive assert that I won't enable yet
because I don't feel I understand SSL_ERROR_WANT_READ enough.
[0]: Look at documentation of SSL_read(), SSL_get_error() and
SSL_CTX_set_mode() (SSL_MODE_AUTO_RETRY section).
Introduce tor_tls_state_changed_callback(), which handles every SSL
state change.
The new function tor_tls_got_server_hello() is called every time we
send a ServerHello during a v2 handshake, and plays the role of the
previous tor_tls_server_info_callback() function.
- Add a tor_process_get_pid() function that returns the PID of a
process_handle_t.
- Conform to make check-spaces.
- Add some more documentation.
- Improve some log messages.
It's too risky to have a function where if you leave one parameter
NULL, it splits up address:port strings, but if you set it, it does
hostname resolution.
Under the new convention, having a tor_addr.*lookup function that
doesn't do hostname resolution is too close for comfort.
I used this script here, and have made no other changes.
s/tor_addr_parse_reverse_lookup_name/tor_addr_parse_PTR_name/g;
s/tor_addr_to_reverse_lookup_name/tor_addr_to_PTR_name/g;
Now let's have "lookup" indicate that there can be a hostname
resolution, and "parse" indicate that there wasn't. Previously, we
had one "lookup" function that did resolution; four "parse" functions,
half of which did resolution; and a "from_str()" function that didn't
do resolution. That's confusing and error-prone!
The code changes in this commit are exactly the result of this perl
script, run under "perl -p -i.bak" :
s/tor_addr_port_parse/tor_addr_port_lookup/g;
s/parse_addr_port(?=[^_])/addr_port_lookup/g;
s/tor_addr_from_str/tor_addr_parse/g;
This patch leaves aton and pton alone: their naming convention and
behavior is is determined by the sockets API.
More renaming may be needed.
Right now we can take the digests only of an RSA key, and only expect to
take the digests of an RSA key. The old tor_cert_get_id_digests() would
return a good set of digests for an RSA key, and an all-zero one for a
non-RSA key. This behavior is too error-prone: it carries the risk that
we will someday check two non-RSA keys for equality and conclude that
they must be equal because they both have the same (zero) "digest".
Instead, let's have tor_cert_get_id_digests() return NULL for keys we
can't handle, and make its callers explicitly test for NULL.
Our keys and x.509 certs are proliferating here. Previously we had:
An ID cert (using the main ID key), self-signed
A link cert (using a shorter-term link key), signed by the ID key
Once proposal 176 and 179 are done, we will also have:
Optionally, a presentation cert (using the link key),
signed by whomever.
An authentication cert (using a shorter-term ID key), signed by
the ID key.
These new keys are managed as part of the tls context infrastructure,
since you want to rotate them under exactly the same circumstances,
and since they need X509 certificates.
GCC 4.2 and maybe other compilers optimize away unsigned integer
overflow checks of the form (foo + bar < foo), for all bar.
Fix one such check in `src/common/OpenBSD_malloc_Linux.c'.
After a stream reached eof, we fclose it, but then
test_util_spawn_background_partial_read() reads from it again, which causes
an error and thus another fclose(). Some platforms are fine with this, others
(e.g. debian-sid-i386) trigger a double-free() error. The actual code used by
Tor (log_from_pipe() and tor_check_port_forwarding()) handle this case
correctly.
Mainly used for testing reading from subprocesses. To be more generic
we now pass in a pointer to a process_handle_t rather than a Windows-
specific HANDLE.
Conventionally in Tor, structs are returned as pointers, so change
tor_spawn_background() to return the process handle in a pointer rather
than as return value.
When we're doing filtering ssl bufferevents, we want the rate-limits
to apply to the lowest level of the bufferevent stack, so that we're
actually limiting bytes sent on the network. Otherwise, we'll read
from the network aggressively, and only limit stuff as we process it.
* Use strcmpstart() instead of strcmp(x,y,strlen(y)).
* Warn the user if the managed proxy failed to launch.
* Improve function documentation.
* Use smartlist_len() instead of n_unconfigured_proxies.
* Split managed_proxy_destroy() to managed_proxy_destroy()
and managed_proxy_destroy_with_transports().
* Constification.
Also remove a few other related warnings that could occur during the ssl
handshake. We do this because the relay operator can't do anything about
them, and they aren't their fault.
We'll still need to tweak it so that it looks for includes and
libraries somewhere more sensible than "where we happened to find
them on Erinn's system"; so that tests and tools get built too;
so that it's a bit documented; and so that we actually try running
the output.
Work done with Erinn Clark.
- pid, stdout/stderr_pipe now encapsulated in process_handle
- read_all replaced by tor_read_all_from_process_stdin/stderr
- waitpid replaced by tor_get_exit_code
Untested on *nix
Previously, if tor_addr_to_str() returned NULL, we would reuse the
last value returned by fmt_addr(). (This could happen if we were
erroneously asked to format an AF_UNSPEC address.) Now instead we
return "???".
This is the meat of proposal 171: we change circuit_is_acceptable()
to require that the connection is compatible with every connection
that has been linked to the circuit; we update circuit_is_better to
prefer attaching streams to circuits in the way that decreases the
circuits' usefulness the least; and we update link_apconn_to_circ()
to do the appropriate bookkeeping.
* Create a function that will get input from a stream, so that we can
communicate with the managed proxy.
* Hackish change to tor_spawn_background() so that we can specify an
environ for our spawn.
- We were reporting the _bottom_ N failing states, not the top N.
- With bufferevents enabled, we logged all TLS states as being "in
bufferevent", which isn't actually informative.
- When we had nothing to report, we reported nothing too loudly.
- Also, we needed documentation.
This code lets us record the state of any outgoing OR connection
that fails before it becomes open, so we can notice if they're all
dying in the same SSL state or the same OR handshake state.
More work is still needed:
- We need documentation
- We need to actually call the code that reports the failure when
we realize that we're having a hard time connecting out or
making circuits.
- We need to periodically clear out all this data -- perhaps,
whenever we build a circuit successfully?
- We'll eventually want to expose it to controllers, perhaps.
Partial implementation of feature 3116.
I'm not one to insist on C's miserly stack limits, but allocating a
256K array on the stack is too much even for me.
Bugfix on 0.2.1.7-alpha. Found by coverity. Fixes CID # 450.
For some inexplicable reason, Coverity departs from its usual
standards of avoiding false positives here, and warns about all
sscanf usage, even when the formatting strings are totally safe.
Addresses CID # 447, 446.
Original message from bug3393:
check_private_dir() to ensure that ControlSocketsGroupWritable is
safe to use. Unfortunately, check_private_dir() only checks against
the currently running user… which can be root until privileges are
dropped to the user and group configured by the User config option.
The attached patch fixes the issue by adding a new effective_user
argument to check_private_dir() and updating the callers. It might
not be the best way to fix the issue, but it did in my tests.
(Code by lunar; changelog by nickm)
George Kadianakis notes that if you give crypto_rand_int() a value
above INT_MAX, it can return a negative number, which is not what
the documentation would imply.
The simple solution is to assert that the input is in [1,INT_MAX+1].
If in the future we need a random-value function that can return
values up to UINT_MAX, we can add one.
Fixes bug 3306; bugfix on 0.2.2pre14.
When we added the check for key size, we required that the keys be
128 bytes. But RSA_size (which defers to BN_num_bytes) will return
128 for keys of length 1017..1024. This patch adds a new
crypto_pk_num_bits() that returns the actual number of significant
bits in the modulus, and uses that to enforce key sizes.
Also, credit the original bug3318 in the changes file.
An elusive compile-error (MingW-gcc v4.50 on Win_XP); a missing
comma (!) and a typo ('err_msg' at line 277 changed to 'errmsg').
Aso changed the format for 'err_code' at line 293 into a "%ld" to suppress
a warning. How did this go unnoticed for ~1 month? Btw. This is my 1st ever
'git commit', so it better work.
The conflicts were mainly caused by the routerinfo->node transition.
Conflicts:
src/or/circuitbuild.c
src/or/command.c
src/or/connection_edge.c
src/or/directory.c
src/or/dirserv.c
src/or/relay.c
src/or/rendservice.c
src/or/routerlist.c
