I added this so I could write a unit test for ServerTransportOptions,
but it incidentally exercises the succeed-on-defaults case of
options_validate too.
Better tests for upper bounds, and for failing cases.
Also, change the function's interface to take a buffer length rather
than a maximum length, and then NUL-terminate: functions that don't
NUL-terminate are trouble waiting to happen.
The only thing that used format_helper_exit_status on win32 was the
unit tests. This caused an error when we tried to leave a static
format_helper_exit_status lying around in a production object file.
The easiest solution is to admit that this way of dealing with process
exit status is Unix-only.
If you pass the --enable-coverage flag on the command line, we build
our testing binaries with appropriate options eo enable coverage
testing. We also build a "tor-cov" binary that has coverage enabled,
for integration tests.
On recent OSX versions, test coverage only works with clang, not gcc.
So we warn about that.
Also add a contrib/coverage script to actually run gcov with the
appropriate options to generate useful .gcov files. (Thanks to
automake, the .o files will not have the names that gcov expects to
find.)
Also, remove generated gcda and gcno files on clean.
We previously used FILENAME_PRIVATE identifiers mostly for
identifiers exposed only to the unit tests... but also for
identifiers exposed to the benchmarker, and sometimes for
identifiers exposed to a similar module, and occasionally for no
really good reason at all.
Now, we use FILENAME_PRIVATE identifiers for identifiers shared by
Tor and the unit tests. They should be defined static when we
aren't building the unit test, and globally visible otherwise. (The
STATIC macro will keep us honest here.)
For identifiers used only by the unit tests and never by Tor at all,
on the other hand, we wrap them in #ifdef TOR_UNIT_TESTS.
This is not the motivating use case for the split test/non-test
build system; it's just a test example to see how it works, and to
take a chance to clean up the code a little.
This is mainly a matter of automake trickery: we build each static
library in two versions now: one with the TOR_UNIT_TESTS macro
defined, and one without. When TOR_UNIT_TESTS is defined, we can
enable mocking and expose more functions. When it's not defined, we
can lock the binary down more.
The alternatives would be to have alternate build modes: a "testing
configuration" for building the libraries with test support, and a
"production configuration" for building them without. I don't favor
that approach, since I think it would mean more people runnning
binaries build for testing, or more people not running unit tests.
Is it possible that *every* attempt to replace the microdesc cache on
windows 7 is going to fail because of our lack of FILE_SHARE_DELETE
while opening the file? If so, this test will catch #2077 and let us
know when it's fixed.
This is a fix for bug 8844, where eugenis correctly notes that there's
a sentinel value at the end of the list-of-freelists that's never
actually checked. It's a bug since the first version of the chunked
buffer code back in 0.2.0.16-alpha.
This would probably be a crash bug if it ever happens, but nobody's
ever reported something like this, so I'm unsure whether it can occur.
It would require write_to_buf, write_to_buf_zlib, read_to_buf, or
read_to_buf_tls to get an input size of more than 32K. Still, it's a
good idea to fix this kind of thing!
It sure is a good thing we can run each test in its own process, or
else the amount of setup I needed to do to make this thing work
would have broken all the other tests.
Test mocking would have made this easier to write too.
If geoip_format_bridge_stats() returned NULL when it should have
returned a string, we would have tried to deref NULL, and died. Not
a big deal in the unit tests, but still worth fixing.
Found by coverity; This is CID 743384.
- Document the key=value format.
- Constify equal_sign_pos.
- Pass some strings that are about to be logged to escape().
- Update documentation and fix some bugs in tor_escape_str_for_socks_arg().
- Use string_is_key_value() in parse_bridge_line().
- Parenthesize a forgotten #define
- Add some more comments.
- Add some more unit test cases.
Right now, all our curve25519 backends ignore the high bit of the
public key. But possibly, others could treat the high bit of the
public key as encoding out-of-bounds values, or as something to be
preserved. This could be used to distinguish clients with different
backends, at the cost of killing a circuit.
As a workaround, let's just clear the high bit of each public key
indiscriminately before we use it. Fix for bug 8121, reported by
rransom. Bugfix on 0.2.4.8-alpha.
When we implemented #5823 and removed v2 directory request info, we
never actually changed the unit tests not to expect it.
Fixes bug 8084; bug not in any released version of Tor.
The implementation is pretty straightforward: parse_extended_hostname() is
modified to drop any leading components from an address like
'foo.aaaaaaaaaaaaaaaa.onion'.
This is an automatically generated commit, from the following perl script,
run with the options "-w -i -p".
s/smartlist_string_num_isin/smartlist_contains_int_as_string/g;
s/smartlist_string_isin((?:_case)?)/smartlist_contains_string$1/g;
s/smartlist_digest_isin/smartlist_contains_digest/g;
s/smartlist_isin/smartlist_contains/g;
s/digestset_isin/digestset_contains/g;
As elsewhere, it makes sense when adding or extending a cell type to
actually make the code to parse it into a separate tested function.
This commit doesn't actually make anything use these new functions;
that's for a later commit.
I'm going to want a generic "onionskin" type and set of wrappers, and
for that, it will be helpful to isolate the different circuit creation
handshakes. Now the original handshake is in onion_tap.[ch], the
CREATE_FAST handshake is in onion_fast.[ch], and onion.[ch] now
handles the onion queue.
This commit does nothing but move code and adjust header files.
Previously, we only used the strong OS entropy source as part of
seeding OpenSSL's RNG. But with curve25519, we'll have occasion to
want to generate some keys using extremely-good entopy, as well as the
means to do so. So let's!
This patch refactors the OS-entropy wrapper into its own
crypto_strongest_rand() function, and makes our new
curve25519_secret_key_generate function try it as appropriate.
The ntor handshake--described in proposal 216 and in a paper by
Goldberg, Stebila, and Ustaoglu--gets us much better performance than
our current approach.
We want to use donna-c64 when we have a GCC with support for
64x64->uint128_t multiplying. If not, we want to use libnacl if we
can, unless it's giving us the unsafe "ref" implementation. And if
that isn't going to work, we'd like to use the
portable-and-safe-but-slow 32-bit "donna" implementation.
We might need more library searching for the correct libnacl,
especially once the next libnacl release is out -- it's likely to have
bunches of better curve25519 implementations.
I also define a set of curve25519 wrapper functions, though it really
shouldn't be necessary.
We should eventually make the -donna*.c files get build with
-fomit-frame-pointer, since that can make a difference.
With an IPv6 virtual address map, we can basically hand out a new
IPv6 address for _every_ address we connect to. That'll be cool, and
will let us maybe get around prop205 issues.
This uses some fancy logic to try to make the code paths in the ipv4
and the ipv6 case as close as possible, and moves to randomly
generated addresses so we don't need to maintain those stupid counters
that will collide if Tor restarts but apps don't.
Also has some XXXX items to fix to make this useful. More design
needed.
(This is part 2 of making DNS cache use enabled/disabled on a
per-client port basis. This implements the CacheIPv[46]DNS options,
but not the UseCachedIPv[46] ones.)
This is a customizable extract-and-expand HMAC-KDF for deriving keys.
It derives from RFC5869, which derives its rationale from Krawczyk,
H., "Cryptographic Extraction and Key Derivation: The HKDF Scheme",
Proceedings of CRYPTO 2010, 2010, <http://eprint.iacr.org/2010/264>.
I'm also renaming the existing KDF, now that Tor has two of them.
This is the key derivation scheme specified in ntor.
There are also unit tests.
IPv4-only exits have an implicit "reject [::]/0", which was making
policy_is_reject_star() return 1 for them, making us refuse to do
hostname lookups.
This fix chanes policy_is_reject_star() to ask about which family we meant.
This makes it so we can handle getting an IPv6 in the 3 different
formats we specified it for in RESOLVED cells,
END_STREAM_REASON_EXITPOLICY cells, and CONNECTED cells.
We don't cache IPv6 addresses yet, since proposal 205 isn't
implemented.
There's a refactored function for parsing connected cells; it has unit
tests.
Now, "accept *:80" means "accept all addresses on port 80", and not
just IPv4. For just v4, say "accept *4:80"; for just v6 say "accept
*6:80".
We can parse these policies from torrc just fine, and we should be
successfully keeping them out of descriptors for now.
We also now include appropriate IPv6 addresses in "reject private:*"
By default, "*" means "All IPv4 addresses" with
tor_addr_parse_mask_ports, so I won't break anything. But if the new
EXTENDED_STAR flag is provided, then * means "any address", *4 means
"any IPv4 address" (that is, 0.0.0.0/0), and "*6" means "any IPv6
address" (that is, [::]/0).
This is going to let us have a syntax for specifying exit policies in
torrc that won't drive people mad.
Also, add a bunch of unit tests for tor_addr_parse_mask_ports to test
these new features, and to increase coverage.
This is part of what's needed to build without warnings on mingw64:
it was warning about the cast from void* to long that happened in
the places we were using test_{n,}eq on pointers.
The alternative here would have been to broaden tt_int_op to accept
a long long or an intptr_t, but that's less correct (since pointers
aren't integers), and would hurt the portability of tinytest a
little.
Fixes part of 7260.
Our implementation of parse_short_policy was screwed up: it would
ignore the last character of every short policy. Obviously, that's
broken.
This patch fixes the busted behavior, and adds a bunch of unit tests
to make sure the rest of that function is okay.
Fixes bug 7192; fix on 0.2.3.1-alpha.
In C, we technically aren't supposed to define our own things that
start with an underscore.
This is a purely machine-generated commit. First, I ran this script
on all the headers in src/{common,or,test,tools/*}/*.h :
==============================
use strict;
my %macros = ();
my %skipped = ();
FILE: for my $fn (@ARGV) {
my $f = $fn;
if ($fn !~ /^\.\//) {
$f = "./$fn";
}
$skipped{$fn} = 0;
open(F, $fn);
while (<F>) {
if (/^#ifndef ([A-Za-z0-9_]+)/) {
$macros{$fn} = $1;
next FILE;
}
}
}
print "#!/usr/bin/perl -w -i -p\n\n";
for my $fn (@ARGV) {
if (! exists $macros{$fn}) {
print "# No macro known for $fn!\n" if (!$skipped{$fn});
next;
}
if ($macros{$fn} !~ /_H_?$/) {
print "# Weird macro for $fn...\n";
}
my $goodmacro = uc $fn;
$goodmacro =~ s#.*/##;
$goodmacro =~ s#[\/\-\.]#_#g;
print "s/(?<![A-Za-z0-9_])$macros{$fn}(?![A-Za-z0-9_])/TOR_${goodmacro}/g;\n"
}
==============================
It produced the following output, which I then re-ran on those same files:
==============================
s/(?<![A-Za-z0-9_])_TOR_ADDRESS_H(?![A-Za-z0-9_])/TOR_ADDRESS_H/g;
s/(?<![A-Za-z0-9_])_TOR_AES_H(?![A-Za-z0-9_])/TOR_AES_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_H(?![A-Za-z0-9_])/TOR_COMPAT_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_LIBEVENT_H(?![A-Za-z0-9_])/TOR_COMPAT_LIBEVENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTAINER_H(?![A-Za-z0-9_])/TOR_CONTAINER_H/g;
s/(?<![A-Za-z0-9_])_TOR_CRYPTO_H(?![A-Za-z0-9_])/TOR_CRYPTO_H/g;
s/(?<![A-Za-z0-9_])TOR_DI_OPS_H(?![A-Za-z0-9_])/TOR_DI_OPS_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMAREA_H(?![A-Za-z0-9_])/TOR_MEMAREA_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMPOOL_H(?![A-Za-z0-9_])/TOR_MEMPOOL_H/g;
s/(?<![A-Za-z0-9_])TOR_PROCMON_H(?![A-Za-z0-9_])/TOR_PROCMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORGZIP_H(?![A-Za-z0-9_])/TOR_TORGZIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORINT_H(?![A-Za-z0-9_])/TOR_TORINT_H/g;
s/(?<![A-Za-z0-9_])_TOR_LOG_H(?![A-Za-z0-9_])/TOR_TORLOG_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORTLS_H(?![A-Za-z0-9_])/TOR_TORTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_UTIL_H(?![A-Za-z0-9_])/TOR_UTIL_H/g;
s/(?<![A-Za-z0-9_])_TOR_BUFFERS_H(?![A-Za-z0-9_])/TOR_BUFFERS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_H(?![A-Za-z0-9_])/TOR_CHANNEL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_TLS_H(?![A-Za-z0-9_])/TOR_CHANNELTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITBUILD_H(?![A-Za-z0-9_])/TOR_CIRCUITBUILD_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITLIST_H(?![A-Za-z0-9_])/TOR_CIRCUITLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_EWMA_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_EWMA_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITUSE_H(?![A-Za-z0-9_])/TOR_CIRCUITUSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMMAND_H(?![A-Za-z0-9_])/TOR_COMMAND_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONFIG_H(?![A-Za-z0-9_])/TOR_CONFIG_H/g;
s/(?<![A-Za-z0-9_])TOR_CONFPARSE_H(?![A-Za-z0-9_])/TOR_CONFPARSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_EDGE_H(?![A-Za-z0-9_])/TOR_CONNECTION_EDGE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_H(?![A-Za-z0-9_])/TOR_CONNECTION_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_OR_H(?![A-Za-z0-9_])/TOR_CONNECTION_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTROL_H(?![A-Za-z0-9_])/TOR_CONTROL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CPUWORKER_H(?![A-Za-z0-9_])/TOR_CPUWORKER_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRECTORY_H(?![A-Za-z0-9_])/TOR_DIRECTORY_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRSERV_H(?![A-Za-z0-9_])/TOR_DIRSERV_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRVOTE_H(?![A-Za-z0-9_])/TOR_DIRVOTE_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNS_H(?![A-Za-z0-9_])/TOR_DNS_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNSSERV_H(?![A-Za-z0-9_])/TOR_DNSSERV_H/g;
s/(?<![A-Za-z0-9_])TOR_EVENTDNS_TOR_H(?![A-Za-z0-9_])/TOR_EVENTDNS_TOR_H/g;
s/(?<![A-Za-z0-9_])_TOR_GEOIP_H(?![A-Za-z0-9_])/TOR_GEOIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_HIBERNATE_H(?![A-Za-z0-9_])/TOR_HIBERNATE_H/g;
s/(?<![A-Za-z0-9_])_TOR_MAIN_H(?![A-Za-z0-9_])/TOR_MAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_MICRODESC_H(?![A-Za-z0-9_])/TOR_MICRODESC_H/g;
s/(?<![A-Za-z0-9_])_TOR_NETWORKSTATUS_H(?![A-Za-z0-9_])/TOR_NETWORKSTATUS_H/g;
s/(?<![A-Za-z0-9_])_TOR_NODELIST_H(?![A-Za-z0-9_])/TOR_NODELIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_NTMAIN_H(?![A-Za-z0-9_])/TOR_NTMAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_ONION_H(?![A-Za-z0-9_])/TOR_ONION_H/g;
s/(?<![A-Za-z0-9_])_TOR_OR_H(?![A-Za-z0-9_])/TOR_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_POLICIES_H(?![A-Za-z0-9_])/TOR_POLICIES_H/g;
s/(?<![A-Za-z0-9_])_TOR_REASONS_H(?![A-Za-z0-9_])/TOR_REASONS_H/g;
s/(?<![A-Za-z0-9_])_TOR_RELAY_H(?![A-Za-z0-9_])/TOR_RELAY_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCLIENT_H(?![A-Za-z0-9_])/TOR_RENDCLIENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCOMMON_H(?![A-Za-z0-9_])/TOR_RENDCOMMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDMID_H(?![A-Za-z0-9_])/TOR_RENDMID_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDSERVICE_H(?![A-Za-z0-9_])/TOR_RENDSERVICE_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPHIST_H(?![A-Za-z0-9_])/TOR_REPHIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPLAYCACHE_H(?![A-Za-z0-9_])/TOR_REPLAYCACHE_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTER_H(?![A-Za-z0-9_])/TOR_ROUTER_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERLIST_H(?![A-Za-z0-9_])/TOR_ROUTERLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERPARSE_H(?![A-Za-z0-9_])/TOR_ROUTERPARSE_H/g;
s/(?<![A-Za-z0-9_])TOR_ROUTERSET_H(?![A-Za-z0-9_])/TOR_ROUTERSET_H/g;
s/(?<![A-Za-z0-9_])TOR_STATEFILE_H(?![A-Za-z0-9_])/TOR_STATEFILE_H/g;
s/(?<![A-Za-z0-9_])_TOR_STATUS_H(?![A-Za-z0-9_])/TOR_STATUS_H/g;
s/(?<![A-Za-z0-9_])TOR_TRANSPORTS_H(?![A-Za-z0-9_])/TOR_TRANSPORTS_H/g;
s/(?<![A-Za-z0-9_])_TOR_TEST_H(?![A-Za-z0-9_])/TOR_TEST_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_NATPMP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_NATPMP_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_UPNP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_UPNP_H/g;
==============================
The rationale for treating these files differently is that we should
be checking upstream for changes as applicable, and merging changes
upstream as warranted.
add read_file_to_str_until_eof which is used by read_file_to_str
if the file happens to be a FIFO.
change file_status() to return FN_FILE if st_mode matches S_IFIFO
(on not-windows) so that init_key_from_file() will read from a FIFO.
This would be undefined behavior if it happened. (It can't actually
happen as we're using round_to_power_of_2, since we would have to
be trying to allocate exabytes of data.)
While we're at it, fix the behavior of round_to_power_of_2(0),
and document the function better.
Fix for bug 6831.
Define new new consensus method 14 adding "a" lines to vote and
consensus documents.
From proposal 186:
As with other data in the vote derived from the descriptor, the
consensus will include whichever set of "a" lines are given by the
most authorities who voted for the descriptor digest that will be
used for the router.
This patch implements this.
This gives us a few benefits:
1) make -j clean all
this will start working, as it should. It currently doesn't.
2) increased parallel build
recursive make will max out at number of files in a directory,
non-recursive make doesn't have such a limitation
3) Removal of duplicate information in make files,
less error prone
I've also slightly updated how we call AM_INIT_AUTOMAKE, as the way
that was used was not only deprecated but will be *removed* in the next
major automake release (1.13).... so probably best that we can continue
to bulid tor without requiring old automake.
(see http://www.gnu.org/software/automake/manual/html_node/Public-Macros.html )
For more reasons why, see resources such as:
http://miller.emu.id.au/pmiller/books/rmch/
The SMARTLIST_FOREACH macro is more convenient than BEGIN/END when
you have a nice short loop body, but using it for long bodies makes
your preprocessor tell the compiler that all the code is on the same
line. That causes grief, since compiler warnings and debugger lines
will all refer to that one line.
So, here's a new style rule: SMARTLIST_FOREACH blocks need to be
short.
Because the string output was no longer equal in length to
HEX_ERRNO_SIZE, the write() call would add some extra spaces and
maybe a NUL, and the NUL would trigger an assert in
get_string_from_pipe.
Fixes bug 6225; bug not in any released version of Tor.
These include:
- Having a weird in_addr that can't be initialized with {0}
- Needing INVALID_HANDLE_VALUE instead of -1 for file handles.
- Having a weird dependent definition for struct stat.
- pid is signed, not unsigned.
This fixes a warning in efb8a09f, where Debain Lenny's GCC doesn't get
that
for (i=0; i<3; ++i) {
const char *p;
switch(i) {
case 0:
p="X"; break;
case 1:
p="Y"; break;
case 2:
p="Z"; break;
}
printf("%s\n", p);
}
will never try to print an uninitialezed value.
Found by buildbots. Bug in no released versions of Tor.
Conflicts:
src/test/test_util.c
Merge the unit tests; I added some when I did this branch against
0.2.2, and then the test format changed and master added more tests.
