(If we don't restrict rename, there's not much point in restricting
open, since an attacker could always use rename to make us open
whatever they want.)
A new set of unit test cases are provided, as well as introducing
an alternative paradigm and macros to support it. Primarily, each test
case is given its own namespace, in order to isolate tests from each
other. We do this by in the usual fashion, by appending module and
submodule names to our symbols. New macros assist by reducing friction
for this and other tasks, like overriding a function in the global
namespace with one in the current namespace, or declaring integer
variables to assist tracking how many times a mock has been called.
A set of tests for a small-scale module has been included in this
commit, in order to highlight how the paradigm can be used. This
suite gives 100% coverage to status.c in test execution.
When we successfully create a usable circuit after it previously
timed out for a certain amount of time, we should make sure that
our public IP address hasn't changed and update our descriptor.
Most of these are simple. The only nontrivial part is that our
pattern for using ENUM_BF was confusing doxygen by making declarations
that didn't look like declarations.
Back in 5e762e6a5c, non-exit servers
stopped launching DNS requests for users. So there's no need for them
to see if their DNS answers are hijacked.
Patch from Matt Pagan. I think this is a 965 fix.
It's increasingly apparent that we want to make sure we initialize our
PRNG nice and early, or else OpenSSL will do it for us. (OpenSSL
doesn't do _too_ bad a job, but it's nice to do it ourselves.)
We'll also need this for making sure we initialize the siphash key
before we do any hashes.
The remaining vestige is that we continue to publish the V2dir flag,
and that, for the controller, we continue to emit v2 directory
formats when requested.
We had accidentially grown two fake ones: one for backtrace.c, and one
for sandbox.c. Let's do this properly instead.
Now, when we configure logs, we keep track of fds that should get told
about bad stuff happening from signal handlers. There's another entry
point for these that avoids using non-signal-handler-safe functions.
On platforms with the backtrace/backtrace_symbols_fd interface, Tor
can now dump stack traces on assertion failure. By default, I log
them to DataDir/stack_dump and to stderr.
We previously used FILENAME_PRIVATE identifiers mostly for
identifiers exposed only to the unit tests... but also for
identifiers exposed to the benchmarker, and sometimes for
identifiers exposed to a similar module, and occasionally for no
really good reason at all.
Now, we use FILENAME_PRIVATE identifiers for identifiers shared by
Tor and the unit tests. They should be defined static when we
aren't building the unit test, and globally visible otherwise. (The
STATIC macro will keep us honest here.)
For identifiers used only by the unit tests and never by Tor at all,
on the other hand, we wrap them in #ifdef TOR_UNIT_TESTS.
This is not the motivating use case for the split test/non-test
build system; it's just a test example to see how it works, and to
take a chance to clean up the code a little.
A new option TestingV3AuthVotingStartOffset is added which offsets the
starting time of the voting interval. This is possible only when
TestingTorNetwork is set.
This patch makes run_scheduled_events() check for new consensus
downloads every second when TestingTorNetwork, instead of every
minute. This should be fine, see #8532 for reasoning.
This patch also brings MIN_VOTE_SECONDS and MIN_DIST_SECONDS down from
20 to 2 seconds, unconditionally. This makes sanity checking of
misconfiguration slightly less sane.
Addresses #8532.
Relays used to check every 10 to 60 seconds, as an accidental side effect
of calling directory_fetches_from_authorities() when considering doing
a directory fetch. The fix for bug 1992 removes that side effect. At the
same time, bridge relays never had the side effect, leading to confused
bridge operators who tried crazy tricks to get their bridges to notice
IP address changes (see ticket 1913).
The new behavior is to reinstate an every-60-seconds check for both
public relays and bridge relays, now that the side effect is gone.
We need a weak RNG in a couple of places where the strong RNG is
both needless and too slow. We had been using the weak RNG from our
platform's libc implementation, but that was problematic (because
many platforms have exceptionally horrible weak RNGs -- like, ones
that only return values between 0 and SHORT_MAX) and because we were
using it in a way that was wrong for LCG-based weak RNGs. (We were
counting on the low bits of the LCG output to be as random as the
high ones, which isn't true.)
This patch adds a separate type for a weak RNG, adds an LCG
implementation for it, and uses that exclusively where we had been
using the platform weak RNG.
This is meant to avoid conflict with the built-in log() function in
math.h. It resolves ticket 7599. First reported by dhill.
This was generated with the following perl script:
#!/usr/bin/perl -w -i -p
s/\blog\(LOG_(ERR|WARN|NOTICE|INFO|DEBUG)\s*,\s*/log_\L$1\(/g;
s/\blog\(/tor_log\(/g;
This is an automatically generated commit, from the following perl script,
run with the options "-w -i -p".
s/smartlist_string_num_isin/smartlist_contains_int_as_string/g;
s/smartlist_string_isin((?:_case)?)/smartlist_contains_string$1/g;
s/smartlist_digest_isin/smartlist_contains_digest/g;
s/smartlist_isin/smartlist_contains/g;
s/digestset_isin/digestset_contains/g;
Add handle_fw_helper_output(), a function responsible for parsing the
output of tor-fw-helper. Refactor tor_check_port_forwarding() and
run_scheduled_events() accordingly too.
We now issue warnings when we get control output from tor-fw-helper,
and we log the verbose output of tor-fw-helper in LOG_INFO.
Conflicts:
src/common/util.c
The SMARTLIST_FOREACH macro is more convenient than BEGIN/END when
you have a nice short loop body, but using it for long bodies makes
your preprocessor tell the compiler that all the code is on the same
line. That causes grief, since compiler warnings and debugger lines
will all refer to that one line.
So, here's a new style rule: SMARTLIST_FOREACH blocks need to be
short.
We now catch bare {s that should be on the previous line with a do,
while, if, or for, and elses that should share a line with their
preceding }.
That is,
if (foo)
{
and
if (foo) {
...
}
else
are now detected.
We should think about maybe making Tor uncrustify-clean some day,
but configuring uncrustify is an exercise in bizarreness, and
reformatting huge gobs of Tor is always painful.
It turns out this can happen. Even though there is no reason for
connections to be marked but reading, we leave them reading anyway,
so warning here is unwarranted. Let's turn that back on once we do
something sensible and disable reading when we mark. Bugfix for
6203 on Tor 0.2.3.17-beta.
Thanks to cypherpunks for pointing out the general stupidity of the
original code here.
The warning message of validate_pluggable_transports_config() is
superseded by the changes in the warning message of
connection_or_connect() when the proxy credentials can't be found.
There is a bug causing busy loops in Libevent and infinite loops in
the Shadow simulator. A connection that is marked for close, wants
to flush, is held open to flush, but is rate limited (the token
bucket is empty) triggers the bug.
This commit fixes the bug. Details are below.
This currently happens on read and write callbacks when the active
socket is marked for close. In this case, Tor doesn't actually try
to complete the read or write (it returns from those methods when
marked), but instead tries to clear the connection with
conn_close_if_marked(). Tor will not close a marked connection that
contains data: it must be flushed first. The bug occurs when this
flush operation on the marked connection can not occur because the
connection is rate-limited (its write token bucket is empty).
The fix is to detect when rate limiting is preventing a marked
connection from properly flushing. In this case, it should be
flagged as read/write_blocked_on_bandwidth and the read/write events
de-registered from Libevent. When the token bucket gets refilled, it
will check the associated read/write_blocked_on_bandwidth flag, and
add the read/write event back to Libevent, which will cause it to
fire. This time, it will be properly flushed and closed.
The reason that both read and write events are both de-registered
when the marked connection can not flush is because both result in
the same behavior. Both read/write events on marked connections will
never again do any actual reads/writes, and are only useful to
trigger the flush and close the connection. By setting the
associated read/write_blocked_on_bandwidth flag, we ensure that the
event will get added back to Libevent, properly flushed, and closed.
Why is this important? Every Shadow event occurs at a discrete time
instant. If Tor does not properly deregister Libevent events that
fire but result in Tor essentially doing nothing, Libevent will
repeatedly fire the event. In Shadow this means infinite loop,
outside of Shadow this means wasted CPU cycles.
Also, try to resolve some doxygen issues. First, define a magic
"This is doxygen!" macro so that we take the correct branch in
various #if/#else/#endifs in order to get the right documentation.
Second, add in a few grouping @{ and @} entries in order to get some
variables and fields to get grouped together.
Previously, we skipped everything that got invoked from
options_init_from_torrc. But some of the stuff in
options_act_reversible and options_act is actually important, like
reopening the logs.
Now, a SIGHUP always makes the effects of an options_set() happen,
even though the options haven't changed.
Fix for bug 5095; bugfix on 0.2.1.9-alpha, which introduced
__ReloadTorrcOnSIGHUP.
This commit is completely mechanical; I used this perl script to make it:
#!/usr/bin/perl -w -i.bak -p
if (/^\s*\#/) {
s/MS_WINDOWS/_WIN32/g;
s/\bWIN32\b/_WIN32/g;
}
Preprocessor directives should not be put inside the arguments
of a macro. This is not supported on older GCC releases (< 3.3)
thus broke compilation on Haiku (running gcc2).
This code handles the new ORPort options, and incidentally makes all
remaining port types use the new port configuration systems.
There are some rough edges! It doesn't do well in the case where your
Address says one thing but you say to Advertise another ORPort. It
doesn't handle AllAddrs. It doesn't actually advertise anything besides
the first listed advertised IPv4 ORPort and DirPort. It doesn't do
port forwarding to them either.
It's not tested either, it needs more documentation, and it probably
forgets to put the milk back in the refrigerator.
Some controllers want this so they can mess with Tor's configuration
for a while via the control port before actually letting Tor out of
the house.
We do this with a new DisableNetwork option, that prevents Tor from
making any outbound connections or binding any non-control
listeners. Additionally, it shuts down the same functionality as
shuts down when we are hibernating, plus the code that launches
directory downloads.
To make sure I didn't miss anything, I added a clause straight to
connection_connect, so that we won't even try to open an outbound
socket when the network is disabled. In my testing, I made this an
assert, but since I probably missed something, I've turned it into a
BUG warning for testing.
We used to do init_keys() if DynamicDHGroups changed after a HUP, so
that the dynamic DH modulus was stored on the disk. Since we are now
doing dynamic DH modulus storing in crypto.c, we can simply initialize
the TLS context and be good with it.
Introduce a new function router_initialize_tls_context() which
initializes the TLS context and use it appropriately.
When running with IOCP, we are in theory able to use userspace-
allocated buffers to avoid filling up the stingy amount of kernel
space allocated for sockets buffers.
The bufferevent_async implementation in Libevent provides this
ability, in theory. (There are likely to be remaining bugs). This
patch adds a new option that, when using IOCP bufferevents, sets
each socket's send and receive buffers to 0, so that we should use
this ability.
When all the bugs are worked out here, if we are right about bug 98,
this might solve or mitigate bug 98.
This option is experimental and will likely require lots of testing
and debugging.
This is used for the bridge authority currently, to get a better
intuition on how many descriptors are actually fetched from it and how
many fetches happen in total.
Implements ticket 4200.
With managed proxies you would always get the error message:
"You have a Bridge line using the X pluggable transport, but there
doesn't seem to be a corresponding ClientTransportPlugin line."
because the check happened directly after parse_client_transport_line()
when managed proxies were not fully configured and their transports
were not registered.
The fix is to move the validation to run_scheduled_events() and make
sure that all managed proxies are configured first.
Right now we only force a new descriptor upload every 18 hours.
This can make servers become unlisted if they upload a descriptor at
time T which the authorities reject as being "too similar" to one
they uploaded before. Nothing will actually make the server upload a
new descriptor later on, until another 18 hours have passed.
This patch changes the upload behavior so that the 18 hour interval
applies only when we're listed in a live consensus with a descriptor
published within the last 18 hours. Otherwise--if we're not listed
in the live consensus, or if we're listed with a publication time
over 18 hours in the past--we upload a new descriptor every 90
minutes.
This is an attempted bugfix for #3327. If we merge it, it should
obsolete #535.
Conflicts:
src/or/connection.c
src/or/connection_edge.c
src/or/connection_edge.h
src/or/dnsserv.c
Some of these were a little tricky, since they touched code that
changed because of the prop171 fixes.
The "nym epoch" of a stream is defined as the number of times that
NEWNYM had been called before the stream was opened. All streams
are isolated by nym epoch.
This feature should be redundant with existing signewnym stuff, but
it provides a good belt-and-suspenders way for us to avoid ever
letting any circuit type bypass signewnym.
* Add some utility transport functions in circuitbuild.[ch] so that we
can use them from pt.c.
* Make the accounting system consider traffic coming from proxies.
* Make sure that we only fetch bridge descriptors when all the
transports are configured.
This lets us make a lot of other stuff const, allows the compiler to
generate (slightly) better code, and will make me get slightly fewer
patches from folks who stick mutable stuff into or_options_t.
const: because not every input is an output!
On win64, sockets are of type UINT_PTR; on win32 they're u_int;
elsewhere they're int. The correct windows way to check a socket for
being set is to compare it with INVALID_SOCKET; elsewhere you see if
it is negative.
On Libevent 2, all callbacks take sockets as evutil_socket_t; we've
been passing them int.
This patch should fix compilation and correctness when built for
64-bit windows. Fixes bug 3270.
When we configure a new bridge via the controller, don't wait up to ten
seconds before trying to fetch its descriptor. This wasn't so bad when
you listed your bridges in torrc, but it's dreadful if you configure
your bridges via vidalia.
If you really want to purge the client DNS cache, the TrackHostExits
mappings, and the virtual address mappings, you should be using NEWNYM
instead.
Fixes bug 1345; bugfix on Tor 0.1.0.1-rc.
Note that this needs more work: now that we aren't nuking the
transient addressmap entries on HUP, we need to make sure that
configuration changes to VirtualAddressMap and TrackHostExits actually
have a reasonable effect.
To make sure that a server learns if its IP has changed, the server
sometimes launches authority.z descriptor fetches from
update_router_descriptor_downloads. That's nice, but we're moving
towards a situation where update_router_descriptor_downloads doesn't
always get called. So this patch breaks the authority.z
check-and-fetch into a new function.
This function also renames last_routerdesc_download to a more
appropriate last_descriptor_download, and adds a new
update_all_descriptor_downloads() function.
(For now, this is unnecessary, since servers don't actually use
microdescriptors. But that could change, or bridges could start
using microdescriptors, and then we'll be glad this is refactored
nicely.)
Previously we ensured that it would get called periodically by doing
it from inside the code that added microdescriptors. That won't work
though: it would interfere with our code that tried to read microdescs
from disk initially. Instead, we should consider rebuilding the cache
periodically, and on startup.
Instead of just saying "boogity boogity!" let's actually warn people
that they need to configure stuff right to be safe, and point them
at instructions for how to do that.
Resolves bug 2474.
This is a tweak to the bug2917 fix. Basically, if we want to simulate
a signal arriving in the controller, we shouldn't have to pretend that
we're Libevent, or depend on how Tor sets up its Libevent callbacks.
While doing so, get rid of the now unnecessary function
control_signal_act().
Fixes bug 2917, reported by Robert Ransom. Bugfix on commit
9b4aa8d2ab. This patch is loosely based on
a patch by Robert (Changelog entry).
We've got millisecond timers now, we might as well use them.
This change won't actually make circuits get expiered with microsecond
precision, since we only call the expiry functions once per second.
Still, it should avoid the situation where we have a circuit get
expired too early because of rounding.
A couple of the expiry functions now call tor_gettimeofday: this
should be cheap since we're only doing it once per second. If it gets
to be called more often, though, we should onsider having the current
time be an argument again.
Name the magic value "10" rather than re-deriving it.
Comment more.
Use the pattern that works for periodic timers, not the pattern that
doesn't work. ;)
Sets:
* Documentation
* Logging domain
* Configuration option
* Scheduled event
* Makefile
It also creates status.c and the log_heartbeat() function.
All code was written by Sebastian Hahn. Commit message was
written by me (George Kadianakis).
This was originally a patch provided by pipe
(http://www.mail-archive.com/or-talk@freehaven.net/msg13085.html) to
provide a method for controllers to query the total amount of traffic
tor has handled (this is a frequently requested piece of information
by relay operators).
Our old code correctly called bufferevent_flush() on linked
connections to make sure that the other side got an EOF event... but
it didn't call bufferevent_flush() when the connection wasn't
hold_open_until_flushed. Directory connections don't use
hold_open_until_flushed, so the linked exit connection never got an
EOF, so they never sent a RELAY_END cell to the client, and the
client never concluded that data had arrived.
The solution is to make the bufferevent_flush() code apply to _all_
closing linked conns whose partner is not already marked for close.
* Make tor_tls_context_new internal to tortls.c, and return the new
tor_tls_context_t from it.
* Add a public tor_tls_context_init wrapper function to replace it.
The node_t type is meant to serve two key functions:
1) Abstracting difference between routerinfo_t and microdesc_t
so that clients can use microdesc_t instead of routerinfo_t.
2) Being a central place to hold mutable state about nodes
formerly held in routerstatus_t and routerinfo_t.
This patch implements a nodelist type that holds a node for every
router that we would consider using.
Specifically, a circ attempt that we'd launched while the network was
down could timeout after we've marked our entrynodes up, marking them
back down again. The fix is to annotate as bad the OR conns that were
around before we did the retry, so if a circuit that's attached to them
times out we don't do anything about it.
This requires the latest Git version of Libevent as of 24 March 2010.
In the future, we'll just say it requires Libevent 2.0.5-alpha or
later.
Since Libevent doesn't yet support hierarchical rate limit groups,
there isn't yet support for tracking relayed-bytes separately when
using the bufferevent system. If a future version does add support
for hierarchical buckets, we can add that back in.
When we introduced the code to close non-open OR connections after
KeepalivePeriod had passed, we replaced some code that said
if (!connection_is_open(conn)) {
/* let it keep handshaking forever */
} else if (do other tests here) {
...
with new code that said
if (!connection_is_open(conn) && past_keepalive) {
/* let it keep handshaking forever */
} else if (do other tests here) {
...
This was a mistake, since it made all the other tests start applying
to non-open connections, thus causing bug 1840, where non-open
connections get closed way early.
Fixes bug 1840. Bugfix on 0.2.1.26 (commit 67b38d50).
With this patch we stop scheduling when we should write statistics using a
single timestamp in run_scheduled_events(). Instead, we remember when a
statistics interval starts separately for each statistic type in geoip.c
and rephist.c. Every time run_scheduled_events() tries to write stats to
disk, it learns when it should schedule the next such attempt.
This patch also enables all statistics to be stopped and restarted at a
later time.
This patch comes with a few refactorings, some of which were not easily
doable without the patch.
The next series of commits begins addressing the issue that we're
currently including the complete or.h file in all of our source files.
To change that, we're splitting function definitions into new header
files (one header file per source file).
These timers behave better with non-monotonic clocks than our old
ones, and also try harder to make once-per-second events get called
one second apart, rather than one-plus-epsilon seconds apart.
This fixes bug 943 for everybody using Libevent 2.0 or later.
Most of the changes here are switches to use APIs available on Windows
CE. The most pervasive change is that Windows CE only provides the
wide-character ("FooW") variants of most of the windows function, and
doesn't support the older ASCII verions at all.
This patch will require use of the wcecompat library to get working
versions of the posix-style fd-based file IO functions.
[commit message by nickm]
