Right now, Tor routers don't save the maxima values from the
bw_history_t between sessions. That's no good, since we use those
values to determine bandwidth. This code adds a new BWHist.*Maximum
set of values to the state file. If they're not present, we estimate
them by taking the observed total bandwidth and dividing it by the
period length, which provides a lower bound.
This should fix bug 1863. I'm calling it a feature.
Previously, our state parsing code would fail to parse a bwhist
correctly if the Interval was anything other than the default
hardcoded 15 minutes. This change makes the parsing less incorrect,
though the resulting history array might get strange values in it if
the intervals don't match the one we're using. (That is, if stuff was
generated in 15 minute intervals, and we read it into an array that
expects 30 minute intervals, we're fine, since values can be combined
pairwise. But if we generate data at 30 minute intervals and read it
into 15 minute intervals, alternating buckets will be empty.)
Bugfix on 0.1.1.11-alpha.
The trick of looping from i=0..4 , switching on i to set up some
variables, then running some common code is much better expressed by
just calling a function 4 times with 4 sets of arguments. This should
make the code a little easier to follow and maintain here.
Our checks that we don't exceed the 50 KB size limit of extra-info
descriptors apparently failed. This patch fixes these checks and reserves
another 250 bytes for appending the signature. Fixes bug 2183.
Mainly, this comes from turning two lists that needed to be kept in
synch into a single list of structs. This should save a little RAM,
and make the code simpler.
There's no reason to keep a time_t and a struct timeval to represent
the same value: highres_created.tv_sec was the same as timestamp_created.
This should save a few bytes per circuit.
Some of these functions only work for routerinfo-based nodes, and as
such are only usable for advisory purposes. Fortunately, our uses
of them are compatible with this limitation.
A node_t is an abstraction over routerstatus_t, routerinfo_t, and
microdesc_t. It should try to present a consistent interface to all
of them. There should be a node_t for a server whenever there is
* A routerinfo_t for it in the routerlist
* A routerstatus_t in the current_consensus.
(note that a microdesc_t alone isn't enough to make a node_t exist,
since microdescriptors aren't usable on their own.)
There are three ways to get a node_t right now: looking it up by ID,
looking it up by nickname, and iterating over the whole list of
microdescriptors.
All (or nearly all) functions that are supposed to return "a router"
-- especially those used in building connections and circuits --
should return a node_t, not a routerinfo_t or a routerstatus_t.
A node_t should hold all the *mutable* flags about a node. This
patch moves the is_foo flags from routerinfo_t into node_t. The
flags in routerstatus_t remain, but they get set from the consensus
and should not change.
Some other highlights of this patch are:
* Looking up routerinfo and routerstatus by nickname is now
unified and based on the "look up a node by nickname" function.
This tries to look only at the values from current consensus,
and not get confused by the routerinfo_t->is_named flag, which
could get set for other weird reasons. This changes the
behavior of how authorities (when acting as clients) deal with
nodes that have been listed by nickname.
* I tried not to artificially increase the size of the diff here
by moving functions around. As a result, some functions that
now operate on nodes are now in the wrong file -- they should
get moved to nodelist.c once this refactoring settles down.
This moving should happen as part of a patch that moves
functions AND NOTHING ELSE.
* Some old code is now left around inside #if 0/1 blocks, and
should get removed once I've verified that I don't want it
sitting around to see how we used to do things.
There are still some unimplemented functions: these are flagged
with "UNIMPLEMENTED_NODELIST()." I'll work on filling in the
implementation here, piece by piece.
I wish this patch could have been smaller, but there did not seem to
be any piece of it that was independent from the rest. Moving flags
forces many functions that once returned routerinfo_t * to return
node_t *, which forces their friends to change, and so on.
With this patch we stop scheduling when we should write statistics using a
single timestamp in run_scheduled_events(). Instead, we remember when a
statistics interval starts separately for each statistic type in geoip.c
and rephist.c. Every time run_scheduled_events() tries to write stats to
disk, it learns when it should schedule the next such attempt.
This patch also enables all statistics to be stopped and restarted at a
later time.
This patch comes with a few refactorings, some of which were not easily
doable without the patch.
The HSAuthorityRecordStats option was used to track statistics of overall
hidden service usage on the version 0 hidden service authorities. With the
version 2 hidden service directories being deployed and version 0
descriptors being phased out, these statistics are not as useful anymore.
Goodbye, you fine piece of software; my first major code contribution to
Tor.
Some *_free functions threw asserts when passed NULL. Now all of them
accept NULL as input and perform no action when called that way.
This gains us consistence for our free functions, and allows some
code simplifications where an explicit null check is no longer necessary.
Do not segfault when writing buffer stats when we haven't observed a
single circuit to report about. This is a minor bug that would only show
up in testing environments with no traffic and with reduced stats
intervals.
- Avoid memmoving 0 bytes which might lead to compiler warnings.
- Don't require relays to be entry node AND bridge at the same to time to
record clients.
- Fix a memory leak when writing dirreq-stats.
- Don't say in the stats files that measurement intervals are twice as long
as they really are.
- Reduce minimum observation time for requests to 12 hours, or we might
never record usage.
- Clear exit stats correctly after writing them, or we accumulate old stats
over time.
- Reset interval start for buffer stats, too.
- Refactor geoip.c by moving duplicate code into rotate_request_period().
- Don't leak memory when cleaning up cell queues.
- Make sure that exit_(streams|bytes_(read|written)) are initialized in all
places accessing these arrays.
- Read only the last block from *stats files and ensure that its timestamp
is not more than 25 hours in the past and not more than 1 hour in the
future.
- Stop truncating the last character when reading *stats files.
The only thing that's left now is to avoid reading whole *stats files into
memory.
Introduce a threshold of 0.01% of bytes that must be read and written per
port in order to be included in the statistics. Otherwise we cannot include
these statistics in extra-info documents, because they are too big.
Change the labels "-written" and "-read" so that the meanings are as
intended.
when we write out our stability info, detect relays that have slipped
through the cracks. log about them and correct the problem.
if we continue to see a lot of these over time, it means there's another
spot where relays fall out of the routerlist without being marked as
unreachable.
The subversion $Id$ fields made every commit force a rebuild of
whatever file got committed. They were not actually useful for
telling the version of Tor files in the wild.
svn:r17867
would stop building circuits and start refusing connections after
24 hours, since we false believed that Tor was dormant. Reported
by nwf; bugfix on 0.1.2.x.
svn:r13583
Fix all but 2 DOCDOC items; defer many XXX020s (particularly those where fixing them would fix no bugs at the risk of introducing some bugs).
svn:r13529
Re-tune mempool parametes based on testing on peacetime: use smaller chuncks, free them a little more aggressively, and try very hard to concentrate allocations on fuller chunks. Also, lots of new documentation.
svn:r13484
Add a bunch more code documentation; change the interface of fetch_var_cell_from_buf() so it takes the current link protocol into account and can't get confused by weird command bytes on v1 connections.
svn:r13430
Add more documentation; change the behavior of read_to_buf_tls to be more consistent. Note a longstanding problem with current read/write interfaces.
svn:r13407
Mess with the formula for the Guard flag again. Now it requires that you be in the most familiar 7/8 of nodes, and have above median wfu for that 7/8th. See spec for details. Also, log thresholds better.
svn:r12440
Make unverified-consensus get removed when it is accepted or rejected. Make a new get_datadir_fname*() set of functions to eliminate the common code of "get the options, get the datadir, append some stuff".
svn:r12000
Save weighted-fractional-uptime to disk as well as MTBF. Bump the version on rouer-stability: downgrading to versions earlier than this one will lose your WFU data.
svn:r11835
Add a bunch of function documentation; clean up a little code; fix some XXXXs; tag the nonsensical EXTRAINFO_PURPOSE_GENERAL as nonsesnse; note another bit of "do not cache special routers" code to nuke.
svn:r11761
Finish implementing and documenting proposal 108: Authorities now use MTBF data to set their stability flags, once they have at least 4 days of data to use.
svn:r11240
Clean up MTBF storage code. Do not count times that we have been down toward the current run. Handle backward timewarps correctly. Store MTBF data on exit in addition to periodically.
svn:r11225
When we are loading state info from disk, never believe any date in the future. Doing so can keep us from retrying guards, rotating onion keys, storing bandwidth info, etc. Fixes bug 434, and others. Backport candidate, once it has been tested.
svn:r11166
Link note_router_reachable and note_router_unreachable to mtbf code. decouple mtbf from connect/disconnect. log it in USR1. do not blow it away on cleanup if we are an authority.
svn:r11151
Minor cleanups on hidden service usage patch from Karsten: tidy documentation; make free_all idempotent (and safe to call even if we have not yet initialized rephist); and stop using "l" as a variable name (it is too easy to confuse with "1").
svn:r10068
Initial version of patch from Karsten Loesing: Add an HSAuthorityRecordStats option to track statistics of overall hidden service usage without logging information that would be useful to an attacker.
svn:r10067
Removing the last DOCDOC comment hurt so much that I had to use Doxygen to identify undocumented macros and comments, and add 150 more DOCDOCs to point out where they were. Oops. Hey, kids! Fixing some of these could be your first Tor patch!
svn:r9477
Round stored/transmitted values for bandwidth usage. This might make some attacks work less well. This might well be voodoo, but it gives me a warm fuzzy feeling.
svn:r9048
Revise logic used to flush state to disk. Now, we try to batch non-urgent changes so that we do not do too many writes, and we save very-non-urgent changes every once in a rare while, and we never save more than once per second.
svn:r9047
Change logging format of state file to only include non-default values. Adjust clients to never store bandwidth history in the state file. (Possible backport candidate.)
svn:r9043
1) Surround all constants by (parens), whether we'll be using them
in a denominator or not.
2) Express all time periods as products (24*60*60), not as multiplied-out
constants (86400).
3) Comments like "(60*60) /* one hour */" are as pointless as comments
like "c = a + b; /* set c to the sum of a and b */". Remove them.
4) All time periods should be #defined constants, not given inline.
5) All time periods should have doxygen comments.
6) All time periods, unless specified, are in seconds. It's not necessary
to say so.
To summarize, the old (lack of) style would allow:
#define FOO_RETRY_INTERVAL 60*60 /* one hour (seconds) */
next_try = now + 3600;
The new style is:
/** How often do we reattempt foo? */
#define FOO_RETRY_INTERVAL (60*60)
next_try = now + RETRY_INTERVAL;
svn:r6142
separately. It's important to keep them separate because internal
circuits have their last hops picked like middle hops, rather than like
exit hops. So exiting on them will break the user's expectations.
- Stop cannibalizing internal circuits for general exits, and stop
cannibalizing exit circuits for rendezvous stuff.
- Don't let new exit streams attach to internal circuits.
- When deciding if we have enough circuits for internal and for exit,
don't count the wrong ones.
- Treat predicted resolves as predicted port 80 exits.
svn:r5457
to the exit policy of the last hop. Intro and rendezvous circs must
be internal circs, to avoid leaking information. Resolve and connect
streams can use internal circs if they want.
New circuit pooling algorithm: make sure to have enough circs around
to satisfy any predicted ports, and also make sure to have 2 internal
circs around if we've required internal circs lately (with high uptime
if we've seen that lately).
Split NewCircuitPeriod config option into NewCircuitPeriod (30 secs),
which describes how often we retry making new circuits if current ones
are dirty, and MaxCircuitDirtiness (10 mins), which describes how long
we're willing to make use of an already-dirty circuit.
Once rendezvous circuits are established, keep using the same circuit as
long as you attach a new stream to it at least every 10 minutes. (So web
browsing doesn't require you to build new rend circs every 30 seconds.)
Cannibalize GENERAL circs to be C_REND, C_INTRO, S_INTRO, and S_REND
circ as necessary, if there are any completed ones lying around when
we try to launch one.
Re-instate the ifdef's to use version-0 style introduce cells, since
there was yet another bug in handling version-1 style. We'll try switching
over again after 0.0.9 is obsolete.
Bugfix: when choosing an exit node for a new non-internal circ, don't take
into account whether it'll be useful for any pending x.onion addresses --
it won't.
Bugfix: we weren't actually publishing the hidden service descriptor when
it became dirty. So we only published it every 20 minutes or so, which
means when you first start your Tor, the hidden service will seem broken.
svn:r3360
Stop treating the uint16_t's as null-terminated strings,
and stop looking at the byte after them to see if it's null,
because sometimes you're not allowed to look there.
svn:r3108
that will handle each such port. (We can extend this to include addresses
if exit policies shift to require that.) Seed us with port 80 so web
browsers won't complain that Tor is "slow to start up".
This was necessary because our old circuit building strategy just involved
counting circuits, and as time went by we would build up a big pile of
circuits that had peculiar exit policies (e.g. only exit to 9001-9100)
which would take up space in the circuit pile but never get used.
Fix router_compare_addr_to_addr_policy: it was not treating a port of *
as always matching, so we were picking reject *:* nodes as exit nodes too.
If you haven't used a clean circuit in an hour, throw it away, just to
be on the safe side.
This means after 6 hours a totally unused Tor client will have no
circuits open.
svn:r3078
