has a full tls record available. perhaps this will make OR outbufs not grow
as huge except in rare cases, thus saving lots of cpu time plus memory.
svn:r4343
on five minutes since last successful write. But if you have bytes queued
already, and they're not getting through, we were adding a new keepalive
every second. This was bad.
svn:r4296
when building testing circs for orport testing, require high-bandwidth
nodes, so fewer circs fail. complain about unreachable orport separately
from unreachable dirport.
svn:r3935
Add 'testing' circuit purpose, for reachability testing.
Notice when our IP changes, and reset stats.
Try to pull down a directory via Tor to see if our DirPort is working.
Try to extend a circuit back to us to see if our ORPort is working.
Only publish a descriptor if they're both reachable.
These mostly work, and I'd better get them in before I cause conflicts.
svn:r3703
Add a new TrackHostExits directive to trigger addressmaps for
certain incoming socks addresses, for sites that break when your exit
keeps changing.
Redo the client-side dns cache so it's just an addressmap too.
svn:r3641
hasn't been connection_added yet. This happens when an exit conn
is in dns_wait and we get a relay end cell for it before it finishes.
We were silently leaking each of these marked conns in 0.0.9.x. Now
we actually free them.
svn:r3470
to the exit policy of the last hop. Intro and rendezvous circs must
be internal circs, to avoid leaking information. Resolve and connect
streams can use internal circs if they want.
New circuit pooling algorithm: make sure to have enough circs around
to satisfy any predicted ports, and also make sure to have 2 internal
circs around if we've required internal circs lately (with high uptime
if we've seen that lately).
Split NewCircuitPeriod config option into NewCircuitPeriod (30 secs),
which describes how often we retry making new circuits if current ones
are dirty, and MaxCircuitDirtiness (10 mins), which describes how long
we're willing to make use of an already-dirty circuit.
Once rendezvous circuits are established, keep using the same circuit as
long as you attach a new stream to it at least every 10 minutes. (So web
browsing doesn't require you to build new rend circs every 30 seconds.)
Cannibalize GENERAL circs to be C_REND, C_INTRO, S_INTRO, and S_REND
circ as necessary, if there are any completed ones lying around when
we try to launch one.
Re-instate the ifdef's to use version-0 style introduce cells, since
there was yet another bug in handling version-1 style. We'll try switching
over again after 0.0.9 is obsolete.
Bugfix: when choosing an exit node for a new non-internal circ, don't take
into account whether it'll be useful for any pending x.onion addresses --
it won't.
Bugfix: we weren't actually publishing the hidden service descriptor when
it became dirty. So we only published it every 20 minutes or so, which
means when you first start your Tor, the hidden service will seem broken.
svn:r3360
poll-but-sometimes-select mess. This will let us use faster async cores
(like epoll, kpoll, and /dev/poll), and hopefully work better on Windows
too.
There are some fairly nasty changes to main.c here; this will almost
certainly break something. But hey, that's what alphas are for.
svn:r3341
tried they were all unreachable, assume we are not connected to
the network.
when an application request comes in during this state, be
optimistic and assume we just reconnected. fetch a new directory
and if it works, begin making circuits.
svn:r3327
make it clearer which warns are bugs,
make the control log event match its specification,
point out a bug in how we deal with failure when renewing the tls context.
svn:r3138
Put the check-if-requested-exitrouter-will-reject-us code in the
circuit_attach loop, so it gets checked periodically and not just
once at the beginning. This is useful in case the routerlist changes,
but also in case the address gets resolved into something that we learn
we'll reject.
svn:r3039
- DirFetchPeriod for fetching full directory,
- StatusFetchPeriod for fetching running-routers,
- DirPostPeriod for posting server descriptor,
- RendPostPeriod for posting hidden service descriptors.
Also make sure the hidden service descriptors are at a random
offset from each other, to hinder linkability.
svn:r2889
Split logic to initiate dirfetch, running-routers fetch, and
descriptor post. arma: There are some XXXs here that raise design
questions which we should solve before the next release.
The biggest problem is this: Right now, the directory is about 50X as
large as running-routers uncompressed, and about 36X as large
compressed. Assuming:
- everybody gets the compressed version of everything,
- everybody gets cached directories from random dirservers and
uncached r-r from authdirservers
- everybody downloads r-r at the same rate they now download dirs,
then using r-r from will *increase* authdirserver directory bandwidth usage
if there are significantly more caches than authdirservers.
I think it's safe to leave this in for now, since there aren't 3x36 caching
dirservers, but we should make everybody with a dirport cache running-routers
soon. But I could be wrong.
svn:r2872
Now we can try setting an option but back out if it fails to parse, or
if it's disallowed (e.g. changing RunAsDaemon from 1 to 0).
Use parse_line_from_str rather than parse_line_from_file.
svn:r2692
gave us a sigpipe, and we logged that we were ignoring it,
causing us to fail to log that, and delete the log entry. Then
when the signal handler exited, we proceeded to delete the log
entry that had already been deleted.
Now we make sure to only log inside our signal handler if we'll
be exit()ing right after.
svn:r2388
- make clients cache directories and use them to seed their router lists
at startup. This means clients have a datadir again.
- Introduce a global_write_bucket. We need to respond better to exhausting
it.
- Remove the last vestiges of LinkPadding and TrafficShaping.
- Configuration infrastructure support for warning on obsolete options.
- Refactor directory header parsing to use smartlist_split_string.
- Respond to content-encoding headers by trying to uncompress as appropriate.
- Reply with a deflated directory when a client asks for "dir.z".
(We could use allow-encodings instead, but allow-encodings isn't
specified in HTTP 1.0.)
svn:r2335
o and/or while avoiding unreliable nodes, depending on goals
o 'fascistfirewall' option to pick dirservers on port 80 and ORs on
port 443.
o if a begin failed due to exit policy, but we believe the IP should have been allowed, switch that router to exitpolicy reject *:* until we get our next directory.
svn:r2231
if we get a sigchld but all our children are gone by the time we get
around to reaping them (i don't think this should ever happen, but it
just did), then we'd loop forever trying to reap them.
svn:r2141
introduce an authdir_mode() macro to match the others.
don't initialize uptime to the number of seconds since 1970.
non-authoritative dirservers don't cache their directory on disk.
make only authdirservers use clique_mode.
only read approved-routers file if you're an authdirserver.
even authdirservers fetch a new directory in do_hup.
retry_all_connections() is now called retry_all_listeners().
router_parse_list_from_string() correctly reports the router number
it's working on.
only call dirserv_add_own_fingerprint() and
dirserv_add_descriptor() on startup if we're an authdirserver.
if AuthDir and !ORPort then fail.
if AuthDir and ClientOnly then fail.
svn:r2061
More docs and (way more!) testing needed.
Done:
- Authdirservers down directories from others.
- Generate and use running-routers lists
- Cache directories; store across reboots.
- Refactor directory parsing a bit; note potential trouble spots.
svn:r1985
bugfix: actually complain if we duplicate mark-for-close a circuit
add more logging for relay ends that claim dns resolve failed, so we can
find out why they're not being retried.
svn:r1798
otherwise they might have a socket -1, and if we leave them around
they'll muck up poll/select.
i think this was the cause of our win32 and os x fakepoll crashes,
and probably would cause other errors down the road.
svn:r1786
bunch of half-initialized services. This should solve half of
weasel's current bug. The crash is the other half.
(arma: should we also call rend_services_init on hup?)
svn:r1578
choose an intro point, connect to it,
choose a rend point, connect to it and establish a cookie,
get an ack from the rendezvous point,
and know when both circs are ready for her.
APConns don't use conn->purpose anymore
don't initiate a renddesc lookup if one is already in progress
also fix a buffer overflow in nickname parsing (only exploitable
by the operator though)
svn:r1471
let you specify the purpose of circuits you launch
bugfix: we used to be declaring the first successful circuit too early -- when the first hop finished. now we're more accurate.
svn:r1421
This allows us to do a directory connection *through* tor just
as if we're doing it as an application.
Make ap_conns tolerate it when the application sends stuff before
The socks handshake is done (it just buffers it).
Tell directory_initiate_command the length of the payload (because
it might include nuls).
Add a directory_has_arrived function to, for example, start building
the rendezvous service descriptor.
svn:r1412
* read all the time (before we would ignore eof sometimes, oops)
* we can handle different urls now
* send back 404 for an un-handled url
* commands initiated by the client can handle payloads now
* introduce conn->purpose to avoid exponential state-space explosion
svn:r1400
this resolves a subtle bug where tor clients were preferentially using
the directory servers, since when they start building circuits they
know only about the directory servers
on the other hand, it now takes longer after startup before there's
a working circuit. so it goes.
svn:r1350
successful/failed connections, successful/failed extends, and
connection uptimes.
It's still not done: more tests are needed, and not everything calls
connection/circuit_mark_for_close properly. This skews the results.
Also, there needs to be a 'testing' mode for non-OP ORs, where they
periodically build circuits just to test whether extends work.
svn:r1313
we were never writing anything when hold_open_until_flushed was set,
since conn_write returns early if marked_for_conn is set.
seems a bit better now.
svn:r1214
who wants to shut down a connection calls connection_mark_for_close instead
of setting marked_for_close to 1. This automatically removes the connection
from the DNS cache if needed, sends a RELAY END cell if appropriate, and can
be changed to do whatever else is needed.
Still to do:
- The same for circuits, maybe.
- Add some kind of hold_connection_open_until_flushed flag, maybe.
- Change stuff that closes connections with return -1 to use mark_for_close,
maybe.
svn:r1145
add some more data to be flushed but never turn POLLOUT on. not sure
how commonly this bug was hit, but it would be a doozy.
Also add some asserts to see if it happens elsewhere.
svn:r1142
directory is the string that dirserv.c and directory.c deal with
routerlist is routerinfo's that are bundled together in routers.c
rename some of the get_routerlist functions to set_routerlist
preparing to break into router.c for stuff the router does,
and routerlist.c for handling routerlist.
svn:r886
Increment failure counts only when circuits close without having been built.
Reset failure counts only on the second, and when circuits are done building.
svn:r847
Move writing of pidfile after daemonizing, and also after setting the [ug]id:
This means that the tor user needs write priviliges to the pidfile location.
It needs it for unlinking the pidfile anyway.
svn:r846
bugfix: keep going when a circ fails in circuit_n_conn_open
(make circuit_enumerate_by_naddr_nport obsolete)
bugfix: make circuit_n_conn_open only look at circ's that start at us
bugfix: only try circuit_n_conn_open if we're an OP. Otherwise we
expect connections to always already be up.
bugfix: when choosing path length, pay attention to whether the directory
says a router is down.
bugfix: when picking good exit, skip routers which are known to be down
(more work needs to be done on this one)
svn:r838
setuid, because after we setuid we don't have the priviledges we
need to setgid anymore, duh. merged switch_user() and
switch_group() into switch_id(), since that code has to be wound
together.
- return -1 from switch_id() if it's not defined to do anything else.
- moved daemoinize(), write_pidfile(), and switch_id() from main.c to
util.c
svn:r656
setuid and setgid respectively, and die if it can't.
(If the User option is set, tor will setgid to the user's gid as well.)
This happens after the pidfile is created, so that in cases where tor
needs to be root to work with the pidfile, it will at least be able to
create it, although it won't be able to delete it. That sucks, but
it's somewhat better than not being able to create the pidfile in the
first place.
svn:r652
If DebugLogFile is specified, log to it at -l debug
If LogFile is specified, log to it at the -l from the commandline
(default info)
If no LogFile *and* not a Daemon, then log to stdout.
Make conn->s = -1 by default (this might break things)
When kill -USR1, prefer to log at INFO, but make sure they always see it.
svn:r596
Improve debugging output on fingerprint checking.
Make sure to add our own fingerprint to the fingerprint list _before_
adding our own descriptor, or else we'll reject ourself.
Don't call a directory invalid just because we have a newer descriptor
for some router.
Use router_get_dir_hash to generate hashes for signed directories.
Make sure we add our own descriptor successfully.
Don't fall-through on failed base64-endode.
svn:r514
fix a variety of seg faults
don't try to list OPs in running-routers
write cached-directory to disk when rebuilding the dir
on boot, dirservers load approved-routers file
on boot, dirservers load cached directory file
svn:r508
ERR is if something fatal just happened
WARNING is something bad happened, but we're still running. The bad thing
is either a bug in the code, an attack or buggy protocol/implementation
of the remote peer, etc. The operator should examine the bad thing and
try to correct it.
(No error or warning messages should be expected. I expect most people
to run on -l warning eventually.)
NOTICE is never ever used.
INFO means something happened (maybe bad, maybe ok), but there's nothing
you need to (or can) do about it.
DEBUG is for everything louder than INFO.
svn:r486
redo all the config files for the new format (we'll redo them again soon)
fix (another! yuck) segfault in log_fn when input is too large
tor_tls_context_new() returns -1 for error, not NULL
fix segfault in check_conn_marked() on conn's that die during tls handshake
make ORs also initialize conn from router when we're the receiving node
make non-dirserver ORs upload descriptor to every dirserver on startup
add our local address to the descriptor
add Content-Length field to POST command
revert the Content-Length search in fetch_from_buf_http() to previous code
fix segfault in memmove in fetch_from_buf_http()
raise maximum allowed headers/body size in directory.c
svn:r484
'buf_t' is now an opaque type defined in buffers.c .
Router descriptors now include all keys; routers generate keys as
needed on startup (in a newly defined "data directory"), and generate
their own descriptors. Descriptors are now self-signed.
Implementation is not complete: descriptors are never published; and
upon receiving a descriptor, the directory doesn't do anything with
it.
At least "routers.or" and orkeygen are now obsolete, BTW.
svn:r483
Fixed up the assert_*_ok funcs some (more work remains)
Changed config so it reads either /etc/torrc or the -f arg, never both
Finally tracked down a nasty bug with our use of tls:
It turns out that if you ask SSL_read() for no more than n bytes, it
will read the entire record from the network (and maybe part of the next
record, I'm not sure), give you n bytes of it, and keep the remaining
bytes internally. This is fine, except our poll-for-read looks at the
network, and there are no bytes pending on the network, so we never know
to ask SSL_read() for more bytes. Currently I've hacked it so if we ask
for n bytes and it returns n bytes, then it reads again right then. This
will interact poorly with our rate limiting; we need a cleaner solution.
svn:r481
deal with content-length headers better when reading http
don't assume struct socks4_info is a packed struct
fail the socks handshake if destip is zero
flesh out conn_state_to_string() for dir conn
fix typo (bug) in connection_handle_read()
directory get is now called fetch, post is now upload
reopen logs on sighup
svn:r475
this paves the way for supporting socks5 and other handshakes
it also removes those pesky AP-only variables from connection_t
also hacked a fix for a bug where some streams weren't ending properly --
maybe because marked connections weren't flushing properly?
svn:r472
your client exits if you're running a version not in the
directory's list of acceptable versions (unless you have a
config variable set to override).
svn:r408
(expiry time set to 100 seconds so we can play with it)
exit connections are now informed when pending resolves fail
we kill off the oldest busy worker when we're under attack and need to
resolve something new
svn:r356
i've eliminated the master dns process, so now the workers just
act like regular connections and are handled by the normal pollarray.
everything seems to still work. ;)
svn:r327
Or at least, directories get generated, signed, download, and checked, with
nobody seeming to crash.
In config/*, added 'signing-key' blocks to dirservers and routers.or, so
that everyone will know about the directories' signing keys.
In or/directory.c, refrained from using a dirserver's signing key when
no such key is known; added more debugging output.
In or/main.c, added debugging output and fixed a few logic errors.
In or/routers.c, added debugging output and prevented a segfault on
routers_resolve_directory. The interleaving of arrays and lists on
routerinfo_t is still messy, but at least it seems to work again.
svn:r278
on startup, it forks off a master dns handler, which forks off dns
slaves (like the apache model). slaves as spawned as load increases,
and then reused. excess slaves are not ever killed, currently.
implemented topics. each topic has a receive window in each direction
at each edge of the circuit, and sends sendme's at the data level, as
per before. each circuit also has receive windows in each direction at
each hop; an edge sends a circuit-level sendme as soon as enough data
cells have arrived (regardless of whether the data cells were flushed
to the exit conns). removed the 'connected' cell type, since it's now
a topic command within data cells.
at the edge of the circuit, there can be multiple connections associated
with a single circuit. you find them via the linked list conn->next_topic.
currently each new ap connection starts its own circuit, so we ought
to see comparable performance to what we had before. but that's only
because i haven't written the code to reattach to old circuits. please
try to break it as-is, and then i'll make it reuse the same circuit and
we'll try to break that.
svn:r152
we also queue data cells destined for a circuit that is
pending, and process them once the circuit opens
destroys reach into the queue and remove the pending onion,
along with its collected data cells
svn:r142
now tor can be run safely inside nat'ed areas that kill idle
connections; and the proxy can handle when you suspend your laptop
and then emerge hours later from a new domain.
svn:r125
prkey is only fetched when it's needed
tor nodes who aren't dirservers now fetch directories and autoconnect
to new nodes listed in the directory
default role is a non-dirserver node
svn:r120
revamped the router reading section
reference counting for crypto pk env's (so we can dup them)
we now read and write pem pk keys from string rather than from FILE*,
in anticipation of fetching directories over a socket
(so now on startup we slurp in the whole file, then parse it as a string)
fixed a bug in the proxy side, where you could get some circuits
wedged if they showed up while the connection was being made
svn:r110
httpap is obsolete; we support privoxy directly now!
smtpap is obsolete; need to find a good socks4a-enabled smtp proxy/client
I dub thee 0.0.1.
svn:r107
I modified new_route so we don't pick twins back-to-back in the path.
I also had to patch my previous uses of connection_twin_get_by_addr_port()
because they assumed that "addr" and "port" would be the same for a twin
as for the original router.
svn:r56
Servers are allowed to send 100 cells initially, and can't send more until
they receive a 'sendme' cell from that direction, indicating that they
can send 10 more cells. As it currently stands, the exit node quickly
runs out of window, and sends bursts of 10 whenever a sendme cell gets
to him. This is much much much faster (and more flexible) than the old
"give each circuit 1 kB/s and hope nothing overflows" approach.
Also divided out the connection_watch_events into stop_reading,
start_writing, etc. That way we can control them separately.
svn:r54
we're now much more robust when bandwidth varies: instead of forcing a
fixed bandwidth on the link, we instead use what the link will give us,
up to our bandwidth.
svn:r53
Each socket reads at most 'bandwidth' bytes per second sustained, but
can handle bursts of up to 10*bandwidth bytes.
Cells are now sent out at evenly-spaced intervals, with padding sent
out otherwise. Set Linkpadding=0 in the rc file to send cells as soon
as they're available (and to never send padding cells).
Added license/copyrights statements at the top of most files.
router->min and router->max have been merged into a single 'bandwidth'
value. We should make the routerinfo_t reflect this (want to do that,
Mat?)
As the bandwidth increases, and we want to stop sleeping more and more
frequently to send a single cell, cpu usage goes up. At 128kB/s we're
pretty much calling poll with a timeout of 1ms or even 0ms. The current
code takes a timeout of 0-9ms and makes it 10ms. prepare_for_poll()
handles everything that should have happened in the past, so as long as
our buffers don't get too full in that 10ms, we're ok.
Speaking of too full, if you run three servers at 100kB/s with -l debug,
it spends too much time printing debugging messages to be able to keep
up with the cells. The outbuf ultimately fills up and it kills that
connection. If you run with -l err, it works fine up through 500kB/s and
probably beyond. Down the road we'll want to teach it to recognize when
an outbuf is getting full, and back off.
svn:r50
basically, a twin is a router which is different except it shares
the same keypair. so in cases where we want to find a "next router"
and all we really care is that it can decrypt the next onion layer,
then a twin is just as good.
we still need to decide how to mark twins in the routerinfo_t and in
the routers config file.
svn:r30
The 'or' process can now be told (by the global_role variable) what
roles this server should play -- connect to all ORs, listen for ORs,
listen for OPs, listen for APs, or any combination.
* everything in /src/op/ is now obsolete.
* connection_ap.c now handles all interactions with application proxies
* "port" is now or_port, op_port, ap_port. But routers are still always
referenced (say, in conn_get_by_addr_port()) by addr / or_port. We
should make routers.c actually read these new ports (currently I've
kludged it so op_port = or_port+10, ap_port=or_port+20)
* circuits currently know if they're at the beginning of the path because
circ->cpath is set. They use this instead for crypts (both ways),
if it's set.
* I still obey the "send a 0 back to the AP when you're ready" protocol,
but I think we should phase it out. I can simply not read from the AP
socket until I'm ready.
I need to do a lot of cleanup work here, but the code appears to work, so
now's a good time for a checkin.
svn:r22
