Commit Graph

19180 Commits

Author SHA1 Message Date
Matt Traudt
ae9e6b547d sched: add comment about how we determine if a socket should write 2017-09-15 11:40:59 -04:00
Matt Traudt
501c58187d sched: add more per-socket limit documentation; int fix 2017-09-15 11:40:59 -04:00
David Goulet
513d642a66 sched: Revert IF_BUG_ONCE() to tor_assert()
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
1033e14a69 sched: Define SCHEDULER_KIST_PRIVATE for more encapsulation
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
c910610701 sched: Improve logging if KIST is disabled
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
0d0b677b64 test: Fix unit tests with latest scheduler changes
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
734dbfa590 sched: Make the scheduler object static
Each type of scheduler implements its own static scheduler_t object and
returns a reference to it.

This commit also makes it a const pointer that is it can only change inside
the scheduler type subsystem but not outside for extra protection.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
bd34a0d30f sched: Compare channel pointer in the socket table
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
e0984b8a51 sched: Make the outbuf table local to run()
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
ba08f8a4ed sched: Make check-spaces happy
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
3d2d8d6841 sched: Extra careful in channel_outbuf_length()
In case we don't have a connection object in the channel, just be careful and
recover.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
14cd8bdd4b sched: Don't expose the global scheduler libevent object
Instead, add wrappers to do the needed action the different scheduler needs
with the libevent object.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
84d18f70f3 sched: Always call on_channel_free() regardless of state
A channel can bounce in the scheduler and bounce out with the IDLE state which
means that if it came in the scheduler once, it has socket information that
needs to be freed from the global hash table.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
7cc9621d11 sched: Add Schedulers torrc option
This option is a list of possible scheduler type tor can use ordered by
priority. Its default value is "KIST,KISTLite,Vanilla" which means that KIST
will be used first and if unavailable will fallback to KISTLite and so on.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
Matt Traudt
6ff8c86ac6 sched: change most asserts to non-fatal BUGs 2017-09-15 11:40:59 -04:00
Matt Traudt
4bc97c6431 sched: revisist compatibility on non-linux systems
Wrap things in HAVE_KIST_SUPPORT until Tor compiles and tests cleanly on
my OS X machine.
2017-09-15 11:40:59 -04:00
Matt Traudt
8ccb2c106a sched: rename scheduler to the_scheduler 2017-09-15 11:40:59 -04:00
Matt Traudt
7f1d643083 sched: validate KIST sched options 2017-09-15 11:40:59 -04:00
Matt Traudt
61fc9c41ad sched: switch to monotonic time; add/fix comments, style, and logs msgs 2017-09-15 11:40:59 -04:00
David Goulet
6e598bbcd8 sched: Add sandbox support for KIST
Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
a06f2a0509 sched: Fallback to naive approach if no runtime KIST support
It is possible that tor was compiled with KIST support but the running kernel
has no support for it. In that case, fallback to a naive approach and flag
that we have no kernel support.

At this commit, if the kernel support is disabled, there are no ways to come
back from it other than restarting tor with a kernel that supporst KIST.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
David Goulet
8424c4f35b sched: Detect KIST support at compile time
Add a detection for the KIST scheduler in our build system and set
HAVE_KIST_SUPPORT if available.

Adapt the should use kist function with this new compile option.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
Matt Traudt
dde358667d sched: Implement the KIST scheduler
Closes #12541

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
Matt Traudt
2034e0d1d4 sched: Groundwork before KIST implementation
- HT_FOREACH_FN defined in an additional place because nickm did that
  in an old kist prototype
- Make channel_more_to_flush mockable for future sched tests
- Add empty scheduler_{vanilla,kist}.c files and put in include.am

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
Matt Traudt
b2c56eacdd sched: Remove vanilla sched options that will be going away
- massive change to src/tgest/test_options.c since the sched options
  were added all over the place in it
- removing the sched options caused some tests to pass/fail in new ways
  so I assumed current behavior is correct and made them pass again
   - ex: "ConnLimit must be greater" lines
   - ex: "Authoritative directory servers must" line
- remove test_options_validate__scheduler in prep for new sched tests

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-15 11:40:59 -04:00
Matt Traudt
91c7bebfa2 consensus: Add a generic notification function on new consensus
Some groundwork for the KIST scheduler implementation.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-07 11:49:38 -04:00
Nick Mathewson
1502bf03fd Add a module comment to util_bug.h
Closes ticket 22824.
2017-09-07 09:31:30 -04:00
Nick Mathewson
1e079ec30d Merge branch 'maint-0.3.1' 2017-09-07 08:51:16 -04:00
Nick Mathewson
9c404602a0 Add a stack trace to the warning at issue with 23105.
With luck, this will help us diagnose 23105 and fix it.

I also added a stack trace to the warning right before it, since why
not.
2017-09-07 08:49:11 -04:00
Nick Mathewson
ae7cb3349e Merge branch 'bug23361_032_01_squashed2' 2017-09-07 08:35:35 -04:00
David Goulet
c527cde82f prop224: Pick rendezvous point of protover HSRend=2
Version 3 hidden service needs rendezvous point that have the protocol version
HSRend >= 2 else the rendezvous cells are rejected.

Fixes #23361

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-09-07 08:35:28 -04:00
Nick Mathewson
ab18e5e5fc Better error on failure to load seccomp2 sandbox
There are two reasons this is likeliest to happen -- no kernel
support, and some bug in Tor.  We'll ask people to check the former
before they report. Closes 23090.
2017-09-06 14:23:47 -04:00
Nick Mathewson
9261f612ad Merge remote-tracking branch 'dgoulet/bug23123_032_01' 2017-09-06 14:17:43 -04:00
Nick Mathewson
342712b9ef Make url-canonicalizer canonicalize correctly. 2017-09-05 15:02:16 -04:00
Nick Mathewson
4e02d02076 Fix an erroneous ! 2017-09-05 14:55:13 -04:00
Nick Mathewson
4ce60dab00 Resolve inconsistencies between buf refactor and HTTP connect 2017-09-05 14:41:35 -04:00
Nick Mathewson
73b0e2e6fd Merge branch 'http_tunnel_squashed' 2017-09-05 14:34:29 -04:00
Nick Mathewson
ad9190404b Add a fuzzer for HTTP CONNECT 2017-09-05 14:34:09 -04:00
Nick Mathewson
53e4272b65 Add stream isolation support for HTTP CONNECT tunnels
I'm doing this using the Proxy-Authorization: header to support
clients that understand it, and with a new tor-specific header that
makes more sense for our use.
2017-09-05 14:34:09 -04:00
Nick Mathewson
4b30ae1581 Add support for HTTP Connect tunnels 2017-09-05 14:34:09 -04:00
Nick Mathewson
eda79c2f78 Export http-command parsing functions. 2017-09-05 14:34:09 -04:00
Nick Mathewson
c0b9f594b6 Make preferred_chunk_size nonstatic, and add a prefix to it 2017-09-05 14:17:18 -04:00
Nick Mathewson
6ec5059723 Refactor buffer APIs to put a buf_t first.
By convention, a function that frobs a foo_t should be called
foo_frob, and it should have a foo_t * as its first argument.  But
for many of the buf_t functions, the buf_t was the final argument,
which is silly.
2017-09-05 13:57:51 -04:00
Nick Mathewson
d61da9e61f Repair wide lines from previous commit. 2017-09-05 13:57:51 -04:00
Nick Mathewson
4a7e90adc5 Repair buffer API so everything starts with buf_.
Our convention is that functions which manipulate a type T should be
named T_foo.  But the buffer functions were super old, and followed
all kinds of conventions.  Now they're uniform.

Here's the perl I used to do this:

\#!/usr/bin/perl -w -i -p

s/read_to_buf\(/buf_read_from_socket\(/;
s/flush_buf\(/buf_flush_to_socket\(/;
s/read_to_buf_tls\(/buf_read_from_tls\(/;
s/flush_buf_tls\(/buf_flush_to_tls\(/;
s/write_to_buf\(/buf_add\(/;
s/write_to_buf_compress\(/buf_add_compress\(/;
s/move_buf_to_buf\(/buf_move_to_buf\(/;
s/peek_from_buf\(/buf_peek\(/;
s/fetch_from_buf\(/buf_get_bytes\(/;
s/fetch_from_buf_line\(/buf_get_line\(/;
s/fetch_from_buf_line\(/buf_get_line\(/;
s/buf_remove_from_front\(/buf_drain\(/;
s/peek_buf_startswith\(/buf_peek_startswith\(/;
s/assert_buf_ok\(/buf_assert_ok\(/;
2017-09-05 13:57:51 -04:00
Nick Mathewson
336aa21e37 Move buffers.c and buffers_tls.c into src/common
These are no longer tor-specific, so they can be part of the
infrastructure.
2017-09-05 13:57:51 -04:00
Nick Mathewson
150089cbd7 Move the tls parts of buffers.c into buffers_tls.c 2017-09-05 13:57:51 -04:00
Nick Mathewson
5921b465e7 Make buffers.c independent of or.h
Also, put ext_or function in new module; it had accidentally gotten
into proto_socks.c
2017-09-05 13:57:51 -04:00
Nick Mathewson
f28e314b0d Make buf_pullup() expose the pulled-up data.
This lets us drop the testing-only function buf_get_first_chunk_data(),
and lets us implement proto_http and proto_socks without looking at
buf_t internals.
2017-09-05 13:57:51 -04:00
Nick Mathewson
cddac959e7 Replace buf->datalen usage in proto_*.c with buf_datalen() call.
This lets us remove BUFFERS_PRIVATE from two of the modules.
2017-09-05 13:57:51 -04:00