Commit Graph

30868 Commits

Author SHA1 Message Date
Alexander Færøy
f983a60a6c Copy (zlib1|libssp-0).dll to \src\test\ to run test-process.exe.
This patch ensures that AppVeyor copies over libssp-0.dll and zlib1.dll
to src/test/ to make sure we can run text-process.exe from our slow
tests.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
6e508e9eb4 Fix tests on kqueue() based platforms.
This patch disables fork()'ing of the slow process tests. This fixes the
tests on the MacOS and other kqueue() based platforms.

Without this patch the main loop exits eearly with EBADF as error.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ccc1963890 Move remaining code from subprocess.{h,c} to more appropriate places.
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.

We also delete the now empty subprocess module files.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
f7d13425fc Delete old process_handle_t code.
This patch removes the old process_handle_t code. Everything should by
now be using the process_t interface.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
289ed0849d Add tests for process environment functionality of process_t.
This patch adds tests for the process_environment_t interaction in
process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
9b6a10a26f Add slow test for process_t for main loop interaction.
This patch adds test cases for process_t which uses Tor's main loop.
This allows us to test that the callbacks are actually invoked by the
main loop when we expect them.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e3ceaebba2 Add support for logging messages from pluggable transports.
This patch adds support for the "LOG" protocol message from a pluggable
transport. This allows pluggable transport developers to relay log
messages from their binary to Tor, which will both emit them as log
messages from the Tor process itself, but also pass them on via the
control port.

See: https://bugs.torproject.org/28180
See: https://bugs.torproject.org/28181
See: https://bugs.torproject.org/28182
2018-12-17 16:39:28 -05:00
Alexander Færøy
bfb94dd2ca Use process_t for managed proxies.
This patch makes the managed proxy subsystem use the process_t data
structure such that we can get events from the PT process while Tor is
running and not just when the PT process is being configured.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ad4cc89c5d Add "PT" log domain.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
b0d268a822 Add process_reset_environment() to the Process subsystem.
This patch adds a new function that allows us to reset the environment
of a given process_t with a list of key/value pairs.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
4f611a1df7 Add process_terminate().
This patch adds support for process termination to the Process
subsystem.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
338137221c Make sure we call process_notify_event_exit() as the last thing in different callbacks.
This patch makes sure that we call process_notify_event_exit() after we
have done any modifications we need to do to the state of a process_t.
This allows application developers to call process_free() in the
exit_callback of the process.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e982fb1dae Add documentation for the is_socket and error argument of read_to_chunk().
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
89393a77e5 Add process_get_pid() to the Process subsystem.
This patch adds support for getting the unique process identifier from a
given process_t. This patch implements both support for both the Unix
and Microsoft Windows backend.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
bb784cf4f3 Add Windows backend for the Process subsystem.
This patch adds support for Microsoft Windows in the Process subsystem.

Libevent does not support mixing different types of handles (sockets,
named pipes, etc.) on Windows in its core event loop code. This have
historically meant that Tor have avoided attaching any non-networking
handles to the event loop. This patch uses a slightly different approach
to roughly support the same features for the Process subsystem as we do
with the Unix backend.

In this patch we use Windows Extended I/O functions (ReadFileEx() and
WriteFileEx()) which executes asynchronously in the background and
executes a completion routine when the scheduled read or write operation
have completed. This is much different from the Unix backend where the
operating system signals to us whenever a file descriptor is "ready" to
either being read from or written to.

To make the Windows operating system execute the completion routines of
ReadFileEx() and WriteFileEx() we must get the Tor process into what
Microsoft calls an "alertable" state. To do this we execute SleepEx()
with a zero millisecond sleep time from a main loop timer that ticks
once a second.  This moves the process into the "alertable" state and
when we return from the zero millisecond timeout all the outstanding I/O
completion routines will be called and we can schedule the next reads
and writes.

The timer loop is also responsible for detecting whether our child
processes have terminated since the last timer tick.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
2e957027e2 Add Unix backend for the Process subsystem.
This patch adds the Unix backend for the Process subsystem. The Unix
backend attaches file descriptors from the child process's standard in,
out and error to Tor's libevent based main loop using traditional Unix
pipes. We use the already available `waitpid` module to get events
whenever the child process terminates.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
35509978dd Add new Process subsystem.
This patch adds a new Process subsystem for running external programs in
the background of Tor. The design is focused around a new type named
`process_t` which have an API that allows the developer to easily write
code that interacts with the given child process. These interactions
includes:

- Easy API for writing output to the child process's standard input
  handle.
- Receive callbacks whenever the child has output on either its standard
  output or standard error handles.
- Receive callback when the child process terminates.

We also support two different "protocols" for handling output from the
child process. The default protocol is the "line" protocol where the
process output callbacks will be invoked only when there is complete
lines (either "\r\n" or "\n" terminated). We also support the "raw"
protocol where the read callbacks will get whatever the operating system
delivered to us in a single read operation.

This patch does not include any operating system backends, but the Unix
and Windows backends will be included in separate commits.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
2b41b857bd Add LD_PROCESS as log domain.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
31b3a6577c Add buf_flush_to_pipe() and buf_read_from_pipe().
This patch adds two new functions: buf_flush_to_pipe() and
buf_read_from_pipe(), which makes use of our new buf_flush_to_fd() and
buf_read_from_fd() functions.

See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
771930b84c Refactor buf_read_from_socket() into buf_read_from_fd().
This patch refactors buf_read_from_socket() into buf_read_from_fd(), and
creates a specialized function for buf_read_from_socket(), which uses
buf_read_from_fd().

See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
c71f9df07b Refactor buf_flush_to_socket() into buf_flush_to_fd().
This patch refactors buf_flush_to_socket() into buf_flush_to_fd() and
creates a specialization function for buf_flush_to_socket() that makes
use of buf_flush_to_fd().

See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
340260281a Refactor flush_chunk() to work on pipes as well as sockets.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
5f26ae833e Refactor read_to_chunk() such that it supports both pipes and sockets.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
2a3eef4404 Remove unused int pid member of managed_proxy_t.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Nick Mathewson
9c2b114b2e Merge branch 'maint-0.3.5' 2018-11-27 12:03:50 -05:00
Nick Mathewson
72c1303cef Merge branch 'maint-0.3.4' into maint-0.3.5 2018-11-27 12:03:50 -05:00
Alexander Færøy
36c3235534 Explicitly specify path to OpenSSL for Appveyor.
This patch explicitly specifies the path to our OpenSSL dependency and
disables the installation of an external OpenSSL version and instead
uses the OpenSSL version available from the MinGW environments.

See: https://bugs.torproject.org/28574
2018-11-27 14:53:50 +01:00
Nick Mathewson
8a15d0f69b Merge branch 'maint-0.3.5' 2018-11-26 17:25:28 -05:00
Nick Mathewson
feb41b7c30 Merge remote-tracking branch 'teor/bug28096-035-squashed' into maint-0.3.5 2018-11-26 17:24:41 -05:00
Nick Mathewson
c292e505ff Merge remote-tracking branch 'tor-github/pr/539' 2018-11-26 17:22:37 -05:00
Nick Mathewson
fc1ad9ab65 Merge remote-tracking branch 'tor-github/pr/495' 2018-11-26 17:17:40 -05:00
Nick Mathewson
e12fdeb181 Changes file for "Dormant Mode" (28335, 2149). 2018-11-26 16:39:44 -05:00
Nick Mathewson
7d8e0cc9ab Merge branch 'dormant_v2_squashed' 2018-11-26 16:33:31 -05:00
Nick Mathewson
02843c4a4e Test for check_network_participation_callback() 2018-11-26 16:32:40 -05:00
Nick Mathewson
55512ef022 Test netstatus.c tracking of user participation status 2018-11-26 16:32:40 -05:00
Nick Mathewson
3743f79695 Add options to control dormant-client feature.
The DormantClientTimeout option controls how long Tor will wait before
going dormant.  It also provides a way to disable the feature by setting
DormantClientTimeout to e.g. "50 years".

The DormantTimeoutDisabledByIdleStreams option controls whether open but
inactive streams count as "client activity".  To implement it, I had to
make it so that reading or writing on a client stream *always* counts as
activity.

Closes ticket 28429.
2018-11-26 16:32:40 -05:00
David Goulet
2b9a907bdc Merge branch 'maint-0.3.5' 2018-11-26 14:59:24 -05:00
David Goulet
056ffdec4c Merge remote-tracking branch 'dgoulet/ticket28128_035_01' into maint-0.3.5 2018-11-26 14:59:10 -05:00
Taylor R Campbell
997a8b0ca7 Create a temporary directory for tor's DataDirectory in test_rebind.
Fixes #28562.

While here, put the argument count test and usage message _before_ we
attempt to read from sys.argv.
2018-11-23 12:51:39 -05:00
teor
3741f9e524
Fix a comment typo in test_hs_common.c 2018-11-22 16:56:06 +10:00
Nick Mathewson
469f47ef8d Fix a fun heisenbug in memoize_protover_flags()
After we clear the protover map for getting full, we need to
re-create it, since we are about to use it.

This is a bugfix for bug 28558. It is a bugfix for the code from
ticket 27225, which is not in any released Tor.  Found by Google
OSS-Fuzz, as issue 11475.
2018-11-21 07:56:33 -05:00
Nick Mathewson
b406fdae4f Merge branch 'ticket25341_merged_v2' 2018-11-20 13:14:56 -05:00
Nick Mathewson
a9820f072b Merge branch 'maint-0.3.4' into maint-0.3.5 2018-11-20 13:10:38 -05:00
Nick Mathewson
56f24889a2 Merge branch 'maint-0.3.5' 2018-11-20 13:10:38 -05:00
Nick Mathewson
9da8cd0bc3 Merge remote-tracking branch 'teor/ticket28459-034' into maint-0.3.4 2018-11-20 13:10:18 -05:00
Nick Mathewson
34cadefe34 Merge branch 'maint-0.3.5' 2018-11-20 09:04:35 -05:00
Taylor Yu
0489288aa2 Update control_free_all() for #27169
Reset the added bootstrap tracking state introduced by ticket 27169.
Fixes bug 28524; bugfix on 0.3.5.1-alpha.
2018-11-19 15:48:08 -06:00
Nick Mathewson
8183640ada Merge branch 'maint-0.3.5' 2018-11-19 10:18:57 -05:00
Alexander Færøy
3260914db0 Add missing library to build tor-print-ed-signing-cert.
To succesful compile tor-print-ed-signing-cert.exe on Windows we
sometimes need to include the @TOR_LIB_GDI@ library.

See: https://bugs.torproject.org/28485
2018-11-19 10:18:44 -05:00
Nick Mathewson
a4f2075170 Merge remote-tracking branch 'isis/bug25341' into ticket25341_merged_v2 2018-11-19 09:22:30 -05:00