Commit Graph

512 Commits

Author SHA1 Message Date
Nick Mathewson
0981ac4c59 Log more messages when failing to decode RSA keys
We log these messages at INFO level, except when we are reading a
private key from a file, in which case we log at WARN.

This fixes a regression from when we re-wrote our PEM code to be
generic between nss and openssl.

Fixes bug 29042, bugfix on 0.3.5.1-alpha.
2019-01-16 14:35:30 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
efe55b8898 Bump copyright date to 2019. 2019-01-16 12:32:32 -05:00
Nick Mathewson
b169c8c14f Merge remote-tracking branch 'asn-github/adaptive_padding-final' 2019-01-14 14:48:00 -05:00
Nick Mathewson
99947c3ce0 Merge remote-tracking branch 'teor/bug29015' 2019-01-14 13:59:07 -05:00
Kris Katterjohn
df1a8a657e Actually close the stdout pipe on error in process_unix_exec
When cleaning up after an error in process_unix_exec, the stdin
pipe was being double closed instead of closing both the stdin
and stdout pipes.  This occurred in two places.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-11 18:55:09 -05:00
Nick Mathewson
5dd926caa6 Include the beginnings of a FAQ about which timer to use. 2019-01-11 09:08:51 -05:00
Taylor R Campbell
d82a8a7f9d Add some more type checking.
NOTE: This commit breaks the build, because there was a mistake in an
earlier change of exactly the sort that this is meant to detect!  I'm
leaving it broken for illustration.
2019-01-11 14:41:28 +02:00
Taylor R Campbell
0f8253bddb Use the distribution abstraction as an abstraction. 2019-01-11 14:40:50 +02:00
Taylor R Campbell
531df9590d Move ceil call back into the geometric sampler.
Test exactly what the geometric sampler returns, because that's what
the downstream callers of it are going to use.

While here, also assert that the geometric sampler returns a positive
integer.  (Our geometric distribution is the one suported on {1, 2,
3, ...} that returns the number of trials before the first success,
not the one supported on {0, 1, 2, ...} that returns the number of
failures before the first success.)
2019-01-11 14:40:50 +02:00
George Kadianakis
e0e0338dc4 Rename crypto_rand_uint32() -> crypto_rand_u32()
See https://github.com/torproject/tor/pull/624#discussion_r246453777
2019-01-10 13:06:08 +02:00
Nick Mathewson
0ed966fde1 Merge branch 'maint-0.3.5' 2019-01-09 09:38:27 -05:00
Nick Mathewson
694e3c57fd Merge remote-tracking branch 'tor-github/pr/636' into maint-0.3.5 2019-01-09 09:38:22 -05:00
Nick Mathewson
e6a4401200 Merge branch 'maint-0.3.5' 2019-01-09 08:52:10 -05:00
Kris Katterjohn
c13a81f7f9 Fix (and make consistent) the use of OpenBSD preprocessor macro tests
Prior to this commit, the testsuite was failing on OpenBSD.  After
this commit the testsuite runs fine on OpenBSD.

It was previously decided to test for the OpenBSD macro (rather than
__OpenBSD__, etc.) because OpenBSD forks seem to have the former
macro defined.  sys/param.h must be included for the OpenBSD macro
definition; however, many files tested for the OpenBSD macro without
having this header included.

This commit includes sys/param.h in the files where the OpenBSD macro
is used (and sys/param.h is not already included), and it also
changes some instances of the __OpenBSD__ macro to OpenBSD.

See commit 27df23abb6 which changed
everything to use OpenBSD instead of __OpenBSD__ or OPENBSD.  See
also tickets #6982 and #20980 (the latter ticket is where it was
decided to use the OpenBSD macro).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-09 08:51:57 -05:00
Nick Mathewson
f9a7701ff1 Merge branch 'ticket28856_v2' 2019-01-09 08:46:40 -05:00
teor
671c34d9b4
lib/net: improve the docs for tor_{ersatz_,}socketpair()
Add some details about IP family support, and point to
tor_socketpair() from tor_ersatz_socketpair().

Closes ticket 29015.
2019-01-08 14:45:17 +10:00
Kris Katterjohn
c59b572e93 Fix tor_ersatz_socketpair on IPv6-only systems
In get_local_listener used by tor_ersatz_socketpair, the address
family used when binding the IPv6 socket was AF_INET instead of
AF_INET6.

Fixes bug 28995; bugfix on 0.3.5.1-alpha.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-07 17:28:21 -06:00
Nick Mathewson
4e4f93d364 Add a #ifdef HAVE_UNISTD_H check to buffers.c
Reported on tor-dev by Gisle Vanem.  Bug not in any released Tor

(The suggested patch used _MSC_VER, but that's not how we do stuff
with autoconf.  With autoconf, you detect the feature you want,
rather than trying to list all the systems that do or do not have
it.)
2019-01-03 09:52:19 -05:00
Nick Mathewson
9ba690e33f Merge branch 'maint-0.3.5' 2019-01-03 09:45:56 -05:00
Nick Mathewson
abdc6aede2 Merge branch 'ticket28851_035_squashed' into maint-0.3.5 2019-01-03 09:45:53 -05:00
Nick Mathewson
b82717b273 Stop re-checking our hardcoded dh parameters on every startup
Closes ticket 28851.
2019-01-03 09:45:43 -05:00
Nick Mathewson
47176eb678 Merge branch 'maint-0.3.5' 2019-01-03 09:44:15 -05:00
Nick Mathewson
98736cf36a Merge remote-tracking branch 'public/ticket28838_035' into maint-0.3.5 2019-01-03 09:44:10 -05:00
Nick Mathewson
27853938a1 Merge branch 'maint-0.3.5' 2019-01-03 09:02:40 -05:00
Nick Mathewson
ed62f0fa15 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-01-03 09:02:39 -05:00
Nick Mathewson
3e7f13a4ef Merge branch 'maint-0.3.5' 2019-01-02 16:15:39 -05:00
Nick Mathewson
b33bcb1e63 Add an errno.h include to freespace.c to fix bug 28974. 2019-01-02 15:19:52 -05:00
George Kadianakis
926fc93be5 Concentrate all TOR_USEC_PER_SEC definitions in a single header file.
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
George Kadianakis
2ccf326837 Implement and test probability distributions used by WTF-PAD.
This project introduces the prob_distr.c subsystem which implements all the
probability distributions that WTF-PAD needs. It also adds unittests for all of
them.

Code and tests courtesy of Riastradh.

Co-authored-by: Taylor R Campbell <campbell+tor@mumble.net>
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
George Kadianakis
a336d816a6 Circuit padding tests.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:25:55 +02:00
Mike Perry
3ba7581129 Provide a smartlist reverse-order traversal.
We need this for padding negotiation so that we can have later machine
revisions supercede earlier ones.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:09:13 +02:00
Nick Mathewson
70dd6d07bb Merge branch 'orconn-tracker_squashed' 2018-12-21 14:22:11 -05:00
Nick Mathewson
fd58e5e498 Fix priority on process subsystem level: it uses "net" 2018-12-21 14:12:20 -05:00
Nick Mathewson
ab4395d082 Merge branch 'ticket28847' 2018-12-21 13:26:47 -05:00
Alexander Færøy
2322b56389 Fix typo in time_sys.h. 2018-12-21 13:26:38 -05:00
Alexander Færøy
cf4b3dbd44 Use the subsystem list to initialize and shutdown process module.
This patch makes the process module use the subsystem list for
initializing and shutting down.

See: https://bugs.torproject.org/28847
2018-12-21 13:26:38 -05:00
Taylor Yu
b0f974633a Add LD_BTRACK log domain for bootstrap tracker
Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Nick Mathewson
5c85ba3077 Merge remote-tracking branch 'tor-github/pr/608' 2018-12-20 11:42:26 -05:00
Alexander Færøy
f7e175db57 Forward declare smartlist_t in process.h
This allows other libraries to include process.h without including
the smartlist_t headers first.

See: https://bugs.torproject.org/28847
2018-12-20 14:36:04 +01:00
Alexander Færøy
7762088967 No need to log ordinary EOF conditions as LOG_WARN.
Let's not use log_warn() when a pipe is closed under what should be
considered normal conditions.

See: https://bugs.torproject.org/28179
2018-12-20 13:12:53 +01:00
Alexander Færøy
412fbe9f17 Make example CancelIoEx() code use CancelIo().
This patch changes the CancelIoEx() example code to use CancelIo(),
which is available for older versions of Windows too. I still think the
kernel handles this nicely by sending broken pipes if either side
closes the pipe while I/O operations are pending.

See: https://bugs.torproject.org/28179
2018-12-20 13:11:24 +01:00
Alexander Færøy
f58e597d42 Handle ERROR_BROKEN_PIPE in completion routines.
Handle `ERROR_BROKEN_PIPE` from ReadFileEx() and WriteFileEx() in
process_win32_stdin_write_done() and
process_win32_handle_read_completion() instead of in the early handler.
This most importantmly makes sure that `reached_eof` is set to true when
these errors appears.

See: https://bugs.torproject.org/28179
2018-12-20 13:04:49 +01:00
Alexander Færøy
36e24782f8 Remember to set reached_eof when our handles are reporting errors.
This patch adds some missing calls to set `reached_eof` of our handles
when various error conditions happens or when we close our handle (which
happens at `process_terminate()`.

See: https://bugs.torproject.org/28179
2018-12-20 13:02:22 +01:00
Alexander Færøy
c6e041e3d8 Handle errors even after success from ReadFileEx() and WriteFileEx().
This patch adds some additional error checking after calls to
ReadFileEx() and WriteFileEx(). I have not managed to get this code to
reach the branch where `error_code` is NOT `ERROR_SUCCESS`, but MSDN
says one should check for this condition so we do so just to be safe.

See: https://bugs.torproject.org/28179
2018-12-20 12:57:20 +01:00
Alexander Færøy
44586a89ef Delay checking process for termination until both stdout and stderr are closed.
This patch makes us delay checking for whether we have an exit code
value (via GetExitCodeProcess()) until both stdout and stderr have been
closed by the operating system either by the process itself or by
process cleanup after termination.

See: https://bugs.torproject.org/28179
2018-12-20 12:53:28 +01:00
Alexander Færøy
1d8dcb416c Remember to close the child process' ends of the pipes.
This prevents us from leaking the HANDLE for stdout, stderr, and stdin.

See: https://bugs.torproject.org/28179
2018-12-20 12:47:04 +01:00
Nick Mathewson
bb091da1e7 Merge branch 'ticket28839_v2_squashed' 2018-12-18 18:59:05 -05:00
Alexander Færøy
ca7a2ecc51 Avoid breaking the event loop prematurely.
This patch makes sure that we terminate the event loop from the event
loop timer instead of directly in the process' exit handler. This allows
us to run the event loop an additional time to ensure that the SleepEx()
call on Windows is called and the data from stdout/stderr is delivered
to us.

Additionally we ensure that we don't try to read or write data from a
Unix process that have been terminated in the main loop, since its file
descriptors are closed at that time.

See: https://bugs.torproject.org/28179
2018-12-18 13:35:29 -05:00
Nick Mathewson
8a01f0eaab lib/process may include lib/buf. 2018-12-17 17:58:49 -05:00
Nick Mathewson
4ad59bfbc2 Update location of buffers.h 2018-12-17 17:01:50 -05:00
Nick Mathewson
e969d9c6b4 Merge branch 'ticket28179_squashed' into ticket28179_squashed_merged 2018-12-17 16:41:01 -05:00
Alexander Færøy
c8b8b15f0e Ensure that line_size >= 1 before trying to trim input string.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
fab22509d7 Make Windows process event timer API available for dormant interface.
This patch changes the API of the Windows backend of the Process
subsystem to allow the dormant interface to disable the Process event
timer.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
a33a77d9cd Document the format of process_t::arguments.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
0d796cce17 Use errno directly if we are not reading/writing from/to a socket.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
cacdd29087 Use const char * instead of char * for line parameter for process callbacks.
This patch changes the type definition of the process callbacks to use
`const char *` instead of `char *`.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
5585cbd08f Change the Process exit_callback to return bool.
This patch changes our process_t's exit_callback to return a boolean
value.  If the returned value is true, the process subsystem will call
process_free() on the given process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
22cb3c6ce9 Call close() on stdin/stdout/stderr in process_terminate().
Call close() on all process handles after we have called kill(pid,
SIGTERM).

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ccc1963890 Move remaining code from subprocess.{h,c} to more appropriate places.
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.

We also delete the now empty subprocess module files.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
f7d13425fc Delete old process_handle_t code.
This patch removes the old process_handle_t code. Everything should by
now be using the process_t interface.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ad4cc89c5d Add "PT" log domain.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
b0d268a822 Add process_reset_environment() to the Process subsystem.
This patch adds a new function that allows us to reset the environment
of a given process_t with a list of key/value pairs.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
4f611a1df7 Add process_terminate().
This patch adds support for process termination to the Process
subsystem.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
338137221c Make sure we call process_notify_event_exit() as the last thing in different callbacks.
This patch makes sure that we call process_notify_event_exit() after we
have done any modifications we need to do to the state of a process_t.
This allows application developers to call process_free() in the
exit_callback of the process.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e982fb1dae Add documentation for the is_socket and error argument of read_to_chunk().
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
89393a77e5 Add process_get_pid() to the Process subsystem.
This patch adds support for getting the unique process identifier from a
given process_t. This patch implements both support for both the Unix
and Microsoft Windows backend.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
bb784cf4f3 Add Windows backend for the Process subsystem.
This patch adds support for Microsoft Windows in the Process subsystem.

Libevent does not support mixing different types of handles (sockets,
named pipes, etc.) on Windows in its core event loop code. This have
historically meant that Tor have avoided attaching any non-networking
handles to the event loop. This patch uses a slightly different approach
to roughly support the same features for the Process subsystem as we do
with the Unix backend.

In this patch we use Windows Extended I/O functions (ReadFileEx() and
WriteFileEx()) which executes asynchronously in the background and
executes a completion routine when the scheduled read or write operation
have completed. This is much different from the Unix backend where the
operating system signals to us whenever a file descriptor is "ready" to
either being read from or written to.

To make the Windows operating system execute the completion routines of
ReadFileEx() and WriteFileEx() we must get the Tor process into what
Microsoft calls an "alertable" state. To do this we execute SleepEx()
with a zero millisecond sleep time from a main loop timer that ticks
once a second.  This moves the process into the "alertable" state and
when we return from the zero millisecond timeout all the outstanding I/O
completion routines will be called and we can schedule the next reads
and writes.

The timer loop is also responsible for detecting whether our child
processes have terminated since the last timer tick.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
2e957027e2 Add Unix backend for the Process subsystem.
This patch adds the Unix backend for the Process subsystem. The Unix
backend attaches file descriptors from the child process's standard in,
out and error to Tor's libevent based main loop using traditional Unix
pipes. We use the already available `waitpid` module to get events
whenever the child process terminates.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
35509978dd Add new Process subsystem.
This patch adds a new Process subsystem for running external programs in
the background of Tor. The design is focused around a new type named
`process_t` which have an API that allows the developer to easily write
code that interacts with the given child process. These interactions
includes:

- Easy API for writing output to the child process's standard input
  handle.
- Receive callbacks whenever the child has output on either its standard
  output or standard error handles.
- Receive callback when the child process terminates.

We also support two different "protocols" for handling output from the
child process. The default protocol is the "line" protocol where the
process output callbacks will be invoked only when there is complete
lines (either "\r\n" or "\n" terminated). We also support the "raw"
protocol where the read callbacks will get whatever the operating system
delivered to us in a single read operation.

This patch does not include any operating system backends, but the Unix
and Windows backends will be included in separate commits.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Nick Mathewson
29254812a3 Remove strcmp_len(): it is now unused
(See 28856.)
2018-12-17 09:04:25 -05:00
Nick Mathewson
3c35c0d441 Add a function to provide an upper bound on base64 decoded length 2018-12-14 13:51:51 -05:00
Nick Mathewson
041e9235c1 Lower the loop_max constant in curve25519_basepoint_spot_check()
The point of this function is to make sure that the ed25519-based
implementation of curve25519_basepoint() actually works when we
start tor, and use the regular fallback implementation if it
doesn't.  But it accounts for 9% of our startup time in the case
when we have directory information, and I think it's safe to make
the test shorter.  After all, it has yet to find any actual bugs in
curved25519_scalarmult_basepoint_donna() on any platforms.

Closes ticket 28838.
2018-12-13 11:26:09 -05:00
Nick Mathewson
69264f96f3 Merge branch 'dormant_persist_squashed' 2018-12-13 08:26:10 -05:00
Nick Mathewson
b5c04173c8 Change interaction between dormant mode and clock jumps.
When the clock jumps, and we have a record of last user activity,
adjust that record.  This way if I'm inactive for 10 minutes and
then the laptop is sleeping for an hour, I'll still count as having
been inactive for 10 minutes.

Previously, we treat every jump as if it were activity, which is
ridiculous, and would prevent a Tor instance with a jumpy clock from
ever going dormant.
2018-12-13 08:25:54 -05:00
Nick Mathewson
f0a8664677 Add code to parse K=V lines into config_line_t format.
Closes ticket 28755
2018-12-12 11:07:08 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
Nick Mathewson
701eaef980 Move net.inet.ip.random_id code to lib/net/ 2018-12-01 11:36:03 -05:00
Alexander Færøy
2b41b857bd Add LD_PROCESS as log domain.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
31b3a6577c Add buf_flush_to_pipe() and buf_read_from_pipe().
This patch adds two new functions: buf_flush_to_pipe() and
buf_read_from_pipe(), which makes use of our new buf_flush_to_fd() and
buf_read_from_fd() functions.

See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
771930b84c Refactor buf_read_from_socket() into buf_read_from_fd().
This patch refactors buf_read_from_socket() into buf_read_from_fd(), and
creates a specialized function for buf_read_from_socket(), which uses
buf_read_from_fd().

See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
c71f9df07b Refactor buf_flush_to_socket() into buf_flush_to_fd().
This patch refactors buf_flush_to_socket() into buf_flush_to_fd() and
creates a specialization function for buf_flush_to_socket() that makes
use of buf_flush_to_fd().

See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
340260281a Refactor flush_chunk() to work on pipes as well as sockets.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Alexander Færøy
5f26ae833e Refactor read_to_chunk() such that it supports both pipes and sockets.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Nick Mathewson
8a15d0f69b Merge branch 'maint-0.3.5' 2018-11-26 17:25:28 -05:00
teor
44ced9b750 Merge branch 'bug28096-029-squashed' into bug28096-035-squashed
Move the get_uname() changes from src/common/compat.c to
src/lib/osinfo/uname.c
2018-11-15 12:23:29 +10:00
Nick Mathewson
eaff47352a Make sure sandbox-related getaddrinfo() functions always exist. 2018-11-14 16:12:29 -05:00
Nick Mathewson
e429e31ad1 Normalize .may_include to always have paths, and paths to include 2018-11-14 16:07:35 -05:00
Nick Mathewson
c0a7527eb8 Remove dependency on lib/net from lib/sandbox.
This was trivial, and the easiest way to remove the remaining
.may_include circularities.
2018-11-14 16:07:03 -05:00
Nick Mathewson
f6b8c7da66 Move buffers.c out of lib/containers to resolve a circularity. 2018-11-14 16:07:03 -05:00
Nick Mathewson
c9f9c9bc49 Make memarea use smartlist_core, not container. 2018-11-14 16:07:03 -05:00
Nick Mathewson
a6a7a1f3ed Merge branch 'maint-0.3.5' 2018-11-13 16:48:26 -05:00
Nick Mathewson
021187f915 Merge branch 'bug28183_029' into maint-0.3.5 2018-11-13 16:48:21 -05:00
Nick Mathewson
d000e798ac Merge branch 'maint-0.3.5' 2018-11-13 08:27:29 -05:00
Nick Mathewson
ae4c94bb64 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-11-13 08:27:29 -05:00
Nick Mathewson
d1e9285b1d Merge branch 'maint-0.3.5' 2018-11-11 18:05:45 -05:00
Nick Mathewson
896d0ebb99 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-11-11 18:05:45 -05:00
Nick Mathewson
bf4f55a13d Merge branch 'subsystems' 2018-11-09 15:01:49 -05:00
Nick Mathewson
c6336727ca Rename subsystem callback functions to make them consistent 2018-11-09 11:12:12 -05:00
Nick Mathewson
e80595f562 fixup! Make initialization for the "err" library into a subsystem.
Check for failure to install backtrace handler.
2018-11-09 11:00:31 -05:00