Commit Graph

546 Commits

Author SHA1 Message Date
rl1987
f236c9e7f9 Introduce tor_assertf() to allow logging extra error message on assert failure
With format string support!
2019-03-05 16:46:40 +02:00
George Kadianakis
c5da1f1cd5 Merge branch 'tor-github/pr/723' 2019-03-04 18:55:28 +02:00
teor
f186f21a4e
doc: Fix an incorrect comment about calling FreeLibrary() on Windows
There's an incorrect comment in compat_time.c that suggests we call
FreeLibrary() before we're done using the library's functions.
See 29642 for background.

Closes ticket 29643.
2019-03-04 11:29:15 +10:00
David Goulet
13e93bdfd5 Merge branch 'tor-github/pr/718' 2019-03-01 09:36:23 -05:00
Nick Mathewson
64f594499a Document crypto_fast_rng_one_in_n. 2019-03-01 08:20:54 -05:00
David Goulet
b402a0887f Merge branch 'tor-github/pr/655' 2019-02-26 11:30:44 -05:00
George Kadianakis
7fbfdf2af7 Merge branch 'tor-github/pr/611' 2019-02-26 12:33:23 +02:00
Nick Mathewson
065e7da8e6 Re-enable and fix unit test for nofork mappings
This test was previously written to use the contents of the system
headers to decide whether INHERIT_NONE or INHERIT_ZERO was going to
work.  But that won't work across different environments, such as
(for example) when the kernel doesn't match the headers.  Instead,
we add a testing-only feature to the code to track which of these
options actually worked, and verify that it behaved as we expected.

Closes ticket 29541; bugfix not on any released version of Tor.
2019-02-25 08:55:25 -05:00
Nick Mathewson
b3416476b4 Remove all uses of weak_rng.
I'm not removing the weak_rng code itself yet, since it is possible
that we will want to revert one of these.
2019-02-20 12:24:34 -05:00
Nick Mathewson
97b9dfe305 Add a convenience macro to get a fast one-in-n calculation 2019-02-20 12:21:05 -05:00
Nick Mathewson
b25cd5cfe1 Implement code to manage a per-thread instance of crypto_fast_rng()
The subsystems API makes this really simple, fortunately.

Closes ticket 29536
2019-02-19 15:36:08 -05:00
David Goulet
6c173d00f5 Merge branch 'tor-github/pr/702' 2019-02-14 11:43:10 -05:00
George Kadianakis
00b073b1bc Merge branch 'maint-0.3.5' 2019-02-14 18:01:07 +02:00
George Kadianakis
d83c299eba Merge branch 'tor-github/pr/689' into maint-0.3.5 2019-02-14 18:00:05 +02:00
Nick Mathewson
f3cbd6426c Implement a fast aes-ctr prng
This module is currently implemented to use the same technique as
libottery (later used by the bsds' arc4random replacement), using
AES-CTR-256 as its underlying stream cipher.  It's backtracking-
resistant immediately after each call, and prediction-resistant
after a while.

Here's how it works:

We generate psuedorandom bytes using AES-CTR-256.  We generate BUFLEN bytes
at a time.  When we do this, we keep the first SEED_LEN bytes as the key
and the IV for our next invocation of AES_CTR, and yield the remaining
BUFLEN - SEED_LEN bytes to the user as they invoke the PRNG.  As we yield
bytes to the user, we clear them from the buffer.

Every RESEED_AFTER times we refill the buffer, we mix in an additional
SEED_LEN bytes from our strong PRNG into the seed.

If the user ever asks for a huge number of bytes at once, we pull SEED_LEN
bytes from the PRNG and use them with our stream cipher to fill the user's
request.
2019-02-14 09:26:40 -05:00
Nick Mathewson
622a9a8a36 Extract the common body of our random-int functions into a macro
This is the second part of refactoring the random-int-in-range code.
2019-02-14 09:26:40 -05:00
David Goulet
d5de1a0a55 Merge branch 'tor-github/pr/651' 2019-02-13 11:02:02 -05:00
David Goulet
6efc2a0e1f Merge branch 'tor-github/pr/650' into maint-0.3.5 2019-02-13 10:56:24 -05:00
Nick Mathewson
49ec29044d Add more openssl includes to fix no-deprecated compilation
Closes ticket 29026; patch from Mangix.
2019-02-08 08:51:23 -05:00
Nick Mathewson
b9abdcd6bc Fix compilation when openssl is compiled without engine support.
Patch from Mangix. Closes part of ticket 29026.
2019-02-08 08:50:43 -05:00
Nick Mathewson
a49149fc13 Extract numeric CSPRNG functions into a new module.
Some of the code for getting a random value within a range wants to
be shared between crypto_rand() and the new crypto_fast_rng() code.
2019-02-06 22:06:05 -05:00
Nick Mathewson
21d184a184 Remove extraneous #if/#endif wrapper in crypto_rand.c
I don't know how this got here, but this kind of a wrapper only
belongs in a header file.
2019-02-06 22:05:39 -05:00
Nick Mathewson
8ca808f81d Code for anonymous mappings via mmap() or CreateFileMapping().
Using an anonymous mmap() is a good way to get pages that we can set
kernel-level flags on, like minherit() or madvise() or mlock().
We're going to use that so that we can make uninheritable locked
pages to store PRNG data.
2019-02-06 22:03:30 -05:00
Nick Mathewson
d23704bf26 Merge branch 'ticket28668_035' into ticket28668_040 2019-01-23 14:50:22 -05:00
Nick Mathewson
6144cf99ad Capture more BUG warnings in util/time test
These are ones that happen on windows only.

Fixes bug 29161.
2019-01-23 12:37:12 -05:00
Peter Gerber
db3ee1d862
Allow getsockopt(…, SOL_SOCKET, SO_ACCEPTCONN, …) in sandbox
SO_ACCEPTCONN checks whether socket listening is enabled and is
used ever since 9369152aae has been merged.

Closes ticket #29150
2019-01-22 21:51:25 +00:00
Nick Mathewson
21dd3ece62 Merge branch 'maint-0.3.5' 2019-01-22 11:55:56 -05:00
Nick Mathewson
4159d103d6 Merge branch 'bug29042_035' into maint-0.3.5 2019-01-22 11:55:53 -05:00
Nick Mathewson
9b0dd1ae04 Add a function to compute an XOF in one shot.
Motivation:
  1. It's convenient.
  2. It's all that openssl supports.

Part of 28837.
2019-01-17 12:43:20 -05:00
Nick Mathewson
c393171403 Use openssl's version of sha3 when available.
Part of 28837.
2019-01-17 12:43:20 -05:00
Nick Mathewson
16430fdaf5 Merge branch 'maint-0.3.5' 2019-01-16 14:43:53 -05:00
Kris Katterjohn
cbcf75d728 Update and fix a couple of comments in meminfo.c
This fixes a typo and also notes that HW_PHYSMEM64 is defined on
NetBSD (not just OpenBSD).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-16 14:40:15 -05:00
Kris Katterjohn
0f3d88a159 Use HW_PHYSMEM instead of HW_USERMEM in get_total_system_memory_impl
The code checked for sysctl being available and HW_PHYSMEM being
defined, but HW_USERMEM was actually being used with sysctl instead
of HW_PHYSMEM.

The case for OpenBSD, etc. use HW_PHYSMEM64 (which is obviously a
64-bit variant of HW_PHYSMEM) and the case for OSX uses HW_MEMSIZE
(which appears to be a 64-bit variant of HW_PHYSMEM).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-16 14:40:15 -05:00
Nick Mathewson
0981ac4c59 Log more messages when failing to decode RSA keys
We log these messages at INFO level, except when we are reading a
private key from a file, in which case we log at WARN.

This fixes a regression from when we re-wrote our PEM code to be
generic between nss and openssl.

Fixes bug 29042, bugfix on 0.3.5.1-alpha.
2019-01-16 14:35:30 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
efe55b8898 Bump copyright date to 2019. 2019-01-16 12:32:32 -05:00
Nick Mathewson
b169c8c14f Merge remote-tracking branch 'asn-github/adaptive_padding-final' 2019-01-14 14:48:00 -05:00
Nick Mathewson
99947c3ce0 Merge remote-tracking branch 'teor/bug29015' 2019-01-14 13:59:07 -05:00
Kris Katterjohn
df1a8a657e Actually close the stdout pipe on error in process_unix_exec
When cleaning up after an error in process_unix_exec, the stdin
pipe was being double closed instead of closing both the stdin
and stdout pipes.  This occurred in two places.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-11 18:55:09 -05:00
Nick Mathewson
5dd926caa6 Include the beginnings of a FAQ about which timer to use. 2019-01-11 09:08:51 -05:00
Taylor R Campbell
d82a8a7f9d Add some more type checking.
NOTE: This commit breaks the build, because there was a mistake in an
earlier change of exactly the sort that this is meant to detect!  I'm
leaving it broken for illustration.
2019-01-11 14:41:28 +02:00
Taylor R Campbell
0f8253bddb Use the distribution abstraction as an abstraction. 2019-01-11 14:40:50 +02:00
Taylor R Campbell
531df9590d Move ceil call back into the geometric sampler.
Test exactly what the geometric sampler returns, because that's what
the downstream callers of it are going to use.

While here, also assert that the geometric sampler returns a positive
integer.  (Our geometric distribution is the one suported on {1, 2,
3, ...} that returns the number of trials before the first success,
not the one supported on {0, 1, 2, ...} that returns the number of
failures before the first success.)
2019-01-11 14:40:50 +02:00
George Kadianakis
e0e0338dc4 Rename crypto_rand_uint32() -> crypto_rand_u32()
See https://github.com/torproject/tor/pull/624#discussion_r246453777
2019-01-10 13:06:08 +02:00
Nick Mathewson
0ed966fde1 Merge branch 'maint-0.3.5' 2019-01-09 09:38:27 -05:00
Nick Mathewson
694e3c57fd Merge remote-tracking branch 'tor-github/pr/636' into maint-0.3.5 2019-01-09 09:38:22 -05:00
Nick Mathewson
e6a4401200 Merge branch 'maint-0.3.5' 2019-01-09 08:52:10 -05:00
Kris Katterjohn
c13a81f7f9 Fix (and make consistent) the use of OpenBSD preprocessor macro tests
Prior to this commit, the testsuite was failing on OpenBSD.  After
this commit the testsuite runs fine on OpenBSD.

It was previously decided to test for the OpenBSD macro (rather than
__OpenBSD__, etc.) because OpenBSD forks seem to have the former
macro defined.  sys/param.h must be included for the OpenBSD macro
definition; however, many files tested for the OpenBSD macro without
having this header included.

This commit includes sys/param.h in the files where the OpenBSD macro
is used (and sys/param.h is not already included), and it also
changes some instances of the __OpenBSD__ macro to OpenBSD.

See commit 27df23abb6 which changed
everything to use OpenBSD instead of __OpenBSD__ or OPENBSD.  See
also tickets #6982 and #20980 (the latter ticket is where it was
decided to use the OpenBSD macro).

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-09 08:51:57 -05:00
Nick Mathewson
f9a7701ff1 Merge branch 'ticket28856_v2' 2019-01-09 08:46:40 -05:00
teor
671c34d9b4
lib/net: improve the docs for tor_{ersatz_,}socketpair()
Add some details about IP family support, and point to
tor_socketpair() from tor_ersatz_socketpair().

Closes ticket 29015.
2019-01-08 14:45:17 +10:00
Kris Katterjohn
c59b572e93 Fix tor_ersatz_socketpair on IPv6-only systems
In get_local_listener used by tor_ersatz_socketpair, the address
family used when binding the IPv6 socket was AF_INET instead of
AF_INET6.

Fixes bug 28995; bugfix on 0.3.5.1-alpha.

Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
2019-01-07 17:28:21 -06:00
Nick Mathewson
4e4f93d364 Add a #ifdef HAVE_UNISTD_H check to buffers.c
Reported on tor-dev by Gisle Vanem.  Bug not in any released Tor

(The suggested patch used _MSC_VER, but that's not how we do stuff
with autoconf.  With autoconf, you detect the feature you want,
rather than trying to list all the systems that do or do not have
it.)
2019-01-03 09:52:19 -05:00
Nick Mathewson
9ba690e33f Merge branch 'maint-0.3.5' 2019-01-03 09:45:56 -05:00
Nick Mathewson
abdc6aede2 Merge branch 'ticket28851_035_squashed' into maint-0.3.5 2019-01-03 09:45:53 -05:00
Nick Mathewson
b82717b273 Stop re-checking our hardcoded dh parameters on every startup
Closes ticket 28851.
2019-01-03 09:45:43 -05:00
Nick Mathewson
47176eb678 Merge branch 'maint-0.3.5' 2019-01-03 09:44:15 -05:00
Nick Mathewson
98736cf36a Merge remote-tracking branch 'public/ticket28838_035' into maint-0.3.5 2019-01-03 09:44:10 -05:00
Nick Mathewson
27853938a1 Merge branch 'maint-0.3.5' 2019-01-03 09:02:40 -05:00
Nick Mathewson
ed62f0fa15 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-01-03 09:02:39 -05:00
Nick Mathewson
3e7f13a4ef Merge branch 'maint-0.3.5' 2019-01-02 16:15:39 -05:00
Nick Mathewson
b33bcb1e63 Add an errno.h include to freespace.c to fix bug 28974. 2019-01-02 15:19:52 -05:00
George Kadianakis
926fc93be5 Concentrate all TOR_USEC_PER_SEC definitions in a single header file.
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
George Kadianakis
2ccf326837 Implement and test probability distributions used by WTF-PAD.
This project introduces the prob_distr.c subsystem which implements all the
probability distributions that WTF-PAD needs. It also adds unittests for all of
them.

Code and tests courtesy of Riastradh.

Co-authored-by: Taylor R Campbell <campbell+tor@mumble.net>
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
George Kadianakis
a336d816a6 Circuit padding tests.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:25:55 +02:00
Mike Perry
3ba7581129 Provide a smartlist reverse-order traversal.
We need this for padding negotiation so that we can have later machine
revisions supercede earlier ones.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:09:13 +02:00
Nick Mathewson
70dd6d07bb Merge branch 'orconn-tracker_squashed' 2018-12-21 14:22:11 -05:00
Nick Mathewson
fd58e5e498 Fix priority on process subsystem level: it uses "net" 2018-12-21 14:12:20 -05:00
Nick Mathewson
ab4395d082 Merge branch 'ticket28847' 2018-12-21 13:26:47 -05:00
Alexander Færøy
2322b56389 Fix typo in time_sys.h. 2018-12-21 13:26:38 -05:00
Alexander Færøy
cf4b3dbd44 Use the subsystem list to initialize and shutdown process module.
This patch makes the process module use the subsystem list for
initializing and shutting down.

See: https://bugs.torproject.org/28847
2018-12-21 13:26:38 -05:00
Taylor Yu
b0f974633a Add LD_BTRACK log domain for bootstrap tracker
Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Nick Mathewson
5c85ba3077 Merge remote-tracking branch 'tor-github/pr/608' 2018-12-20 11:42:26 -05:00
Nick Mathewson
a517daa56f base32_decode(): Return number of bytes written on success.
This makes it consistent with base64_decode().

Closes ticket 28913.
2018-12-20 08:36:25 -05:00
Alexander Færøy
f7e175db57 Forward declare smartlist_t in process.h
This allows other libraries to include process.h without including
the smartlist_t headers first.

See: https://bugs.torproject.org/28847
2018-12-20 14:36:04 +01:00
Alexander Færøy
7762088967 No need to log ordinary EOF conditions as LOG_WARN.
Let's not use log_warn() when a pipe is closed under what should be
considered normal conditions.

See: https://bugs.torproject.org/28179
2018-12-20 13:12:53 +01:00
Alexander Færøy
412fbe9f17 Make example CancelIoEx() code use CancelIo().
This patch changes the CancelIoEx() example code to use CancelIo(),
which is available for older versions of Windows too. I still think the
kernel handles this nicely by sending broken pipes if either side
closes the pipe while I/O operations are pending.

See: https://bugs.torproject.org/28179
2018-12-20 13:11:24 +01:00
Alexander Færøy
f58e597d42 Handle ERROR_BROKEN_PIPE in completion routines.
Handle `ERROR_BROKEN_PIPE` from ReadFileEx() and WriteFileEx() in
process_win32_stdin_write_done() and
process_win32_handle_read_completion() instead of in the early handler.
This most importantmly makes sure that `reached_eof` is set to true when
these errors appears.

See: https://bugs.torproject.org/28179
2018-12-20 13:04:49 +01:00
Alexander Færøy
36e24782f8 Remember to set reached_eof when our handles are reporting errors.
This patch adds some missing calls to set `reached_eof` of our handles
when various error conditions happens or when we close our handle (which
happens at `process_terminate()`.

See: https://bugs.torproject.org/28179
2018-12-20 13:02:22 +01:00
Alexander Færøy
c6e041e3d8 Handle errors even after success from ReadFileEx() and WriteFileEx().
This patch adds some additional error checking after calls to
ReadFileEx() and WriteFileEx(). I have not managed to get this code to
reach the branch where `error_code` is NOT `ERROR_SUCCESS`, but MSDN
says one should check for this condition so we do so just to be safe.

See: https://bugs.torproject.org/28179
2018-12-20 12:57:20 +01:00
Alexander Færøy
44586a89ef Delay checking process for termination until both stdout and stderr are closed.
This patch makes us delay checking for whether we have an exit code
value (via GetExitCodeProcess()) until both stdout and stderr have been
closed by the operating system either by the process itself or by
process cleanup after termination.

See: https://bugs.torproject.org/28179
2018-12-20 12:53:28 +01:00
Alexander Færøy
1d8dcb416c Remember to close the child process' ends of the pipes.
This prevents us from leaking the HANDLE for stdout, stderr, and stdin.

See: https://bugs.torproject.org/28179
2018-12-20 12:47:04 +01:00
Nick Mathewson
bb091da1e7 Merge branch 'ticket28839_v2_squashed' 2018-12-18 18:59:05 -05:00
Alexander Færøy
ca7a2ecc51 Avoid breaking the event loop prematurely.
This patch makes sure that we terminate the event loop from the event
loop timer instead of directly in the process' exit handler. This allows
us to run the event loop an additional time to ensure that the SleepEx()
call on Windows is called and the data from stdout/stderr is delivered
to us.

Additionally we ensure that we don't try to read or write data from a
Unix process that have been terminated in the main loop, since its file
descriptors are closed at that time.

See: https://bugs.torproject.org/28179
2018-12-18 13:35:29 -05:00
Nick Mathewson
8a01f0eaab lib/process may include lib/buf. 2018-12-17 17:58:49 -05:00
Nick Mathewson
4ad59bfbc2 Update location of buffers.h 2018-12-17 17:01:50 -05:00
Nick Mathewson
e969d9c6b4 Merge branch 'ticket28179_squashed' into ticket28179_squashed_merged 2018-12-17 16:41:01 -05:00
Alexander Færøy
c8b8b15f0e Ensure that line_size >= 1 before trying to trim input string.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
fab22509d7 Make Windows process event timer API available for dormant interface.
This patch changes the API of the Windows backend of the Process
subsystem to allow the dormant interface to disable the Process event
timer.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
a33a77d9cd Document the format of process_t::arguments.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
0d796cce17 Use errno directly if we are not reading/writing from/to a socket.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
cacdd29087 Use const char * instead of char * for line parameter for process callbacks.
This patch changes the type definition of the process callbacks to use
`const char *` instead of `char *`.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
5585cbd08f Change the Process exit_callback to return bool.
This patch changes our process_t's exit_callback to return a boolean
value.  If the returned value is true, the process subsystem will call
process_free() on the given process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
22cb3c6ce9 Call close() on stdin/stdout/stderr in process_terminate().
Call close() on all process handles after we have called kill(pid,
SIGTERM).

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ccc1963890 Move remaining code from subprocess.{h,c} to more appropriate places.
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.

We also delete the now empty subprocess module files.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
f7d13425fc Delete old process_handle_t code.
This patch removes the old process_handle_t code. Everything should by
now be using the process_t interface.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ad4cc89c5d Add "PT" log domain.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
b0d268a822 Add process_reset_environment() to the Process subsystem.
This patch adds a new function that allows us to reset the environment
of a given process_t with a list of key/value pairs.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
4f611a1df7 Add process_terminate().
This patch adds support for process termination to the Process
subsystem.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
338137221c Make sure we call process_notify_event_exit() as the last thing in different callbacks.
This patch makes sure that we call process_notify_event_exit() after we
have done any modifications we need to do to the state of a process_t.
This allows application developers to call process_free() in the
exit_callback of the process.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e982fb1dae Add documentation for the is_socket and error argument of read_to_chunk().
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00