Commit Graph

4317 Commits

Author SHA1 Message Date
Nick Mathewson
7465ea4ad9 Remove some LCOV_EXCL stuff that I think may be testable after all.
This is partial revert on 22286.

Also, tweak some log messages to be distinct.
2017-08-08 10:08:06 -04:00
Nick Mathewson
6121ca16bc Merge remote-tracking branch 'ahf/bugs/22286' into maint-0.3.1 2017-08-08 10:03:08 -04:00
Roger Dingledine
2032b9b1b1 fix typo in comment 2017-08-07 00:22:27 -04:00
Nick Mathewson
89407bedf8 Tweak usage of get_current_working_dir() for tor_malloc() paranoia.
We assume that tor_free() is not required to be compatible with
the platform malloc(), so we need to use a strdup here.
2017-08-04 12:26:35 -04:00
cypherpunks
bfe740f065 Refactor retrieving the current working directory
The GNU C Library (glibc) offers an function which allocates the
necessary memory automatically [0]. When it is available, we use that.

Otherwise we depend upon the `getcwd` function which requires a
preallocated buffer (and its size). This function was used incorrectly
by depending on the initial buffer size being big enough and otherwise
failing to return the current working directory. The proper way of
getting the current working directory requires a loop which doubles the
buffer size if `getcwd` requires it. This code was copied from [1] with
modifications to fit the context.

[0] https://www.gnu.org/software/hurd/hurd/porting/guidelines.html
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/getcwd.html
2017-08-04 12:22:53 -04:00
Neel Chauhan
5ee6ca8da2 Switch to offsetof() 2017-08-03 08:56:35 -04:00
Nick Mathewson
accb734c5f Turn base < 0 into a BUG() in our long-parse functions. 2017-07-28 10:26:06 -04:00
Nick Mathewson
602c52cad4 Coverity deadcode shenanigans on BUG() macro.
We don't actually want Coverity to complain when a BUG() check can
never fail, since such checks can prevent us from introducing bugs
later on.

Closes ticket 23054. Closes CID 1415720, 1415724.
2017-07-28 10:02:38 -04:00
Nick Mathewson
baf53300d7 Merge branch 'maint-0.3.1' 2017-07-27 20:33:01 -04:00
Nick Mathewson
af3079a492 Try to work around a compile warning in workqueue.c 2017-07-27 20:32:59 -04:00
Nick Mathewson
15ed1c0c83 Merge branch 'maint-0.3.1' 2017-07-27 16:30:52 -04:00
Nick Mathewson
ba334c00da Merge branch 'multi-priority_squashed' into maint-0.3.1 2017-07-27 16:29:34 -04:00
Nick Mathewson
f5a852de91 Note that threadpool_queue_work...() can't actually return NULL 2017-07-27 16:28:59 -04:00
Nick Mathewson
bddea78ded Fix a pair of stale comments in workqueue.c
These comments said that each thread had a separate queue, but we
haven't been using that design for some while.
2017-07-27 16:28:59 -04:00
Nick Mathewson
efadebf7c3 Make the chance for priority inversion thread-specific
Instead of choosing a lower-priority job with a 1/37 chance, have
the chance be 1/37 for half the threads, and 1/2147483647 for the
other half.  This way if there are very slow jobs of low priority,
they shouldn't be able to grab all the threads when there is better
work to do.
2017-07-27 16:28:59 -04:00
Nick Mathewson
10e0bff4ca Add support for multi-priority workqueues
Each piece of queued work now has an associated priority value; each
priority goes on a separate queue.

With probability (N-1)/N, the workers will take work from the highest
priority nonempty queue.  Otherwise, they'll look for work in a
queue of lower priority.  This behavior is meant to prevent
starvation for lower-priority tasks.
2017-07-27 16:28:05 -04:00
Nick Mathewson
ced2dd5f92 Merge branch 'maint-0.3.0' into maint-0.3.1 2017-07-27 08:23:37 -04:00
Nick Mathewson
b387dd364f Merge branch 'maint-0.3.1' 2017-07-27 08:23:37 -04:00
Nick Mathewson
ad35e595e5 Merge branch 'maint-0.2.9' into maint-0.3.0 2017-07-27 08:23:36 -04:00
Nick Mathewson
24ddf5862e Merge remote-tracking branch 'public/bug20247_029' into maint-0.2.9 2017-07-27 08:23:34 -04:00
Nick Mathewson
8d3c3f039d Merge branch 'maint-0.3.1' 2017-07-26 12:58:22 -04:00
Nick Mathewson
431c8d09ee Merge branch 'maint-0.3.0' into maint-0.3.1 2017-07-26 12:58:22 -04:00
Nick Mathewson
18734d3b25 Merge branch 'maint-0.2.9' into maint-0.3.0 2017-07-26 12:58:22 -04:00
Nick Mathewson
32b9edeb91 Fix build warnings from Coverity related to our BUG macro
In the Linux kernel, the BUG() macro causes an instant panic.  Our
BUG() macro is different, however: it generates a nonfatal assertion
failure, and is usable as an expression.

Additionally, this patch tells util_bug.h to make all assertion
failures into fatal conditions when we're building with a static
analysis tool, so that the analysis tool can look for instances
where they're reachable.

Fixes bug 23030.
2017-07-26 12:57:49 -04:00
Nick Mathewson
8b5b3b5fb4 Merge branch 'maint-0.3.1' 2017-07-26 12:54:41 -04:00
Nick Mathewson
30a98c765f Merge branch 'bug22927_031' into maint-0.3.1 2017-07-26 12:54:37 -04:00
Nick Mathewson
3c017e823b Merge branch 'maint-0.2.9' into maint-0.3.0 2017-07-26 12:53:38 -04:00
Nick Mathewson
5141360099 Merge branch 'maint-0.3.1' 2017-07-26 12:53:38 -04:00
Nick Mathewson
d068f3359f Merge branch 'maint-0.3.0' into maint-0.3.1 2017-07-26 12:53:38 -04:00
Nick Mathewson
fca1934c88 Suppress clang4-specific -Wdouble-promotion warnings
Wow, it sure seems like some compilers can't implement isnan() and
friends in a way that pleases themselves!

Fixes bug 22915. Bug trigged by 0.2.8.1-alpha and later; caused by
clang 4.
2017-07-26 12:53:00 -04:00
Nick Mathewson
6c8c973191 Rename the hybrid_encrypt/decrypt functions; label them as dangerous
We need to keep these around for TAP and old-style hidden services,
but they're obsolete, and we shouldn't encourage anyone to use them.
So I've added "obsolete" to their names, and a comment explaining
what the problem is.

Closes ticket 23026.
2017-07-24 14:34:53 -04:00
Nick Mathewson
db1664e593 Improve comment about why we disable TLS compression.
Closes bug 22964.  Based on Teor's replacement there, but tries
to put the comment in a more logical place, and explain why we're
actually disabling compression in the first place.
2017-07-24 14:17:16 -04:00
Roger Dingledine
ec5b30ca0b fix whitespace issue 2017-07-23 00:57:10 -04:00
Nick Mathewson
2ae51ed5e2 Fix zstd 1.3.0 trouble: Be more respectful of its state machine
In zstd 1.3.0, once you have called ZSTD_endStream and been told
that your putput buffer is full, it really doesn't want you to call
ZSTD_compressStream again.  ZSTD 1.2.0 didn't seem to mind about
this.

This patch fixes the issue by making sure never to call
ZSTD_endStream if there's any more data on the input buffer to
process, by flushing even when we're about to call "endStream", and
by never calling "compress" or "flush" after "endStream".
2017-07-14 16:31:29 -04:00
Nick Mathewson
ec29cae8d7 Merge branch 'maint-0.3.1' 2017-07-13 16:52:20 -04:00
Nick Mathewson
63ceadb485 Use LANG_ENGLISH in windows error messages
This change prevents us from generating corrupt messages when we
are confused about codepage settings, and makes Windows errors
consistent with the rest of our logs.

Fixes bug 22520; bugfix on 0.1.2.8-alpha.  Patch from "Vort".
2017-07-07 13:12:45 -04:00
Nick Mathewson
c387cc5022 Merge branch 'ticket21859_032_01_squashed' 2017-07-07 11:17:53 -04:00
George Kadianakis
f35f52e869 Hide crypto_digest_t again and use an accessor for tests. 2017-07-07 11:12:27 -04:00
George Kadianakis
43a73f6eb6 test: Crypto groundwork for e2e circuit unittests.
- Move some crypto structures so that they are visible by tests.

- Introduce a func to count number of hops in cpath which will be used
  by the tests.

- Mark a function as mockable.
2017-07-07 11:12:26 -04:00
Nick Mathewson
55777b3ff9 Merge branch 'maint-0.2.9' into maint-0.3.0 2017-07-05 16:11:48 -04:00
Nick Mathewson
0dc7d68bb5 Merge branch 'maint-0.3.1' 2017-07-05 16:11:48 -04:00
Nick Mathewson
15b13578e8 Merge branch 'maint-0.3.0' into maint-0.3.1 2017-07-05 16:11:48 -04:00
Nick Mathewson
dfc0614840 Only disable -Wfloat-conversion on mingw when it exists.
The 22081 fix disabled -Wfloat-conversion, but -Wfloat-conversion
didn't exist in every relevant mingw; it was added in GCC 4.9.x some
time, if the documentation can be trusted.

Bug not in any released version of tor.
2017-07-05 16:10:45 -04:00
Nick Mathewson
13024c7932 Merge branch 'maint-0.3.1' 2017-07-05 15:57:09 -04:00
Nick Mathewson
9383fa3851 Fix mixed-sign comparison warning in fix for 22797. 2017-07-05 15:56:57 -04:00
Nick Mathewson
546f5b364b Merge branch 'maint-0.2.9' into maint-0.3.0 2017-07-05 13:43:31 -04:00
Nick Mathewson
15fddaffd5 Merge branch 'maint-0.3.1' 2017-07-05 13:43:31 -04:00
Nick Mathewson
5434b2451e Merge branch 'maint-0.3.0' into maint-0.3.1 2017-07-05 13:43:31 -04:00
Nick Mathewson
32c0066e4b Merge branch 'maint-0.2.8' into maint-0.2.9 2017-07-05 13:43:21 -04:00
Nick Mathewson
5ff0f1ab9e Merge branch 'maint-0.2.7-redux' into maint-0.2.8 2017-07-05 13:42:47 -04:00
Nick Mathewson
6cd6d488dc Merge branch 'maint-0.2.6' into maint-0.2.7-redux 2017-07-05 13:42:37 -04:00
Nick Mathewson
f6420bceec Merge branch 'maint-0.2.5' into maint-0.2.6 2017-07-05 13:42:32 -04:00
Nick Mathewson
ff8c230d7c Merge branch 'maint-0.2.4' into maint-0.2.5 2017-07-05 13:42:26 -04:00
Nick Mathewson
16d2bce893 Allow setsockopt(IPV6_V6ONLY) in sandbox.
Fixes bug 20247.  We started setting V6ONLY in 0.2.3.13-alpha and
added the sandbox on 0.2.5.1-alpha.
2017-07-05 13:09:21 -04:00
Nick Mathewson
8bc70a2ad2 Merge branch 'maint-0.2.9' into maint-0.3.0 2017-07-05 11:19:03 -04:00
Nick Mathewson
d4f08c74fe Merge branch 'maint-0.3.1' 2017-07-05 11:19:03 -04:00
Nick Mathewson
0f97f963e3 Merge branch 'maint-0.3.0' into maint-0.3.1 2017-07-05 11:19:03 -04:00
Nick Mathewson
bb97f680e7 Merge branch 'bug22801_028' into maint-0.2.9 2017-07-05 11:18:59 -04:00
Nick Mathewson
e04cc7e27f Merge branch 'maint-0.3.1' 2017-07-05 11:16:51 -04:00
Nick Mathewson
e88aa98451 Merge branch 'teor-bug22797-025' into maint-0.3.1 2017-07-05 11:16:30 -04:00
teor
878e0d45a5 Always allow extra file descriptors when setting the connection maximum
When setting the maximum number of connections allowed by the OS,
always allow some extra file descriptors for other files.

Fixes bug 22797; bugfix on 0.2.0.10-alpha.
2017-07-05 11:15:10 -04:00
cypherpunks
c79e286386 Use the proper syscall in sandbox error messages
Fixes #22750.
2017-07-05 09:56:28 -04:00
Nick Mathewson
bb3f74e66b Fix assertion failure related to openbsd strtol().
Fixes bug 22789; bugfix on 0.2.3.8-alpha.
2017-07-03 11:22:27 -04:00
Nick Mathewson
5361032219 Fix -Wfloat-conversion C warnings on mingw in clamp_double_to_int64.
We just have to suppress these warnings: Mingw's math.h uses gcc's
__builtin_choose_expr() facility to declare isnan, isfinite, and
signbit.  But as implemented in at least some versions of gcc,
__builtin_choose_expr() can generate type warnings even from
branches that are not taken.

Fixes bug 22801; bugfix on 0.2.8.1-alpha.
2017-07-03 10:59:31 -04:00
Nick Mathewson
2c718c1a12 Merge branch 'maint-0.3.1' 2017-06-29 10:43:50 -04:00
Nick Mathewson
a088a08eeb Log real error message when unable to remove a storagedir file
Attempts to help diagnose 22752.
2017-06-28 14:24:27 -04:00
Nick Mathewson
01404d7c52 Merge branch 'maint-0.3.1' 2017-06-28 12:25:09 -04:00
Alexander Færøy
c239b2fc9c Fix crash in LZMA module when the Sandbox is enabled.
This patch fixes a crash in our LZMA module where liblzma will allocate
slightly more data than it is allowed to by its limit, which leads to a
crash.

See: https://bugs.torproject.org/22751
2017-06-28 10:00:24 -04:00
George Kadianakis
0d9873ac0d ed25519: Check retval of unpack_negative_vartime in donna. 2017-06-28 14:58:22 +03:00
Nick Mathewson
733ce556ad Merge branch 'asn_bug22006_final_squashed' 2017-06-27 18:21:46 -04:00
Nick Mathewson
3f94041589 no newlines in log messages. 2017-06-27 18:21:35 -04:00
Nick Mathewson
7fff6cfead Merge branch 'asn_bug22006_final_squashed' 2017-06-27 17:19:08 -04:00
George Kadianakis
559658ff1c ed25519: Add func that checks for torsion component in pubkeys.
See https://lists.torproject.org/pipermail/tor-dev/2017-April/012213.html .
2017-06-27 17:17:58 -04:00
Nick Mathewson
82a68be69a Merge branch 'maint-0.3.1' 2017-06-26 10:32:57 -04:00
Nick Mathewson
0c7c49483f Fix a coverity warning about a no-op assert with-64 bit size_t
This is CID 1403400
2017-06-26 10:31:13 -04:00
Nick Mathewson
acbe16a9a2 Log even more to try to debug 22719 2017-06-24 13:40:02 -04:00
Nick Mathewson
6caf924605 Add an additional log message to try to diagnose #22719 2017-06-24 11:27:46 -04:00
Nick Mathewson
e51e7bd38b Merge branch 'bug22502_redux_031' into maint-0.3.1 2017-06-20 20:27:48 -04:00
Nick Mathewson
c999e84436 Merge branch 'bug22672_031' into maint-0.3.1 2017-06-20 20:26:45 -04:00
Nick Mathewson
392e5457b8 Merge remote-tracking branch 'argonblue/bug22638' into maint-0.3.1 2017-06-20 13:35:38 -04:00
Nick Mathewson
9328bd524e Enforce the rule that COMPRESS_OK means progress was made.
If COMPRESS_OK occurs but data is neither consumed nor generated,
treat it as a BUG and a COMPRESS_ERROR.

This change is meant to prevent infinite loops in the case where
we've made a mistake in one of our compression backends.

Closes ticket 22672.
2017-06-20 12:26:57 -04:00
Nick Mathewson
945256188a mingw/windows printf lacks %zd ; use %lu and casts instead
(This approach can lose accuracy, but it's only in debug-level messages.)

Fixes windows compilation. Bugfix on recent compress.c changes; bug
not in any released Tor.
2017-06-20 12:12:55 -04:00
Taylor Yu
25edb41e6f Fix compress_none.c header comment
The Doxygen \file markup for compress_none.c had the wrong filename.
Fixes #22638.
2017-06-20 11:27:17 -04:00
Nick Mathewson
eb632afb17 Correct the fix to bug 22629 to permit trailing non-garbage
This change makes it so that we can decompress concatenated zstd
outputs.
2017-06-20 10:24:22 -04:00
Nick Mathewson
1c0459f19a Merge remote-tracking branch 'teor/bug22502' into maint-0.3.1 2017-06-20 10:04:16 -04:00
Nick Mathewson
eff5e29404 Merge branch 'maint-0.3.0' into maint-0.3.1 2017-06-19 13:52:19 -04:00
Nick Mathewson
71c701927a Merge branch 'maint-0.2.9' into maint-0.3.0 2017-06-19 13:52:19 -04:00
Nick Mathewson
59f29970fa Permit the fchmod system call.
Fixes bug 22516; bugfix on 0.2.5.4-alpha.
2017-06-16 14:03:02 -04:00
teor
7d535ea9d3
Add extra logging during compression and decompression
This helps diagnose failures.

Part of #22502.
2017-06-16 09:48:18 +10:00
teor
cbaf0c049c
Return TOR_COMPRESS_BUFFER_FULL when zstd has additional input
Fixes #22628.
2017-06-16 09:47:32 +10:00
teor
617e1da636
Remove a redundant conditional in tor_zstd_compress_process
Part of #22502
2017-06-16 09:46:46 +10:00
teor
7605bd528e
Move a comment to the right place in tor_zstd_compress_process
Part of #22502
2017-06-16 09:45:58 +10:00
teor
952c9073ad
Check for trailing input garbage in tor_compress_impl() when decompressing
Fixes #22629.
2017-06-16 09:41:29 +10:00
teor
8e1b37a4aa
Check if tor_compress_new() returns NULL in tor_compress_impl()
Partial fix to 22626.
2017-06-16 09:38:18 +10:00
Nick Mathewson
e3b1573be6 Merge branch 'maint-0.3.0' 2017-06-05 15:52:06 -04:00
Nick Mathewson
d1c1dc229e Merge branch 'maint-0.2.9' into maint-0.3.0 2017-06-05 15:44:12 -04:00
Nick Mathewson
ec84fc1d8e Improve documentation on get_{peer,own}_certificate()
Make it clear that we're returning a newly allocated copy.
2017-06-05 15:27:33 -04:00
Nick Mathewson
39b7e89c28 Test prerequisites: function to dup a cert, make get_own_cert mockable. 2017-06-05 15:27:33 -04:00
Nick Mathewson
50facb40bb On v3 link handshake, send the correct link certificate
Previously we'd send the _current_ link certificate, which would
cause a handshaking failure when the TLS context rotated.
2017-06-05 15:27:33 -04:00
Nick Mathewson
0fbe1a2c6f Merge remote-tracking branch 'teor/bug22424' 2017-05-30 08:50:45 -04:00
Daniel Pinto
94d321120e Replace 3-star pointer with 2-star pointer 2017-05-28 20:24:48 +01:00
teor
79725289e1
If we do underflow the know usage of a storage, recalculate it
Fixes bug #22424 on 0.3.1.1-alpha.
2017-05-28 22:34:43 +10:00
teor
69b234a0a8
Refactor storage usage reductions into a static function
No behaviour change.

Part of #22424.
2017-05-28 22:28:43 +10:00
teor
334fe6bb6b
Don't underflow usage when it is unknown and a file is removed
Part of #22424.
2017-05-28 22:16:00 +10:00
teor
9e36b0beb9
Always check for usage underflow when removing a file in storage.c
Part of #22424.
2017-05-28 22:12:09 +10:00
teor
f6841ae263
Fix comment typos in storage.c 2017-05-28 22:11:22 +10:00
Daniel Pinto
f8ccf8d9a9 Fix crash with %include
Fixes crash when including a folder that contains a non-empty file
without any values followed by any other non-empty file.
2017-05-28 09:53:14 +01:00
Nick Mathewson
6fcaf83c98 Cleanup MOCK_IMPL (etc) to be findable with etags
A fair number of our mock_impl declarations were messed up so that
even our special AM_ETAGSFLAGS couldn't find them.

This should be a whitespace-only patch.
2017-05-26 14:07:06 -04:00
teor
af98b862a5
Fix comments of functions that return tor_snprintf
No code changes needed: in the places where we actually check the
return value of these functions, we handle it correctly.
2017-05-23 18:44:45 +10:00
Roger Dingledine
0698a0beca fix two typos in comments
closes ticket 22322
2017-05-22 01:43:52 -04:00
Nick Mathewson
d950ad0dfd Remove call to get_unquoted_path in config_process_include()
parse_config_line_from_str_verbose() already looks for strings
that are surrounded by quotes, and processes them with
unescape_string().  So things were getting decoded twice, which was
(in turn) playing havoc with backslashes on Windows.
2017-05-19 14:09:51 -04:00
Nick Mathewson
ff1af5550a strlen() returns size_t 2017-05-19 08:54:56 -04:00
Nick Mathewson
69ef94820b Merge branch 'add_rust_squashed' 2017-05-19 08:47:18 -04:00
Sebastian Hahn
f8ef7c65d1 Add some Rust utility functions and print support
This gives an indication in the log that Tor was built with Rust
support, as well as laying some groundwork for further string-returning
APIs to be converted to Rust
2017-05-19 08:47:10 -04:00
Nick Mathewson
92d335b3dc Merge remote-tracking branch 'jigsaw/torrc-dir-fix-1922_squashed2' 2017-05-19 08:46:13 -04:00
Daniel Pinto
ba3a5f82f1 Add support for %include funcionality on torrc #1922
config_get_lines is now split into two functions:
 - config_get_lines which is the same as before we had %include
 - config_get_lines_include which actually processes %include
2017-05-18 23:44:16 +01:00
Alexander Færøy
fcf836d239 Add coverage markers in Zstd + LZMA compression backends.
See: https://bugs.torproject.org/22286
2017-05-17 13:23:54 +00:00
Alexander Færøy
77511aed6c Fix whitespace issue.
See: https://bugs.torproject.org/22286
2017-05-17 13:23:46 +00:00
Daniel Pinto
e04da2828d Fixed error on BASEXX_NOPAD LEN and BUFSIZE macros #21872 2017-05-17 00:22:11 +01:00
Alexander Færøy
3a05687c6d
Add API for getting human readable descriptions of a compress_method_t
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Nick Mathewson
95fa7d1cf8 In channelpadding tests that touch libevent, call event_reinit().
This is necessary to avoid crashes and test failures on kevent-based
systems.

Fixes bug 22209; bug not in any released Tor.
2017-05-10 11:01:13 -04:00
Nick Mathewson
35025ee51f Merge branch 'maint-0.3.0' 2017-05-08 13:40:41 -04:00
Nick Mathewson
d792d2a14d Merge branch 'maint-0.2.9' into maint-0.3.0 2017-05-08 13:40:26 -04:00
Nick Mathewson
584ab1f29b Merge branch 'compress_none_v2_squashed' 2017-05-02 08:33:35 -04:00
Nick Mathewson
1bc21111d8 Treat the identity transformation as another kind of compression.
This will allow us to treat NO_METHOD as a real compression method,
and to simplify code that currently does

   if (compressing) {
      compress
   } else {
      copy
   }
2017-05-02 08:31:32 -04:00
Nick Mathewson
c486ef57a3 Rename x-lzma to x-tor-lzma
We shouldn't call it lzma, because we are imposing a limit on the
memory needed for decoding.
2017-05-01 15:31:28 -04:00
Nick Mathewson
4837421d7c Merge remote-tracking branch 'ahf/bugs/21665' 2017-05-01 14:22:49 -04:00
Nick Mathewson
531835f561 Increase MALLOC_MP_LIM to 16MB
Increase the maximum allowed size passed to mprotect(PROT_WRITE)
from 1MB to 16MB. This was necessary with the glibc allocator
in order to allow worker threads to allocate more memory --
which in turn is necessary because of our new use of worker
threads for compression.

Closes ticket #22096. Found while working on #21648.
2017-04-28 10:55:10 -04:00
Nick Mathewson
00a12337ff Merge branches 'consdiffmgr_orig_squashed' and 'actually_compute_diffs_squashed' 2017-04-27 21:43:06 -04:00
Nick Mathewson
7a0964279f Functionality to ensure there is space to add files to cache. 2017-04-27 21:40:13 -04:00
Nick Mathewson
16d6ab6640 Fix use-after-free bug in storage_dir sandbox code. 2017-04-27 21:40:12 -04:00
Alexander Færøy
0672b33f1e
Enforce 16 MB upper bound of memory usage in LZMA decoder.
This patch changes two things in our LZMA compression backend:

- We lower the preset values for all `compression_level_t` values to
  ensure that we can run the LZMA decoder with less than 65 MB of memory
  available. This seems to have a small impact on the real world usage
  and fits well with our needs.

- We set the upper bound of memory usage for the LZMA decoder to 16 MB.

See: https://bugs.torproject.org/21665
2017-04-27 20:09:20 +02:00
Alexander Færøy
e5122b91a9
Only compare the first 3 bytes when trying to detect LZMA compression.
This patch changes the logic in `detect_compression_method()` to only
use the 3 first bytes when checking if a given input is LZMA encoded.
2017-04-27 20:07:08 +02:00
Nick Mathewson
10a4f9cd07 Merge branch 'parse_accept_encoding' 2017-04-27 11:31:31 -04:00
Nick Mathewson
2903c329aa Move the "supported compression bitmask" into compress.[ch] 2017-04-27 11:30:51 -04:00
Nick Mathewson
49deb1e1b8 Document and test nul-terminating behavior of tor_uncompress()
We added this as a safety feature, but there are a few places in the
code that actually depend on it.
2017-04-27 10:59:48 -04:00
Nick Mathewson
39cfaba9e2 Fix handling of "final" flag in zstd decompression
We were returning "DONE" on truncated input streams, which was not
what we wanted.
2017-04-27 10:42:05 -04:00
Nick Mathewson
7fb9586953 Fix compilation when lzma or zstd is absent 2017-04-26 15:00:40 -04:00
Nick Mathewson
4038202f89 Avoid a warning from the use of floating-point in zstd
Replace "(preset - 0.5) * 1mb" with "preset * 1mb - 0.5 mb", to
avoid warning about converting double to size_t.
2017-04-26 14:21:45 -04:00
Nick Mathewson
be0557f759 Merge remote-tracking branch 'ahf/bugs/22066' 2017-04-26 14:20:01 -04:00
Alexander Færøy
e42c204f67
Approximate memory usage needed for the Zstandard backend.
This patch adds support for measuring the approximated memory usage by
the individual `tor_zstd_compress_state_t` object instances.

See: https://bugs.torproject.org/22066
2017-04-26 19:54:18 +02:00
Alexander Færøy
2aa28e7cb7
Better documentation for tor_uncompress().
This patch fixes the documentation string for `tor_uncompress()` to
ensure that it does not explicitly mention zlib or gzip since we now
support multiple compression backends.
2017-04-26 19:54:18 +02:00
Alexander Færøy
341824687a
Approximate memory usage needed for the LZMA backend.
This patch adds support for measuring the approximated memory usage by
the individual `tor_lzma_compress_state_t` object instances.

The LZMA library provides the functions `lzma_easy_encoder_memusage()`
and `lzma_easy_decoder_memusage()` which is used to find the estimated
usage in bytes.

See: https://bugs.torproject.org/22066
2017-04-26 19:54:18 +02:00
Nick Mathewson
99e943998d Add getpid() to the seccomp2 sandbox.
We hadn't needed this before, because most getpid() callers on Linux
were looking at the vDSO version of getpid().  I don't know why at
least one version of OpenSSL seems to be ignoring the vDSO, but this
change should fix it.

Fixes bug 21943; bugfix on 0.2.5.1-alpha when the sandbox was
introduced.
2017-04-26 12:56:06 -04:00
Sebastian Hahn
71c8974af0 Fix coverity cid 1405509
Locking in the init function is not necessary, but coverity gets
confused about it. So let's trick it.
2017-04-26 08:48:24 +02:00
Alexander Færøy
08d86e8408
Store compression overhead from tor_compress_state_t.
The `tor_compress_state_t` data-type is used as a wrapper around the
more specialized state-types used by the various compression backends.
This patch ensures that the overhead of this "thin" wrapper type is
included in the value returned by `tor_compress_get_total_allocation()`.

See: https://bugs.torproject.org/22066
2017-04-26 02:56:21 +02:00
Alexander Færøy
fac8ac0e4a
Remove unused header from the Zstandard compression backend.
Since we stopped looking at Zstandard error codes there is no need to
include the zstd_errors.h header file anymore.
2017-04-26 02:54:34 +02:00
Nick Mathewson
2655a72d89 Use x-lzma, not x-lzma2, as the identifier 2017-04-25 19:00:52 -04:00
Nick Mathewson
0274ea749a Function to convert compression methods to/from strings. 2017-04-25 16:47:46 -04:00
Nick Mathewson
43db91bd87 Teach cov-exclude to detect runaway LCOV_EXCL_START lines
Also, fix two instances of runaway LCOV_EXCL_START lines.
2017-04-25 10:59:19 -04:00
Nick Mathewson
7a37cbbea3 Whitespace fixes. 2017-04-25 10:54:34 -04:00
Nick Mathewson
49868340f7 Merge branch 'unified_compress_squashed' 2017-04-25 10:51:13 -04:00
Nick Mathewson
49a5b50b31 zlib: Turn UINT_MAX overrun into an error, not an assert. 2017-04-25 10:50:50 -04:00
Nick Mathewson
4b01b45ec1 Add a "best compression" flag. 2017-04-25 10:50:50 -04:00
Nick Mathewson
880fb3e3a9 Combine all *compress/*uncompress backend function into one
Since we have a streaming API for each compression backend, we don't
need a non-streaming API for each: we can build a common
non-streaming API at the front-end.
2017-04-25 10:50:50 -04:00
Nick Mathewson
4266ec766a Use atomic counters for compressor allocation. 2017-04-25 10:29:07 -04:00
Alexander Færøy
91dd4a00f7
Rename shadowing variable in compress_zlib.c.
This patch renames the `compress` parameter of the
`tor_zlib_compress_new()` function to `_compress` to avoid shadowing the
`compress()` function in zlib.h.
2017-04-25 14:56:55 +02:00
Nick Mathewson
ba405f86bf Merge branch 'ahf_prop278_21662_squashed' 2017-04-25 08:12:59 -04:00
Alexander Færøy
2210b330e7 Fix newlines in compression headers.
See: https://bugs.torproject.org/21662
2017-04-25 08:11:32 -04:00
Alexander Færøy
cf912259ba Remove tor_compress_memory_level().
This patch splits up `tor_compress_memory_level()` into static functions
in the individual compression backends, which allows us to tune the
values per compression backend rather than globally.

See: https://bugs.torproject.org/21662
2017-04-25 08:11:32 -04:00
Alexander Færøy
69a41e8bc6 Use switch-statement in tor_{compress,uncompress}.
Use a switch-statement in `tor_compress()` and `tor_uncompress()` for
the given `compress_method_t` parameter. This allows us to have the
compiler detect if we forgot add a handler in these functions for a
newly added enumeration value.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:10 -04:00
Alexander Færøy
6b905b38bb Add API entry-point for getting compression method version numbers.
This patch adds `tor_compress_version_str()` and
`tor_compress_header_version_str()` to get the version strings of the
different compression schema providers. Both functions returns `NULL` in
case a given `compress_method_t` is unknown or unsupported.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:10 -04:00
Alexander Færøy
1c77d8690c Add function to check if a given compression method is supported.
This patch adds support for checking if a given `compress_method_t` is
supported by the currently running Tor instance using
`tor_compress_supports_method()`.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
04682d302a Add tor_compress_get_total_allocation() function.
This patch adds the `tor_compress_get_total_allocation()` which returns
an approximate number of bytes currently in use by all the different
compression backends.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
380736d045 Add Zstandard support.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
bf1c07cb07 Add LZMA support.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:06 -04:00
Alexander Færøy
c171af0487 Use a switch-statement when checking for compression method.
This patch changes the way `tor_compress_new()`,
`tor_compress_process()`, and `tor_compress_free()` handles different
compression methods. This should give us compiler warnings in case an
additional compression method is added, but the developer forgets to add
handlers in the three aforementioned functions.

See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Alexander Færøy
300ac49685 Add compress_zlib.obj to src/common/Makefile.nmake.
See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Alexander Færøy
04583df452 Rename the torgzip module to compress.
See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Alexander Færøy
9d5bc1a935 Move zlib compression code into its own module.
This patch refactors the `torgzip` module to allow us to extend a common
compression API to support multiple compression backends.

Additionally we move the gzip/zlib code into its own module under the
name `compress_zlib`.

See https://bugs.torproject.org/21664
2017-04-25 08:06:01 -04:00
Alexander Færøy
e6c6606a17
Expose tor_compress_is_compression_bomb() as part of the public compression API.
This patch exposes the old `is_compression_bomb()` function as a public
API as part of the compression module.

See https://bugs.torproject.org/21663
2017-04-18 03:12:19 +02:00
Alexander Færøy
6b5172bcf2
Expose tor_compress_memory_level() as a public function in the compression module.
This patch makes the internal `get_memlevel()` a part of the public
compression API as `tor_compress_memory_level()`.

See https://bugs.torproject.org/21663
2017-04-18 02:23:25 +02:00
Alexander Færøy
3c4459bcbf
Refactor the streaming compression code.
This patch refactors our streaming compression code to allow us to
extend it with non-zlib/non-gzip based compression schemas.

See https://bugs.torproject.org/21663
2017-04-17 14:57:37 +02:00
Alexander Færøy
44cb86adbe
Rename tor_gzip_{compress,uncompress} to tor_{compress,uncompress}.
To allow us to use the API name `tor_compress` and `tor_uncompress` as
the main entry-point for all compression/uncompression and not just gzip
and zlib.

See https://bugs.torproject.org/21663
2017-04-17 14:29:10 +02:00
Alexander Færøy
e8b025dfc3
Rename zlib_compression_level_t to compression_level_t.
See https://bugs.torproject.org/21663
2017-04-17 14:22:13 +02:00
Alexander Færøy
7460b9755a
Remove unused function is_gzip_supported().
This patch removes the unused `is_gzip_supported()` and changes the
documentation string around the `compress_method_t` enumeration to
explicitly state that both `ZLIB_METHOD` and `GZIP_METHOD` are both
always supported.

Zlib version 1.2.0 was released on the 9'th of March, 2003 according to
their ChangeLog.

See https://bugs.torproject.org/21663
2017-04-17 14:11:35 +02:00
Nick Mathewson
b022ea32a6 Expand cpuworker API to allow other work types 2017-04-15 11:21:33 -04:00
Nick Mathewson
222122450c Add a config_line_prepend() function 2017-04-15 11:21:32 -04:00
Taylor Yu
2b38bb28b4 Remove base64_decode_nopad()
Remove base64_decode_nopad() because it is redundant now that
base64_decode() correctly handles both padded and unpadded base64
encodings with "right-sized" output buffers.
2017-04-07 18:18:23 -04:00
Taylor Yu
f15818f280 Make base64_decode() check actual decoded length
base64_decode() was applying an overly conservative check on the
output buffer length that could incorrectly produce an error if the
input encoding contained padding or newlines.  Fix this by checking
the output buffer length against the actual decoded length produced
during decoding.
2017-04-07 18:13:22 -04:00
Nick Mathewson
05ef3b959d Merge branch 'maint-0.3.0' 2017-04-07 14:03:40 -04:00
Nick Mathewson
f1613b53c5 Merge remote-tracking branch 'public/bug21894_029' into maint-0.3.0 2017-04-07 14:03:25 -04:00
Nick Mathewson
f0fa7dcdf0 Merge branch 'ticket21842_squashed' 2017-04-07 13:21:15 -04:00
Nick Mathewson
7b60f0129a Remove tor-checkkey as obsolete
CVE-2008-0166 is long gone, and we no longer need a helper tool to
dump out public key moduli so folks can detect it.

Closes ticket 21842.
2017-04-07 13:21:07 -04:00
Nick Mathewson
4812441d34 Never read off the end of a buffer in base32_encode()
When we "fixed" #18280 in 4e4a7d2b0c
in 0291 it appears that we introduced a bug: The base32_encode
function can read off the end of the input buffer, if the input
buffer size modulo 5 is not equal to 0 or 3.

This is not completely horrible, for two reasons:
   * The extra bits that are read are never actually used: so this
     is only a crash when asan is enabled, in the worst case.  Not a
     data leak.

   * The input sizes passed to base32_encode are only ever multiples
      of 5. They are all either DIGEST_LEN (20), REND_SERVICE_ID_LEN
      (10), sizeof(rand_bytes) in addressmap.c (10), or an input in
      crypto.c that is forced to a multiple of 5.

So this bug can't actually trigger in today's Tor.

Closes bug 21894; bugfix on 0.2.9.1-alpha.
2017-04-07 10:47:16 -04:00
Nick Mathewson
d33c3627f4 Comment fix. (Catalyst spotted this) 2017-04-07 10:00:08 -04:00
Nick Mathewson
61f1838cdc Merge branch 'isolate_openssl' 2017-04-07 09:58:21 -04:00
Nick Mathewson
1e54bdd48a Make the warnings about terminating nuls a bit stronger
It looks like 32_encoded_size/64_encode_size APIs are inconsistent
not only in the number of "d"s they have, but also in whether they
count the terminating NUL.  Taylor noted this in 86477f4e3f,
but I think we should note the inconsistently more loudly in order
to avoid trouble.

(I ran into trouble with this when writing 30b13fd82e243713c6a0d.)
2017-04-07 09:51:08 -04:00
Taylor Yu
86477f4e3f Use baseXX length macros in baseXX_encode()
Use the new baseXX length macros to clean up the length checks in the
baseXX_encode() functions.
2017-04-06 17:43:16 -04:00
Taylor Yu
e7f40baade Add macros for baseXX encoding lengths 2017-04-06 15:43:55 -04:00
Taylor Yu
c5adab0258 Make CEIL_DIV() slightly more overflow-safe 2017-04-06 13:54:46 -04:00
Nick Mathewson
5567a80caa Explain config_line_find() behavior on duplicates. 2017-04-06 11:48:41 -04:00
Nick Mathewson
89b673f24b Add an assertion to config_line_append(). 2017-04-06 11:48:41 -04:00
Nick Mathewson
37bb3b31fa Add some "labeled storagedir" abstractions.
These add a tiny bit of structure on top of the regular storagedir
abstractions in order to store key-value lists at the head of each
document.
2017-04-06 11:48:40 -04:00
Nick Mathewson
b6f22524ee Move config line parsing function to confline.c 2017-04-04 11:11:01 -04:00
Nick Mathewson
c2947dbb86 Move config_line_t functions from confparse.c into common.
I'm doing this to storagedir to used config_line_t.
2017-04-04 11:11:01 -04:00
Nick Mathewson
28f9b68e87 Move "change cert expiration and re-sign" fn into tortls.c
This lets test_link_handshake stop including openssl headers.
2017-03-31 10:04:45 -04:00
Nick Mathewson
5ca0d6daf0 Mark many private tortls.h APIs as openssl-only.
This change lets us remove the openssl/ssl.h include from
test_link_handshake.c.
2017-03-31 10:04:45 -04:00
Nick Mathewson
a8b9aba912 Change many tortls.h declarations of private APIs to use structs
This change makes it so those those APIs will not require prior
inclusion of openssl headers.  I've left some APIs alone-- those
will change to be extra-private.
2017-03-31 10:04:45 -04:00
Nick Mathewson
3fc4f81de3 Make crypto_ed25519.c no longer depend on openssl
Now it calls through our own crypto API.
2017-03-31 10:04:44 -04:00
Nick Mathewson
e7506c03cf Isolate dmalloc/openssl bridge code to crypto.c
This makes it so main.c, and the rest of src/or, no longer need to
include any openssl headers.
2017-03-31 10:04:44 -04:00
Nick Mathewson
10888dcac6 Merge remote-tracking branch 'ahf/bugs/21757' 2017-03-27 15:02:48 +02:00
Nick Mathewson
1d617e3ed0 Check and cast st_size to size_t in storagedir code
This prevents an i386 compilation warning and fixes bug 21828. Bug not
in any released Tor.
2017-03-27 10:40:15 +02:00
Alexander Færøy
a28be68cf2
Split strings at newline in tor_get_lines_from_handle().
This patch fixes a regression described in bug #21757 that first
appeared after commit 6e78ede73f which was an attempt to fix bug #21654.

When switching from buffered I/O to direct file descriptor I/O our
output strings from get_string_from_pipe() might contain newline
characters (\n). In this patch we modify tor_get_lines_from_handle() to
ensure that the function splits the newly read string at the newline
character and thus might return multiple lines from a single call to
get_string_from_pipe().

Additionally, we add a test case to test_util_string_from_pipe() to
ensure that get_string_from_pipe() correctly returns multiple lines in a
single call.

See: https://bugs.torproject.org/21757
See: https://bugs.torproject.org/21654
2017-03-17 04:27:12 +01:00
Nick Mathewson
45d008c4aa Merge branch 'storagedir_squashed' 2017-03-16 16:03:13 -04:00
Nick Mathewson
ee253e392a Define a "storagedir" abstraction to hold numerous similar files
We could use one of these for holding "junk" descriptors and
unparseable things -- but we'll _need_ it for having cached
consensuses and diffs between them.
2017-03-16 16:01:59 -04:00
Nick Mathewson
877468c781 Now, resolve a new 64-bit warning in compat_threads.c 2017-03-16 15:34:57 -04:00
Nick Mathewson
4a4f1e44af Merge branch 'prop140_21643_diff_only_squashed' 2017-03-16 14:58:43 -04:00
Daniel Martí
590ffdb2c9 Consensus diff backend from Daniel Martí GSOC project.
(This commit was extracted by nickm based on the final outcome of
the project, taking only the changes in the files touched by this
commit from the consdiff_rebased branch.  The directory-system
changes are going to get worked on separately.)
2017-03-16 14:38:28 -04:00
Nick Mathewson
7505f452c8 Run the copyright update script. 2017-03-15 16:13:17 -04:00
Nick Mathewson
c64adaaf90 Merge branch 'bug21540_029_v2' 2017-03-15 12:17:15 -04:00
Nick Mathewson
44514058b9 Correctly handle fd-drain errors on windows workqueues
Windows doesn't let you check the socket error for a socket with
WSAGetLastError() and getsockopt(SO_ERROR).  But
getsockopt(SO_ERROR) clears the error on the socket, so you can't
call it more than once per error.

When we introduced recv_ni to help drain alert sockets, back in
0.2.6.3-alpha, we had the failure path for recv_ni call getsockopt()
twice, though: once to check for EINTR and one to check for EAGAIN.
Of course, we never got the eagain, so we treated it as an error,
and warned about: "No error".

The fix here is to have these functions return -errno on failure.

Fixes bug 21540; bugfix on 0.2.6.3-alpha.
2017-03-15 12:16:17 -04:00
Nick Mathewson
c000c7d118 Create logfiles in mode 0640
Patch from toralf; closes 21729.
2017-03-14 11:36:53 -04:00
Nick Mathewson
dc37ea8b98 Add a null statement to prevent some SMARTLIST_FOREACH() bugs.
So we require that SMARTLIST_FOREACH_END() have the name of the loop
variable in it.  But right now the only enforcement for that is to
clear the variable at the end of the loop, which is really not
sufficient: I spent 45 minutes earlier today debugging an issue
where I had said:
   SMARTLIST_FOREACH_BEGIN(spool, spooled_resource_t *, spooled) {
     ...
   } SMARTLIST_FOREACH_END(spool);

This patch makes it so that ONLY loop variables can be used, by
referring to the _sl_idx variable.
2017-03-13 16:43:30 -04:00
Alexander Færøy
02fc0a5ecf
Remove fgets() compatbility function and related tests.
This patch removes the `tor_fgets()` wrapper around `fgets(3)` since it
is no longer needed. The function was created due to inconsistency
between the returned values of `fgets(3)` on different versions of Unix
when using `fgets(3)` on non-blocking file descriptors, but with the
recent changes in bug #21654 we switch from unbuffered to direct I/O on
non-blocking file descriptors in our utility module.

We continue to use `fgets(3)` directly in the geoip and dirserv module
since this usage is considered safe.

This patch also removes the test-case that was created to detect
differences in the implementation of `fgets(3)` as well as the changes
file since these changes was not included in any releases yet.

See: https://bugs.torproject.org/21654
2017-03-09 00:10:18 +01:00
Alexander Færøy
02ef06516e
Use less-than instead of not-equal-to for comparison in read loops.
This patch changes a number of read loops in the util module to use
less-than comparison instead of not-equal-to comparison. We do this in
the case that we have a bug elsewhere that might cause `numread` to
become larger than `count` and thus become an infinite loop.
2017-03-09 00:10:17 +01:00
Alexander Færøy
6e78ede73f
Remove buffered I/O stream usage in process_handle_t.
This patch removes the buffered I/O stream usage in process_handle_t and
its related utility functions. This simplifies the code and avoids racy
code where we used buffered I/O on non-blocking file descriptors.

See: https://bugs.torproject.org/21654
2017-03-09 00:10:17 +01:00
Alexander Færøy
86de065aee
Use read(2) instead of fgets(3) when reading process output.
This patch modifies `tor_read_all_handle()` to use read(2) instead of
fgets(3) when reading the stdout from the child process. This should
eliminate the race condition that can be triggered in the 'slow/util/*'
tests on slower machines running OpenBSD, FreeBSD and HardenedBSD.

See: https://bugs.torproject.org/21654
2017-03-08 02:08:44 +01:00
Alexander Færøy
3dca5a6e71
Use tor_fgets() instead of fgets().
This patch changes our use of fgets() to tor_fgets() for more consistent
error handling across different versions of the C library.
2017-03-01 21:26:27 +01:00
Alexander Færøy
498287b3c3
Add compatibility function for fgets(3).
This patch adds the `tor_fgets()` function to our compatibility layer.

`tor_fgets()` adds an additional check for whether the error-bit have
been enabled for the given file stream, if that is the case and `errno`
is set to `EAGAIN` we make sure that we always return NULL.

Unfortunately `fgets(3)` behaves differently on different versions of
the C library.

See: https://bugs.torproject.org/21416
See: https://bugs.torproject.org/20988
2017-03-01 21:26:26 +01:00
Nick Mathewson
c0aa7ac5ac Merge branch 'disable_memory_sentinels_squashed' 2017-02-27 16:25:25 -05:00
Nick Mathewson
b923c4dc9f Code to disable memory sentinels for fuzzing
This feature makes it possible to turn off memory sentinels (like
those used for safety in buffers.c and memarea.c) when fuzzing, so
that we can catch bugs that they would otherwise prevent.
2017-02-27 16:25:10 -05:00
Nick Mathewson
6747c62386 Merge branch 'bug21420_029_squashed' into maint-0.3.0 2017-02-27 11:20:39 -05:00
Nick Mathewson
f6e5a658df Revise the logic for picking the start time for link certs
Since 0.2.4.11-alpha (in 0196647970) we've tried to randomize
the start time to up to some time in the past.  But unfortunately we
allowed the start time to be in the future as well, which isn't
really legit.

The new behavior lets the start time be be up to
MAX(cert_lifetime-2days, 0) in the past, but never in the future.

Fixes bug 21420; bugfix on 0.2.4.11-alpha.
2017-02-27 11:19:54 -05:00
Nick Mathewson
7e469c1002 Merge branch 'bug20894_029_v3' 2017-02-14 19:10:20 -05:00
Nick Mathewson
c4f2faf301 Don't atoi off the end of a buffer chunk.
Fixes bug 20894; bugfix on 0.2.0.16-alpha.

We already applied a workaround for this as 20834, so no need to
freak out (unless you didn't apply 20384 yet).
2017-02-14 16:38:47 -05:00
Nick Mathewson
e91bb84a91 Merge branch 'maint-0.2.6' into maint-0.2.7-redux
maint-0.2.7-redux is an attempt to try to re-create a plausible
maint-0.2.7 branch.  I've started from the tor-0.2.7.6, and then I
merged maint-0.2.6 into the branch.

This has produced 2 conflicts: one related to the
rendcommon->rendcache move, and one to the authority refactoring.
2017-02-07 09:59:12 -05:00
Nick Mathewson
e6965f78b8 Merge branch 'maint-0.2.5' into maint-0.2.6 2017-02-07 08:54:54 -05:00
Nick Mathewson
6b37512dc7 Merge branch 'maint-0.2.4' into maint-0.2.5 2017-02-07 08:54:47 -05:00
Nick Mathewson
d6eae78e29 Merge remote-tracking branch 'public/bug19152_024_v2' into maint-0.2.4 2017-02-07 08:47:11 -05:00
Nick Mathewson
8936c50d83 Merge branch 'maint-0.2.5' into maint-0.2.6 2017-02-07 08:39:07 -05:00
Nick Mathewson
05ec055c41 Merge branch 'maint-0.2.4' into maint-0.2.5 2017-02-07 08:38:59 -05:00
Nick Mathewson
51675f97d3 Merge remote-tracking branch 'public/bug17404_024' into maint-0.2.4 2017-02-07 08:37:07 -05:00
Nick Mathewson
332543baed Merge branch 'maint-0.2.4' into maint-0.2.5 2017-02-07 08:34:08 -05:00
Nick Mathewson
6cb8c0fd4e Refine the memwipe() arguments check for 18089 a little more.
We still silently ignore
     memwipe(NULL, ch, 0);
and
     memwipe(ptr, ch, 0);  /* for ptr != NULL */

But we now assert on:
     memwipe(NULL, ch, 30);
2017-02-07 08:33:51 -05:00
teor (Tim Wilson-Brown)
fb7d1f41b4 Make memwipe() do nothing when passed a NULL pointer or zero size
Check size argument to memwipe() for underflow.

Closes bug #18089. Reported by "gk", patch by "teor".
Bugfix on 0.2.3.25 and 0.2.4.6-alpha (#7352),
commit 49dd5ef3 on 7 Nov 2012.
2017-02-07 08:33:39 -05:00
cypherpunks
27df23abb6 Use the standard OpenBSD preprocessor definition 2017-02-03 09:37:39 -05:00
Nick Mathewson
558c04f5b1 Merge branch 'combined-fuzzing-v4' 2017-01-30 08:40:46 -05:00
Nick Mathewson
e2aeaeb76c Make a bunch of signature/digest-checking functions mockable 2017-01-30 08:37:22 -05:00
Nick Mathewson
67eb6470d7 Merge branches 'server_ciphers' and 'ciphers.inc' 2017-01-27 16:45:18 -05:00
Nick Mathewson
ad382049ed Fix a signed/unsigned comparison warning 2017-01-25 13:23:08 -05:00
junglefowl
373d9aff7a Fail if file is too large to mmap.
If tor_mmap_file is called with a file which is larger than SIZE_MAX,
only a small part of the file will be memory-mapped due to integer
truncation.

This can only realistically happen on 32 bit architectures with large
file support.
2017-01-25 13:21:44 -05:00
Nick Mathewson
363be43df3 Re-run gen_server_ciphers 2017-01-24 15:30:35 -05:00
Nick Mathewson
4f1dc34e36 Regenerate ciphers.inc 2017-01-24 15:05:35 -05:00
Nick Mathewson
2e656d19e5 comment fix from pastly 2017-01-11 15:12:59 -05:00
Nick Mathewson
ac3b559e93 Merge branch 'bug20569_030_02_squashed' 2017-01-11 12:52:52 -05:00
David Goulet
870b5e2227 hs: Use AES-256 for v3 descriptor
That key size is taken from proposal 224 thus specified in the protocol.

Closes #20569

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-01-11 12:52:34 -05:00
Nick Mathewson
4f1a5da473 Merge remote-tracking branch 'public/ticket21037' 2017-01-11 09:17:21 -05:00
Nick Mathewson
f23ec14d62 Make tor_addr_lookup() mockable. 2017-01-03 10:11:15 -05:00
Nick Mathewson
6aac6c6bee Make ed25519_fmt() log 0-valued keys more nicely.
Because <unset> makes more sense than AAAAAAAAAAAAAAAAAAA...

(I have indeed verified that ed25519_fmt() is only used for
logging. This patch also clarifies the intention that ed25519_fmt()
is only for logging.

Closes ticket 21037.
2017-01-02 12:31:15 -05:00
Nick Mathewson
fda7f00321 Note memset as redundant; see 20764. 2016-12-23 09:59:41 -05:00
Nick Mathewson
cea1a4e19d This is no longer inline. 2016-12-23 09:50:44 -05:00
Hans Jerry Illikainen
a23fd15786 Fix unreachable heap corruption in base64_decode()
Give size_mul_check() external linkage and use it in base64_decode() to
avoid a potential integer wrap.

Closes #19222
2016-12-23 09:47:09 -05:00
Nick Mathewson
a6eaa121e4 Merge branch 'maint-0.2.9' 2016-12-23 08:06:19 -05:00
Nick Mathewson
a757f76967 Withstand failures in CLOCK_MONOTONIC_COARSE
This came up on #21035, where somebody tried to build on a linux
system with kernel headers including CLOCK_MONOTONIC_COARSE, then
run on a kernel that didn't support it.

I've adopted a belt-and-suspenders approach here: we detect failures
at initialization time, and we also detect (loudly) failures later on.

Fixes bug 21035; bugfix on 0.2.9.1-alpha when we started using
monotonic time.
2016-12-21 08:17:26 -05:00
Roger Dingledine
51ee549a90 fix typos and trivial syntax problems 2016-12-18 04:06:02 -05:00
Roger Dingledine
f3d056ab16 clarify debug-level log while initializing entropy
I got confused when I saw my Tor saying it was opening a file
that doesn't exist. It turns out it isn't opening it, it's just
calling open() on it and then moving on when it's not there.
2016-12-18 03:48:31 -05:00
Nick Mathewson
2a00110e5b Revert "Stop checking whether environ is declared."
This reverts commit 954eeda619.

Apparently, OpenBSD is what expects you to declare environ
yourself.  So 19142 is a wontfix.
2016-12-16 12:16:52 -05:00
Nick Mathewson
2cee38f76a Merge branch 'prop271_030_v1_squashed' 2016-12-16 11:20:59 -05:00
Nick Mathewson
1d52ac4d3f Lay down some infrastructure for bridges in the New Guard Order.
This includes:
  * making bridge_info_t exposed but opaque
  * allowing guards where we don't know an identity
  * making it possible to learn the identity of a guard
  * creating a guard that lacks a node_t
  * remembering a guard's address and port.
  * Looking up a guard by address and port.
  * Only enforcing the rule that we need a live consensus to update
    the "listed" status for guards when we are not using bridges.
2016-12-16 11:06:18 -05:00
Nick Mathewson
df6c475e59 Merge remote-tracking branch 'public/ticket19142' 2016-12-16 10:27:27 -05:00
Nick Mathewson
c838d34921 Merge branch 'dgoulet_ticket19043_030_03_squashed' 2016-12-14 15:28:28 -05:00
David Goulet
118691cd47 crypto: Change crypto_mac_sha3_256 to use the key length in the construction
Signed-off-by: David Goulet <dgoulet@torproject.org>
2016-12-14 15:18:40 -05:00
George Kadianakis
297213825b prop224: Add unittests handling v3 ESTABLISH_INTRO cells.
Test for both v2 and v3 ESTABLISH_INTRO handling.
2016-12-14 15:18:40 -05:00
George Kadianakis
b9010c8bf5 prop224 prepwork: Introduce HMAC-SHA3 function. 2016-12-14 15:17:57 -05:00
Nick Mathewson
954eeda619 Stop checking whether environ is declared.
There seems to be pretty good evidence that it's always declared,
and that checking for it is pointless.

Closes ticket 19142.
2016-12-12 10:55:10 -05:00
Nick Mathewson
b659ffe9ac Merge remote-tracking branch 'jryans/log-severity' 2016-12-12 09:46:07 -05:00
Nick Mathewson
e1f00c5f86 whitespace cleanups 2016-12-08 16:53:29 -05:00
Nick Mathewson
e93234af70 Merge branch 'feature15056_v1_squashed' 2016-12-08 16:49:24 -05:00
Nick Mathewson
937aef48ee Add an ed25519_copy; use it in a couple of places dgoulet suggested. 2016-12-08 16:48:01 -05:00
Nick Mathewson
2cdd24ddd6 Helper function for logging ed25519 public keys. 2016-12-08 16:47:58 -05:00
Nick Mathewson
10baf2c684 Backport the other sierra fix in 20865.
They added clock_gettime(), but with tv_nsec as a long, whereas
tv_usec is a __darwin_suseconds_t (a.k.a. 'int').  Now, why would
they do that? Are they preparing for a world where there are more
than 2 billion nanoseconds per second?  Are they planning for having
int be less than 32 bits again?  Or are they just not paying
attention to the Darwin API?

Also, they forgot to mark clock_gettime() as Sierra-only, so even
if we fixed the issue here, we'd still be stick with portability
breakage like we were for 0.2.9.

So, just disable clock_gettime() on apple.
2016-12-07 18:24:28 -05:00
Nick Mathewson
129cee1c75 Merge branch 'maint-0.2.9' 2016-12-07 10:52:28 -05:00
Nick Mathewson
d6ca36defa Merge branch 'bug20710_025' into maint-0.2.9 2016-12-07 10:52:12 -05:00
J. Ryan Stinnett
9b2b799d82 Accept non-space whitespace characters in log severity syntax.
Adds a test_config_parse_log_severity unit test to verify behavior.

Fixes #19965.
2016-12-06 11:11:43 -10:00
Nick Mathewson
daeb633825 whitespace fix 2016-12-05 10:31:10 -05:00
Nick Mathewson
f92630941a Merge remote-tracking branch 'chelseakomlo/20717_hashing_api_bug' 2016-12-05 10:27:16 -05:00
Nick Mathewson
cc34ba1cec Merge branch 'getentropy_028' into maint-0.2.8 2016-12-05 10:06:16 -05:00
Nick Mathewson
714aeedc52 20865: Don't use getentropy() on OSX Sierra.
Tor 0.2.9 has a broader range of fixes and workarounds here, but for
0.2.8, we're just going to maintain the existing behavior.

(The alternative would be to backport both
1eba088054 and
16fcbd21c9 , but the latter is kind of
a subtle kludge in the configure.ac script, and I'm not a fan of
backporting that kind of thing.)
2016-12-05 10:02:33 -05:00
Nick Mathewson
2a365413eb Always Use EVP_aes_*_ctr() with openssl 1.1
(OpenSSL 1.1 makes EVP_CIPHER_CTX opaque, _and_ adds acceleration
for counter mode on more architectures.  So it won't work if we try
the older approach, and it might help if we try the newer one.)

Fixes bug 20588.
2016-12-05 07:54:22 -05:00
J. Ryan Stinnett
7ffa95abd9 Clarify that ClientRejectInternalAddresses also rejects mDNS *.local hosts
Fixes #17070.
2016-12-03 21:10:40 -06:00
Nick Mathewson
e6facbfe7a Add accessor for inspecting timer callbacks. 2016-12-02 12:15:07 -05:00
Nick Mathewson
6a069959c6 Fix major errors in freeing getaddrinfo sandbox cache
Patch from cypherpunks. Fixes bug 20710; bugfix on 0.2.5.5-alpha.
2016-12-01 10:36:02 -05:00
teor
1e8f68a9c7 Add an extra warning message to check_private_dir 2016-12-01 09:51:19 -05:00
Nick Mathewson
21c47c4410 Add a smartlist_remove_keeporder() function, with tests. 2016-11-30 14:42:52 -05:00
Nick Mathewson
bf64564e37 Add a GUARD log domain, for use with new guards code 2016-11-30 14:42:52 -05:00
Nick Mathewson
539eba0a4b Teach parse_iso_time about the spaceless variant.
(We previously added support for generating the spaceless
2016-11-14T19:58:12 variant, but not for actually parsing it.)
2016-11-30 14:42:52 -05:00
Chelsea H. Komlo
e01b09d5ce
crypto_digest512 returns expected error value of -1 2016-11-24 12:14:54 -05:00
Chelsea H. Komlo
9d9110f65d
crypto_digest256 returns expected error value of -1 2016-11-24 12:13:07 -05:00
Chelsea H. Komlo
276d07a88a
crypto_digest returns expected error value of -1 2016-11-24 10:01:03 -05:00
Nick Mathewson
4614f8e681 Merge remote-tracking branch 'teor/fix-mingw-pagesize' 2016-11-22 18:29:50 -05:00
Fabian Keil
db2dd8434e finish_writing_to_file_impl(): Remove temporary file if replacing the existing one failed
Signed-off-by: David Goulet <dgoulet@torproject.org>
2016-11-16 11:25:26 -05:00
Nick Mathewson
431565e053 Helper function to see if an ed25519 pk is set. 2016-11-10 09:43:27 -05:00
Nick Mathewson
31f41fe096 Merge branch 'maint-0.2.9' 2016-11-08 18:45:10 -05:00
Nick Mathewson
286fa94064 Use va_copy() in pure-windows version of tor_asprintf().
It's not okay to use the same varargs list twice, and apparently
some windows build environments produce code here that would leave
tor_asprintf() broken. Fix for bug 20560; bugfix on 0.2.2.11-alpha
when tor_asprintf() was introduced.
2016-11-08 18:44:06 -05:00
Nick Mathewson
3e3040a5d9 Merge branch 'maint-0.2.9'
Conflicts:
	src/or/rendservice.c
2016-11-07 16:31:40 -05:00
Nick Mathewson
c2fc0941a5 Merge remote-tracking branch 'teor/bug20484_029_v2' into maint-0.2.9 2016-11-07 16:12:13 -05:00
Nick Mathewson
a415fee58a Merge branch 'maint-0.2.9' 2016-11-07 09:09:06 -05:00
Nick Mathewson
0bd55ed96a Always Use EVP_aes_*_ctr() with openssl 1.1
(OpenSSL 1.1 makes EVP_CIPHER_CTX opaque, _and_ adds acceleration
for counter mode on more architectures.  So it won't work if we try
the older approach, and it might help if we try the newer one.)

Fixes bug 20588.
2016-11-06 21:01:25 -05:00
Nick Mathewson
59f4cae68c Merge branch 'maint-0.2.8' into maint-0.2.9 2016-11-03 18:36:43 -04:00
Nick Mathewson
61bdc452b0 Merge branch 'bug20551_028' into maint-0.2.8 2016-11-03 18:36:25 -04:00
Nick Mathewson
3bb49c0110 Merge branch 'maint-0.2.8' into maint-0.2.9 2016-11-03 15:41:04 -04:00
Nick Mathewson
957bdc4a42 Merge branch 'bug20553_028' 2016-11-03 10:52:21 -04:00
Nick Mathewson
9b18b215bb Work around a behavior change in openssl's BUF_MEM code
In our code to write public keys to a string, for some unfathomable
reason since 253f0f160e, we would allocate a memory BIO, then
set the NOCLOSE flag on it, extract its memory buffer, and free it.
Then a little while later we'd free the memory buffer with
BUF_MEM_free().

As of openssl 1.1 this doesn't work any more, since there is now a
BIO_BUF_MEM structure that wraps the BUF_MEM structure.  This
BIO_BUF_MEM doesn't get freed in our code.

So, we had a memory leak!

Is this an openssl bug?  Maybe.  But our code was already pretty
silly.  Why mess around with the NOCLOSE flag here when we can just
keep the BIO object around until we don't need the buffer any more?

Fixes bug 20553; bugfix on 0.0.2pre8
2016-11-03 10:51:10 -04:00
Nick Mathewson
1eef543f9d Merge branch 'bug20551_028' 2016-11-03 09:37:44 -04:00
Nick Mathewson
464783a8dc Use explicit casts to avoid warnings when building with openssl 1.1
fixes bug 20551; bugfix on 0.2.1.1-alpha
2016-11-03 09:35:41 -04:00
Nick Mathewson
d9ca4e20bd Merge branch 'feature_15055_v2' 2016-11-03 08:44:46 -04:00
Nick Mathewson
f156156d56 Audit use of tor_tls_cert_get_key().
This function is allowed to return NULL if the certified key isn't
RSA. But in a couple of places we were treating this as a bug or
internal error, and in one other place we weren't checking for it at
all!

Caught by Isis during code review for #15055.  The serious bug was
only on the 15055 branch, thank goodness.
2016-11-03 08:40:11 -04:00
Nick Mathewson
70e7d28b3e Generate our x509 certificates using sha256, not sha1.
All supported Tors (0.2.4+) require versions of openssl that can
handle this.

Now that our link certificates are RSA2048, this might actually help
vs fingerprinting a little.
2016-11-03 08:40:10 -04:00
Nick Mathewson
67e66898d2 For testing: add a tor_x509_cert_dup(). 2016-11-03 08:39:31 -04:00
Nick Mathewson
e64bac6eb4 Increase TLS RSA link key length to 2048 bits
Oddly, nothing broke.

Closes ticket 13752.
2016-11-03 08:39:30 -04:00
Nick Mathewson
0b4221f98d Make the current time an argument to x509 cert-checking functions
This makes the code a bit cleaner by having more of the functions be
pure functions that don't depend on the current time.
2016-11-03 08:37:22 -04:00
Nick Mathewson
b004ff45d7 New authentication types to use RFC5705.
See proposal 244.  This feature lets us stop looking at the internals
of SSL objects, *and* should let us port better to more SSL libraries,
if they have RFC5705 support.

Preparatory for #19156
2016-11-03 08:37:20 -04:00
teor
8f465808a0
Check for getpagesize before using it to mmap files
This fixes compilation in some MinGW environments.

Fixes bug 20530; bugfix on commit bf72878 in tor-0.1.2.1-alpha.
Reported by "ice".
2016-11-03 08:44:57 +11:00
teor
2f48693663
Improve comments in check_private_dir and onion poisoning
Comment changes only
2016-11-02 14:11:26 +11:00
Nick Mathewson
68a27dad43 Merge branch 'maint-0.2.9' 2016-10-31 16:33:12 -04:00
Nick Mathewson
becc957839 Actually clamp the number of detected CPUs to 16.
Previously we said we did, but didn't.

Fixes #19968; bugfix on 0.2.3.1-alpha.
2016-10-31 14:19:39 -04:00
overcaffeinated
265d5446fa Automated change to use smartlist_add_strdup
Use the following coccinelle script to change uses of
smartlist_add(sl, tor_strdup(str)) to
smartlist_add_strdup(sl, string) (coccinelle script from nickm
via bug 20048):

@@
expression a;
expression b;
@@
- smartlist_add
+ smartlist_add_strdup
   (a,
- tor_strdup(
   b
- )
  )
2016-10-27 10:26:06 +01:00
overcaffeinated
b8b8b6b70e Add implementation of smartlist_add_strdup
Add smartlist_add_strdup(sl, string) - replaces the use of
smartlist_add(sl, tor_strdup(string)). Fixes bug 20048.
2016-10-27 10:12:28 +01:00
Nick Mathewson
f3174428e2 Fix a syntax problem 2016-10-17 10:25:13 -04:00
Nick Mathewson
aae034d13e Write a bunch of module documentation.
This commit adds or improves the module-level documenation for:

  buffers.c circuitstats.c command.c connection_edge.c control.c
  cpuworker.c crypto_curve25519.c crypto_curve25519.h
  crypto_ed25519.c crypto_format.c dircollate.c dirserv.c dns.c
  dns_structs.h fp_pair.c geoip.c hibernate.c keypin.c ntmain.c
  onion.c onion_fast.c onion_ntor.c onion_tap.c periodic.c
  protover.c protover.h reasons.c rephist.c replaycache.c
  routerlist.c routerparse.c routerset.c statefile.c status.c
  tor_main.c workqueue.c

In particular, I've tried to explain (for each documented module)
what each module does, what's in it, what the big idea is, why it
belongs in Tor, and who calls it.  In a few cases, I've added TODO
notes about refactoring opportunities.

I've also renamed an argument, and fixed a few DOCDOC comments.
2016-10-17 10:16:59 -04:00
Nick Mathewson
af70e43131 Merge remote-tracking branch 'public/spaces_in_unix_addrs' 2016-10-14 10:21:41 -04:00
Nick Mathewson
2e7e635c59 Switch from "AF_UNIX is always equal" to "always unequal" to avoid wacky bugs. See discussion on 20261 2016-10-11 11:11:21 -04:00
Nick Mathewson
d25fed5174 Merge remote-tracking branch 'yawning-schwanenlied/bug20261' 2016-10-11 11:08:20 -04:00
paolo.ingls@gmail.com
ab78a4df93 torrc parsing b0rks on carriage-return
(Specifically, carriage return after a quoted value in a config
line. Fixes bug 19167; bugfix on 0.2.0.16-alpha when we introduced
support for quoted values. Unit tests, changes file, and this
parenthetical by nickm.)
2016-10-11 09:25:22 -04:00
Yawning Angel
7b2c856785 Bug 20261: Treat AF_UNIX addresses as equal when comparing them.
This is a kludge to deal with the fact that `tor_addr_t` doesn't contain
`sun_path`.  This currently ONLY happens when circuit isolation is being
checked, for an isolation mode that is force disabled anyway, so the
kludge is "ugly but adequate", but realistically, making `tor_addr_t`
and the AF_UNIX SocksPort code do the right thing is probably the better
option.
2016-10-10 20:57:45 +00:00
Nick Mathewson
850ec1e282 Stop implying that we support openssl 1.0.0; we don't.
Closes ticket 20303.

The LIBRESSL_VERSION_NUMBER check is needed because if our openssl
is really libressl, it will have an openssl version number we can't
really believe.
2016-10-06 12:58:49 -04:00
Nick Mathewson
05aed5b635 Allow a unix: address to contain a C-style quoted string.
Feature 18753 -- all this to allow spaces.
2016-10-04 15:43:20 -04:00
cypherpunks
3b2f012e28 Avoid reordering IPv6 interface addresses
When deleting unsuitable addresses in get_interface_address6_list(), to
avoid reordering IPv6 interface addresses and keep the order returned by
the OS, use SMARTLIST_DEL_CURRENT_KEEPORDER() instead of
SMARTLIST_DEL_CURRENT().

This issue was reported by René Mayrhofer.

[Closes ticket 20163; changes file written by teor. This paragraph
added by nickm]
2016-10-03 13:50:27 -04:00
Nick Mathewson
a633baf632 Merge branch 'osx_sierra_028' 2016-09-24 13:33:09 -07:00
Nick Mathewson
951638a06d Fix pthread_cond_timedwait() on OSX Sierra
Sierra provides clock_gettime(), but not pthread_condattr_setclock.
So we had better lot try to use CLOCK_MONOTONIC as our source for
time when waiting, since we ccan never actually tell the condition
that we mean CLOCK_MONOTONIC.

This isn't a tor bug yet, since we never actually pass a timeout to
tor_cond_wait() outside of the unit tests.
2016-09-24 09:12:00 -07:00
Nick Mathewson
1eba088054 Fix compilation on OSX Sierra (10.12) 2016-09-24 08:48:47 -07:00
Nick Mathewson
4c69ba5895 Fix conflicting types errors for aes.c. 2016-09-22 08:52:42 -04:00
Nick Mathewson
6cb9c2cf77 Add support for AES256 and AES192
(This will be used by prop224)
2016-09-16 11:21:33 -04:00
Nick Mathewson
83129031b1 Remove a needless level of indirection from crypto_cipher_t
Now that crypto_cipher_t only contains a pointer, it no longer
has any reason for an independent existence.
2016-09-16 10:20:08 -04:00
Nick Mathewson
ff116b7808 Simplify the crypto_cipher_t interface and structure
Previously, the IV and key were stored in the structure, even though
they mostly weren't needed.  The only purpose they had was to
support a seldom-used API where you could pass NULL when creating
a cipher in order to get a random key/IV, and then pull that key/IV
back out.

This saves 32 bytes per AES instance, and makes it easier to support
different key lengths.
2016-09-16 10:12:30 -04:00
Nick Mathewson
981d0a24b8 In aes.c, support 192-bit and 256-bit keys.
Also, change the input types for aes_new_cipher to be unsigned,
as they should have been all along.
2016-09-16 09:51:51 -04:00
Nick Mathewson
b88f918227 Remove an extraneous parenthesis in IF_BUG_OHNCE__
Fixes bug 20141; bugfix on 0.2.9.1-alpha.

Patch from Gisle Vanem.
2016-09-14 10:53:49 -04:00
Nick Mathewson
b488bd54ba Merge remote-tracking branch 'public/bug20063' 2016-09-13 11:25:59 -04:00
Nick Mathewson
4b182dfc23 Merge remote-tracking branch 'public/ticket19998' 2016-09-13 08:54:43 -04:00
Nick Mathewson
64521a9d35 Merge remote-tracking branch 'public/solaris_warnings_028' 2016-09-11 16:52:24 -04:00
Nick Mathewson
4c55e8a58f Fix cases where the tests were doing closesocket() on a non-socket
These seem to have caused warnings on windows. Hmmm.
2016-09-09 10:28:12 -04:00
Nick Mathewson
5e30e26c6d Chop another ~93 RSA key generations out of the unit tests
We have a mock for our RSA key generation function, so we now wire
it to pk_generate(). This covers all the cases that were not using
pk_generate() before -- all ~93 of them.
2016-09-09 09:45:50 -04:00
Nick Mathewson
3269307daf Treat all nonfatal assertion failures as unit test failures.
Part of 19999.
2016-09-08 13:27:30 -04:00
Nick Mathewson
d09723ad19 Add facility to suppress/capture tor_bug_occurred_() messages in unit tests. 2016-09-06 21:01:17 -04:00
Nick Mathewson
2b39c927c7 Add !(...) to BUG() log messages
They use the same code as reporting assertion failures, so we should
invert the sense of what we're "asserting".

Fixes bug 20093.
2016-09-06 21:00:51 -04:00
Nick Mathewson
5927ed8d33 checkSpace.pl now forbids more identifiers.
The functions it warns about are:
  assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
  strdup, strndup, calloc.

Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
2016-09-06 12:35:37 -04:00
teor (Tim Wilson-Brown)
41ad244dd6
Fix a comment typo in smartlist_get_most_frequent_() 2016-09-06 17:49:44 +10:00
Nick Mathewson
b9a43c8f43 For me, asan/ubsan require more syscalls.
Permit sched_yield() and sigaltstack() in the sandbox.

Closes ticket 20063
2016-09-05 14:25:58 -04:00
Nick Mathewson
c2d1356739 Change servers to never pick 3DES.
Closes ticket 19998.
2016-09-05 14:09:14 -04:00
Nick Mathewson
251b348d7b It is not a bug to attempt to base32_decode a non-base32 string
(Rationale: it isn't a bug to try this for a base16 or base64
string. It's just a bad input that we're detecting.)
2016-08-31 14:31:00 -04:00
Nick Mathewson
69dce09031 Do not call tor_tls_server_info_callback(NULL) from tests.
This isn't valid behavior, and it causes a crash when you run
the unit tests at --debug.

I've added an IF_BUG_ONCE() check for this case.
2016-08-31 13:18:13 -04:00
Nick Mathewson
f74916a98f setup_capture_of_logs: no longer suppress log messages
Previously setup_capture_of_logs would prevent log messages from
going to the console entirely.  That's a problem, since sometimes
log messages are bugs!  Now setup_capture_of_logs() acts sensibly.

If you really do need to keep a message from going to the console
entirely, there is setup_full_capture_of_logs().  But only use that
if you're prepared to make sure that there are no extraneous
messages generated at all.
2016-08-31 12:51:22 -04:00
Nick Mathewson
40d05983c4 Fix some comments in sandbox.c
Closes ticket 19942; patch from "cypherpunks"
2016-08-23 10:02:11 -04:00
Nick Mathewson
a3d419634b Merge remote-tracking branch 'asn/bug19872_v2' 2016-08-23 08:50:32 -04:00
George Kadianakis
b8bfdf638e Introduce ed25519_{sign,checksig}_prefixed functions(). 2016-08-23 14:53:01 +03:00
Nick Mathewson
8f2f06c9b3 Merge branch 'maint-0.2.8' 2016-08-19 19:35:39 -04:00
Nick Mathewson
49843c980a Avoid confusing GCC 4.2.1 by saying "int foo()... inline int foo() {...}"
Fixes bug 19903; bugfix on 0.2.8.1-alpha.
2016-08-19 19:34:39 -04:00
Nick Mathewson
7f145b54af Merge remote-tracking branch 'public/Fix_19450' 2016-08-12 16:11:28 -04:00
cypherpunks
8d67c079b4 Fix integer overflows in the conversion tables 2016-08-12 14:18:01 -04:00
Nick Mathewson
e788c577f1 Only use evutil_secure_rng_add_bytes() when present.
OpenBSD removes this function, and now that Tor requires Libevent 2,
we should also support the OpenBSD Libevent 2.

Fixes bug 19904; bugfix on 0.2.5.4-alpha.
2016-08-11 20:37:18 -04:00
Nick Mathewson
60997a00e8 Fix a bug in the old-gcc version of ENABLE_GCC_WARNING
Fixes bug 19901; bugfix on 0.2.9.1-alpha.
2016-08-11 19:58:13 -04:00
Nick Mathewson
4d4ccc505b Search for remaining references to 'bufferevent'.
Remove or adjust as appropriate.
2016-08-02 13:59:47 -04:00
Nick Mathewson
88a7a02728 Bufferevent removal: remove more bufferevent-only options
(All this IOCP stuff was bufferevent-only.)
2016-08-02 13:33:08 -04:00
Nick Mathewson
8fd6b0fc46 Remove USE_BUFFEREVENTS code outside src/or 2016-08-02 13:22:06 -04:00
Andrea Shepard
1995328a3d Keep make check-spaces happy 2016-07-29 05:05:12 +00:00
Nick Mathewson
dffc6910b1 Three more -Wshadow fixes. 2016-07-28 11:24:03 -04:00
Nick Mathewson
94bff894f9 Fix a large pile of solaris warnings for bug 19767.
In nearly all cases, this is a matter of making sure that we include
orconfig.h before we include any standard c headers.
2016-07-28 10:47:46 -04:00
Nick Mathewson
9fe6fea1cc Fix a huge pile of -Wshadow warnings.
These appeared on some of the Jenkins platforms. Apparently some
GCCs care when you shadow globals, and some don't.
2016-07-28 10:22:10 -04:00
Nick Mathewson
a8676b1ede Merge branch 'bug18902_squashed' 2016-07-28 06:59:03 -04:00
Nick Mathewson
4757303873 Fix all -Wshadow warnings on Linux
This is a partial fix for 18902.
2016-07-28 06:58:44 -04:00
Nick Mathewson
d97fca16d0 Fix an integer overflow related to monotonic time on windows.
To maintain precision, to get nanoseconds, we were multiplying our
tick count by a billion, then dividing by ticks-per-second.  But
that apparently isn't such a great idea, since ticks-per-second is
sometimes a billion on its own, so our intermediate result was
giving us attoseconds.

When you're counting in attoseconds, you can only fit about 9
seconds into an int64_t, which is not so great for our purposes.

Instead, we now simplify the 1000000000/1000000000 fraction before
we start messing with nanoseconds.  This has potential to mess us
up if some future MS version declares that performance counters will
use 1,000,000,007 units per second, but let's burn that bridge when
we come to it.
2016-07-26 11:23:58 -04:00
Nick Mathewson
09c25697d7 Add a function to simplify a fraction.
Apparently remembering euclid's algorithm does pay off sooner or later.
2016-07-26 11:23:34 -04:00
Nick Mathewson
90ca446048 Remove windows debugging prints: it was an integer overflow hitting ftrapv 2016-07-26 11:07:53 -04:00
Nick Mathewson
019b7ddb9f fix identifier mistake :( 2016-07-26 10:44:51 -04:00
Nick Mathewson
160d2c6aab Redux^3: Temporarily add windows verbosity to track down jenkins failures 2016-07-26 10:36:44 -04:00
Nick Mathewson
0cef69713c Redux^2: Temporarily add windows verbosity to track down jenkins failures 2016-07-26 10:04:40 -04:00
Nick Mathewson
fb7f90c181 Tweaks on 19435 fix:
* Raise limit: 16k isn't all that high.
   * Don't log when limit exceded; log later on.
   * Say "over" when we log more than we say we log.
   * Add target version to changes file
2016-07-26 09:59:48 -04:00
Ivan Markin
77459b97aa Fix integer overflow in the rate-limiter (#19435). 2016-07-26 09:49:40 -04:00
Nick Mathewson
264fb7eb82 debugging: print ticks-per-second on windows. is it 0? 2016-07-26 09:44:41 -04:00
Nick Mathewson
1033713c9c Temporarily add some windows verbosity to track down unit test failure on jenkins. 2016-07-26 08:56:55 -04:00
Nick Mathewson
3f9c036821 Try a little harder to work around mingw clock_gettime weirdness 2016-07-26 08:22:37 -04:00
Peter Palfrader
518c8fe0ec Also ship compat_time.h in release tarballs. Fixes bug #19746 2016-07-25 09:07:29 +02:00
Nick Mathewson
53f9f71985 ug no, the RIGHT fix. 2016-07-21 15:29:56 +02:00
Nick Mathewson
9c210d0e81 Avoid infinite stack explosion in windows monotime.
[init calls get calls init calls get calls init.... ]
2016-07-21 15:26:05 +02:00
Nick Mathewson
1d0775684d Once more, 32-bit fixes on monotime mocking 2016-07-21 14:32:15 +02:00
Nick Mathewson
9c87869dde Merge branch 'maint-0.2.8' 2016-07-21 14:15:19 +02:00
Nick Mathewson
f1973e70a4 Coverity hates it when we do "E1 ? E2 : E2".
It says, 'Incorrect expression (IDENTICAL_BRANCHES)'

Fix for CID 1364127. Not in any released Tor.
2016-07-21 14:14:33 +02:00
Nick Mathewson
22314f9050 loony mingwcross bug: insist we dont have clock_gettime. 2016-07-21 14:09:00 +02:00
Nick Mathewson
852cff043b fix monotime test mocking on 32-bit systems 2016-07-21 14:05:29 +02:00
Nick Mathewson
2d26b1a549 Actually make monotonic time functions mockable.
This is different from making the functions mockable, since
monotime_t is opaque and so providing mocks for the functions is
really hard.
2016-07-21 07:02:33 -04:00
Nick Mathewson
72a1f0180d Revert "Make the monotonic{_coarse,}_get() functions mockable."
This reverts commit 2999f0b33f.
2016-07-21 10:30:21 +02:00
Nick Mathewson
2999f0b33f Make the monotonic{_coarse,}_get() functions mockable. 2016-07-21 10:25:23 +02:00
Nick Mathewson
558f7d3701 Merge branch 'monotonic_v2_squashed' 2016-07-19 11:42:26 +02:00
Nick Mathewson
6ba415d400 Make sure initialized_at is initialized before use. 2016-07-19 11:40:47 +02:00
Nick Mathewson
2a217ef723 Expose monotonic time ratchet functions for testing. 2016-07-19 11:40:47 +02:00
Nick Mathewson
7bc4ca7de9 Remove tor_gettimeofday_cached_monotonic as broken and unneeded 2016-07-19 11:40:47 +02:00
Nick Mathewson
6a2002fc09 convert timers.c to use real monotonic time. 2016-07-19 11:40:46 +02:00
Nick Mathewson
dc6f5d1dc1 Basic portable monotonic timer implementation
This code uses QueryPerformanceCounter() [**] on Windows,
mach_absolute_time() on OSX, clock_gettime() where available, and
gettimeofday() [*] elsewhere.

Timer types are stored in an opaque OS-specific format; the only
supported operation is to compute the difference between two timers.

[*] As you know, gettimeofday() isn't monotonic, so we include
a simple ratchet function to ensure that it only moves forward.

[**] As you may not know, QueryPerformanceCounter() isn't actually
always as monotonic as you might like it to be, so we ratchet that
one too.

We also include a "coarse monotonic timer" for cases where we don't
actually need high-resolution time.  This is GetTickCount{,64}() on
Windows, clock_gettime(CLOCK_MONOTONIC_COARSE) on Linux, and falls
back to regular monotonic time elsewhere.
2016-07-19 11:40:46 +02:00
Nick Mathewson
c138c9a2be Merge branch 'maint-0.2.8' 2016-07-17 13:55:04 -04:00
Nick Mathewson
fbae15a856 Merge remote-tracking branch 'weasel/bug19660' into maint-0.2.8 2016-07-17 13:54:40 -04:00
Nick Mathewson
9932544297 Merge branch 'maint-0.2.8' 2016-07-13 09:19:35 -04:00
Nick Mathewson
bb731ca665 Merge remote-tracking branch 'Jigsaw52/seccomp-fix-18397' into maint-0.2.8 2016-07-13 09:16:59 -04:00
Peter Palfrader
36b06be738 Add (SOCK_DGRAM, IPPROTO_UDP) sockets to the sandboxing whitelist
If we did not find a non-private IPaddress by iterating over interfaces,
we would try to get one via
get_interface_address6_via_udp_socket_hack().  This opens a datagram
socket with IPPROTO_UDP.  Previously all our datagram sockets (via
libevent) used IPPROTO_IP, so we did not have that in the sandboxing
whitelist.  Add (SOCK_DGRAM, IPPROTO_UDP) sockets to the sandboxing
whitelist.  Fixes bug 19660.
2016-07-11 09:37:01 +02:00
Daniel Pinto
20e89453fd Adds missing syscalls to seccomp filter.
Fixes #18397 which prevented tor starting with Sandbox 1.
2016-07-09 00:36:37 +01:00
Nick Mathewson
aa971c5924 Move our "what time is it now" compat functions into a new module
I'm not moving our "format and parse the time" functions, since
those have been pretty volatile over the last couple of years.
2016-07-08 10:38:59 -04:00
Nick Mathewson
466259eb50 Merge remote-tracking branch 'sebastian/libevent2' 2016-07-08 09:57:31 -04:00
Nick Mathewson
1135405c8c Fix a variable-shadowing bug in check_private_dir
We introduded a shadowed variable, thereby causing a log message to
be wrong. Fixes 19578. I believe the bug was introduced by
54d7d31cba in 0.2.2.29-beta.
2016-07-05 13:43:58 -04:00
Nick Mathewson
8ba4ba0a74 Grammar.
I grepped and hand-inspected the "it's" instances, to see if any
were supposed to be possessive.  While doing that, I found a
"the the", so I grepped to see if there were any more.
2016-07-05 12:10:12 -04:00
Sebastian Hahn
bd1a85cc91 Address review comments from cypherpunks 2016-07-04 16:34:40 +02:00
Sebastian Hahn
265e40b481 Raise libevent dependency to 2.0.10-stable or newer
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
2016-07-04 12:40:09 +02:00
Sebastian Hahn
ec6ea66240 Remove two wrong comments 2016-07-04 12:26:14 +02:00
Nick Mathewson
aaa3129043 Merge remote-tracking branch 'dgoulet/ticket16943_029_05-squashed'
Trivial Conflicts:
	src/or/or.h
	src/or/routerparse.c
2016-07-01 15:29:05 -04:00
Nick Mathewson
c6846d7bf0 Merge remote-tracking branch 'andrea/bug18322_v3_squashed' 2016-06-30 11:18:00 -04:00
Andrea Shepard
f99c9df02b Make things mockable for dump_desc_populate_fifo_from_directory() unit test 2016-06-30 07:03:26 +00:00
Andrea Shepard
42f089473a Unit test for dump_desc_populate_one_file() 2016-06-30 07:03:26 +00:00
Andrea Shepard
2154160a24 Add support for mocking functions declared with attributes without causing gcc warnings 2016-06-30 07:03:26 +00:00
Andrea Shepard
38cced90ef Move unparseable descriptor dumps into subdirectory of DataDir 2016-06-30 07:03:25 +00:00
Andrea Shepard
4e4a760491 Add extern support for file-scope variables in testsupport.h 2016-06-30 07:03:25 +00:00
Andrea Shepard
17ed2fed68 Expose dump_desc() to the test suite and make things it calls mockable 2016-06-30 07:03:24 +00:00
teor (Tim Wilson-Brown)
aae14f8346
Fix bug19483: avoid range checks when they are always true
Some compilers are smart enough to work out that comparisons to
LONG_MAX are a no-op on L64.
2016-06-30 09:29:14 +10:00
teor (Tim Wilson-Brown)
4234ca3bf2
Improve overflow checks in tv_udiff and tv_mdiff
Validate that tv_usec inputs to tv_udiff and tv_mdiff are in range.

Do internal calculations in tv_udiff and tv_mdiff in 64-bit,
which makes the function less prone to integer overflow,
particularly on platforms where long and time_t are 32-bit,
but tv_sec is 64-bit, like some BSD configurations.

Check every addition and subtraction that could overflow.
2016-06-29 17:23:24 +10:00
teor (Tim Wilson-Brown)
2e51608a8b
Fix an integer overflow bug in the tv_mdiff range check
The temporary second used for rounding can cause overflow,
depending on the order the compiler performs the operations.
2016-06-29 12:53:50 +10:00
Nick Mathewson
2197bfcc6a Merge branch 'maint-0.2.8' 2016-06-27 13:17:42 -04:00
Yawning Angel
0116eae59a Bug19499: Fix GCC warnings when building against bleeding edge OpenSSL.
The previous version of the new accessors didn't specify const but it
was changed in master.
2016-06-24 22:20:41 +00:00
David Goulet
9744a40f7a Add tor_htonll/ntohll functions
Signed-off-by: David Goulet <dgoulet@ev0ke.net>
2016-06-20 15:26:58 -04:00
David Goulet
49e8f47505 util: zero target buffer of base*_encode/decode
Make sure to memset(0) the destination buffer so we don't leave any
uninitialized data.

Fixes #19462

Signed-off-by: David Goulet <dgoulet@torproject.org>
2016-06-20 13:47:19 -04:00
Nick Mathewson
603cb712ef Small coverage improvements on compat.c 2016-06-20 11:03:13 -04:00
Nick Mathewson
ba28da8de5 compat.c coverage: simplify under-tested alloc_getcwd.
Yes, HURD lacks PATH_MAX.  But we already limited the maximum buffer
to 4096, so why not just use that?
2016-06-20 10:47:31 -04:00
Nick Mathewson
2b74e13a7c More coverage in backtrace.c 2016-06-20 10:31:36 -04:00
Nick Mathewson
6cedd49323 Merge branch 'bug14013_029_01_squashed' 2016-06-20 08:48:09 -04:00
nikkolasg
568dc27a19 Make base16_decodes return number of decoded bytes
base16_decodes() now returns the number of decoded bytes. It's interface
changes from returning a "int" to a "ssize_t". Every callsite now checks the
returned value.

Fixes #14013

Signed-off-by: David Goulet <dgoulet@torproject.org>
2016-06-20 08:44:58 -04:00
Nick Mathewson
1160ac1283 Changes file for 19063; use the BUG macro 2016-06-19 12:38:15 -04:00
U+039b
58e6a6aaeb Fix #19063: Add check in utility macro 2016-06-19 12:34:49 -04:00
Nick Mathewson
81cfd5c9a1 Merge branch 'zlib_coverage_squashed' 2016-06-19 12:20:38 -04:00
Nick Mathewson
5a725dab0a Mark some torgzip lines as unreachable/untestable. 2016-06-19 12:20:24 -04:00
Nick Mathewson
358fc026d9 Remove a ridiculous realloc call from torgzip.c
realloc()ing a thing in order to try to save memory on it just
doesn't make sense with today's allocators.  Instead, let's use the
fact that whenever we decompress something, either it isn't too big,
or we chop it up, or we reallocate it.
2016-06-19 12:20:24 -04:00
Nick Mathewson
808015316a Remove support for zlib <= 1.1
zlib 1.2 came out in 2003; earlier versions should be dead by now.

Our workaround code was only preventing us from using the gzip
encoding (if we decide to do so), and having some dead code linger
around in torgzip.c
2016-06-19 12:20:24 -04:00
Nick Mathewson
b421648da2 Merge remote-tracking branch 'public/thread_coverage' 2016-06-19 12:15:55 -04:00
Andrea Shepard
033cf30b3c Keep make check-spaces happy 2016-06-18 16:07:44 +00:00
Nick Mathewson
48b25e6811 Merge branch 'bug18280_029_03_nm_squashed' 2016-06-17 13:53:57 -04:00
David Goulet
4e4a7d2b0c Fix base32 API to take any source length in bytes
Fixes #18280

Signed-off-by: David Goulet <dgoulet@ev0ke.net>
2016-06-17 13:53:47 -04:00
cypherpunks
94762e37b9 Use the Autoconf macro AC_USE_SYSTEM_EXTENSIONS
The Autoconf macro AC_USE_SYSTEM_EXTENSIONS defines preprocessor macros
which turn on extensions to C and POSIX. The macro also makes it easier
for developers to use the extensions without needing (or forgetting) to
define them manually.

The macro can be safely used because it was introduced in Autoconf 2.60
and Tor requires Autoconf 2.63 and above.
2016-06-17 10:17:44 -04:00
Nick Mathewson
a8c766220f Mark an abort() as unreachable. 2016-06-16 15:53:05 -04:00
Nick Mathewson
c9ea9de806 Remove parse_config_line_from_str alias
All of our code just uses parse_config_line_from_str_verbose.
2016-06-16 15:40:56 -04:00
Nick Mathewson
128ab31c64 Mark code unreachable in unescape_string()
Also, add tests for it in case someday it does become reachable.
2016-06-16 15:36:08 -04:00
Nick Mathewson
5c596cdbc0 Tests for message rate-limiting
Also note a bug in the rate-limiting message.
2016-06-16 11:54:50 -04:00
Nick Mathewson
9b0bd65f22 I believe I found some dead code in our time parsing functions 2016-06-16 11:14:12 -04:00
Nick Mathewson
79370914d1 tests for size_mul_check__()
it's important, and we should make sure we got it right.
2016-06-16 10:43:01 -04:00
Nick Mathewson
41cb26c169 Correct the rounding behavior on tv_mdiff.
Fix for bug 19428.
2016-06-16 10:16:04 -04:00