Commit Graph

3907 Commits

Author SHA1 Message Date
Nick Mathewson
e04cc7e27f Merge branch 'maint-0.3.1' 2017-07-05 11:16:51 -04:00
Nick Mathewson
e88aa98451 Merge branch 'teor-bug22797-025' into maint-0.3.1 2017-07-05 11:16:30 -04:00
teor
878e0d45a5 Always allow extra file descriptors when setting the connection maximum
When setting the maximum number of connections allowed by the OS,
always allow some extra file descriptors for other files.

Fixes bug 22797; bugfix on 0.2.0.10-alpha.
2017-07-05 11:15:10 -04:00
cypherpunks
c79e286386 Use the proper syscall in sandbox error messages
Fixes #22750.
2017-07-05 09:56:28 -04:00
Nick Mathewson
bb3f74e66b Fix assertion failure related to openbsd strtol().
Fixes bug 22789; bugfix on 0.2.3.8-alpha.
2017-07-03 11:22:27 -04:00
Nick Mathewson
5361032219 Fix -Wfloat-conversion C warnings on mingw in clamp_double_to_int64.
We just have to suppress these warnings: Mingw's math.h uses gcc's
__builtin_choose_expr() facility to declare isnan, isfinite, and
signbit.  But as implemented in at least some versions of gcc,
__builtin_choose_expr() can generate type warnings even from
branches that are not taken.

Fixes bug 22801; bugfix on 0.2.8.1-alpha.
2017-07-03 10:59:31 -04:00
Nick Mathewson
2c718c1a12 Merge branch 'maint-0.3.1' 2017-06-29 10:43:50 -04:00
Nick Mathewson
a088a08eeb Log real error message when unable to remove a storagedir file
Attempts to help diagnose 22752.
2017-06-28 14:24:27 -04:00
Nick Mathewson
01404d7c52 Merge branch 'maint-0.3.1' 2017-06-28 12:25:09 -04:00
Alexander Færøy
c239b2fc9c Fix crash in LZMA module when the Sandbox is enabled.
This patch fixes a crash in our LZMA module where liblzma will allocate
slightly more data than it is allowed to by its limit, which leads to a
crash.

See: https://bugs.torproject.org/22751
2017-06-28 10:00:24 -04:00
George Kadianakis
0d9873ac0d ed25519: Check retval of unpack_negative_vartime in donna. 2017-06-28 14:58:22 +03:00
Nick Mathewson
733ce556ad Merge branch 'asn_bug22006_final_squashed' 2017-06-27 18:21:46 -04:00
Nick Mathewson
3f94041589 no newlines in log messages. 2017-06-27 18:21:35 -04:00
Nick Mathewson
7fff6cfead Merge branch 'asn_bug22006_final_squashed' 2017-06-27 17:19:08 -04:00
George Kadianakis
559658ff1c ed25519: Add func that checks for torsion component in pubkeys.
See https://lists.torproject.org/pipermail/tor-dev/2017-April/012213.html .
2017-06-27 17:17:58 -04:00
Nick Mathewson
82a68be69a Merge branch 'maint-0.3.1' 2017-06-26 10:32:57 -04:00
Nick Mathewson
0c7c49483f Fix a coverity warning about a no-op assert with-64 bit size_t
This is CID 1403400
2017-06-26 10:31:13 -04:00
Nick Mathewson
acbe16a9a2 Log even more to try to debug 22719 2017-06-24 13:40:02 -04:00
Nick Mathewson
6caf924605 Add an additional log message to try to diagnose #22719 2017-06-24 11:27:46 -04:00
Nick Mathewson
e51e7bd38b Merge branch 'bug22502_redux_031' into maint-0.3.1 2017-06-20 20:27:48 -04:00
Nick Mathewson
c999e84436 Merge branch 'bug22672_031' into maint-0.3.1 2017-06-20 20:26:45 -04:00
Nick Mathewson
392e5457b8 Merge remote-tracking branch 'argonblue/bug22638' into maint-0.3.1 2017-06-20 13:35:38 -04:00
Nick Mathewson
9328bd524e Enforce the rule that COMPRESS_OK means progress was made.
If COMPRESS_OK occurs but data is neither consumed nor generated,
treat it as a BUG and a COMPRESS_ERROR.

This change is meant to prevent infinite loops in the case where
we've made a mistake in one of our compression backends.

Closes ticket 22672.
2017-06-20 12:26:57 -04:00
Nick Mathewson
945256188a mingw/windows printf lacks %zd ; use %lu and casts instead
(This approach can lose accuracy, but it's only in debug-level messages.)

Fixes windows compilation. Bugfix on recent compress.c changes; bug
not in any released Tor.
2017-06-20 12:12:55 -04:00
Taylor Yu
25edb41e6f Fix compress_none.c header comment
The Doxygen \file markup for compress_none.c had the wrong filename.
Fixes #22638.
2017-06-20 11:27:17 -04:00
Nick Mathewson
eb632afb17 Correct the fix to bug 22629 to permit trailing non-garbage
This change makes it so that we can decompress concatenated zstd
outputs.
2017-06-20 10:24:22 -04:00
Nick Mathewson
1c0459f19a Merge remote-tracking branch 'teor/bug22502' into maint-0.3.1 2017-06-20 10:04:16 -04:00
Nick Mathewson
eff5e29404 Merge branch 'maint-0.3.0' into maint-0.3.1 2017-06-19 13:52:19 -04:00
Nick Mathewson
71c701927a Merge branch 'maint-0.2.9' into maint-0.3.0 2017-06-19 13:52:19 -04:00
Nick Mathewson
59f29970fa Permit the fchmod system call.
Fixes bug 22516; bugfix on 0.2.5.4-alpha.
2017-06-16 14:03:02 -04:00
teor
7d535ea9d3
Add extra logging during compression and decompression
This helps diagnose failures.

Part of #22502.
2017-06-16 09:48:18 +10:00
teor
cbaf0c049c
Return TOR_COMPRESS_BUFFER_FULL when zstd has additional input
Fixes #22628.
2017-06-16 09:47:32 +10:00
teor
617e1da636
Remove a redundant conditional in tor_zstd_compress_process
Part of #22502
2017-06-16 09:46:46 +10:00
teor
7605bd528e
Move a comment to the right place in tor_zstd_compress_process
Part of #22502
2017-06-16 09:45:58 +10:00
teor
952c9073ad
Check for trailing input garbage in tor_compress_impl() when decompressing
Fixes #22629.
2017-06-16 09:41:29 +10:00
teor
8e1b37a4aa
Check if tor_compress_new() returns NULL in tor_compress_impl()
Partial fix to 22626.
2017-06-16 09:38:18 +10:00
Nick Mathewson
e3b1573be6 Merge branch 'maint-0.3.0' 2017-06-05 15:52:06 -04:00
Nick Mathewson
d1c1dc229e Merge branch 'maint-0.2.9' into maint-0.3.0 2017-06-05 15:44:12 -04:00
Nick Mathewson
ec84fc1d8e Improve documentation on get_{peer,own}_certificate()
Make it clear that we're returning a newly allocated copy.
2017-06-05 15:27:33 -04:00
Nick Mathewson
39b7e89c28 Test prerequisites: function to dup a cert, make get_own_cert mockable. 2017-06-05 15:27:33 -04:00
Nick Mathewson
50facb40bb On v3 link handshake, send the correct link certificate
Previously we'd send the _current_ link certificate, which would
cause a handshaking failure when the TLS context rotated.
2017-06-05 15:27:33 -04:00
Nick Mathewson
0fbe1a2c6f Merge remote-tracking branch 'teor/bug22424' 2017-05-30 08:50:45 -04:00
Daniel Pinto
94d321120e Replace 3-star pointer with 2-star pointer 2017-05-28 20:24:48 +01:00
teor
79725289e1
If we do underflow the know usage of a storage, recalculate it
Fixes bug #22424 on 0.3.1.1-alpha.
2017-05-28 22:34:43 +10:00
teor
69b234a0a8
Refactor storage usage reductions into a static function
No behaviour change.

Part of #22424.
2017-05-28 22:28:43 +10:00
teor
334fe6bb6b
Don't underflow usage when it is unknown and a file is removed
Part of #22424.
2017-05-28 22:16:00 +10:00
teor
9e36b0beb9
Always check for usage underflow when removing a file in storage.c
Part of #22424.
2017-05-28 22:12:09 +10:00
teor
f6841ae263
Fix comment typos in storage.c 2017-05-28 22:11:22 +10:00
Daniel Pinto
f8ccf8d9a9 Fix crash with %include
Fixes crash when including a folder that contains a non-empty file
without any values followed by any other non-empty file.
2017-05-28 09:53:14 +01:00
Nick Mathewson
6fcaf83c98 Cleanup MOCK_IMPL (etc) to be findable with etags
A fair number of our mock_impl declarations were messed up so that
even our special AM_ETAGSFLAGS couldn't find them.

This should be a whitespace-only patch.
2017-05-26 14:07:06 -04:00
teor
af98b862a5
Fix comments of functions that return tor_snprintf
No code changes needed: in the places where we actually check the
return value of these functions, we handle it correctly.
2017-05-23 18:44:45 +10:00
Roger Dingledine
0698a0beca fix two typos in comments
closes ticket 22322
2017-05-22 01:43:52 -04:00
Nick Mathewson
d950ad0dfd Remove call to get_unquoted_path in config_process_include()
parse_config_line_from_str_verbose() already looks for strings
that are surrounded by quotes, and processes them with
unescape_string().  So things were getting decoded twice, which was
(in turn) playing havoc with backslashes on Windows.
2017-05-19 14:09:51 -04:00
Nick Mathewson
ff1af5550a strlen() returns size_t 2017-05-19 08:54:56 -04:00
Nick Mathewson
69ef94820b Merge branch 'add_rust_squashed' 2017-05-19 08:47:18 -04:00
Sebastian Hahn
f8ef7c65d1 Add some Rust utility functions and print support
This gives an indication in the log that Tor was built with Rust
support, as well as laying some groundwork for further string-returning
APIs to be converted to Rust
2017-05-19 08:47:10 -04:00
Nick Mathewson
92d335b3dc Merge remote-tracking branch 'jigsaw/torrc-dir-fix-1922_squashed2' 2017-05-19 08:46:13 -04:00
Daniel Pinto
ba3a5f82f1 Add support for %include funcionality on torrc #1922
config_get_lines is now split into two functions:
 - config_get_lines which is the same as before we had %include
 - config_get_lines_include which actually processes %include
2017-05-18 23:44:16 +01:00
Daniel Pinto
e04da2828d Fixed error on BASEXX_NOPAD LEN and BUFSIZE macros #21872 2017-05-17 00:22:11 +01:00
Alexander Færøy
3a05687c6d
Add API for getting human readable descriptions of a compress_method_t
See: https://bugs.torproject.org/21667
2017-05-12 17:18:45 +02:00
Nick Mathewson
95fa7d1cf8 In channelpadding tests that touch libevent, call event_reinit().
This is necessary to avoid crashes and test failures on kevent-based
systems.

Fixes bug 22209; bug not in any released Tor.
2017-05-10 11:01:13 -04:00
Nick Mathewson
35025ee51f Merge branch 'maint-0.3.0' 2017-05-08 13:40:41 -04:00
Nick Mathewson
d792d2a14d Merge branch 'maint-0.2.9' into maint-0.3.0 2017-05-08 13:40:26 -04:00
Nick Mathewson
584ab1f29b Merge branch 'compress_none_v2_squashed' 2017-05-02 08:33:35 -04:00
Nick Mathewson
1bc21111d8 Treat the identity transformation as another kind of compression.
This will allow us to treat NO_METHOD as a real compression method,
and to simplify code that currently does

   if (compressing) {
      compress
   } else {
      copy
   }
2017-05-02 08:31:32 -04:00
Nick Mathewson
c486ef57a3 Rename x-lzma to x-tor-lzma
We shouldn't call it lzma, because we are imposing a limit on the
memory needed for decoding.
2017-05-01 15:31:28 -04:00
Nick Mathewson
4837421d7c Merge remote-tracking branch 'ahf/bugs/21665' 2017-05-01 14:22:49 -04:00
Nick Mathewson
531835f561 Increase MALLOC_MP_LIM to 16MB
Increase the maximum allowed size passed to mprotect(PROT_WRITE)
from 1MB to 16MB. This was necessary with the glibc allocator
in order to allow worker threads to allocate more memory --
which in turn is necessary because of our new use of worker
threads for compression.

Closes ticket #22096. Found while working on #21648.
2017-04-28 10:55:10 -04:00
Nick Mathewson
00a12337ff Merge branches 'consdiffmgr_orig_squashed' and 'actually_compute_diffs_squashed' 2017-04-27 21:43:06 -04:00
Nick Mathewson
7a0964279f Functionality to ensure there is space to add files to cache. 2017-04-27 21:40:13 -04:00
Nick Mathewson
16d6ab6640 Fix use-after-free bug in storage_dir sandbox code. 2017-04-27 21:40:12 -04:00
Alexander Færøy
0672b33f1e
Enforce 16 MB upper bound of memory usage in LZMA decoder.
This patch changes two things in our LZMA compression backend:

- We lower the preset values for all `compression_level_t` values to
  ensure that we can run the LZMA decoder with less than 65 MB of memory
  available. This seems to have a small impact on the real world usage
  and fits well with our needs.

- We set the upper bound of memory usage for the LZMA decoder to 16 MB.

See: https://bugs.torproject.org/21665
2017-04-27 20:09:20 +02:00
Alexander Færøy
e5122b91a9
Only compare the first 3 bytes when trying to detect LZMA compression.
This patch changes the logic in `detect_compression_method()` to only
use the 3 first bytes when checking if a given input is LZMA encoded.
2017-04-27 20:07:08 +02:00
Nick Mathewson
10a4f9cd07 Merge branch 'parse_accept_encoding' 2017-04-27 11:31:31 -04:00
Nick Mathewson
2903c329aa Move the "supported compression bitmask" into compress.[ch] 2017-04-27 11:30:51 -04:00
Nick Mathewson
49deb1e1b8 Document and test nul-terminating behavior of tor_uncompress()
We added this as a safety feature, but there are a few places in the
code that actually depend on it.
2017-04-27 10:59:48 -04:00
Nick Mathewson
39cfaba9e2 Fix handling of "final" flag in zstd decompression
We were returning "DONE" on truncated input streams, which was not
what we wanted.
2017-04-27 10:42:05 -04:00
Nick Mathewson
7fb9586953 Fix compilation when lzma or zstd is absent 2017-04-26 15:00:40 -04:00
Nick Mathewson
4038202f89 Avoid a warning from the use of floating-point in zstd
Replace "(preset - 0.5) * 1mb" with "preset * 1mb - 0.5 mb", to
avoid warning about converting double to size_t.
2017-04-26 14:21:45 -04:00
Nick Mathewson
be0557f759 Merge remote-tracking branch 'ahf/bugs/22066' 2017-04-26 14:20:01 -04:00
Alexander Færøy
e42c204f67
Approximate memory usage needed for the Zstandard backend.
This patch adds support for measuring the approximated memory usage by
the individual `tor_zstd_compress_state_t` object instances.

See: https://bugs.torproject.org/22066
2017-04-26 19:54:18 +02:00
Alexander Færøy
2aa28e7cb7
Better documentation for tor_uncompress().
This patch fixes the documentation string for `tor_uncompress()` to
ensure that it does not explicitly mention zlib or gzip since we now
support multiple compression backends.
2017-04-26 19:54:18 +02:00
Alexander Færøy
341824687a
Approximate memory usage needed for the LZMA backend.
This patch adds support for measuring the approximated memory usage by
the individual `tor_lzma_compress_state_t` object instances.

The LZMA library provides the functions `lzma_easy_encoder_memusage()`
and `lzma_easy_decoder_memusage()` which is used to find the estimated
usage in bytes.

See: https://bugs.torproject.org/22066
2017-04-26 19:54:18 +02:00
Nick Mathewson
99e943998d Add getpid() to the seccomp2 sandbox.
We hadn't needed this before, because most getpid() callers on Linux
were looking at the vDSO version of getpid().  I don't know why at
least one version of OpenSSL seems to be ignoring the vDSO, but this
change should fix it.

Fixes bug 21943; bugfix on 0.2.5.1-alpha when the sandbox was
introduced.
2017-04-26 12:56:06 -04:00
Sebastian Hahn
71c8974af0 Fix coverity cid 1405509
Locking in the init function is not necessary, but coverity gets
confused about it. So let's trick it.
2017-04-26 08:48:24 +02:00
Alexander Færøy
08d86e8408
Store compression overhead from tor_compress_state_t.
The `tor_compress_state_t` data-type is used as a wrapper around the
more specialized state-types used by the various compression backends.
This patch ensures that the overhead of this "thin" wrapper type is
included in the value returned by `tor_compress_get_total_allocation()`.

See: https://bugs.torproject.org/22066
2017-04-26 02:56:21 +02:00
Alexander Færøy
fac8ac0e4a
Remove unused header from the Zstandard compression backend.
Since we stopped looking at Zstandard error codes there is no need to
include the zstd_errors.h header file anymore.
2017-04-26 02:54:34 +02:00
Nick Mathewson
2655a72d89 Use x-lzma, not x-lzma2, as the identifier 2017-04-25 19:00:52 -04:00
Nick Mathewson
0274ea749a Function to convert compression methods to/from strings. 2017-04-25 16:47:46 -04:00
Nick Mathewson
43db91bd87 Teach cov-exclude to detect runaway LCOV_EXCL_START lines
Also, fix two instances of runaway LCOV_EXCL_START lines.
2017-04-25 10:59:19 -04:00
Nick Mathewson
7a37cbbea3 Whitespace fixes. 2017-04-25 10:54:34 -04:00
Nick Mathewson
49868340f7 Merge branch 'unified_compress_squashed' 2017-04-25 10:51:13 -04:00
Nick Mathewson
49a5b50b31 zlib: Turn UINT_MAX overrun into an error, not an assert. 2017-04-25 10:50:50 -04:00
Nick Mathewson
4b01b45ec1 Add a "best compression" flag. 2017-04-25 10:50:50 -04:00
Nick Mathewson
880fb3e3a9 Combine all *compress/*uncompress backend function into one
Since we have a streaming API for each compression backend, we don't
need a non-streaming API for each: we can build a common
non-streaming API at the front-end.
2017-04-25 10:50:50 -04:00
Nick Mathewson
4266ec766a Use atomic counters for compressor allocation. 2017-04-25 10:29:07 -04:00
Alexander Færøy
91dd4a00f7
Rename shadowing variable in compress_zlib.c.
This patch renames the `compress` parameter of the
`tor_zlib_compress_new()` function to `_compress` to avoid shadowing the
`compress()` function in zlib.h.
2017-04-25 14:56:55 +02:00
Nick Mathewson
ba405f86bf Merge branch 'ahf_prop278_21662_squashed' 2017-04-25 08:12:59 -04:00
Alexander Færøy
2210b330e7 Fix newlines in compression headers.
See: https://bugs.torproject.org/21662
2017-04-25 08:11:32 -04:00
Alexander Færøy
cf912259ba Remove tor_compress_memory_level().
This patch splits up `tor_compress_memory_level()` into static functions
in the individual compression backends, which allows us to tune the
values per compression backend rather than globally.

See: https://bugs.torproject.org/21662
2017-04-25 08:11:32 -04:00