Commit Graph

18133 Commits

Author SHA1 Message Date
Nick Mathewson
166aa8d741 Have a separate entry point for each compresion test 2017-04-27 10:25:52 -04:00
Nick Mathewson
36e62ae7e2 Merge remote-tracking branch 'ahf/bugs/22085' 2017-04-27 10:12:48 -04:00
Nick Mathewson
33a2fd065d Merge branch 'dirreq' 2017-04-27 10:08:32 -04:00
Alexander Færøy
e0a3819dcc
Test support for multiple compression backends in the buffer code.
This patch refactors the buffer compression tests to support multiple
comprssion backends.

See: https://bugs.torproject.org/22085
2017-04-27 15:51:14 +02:00
Alexander Færøy
22e6ad6f26
Clean up mentions of 'zlib' and rename the mentions to 'compressed'.
This patch cleans up in various places where 'zlib' is mentioned.
2017-04-27 15:51:14 +02:00
Alexander Færøy
7bececbd69
Refactor compression tests into a single test.
This patch refactors our compression tests such that deflate, gzip,
lzma, and zstd are all tested using the same code.

Additionally we use run-time checks to see if the given compression
method is supported instead of using HAVE_LZMA and HAVE_ZSTD.

See: https://bugs.torproject.org/22085
2017-04-27 15:51:14 +02:00
Nick Mathewson
b4fe0a6a03 Improve control flow in authority_certs_fetch_resource_impl 2017-04-27 09:30:08 -04:00
Nick Mathewson
52316f9969 Include UPLOAD_RENDDESC_V2 in PURPOSE_IS_UPLOAD
This was only used in one place before, and it's safe to update it.
2017-04-27 09:27:00 -04:00
Nick Mathewson
7fb9586953 Fix compilation when lzma or zstd is absent 2017-04-26 15:00:40 -04:00
Nick Mathewson
4038202f89 Avoid a warning from the use of floating-point in zstd
Replace "(preset - 0.5) * 1mb" with "preset * 1mb - 0.5 mb", to
avoid warning about converting double to size_t.
2017-04-26 14:21:45 -04:00
Nick Mathewson
be0557f759 Merge remote-tracking branch 'ahf/bugs/22066' 2017-04-26 14:20:01 -04:00
Alexander Færøy
e42c204f67
Approximate memory usage needed for the Zstandard backend.
This patch adds support for measuring the approximated memory usage by
the individual `tor_zstd_compress_state_t` object instances.

See: https://bugs.torproject.org/22066
2017-04-26 19:54:18 +02:00
Alexander Færøy
2aa28e7cb7
Better documentation for tor_uncompress().
This patch fixes the documentation string for `tor_uncompress()` to
ensure that it does not explicitly mention zlib or gzip since we now
support multiple compression backends.
2017-04-26 19:54:18 +02:00
Alexander Færøy
341824687a
Approximate memory usage needed for the LZMA backend.
This patch adds support for measuring the approximated memory usage by
the individual `tor_lzma_compress_state_t` object instances.

The LZMA library provides the functions `lzma_easy_encoder_memusage()`
and `lzma_easy_decoder_memusage()` which is used to find the estimated
usage in bytes.

See: https://bugs.torproject.org/22066
2017-04-26 19:54:18 +02:00
Sebastian Hahn
71c8974af0 Fix coverity cid 1405509
Locking in the init function is not necessary, but coverity gets
confused about it. So let's trick it.
2017-04-26 08:48:24 +02:00
Sebastian Hahn
ee478bdf38 Fix coverity cid 1405510
This is a false positive, but let's appease coverity.
2017-04-26 08:45:38 +02:00
Alexander Færøy
08d86e8408
Store compression overhead from tor_compress_state_t.
The `tor_compress_state_t` data-type is used as a wrapper around the
more specialized state-types used by the various compression backends.
This patch ensures that the overhead of this "thin" wrapper type is
included in the value returned by `tor_compress_get_total_allocation()`.

See: https://bugs.torproject.org/22066
2017-04-26 02:56:21 +02:00
Alexander Færøy
fac8ac0e4a
Remove unused header from the Zstandard compression backend.
Since we stopped looking at Zstandard error codes there is no need to
include the zstd_errors.h header file anymore.
2017-04-26 02:54:34 +02:00
Nick Mathewson
2655a72d89 Use x-lzma, not x-lzma2, as the identifier 2017-04-25 19:00:52 -04:00
Nick Mathewson
0274ea749a Function to convert compression methods to/from strings. 2017-04-25 16:47:46 -04:00
Alexander Færøy
236cf49670
Add stub function for libor-trace.
OS X's ar(1) doesn't allow us to create an archive with no object files.
This patch adds a stub file with a stub function in it to make OS X
happy again.
2017-04-25 18:02:19 +02:00
Nick Mathewson
43db91bd87 Teach cov-exclude to detect runaway LCOV_EXCL_START lines
Also, fix two instances of runaway LCOV_EXCL_START lines.
2017-04-25 10:59:19 -04:00
Nick Mathewson
7a37cbbea3 Whitespace fixes. 2017-04-25 10:54:34 -04:00
Nick Mathewson
49868340f7 Merge branch 'unified_compress_squashed' 2017-04-25 10:51:13 -04:00
Nick Mathewson
49a5b50b31 zlib: Turn UINT_MAX overrun into an error, not an assert. 2017-04-25 10:50:50 -04:00
Nick Mathewson
4b01b45ec1 Add a "best compression" flag. 2017-04-25 10:50:50 -04:00
Nick Mathewson
880fb3e3a9 Combine all *compress/*uncompress backend function into one
Since we have a streaming API for each compression backend, we don't
need a non-streaming API for each: we can build a common
non-streaming API at the front-end.
2017-04-25 10:50:50 -04:00
Nick Mathewson
232c9e14a8 Merge branch 'atomic_counters' 2017-04-25 10:46:23 -04:00
Nick Mathewson
ec7c512d27 Merge remote-tracking branch 'dgoulet/bug21293_031_01' 2017-04-25 10:43:53 -04:00
David Goulet
cb8ac1f331 trace: Add a basic event-tracing infrastructure.
This commit adds the src/trace directory containing the basics for our tracing
subsystem. It is not used in the code base. The "src/trace/debug.h" file
contains an example on how we can map our tor trace events to log_debug().

The tracing subsystem can only be enabled by tracing framework at compile
time. This commit introduces the "--enable-tracing-debug" option that will
make all "tor_trace()" function be maped to "log_debug()".

Closes #13802

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-25 10:37:31 -04:00
Nick Mathewson
4266ec766a Use atomic counters for compressor allocation. 2017-04-25 10:29:07 -04:00
David Goulet
368b091329 relay: Change LD_BUG log to LD_PROTOCOL_WARN
That log statement can be triggered if somebody on the Internet behaves badly
which is possible with buggy implementation for instance.

Fixes #21293

Signed-off-by: David Goulet <dgoulet@torproject.org>
2017-04-25 09:27:34 -04:00
Alexander Færøy
91dd4a00f7
Rename shadowing variable in compress_zlib.c.
This patch renames the `compress` parameter of the
`tor_zlib_compress_new()` function to `_compress` to avoid shadowing the
`compress()` function in zlib.h.
2017-04-25 14:56:55 +02:00
Nick Mathewson
ba405f86bf Merge branch 'ahf_prop278_21662_squashed' 2017-04-25 08:12:59 -04:00
Alexander Færøy
2210b330e7 Fix newlines in compression headers.
See: https://bugs.torproject.org/21662
2017-04-25 08:11:32 -04:00
Alexander Færøy
cf912259ba Remove tor_compress_memory_level().
This patch splits up `tor_compress_memory_level()` into static functions
in the individual compression backends, which allows us to tune the
values per compression backend rather than globally.

See: https://bugs.torproject.org/21662
2017-04-25 08:11:32 -04:00
Alexander Færøy
69a41e8bc6 Use switch-statement in tor_{compress,uncompress}.
Use a switch-statement in `tor_compress()` and `tor_uncompress()` for
the given `compress_method_t` parameter. This allows us to have the
compiler detect if we forgot add a handler in these functions for a
newly added enumeration value.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:10 -04:00
Alexander Færøy
c2d1d949de Use tor_compress_supports_method() before printing library versions.
This patch ensures that Tor checks if a given compression method is
supported before printing the version string when calling `tor
--library-versions`.

Additionally, we use the `tor_compress_supports_method()` to check if a
given version is supported for Tor's start-up version string, but here
we print "N/A" if a given compression method is unavailable.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:10 -04:00
Alexander Færøy
6b905b38bb Add API entry-point for getting compression method version numbers.
This patch adds `tor_compress_version_str()` and
`tor_compress_header_version_str()` to get the version strings of the
different compression schema providers. Both functions returns `NULL` in
case a given `compress_method_t` is unknown or unsupported.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:10 -04:00
Alexander Færøy
1c77d8690c Add function to check if a given compression method is supported.
This patch adds support for checking if a given `compress_method_t` is
supported by the currently running Tor instance using
`tor_compress_supports_method()`.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
04682d302a Add tor_compress_get_total_allocation() function.
This patch adds the `tor_compress_get_total_allocation()` which returns
an approximate number of bytes currently in use by all the different
compression backends.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
be4dc54634 Display LZMA and Zstandard versions when starting Tor.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
2fa7b722ce Show liblzma and libzstd versions in tor --library-versions.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
380736d045 Add Zstandard support.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
ce1feae9d9 Add --enable-zstd to our configure script.
This patch adds support for enabling support for Zstandard to our configure
script. By default, the --enable-zstd option is set to "auto" which means if
libzstd is available we'll build Tor with Zstandard support.

See: https://bugs.torproject.org/21662
2017-04-25 08:10:09 -04:00
Alexander Færøy
bf1c07cb07 Add LZMA support.
See: https://bugs.torproject.org/21662
2017-04-25 08:10:06 -04:00
Alexander Færøy
157af1d26e Add --enable-lzma to our configure script.
This patch adds support for enabling support for LZMA to our configure
script. By default, the --enable-lzma option is set to "auto" which
means if liblzma is available we'll build Tor with LZMA support.

See: https://bugs.torproject.org/21662
2017-04-25 08:06:02 -04:00
Alexander Færøy
c171af0487 Use a switch-statement when checking for compression method.
This patch changes the way `tor_compress_new()`,
`tor_compress_process()`, and `tor_compress_free()` handles different
compression methods. This should give us compiler warnings in case an
additional compression method is added, but the developer forgets to add
handlers in the three aforementioned functions.

See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Alexander Færøy
300ac49685 Add compress_zlib.obj to src/common/Makefile.nmake.
See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00
Alexander Færøy
04583df452 Rename the torgzip module to compress.
See https://bugs.torproject.org/21663
2017-04-25 08:06:01 -04:00