The GNU C Library (glibc) offers an function which allocates the
necessary memory automatically [0]. When it is available, we use that.
Otherwise we depend upon the `getcwd` function which requires a
preallocated buffer (and its size). This function was used incorrectly
by depending on the initial buffer size being big enough and otherwise
failing to return the current working directory. The proper way of
getting the current working directory requires a loop which doubles the
buffer size if `getcwd` requires it. This code was copied from [1] with
modifications to fit the context.
[0] https://www.gnu.org/software/hurd/hurd/porting/guidelines.html
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/getcwd.html
We don't actually want Coverity to complain when a BUG() check can
never fail, since such checks can prevent us from introducing bugs
later on.
Closes ticket 23054. Closes CID 1415720, 1415724.
Instead of choosing a lower-priority job with a 1/37 chance, have
the chance be 1/37 for half the threads, and 1/2147483647 for the
other half. This way if there are very slow jobs of low priority,
they shouldn't be able to grab all the threads when there is better
work to do.
Each piece of queued work now has an associated priority value; each
priority goes on a separate queue.
With probability (N-1)/N, the workers will take work from the highest
priority nonempty queue. Otherwise, they'll look for work in a
queue of lower priority. This behavior is meant to prevent
starvation for lower-priority tasks.
In the Linux kernel, the BUG() macro causes an instant panic. Our
BUG() macro is different, however: it generates a nonfatal assertion
failure, and is usable as an expression.
Additionally, this patch tells util_bug.h to make all assertion
failures into fatal conditions when we're building with a static
analysis tool, so that the analysis tool can look for instances
where they're reachable.
Fixes bug 23030.
Wow, it sure seems like some compilers can't implement isnan() and
friends in a way that pleases themselves!
Fixes bug 22915. Bug trigged by 0.2.8.1-alpha and later; caused by
clang 4.
We need to keep these around for TAP and old-style hidden services,
but they're obsolete, and we shouldn't encourage anyone to use them.
So I've added "obsolete" to their names, and a comment explaining
what the problem is.
Closes ticket 23026.
Closes bug 22964. Based on Teor's replacement there, but tries
to put the comment in a more logical place, and explain why we're
actually disabling compression in the first place.
In zstd 1.3.0, once you have called ZSTD_endStream and been told
that your putput buffer is full, it really doesn't want you to call
ZSTD_compressStream again. ZSTD 1.2.0 didn't seem to mind about
this.
This patch fixes the issue by making sure never to call
ZSTD_endStream if there's any more data on the input buffer to
process, by flushing even when we're about to call "endStream", and
by never calling "compress" or "flush" after "endStream".
This change prevents us from generating corrupt messages when we
are confused about codepage settings, and makes Windows errors
consistent with the rest of our logs.
Fixes bug 22520; bugfix on 0.1.2.8-alpha. Patch from "Vort".
- Move some crypto structures so that they are visible by tests.
- Introduce a func to count number of hops in cpath which will be used
by the tests.
- Mark a function as mockable.
The 22081 fix disabled -Wfloat-conversion, but -Wfloat-conversion
didn't exist in every relevant mingw; it was added in GCC 4.9.x some
time, if the documentation can be trusted.
Bug not in any released version of tor.
When setting the maximum number of connections allowed by the OS,
always allow some extra file descriptors for other files.
Fixes bug 22797; bugfix on 0.2.0.10-alpha.
We just have to suppress these warnings: Mingw's math.h uses gcc's
__builtin_choose_expr() facility to declare isnan, isfinite, and
signbit. But as implemented in at least some versions of gcc,
__builtin_choose_expr() can generate type warnings even from
branches that are not taken.
Fixes bug 22801; bugfix on 0.2.8.1-alpha.
This patch fixes a crash in our LZMA module where liblzma will allocate
slightly more data than it is allowed to by its limit, which leads to a
crash.
See: https://bugs.torproject.org/22751
If COMPRESS_OK occurs but data is neither consumed nor generated,
treat it as a BUG and a COMPRESS_ERROR.
This change is meant to prevent infinite loops in the case where
we've made a mistake in one of our compression backends.
Closes ticket 22672.
(This approach can lose accuracy, but it's only in debug-level messages.)
Fixes windows compilation. Bugfix on recent compress.c changes; bug
not in any released Tor.
A fair number of our mock_impl declarations were messed up so that
even our special AM_ETAGSFLAGS couldn't find them.
This should be a whitespace-only patch.
parse_config_line_from_str_verbose() already looks for strings
that are surrounded by quotes, and processes them with
unescape_string(). So things were getting decoded twice, which was
(in turn) playing havoc with backslashes on Windows.
This gives an indication in the log that Tor was built with Rust
support, as well as laying some groundwork for further string-returning
APIs to be converted to Rust
config_get_lines is now split into two functions:
- config_get_lines which is the same as before we had %include
- config_get_lines_include which actually processes %include
This will allow us to treat NO_METHOD as a real compression method,
and to simplify code that currently does
if (compressing) {
compress
} else {
copy
}
Increase the maximum allowed size passed to mprotect(PROT_WRITE)
from 1MB to 16MB. This was necessary with the glibc allocator
in order to allow worker threads to allocate more memory --
which in turn is necessary because of our new use of worker
threads for compression.
Closes ticket #22096. Found while working on #21648.
This patch changes two things in our LZMA compression backend:
- We lower the preset values for all `compression_level_t` values to
ensure that we can run the LZMA decoder with less than 65 MB of memory
available. This seems to have a small impact on the real world usage
and fits well with our needs.
- We set the upper bound of memory usage for the LZMA decoder to 16 MB.
See: https://bugs.torproject.org/21665
This patch adds support for measuring the approximated memory usage by
the individual `tor_zstd_compress_state_t` object instances.
See: https://bugs.torproject.org/22066
This patch fixes the documentation string for `tor_uncompress()` to
ensure that it does not explicitly mention zlib or gzip since we now
support multiple compression backends.
This patch adds support for measuring the approximated memory usage by
the individual `tor_lzma_compress_state_t` object instances.
The LZMA library provides the functions `lzma_easy_encoder_memusage()`
and `lzma_easy_decoder_memusage()` which is used to find the estimated
usage in bytes.
See: https://bugs.torproject.org/22066
We hadn't needed this before, because most getpid() callers on Linux
were looking at the vDSO version of getpid(). I don't know why at
least one version of OpenSSL seems to be ignoring the vDSO, but this
change should fix it.
Fixes bug 21943; bugfix on 0.2.5.1-alpha when the sandbox was
introduced.
The `tor_compress_state_t` data-type is used as a wrapper around the
more specialized state-types used by the various compression backends.
This patch ensures that the overhead of this "thin" wrapper type is
included in the value returned by `tor_compress_get_total_allocation()`.
See: https://bugs.torproject.org/22066
Since we have a streaming API for each compression backend, we don't
need a non-streaming API for each: we can build a common
non-streaming API at the front-end.
This patch renames the `compress` parameter of the
`tor_zlib_compress_new()` function to `_compress` to avoid shadowing the
`compress()` function in zlib.h.
This patch splits up `tor_compress_memory_level()` into static functions
in the individual compression backends, which allows us to tune the
values per compression backend rather than globally.
See: https://bugs.torproject.org/21662
Use a switch-statement in `tor_compress()` and `tor_uncompress()` for
the given `compress_method_t` parameter. This allows us to have the
compiler detect if we forgot add a handler in these functions for a
newly added enumeration value.
See: https://bugs.torproject.org/21662
This patch adds `tor_compress_version_str()` and
`tor_compress_header_version_str()` to get the version strings of the
different compression schema providers. Both functions returns `NULL` in
case a given `compress_method_t` is unknown or unsupported.
See: https://bugs.torproject.org/21662
This patch adds support for checking if a given `compress_method_t` is
supported by the currently running Tor instance using
`tor_compress_supports_method()`.
See: https://bugs.torproject.org/21662
This patch adds the `tor_compress_get_total_allocation()` which returns
an approximate number of bytes currently in use by all the different
compression backends.
See: https://bugs.torproject.org/21662
This patch changes the way `tor_compress_new()`,
`tor_compress_process()`, and `tor_compress_free()` handles different
compression methods. This should give us compiler warnings in case an
additional compression method is added, but the developer forgets to add
handlers in the three aforementioned functions.
See https://bugs.torproject.org/21663
This patch refactors the `torgzip` module to allow us to extend a common
compression API to support multiple compression backends.
Additionally we move the gzip/zlib code into its own module under the
name `compress_zlib`.
See https://bugs.torproject.org/21664
This patch makes the internal `get_memlevel()` a part of the public
compression API as `tor_compress_memory_level()`.
See https://bugs.torproject.org/21663
This patch refactors our streaming compression code to allow us to
extend it with non-zlib/non-gzip based compression schemas.
See https://bugs.torproject.org/21663
To allow us to use the API name `tor_compress` and `tor_uncompress` as
the main entry-point for all compression/uncompression and not just gzip
and zlib.
See https://bugs.torproject.org/21663
This patch removes the unused `is_gzip_supported()` and changes the
documentation string around the `compress_method_t` enumeration to
explicitly state that both `ZLIB_METHOD` and `GZIP_METHOD` are both
always supported.
Zlib version 1.2.0 was released on the 9'th of March, 2003 according to
their ChangeLog.
See https://bugs.torproject.org/21663
Remove base64_decode_nopad() because it is redundant now that
base64_decode() correctly handles both padded and unpadded base64
encodings with "right-sized" output buffers.
base64_decode() was applying an overly conservative check on the
output buffer length that could incorrectly produce an error if the
input encoding contained padding or newlines. Fix this by checking
the output buffer length against the actual decoded length produced
during decoding.
When we "fixed" #18280 in 4e4a7d2b0c
in 0291 it appears that we introduced a bug: The base32_encode
function can read off the end of the input buffer, if the input
buffer size modulo 5 is not equal to 0 or 3.
This is not completely horrible, for two reasons:
* The extra bits that are read are never actually used: so this
is only a crash when asan is enabled, in the worst case. Not a
data leak.
* The input sizes passed to base32_encode are only ever multiples
of 5. They are all either DIGEST_LEN (20), REND_SERVICE_ID_LEN
(10), sizeof(rand_bytes) in addressmap.c (10), or an input in
crypto.c that is forced to a multiple of 5.
So this bug can't actually trigger in today's Tor.
Closes bug 21894; bugfix on 0.2.9.1-alpha.
It looks like 32_encoded_size/64_encode_size APIs are inconsistent
not only in the number of "d"s they have, but also in whether they
count the terminating NUL. Taylor noted this in 86477f4e3f,
but I think we should note the inconsistently more loudly in order
to avoid trouble.
(I ran into trouble with this when writing 30b13fd82e243713c6a0d.)
