Add a TOR_PUSH_DELAY variable to git-push-all.sh, which makes the script
push master and maint branches with a delay between each branch. These
delays trigger the CI jobs in a set order, which should show the most
likely failures first.
Also:
* make pushes atomic by default, and
* make the script pass any command-line arguments to git push.
Closes ticket 29879.
On some windows builds, time_t is 64 bits but long is not. This is
causing appveyor builds to fail.
Also, one of our uses of labs() on time_t was logically incorrect:
it was telling us to accept NETINFO cells up to three minutes
_before_ the message they were responding to, which doesn't make
sense.
This patch adds a time_abs() function that we should eventually move
to intmath.h or something. For now, though, it will make merges
easier to have it file-local in channeltls.c.
Fixes bug 31343; bugfix on 0.2.4.4-alpha.
I'm using 500 as a file size limit, and 15 as an include limit.
This affects comparatively few files, but I think they are the worst
ones.
Closes ticket 31175.
Padding circuits were regular cells that got closed before their padding
machine could finish. This means that they can still receive regular cells from
their past life, but they have no way or reason to answer them anymore. Hence
let's ignore them before they even get to the proper subsystems.
There is other code that uses this value, and some of it is
apparently reachable from inside router_dir_info_changed(), which
routerlist_free() apparently calls. (ouch!) This is a minimal fix
to try to resolve the issue without causing other problems.
Fixes bug 31003. I'm calling this a bugfix on 0.1.2.2-alpha, where
the call to router_dir_info_changed() was added to routerlist_free().
When we added LD_MESG, we created a conflict with the LD_NO_MOCK
flag. We now need 64 bits for log domains in order to fix this
issue.
Fixes bug 31080; bugfix on 0.4.1.1-alpha.
Coverity has had trouble figuring out our csiphash implementation,
and has given spurious warnings about its behavior.
This patch changes the csiphash implementation when coverity is in
use, so that coverity can figure out that we are not about to read
beyond the provided input.
Closes ticket 31025.
Coverity doesn't understand that if begin_cell_parse() returns 0 and
sets is_begindir to 0, its address field will always be set.
Fixes bug 30126; bugfix on 0.2.4.7-alpha; Fixes CID 1447296.
Replace the 157 fallbacks originally introduced in Tor 0.3.5.6-rc
in December 2018 (of which ~122 were still functional), with a
list of 148 fallbacks (70 new, 78 existing, 79 removed) generated
in June 2019.
Closes ticket 28795.
Note that we created extra lists from DE and US, but they didn't add
any more fallbacks when we tried to merge them.
When parsing addreses via Tor's internal address:port parsing and
DNS lookup APIs, require IPv6 addresses with ports to have square
brackets.
But allow IPv6 addresses without ports, whether or not they have
square brackets.
Fixes bug 30721; bugfix on 0.2.1.5-alpha.
When parsing addreses via Tor's internal DNS lookup API:
* reject IPv4 addresses in square brackets (with or without a port),
* accept IPv6 addresses in square brackets (with or without a port), and
* accept IPv6 addresses without square brackets, as long as they have no port.
This change completes the work started in 23082, making address parsing
consistent between tor's internal DNS lookup and address parsing APIs.
Fixes bug 30721; bugfix on 0.2.1.5-alpha.
Right now, this has been done at a high level by confparse.c, but it
makes more sense to lower it.
This API is radically un-typesafe as it stands; we'll be wrapping it
in a safer API as we do #30914 and lower the struct manipulation
code as well.
Closes ticket 30864.
If the signature data was removed, but the keyword was kept, this could
result in an unparseable extra info file.
Fixes bug 30958; bugfix on 0.2.7.2-alpha.
Always publish bridge pluggable transport information in the extra info
descriptor, even if ExtraInfoStatistics is 0. This information is
needed by BridgeDB.
Fixes bug 30956; bugfix on 0.4.1.1-alpha.
This will effectively also deny any bridge to be used as a single hop to the
introduction point since bridge do not authenticate like clients.
Fixes#24963
Signed-off-by: David Goulet <dgoulet@torproject.org>
When we consider all circuits in "waiting for guard" state to be promoted to
an "open" state, we were considering all circuits, even the one marked for
close.
This ultiamtely triggers a "circuit_has_opened()" called on the circuit that
is marked for close which then leads to possible undesirable behaviors within
a subsystem.
For instance, the HS subsystem would be unable to find the authentication key
of the introduction point circuit leading to a BUG() warning and a duplicate
mark for close on the circuit.
This commit also adds a unit test to make sure we never select marked for
close circuits when upgrading its guard state from waiting for guard to open.
Fixes#30871
Signed-off-by: David Goulet <dgoulet@torproject.org>
When we consider all circuits in "waiting for guard" state to be promoted to
an "open" state, we were considering all circuits, even the one marked for
close.
This ultiamtely triggers a "circuit_has_opened()" called on the circuit that
is marked for close which then leads to possible undesirable behaviors within
a subsystem.
For instance, the HS subsystem would be unable to find the authentication key
of the introduction point circuit leading to a BUG() warning and a duplicate
mark for close on the circuit.
This commit also adds a unit test to make sure we never select marked for
close circuits when upgrading its guard state from waiting for guard to open.
Fixes#30871
Signed-off-by: David Goulet <dgoulet@torproject.org>
When we consider all circuits in "waiting for guard" state to be promoted to
an "open" state, we were considering all circuits, even the one marked for
close.
This ultiamtely triggers a "circuit_has_opened()" called on the circuit that
is marked for close which then leads to possible undesirable behaviors within
a subsystem.
For instance, the HS subsystem would be unable to find the authentication key
of the introduction point circuit leading to a BUG() warning and a duplicate
mark for close on the circuit.
This commit also adds a unit test to make sure we never select marked for
close circuits when upgrading its guard state from waiting for guard to open.
Fixes#30871
Signed-off-by: David Goulet <dgoulet@torproject.org>
Note that the event base object is _not_ created from the initialize
function, since it is configuration-dependent. This will wait until
configuration is integrated into subsystems.
Closes ticket 30806.
"make test-network-all" shows the warnings from each test-network.sh
run on the console, so developers see new warnings early.
Improve the documentation for this feature, and rename a Makefile
variable so the code is self-documenting.
Fixes bug 30455; bugfix on 0.3.0.4-rc.
This is to avoid having two sendme.{c|h} in the repository since the subsystem
is implemented in src/core/or/sendme.{c|h}.
Fixes#30769
Signed-off-by: David Goulet <dgoulet@torproject.org>
Set the env vars:
* TOR_MASTER_NAME to override the tor master branch name, and
* TOR_WKT_NAME to override the worktree path
Fixes bug 30841; bugfix on 0.4.0.1-alpha.
Skip test_rebind when the TOR_SKIP_TEST_REBIND environmental variable
is set.
Skip test_rebind on macOS in Travis builds, because it is unreliable
on macOS on Travis.
Fixes bug 30713; bugfix on 0.3.5.1-alpha.
Adds ROUTER_AUTHDIR_BUG_ANNOTATIONS to was_router_added_t.
The out-of-order numbering is deliberate: it will be fixed by later commits
for 16564.
Fixes bug 30780; bugfix on 0.2.0.8-alpha.
Leave the other rend and hs_ident data around until circuit free, since code
may still try to inspect it after marking the circuit for close. The
circuitmap is the important thing to clean up, since repurposed
intropoints must be removed from this map to ensure validity.
Leave the other rend and hs_ident data around until circuit free, since code
may still try to inspect it after marking the circuit for close. The
circuitmap is the important thing to clean up, since repurposed
intropoints must be removed from this map to ensure validity.
If "Log debug ..." is not set, the decrement never happens. This lead to the
package/deliver window to be out of sync at the stream level and thus breaking
the connection after 50+ cells.
Fixes#30628
Signed-off-by: David Goulet <dgoulet@torproject.org>
Previously, we were looking at our global settings to see what kind
of proxy we had. But doing this would sometimes give us the wrong
results when we had ClientTransportPlugin configured but we weren't
using it for a particular connection. In several places in the
code, we had added checks to see if we were _really_ using a PT or
whether we were using a socks proxy, but we had forgotten to do so
in at least once case. Instead, since every time we call this
function we are asking about a single connection, it is probably
best just to make this function connection-specific.
Fixes bug 29670; bugfix on 0.2.6.2-alpha.
When we repurpose a hidden service circuit, we need to clean up from the HS
circuit map and any HS related data structured contained in the circuit.
This commit adds an helper function that does it when repurposing a hidden
service circuit.
Fixes#29034
Signed-off-by: David Goulet <dgoulet@torproject.org>
Proposal 289 prevents SENDME-flooding by requiring the other side to
authenticate the data it has received. But this data won't actually
be random if they are downloading a known resource. "No problem",
we said, "let's fell the empty parts of our cells with some
randomness!" and we did that in #26871.
Unfortunately, if the relay data payloads are all completely full,
there won't be any empty parts for us to randomize.
Therefore, we now pick random "randomness windows" between
CIRCWINDOW_INCREMENT/2 and CIRCWINDOW_INCREMENT. We remember whether we have
sent a cell containing at least 16 bytes of randomness in that window. If we
haven't, then when the window is exhausted, we send one. (This window approach
is designed to lower the number of rng checks we have to do. The number 16 is
pulled out of a hat to change the attacker's guessing difficulty to
"impossible".)
Implements 28646.
Previously we purged it in 1-hour increments -- but one-hour is the
maximum TTL for the cache! Now we do it in 25%-TTL increments.
Fixes bug 29617; bugfix on 0.3.5.1-alpha.
This is the first half of implementing proposal 301. The
RecommendedPackages torrc option is marked as obsolete and
the test cases for the option removed. Additionally, the code relating
to generating and formatting package lines in votes is removed.
These lines may still appear in votes from other directory authorities
running earlier versions of the code and so consensuses may still
contain package lines. A new consensus method will be needed to stop
including package lines in consensuses.
Fixes: #28465
Fortunately, in 0.3.5.1-alpha we improved logging for various
failure cases involved with onion service client auth.
Unfortunately, for this one, we freed the file right before logging
its name.
Fortunately, tor_free() sets its pointer to NULL, so we didn't have
a use-after-free bug.
Unfortunately, passing NULL to %s is not defined.
Fortunately, GCC 9.1.1 caught the issue!
Unfortunately, nobody has actually tried building Tor with GCC 9.1.1
before. Or if they had, they didn't report the warning.
Fixes bug 30475; bugfix on 0.3.5.1-alpha.
For memeq and friends, "tor_" indicates constant-time and "fast_"
indicates optimized. I'm fine with leaving the constant-time
"safe_mem_is_zero" with its current name, but the "tor_" prefix on
the current optimized version is misleading.
Also, make the tor_digest*_is_zero() uniformly constant-time, and
add a fast_digest*_is_zero() version to use as needed.
A later commit in this branch will fix all the users of
tor_mem_is_zero().
Closes ticket 30309.
The nodelist_idx for each node_t serves as a unique identifier for
the node, so we can use a bitarray to hold all the excluded
nodes, and then remove them from the smartlist.
Previously use used smartlist_subtract(sl, excluded), which is
O(len(sl)*len(excluded)).
We can use this function in other places too, but this is the one
that showed up on the profiles of 30291.
Closes ticket 30307.
The first line break in particular was mishandled: it was discarded
if no arguments came before it, which made it impossible to
distinguish arguments from the first line of the body.
To solve this, we need to allocate a copy of the command rather than
using NUL to separate it, since we might have "COMMAND\n" as our input.
Fixes ticket 29984.
In this patch we lower the log level of the failures for the three calls
to unlink() in networkstatus_set_current_consensus(). These errors might
trigger on Windows because the memory mapped consensus file keeps the
file in open state even after we have close()'d it. Windows will then
error on the unlink() call with a "Permission denied" error.
The consequences of ignoring these errors is that we leave an unused
file around on the file-system, which is an easier way to fix this
problem right now than refactoring networkstatus_set_current_consensus().
See: https://bugs.torproject.org/29930
In "make test-network-all", test IPv6-only v3 single onion services,
using the chutney network single-onion-v23-ipv6-md. This test will
not pass until 23588 has been merged.
Closes ticket 27251.
Stop ignoring IPv6 link specifiers sent to v3 onion services.
v3 onion service IPv6 support is still incomplete, see 23493 for
details.
Fixes bug 23588; bugfix on 0.3.2.1-alpha.
Patch by Neel Chauhan.
When releasing OpenSSL patch-level maintenance updates,
we do not want to rebuild binaries using it.
And since they guarantee ABI stability, we do not have to.
Without this patch, warning messages were produced
that confused users:
https://bugzilla.opensuse.org/show_bug.cgi?id=1129411
Fixes bug 30190; bugfix on 0.2.4.2-alpha commit 7607ad2bec
Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
The smartlist functions take great care to reset unused pointers inside
the smartlist memory to NULL.
The function smartlist_remove_keeporder does not clear memory in such
way when elements have been removed. Therefore call memset after the
for-loop that removes elements. If no element is removed, it is
effectively a no-op.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Previously, our use of abort() would break anywhere that we didn't
include stdlib.h. This was especially troublesome in case where
tor_assert_nonfatal() was used with ALL_BUGS_ARE_FATAL, since that
one seldom gets tested.
As an alternative, we could have just made this header include
stdlib.h. But that seems bloaty.
Fixes bug 30189; bugfix on 0.3.4.1-alpha.
Fixes bug 29922; bugfix on 0.2.9.3-alpha when we tried to capture
all these warnings. No need to backport any farther than 0.3.5,
though -- these warnings don't cause test failures before then.
This one was tricky to find because apparently it only happened on
_some_ windows builds.
In current NSS versions, these ciphersuites don't work with
SSL_ExportKeyingMaterial(), which was causing relays to fail when
they tried to negotiate the v3 link protocol authentication.
Fixes bug 29241; bugfix on 0.4.0.1-alpha.
When classifying a client's selection of TLS ciphers, if the client
ciphers are not yet available, do not cache the result. Previously,
we had cached the unavailability of the cipher list and never looked
again, which in turn led us to assume that the client only supported
the ancient V1 link protocol. This, in turn, was causing Stem
integration tests to stall in some cases. Fixes bug 30021; bugfix
on 0.2.4.8-alpha.
When we fixed 28614, our answer was "if we failed to load the
consensus on windows and it had a CRLF, retry it." But we logged
the failure at "warn", and we only logged the retry at "info".
Now we log the retry at "notice", with more useful information.
Fixes bug 30004.
Previously we used time(NULL) to set the Expires: header in our HTTP
responses. This made the actual contents of that header untestable,
since the unit tests have no good way to override time(), or to see
what time() was at the exact moment of the call to time() in
dircache.c.
This gave us a race in dir_handle_get/status_vote_next_bandwidth,
where the time() call in dircache.c got one value, and the call in
the tests got another value.
I'm applying our regular solution here: using approx_time() so that
the value stays the same between the code and the test. Since
approx_time() is updated on every event callback, we shouldn't be
losing any accuracy here.
Fixes bug 30001. Bug introduced in fb4a40c32c4a7e5; not in any
released Tor.
Let's use the same function exit point for BUG() codepath that we're using
for every other exit condition. That way, we're not forgetting to clean up
the memarea.
Previously, or_connection_t did not record whether or not the
connection uses a pluggable transport. Instead, it stored the
underlying proxy protocol of the pluggable transport in
proxy_type. This made bootstrap reporting treat pluggable transport
connections as plain proxy connections.
Store a separate bit indicating whether a pluggable transport is in
use, and decode this during bootstrap reporting.
Fixes bug 28925; bugfix on 0.4.0.1-alpha.
When NULL is given to lpApplicationName we enable Windows' "magical"
path interpretation logic, which makes Tor 0.4.x behave in the same way
as previous Tor versions did when it comes to executing binaries in
different system paths.
For more information about this have a look at the CreateProcessA()
documentation on MSDN -- especially the string interpretation example is
useful to understand this issue.
This bug was introduced in commit bfb94dd2ca.
See: https://bugs.torproject.org/29874
so that the relays that would be "excluded" from the bandwidth
file because of something failed can be included to diagnose what
failed, without still including these relays in the bandwidth
authorities vote.
Closes#29806.
The previous commit introduces these changes:
Stop warning about undefined behavior in the probability distribution
tests. Float division by zero may technically be undefined behaviour in
C, but it's well-defined in IEEE 754.
Partial backport of 29298.
Closes ticket 29527; bugfix on 0.4.0.1-alpha.
Allow connections to single onion services to remain idle without being
disconnected.
Relays acting as rendezvous points for single onion services were
mistakenly closing idle established rendezvous circuits after 60 seconds,
thinking that they are unused directory-fetching circuits that had served
their purpose.
Fixes bug 29665; bugfix on 0.2.1.26.
Allow connections to single onion services to remain idle without being
disconnected.
Relays acting as rendezvous points for single onion services were
mistakenly closing idle established rendezvous circuits after 60 seconds,
thinking that they are unused directory-fetching circuits that had served
their purpose.
Fixes bug 29665; bugfix on 0.2.1.26.
- every .h file needs an #ifndef/#define pair.
- They must refer to the same macro.
- The guard macros that they refer to must be unique across all headers.
Cleanup some bugs discovered during 23576:
* stop copying the first 20 characters of a 40-character hex string
to a binary fingerprint
* stop putting IPv6 addresses in a variable called "ipv4"
* explain why we do a duplicate tt_int_op() to deliberately fail and
print a value
Fixes bug 29243; bugfix on 0.3.2.1-alpha.
The previous commits for 23576 confused hs_desc_link_specifier_t
and link_specifier_t. Removing hs_desc_link_specifier_t fixes this
confusion.
Fixes bug 22781; bugfix on 0.3.2.1-alpha.
Refactor the shared random state's memory management so that it actually
takes ownership of the shared random value pointers.
Fixes bug 29706; bugfix on 0.2.9.1-alpha.
Stop leaking parts of the shared random state in the shared-random unit
tests. The previous fix in 29599 was incomplete.
Fixes bug 29706; bugfix on 0.2.9.1-alpha.
Turns out that when reloading a tor configured with hidden service(s), we
weren't copying all the needed information between the old service object to
the new one.
For instance, the desc_is_dirty timestamp wasn't which could lead to the
service uploading its descriptor much later than it would need to.
The replaycache wasn't also moved over and some intro point information as
well.
Fixes#23790
Signed-off-by: David Goulet <dgoulet@torproject.org>
This commit also explicitly set the value of the PRT enum so we can match/pin
the C enum values to the Rust one in protover/ffi.rs.
Fixes#29631
Signed-off-by: David Goulet <dgoulet@torproject.org>
* Move out code that depends on NSS to crypto_digest_nss.c
* Move out code that depends on OpenSSL to crypto_digest_openssl.c
* Keep the general code that is not specific to any of the above in
crypto_digest.c
Prior to #23100, we were not counting HS circuit build times in our
calculation of the timeout. This could lead to a condition where our timeout
was set too low, based on non HS circuit build times, and then we would
abandon all HS circuits, storing no valid timeouts in the histogram.
This commit avoids the assert.
When "auto" was used for the port number for a listening socket, the
message logged after opening the socket would incorrectly say port 0
instead of the actual port used.
Fixes bug 29144; bugfix on 0.3.5.1-alpha
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
Skip the Appveyor 32-bit Windows Server 2016 job, and 64-bit Windows
Server 2012 R2 job. The remaining 2 jobs still provide coverage of
64/32-bit, and Windows Server 2016/2012 R2.
Also set fast_finish, so failed jobs terminate the build immediately.
Fixes bug 29601; bugfix on 0.3.5.4-alpha.
This patch fixes a crash bug (assertion failure) in the PT subsystem
that could get triggered if the user cancels bootstrap via the UI in
TorBrowser. This would cause Tor to call `managed_proxy_destroy()` which
called `process_free()` after it had called `process_terminate()`. This
leads to a crash when the various process callbacks returns with data
after the `process_t` have been freed using `process_free()`.
We solve this issue by ensuring that everywhere we call
`process_terminate()` we make sure to detach the `managed_proxy_t` from
the `process_t` (by calling `process_set_data(process, NULL)`) and avoid
calling `process_free()` at all in the transports code. Instead we just
call `process_terminate()` and let the process exit callback in
`managed_proxy_exit_callback()` handle the `process_free()` call by
returning true to the process subsystem.
See: https://bugs.torproject.org/29562
When IPv4Only (IPv6Only) was used but the address could not be
interpreted as a IPv4 (IPv6) address, the error message referred
to the wrong IP version.
This also fixes up the error-checking logic so it's more precise
about what's being checked.
Fixes bug 13221; bugfix on 0.2.3.9-alpha
Signed-off-by: Kris Katterjohn <katterjohn@gmail.com>
KIST works by computing how much should be allowed to write to the kernel for
a given socket, and then it writes that amount to the outbuf.
The problem is that it could be possible that the outbuf already has lots of
data in it from a previous scheduling round (because the kernel is full/busy
and Tor was not able to flush the outbuf yet). KIST ignores that the outbuf
has been filling (is above its "highwater") and writes more anyway. The end
result is that the outbuf length would exceed INT_MAX, hence causing an
assertion error and a corresponding "Bug()" message to get printed to the
logs.
This commit makes it for KIST to take into account the outbuf length when
computing the available space.
Bug found and patch by Rob Jansen.
Closes#29168. TROVE-2019-001.
Signed-off-by: David Goulet <dgoulet@torproject.org>
