Commit Graph

404 Commits

Author SHA1 Message Date
Taylor Yu
936c93e562 Hook up control_event_bootstrap() to btrack_orconn
Replace a few invocations of control_event_bootstrap() with calls from
the bootstrap tracker subsystem.  This mostly leaves behavior
unchanged.  The actual behavior changes come in the next commit.

Part of ticket 27167.
2018-12-21 14:15:35 -05:00
Taylor Yu
9d29abb34e Add a comment about bto_update_best. 2018-12-21 14:15:21 -05:00
Taylor Yu
b0ae6a332a Add bootstrap tracker subsystem
Add a tracker for bootstrap progress, tracking events related to
origin circuit and ORCONN states.  This uses the ocirc_event and
orconn_event publish-subscribe subsystems.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
a0b4fa1f16 Add origin circuit event pubsub system
Add a publish-subscribe subsystem to publish messages about changes to
origin circuits.

Functions in circuitbuild.c and circuitlist.c publish messages to this
subsystem.

Move circuit event constants out of control.h so that subscribers
don't have to include all of control.h to take actions based on
messages they receive.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
271b50f54a Add ORCONN event pubsub system
Add a publish-subscribe subsystem to publish messages about changes to
OR connections.

connection_or_change_state() in connection_or.c and
control_event_or_conn_event() in control.c publish messages to this
subsystem via helper functions.

Move state constants from connection_or.h to orconn_state.h so that
subscribers don't have to include all of connection_or.h to take
actions based on changes in OR connection state.  Move event constants
from control.h for similar reasons.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Alexander Færøy
ab0d7d2dd4 Escape the PT K/V data before sending it to the logger.
See: https://bugs.torproject.org/28846
2018-12-20 19:05:50 +01:00
Alexander Færøy
4efe4cc2f9 Add support for STATUS messages from Pluggable Transports.
This patch adds support for the new STATUS message that PT's can emit
from their standard out. The STATUS message uses the `config_line_t` K/V
format that was recently added in Tor.

See: https://bugs.torproject.org/28846
2018-12-20 03:55:02 +01:00
Alexander Færøy
426c52b377 Use K/V parser to handle LOG messages for pluggable transports.
This patch changes the LOG pluggable transport message to use the recent
K/V parser that landed in Tor. This allows PT's to specify the log
severity level as well as the message. A mapping between the PT log
severity levels and Tor's log serverity level is provided.

See: https://bugs.torproject.org/28846
2018-12-20 03:41:28 +01:00
Nick Mathewson
bb091da1e7 Merge branch 'ticket28839_v2_squashed' 2018-12-18 18:59:05 -05:00
Nick Mathewson
7113a339dc Avoid a needless decode/re-encode step in assigning onion keys
Previously we had decoded the asn.1 to get a public key, and then
discarded the asn.1 so that we had to re-encode the key to store it
in the onion_pkey field of a microdesc_t or routerinfo_t.

Now we can just do a tor_memdup() instead, which should be loads
faster.
2018-12-18 18:58:08 -05:00
Nick Mathewson
0556942284 Use a single path for all PEM-like objects in get_next_token()
Previously, we would decode the PEM wrapper for keys twice: once in
get_next_token, and once later in PEM decode.  Now we just do all of
the wrapper and base64 stuff in get_next_token, and store the
base64-decoded part in the token object for keys and non-keys alike.

This change should speed up parsing slightly by letting us skip a
bunch of stuff in crypto_pk_read_*from_string(), including the tag
detection parts of pem_decode(), and an extra key allocation and
deallocation pair.

Retaining the base64-decoded part in the token object will allow us
to speed up our microdesc parsing, since it is the asn1 portion that
we actually want to retain.
2018-12-18 18:58:08 -05:00
Nick Mathewson
c61cd5775c Revert "Log bootstrap tag names"
This reverts commit 1b855af5e3.
2018-12-18 08:09:43 -05:00
Nick Mathewson
e969d9c6b4 Merge branch 'ticket28179_squashed' into ticket28179_squashed_merged 2018-12-17 16:41:01 -05:00
Alexander Færøy
651cdd05b7 Add an additional space when we check for the PROTO_LOG handler.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
cacdd29087 Use const char * instead of char * for line parameter for process callbacks.
This patch changes the type definition of the process callbacks to use
`const char *` instead of `char *`.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ec2ae3ed8b Change EVENT_TRANSPORT_LOG to EVENT_PT_LOG.
This patch changes our EVENT_TRANSPORT_LOG event to be EVENT_PT_LOG. The
new message includes the path to the PT executable instead of the
transport name, since one PT binary can include multiple transport they
sometimes might need to log messages that are not specific to a given
transport.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
5585cbd08f Change the Process exit_callback to return bool.
This patch changes our process_t's exit_callback to return a boolean
value.  If the returned value is true, the process subsystem will call
process_free() on the given process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ccc1963890 Move remaining code from subprocess.{h,c} to more appropriate places.
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.

We also delete the now empty subprocess module files.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
f7d13425fc Delete old process_handle_t code.
This patch removes the old process_handle_t code. Everything should by
now be using the process_t interface.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e3ceaebba2 Add support for logging messages from pluggable transports.
This patch adds support for the "LOG" protocol message from a pluggable
transport. This allows pluggable transport developers to relay log
messages from their binary to Tor, which will both emit them as log
messages from the Tor process itself, but also pass them on via the
control port.

See: https://bugs.torproject.org/28180
See: https://bugs.torproject.org/28181
See: https://bugs.torproject.org/28182
2018-12-17 16:39:28 -05:00
Alexander Færøy
bfb94dd2ca Use process_t for managed proxies.
This patch makes the managed proxy subsystem use the process_t data
structure such that we can get events from the PT process while Tor is
running and not just when the PT process is being configured.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Nick Mathewson
16199a54a2 Check hostname before using it in send_resolved_hostname_cell()
Also, turn an absent hostname into a BUG(), not a crash.

Found by scan-build.

Closes ticket 28879; bugfix on 0.1.2.7-alpha
2018-12-17 09:15:37 -05:00
Nick Mathewson
a0fad3981e Replace use of strcmp_len() with new mem_eq_token().
The strcmp_len() function was somewhat misconceived, since we're
only using it to test whether a length+extent string is equal to a
NUL-terminated string or not.  By simplifying it and making it
inlined, we should be able to make it a little faster.

(It *does* show up in profiles.)

Closes ticket 28856.
2018-12-17 09:03:04 -05:00
Nick Mathewson
9dc53bc68f Remove a needless memset() in get_token_arguments()
I believe we originally added this for "just in case" safety, but it
isn't actually needed -- we never copy uninitialized stack here.
What's more, this one memset is showing up on our startup profiles,
so we ought to remove it.

Closes ticket 28852.
2018-12-14 14:48:12 -05:00
Nick Mathewson
6dc90d290d Use 25% less RAM for base64-encoded directory objects
We were allocating N bytes to decode an N-byte base64 encoding,
when 3N/4 would have been enough.
2018-12-14 13:51:51 -05:00
Nick Mathewson
4bc3983f64 Add a DROPOWNERSHIP controller command to undo TAKEOWNERSHIP.
Closes ticket 28843.
2018-12-13 19:35:02 -05:00
Nick Mathewson
f8dac5c900 Merge branch 'maint-0.3.5' 2018-12-13 19:01:29 -05:00
Nick Mathewson
94a7998158 Merge remote-tracking branch 'tlyu-github/ticket28731-035' into maint-0.3.5 2018-12-13 18:57:00 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
ce501a529f Merge remote-tracking branch 'catalyst-github/ticket27402' 2018-12-11 09:37:41 -05:00
Taylor Yu
1b855af5e3 Log bootstrap tag names
Add the bootstrap tag name to the log messages, so people
troubleshooting connection problems can look up a symbol instead of a
number.  Closes ticket 28731.
2018-12-10 17:22:28 -06:00
Nick Mathewson
46a321fbdd Merge branch 'maint-0.3.5' 2018-12-05 10:25:12 -05:00
Nick Mathewson
967efc0d28 Merge remote-tracking branch 'tor-github/pr/546' into maint-0.3.5 2018-12-05 10:23:28 -05:00
Nick Mathewson
1eb3719a62 Merge remote-tracking branch 'public/prop298' 2018-12-05 09:43:03 -05:00
Nick Mathewson
1f95e80351 Merge branch 'prop293_squashed' 2018-12-05 09:24:51 -05:00
Nick Mathewson
a2f81b644b Write tests for mark_my_descriptor_dirty_if_too_old() 2018-12-05 09:24:45 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
David Goulet
cec616a0c8 hs-v3: Don't BUG() if descriptor is found on SOCKS connection retry
When retrying all SOCKS connection because new directory information just
arrived, do not BUG() if a connection in state AP_CONN_STATE_RENDDESC_WAIT is
found to have a usable descriptor.

There is a rare case when this can happen as detailed in #28669 so the right
thing to do is put that connection back in circuit wait state so the
descriptor can be retried.

Fixes #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:34:04 -05:00
David Goulet
43bd4d7509 hs-v3: Add the helper function mark_conn_as_waiting_for_circuit
This helper function marks an entry connection as pending for a circuit and
changes its state to AP_CONN_STATE_CIRCUIT_WAIT. The timestamps are set to
now() so it can be considered as new.

No behaviour change, this helper function will be used in next commit.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:34:04 -05:00
David Goulet
00b59d9281 conn: Use connection_ap_mark_as_waiting_for_renddesc()
Use the helper function connection_ap_mark_as_waiting_for_renddesc()
introduced in previous commit everywhere in the code where an AP connection
state is transitionned to AP_CONN_STATE_RENDDESC_WAIT.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:10:00 -05:00
Nick Mathewson
417a324a85 Make input argument const in set_routerstatus_from_routerinfo. 2018-12-03 12:34:29 -05:00
Nick Mathewson
32213fa9ad Keep list of dirauth flags in sync between dirvote.c and fuzz_vrs.c
Suggested by Teor on PR
2018-12-03 12:18:45 -05:00
Nick Mathewson
8221b5d587 Merge remote-tracking branch 'tor-github/pr/559' 2018-12-02 19:38:40 -05:00
teor
612b21b8ea
comment: replace cached-routers with cached-descriptors
cached-routers has been gone for a long time
2018-12-03 10:19:34 +10:00
Nick Mathewson
2b2b97484a Merge branch 'ticket27490a_squashed' 2018-12-01 20:32:18 -05:00
Neel Chauhan
822cb93cab Add new option ClientAutoIPv6ORPort to switch between IPv4 and IPv6 OR ports 2018-12-01 14:55:57 -05:00
Nick Mathewson
51d94cea33 Merge branch 'maint-0.3.5' 2018-12-01 11:26:55 -05:00
Nick Mathewson
7e9985b75a Merge remote-tracking branch 'tor-github/pr/536' 2018-12-01 11:24:02 -05:00
Taylor Yu
7685f8ad35 Use table lookup for bootstrap_status_to_string
It also no longer distinguishes the case of internal-only paths, which
was often wrong anyway.  Closes ticket 27402.
2018-11-30 16:54:01 -06:00
Taylor Yu
1fe6507d29 Split bootstrap event reporting out of control.c
Part of ticket 27402.
2018-11-30 16:49:44 -06:00
Neel Chauhan
d18a167ff3 sr: Switch from tor_assert() to BUG()
Closes #19566

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-11-30 12:16:18 -05:00
teor
7a45bc74a4 Dir: when Tor's clock is behind, use a future consensus to bootstrap
When Tor's clock is behind the clocks on the authorities, allow Tor to
bootstrap successfully.

Fixes bug 28591; bugfix on 0.2.0.9-alpha.
2018-11-29 00:50:24 +10:00
Alexander Færøy
2a3eef4404 Remove unused int pid member of managed_proxy_t.
See: https://bugs.torproject.org/28179
2018-11-27 19:31:08 +01:00
Nick Mathewson
7d8e0cc9ab Merge branch 'dormant_v2_squashed' 2018-11-26 16:33:31 -05:00
Nick Mathewson
02843c4a4e Test for check_network_participation_callback() 2018-11-26 16:32:40 -05:00
Nick Mathewson
881b85cf32 Treat the StaleDesc flag as making our descriptor dirty.
Relay side of prop293.
2018-11-25 10:12:20 -05:00
Nick Mathewson
36f808c936 Vote on the StaleDesc flag from prop293
The StaleDesc flag tells relays that they need to upload a new
descriptor soon, or they will drop out of the consensus.
2018-11-25 10:05:13 -05:00
Nick Mathewson
7da06e43da No longer exit for missing required protocolversions on an old consensus
Specifically, if the consensus is older than the (estimted or
measured) release date for this version of tor, we assume that the
required versions may have changed in between that consensus and
this release.

Implements ticket 27735 and proposal 297.
2018-11-24 20:44:37 -05:00
Nick Mathewson
05dee063c8 Emit router families in canonical form
This patch has routers use the same canonicalization logic as
authorities when encoding their family lists.  Additionally, they
now warn if any router in their list is given by nickname, since
that's error-prone.

This patch also adds some long-overdue tests for family formatting.
2018-11-24 16:35:58 -05:00
Nick Mathewson
f82eb6269f Extract get_declared_family() into its own function.
This will help as we refactor it.
2018-11-24 12:11:40 -05:00
Nick Mathewson
0a0c612b79 Add a consensus method in which md families get canonicalized.
Implements prop298. Closes ticket 28266.
2018-11-24 12:01:09 -05:00
Nick Mathewson
d29e3a02d5 Add a function to canonicalize nodefamilies per prop298
This is the same as the regular canonical nodefamily format, except
that unrecognized elements are preserved.
2018-11-24 10:53:38 -05:00
Nick Mathewson
0e9a963b6b Revise nodefamily.c to match proposal 298
Prop298 says that family entries should be formatted with
$hexids in uppercase, nicknames in lower case, $hexid~names
truncated, and everything sorted lexically.  These changes implement
that ordering for nodefamily.c.

We don't _strictly speaking_ need to nodefamily.c formatting use
this for prop298 microdesc generation, but it seems silly to have
two separate canonicalization algorithms.
2018-11-24 10:30:15 -05:00
teor
cebc39bcd5
Test: make unit tests use a reasonably live consensus
Cleanup after 24661.
2018-11-22 16:54:46 +10:00
teor
657618ba9b
Entry Nodes: Mark outdated dirservers in reasonably live consensuses
Fixes bug 28569; bugfix on Tor 0.3.2.5-alpha.
2018-11-22 16:54:34 +10:00
teor
d1ac5613fc
Entry Nodes: Use a reasonably live consensus to select guards
Fixes bug 24661; bugfix on 0.3.0.1-alpha.
2018-11-22 16:54:22 +10:00
Nick Mathewson
34cadefe34 Merge branch 'maint-0.3.5' 2018-11-20 09:04:35 -05:00
Taylor Yu
0489288aa2 Update control_free_all() for #27169
Reset the added bootstrap tracking state introduced by ticket 27169.
Fixes bug 28524; bugfix on 0.3.5.1-alpha.
2018-11-19 15:48:08 -06:00
Nick Mathewson
48b08f0592 Merge branch 'ticket27359_v2_squashed' 2018-11-19 08:26:49 -05:00
Nick Mathewson
4f9548f893 Expose more nodelist.c functions to tests 2018-11-19 08:26:10 -05:00
Nick Mathewson
426c9561c5 Use nodefamily_t in microdescriptors.
Closes ticket 27359.
2018-11-19 08:26:10 -05:00
Nick Mathewson
83be4d2bbd Backend for compact node-family representation.
This representation is meant to save memory in microdescriptors --
we can't use it in routerinfo_t yet, since those families need to be
encoded losslessly for directory voting to work.

This representation saves memory in three ways:
   1. It uses only one allocation per family.  (The old way used a
      smartlist (2 allocs) plus one strdup per entry.)
   2. It stores identity digests in binary, not hex.
   3. It keeps families in a canonical format, memoizes, and
      reference-counts them.

Part of #27359.
2018-11-19 08:26:10 -05:00
Nick Mathewson
35558c39dd Merge remote-tracking branch 'dgoulet/ticket27471_035_02' into maint-0.3.5 2018-11-16 08:57:56 -05:00
Nick Mathewson
2c15b65381 Make the NET_PARTICIPANT role dependent on user activity
This patch implements all of 28337, except for the part where we
turn off the role if we've been idle for a long time.
2018-11-15 11:17:22 -05:00
Nick Mathewson
4bf79fa4fa Turn second_elapsed_callback into a normal periodic event. 2018-11-15 11:17:22 -05:00
Nick Mathewson
303e5c70e0 Move the responsibility for delayed shutdown into the mainloop
This is part of 28422, so we don't have to call
consider_hibernation() once per second when we're dormant.

This commit does not remove delayed shutdown from hibernate.c: it
uses it as a backup shutdown mechanism, in case the regular shutdown
timer mechanism fails for some reason.
2018-11-15 11:17:22 -05:00
Nick Mathewson
a0380b705d Move control_per_second_events() into a callback with its own role
Part of making extra-dormant mode work; closes ticket 28421.
2018-11-15 11:17:22 -05:00
Nick Mathewson
f6b8c7da66 Move buffers.c out of lib/containers to resolve a circularity. 2018-11-14 16:07:03 -05:00
Nick Mathewson
cec58ae55c Merge branch 'maint-0.3.5' 2018-11-14 07:56:52 -05:00
Nick Mathewson
a58b19465d Merge remote-tracking branch 'teor/bug28441-035' into maint-0.3.5 2018-11-14 07:56:48 -05:00
Nick Mathewson
586c3a7c90 Merge branch 'maint-0.3.5' 2018-11-14 07:42:56 -05:00
teor
16ca6fdfdb
log: stop talking about the Named flag in log messages
Clients have ignored the Named flag since 0.3.2.

Fixes bug 28441; bugfix on 0.3.2.1-alpha.
2018-11-14 18:16:34 +10:00
teor
a7aa3f76ec
comment: Fix a typo in nodes_in_same_family() 2018-11-14 18:06:05 +10:00
David Goulet
342f2b1873 Merge branch 'tor-github/pr/501' 2018-11-13 10:48:23 -05:00
David Goulet
8fb318860e Merge branch 'maint-0.3.5' 2018-11-13 10:43:03 -05:00
David Goulet
6f2151be9a Merge branch 'tor-github/pr/487' into maint-0.3.5 2018-11-13 10:37:25 -05:00
Neel Chauhan
ec93385cb2 Comment for rend_cache_failure in feature/rend/rendcache.c: "usuable" should be "usable" 2018-11-13 10:33:51 -05:00
Nick Mathewson
bf4f55a13d Merge branch 'subsystems' 2018-11-09 15:01:49 -05:00
Nick Mathewson
8e3bae566a Merge branch 'maint-0.3.5' 2018-11-07 16:27:26 -05:00
Nick Mathewson
212bd9778b Make the hibernate.c changes for systemd compile. 2018-11-07 16:27:20 -05:00
Nick Mathewson
8020d6fb05 Merge remote-tracking branch 'tor-github/pr/464' 2018-11-06 15:33:25 -05:00
Nick Mathewson
ba28704b29 Merge branch 'maint-0.3.5' 2018-11-06 15:22:11 -05:00
Nick Mathewson
c60f3ea607 Merge remote-tracking branch 'tor-github/pr/474' into maint-0.3.5 2018-11-06 15:21:45 -05:00
Nick Mathewson
6e7ff8cba0 Move the code that knows our tor version into a lowest-level lib 2018-11-05 09:22:02 -05:00
Nick Mathewson
674ef53a7e Add a warning if we can't write networkstatus-bridges
Fixes CID 1440818.
2018-11-02 13:32:43 -04:00
Nick Mathewson
18a4eaf5c1 Avoid mmap leak if we get a consensus diff we can't use.
Fixes CID 1440819; bug not in any released Tor.
2018-11-02 13:30:55 -04:00
Nick Mathewson
865514e66e Merge branch 'ticket28100_squashed' 2018-11-02 13:19:24 -04:00
Alex Xu (Hello71)
1b75de85b3 Don't overwrite the Content-Type when compressing 2018-11-02 13:19:14 -04:00
Fernando Fernandez Mancera
f60607ee96 Improve log message in hs_service.c
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
2018-11-01 12:40:52 +02:00
Nick Mathewson
383b44553c Merge branch 'ticket24393_036_01_squashed' 2018-10-31 11:07:15 -04:00