Commit Graph

462 Commits

Author SHA1 Message Date
teor
53b49d1a35 test_dir: Unit tests for RSA-only router and extrainfo descriptor creation
Tests 29017 and 29018.
2019-02-19 21:44:40 +10:00
teor
a9f852a0f6 router: Document the additional config and state used to dump descriptors
Also, explicitly state when routerinfos and extra-infos are signed.
And tidy up some other comments.

Preparation for testing 29017 and 20918.
2019-02-19 21:43:07 +10:00
teor
a1f8558628 router: Move extrainfo signing into its own function
This refactoring improves the structure of router_build_fresh_descriptor().

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
9cab988696 router: eliminate router_update_info_send_unencrypted()
Remove router_update_info_send_unencrypted(), and move its code into the
relevant functions.

Then, re-use an options pointer.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
af0a43be2c router: eliminate tiny router_build_fresh_descriptor() static functions
Remove some tiny static functions called by router_build_fresh_descriptor(),
and move their code into more relevant functions.

Then, give router_update_{router,extra}info_descriptor_body identical layouts.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
a65c101973 router: check for NULL in router_build_fresh_descriptor() static functions
Make sure that these static functions aren't passed NULL.
If they are, log a BUG() warning, and return an error.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:43 +10:00
teor
f19b64dce9 router: refactor router_build_fresh_descriptor() static function interfaces
Tidy the arguments and return values of these functions, and clean up their
memory management.

Preparation for testing 29017 and 20918.
2019-02-19 21:41:36 +10:00
teor
6c5a506cdb router: split router_build_fresh_descriptor() into static functions
Split the body of router_build_fresh_descriptor() into static functions,
by inserting function prologues and epilogues between existing sections.

Write a new body for router_build_fresh_descriptor() that calls the new
static functions.

Initial refactor with no changes to the body of the old
router_build_fresh_descriptor(), except for the split.

Preparation for testing 29017 and 20918.
2019-02-19 19:05:43 +10:00
teor
a798bd40fb stats: Stop reporting statistics when ExtraInfoStatistics is 0
When ExtraInfoStatistics is 0, stop including bandwidth usage statistics,
GeoIPFile hashes, ServerTransportPlugin lines, and bridge statistics
by country in extra-info documents.

Fixes bug 29018; bugfix on 0.2.4.1-alpha (and earlier versions).
2019-02-19 19:01:44 +10:00
teor
361738c964 Merge branch 'bug29017-033' into bug29017-master-merge 2019-02-19 18:59:16 +10:00
Roger Dingledine
94f7e53d04 fix a bootstrapping string typo
introduced in 85542ee5

next step is to fix it in torspec too
2019-02-17 16:56:13 -05:00
George Kadianakis
9bfe4ed6dd Merge branch 'tor-github/pr/536' into maint-0.3.5 2019-02-14 17:39:34 +02:00
juga0
ec7da50ab4 dirvote: Add the bandwidth file digest in the vote 2019-02-13 12:26:00 +00:00
juga0
28490fa23e test: Add test to get the digest of a bw file 2019-02-13 12:26:00 +00:00
juga0
fc3e90a7b6 bwauth: Add function to get the digest of a bw file 2019-02-13 12:26:00 +00:00
David Goulet
95e5f8fe03 Merge branch 'tor-github/pr/671' 2019-02-12 13:02:30 -05:00
Nick Mathewson
72b978c3a5 On windows, if we fail to load a consensus and it has a CRLF, retry.
Fixes bug 28614; bugfix on 0.4.0.1-alpha when we started mmapping
the consensus.
2019-02-12 12:57:33 -05:00
rl1987
4c10221332 Use compress_dir_buf_add() function in a few places 2019-02-09 16:46:31 +02:00
Nick Mathewson
7f59b9fb1f Merge branch 'maint-0.3.5' 2019-02-08 08:37:46 -05:00
Nick Mathewson
ab65347819 Merge branch 'ticket29040_1_changes' into maint-0.3.5 2019-02-08 08:37:43 -05:00
teor
6170d3fcf1 hs: Onion services put IPv6 addresses in service descriptors
Rewrite service_intro_point_new() to take a node_t. Since
node_get_link_specifier_smartlist() supports IPv6 link specifiers,
this refactor adds IPv6 addresses to onion service descriptors.

Part of 23576, implements 26992.
2019-01-31 07:53:22 +01:00
teor
cdda3dc484 hs: Move get_lspecs_from_node to nodelist.c
Also:
* rename to node_get_link_specifier_smartlist
* rewrite to return a smartlist
* add link_specifier_smartlist_free

Part of 23576.
2019-01-30 15:15:41 +01:00
Nick Mathewson
bbd893d6bd Write consensus files in binary mode
This will help us out on windows now that we mmap files.  Fixes part
of ticket 28614.
2019-01-29 16:18:41 +01:00
Neel Chauhan
c985940de9 Add version 3 onion service support to HSFETCH 2019-01-24 10:22:41 -05:00
Suphanat Chunhapanya
238a9080c6 hs-v3: add an option param to safe log functions
We add an option param to safe_str and safe_str_client because in
some case we need to use those functions before global_options is set.
2019-01-24 04:31:18 +07:00
Suphanat Chunhapanya
8de735f068 hs-v3: fix use after free in client auth config
We accidentally use `auth` after freeing it in
client_service_authorization_free. The way to solve it is to
free after using it.
2019-01-24 04:31:07 +07:00
Nick Mathewson
adeeb8841e Merge branch 'maint-0.3.5' 2019-01-23 11:18:14 -05:00
rl1987
712a622fce Log an HSDesc we failed to parse at Debug loglevel 2019-01-23 10:37:10 -05:00
Nick Mathewson
d1af4d65df Merge branch 'maint-0.3.5' 2019-01-18 12:25:08 -05:00
Nick Mathewson
a8580a6836 Merge branch 'maint-0.3.4' into maint-0.3.5 2019-01-18 12:25:08 -05:00
Nick Mathewson
f632335feb Fix users of base32_decode to check for expected length in return.
Also, when we log about a failure from base32_decode(), we now
say that the length is wrong or that the characters were invalid:
previously we would just say that there were invalid characters.

Follow-up on 28913 work.
2019-01-17 13:32:19 -05:00
Nick Mathewson
77712a5fa2 Merge remote-tracking branch 'tor-github/pr/645' 2019-01-17 12:04:50 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
efe55b8898 Bump copyright date to 2019. 2019-01-16 12:32:32 -05:00
Nick Mathewson
b169c8c14f Merge remote-tracking branch 'asn-github/adaptive_padding-final' 2019-01-14 14:48:00 -05:00
Nick Mathewson
691dec5d46 Merge branch 'maint-0.3.5' 2019-01-14 14:02:42 -05:00
rl1987
9d9e71824c Rework rep_hist_log_link_protocol_counts() 2019-01-14 14:01:55 -05:00
Alexander Færøy
59a88b3c3a Rename TYPE to TRANSPORT in PT STATUS messages.
See: https://bugs.torproject.org/28181
2019-01-14 18:35:13 +01:00
Nick Mathewson
d21fa48cac Merge branch 'maint-0.3.5' 2019-01-11 18:53:24 -05:00
Nick Mathewson
efd765a948 Merge remote-tracking branch 'tor-github/pr/563' into maint-0.3.5 2019-01-11 18:53:18 -05:00
Nick Mathewson
2f0e187191 Merge remote-tracking branch 'tor-github/pr/627' 2019-01-09 15:46:14 -05:00
Nick Mathewson
578a93bbbc Merge branch 'ticket28843' 2019-01-09 09:49:25 -05:00
Nick Mathewson
d890ec29aa Merge branch 'maint-0.3.5' 2019-01-09 09:35:40 -05:00
rl1987
380ae2b74b Stop logging 'Your Guard' in circpathbias.c 2019-01-09 09:35:36 -05:00
Nick Mathewson
3df1d29d9b Merge branch 'bug28591_035_squashed' 2019-01-09 09:33:31 -05:00
teor
44db455cc8 Dir: allow directory mirrors to serve future consensuses
When Tor's clock is behind the clocks on the authorities, allow Tor to
serve future consensuses.

Fixes bug 28654; bugfix on 0.3.0.1-alpha.
2019-01-09 09:32:28 -05:00
Nick Mathewson
f9a7701ff1 Merge branch 'ticket28856_v2' 2019-01-09 08:46:40 -05:00
Taylor Yu
7e75311714 Fix typo in bootstrap message
The message for the "ap_conn_proxy" bootstrap status tag was missing
some text.  Fixes bug 28929.  Bug not in any released Tor.
2019-01-04 17:27:28 -06:00
Nick Mathewson
98a1b8770c Merge branch 'maint-0.3.5' 2019-01-03 21:34:43 -05:00
Nick Mathewson
c75cf802d3 Merge branch 'ticket28879' into maint-0.3.5 2019-01-03 21:34:35 -05:00
Nick Mathewson
a7cbbf279e Merge branch 'ticket28852' 2019-01-03 21:31:08 -05:00
rl1987
dbf1725a13 Completely remove 'GETINFO status/version/num-{concurring,versioning}' 2019-01-03 10:36:38 +02:00
George Kadianakis
926fc93be5 Concentrate all TOR_USEC_PER_SEC definitions in a single header file.
Co-authored-by: Mike Perry <mikeperry-git@torproject.org>
2019-01-02 15:25:55 +02:00
Mike Perry
8ad497bb57 Config option to specify specific MiddleNodes.
Hope is this will make it easier to test on the live tor network.

Does not need to be merged if we don't want to, but will come in handy
for researchers.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:25:55 +02:00
Mike Perry
659a4f06d4 Circuit padding ProtoVer plumbing.
This helps us to determine if a middle node can pad to us or not.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:16 +02:00
Mike Perry
70e9245f6f Initialize circuit padding machines and global state.
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2019-01-02 15:12:13 +02:00
Taylor Yu
85542ee5a0 The big bootstrap phase redefinition
Redefine the set of bootstrap phases to allow display of finer-grained
progress in the early connection stages of connecting to a relay.

This includes adding intermediate phases for proxy and PT connections.

Also add a separate new phase to indicate obtaining enough directory
info to build circuits so we can report that independently of actually
initiating an ORCONN to build the first application circuit.
Previously, we would claim to be connecting to a relay when we had
merely finished obtaining directory info.

Part of ticket 27167.
2018-12-21 14:15:35 -05:00
Taylor Yu
936c93e562 Hook up control_event_bootstrap() to btrack_orconn
Replace a few invocations of control_event_bootstrap() with calls from
the bootstrap tracker subsystem.  This mostly leaves behavior
unchanged.  The actual behavior changes come in the next commit.

Part of ticket 27167.
2018-12-21 14:15:35 -05:00
Taylor Yu
9d29abb34e Add a comment about bto_update_best. 2018-12-21 14:15:21 -05:00
Taylor Yu
b0ae6a332a Add bootstrap tracker subsystem
Add a tracker for bootstrap progress, tracking events related to
origin circuit and ORCONN states.  This uses the ocirc_event and
orconn_event publish-subscribe subsystems.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
a0b4fa1f16 Add origin circuit event pubsub system
Add a publish-subscribe subsystem to publish messages about changes to
origin circuits.

Functions in circuitbuild.c and circuitlist.c publish messages to this
subsystem.

Move circuit event constants out of control.h so that subscribers
don't have to include all of control.h to take actions based on
messages they receive.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Taylor Yu
271b50f54a Add ORCONN event pubsub system
Add a publish-subscribe subsystem to publish messages about changes to
OR connections.

connection_or_change_state() in connection_or.c and
control_event_or_conn_event() in control.c publish messages to this
subsystem via helper functions.

Move state constants from connection_or.h to orconn_state.h so that
subscribers don't have to include all of connection_or.h to take
actions based on changes in OR connection state.  Move event constants
from control.h for similar reasons.

Part of ticket 27167.
2018-12-20 18:46:17 -06:00
Alexander Færøy
ab0d7d2dd4 Escape the PT K/V data before sending it to the logger.
See: https://bugs.torproject.org/28846
2018-12-20 19:05:50 +01:00
Nick Mathewson
a517daa56f base32_decode(): Return number of bytes written on success.
This makes it consistent with base64_decode().

Closes ticket 28913.
2018-12-20 08:36:25 -05:00
Alexander Færøy
4efe4cc2f9 Add support for STATUS messages from Pluggable Transports.
This patch adds support for the new STATUS message that PT's can emit
from their standard out. The STATUS message uses the `config_line_t` K/V
format that was recently added in Tor.

See: https://bugs.torproject.org/28846
2018-12-20 03:55:02 +01:00
Alexander Færøy
426c52b377 Use K/V parser to handle LOG messages for pluggable transports.
This patch changes the LOG pluggable transport message to use the recent
K/V parser that landed in Tor. This allows PT's to specify the log
severity level as well as the message. A mapping between the PT log
severity levels and Tor's log serverity level is provided.

See: https://bugs.torproject.org/28846
2018-12-20 03:41:28 +01:00
Nick Mathewson
bb091da1e7 Merge branch 'ticket28839_v2_squashed' 2018-12-18 18:59:05 -05:00
Nick Mathewson
7113a339dc Avoid a needless decode/re-encode step in assigning onion keys
Previously we had decoded the asn.1 to get a public key, and then
discarded the asn.1 so that we had to re-encode the key to store it
in the onion_pkey field of a microdesc_t or routerinfo_t.

Now we can just do a tor_memdup() instead, which should be loads
faster.
2018-12-18 18:58:08 -05:00
Nick Mathewson
0556942284 Use a single path for all PEM-like objects in get_next_token()
Previously, we would decode the PEM wrapper for keys twice: once in
get_next_token, and once later in PEM decode.  Now we just do all of
the wrapper and base64 stuff in get_next_token, and store the
base64-decoded part in the token object for keys and non-keys alike.

This change should speed up parsing slightly by letting us skip a
bunch of stuff in crypto_pk_read_*from_string(), including the tag
detection parts of pem_decode(), and an extra key allocation and
deallocation pair.

Retaining the base64-decoded part in the token object will allow us
to speed up our microdesc parsing, since it is the asn1 portion that
we actually want to retain.
2018-12-18 18:58:08 -05:00
Nick Mathewson
c61cd5775c Revert "Log bootstrap tag names"
This reverts commit 1b855af5e3.
2018-12-18 08:09:43 -05:00
Nick Mathewson
e969d9c6b4 Merge branch 'ticket28179_squashed' into ticket28179_squashed_merged 2018-12-17 16:41:01 -05:00
Alexander Færøy
651cdd05b7 Add an additional space when we check for the PROTO_LOG handler.
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
cacdd29087 Use const char * instead of char * for line parameter for process callbacks.
This patch changes the type definition of the process callbacks to use
`const char *` instead of `char *`.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ec2ae3ed8b Change EVENT_TRANSPORT_LOG to EVENT_PT_LOG.
This patch changes our EVENT_TRANSPORT_LOG event to be EVENT_PT_LOG. The
new message includes the path to the PT executable instead of the
transport name, since one PT binary can include multiple transport they
sometimes might need to log messages that are not specific to a given
transport.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
5585cbd08f Change the Process exit_callback to return bool.
This patch changes our process_t's exit_callback to return a boolean
value.  If the returned value is true, the process subsystem will call
process_free() on the given process_t.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
ccc1963890 Move remaining code from subprocess.{h,c} to more appropriate places.
This patch moves the remaining code from subprocess.{h,c} to more
appropriate places in the process.c and process_win32.c module.

We also delete the now empty subprocess module files.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
f7d13425fc Delete old process_handle_t code.
This patch removes the old process_handle_t code. Everything should by
now be using the process_t interface.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e3ceaebba2 Add support for logging messages from pluggable transports.
This patch adds support for the "LOG" protocol message from a pluggable
transport. This allows pluggable transport developers to relay log
messages from their binary to Tor, which will both emit them as log
messages from the Tor process itself, but also pass them on via the
control port.

See: https://bugs.torproject.org/28180
See: https://bugs.torproject.org/28181
See: https://bugs.torproject.org/28182
2018-12-17 16:39:28 -05:00
Alexander Færøy
bfb94dd2ca Use process_t for managed proxies.
This patch makes the managed proxy subsystem use the process_t data
structure such that we can get events from the PT process while Tor is
running and not just when the PT process is being configured.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Nick Mathewson
16199a54a2 Check hostname before using it in send_resolved_hostname_cell()
Also, turn an absent hostname into a BUG(), not a crash.

Found by scan-build.

Closes ticket 28879; bugfix on 0.1.2.7-alpha
2018-12-17 09:15:37 -05:00
Nick Mathewson
a0fad3981e Replace use of strcmp_len() with new mem_eq_token().
The strcmp_len() function was somewhat misconceived, since we're
only using it to test whether a length+extent string is equal to a
NUL-terminated string or not.  By simplifying it and making it
inlined, we should be able to make it a little faster.

(It *does* show up in profiles.)

Closes ticket 28856.
2018-12-17 09:03:04 -05:00
Nick Mathewson
9dc53bc68f Remove a needless memset() in get_token_arguments()
I believe we originally added this for "just in case" safety, but it
isn't actually needed -- we never copy uninitialized stack here.
What's more, this one memset is showing up on our startup profiles,
so we ought to remove it.

Closes ticket 28852.
2018-12-14 14:48:12 -05:00
Nick Mathewson
6dc90d290d Use 25% less RAM for base64-encoded directory objects
We were allocating N bytes to decode an N-byte base64 encoding,
when 3N/4 would have been enough.
2018-12-14 13:51:51 -05:00
Nick Mathewson
4bc3983f64 Add a DROPOWNERSHIP controller command to undo TAKEOWNERSHIP.
Closes ticket 28843.
2018-12-13 19:35:02 -05:00
Nick Mathewson
f8dac5c900 Merge branch 'maint-0.3.5' 2018-12-13 19:01:29 -05:00
Nick Mathewson
94a7998158 Merge remote-tracking branch 'tlyu-github/ticket28731-035' into maint-0.3.5 2018-12-13 18:57:00 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
ce501a529f Merge remote-tracking branch 'catalyst-github/ticket27402' 2018-12-11 09:37:41 -05:00
Taylor Yu
1b855af5e3 Log bootstrap tag names
Add the bootstrap tag name to the log messages, so people
troubleshooting connection problems can look up a symbol instead of a
number.  Closes ticket 28731.
2018-12-10 17:22:28 -06:00
Nick Mathewson
46a321fbdd Merge branch 'maint-0.3.5' 2018-12-05 10:25:12 -05:00
Nick Mathewson
967efc0d28 Merge remote-tracking branch 'tor-github/pr/546' into maint-0.3.5 2018-12-05 10:23:28 -05:00
Nick Mathewson
1eb3719a62 Merge remote-tracking branch 'public/prop298' 2018-12-05 09:43:03 -05:00
Nick Mathewson
1f95e80351 Merge branch 'prop293_squashed' 2018-12-05 09:24:51 -05:00
Nick Mathewson
a2f81b644b Write tests for mark_my_descriptor_dirty_if_too_old() 2018-12-05 09:24:45 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
David Goulet
cec616a0c8 hs-v3: Don't BUG() if descriptor is found on SOCKS connection retry
When retrying all SOCKS connection because new directory information just
arrived, do not BUG() if a connection in state AP_CONN_STATE_RENDDESC_WAIT is
found to have a usable descriptor.

There is a rare case when this can happen as detailed in #28669 so the right
thing to do is put that connection back in circuit wait state so the
descriptor can be retried.

Fixes #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:34:04 -05:00
David Goulet
43bd4d7509 hs-v3: Add the helper function mark_conn_as_waiting_for_circuit
This helper function marks an entry connection as pending for a circuit and
changes its state to AP_CONN_STATE_CIRCUIT_WAIT. The timestamps are set to
now() so it can be considered as new.

No behaviour change, this helper function will be used in next commit.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:34:04 -05:00
David Goulet
00b59d9281 conn: Use connection_ap_mark_as_waiting_for_renddesc()
Use the helper function connection_ap_mark_as_waiting_for_renddesc()
introduced in previous commit everywhere in the code where an AP connection
state is transitionned to AP_CONN_STATE_RENDDESC_WAIT.

Part of #28669

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-12-04 14:10:00 -05:00
Nick Mathewson
417a324a85 Make input argument const in set_routerstatus_from_routerinfo. 2018-12-03 12:34:29 -05:00
Nick Mathewson
32213fa9ad Keep list of dirauth flags in sync between dirvote.c and fuzz_vrs.c
Suggested by Teor on PR
2018-12-03 12:18:45 -05:00