Commit Graph

37258 Commits

Author SHA1 Message Date
Nick Mathewson
bc21ed3290 Merge remote-tracking branch 'tor-gitlab/mr/316' into maint-0.4.5 2021-02-22 15:37:31 -05:00
David Goulet
d98c77b78e relay: Reduce streaming compression ratio from HIGH to LOW
Fixes #40301

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-22 14:47:42 -05:00
Alexander Færøy
a4df1e8ea4 Merge branch 'maint-0.4.5' 2021-02-22 19:13:12 +00:00
Alexander Færøy
26c2e843f9 Merge remote-tracking branch 'tor-gitlab/mr/309' into maint-0.4.5 2021-02-22 19:12:53 +00:00
Nick Mathewson
85dcd4ce07 Use rangemap 0.1.10
0.1.9 fixed a range-coalescing bug; 0.1.10 fixed a performance
regression.
2021-02-22 13:22:38 -05:00
Nick Mathewson
dcad439e94 script to generate a commit for an updated geoip file 2021-02-22 12:41:03 -05:00
Nick Mathewson
0d4237839b Rust tool to convert IPFire Location dump into CSV format.
The IPFire people provide a tool that collects data from several
top-level sources, combines it into a single database, and annotates
it with optional overrides.  This tool transforms the "dump" format
of their database into the form Tor expects.
2021-02-22 12:25:18 -05:00
David Goulet
4d7f31b964 relay: Move log notice after suggested address lookup
When trying to find our address to publish, we would log notice if we couldn't
find it from the cache but then we would look at the suggested cache (which
contains the address from the authorities) in which we might actually have the
address.

Thus that log notice was misplaced. Move it down after the suggested address
cache lookup.

Closes #40300

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-22 09:13:54 -05:00
David Goulet
9541ed63a1 relay: Only authorities publish a DirPort
Relay will always publish 0 as DirPort value in their descriptor from now on
except authorities.

Related to #40282

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-22 08:52:15 -05:00
David Goulet
38649b4f95 relay: Remove dirport reachability self test
Regular relays are about to get their DirPort removed so that reachability
test is not useful anymore

Authorities will still use the DirPort but because network reentry towards
their DirPort is now denied network wide, this test is not useful anymore and
so it should simply be considered reachable at all time.

Part of #40282

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-22 08:52:06 -05:00
George Kadianakis
c0589d06be Fix a test failure in test_hs_control_add_onion_helper_add_service().
This bug made the pipeline fail. It basically tries to access a service we just
freed because it's still on the service list.

It only occurs about once every 10 tests and it looks like this:

$ ./src/test/test hs_control/hs_control_add_onion_helper_add_service
hs_control/hs_control_add_onion_helper_add_service: [forking] =================================================================
==354311==ERROR: AddressSanitizer: heap-use-after-free on address 0x613000000940 at pc 0x55a159251b03 bp 0x7ffc6abb5b30 sp 0x7ffc6abb5b28
READ of size 8 at 0x613000000940 thread T0
^[[A
    #0 0x55a159251b02 in hs_service_ht_HT_FIND_P_ src/feature/hs/hs_service.c:153
    #1 0x55a159251b02 in hs_service_ht_HT_FIND src/feature/hs/hs_service.c:153
    #2 0x55a159251b02 in find_service src/feature/hs/hs_service.c:175
    #3 0x55a159251c2c in register_service src/feature/hs/hs_service.c:188
    #4 0x55a159262379 in hs_service_add_ephemeral src/feature/hs/hs_service.c:3811
    #5 0x55a158e865e6 in test_hs_control_add_onion_helper_add_service src/test/test_hs_control.c:847
    #6 0x55a1590fe77b in testcase_run_bare_ src/ext/tinytest.c:107
    #7 0x55a1590fee98 in testcase_run_forked_ src/ext/tinytest.c:201
    #8 0x55a1590fee98 in testcase_run_one src/ext/tinytest.c:267
    #9 0x55a1590ffb06 in tinytest_main src/ext/tinytest.c:454
    #10 0x55a158b1b1a4 in main src/test/testing_common.c:420
    #11 0x7f7f06f8dd09 in __libc_start_main ../csu/libc-start.c:308
    #12 0x55a158b21f69 in _start (/home/f/Computers/tor/mytor/src/test/test+0x372f69)

0x613000000940 is located 64 bytes inside of 344-byte region [0x613000000900,0x613000000a58)
freed by thread T0 here:
    #0 0x7f7f0774ab6f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:123
    #1 0x55a158e86508 in test_hs_control_add_onion_helper_add_service src/test/test_hs_control.c:838
    #2 0x55a1590fe77b in testcase_run_bare_ src/ext/tinytest.c:107
    #3 0x55a1590fee98 in testcase_run_forked_ src/ext/tinytest.c:201
    #4 0x55a1590fee98 in testcase_run_one src/ext/tinytest.c:267
    #5 0x55a1590ffb06 in tinytest_main src/ext/tinytest.c:454
    #6 0x55a158b1b1a4 in main src/test/testing_common.c:420
    #7 0x7f7f06f8dd09 in __libc_start_main ../csu/libc-start.c:308

previously allocated by thread T0 here:
    #0 0x7f7f0774ae8f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x55a15948b728 in tor_malloc_ src/lib/malloc/malloc.c:45
    #2 0x55a15948b7c0 in tor_malloc_zero_ src/lib/malloc/malloc.c:71
    #3 0x55a159261bb5 in hs_service_new src/feature/hs/hs_service.c:4290
    #4 0x55a159261f49 in hs_service_add_ephemeral src/feature/hs/hs_service.c:3758
    #5 0x55a158e8619f in test_hs_control_add_onion_helper_add_service src/test/test_hs_control.c:832
    #6 0x55a1590fe77b in testcase_run_bare_ src/ext/tinytest.c:107
    #7 0x55a1590fee98 in testcase_run_forked_ src/ext/tinytest.c:201
    #8 0x55a1590fee98 in testcase_run_one src/ext/tinytest.c:267
    #9 0x55a1590ffb06 in tinytest_main src/ext/tinytest.c:454
    #10 0x55a158b1b1a4 in main src/test/testing_common.c:420
    #11 0x7f7f06f8dd09 in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: heap-use-after-free src/feature/hs/hs_service.c:153 in hs_service_ht_HT_FIND_P_
Shadow bytes around the buggy address:
  0x0c267fff80d0: 00 00 00 00 00 00 00 00 00 00 00 00 fa fa fa fa
  0x0c267fff80e0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c267fff80f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff8100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff8110: 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
=>0x0c267fff8120: fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd
  0x0c267fff8130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c267fff8140: fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa
  0x0c267fff8150: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c267fff8160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c267fff8170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==354311==ABORTING
[Lost connection!]
  [hs_control_add_onion_helper_add_service FAILED]
1/1 TESTS FAILED. (0 skipped)
2021-02-22 13:31:29 +02:00
George Kadianakis
428819f5dd Fix CID 1473232 in connection_ap_handle_onion().
Now that v2 is off the table, 'rend_cache_lookup_result' is useless in
connection_ap_handle_onion() because it can only take the ENOENT value.  Let's
remove that helper variable and handle the ENOENT case specifically when we
check the cache.

Also remove the 'onion_address' helper variable.
2021-02-22 12:58:23 +02:00
George Kadianakis
32fc8a116a Refactoring: Remove 'addresstype' from connection_ap_handle_onion().
It's all v3 now.

Preparation for fixing CID 1473232.
2021-02-22 12:58:18 +02:00
George Kadianakis
c0a2330314 Fix CID 1473233 in handle_control_hsfetch().
With v2 support for HSFETCH gone, we only support v3 addresses. We don't
support v2 descriptor IDs anymore and hence we can remove that code.

The code removed would ensure that if a v2 descriptor ID was provided, the user
also had to provide HSDirs explicitly.

In the v3 case, the code should work even if no HSDirs are provided, and Tor
would find the HSDirs itself.
2021-02-22 12:57:43 +02:00
Neel Chauhan
88559aca2d Make dirauths vote the Sybil flag when other flags are zeroed out 2021-02-19 10:44:11 -08:00
David Goulet
8ccfd4a51a hs-v2: Warn of v2 obsolete if configured
For a user using "HiddenServiceVersion 2", a log warning is emitted indicating
that v2 is now obsolete instead of a confusing message saying that the version
is not supported.

Also, if an introduction point gets a legacy (v2) ESTABLISH_INTRO, we'll
simply close the circuit without emitting a protocol warning log onto the
relay.

Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
6e6ad44262 test: Exclude HSv2 test from Stem CI driver
Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
8577243cd2 hs: Remove last artefact of HSv2 in the code
Closes #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
9782cd2c9b hs: Remove v2 code from HS circuitmap
Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
897ec8a34c hs: Remove v2 stats
We still keep v2 rendezvous stats since we will allow them until the network
has entirely phased out.

Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
9f2e74cac7 test: Change chutney test to use HSv3 only
Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
330aec798b man: Cleanup of v2 options and config
Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
90c1a1f8b1 or: Cleanup or.h header of HSv2 code
Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
ca04e9b8ed hs: Remove introduction point v2 support
Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:49 -05:00
David Goulet
2c865542b6 hs-v2: Removal of service and relay support
This is unfortunately massive but both functionalities were extremely
intertwined and it would have required us to actually change the HSv2 code in
order to be able to split this into multiple commits.

After this commit, there are still artefacts of v2 in the code but there is no
more support for service, intro point and HSDir.

The v2 support for rendezvous circuit is still available since that code is
the same for the v3 and we will leave it in so if a client is able to
rendezvous on v2 then it can still transfer traffic. Once the entire network
has moved away from v2, we can remove v2 rendezvous point support.

Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:20:48 -05:00
David Goulet
a34885bc80 hs: Rename service config port object
Remove it from rendservice.c and move everything related to hs_common.{c|h}.

Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:19:08 -05:00
David Goulet
2444629c86 hs-v2: Remove client support
Related to #40266

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-19 13:19:08 -05:00
Nick Mathewson
ede88c374c Disable the dump_desc() function.
It can be called with strings that should have been
length-delimited, but which in fact are not.  This can cause a
CPU-DoS bug or, in a worse case, a crash.

Since this function isn't essential, the best solution for older
Tors is to just turn it off.

Fixes bug 40286; bugfix on 0.2.2.1-alpha when dump_desc() was
introduced.
2021-02-19 12:31:19 -05:00
George Kadianakis
67938e0198 Merge branch 'maint-0.4.5' 2021-02-19 13:26:52 +02:00
George Kadianakis
03c686563b Merge remote-tracking branch 'tor-gitlab/mr/308' into maint-0.4.5 2021-02-19 13:26:33 +02:00
George Kadianakis
cf1f03fe1d Merge remote-tracking branch 'tor-gitlab/mr/308' 2021-02-19 13:25:47 +02:00
George Kadianakis
e8cce1058c Merge remote-tracking branch 'tor-gitlab/mr/305' 2021-02-19 13:24:34 +02:00
Mike Perry
b2f025cb56 Merge branch 'bug40168+34088-035-v3' into bug40168+34088-035-v3-master 2021-02-18 18:17:18 +00:00
Mike Perry
917f8beb54 Add CBT unit test for Xm and alpha estimation. 2021-02-18 11:21:25 -06:00
Mike Perry
a0690f079d Update documentation for the number of modes for Xm estimator. 2021-02-18 11:21:25 -06:00
Mike Perry
525195a715 Bug 34088: Changes file 2021-02-18 11:21:25 -06:00
Mike Perry
d16b3d12a1 Bug 34088: Remove max timeout calculation and warning.
With the maximum likelihood estimator for alpha from #40168, we no longer need
max_time to calculate alpha.
2021-02-18 11:21:25 -06:00
Mike Perry
1fadb1bdf1 Bug 40168 changes file 2021-02-18 11:21:25 -06:00
Mike Perry
ed9d60cb92 Fix Xm mode calculation to properly average N=10 modes.
This is still fast enough. ~100usec on my laptop with 1000 build times.
2021-02-18 11:21:25 -06:00
Mike Perry
406400a74d Lower circuit build time bin width to 10ms.
50ms is not enough resolution. CBT can be as low as 80ms in datacenter
clients close to their relays.
2021-02-18 11:21:25 -06:00
Mike Perry
86acd4d940 Log circuit timeout in milliseconds 2021-02-18 11:21:25 -06:00
Mike Perry
c90b0cb6fb Raise the circuit close time quantile to 99.
This should allow us to more accurately estimate pareto parameters
without relying on "right-censorship" of circuit build timeout values.
2021-02-18 11:21:25 -06:00
Mike Perry
761dd9f2ab Lower min circ timeout from 1.5s to bin width (10ms) 2021-02-18 11:21:25 -06:00
Mike Perry
37b2159150 Completely ignore abandoned circs from circ timeout calc
This prevents the timeout curve from getting spread out as much, resulting in
more accurate timeout values for quantiles from 60-80.
2021-02-18 11:21:25 -06:00
David Goulet
2709828494 Merge branch 'tor-gitlab/mr/311' 2021-02-17 13:22:36 -05:00
Nick Mathewson
aebd0a4080 Make our configure.ac script use the portable = comparison
Back in 0.4.5.1-alpha we added an `==` comparison, which isn't
portable.

Fixes bug #40298; bugfix on 0.4.5.1-alpha.
2021-02-17 12:36:23 -05:00
Roger Dingledine
8a8045c788 relay: No longer test dirport reachability for authorities
Now that exit relays don't allow exit connections to directory authority
DirPorts, the follow-up step is to make directory authorities stop doing
DirPort reachability checks.

Fixes #40287

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-17 10:28:18 -05:00
David Goulet
9f61d9238c metrics: Handle the connection finished flushing event
Turns out, we forgot to add the METRICS connection type fo the finished
flushing handler.

Fixes #40295

Signed-off-by: David Goulet <dgoulet@torproject.org>
2021-02-17 09:54:05 -05:00
Nick Mathewson
1b74c49048 copy changelog entries forward 2021-02-15 13:09:30 -05:00
Nick Mathewson
67fdb1cd55 Merge branch 'maint-0.4.5'
"ours" to avoid version bump.
2021-02-15 13:00:20 -05:00