release: ChangeLog and ReleaseNotes for 0.4.7.7

ChangeLog

@@ -1,3 +1,27 @@
+Changes in version 0.4.7.7 - 2022-04-27
+  This is the first stable version of the 0.4.7.x series. This series includes
+  several major bugfixes from previous series and one massive new feature:
+  congestion control.
+
+  Congestion control should improve traffic speed and stability on the network
+  once a majority of Exit upgrade. You can find more details about it in
+  proposal 324 in the torspec.git repository.
+
+  For a complete list of changes since 0.4.6.10, see the ReleaseNotes file.
+
+  o Minor features (fallbackdir):
+    - Regenerate fallback directories generated on April 27, 2022.
+
+  o Minor features (geoip data):
+    - Update the geoip files to match the IPFire Location Database, as
+      retrieved on 2022/04/27.
+
+  o Minor bugfixes (congestion control, client side logs):
+    - Demote a warn about 1-hop circuits using congestion control down to
+      info; Demote the 4-hop case to notice. Fixes bug 40598; bugfix on
+      0.4.5-alpha.
+
+
 Changes in version 0.4.7.6-rc - 2022-04-07
   This is the first release candidate of the 0.4.7.x series. Only one minor
   bugfix went in since the last alpha couple weeks ago. We strongly recommend

ReleaseNotes

@@ -2,6 +2,426 @@ This document summarizes new features and bugfixes in each stable
 release of Tor. If you want to see more detailed descriptions of the
 changes in each development snapshot, see the ChangeLog file.
 
+Changes in version 0.4.7.7 - 2022-04-27
+  This is the first stable version of the 0.4.7.x series. This series
+  includes several major bugfixes from previous series and several features
+  including one massive new one: congestion control.
+
+  Congestion control, among other things, should improve traffic speed and
+  stability on the network once a majority of Exit upgrade. You can find more
+  details about it in proposal 324 in the torspec.git repository.
+
+  Below are all the changes of the 0.4.7.x series:
+
+  o Major features (congestion control):
+    - Implement support for flow control over congestion controlled
+      circuits. This work comes from proposal 324. Closes ticket 40450.
+
+  o Major features (relay, client, onion services):
+    - Implement RTT-based congestion control for exits and onion
+      services, from Proposal 324. Disabled by default. Enabled by the
+      'cc_alg' consensus parameter. Closes ticket 40444.
+
+  o Major features (directory authority):
+    - Add a new consensus method to handle MiddleOnly specially. When
+      enough authorities are using this method, then any relay tagged
+      with the MiddleOnly flag will have its Exit, Guard, HSDir, and
+      V2Dir flags automatically cleared, and will have its BadExit flag
+      automatically set. Implements part of proposal 335.
+    - Authorities can now be configured to label relays as "MiddleOnly".
+      When voting for this flag, authorities automatically vote against
+      Exit, Guard, HSDir, and V2Dir; and in favor of BadExit. Implements
+      part of proposal 335. Based on a patch from Neel Chauhan.
+
+  o Major features (Proposal 332, onion services, guard selection algorithm):
+    - Clients and onion services now choose four long-lived "layer 2"
+      guard relays for use as the middle hop in all onion circuits.
+      These relays are kept in place for a randomized duration averaging
+      1 week. This mitigates guard discovery attacks against clients and
+      short-lived onion services such as OnionShare. Long-lived onion
+      services that need high security should still use the Vanguards
+      addon (https://github.com/mikeperry-tor/vanguards). Closes ticket
+      40363; implements proposal 333.
+
+  o Major bugfix (relay, metrics):
+    - On the MetricsPort, the DNS error statistics are not reported by
+      record type ("record=...") anymore due to a libevent bug
+      (https://github.com/libevent/libevent/issues/1219). Fixes bug
+      40490; bugfix on 0.4.7.1-alpha.
+
+  o Major bugfixes (bridges):
+    - Make Tor work reliably again when you have multiple bridges
+      configured and one or more of them are unreachable. The problem
+      came because we require that we have bridge descriptors for both
+      of our first two bridges (else we refuse to try to connect), but
+      in some cases we would wait three hours before trying to fetch
+      these missing descriptors, and/or never recover when we do try to
+      fetch them. Fixes bugs 40396 and 40495; bugfix on 0.3.0.5-rc
+      and 0.3.2.1-alpha.
+
+  o Major bugfixes (client):
+    - Stop caching TCP connect failures to relays/bridges when we
+      initiated the connection as a client. Now we only cache connect
+      failures as a relay or bridge when we initiated them because of an
+      EXTEND request. Declining to re-attempt the client-based
+      connections could cause problems when we lose connectivity and try
+      to reconnect. Fixes bug 40499; bugfix on 0.3.3.4-alpha.
+
+  o Major bugfixes (onion service, congestion control):
+    - Fix the onion service upload case where the congestion control
+      parameters were not added to the right object. Fixes bug 40586;
+      bugfix on 0.4.7.4-alpha.
+
+  o Major bugfixes (relay, DNS):
+    - Lower the DNS timeout from 3 attempts at 5 seconds each to 2
+      attempts at 1 seconds each. Two new consensus parameters were
+      added to control these values. This change should improve observed
+      performance under DNS load; see ticket for more details. Fixes bug
+      40312; bugfix on 0.3.5.1-alpha.
+
+  o Major bugfixes (relay, overload state):
+    - Relays report the general overload state for DNS timeout errors
+      only if X% of all DNS queries over Y seconds are errors. Before
+      that, it only took 1 timeout to report the overload state which
+      was just too low of a threshold. The X and Y values are 1% and 10
+      minutes respectively but they are also controlled by consensus
+      parameters. Fixes bug 40491; bugfix on 0.4.6.1-alpha.
+
+  o Major bugfixes (relay, overload):
+    - Do not trigger a general overload on DNS timeout. Even after
+      fixing 40527, some code remained that triggered the overload.
+      Fixes bug 40564; bugfix on 0.4.7.1-alpha.
+    - Change the MetricsPort DNS "timeout" label to be "tor_timeout" in
+      order to indicate that this was a DNS timeout from tor perspective
+      and not the DNS server itself.
+    - Deprecate overload_dns_timeout_period_secs and
+      overload_dns_timeout_scale_percent consensus parameters as well.
+      They were used to assess the overload state which is no more now.
+    - Don't make Tor DNS timeout trigger an overload general state.
+      These timeouts are different from DNS server timeout. They have to
+      be seen as timeout related to UX and not because of a network
+      problem. Fixes bug 40527; bugfix on 0.4.6.1-alpha.
+
+  o Minor feature (authority, relay):
+    - Reject End-Of-Life relays running version 0.4.2.x, 0.4.3.x,
+      0.4.4.x and 0.4.5 alphas/rc. Closes ticket 40480.
+    - Reject End-Of-Life relays running version 0.3.5.x. Closes
+      ticket 40559.
+
+  o Minor feature (onion service v2):
+    - Onion service v2 addresses are now not recognized anymore by tor
+      meaning a bad hostname is returned when attempting to pass it on a
+      SOCKS connection. No more deprecation log is emitted client side.
+      Closes ticket 40476.
+    - See https://blog.torproject.org/v2-deprecation-timeline for
+      details on how to transition from v2 to v3.
+
+  o Minor feature (reproducible build):
+    - The repository can now build reproducible tarballs which adds the
+      build command "make dist-reprod" for that purpose. Closes
+      ticket 26299.
+
+  o Minor features (bridge testing support):
+    - Let external bridge reachability testing tools discard cached
+      bridge descriptors when setting new bridges, so they can be sure
+      to get a clean reachability test. Implements ticket 40209.
+
+  o Minor features (compilation):
+    - Give an error message if trying to build with a version of
+      LibreSSL known not to work with Tor. (There's an incompatibility
+      with LibreSSL versions 3.2.1 through 3.4.0 inclusive because of
+      their incompatibility with OpenSSL 1.1.1's TLSv1.3 APIs.) Closes
+      ticket 40511.
+
+  o Minor features (control port):
+    - Provide congestion control fields on CIRC_BW and STREAM control
+      port events, for use by sbws. Closes ticket 40568.
+
+  o Minor features (fallbackdir):
+    - Regenerate fallback directories generated on April 27, 2022.
+
+  o Minor features (fuzzing):
+    - When building with --enable-libfuzzer, use a set of compiler flags
+      that works with more recent versions of the library. Previously we
+      were using a set of flags from 2017. Closes ticket 40407.
+
+  o Minor features (geoip data):
+    - Update the geoip files to match the IPFire Location Database, as
+      retrieved on 2022/04/27.
+
+  o Minor features (logging, heartbeat):
+    - When a relay receives a cell that isn't encrypted properly for it,
+      but the relay is the last hop on the circuit, the relay now counts
+      how many cells of this kind it receives, on how many circuits, and
+      reports this information in the log. Previously, we'd log each
+      cell at PROTOCOL_WARN level, which is far too verbose to be
+      useful. Fixes part of ticket 40400.
+
+  o Minor features (portability):
+    - Try to prevent a compiler warning about printf arguments that
+      could sometimes occur on MSYS2 depending on the configuration.
+      Closes ticket 40355.
+
+  o Minor features (testing configuration):
+    - When TestingTorNetwork is enabled, skip the permissions check on
+      hidden service directories. Closes ticket 40338.
+    - On a testing network, relays can now use the
+      TestingMinTimeToReportBandwidth option to change the smallest
+      amount of time over which they're willing to report their observed
+      maximum bandwidth. Previously, this was fixed at 1 day. For
+      safety, values under 2 hours are only supported on testing
+      networks. Part of a fix for ticket 40337.
+    - Relays on testing networks no longer rate-limit how frequently
+      they are willing to report new bandwidth measurements. Part of a
+      fix for ticket 40337.
+    - Relays on testing networks now report their observed bandwidths
+      immediately from startup. Previously, they waited until they had
+      been running for a full day. Closes ticket 40337.
+
+  o Minor features (testing):
+    - We now have separate fuzzers for the inner layers of v3 onion
+      service descriptors, to prevent future bugs like 40392. Closes
+      ticket 40488.
+
+  o Minor bugfix (CI, onion service):
+    - Exclude onion service version 2 Stem tests in our CI. Fixes bug
+      40500; bugfix on 0.3.2.1-alpha.
+
+  o Minor bugfix (logging):
+    - Update a log notice dead URL to a working one. Fixes bug 40544;
+      bugfix on 0.3.5.1-alpha.
+
+  o Minor bugfix (pluggable transport):
+    - Do not kill a managed proxy if one of its transport configurations
+      emits a method error. Instead log a warning and continue processing
+      method arguments. Fixes bug 7362; bugfix on 0.2.3.6-alpha.
+
+  o Minor bugfix (relay):
+    - Remove the HSDir and HSIntro onion service v2 protocol versions so
+      relay stop advertising that they support them. Fixes bug 40509;
+      bugfix on 0.3.5.17.
+
+  o Minor bugfixes (bridges):
+    - When we don't yet have a descriptor for one of our bridges,
+      disable the entry guard retry schedule on that bridge. The entry
+      guard retry schedule and the bridge descriptor retry schedule can
+      conflict, e.g. where we mark a bridge as "maybe up" yet we don't
+      try to fetch its descriptor yet, leading Tor to wait (refusing to
+      do anything) until it becomes time to fetch the descriptor. Fixes
+      bug 40497; bugfix on 0.3.0.3-alpha.
+
+  o Minor bugfixes (cell scheduling):
+    - Avoid writing empty payload with NSS write.
+    - Don't attempt to write 0 bytes after a cell scheduling loop. No
+      empty payload was put on the wire. Fixes bug 40548; bugfix
+      on 0.3.5.1-alpha.
+
+  o Minor bugfixes (circuit padding):
+    - Don't send STOP circuit padding cells when the other side has
+      already shut down the corresponding padding machine. Fixes bug
+      40435; bugfix on 0.4.0.1-alpha.
+
+  o Minor bugfixes (compatibility):
+    - Fix compatibility with the most recent Libevent versions, which no
+      longer have an evdns_set_random_bytes() function. Because this
+      function has been a no-op since Libevent 2.0.4-alpha, it is safe
+      for us to just stop calling it. Fixes bug 40371; bugfix
+      on 0.2.1.7-alpha.
+
+  o Minor bugfixes (compilation):
+    - Fix compilation error when __NR_time is not defined. Fixes bug
+      40465; bugfix on 0.2.5.5-alpha. Patch by Daniel Pinto.
+    - Fix our configuration logic to detect whether we had OpenSSL 3:
+      previously, our logic was reversed. This has no other effect than
+      to change whether we suppress deprecated API warnings. Fixes bug
+      40429; bugfix on 0.3.5.13.
+    - Resume being able to build on old / esoteric gcc versions. Fixes
+      bug 40550; bugfix on 0.4.7.1-alpha.
+
+  o Minor bugfixes (compiler warnings):
+    - Fix couple compiler warnings on latest Ubuntu Jammy. Fixes bug
+      40516; bugfix on 0.3.5.1-alpha.
+
+  o Minor bugfixes (congestion control, client side logs):
+    - Demote a warn about 1-hop circuits using congestion control down
+      to info; Demote the 4-hop case to notice. Fixes bug 40598; bugfix
+      on 0.4.5-alpha.
+
+  o Minor bugfixes (control, sandbox):
+    - Allows the control command SAVECONF to succeed when the seccomp
+      sandbox is enabled. Makes SAVECONF keep only one backup file, to
+      simplify implementation. Fixes bug 40317; bugfix on 0.2.5.4-alpha.
+      Patch by Daniel Pinto.
+
+  o Minor bugfixes (controller, path bias):
+    - When a circuit's path is specified, in full or in part, from the
+      controller API, do not count that circuit towards our path-bias
+      calculations. (Doing so was incorrect, since we cannot tell
+      whether the controller is selecting relays randomly.) Resolves a
+      "Bug" warning. Fixes bug 40515; bugfix on 0.2.4.10-alpha.
+
+  o Minor bugfixes (dirauth, bandwidth scanner):
+    - Add the AuthDirDontVoteOnDirAuthBandwidth dirauth config parameter
+      to avoid voting on bandwidth scanner weights to v3 directory
+      authorities. Fixes bug 40471; bugfix on 0.2.2.1-alpha. Patch by
+      Neel Chauhan.
+
+  o Minor bugfixes (DNSPort, dormant mode):
+    - A request on the DNSPort now wakes up a dormant tor. Fixes bug
+      40577; bugfix on 0.3.5.1-alpha.
+
+  o Minor bugfixes (fragile-hardening, sandbox):
+    - When building with --enable-fragile-hardening, add or relax Linux
+      seccomp rules to allow AddressSanitizer to execute normally if the
+      process terminates with the sandbox active. This has the side
+      effect of disabling the filtering of file- and directory-open
+      requests on most systems and dilutes the effectiveness of the
+      sandbox overall, as a wider range of system calls must be
+      permitted. Fixes bug 11477; bugfix on 0.2.5.4-alpha.
+
+  o Minor bugfixes (heartbeat):
+    - Adjust the heartbeat log message about distinct clients to
+      consider the HeartbeatPeriod rather than a flat 6-hour delay.
+      Fixes bug 40330; bugfix on 0.2.6.3-alpha.
+
+  o Minor bugfixes (logging):
+    - If a channel has never received or transmitted a cell, or seen a
+      client, do not calculate time diffs against 1/1/1970 but log a
+      better prettier message. Fixes bug 40182; bugfix on 0.2.4.4.
+    - When we no longer have enough directory information to use the
+      network, we would log a notice-level message -- but we would not
+      reliably log a message when we recovered and resumed using the
+      network. Now make sure there is always a corresponding message
+      about recovering. Fixes bug 40496; bugfix on 0.3.5.1-alpha.
+
+  o Minor bugfixes (logging, relay):
+    - Add spaces between the "and" when logging the "Your server has not
+      managed to confirm reachability for its" on dual-stack relays.
+      Fixes bug 40453; bugfix on 0.4.5.1-alpha. Patch by Neel Chauhan.
+
+  o Minor bugfixes (metrics port, onion service):
+    - Fix the metrics with a port label to be unique. Before this, all
+      ports of an onion service would be on the same line which violates
+      the Prometheus rules of unique labels. Fixes bug 40581; bugfix
+      on 0.4.5.1-alpha.
+
+  o Minor bugfixes (onion service congestion control):
+    - Avoid a non-fatal assertion failure in the case where we fail to
+      set up congestion control on a rendezvous circuit. This could
+      happen naturally if a cache entry expired at an unexpected time.
+      Fixes bug 40576; bugfix on 0.4.7.4-alpha.
+
+  o Minor bugfixes (onion service):
+    - Do not flag an HSDir as non-running in case the descriptor upload
+      or fetch fails. An onion service closes pending directory
+      connections before uploading a new descriptor which leads to
+      wrongly flagging many relays and thus affecting circuit path
+      selection. Fixes bug 40434; bugfix on 0.2.0.13-alpha.
+    - Fix a warning BUG that would occur often on heavily loaded onion
+      service leading to filling the logs with useless warnings. Fixes
+      bug 34083; bugfix on 0.3.2.1-alpha.
+
+  o Minor bugfixes (onion service, client):
+    - Fix a rare but fatal assertion failure due to a guard subsystem
+      recursion triggered by the onion service client. Fixes bug 40579;
+      bugfix on 0.3.5.1-alpha.
+
+  o Minor bugfixes (onion service, config):
+    - Fix a memory leak for a small config line string that could occur
+      if the onion service failed to be configured from file properly.
+      Fixes bug 40484; bugfix on 0.3.2.1-alpha.
+
+  o Minor bugfixes (onion service, TROVE-2021-008):
+    - Only log v2 access attempts once total, in order to not pollute
+      the logs with warnings and to avoid recording the times on disk
+      when v2 access was attempted. Note that the onion address was
+      _never_ logged. This counts as a Low-severity security issue.
+      Fixes bug 40474; bugfix on 0.4.5.8.
+    - Note that due to #40476 which removes v2 support entirely, this
+      log line is not emitted anymore. We still mention this in the
+      changelog because it is a Low-severity TROVE.
+
+  o Minor bugfixes (performance, DoS):
+    - Fix one case of a not-especially viable denial-of-service attack
+      found by OSS-Fuzz in our consensus-diff parsing code. This attack
+      causes a lot small of memory allocations and then immediately
+      frees them: this is only slow when running with all the sanitizers
+      enabled. Fixes one case of bug 40472; bugfix on 0.3.1.1-alpha.
+
+  o Minor bugfixes (relay):
+    - Reject IPv6-only DirPorts. Our reachability self-test forces
+      DirPorts to be IPv4, but our configuration parser allowed them to
+      be IPv6-only, which led to an assertion failure. Fixes bug 40494;
+      bugfix on 0.4.5.1-alpha.
+
+  o Minor bugfixes (relay, overload):
+    - Decide whether to signal overload based on a fraction and
+      assessment period of ntor handshake drops. Previously, a single
+      drop could trigger an overload state, which caused many false
+      positives. Fixes bug 40560; bugfix on 0.4.7.1-alpha.
+
+  o Minor bugfixes (sandbox):
+    - Fix the sandbox on i386 by modifying it to allow the
+      "clock_gettime64" and "statx" system calls and to filter the
+      "chown32" and "stat64" system calls in place of "chown" and
+      "stat", respectively. Fixes bug 40505; bugfix on 0.2.5.4-alpha.
+
+  o Minor bugfixes (statistics):
+    - Fix a fencepost issue when we check stability_last_downrated where
+      we called rep_hist_downrate_old_runs() twice. Fixes bug 40394;
+      bugfix on 0.2.0.5-alpha. Patch by Neel Chauhan.
+
+  o Minor bugfixes (tests):
+    - Fix a bug that prevented some tests from running with the correct
+      names. Fixes bug 40365; bugfix on 0.4.3.1-alpha.
+
+  o Minor bugfixes (usability):
+    - Do not log "RENDEZVOUS1 cell with unrecognized rendezvous cookie"
+      at LOG_PROTOCOL_WARN; instead log it at DEBUG. This warning can
+      happen naturally if a client gives up on a rendezvous circuit
+      after sending INTRODUCE1. Fixes part of bug 40400; bugfix
+      on 0.1.1.13-alpha.
+    - Do not log "circuit_receive_relay_cell failed" at
+      LOG_PROTOCOL_WARN; instead log it at DEBUG. In every case where we
+      would want to log this as a protocol warning, we are already
+      logging another warning from inside circuit_receive_relay_cell.
+      Fixes part of bug 40400; bugfix on 0.1.1.9-alpha.
+
+  o Code simplification and refactoring:
+    - Lower the official maximum for "guard-extreme-restriction-percent"
+      to 100. This has no effect on when the guard code will generate a
+      warning, but it makes the intent of the option clearer. Fixes bug
+      40486; bugfix on 0.3.0.1-alpha.
+
+  o Documentation:
+    - Add links to original tor design paper and anonbib to
+      docs/HACKING/README.1st.md. Closes ticket 33742. Patch from
+      Emily Bones.
+    - Describe the "fingerprint-ed25519" file in the tor.1 man page.
+      Fixes bug 40467; bugfix on 0.4.3.1-alpha. Patch by Neel Chauhan.
+    - Provide an improved version of the tor-exit-notice.html file for
+      exit relays to use as a landing page. The text is unchanged, but
+      the page design and layout are significantly modernized, and
+      several links are fixed. Patch from "n_user"; closes ticket 40529.
+
+  o Testing:
+    - Add unit tests for the Linux seccomp sandbox. Resolves
+      issue 16803.
+
+  o Code simplification and refactoring (rust):
+    - Remove Rust support and its associated code. It is unsupported and
+      Rust focus should be shifted to arti. Closes ticket 40469.
+
+  o Documentation (man, relay):
+    - Missing "OverloadStatistics" in tor.1 manpage. Fixes bug 40504;
+      bugfix on 0.4.6.1-alpha.
+
+  o Testing (CI, chutney):
+    - Bump the data size that chutney transmits to 5MBytes in order to
+      trigger the flow control and congestion window code. Closes
+      ticket 40485.
+
+
 Changes in version 0.4.6.10 - 2022-02-04
   This version contains minor bugfixes but one in particular is that relays
   don't advertise onion service v2 support at the protocol version level.

changes/bug40598

@@ -1,5 +0,0 @@
-  o Minor bugfixes (congestion control, client side logs):
-    - Demote a warn about 1-hop circuits using congestion control down to
-      info; Demote the 4-hop case to notice. Fixes bug 40598; bugfix on
-      0.4.5-alpha.
-

changes/fallbackdirs-2022-04-27

@@ -1,2 +0,0 @@
-  o Minor features (fallbackdir):
-    - Regenerate fallback directories generated on April 27, 2022.

changes/geoip-2022-04-27

@@ -1,3 +0,0 @@
-  o Minor features (geoip data):
-    - Update the geoip files to match the IPFire Location Database,
-      as retrieved on 2022/04/27.