diff --git a/.gitignore b/.gitignore index 46917c311f..9dcf3a4fe6 100644 --- a/.gitignore +++ b/.gitignore @@ -108,8 +108,10 @@ # /src/common/ /src/common/Makefile /src/common/Makefile.in +/src/common/common_sha1.i /src/common/libor.a /src/common/libor-crypto.a +/src/common/libor-event.a # /src/config/ /src/config/Makefile @@ -121,6 +123,7 @@ # /src/or/ /src/or/Makefile /src/or/Makefile.in +/src/or/or_sha1.i /src/or/micro-revision.* /src/or/tor /src/or/test diff --git a/ChangeLog b/ChangeLog index ed76e22f95..733e222253 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,7 +1,120 @@ -Changes in version 0.2.1.19 - 2009-07-28 +Changes in version 0.2.2.1-alpha - 2009-08-26 + o Security fixes: + - Start the process of disabling ".exit" address notation, since it + can be used for a variety of esoteric application-level attacks + on users. To reenable it, set "AllowDotExit 1" in your torrc. Fix + on 0.0.9rc5. + + o New directory authorities: + - Set up urras (run by Jacob Appelbaum) as the seventh v3 directory + authority. + + o Major features: + - New AccelName and AccelDir options add support for dynamic OpenSSL + hardware crypto acceleration engines. + - Tor now supports tunneling all of its outgoing connections over + a SOCKS proxy, using the SOCKS4Proxy and/or SOCKS5Proxy + configuration options. Code by Christopher Davis. + o Major bugfixes: - - Make accessing hidden services on 0.2.1.x work right - again. Bugfix on 0.2.1.3-alpha; workaround for bug 1038. + - Send circuit or stream sendme cells when our window has decreased + by 100 cells, not when it has decreased by 101 cells. Bug uncovered + by Karsten when testing the "reduce circuit window" performance + patch. Bugfix on the 54th commit on Tor -- from July 2002, + before the release of Tor 0.0.0. This is the new winner of the + oldest-bug prize. + + o New options for gathering stats safely: + - Directories that set "DirReqStatistics 1" write statistics on + directory request to disk every 24 hours. As compared to the + --enable-geoip-stats flag in 0.2.1.x, there are a few improvements: + 1) stats are written to disk exactly every 24 hours; 2) estimated + shares of v2 and v3 requests are determined as mean values, not at + the end of a measurement period; 3) unresolved requests are listed + with country code '??'; 4) directories also measure download times. + - Exit nodes that set "ExitPortStatistics 1" write statistics on the + number of exit streams and transferred bytes per port to disk every + 24 hours. + - Relays that set "CellStatistics 1" write statistics on how long + cells spend in their circuit queues to disk every 24 hours. + - Entry nodes that set "EntryStatistics 1" write statistics on the + rough number and origins of connecting clients to disk every 24 + hours. + - Relays that write any of the above statistics to disk and set + "ExtraInfoStatistics 1" include the past 24 hours of statistics in + their extra-info documents. + + o Minor features: + - New --digests command-line switch to output the digests of the + source files Tor was built with. + - The "torify" script now uses torsocks where available. + - The memarea code now uses a sentinel value at the end of each area + to make sure nothing writes beyond the end of an area. This might + help debug some conceivable causes of bug 930. + - Time and memory units in the configuration file can now be set to + fractional units. For example, "2.5 GB" is now a valid value for + AccountingMax. + - Certain Tor clients (such as those behind check.torproject.org) may + want to fetch the consensus in an extra early manner. To enable this + a user may now set FetchDirInfoExtraEarly to 1. This also depends on + setting FetchDirInfoEarly to 1. Previous behavior will stay the same + as only certain clients who must have this information sooner should + set this option. + - Instead of adding the svn revision to the Tor version string, report + the git commit (when we're building from a git checkout). + + o Minor bugfixes: + - If any the v3 certs we download are unparseable, we should actually + notice the failure so we don't retry indefinitely. Bugfix on + 0.2.0.x; reported by "rotator". + - If the cached cert file is unparseable, warn but don't exit. + - Fix possible segmentation fault on directory authorities. Bugfix on + 0.2.1.14-rc. + - When Tor fails to parse a descriptor of any kind, dump it to disk. + Might help diagnosing bug 1051. + + o Deprecated and removed features: + - The controller no longer accepts the old obsolete "addr-mappings/" + or "unregistered-servers-" GETINFO values. + - Hidden services no longer publish version 0 descriptors, and clients + do not request or use version 0 descriptors. However, the old hidden + service authorities still accept and serve version 0 descriptors + when contacted by older hidden services/clients. + - The EXTENDED_EVENTS and VERBOSE_NAMES controller features are now + always on; using them is necessary for correct forward-compatible + controllers. + - Remove support for .noconnect style addresses. Nobody was using + them, and they provided another avenue for detecting Tor users + via application-level web tricks. + + +Changes in version 0.2.1.20 - 2009-??-?? + o Major bugfixes: + - Send circuit or stream sendme cells when our window has decreased + by 100 cells, not when it has decreased by 101 cells. Bug uncovered + by Karsten when testing the "reduce circuit window" performance + patch. Bugfix on the 54th commit on Tor -- from July 2002, + before the release of Tor 0.0.0. This is the new winner of the + oldest-bug prize. + + o New directory authorities: + - Set up urras (run by Jacob Appelbaum) as the seventh v3 directory + authority. + + o Minor bugfixes: + - Fix a signed/unsigned compile warning in 0.2.1.19. + - Fix possible segmentation fault on directory authorities. Bugfix on + 0.2.1.14-rc. + + +Changes in version 0.2.1.19 - 2009-07-28 + Tor 0.2.1.19 fixes a major bug with accessing and providing hidden + services on Tor 0.2.1.3-alpha through 0.2.1.18. + + o Major bugfixes: + - Make accessing hidden services on 0.2.1.x work right again. + Bugfix on 0.2.1.3-alpha; workaround for bug 1038. Diagnosis and + part of patch provided by "optimist". o Minor features: - When a relay/bridge is writing out its identity key fingerprint to @@ -20,6 +133,12 @@ Changes in version 0.2.1.19 - 2009-07-28 Changes in version 0.2.1.18 - 2009-07-24 + Tor 0.2.1.18 lays the foundations for performance improvements, + adds status events to help users diagnose bootstrap problems, adds + optional authentication/authorization for hidden services, fixes a + variety of potential anonymity problems, and includes a huge pile of + other features and bug fixes. + o Build fixes: - Add LIBS=-lrt to Makefile.am so the Tor RPMs use a static libevent. diff --git a/Doxyfile.in b/Doxyfile.in index b4d21c334d..24355f5f04 100644 --- a/Doxyfile.in +++ b/Doxyfile.in @@ -1,4 +1,3 @@ -# $Id$ # Doxyfile 1.5.1 # This file describes the settings to be used by the documentation system diff --git a/Makefile.am b/Makefile.am index 39ea570dd7..aa679e556e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,4 +1,3 @@ -# $Id$ # Copyright (c) 2001-2004, Roger Dingledine # Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson # Copyright (c) 2007-2009, The Tor Project, Inc. diff --git a/ReleaseNotes b/ReleaseNotes index f4e1a00cfb..bd2e3090a3 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -3,7 +3,38 @@ This document summarizes new features and bugfixes in each stable release of Tor. If you want to see more detailed descriptions of the changes in each development snapshot, see the ChangeLog file. +Changes in version 0.2.1.19 - 2009-07-28 + Tor 0.2.1.19 fixes a major bug with accessing and providing hidden + services. + + o Major bugfixes: + - Make accessing hidden services on 0.2.1.x work right again. + Bugfix on 0.2.1.3-alpha; workaround for bug 1038. Diagnosis and + part of patch provided by "optimist". + + o Minor features: + - When a relay/bridge is writing out its identity key fingerprint to + the "fingerprint" file and to its logs, write it without spaces. Now + it will look like the fingerprints in our bridges documentation, + and confuse fewer users. + + o Minor bugfixes: + - Relays no longer publish a new server descriptor if they change + their MaxAdvertisedBandwidth config option but it doesn't end up + changing their advertised bandwidth numbers. Bugfix on 0.2.0.28-rc; + fixes bug 1026. Patch from Sebastian. + - Avoid leaking memory every time we get a create cell but we have + so many already queued that we refuse it. Bugfix on 0.2.0.19-alpha; + fixes bug 1034. Reported by BarkerJr. + + Changes in version 0.2.1.18 - 2009-07-24 + Tor 0.2.1.18 lays the foundations for performance improvements, + adds status events to help users diagnose bootstrap problems, adds + optional authentication/authorization for hidden services, fixes a + variety of potential anonymity problems, and includes a huge pile of + other features and bug fixes. + o Major features (clients): - Start sending "bootstrap phase" status events to the controller, so it can keep the user informed of progress fetching directory diff --git a/acinclude.m4 b/acinclude.m4 index 47d421c260..766ca1e62e 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -1,4 +1,3 @@ -dnl $Id$ dnl Helper macros for Tor configure.in dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson diff --git a/configure.in b/configure.in index 5b30198f22..41d8b2d0e4 100644 --- a/configure.in +++ b/configure.in @@ -1,11 +1,10 @@ -dnl $Id$ dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson dnl Copyright (c) 2007-2008, The Tor Project, Inc. dnl See LICENSE for licensing information AC_INIT -AM_INIT_AUTOMAKE(tor, 0.2.1.19) +AM_INIT_AUTOMAKE(tor, 0.2.2.1-alpha) AM_CONFIG_HEADER(orconfig.h) AC_CANONICAL_HOST @@ -86,13 +85,6 @@ case $host in ;; esac -AC_ARG_ENABLE(geoip-stats, - AS_HELP_STRING(--enable-geoip-stats, enable code for directories to collect per-country statistics)) - -if test "$enable_geoip_stats" = "yes"; then - AC_DEFINE(ENABLE_GEOIP_STATS, 1, [Defined if we try to collect per-country statistics]) -fi - AC_ARG_ENABLE(gcc-warnings, AS_HELP_STRING(--enable-gcc-warnings, enable verbose warnings)) @@ -108,6 +100,12 @@ AC_PROG_CPP AC_PROG_MAKE_SET AC_PROG_RANLIB +dnl autoconf 2.59 appears not to support AC_PROG_SED +AC_CHECK_PROG([SED],[sed],[sed],[/bin/false]) + +AC_PATH_PROG([SHA1SUM], [sha1sum], none) +AC_PATH_PROG([OPENSSL], [openssl], none) + TORUSER=_tor AC_ARG_WITH(tor-user, [ --with-tor-user=NAME Specify username for tor daemon ], @@ -270,15 +268,19 @@ save_CPPFLAGS="$CPPFLAGS" LIBS="-levent $TOR_LIB_WS32 $LIBS" LDFLAGS="$TOR_LDFLAGS_libevent $LDFLAGS" CPPFLAGS="$TOR_CPPFLAGS_libevent $CPPFLAGS" -AC_CHECK_FUNCS(event_get_version event_get_method event_set_log_callback) +AC_CHECK_FUNCS(event_get_version event_get_version_number event_get_method event_set_log_callback evdns_set_outgoing_bind_address event_base_loopexit) AC_CHECK_MEMBERS([struct event.min_heap_idx], , , [#include ]) +AC_CHECK_HEADERS(event2/event.h event2/dns.h) + LIBS="$save_LIBS" LDFLAGS="$save_LDFLAGS" CPPFLAGS="$save_CPPFLAGS" +AM_CONDITIONAL(USE_EXTERNAL_EVDNS, test x$ac_cv_header_event2_dns_h = xyes) + dnl ------------------------------------------------------ dnl Where do you live, openssl? And how do we call you? diff --git a/contrib/checkOptionDocs.pl b/contrib/checkOptionDocs.pl index ca3fba55e3..c2e8757362 100755 --- a/contrib/checkOptionDocs.pl +++ b/contrib/checkOptionDocs.pl @@ -1,5 +1,4 @@ #!/usr/bin/perl -w -# $Id use strict; my %options = (); diff --git a/contrib/checkSpace.pl b/contrib/checkSpace.pl index 37f079c52b..db061a0828 100755 --- a/contrib/checkSpace.pl +++ b/contrib/checkSpace.pl @@ -42,9 +42,8 @@ for $fn (@ARGV) { $lastnil = 0; } ## Terminals are still 80 columns wide in my world. I refuse to - ## accept double-line lines. Except, of course, svn Id tags - ## can make us go long. - if (/^.{80}/ && !/\$Id: /) { + ## accept double-line lines. + if (/^.{80}/) { print " Wide:$fn:$.\n"; } ### Juju to skip over comments and strings, since the tests diff --git a/contrib/cross.sh b/contrib/cross.sh index e660be780d..af68755dbf 100755 --- a/contrib/cross.sh +++ b/contrib/cross.sh @@ -1,5 +1,4 @@ #!/bin/bash -# $Id$ # Copyright 2006 Michael Mohr with modifications by Roger Dingledine # See LICENSE for licensing information. diff --git a/contrib/directory-archive/fetch-all b/contrib/directory-archive/fetch-all index fabcbb8bc4..dfa5a1b3e5 100755 --- a/contrib/directory-archive/fetch-all +++ b/contrib/directory-archive/fetch-all @@ -30,8 +30,8 @@ DIRSERVERS="" DIRSERVERS="$DIRSERVERS 86.59.21.38:80" # tor26 DIRSERVERS="$DIRSERVERS 128.31.0.34:9031" # moria1 DIRSERVERS="$DIRSERVERS 128.31.0.34:9032" # moria2 -#DIRSERVERS="$DIRSERVERS 140.247.60.64:80" # lefkada DIRSERVERS="$DIRSERVERS 194.109.206.212:80" # dizum + DATEDIR=$(date "+%Y/%m/%d") TIME=$(date "+%Y%m%d-%H%M%S") diff --git a/contrib/directory-archive/fetch-all-v3 b/contrib/directory-archive/fetch-all-v3 index 02b3d5d9e8..a4746e02cf 100755 --- a/contrib/directory-archive/fetch-all-v3 +++ b/contrib/directory-archive/fetch-all-v3 @@ -35,6 +35,7 @@ DIRSERVERS="$DIRSERVERS 80.190.246.100:80" # gabelmoo DIRSERVERS="$DIRSERVERS 194.109.206.212:80" # dizum #DIRSERVERS="$DIRSERVERS 128.31.0.34:9032" # moria2 DIRSERVERS="$DIRSERVERS 213.73.91.31:80" # dannenberg +DIRSERVERS="$DIRSERVERS 208.83.223.34:443" # urras TIME=$(date "+%Y%m%d-%H%M%S") . fetch-all-functions diff --git a/contrib/id_to_fp.c b/contrib/id_to_fp.c index 73395e16c1..55b025dfaf 100644 --- a/contrib/id_to_fp.c +++ b/contrib/id_to_fp.c @@ -1,5 +1,4 @@ /* Copyright 2006 Nick Mathewson; see LICENSE for licensing information */ -/* $Id$ */ /* id_to_fp.c : Helper for directory authority ops. When somebody sends us * a private key, this utility converts the private key into a fingerprint diff --git a/contrib/nagios-check-tor-authority-cert b/contrib/nagios-check-tor-authority-cert index 0e2c1d06c4..46dc7284b7 100755 --- a/contrib/nagios-check-tor-authority-cert +++ b/contrib/nagios-check-tor-authority-cert @@ -8,8 +8,6 @@ # Usage: nagios-check-tor-authority-cert # e.g.: nagios-check-tor-authority-cert A9AC67E64B200BBF2FA26DF194AC0469E2A948C6 -# $Id$ - # Copyright (c) 2008 Peter Palfrader # # Permission is hereby granted, free of charge, to any person obtaining diff --git a/contrib/osx/package.sh b/contrib/osx/package.sh index 040c7cd4c9..eeb0f95078 100644 --- a/contrib/osx/package.sh +++ b/contrib/osx/package.sh @@ -1,5 +1,4 @@ #!/bin/sh -# $Id$ # Copyright 2004-2005 Nick Mathewson. # Copyright 2005-2007 Andrew Lewman # Copyright 2008 The Tor Project, Inc. diff --git a/contrib/osx/uninstall_tor_bundle.sh b/contrib/osx/uninstall_tor_bundle.sh index 9089407edb..b7c439429e 100755 --- a/contrib/osx/uninstall_tor_bundle.sh +++ b/contrib/osx/uninstall_tor_bundle.sh @@ -134,7 +134,7 @@ fi ## clean up echo ". Cleaning up" rm -rf $TEMP_BOM_CONTENTS -rm -rf /Library/Privoxy/ /Library/StartupItems/Privoxy/ /Library/Tor/ /Library/StartupItems/Tor/ /Library/Torbutton/ +rm -rf /Library/Privoxy/ /Library/StartupItems/Privoxy/ /Library/Tor/ /Library/StartupItems/Tor/ /Library/Torbutton/ /Library/Receipts/Privoxy.pkg /Library/Receipts/torbutton.pkg /Library/Receipts/Tor.pkg /Library/Receipts/Vidalia.pkg /Library/Receipts/TorStartup.pkg echo ". Finished" diff --git a/contrib/polipo/package.sh b/contrib/polipo/package.sh index 83f74212be..4ec72c81d8 100644 --- a/contrib/polipo/package.sh +++ b/contrib/polipo/package.sh @@ -1,5 +1,4 @@ #!/bin/sh -# $Id: package.sh 8992 2006-12-23 03:12:09Z phobos $ # Copyright 2004-2005 Nick Mathewson & Andrew Lewman. # Copyright 2005-2008 Andrew Lewman # This is licensed under the Modified BSD License. diff --git a/contrib/rc.subr b/contrib/rc.subr index 117ae71d47..d757e89528 100644 --- a/contrib/rc.subr +++ b/contrib/rc.subr @@ -1,5 +1,4 @@ #!/bin/sh -# $Id$ # $FreeBSD: ports/security/tor-devel/files/tor.in,v 1.1 2006/02/17 22:21:25 mnag Exp $ # # (rc.subr written by Peter Thoenen for Net/FreeBSD) diff --git a/contrib/tor-mingw.nsi.in b/contrib/tor-mingw.nsi.in index 053c142159..9dcbc6bac5 100644 --- a/contrib/tor-mingw.nsi.in +++ b/contrib/tor-mingw.nsi.in @@ -9,7 +9,7 @@ !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.2.1.19" +!define VERSION "0.2.2.1-alpha" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" @@ -216,6 +216,7 @@ Function un.InstallFiles Delete "$INSTDIR\tor.ico" Delete "$SMSTARTUP\Tor.lnk" Delete "$INSTDIR\Uninstall.exe" + Delete "$INSTDIR\geoip" FunctionEnd Function un.InstallDirectories diff --git a/contrib/tor-resolve.py b/contrib/tor-resolve.py index 919bc876cc..47ae1a0c38 100755 --- a/contrib/tor-resolve.py +++ b/contrib/tor-resolve.py @@ -1,5 +1,4 @@ #!/usr/bin/python -#$Id$ import socket import struct diff --git a/contrib/torify.1 b/contrib/torify.1 index b08d468451..9ae4e40d9d 100644 --- a/contrib/torify.1 +++ b/contrib/torify.1 @@ -1,22 +1,27 @@ .TH torify 1 "" Jan-2009 "" -.\" manual page by Peter Palfrader +.\" manual page by Peter Palfrader and Jacob Appelbaum .SH NAME .LP -torify \- wrapper for tsocks and tor +torify \- wrapper for torsocks or tsocks and tor .SH SYNOPSIS \fBtorify\fP\ \fIapplication\fP\ [\fIapplication's\ arguments\fP] .SH DESCRIPTION -\fBtorify\fR is a simple wrapper that calls tsocks with a tor specific +\fBtorify\fR is a simple wrapper that attempts to find the best underlying Tor +wrapper available on a system. It calls torsocks or tsocks with a tor specific configuration file. +torsocks is an improved wrapper that explictly rejects UDP, safely resolves DNS +lookups and properly socksifies your TCP connections. + tsocks itself is a wrapper between the tsocks library and the application that you would like to run socksified. -Please note that since tsocks uses LD_PRELOAD, torify cannot be applied +Please note that since both method use LD_PRELOAD, torify cannot be applied to suid binaries. +.SH WARNING You should also be aware that the way tsocks currently works only TCP connections are socksified. Be aware that this will in most circumstances not include hostname lookups which would still be routed through your @@ -25,8 +30,13 @@ normal system resolver to your usual resolving nameservers. The The Tor FAQ at https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ might have further information on this subject. +When used with torsocks, torify should not leak DNS requests or UDP data. + +Both will leak ICMP data. + .SH SEE ALSO .BR tor (1), .BR tor-resolve (1), +.BR torsocks (1), .BR tsocks (1), .BR tsocks.conf (5). diff --git a/contrib/torify.in b/contrib/torify.in index 05645fd07c..5bf7d4dbcf 100755 --- a/contrib/torify.in +++ b/contrib/torify.in @@ -10,36 +10,67 @@ # Define and ensure we have tsocks # XXX: what if we don't have which? +TORSOCKS="`which torsocks`" TSOCKS="`which tsocks`" +PROG="" if [ ! -x "$TSOCKS" ] then - echo "$0: Can't find tsocks in PATH. Perhaps you haven't installed it?" >&2 - exit 1 + echo "$0: Can't find tsocks in PATH. Perhaps you haven't installed it?" >&2 +else + PROG=$TSOCKS +fi +if [ ! -x "$TORSOCKS" ] +then + echo "$0: Can't find torsocks in PATH. Perhaps you haven't installed it?" >&2 +else + PROG=$TORSOCKS +fi + +if [ ! -x "$PROG" ] +then + echo "$0: Can't find the required tor helpers in our PATH. Perhaps you haven't installed them?" >&2 + exit 1; fi # Check for any argument list if [ "$#" = 0 ] then - echo "Usage: $0 [...]" >&2 - exit 1 + echo "Usage: $0 [-hv] [...]" >&2 + exit 1 fi if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] ) then - echo "Usage: $0 [...]" - exit 0 + echo "Usage: $0 [-hv] [...]" + exit 0 fi -# Define our tsocks config file -TSOCKS_CONF_FILE="@CONFDIR@/tor-tsocks.conf" -export TSOCKS_CONF_FILE - -# Check that we've got a tsocks config file -if [ -r "$TSOCKS_CONF_FILE" ] +if [ "$1" = "-v" ] || [ "$1" = "--verbose" ] then - exec tsocks "$@" - echo "$0: Failed to exec tsocks $@" >&2 - exit 1 -else - echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2 - exit 1 + echo "We're armed with the following tsocks: $TSOCKS" + echo "We're armed with the following torsocks: $TORSOCKS" + echo "We're attempting to use $PROG for all tor action." + shift 1 +fi + +if [ "$PROG" = "$TSOCKS" ] +then + # Define our tsocks config file + TSOCKS_CONF_FILE="/etc/tor/tor-tsocks.conf" + export TSOCKS_CONF_FILE + + # Check that we've got a tsocks config file + if [ -r "$TSOCKS_CONF_FILE" ] + then + echo "WARNING: tsocks is known to leak DNS and UDP data." >&2 + exec tsocks "$@" + echo "$0: Failed to exec tsocks $@" >&2 + exit 1 + else + echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2 + exit 1 + fi +fi +if [ "$PROG" = "$TORSOCKS" ] +then + exec torsocks "$@" fi diff --git a/debian/changelog b/debian/changelog index 00eae0f0ac..6673fe01b0 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,11 @@ +tor (0.2.2.1-alpha-1) experimental; urgency=low + + * New upstream version. + * Forward port patches/03_tor_manpage_in_section_8.dpatch. + * Forward port patches/06_add_compile_time_defaults.dpatch. + + -- Peter Palfrader Thu, 03 Sep 2009 15:10:26 +0200 + tor (0.2.1.19-1) unstable; urgency=low * New upstream version. diff --git a/debian/patches/03_tor_manpage_in_section_8.dpatch b/debian/patches/03_tor_manpage_in_section_8.dpatch index 28bbf957c0..ece1fe67b2 100755 --- a/debian/patches/03_tor_manpage_in_section_8.dpatch +++ b/debian/patches/03_tor_manpage_in_section_8.dpatch @@ -26,20 +26,21 @@ exit 0 diff -urNad tor-0.1.1.5/contrib/torify.1 /tmp/dpep.fOA3Mm/tor-0.1.1.5/contrib/torify.1 --- tor-0.1.1.5/contrib/torify.1 +++ /tmp/dpep.fOA3Mm/tor-0.1.1.5/contrib/torify.1 -@@ -18,6 +18,6 @@ +@@ -35,7 +35,7 @@ to suid binaries. .SH SEE ALSO -.BR tor (1), +.BR tor (8), .BR tor-resolve (1), + .BR torsocks (1), .BR tsocks (1), diff -urNad tor-0.1.1.5/doc/tor.1.in /tmp/dpep.fOA3Mm/tor-0.1.1.5/doc/tor.1.in --- tor-0.1.1.5/doc/tor.1.in +++ /tmp/dpep.fOA3Mm/tor-0.1.1.5/doc/tor.1.in @@ -1,4 +1,4 @@ --.TH TOR 1 "January 2009" "TOR" -+.TH TOR 8 "January 2009" "TOR" +-.TH TOR 1 "August 2009" "TOR" ++.TH TOR 8 "August 2009" "TOR" .SH NAME tor \- The second-generation onion router .SH SYNOPSIS diff --git a/debian/patches/06_add_compile_time_defaults.dpatch b/debian/patches/06_add_compile_time_defaults.dpatch index a2472d1db8..e64d4618af 100755 --- a/debian/patches/06_add_compile_time_defaults.dpatch +++ b/debian/patches/06_add_compile_time_defaults.dpatch @@ -23,9 +23,9 @@ esac exit 0 @DPATCH@ -diff -urNad tor-trunk~/src/or/config.c tor-trunk/src/or/config.c ---- tor-trunk~/src/or/config.c 2009-01-18 01:47:33.000000000 +0100 -+++ tor-trunk/src/or/config.c 2009-02-05 00:25:17.614844812 +0100 +diff -urNad tor~/src/or/config.c tor/src/or/config.c +--- tor~/src/or/config.c 2009-09-03 15:05:41.000000000 +0200 ++++ tor/src/or/config.c 2009-09-03 15:09:37.662104166 +0200 @@ -12,6 +12,7 @@ #define CONFIG_PRIVATE @@ -34,16 +34,17 @@ diff -urNad tor-trunk~/src/or/config.c tor-trunk/src/or/config.c #ifdef MS_WINDOWS #include #endif -@@ -711,6 +712,8 @@ - #if defined(HAVE_EVENT_GET_VERSION) && defined(HAVE_EVENT_GET_METHOD) - static void check_libevent_version(const char *m, int server); - #endif +@@ -717,6 +718,9 @@ + static void init_libevent(void); + static int opt_streq(const char *s1, const char *s2); + +static int debian_running_as_debiantor(); +static int debian_config_fix_defaults(); - ++ /** Magic value for or_options_t. */ #define OR_OPTIONS_MAGIC 9090909 -@@ -3917,6 +3920,9 @@ + +@@ -4086,6 +4090,9 @@ char *command_arg = NULL; char *errmsg=NULL; @@ -53,7 +54,7 @@ diff -urNad tor-trunk~/src/or/config.c tor-trunk/src/or/config.c if (argv) { /* first time we're called. save command line args */ backup_argv = argv; backup_argc = argc; -@@ -5307,3 +5313,62 @@ +@@ -5304,3 +5311,62 @@ return 0; } diff --git a/doc/HACKING b/doc/HACKING index 50b5d80d18..3d3f2c1dfc 100644 --- a/doc/HACKING +++ b/doc/HACKING @@ -11,12 +11,20 @@ 0.1. Useful command-lines that are non-trivial to reproduce but can help with tracking bugs or leaks. +0.1.1. Dmalloc + dmalloc -l ~/dmalloc.log (run the commands it tells you) ./configure --with-dmalloc +0.2.2. Valgrind + valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor +(Note that if you get a zillion openssl warnings, you will also need to + pass --undef-value-errors=no to valgrind, or rebuild your openssl + with -DPURIFY.) + 0.2. Running gcov for unit test coverage make clean diff --git a/doc/TODO b/doc/TODO index fd023c8bf5..194d6507bc 100644 --- a/doc/TODO +++ b/doc/TODO @@ -4,8 +4,8 @@ We've split out our TODO into three files: TODO.02x is the list of items we're planning to get done in the next stable release. -TODO.external is the list of external constraints and deliverables that -we all need to keep in mind. +TODO.external lives in svn under /projects/todo/. It's the list of +external constraints and deliverables that we all need to keep in mind. TODO.future is the list of other items we plan to get to in later releases. diff --git a/doc/TODO.021 b/doc/TODO.021 index 881ba5ee4b..37c5b9845b 100644 --- a/doc/TODO.021 +++ b/doc/TODO.021 @@ -1,4 +1,3 @@ -$Id$ Legend: SPEC!! - Not specified SPEC - Spec not finalized diff --git a/doc/TODO.022 b/doc/TODO.022 index 3eeae006cb..f4fe2ebb2a 100644 --- a/doc/TODO.022 +++ b/doc/TODO.022 @@ -8,14 +8,17 @@ NOTE 2: It's easy to list stuff like this with no time estimates and 0.2.2, figure out how long the stuff we want will take, and triage accordingly, or vice versa. -- Design +- Design only - Begin design work for UDP transition; identify areas where we need to make changes or instrument stuff early. + [multiple weeks, ongoing. Need to do a draft early.] - Performance, mostly protocol-neutral. - Work with Libevent 2.0's bufferevent interface - Identify any performance stuff we need to push back into libevent to make it as fast as we want. + - Get a decent rate-limiting feature into Libevent + - Get openssl support into Libevent. - Revise how we do bandwidth limiting and round-robining between circuits on a connection. @@ -30,21 +33,76 @@ NOTE 2: It's easy to list stuff like this with no time estimates and - Figure out good ways to instrument Tor internals so we can tell how well our bandwidth and flow-control stuff is actually working. + - What ports eat the bandwidth? + - How full do queues get? + - How much latency do queues get? -- Features + - Rate limit at clients: + - Give clients an upper bound on how much they're willing to use + the network if they're not relaying? + - ... or group client circuits by IP at the server and rate-limit + like that. + + - Use if-modified-since to download consensuses + + +- Other features - Proposals to implement: - - 146: reflect long-term stability + - 146: reflect long-term stability in consensuses - 147: Stop using v2 directories to generate v3 votes. + - Start pinging as soon as we learn about a relay, not on a + 22-minute cycle. Prioritize new and volatile relays for + testing. - Proposals to improve and implement - 158: microdescriptors + o Revise proposal + - Implement + o 160: list bandwidth in consensus + o Finish proposal + o and actually set it reasonably + o and actually use it. - Proposals to improve and implement if not broken - - IPv6 support. (Parts of 117, but figure out how to handle DNS + D IPv6 support. (Parts of 117, but figure out how to handle DNS requests.) - 140: Directory diffs + - Need a decent simple C diff implementation. + - Need a decent simple C ed patch implementation. - 149: learn info from netinfo cells. - - 134: handle authority fragmentation (Needs more analysis) + o Start discussion + - Revise proposal based on discussion. + X 134: handle authority fragmentation (Needs more analysis) + - 165: Easy migration for voting authority sets + - 163: Detect client-status better + o Write proposal + - Possibly implement, depending on discussion. + - 164: Have authorities report relay and voting status better: make it + easy to answer, "Why is my server not listed/not Guard/not + Running/etc" + o Write proposal + - Possibly implement, depending on discussion + - 162: Have consensuses come in multiple "flavours". + o Write proposal + - Possibly implement, depending on discussion. + + - Needs a proposal, or at least some design + - Weaken the requirements for being a Guard, based on K's + measurements. +K - Finish measurements +K? - Write proposal + - Adaptive timeouts for giving up on circuits and streams. +M - Revise proposal 151 + - Downweight guards more sensibly: be more forgiving about using + Guard nodes as non-first-hop. + - Write proposal. + - Lagged weight updates in consensuses: don't just move abruptly. +M? - Write proposal + d Don't kill a circuit on the first failed extend. + +- Installers + - Switch to MSI on win32 + - Use Thandy, perhaps? - Deprecations - Make .exit safe, or make it off-by-default. diff --git a/doc/TODO.external b/doc/TODO.external index c02d6aca54..2e7e536efc 100644 --- a/doc/TODO.external +++ b/doc/TODO.external @@ -1,196 +1,4 @@ -$Id$ -Legend: -SPEC!! - Not specified -SPEC - Spec not finalized -N - nick claims -R - arma claims -P - phobos claims -S - Steven claims -E - Matt claims -M - Mike claims -J - Jeff claims -I - ioerror claims -W - weasel claims -K - Karsten claims -C - coderman claims - - Not done - * Top priority - . Partially done - o Done - d Deferrable - D Deferred - X Abandoned -======================================================================= - -External constraints: - -For June/July: -NR - Work more on Paul's NRL research problem. - -For March 22: -I * Email auto-responder - * teach gettor how to ask for (and attach) split files. - -K . Metrics. - . With Mike's help, use Torflow to start doing monthly rudimentary - performance evaluations: - . Circuit throughput and latency - - Measure via Broadband and dialup - . Publish a report addressing key long-term metrics questions: - . What metrics should we present? - . What data are available for these metrics? - . What data are missing, and can collect them safely? Can we - publish them safely? - . What systems are available to present this data? - -E . Vidalia improvements - o Vidalia displays by-country user summary for bridge operators -? - write a help page for vidalia, "what is this" - -For mid August: - -Section 0, items that didn't make it into the original roadmap: - -0.1, installers and packaging -C . i18n for the msi bundle files -P . more consistent TBB builds -IC- get a buildbot up again. Have Linux and BSD build machines. - (Windows would be nice but realistically will come later.) -E - Get Tor to work properly on the iPhone. - -3.1, performance work. [Section numbers in here are from performance.pdf] - - High-priority items from performance.pdf -RS - 1.2, new circuit window sizes. make the default package window lower. -R+ - 2.1, squeeze loud circuits - - Evaluate the code to see what stats we can keep about circuit use. - - Write proposals for various meddling. Look at the research papers - that Juliusz pointed us to. Ask our systems friends. Plan to put - a lot of the parameters in the consensus, so we can tune it with - short turnaround times. -E+ - 2.5, Change Vidalia's default exit policy to not click "other - protocols". Or choose not to. Think this through first. -R+ - 2.6, Tell users not to file-share. - - Put statement on the Tor front page - - Put statement on the download pages too - - And the FAQ - - 3.1.2, Tor weather -I - Implement time-to-notification (immediate, a day, a week) -I - Get a relay operator mailing list going, with a plan and supporting - scripts and so on. -R - Link to them from the Tor relay page -R - and the torrc.sample? -SM - 4.1, balance traffic better - - Steven and Mike should decide if we should do Steven's plan - (rejigger the bandwidth numbers at the authorities based on - Steven's algorithm), or Mike's plan (relay scanning to identify - the unbalanced relays and fix them on the fly), or both. - - Figure out how to actually modify bandwidths in the consensus. We - may need to change the consensus voting algorithm to decide what - bandwidth to advertise based on something other than median: - if 7 authorities provide bandwidths, and 2 are doing scanning, - then the 5 that aren't scanning will outvote any changes. Should - all 7 scan? Should only some vote? Extra points if it doesn't - change all the numbers every new consensus, so consensus diffing - is still practical. -? - 4.5, Older entry guards are overloaded - - Pick a conservative timeout like a month, and implement. -M - 5.2, better timeouts for giving up on circuits/streams - - clients gather data about circuit timeouts, and then abandon - circuits that take more than a std dev above that. - -4.1, IOCP / libevent / windows / tor -N - get it working for nick -N - put out a release so other people can start testing it. -N - both the libevent buffer abstraction, and the - tor-uses-libevent-buffer-abstraction. Unless we think that's - unreachable for this milestone? - -4.2.1, risks from becoming a relay -S - Have a clear plan for how users who become relays will be safe, - and be confident that we can build this plan. - - evaluate all the various attacks that are made possible by relaying. - specifically, see "relaying-traffic attacks" in 6.6. - - identify and evaluate ways to make them not a big deal - - setting a low RelayBandwidth - - Nick Hopper's FC08 paper suggesting that we should do a modified - round-robin so we leak less about other circuits - - instructing clients to disable pings in their firewall, etc - - pick the promising ones, improve them so they're even better, and - spec them out so we know how to build them and how much effort is - involved in building them. - -4.5, clients download less directory info -N * deploy proposal 158. -N - decide whether to do proposal 140. if so, construct an implementation - plan for how we'll do it. if not, explain why not. - -5.1, Normalize TLS fingerprint -N o write a draft list of possible attacks for this section, with - estimates about difficulty of attack, difficulty of solution, etc -N - revisit the list and revise our plans as needed -NR- put up a blog post about the two contradictory conclusions: we can - discuss the theory of arms races, and our quandry, without revealing - any specific vulnerabilities. (or decide not to put up a blog post, - and explain why not.) - -5.5, email autoresponder -I . maintenance and keeping it running - -5.7.2, metrics - -XXX. - -6.2, Vidalia work -E - add breakpad support or similar for windows debugging -E o let vidalia change languages without needing a restart -E - Implement the status warning event interface started for the - phase one deliverables. -E - Work with Steve Tyree on building a Vidalia plugin API to enable - building Herdict and TBB plugins. - -6.3, Node scanning -M - Steps toward automation - - Set up email list for results - - Map failure types to potential BadExit lines -M - Improve the ability of SoaT to mimic various real web browsers - - randomizing user agents and locale strings - - caching, XMLHTTPRequest, form posting, content sniffing - - Investigate ideas like running Chrome/xulrunner in parallel -M - Other protocols - - SSH, IMAPS, POPS, SMTPS -M - Add ability to geolocalize exit selection based on scanner location - - Use this to rescan dynamic urls filtered by the URL filter - -6.4, Torbutton development -M - Resolve extension conflicts and other high priority bugs -M - Fix or hack around ugly firefox bugs, especially Timezone issue. - Definitely leaning towards "hack around" unless we see some - level of love from Mozilla. -M - Vidalia New Nym Integration - - Implement for Torbutton to pick up on Vidalia's NEWNYM and clear - cookies based on FoeBud's source - - Do this in such a way that we could adapt polipo to purge cache - if we were so inclined -M - Write up a summary of our options for dealing with the google - you-must-solve-a-captcha-to-search problem, and pick one as our - favorite option. - -6.6, Evaluate new anonymity attacks -S - relaying-traffic attacks - - original murdoch-danezis attack - - nick hopper's latency measurement attack - - columbia bandwidth measurement attack - - christian grothoff's long-circuit attack -S - client attacks - - website fingerprinting - -7.1, Tor VM Research, analysis, and prototyping -C . Get a working package out, meaning other people are testing it. - -7.2, Tor Browser Bundle -I - Port to one of OS X or Linux, and start the port to the other. -I . Make it the recommended Tor download on Windows -I - Make sure it's easy to un-brand TBB in case Firefox asks us to -I - Evaluate CCC's Freedom Stick +[This file moved to svn in /projects/todo/. More people can edit +it more easily there. -RD] diff --git a/doc/TODO.future b/doc/TODO.future index 64169ecfec..a6cc95150e 100644 --- a/doc/TODO.future +++ b/doc/TODO.future @@ -1,4 +1,3 @@ -$Id$ Legend: SPEC!! - Not specified SPEC - Spec not finalized diff --git a/doc/design-paper/latex8.bst b/doc/design-paper/latex8.bst index 2dd3249633..bae8e209ee 100644 --- a/doc/design-paper/latex8.bst +++ b/doc/design-paper/latex8.bst @@ -1,8 +1,6 @@ % --------------------------------------------------------------- % -% $Id$ -% % by Paolo.Ienne@di.epfl.ch % diff --git a/doc/design-paper/usenix.sty b/doc/design-paper/usenix.sty index 4442f11574..575c854e77 100644 --- a/doc/design-paper/usenix.sty +++ b/doc/design-paper/usenix.sty @@ -5,8 +5,6 @@ % \usepackage{usenix-2e} % and put {\rm ....} around the author names. % -% $Id$ -% % The following definitions are modifications of standard article.sty % definitions, arranged to do a better job of matching the USENIX % guidelines. diff --git a/doc/spec/address-spec.txt b/doc/spec/address-spec.txt index 2a84d857e6..2e1aff2b8a 100644 --- a/doc/spec/address-spec.txt +++ b/doc/spec/address-spec.txt @@ -1,4 +1,3 @@ -$Id$ Special Hostnames in Tor Nick Mathewson @@ -34,10 +33,13 @@ $Id$ "www.google.com.foo.exit=64.233.161.99.foo.exit" to speed subsequent lookups. + The .exit notation is disabled by default as of Tor 0.2.2.1-alpha, due + to potential application-level attacks. + EXAMPLES: www.example.com.exampletornode.exit - Connect to www.example.com from the node called "exampletornode." + Connect to www.example.com from the node called "exampletornode". exampletornode.exit @@ -54,15 +56,3 @@ $Id$ When Tor sees an address in this format, it tries to look up and connect to the specified hidden service. See rend-spec.txt for full details. -4. .noconnect - - SYNTAX: [string].noconnect - - When Tor sees an address in this format, it immediately closes the - connection without attaching it to any circuit. This is useful for - controllers that want to test whether a given application is indeed using - the same instance of Tor that they're controlling. - -5. [XXX Is there a ".virtual" address that we expose too, or is that -just intended to be internal? -RD] - diff --git a/doc/spec/bridges-spec.txt b/doc/spec/bridges-spec.txt index 4a9b373c8e..647118815c 100644 --- a/doc/spec/bridges-spec.txt +++ b/doc/spec/bridges-spec.txt @@ -1,4 +1,3 @@ -$Id$ Tor bridges specification diff --git a/doc/spec/control-spec-v0.txt b/doc/spec/control-spec-v0.txt index faf75a64a4..3515d395a6 100644 --- a/doc/spec/control-spec-v0.txt +++ b/doc/spec/control-spec-v0.txt @@ -1,4 +1,3 @@ -$Id$ TC: A Tor control protocol (Version 0) diff --git a/doc/spec/control-spec.txt b/doc/spec/control-spec.txt index 576c5dcd53..fc4242ea16 100644 --- a/doc/spec/control-spec.txt +++ b/doc/spec/control-spec.txt @@ -1,4 +1,3 @@ -$Id$ TC: A Tor control protocol (Version 1) @@ -220,7 +219,7 @@ $Id$ "INFO" / "NOTICE" / "WARN" / "ERR" / "NEWDESC" / "ADDRMAP" / "AUTHDIR_NEWDESCS" / "DESCCHANGED" / "STATUS_GENERAL" / "STATUS_CLIENT" / "STATUS_SERVER" / "GUARD" / "NS" / "STREAM_BW" / - "CLIENTS_SEEN" + "CLIENTS_SEEN" / "NEWCONSENSUS" Any events *not* listed in the SETEVENTS line are turned off; thus, sending SETEVENTS with an empty body turns off all event reporting. @@ -503,7 +502,7 @@ $Id$ start and the rest of the interval respectively. The 'interval-start' and 'interval-end' fields are the borders of the current interval; the 'interval-wake' field is the time within the current interval (if any) - where we plan[ned] to start being active. + where we plan[ned] to start being active. The times are GMT. "config/names" A series of lines listing the available configuration options. Each is @@ -563,14 +562,14 @@ $Id$ states. See Section 4.1.10 for explanations. (Only a few of the status events are available as getinfo's currently. Let us know if you want more exposed.) - "status/reachability/or" + "status/reachability-succeeded/or" 0 or 1, depending on whether we've found our ORPort reachable. - "status/reachability/dir" + "status/reachability-succeeded/dir" 0 or 1, depending on whether we've found our DirPort reachable. - "status/reachability" + "status/reachability-succeeded" "OR=" ("0"/"1") SP "DIR=" ("0"/"1") - Combines status/reachability/*; controllers MUST ignore unrecognized - elements in this entry. + Combines status/reachability-succeeded/*; controllers MUST ignore + unrecognized elements in this entry. "status/bootstrap-phase" Returns the most recent bootstrap phase status event sent. Specifically, it returns a string starting with either @@ -774,9 +773,8 @@ $Id$ Same as passing 'EXTENDED' to SETEVENTS; this is the preferred way to request the extended event syntax. - This will not be always-enabled until at least two stable releases - after 0.1.2.3-alpha, the release where it was first used for - anything. + This feature was first used in 0.1.2.3-alpha. It is always-on in + Tor 0.2.2.1-alpha and later. VERBOSE_NAMES @@ -787,8 +785,9 @@ $Id$ LongName format includes a Fingerprint, an indication of Named status, and a Nickname (if one is known). - This will not be always-enabled until at least two stable releases - after 0.1.2.2-alpha, the release where it was first available. + This will not be always-enabled until at least two stable + releases after 0.1.2.2-alpha, the release where it was first + available. It is always-on in Tor 0.2.2.1-alpha and later. 3.20. RESOLVE diff --git a/doc/spec/dir-spec-v1.txt b/doc/spec/dir-spec-v1.txt index 286df664e2..a92fc7999a 100644 --- a/doc/spec/dir-spec-v1.txt +++ b/doc/spec/dir-spec-v1.txt @@ -1,4 +1,3 @@ -$Id$ Tor Protocol Specification diff --git a/doc/spec/dir-spec-v2.txt b/doc/spec/dir-spec-v2.txt index 4873c4a728..d1be27f3db 100644 --- a/doc/spec/dir-spec-v2.txt +++ b/doc/spec/dir-spec-v2.txt @@ -1,4 +1,3 @@ -$Id$ Tor directory protocol, version 2 diff --git a/doc/spec/dir-spec.txt b/doc/spec/dir-spec.txt index 9a2a62bc46..16f121a19a 100644 --- a/doc/spec/dir-spec.txt +++ b/doc/spec/dir-spec.txt @@ -1,4 +1,3 @@ -$Id$ Tor directory protocol, version 3 @@ -594,7 +593,7 @@ $Id$ "allow-single-hop-exits" - [At most one.] + [At most once.] Present only if the router allows single-hop circuits to make exit connections. Most Tor servers do not support this: this is @@ -642,6 +641,200 @@ $Id$ "geoip-start" is the time at which we began collecting geoip statistics. + "dirreq-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + A "dirreq-stats-end" line, as well as any other "dirreq-*" line, + is only added when the relay has opened its Dir port and after 24 + hours of measuring directory requests. + + "dirreq-v2-ips" CC=N,CC=N,... NL + [At most once.] + "dirreq-v3-ips" CC=N,CC=N,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + unique IP addresses that have connected from that country to + request a v2/v3 network status, rounded up to the nearest multiple + of 8. Only those IP addresses are counted that the directory can + answer with a 200 OK status code. + + "dirreq-v2-reqs" CC=N,CC=N,... NL + [At most once.] + "dirreq-v3-reqs" CC=N,CC=N,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + requests for v2/v3 network statuses from that country, rounded up + to the nearest multiple of 8. Only those requests are counted that + the directory can answer with a 200 OK status code. + + "dirreq-v2-share" num% NL + [At most once.] + "dirreq-v3-share" num% NL + [At most once.] + + The share of v2/v3 network status requests that the directory + expects to receive from clients based on its advertised bandwidth + compared to the overall network bandwidth capacity. Shares are + formatted in percent with two decimal places. Shares are + calculated as means over the whole 24-hour interval. + + "dirreq-v2-resp" status=num,... NL + [At most once.] + "dirreq-v3-resp" status=nul,... NL + [At most once.] + + List of mappings from response statuses to the number of requests + for v2/v3 network statuses that were answered with that response + status, rounded up to the nearest multiple of 4. Only response + statuses with at least 1 response are reported. New response + statuses can be added at any time. The current list of response + statuses is as follows: + + "ok": a network status request is answered; this number + corresponds to the sum of all requests as reported in + "dirreq-v2-reqs" or "dirreq-v3-reqs", respectively, before + rounding up. + "not-enough-sigs: a version 3 network status is not signed by a + sufficient number of requested authorities. + "unavailable": a requested network status object is unavailable. + "not-found": a requested network status is not found. + "not-modified": a network status has not been modified since the + If-Modified-Since time that is included in the request. + "busy": the directory is busy. + + "dirreq-v2-direct-dl" key=val,... NL + [At most once.] + "dirreq-v3-direct-dl" key=val,... NL + [At most once.] + "dirreq-v2-tunneled-dl" key=val,... NL + [At most once.] + "dirreq-v3-tunneled-dl" key=val,... NL + [At most once.] + + List of statistics about possible failures in the download process + of v2/v3 network statuses. Requests are either "direct" + HTTP-encoded requests over the relay's directory port, or + "tunneled" requests using a BEGIN_DIR cell over the relay's OR + port. The list of possible statistics can change, and statistics + can be left out from reporting. The current list of statistics is + as follows: + + Successful downloads and failures: + + "complete": a client has finished the download successfully. + "timeout": a download did not finish within 10 minutes after + starting to send the response. + "running": a download is still running at the end of the + measurement period for less than 10 minutes after starting to + send the response. + + Download times: + + "min", "max": smallest and largest measured bandwidth in B/s. + "d[1-4,6-9]": 1st to 4th and 6th to 9th decile of measured + bandwidth in B/s. For a given decile i, i/10 of all downloads + had a smaller bandwidth than di, and (10-i)/10 of all downloads + had a larger bandwidth than di. + "q[1,3]": 1st and 3rd quartile of measured bandwidth in B/s. One + fourth of all downloads had a smaller bandwidth than q1, one + fourth of all downloads had a larger bandwidth than q3, and the + remaining half of all downloads had a bandwidth between q1 and + q3. + "md": median of measured bandwidth in B/s. Half of the downloads + had a smaller bandwidth than md, the other half had a larger + bandwidth than md. + + "entry-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + An "entry-stats-end" line, as well as any other "entry-*" + line, is first added after the relay has been running for at least + 24 hours. + + "entry-ips" CC=N,CC=N,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + unique IP addresses that have connected from that country to the + relay and which are no known other relays, rounded up to the + nearest multiple of 8. + + "cell-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + A "cell-stats-end" line, as well as any other "cell-*" line, + is first added after the relay has been running for at least 24 + hours. + + "cell-processed-cells" num,...,num NL + [At most once.] + + Mean number of processed cells per circuit, subdivided into + deciles of circuits by the number of cells they have processed in + descending order from loudest to quietest circuits. + + "cell-queued-cells" num,...,num NL + [At most once.] + + Mean number of cells contained in queues by circuit decile. These + means are calculated by 1) determining the mean number of cells in + a single circuit between its creation and its termination and 2) + calculating the mean for all circuits in a given decile as + determined in "cell-processed-cells". Numbers have a precision of + two decimal places. + + "cell-time-in-queue" num,...,num NL + [At most once.] + + Mean time cells spend in circuit queues in milliseconds. Times are + calculated by 1) determining the mean time cells spend in the + queue of a single circuit and 2) calculating the mean for all + circuits in a given decile as determined in + "cell-processed-cells". + + "cell-circuits-per-decile" num NL + [At most once.] + + Mean number of circuits that are included in any of the deciles, + rounded up to the next integer. + + "exit-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + An "exit-stats-end" line, as well as any other "exit-*" line, is + first added after the relay has been running for at least 24 hours + and only if the relay permits exiting (where exiting to a single + port and IP address is sufficient). + + "exit-kibibytes-written" port=N,port=N,... NL + [At most once.] + "exit-kibibytes-read" port=N,port=N,... NL + [At most once.] + + List of mappings from ports to the number of kibibytes that the + relay has written to or read from exit connections to that port, + rounded up to the next full kibibyte. + + "exit-streams-opened" port=N,port=N,... NL + [At most once.] + + List of mappings from ports to the number of opened exit streams + to that port, rounded up to the nearest multiple of 4. + "router-signature" NL Signature NL [At end, exactly once.] @@ -798,7 +991,7 @@ $Id$ documents are described in section XXX below. Status documents contain a preamble, an authority section, a list of - router status entries, and one more footers signature, in that order. + router status entries, and one or more footer signature, in that order. Unlike other formats described above, a SP in these documents must be a single space character (hex 20). @@ -1030,13 +1223,20 @@ $Id$ descriptors if they would cause "v" lines to be over 128 characters long. - "w" SP "Bandwidth=" INT NL + "w" SP "Bandwidth=" INT [SP "Measured=" INT] NL [At most once.] An estimate of the bandwidth of this server, in an arbitrary unit (currently kilobytes per second). Used to weight router - selection. Other weighting keywords may be added later. + selection. + + Additionally, the Measured= keyword is present in votes by + participating bandwidth measurement authorites to indicate + a measured bandwidth currently produced by measuring stream + capacities. + + Other weighting keywords may be added later. Clients MUST ignore keywords they do not recognize. "p" SP ("accept" / "reject") SP PortList NL @@ -1179,6 +1379,13 @@ $Id$ rate limit from the router descriptor. It is given in kilobytes per second, and capped at some arbitrary value (currently 10 MB/s). + The Measured= keyword on a "w" line vote is currently computed + by multiplying the previous published consensus bandwidth by the + ratio of the measured average node stream capacity to the network + average. If 3 or more authorities provide a Measured= keyword for + a router, the authorites produce a consensus containing a "w" + Bandwidth= keyword equal to the median of the Measured= votes. + The ports listed in a "p" line should be taken as those ports for which the router's exit policy permits 'most' addresses, ignoring any accept not for all addresses, ignoring all rejects for private @@ -1261,6 +1468,11 @@ $Id$ one, breaking ties in favor of the lexicographically larger vote.) The port list is encoded as specified in 3.4.2. + * If consensus-method 6 or later is in use and if 3 or more + authorities provide a Measured= keyword in their votes for + a router, the authorities produce a consensus containing a + Bandwidth= keyword equal to the median of the Measured= votes. + The signatures at the end of a consensus document are sorted in ascending order by identity digest. @@ -1281,6 +1493,7 @@ $Id$ "3" -- Added legacy ID key support to aid in authority ID key rollovers "4" -- No longer list routers that are not running in the consensus "5" -- adds support for "w" and "p" lines. + "6" -- Prefers measured bandwidth values rather than advertised Before generating a consensus, an authority must decide which consensus method to use. To do this, it looks for the highest version number diff --git a/doc/spec/path-spec.txt b/doc/spec/path-spec.txt index dceb21dad7..78f3b63bcb 100644 --- a/doc/spec/path-spec.txt +++ b/doc/spec/path-spec.txt @@ -1,4 +1,3 @@ -$Id$ Tor Path Specification @@ -72,6 +71,24 @@ of their choices. is unknown (usually its target IP), but we believe the path probably supports the request according to the rules given below. +1.1. A server's bandwidth + + Old versions of Tor did not report bandwidths in network status + documents, so clients had to learn them from the routers' advertised + server descriptors. + + For versions of Tor prior to 0.2.1.17-rc, everywhere below where we + refer to a server's "bandwidth", we mean its clipped advertised + bandwidth, computed by taking the smaller of the 'rate' and + 'observed' arguments to the "bandwidth" element in the server's + descriptor. If a router's advertised bandwidth is greater than + MAX_BELIEVABLE_BANDWIDTH (currently 10 MB/s), we clipped to that + value. + + For more recent versions of Tor, we take the bandwidth value declared + in the consensus, and fall back to the clipped advertised bandwidth + only if the consensus does not have bandwidths listed. + 2. Building circuits 2.1. When we build @@ -179,16 +196,13 @@ of their choices. multiple candidates for a path element, we choose randomly. For "fast" circuits, we pick a given router as an exit with probability - proportional to its advertised bandwidth [the smaller of the 'rate' and - 'observed' arguments to the "bandwidth" element in its descriptor]. If a - router's advertised bandwidth is greater than MAX_BELIEVABLE_BANDWIDTH - (currently 10 MB/s), we clip to that value. + proportional to its bandwidth. For non-exit positions on "fast" circuits, we pick routers as above, but - we weight the clipped advertised bandwidth of Exit-flagged nodes depending + we weight the bandwidth of Exit-flagged nodes depending on the fraction of bandwidth available from non-Exit nodes. Call the - total clipped advertised bandwidth for Exit nodes under consideration E, - and the total clipped advertised bandwidth for all nodes under + total bandwidth for Exit nodes under consideration E, + and the total bandwidth for all nodes under consideration T. If E