Allow cached or outdated Onionoo data to be used to choose
fallback directories, as long as it's less than a day old.
Modify last modified date checks in preparation for Onionoo change
"Tor has included a feature to fetch the initial consensus from nodes
other than the authorities for a while now. We just haven't shipped a
list of alternate locations for clients to go to yet.
Reasons why we might want to ship tor with a list of additional places
where clients can find the consensus is that it makes authority
reachability and BW less important.
We want them to have been around and using their current key, address,
and port for a while now (120 days), and have been running, a guard,
and a v2 directory mirror for most of that time."
Features:
* whitelist and blacklist for an opt-in/opt-out trial.
* excludes BadExits, tor versions that aren't recommended, and low
consensus weight directory mirrors.
* reduces the weighting of Exits to avoid overloading them.
* places limits on the weight of any one fallback.
* includes an IPv6 address and orport for each FallbackDir, as
implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks
until #17840 is merged.)
* generated output includes timestamps & Onionoo URL for traceability.
* unit test ensures that we successfully load all included default
fallback directories.
Closes ticket #15775. Patch by "teor".
OnionOO script by "weasel", "teor", "gsathya", and "karsten".
These scripts are now a little more bulletproof, cache data a little
better, and generate more information. Notably, they search for the
vectors or edges to cut that would lower the size of the largest
SCC.
Additional fixes to make the change work;
- fix Python 2 vs 3 issues
- fix some PEP 8 warnings
- handle paths with numbers correctly
- mention the make rule in doc/HACKING.
I've copied the entries from changes/, labeled the ones that also
appeared in 0.2.4.22, sorted them lightly with a python script
(added to maint), and combined sections with the same name.
I didn't combine sections without a description (e.g. "Minor
bugfixes:"), since we'll probably add a description to those.
(I had a bad clone of Knuth's algorithm sitting around in an old code
repository of mine. I added orphan detection and smarter hyphenation;
it seems to give marginally better results than we had before.)
We've accumulated a lot of cruft in this directory over the years: so
much, that it passed the point of being so disorganized that we no
longer browsed through it to see how bad it had gotten.
This patch (based on changes by rl1987) tries to remove the most
useless items, and split the others into reasonable directories. It
creates a new scripts/ directory for maint and test scripts.
This patch was generated with the script below. No other changes are made in
this patch.
#############
# new directories
mkdir -p contrib/test-tools
mkdir -p contrib/or-tools
mkdir -p contrib/dirauth-tools
mkdir -p contrib/operator-tools
mkdir -p contrib/client-tools
mkdir -p contrib/test-tools
mkdir -p contrib/dist
mkdir -p contrib/dist/suse
mkdir -p contrib/win32build
mkdir -p scripts/maint
mkdir -p scripts/test
############
# Deleted -- nobody who wants this is going to be looking for it here any
# longer. Also, nobody wants it.
git rm contrib/auto-naming/README
# Deleted: We no longer do polipo.
git rm contrib/polipo/Makefile.mingw
git rm contrib/polipo/README
git rm contrib/polipo/polipo-mingw.nsi
# We haven't even tried to run this for ages. It is a relic of a bygone era
git rm contrib/mdd.py
# contrib/dir-tools/directory-archive/
# Tools for running a directory archive. No longer used - deleting them.
git rm contrib/directory-archive/crontab.sample
git rm contrib/directory-archive/fetch-all
git rm contrib/directory-archive/fetch-all-v3
git rm contrib/directory-archive/tar-them-up
git rm contrib/directory-archive/fetch-all-functions
git rm contrib/directory-archive/sort-into-month-folder
# This appears to be related to very old windows packaging stuff.
git rm contrib/bundle.nsi
git rm contrib/package_nsis-weasel.sh
git rm contrib/package_nsis.sh
git rm contrib/netinst.nsi
git rm contrib/torinst32.ico
git rm contrib/xenobite.ico
# This should not be needed for cross-compilation any more, should it?
git rm contrib/cross.sh
# I don't think anyone ever used this.
git rm contrib/make-signature.sh
# These are attempts to send tor controller commands from the command-line.
# They don't support modern authentication.
git rm contrib/tor-ctrl.sh
# this is for fetching about a tor server from a dirauth. But it
# doesn't authenticate the dirauth: yuck.
git rm contrib/sd
# wow, such unused, very perl4.
git rm contrib/tor-stress
####### contrib/dirauth-tools/
# Tools for running a directory authority
git mv contrib/add-tor contrib/dirauth-tools/
git mv contrib/nagios-check-tor-authority-cert contrib/dirauth-tools/
#######
# contrib/or-tools/
# Tools for examining relays
git mv contrib/check-tor contrib/or-tools/check-tor
git mv contrib/checksocks.pl contrib/or-tools/checksocks.pl
git mv contrib/exitlist contrib/or-tools/exitlist
#######
# contrib/operator-tools
# Tools for running a relay.
git mv contrib/linux-tor-prio.sh contrib/operator-tools/linux-tor-prio.sh
git mv contrib/tor-exit-notice.html contrib/operator-tools/tor-exit-notice.html
git mv contrib/tor.logrotate.in contrib/operator-tools/
######
# contrib/dist
git mv contrib/rc.subr contrib/dist/
git mv contrib/tor.sh.in contrib/dist/
git mv contrib/torctl.in contrib/dist/
git mv contrib/suse/* contrib/dist/suse/
######
# client-tools
git mv contrib/torify contrib/client-tools/torify
git mv contrib/tor-resolve.py contrib/client-tools/
######
# win32build
git mv contrib/package_nsis-mingw.sh contrib/win32build/
git mv contrib/tor.nsi.in contrib/win32build/
# Erinn didn't ask for this...
git mv contrib/tor-mingw.nsi.in contrib/win32build/
git mv contrib/tor.ico contrib/win32build/
######
# scripts/test
git mv contrib/cov-blame scripts/test/cov-blame
git mv contrib/cov-diff scripts/test/cov-diff
git mv contrib/coverage scripts/test/coverage
git mv contrib/scan-build.sh scripts/test/
######## scripts/maint
# Maintainance scripts
#
# These are scripts for developers to use when hacking on Tor. They mostly
# look at the Tor source in one way or another.
git mv contrib/findMergedChanges.pl scripts/maint/findMergedChanges.pl
git mv contrib/checkOptionDocs.pl scripts/maint/checkOptionDocs.pl
git mv contrib/checkSpace.pl scripts/maint/checkSpace.pl
git mv contrib/redox.py scripts/maint/redox.py
git mv contrib/updateVersions.pl scripts/maint/updateVersions.pl
git mv contrib/checkLogs.pl scripts/maint/checkLogs.pl
git mv contrib/format_changelog.py scripts/maint/
