Commit Graph

648 Commits

Author SHA1 Message Date
teor (Tim Wilson-Brown)
efd2c9d05b Fix a display issue in updateFallbackDirs.py output 2016-01-18 12:36:49 +11:00
Nick Mathewson
d062baac86 Merge remote-tracking branch 'teor/bug17888' 2016-01-12 12:26:06 -05:00
Nick Mathewson
7616922e81 Merge remote-tracking branch 'teor/bug17887-v2' 2016-01-12 10:47:54 -05:00
teor (Tim Wilson-Brown)
1c139f60d2 Don't limit fallback directory candidates to 500
Only impose the limit when selecting fallback directories for
inclusion in tor.
2016-01-12 10:10:05 -05:00
teor (Tim Wilson-Brown)
da968e558e Fix various issues with fallback directory data handling
* support maximum history age in _avg_generic_history()
* fix division-by-zero trap in _avg_generic_history()
* skip missing (i.e. null/None) intervals in _avg_generic_history()
* Python timedelta.total_seconds() function not available in 2.6;
  replace with equivalent expression
* set DEBUG logging level to make relay exclusion reasons visible
* move CUTOFF_GUARD test to end in order to expose more exclusion
  reasons

Patch by "starlight", merge modifications by "teor".
2016-01-12 10:10:05 -05:00
teor (Tim Wilson-Brown)
bc2bed8979 Don't reduce the weight of exits selected as fallback directories
When selecting exits as fallback directories, don't reduce
their weights.

Closes ticket #17888.
2016-01-12 13:56:45 +11:00
teor (Tim Wilson-Brown)
430181fad2 Allow fallback directory selection to use day-old data
Allow cached or outdated Onionoo data to be used to choose
fallback directories, as long as it's less than a day old.

Modify last modified date checks in preparation for Onionoo change
2016-01-12 13:51:25 +11:00
teor
4c1c2a313d Add Fallback Directory Candidate Selection Script
"Tor has included a feature to fetch the initial consensus from nodes
 other than the authorities for a while now. We just haven't shipped a
 list of alternate locations for clients to go to yet.

 Reasons why we might want to ship tor with a list of additional places
 where clients can find the consensus is that it makes authority
 reachability and BW less important.

 We want them to have been around and using their current key, address,
 and port for a while now (120 days), and have been running, a guard,
 and a v2 directory mirror for most of that time."

Features:
* whitelist and blacklist for an opt-in/opt-out trial.
* excludes BadExits, tor versions that aren't recommended, and low
  consensus weight directory mirrors.
* reduces the weighting of Exits to avoid overloading them.
* places limits on the weight of any one fallback.
* includes an IPv6 address and orport for each FallbackDir, as
  implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks
  until #17840 is merged.)
* generated output includes timestamps & Onionoo URL for traceability.
* unit test ensures that we successfully load all included default
  fallback directories.

Closes ticket #15775. Patch by "teor".
OnionOO script by "weasel", "teor", "gsathya", and "karsten".
2015-12-16 05:54:40 +11:00
Nick Mathewson
1eb2106375 Document callgraph analysis code 2015-08-25 11:53:20 -04:00
Sebastian Hahn
1633d1ad1d Remove tor-fw-helper more thoroughly 2015-08-21 10:36:53 -04:00
Sebastian Hahn
38601f06b7 Observe workqueue_reply_t in check-spaces 2015-08-21 10:36:53 -04:00
Nick Mathewson
bb46630513 Hack up the scripts/maint/*callgraph* scripts to do more, better
These scripts are now a little more bulletproof, cache data a little
better, and generate more information.  Notably, they search for the
vectors or edges to cut that would lower the size of the largest
SCC.
2015-08-15 22:37:32 -04:00
Nick Mathewson
def5883bbb Update callgraph code to find and output strongly connected components 2015-08-10 12:11:34 -04:00
Nick Mathewson
26c9691186 Check in some scripts I use for callgraph analysis. 2015-07-30 12:23:28 -04:00
Nick Mathewson
993b4b8918 Fix a bug in format_changelog, in a silly way 2015-05-05 18:23:56 -04:00
Nick Mathewson
e81951c489 Add collation/splitting support to sortChanges script 2015-05-05 11:24:01 -04:00
cypherpunks
e85ba7459e Revive updateVersions.pl with make update-versions. 2015-03-14 13:00:06 -04:00
cypherpunks
eeb753e871 Make check-docs work from out-of-tree builds. 2015-03-14 13:00:06 -04:00
cypherpunks
034f51dd02 Be exact about the number of spaces in the header.
Also fixes changes files that warn on this.
2015-03-09 09:00:40 -04:00
cypherpunks
9dc90a5b7b Add check-changes rule for checking formatting of changes files.
Additional fixes to make the change work;
- fix Python 2 vs 3 issues
- fix some PEP 8 warnings
- handle paths with numbers correctly
- mention the make rule in doc/HACKING.
2015-03-09 09:00:12 -04:00
Nick Mathewson
94a877381d Check for # characters in lintchanges 2015-02-19 09:54:09 -05:00
Nick Mathewson
fc0febc5c6 Commit the update-copyrights script.
(I'm tired of rewriting this by hand every January)
2015-01-02 14:28:21 -05:00
Nick Mathewson
f54e54b0b4 Bump copyright dates to 2015, in case someday this matters. 2015-01-02 14:27:39 -05:00
Nick Mathewson
845d92295f have lintchanges check header format. 2014-12-22 10:00:34 -05:00
Nick Mathewson
441a481bb8 Resolve issues in changes files 2014-12-22 09:49:33 -05:00
Nick Mathewson
8b532a8c81 Short python script to lint the changes files 2014-12-22 09:34:55 -05:00
Nick Mathewson
a28df3fb67 Merge remote-tracking branch 'andrea/cmux_refactor_configurable_threshold'
Conflicts:
	src/or/or.h
	src/test/Makefile.nmake
2014-11-27 22:39:46 -05:00
Nick Mathewson
5ad8721838 Repair format_changelog.py drupal breaks a little more. Drupal is weird. 2014-10-31 09:13:08 -04:00
Nick Mathewson
96e3936fb7 Teach format_changelog.py --blog about drupal <!--break--> 2014-10-30 17:08:42 -04:00
Nick Mathewson
0eb1886692 Twiddle the format_changelog.py option defaults. 2014-10-30 16:54:10 -04:00
Nick Mathewson
c11ce1ca5b Reinvoke format_changelog.py
Also, tweak it so that it puts major deprecations and requirements
early in the changelog.
2014-10-29 10:19:10 -04:00
Nick Mathewson
4e7046cfb2 Teach format_changelog.py to emit HTML. 2014-10-27 10:32:25 -04:00
Nick Mathewson
6c5db03e02 format_changelog: Sort sections case-insensitively 2014-10-19 14:19:22 -04:00
Nick Mathewson
5344093829 format_changelog.py: fix spelling of "hyphenatable" 2014-10-19 12:59:17 -04:00
Nick Mathewson
29b0e82b5a format_changelog.py: tweak hyphenation rules
Mark all our generated command names, and anything with a
double-hyphen, as unhyphenatable.
2014-10-19 12:57:57 -04:00
Nick Mathewson
45f534e6df Teach format_changelog to sort and collate sections.
Give it options to turn off collation and/or wrapping.
2014-10-19 12:44:19 -04:00
Nick Mathewson
1405fbcc58 format_changelog.py: learn about the ReleaseNotes format 2014-10-19 11:47:16 -04:00
Andrea Shepard
7674308f62 Make 'make check-spaces' not complain about function pointers returning size_t or double 2014-09-30 22:49:02 -07:00
Nick Mathewson
3d6652f3d1 Improve comments in checkSpace.pl to explain how to fix its warnings 2014-08-18 10:03:14 -04:00
Nick Mathewson
2050846312 Bring remaining 0.2.5.5-alpha entries into changelog 2014-06-16 15:00:35 -04:00
Nick Mathewson
d92a01e820 Reformat 0255 changelog. Tweak formatter script. 2014-05-29 11:30:15 -04:00
Nick Mathewson
413a442f57 Start on the 0.2.5.5-alpha changelog.
I've copied the entries from changes/, labeled the ones that also
appeared in 0.2.4.22, sorted them lightly with a python script
(added to maint), and combined sections with the same name.

I didn't combine sections without a description (e.g. "Minor
bugfixes:"), since we'll probably add a description to those.
2014-05-29 11:21:23 -04:00
Nick Mathewson
747f368c6d make the changelog text wrapper respect arma's no-initial-openparen rule 2014-05-14 12:56:09 -04:00
Nick Mathewson
2e1ac274ab Script to detect unused autoconf outputs
Should help speed up mingw builds by a percent or two.
2014-05-07 03:56:51 -04:00
Nick Mathewson
8a173635bc Tweak the changelog formatter a little.
(I had a bad clone of Knuth's algorithm sitting around in an old code
repository of mine.  I added orphan detection and smarter hyphenation;
it seems to give marginally better results than we had before.)
2014-05-02 12:50:23 -04:00
Nick Mathewson
3266f04925 Fix the check-docs script
We broke it when we added anchors to the manpage.

This patch fixes it, and makes it sorta detect missing anchors.
2014-04-28 12:07:57 -04:00
Nick Mathewson
703ad69587 Deal with the aftermath of sorting contrib
This basically amounts to grepping for every file that mentioned
contrib and adjusting its references to refer to the right place.
2014-04-28 11:59:55 -04:00
Nick Mathewson
9230bc7c65 Clean the contrib directory with torch and machete.
We've accumulated a lot of cruft in this directory over the years: so
much, that it passed the point of being so disorganized that we no
longer browsed through it to see how bad it had gotten.

This patch (based on changes by rl1987) tries to remove the most
useless items, and split the others into reasonable directories.  It
creates a new scripts/ directory for maint and test scripts.

This patch was generated with the script below.  No other changes are made in
this patch.

#############
# new directories
mkdir -p contrib/test-tools
mkdir -p contrib/or-tools
mkdir -p contrib/dirauth-tools
mkdir -p contrib/operator-tools
mkdir -p contrib/client-tools
mkdir -p contrib/test-tools
mkdir -p contrib/dist
mkdir -p contrib/dist/suse
mkdir -p contrib/win32build

mkdir -p scripts/maint
mkdir -p scripts/test

############
# Deleted -- nobody who wants this is going to be looking for it here any
# longer.  Also, nobody wants it.
git rm contrib/auto-naming/README

# Deleted: We no longer do polipo.
git rm contrib/polipo/Makefile.mingw
git rm contrib/polipo/README
git rm contrib/polipo/polipo-mingw.nsi

# We haven't even tried to run this for ages. It is a relic of a bygone era
git rm contrib/mdd.py

# contrib/dir-tools/directory-archive/
# Tools for running a directory archive. No longer used - deleting them.
git rm contrib/directory-archive/crontab.sample
git rm contrib/directory-archive/fetch-all
git rm contrib/directory-archive/fetch-all-v3
git rm contrib/directory-archive/tar-them-up
git rm contrib/directory-archive/fetch-all-functions
git rm contrib/directory-archive/sort-into-month-folder

# This appears to be related to very old windows packaging stuff.
git rm contrib/bundle.nsi
git rm contrib/package_nsis-weasel.sh
git rm contrib/package_nsis.sh
git rm contrib/netinst.nsi
git rm contrib/torinst32.ico
git rm contrib/xenobite.ico

# This should not be needed for cross-compilation any more, should it?
git rm contrib/cross.sh

# I don't think anyone ever used this.
git rm contrib/make-signature.sh

# These are attempts to send tor controller commands from the command-line.
# They don't support modern authentication.
git rm contrib/tor-ctrl.sh

# this is for fetching about a tor server from a dirauth. But it
# doesn't authenticate the dirauth: yuck.
git rm contrib/sd

# wow, such unused, very perl4.
git rm contrib/tor-stress

####### contrib/dirauth-tools/
# Tools for running a directory authority

git mv contrib/add-tor contrib/dirauth-tools/
git mv contrib/nagios-check-tor-authority-cert contrib/dirauth-tools/

#######
# contrib/or-tools/
# Tools for examining relays
git mv contrib/check-tor contrib/or-tools/check-tor
git mv contrib/checksocks.pl contrib/or-tools/checksocks.pl
git mv contrib/exitlist contrib/or-tools/exitlist

#######
# contrib/operator-tools

# Tools for running a relay.
git mv contrib/linux-tor-prio.sh contrib/operator-tools/linux-tor-prio.sh
git mv contrib/tor-exit-notice.html contrib/operator-tools/tor-exit-notice.html
git mv contrib/tor.logrotate.in contrib/operator-tools/

######
# contrib/dist

git mv contrib/rc.subr contrib/dist/
git mv contrib/tor.sh.in contrib/dist/
git mv contrib/torctl.in contrib/dist/
git mv contrib/suse/* contrib/dist/suse/

######
# client-tools
git mv contrib/torify contrib/client-tools/torify
git mv contrib/tor-resolve.py contrib/client-tools/

######
# win32build

git mv contrib/package_nsis-mingw.sh contrib/win32build/
git mv contrib/tor.nsi.in contrib/win32build/
# Erinn didn't ask for this...
git mv contrib/tor-mingw.nsi.in contrib/win32build/
git mv contrib/tor.ico contrib/win32build/

######
# scripts/test
git mv contrib/cov-blame scripts/test/cov-blame
git mv contrib/cov-diff scripts/test/cov-diff
git mv contrib/coverage scripts/test/coverage
git mv contrib/scan-build.sh scripts/test/

######## scripts/maint
# Maintainance scripts
#
# These are scripts for developers to use when hacking on Tor.  They mostly
# look at the Tor source in one way or another.
git mv contrib/findMergedChanges.pl scripts/maint/findMergedChanges.pl
git mv contrib/checkOptionDocs.pl scripts/maint/checkOptionDocs.pl
git mv contrib/checkSpace.pl scripts/maint/checkSpace.pl
git mv contrib/redox.py scripts/maint/redox.py
git mv contrib/updateVersions.pl scripts/maint/updateVersions.pl
git mv contrib/checkLogs.pl scripts/maint/checkLogs.pl
git mv contrib/format_changelog.py scripts/maint/
2014-04-28 11:34:53 -04:00