Commit Graph

166 Commits

Author SHA1 Message Date
Nick Mathewson
b64c224362 fix crash in lintChanges.py 2016-09-22 08:33:09 -04:00
Nick Mathewson
5927ed8d33 checkSpace.pl now forbids more identifiers.
The functions it warns about are:
  assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
  strdup, strndup, calloc.

Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
2016-09-06 12:35:37 -04:00
Nick Mathewson
5aa6a19de5 Update lintChanges script to understand 'Minor features (debugging)'
Previously we would give a warning because we didn't say "Fixes bug
Foo" on these.
2016-08-02 11:40:08 -04:00
Nick Mathewson
f3575a45ce Merge branch 'maint-0.2.8' 2016-07-29 10:23:38 -04:00
teor (Tim Wilson-Brown)
1c4a2dd208 Remove a fallback that was on the hardcoded list, then opted-out
The fallback was added in 0.2.8.2-alpha.
2016-07-29 10:23:15 -04:00
Sebastian Hahn
265e40b481 Raise libevent dependency to 2.0.10-stable or newer
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
2016-07-04 12:40:09 +02:00
Nick Mathewson
f42dbc4e26 Merge branch 'maint-0.2.8' 2016-06-28 19:15:43 -04:00
teor (Tim Wilson-Brown)
26146dbe9e Comment-out fallbacks in a way the stem fallback parser understands
If we manually remove fallbacks in C by adding '/*' and '*/' on separate
lines, stem still parses them as being present, because it only looks at
the start of a line.

Add a comment to this effect in the generated source code.
2016-06-28 19:15:08 -04:00
teor (Tim Wilson-Brown)
dae442a382
Whitelist a fallback's new IPv6 address
The IPv4-only entry will be used for 0.2.8.
The IPv4 and IPv6 entry will be considered in 0.2.9.
2016-06-23 10:38:52 +10:00
teor (Tim Wilson-Brown)
828e2e1a2e
Remove a fallback that changed DirPort
The operator has confirmed that the DirPort change is perament.
The relay will be reconsidered as a fallback in 0.2.9.
2016-06-23 10:38:03 +10:00
Nick Mathewson
61dac57aa3 Merge branch 'maint-0.2.8' 2016-06-22 08:20:11 -04:00
teor (Tim Wilson-Brown)
69635e41c8
Remove and blacklist 3 fallbacks which disappeared
Blacklist them in case they appear again.
2016-06-22 12:18:10 +10:00
teor (Tim Wilson-Brown)
7e9532b9be
Remove and blacklist 4 fallbacks which are unsuitable
Remove a fallback that changed its fingerprint after it was listed
This happened after to a software update:
https://lists.torproject.org/pipermail/tor-relays/2016-June/009473.html

Remove a fallback that changed IPv4 address

Remove two fallbacks that were slow to deliver consensuses,
we can't guarantee they'll be fast in future.

Blacklist all these fallbacks until operators confirm they're stable.
2016-06-22 12:16:57 +10:00
teor (Tim Wilson-Brown)
0a79e92914
Update the fallback whitelist and blacklist
Operators have sent emails asking to have their relays added or
removed from the fallback list. Since none of the blacklisted
relays are in the hard-coded falback list, it does not need to
be changed.
2016-06-22 11:56:40 +10:00
teor (Tim Wilson-Brown)
4090612246
Avoid errors in updateFallbackDirs.py when there are no fallbacks 2016-06-22 11:56:31 +10:00
teor (Tim Wilson-Brown)
6ce53668f4
Document how to test the hard-coded fallback list 2016-06-22 11:56:27 +10:00
Nick Mathewson
87593702eb roger says this url is better 2016-05-27 15:11:11 -04:00
Nick Mathewson
3934e78bb9 Make format_changelog.py add links to bugs 2016-05-27 09:26:49 -04:00
Nick Mathewson
5dbb0bdae9 Merge branch 'maint-0.2.8' 2016-05-16 08:33:53 -04:00
teor (Tim Wilson-Brown)
ab06b79cba Simplify string cleansing in fallback update script
No behavioural change
2016-04-29 11:59:46 +10:00
teor (Tim Wilson-Brown)
d41f92b006 Improve logging in fallback update script
Log a notice just before the script is about to perform a
potentially time-consuming operation

Clarify the warning when py2-ipaddress isn't found

Make log levels more consistent

No behavioural change (just logging)
2016-04-29 11:59:46 +10:00
teor (Tim Wilson-Brown)
46d813922a Improve comments in fallback update script 2016-04-29 11:43:55 +10:00
Nick Mathewson
fb9c9e04f0 Merge branch 'maint-0.2.8' 2016-04-26 19:27:39 -04:00
teor (Tim Wilson-Brown)
de08f2d96b Whitelist additional fallbacks after creating April 2016 list 2016-04-26 19:26:22 -04:00
teor (Tim Wilson-Brown)
ba7691071e Report fallback directory detail changes when rebuilding list
As well as the existing reports of IPv6 address additions or removals,
the script now warns when keys change but IPv4:ORPort or
IPv6:IPv6ORPort remain the same.

Existing checks for other whitelist detail changes have also
been re-worded and upgraded to warnings.

This makes it easier for changes to be identified so operators can
be contacted to confirm whether the change is stable.
2016-04-26 19:26:22 -04:00
teor (Tim Wilson-Brown)
8b90d45f78 Add fallbacks to white/blacklist from operator responses
Also add misbehaving relays to updateFallbackDirs.py blacklist,
but leave them commented out in case it's a transient issue,
or it's been resolved by the download check fixes. (These
relays hang stem's downloader. It's unlikely they'll ever help
clients.)
2016-04-26 19:26:21 -04:00
teor (Tim Wilson-Brown)
c157a31ee8 Limit fallbacks from the same operator
Use IP address, effective family, and contact info to
discover and limit fallbacks to one per operator.

Also analyse netblock, ports, IP version, and Exit flag,
and print the results. Don't exclude any fallbacks from
the list because of netblocks, ports, IP version, or
Exit flag.
2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
999834324b Simplify fallback list output
When creating the list of fallbacks for a release:
* don't output fallback name and contact
* sort fallbacks by fingerprint
2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
78ec782f76 Select fallbacks by bandwidth rather than consensus weight
But as advertised bandwidth is controlled by relays,
use consensus weight and median weight to bandwidth ratio
to approximate measured bandwidth.

Includes minor comment changes and parameter reordering.
2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
e115523c98 Only count active fallbacks in updateFallbackDirs.py
Previously, we would cut the list down to 100 fallbacks,
then check if they could serve a consensus, and comment
them out if they couldn't.
This would leave us with fewer than 100 active fallbacks.

Now, we stop when there are 100 active fallbacks.

Also count fallbacks with identical contact info.
Also fix minor logging issues.
2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
205a641e1d Remove fallback directory weights
Give each fallback a set weight of 10.0 for client selection.

Fallbacks must have at least 3000 consensus weight.
This is (nominally) 100 times the expected extra load of
20 kilobytes per second (50 GB per month).

Fixes issue #17905.
2016-04-15 12:21:32 +10:00
teor (Tim Wilson-Brown)
7e1b8ae79c Improve fallback selection and output
Improve the download test:
* Allow IPv4 DirPort checks to be turned off.
* Add a timeout to stem's consensus download.
* Actually check for download errors, rather than ignoring them.
* Simplify the timeout and download error checking logic.

Tweak whitelist/blacklist checks to be more robust.
Improve logging, make it warn by default.

Cleanse fallback comments more thoroughly:
* non-printables (yes, ContactInfo can have these)
* // comments (don't rely on newlines to prevent // */ escapes)
2016-04-15 12:21:32 +10:00
Nick Mathewson
4043f2c95f Adopt the LCOV convention for marking lines as unreachable by tests.
Document this convention.

Add a script to post-process .gcov files in order to stop nagging us
about excluded lines.

Teach cov-diff to handle these post-processed files.

Closes ticket 16792
2016-04-12 21:12:10 -04:00
Nick Mathewson
560450f2fb helper script to highlight undocumented members 2016-03-26 10:31:59 -04:00
Nick Mathewson
52bc921402 Fix (most) lintChanges warnings on master. 2016-03-21 11:14:57 -04:00
Nick Mathewson
69fc025e95 Merge remote-tracking branch 'teor/fallbacks-201602-v2' 2016-02-28 15:51:22 +01:00
Nick Mathewson
57699de005 Update the copyright year. 2016-02-27 18:48:19 +01:00
teor (Tim Wilson-Brown)
e2202146d1 Update default fallback directories for 0.2.8.2-alpha (Feb 2016)
Allow fallback directories which have been stable for 7 days
to work around #18050, which causes relays to submit descriptors
with 0 DirPorts when restarted. (Particularly during Tor version
upgrades.)

Ignore low fallback directory count in alpha builds.
Set the target count to 50.
2016-02-27 10:04:00 +01:00
teor (Tim Wilson-Brown)
a6017d61e1 Fix minor fallback directory script issues with time zones 2016-02-27 10:03:59 +01:00
teor (Tim Wilson-Brown)
9580036986 Check if fallback directories can serve the consensus fast enough 2016-02-27 10:03:59 +01:00
Nick Mathewson
2123f0a2f9 Make the callgraph module-aware 2016-02-10 15:09:10 -05:00
Nick Mathewson
f09d03c384 use sortChanges to fold changes files into changelog. No additional editing or formatting yet. 2016-02-01 16:56:20 -05:00
Nick Mathewson
deea08c13f Fix all the warnings from lintChanges.py 2016-02-01 16:46:29 -05:00
teor (Tim Wilson-Brown)
ab3c86479a Add default fallback directories for the 0.2.8 alpha releases
Allow fallback directories which have been stable for 30 days
to work around #18050, which causes relays to submit descriptors
with 0 DirPorts when restarted. (Particularly during Tor version
upgrades.)

Ignore low fallback directory count in alpha builds.
2016-01-18 20:15:59 -05:00
teor (Tim Wilson-Brown)
efd2c9d05b Fix a display issue in updateFallbackDirs.py output 2016-01-18 12:36:49 +11:00
Nick Mathewson
d062baac86 Merge remote-tracking branch 'teor/bug17888' 2016-01-12 12:26:06 -05:00
Nick Mathewson
7616922e81 Merge remote-tracking branch 'teor/bug17887-v2' 2016-01-12 10:47:54 -05:00
teor (Tim Wilson-Brown)
1c139f60d2 Don't limit fallback directory candidates to 500
Only impose the limit when selecting fallback directories for
inclusion in tor.
2016-01-12 10:10:05 -05:00
teor (Tim Wilson-Brown)
da968e558e Fix various issues with fallback directory data handling
* support maximum history age in _avg_generic_history()
* fix division-by-zero trap in _avg_generic_history()
* skip missing (i.e. null/None) intervals in _avg_generic_history()
* Python timedelta.total_seconds() function not available in 2.6;
  replace with equivalent expression
* set DEBUG logging level to make relay exclusion reasons visible
* move CUTOFF_GUARD test to end in order to expose more exclusion
  reasons

Patch by "starlight", merge modifications by "teor".
2016-01-12 10:10:05 -05:00
teor (Tim Wilson-Brown)
bc2bed8979 Don't reduce the weight of exits selected as fallback directories
When selecting exits as fallback directories, don't reduce
their weights.

Closes ticket #17888.
2016-01-12 13:56:45 +11:00
teor (Tim Wilson-Brown)
430181fad2 Allow fallback directory selection to use day-old data
Allow cached or outdated Onionoo data to be used to choose
fallback directories, as long as it's less than a day old.

Modify last modified date checks in preparation for Onionoo change
2016-01-12 13:51:25 +11:00
teor
4c1c2a313d Add Fallback Directory Candidate Selection Script
"Tor has included a feature to fetch the initial consensus from nodes
 other than the authorities for a while now. We just haven't shipped a
 list of alternate locations for clients to go to yet.

 Reasons why we might want to ship tor with a list of additional places
 where clients can find the consensus is that it makes authority
 reachability and BW less important.

 We want them to have been around and using their current key, address,
 and port for a while now (120 days), and have been running, a guard,
 and a v2 directory mirror for most of that time."

Features:
* whitelist and blacklist for an opt-in/opt-out trial.
* excludes BadExits, tor versions that aren't recommended, and low
  consensus weight directory mirrors.
* reduces the weighting of Exits to avoid overloading them.
* places limits on the weight of any one fallback.
* includes an IPv6 address and orport for each FallbackDir, as
  implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks
  until #17840 is merged.)
* generated output includes timestamps & Onionoo URL for traceability.
* unit test ensures that we successfully load all included default
  fallback directories.

Closes ticket #15775. Patch by "teor".
OnionOO script by "weasel", "teor", "gsathya", and "karsten".
2015-12-16 05:54:40 +11:00
Nick Mathewson
a02f6e3d23 update codegen script to work with latest mozilla 2015-10-07 16:24:30 -04:00
Nick Mathewson
216024261b add more clang checkers; find no new bugs. 2015-09-22 11:37:50 -04:00
Nick Mathewson
742514be2f scan-build is more mature now; no need to hardcode the one in my /opt/clang-3.4 2015-09-22 09:58:16 -04:00
Nick Mathewson
1eb2106375 Document callgraph analysis code 2015-08-25 11:53:20 -04:00
Sebastian Hahn
1633d1ad1d Remove tor-fw-helper more thoroughly 2015-08-21 10:36:53 -04:00
Sebastian Hahn
38601f06b7 Observe workqueue_reply_t in check-spaces 2015-08-21 10:36:53 -04:00
Nick Mathewson
bb46630513 Hack up the scripts/maint/*callgraph* scripts to do more, better
These scripts are now a little more bulletproof, cache data a little
better, and generate more information.  Notably, they search for the
vectors or edges to cut that would lower the size of the largest
SCC.
2015-08-15 22:37:32 -04:00
Nick Mathewson
def5883bbb Update callgraph code to find and output strongly connected components 2015-08-10 12:11:34 -04:00
Nick Mathewson
ebff18d18e Allow cov-display to take directories as arguments 2015-08-05 11:06:41 -04:00
Nick Mathewson
26c9691186 Check in some scripts I use for callgraph analysis. 2015-07-30 12:23:28 -04:00
Nick Mathewson
90701c1de4 Add my script to dump per-file or per-function coverage 2015-07-16 11:55:44 -04:00
Nick Mathewson
e71c8801cf more generic scan-build script 2015-05-29 13:50:05 -04:00
Nick Mathewson
1b52e95028 Merge branch '12498_ed25519_keys_v6'
Fixed numerous conflicts, and ported code to use new base64 api.
2015-05-28 11:04:33 -04:00
Nick Mathewson
79db24b3d5 Sign extrainfo documents with ed25519
Extrainfo documents are now ed-signed just as are router
descriptors, according to proposal 220.  This patch also includes
some more tests for successful/failing parsing, and fixes a crash
bug in ed25519 descriptor parsing.
2015-05-28 10:42:22 -04:00
Nick Mathewson
b600b68b20 Revise makedesc.py: teach it how to emit ed signatures and crosscerts
Also, add a trivial ed25519-signed routerinfo to the tests.
2015-05-28 10:41:50 -04:00
Nick Mathewson
df05e195ee Add trunnel-generated items for link handshake code.
This includes the link handshake variations for proposal220.

We'll use this for testing first, and then use it to extend our
current code to support prop220.
2015-05-28 10:41:49 -04:00
Nick Mathewson
993b4b8918 Fix a bug in format_changelog, in a silly way 2015-05-05 18:23:56 -04:00
Nick Mathewson
e81951c489 Add collation/splitting support to sortChanges script 2015-05-05 11:24:01 -04:00
cypherpunks
e85ba7459e Revive updateVersions.pl with make update-versions. 2015-03-14 13:00:06 -04:00
cypherpunks
eeb753e871 Make check-docs work from out-of-tree builds. 2015-03-14 13:00:06 -04:00
cypherpunks
034f51dd02 Be exact about the number of spaces in the header.
Also fixes changes files that warn on this.
2015-03-09 09:00:40 -04:00
cypherpunks
9dc90a5b7b Add check-changes rule for checking formatting of changes files.
Additional fixes to make the change work;
- fix Python 2 vs 3 issues
- fix some PEP 8 warnings
- handle paths with numbers correctly
- mention the make rule in doc/HACKING.
2015-03-09 09:00:12 -04:00
Nick Mathewson
94a877381d Check for # characters in lintchanges 2015-02-19 09:54:09 -05:00
Nick Mathewson
fc0febc5c6 Commit the update-copyrights script.
(I'm tired of rewriting this by hand every January)
2015-01-02 14:28:21 -05:00
Nick Mathewson
f54e54b0b4 Bump copyright dates to 2015, in case someday this matters. 2015-01-02 14:27:39 -05:00
Nick Mathewson
845d92295f have lintchanges check header format. 2014-12-22 10:00:34 -05:00
Nick Mathewson
441a481bb8 Resolve issues in changes files 2014-12-22 09:49:33 -05:00
Nick Mathewson
8b532a8c81 Short python script to lint the changes files 2014-12-22 09:34:55 -05:00
Nick Mathewson
a28df3fb67 Merge remote-tracking branch 'andrea/cmux_refactor_configurable_threshold'
Conflicts:
	src/or/or.h
	src/test/Makefile.nmake
2014-11-27 22:39:46 -05:00
Mansour Moufid
3ab2c865bf Add a rule to the calloc semantic patch for argument ordering. 2014-11-02 11:54:42 -05:00
Mansour Moufid
533790ca77 The second argument to tor_calloc should be a constant.
Just like the conventional calloc.
2014-11-02 11:54:42 -05:00
Mansour Moufid
a746081f38 Refactor the calloc semantic patch.
This does not change its effects.
2014-11-02 11:54:42 -05:00
Nick Mathewson
5ad8721838 Repair format_changelog.py drupal breaks a little more. Drupal is weird. 2014-10-31 09:13:08 -04:00
Nick Mathewson
96e3936fb7 Teach format_changelog.py --blog about drupal <!--break--> 2014-10-30 17:08:42 -04:00
Nick Mathewson
0eb1886692 Twiddle the format_changelog.py option defaults. 2014-10-30 16:54:10 -04:00
Nick Mathewson
c11ce1ca5b Reinvoke format_changelog.py
Also, tweak it so that it puts major deprecations and requirements
early in the changelog.
2014-10-29 10:19:10 -04:00
Nick Mathewson
4e7046cfb2 Teach format_changelog.py to emit HTML. 2014-10-27 10:32:25 -04:00
Nick Mathewson
6c5db03e02 format_changelog: Sort sections case-insensitively 2014-10-19 14:19:22 -04:00
Nick Mathewson
5344093829 format_changelog.py: fix spelling of "hyphenatable" 2014-10-19 12:59:17 -04:00
Nick Mathewson
29b0e82b5a format_changelog.py: tweak hyphenation rules
Mark all our generated command names, and anything with a
double-hyphen, as unhyphenatable.
2014-10-19 12:57:57 -04:00
Nick Mathewson
45f534e6df Teach format_changelog to sort and collate sections.
Give it options to turn off collation and/or wrapping.
2014-10-19 12:44:19 -04:00
Nick Mathewson
1405fbcc58 format_changelog.py: learn about the ReleaseNotes format 2014-10-19 11:47:16 -04:00
Nick Mathewson
48af23eb0d Emit fewer spurious lines in cov-diff output. 2014-10-14 09:41:40 -04:00
Nick Mathewson
b42d811718 Commit the script I used to generate signed ri and ei documents 2014-10-13 14:30:03 -04:00
Andrea Shepard
7674308f62 Make 'make check-spaces' not complain about function pointers returning size_t or double 2014-09-30 22:49:02 -07:00
Nick Mathewson
50d15e06b3 Use --require-version to prevent running trunnel pre-1.2
(Also, regenerate trunnel stuff with trunnel 1.2.  This just adds a
few comments to our output.)
2014-09-25 14:49:00 -04:00
Nick Mathewson
d303d6244e Have the run_trunnel.sh script tell trunnel to update src/ext/trunnel too 2014-09-25 13:16:35 -04:00
Nick Mathewson
1b13139709 Add a script to run trunnel on the trunnel files.
Also, re-run the latest trunnel.

Closes ticket 13242
2014-09-25 12:32:08 -04:00