nihilist/tor
1
0
Fork 0
mirror of https://gitlab.torproject.org/tpo/core/tor.git synced 2024-11-30 23:53:32 +01:00
Code Issues Packages Projects Releases Wiki Activity
37,650 Commits 53 Branches 630 Tags 128 MiB
ecb9cee266
Commit Graph

962 Commits

Author SHA1 Message Date
Nick Mathewson
0fa3dc3228 begin_cell_parse(): Add an assertion to please coverity. 
Coverity doesn't understand that if begin_cell_parse() returns 0 and
sets is_begindir to 0, its address field will always be set.

Fixes bug 30126; bugfix on 0.2.4.7-alpha; Fixes CID 1447296.
 2019-06-28 11:29:51 -04:00
Nick Mathewson
089ef46063 Merge branch 'bug30721_squashed' 2019-06-26 09:57:20 -04:00
teor
fb93646c1c resolve: split sub-functions out of tor_addr_lookup() 
And remove the practracker exception for tor_addr_lookup().

Cleanup after 30721.
 2019-06-26 09:55:37 -04:00
Nick Mathewson
0fe9657c8c Merge remote-tracking branch 'tor-github/pr/1119' 2019-06-26 09:50:00 -04:00
Nick Mathewson
a91ed23403 Use structvar to find the types for config vars. 2019-06-25 12:51:25 -04:00
teor
c131b0763e
stats: add comments about the required chunk structure in extra info files 
These comments should prevent future instances of 30958.

And allow a larger file in practracker.

Follow up after 30958.
 2019-06-25 12:30:59 +10:00
Nick Mathewson
c60a85d22a Add a "typed_var" abstraction to implement lvalue access in C. 
Right now, this has been done at a high level by confparse.c, but it
makes more sense to lower it.

This API is radically un-typesafe as it stands; we'll be wrapping it
in a safer API as we do #30914 and lower the struct manipulation
code as well.

Closes ticket 30864.
 2019-06-24 17:50:43 -04:00
Nick Mathewson
81d16d8d0c checkSpace.pl: Allow 'bool' before a space and an open-paren 
We need this so we can declare function pointers returning bool
without upsetting our style checker. :/
 2019-06-24 15:11:57 -04:00
teor
45be44ed9c stats: Split extrainfo_dump_to_string() into smaller functions. 
Closes ticket 30956.
 2019-06-24 20:47:44 +10:00
David Goulet
9644f3462a Make the great grand practracker happy... 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-06-19 07:42:30 -04:00
David Goulet
05b6f73f12 Make the great grand practracker happy... 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-06-19 07:41:45 -04:00
Nick Mathewson
93ca08d66c Merge branch 'ticket29533_squashed' 2019-06-17 08:56:13 -04:00
rl1987
e8da65ee91 Fix SC2034 in pre-push.git-hook 2019-06-17 08:56:06 -04:00
Nick Mathewson
52d386c9b0 Merge remote-tracking branch 'tor-github/pr/1100' 2019-06-17 08:54:12 -04:00
teor
16e71c7fb0
practracker: accept an extra line in nt_service_install() 
Part of 30799.
 2019-06-13 18:23:50 +10:00
George Kadianakis
438b7eec85 Merge branch 'tor-github/pr/1099' 2019-06-12 13:02:49 +03:00
David Goulet
e9d99d2e15 Merge branch 'tor-github/pr/1083' 2019-06-11 11:43:15 -04:00
Nick Mathewson
eb02c323eb Merge branch 'maint-0.4.1' 2019-06-11 08:41:55 -04:00
teor
396134188f
Stop hard-coding env vars in the git scripts 
Set the env vars:
* TOR_MASTER_NAME to override the tor master branch name, and
* TOR_WKT_NAME to override the worktree path

Fixes bug 30841; bugfix on 0.4.0.1-alpha.
 2019-06-11 14:34:44 +10:00
teor
973800b847
scripts/git: Stop hard-coding the bash path in the git scripts 
Some OSes don't have bash in /usr/bin, others have an ancient bash at
this path.

Fixes bug 30840; bugfix on 0.4.0.1-alpha.
 2019-06-11 14:29:10 +10:00
Nick Mathewson
0635170cf1 Add an environment variable to set GIT_PATH in git scripts 2019-06-10 09:05:26 -04:00
Nick Mathewson
bbddda3a18 push-all: Use TOR_UPSTREAM_REMOTE_NAME to override UPSTREAM_BRANCH 
We already allow this override in our pre-push hook, so let's allow
it here too.

(I call my upstream branches "origin")
 2019-06-10 09:01:17 -04:00
Nick Mathewson
46da530431 Update git scripts for new versions 
{maint,release}-0.4.1 are now real

{maint,release}-0.3.4 are now EOL

Closes ticket 30822
 2019-06-10 09:01:17 -04:00
teor
6be9d3aed8
practracker: accept one extra line in routerlist.c 
practracker exception for 30780.
 2019-06-06 18:24:17 +10:00
teor
b07b1a4f6d
practracker: accept 3 extra lines in router_parse_entry_from_string() 
practracker exception for bug 30781.
 2019-06-06 13:42:30 +10:00
Nick Mathewson
2926f49b24 Regenerate practracker excpetions file 2019-06-05 09:34:14 -04:00
Nick Mathewson
ae490189f8 practracker update 2019-06-05 09:06:43 -04:00
Nick Mathewson
0c16ec4c07 Merge branch 'bug30286_2_squashed' 2019-06-05 08:32:39 -04:00
rl1987
e8aab46a3a Document TOR_UPSTREAM_REMOTE_NAME env variable 2019-06-05 08:29:55 -04:00
George Kadianakis
99bf3d8e14 Merge branch 'tor-github/pr/1072' 2019-06-05 14:40:38 +03:00
Nick Mathewson
dc5cdde60c update practracker 2019-06-04 08:29:43 -04:00
Nick Mathewson
5e594831c7 Practracker excpetions. 2019-05-31 12:41:44 -04:00
George Kadianakis
70553a9134 Merge branch 'tor-github/pr/1037' 2019-05-29 21:25:32 +03:00
David Goulet
ff9aa32143 Merge branch 'tor-github/pr/1047' 2019-05-28 14:59:07 -04:00
Nick Mathewson
0bc1241494 Make sure that we send at least some random data in RELAY_DATA cells 
Proposal 289 prevents SENDME-flooding by requiring the other side to
authenticate the data it has received.  But this data won't actually
be random if they are downloading a known resource.  "No problem",
we said, "let's fell the empty parts of our cells with some
randomness!" and we did that in #26871.

Unfortunately, if the relay data payloads are all completely full,
there won't be any empty parts for us to randomize.

Therefore, we now pick random "randomness windows" between
CIRCWINDOW_INCREMENT/2 and CIRCWINDOW_INCREMENT. We remember whether we have
sent a cell containing at least 16 bytes of randomness in that window.  If we
haven't, then when the window is exhausted, we send one.  (This window approach
is designed to lower the number of rng checks we have to do.  The number 16 is
pulled out of a hat to change the attacker's guessing difficulty to
"impossible".)

Implements 28646.
 2019-05-27 14:20:07 +03:00
Nick Mathewson
7893f2cd73 cov-test-determinism: use the same RNG seed as in travis.yml 
We added this facility so that we could get deterministic PRNG
behavior for coverage testing on tests that use a replaced PRNG.
We need to have our coverage determinism tool test for this as well.
 2019-05-23 14:36:01 -04:00
Nick Mathewson
e6b862e6a8 Merge branch 'ticket30428_041_02_squashed' 2019-05-22 11:48:43 -04:00
David Goulet
3835a3acf5 sendme: Properly record SENDMEs on both edges 
Turns out that we were only recording the "b_digest" but to have
bidirectionnal authenticated SENDMEs, we need to use the "f_digest" in the
forward cell situation.

Because of the cpath refactoring, this commit plays with the crypt_path_ and
relay_crypto_t API a little bit in order to respect the abstractions.

Previously, we would record the cell digest as the SENDME digest in the
decrypt cell function but to avoid code duplication (both directions needs to
record), we now do that right after iff the cell is recognized (at the edge).
It is now done in circuit_receive_relay_cell() instead.

We now also record the cell digest as the SENDME digest in both relay cell
encryption functions since they are split depending on the direction.
relay_encrypt_cell_outbound() and relay_encrypt_cell_inbound() need to
consider recording the cell digest depending on their direction (f vs b
digest).

Fixes #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-22 11:47:20 -04:00
David Goulet
59b9eecc19 sendme: Record cell digest on both client and exit 
It turns out that only the exit side is validating the authenticated SENDME v1
logic and never the client side. Which means that if a client ever uploaded
data towards an exit, the authenticated SENDME logic wouldn't apply.

For this to work, we have to record the cell digest client side as well which
introduced a new function that supports both type of edges.

This also removes a test that is not valid anymore which was that we didn't
allow cell recording on an origin circuit (client).

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-22 11:47:20 -04:00
Nick Mathewson
66eb0a5a32 updateCopyright: look at the current year. 2019-05-20 12:02:20 -04:00
Nick Mathewson
2f31c8146f rectify_include_paths: warn instead of aborting on duplicate headers 
We have two sendme.h files at the moment; we should fix that, but
not in this branch.
 2019-05-20 11:52:45 -04:00
Mike Perry
84274000d8 Yes, these functions really do have to be this long. 2019-05-16 20:29:09 +00:00
David Goulet
39a14421b1 Merge branch 'tor-github/pr/1021' 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-15 15:35:22 -04:00
George Kadianakis
d71fa707dd Merge branch 'bug28780-squashed3-rebased' into bug28780_rebase 2019-05-15 16:46:51 +03:00
Mike Perry
ff2a980935 The practracker beatings will continue until the files get smaller. 2019-05-15 16:44:59 +03:00
Mike Perry
5f47d582d5 Practracker beatings are even more fun when they get caused mid-PR due to a github auto-rebase of a PR 
Because github PRs choose the most recent origin/master at the time of the PR
(and for any fixups pushed to a PR later to send to CI), there are tons of
conflicts and unexpected practracker issues.

This means CI can suddenly fail after fixups to a branch that pass locally.

Then CI fails and we have to close and re-open the PR.
 2019-05-15 15:10:48 +03:00
Nick Mathewson
b9f50a2d77 update practracker for tor_init 2019-05-14 19:56:20 -04:00
Nick Mathewson
dd537ba35f Update practracker for 30452 2019-05-14 19:22:35 -04:00
Nick Mathewson
43d4119454 Merge remote-tracking branch 'tor-github/pr/1004' 2019-05-14 11:43:10 -04:00
Nick Mathewson
9ad2eb8f73 Merge branch 'bug28683_30173_29203_squashed' 2019-05-13 14:33:31 -04:00
Mike Perry
507df74b31 The practracker beatings will continue until the functions get smaller. 2019-05-13 14:30:35 -04:00
Nick Mathewson
c6523a6398 Merge remote-tracking branch 'tor-github/pr/998' 2019-05-13 14:25:54 -04:00
rl1987
6f1527f888 pre-push.git-hook: Check remote name to see if it's upstream 2019-05-11 18:58:14 +03:00
George Kadianakis
501d1ae0bd Merge branch 'tor-github/pr/973' 2019-05-10 12:49:01 +03:00
Neel Chauhan
3cafdeb8c0 Only call tor_addr_parse() in circuit_is_acceptable() when needed 2019-05-07 11:52:56 -04:00
George Kadianakis
7f2cd6545c Hiding crypt_path_t: Hide 'crypto' usage in sendme.c 2019-05-03 18:29:51 +03:00
George Kadianakis
593b7726e9 Hiding crypt_path_t: Trivial changes to satisfy check-local. 2019-05-03 18:15:26 +03:00
David Goulet
b3492d53c3 Merge branch 'tor-github/pr/984' 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-03 10:56:12 -04:00
George Kadianakis
b2c2cb9287 Merge branch 'tor-github/pr/986' 2019-05-02 18:12:52 +03:00
Nick Mathewson
0f365e2f46 practracker updates. 2019-05-02 09:22:13 -04:00
Mike Perry
e1771aeb51 The practracker beatings will continue until our files get smaller. 2019-05-01 21:04:40 +00:00
rl1987
d91deeee45 pre-push.git-hook: Allow fixup and squash commits when pushing to non-upstream branches 2019-05-01 14:01:30 +03:00
Taylor Yu
58ec88e806 Coccinelle scripts for control.c refactor 2019-04-30 13:18:46 -05:00
Taylor Yu
8e7316bae4 Split reply formatting out of control_fmt.c 
Split the core reply formatting code out of control_fmt.c into
control_proto.c.  The remaining code in control_format.c deals with
specific subsystems and will eventually move to join those subsystems.
 2019-04-30 13:18:46 -05:00
George Kadianakis
9084a90b00 Merge branch 'tor-github/pr/936' 2019-04-30 19:21:15 +03:00
George Kadianakis
a44aca5453 Merge branch 'tor-github/pr/993' 2019-04-30 19:13:57 +03:00
George Kadianakis
86f8dfe419 Merge branch 'tor-github/pr/983' 2019-04-30 19:13:30 +03:00
David Goulet
43c119fedb Merge branch 'tor-github/pr/980' 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-04-30 11:50:36 -04:00
Nick Mathewson
b5a62b1ef5 Move dirauth periodic events into dirauth module. 
Closes ticket 30294.
 2019-04-30 11:14:59 -04:00
David Goulet
535ba0d7c5 practracker: Update exceptions for #26288 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-04-29 12:27:53 -04:00
Nick Mathewson
1d44ac9acd Make nodelist_get_list() return a const pointer. 2019-04-26 10:36:49 -04:00
Nick Mathewson
01b07c548b Use parsing code for the simpler controller commands. 
(This should be all of the command that work nicely with positional
arguments only.)

Some of these commands should probably treat extra arguments as
incorrect, but for now I'm trying to be careful not to break
any existing users.
 2019-04-25 14:13:03 -04:00
Nick Mathewson
dbfe1a14e4 When parsing a multiline controller command, be careful with linebreaks 
The first line break in particular was mishandled: it was discarded
if no arguments came before it, which made it impossible to
distinguish arguments from the first line of the body.

To solve this, we need to allocate a copy of the command rather than
using NUL to separate it, since we might have "COMMAND\n" as our input.

Fixes ticket 29984.
 2019-04-25 14:13:03 -04:00
Nick Mathewson
d0a0f3e8cd Allow do_resolve() to be longer. 2019-04-24 14:15:18 -04:00
David Goulet
4587e67dfd Merge branch 'tor-github/pr/930' 2019-04-24 09:39:32 -04:00
teor
de91b83849
practracker: Accept ~80 extra lines in src/core/or/policies.c 
Part of 23588.
 2019-04-24 17:31:02 +10:00
Nick Mathewson
15d4238383 Merge remote-tracking branch 'tor-github/pr/944' 2019-04-23 15:39:23 -04:00
Nick Mathewson
b0459de217 Merge branch 'ticket30051_take2_squashed' 2019-04-23 12:51:44 -04:00
rl1987
a85cd5b41b Call practracker from pre-push and pre-commit hooks 2019-04-23 12:51:29 -04:00
Nick Mathewson
a7599c5be2 Merge remote-tracking branch 'tor-github/pr/962' 2019-04-23 12:48:37 -04:00
teor
bffba9d26f
practracker: accept more lines in microdescs_parse_from_string() 
Part of 28223.
 2019-04-19 10:34:16 +10:00
George Kadianakis
78223ab0fc Merge branch 'tor-github/pr/938' 2019-04-18 13:23:32 +03:00
George Kadianakis
2db9bb02c7 Merge branch 'tor-github/pr/925' 2019-04-18 13:22:54 +03:00
teor
f12b990bbf
practracker: accept the extra 25 line string from 27821 2019-04-17 18:44:26 +10:00
Neel Chauhan
f643020e64 Update practracker exceptions.txt for policies.c 2019-04-16 09:16:52 -04:00
Nick Mathewson
82b690d7da Merge remote-tracking branch 'tor-github/pr/910' 2019-04-16 08:09:20 -04:00
Neel Chauhan
d4f980d29a Update exceptions.txt for Bug #29613 2019-04-14 14:27:29 -04:00
Nick Mathewson
7332346002 Changes file and practracker updates for 30149. 2019-04-11 18:58:44 -04:00
Nick Mathewson
e39b53ef7d changes file and practracker updates for 30147. 2019-04-11 17:59:21 -04:00
Nick Mathewson
98e08b452f Fix pre-commit hook to correctly allow empty changes files. 
Fixes bug 30120; bugfix not in any released Tor.
 2019-04-10 11:49:23 -04:00
teor
5722c6d12d
scripts: In git-pull-all.sh, also fetch the latest tor-github pull requests 
Implements ticket 30114.
 2019-04-10 19:26:47 +10:00
teor
60c46c6cd0
practracker: accept 4 extra lines due to 30041 2019-04-10 18:29:11 +10:00
teor
7741b21d0e
practracker: accept 6 extra lines in tortls_nss.c:tor_tls_context_new() 
These lines were added to fix bug 29241.
 2019-04-06 12:26:30 +10:00
rl1987
705c1c28c7 Call pre-commit hook from pre-push script 2019-04-05 18:59:48 +03:00
Nick Mathewson
f021ca2d52 practracker: allow config.c to be a touch larger. 2019-04-05 09:59:45 -04:00
Nick Mathewson
ee6f54ff3f Merge remote-tracking branch 'tor-github/pr/860' 2019-04-03 08:33:40 -04:00
Nick Mathewson
f0e39df5ce allow circuituse.c to get even longer. 2019-03-29 14:30:33 -04:00
teor
a10d4adc25
Stop assuming that /usr/bin/python3 exists 
For scripts that work with python2, use /usr/bin/python.
Otherwise, use /usr/bin/env python3.

Fixes bug 29913; bugfix on 0.2.5.3-alpha.
 2019-03-27 11:07:55 +10:00
Nick Mathewson
7b9732063c practracker updates from messaging_v3 merge 
(main.c is a bit better, but shutdown.c is ugly)
 2019-03-26 20:14:21 -04:00
teor
beceb079e1
practracker: regen in master, for bug28925 merged to 0.4.0 and later 
python3 scripts/maint/practracker/practracker.py --regen
 2019-03-27 09:40:03 +10:00
Nick Mathewson
7502e5467b Regenerate practracker file from scratch. 
Closes ticket 29912.

Since this is the first time that practracker has had a stable
output order, this diff will be larger than usual.
 2019-03-26 19:28:35 -04:00
Nick Mathewson
300e7d8c99 Merge branch 'practracker_regen' 2019-03-26 19:27:54 -04:00
George Kadianakis
989b6325d6 Merge branch 'tor-github/pr/842' 2019-03-26 16:41:07 +02:00
Nick Mathewson
e028ec6bb7 Add new exceptions.txt entries 2019-03-26 09:43:46 -04:00
Nick Mathewson
39e4494344 practracker: update usage note in docstring 2019-03-26 08:42:14 -04:00
teor
ec736f8729
Merge remote-tracking branch 'tor-github/pr/833' 2019-03-26 13:02:37 +10:00
George Kadianakis
473decb246 Merge branch 'tor-github/pr/839' 2019-03-26 01:17:58 +02:00
Nick Mathewson
c2643842a9 practracker: add ability to regenerate exceptions file. 
Also add a useful argument parser.
 2019-03-25 16:09:11 -04:00
Nick Mathewson
0260e0f6fc practracker: pass sys.argv to main() as an argument 2019-03-25 16:09:11 -04:00
Nick Mathewson
301e3f22ef Practracker: add a string explaining the excptions file. 2019-03-25 16:09:11 -04:00
Nick Mathewson
c81b2b09ea Merge branch 'practracker_comments' 2019-03-25 15:18:36 -04:00
Nick Mathewson
61cebb2035 Minimize the includes in control.c 2019-03-25 14:14:56 -04:00
Nick Mathewson
2917ecaa97 Split command-handling and authentication from control.c 2019-03-25 14:06:56 -04:00
Nick Mathewson
4754e9058b Split getinfo handling into a new control_getinfo.c 2019-03-25 12:49:24 -04:00
Nick Mathewson
a49f506e05 Split all controller events code into a new control_events.c 
Also, split the formatting code shared by control.c and
control_events.c into controller_fmt.c.
 2019-03-25 12:11:59 -04:00
Nick Mathewson
135b51c9d3 practracker: allow comments in exceptions file 
Also, distinguish between empty lines (which we should ignore)
and incorrect lines (which we should warn about).
 2019-03-25 09:28:24 -04:00
Nick Mathewson
a20a2025a5 practracker: sort filenames and directories. 
This helps ensure that we'll get output in a stable order.

Closes ticket 29882.
 2019-03-25 09:08:04 -04:00
teor
d4d541c53c
Merge remote-tracking branch 'tor-github/pr/785' 2019-03-25 14:01:20 +10:00
Taylor Yu
307c156fbe Set file encoding in practracker.py 
Explicitly set the file encoding to UTF-8 in practracker.py, to avoid
problems in some CI environments.  Fixes bug 29789; bug not in any
released Tor.
 2019-03-22 17:51:55 -05:00
George Kadianakis
0aaeec19e7 Satisfy practracker broken by #29665 and #28656. 2019-03-15 17:13:06 +02:00
Nick Mathewson
2d76945052 practracker: Create an exception for an as-yet-nonexistent file. 
We need a better way to do this.
 2019-03-14 09:35:23 -04:00
George Kadianakis
a55c89c475 practracker: Be compatible with python2 which is used by travis/jenkins. 2019-03-14 02:15:32 +02:00
George Kadianakis
26b0d95397 Merge branch 'tor-github/pr/780' 2019-03-13 16:07:19 +02:00
Nick Mathewson
17ff69a268 Merge branch 'bug29221_more_squashed' 2019-03-13 09:30:02 -04:00
George Kadianakis
3e38efdf16 practracker: Fix duplicate exceptions and handle them more "gracefully". 2019-03-13 09:27:29 -04:00
Nick Mathewson
21c6e295ca grandfather in two more functions 2019-03-13 09:27:29 -04:00
Nick Mathewson
e2512950b6 Improve failure message from check-best-practices 2019-03-13 09:27:29 -04:00
Nick Mathewson
f0302d51ab practracker: Be more careful about excluding "confusing terms" 
Previously we excluded any line containing one of these terms from
consideration as the start or end of a function.  Now we're more
careful, and we only ignore these terms when they appear to be
starting a function definition.
 2019-03-13 09:27:29 -04:00
George Kadianakis
ec8c5b3fea practracker: Improvements based on last Nick's review. 2019-03-13 09:27:29 -04:00
George Kadianakis
157f7ba93e practracker: Update exceptions file :) 2019-03-13 09:27:29 -04:00
George Kadianakis
8c9835c6e5 practracker: Normalize filesystem paths across Windows and Posix. 
This was causing issues because the exceptions file is written using Posix
paths, whereas practracker in Windows was trying to match Windows paths ("\"
instead of "/").
 2019-03-13 09:27:29 -04:00
George Kadianakis
8bacc1dad1 practracker: Improve documentation in problem.py . 2019-03-13 09:27:29 -04:00
George Kadianakis
4795f2d3a0 Fold in an initial practracker exceptions file. 2019-03-13 09:27:29 -04:00
George Kadianakis
58de565988 Call practracker as part of check-local. 
- Introduce 'make check-best-practices'.
- Fix up Tor topdir etc to work with the way 'make check-local' gets called.
- Make practracker less likely to print useless stuff.
 2019-03-13 09:27:24 -04:00
Nick Mathewson
95209be861 Make checkSpace.pl check guard macros: 
- every .h file needs an #ifndef/#define pair.
  - They must refer to the same macro.
  - The guard macros that they refer to must be unique across all headers.
 2019-03-12 15:19:28 -04:00
Nick Mathewson
a6dd893e76 Fix shellcheck warnings in pull-all/merge-all scripts 
This appears at first glance to be a shellcheck bug.

Closes 29747.  Bugfix not in any released Tor.
 2019-03-12 10:50:54 -04:00
rl1987
0befdb8a35 Disable git diff pagination 2019-03-10 18:28:07 +02:00
rl1987
537692c1e3 Using diff -u in check_for_diffs 2019-03-10 18:28:07 +02:00
rl1987
5f253f6a47 Iterate over contents of scripts/git with check_for_script_update function 2019-03-10 18:28:06 +02:00
rl1987
888bb9508b Move all git maintenance scripts to separate directory 2019-03-10 18:28:06 +02:00
rl1987
7b5f31f2d6 Mention what file has changed 2019-03-10 18:28:06 +02:00
rl1987
88633fad5b Write a comment for post-merge.git-hook 2019-03-10 18:28:06 +02:00
rl1987
bb8b2f47d0 Also print changes in git helper scripts, if any 2019-03-10 18:28:06 +02:00
rl1987
2d3ef34dce Add post-merge git hook to warn about git hooks being updated in the repo 2019-03-10 18:28:06 +02:00
Nick Mathewson
8fd20df455 Merge remote-tracking branch 'tor-github/pr/743' 2019-03-08 10:07:25 -05:00
David Goulet
db2e916afe Merge branch 'tor-github/pr/717' 2019-03-05 14:26:59 -05:00
rl1987
7f0516022b Also disallow pushing to/from upstream branch when branch names do not match 2019-03-05 14:26:09 -05:00
rl1987
0deea98d02 Improve pre-push.git-hook description 2019-03-05 14:26:09 -05:00
rl1987
f3eac74ed9 In pre-push hook script, actually check local and remote refs 2019-03-05 14:26:09 -05:00
rl1987
ae5a0f39cd Update git pre-push hook so that only upstream branches can get pushed to origin 2019-03-05 14:26:09 -05:00
David Goulet
436e0c8ca4 maint: Make git scripts fetch origin once 
Replace the "git pull" by one single "git fetch origin" and then "git merge"
into each defined branches.

This speeds up the process considerably.

Closes #29616

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-03-01 09:24:46 -05:00
David Goulet
9256b02cc8 maint: Remove 0.3.3 branch from git scripts 
It is EOL today: Febuary 28th, 2019

No more releases will occur.

Closes #29616

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-02-28 13:16:41 -05:00
George Kadianakis
cec555f44c Merge branch 'tor-github/pr/720' 2019-02-28 12:32:16 +02:00
George Kadianakis
31c1d91ffb Exit with 1 if new issues were found. Also work with python3. 2019-02-27 19:31:41 +02:00
George Kadianakis
2a722e2841 Improve metrics implementation. 
No longer messes up with MOCK_IMPL. Also update the tests to show that. We are
still being innacurate on the line count in some cases, but that doesnt matter
so much.
 2019-02-27 19:31:15 +02:00
George Kadianakis
26c4f6cfd0 Add more intelligent problem tracking. 2019-02-27 18:34:13 +02:00
George Kadianakis
371ea65c08 Improve #include counting func and move it to metrics.py. 2019-02-27 17:05:00 +02:00
George Kadianakis
a7684fcb57 Add some failing unittests for the function length metric. 2019-02-27 15:22:24 +02:00
George Kadianakis
17dd316749 Initial commit of practracker.py . 2019-02-27 15:22:24 +02:00
David Goulet
e1ad22643e maint: Helpful scripts for tor maintenance 
Closes #29391

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-02-26 10:52:25 -05:00
Nick Mathewson
b3b737b875 Update pre-commit.git-hook for ticket 29553 
- handle older source layout
 - handle empty changes directories
 - "set -e" so that  we exit if there's a problem.
 2019-02-21 13:35:33 -05:00
Nick Mathewson
a036c0187a cov-diff: better handle the case where a file stops having coverage 2019-02-08 11:22:55 -05:00
Nick Mathewson
f4c76661d1 Add a script to check for coverage nondeterminism 
Closes ticket 29436.
 2019-02-08 11:22:24 -05:00
Nick Mathewson
f2b646a2b6 Merge branch 'ticket29435_035' into ticket29435_040 2019-02-08 10:47:04 -05:00
Nick Mathewson
192b312c97 Fix coverage script. 
It was looking for object files made with the old automake
directorations, but those changed when we split up our libraries.

Fixes bug 29435; bugfix on 0.3.5.1-alpha.
 2019-02-08 10:43:38 -05:00
Nick Mathewson
8b254c1c70 Merge branch 'precommit_squashed' 2019-01-24 11:08:10 -05:00
Nick Mathewson
dd22c84ccf Only run the checkIncludes.py script if it exists 
(It was added in 0.3.5)
 2019-01-24 08:06:09 -05:00
Nick Mathewson
107e50a90f lintChanges.py didn't take a directory argument in 0.2.9 2019-01-24 08:05:42 -05:00
rl1987
611e8dcda7 Also run checkInclude.py before every commit 2019-01-24 08:03:17 -05:00
rl1987
99fffc6c2f Add a pre-commit hook that runs code and changelog entry formatting checks 2019-01-24 08:03:17 -05:00
Nick Mathewson
00bc18b847 Merge remote-tracking branch 'tor-github/pr/653' 2019-01-22 08:13:14 -05:00
Nick Mathewson
aaef95ada5 format_changelog: treat "issue NNNN" as a track ticket 2019-01-18 10:15:02 -05:00
Nick Mathewson
7e1f893460 lintChanges.py: Two python re usage fixes 2019-01-17 08:46:06 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
9473a386c4 Update copyright script for 2019 2019-01-16 12:33:09 -05:00
Nick Mathewson
efe55b8898 Bump copyright date to 2019. 2019-01-16 12:32:32 -05:00
Nick Mathewson
2dd23086f1 Remove fallback scripts and whitelist 
They have been extracted to a new fallback-scripts.git repository.

Closes ticket 27914.
 2019-01-15 19:20:25 -05:00
Nick Mathewson
31e784915e Use [ -n "$x" ] in place of [ ! -z "$x" ] in chutney-git-bisect 
Closes 29099.
 2019-01-15 18:26:08 -05:00
rl1987
d47e7863bd Depending on script mode, either use 'whitelist' or 'fallback list' 2019-01-15 08:25:29 -05:00
rl1987
7fbe7a2563 In updateFallbackDirs.py, say 'fallback list' instead of 'whitelist' 2019-01-15 08:25:29 -05:00
Nick Mathewson
4afc6b172a Merge branch 'ticket28755_v2_squashed' 2018-12-12 11:07:38 -05:00
Nick Mathewson
5200df8557 Copy the nss-related changes into fuzzing_include_am.py. 2018-12-12 11:07:08 -05:00
Nick Mathewson
845e8dbe59 Fuzzing module for various string operations, currently focusing on 
encoding and decoding.

There are bunches of places where we don't want to invest in a full
fuzzer, but we would like to make sure that some string operation
can handle all its possible inputs.  This fuzzer uses the first byte
of its input to decide what to do with the rest of the input.  Right
now, all the possibilities are decoding a string, and seeing whether
it is decodeable.  If it is, we try to re-encode it and do the whole
thing again, to make sure we get the same result.

This turned up a lot of bugs in the key-value parser, and I think it
will help in other cases too.

Closes ticket 28808.
 2018-12-12 11:07:08 -05:00
Nick Mathewson
91be6a77e9 Merge remote-tracking branch 'teor/ticket24838' 2018-12-11 10:04:16 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
58d2ced572 Merge remote-tracking branch 'teor/24805-phoul' 2018-12-11 09:38:27 -05:00
Nick Mathewson
b65166750d Merge remote-tracking branch 'rl1987/ticket28007_take2' 2018-12-11 09:25:46 -05:00
rl1987
da264f7c76 Let's not double-quote OUTPUTARG after all 2018-12-08 15:17:22 +02:00
teor
6bc5c06dc2 Fallbacks: accept relays that are a fuzzy match to the whitelist 
If a relay matches at least one fingerprint, IPv4 address, or IPv6
address in the fallback whitelist, it can become a fallback. This
reduces the work required to keep the list up to date.

Closes ticket 28768.
 2018-12-07 16:30:24 +10:00
teor
75b5cc047d Fallbacks: refactor is_in_whitelist() to take an exact match boolean 
No behaviour change.

Preparation for 24838.
 2018-12-07 15:54:32 +10:00
teor
7f3a7d9a27 Fallbacks: Rename CONSENSUS_EXPIRY_TOLERANCE to REASONABLY_LIVE_TIME 
Cleanup after 28768.
 2018-12-07 15:54:32 +10:00
teor
c3fe405e21 Fallbacks: use a 24 hour future consensus tolerance 
Tor clients on 0.3.5.6-rc? and later will use a consensus that will become
valid up to 24 hours in the future.

Clients on 0.3.5.5-alpha? and earlier won't accept future consensuses.

Update the fallback expiry tolerance to match tor's checks.

Part of 28768, follow-up on 28591.
 2018-12-07 15:54:32 +10:00
teor
766fd6cf76 Fallbacks: use a 24 hour consensus expiry tolerance 
Tor clients will use a consensus that expired up to 24 hours ago.

Clients on 0.3.5.5-alpha? and earlier won't select guards from an expired
consensus, but they can still bootstrap if they have existing guards.

Update the fallback expiry tolerance to match tor's checks.

Part of 28768, follow-up on 24661.
 2018-12-07 15:54:32 +10:00
teor
d9f36d3e92 Fallbacks: minor script comment changes 2018-12-07 15:45:48 +10:00
Nick Mathewson
d4f558e385 Merge remote-tracking branch 'tor-github/pr/531' 2018-12-05 09:02:49 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
rl1987
9c90bddc42 Mention --no-verify in error message 2018-11-29 11:10:48 +02:00
rl1987
7c6dc28881 Improve comment at the top 2018-11-29 11:10:30 +02:00
rl1987
e2b418bab5 Rename to pre-push.git-hook 2018-11-29 10:56:56 +02:00
rl1987
59001a69c9 Address SC2086 in scan-build.sh 
We can safely silence SC2086 warning on $CHECKERS, as contents of that
is hardcoded into script, and we don't want to require Bash to use Bash
array here. Double-quote $OUTPUTARG, as it's value depends on environment
variable.
 2018-11-26 18:03:36 +02:00
rl1987
30f8b49d3b Silence SC2034 shellcheck checker for EXTRA_CHECKERS and NOISY_CHECKERS variables 2018-11-26 17:46:05 +02:00
Nick Mathewson
b16d6453ad Rewrite updateVersions script in Python, add datestamp functionality. 
This updateVersions.pl script was one of the only essential perl
scripts left in out maint system, and was the only one that used
autoconf to fill in the script.

This script adds a feature to define an APPROX_RELEASE_DATE macro
that is updated when the version changes.  We'll use this to
implement prop297, so that we have an accurate view of when a
release date happens.
 2018-11-24 20:44:37 -05:00
Colin Childs
6d2c2be291 Add CrackerboxPalace fallback 2018-11-21 20:36:08 -06:00
rl1987
ba2a9988cc Fix SC2230 warning 2018-11-20 16:57:57 +02:00
rl1987
27e982b470 Fix SC2046 and SC2086 warnings 2018-11-20 16:55:35 +02:00
rl1987
b2053cfc44 Also disallow fixup/squash commits in maint-* and release-* 2018-11-20 16:37:30 +02:00
rl1987
7dd515b0e0 Silence SC1117 warnings 2018-11-20 16:22:47 +02:00
rl1987
ffee0a6384 Add pre-push git hook to prevent fixup and squash commits from ending up in master 2018-11-20 15:40:52 +02:00
Nick Mathewson
a8df39e955 Merge remote-tracking branch 'tor-github/pr/519' 2018-11-19 08:47:03 -05:00
rl1987
d9f7cb3f62 Fix shellcheck issues in cov-diff - SC2231, SC2006 and SC2086 2018-11-15 19:37:07 +02:00
rl1987
4fc584f20e Fix one last SC2086 2018-11-15 19:35:56 +02:00
rl1987
7c04b00e65 Fix most instances of SC2086 warning 2018-11-15 19:35:56 +02:00
rl1987
ce1b018ab7 Fix shellcheck SC2006 warnings in 'coverage' script 2018-11-15 19:35:56 +02:00
Nick Mathewson
fae29f7b42 Make the topological-sort output off by default 2018-11-14 16:07:36 -05:00
Nick Mathewson
c3c8c926bf Update the check-includes script to enforce some naming and no-circularity rules 2018-11-14 16:07:36 -05:00
rl1987
5b48af9c4c Fix all instances of shellcheck warning SC2006 2018-11-04 19:34:46 +02:00
Nick Mathewson
f27279f857 Merge remote-tracking branch 'tor-github/pr/449' 2018-11-02 13:11:26 -04:00
Nick Mathewson
d671e3f513 Merge remote-tracking branch 'tor-github/pr/448' 2018-11-02 13:09:45 -04:00
rl1987
6aef0ce94e Fix shellcheck issue SC2217 in run_calltool.sh 2018-10-29 11:37:16 +02:00
rl1987
eab81b12e9 Fix shellcheck warning SC2086 in run_calltool.sh 2018-10-29 11:31:03 +02:00
rl1987
a0dd6bfdb0 run_trunnel.sh: Use 'find -exec' instead of a 'for' loop 
This fixes shellcheck warnings SC2044 and SC2006.
 2018-10-29 10:54:31 +02:00
rl1987
635312fc2a Silence shellcheck SC2086 in run_trunnel.sh 2018-10-29 10:45:14 +02:00
Colin Childs
17f4388b0f Remove mullbinde9 from fallback list 2018-10-15 13:47:31 -05:00
Colin Childs
9b946c13a2 Adding hviv104 to fallback list 2018-10-15 11:43:02 -05:00
Colin Childs
5033e950cc Adding Quake to fallback list 2018-10-12 15:25:17 -05:00
Colin Childs
57bba19bbe Remove aurora and chulak from fallback list 2018-10-12 15:21:32 -05:00
rl1987
8b5ad246e8 Fix issues that shellcheck found in chutney-git-bisect.sh 2018-10-11 20:28:11 +03:00
Colin Childs
bc68e80e0a Adding fallback relay 2018-09-20 13:48:45 -05:00
Colin Childs
1663a1dd63 Adjusting IP of fallback 2018-09-19 15:39:39 -05:00
teor
3eafa61f63 check-changes: Warn about bugfixes on future releases 
Warn when bugfix changes files say that the bug is in a future release.

Closes ticket 27761.
 2018-09-18 20:40:50 +10:00
teor
7e862c3ec0 check-changes: Check bugfix version formatting 
Check that bugfix versions in changes files look like Tor versions
from the versions spec.

Part of ticket 27761.
 2018-09-18 20:40:44 +10:00
Colin Childs
cb1891b412 Adding 1 new fallback 2018-09-12 13:39:49 -05:00
Nick Mathewson
b4b8fa4899 Set the file encoding in checkIncludes.py with Python3 2018-08-28 16:25:54 -04:00
Colin Childs
f0792537e9 Adding new fallback 2018-08-28 13:05:59 -05:00
teor
4217dc0558
Add scripts/test/chutney-git-bisect.sh, for bisecting using chutney 
Supports bisection on 0.3.4 and earlier.
Recommend that users copy the script before bisecting.

Implements ticket 27211.
 2018-08-24 12:46:04 +10:00
Colin Childs
51f8db1d92 Drop relays from fallback list that were terminated by provider 2018-08-16 14:32:23 -05:00
Colin Childs
9aceb92839 Adding 2 new fallback mirrors 2018-08-10 15:19:03 -05:00
Colin Childs
0260b72005 Adding relay from Edmond 2018-08-03 09:49:03 -05:00
Nick Mathewson
d66cfadbd3 Merge remote-tracking branch 'teor/bug26979' 2018-07-31 08:26:15 -04:00
Nick Mathewson
de4d7c3837 Merge branch 'ticket26447' 2018-07-31 08:21:10 -04:00
teor
c3fca338a2
Appveyor CI: always use HEAD for the short commit 
Part of 26979.
 2018-07-31 12:14:40 +10:00
teor
c6973aeccb
Appveyor CI: sort environmental variables 
To avoid future duplicates.
 2018-07-31 11:54:02 +10:00
teor
02a45b3ea4
Appveyor CI: fix some typos 2018-07-31 11:49:11 +10:00
teor
019c31bbf8
Appveyor CI: Generate correct tag names 
Part of 26979.
 2018-07-31 11:47:33 +10:00
teor
3d3e62d147
Appveyor CI: Switch to one URL per line 
Part of 26979.
 2018-07-31 11:37:11 +10:00
teor
9b41eb2bc3
Appveyor CI: Generate correct branches and URLs for pull requests 
Part of 26979.
 2018-07-31 11:13:20 +10:00
teor
861db8c4ae
Appveyor CI: Make short commits 10 hexdigits long 
That's what git does for tor.

Part of 26979.
 2018-07-31 11:13:17 +10:00
teor
fc9f8c28fb
Appveyor CI: Fix GitHub provider detection 
Part of 26979.
 2018-07-31 11:13:10 +10:00
Nick Mathewson
4833717d69 Merge branch 'maint-0.3.4' 2018-07-17 14:46:16 -04:00
rl1987
8645647250 Fix build to work with --disable-unittests again 2018-07-17 12:09:34 +00:00
Colin Childs
5c2431a343 Add mirrors from EmeraldOnion 2018-07-16 12:25:29 -05:00
Colin Childs
831abc0e10 Adding 3 more fallback mirrors 2018-07-16 12:25:29 -05:00
Colin Childs
226816df55 Fixing spacing issue in whitelist 2018-07-16 12:25:29 -05:00
Colin Childs
eca58aec76 Updating whitelist to match operator feedback 2018-07-16 12:25:29 -05:00
Colin Childs
7479f75b95 Update whitelist to match feedback from operators 2018-07-16 12:25:29 -05:00
Colin Childs
04de9443fe Update fallback lists to match #24805 2018-07-16 12:25:29 -05:00
Nick Mathewson
2d0e1cef20 Add fuzzing wrapper for fetch_from_buf_socks() 2018-07-12 14:20:46 -04:00
Nick Mathewson
413b73450c Merge remote-tracking branch 'imnotbad/bug26502' 2018-07-12 09:30:01 -04:00
Nick Mathewson
ee12286222 Add a little documentation to checkIncludes.py. 
Someday people might be glad I did.
 2018-07-09 16:02:34 -04:00
Nick Mathewson
294f580f04 Re-wrap the error message emitted by checkIncludes.py 2018-07-09 15:48:55 -04:00
Nick Mathewson
3145e469d8 Make checkIncludes work on python 2.7 2018-07-09 15:46:39 -04:00
Kaidan
85fa69297c Remove small piece of unneeded documentation 2018-07-08 07:05:31 +10:00
Kaidan
5f09708046 #26502: Stop using fallback blacklist 2018-07-08 06:54:14 +10:00
Nick Mathewson
f720a5a439 Fix everything that previously referred to src/or 2018-07-05 17:15:50 -04:00
Nick Mathewson
81cb0afb2b Start splitting src/or 
This is a very gentle commit that just lays the groundwork in the
build system: it puts the include files to build libtor-app.a into
src/core, and to build the tor executable into src/app.  The
executable is now "src/app/tor".
 2018-07-05 17:15:50 -04:00
Nick Mathewson
f7eff3008d Fix fuzzing_include_am.py 2018-07-05 15:22:17 -04:00
Nick Mathewson
83a4946e7b Prune the .may_include files a bit; detect unused lines in them 2018-07-01 18:14:28 -04:00
Nick Mathewson
c03958dbc1 Update the coverage script to look at new file locations 2018-07-01 10:18:48 -04:00
Nick Mathewson
3808e3015a Merge branch 'maint-0.3.3' into maint-0.3.4 2018-06-29 23:12:03 -04:00
Nick Mathewson
72ee0b6f79 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-06-29 23:12:03 -04:00
Nick Mathewson
3c97b148a4 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-06-29 23:12:03 -04:00
Nick Mathewson
2968c716d7 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-06-29 23:11:47 -04:00
Nick Mathewson
1953a7e61d Finally remove our EOL@EOF check. 
This check was added by mistake long ago.  It will be nice to see
these EOLs disappear from our code over time.

(backport from master)
 2018-06-29 23:11:02 -04:00
Nick Mathewson
860b9a9918 Remove some accidentally committed code in checkIncludes.py 2018-06-26 12:04:24 -04:00
Nick Mathewson
405fa42e8a Another windows include 2018-06-22 13:52:30 -04:00
Nick Mathewson
eb784aa9ea add license statement to checkincludes, and have it return an error code 2018-06-21 14:29:00 -04:00
Nick Mathewson
999f7984e1 New script to check includes for modularity violations 
Includes configuration files to enforce these rules on lib and
common.  Of course, "common" *is* a modularity violation right now,
so these rules aren't as strict as I would like them to be.
 2018-06-21 14:05:33 -04:00
Nick Mathewson
0dab29ce10 Run rectify_include_paths.py 2018-06-20 09:35:05 -04:00
Nick Mathewson
178b738be0 Script to replace include paths with full paths under src/ 
This will let us move around header files without having to fix up
all the include sites manually.
 2018-06-20 09:25:16 -04:00
Nick Mathewson
fb0019daf9 Update copyrights to 2018. 2018-06-20 08:13:28 -04:00
Nick Mathewson
67dc83fa75 New check-spaces rule: Our C files may not have duplicate names. 2018-06-18 16:17:11 -04:00
Nick Mathewson
6af248f27f Finally remove our EOL@EOF check. 
This check was added by mistake long ago.  It will be nice to see
these EOLs disappear from our code over time.
 2018-06-18 16:10:47 -04:00
Nick Mathewson
d7bbfd0f62 Fix various typos 2018-05-30 07:57:22 -07:00
Nick Mathewson
e053c71e3e Merge branch 'appveyor_min_029_v2' into appveyor_min_034_v2 2018-05-22 09:57:12 -04:00
Nick Mathewson
459ab3650c Python script for IRC notifications from appveyor 
Original by Christopher Wood, with modifications by nexB, Isis
Lovecruft, and Marcin Cieślak.
 2018-05-21 17:01:11 -04:00
Nick Mathewson
db94d7fed2 Update cov-diff to handle new gcov, and remove timestamps 
The new gcov sometimes emits *s if there is a line containing
multiple basic blocks, and some are not executed.  (The gcov
documentation says something weird about this point, so I'm trying
to interpret it into the compilerese that I'm familiar with.)
That's bug 26101.

Also, when we're looking for unique variations in our coverage, we
would like cov-diff to suppress timestamps on the diffs.  That's bug
26102.

Both of these are bugfixes on 0.2.5.1-alpha when the cov-diff script
was introduced.

My apologies for the perl.
 2018-05-14 18:04:10 -04:00
Nick Mathewson
c9c26d0c31 Remove a comment about 0.2.8 that will soon be irrelevant. 
(It becomes irrelevant on 15 May 2018 when 0.2.5 is EOL.)
 2018-04-18 10:44:25 -04:00
Isis Lovecruft
9799394375 maint: Add script for vendoring/updating Rust dependencies. 
* ADD documentation for usage both inside the script and in
   doc/HACKING/CodingStandardsRust.md
 * FIXES part of #25310: https://bugs.torproject.org/25310
 2018-03-21 17:04:28 -04:00
Deepesh Pathak
ca6682f3f8 Fix spelling mistakes corresponding to ticket #23650 2018-02-07 10:41:57 -05:00
Nick Mathewson
455cc12b51 Update scan-build.sh to work with the latest clangalyzer 2018-01-16 08:56:07 -05:00
teor
65520f1bde
Stop logging excessive information about fallback netblocks 
Implements ticket 24791
 2018-01-05 13:28:17 +11:00
teor
d30d3fe8ed
Adjust fallback selection parameters 
Avoid selecting fallbacks that change their IP addresses too often.

Select more fallbacks by ignoring the Guard flag, and allowing lower
cutoffs for the Running and V2Dir flags. Also allow a lower bandwidth,
and a higher number of fallbacks per operator (5% of the list).

Implements ticket 24785.
 2018-01-05 13:28:13 +11:00
teor
befcc56d84
Update dependency list for the fallback script 
Comment-only change.

Implements #24711.
 2018-01-05 13:28:09 +11:00
teor
91a8b37a2e
Make updateFallbackDirs.py search harder for python 
(Some OSs don't put it in /usr/bin.)

Fixes bug 24708; bugfix on 0.2.8.1-alpha.
 2018-01-05 13:28:05 +11:00
teor
2ae33e486e
Revise the generateFallbackDirLine.py script to use descriptors 
Patch by atagar, part of #24706.
 2018-01-05 13:28:01 +11:00
teor
e244738bbd
Add fallback directory mirror helper scripts 
Add the generateFallbackDirLine.py script for automatically generating
fallback directory mirror lines from relay fingerprints. No more typos!

Add the lookupFallbackDirContact.py script for automatically looking up
operator contact info from relay fingerprints.

Implements ticket 24706.
 2018-01-05 13:27:55 +11:00
teor
9864fcd4e3
Add a generated timestamp to the fallback directory header 
This can act as a revision counter, as requested by atagar in 24742.

Part of 24725.
 2018-01-05 13:27:47 +11:00
teor
d8ab1b803a
Remove an incorrect C comment from updateFallbackDirs.py's list generation code 
Part of #24725.
 2018-01-05 13:27:39 +11:00
teor
b6033f5734
Add a delimiter to the end of each fallback entry 
This helps external parsers.

Also, add comments indicating where to add new fields in the fallback
format.

Part of 24725.
 2018-01-05 13:27:32 +11:00
teor
beedf5fd81
Remove commas and equals signs from external string inputs to the fallback list 
This makes sure that list parsers only see one comma per fallback entry,
and only see one equals sign per field.

Implements ticket 24726.
 2018-01-05 13:27:22 +11:00
teor
6a27fc6245
Add an extrainfo cache flag for each fallback in a C comment 
This allows stem to use fallbacks to fetch extra-info documents,
rather than using authorities.

Implements ticket 22759.
 2018-01-05 13:27:14 +11:00
teor
561f18e724
Add a nickname to each fallback in a C comment 
This makes it easier for operators to find their relays, and allows stem to
use nicknames to identify fallbacks.

Implements ticket 24600.
 2018-01-05 13:27:07 +11:00
teor
ac5058ac8a
Remove the "weight=10" line from fallback directory mirror entries 
This removes some redundant repeated lines.

Ticket 24681 will maintain the current fallback weights by changing
Tor's default fallback weight to 10.

Implements ticket 24679.
 2018-01-05 13:26:53 +11:00
teor
8d226a2c7b
Add a type and version header to the fallback directory mirror file 
This helps external parsers like stem and Relay Search.

Implements ticket 24725.
 2018-01-05 13:26:44 +11:00
teor
5045dde956
Stop refusing fallbacks on the same machine 
We only occasionally checked for fallbacks on the same machine.
And I'm not convinced it makes much of a difference with ~150 fallbacks.

Part of #22321.
 2018-01-05 13:26:34 +11:00
teor
23dddaf49b
Forgive fallback operators that were blacklisted due to a details change 
(But have since run a stable relay.)

Part of #22321.
 2018-01-05 13:26:30 +11:00
teor
a9c0be62a9
Some fallbacks changed their details: assume the changes are permanent 
We have already updated the details for operators who replied to:
https://lists.torproject.org/pipermail/tor-relays/2017-December/013988.html

Closes #24678.
 2018-01-05 13:26:26 +11:00
teor
f2a9019569
Update fallback directory mirror details based on opt-ins and opt-outs 
https://lists.torproject.org/pipermail/tor-relays/2017-December/013898.html

Part of 22321.
 2018-01-05 13:26:22 +11:00
Matt Traudt
90fd4566ad
Update fallback whitelist and blacklist based on 2017 changes 
This covers #22321 comments 3-14, including some child tickets.

Part of #22321. Patch by pastly.
 2018-01-05 13:26:18 +11:00
teor
9a0cf2376d
Update comment and add changes file for 20942 2018-01-05 13:26:14 +11:00
Dominique Ingoglia
23979d7682
Set CONSENSUS_EXPIRY_TOLERANCE to 0 2018-01-05 13:26:10 +11:00
Nick Mathewson
8e5450b752 Remove merge-conflict detritus 2017-12-20 10:15:00 -05:00
teor
1a29ad611c
Make fallback script usage instructions create a log file 
Comment-only change.

Closes 24682.
 2017-12-20 12:36:31 +11:00
Nick Mathewson
f755f9b9e6 Merge branch 'maint-0.3.2' 2017-11-05 14:41:53 -05:00
Nick Mathewson
f46cdb3ff8 lintchanges: Allow 'fixes bugs a, b, and c' 2017-11-05 14:41:39 -05:00
Roger Dingledine
b601eeda0e all the other lines here had a tab; make this one blend in 2017-10-31 23:07:47 -04:00
David Goulet
4d2de821ce Revert "Temporarily disable compilation of the v3 hs fuzzing code" 
This reverts commit 5ef656e7d1.
 2017-10-26 09:41:47 -04:00
Nick Mathewson
9298749088 Add dirauth changes as a lintchanges category 2017-10-17 13:58:21 -04:00
Taylor Yu
e31ffb2741 Don't move gcov output to root directory 
In scripts/test/coverage, avoid attempting to move gcov output files
to the root directory when given no arguments.  Fixes bug 23741.
 2017-10-02 16:09:28 -05:00
Nick Mathewson
9727a9248a Improve cov-diff script to account for new files. 2017-09-28 08:53:16 -04:00
Nick Mathewson
3c04c8317f Make check-spaces part of the standard "make check" process 
To do this, we had to make sure it passes when the changes directory
is empty.  I also tried to improve the quality of the output, and
fix some false-positive cases.  Let's see how this goes!

Closes ticket 23564.
 2017-09-21 16:25:24 -04:00
Nick Mathewson
1f602e8643 plural in manpage 2017-09-19 08:48:39 -04:00
Nick Mathewson
2b5bc5613c Merge remote-tracking branch 'ewongbb/chkdblcolon' 2017-09-18 15:04:14 -04:00
Edmund Wong
f0e87ff9eb Add check for double-semi-colons at the end of a line 2017-09-18 15:33:24 +08:00
Nick Mathewson
76c33f7ff4 Merge branch 'scan-build-032' 2017-09-15 16:40:11 -04:00
Nick Mathewson
40ce9819dd update script to use /* */ comments 2017-09-15 16:11:48 -04:00
Nick Mathewson
5d06bbc66f Merge branch 'annotate_ifdefs' 2017-09-15 16:07:37 -04:00
Nick Mathewson
a28e239b17 Teach the scan-build script to output to a chosen directory 2017-09-12 21:42:36 -04:00
Nick Mathewson
f9f3014ce6 Add more checkers to scan-build. 2017-09-12 21:32:42 -04:00
Nick Mathewson
4097d646d8 Use all+only the default checkers in scan-build script 2017-09-12 19:03:04 -04:00
David Goulet
87585ebd2d test: Add an HS v3 reachability unit test 
This is a large and important unit test for the hidden service version
3! It tests the service reachability for a client using different
consensus timings and makes sure that the computed hashring is the same
on both side so it is actually reachable.

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2017-09-08 19:07:00 +03:00
Nick Mathewson
f117da3ea0 Whoops -- I messed a calltool target 2017-09-07 11:03:22 -04:00
Nick Mathewson
283be00b0b Add support for running "calltool" on Tor via "make callgraph". 
Closes #19307.
 2017-09-07 10:51:02 -04:00
Nick Mathewson
f8c14a987c Add a python script to annotate our #elses and #endifs 2017-09-06 11:30:37 -04:00
Nick Mathewson
73b0e2e6fd Merge branch 'http_tunnel_squashed' 2017-09-05 14:34:29 -04:00
Nick Mathewson
ad9190404b Add a fuzzer for HTTP CONNECT 2017-09-05 14:34:09 -04:00
Nick Mathewson
5ef656e7d1 Temporarily disable compilation of the v3 hs fuzzing code 
Turns out, it wasn't up-to-date with the latest v3 hs API :(
 2017-08-28 14:12:56 -04:00
Nick Mathewson
18bc7fa70b Merge remote-tracking branch 'haxxpop/fuzzing-hsv3' 2017-08-28 13:46:24 -04:00
Alexander Færøy
9e1fa95920 Add Coccinelle patch for replacing NULL/non-NULL tt_assert(). 
This patch replaces tt_assert() checks for NULL/non-NULL values with
tt_ptr_op().
 2017-08-24 15:22:20 -04:00
Alexander Færøy
7666cd8881 Add Coccinelle patch for replacing tt_assert() usage on integer types. 
This patch replaces tt_assert() comparison of integers and unsigned
integers with their respective tt_int_op or tt_uint_op counterpart.
 2017-08-24 15:21:43 -04:00
Alexander Færøy
3fd68b249b Add Coccinelle patch for replacing tt_assert(0) with tt_abort(). 2017-08-24 15:21:07 -04:00
Alexander Færøy
25eaf77033 Add Coccinelle patch for detecing places where CEIL_DIV should be used. 2017-08-24 15:20:11 -04:00
Alexander Færøy
7c3e980fb9 Add script for cleaning op operator usage in test files. 
This patch adds a script written by Nick for bug #13172 to clean up the
usage of ==, !=, <, >, <=, and >= by replacing them with their symbolic
OP_* counterpart. This will ensure that a tool like Coccinelle doesn't
get confused and silently ignore large blocks of code.
 2017-08-24 15:19:11 -04:00
Ties Stuij
2e99f839e9 22839: Build tor with rust enabled on win 
- make tor_util static library name configurable
- fix Rust libary dependency order for Windows
 2017-08-21 15:08:24 -04:00
Suphanat Chunhapanya
97347b1109 Fuzz outer layer of hsv3 descriptor 
The code in fuzz_hsdescv3.c fuzzes the unencrypted layer of the hsv3
descriptor. We need to fuzz the encrypted layer later.
 2017-08-13 17:50:51 +07:00
Nick Mathewson
d011126467 Update fuzzing_include_am to include updates from include.am 2017-07-03 13:24:57 -04:00
Nick Mathewson
80ad374b84 Remove old callgraph scripts; recommend calltool instead. 2017-06-14 17:44:15 -04:00
teor
09cd788869
Update fallback whitelist and blacklist based on operator emails 
Part of 21283.
 2017-05-16 18:12:47 +10:00
teor
cd687a0a2c
Update fallback stability requirements 
Decrease the minimum number of fallbacks to 100.

Final fix to 20913.
 2017-05-16 18:12:10 +10:00
teor
0e81e2114a
Merge branch 'fallbacks-201702-v3' into fallbacks-2017-05 2017-05-16 18:03:00 +10:00
teor
3f28824931
Make the usage example in updateFallbackDirs.py actually work 
And explain what it does.

Fixes bug 22270 in 0.3.0.3-alpha.
 2017-05-16 17:35:47 +10:00
Nick Mathewson
43db91bd87 Teach cov-exclude to detect runaway LCOV_EXCL_START lines 
Also, fix two instances of runaway LCOV_EXCL_START lines.
 2017-04-25 10:59:19 -04:00
Nick Mathewson
6657fe1e54 Generate src/test/fuzz/include.am from a script 
It was very error-prone to maintain this by hand.
 2017-03-16 17:14:10 -04:00
Nick Mathewson
7505f452c8 Run the copyright update script. 2017-03-15 16:13:17 -04:00
Nick Mathewson
c6b97b3da0 Update updateCopyright to 2017 2017-03-15 16:11:50 -04:00
Nick Mathewson
1453bda67b Update changelog sorting rules to know about dirauth changes 2017-02-28 10:12:17 -05:00
teor
28d523fcd5
Move a relay to the blacklist because its details changed 
(And its contact email address no longer works.)
 2017-02-27 13:13:32 +11:00
teor
1f26b6db57
Update the fallback whitelist and blacklist based on operator emails 2017-02-27 13:13:28 +11:00
teor
71b79d6bf3
Modify fallback stability requirements 
Increase the fallback stability requirement to 30 days.
When this was at 7 days, we chose far too many unstable fallbacks.

Decrease the guard flag requirement to 0.8.
When this was at 0.9, we lost too many fallbacks due to version upgrades.
(The running and v2dir flags ensure DirPorts are available to clients.)

Partial fixes to #20913.
 2017-02-27 13:13:17 +11:00
Nick Mathewson
67eb6470d7 Merge branches 'server_ciphers' and 'ciphers.inc' 2017-01-27 16:45:18 -05:00
Nick Mathewson
26a9787aa5 MOre gen_server_ciphers improvements: 
- Remove DES from mandatory list.
  - always put the closing ; on its own line.
 2017-01-27 16:15:26 -05:00
Suphanat Chunhapanya
1e26d6dbe8 Add the output file to the usage 2017-01-25 13:11:57 -05:00
Suphanat Chunhapanya
e6eaf14350 Add usage for check_existing 2017-01-25 13:11:57 -05:00
Suphanat Chunhapanya
d97792d596 Logging existing fallbacks at warning level 2017-01-25 13:11:57 -05:00
Suphanat Chunhapanya
3fd58aa98d Automate checking existing fallbacks 2017-01-25 13:11:56 -05:00
Nick Mathewson
8a9eca1267 Teach gen_server_ciphers about CCM and Chacha. 
Also, teach it to not use 3DES any more.
 2017-01-24 15:29:26 -05:00
Nick Mathewson
3db28eaea2 Update ciphers.inc script to understand openssl-1.1 2017-01-24 15:02:36 -05:00
cypherpunks
99cbadf143 Warn on Tor versions with the 'tor-' prefix 
Closes ticket 21096.
 2017-01-11 09:16:20 -05:00
cypherpunks
fa0792c3a8 Fix coding style issues in the changes file linter 
Found using flake8 which gave the following output;

lintChanges.py:10:13: E225 missing whitespace around operator
lintChanges.py:25:1: E302 expected 2 blank lines, found 1
lintChanges.py:46:5: F841 local variable 'lines' is assigned to but never used
lintChanges.py:52:32: E228 missing whitespace around modulo operator
lintChanges.py:53:11: E201 whitespace after '('
lintChanges.py:56:41: E228 missing whitespace around modulo operator

Part of ticket 21096.
 2017-01-11 09:16:19 -05:00
teor
a0bbd72caa
Add and update fallback directory details 
These updates were made after the December 2016 list was created.
 2016-12-21 08:15:21 +11:00
teor
e9b7308ea5
Blacklist a fallback that was removed by the operator 2016-12-19 18:15:06 +11:00
teor
fcf19f8b54
Update fallback whitelist and blacklist 
Based on:
* fallback directory mirror checks,
* operator emails and tickets, both before and after the Dec 2016 emails:
https://lists.torproject.org/pipermail/tor-relays/2016-December/011113.html
https://lists.torproject.org/pipermail/tor-relays/2016-December/011330.html
 2016-12-19 15:06:46 +11:00
teor
53ec087450
Avoid an error in the fallback script when a fallback doesn't have any uptime 
Sometimes, the fallback generation script doesn't add attributes to the
fallbacks in the list. If this happens, log an error, and avoid selecting
that fallback.

This is a rare issue: it should not change selection behaviour.

Fixes issue #20945.
 2016-12-19 15:06:04 +11:00
teor
654367f026
Allow fallbacks serving consensuses that expired less than 24 hours ago 
This works around #20909, where relays serve stale consensuses for a short
time, and then recover.

Update to the fix for #20539.
 2016-12-19 15:06:00 +11:00
teor
2d2bbaf259
Avoid checking fallback candidates' DirPorts if they are down in OnionOO 
Exclude relays that have been down for 1 or more days from the fallback
candidate list.

When a relay operator has multiple relays, this prioritises relays that are
up over relays that are down.

Fixes issue #20926.
 2016-12-19 15:05:56 +11:00
teor
4c832bcfac
Be more tolerant of a decreased number of fallbacks 2016-12-19 15:05:53 +11:00
teor
124c342364
Reduce fallback bandwidth requirement to 1 MByte/s 2016-12-19 15:05:49 +11:00
teor
35da99a712
Allow 3 fallbacks per operator 
This is safe now we are choosing 200 fallbacks.

Closes ticket 20912.
 2016-12-19 15:05:45 +11:00
teor
ee3e8fc3e9
Require fallbacks to have 90% Running, V2Dir, and Guard flags 
This allows 73% of clients to bootstrap in the first 5 seconds without
contacting an authority.

Part of #18828.
 2016-12-19 15:05:40 +11:00
teor
396bddaa4c
Require fallback directories to have the same address and port for 7 days 
7 days is a tradeoff between the expected time between major Tor releases,
which is 6 months, and the number of relays with enough stability.

Relays whose OnionOO stability timer is reset on restart by bug #18050
should upgrade to Tor 0.2.8.7 or later, which has a fix for this issue.

Closes ticket #20880; maintains short-term fix in e220214 in tor-0.2.8.2-alpha.
 2016-12-19 15:05:36 +11:00
teor
9629a25d10
Display the fingerprint when downloading consensuses from fallbacks 2016-12-19 15:05:33 +11:00
teor
ced50aff7e
Update fallback whitelist based on relay descriptors 
These updates assume that the changes were intentional and permanent.
The operators were emailed to confirm.
 2016-12-19 15:05:29 +11:00
teor
8381d928cf
Exclude relays that deliver an expired consensus from the fallback list 
Part of #20539, based on #20501.
 2016-12-19 15:05:21 +11:00
teor
243d6fa0c7
Exclude relay versions affected by #20499 from the fallback list 
Part of #20539, based on #20509.
 2016-12-19 15:05:18 +11:00
teor (Tim Wilson-Brown)
49df83cc03
Update fallback whitelist and blacklist 
Update fallback whitelist and blacklist based on:
* pre-0.2.9 checks
* operator opt-ins and opt-outs, via emails and tickets
 2016-12-19 15:05:10 +11:00
teor
864a8eb283
Make fallback sort order configurable 
Closes issue #20882.
 2016-12-19 15:05:06 +11:00
teor
31e1439642
Select 200 fallback directories by default for each release 
Closes ticket #20881.
 2016-12-19 15:05:02 +11:00
teor
4eba30ca59
Provide bandwidth and consensus weight for each candidate fallback 
And make it clear that updateFallbackDirs.py outputs bandwidth in
megabytes per second.

Closes #20878.
 2016-12-19 15:04:58 +11:00
teor
6ed8e3764b
Add a missed return to fallbackdir_comment() in updateFallbackDirs.py 
Closes ticket #20877.
 2016-12-19 15:04:54 +11:00
cypherpunks
e8760b6e51 Use the new message function 
Substitutions were made using `sed -e 's/print/msg/g'`.
 2016-12-16 10:52:14 -05:00
cypherpunks
4b2516313e Add a function to keep track of found errors 2016-12-16 10:52:14 -05:00
cypherpunks
44db6461b5 Restrict unsafe constructs and enable all warnings 2016-12-16 10:52:14 -05:00
cypherpunks
f43e56a6d0 Declare all variables to be local 
Without the 'my' keyword the variables are global and Perl requires
global symbols to include an explicit package name.
 2016-12-16 10:52:14 -05:00
cypherpunks
67e1eff321 Convert tabs to spaces 2016-12-16 10:52:14 -05:00
Nick Mathewson
bc68eedd79 Update to trunnel 1.5.1 2016-12-08 16:59:25 -05:00
David Goulet
0ba3444b4a Make trunnel find files recursively 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2016-10-31 16:03:28 -04:00
Nick Mathewson
b64c224362 fix crash in lintChanges.py 2016-09-22 08:33:09 -04:00
Nick Mathewson
5927ed8d33 checkSpace.pl now forbids more identifiers. 
The functions it warns about are:
  assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
  strdup, strndup, calloc.

Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
 2016-09-06 12:35:37 -04:00
Nick Mathewson
5aa6a19de5 Update lintChanges script to understand 'Minor features (debugging)' 
Previously we would give a warning because we didn't say "Fixes bug
Foo" on these.
 2016-08-02 11:40:08 -04:00
Nick Mathewson
f3575a45ce Merge branch 'maint-0.2.8' 2016-07-29 10:23:38 -04:00
teor (Tim Wilson-Brown)
1c4a2dd208 Remove a fallback that was on the hardcoded list, then opted-out 
The fallback was added in 0.2.8.2-alpha.
 2016-07-29 10:23:15 -04:00
Sebastian Hahn
265e40b481 Raise libevent dependency to 2.0.10-stable or newer 
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
 2016-07-04 12:40:09 +02:00
Nick Mathewson
f42dbc4e26 Merge branch 'maint-0.2.8' 2016-06-28 19:15:43 -04:00
teor (Tim Wilson-Brown)
26146dbe9e Comment-out fallbacks in a way the stem fallback parser understands 
If we manually remove fallbacks in C by adding '/*' and '*/' on separate
lines, stem still parses them as being present, because it only looks at
the start of a line.

Add a comment to this effect in the generated source code.
 2016-06-28 19:15:08 -04:00
teor (Tim Wilson-Brown)
dae442a382
Whitelist a fallback's new IPv6 address 
The IPv4-only entry will be used for 0.2.8.
The IPv4 and IPv6 entry will be considered in 0.2.9.
 2016-06-23 10:38:52 +10:00
teor (Tim Wilson-Brown)
828e2e1a2e
Remove a fallback that changed DirPort 
The operator has confirmed that the DirPort change is perament.
The relay will be reconsidered as a fallback in 0.2.9.
 2016-06-23 10:38:03 +10:00
Nick Mathewson
61dac57aa3 Merge branch 'maint-0.2.8' 2016-06-22 08:20:11 -04:00
teor (Tim Wilson-Brown)
69635e41c8
Remove and blacklist 3 fallbacks which disappeared 
Blacklist them in case they appear again.
 2016-06-22 12:18:10 +10:00
teor (Tim Wilson-Brown)
7e9532b9be
Remove and blacklist 4 fallbacks which are unsuitable 
Remove a fallback that changed its fingerprint after it was listed
This happened after to a software update:
https://lists.torproject.org/pipermail/tor-relays/2016-June/009473.html

Remove a fallback that changed IPv4 address

Remove two fallbacks that were slow to deliver consensuses,
we can't guarantee they'll be fast in future.

Blacklist all these fallbacks until operators confirm they're stable.
 2016-06-22 12:16:57 +10:00
teor (Tim Wilson-Brown)
0a79e92914
Update the fallback whitelist and blacklist 
Operators have sent emails asking to have their relays added or
removed from the fallback list. Since none of the blacklisted
relays are in the hard-coded falback list, it does not need to
be changed.
 2016-06-22 11:56:40 +10:00
teor (Tim Wilson-Brown)
4090612246
Avoid errors in updateFallbackDirs.py when there are no fallbacks 2016-06-22 11:56:31 +10:00
teor (Tim Wilson-Brown)
6ce53668f4
Document how to test the hard-coded fallback list 2016-06-22 11:56:27 +10:00
Nick Mathewson
87593702eb roger says this url is better 2016-05-27 15:11:11 -04:00
Nick Mathewson
3934e78bb9 Make format_changelog.py add links to bugs 2016-05-27 09:26:49 -04:00
Nick Mathewson
5dbb0bdae9 Merge branch 'maint-0.2.8' 2016-05-16 08:33:53 -04:00
teor (Tim Wilson-Brown)
ab06b79cba Simplify string cleansing in fallback update script 
No behavioural change
 2016-04-29 11:59:46 +10:00
teor (Tim Wilson-Brown)
d41f92b006 Improve logging in fallback update script 
Log a notice just before the script is about to perform a
potentially time-consuming operation

Clarify the warning when py2-ipaddress isn't found

Make log levels more consistent

No behavioural change (just logging)
 2016-04-29 11:59:46 +10:00
teor (Tim Wilson-Brown)
46d813922a Improve comments in fallback update script 2016-04-29 11:43:55 +10:00
Nick Mathewson
fb9c9e04f0 Merge branch 'maint-0.2.8' 2016-04-26 19:27:39 -04:00
teor (Tim Wilson-Brown)
de08f2d96b Whitelist additional fallbacks after creating April 2016 list 2016-04-26 19:26:22 -04:00
teor (Tim Wilson-Brown)
ba7691071e Report fallback directory detail changes when rebuilding list 
As well as the existing reports of IPv6 address additions or removals,
the script now warns when keys change but IPv4:ORPort or
IPv6:IPv6ORPort remain the same.

Existing checks for other whitelist detail changes have also
been re-worded and upgraded to warnings.

This makes it easier for changes to be identified so operators can
be contacted to confirm whether the change is stable.
 2016-04-26 19:26:22 -04:00
teor (Tim Wilson-Brown)
8b90d45f78 Add fallbacks to white/blacklist from operator responses 
Also add misbehaving relays to updateFallbackDirs.py blacklist,
but leave them commented out in case it's a transient issue,
or it's been resolved by the download check fixes. (These
relays hang stem's downloader. It's unlikely they'll ever help
clients.)
 2016-04-26 19:26:21 -04:00
teor (Tim Wilson-Brown)
c157a31ee8 Limit fallbacks from the same operator 
Use IP address, effective family, and contact info to
discover and limit fallbacks to one per operator.

Also analyse netblock, ports, IP version, and Exit flag,
and print the results. Don't exclude any fallbacks from
the list because of netblocks, ports, IP version, or
Exit flag.
 2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
999834324b Simplify fallback list output 
When creating the list of fallbacks for a release:
* don't output fallback name and contact
* sort fallbacks by fingerprint
 2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
78ec782f76 Select fallbacks by bandwidth rather than consensus weight 
But as advertised bandwidth is controlled by relays,
use consensus weight and median weight to bandwidth ratio
to approximate measured bandwidth.

Includes minor comment changes and parameter reordering.
 2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
e115523c98 Only count active fallbacks in updateFallbackDirs.py 
Previously, we would cut the list down to 100 fallbacks,
then check if they could serve a consensus, and comment
them out if they couldn't.
This would leave us with fewer than 100 active fallbacks.

Now, we stop when there are 100 active fallbacks.

Also count fallbacks with identical contact info.
Also fix minor logging issues.
 2016-04-15 12:26:37 +10:00
teor (Tim Wilson-Brown)
205a641e1d Remove fallback directory weights 
Give each fallback a set weight of 10.0 for client selection.

Fallbacks must have at least 3000 consensus weight.
This is (nominally) 100 times the expected extra load of
20 kilobytes per second (50 GB per month).

Fixes issue #17905.
 2016-04-15 12:21:32 +10:00
teor (Tim Wilson-Brown)
7e1b8ae79c Improve fallback selection and output 
Improve the download test:
* Allow IPv4 DirPort checks to be turned off.
* Add a timeout to stem's consensus download.
* Actually check for download errors, rather than ignoring them.
* Simplify the timeout and download error checking logic.

Tweak whitelist/blacklist checks to be more robust.
Improve logging, make it warn by default.

Cleanse fallback comments more thoroughly:
* non-printables (yes, ContactInfo can have these)
* // comments (don't rely on newlines to prevent // */ escapes)
 2016-04-15 12:21:32 +10:00
Nick Mathewson
4043f2c95f Adopt the LCOV convention for marking lines as unreachable by tests. 
Document this convention.

Add a script to post-process .gcov files in order to stop nagging us
about excluded lines.

Teach cov-diff to handle these post-processed files.

Closes ticket 16792
 2016-04-12 21:12:10 -04:00
Nick Mathewson
560450f2fb helper script to highlight undocumented members 2016-03-26 10:31:59 -04:00
Nick Mathewson
52bc921402 Fix (most) lintChanges warnings on master. 2016-03-21 11:14:57 -04:00
Nick Mathewson
69fc025e95 Merge remote-tracking branch 'teor/fallbacks-201602-v2' 2016-02-28 15:51:22 +01:00
Nick Mathewson
57699de005 Update the copyright year. 2016-02-27 18:48:19 +01:00
teor (Tim Wilson-Brown)
e2202146d1 Update default fallback directories for 0.2.8.2-alpha (Feb 2016) 
Allow fallback directories which have been stable for 7 days
to work around #18050, which causes relays to submit descriptors
with 0 DirPorts when restarted. (Particularly during Tor version
upgrades.)

Ignore low fallback directory count in alpha builds.
Set the target count to 50.
 2016-02-27 10:04:00 +01:00
teor (Tim Wilson-Brown)
a6017d61e1 Fix minor fallback directory script issues with time zones 2016-02-27 10:03:59 +01:00
teor (Tim Wilson-Brown)
9580036986 Check if fallback directories can serve the consensus fast enough 2016-02-27 10:03:59 +01:00
Nick Mathewson
2123f0a2f9 Make the callgraph module-aware 2016-02-10 15:09:10 -05:00
Nick Mathewson
f09d03c384 use sortChanges to fold changes files into changelog. No additional editing or formatting yet. 2016-02-01 16:56:20 -05:00
Nick Mathewson
deea08c13f Fix all the warnings from lintChanges.py 2016-02-01 16:46:29 -05:00
teor (Tim Wilson-Brown)
ab3c86479a Add default fallback directories for the 0.2.8 alpha releases 
Allow fallback directories which have been stable for 30 days
to work around #18050, which causes relays to submit descriptors
with 0 DirPorts when restarted. (Particularly during Tor version
upgrades.)

Ignore low fallback directory count in alpha builds.
 2016-01-18 20:15:59 -05:00
teor (Tim Wilson-Brown)
efd2c9d05b Fix a display issue in updateFallbackDirs.py output 2016-01-18 12:36:49 +11:00
Nick Mathewson
d062baac86 Merge remote-tracking branch 'teor/bug17888' 2016-01-12 12:26:06 -05:00
Nick Mathewson
7616922e81 Merge remote-tracking branch 'teor/bug17887-v2' 2016-01-12 10:47:54 -05:00
teor (Tim Wilson-Brown)
1c139f60d2 Don't limit fallback directory candidates to 500 
Only impose the limit when selecting fallback directories for
inclusion in tor.
 2016-01-12 10:10:05 -05:00
teor (Tim Wilson-Brown)
da968e558e Fix various issues with fallback directory data handling 
* support maximum history age in _avg_generic_history()
* fix division-by-zero trap in _avg_generic_history()
* skip missing (i.e. null/None) intervals in _avg_generic_history()
* Python timedelta.total_seconds() function not available in 2.6;
  replace with equivalent expression
* set DEBUG logging level to make relay exclusion reasons visible
* move CUTOFF_GUARD test to end in order to expose more exclusion
  reasons

Patch by "starlight", merge modifications by "teor".
 2016-01-12 10:10:05 -05:00
teor (Tim Wilson-Brown)
bc2bed8979 Don't reduce the weight of exits selected as fallback directories 
When selecting exits as fallback directories, don't reduce
their weights.

Closes ticket #17888.
 2016-01-12 13:56:45 +11:00
teor (Tim Wilson-Brown)
430181fad2 Allow fallback directory selection to use day-old data 
Allow cached or outdated Onionoo data to be used to choose
fallback directories, as long as it's less than a day old.

Modify last modified date checks in preparation for Onionoo change
 2016-01-12 13:51:25 +11:00
teor
4c1c2a313d Add Fallback Directory Candidate Selection Script 
"Tor has included a feature to fetch the initial consensus from nodes
 other than the authorities for a while now. We just haven't shipped a
 list of alternate locations for clients to go to yet.

 Reasons why we might want to ship tor with a list of additional places
 where clients can find the consensus is that it makes authority
 reachability and BW less important.

 We want them to have been around and using their current key, address,
 and port for a while now (120 days), and have been running, a guard,
 and a v2 directory mirror for most of that time."

Features:
* whitelist and blacklist for an opt-in/opt-out trial.
* excludes BadExits, tor versions that aren't recommended, and low
  consensus weight directory mirrors.
* reduces the weighting of Exits to avoid overloading them.
* places limits on the weight of any one fallback.
* includes an IPv6 address and orport for each FallbackDir, as
  implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks
  until #17840 is merged.)
* generated output includes timestamps & Onionoo URL for traceability.
* unit test ensures that we successfully load all included default
  fallback directories.

Closes ticket #15775. Patch by "teor".
OnionOO script by "weasel", "teor", "gsathya", and "karsten".
 2015-12-16 05:54:40 +11:00
Nick Mathewson
a02f6e3d23 update codegen script to work with latest mozilla 2015-10-07 16:24:30 -04:00
Nick Mathewson
216024261b add more clang checkers; find no new bugs. 2015-09-22 11:37:50 -04:00
Nick Mathewson
742514be2f scan-build is more mature now; no need to hardcode the one in my /opt/clang-3.4 2015-09-22 09:58:16 -04:00
Nick Mathewson
1eb2106375 Document callgraph analysis code 2015-08-25 11:53:20 -04:00
Sebastian Hahn
1633d1ad1d Remove tor-fw-helper more thoroughly 2015-08-21 10:36:53 -04:00
Sebastian Hahn
38601f06b7 Observe workqueue_reply_t in check-spaces 2015-08-21 10:36:53 -04:00
Nick Mathewson
bb46630513 Hack up the scripts/maint/*callgraph* scripts to do more, better 
These scripts are now a little more bulletproof, cache data a little
better, and generate more information.  Notably, they search for the
vectors or edges to cut that would lower the size of the largest
SCC.
 2015-08-15 22:37:32 -04:00
Nick Mathewson
def5883bbb Update callgraph code to find and output strongly connected components 2015-08-10 12:11:34 -04:00
Nick Mathewson
ebff18d18e Allow cov-display to take directories as arguments 2015-08-05 11:06:41 -04:00
Nick Mathewson
26c9691186 Check in some scripts I use for callgraph analysis. 2015-07-30 12:23:28 -04:00
Nick Mathewson
90701c1de4 Add my script to dump per-file or per-function coverage 2015-07-16 11:55:44 -04:00
Nick Mathewson
e71c8801cf more generic scan-build script 2015-05-29 13:50:05 -04:00
Nick Mathewson
1b52e95028 Merge branch '12498_ed25519_keys_v6' 
Fixed numerous conflicts, and ported code to use new base64 api.
 2015-05-28 11:04:33 -04:00
Nick Mathewson
79db24b3d5 Sign extrainfo documents with ed25519 
Extrainfo documents are now ed-signed just as are router
descriptors, according to proposal 220.  This patch also includes
some more tests for successful/failing parsing, and fixes a crash
bug in ed25519 descriptor parsing.
 2015-05-28 10:42:22 -04:00
Nick Mathewson
b600b68b20 Revise makedesc.py: teach it how to emit ed signatures and crosscerts 
Also, add a trivial ed25519-signed routerinfo to the tests.
 2015-05-28 10:41:50 -04:00
Nick Mathewson
df05e195ee Add trunnel-generated items for link handshake code. 
This includes the link handshake variations for proposal220.

We'll use this for testing first, and then use it to extend our
current code to support prop220.
 2015-05-28 10:41:49 -04:00
Nick Mathewson
993b4b8918 Fix a bug in format_changelog, in a silly way 2015-05-05 18:23:56 -04:00
Nick Mathewson
e81951c489 Add collation/splitting support to sortChanges script 2015-05-05 11:24:01 -04:00
cypherpunks
e85ba7459e Revive updateVersions.pl with make update-versions. 2015-03-14 13:00:06 -04:00
cypherpunks
eeb753e871 Make check-docs work from out-of-tree builds. 2015-03-14 13:00:06 -04:00
cypherpunks
034f51dd02 Be exact about the number of spaces in the header. 
Also fixes changes files that warn on this.
 2015-03-09 09:00:40 -04:00
cypherpunks
9dc90a5b7b Add check-changes rule for checking formatting of changes files. 
Additional fixes to make the change work;
- fix Python 2 vs 3 issues
- fix some PEP 8 warnings
- handle paths with numbers correctly
- mention the make rule in doc/HACKING.
 2015-03-09 09:00:12 -04:00
Nick Mathewson
94a877381d Check for # characters in lintchanges 2015-02-19 09:54:09 -05:00
Nick Mathewson
fc0febc5c6 Commit the update-copyrights script. 
(I'm tired of rewriting this by hand every January)
 2015-01-02 14:28:21 -05:00
Nick Mathewson
f54e54b0b4 Bump copyright dates to 2015, in case someday this matters. 2015-01-02 14:27:39 -05:00
Nick Mathewson
845d92295f have lintchanges check header format. 2014-12-22 10:00:34 -05:00
Nick Mathewson
441a481bb8 Resolve issues in changes files 2014-12-22 09:49:33 -05:00
Nick Mathewson
8b532a8c81 Short python script to lint the changes files 2014-12-22 09:34:55 -05:00
Nick Mathewson
a28df3fb67 Merge remote-tracking branch 'andrea/cmux_refactor_configurable_threshold' 
Conflicts:
	src/or/or.h
	src/test/Makefile.nmake
 2014-11-27 22:39:46 -05:00
Mansour Moufid
3ab2c865bf Add a rule to the calloc semantic patch for argument ordering. 2014-11-02 11:54:42 -05:00
Mansour Moufid
533790ca77 The second argument to tor_calloc should be a constant. 
Just like the conventional calloc.
 2014-11-02 11:54:42 -05:00
Mansour Moufid
a746081f38 Refactor the calloc semantic patch. 
This does not change its effects.
 2014-11-02 11:54:42 -05:00
Nick Mathewson
5ad8721838 Repair format_changelog.py drupal breaks a little more. Drupal is weird. 2014-10-31 09:13:08 -04:00
Nick Mathewson
96e3936fb7 Teach format_changelog.py --blog about drupal <!--break--> 2014-10-30 17:08:42 -04:00
Nick Mathewson
0eb1886692 Twiddle the format_changelog.py option defaults. 2014-10-30 16:54:10 -04:00
Nick Mathewson
c11ce1ca5b Reinvoke format_changelog.py 
Also, tweak it so that it puts major deprecations and requirements
early in the changelog.
 2014-10-29 10:19:10 -04:00
Nick Mathewson
4e7046cfb2 Teach format_changelog.py to emit HTML. 2014-10-27 10:32:25 -04:00
Nick Mathewson
6c5db03e02 format_changelog: Sort sections case-insensitively 2014-10-19 14:19:22 -04:00
Nick Mathewson
5344093829 format_changelog.py: fix spelling of "hyphenatable" 2014-10-19 12:59:17 -04:00
Nick Mathewson
29b0e82b5a format_changelog.py: tweak hyphenation rules 
Mark all our generated command names, and anything with a
double-hyphen, as unhyphenatable.
 2014-10-19 12:57:57 -04:00
Nick Mathewson
45f534e6df Teach format_changelog to sort and collate sections. 
Give it options to turn off collation and/or wrapping.
 2014-10-19 12:44:19 -04:00
Nick Mathewson
1405fbcc58 format_changelog.py: learn about the ReleaseNotes format 2014-10-19 11:47:16 -04:00
Nick Mathewson
48af23eb0d Emit fewer spurious lines in cov-diff output. 2014-10-14 09:41:40 -04:00
Nick Mathewson
b42d811718 Commit the script I used to generate signed ri and ei documents 2014-10-13 14:30:03 -04:00
Andrea Shepard
7674308f62 Make 'make check-spaces' not complain about function pointers returning size_t or double 2014-09-30 22:49:02 -07:00
Nick Mathewson
50d15e06b3 Use --require-version to prevent running trunnel pre-1.2 
(Also, regenerate trunnel stuff with trunnel 1.2.  This just adds a
few comments to our output.)
 2014-09-25 14:49:00 -04:00
Nick Mathewson
d303d6244e Have the run_trunnel.sh script tell trunnel to update src/ext/trunnel too 2014-09-25 13:16:35 -04:00
Nick Mathewson
1b13139709 Add a script to run trunnel on the trunnel files. 
Also, re-run the latest trunnel.

Closes ticket 13242
 2014-09-25 12:32:08 -04:00
Nick Mathewson
1bb10353ff Add coccinelle script to remove pointless callocs 2014-09-16 11:11:17 -04:00
Nick Mathewson
ef9a0d2048 Add script to detect and remove unCish malloc-then-cast pattern 
Also, apply it.
 2014-09-16 10:57:00 -04:00
Nick Mathewson
3d6652f3d1 Improve comments in checkSpace.pl to explain how to fix its warnings 2014-08-18 10:03:14 -04:00
Nick Mathewson
be231b0db1 Add scripts/coccinelle to scripts/README 2014-08-15 11:03:15 -04:00