nihilist/tor
1
0
Fork 0
mirror of https://gitlab.torproject.org/tpo/core/tor.git synced 2024-11-24 04:13:28 +01:00
Code Issues Packages Projects Releases Wiki Activity
35,865 Commits 53 Branches 630 Tags 128 MiB
bf0e2ae0d0
Commit Graph

605 Commits

Author SHA1 Message Date
Nick Mathewson
e57209dc32 Regenerate the practracker exceptions.txt file 2019-08-01 10:25:42 -04:00
Nick Mathewson
a4e4896e66 practracker: restore exceptions.txt header when running --regen 2019-08-01 10:25:20 -04:00
David Goulet
c4864de573 Merge branch 'tor-github/pr/1177' 2019-08-01 10:18:10 -04:00
Nick Mathewson
19536fd18d practracker: replaces "overstrict" with "overbroad" 
I had the logic reversed here.
 2019-08-01 09:35:33 -04:00
Nick Mathewson
3221dc1b32 Lower check of TOR_DISABLE_PRACTRACKER 
Since we sometimes call practracker directly, that's where we should
check the TOR_DISABLE_PRACTRACKER envvar.
 2019-08-01 08:40:56 -04:00
Nick Mathewson
8d3f3e5d30 Practracker: add an integration test. 
This test runs practracker with a set of 0 thresholds, to make sure
that it enumerates all its values right.  It tries running with an
empty exceptions file, and with an exceptions file that covers
_some_ of the data, and it makes sure that the outputs are as expected.
 2019-07-30 12:14:46 -04:00
Nick Mathewson
a79e2c2975 practracker: better warning/regen handling 
Now that there is only one toplevel place where we print problems,
we can redirect just that one print to a file when we are
regenerating the exceptions.txt file.  Previously we redirected
sys.stdout, which is naughty, and forced us to send warnings (and
warnings alone) to stderr.
 2019-07-30 11:54:05 -04:00
Nick Mathewson
3f303c102a Practracker: new flags to control output. 
These flags let you suppress the message about the number of
problems and warnings, and let you control the thresholds above
which something counts as a problem.

I need this for testing.
 2019-07-30 11:49:50 -04:00
Nick Mathewson
31a0b81854 practracker: Remove problemvault global. 2019-07-30 10:17:56 -04:00
Nick Mathewson
65cb4fead5 practracker: Move the warning/error distinction to a higher level. 
Previously warnings were generated by magic inside ProblemVault; now
they're printed on demand.
 2019-07-30 10:17:47 -04:00
Nick Mathewson
bcef6a5802 practracker: Refactor flow to use generators 
Instead of having "consider" functions that have to call a global
ProblemVault, we can now generate all the metrics for the code
separately from the decision about what to do for them.
 2019-07-30 09:24:41 -04:00
Nick Mathewson
ec13a727b0 practracker: Rename "Problem" to "Item". 
I'm about to refactor the code into a set of iterators that yield
*all* the metrics for the code, and then add a filter on top of that
to return the problems.
 2019-07-30 09:03:58 -04:00
David Goulet
57f955a38b practracker: Make it happing for circuitpadding.c 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-07-25 10:28:38 -04:00
Nick Mathewson
f306d12b58 Refactor handling of TestingTorNetwork 
Previously, when TestingTorNetwork was set, we would manually adjust
the initvalue members of a bunch of other config_var_t, and then
re-run the early parts or parsing the options.

Now we treat the initvalue fields as immutable, but instead assign
to them in options_init(), as early as possible.  Rather than
re-running the early parts of options, we just re-call the
options_init_from_string() function.

This patch de-kludges some of our code pretty handily.  I think it
could later handle authorities and fallbacks, but for now I think we
should leave those alone.
 2019-07-24 15:21:56 -04:00
Nick Mathewson
f6db290e5a Merge remote-tracking branch 'tor-github/pr/1186' 2019-07-23 12:30:19 -04:00
David Goulet
ef2dd1ba96 Merge branch 'tor-github/pr/1116' 2019-07-23 09:46:29 -04:00
George Kadianakis
fdfee3d06f Changes file for #31113. 2019-07-23 12:44:43 +03:00
Nick Mathewson
d6a3636cdc Add a TOR_DISABLE_PRACTRACKER envvar for use by folks who don't care 
Fixes part of bug 30752
 2019-07-18 09:28:08 -04:00
Nick Mathewson
6303c9aa26 Practracker: add tolerances for exceptions 
When an exception is present, we can now violate the limit by a little
bit and only produce a warning.  The strict flag overrides this
behavior.

I've given file sizes a 2% tolerances and function sizes/include
counts a 10% tolerance.

Part of 30752
 2019-07-18 09:28:08 -04:00
Nick Mathewson
a5e1fa3a03 Practracker: add a --list-overstrict option 
This option lists every exception that is stricter than it needs to
be.

Part of 30752
 2019-07-18 09:28:08 -04:00
Nick Mathewson
43f163de80 Practracker: improve exclude-directory logic 
Instead of excluding directories at the last minute if they happen
to appear in our filenames, we exclude them early, before recursing
into all their subdirectories.

Part of 29746.
 2019-07-17 14:33:49 +02:00
Nick Mathewson
f93057fc0a Pracktracker: give the number of new errors found. 
Part of 29746.
 2019-07-17 14:33:49 +02:00
Nick Mathewson
86d3d310f5 Practracker: allow tabs in include lines 
This isn't actually something that Tor does, but it's cleaner to do
it this way.  Part of 29746.
 2019-07-17 14:33:49 +02:00
Nick Mathewson
4c09532996 Fix practracker_tests.py and practracker line counting. 
The practracker_tests.py unit test file called a function by its old
name.

Also, practracker counted functions as starting one line after the
function name, and ending with the closing brace.  Now they start
with the open brace and end with the closing brace.
 2019-07-17 14:33:49 +02:00
George Kadianakis
5303dbe624 Merge branch 'tor-github/pr/1152' 2019-07-04 17:14:06 +03:00
Nick Mathewson
fdbd139495 Merge remote-tracking branch 'tor-github/pr/1136' 2019-07-02 13:33:50 -04:00
Nick Mathewson
0fa3dc3228 begin_cell_parse(): Add an assertion to please coverity. 
Coverity doesn't understand that if begin_cell_parse() returns 0 and
sets is_begindir to 0, its address field will always be set.

Fixes bug 30126; bugfix on 0.2.4.7-alpha; Fixes CID 1447296.
 2019-06-28 11:29:51 -04:00
Nick Mathewson
089ef46063 Merge branch 'bug30721_squashed' 2019-06-26 09:57:20 -04:00
teor
fb93646c1c resolve: split sub-functions out of tor_addr_lookup() 
And remove the practracker exception for tor_addr_lookup().

Cleanup after 30721.
 2019-06-26 09:55:37 -04:00
Nick Mathewson
0fe9657c8c Merge remote-tracking branch 'tor-github/pr/1119' 2019-06-26 09:50:00 -04:00
Nick Mathewson
a91ed23403 Use structvar to find the types for config vars. 2019-06-25 12:51:25 -04:00
teor
c131b0763e
stats: add comments about the required chunk structure in extra info files 
These comments should prevent future instances of 30958.

And allow a larger file in practracker.

Follow up after 30958.
 2019-06-25 12:30:59 +10:00
Nick Mathewson
c60a85d22a Add a "typed_var" abstraction to implement lvalue access in C. 
Right now, this has been done at a high level by confparse.c, but it
makes more sense to lower it.

This API is radically un-typesafe as it stands; we'll be wrapping it
in a safer API as we do #30914 and lower the struct manipulation
code as well.

Closes ticket 30864.
 2019-06-24 17:50:43 -04:00
Nick Mathewson
81d16d8d0c checkSpace.pl: Allow 'bool' before a space and an open-paren 
We need this so we can declare function pointers returning bool
without upsetting our style checker. :/
 2019-06-24 15:11:57 -04:00
teor
45be44ed9c stats: Split extrainfo_dump_to_string() into smaller functions. 
Closes ticket 30956.
 2019-06-24 20:47:44 +10:00
David Goulet
9644f3462a Make the great grand practracker happy... 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-06-19 07:42:30 -04:00
David Goulet
05b6f73f12 Make the great grand practracker happy... 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-06-19 07:41:45 -04:00
Nick Mathewson
52d386c9b0 Merge remote-tracking branch 'tor-github/pr/1100' 2019-06-17 08:54:12 -04:00
teor
16e71c7fb0
practracker: accept an extra line in nt_service_install() 
Part of 30799.
 2019-06-13 18:23:50 +10:00
David Goulet
e9d99d2e15 Merge branch 'tor-github/pr/1083' 2019-06-11 11:43:15 -04:00
teor
6be9d3aed8
practracker: accept one extra line in routerlist.c 
practracker exception for 30780.
 2019-06-06 18:24:17 +10:00
teor
b07b1a4f6d
practracker: accept 3 extra lines in router_parse_entry_from_string() 
practracker exception for bug 30781.
 2019-06-06 13:42:30 +10:00
Nick Mathewson
2926f49b24 Regenerate practracker excpetions file 2019-06-05 09:34:14 -04:00
Nick Mathewson
ae490189f8 practracker update 2019-06-05 09:06:43 -04:00
George Kadianakis
99bf3d8e14 Merge branch 'tor-github/pr/1072' 2019-06-05 14:40:38 +03:00
Nick Mathewson
dc5cdde60c update practracker 2019-06-04 08:29:43 -04:00
Nick Mathewson
5e594831c7 Practracker excpetions. 2019-05-31 12:41:44 -04:00
George Kadianakis
70553a9134 Merge branch 'tor-github/pr/1037' 2019-05-29 21:25:32 +03:00
Nick Mathewson
0bc1241494 Make sure that we send at least some random data in RELAY_DATA cells 
Proposal 289 prevents SENDME-flooding by requiring the other side to
authenticate the data it has received.  But this data won't actually
be random if they are downloading a known resource.  "No problem",
we said, "let's fell the empty parts of our cells with some
randomness!" and we did that in #26871.

Unfortunately, if the relay data payloads are all completely full,
there won't be any empty parts for us to randomize.

Therefore, we now pick random "randomness windows" between
CIRCWINDOW_INCREMENT/2 and CIRCWINDOW_INCREMENT. We remember whether we have
sent a cell containing at least 16 bytes of randomness in that window.  If we
haven't, then when the window is exhausted, we send one.  (This window approach
is designed to lower the number of rng checks we have to do.  The number 16 is
pulled out of a hat to change the attacker's guessing difficulty to
"impossible".)

Implements 28646.
 2019-05-27 14:20:07 +03:00
Nick Mathewson
e6b862e6a8 Merge branch 'ticket30428_041_02_squashed' 2019-05-22 11:48:43 -04:00
David Goulet
3835a3acf5 sendme: Properly record SENDMEs on both edges 
Turns out that we were only recording the "b_digest" but to have
bidirectionnal authenticated SENDMEs, we need to use the "f_digest" in the
forward cell situation.

Because of the cpath refactoring, this commit plays with the crypt_path_ and
relay_crypto_t API a little bit in order to respect the abstractions.

Previously, we would record the cell digest as the SENDME digest in the
decrypt cell function but to avoid code duplication (both directions needs to
record), we now do that right after iff the cell is recognized (at the edge).
It is now done in circuit_receive_relay_cell() instead.

We now also record the cell digest as the SENDME digest in both relay cell
encryption functions since they are split depending on the direction.
relay_encrypt_cell_outbound() and relay_encrypt_cell_inbound() need to
consider recording the cell digest depending on their direction (f vs b
digest).

Fixes #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-22 11:47:20 -04:00
David Goulet
59b9eecc19 sendme: Record cell digest on both client and exit 
It turns out that only the exit side is validating the authenticated SENDME v1
logic and never the client side. Which means that if a client ever uploaded
data towards an exit, the authenticated SENDME logic wouldn't apply.

For this to work, we have to record the cell digest client side as well which
introduced a new function that supports both type of edges.

This also removes a test that is not valid anymore which was that we didn't
allow cell recording on an origin circuit (client).

Part of #30428

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-22 11:47:20 -04:00
Nick Mathewson
66eb0a5a32 updateCopyright: look at the current year. 2019-05-20 12:02:20 -04:00
Nick Mathewson
2f31c8146f rectify_include_paths: warn instead of aborting on duplicate headers 
We have two sendme.h files at the moment; we should fix that, but
not in this branch.
 2019-05-20 11:52:45 -04:00
Mike Perry
84274000d8 Yes, these functions really do have to be this long. 2019-05-16 20:29:09 +00:00
David Goulet
39a14421b1 Merge branch 'tor-github/pr/1021' 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-15 15:35:22 -04:00
George Kadianakis
d71fa707dd Merge branch 'bug28780-squashed3-rebased' into bug28780_rebase 2019-05-15 16:46:51 +03:00
Mike Perry
ff2a980935 The practracker beatings will continue until the files get smaller. 2019-05-15 16:44:59 +03:00
Mike Perry
5f47d582d5 Practracker beatings are even more fun when they get caused mid-PR due to a github auto-rebase of a PR 
Because github PRs choose the most recent origin/master at the time of the PR
(and for any fixups pushed to a PR later to send to CI), there are tons of
conflicts and unexpected practracker issues.

This means CI can suddenly fail after fixups to a branch that pass locally.

Then CI fails and we have to close and re-open the PR.
 2019-05-15 15:10:48 +03:00
Nick Mathewson
b9f50a2d77 update practracker for tor_init 2019-05-14 19:56:20 -04:00
Nick Mathewson
dd537ba35f Update practracker for 30452 2019-05-14 19:22:35 -04:00
Nick Mathewson
43d4119454 Merge remote-tracking branch 'tor-github/pr/1004' 2019-05-14 11:43:10 -04:00
Nick Mathewson
9ad2eb8f73 Merge branch 'bug28683_30173_29203_squashed' 2019-05-13 14:33:31 -04:00
Mike Perry
507df74b31 The practracker beatings will continue until the functions get smaller. 2019-05-13 14:30:35 -04:00
Nick Mathewson
c6523a6398 Merge remote-tracking branch 'tor-github/pr/998' 2019-05-13 14:25:54 -04:00
George Kadianakis
501d1ae0bd Merge branch 'tor-github/pr/973' 2019-05-10 12:49:01 +03:00
Neel Chauhan
3cafdeb8c0 Only call tor_addr_parse() in circuit_is_acceptable() when needed 2019-05-07 11:52:56 -04:00
George Kadianakis
7f2cd6545c Hiding crypt_path_t: Hide 'crypto' usage in sendme.c 2019-05-03 18:29:51 +03:00
George Kadianakis
593b7726e9 Hiding crypt_path_t: Trivial changes to satisfy check-local. 2019-05-03 18:15:26 +03:00
David Goulet
b3492d53c3 Merge branch 'tor-github/pr/984' 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-05-03 10:56:12 -04:00
George Kadianakis
b2c2cb9287 Merge branch 'tor-github/pr/986' 2019-05-02 18:12:52 +03:00
Nick Mathewson
0f365e2f46 practracker updates. 2019-05-02 09:22:13 -04:00
Mike Perry
e1771aeb51 The practracker beatings will continue until our files get smaller. 2019-05-01 21:04:40 +00:00
Taylor Yu
8e7316bae4 Split reply formatting out of control_fmt.c 
Split the core reply formatting code out of control_fmt.c into
control_proto.c.  The remaining code in control_format.c deals with
specific subsystems and will eventually move to join those subsystems.
 2019-04-30 13:18:46 -05:00
George Kadianakis
9084a90b00 Merge branch 'tor-github/pr/936' 2019-04-30 19:21:15 +03:00
George Kadianakis
a44aca5453 Merge branch 'tor-github/pr/993' 2019-04-30 19:13:57 +03:00
George Kadianakis
86f8dfe419 Merge branch 'tor-github/pr/983' 2019-04-30 19:13:30 +03:00
David Goulet
43c119fedb Merge branch 'tor-github/pr/980' 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-04-30 11:50:36 -04:00
Nick Mathewson
b5a62b1ef5 Move dirauth periodic events into dirauth module. 
Closes ticket 30294.
 2019-04-30 11:14:59 -04:00
David Goulet
535ba0d7c5 practracker: Update exceptions for #26288 
Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-04-29 12:27:53 -04:00
Nick Mathewson
1d44ac9acd Make nodelist_get_list() return a const pointer. 2019-04-26 10:36:49 -04:00
Nick Mathewson
01b07c548b Use parsing code for the simpler controller commands. 
(This should be all of the command that work nicely with positional
arguments only.)

Some of these commands should probably treat extra arguments as
incorrect, but for now I'm trying to be careful not to break
any existing users.
 2019-04-25 14:13:03 -04:00
Nick Mathewson
dbfe1a14e4 When parsing a multiline controller command, be careful with linebreaks 
The first line break in particular was mishandled: it was discarded
if no arguments came before it, which made it impossible to
distinguish arguments from the first line of the body.

To solve this, we need to allocate a copy of the command rather than
using NUL to separate it, since we might have "COMMAND\n" as our input.

Fixes ticket 29984.
 2019-04-25 14:13:03 -04:00
Nick Mathewson
d0a0f3e8cd Allow do_resolve() to be longer. 2019-04-24 14:15:18 -04:00
teor
de91b83849
practracker: Accept ~80 extra lines in src/core/or/policies.c 
Part of 23588.
 2019-04-24 17:31:02 +10:00
Nick Mathewson
15d4238383 Merge remote-tracking branch 'tor-github/pr/944' 2019-04-23 15:39:23 -04:00
Nick Mathewson
a7599c5be2 Merge remote-tracking branch 'tor-github/pr/962' 2019-04-23 12:48:37 -04:00
teor
bffba9d26f
practracker: accept more lines in microdescs_parse_from_string() 
Part of 28223.
 2019-04-19 10:34:16 +10:00
teor
f12b990bbf
practracker: accept the extra 25 line string from 27821 2019-04-17 18:44:26 +10:00
Neel Chauhan
f643020e64 Update practracker exceptions.txt for policies.c 2019-04-16 09:16:52 -04:00
Neel Chauhan
d4f980d29a Update exceptions.txt for Bug #29613 2019-04-14 14:27:29 -04:00
Nick Mathewson
7332346002 Changes file and practracker updates for 30149. 2019-04-11 18:58:44 -04:00
Nick Mathewson
e39b53ef7d changes file and practracker updates for 30147. 2019-04-11 17:59:21 -04:00
teor
60c46c6cd0
practracker: accept 4 extra lines due to 30041 2019-04-10 18:29:11 +10:00
teor
7741b21d0e
practracker: accept 6 extra lines in tortls_nss.c:tor_tls_context_new() 
These lines were added to fix bug 29241.
 2019-04-06 12:26:30 +10:00
Nick Mathewson
f021ca2d52 practracker: allow config.c to be a touch larger. 2019-04-05 09:59:45 -04:00
Nick Mathewson
ee6f54ff3f Merge remote-tracking branch 'tor-github/pr/860' 2019-04-03 08:33:40 -04:00
Nick Mathewson
f0e39df5ce allow circuituse.c to get even longer. 2019-03-29 14:30:33 -04:00
teor
a10d4adc25
Stop assuming that /usr/bin/python3 exists 
For scripts that work with python2, use /usr/bin/python.
Otherwise, use /usr/bin/env python3.

Fixes bug 29913; bugfix on 0.2.5.3-alpha.
 2019-03-27 11:07:55 +10:00
Nick Mathewson
7b9732063c practracker updates from messaging_v3 merge 
(main.c is a bit better, but shutdown.c is ugly)
 2019-03-26 20:14:21 -04:00
teor
beceb079e1
practracker: regen in master, for bug28925 merged to 0.4.0 and later 
python3 scripts/maint/practracker/practracker.py --regen
 2019-03-27 09:40:03 +10:00
Nick Mathewson
7502e5467b Regenerate practracker file from scratch. 
Closes ticket 29912.

Since this is the first time that practracker has had a stable
output order, this diff will be larger than usual.
 2019-03-26 19:28:35 -04:00
Nick Mathewson
300e7d8c99 Merge branch 'practracker_regen' 2019-03-26 19:27:54 -04:00
George Kadianakis
989b6325d6 Merge branch 'tor-github/pr/842' 2019-03-26 16:41:07 +02:00
Nick Mathewson
e028ec6bb7 Add new exceptions.txt entries 2019-03-26 09:43:46 -04:00
Nick Mathewson
39e4494344 practracker: update usage note in docstring 2019-03-26 08:42:14 -04:00
teor
ec736f8729
Merge remote-tracking branch 'tor-github/pr/833' 2019-03-26 13:02:37 +10:00
George Kadianakis
473decb246 Merge branch 'tor-github/pr/839' 2019-03-26 01:17:58 +02:00
Nick Mathewson
c2643842a9 practracker: add ability to regenerate exceptions file. 
Also add a useful argument parser.
 2019-03-25 16:09:11 -04:00
Nick Mathewson
0260e0f6fc practracker: pass sys.argv to main() as an argument 2019-03-25 16:09:11 -04:00
Nick Mathewson
301e3f22ef Practracker: add a string explaining the excptions file. 2019-03-25 16:09:11 -04:00
Nick Mathewson
c81b2b09ea Merge branch 'practracker_comments' 2019-03-25 15:18:36 -04:00
Nick Mathewson
61cebb2035 Minimize the includes in control.c 2019-03-25 14:14:56 -04:00
Nick Mathewson
2917ecaa97 Split command-handling and authentication from control.c 2019-03-25 14:06:56 -04:00
Nick Mathewson
4754e9058b Split getinfo handling into a new control_getinfo.c 2019-03-25 12:49:24 -04:00
Nick Mathewson
a49f506e05 Split all controller events code into a new control_events.c 
Also, split the formatting code shared by control.c and
control_events.c into controller_fmt.c.
 2019-03-25 12:11:59 -04:00
Nick Mathewson
135b51c9d3 practracker: allow comments in exceptions file 
Also, distinguish between empty lines (which we should ignore)
and incorrect lines (which we should warn about).
 2019-03-25 09:28:24 -04:00
Nick Mathewson
a20a2025a5 practracker: sort filenames and directories. 
This helps ensure that we'll get output in a stable order.

Closes ticket 29882.
 2019-03-25 09:08:04 -04:00
teor
d4d541c53c
Merge remote-tracking branch 'tor-github/pr/785' 2019-03-25 14:01:20 +10:00
Taylor Yu
307c156fbe Set file encoding in practracker.py 
Explicitly set the file encoding to UTF-8 in practracker.py, to avoid
problems in some CI environments.  Fixes bug 29789; bug not in any
released Tor.
 2019-03-22 17:51:55 -05:00
George Kadianakis
0aaeec19e7 Satisfy practracker broken by #29665 and #28656. 2019-03-15 17:13:06 +02:00
Nick Mathewson
2d76945052 practracker: Create an exception for an as-yet-nonexistent file. 
We need a better way to do this.
 2019-03-14 09:35:23 -04:00
George Kadianakis
a55c89c475 practracker: Be compatible with python2 which is used by travis/jenkins. 2019-03-14 02:15:32 +02:00
George Kadianakis
26b0d95397 Merge branch 'tor-github/pr/780' 2019-03-13 16:07:19 +02:00
Nick Mathewson
17ff69a268 Merge branch 'bug29221_more_squashed' 2019-03-13 09:30:02 -04:00
George Kadianakis
3e38efdf16 practracker: Fix duplicate exceptions and handle them more "gracefully". 2019-03-13 09:27:29 -04:00
Nick Mathewson
21c6e295ca grandfather in two more functions 2019-03-13 09:27:29 -04:00
Nick Mathewson
e2512950b6 Improve failure message from check-best-practices 2019-03-13 09:27:29 -04:00
Nick Mathewson
f0302d51ab practracker: Be more careful about excluding "confusing terms" 
Previously we excluded any line containing one of these terms from
consideration as the start or end of a function.  Now we're more
careful, and we only ignore these terms when they appear to be
starting a function definition.
 2019-03-13 09:27:29 -04:00
George Kadianakis
ec8c5b3fea practracker: Improvements based on last Nick's review. 2019-03-13 09:27:29 -04:00
George Kadianakis
157f7ba93e practracker: Update exceptions file :) 2019-03-13 09:27:29 -04:00
George Kadianakis
8c9835c6e5 practracker: Normalize filesystem paths across Windows and Posix. 
This was causing issues because the exceptions file is written using Posix
paths, whereas practracker in Windows was trying to match Windows paths ("\"
instead of "/").
 2019-03-13 09:27:29 -04:00
George Kadianakis
8bacc1dad1 practracker: Improve documentation in problem.py . 2019-03-13 09:27:29 -04:00
George Kadianakis
4795f2d3a0 Fold in an initial practracker exceptions file. 2019-03-13 09:27:29 -04:00
George Kadianakis
58de565988 Call practracker as part of check-local. 
- Introduce 'make check-best-practices'.
- Fix up Tor topdir etc to work with the way 'make check-local' gets called.
- Make practracker less likely to print useless stuff.
 2019-03-13 09:27:24 -04:00
Nick Mathewson
95209be861 Make checkSpace.pl check guard macros: 
- every .h file needs an #ifndef/#define pair.
  - They must refer to the same macro.
  - The guard macros that they refer to must be unique across all headers.
 2019-03-12 15:19:28 -04:00
Nick Mathewson
a6dd893e76 Fix shellcheck warnings in pull-all/merge-all scripts 
This appears at first glance to be a shellcheck bug.

Closes 29747.  Bugfix not in any released Tor.
 2019-03-12 10:50:54 -04:00
rl1987
888bb9508b Move all git maintenance scripts to separate directory 2019-03-10 18:28:06 +02:00
rl1987
7b5f31f2d6 Mention what file has changed 2019-03-10 18:28:06 +02:00
rl1987
88633fad5b Write a comment for post-merge.git-hook 2019-03-10 18:28:06 +02:00
rl1987
bb8b2f47d0 Also print changes in git helper scripts, if any 2019-03-10 18:28:06 +02:00
rl1987
2d3ef34dce Add post-merge git hook to warn about git hooks being updated in the repo 2019-03-10 18:28:06 +02:00
Nick Mathewson
8fd20df455 Merge remote-tracking branch 'tor-github/pr/743' 2019-03-08 10:07:25 -05:00
David Goulet
db2e916afe Merge branch 'tor-github/pr/717' 2019-03-05 14:26:59 -05:00
rl1987
7f0516022b Also disallow pushing to/from upstream branch when branch names do not match 2019-03-05 14:26:09 -05:00
rl1987
0deea98d02 Improve pre-push.git-hook description 2019-03-05 14:26:09 -05:00
rl1987
f3eac74ed9 In pre-push hook script, actually check local and remote refs 2019-03-05 14:26:09 -05:00
rl1987
ae5a0f39cd Update git pre-push hook so that only upstream branches can get pushed to origin 2019-03-05 14:26:09 -05:00
David Goulet
436e0c8ca4 maint: Make git scripts fetch origin once 
Replace the "git pull" by one single "git fetch origin" and then "git merge"
into each defined branches.

This speeds up the process considerably.

Closes #29616

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-03-01 09:24:46 -05:00
David Goulet
9256b02cc8 maint: Remove 0.3.3 branch from git scripts 
It is EOL today: Febuary 28th, 2019

No more releases will occur.

Closes #29616

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-02-28 13:16:41 -05:00
George Kadianakis
cec555f44c Merge branch 'tor-github/pr/720' 2019-02-28 12:32:16 +02:00
George Kadianakis
31c1d91ffb Exit with 1 if new issues were found. Also work with python3. 2019-02-27 19:31:41 +02:00
George Kadianakis
2a722e2841 Improve metrics implementation. 
No longer messes up with MOCK_IMPL. Also update the tests to show that. We are
still being innacurate on the line count in some cases, but that doesnt matter
so much.
 2019-02-27 19:31:15 +02:00
George Kadianakis
26c4f6cfd0 Add more intelligent problem tracking. 2019-02-27 18:34:13 +02:00
George Kadianakis
371ea65c08 Improve #include counting func and move it to metrics.py. 2019-02-27 17:05:00 +02:00
George Kadianakis
a7684fcb57 Add some failing unittests for the function length metric. 2019-02-27 15:22:24 +02:00
George Kadianakis
17dd316749 Initial commit of practracker.py . 2019-02-27 15:22:24 +02:00
David Goulet
e1ad22643e maint: Helpful scripts for tor maintenance 
Closes #29391

Signed-off-by: David Goulet <dgoulet@torproject.org>
 2019-02-26 10:52:25 -05:00
Nick Mathewson
b3b737b875 Update pre-commit.git-hook for ticket 29553 
- handle older source layout
 - handle empty changes directories
 - "set -e" so that  we exit if there's a problem.
 2019-02-21 13:35:33 -05:00
Nick Mathewson
8b254c1c70 Merge branch 'precommit_squashed' 2019-01-24 11:08:10 -05:00
Nick Mathewson
dd22c84ccf Only run the checkIncludes.py script if it exists 
(It was added in 0.3.5)
 2019-01-24 08:06:09 -05:00
Nick Mathewson
107e50a90f lintChanges.py didn't take a directory argument in 0.2.9 2019-01-24 08:05:42 -05:00
rl1987
611e8dcda7 Also run checkInclude.py before every commit 2019-01-24 08:03:17 -05:00
rl1987
99fffc6c2f Add a pre-commit hook that runs code and changelog entry formatting checks 2019-01-24 08:03:17 -05:00
Nick Mathewson
00bc18b847 Merge remote-tracking branch 'tor-github/pr/653' 2019-01-22 08:13:14 -05:00
Nick Mathewson
aaef95ada5 format_changelog: treat "issue NNNN" as a track ticket 2019-01-18 10:15:02 -05:00
Nick Mathewson
7e1f893460 lintChanges.py: Two python re usage fixes 2019-01-17 08:46:06 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
9473a386c4 Update copyright script for 2019 2019-01-16 12:33:09 -05:00
Nick Mathewson
2dd23086f1 Remove fallback scripts and whitelist 
They have been extracted to a new fallback-scripts.git repository.

Closes ticket 27914.
 2019-01-15 19:20:25 -05:00
rl1987
d47e7863bd Depending on script mode, either use 'whitelist' or 'fallback list' 2019-01-15 08:25:29 -05:00
rl1987
7fbe7a2563 In updateFallbackDirs.py, say 'fallback list' instead of 'whitelist' 2019-01-15 08:25:29 -05:00
Nick Mathewson
91be6a77e9 Merge remote-tracking branch 'teor/ticket24838' 2018-12-11 10:04:16 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
58d2ced572 Merge remote-tracking branch 'teor/24805-phoul' 2018-12-11 09:38:27 -05:00
teor
6bc5c06dc2 Fallbacks: accept relays that are a fuzzy match to the whitelist 
If a relay matches at least one fingerprint, IPv4 address, or IPv6
address in the fallback whitelist, it can become a fallback. This
reduces the work required to keep the list up to date.

Closes ticket 28768.
 2018-12-07 16:30:24 +10:00
teor
75b5cc047d Fallbacks: refactor is_in_whitelist() to take an exact match boolean 
No behaviour change.

Preparation for 24838.
 2018-12-07 15:54:32 +10:00
teor
7f3a7d9a27 Fallbacks: Rename CONSENSUS_EXPIRY_TOLERANCE to REASONABLY_LIVE_TIME 
Cleanup after 28768.
 2018-12-07 15:54:32 +10:00
teor
c3fe405e21 Fallbacks: use a 24 hour future consensus tolerance 
Tor clients on 0.3.5.6-rc? and later will use a consensus that will become
valid up to 24 hours in the future.

Clients on 0.3.5.5-alpha? and earlier won't accept future consensuses.

Update the fallback expiry tolerance to match tor's checks.

Part of 28768, follow-up on 28591.
 2018-12-07 15:54:32 +10:00
teor
766fd6cf76 Fallbacks: use a 24 hour consensus expiry tolerance 
Tor clients will use a consensus that expired up to 24 hours ago.

Clients on 0.3.5.5-alpha? and earlier won't select guards from an expired
consensus, but they can still bootstrap if they have existing guards.

Update the fallback expiry tolerance to match tor's checks.

Part of 28768, follow-up on 24661.
 2018-12-07 15:54:32 +10:00
teor
d9f36d3e92 Fallbacks: minor script comment changes 2018-12-07 15:45:48 +10:00
Nick Mathewson
d4f558e385 Merge remote-tracking branch 'tor-github/pr/531' 2018-12-05 09:02:49 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
rl1987
9c90bddc42 Mention --no-verify in error message 2018-11-29 11:10:48 +02:00
rl1987
7c6dc28881 Improve comment at the top 2018-11-29 11:10:30 +02:00
rl1987
e2b418bab5 Rename to pre-push.git-hook 2018-11-29 10:56:56 +02:00
Nick Mathewson
b16d6453ad Rewrite updateVersions script in Python, add datestamp functionality. 
This updateVersions.pl script was one of the only essential perl
scripts left in out maint system, and was the only one that used
autoconf to fill in the script.

This script adds a feature to define an APPROX_RELEASE_DATE macro
that is updated when the version changes.  We'll use this to
implement prop297, so that we have an accurate view of when a
release date happens.
 2018-11-24 20:44:37 -05:00
Colin Childs
6d2c2be291 Add CrackerboxPalace fallback 2018-11-21 20:36:08 -06:00
rl1987
ba2a9988cc Fix SC2230 warning 2018-11-20 16:57:57 +02:00
rl1987
27e982b470 Fix SC2046 and SC2086 warnings 2018-11-20 16:55:35 +02:00
rl1987
b2053cfc44 Also disallow fixup/squash commits in maint-* and release-* 2018-11-20 16:37:30 +02:00
rl1987
7dd515b0e0 Silence SC1117 warnings 2018-11-20 16:22:47 +02:00
rl1987
ffee0a6384 Add pre-push git hook to prevent fixup and squash commits from ending up in master 2018-11-20 15:40:52 +02:00
Nick Mathewson
fae29f7b42 Make the topological-sort output off by default 2018-11-14 16:07:36 -05:00
Nick Mathewson
c3c8c926bf Update the check-includes script to enforce some naming and no-circularity rules 2018-11-14 16:07:36 -05:00
rl1987
5b48af9c4c Fix all instances of shellcheck warning SC2006 2018-11-04 19:34:46 +02:00
rl1987
6aef0ce94e Fix shellcheck issue SC2217 in run_calltool.sh 2018-10-29 11:37:16 +02:00
rl1987
eab81b12e9 Fix shellcheck warning SC2086 in run_calltool.sh 2018-10-29 11:31:03 +02:00
Colin Childs
17f4388b0f Remove mullbinde9 from fallback list 2018-10-15 13:47:31 -05:00
Colin Childs
9b946c13a2 Adding hviv104 to fallback list 2018-10-15 11:43:02 -05:00
Colin Childs
5033e950cc Adding Quake to fallback list 2018-10-12 15:25:17 -05:00
Colin Childs
57bba19bbe Remove aurora and chulak from fallback list 2018-10-12 15:21:32 -05:00
Colin Childs
bc68e80e0a Adding fallback relay 2018-09-20 13:48:45 -05:00
Colin Childs
1663a1dd63 Adjusting IP of fallback 2018-09-19 15:39:39 -05:00
teor
3eafa61f63 check-changes: Warn about bugfixes on future releases 
Warn when bugfix changes files say that the bug is in a future release.

Closes ticket 27761.
 2018-09-18 20:40:50 +10:00
teor
7e862c3ec0 check-changes: Check bugfix version formatting 
Check that bugfix versions in changes files look like Tor versions
from the versions spec.

Part of ticket 27761.
 2018-09-18 20:40:44 +10:00
Colin Childs
cb1891b412 Adding 1 new fallback 2018-09-12 13:39:49 -05:00
Nick Mathewson
b4b8fa4899 Set the file encoding in checkIncludes.py with Python3 2018-08-28 16:25:54 -04:00
Colin Childs
f0792537e9 Adding new fallback 2018-08-28 13:05:59 -05:00
Colin Childs
51f8db1d92 Drop relays from fallback list that were terminated by provider 2018-08-16 14:32:23 -05:00
Colin Childs
9aceb92839 Adding 2 new fallback mirrors 2018-08-10 15:19:03 -05:00
Colin Childs
0260b72005 Adding relay from Edmond 2018-08-03 09:49:03 -05:00
Nick Mathewson
de4d7c3837 Merge branch 'ticket26447' 2018-07-31 08:21:10 -04:00
Colin Childs
5c2431a343 Add mirrors from EmeraldOnion 2018-07-16 12:25:29 -05:00
Colin Childs
831abc0e10 Adding 3 more fallback mirrors 2018-07-16 12:25:29 -05:00
Colin Childs
226816df55 Fixing spacing issue in whitelist 2018-07-16 12:25:29 -05:00
Colin Childs
eca58aec76 Updating whitelist to match operator feedback 2018-07-16 12:25:29 -05:00
Colin Childs
7479f75b95 Update whitelist to match feedback from operators 2018-07-16 12:25:29 -05:00
Colin Childs
04de9443fe Update fallback lists to match #24805 2018-07-16 12:25:29 -05:00
Nick Mathewson
413b73450c Merge remote-tracking branch 'imnotbad/bug26502' 2018-07-12 09:30:01 -04:00
Nick Mathewson
ee12286222 Add a little documentation to checkIncludes.py. 
Someday people might be glad I did.
 2018-07-09 16:02:34 -04:00
Nick Mathewson
294f580f04 Re-wrap the error message emitted by checkIncludes.py 2018-07-09 15:48:55 -04:00
Nick Mathewson
3145e469d8 Make checkIncludes work on python 2.7 2018-07-09 15:46:39 -04:00
Kaidan
85fa69297c Remove small piece of unneeded documentation 2018-07-08 07:05:31 +10:00
Kaidan
5f09708046 #26502: Stop using fallback blacklist 2018-07-08 06:54:14 +10:00
Nick Mathewson
f720a5a439 Fix everything that previously referred to src/or 2018-07-05 17:15:50 -04:00
Nick Mathewson
81cb0afb2b Start splitting src/or 
This is a very gentle commit that just lays the groundwork in the
build system: it puts the include files to build libtor-app.a into
src/core, and to build the tor executable into src/app.  The
executable is now "src/app/tor".
 2018-07-05 17:15:50 -04:00
Nick Mathewson
83a4946e7b Prune the .may_include files a bit; detect unused lines in them 2018-07-01 18:14:28 -04:00
Nick Mathewson
860b9a9918 Remove some accidentally committed code in checkIncludes.py 2018-06-26 12:04:24 -04:00
Nick Mathewson
405fa42e8a Another windows include 2018-06-22 13:52:30 -04:00
Nick Mathewson
eb784aa9ea add license statement to checkincludes, and have it return an error code 2018-06-21 14:29:00 -04:00
Nick Mathewson
999f7984e1 New script to check includes for modularity violations 
Includes configuration files to enforce these rules on lib and
common.  Of course, "common" *is* a modularity violation right now,
so these rules aren't as strict as I would like them to be.
 2018-06-21 14:05:33 -04:00
Nick Mathewson
0dab29ce10 Run rectify_include_paths.py 2018-06-20 09:35:05 -04:00
Nick Mathewson
178b738be0 Script to replace include paths with full paths under src/ 
This will let us move around header files without having to fix up
all the include sites manually.
 2018-06-20 09:25:16 -04:00
Nick Mathewson
fb0019daf9 Update copyrights to 2018. 2018-06-20 08:13:28 -04:00
Nick Mathewson
67dc83fa75 New check-spaces rule: Our C files may not have duplicate names. 2018-06-18 16:17:11 -04:00
Nick Mathewson
6af248f27f Finally remove our EOL@EOF check. 
This check was added by mistake long ago.  It will be nice to see
these EOLs disappear from our code over time.
 2018-06-18 16:10:47 -04:00
Nick Mathewson
c9c26d0c31 Remove a comment about 0.2.8 that will soon be irrelevant. 
(It becomes irrelevant on 15 May 2018 when 0.2.5 is EOL.)
 2018-04-18 10:44:25 -04:00
Isis Lovecruft
9799394375 maint: Add script for vendoring/updating Rust dependencies. 
* ADD documentation for usage both inside the script and in
   doc/HACKING/CodingStandardsRust.md
 * FIXES part of #25310: https://bugs.torproject.org/25310
 2018-03-21 17:04:28 -04:00
Deepesh Pathak
ca6682f3f8 Fix spelling mistakes corresponding to ticket #23650 2018-02-07 10:41:57 -05:00
teor
65520f1bde
Stop logging excessive information about fallback netblocks 
Implements ticket 24791
 2018-01-05 13:28:17 +11:00
teor
d30d3fe8ed
Adjust fallback selection parameters 
Avoid selecting fallbacks that change their IP addresses too often.

Select more fallbacks by ignoring the Guard flag, and allowing lower
cutoffs for the Running and V2Dir flags. Also allow a lower bandwidth,
and a higher number of fallbacks per operator (5% of the list).

Implements ticket 24785.
 2018-01-05 13:28:13 +11:00
teor
befcc56d84
Update dependency list for the fallback script 
Comment-only change.

Implements #24711.
 2018-01-05 13:28:09 +11:00
teor
91a8b37a2e
Make updateFallbackDirs.py search harder for python 
(Some OSs don't put it in /usr/bin.)

Fixes bug 24708; bugfix on 0.2.8.1-alpha.
 2018-01-05 13:28:05 +11:00
teor
2ae33e486e
Revise the generateFallbackDirLine.py script to use descriptors 
Patch by atagar, part of #24706.
 2018-01-05 13:28:01 +11:00
teor
e244738bbd
Add fallback directory mirror helper scripts 
Add the generateFallbackDirLine.py script for automatically generating
fallback directory mirror lines from relay fingerprints. No more typos!

Add the lookupFallbackDirContact.py script for automatically looking up
operator contact info from relay fingerprints.

Implements ticket 24706.
 2018-01-05 13:27:55 +11:00
teor
9864fcd4e3
Add a generated timestamp to the fallback directory header 
This can act as a revision counter, as requested by atagar in 24742.

Part of 24725.
 2018-01-05 13:27:47 +11:00
teor
d8ab1b803a
Remove an incorrect C comment from updateFallbackDirs.py's list generation code 
Part of #24725.
 2018-01-05 13:27:39 +11:00
teor
b6033f5734
Add a delimiter to the end of each fallback entry 
This helps external parsers.

Also, add comments indicating where to add new fields in the fallback
format.

Part of 24725.
 2018-01-05 13:27:32 +11:00
teor
beedf5fd81
Remove commas and equals signs from external string inputs to the fallback list 
This makes sure that list parsers only see one comma per fallback entry,
and only see one equals sign per field.

Implements ticket 24726.
 2018-01-05 13:27:22 +11:00