Commit Graph

505 Commits

Author SHA1 Message Date
Nick Mathewson
c2643842a9 practracker: add ability to regenerate exceptions file.
Also add a useful argument parser.
2019-03-25 16:09:11 -04:00
Nick Mathewson
0260e0f6fc practracker: pass sys.argv to main() as an argument 2019-03-25 16:09:11 -04:00
Nick Mathewson
301e3f22ef Practracker: add a string explaining the excptions file. 2019-03-25 16:09:11 -04:00
Nick Mathewson
c81b2b09ea Merge branch 'practracker_comments' 2019-03-25 15:18:36 -04:00
Nick Mathewson
61cebb2035 Minimize the includes in control.c 2019-03-25 14:14:56 -04:00
Nick Mathewson
2917ecaa97 Split command-handling and authentication from control.c 2019-03-25 14:06:56 -04:00
Nick Mathewson
4754e9058b Split getinfo handling into a new control_getinfo.c 2019-03-25 12:49:24 -04:00
Nick Mathewson
a49f506e05 Split all controller events code into a new control_events.c
Also, split the formatting code shared by control.c and
control_events.c into controller_fmt.c.
2019-03-25 12:11:59 -04:00
Nick Mathewson
135b51c9d3 practracker: allow comments in exceptions file
Also, distinguish between empty lines (which we should ignore)
and incorrect lines (which we should warn about).
2019-03-25 09:28:24 -04:00
Nick Mathewson
a20a2025a5 practracker: sort filenames and directories.
This helps ensure that we'll get output in a stable order.

Closes ticket 29882.
2019-03-25 09:08:04 -04:00
teor
d4d541c53c
Merge remote-tracking branch 'tor-github/pr/785' 2019-03-25 14:01:20 +10:00
Taylor Yu
307c156fbe Set file encoding in practracker.py
Explicitly set the file encoding to UTF-8 in practracker.py, to avoid
problems in some CI environments.  Fixes bug 29789; bug not in any
released Tor.
2019-03-22 17:51:55 -05:00
George Kadianakis
0aaeec19e7 Satisfy practracker broken by #29665 and #28656. 2019-03-15 17:13:06 +02:00
Nick Mathewson
2d76945052 practracker: Create an exception for an as-yet-nonexistent file.
We need a better way to do this.
2019-03-14 09:35:23 -04:00
George Kadianakis
a55c89c475 practracker: Be compatible with python2 which is used by travis/jenkins. 2019-03-14 02:15:32 +02:00
George Kadianakis
26b0d95397 Merge branch 'tor-github/pr/780' 2019-03-13 16:07:19 +02:00
Nick Mathewson
17ff69a268 Merge branch 'bug29221_more_squashed' 2019-03-13 09:30:02 -04:00
George Kadianakis
3e38efdf16 practracker: Fix duplicate exceptions and handle them more "gracefully". 2019-03-13 09:27:29 -04:00
Nick Mathewson
21c6e295ca grandfather in two more functions 2019-03-13 09:27:29 -04:00
Nick Mathewson
e2512950b6 Improve failure message from check-best-practices 2019-03-13 09:27:29 -04:00
Nick Mathewson
f0302d51ab practracker: Be more careful about excluding "confusing terms"
Previously we excluded any line containing one of these terms from
consideration as the start or end of a function.  Now we're more
careful, and we only ignore these terms when they appear to be
starting a function definition.
2019-03-13 09:27:29 -04:00
George Kadianakis
ec8c5b3fea practracker: Improvements based on last Nick's review. 2019-03-13 09:27:29 -04:00
George Kadianakis
157f7ba93e practracker: Update exceptions file :) 2019-03-13 09:27:29 -04:00
George Kadianakis
8c9835c6e5 practracker: Normalize filesystem paths across Windows and Posix.
This was causing issues because the exceptions file is written using Posix
paths, whereas practracker in Windows was trying to match Windows paths ("\"
instead of "/").
2019-03-13 09:27:29 -04:00
George Kadianakis
8bacc1dad1 practracker: Improve documentation in problem.py . 2019-03-13 09:27:29 -04:00
George Kadianakis
4795f2d3a0 Fold in an initial practracker exceptions file. 2019-03-13 09:27:29 -04:00
George Kadianakis
58de565988 Call practracker as part of check-local.
- Introduce 'make check-best-practices'.
- Fix up Tor topdir etc to work with the way 'make check-local' gets called.
- Make practracker less likely to print useless stuff.
2019-03-13 09:27:24 -04:00
Nick Mathewson
95209be861 Make checkSpace.pl check guard macros:
- every .h file needs an #ifndef/#define pair.
  - They must refer to the same macro.
  - The guard macros that they refer to must be unique across all headers.
2019-03-12 15:19:28 -04:00
Nick Mathewson
a6dd893e76 Fix shellcheck warnings in pull-all/merge-all scripts
This appears at first glance to be a shellcheck bug.

Closes 29747.  Bugfix not in any released Tor.
2019-03-12 10:50:54 -04:00
rl1987
0befdb8a35 Disable git diff pagination 2019-03-10 18:28:07 +02:00
rl1987
537692c1e3 Using diff -u in check_for_diffs 2019-03-10 18:28:07 +02:00
rl1987
5f253f6a47 Iterate over contents of scripts/git with check_for_script_update function 2019-03-10 18:28:06 +02:00
rl1987
888bb9508b Move all git maintenance scripts to separate directory 2019-03-10 18:28:06 +02:00
rl1987
7b5f31f2d6 Mention what file has changed 2019-03-10 18:28:06 +02:00
rl1987
88633fad5b Write a comment for post-merge.git-hook 2019-03-10 18:28:06 +02:00
rl1987
bb8b2f47d0 Also print changes in git helper scripts, if any 2019-03-10 18:28:06 +02:00
rl1987
2d3ef34dce Add post-merge git hook to warn about git hooks being updated in the repo 2019-03-10 18:28:06 +02:00
Nick Mathewson
8fd20df455 Merge remote-tracking branch 'tor-github/pr/743' 2019-03-08 10:07:25 -05:00
David Goulet
db2e916afe Merge branch 'tor-github/pr/717' 2019-03-05 14:26:59 -05:00
rl1987
7f0516022b Also disallow pushing to/from upstream branch when branch names do not match 2019-03-05 14:26:09 -05:00
rl1987
0deea98d02 Improve pre-push.git-hook description 2019-03-05 14:26:09 -05:00
rl1987
f3eac74ed9 In pre-push hook script, actually check local and remote refs 2019-03-05 14:26:09 -05:00
rl1987
ae5a0f39cd Update git pre-push hook so that only upstream branches can get pushed to origin 2019-03-05 14:26:09 -05:00
David Goulet
436e0c8ca4 maint: Make git scripts fetch origin once
Replace the "git pull" by one single "git fetch origin" and then "git merge"
into each defined branches.

This speeds up the process considerably.

Closes #29616

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-03-01 09:24:46 -05:00
David Goulet
9256b02cc8 maint: Remove 0.3.3 branch from git scripts
It is EOL today: Febuary 28th, 2019

No more releases will occur.

Closes #29616

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-02-28 13:16:41 -05:00
George Kadianakis
cec555f44c Merge branch 'tor-github/pr/720' 2019-02-28 12:32:16 +02:00
George Kadianakis
31c1d91ffb Exit with 1 if new issues were found. Also work with python3. 2019-02-27 19:31:41 +02:00
George Kadianakis
2a722e2841 Improve metrics implementation.
No longer messes up with MOCK_IMPL. Also update the tests to show that. We are
still being innacurate on the line count in some cases, but that doesnt matter
so much.
2019-02-27 19:31:15 +02:00
George Kadianakis
26c4f6cfd0 Add more intelligent problem tracking. 2019-02-27 18:34:13 +02:00
George Kadianakis
371ea65c08 Improve #include counting func and move it to metrics.py. 2019-02-27 17:05:00 +02:00
George Kadianakis
a7684fcb57 Add some failing unittests for the function length metric. 2019-02-27 15:22:24 +02:00
George Kadianakis
17dd316749 Initial commit of practracker.py . 2019-02-27 15:22:24 +02:00
David Goulet
e1ad22643e maint: Helpful scripts for tor maintenance
Closes #29391

Signed-off-by: David Goulet <dgoulet@torproject.org>
2019-02-26 10:52:25 -05:00
Nick Mathewson
b3b737b875 Update pre-commit.git-hook for ticket 29553
- handle older source layout
 - handle empty changes directories
 - "set -e" so that  we exit if there's a problem.
2019-02-21 13:35:33 -05:00
Nick Mathewson
a036c0187a cov-diff: better handle the case where a file stops having coverage 2019-02-08 11:22:55 -05:00
Nick Mathewson
f4c76661d1 Add a script to check for coverage nondeterminism
Closes ticket 29436.
2019-02-08 11:22:24 -05:00
Nick Mathewson
f2b646a2b6 Merge branch 'ticket29435_035' into ticket29435_040 2019-02-08 10:47:04 -05:00
Nick Mathewson
192b312c97 Fix coverage script.
It was looking for object files made with the old automake
directorations, but those changed when we split up our libraries.

Fixes bug 29435; bugfix on 0.3.5.1-alpha.
2019-02-08 10:43:38 -05:00
Nick Mathewson
8b254c1c70 Merge branch 'precommit_squashed' 2019-01-24 11:08:10 -05:00
Nick Mathewson
dd22c84ccf Only run the checkIncludes.py script if it exists
(It was added in 0.3.5)
2019-01-24 08:06:09 -05:00
Nick Mathewson
107e50a90f lintChanges.py didn't take a directory argument in 0.2.9 2019-01-24 08:05:42 -05:00
rl1987
611e8dcda7 Also run checkInclude.py before every commit 2019-01-24 08:03:17 -05:00
rl1987
99fffc6c2f Add a pre-commit hook that runs code and changelog entry formatting checks 2019-01-24 08:03:17 -05:00
Nick Mathewson
00bc18b847 Merge remote-tracking branch 'tor-github/pr/653' 2019-01-22 08:13:14 -05:00
Nick Mathewson
aaef95ada5 format_changelog: treat "issue NNNN" as a track ticket 2019-01-18 10:15:02 -05:00
Nick Mathewson
7e1f893460 lintChanges.py: Two python re usage fixes 2019-01-17 08:46:06 -05:00
Nick Mathewson
2f683465d4 Bump copyright date to 2019 2019-01-16 12:33:22 -05:00
Nick Mathewson
9473a386c4 Update copyright script for 2019 2019-01-16 12:33:09 -05:00
Nick Mathewson
efe55b8898 Bump copyright date to 2019. 2019-01-16 12:32:32 -05:00
Nick Mathewson
2dd23086f1 Remove fallback scripts and whitelist
They have been extracted to a new fallback-scripts.git repository.

Closes ticket 27914.
2019-01-15 19:20:25 -05:00
Nick Mathewson
31e784915e Use [ -n "$x" ] in place of [ ! -z "$x" ] in chutney-git-bisect
Closes 29099.
2019-01-15 18:26:08 -05:00
rl1987
d47e7863bd Depending on script mode, either use 'whitelist' or 'fallback list' 2019-01-15 08:25:29 -05:00
rl1987
7fbe7a2563 In updateFallbackDirs.py, say 'fallback list' instead of 'whitelist' 2019-01-15 08:25:29 -05:00
Nick Mathewson
4afc6b172a Merge branch 'ticket28755_v2_squashed' 2018-12-12 11:07:38 -05:00
Nick Mathewson
5200df8557 Copy the nss-related changes into fuzzing_include_am.py. 2018-12-12 11:07:08 -05:00
Nick Mathewson
845e8dbe59 Fuzzing module for various string operations, currently focusing on
encoding and decoding.

There are bunches of places where we don't want to invest in a full
fuzzer, but we would like to make sure that some string operation
can handle all its possible inputs.  This fuzzer uses the first byte
of its input to decide what to do with the rest of the input.  Right
now, all the possibilities are decoding a string, and seeing whether
it is decodeable.  If it is, we try to re-encode it and do the whole
thing again, to make sure we get the same result.

This turned up a lot of bugs in the key-value parser, and I think it
will help in other cases too.

Closes ticket 28808.
2018-12-12 11:07:08 -05:00
Nick Mathewson
91be6a77e9 Merge remote-tracking branch 'teor/ticket24838' 2018-12-11 10:04:16 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
58d2ced572 Merge remote-tracking branch 'teor/24805-phoul' 2018-12-11 09:38:27 -05:00
Nick Mathewson
b65166750d Merge remote-tracking branch 'rl1987/ticket28007_take2' 2018-12-11 09:25:46 -05:00
rl1987
da264f7c76 Let's not double-quote OUTPUTARG after all 2018-12-08 15:17:22 +02:00
teor
6bc5c06dc2 Fallbacks: accept relays that are a fuzzy match to the whitelist
If a relay matches at least one fingerprint, IPv4 address, or IPv6
address in the fallback whitelist, it can become a fallback. This
reduces the work required to keep the list up to date.

Closes ticket 28768.
2018-12-07 16:30:24 +10:00
teor
75b5cc047d Fallbacks: refactor is_in_whitelist() to take an exact match boolean
No behaviour change.

Preparation for 24838.
2018-12-07 15:54:32 +10:00
teor
7f3a7d9a27 Fallbacks: Rename CONSENSUS_EXPIRY_TOLERANCE to REASONABLY_LIVE_TIME
Cleanup after 28768.
2018-12-07 15:54:32 +10:00
teor
c3fe405e21 Fallbacks: use a 24 hour future consensus tolerance
Tor clients on 0.3.5.6-rc? and later will use a consensus that will become
valid up to 24 hours in the future.

Clients on 0.3.5.5-alpha? and earlier won't accept future consensuses.

Update the fallback expiry tolerance to match tor's checks.

Part of 28768, follow-up on 28591.
2018-12-07 15:54:32 +10:00
teor
766fd6cf76 Fallbacks: use a 24 hour consensus expiry tolerance
Tor clients will use a consensus that expired up to 24 hours ago.

Clients on 0.3.5.5-alpha? and earlier won't select guards from an expired
consensus, but they can still bootstrap if they have existing guards.

Update the fallback expiry tolerance to match tor's checks.

Part of 28768, follow-up on 24661.
2018-12-07 15:54:32 +10:00
teor
d9f36d3e92 Fallbacks: minor script comment changes 2018-12-07 15:45:48 +10:00
Nick Mathewson
d4f558e385 Merge remote-tracking branch 'tor-github/pr/531' 2018-12-05 09:02:49 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
rl1987
9c90bddc42 Mention --no-verify in error message 2018-11-29 11:10:48 +02:00
rl1987
7c6dc28881 Improve comment at the top 2018-11-29 11:10:30 +02:00
rl1987
e2b418bab5 Rename to pre-push.git-hook 2018-11-29 10:56:56 +02:00
rl1987
59001a69c9 Address SC2086 in scan-build.sh
We can safely silence SC2086 warning on $CHECKERS, as contents of that
is hardcoded into script, and we don't want to require Bash to use Bash
array here. Double-quote $OUTPUTARG, as it's value depends on environment
variable.
2018-11-26 18:03:36 +02:00
rl1987
30f8b49d3b Silence SC2034 shellcheck checker for EXTRA_CHECKERS and NOISY_CHECKERS variables 2018-11-26 17:46:05 +02:00
Nick Mathewson
b16d6453ad Rewrite updateVersions script in Python, add datestamp functionality.
This updateVersions.pl script was one of the only essential perl
scripts left in out maint system, and was the only one that used
autoconf to fill in the script.

This script adds a feature to define an APPROX_RELEASE_DATE macro
that is updated when the version changes.  We'll use this to
implement prop297, so that we have an accurate view of when a
release date happens.
2018-11-24 20:44:37 -05:00
Colin Childs
6d2c2be291 Add CrackerboxPalace fallback 2018-11-21 20:36:08 -06:00
rl1987
ba2a9988cc Fix SC2230 warning 2018-11-20 16:57:57 +02:00
rl1987
27e982b470 Fix SC2046 and SC2086 warnings 2018-11-20 16:55:35 +02:00
rl1987
b2053cfc44 Also disallow fixup/squash commits in maint-* and release-* 2018-11-20 16:37:30 +02:00
rl1987
7dd515b0e0 Silence SC1117 warnings 2018-11-20 16:22:47 +02:00
rl1987
ffee0a6384 Add pre-push git hook to prevent fixup and squash commits from ending up in master 2018-11-20 15:40:52 +02:00
Nick Mathewson
a8df39e955 Merge remote-tracking branch 'tor-github/pr/519' 2018-11-19 08:47:03 -05:00
rl1987
d9f7cb3f62 Fix shellcheck issues in cov-diff - SC2231, SC2006 and SC2086 2018-11-15 19:37:07 +02:00
rl1987
4fc584f20e Fix one last SC2086 2018-11-15 19:35:56 +02:00
rl1987
7c04b00e65 Fix most instances of SC2086 warning 2018-11-15 19:35:56 +02:00
rl1987
ce1b018ab7 Fix shellcheck SC2006 warnings in 'coverage' script 2018-11-15 19:35:56 +02:00
Nick Mathewson
fae29f7b42 Make the topological-sort output off by default 2018-11-14 16:07:36 -05:00
Nick Mathewson
c3c8c926bf Update the check-includes script to enforce some naming and no-circularity rules 2018-11-14 16:07:36 -05:00
rl1987
5b48af9c4c Fix all instances of shellcheck warning SC2006 2018-11-04 19:34:46 +02:00
Nick Mathewson
f27279f857 Merge remote-tracking branch 'tor-github/pr/449' 2018-11-02 13:11:26 -04:00
Nick Mathewson
d671e3f513 Merge remote-tracking branch 'tor-github/pr/448' 2018-11-02 13:09:45 -04:00
rl1987
6aef0ce94e Fix shellcheck issue SC2217 in run_calltool.sh 2018-10-29 11:37:16 +02:00
rl1987
eab81b12e9 Fix shellcheck warning SC2086 in run_calltool.sh 2018-10-29 11:31:03 +02:00
rl1987
a0dd6bfdb0 run_trunnel.sh: Use 'find -exec' instead of a 'for' loop
This fixes shellcheck warnings SC2044 and SC2006.
2018-10-29 10:54:31 +02:00
rl1987
635312fc2a Silence shellcheck SC2086 in run_trunnel.sh 2018-10-29 10:45:14 +02:00
Colin Childs
17f4388b0f Remove mullbinde9 from fallback list 2018-10-15 13:47:31 -05:00
Colin Childs
9b946c13a2 Adding hviv104 to fallback list 2018-10-15 11:43:02 -05:00
Colin Childs
5033e950cc Adding Quake to fallback list 2018-10-12 15:25:17 -05:00
Colin Childs
57bba19bbe Remove aurora and chulak from fallback list 2018-10-12 15:21:32 -05:00
rl1987
8b5ad246e8 Fix issues that shellcheck found in chutney-git-bisect.sh 2018-10-11 20:28:11 +03:00
Colin Childs
bc68e80e0a Adding fallback relay 2018-09-20 13:48:45 -05:00
Colin Childs
1663a1dd63 Adjusting IP of fallback 2018-09-19 15:39:39 -05:00
teor
3eafa61f63 check-changes: Warn about bugfixes on future releases
Warn when bugfix changes files say that the bug is in a future release.

Closes ticket 27761.
2018-09-18 20:40:50 +10:00
teor
7e862c3ec0 check-changes: Check bugfix version formatting
Check that bugfix versions in changes files look like Tor versions
from the versions spec.

Part of ticket 27761.
2018-09-18 20:40:44 +10:00
Colin Childs
cb1891b412 Adding 1 new fallback 2018-09-12 13:39:49 -05:00
Nick Mathewson
b4b8fa4899 Set the file encoding in checkIncludes.py with Python3 2018-08-28 16:25:54 -04:00
Colin Childs
f0792537e9 Adding new fallback 2018-08-28 13:05:59 -05:00
teor
4217dc0558
Add scripts/test/chutney-git-bisect.sh, for bisecting using chutney
Supports bisection on 0.3.4 and earlier.
Recommend that users copy the script before bisecting.

Implements ticket 27211.
2018-08-24 12:46:04 +10:00
Colin Childs
51f8db1d92 Drop relays from fallback list that were terminated by provider 2018-08-16 14:32:23 -05:00
Colin Childs
9aceb92839 Adding 2 new fallback mirrors 2018-08-10 15:19:03 -05:00
Colin Childs
0260b72005 Adding relay from Edmond 2018-08-03 09:49:03 -05:00
Nick Mathewson
d66cfadbd3 Merge remote-tracking branch 'teor/bug26979' 2018-07-31 08:26:15 -04:00
Nick Mathewson
de4d7c3837 Merge branch 'ticket26447' 2018-07-31 08:21:10 -04:00
teor
c3fca338a2
Appveyor CI: always use HEAD for the short commit
Part of 26979.
2018-07-31 12:14:40 +10:00
teor
c6973aeccb
Appveyor CI: sort environmental variables
To avoid future duplicates.
2018-07-31 11:54:02 +10:00
teor
02a45b3ea4
Appveyor CI: fix some typos 2018-07-31 11:49:11 +10:00
teor
019c31bbf8
Appveyor CI: Generate correct tag names
Part of 26979.
2018-07-31 11:47:33 +10:00
teor
3d3e62d147
Appveyor CI: Switch to one URL per line
Part of 26979.
2018-07-31 11:37:11 +10:00
teor
9b41eb2bc3
Appveyor CI: Generate correct branches and URLs for pull requests
Part of 26979.
2018-07-31 11:13:20 +10:00
teor
861db8c4ae
Appveyor CI: Make short commits 10 hexdigits long
That's what git does for tor.

Part of 26979.
2018-07-31 11:13:17 +10:00
teor
fc9f8c28fb
Appveyor CI: Fix GitHub provider detection
Part of 26979.
2018-07-31 11:13:10 +10:00
Nick Mathewson
4833717d69 Merge branch 'maint-0.3.4' 2018-07-17 14:46:16 -04:00
rl1987
8645647250 Fix build to work with --disable-unittests again 2018-07-17 12:09:34 +00:00
Colin Childs
5c2431a343 Add mirrors from EmeraldOnion 2018-07-16 12:25:29 -05:00
Colin Childs
831abc0e10 Adding 3 more fallback mirrors 2018-07-16 12:25:29 -05:00
Colin Childs
226816df55 Fixing spacing issue in whitelist 2018-07-16 12:25:29 -05:00
Colin Childs
eca58aec76 Updating whitelist to match operator feedback 2018-07-16 12:25:29 -05:00
Colin Childs
7479f75b95 Update whitelist to match feedback from operators 2018-07-16 12:25:29 -05:00
Colin Childs
04de9443fe Update fallback lists to match #24805 2018-07-16 12:25:29 -05:00
Nick Mathewson
2d0e1cef20 Add fuzzing wrapper for fetch_from_buf_socks() 2018-07-12 14:20:46 -04:00