Commit Graph

897 Commits

Author SHA1 Message Date
Nick Mathewson
97a9966b04 Update git scripts to include tor-gitlab repository
Analogously to tor-github, we now make a tor-gitlab repository.  It
is set up to disable push direct attempts, and to fetch merge
requests into appropriate branches.

git-pull-all.sh knows how to fetch this repository.
2020-07-09 14:27:10 -04:00
David Goulet
d80c34d214 trace: Move LTTng specific declartion to .inc file
LTTng tracepoint probe declaration is not really following a C standard that
coccinelle and checkSpace.pl likes.

Move everything to a .inc file and standardize the trace_probes_circuit.h
header to include that LTTng specific file if the instrumentation was enabled
at configure time.

Part of #32910

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-07-08 09:18:33 -04:00
David Goulet
bf0e2ae0d0 trace: Add single tracepoint in circuit subsystem
This is the very first tracepoint in tor. It is in the circuit subsystem for
when a new circuit opens.

LTTng instrumentation requires lot more around a tracepoint than USDT thus
this commit only adds one tracepoint in order to outline a base to add more
tracepoints later.

The idea is that we separate subsystem into what LTTng defines as "providers"
so the circuit provider contains the tracepoint definitions for the circuit
subsystem.

Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-07-08 09:10:08 -04:00
George Kadianakis
3bb9b7e743 Merge branch 'tor-github/pr/1934' 2020-07-03 10:50:47 +03:00
David Goulet
887dc6abb0 scripts: Don't run practracker in git hooks
Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-07-02 14:39:44 -04:00
David Goulet
8ae0ecbd6a scripts: Add env variable to disable pre-commit hook
We introduce TOR_EXTRA_PRE_COMMIT_CHECKS environment variable to run the
pre-commit hook. The pre-push git hook will set it in order to run all
pre-commit checks.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-07-02 14:39:44 -04:00
Nick Mathewson
8f362b7bce Extract extend_info manipulation functions into a new file. 2020-07-02 09:51:03 -04:00
David Goulet
7a6e1f2491 scripts: Fix rename_c_identifier.py uncommitted warning
Only warn when we actually want a commit _and_ there are uncommitted changes
found. Else, it is fine to rename if the script is not creating a commit.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-06-30 13:48:22 -04:00
David Goulet
86da0cfd36 Merge branch 'tor-github/pr/1945' 2020-06-24 13:51:51 -04:00
George Kadianakis
279225acdd Merge branch 'tor-github/pr/1929' 2020-06-24 17:20:25 +03:00
David Goulet
6da8c0b4fa addr: Rename resolve_my_address to be v4 specific
Part of #33233

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-06-23 09:25:36 -04:00
Mike Perry
c047c6d5b8 practracker FTL!
Faster Than Light!
2020-06-16 13:44:10 -05:00
George Kadianakis
3cb77a9cca Merge branch 'maint-0.4.4' 2020-06-11 17:29:54 +03:00
Nick Mathewson
f3c82c86f9 Improve our clang-format.sh script
Previously, this script ran over every C file in our source code,
which took up to a minute on my desktop.

Instead, the script now has several modes that it can run in, to
minimize the time spent and improve useful output.  It should now be
suitable for everyday use and git hooks.  I hope.

I've also renamed this script, so that we can keep using it in the
future if we were to move to some tool other than clang-format.
2020-06-11 10:28:03 -04:00
George Kadianakis
52edea121e Fold in a changes file and update exceptions.txt.
Co-authored-by: Florentin Rochet <florentin.rochet@uclouvain.be>
2020-06-11 16:13:25 +03:00
Nick Mathewson
fc16dbf7aa Remove "enable_practracker_in_hooks" for 0.4.4. 2020-06-09 15:47:25 -04:00
Nick Mathewson
354f085e5f Merge remote-tracking branch 'tor-github/pr/1888/head' 2020-06-09 15:44:58 -04:00
Nick Mathewson
eaae5625cb Add 0.4.4 to list of tor branches used by git scripts. 2020-06-09 14:17:15 -04:00
Nick Mathewson
ca8b04f194 Merge branch 'ticket34375' 2020-06-09 14:16:21 -04:00
c
76d07940fd
scripts/maint: Remove obsolete generated tor.sh
Per <https://trac.torproject.org/projects/tor/ticket/34381>, these
contrib scripts were generated by older versions of Tor and shellcheck
will throw warnings if they still exist.
2020-06-08 01:41:36 +00:00
Nick Mathewson
811853c677 Remove 0.4.1 from list of tor branches used by git scripts. 2020-06-03 14:54:21 -04:00
teor
8ec4d9cc3c practracker: Accept extra file lines, enforce a smaller function
Accept extra lines in nodelist and routerlist due to extra features, and
due to refactors that simplify some functions.

Most of the refactor eliminated duplicate code in smaller functions, so
there's only one large function that got smaller.

Part of 34200.
2020-05-18 21:53:52 +10:00
Nick Mathewson
bbfdee4c4e Fix a python warning in format_changelog.py. 2020-05-14 10:37:52 -04:00
teor
de979dd7bd practracker: Allow a few extra lines in circuituse
Part of 33222.
2020-05-13 20:08:29 +10:00
teor
b64972edcd practracker: Accept extra or.h lines 2020-05-11 14:39:42 +10:00
teor
24099680f1 Appease practracker: wrapping costs lines 2020-05-07 20:19:19 +10:00
David Goulet
a25f167072 config: New file resolve_addr.{c|h}
Move a series of function from config.c into that new file which is related to
address resolving.

Part of #33789

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-05-05 14:24:04 -04:00
teor
a202f03c64 practracker: Accept some wrapped long lines
Part of 33222.
2020-04-30 15:15:53 +10:00
teor
efcae919ae
practracker: Accept the connection buf refactor
Accept a few extra lines from the connection_buf_read_from_socket()
refactor.

Cleanup after 33131.
2020-04-30 13:55:26 +10:00
teor
ed4420b4b8 practracker: Allow an extra line in tor_addr_parse_mask_ports()
We added a cast, and wrapped a line.
2020-04-30 06:54:42 +10:00
teor
f5c34d340d
Run practracker regen
Lock-in improvements.
Accept some small function size increases.
Allow a small amount of extra file size increase.
2020-04-14 15:42:52 +10:00
Nick Mathewson
80031db32a Merge remote-tracking branch 'tor-github/pr/1801/head' 2020-04-09 11:50:20 -04:00
teor
75aba30abf relay: Run practracker --regen
Lock in some file and function size improvements.

Accept some small file size and dependency increases.

Part of 33633.
2020-04-09 11:00:04 +10:00
teor
065ccda4f6
Merge branch 'pr1838_squashed' 2020-04-07 17:36:17 +10:00
Nick Mathewson
ea9ff58921
Try rewriting normalize_srcdir to normalize harder. 2020-04-07 17:36:05 +10:00
Nick Mathewson
d3db84c1e7
Make check_subsystem_order work in distcheck builds. 2020-04-07 17:36:02 +10:00
Nick Mathewson
5f49695f94
Make check_subsystem_order work in out-of-tree builds. 2020-04-07 17:35:58 +10:00
Nick Mathewson
9b434b79ce
Add a test script to check subsystem order as part of make check. 2020-04-07 17:35:51 +10:00
teor
a762234ba2
scripts: Ignore editor temporary files
Ignore editor temporary files when autostyling in:
* rectify_include_paths.py
* rename_c_identifier.py

Obviously correct changes to already-reviewed code.
2020-04-03 19:16:57 +10:00
Nick Mathewson
978b7ef45c Teach --check-subsystem-order to take input from stdin 2020-03-26 12:30:30 -04:00
Nick Mathewson
a40d539f7c Extend includes.py to compare topology with subsystem init order. 2020-03-26 12:17:53 -04:00
teor
a34a88c6f1
practracker: Accept changes
Accept a smaller file and function.
Let router.c grow slightly without warning.
Accept an extra comment line.

Cleanup after 32588.
2020-03-21 04:10:24 +10:00
teor
5d53b2317f
practracker: Accept larger dirvote file
Due to improved function comments.
2020-03-21 02:48:51 +10:00
teor
bb3df5f322
practracker: Disable practracker in git hooks
Disable our coding standards best practices tracker in our git hooks.

0.4.3 branches only.

Closes ticket 33678.
2020-03-20 17:32:57 +10:00
teor
78b2984ac9
practracker: Regenerate exceptions file
Accept the current state of the tor codebase, including all the
files and functions that have increased technical debt over the
last few months.

Accepts all the current practracker warnings, so that reviewers
can focus on new warnings.

Closes ticket 33635.
2020-03-19 16:59:52 +10:00
teor
c86d08e3e6
Run "practracker.py --regen-overbroad"
We've successfully resolved a bunch of technical debt over the last
few months. Let's remove the allowances for that debt from the
practracker exceptions.
2020-03-17 23:02:16 +10:00
teor
1d17541cfa
Merge branch 'pr1764_squashed' 2020-03-13 11:14:20 +10:00
Nick Mathewson
78c9ff7052
Add a script to install the scripts in scripts/git appropriately.
To run this, say something like

   TOR_DEVTOOL_DIR=~/bin scripts/git/git-install-tools.sh all

To see what it would do, give it the -n flag.

To get help, give it the -h flag.
2020-03-13 11:14:04 +10:00
David Goulet
df3f2bd9aa hs-v3: Log reasons why service can't upload descriptor
When a service can not upload its descriptor(s), we have no logs on why. This
adds logging for each possible reason for each descriptors.

That logging is emitted every second so it is rate limited for each reason and
per descriptor.

Closes #33400

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-03-11 15:32:11 +02:00
Nick Mathewson
8096f3b254 Revert "scripts: Remove typo in git-list-tor-branches.sh"
This reverts commit a63b414822.

The "typo" wasn't actually a typo.  The "r" in the suffixes for the
release branches is meant to make them distinct from the maint
branches.

Now, _right now_, nothing uses release branch suffixes, but it's
important to keep them distinct if we decide to use them in the
future.
2020-03-09 09:03:51 -04:00
David Goulet
a63b414822 scripts: Remove typo in git-list-tor-branches.sh
Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-03-05 10:29:47 -05:00
David Goulet
a62e3e45e6 Merge branch 'tor-github/pr/1720' 2020-03-05 10:29:42 -05:00
Nick Mathewson
d9152b8a66 Update git scripts to use git-list-tor-branches.sh 2020-03-05 10:29:35 -05:00
Nick Mathewson
42660ea056 Add a script to list the maintained Tor branches in different ways
This will be used for 32121
2020-03-05 10:29:35 -05:00
Nick Mathewson
686494f0f7 Merge branch 'clang_format_prep_3' 2020-03-05 08:23:32 -05:00
Nick Mathewson
caa392a73a Merge remote-tracking branch 'tor-github/pr/1685/head' 2020-02-24 07:45:20 -05:00
Nick Mathewson
7ba7f9c0de Merge branch 'maint-0.4.3' 2020-02-20 10:44:43 -05:00
Nick Mathewson
7b42502869 Merge remote-tracking branch 'tor-github/pr/1757/head' into maint-0.4.3 2020-02-20 10:44:36 -05:00
Nick Mathewson
66bbdadbbe Merge branch 'extract_routerkeys_squashed' 2020-02-20 10:41:12 -05:00
Nick Mathewson
1d068625dd Move relay_handshake.[ch] into src/feature/relay, and make it optional 2020-02-20 10:41:06 -05:00
Nick Mathewson
09fb7987c5 Merge branch 'maint-0.4.3' 2020-02-20 08:50:18 -05:00
Nick Mathewson
dd4fa9b468 Extract relay-only handshake code into its own file.
This is not the only relay-only handshake code, but it is most of
such code that is in connection_or.c.
2020-02-19 19:11:57 -05:00
teor
57ff8d4a53
practracker: Read unicode files when using Python 2
Allow the "practracker" coding best practices checking script to read
unicode files, when using Python 2.

We made the script use unicode literals in 0.4.3.1-alpha, but didn't
change the codec for opening files.

Fixes bug 33374; bugfix on 0.4.3.1-alpha.
2020-02-19 08:22:01 +10:00
teor
d8cbb2965e
Run "practracker --regen-overbroad"
Closes ticket 33373.
2020-02-19 08:13:02 +10:00
David Goulet
17724a7cde Merge branch 'tor-github/pr/1743' 2020-02-18 09:06:13 -05:00
Nick Mathewson
d3c23b079a lintChanges: Tolerate variants that sortChanges knows how to clean up. 2020-02-18 09:01:56 -05:00
Panagiotis Vasilopoulos
674a821fbb
Stop assuming that /usr/bin/python exists
Fixes 33192
2020-02-16 21:58:01 +02:00
Taylor Yu
55f088bb29 Script to check sorting of manpage entries
Add a script to help check the alphabetical ordering of option names
in a manpage.  Closes ticket 33339.
2020-02-14 11:02:16 -06:00
Nick Mathewson
b5ccdd978e Add a dire warning about not reformatting the whole codebase yet. 2020-02-12 18:52:35 -05:00
teor
99d957696b
scripts/git: Stop running the checked-out git scripts
Stop executing the checked-out pre-commit hook from the pre-push hook.
Instead, execute the copy in the user's git dir.

Fixes bug 33284; bugfix on 0.4.1.1-alpha.
2020-02-12 20:37:21 +10:00
David Goulet
2a524662c3 Merge branch 'tor-github/pr/1717' 2020-02-11 10:52:49 -05:00
Nick Mathewson
7404b6fb71 Update scripts/git/*.sh to know about 043 and not about 040. 2020-02-11 10:52:42 -05:00
David Goulet
f0964628e6 Merge branch 'ticket33029_042_01' into ticket33029_043_03
Conflicts:
	doc/tor.1.txt
	src/app/config/config.c
	src/app/config/or_options_st.h
	src/core/mainloop/connection.h

Between 042 and 043, the dirauth options were modularized so this merge commit
address this by moving the AuthDirRejectUncompressedRequests to the module
along with a series of accessors.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-02-11 10:30:29 -05:00
Christian Clauss
9f7e1ccacb Undefined name: make_key() -> make_rsa_key() 2020-02-11 08:35:11 -05:00
Nick Mathewson
5fa62fffee checkSpace: permit wide lines for LCOV_EXCL
We're telling clang-format that a line with LCOV_EXCL must not be
split -- that's fine, but we shouldn't complain when it indents it.
2020-02-06 16:25:56 -05:00
Nick Mathewson
f39ba52029 checkSpace: be more careful about bad function headers.
Previously we would forbid macro indentations like this:

FOO({
  int x;
  })

But clang-format sometimes generates those.
2020-02-06 14:54:04 -05:00
Nick Mathewson
c8fae6b5c8 checkSpace: don't treat an unindented label as starting a function. 2020-02-06 14:33:05 -05:00
Nick Mathewson
8d6f27cea5 checkSpace.pl: Allow space between iteration macros and ().
Clang-format wants to put these in, and they do make sense for
consistency.

Also allow more types.
2020-02-06 14:33:05 -05:00
Nick Mathewson
bfa7607385 checkSpace.pl: Use a data structure for a list of non-function names 2020-02-06 14:33:05 -05:00
Nick Mathewson
1f1d943999 checkspace: allow spaces in cpp directives. 2020-02-06 14:33:05 -05:00
Nick Mathewson
15819cde61 checkSpace.pl: allow {{, ){, and ({. 2020-02-06 14:33:05 -05:00
Nick Mathewson
f1d371bf32 checkSpace: remove the nosplabel test. 2020-02-06 14:33:05 -05:00
Nick Mathewson
573ab70bfe Temporary clang-format configuration and script.
The format is the same as in my previous efforts here.

The script is a little tricky, since it invokes both clang-format
and codetool, and it makes sure that files do not have a changed
mtime unless there is actually some change in the file.
2020-02-06 14:33:05 -05:00
Nick Mathewson
5bd86b50b5 codetool: post-processor for clang-format
This code transformer makes a couple of changes that we want for our
source code, and can be expanded to handle more.
2020-02-06 14:04:19 -05:00
Nick Mathewson
b7bbff0c9c Merge remote-tracking branch 'public/practracker_regen_overbroad_2' 2020-02-06 08:27:12 -05:00
Nick Mathewson
5e963f97b1 practracker: use shutil.move so --regen will work on windows
On windows you can't os.rename() a file if the target filename
already exists.
2020-02-03 13:36:56 -05:00
Nick Mathewson
ec965ba98b practracker: integration tests for --regen and --regen-overbroad 2020-02-03 13:11:58 -05:00
Nick Mathewson
2542a24b63 practracker: add a --regen-overbroad option to remove overbroad exceptions.
Closes ticket 32372.
2020-02-03 13:02:12 -05:00
Christian Clauss
472d58372b
Remove unused import UserDict
Fixes: https://trac.torproject.org/projects/tor/ticket/32791#comment:3
Unblocks: #1686
2020-02-03 17:55:58 +02:00
cclauss
3208a74f90 Use print() function in both Python 2 and Python 3 2020-01-28 01:38:54 +01:00
Nick Mathewson
cda10a2e89 Git hooks: remove support for 0.2.9 directory layout.
We no longer will be pushing or committing things in branches that
use src/common or src/or.

Resolves ticket 32957.
2020-01-23 10:28:09 -05:00
David Goulet
3484608bda practracker: Make it happy
Signed-off-by: David Goulet <dgoulet@torproject.org>
2020-01-21 10:31:29 -05:00
teor
28c8c63de9 add_c_file: Replace asserts with exceptions
Closes 32962.
2020-01-20 13:27:40 +10:00
teor
2c75d4a8d0 add_c_file: Improve script documentation
Part of 32962.
2020-01-20 13:20:14 +10:00
teor
0418bc0cb2 add_c_file: Improve tor source directory checks
Check that the script isn't in a tor build directory, by looking
for a src/include.am file.

Part of 32962.
2020-01-20 13:04:02 +10:00
teor
eb336e23a6
add_c_file: Rename variables based on the type of path
* fname for generic file paths
* tor_fname for paths relative to the top-level tor directory
* src_fname for paths relative to tor's src directory

With prefixes as required to disambiguate different paths of the same
type.

Part of 32962.
2020-01-16 09:27:57 +10:00
teor
b828588499
add_c_file: Simplify some usage of srcdir_file()
Part of 32962.
2020-01-16 09:26:08 +10:00
teor
3d50efcf98
add_c_file: Document the paths used by each part of the script
Most paths are relative to the top-level tor directory, but the
paths in the C and H files are relative to tor's src directory.

Part of 32962.
2020-01-16 09:26:04 +10:00
teor
207d2625ed
add_c_file: Improve path handling and canonicalisation
* distinguish between paths relative to the top-level tor directory,
  and paths relative to tor's src directory
* canonicalise paths before using them
* check that the script is run from the top-level tor directory
* check that the file is being created in tor's src directory

Part of 32962.
2020-01-16 09:25:58 +10:00
Nick Mathewson
4f45ad1394
add_c_file: tolerate ./ in filenames. 2020-01-16 09:23:54 +10:00