nihilist/tor
1
0
Fork 0
mirror of https://gitlab.torproject.org/tpo/core/tor.git synced 2024-11-10 13:13:44 +01:00
Code Issues Packages Projects Releases Wiki Activity

added some markdown formatting

Browse Source
This commit is contained in:
tom lurge 2015-11-05 09:13:53 -05:00 committed by Nick Mathewson
parent 8976e739af
commit 617e0f8d26
7 changed files with 420 additions and 415 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

127
doc/HACKING/CodingStandards.md
View File

@ -1,16 +1,16 @@
Coding conventions for Tor Coding conventions for Tor
-------------------------- ==========================
tl;dr: tl;dr:
* Run configure with '--enable-gcc-warnings' - Run configure with `--enable-gcc-warnings`
* Run 'make check-spaces' to catch whitespace errors - Run `make check-spaces` to catch whitespace errors
* Document your functions - Document your functions
* Write unit tests - Write unit tests
* Add a file in 'changes' for your branch. - Add a file in `changes` for your branch.
Patch checklist Patch checklist
~~~~~~~~~~~~~~~ ---------------
If possible, send your patch as one of these (in descending order of If possible, send your patch as one of these (in descending order of
preference) preference)
@ -21,16 +21,16 @@ preference)
Did you remember... Did you remember...
- To build your code while configured with --enable-gcc-warnings? - To build your code while configured with `--enable-gcc-warnings`?
- To run "make check-spaces" on your code? - To run `make check-spaces` on your code?
- To run "make check-docs" to see whether all new options are on - To run `make check-docs` to see whether all new options are on
the manpage? the manpage?
- To write unit tests, as possible? - To write unit tests, as possible?
- To base your code on the appropriate branch? - To base your code on the appropriate branch?
- To include a file in the "changes" directory as appropriate? - To include a file in the `changes` directory as appropriate?
How we use Git branches How we use Git branches
----------------------- =======================
Each main development series (like 0.2.1, 0.2.2, etc) has its main work Each main development series (like 0.2.1, 0.2.2, etc) has its main work
applied to a single branch. At most one series can be the development series applied to a single branch. At most one series can be the development series
@ -54,13 +54,13 @@ you're working on a new feature, base it on the master branch.
How we log changes How we log changes
------------------ ==================
When you do a commit that needs a ChangeLog entry, add a new file to When you do a commit that needs a ChangeLog entry, add a new file to
the "changes" toplevel subdirectory. It should have the format of a the `changes` toplevel subdirectory. It should have the format of a
one-entry changelog section from the current ChangeLog file, as in one-entry changelog section from the current ChangeLog file, as in
o Major bugfixes: - Major bugfixes:
- Fix a potential buffer overflow. Fixes bug 99999; bugfix on - Fix a potential buffer overflow. Fixes bug 99999; bugfix on
0.3.1.4-beta. 0.3.1.4-beta.
@ -69,110 +69,115 @@ are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code
simplifications and refactoring. Then say what the change does. If simplifications and refactoring. Then say what the change does. If
it's a bugfix, mention what bug it fixes and when the bug was it's a bugfix, mention what bug it fixes and when the bug was
introduced. To find out which Git tag the change was introduced in, introduced. To find out which Git tag the change was introduced in,
you can use "git describe --contains <sha1 of commit>". you can use `git describe --contains <sha1 of commit>`.
If at all possible, try to create this file in the same commit where you are If at all possible, try to create this file in the same commit where you are
making the change. Please give it a distinctive name that no other branch will making the change. Please give it a distinctive name that no other branch will
use for the lifetime of your change. To verify the format of the changes file, use for the lifetime of your change. To verify the format of the changes file,
you can use "make check-changes". you can use `make check-changes`.
When we go to make a release, we will concatenate all the entries When we go to make a release, we will concatenate all the entries
in changes to make a draft changelog, and clear the directory. We'll in changes to make a draft changelog, and clear the directory. We'll
then edit the draft changelog into a nice readable format. then edit the draft changelog into a nice readable format.
What needs a changes file?:: What needs a changes file?
A not-exhaustive list: Anything that might change user-visible
* A not-exhaustive list: Anything that might change user-visible
behavior. Anything that changes internals, documentation, or the build behavior. Anything that changes internals, documentation, or the build
system enough that somebody could notice. Big or interesting code system enough that somebody could notice. Big or interesting code
rewrites. Anything about which somebody might plausibly wonder "when rewrites. Anything about which somebody might plausibly wonder "when
did that happen, and/or why did we do that" 6 months down the line. did that happen, and/or why did we do that" 6 months down the line.
Why use changes files instead of Git commit messages?:: Why use changes files instead of Git commit messages?
Git commit messages are written for developers, not users, and they
* Git commit messages are written for developers, not users, and they
are nigh-impossible to revise after the fact. are nigh-impossible to revise after the fact.
Why use changes files instead of entries in the ChangeLog?:: Why use changes files instead of entries in the ChangeLog?
Having every single commit touch the ChangeLog file tended to create
* Having every single commit touch the ChangeLog file tended to create
zillions of merge conflicts. zillions of merge conflicts.
Whitespace and C conformance Whitespace and C conformance
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ----------------------------
Invoke "make check-spaces" from time to time, so it can tell you about Invoke `make check-spaces` from time to time, so it can tell you about
deviations from our C whitespace style. Generally, we use: deviations from our C whitespace style. Generally, we use:
- Unix-style line endings - Unix-style line endings
- K&R-style indentation - K&R-style indentation
- No space before newlines - No space before newlines
- A blank line at the end of each file - A blank line at the end of each file
- Never more than one blank line in a row - Never more than one blank line in a row
- Always spaces, never tabs - Always spaces, never tabs
- No more than 79-columns per line. - No more than 79-columns per line.
- Two spaces per indent. - Two spaces per indent.
- A space between control keywords and their corresponding paren - A space between control keywords and their corresponding paren
"if (x)", "while (x)", and "switch (x)", never "if(x)", "while(x)", or `if (x)`, `while (x)`, and `switch (x)`, never `if(x)`, `while(x)`, or
"switch(x)". `switch(x)`.
- A space between anything and an open brace. - A space between anything and an open brace.
- No space between a function name and an opening paren. "puts(x)", not - No space between a function name and an opening paren. `puts(x)`, not
"puts (x)". `puts (x)`.
- Function declarations at the start of the line. - Function declarations at the start of the line.
We try hard to build without warnings everywhere. In particular, if you're We try hard to build without warnings everywhere. In particular, if you're
using gcc, you should invoke the configure script with the option using gcc, you should invoke the configure script with the option
"--enable-gcc-warnings". This will give a bunch of extra warning flags to `--enable-gcc-warnings`. This will give a bunch of extra warning flags to
the compiler, and help us find divergences from our preferred C style. the compiler, and help us find divergences from our preferred C style.
Functions to use; functions not to use Functions to use; functions not to use
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --------------------------------------
We have some wrapper functions like tor_malloc, tor_free, tor_strdup, and We have some wrapper functions like `tor_malloc`, `tor_free`, `tor_strdup`, and
tor_gettimeofday; use them instead of their generic equivalents. (They `tor_gettimeofday;` use them instead of their generic equivalents. (They
always succeed or exit.) always succeed or exit.)
You can get a full list of the compatibility functions that Tor provides by You can get a full list of the compatibility functions that Tor provides by
looking through src/common/util*.h and src/common/compat*.h. You can see the looking through `src/common/util*.h` and `src/common/compat*.h`. You can see the
available containers in src/common/containers*.h. You should probably available containers in `src/common/containers*.h`. You should probably
familiarize yourself with these modules before you write too much code, or familiarize yourself with these modules before you write too much code, or
else you'll wind up reinventing the wheel. else you'll wind up reinventing the wheel.
Use 'INLINE' instead of 'inline' -- it's a vestige of an old hack to make Use `INLINE` instead of `inline` -- it's a vestige of an old hack to make
sure that we worked on MSVC6. sure that we worked on MSVC6.
We don't use strcat or strcpy or sprintf of any of those notoriously broken We don't use `strcat` or `strcpy` or `sprintf` of any of those notoriously broken
old C functions. Use strlcat, strlcpy, or tor_snprintf/tor_asprintf instead. old C functions. Use `strlcat`, `strlcpy`, or `tor_snprintf/tor_asprintf` instead.
We don't call memcmp() directly. Use fast_memeq(), fast_memneq(), We don't call `memcmp()` directly. Use `fast_memeq()`, `fast_memneq()`,
tor_memeq(), or tor_memneq() for most purposes. `tor_memeq()`, or `tor_memneq()` for most purposes.
Functions not to write Functions not to write
~~~~~~~~~~~~~~~~~~~~~~ ----------------------
Try to never hand-write new code to parse or generate binary Try to never hand-write new code to parse or generate binary
formats. Instead, use trunnel if at all possible. See formats. Instead, use trunnel if at all possible. See
https://gitweb.torproject.org/trunnel.git/tree https://gitweb.torproject.org/trunnel.git/tree
for more information about trunnel. for more information about trunnel.
For information on adding new trunnel code to Tor, see src/trunnel/README For information on adding new trunnel code to Tor, see src/trunnel/README
Calling and naming conventions Calling and naming conventions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ------------------------------
Whenever possible, functions should return -1 on error and 0 on success. Whenever possible, functions should return -1 on error and 0 on success.
For multi-word identifiers, use lowercase words combined with For multi-word identifiers, use lowercase words combined with
underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and underscores. (e.g., `multi_word_identifier`). Use ALL_CAPS for macros and
constants. constants.
Typenames should end with "_t". Typenames should end with `_t`.
Function names should be prefixed with a module name or object name. (In Function names should be prefixed with a module name or object name. (In
general, code to manipulate an object should be a module with the same name general, code to manipulate an object should be a module with the same name
as the object, so it's hard to tell which convention is used.) as the object, so it's hard to tell which convention is used.)
Functions that do things should have imperative-verb names Functions that do things should have imperative-verb names
(e.g. buffer_clear, buffer_resize); functions that return booleans should (e.g. `buffer_clear`, `buffer_resize`); functions that return booleans should
have predicate names (e.g. buffer_is_empty, buffer_needs_resizing). have predicate names (e.g. `buffer_is_empty`, `buffer_needs_resizing`).
If you find that you have four or more possible return code values, it's If you find that you have four or more possible return code values, it's
probably time to create an enum. If you find that you are passing three or probably time to create an enum. If you find that you are passing three or
@ -180,16 +185,16 @@ more flags to a function, it's probably time to create a flags argument that
takes a bitfield. takes a bitfield.
What To Optimize What To Optimize
~~~~~~~~~~~~~~~~ ----------------
Don't optimize anything if it's not in the critical path. Right now, the Don't optimize anything if it's not in the critical path. Right now, the
critical path seems to be AES, logging, and the network itself. Feel free to critical path seems to be AES, logging, and the network itself. Feel free to
do your own profiling to determine otherwise. do your own profiling to determine otherwise.
Log conventions Log conventions
~~~~~~~~~~~~~~~ ---------------
https://www.torproject.org/docs/faq#LogLevel `https://www.torproject.org/docs/faq#LogLevel`
No error or warning messages should be expected during normal OR or OP No error or warning messages should be expected during normal OR or OP
operation. operation.
@ -206,7 +211,7 @@ option (B).
Doxygen comment conventions Doxygen comment conventions
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ---------------------------
Say what functions do as a series of one or more imperative sentences, as Say what functions do as a series of one or more imperative sentences, as
though you were telling somebody how to be the function. In other words, DO though you were telling somebody how to be the function. In other words, DO

67
doc/HACKING/GettingStarted.md
View File

@ -37,10 +37,11 @@ Getting your first patch into Tor
Once you've reached this point, here's what you need to know. Once you've reached this point, here's what you need to know.
1) Get the source. 1. Get the source.
We keep our source under version control in Git. To get the latest We keep our source under version control in Git. To get the latest
version, run version, run
git clone https://git.torproject.org/git/tor git clone https://git.torproject.org/git/tor
This will give you a checkout of the master branch. If you're This will give you a checkout of the master branch. If you're
@ -49,10 +50,11 @@ Once you've reached this point, here's what you need to know.
git checkout maint-0.2.7 git checkout maint-0.2.7
2) Find your way around the source 2. Find your way around the source
Our overall code structure is explained in the "torguts" documents, Our overall code structure is explained in the "torguts" documents,
currently at currently at
git clone https://git.torproject.org/user/nickm/torguts.git git clone https://git.torproject.org/user/nickm/torguts.git
Find a part of the code that looks interesting to you, and start Find a part of the code that looks interesting to you, and start
@ -64,14 +66,14 @@ Once you've reached this point, here's what you need to know.
If you see something that doesn't make sense, we love to get If you see something that doesn't make sense, we love to get
questions! questions!
3) Find something cool to hack on. 3. Find something cool to hack on.
You may already have a good idea of what you'd like to work on, or You may already have a good idea of what you'd like to work on, or
you might be looking for a way to contribute. you might be looking for a way to contribute.
Many people have gotten started by looking for an area where they Many people have gotten started by looking for an area where they
personally felt Tor was underperforming, and investigating ways to personally felt Tor was underperforming, and investigating ways to
fix it. If you're looking for ideas, you can head to our bug fix it. If you're looking for ideas, you can head to our bug
tracker at trac.torproject.org and look for tickets that have tracker at trac.torproject.org and look for tickets that have
received the "easy" tag: these are ones that developers think would received the "easy" tag: these are ones that developers think would
be pretty simple for a new person to work on. For a bigger be pretty simple for a new person to work on. For a bigger
@ -86,11 +88,12 @@ Once you've reached this point, here's what you need to know.
For your first patch, it is probably NOT a good idea to make For your first patch, it is probably NOT a good idea to make
something huge or invasive. In particular, you should probably something huge or invasive. In particular, you should probably
avoid: avoid:
* Major changes spread across many parts of the codebase.
* Major changes to programming practice or coding style.
* Huge new features or protocol changes.
4) Meet the developers! * Major changes spread across many parts of the codebase.
* Major changes to programming practice or coding style.
* Huge new features or protocol changes.
4. Meet the developers!
We discuss stuff on the tor-dev mailing list and on the #tor-dev We discuss stuff on the tor-dev mailing list and on the #tor-dev
IRC channel on OFTC. We're generally friendly and approachable, IRC channel on OFTC. We're generally friendly and approachable,
@ -103,7 +106,7 @@ Once you've reached this point, here's what you need to know.
better. The time might change in the future, but generally, better. The time might change in the future, but generally,
there's no bad time to talk, and ask us about patch ideas. there's no bad time to talk, and ask us about patch ideas.
5) Do you need to write a design proposal? 5. Do you need to write a design proposal?
If your idea is very large, or it will require a change to Tor's If your idea is very large, or it will require a change to Tor's
protocols, there needs to be a written design proposal before it protocols, there needs to be a written design proposal before it
@ -116,7 +119,7 @@ Once you've reached this point, here's what you need to know.
You might also like to look around the rest of that directory, to You might also like to look around the rest of that directory, to
see more about open and past proposed changes to Tor's behavior. see more about open and past proposed changes to Tor's behavior.
6) Writing your patch 6. Writing your patch
As you write your code, you'll probably want it to fit in with the As you write your code, you'll probably want it to fit in with the
standards of the rest of the Tor codebase so it will be easy for us standards of the rest of the Tor codebase so it will be easy for us
@ -127,7 +130,7 @@ Once you've reached this point, here's what you need to know.
components, remember to divide it into a series of Git commits. A components, remember to divide it into a series of Git commits. A
series of small changes is much easier to review than one big lump. series of small changes is much easier to review than one big lump.
7) Testing your patch 7. Testing your patch
We prefer that all new or modified code have unit tests for it to We prefer that all new or modified code have unit tests for it to
ensure that it runs correctly. Also, all code should actually be ensure that it runs correctly. Also, all code should actually be
@ -137,7 +140,7 @@ Once you've reached this point, here's what you need to know.
in Tor. If you'd like any help writing tests, just ask! We're in Tor. If you'd like any help writing tests, just ask! We're
glad to help out. glad to help out.
8) Submitting your patch 8. Submitting your patch
We review patches through tickets on our bugtracker at We review patches through tickets on our bugtracker at
trac.torproject.org. You can either upload your patches there, or trac.torproject.org. You can either upload your patches there, or
@ -149,7 +152,7 @@ Once you've reached this point, here's what you need to know.
you've done on trac, and then change the status of the ticket to you've done on trac, and then change the status of the ticket to
needs_review. needs_review.
9) Review, Revision, and Merge 9. Review, Revision, and Merge
With any luck, somebody will review your patch soon! If not, you With any luck, somebody will review your patch soon! If not, you
can ask on the IRC channel; sometimes we get really busy and take can ask on the IRC channel; sometimes we get really busy and take
@ -159,29 +162,29 @@ Once you've reached this point, here's what you need to know.
When your patch is reviewed, one of these things will happen: When your patch is reviewed, one of these things will happen:
* The reviewer will say "looks good to me" and your * The reviewer will say "looks good to me" and your
patch will get merged right into Tor. [Assuming we're not patch will get merged right into Tor. [Assuming we're not
in the middle of a code-freeze window. If the codebase is in the middle of a code-freeze window. If the codebase is
frozen, your patch will go into the next release series.] frozen, your patch will go into the next release series.]
* OR the reviewer will say "looks good, just needs some small * OR the reviewer will say "looks good, just needs some small
changes!" And then the reviewer will make those changes, changes!" And then the reviewer will make those changes,
and merge the modified patch into Tor. and merge the modified patch into Tor.
* OR the reviewer will say "Here are some questions and * OR the reviewer will say "Here are some questions and
comments," followed by a bunch of stuff that the reviewer comments," followed by a bunch of stuff that the reviewer
thinks should change in your code, or questions that the thinks should change in your code, or questions that the
reviewer has. reviewer has.
At this point, you might want to make the requested changes At this point, you might want to make the requested changes
yourself, and comment on the trac ticket once you have done yourself, and comment on the trac ticket once you have done
so. Or if you disagree with any of the comments, you should so. Or if you disagree with any of the comments, you should
say so! And if you won't have time to make some of the say so! And if you won't have time to make some of the
changes, you should say that too, so that other developers changes, you should say that too, so that other developers
will be able to pick up the unfinished portion will be able to pick up the unfinished portion.
Congratulations! You have now written your first patch, and gotten Congratulations! You have now written your first patch, and gotten
it integrated into mainline Tor. it integrated into mainline Tor.

179
doc/HACKING/HelpfulTools.md
View File

@ -1,159 +1,149 @@
Useful tools Useful tools
------------ ============
These aren't strictly necessary for hacking on Tor, but they can help track These aren't strictly necessary for hacking on Tor, but they can help track
down bugs. down bugs.
Jenkins Jenkins
~~~~~~~ -------
https://jenkins.torproject.org https://jenkins.torproject.org
Dmalloc Dmalloc
~~~~~~~ -------
The dmalloc library will keep track of memory allocation, so you can find out The dmalloc library will keep track of memory allocation, so you can find out
if we're leaking memory, doing any double-frees, or so on. if we're leaking memory, doing any double-frees, or so on.
dmalloc -l ~/dmalloc.log dmalloc -l -/dmalloc.log
(run the commands it tells you) (run the commands it tells you)
./configure --with-dmalloc ./configure --with-dmalloc
Valgrind Valgrind
~~~~~~~~ --------
valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor
(Note that if you get a zillion openssl warnings, you will also need to (Note that if you get a zillion openssl warnings, you will also need to
pass --undef-value-errors=no to valgrind, or rebuild your openssl pass `--undef-value-errors=no` to valgrind, or rebuild your openssl
with -DPURIFY.) with `-DPURIFY`.)
Coverity Coverity
~~~~~~~~ --------
Nick regularly runs the coverity static analyzer on the Tor codebase. Nick regularly runs the coverity static analyzer on the Tor codebase.
The preprocessor define __COVERITY__ is used to work around instances The preprocessor define `__COVERITY__` is used to work around instances
where coverity picks up behavior that we wish to permit. where coverity picks up behavior that we wish to permit.
clang Static Analyzer clang Static Analyzer
~~~~~~~~~~~~~~~~~~~~~ ---------------------
The clang static analyzer can be run on the Tor codebase using Xcode (WIP) The clang static analyzer can be run on the Tor codebase using Xcode (WIP)
or a command-line build. or a command-line build.
The preprocessor define __clang_analyzer__ is used to work around instances The preprocessor define `__clang_analyzer__` is used to work around instances
where clang picks up behavior that we wish to permit. where clang picks up behavior that we wish to permit.
clang Runtime Sanitizers clang Runtime Sanitizers
~~~~~~~~~~~~~~~~~~~~~~~~ ------------------------
To build the Tor codebase with the clang Address and Undefined Behavior To build the Tor codebase with the clang Address and Undefined Behavior
sanitizers, see the file contrib/clang/sanitize_blacklist.txt. sanitizers, see the file `contrib/clang/sanitize_blacklist.txt`.
Preprocessor workarounds for instances where clang picks up behavior that Preprocessor workarounds for instances where clang picks up behavior that
we wish to permit are also documented in the blacklist file. we wish to permit are also documented in the blacklist file.
Running lcov for unit test coverage Running lcov for unit test coverage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -----------------------------------
Lcov is a utility that generates pretty HTML reports of test code coverage. Lcov is a utility that generates pretty HTML reports of test code coverage.
To generate such a report: To generate such a report:
----- ./configure --enable-coverage
./configure --enable-coverage make
make make coverage-html
make coverage-html $BROWSER ./coverage_html/index.html
$BROWSER ./coverage_html/index.html
-----
This will run the tor unit test suite `./src/test/test` and generate the HTML This will run the tor unit test suite `./src/test/test` and generate the HTML
coverage code report under the directory ./coverage_html/. To change the coverage code report under the directory `./coverage_html/`. To change the
output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`. output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`.
Coverage diffs using lcov are not currently implemented, but are being Coverage diffs using lcov are not currently implemented, but are being
investigated (as of July 2014). investigated (as of July 2014).
Running the unit tests Running the unit tests
~~~~~~~~~~~~~~~~~~~~~~ ----------------------
To quickly run all the tests distributed with Tor: To quickly run all the tests distributed with Tor:
-----
make check make check
-----
To run the fast unit tests only: To run the fast unit tests only:
-----
make test make test
-----
To selectively run just some tests (the following can be combined To selectively run just some tests (the following can be combined
arbitrarily): arbitrarily):
-----
./src/test/test <name_of_test> [<name of test 2>] ... ./src/test/test <name_of_test> [<name of test 2>] ...
./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ... ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ...
./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]... ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]...
-----
To run all tests, including those based on Stem or Chutney: To run all tests, including those based on Stem or Chutney:
-----
make test-full make test-full
-----
To run all tests, including those based on Stem or Chutney that require a To run all tests, including those based on Stem or Chutney that require a
working connection to the internet: working connection to the internet:
-----
make test-full-online make test-full-online
-----
Running gcov for unit test coverage Running gcov for unit test coverage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -----------------------------------
----- ./configure --enable-coverage
./configure --enable-coverage make
make make check
make check # or--- make test-full ? make test-full-online?
# or--- make test-full ? make test-full-online? mkdir coverage-output
mkdir coverage-output ./scripts/test/coverage coverage-output
./scripts/test/coverage coverage-output
-----
(On OSX, you'll need to start with "--enable-coverage CC=clang".) (On OSX, you'll need to start with `--enable-coverage CC=clang`.)
Then, look at the .gcov files in coverage-output. '-' before a line means Then, look at the .gcov files in `coverage-output`. '-' before a line means
that the compiler generated no code for that line. '######' means that the that the compiler generated no code for that line. '######' means that the
line was never reached. Lines with numbers were called that number of times. line was never reached. Lines with numbers were called that number of times.
If that doesn't work: If that doesn't work:
* Try configuring Tor with --disable-gcc-hardening
* You might need to run 'make clean' after you run './configure'. * Try configuring Tor with `--disable-gcc-hardening`
* You might need to run `make clean` after you run `./configure`.
If you make changes to Tor and want to get another set of coverage results, If you make changes to Tor and want to get another set of coverage results,
you can run "make reset-gcov" to clear the intermediary gcov output. you can run `make reset-gcov` to clear the intermediary gcov output.
If you have two different "coverage-output" directories, and you want to see If you have two different `coverage-output` directories, and you want to see
a meaningful diff between them, you can run: a meaningful diff between them, you can run:
----- ./scripts/test/cov-diff coverage-output1 coverage-output2 | less
./scripts/test/cov-diff coverage-output1 coverage-output2 | less
-----
In this diff, any lines that were visited at least once will have coverage In this diff, any lines that were visited at least once will have coverage
"1". This lets you inspect what you (probably) really want to know: which "1". This lets you inspect what you (probably) really want to know: which
untested lines were changed? Are there any new untested lines? untested lines were changed? Are there any new untested lines?
Running integration tests Running integration tests
~~~~~~~~~~~~~~~~~~~~~~~~~ -------------------------
We have the beginnings of a set of scripts to run integration tests using We have the beginnings of a set of scripts to run integration tests using
Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and
run "make test-network". run `make test-network`.
We also have scripts to run integration tests using Stem. To try them, set We also have scripts to run integration tests using Stem. To try them, set
STEM_SOURCE_DIR to your Stem source directory, and run "test-stem". `STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`.
Profiling Tor with oprofile Profiling Tor with oprofile
~~~~~~~~~~~~~~~~~~~~~~~~~~~ ---------------------------
The oprofile tool runs (on Linux only!) to tell you what functions Tor is The oprofile tool runs (on Linux only!) to tell you what functions Tor is
spending its CPU time in, so we can identify performance bottlenecks. spending its CPU time in, so we can identify performance bottlenecks.
@ -165,30 +155,30 @@ Here are some basic instructions
- Build all the libraries you care about with debugging symbols - Build all the libraries you care about with debugging symbols
(probably you only care about libssl, maybe zlib and Libevent). (probably you only care about libssl, maybe zlib and Libevent).
- Copy this tor to a new directory - Copy this tor to a new directory
- Copy all the libraries it uses to that dir too (ldd ./tor will - Copy all the libraries it uses to that dir too (`ldd ./tor` will
tell you) tell you)
- Set LD_LIBRARY_PATH to include that dir. ldd ./tor should now - Set LD_LIBRARY_PATH to include that dir. `ldd ./tor` should now
show you it's using the libs in that dir show you it's using the libs in that dir
- Run that tor - Run that tor
- Reset oprofiles counters/start it - Reset oprofiles counters/start it
* "opcontrol --reset; opcontrol --start", if Nick remembers right. * `opcontrol --reset; opcontrol --start`, if Nick remembers right.
- After a while, have it dump the stats on tor and all the libs - After a while, have it dump the stats on tor and all the libs
in that dir you created. in that dir you created.
* "opcontrol --dump;" * `opcontrol --dump;`
* "opreport -l that_dir/*" * `opreport -l that_dir/*`
- Profit - Profit
Generating and analyzing a callgraph Generating and analyzing a callgraph
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ------------------------------------
1. Run ./scripts/maint/generate_callgraph.sh . This will generate a 1. Run `./scripts/maint/generate_callgraph.sh`. This will generate a
bunch of files in a new ./callgraph directory. bunch of files in a new ./callgraph directory.
2. Run ./scripts/maint/analyze_callgraph.py callgraph/src/*/* . This 2. Run `./scripts/maint/analyze_callgraph.py callgraph/src/*/*`. This
will do a lot of graph operations and then dump out a new will do a lot of graph operations and then dump out a new
"callgraph.pkl" file, containing data in Python's "pickle" format. `callgraph.pkl` file, containing data in Python's 'pickle' format.
3. Run ./scripts/maint/display_callgraph.py . It will display: 3. Run `./scripts/maint/display_callgraph.py`. It will display:
- the number of functions reachable from each function. - the number of functions reachable from each function.
- all strongly-connnected components in the Tor callgraph - all strongly-connnected components in the Tor callgraph
- the largest bottlenecks in the largest SCC in the Tor callgraph. - the largest bottlenecks in the largest SCC in the Tor callgraph.
@ -197,11 +187,11 @@ Note that currently the callgraph generator can't detect calls that pass
through function pointers. through function pointers.
Getting emacs to edit Tor source properly Getting emacs to edit Tor source properly
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -----------------------------------------
Nick likes to put the following snippet in his .emacs file: Nick likes to put the following snippet in his .emacs file:
-----
(add-hook 'c-mode-hook (add-hook 'c-mode-hook
(lambda () (lambda ()
(font-lock-mode 1) (font-lock-mode 1)
@ -221,9 +211,9 @@ Nick likes to put the following snippet in his .emacs file:
(set-variable 'c-basic-offset 8) (set-variable 'c-basic-offset 8)
(set-variable 'tab-width 8)) (set-variable 'tab-width 8))
)))) ))))
-----
You'll note that it defaults to showing all trailing whitespace. The "cond"
You'll note that it defaults to showing all trailing whitespace. The `cond`
test detects whether the file is one of a few C free software projects that I test detects whether the file is one of a few C free software projects that I
often edit, and sets up the indentation level and tab preferences to match often edit, and sets up the indentation level and tab preferences to match
what they want. what they want.
@ -233,26 +223,27 @@ patterns to match where you keep your Tor files.
If you use emacs for editing Tor and nothing else, you could always just say: If you use emacs for editing Tor and nothing else, you could always just say:
-----
(add-hook 'c-mode-hook (add-hook 'c-mode-hook
(lambda () (lambda ()
(font-lock-mode 1) (font-lock-mode 1)
(set-variable 'show-trailing-whitespace t) (set-variable 'show-trailing-whitespace t)
(set-variable 'indent-tabs-mode nil) (set-variable 'indent-tabs-mode nil)
(set-variable 'c-basic-offset 2))) (set-variable 'c-basic-offset 2)))
-----
There is probably a better way to do this. No, we are probably not going There is probably a better way to do this. No, we are probably not going
to clutter the files with emacs stuff. to clutter the files with emacs stuff.
Doxygen Doxygen
~~~~~~~ -------
We use the 'doxygen' utility to generate documentation from our We use the 'doxygen' utility to generate documentation from our
source code. Here's how to use it: source code. Here's how to use it:
1. Begin every file that should be documented with 1. Begin every file that should be documented with
/** /**
* \file filename.c * \file filename.c
* \brief Short description of the file. * \brief Short description of the file.
@ -279,24 +270,24 @@ source code. Here's how to use it:
* \endcode * \endcode
*/ */
3. Make sure to escape the characters "<", ">", "\", "%" and "#" as "\<", 3. Make sure to escape the characters `<`, `>`, `\`, `%` and `#` as `\<`,
"\>", "\\", "\%", and "\#". `\>`, `\\`, `\%` and `\#`.
4. To document structure members, you can use two forms: 4. To document structure members, you can use two forms:
struct foo { struct foo {
/** You can put the comment before an element; */ /** You can put the comment before an element; */
int a; int a;
int b; /**< Or use the less-than symbol to put the comment int b; /**< Or use the less-than symbol to put the comment
* after the element. */ * after the element. */
}; };
5. To generate documentation from the Tor source code, type: 5. To generate documentation from the Tor source code, type:
$ doxygen -g $ doxygen -g
To generate a file called 'Doxyfile'. Edit that file and run to generate a file called `Doxyfile`. Edit that file and run
'doxygen' to generate the API documentation. `doxygen` to generate the API documentation.
6. See the Doxygen manual for more information; this summary just 6. See the Doxygen manual for more information; this summary just
scratches the surface. scratches the surface.

60
doc/HACKING/HowToReview.md
View File

@ -15,71 +15,71 @@ Top-level smell-checks
(Difficulty: easy) (Difficulty: easy)
Does it compile with --enable-gcc-warnings? - Does it compile with `--enable-gcc-warnings`?
Does 'make check-spaces' pass? - Does `make check-spaces` pass?
Does it have a reasonable amount of tests? Do they pass? Do they leak - Does it have a reasonable amount of tests? Do they pass? Do they leak
memory? memory?
Do all the new functions, global variables, types, and structure members have - Do all the new functions, global variables, types, and structure members have
documentation? documentation?
Do all the functions, global variables, types, and structure members with - Do all the functions, global variables, types, and structure members with
modified behavior have modified documentation? modified behavior have modified documentation?
Do all the new torrc options have documentation? - Do all the new torrc options have documentation?
If this changes Tor's behavior on the wire, is there a design proposal? - If this changes Tor's behavior on the wire, is there a design proposal?
Let's look at the code! Let's look at the code!
----------------------- -----------------------
Does the code conform to CodingStandards.txt? - Does the code conform to CodingStandards.txt?
Does the code leak memory? - Does the code leak memory?
If two or more pointers ever point to the same object, is it clear which - If two or more pointers ever point to the same object, is it clear which
pointer "owns" the object? pointer "owns" the object?
Are all allocated resources freed? - Are all allocated resources freed?
Are all pointers that should be const, const? - Are all pointers that should be const, const?
Are #defines used for 'magic' numbers? - Are `#defines` used for 'magic' numbers?
Can you understand what the code is trying to do? - Can you understand what the code is trying to do?
Can you convince yourself that the code really does that? - Can you convince yourself that the code really does that?
Is there duplicated code that could be turned into a function? - Is there duplicated code that could be turned into a function?
Let's look at the documentation! Let's look at the documentation!
-------------------------------- --------------------------------
Does the documentation confirm to CodingStandards.txt? - Does the documentation confirm to CodingStandards.txt?
Does it make sense? - Does it make sense?
Can you predict what the function will do from its documentation? - Can you predict what the function will do from its documentation?
Let's think about security! Let's think about security!
--------------------------- ---------------------------
If there are any arrays, buffers, are you 100% sure that they cannot - If there are any arrays, buffers, are you 100% sure that they cannot
overflow? overflow?
If there is any integer math, can it overflow or underflow? - If there is any integer math, can it overflow or underflow?
If there are any allocations, are you sure there are corresponding - If there are any allocations, are you sure there are corresponding
deallocations? deallocations?
Is there a safer pattern that could be used in any case? - Is there a safer pattern that could be used in any case?
Have they used one of the Forbidden Functions? - Have they used one of the Forbidden Functions?
(Also see your favorite secure C programming guides.) (Also see your favorite secure C programming guides.)

28
doc/HACKING/README.1st.md
View File

@ -5,46 +5,48 @@ In this directory
This directory has helpful information about what you need to know to This directory has helpful information about what you need to know to
hack on Tor! hack on Tor!
First, read 'GettingStarted.md' to learn how to get a start in Tor First, read `GettingStarted.md` to learn how to get a start in Tor
development. development.
If you've decided to write a patch, 'CodingStandards.txt' will give If you've decided to write a patch, `CodingStandards.txt` will give
you a bunch of information about how we structure our code. you a bunch of information about how we structure our code.
It's important to get code right! Reading 'WritingTests.md' will It's important to get code right! Reading `WritingTests.md` will
tell you how to write and run tests in the Tor codebase. tell you how to write and run tests in the Tor codebase.
There are a bunch of other programs we use to help maintain and There are a bunch of other programs we use to help maintain and
develop the codebase: 'HelpfulTools.md' can tell you how to use them develop the codebase: `HelpfulTools.md` can tell you how to use them
with Tor. with Tor.
If it's your job to put out Tor releases, see 'ReleasingTor.md' so If it's your job to put out Tor releases, see `ReleasingTor.md` so
that you don't miss any steps! that you don't miss any steps!
----------------------- -----------------------
For full information on how Tor is supposed to work, look at the files in For full information on how Tor is supposed to work, look at the files in
https://gitweb.torproject.org/torspec.git/tree `https://gitweb.torproject.org/torspec.git/tree`.
For an explanation of how to change Tor's design to work differently, look at For an explanation of how to change Tor's design to work differently, look at
https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt `https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt`.
For the latest version of the code, get a copy of git, and For the latest version of the code, get a copy of git, and
git clone https://git.torproject.org/git/tor git clone https://git.torproject.org/git/tor
We talk about Tor on the tor-talk mailing list. Design proposals and We talk about Tor on the `tor-talk` mailing list. Design proposals and
discussion belong on the tor-dev mailing list. We hang around on discussion belong on the `tor-dev` mailing list. We hang around on
irc.oftc.net, with general discussion happening on #tor and development irc.oftc.net, with general discussion happening on #tor and development
happening on #tor-dev. happening on `#tor-dev`.
The other files in this "HACKING" directory may also be useful as you The other files in this `HACKING` directory may also be useful as you
get started working with Tor. get started working with Tor.
Happy hacking! Happy hacking!
-----------------------
XXXXX also describe XXXXX also describe
doc/HACKING/WritingTests.md doc/HACKING/WritingTests.md

168
doc/HACKING/ReleasingTor.md
View File

@ -4,122 +4,126 @@ Putting out a new release
Here are the steps Roger takes when putting out a new Tor release: Here are the steps Roger takes when putting out a new Tor release:
1) Use it for a while, as a client, as a relay, as a hidden service, 1. Use it for a while, as a client, as a relay, as a hidden service,
and as a directory authority. See if it has any obvious bugs, and and as a directory authority. See if it has any obvious bugs, and
resolve those. resolve those.
1.5) As applicable, merge the maint-X branch into the release-X branch. As applicable, merge the `maint-X` branch into the `release-X` branch.
2) Gather the changes/* files into a changelog entry, rewriting many 2. Gather the `changes/*` files into a changelog entry, rewriting many
of them and reordering to focus on what users and funders would find of them and reordering to focus on what users and funders would find
interesting and understandable. interesting and understandable.
2.1) Make sure that everything that wants a bug number has one. 1. Make sure that everything that wants a bug number has one.
Make sure that everything which is a bugfix says what version Make sure that everything which is a bugfix says what version
it was a bugfix on. it was a bugfix on.
2.2) Concatenate them.
2.3) Sort them by section. Within each section, sort by "version it's
a bugfix on", else by numerical ticket order.
2.4) Clean them up: 2. Concatenate them.
Standard idioms: 3. Sort them by section. Within each section, sort by "version it's
"Fixes bug 9999; bugfix on 0.3.3.3-alpha." a bugfix on", else by numerical ticket order.
One space after a period. 4. Clean them up:
Make stuff very terse Standard idioms:
`Fixes bug 9999; bugfix on 0.3.3.3-alpha.`
Make sure each section name ends with a colon One space after a period.
Describe the user-visible problem right away Make stuff very terse
Mention relevant config options by name. If they're rare or unusual, Make sure each section name ends with a colon
remind people what they're for
Avoid starting lines with open-paren Describe the user-visible problem right away
Present and imperative tense: not past. Mention relevant config options by name. If they're rare or unusual,
remind people what they're for
'Relays', not 'servers' or 'nodes' or 'Tor relays'. Avoid starting lines with open-paren
"Stop FOOing", not "Fix a bug where we would FOO". Present and imperative tense: not past.
Try not to let any given section be longer than about a page. Break up 'Relays', not 'servers' or 'nodes' or 'Tor relays'.
long sections into subsections by some sort of common subtopic. This
guideline is especially important when organizing Release Notes for
new stable releases.
If a given changes stanza showed up in a different release (e.g. "Stop FOOing", not "Fix a bug where we would FOO".
maint-0.2.1), be sure to make the stanzas identical (so people can
distinguish if these are the same change).
2.5) Merge them in. Try not to let any given section be longer than about a page. Break up
long sections into subsections by some sort of common subtopic. This
guideline is especially important when organizing Release Notes for
new stable releases.
2.6) Clean everything one last time. If a given changes stanza showed up in a different release (e.g.
maint-0.2.1), be sure to make the stanzas identical (so people can
distinguish if these are the same change).
2.7) Run ./scripts/maint/format_changelog.py to make it prettier. 5. Merge them in.
3) Compose a short release blurb to highlight the user-facing 6. Clean everything one last time.
changes. Insert said release blurb into the ChangeLog stanza. If it's
a stable release, add it to the ReleaseNotes file too. If we're adding
to a release-0.2.x branch, manually commit the changelogs to the later
git branches too.
4) In maint-0.2.x, bump the version number in configure.ac and run 7. Run `./scripts/maint/format_changelog.py` to make it prettier.
scripts/maint/updateVersions.pl to update version numbers in other
places, and commit. Then merge maint-0.2.x into release-0.2.x.
(NOTE: To bump the version number, edit configure.ac, and then run 3. Compose a short release blurb to highlight the user-facing
either make, or 'perl scripts/maint/updateVersions.pl', depending on changes. Insert said release blurb into the ChangeLog stanza. If it's
a stable release, add it to the ReleaseNotes file too. If we're adding
to a release-0.2.x branch, manually commit the changelogs to the later
git branches too.
4. In `maint-0.2.x`, bump the version number in `configure.ac` and run
`scripts/maint/updateVersions.pl` to update version numbers in other
places, and commit. Then merge `maint-0.2.x` into `release-0.2.x`.
(NOTE: To bump the version number, edit `configure.ac`, and then run
either `make`, or `perl scripts/maint/updateVersions.pl`, depending on
your version.) your version.)
5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait 5. Make dist, put the tarball up somewhere, and tell `#tor` about it. Wait
a while to see if anybody has problems building it. Try to get Sebastian a while to see if anybody has problems building it. Try to get Sebastian
or somebody to try building it on Windows. or somebody to try building it on Windows.
6) Get at least two of weasel/arma/Sebastian to put the new version number 6. Get at least two of weasel/arma/Sebastian to put the new version number
in their approved versions list. in their approved versions list.
7) Sign the tarball, then sign and push the git tag: 7. Sign the tarball, then sign and push the git tag:
gpg -ba <the_tarball>
git tag -u <keyid> tor-0.2.x.y-status
git push origin tag tor-0.2.x.y-status
8a) scp the tarball and its sig to the dist website, i.e. gpg -ba <the_tarball>
/srv/dist-master.torproject.org/htdocs/ on dist-master. When you want git tag -u <keyid> tor-0.2.x.y-status
it to go live, you run "static-update-component dist.torproject.org" git push origin tag tor-0.2.x.y-status
on dist-master.
8b) Edit "include/versions.wmi" and "Makefile" to note the new version. 8. scp the tarball and its sig to the dist website, i.e.
`/srv/dist-master.torproject.org/htdocs/` on dist-master. When you want
it to go live, you run "static-update-component dist.torproject.org"
on dist-master.
9) Email the packagers (cc'ing tor-assistants) that a new tarball is up. Edit `include/versions.wmi` and `Makefile` to note the new version.
9. Email the packagers (cc'ing tor-assistants) that a new tarball is up.
The current list of packagers is: The current list of packagers is:
{weasel,gk,mikeperry} at torproject dot org
{blueness} at gentoo dot org
{paul} at invizbox dot io
{ondrej.mikle} at gmail dot com
{lfleischer} at archlinux dot org
{tails-dev} at doum dot org
10) Add the version number to Trac. To do this, go to Trac, log in, - {weasel,gk,mikeperry} at torproject dot org
select "Admin" near the top of the screen, then select "Versions" from - {blueness} at gentoo dot org
the menu on the left. At the right, there will be an "Add version" - {paul} at invizbox dot io
box. By convention, we enter the version in the form "Tor: - {ondrej.mikle} at gmail dot com
0.2.2.23-alpha" (or whatever the version is), and we select the date as - {lfleischer} at archlinux dot org
the date in the ChangeLog. - {tails-dev} at doum dot org
11) Forward-port the ChangeLog. 10. Add the version number to Trac. To do this, go to Trac, log in,
select "Admin" near the top of the screen, then select "Versions" from
the menu on the left. At the right, there will be an "Add version"
box. By convention, we enter the version in the form "Tor:
0.2.2.23-alpha" (or whatever the version is), and we select the date as
the date in the ChangeLog.
12) Wait up to a day or two (for a development release), or until most 11. Forward-port the ChangeLog.
packages are up (for a stable release), and mail the release blurb and
changelog to tor-talk or tor-announce.
(We might be moving to faster announcements, but don't announce until 12. Wait up to a day or two (for a development release), or until most
the website is at least updated.) packages are up (for a stable release), and mail the release blurb and
changelog to tor-talk or tor-announce.
13) If it's a stable release, bump the version number in the maint-x.y.z (We might be moving to faster announcements, but don't announce until
branch to "newversion-dev", and do a "merge -s ours" merge to avoid the website is at least updated.)
taking that change into master. Do a similar 'merge -s theirs'
13. If it's a stable release, bump the version number in the `maint-x.y.z`
branch to "newversion-dev", and do a `merge -s ours` merge to avoid
taking that change into master. Do a similar `merge -s theirs`
merge to get the change (and only that change) into release. (Some merge to get the change (and only that change) into release. (Some
of the build scripts require that maint merge cleanly into release.) of the build scripts require that maint merge cleanly into release.)

192
doc/HACKING/WritingTests.md
View File

@ -22,92 +22,92 @@ keep from introducing bugs. The major ones are:
How to run these tests How to run these tests
---------------------- ----------------------
=== The easy version ### The easy version
To run all the tests that come bundled with Tor, run "make check" To run all the tests that come bundled with Tor, run `make check`.
To run the Stem tests as well, fetch stem from the git repository, To run the Stem tests as well, fetch stem from the git repository,
set STEM_SOURCE_DIR to the checkout, and run "make test-stem". set `STEM_SOURCE_DIR` to the checkout, and run `make test-stem`.
To run the Chutney tests as well, fetch chutney from the git repository, To run the Chutney tests as well, fetch chutney from the git repository,
set CHUTNEY_PATH to the checkout, and run "make test-network". set `CHUTNEY_PATH` to the checkout, and run `make test-network`.
To run all of the above, run "make test-full". To run all of the above, run `make test-full`.
To run all of the above, plus tests that require a working connection to the To run all of the above, plus tests that require a working connection to the
internet, run "make test-full-online". internet, run `make test-full-online`.
=== Running particular subtests ### Running particular subtests
The Tor unit tests are divided into separate programs and a couple of The Tor unit tests are divided into separate programs and a couple of
bundled unit test programs. bundled unit test programs.
Separate programs are easy. For example, to run the memwipe tests in Separate programs are easy. For example, to run the memwipe tests in
isolation, you just run ./src/test/test-memwipe . isolation, you just run `./src/test/test-memwipe`.
To run tests within the unit test programs, you can specify the name To run tests within the unit test programs, you can specify the name
of the test. The string ".." can be used as a wildcard at the end of the of the test. The string ".." can be used as a wildcard at the end of the
test name. For example, to run all the cell format tests, enter test name. For example, to run all the cell format tests, enter
"./src/test/test cellfmt/..". To run `./src/test/test cellfmt/..`. To run
Many tests that need to mess with global state run in forked subprocesses in Many tests that need to mess with global state run in forked subprocesses in
order to keep from contaminating one another. But when debugging a failing test, order to keep from contaminating one another. But when debugging a failing test,
you might want to run it without forking a subprocess. To do so, use the you might want to run it without forking a subprocess. To do so, use the
"--no-fork" option with a single test. (If you specify it along with `--no-fork` option with a single test. (If you specify it along with
multiple tests, they might interfere.) multiple tests, they might interfere.)
You can turn on logging in the unit tests by passing one of "--debug", You can turn on logging in the unit tests by passing one of `--debug`,
"--info", "--notice", or "--warn". By default only errors are displayed. `--info`, `--notice`, or `--warn`. By default only errors are displayed.
Unit tests are divided into "./src/test/test" and "./src/test/test-slow". Unit tests are divided into `./src/test/test` and `./src/test/test-slow`.
The former are those that should finish in a few seconds; the latter tend to The former are those that should finish in a few seconds; the latter tend to
take more time, and may include CPU-intensive operations, deliberate delays, take more time, and may include CPU-intensive operations, deliberate delays,
and stuff like that. and stuff like that.
=== Finding test coverage ### Finding test coverage
Test coverage is a measurement of which lines your tests actually visit. Test coverage is a measurement of which lines your tests actually visit.
When you configure Tor with the --enable-coverage option, it should When you configure Tor with the `--enable-coverage` option, it should
build with support for coverage in the unit tests, and in a special build with support for coverage in the unit tests, and in a special
"tor-cov" binary. `tor-cov` binary.
Then, run the tests you'd like to see coverage from. If you have old Then, run the tests you'd like to see coverage from. If you have old
coverage output, you may need to run "reset-gcov" first. coverage output, you may need to run `reset-gcov` first.
Now you've got a bunch of files scattered around your build directories Now you've got a bunch of files scattered around your build directories
called "*.gcda". In order to extract the coverage output from them, make a called `*.gcda`. In order to extract the coverage output from them, make a
temporary directory for them and run "./scripts/test/coverage ${TMPDIR}", temporary directory for them and run `./scripts/test/coverage ${TMPDIR}`,
where ${TMPDIR} is the temporary directory you made. This will create a where `${TMPDIR}` is the temporary directory you made. This will create a
".gcov" file for each source file under tests, containing that file's source `.gcov` file for each source file under tests, containing that file's source
annotated with the number of times the tests hit each line. (You'll need to annotated with the number of times the tests hit each line. (You'll need to
have gcov installed.) have gcov installed.)
You can get a summary of the test coverage for each file by running You can get a summary of the test coverage for each file by running
"./scripts/test/cov-display ${TMPDIR}/*" . Each line lists the file's name, `./scripts/test/cov-display ${TMPDIR}/*` . Each line lists the file's name,
the number of uncovered lines, the number of uncovered lines, and the the number of uncovered lines, the number of uncovered lines, and the
coverage percentage. coverage percentage.
For a summary of the test coverage for each _function_, run For a summary of the test coverage for each _function_, run
"./scripts/test/cov-display -f ${TMPDIR}/*" . `./scripts/test/cov-display -f ${TMPDIR}/*`.
=== Comparing test coverage ### Comparing test coverage
Sometimes it's useful to compare test coverage for a branch you're writing to Sometimes it's useful to compare test coverage for a branch you're writing to
coverage from another branch (such as git master, for example). But you coverage from another branch (such as git master, for example). But you
can't run "diff" on the two coverage outputs directly, since the actual can't run `diff` on the two coverage outputs directly, since the actual
number of times each line is executed aren't so important, and aren't wholly number of times each line is executed aren't so important, and aren't wholly
deterministic. deterministic.
Instead, follow the instructions above for each branch, creating a separate Instead, follow the instructions above for each branch, creating a separate
temporary directory for each. Then, run "./scripts/test/cov-diff ${D1} temporary directory for each. Then, run `./scripts/test/cov-diff ${D1}
${D2}", where D1 and D2 are the directories you want to compare. This will ${D2}`, where D1 and D2 are the directories you want to compare. This will
produce a diff of the two directories, with all lines normalized to be either produce a diff of the two directories, with all lines normalized to be either
covered or uncovered. covered or uncovered.
To count new or modified uncovered lines in D2, you can run: To count new or modified uncovered lines in D2, you can run:
"./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' |wc -l ./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' | wc -l
What kinds of test should I write? What kinds of test should I write?
@ -132,18 +132,18 @@ Unit and regression tests: Does this function do what it's supposed to?
Most of Tor's unit tests are made using the "tinytest" testing framework. Most of Tor's unit tests are made using the "tinytest" testing framework.
You can see a guide to using it in the tinytest manual at You can see a guide to using it in the tinytest manual at
https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md
To add a new test of this kind, either edit an existing C file in src/test/, To add a new test of this kind, either edit an existing C file in `src/test/`,
or create a new C file there. Each test is a single function that must or create a new C file there. Each test is a single function that must
be indexed in the table at the end of the file. We use the label "done:" as be indexed in the table at the end of the file. We use the label "done:" as
a cleanup point for all test functions. a cleanup point for all test functions.
(Make sure you read tinytest-manual.md before proceeding.) (Make sure you read `tinytest-manual.md` before proceeding.)
I use the term "unit test" and "regression tests" very sloppily here. I use the term "unit test" and "regression tests" very sloppily here.
=== A simple example ### A simple example
Here's an example of a test function for a simple function in util.c: Here's an example of a test function for a simple function in util.c:
@ -172,20 +172,20 @@ Here's an example of a test function for a simple function in util.c:
This should look pretty familiar to you if you've read the tinytest This should look pretty familiar to you if you've read the tinytest
manual. One thing to note here is that we use the testing-specific manual. One thing to note here is that we use the testing-specific
function "get_fname" to generate a file with respect to a temporary function `get_fname` to generate a file with respect to a temporary
directory that the tests use. You don't need to delete the file; directory that the tests use. You don't need to delete the file;
it will get removed when the tests are done. it will get removed when the tests are done.
Also note our use of OP_EQ instead of == in the tt_int_op() calls. Also note our use of `OP_EQ` instead of `==` in the `tt_int_op()` calls.
We define OP_* macros to use instead of the binary comparison We define `OP_*` macros to use instead of the binary comparison
operators so that analysis tools can more easily parse our code. operators so that analysis tools can more easily parse our code.
(Coccinelle really hates to see == used as a macro argument.) (Coccinelle really hates to see `==` used as a macro argument.)
Finally, remember that by convention, all *_free() functions that Finally, remember that by convention, all `*_free()` functions that
Tor defines are defined to accept NULL harmlessly. Thus, you don't Tor defines are defined to accept NULL harmlessly. Thus, you don't
need to say "if (contents)" in the cleanup block. need to say `if (contents)` in the cleanup block.
=== Exposing static functions for testing ### Exposing static functions for testing
Sometimes you need to test a function, but you don't want to expose Sometimes you need to test a function, but you don't want to expose
it outside its usual module. it outside its usual module.
@ -193,20 +193,20 @@ it outside its usual module.
To support this, Tor's build system compiles a testing version of To support this, Tor's build system compiles a testing version of
each module, with extra identifiers exposed. If you want to each module, with extra identifiers exposed. If you want to
declare a function as static but available for testing, use the declare a function as static but available for testing, use the
macro "STATIC" instead of "static". Then, make sure there's a macro `STATIC` instead of `static`. Then, make sure there's a
macro-protected declaration of the function in the module's header. macro-protected declaration of the function in the module's header.
For example, crypto_curve25519.h contains: For example, `crypto_curve25519.h` contains:
#ifdef CRYPTO_CURVE25519_PRIVATE #ifdef CRYPTO_CURVE25519_PRIVATE
STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret, STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret,
const uint8_t *basepoint); const uint8_t *basepoint);
#endif #endif
The crypto_curve25519.c file and the test_crypto.c file both define The `crypto_curve25519.c` file and the `test_crypto.c` file both define
CRYPTO_CURVE25519_PRIVATE, so they can see this declaration. `CRYPTO_CURVE25519_PRIVATE`, so they can see this declaration.
=== Mock functions for testing in isolation ### Mock functions for testing in isolation
Often we want to test that a function works right, but the function to Often we want to test that a function works right, but the function to
be tested depends on other functions whose behavior is hard to observe, be tested depends on other functions whose behavior is hard to observe,
@ -216,7 +216,7 @@ To write tests for this case, you can replace the underlying functions
with testing stubs while your unit test is running. You need to declare with testing stubs while your unit test is running. You need to declare
the underlying function as 'mockable', as follows: the underlying function as 'mockable', as follows:
MOCK_DECL(returntype, functionname, (argument list)); MOCK_DECL(returntype, functionname, (argument list));
and then later implement it as: and then later implement it as:
@ -229,7 +229,7 @@ For example, if you had a 'connect to remote server' function, you could
declare it as: declare it as:
MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status)); MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status));
When you declare a function this way, it will be declared as normal in When you declare a function this way, it will be declared as normal in
regular builds, but when the module is built for testing, it is declared regular builds, but when the module is built for testing, it is declared
@ -238,16 +238,16 @@ as a function pointer initialized to the actual implementation.
In your tests, if you want to override the function with a temporary In your tests, if you want to override the function with a temporary
replacement, you say: replacement, you say:
MOCK(functionname, replacement_function_name); MOCK(functionname, replacement_function_name);
And later, you can restore the original function with: And later, you can restore the original function with:
UNMOCK(functionname); UNMOCK(functionname);
For more information, see the definitions of this mocking logic in For more information, see the definitions of this mocking logic in
testsupport.h. `testsupport.h`.
=== Okay but what should my tests actually do? ### Okay but what should my tests actually do?
We talk above about "test coverage" -- making sure that your tests visit We talk above about "test coverage" -- making sure that your tests visit
every line of code, or every branch of code. But visiting the code isn't every line of code, or every branch of code. But visiting the code isn't
@ -267,11 +267,11 @@ cases and failure csaes.
For example, consider testing this function: For example, consider testing this function:
/** Remove all elements E from sl such that E==element. Preserve /** Remove all elements E from sl such that E==element. Preserve
* the order of any elements before E, but elements after E can be * the order of any elements before E, but elements after E can be
* rearranged. * rearranged.
*/ */
void smartlist_remove(smartlist_t *sl, const void *element); void smartlist_remove(smartlist_t *sl, const void *element);
In order to test it well, you should write tests for at least all of the In order to test it well, you should write tests for at least all of the
following cases. (These would be black-box tests, since we're only looking following cases. (These would be black-box tests, since we're only looking
@ -298,19 +298,19 @@ When you consider edge cases, you might try:
Now let's look at the implementation: Now let's look at the implementation:
void void
smartlist_remove(smartlist_t *sl, const void *element) smartlist_remove(smartlist_t *sl, const void *element)
{ {
int i; int i;
if (element == NULL) if (element == NULL)
return; return;
for (i=0; i < sl->num_used; i++) for (i=0; i < sl->num_used; i++)
if (sl->list[i] == element) { if (sl->list[i] == element) {
sl->list[i] = sl->list[--sl->num_used]; /* swap with the end */ sl->list[i] = sl->list[--sl->num_used]; /* swap with the end */
i--; /* so we process the new i'th element */ i--; /* so we process the new i'th element */
sl->list[sl->num_used] = NULL; sl->list[sl->num_used] = NULL;
} }
} }
Based on the implementation, we now see three more edge cases to test: Based on the implementation, we now see three more edge cases to test:
@ -319,14 +319,14 @@ Based on the implementation, we now see three more edge cases to test:
* Removing an element from a position other than the end of the list. * Removing an element from a position other than the end of the list.
=== What should my tests NOT do? ### What should my tests NOT do?
Tests shouldn't require a network connection. Tests shouldn't require a network connection.
Whenever possible, tests shouldn't take more than a second. Put the test Whenever possible, tests shouldn't take more than a second. Put the test
into test/slow if it genuinely needs to be run. into test/slow if it genuinely needs to be run.
Tests should not alter global state unless they run with TT_FORK: Tests Tests should not alter global state unless they run with `TT_FORK`: Tests
should not require other tests to be run before or after them. should not require other tests to be run before or after them.
Tests should not leak memory or other resources. To find out if your tests Tests should not leak memory or other resources. To find out if your tests
@ -338,16 +338,16 @@ the test should verify that the documented behavior is implemented, but
should not break if other permissible behavior is later implemented. should not break if other permissible behavior is later implemented.
=== Advanced techniques: Namespaces ### Advanced techniques: Namespaces
Sometimes, when you're doing a lot of mocking at once, it's convenient to Sometimes, when you're doing a lot of mocking at once, it's convenient to
isolate your identifiers within a single namespace. If this were C++, we'd isolate your identifiers within a single namespace. If this were C++, we'd
already have namespaces, but for C, we do the best we can with macros and already have namespaces, but for C, we do the best we can with macros and
token-pasting. token-pasting.
We have some macros defined for this purpose in src/test/test.h. To use We have some macros defined for this purpose in `src/test/test.h`. To use
them, you define NS_MODULE to a prefix to be used for your identifiers, and them, you define `NS_MODULE` to a prefix to be used for your identifiers, and
then use other macros in place of identifier names. See src/test/test.h for then use other macros in place of identifier names. See `src/test/test.h` for
more documentation. more documentation.
@ -357,19 +357,19 @@ Integration tests: Calling Tor from the outside
Some tests need to invoke Tor from the outside, and shouldn't run from the Some tests need to invoke Tor from the outside, and shouldn't run from the
same process as the Tor test program. Reasons for doing this might include: same process as the Tor test program. Reasons for doing this might include:
* Testing the actual behavior of Tor when run from the command line * Testing the actual behavior of Tor when run from the command line
* Testing that a crash-handler correctly logs a stack trace * Testing that a crash-handler correctly logs a stack trace
* Verifying that violating a sandbox or capability requirement will * Verifying that violating a sandbox or capability requirement will
actually crash the program. actually crash the program.
* Needing to run as root in order to test capability inheritance or * Needing to run as root in order to test capability inheritance or
user switching. user switching.
To add one of these, you generally want a new C program in src/test. Add it To add one of these, you generally want a new C program in `src/test`. Add it
to TESTS and noinst_PROGRAMS if it can run on its own and return success or to `TESTS` and `noinst_PROGRAMS` if it can run on its own and return success or
failure. If it needs to be invoked multiple times, or it needs to be failure. If it needs to be invoked multiple times, or it needs to be
wrapped, add a new shell script to TESTS, and the new program to wrapped, add a new shell script to `TESTS`, and the new program to
noinst_PROGRAMS. If you need access to any environment variable from the `noinst_PROGRAMS`. If you need access to any environment variable from the
makefile (eg ${PYTHON} for a python interpreter), then make sure that the makefile (eg `${PYTHON}` for a python interpreter), then make sure that the
makefile exports them. makefile exports them.
Writing integration tests with Stem Writing integration tests with Stem
@ -379,25 +379,25 @@ The 'stem' library includes extensive unit tests for the Tor controller
protocol. protocol.
For more information on writing new tests for stem, have a look around For more information on writing new tests for stem, have a look around
the test/* directory in stem, and find a good example to emulate. You the `test/*` directory in stem, and find a good example to emulate. You
might want to start with might want to start with
https://gitweb.torproject.org/stem.git/tree/test/integ/control/controller.py `https://gitweb.torproject.org/stem.git/tree/test/integ/control/controller.py`
to improve Tor's test coverage. to improve Tor's test coverage.
You can run stem tests from tor with "make test-stem", or see You can run stem tests from tor with `make test-stem`, or see
https://stem.torproject.org/faq.html#how-do-i-run-the-tests . `https://stem.torproject.org/faq.html#how-do-i-run-the-tests`.
System testing with Chutney System testing with Chutney
--------------------------- ---------------------------
The 'chutney' program configures and launches a set of Tor relays, The 'chutney' program configures and launches a set of Tor relays,
authorities, and clients on your local host. It has a 'test network' authorities, and clients on your local host. It has a `test network`
functionality to send traffic through them and verify that the traffic functionality to send traffic through them and verify that the traffic
arrives correctly. arrives correctly.
You can write new test networks by adding them to 'networks'. To add You can write new test networks by adding them to `networks`. To add
them to Tor's tests, add them to the test-network or test-network-all them to Tor's tests, add them to the `test-network` or `test-network-all`
targets in Makefile.am. targets in `Makefile.am`.
(Adding new kinds of program to chutney will still require hacking the (Adding new kinds of program to chutney will still require hacking the
code.) code.)