2020-03-26 16:19:42 +01:00
|
|
|
# Hacking on Rust in Tor
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Getting Started
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Please read or review our documentation on Rust coding standards
|
2020-03-26 15:54:29 +01:00
|
|
|
(`doc/HACKING/CodingStandardsRust.md`) before doing anything.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Please also read
|
2018-02-21 17:55:07 +01:00
|
|
|
[the Rust Code of Conduct](https://www.rust-lang.org/en-US/conduct.html). We
|
|
|
|
aim to follow the good example set by the Rust community and be
|
|
|
|
excellent to one another. Let's be careful with each other, so we can
|
|
|
|
be memory-safe together!
|
|
|
|
|
|
|
|
Next, please contact us before rewriting anything! Rust in Tor is still
|
|
|
|
an experiment. It is an experiment that we very much want to see
|
|
|
|
succeed, so we're going slowly and carefully. For the moment, it's also
|
|
|
|
a completely volunteer-driven effort: while many, if not most, of us are
|
|
|
|
paid to work on Tor, we are not yet funded to write Rust code for Tor.
|
|
|
|
Please be patient with the other people who are working on getting more
|
|
|
|
Rust code into Tor, because they are graciously donating their free time
|
|
|
|
to contribute to this effort.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Resources for learning Rust
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
**Beginning resources**
|
|
|
|
|
|
|
|
The primary resource for learning Rust is
|
|
|
|
[The Book](https://doc.rust-lang.org/book/). If you'd like to start writing
|
|
|
|
Rust immediately, without waiting for anything to install, there is
|
|
|
|
[an interactive browser-based playground](https://play.rust-lang.org/).
|
|
|
|
|
|
|
|
**Advanced resources**
|
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
If you're interested in playing with various Rust compilers and viewing
|
|
|
|
a very nicely displayed output of the generated assembly, there is
|
2017-08-30 01:25:02 +02:00
|
|
|
[the Godbolt compiler explorer](https://rust.godbolt.org/)
|
|
|
|
|
|
|
|
For learning how to write unsafe Rust, read
|
|
|
|
[The Rustonomicon](https://doc.rust-lang.org/nomicon/).
|
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
For learning everything you ever wanted to know about Rust macros, there
|
|
|
|
is
|
2017-08-30 01:25:02 +02:00
|
|
|
[The Little Book of Rust Macros](https://danielkeep.github.io/tlborm/book/index.html).
|
|
|
|
|
2017-09-29 03:04:18 +02:00
|
|
|
For learning more about FFI and Rust, see Jake Goulding's
|
2019-09-20 09:58:30 +02:00
|
|
|
[Rust FFI Omnibus](https://jakegoulding.com/rust-ffi-omnibus/).
|
2017-09-29 03:04:18 +02:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Compiling Tor with Rust enabled
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
You will need to run the `configure` script with the `--enable-rust`
|
|
|
|
flag to explicitly build with Rust. Additionally, you will need to
|
|
|
|
specify where to fetch Rust dependencies, as we allow for either
|
|
|
|
fetching dependencies from Cargo or specifying a local directory.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
**Fetch dependencies from Cargo**
|
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ ./configure --enable-rust --enable-cargo-online-mode
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
**Using a local dependency cache**
|
|
|
|
|
2017-11-28 23:39:53 +01:00
|
|
|
You'll need the following Rust dependencies (as of this writing):
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2018-03-28 12:19:31 +02:00
|
|
|
libc==0.2.39
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2017-11-28 23:39:53 +01:00
|
|
|
We vendor our Rust dependencies in a separate repo using
|
2018-02-21 17:55:07 +01:00
|
|
|
[cargo-vendor](https://github.com/alexcrichton/cargo-vendor). To use
|
|
|
|
them, do:
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ git submodule init
|
|
|
|
$ git submodule update
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
To specify the local directory containing the dependencies, (assuming
|
|
|
|
you are in the top level of the repository) configure tor with:
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ TOR_RUST_DEPENDENCIES='path_to_dependencies_directory' ./configure --enable-rust
|
|
|
|
```
|
2017-11-28 23:39:53 +01:00
|
|
|
|
2020-03-22 13:50:58 +01:00
|
|
|
(Note that `TOR_RUST_DEPENDENCIES` must be the full path to the directory; it
|
2018-02-21 17:55:07 +01:00
|
|
|
cannot be relative.)
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
Assuming you used the above `git submodule` commands and you're in the
|
|
|
|
topmost directory of the repository, this would be:
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ TOR_RUST_DEPENDENCIES=`pwd`/src/ext/rust/crates ./configure --enable-rust
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Identifying which modules to rewrite
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
The places in the Tor codebase that are good candidates for porting to
|
|
|
|
Rust are:
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
1. loosely coupled to other Tor submodules,
|
|
|
|
2. have high test coverage, and
|
|
|
|
3. would benefit from being implemented in a memory safe language.
|
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
Help in either identifying places such as this, or working to improve
|
|
|
|
existing areas of the C codebase by adding regression tests and
|
|
|
|
simplifying dependencies, would be really helpful.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Furthermore, as submodules in C are implemented in Rust, this is a good
|
2018-02-21 17:55:07 +01:00
|
|
|
opportunity to refactor, add more tests, and split modules into smaller
|
|
|
|
areas of responsibility.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
A good first step is to build a module-level callgraph to understand how
|
|
|
|
interconnected your target module is.
|
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ git clone https://git.torproject.org/user/nickm/calltool.git
|
|
|
|
$ cd tor
|
|
|
|
$ CFLAGS=0 ./configure
|
|
|
|
$ ../calltool/src/main.py module_callgraph
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
The output will tell you each module name, along with a set of every module that
|
|
|
|
the module calls. Modules which call fewer other modules are better targets.
|
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Writing your Rust module
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Strive to change the C API as little as possible.
|
|
|
|
|
2020-11-11 12:38:19 +01:00
|
|
|
We are currently targeting Rust stable. (See `CodingStandardsRust.md` for more
|
2018-08-16 10:58:47 +02:00
|
|
|
details.)
|
|
|
|
|
2018-08-16 14:35:27 +02:00
|
|
|
It is on our TODO list to try to cultivate good
|
2018-02-21 17:55:07 +01:00
|
|
|
standing with various distro maintainers of `rustc` and `cargo`, in
|
|
|
|
order to ensure that whatever version we solidify on is readily
|
|
|
|
available.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2018-02-21 17:55:07 +01:00
|
|
|
If parts of your Rust code needs to stay in sync with C code (such as
|
|
|
|
handling enums across the FFI boundary), annonotate these places in a
|
|
|
|
comment structured as follows:
|
2017-11-12 04:26:22 +01:00
|
|
|
|
2020-03-22 13:50:58 +01:00
|
|
|
`/// C_RUST_COUPLED: <path_to_file> <name_of_c_object>`
|
2017-11-12 04:26:22 +01:00
|
|
|
|
2020-03-22 13:50:58 +01:00
|
|
|
Where `<name_of_c_object>` can be an enum, struct, constant, etc. Then,
|
2018-02-21 17:55:07 +01:00
|
|
|
do the same in the C code, to note that rust will need to be changed
|
|
|
|
when the C does.
|
2017-11-12 04:26:22 +01:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Adding your Rust module to Tor's build system
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
0. Your translation of the C module should live in its own crate(s)
|
2020-03-26 15:54:29 +01:00
|
|
|
in the `src/rust/` directory.
|
|
|
|
1. Add your crate to `src/rust/Cargo.toml`, in the
|
2017-08-30 01:25:02 +02:00
|
|
|
`[workspace.members]` section.
|
2017-11-06 18:55:10 +01:00
|
|
|
2. Add your crate's files to src/rust/include.am
|
|
|
|
|
|
|
|
If your crate should be available to C (rather than just being included as a
|
|
|
|
dependency of other Rust modules):
|
|
|
|
0. Declare the crate as a dependency of tor_rust in
|
|
|
|
`src/rust/tor_util/Cargo.toml` and include it in
|
|
|
|
`src/rust/tor_rust/lib.rs`
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## How to test your Rust code
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Everything should be tested full stop. Even non-public functionality.
|
|
|
|
|
2020-03-26 15:54:29 +01:00
|
|
|
Be sure to edit `src/test/test_rust.sh` to add the name of your
|
2018-02-21 17:55:07 +01:00
|
|
|
crate to the `crates` variable! This will ensure that `cargo test` is
|
|
|
|
run on your crate.
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Configure Tor's build system to build with Rust enabled:
|
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ ./configure --enable-fatal-warnings --enable-rust --enable-cargo-online-mode
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Tor's test should be run by doing:
|
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ make check
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
|
|
|
Tor's integration tests should also pass:
|
|
|
|
|
2020-07-13 11:16:51 +02:00
|
|
|
```console
|
|
|
|
$ make test-stem
|
|
|
|
```
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-03-26 16:19:42 +01:00
|
|
|
## Submitting a patch
|
2017-08-30 01:25:02 +02:00
|
|
|
|
2020-03-26 15:54:29 +01:00
|
|
|
Please follow the instructions in `doc/HACKING/GettingStarted.md`.
|