tor/doc/HACKING/design/01.00-lib-overview.md

## Library code in Tor.

Most of Tor's utility code is in modules in the `src/lib` subdirectory.  In
general, this code is not necessarily Tor-specific, but is instead possibly
useful for other applications.

This code includes:

  * Compatibility wrappers, to provide a uniform API across different
    platforms.

  * Library wrappers, to provide a tor-like API over different libraries
    that Tor uses for things like compression and cryptography.

  * Containers, to implement some general-purpose data container types.

The modules in `src/lib` are currently well-factored: each one depends
only on lower-level modules.  You can see an up-to-date list of the
modules sorted from lowest to highest level by running
`./scripts/maint/practracker/includes.py --toposort`.

As of this writing, the library modules are (from lowest to highest
level):

   * `lib/cc` -- Macros for managing the C compiler and
     language. Includes macros for improving compatibility and clarity
     across different C compilers.

   * `lib/version` -- Holds the current version of Tor.

   * `lib/testsupport` -- Helpers for making test-only code and test
     mocking support.

   * `lib/defs` -- Lowest-level constants used in many places across the
     code.

   * `lib/subsys` -- Types used for declaring a "subsystem". A subsystem
     is a module with support for initialization, shutdown,
     configuration, and so on.

   * `lib/conf` -- Types and macros used for declaring configuration
     options.

   * `lib/arch` -- Compatibility functions and macros for handling
     differences in CPU architecture.

   * `lib/err` -- Lowest-level error handling code: responsible for
     generating stack traces, handling raw assertion failures, and
     otherwise reporting problems that might not be safe to report
     via the regular logging module.

   * `lib/malloc` -- Wrappers and utilities for memory management.

   * `lib/intmath` -- Utilities for integer mathematics.

   * `lib/fdio` -- Utilities and compatibility code for reading and
      writing data on file descriptors (and on sockets, for platforms
      where a socket is not a kind of fd).

   * `lib/lock` -- Compatibility code for declaring and using locks.
      Lower-level than the rest of the threading code.

   * `lib/ctime` -- Constant-time implementations for data comparison
     and table lookup, used to avoid timing side-channels from standard
     implementations of memcmp() and so on.

   * `lib/string` -- Low-level compatibility wrappers and utility
     functions for string manipulation.

   * `lib/wallclock` -- Compatibility and utility functions for
     inspecting and manipulating the current (UTC) time.

   * `lib/osinfo` -- Functions for inspecting the version and
     capabilities of the operating system.

   * `lib/smartlist_core` -- The bare-bones pieces of our dynamic array
     ("smartlist") implementation. There are higher-level pieces, but
     these ones are used by (and therefore cannot use) the logging code.

   * `lib/log` -- Implements the logging system used by all higher-level
     Tor code.  You can think of this as the logical "midpoint" of the
     library code: much of the higher-level code is higher-level
     _because_ it uses the logging module, and much of the lower-level
     code is specifically written to avoid having to log, because the
     logging module depends on it.

   * `lib/container` -- General purpose containers, including dynamic arrays
     ("smartlists"), hashtables, bit arrays, weak-reference-like "handles",
     bloom filters, and a bit more.

   * `lib/trace` -- A general-purpose API for introducing
     function-tracing functionality into Tor.  Currently not much used.

   * `lib/thread` -- Threading compatibility and utility functionality,
     other than low-level locks (which are in `lib/lock`) and
     workqueue/threadpool code (which belongs in `lib/evloop`).

   * `lib/term` -- Code for terminal manipulation functions (like
     reading a password from the user).

   * `lib/memarea` -- A data structure for a fast "arena" style allocator,
     where the data is freed all at once.  Used for parsing.

   * `lib/encoding` -- Implementations for encoding data in various
     formats, datatypes, and transformations.

   * `lib/dispatch` -- A general-purpose in-process message delivery
     system.  Used by `lib/pubsub` to implement our inter-module
     publish/subscribe system.

   * `lib/sandbox` -- Our Linux seccomp2 sandbox implementation.

   * `lib/pubsub` -- Code and macros to implement our publish/subscribe
     message passing system.

   * `lib/fs` -- Utility and compatibility code for manipulating files,
     filenames, directories, and so on.

   * `lib/confmgt` -- Code to parse, encode, and manipulate our
     configuration files, state files, and so forth.

   * `lib/crypt_ops` -- Cryptographic operations. This module contains
     wrappers around the cryptographic libraries that we support,
     and implementations for some higher-level cryptographic
     constructions that we use.

   * `lib/meminfo` -- Functions for inspecting our memory usage, if the
     malloc implementation exposes that to us.

   * `lib/time` -- Higher level time functions, including fine-gained and
      monotonic timers.

   * `lib/math` -- Floating-point mathematical utilities, including
     compatibility code, and probability distributions.

   * `lib/buf` -- A general purpose queued buffer implementation,
     similar to the BSD kernel's "mbuf" structure.

   * `lib/net` -- Networking code, including address manipulation,
     compatibility wrappers,

   * `lib/compress` -- A compatibility wrapper around several
     compression libraries, currently including zlib, zstd, and lzma.

   * `lib/geoip` -- Utilities to manage geoip (IP to country) lookups
      and formats.

   * `lib/tls` -- Compatibility wrappers around the library (NSS or
     OpenSSL, depending on configuration) that Tor uses to implement the
     TLS link security protocol.

   * `lib/evloop` -- Tools to manage the event loop and related
     functionality, in order to implement asynchronous networking,
     timers, periodic events, and other scheduling tasks.

   * `lib/process` -- Utilities and compatibility code to launch and
     manage subprocesses.

### What belongs in lib?

In general, if you can imagine some program wanting the functionality
you're writing, even if that program had nothing to do with Tor, your
functionality belongs in lib.

If it falls into one of the existing "lib" categories, your
functionality belongs in lib.

If you are using platform-specific `#ifdef`s to manage compatibility
issues among platforms, you should probably consider whether you can
put your code into lib.