mirror of
https://gitlab.torproject.org/tpo/core/tor.git
synced 2024-11-27 13:53:31 +01:00
Divide 01a-memory.md into doxygen.
This commit is contained in:
parent
4bf73dfa26
commit
fb20618e28
@ -1,103 +0,0 @@
|
|||||||
|
|
||||||
## Memory management
|
|
||||||
|
|
||||||
### Heap-allocation functions: lib/malloc/malloc.h
|
|
||||||
|
|
||||||
Tor imposes a few light wrappers over C's native malloc and free
|
|
||||||
functions, to improve convenience, and to allow wholescale replacement
|
|
||||||
of malloc and free as needed.
|
|
||||||
|
|
||||||
You should never use 'malloc', 'calloc', 'realloc, or 'free' on their
|
|
||||||
own; always use the variants prefixed with 'tor_'.
|
|
||||||
They are the same as the standard C functions, with the following
|
|
||||||
exceptions:
|
|
||||||
|
|
||||||
* `tor_free(NULL)` is a no-op.
|
|
||||||
* `tor_free()` is a macro that takes an lvalue as an argument and sets it to
|
|
||||||
NULL after freeing it. To avoid this behavior, you can use `tor_free_()`
|
|
||||||
instead.
|
|
||||||
* tor_malloc() and friends fail with an assertion if they are asked to
|
|
||||||
allocate a value so large that it is probably an underflow.
|
|
||||||
* It is always safe to `tor_malloc(0)`, regardless of whether your libc
|
|
||||||
allows it.
|
|
||||||
* `tor_malloc()`, `tor_realloc()`, and friends are never allowed to fail.
|
|
||||||
Instead, Tor will die with an assertion. This means that you never
|
|
||||||
need to check their return values. See the next subsection for
|
|
||||||
information on why we think this is a good idea.
|
|
||||||
|
|
||||||
We define additional general-purpose memory allocation functions as well:
|
|
||||||
|
|
||||||
* `tor_malloc_zero(x)` behaves as `calloc(1, x)`, except the it makes clear
|
|
||||||
the intent to allocate a single zeroed-out value.
|
|
||||||
* `tor_reallocarray(x,y)` behaves as the OpenBSD reallocarray function.
|
|
||||||
Use it for cases when you need to realloc() in a multiplication-safe
|
|
||||||
way.
|
|
||||||
|
|
||||||
And specific-purpose functions as well:
|
|
||||||
|
|
||||||
* `tor_strdup()` and `tor_strndup()` behaves as the underlying libc
|
|
||||||
functions, but use `tor_malloc()` instead of the underlying function.
|
|
||||||
* `tor_memdup()` copies a chunk of memory of a given size.
|
|
||||||
* `tor_memdup_nulterm()` copies a chunk of memory of a given size, then
|
|
||||||
NUL-terminates it just to be safe.
|
|
||||||
|
|
||||||
#### Why assert on allocation failure?
|
|
||||||
|
|
||||||
Why don't we allow `tor_malloc()` and its allies to return NULL?
|
|
||||||
|
|
||||||
First, it's error-prone. Many programmers forget to check for NULL return
|
|
||||||
values, and testing for `malloc()` failures is a major pain.
|
|
||||||
|
|
||||||
Second, it's not necessarily a great way to handle OOM conditions. It's
|
|
||||||
probably better (we think) to have a memory target where we dynamically free
|
|
||||||
things ahead of time in order to stay under the target. Trying to respond to
|
|
||||||
an OOM at the point of `tor_malloc()` failure, on the other hand, would involve
|
|
||||||
a rare operation invoked from deep in the call stack. (Again, that's
|
|
||||||
error-prone and hard to debug.)
|
|
||||||
|
|
||||||
Third, thanks to the rise of Linux and other operating systems that allow
|
|
||||||
memory to be overcommitted, you can't actually ever rely on getting a NULL
|
|
||||||
from `malloc()` when you're out of memory; instead you have to use an approach
|
|
||||||
closer to tracking the total memory usage.
|
|
||||||
|
|
||||||
#### Conventions for your own allocation functions.
|
|
||||||
|
|
||||||
Whenever you create a new type, the convention is to give it a pair of
|
|
||||||
`x_new()` and `x_free_()` functions, named after the type.
|
|
||||||
|
|
||||||
Calling `x_free(NULL)` should always be a no-op.
|
|
||||||
|
|
||||||
There should additionally be an `x_free()` macro, defined in terms of
|
|
||||||
`x_free_()`. This macro should set its lvalue to NULL. You can define it
|
|
||||||
using the FREE_AND_NULL macro, as follows:
|
|
||||||
|
|
||||||
```
|
|
||||||
#define x_free(ptr) FREE_AND_NULL(x_t, x_free_, (ptr))
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### Grow-only memory allocation: lib/memarea
|
|
||||||
|
|
||||||
It's often handy to allocate a large number of tiny objects, all of which
|
|
||||||
need to disappear at the same time. You can do this in tor using the
|
|
||||||
memarea.c abstraction, which uses a set of grow-only buffers for allocation,
|
|
||||||
and only supports a single "free" operation at the end.
|
|
||||||
|
|
||||||
Using memareas also helps you avoid memory fragmentation. You see, some libc
|
|
||||||
malloc implementations perform badly on the case where a large number of
|
|
||||||
small temporary objects are allocated at the same time as a few long-lived
|
|
||||||
objects of similar size. But if you use tor_malloc() for the long-lived ones
|
|
||||||
and a memarea for the temporary object, the malloc implementation is likelier
|
|
||||||
to do better.
|
|
||||||
|
|
||||||
To create a new memarea, use `memarea_new()`. To drop all the storage from a
|
|
||||||
memarea, and invalidate its pointers, use `memarea_drop_all()`.
|
|
||||||
|
|
||||||
The allocation functions `memarea_alloc()`, `memarea_alloc_zero()`,
|
|
||||||
`memarea_memdup()`, `memarea_strdup()`, and `memarea_strndup()` are analogous
|
|
||||||
to the similarly-named malloc() functions. There is intentionally no
|
|
||||||
`memarea_free()` or `memarea_realloc()`.
|
|
||||||
|
|
||||||
### Special allocation: lib/malloc/map_anon.h
|
|
||||||
|
|
||||||
TODO: WRITEME.
|
|
@ -2,4 +2,77 @@
|
|||||||
@dir /lib/malloc
|
@dir /lib/malloc
|
||||||
@brief lib/malloc: Wrappers and utilities for memory management.
|
@brief lib/malloc: Wrappers and utilities for memory management.
|
||||||
|
|
||||||
|
|
||||||
|
Tor imposes a few light wrappers over C's native malloc and free
|
||||||
|
functions, to improve convenience, and to allow wholescale replacement
|
||||||
|
of malloc and free as needed.
|
||||||
|
|
||||||
|
You should never use 'malloc', 'calloc', 'realloc, or 'free' on their
|
||||||
|
own; always use the variants prefixed with 'tor_'.
|
||||||
|
They are the same as the standard C functions, with the following
|
||||||
|
exceptions:
|
||||||
|
|
||||||
|
* `tor_free(NULL)` is a no-op.
|
||||||
|
* `tor_free()` is a macro that takes an lvalue as an argument and sets it to
|
||||||
|
NULL after freeing it. To avoid this behavior, you can use `tor_free_()`
|
||||||
|
instead.
|
||||||
|
* tor_malloc() and friends fail with an assertion if they are asked to
|
||||||
|
allocate a value so large that it is probably an underflow.
|
||||||
|
* It is always safe to `tor_malloc(0)`, regardless of whether your libc
|
||||||
|
allows it.
|
||||||
|
* `tor_malloc()`, `tor_realloc()`, and friends are never allowed to fail.
|
||||||
|
Instead, Tor will die with an assertion. This means that you never
|
||||||
|
need to check their return values. See the next subsection for
|
||||||
|
information on why we think this is a good idea.
|
||||||
|
|
||||||
|
We define additional general-purpose memory allocation functions as well:
|
||||||
|
|
||||||
|
* `tor_malloc_zero(x)` behaves as `calloc(1, x)`, except the it makes clear
|
||||||
|
the intent to allocate a single zeroed-out value.
|
||||||
|
* `tor_reallocarray(x,y)` behaves as the OpenBSD reallocarray function.
|
||||||
|
Use it for cases when you need to realloc() in a multiplication-safe
|
||||||
|
way.
|
||||||
|
|
||||||
|
And specific-purpose functions as well:
|
||||||
|
|
||||||
|
* `tor_strdup()` and `tor_strndup()` behaves as the underlying libc
|
||||||
|
functions, but use `tor_malloc()` instead of the underlying function.
|
||||||
|
* `tor_memdup()` copies a chunk of memory of a given size.
|
||||||
|
* `tor_memdup_nulterm()` copies a chunk of memory of a given size, then
|
||||||
|
NUL-terminates it just to be safe.
|
||||||
|
|
||||||
|
#### Why assert on allocation failure?
|
||||||
|
|
||||||
|
Why don't we allow `tor_malloc()` and its allies to return NULL?
|
||||||
|
|
||||||
|
First, it's error-prone. Many programmers forget to check for NULL return
|
||||||
|
values, and testing for `malloc()` failures is a major pain.
|
||||||
|
|
||||||
|
Second, it's not necessarily a great way to handle OOM conditions. It's
|
||||||
|
probably better (we think) to have a memory target where we dynamically free
|
||||||
|
things ahead of time in order to stay under the target. Trying to respond to
|
||||||
|
an OOM at the point of `tor_malloc()` failure, on the other hand, would involve
|
||||||
|
a rare operation invoked from deep in the call stack. (Again, that's
|
||||||
|
error-prone and hard to debug.)
|
||||||
|
|
||||||
|
Third, thanks to the rise of Linux and other operating systems that allow
|
||||||
|
memory to be overcommitted, you can't actually ever rely on getting a NULL
|
||||||
|
from `malloc()` when you're out of memory; instead you have to use an approach
|
||||||
|
closer to tracking the total memory usage.
|
||||||
|
|
||||||
|
#### Conventions for your own allocation functions.
|
||||||
|
|
||||||
|
Whenever you create a new type, the convention is to give it a pair of
|
||||||
|
`x_new()` and `x_free_()` functions, named after the type.
|
||||||
|
|
||||||
|
Calling `x_free(NULL)` should always be a no-op.
|
||||||
|
|
||||||
|
There should additionally be an `x_free()` macro, defined in terms of
|
||||||
|
`x_free_()`. This macro should set its lvalue to NULL. You can define it
|
||||||
|
using the FREE_AND_NULL macro, as follows:
|
||||||
|
|
||||||
|
```
|
||||||
|
#define x_free(ptr) FREE_AND_NULL(x_t, x_free_, (ptr))
|
||||||
|
```
|
||||||
|
|
||||||
**/
|
**/
|
||||||
|
@ -7,4 +7,24 @@ once. This kind of allocation is very fast and avoids fragmentation, at the
|
|||||||
expense of requiring all the data to be freed at the same time. We use this
|
expense of requiring all the data to be freed at the same time. We use this
|
||||||
for parsing and diff calculations.
|
for parsing and diff calculations.
|
||||||
|
|
||||||
|
It's often handy to allocate a large number of tiny objects, all of which
|
||||||
|
need to disappear at the same time. You can do this in tor using the
|
||||||
|
memarea.c abstraction, which uses a set of grow-only buffers for allocation,
|
||||||
|
and only supports a single "free" operation at the end.
|
||||||
|
|
||||||
|
Using memareas also helps you avoid memory fragmentation. You see, some libc
|
||||||
|
malloc implementations perform badly on the case where a large number of
|
||||||
|
small temporary objects are allocated at the same time as a few long-lived
|
||||||
|
objects of similar size. But if you use tor_malloc() for the long-lived ones
|
||||||
|
and a memarea for the temporary object, the malloc implementation is likelier
|
||||||
|
to do better.
|
||||||
|
|
||||||
|
To create a new memarea, use `memarea_new()`. To drop all the storage from a
|
||||||
|
memarea, and invalidate its pointers, use `memarea_drop_all()`.
|
||||||
|
|
||||||
|
The allocation functions `memarea_alloc()`, `memarea_alloc_zero()`,
|
||||||
|
`memarea_memdup()`, `memarea_strdup()`, and `memarea_strndup()` are analogous
|
||||||
|
to the similarly-named malloc() functions. There is intentionally no
|
||||||
|
`memarea_free()` or `memarea_realloc()`.
|
||||||
|
|
||||||
**/
|
**/
|
||||||
|
Loading…
Reference in New Issue
Block a user