Equi-X supports optionally allocating its solver memory using huge
pages, to reduce the virtual memory subsystem overhead required to make
the entire solver buffer live.
Tor doesn't use this feature, since it seems to have no noticeable
performance benefit at this time, but we still included code for it at
compile time. To improve portability, this patch disables huge page
support by default and enables it only in the cmake build system used
for equix benchmarks.
With this patch equix-bench still supports huge pages. Verified using
strace that we're making the hugepage allocation.
There's no fallback for huge pages, so Equi-X initialization will fail
if they are requested and we don't support them for any runtime or
compile-time reason.
Addresses #40843 (NetBSD) but also prevents future porting issues
related to huge pages.
NetBSD includes the idea of a 'maximum protection' per-region,
and an mprotect which exceeds the max protection will be denied.
If we explicitly ask for a maximum which includes execute permission, we
can successfully swap our code buffer's permissions between read-write
and read-execute when each hash program is compiled.
With this patch, the crypto/hashx tests pass on NetBSD 9.
This addresses bug #40844
As suggested by @wiz on #40843, let's add an explicit check to
hashx_vm_alloc_huge() that avoids using a Linux-style default
on NetBSD targets.
This doesn't change the questionable Linux-style default,
but a future patch will disable this code by default so it's not a
portability liability.
(This code is in hashx's VM layer but it's actually only relevant to
equix.)
This addresses bug #40843. Another patch will disable huge pages
by default entirely, but this patch is sufficient to fix the NetBSD
build.
I saw this test fail intermittently due to what seemed like a filesystem
race in docker? The cleanup task was failing with a 'directory not
empty' error, despite trying to do a recursive 'rm'. This patch adds an
'ls' to the same directory, hoping the output might be useful to
diagnose future intermittent failures.
Change 3f66ff9b00 added geoip-db-tool to
the main workspace, so it's no longer using a local lockfile. Move its
lock to the crate root, remove from gitignore, and update it.
(We could also choose to not keep the lockfiles checked in, but it seems
useful to have them in our test and maintenance tooling here.)
Considering a compression bomb before looking for errors led to false negative
log warnings. Instead, it is possible the work failed for whatever reasons
which is not indicative of a compression bomb.
Fixes#40739
Signed-off-by: David Goulet <dgoulet@torproject.org>
