Add simple script for running gitian-build
This shell script creates a docker image for running the gitian-build.py script, to avoid version incompatibilities. The builds may be signed automatically after manually reviewing the build hashes. Only docker.io and apt-cacher-ng are required to be installed on the host. The MacOSX SDK should also be present in this directory if Mac builds are desired. A container named "gitrun" is used to launch the builds, and all output will reside in this container as well.
This commit is contained in:
Howard Chu
parent
8fde011dbe
commit
4571d3686d
85
contrib/gitian/DOCKRUN.md
Normal file
85
contrib/gitian/DOCKRUN.md
Normal file
@ -0,0 +1,85 @@
|
||||
Quick Gitian building with docker
|
||||
=================================
|
||||
|
||||
*Setup instructions for a Gitian build of Monero using Docker.*
|
||||
|
||||
Gitian supports other container mechanisms too but if you have a Debian or
|
||||
Ubuntu-based host the steps can be greatly simplified.
|
||||
|
||||
Preparing the Gitian builder host
|
||||
---------------------------------
|
||||
|
||||
The procedure here will create a docker container for build preparation, as well as
|
||||
for actually running the builds. The only items you must install on your own host
|
||||
are docker and apt-cacher-ng. With docker installed, you should also give yourself
|
||||
permission to use docker by adding yourself to the docker group.
|
||||
|
||||
```bash
|
||||
sudo apt-get install docker.io apt-cacher-ng
|
||||
sudo usermod -aG docker $USER
|
||||
su $USER
|
||||
```
|
||||
|
||||
The final `su` command is needed to start a new shell with your new group membership,
|
||||
since the `usermod` command doesn't affect any existing sessions.
|
||||
|
||||
If you want Mac binaries included in your build, you need to obtain the MacOS SDK:
|
||||
|
||||
```bash
|
||||
curl -O https://bitcoincore.org/depends-sources/sdks/MacOSX10.11.sdk.tar.gz
|
||||
```
|
||||
|
||||
Other User Preparation
|
||||
----------------------
|
||||
|
||||
The final step will be to `gpg` sign the results of your build and upload them to GitHub.
|
||||
Before you can do that, you'll need
|
||||
* a GitHub account.
|
||||
If your GitHub account name is different from your local account name, you must
|
||||
set your GitHub account name for the script to use:
|
||||
|
||||
```bash
|
||||
export GH_USER=<github account name>
|
||||
```
|
||||
|
||||
* PGP keys - if you don't have one already, you can use `gpg --quick-gen-key` to generate it.
|
||||
* a fork of the [gitian.sigs](https://github.com/monero-project/gitian.sigs/) repo on your GitHub account.
|
||||
Please follow the directions there for uploading your key first.
|
||||
|
||||
**Note:** Please ensure your gpg public key is available to check signatures by adding it to the [gitian.sigs/gitian-pubkeys/](https://github.com/monero-project/gitian.sigs/tree/master/gitian-pubkeys) directory in a pull request.
|
||||
|
||||
|
||||
Building the Binaries
|
||||
---------------------
|
||||
|
||||
The dockrun.sh script will do everything to build the binaries. Just specify the
|
||||
version to build as its only argument, e.g.
|
||||
|
||||
```bash
|
||||
./dockrun.sh v0.17.2.3
|
||||
```
|
||||
|
||||
The build should run to completion with no errors, and will display the SHA256 checksums
|
||||
of the resulting binaries. You'll be prompted to check if the sums look good, and if so
|
||||
then the results will be signed, and the signatures will be pushed to GitHub.
|
||||
|
||||
You can also look in the [gitian.sigs](https://github.com/monero-project/gitian.sigs/) repo and / or [getmonero.org release checksums](https://web.getmonero.org/downloads/hashes.txt) to see if others got the same checksum for the same version tag. If there is ever a mismatch -- **STOP! Something is wrong**. Contact others on IRC / GitHub to figure out what is going on.
|
||||
|
||||
|
||||
Other Options
|
||||
-------------
|
||||
|
||||
This script just runs the [gitian-build.py](gitian-build.py) inside a container named `gitrun`.
|
||||
You can set other options for that script by setting the OPT variable when running `dockrun.sh`
|
||||
e.g.
|
||||
|
||||
```bash
|
||||
OPT="-j 8" ./dockrun.sh v0.17.2.3
|
||||
```
|
||||
|
||||
You can also examine the build and install logs by running a shell in the container, e.g.
|
||||
|
||||
```bash
|
||||
docker exec -it gitrun /bin/bash
|
||||
more builder/var/install-linux.log
|
||||
```
|
||||
@ -31,6 +31,8 @@ This guide explains how to set up the environment, and how to start the builds.
|
||||
|
||||
* Gitian gives you the option of using any of 3 different virtualization tools: `kvm`, `docker` or `lxc`. This documentation will only show how to build with `lxc` and `docker` (documentation for `kvm` is welcome). Building with `lxc` is the default, but is more complicated, so we recommend docker your first time.
|
||||
|
||||
* For a shortcut using `docker` follow the instructions in [DOCKRUN.md](DOCKRUN.md) instead
|
||||
of following the rest of this document..
|
||||
|
||||
## Create the gitianuser account
|
||||
|
||||
|
||||
133
contrib/gitian/dockrun.sh
Executable file
133
contrib/gitian/dockrun.sh
Executable file
@ -0,0 +1,133 @@
|
||||
#!/bin/sh
|
||||
|
||||
if [ $# -ne 1 ]; then
|
||||
echo "usage: $0 <version>"
|
||||
exit 1
|
||||
fi
|
||||
VERSION=$1
|
||||
|
||||
DOCKER=`command -v docker`
|
||||
CACHER=`command -v apt-cacher-ng`
|
||||
|
||||
if [ -z "$DOCKER" -o -z "$CACHER" ]; then
|
||||
echo "$0: you must first install docker.io and apt-cacher-ng"
|
||||
echo " e.g. sudo apt-get install docker.io apt-cacher-ng"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
GH_USER=${GH_USER-$USER}
|
||||
|
||||
TAG=gitrun-bionic
|
||||
TAG2=base-bionic-amd64
|
||||
IMAGE=`docker images | grep $TAG`
|
||||
|
||||
WORKDIR=/home/ubuntu
|
||||
|
||||
if [ -z "$IMAGE" ]; then
|
||||
GID=`getent group docker`
|
||||
mkdir -p docker
|
||||
cd docker
|
||||
|
||||
# container for running gitian-build.py
|
||||
cat <<EOF > ${TAG}.Dockerfile
|
||||
FROM ubuntu:bionic
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN echo 'Acquire::http { Proxy "http://172.17.0.1:3142"; };' > /etc/apt/apt.conf.d/50cacher
|
||||
RUN echo "$GID" >> /etc/group
|
||||
RUN apt-get update && apt-get --no-install-recommends -y install lsb-release ruby git make wget docker.io python3 curl
|
||||
|
||||
RUN useradd -ms /bin/bash -U ubuntu -G docker
|
||||
USER ubuntu:docker
|
||||
WORKDIR $WORKDIR
|
||||
|
||||
RUN git clone https://github.com/monero-project/gitian.sigs.git sigs; \
|
||||
git clone https://github.com/devrandom/gitian-builder.git builder; \
|
||||
cd builder; git checkout c0f77ca018cb5332bfd595e0aff0468f77542c23; mkdir -p inputs var; cd inputs; \
|
||||
git clone https://github.com/monero-project/monero
|
||||
|
||||
CMD ["sleep", "infinity"]
|
||||
EOF
|
||||
|
||||
docker build --pull -f ${TAG}.Dockerfile -t $TAG .
|
||||
|
||||
cd ..
|
||||
docker run -v /var/run/docker.sock:/var/run/docker.sock -d --name gitrun $TAG
|
||||
if [ -f MacOSX10.11.sdk.tar.gz ]; then
|
||||
docker cp MacOSX10.11.sdk.tar.gz gitrun:$WORKDIR/builder/inputs/
|
||||
else
|
||||
echo "No MacOS SDK found, Mac builds will be omitted"
|
||||
fi
|
||||
|
||||
fi
|
||||
|
||||
IMAGE=`docker images | grep $TAG2`
|
||||
if [ -z "$IMAGE" ]; then
|
||||
mkdir -p docker
|
||||
cd docker
|
||||
|
||||
# container for actually running each build
|
||||
cat <<EOF > ${TAG2}.Dockerfile
|
||||
FROM ubuntu:bionic
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN echo 'Acquire::http { Proxy "http://172.17.0.1:3142"; };' > /etc/apt/apt.conf.d/50cacher
|
||||
RUN apt-get update && apt-get --no-install-recommends -y install build-essential git language-pack-en \
|
||||
wget lsb-release curl gcc-7 g++-7 gcc g++ binutils-gold pkg-config autoconf libtool automake faketime \
|
||||
bsdmainutils ca-certificates python cmake gperf
|
||||
|
||||
RUN useradd -ms /bin/bash -U ubuntu
|
||||
USER ubuntu:ubuntu
|
||||
WORKDIR $WORKDIR
|
||||
|
||||
CMD ["sleep", "infinity"]
|
||||
EOF
|
||||
|
||||
docker build --pull -f ${TAG2}.Dockerfile -t $TAG2 .
|
||||
|
||||
cd ..
|
||||
|
||||
fi
|
||||
|
||||
RUNNING=`docker ps | grep gitrun`
|
||||
if [ -z "$RUNNING" ]; then
|
||||
BUILT=`docker ps -a | grep gitrun`
|
||||
if [ -z "$BUILT" ]; then
|
||||
docker run -v /var/run/docker.sock:/var/run/docker.sock -d --name gitrun $TAG
|
||||
else
|
||||
docker start gitrun
|
||||
fi
|
||||
fi
|
||||
docker cp gitian-build.py gitrun:$WORKDIR/
|
||||
docker exec -t gitrun ./gitian-build.py -d -b -D -n $OPT $GH_USER $VERSION
|
||||
RC=$?
|
||||
if [ $RC != 0 ]; then
|
||||
exit $RC
|
||||
fi
|
||||
echo "\nBuild Results:\n"
|
||||
docker exec gitrun sh -c "sha256sum out/$VERSION/*"
|
||||
echo "\nIf these results look correct, type \"sign\" to sign them, otherwise ^C to stop now."
|
||||
read check
|
||||
if [ "$check" != "sign" ]; then
|
||||
echo "Not signing, bye."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -d sigs ]; then
|
||||
git clone https://github.com/monero-project/gitian.sigs.git sigs
|
||||
cd sigs
|
||||
git remote add $GH_USER git@github.com:$GH_USER/gitian.sigs.git
|
||||
cd ..
|
||||
fi
|
||||
|
||||
DIRS=`docker exec gitrun sh -c "echo sigs/$VERSION-*"`
|
||||
for i in $DIRS; do
|
||||
docker cp gitrun:$WORKDIR/$i sigs
|
||||
gpg --detach-sign $i/$GH_USER/*.assert
|
||||
done
|
||||
|
||||
cd sigs
|
||||
git checkout -b $VERSION
|
||||
git add $VERSION-*
|
||||
git commit -S -m "Add $GH_USER $VERSION"
|
||||
git push --set-upstream $GH_USER $VERSION
|
||||
Loading…
Reference in New Issue
Block a user