Install Greenbone Vulnerability Manager 11 on Ubuntu 19.04 from source... Part 1

Edits:

2019-10-12 - Splitted this up in two parts, this part (part 1) is for the manager and scanner but with no startups scripts…
2019-10-12 - GVM 11 is not released as of yet when this is written. It’s a git snapshot from a pre GVM 11 that I am trying out here..
2019-10-13 - Rewrote some stuff with redis and gvmd..
2019-10-14 - Did a full rewrite of redis and the new ospd-openvas thingies
2019-10-14 - GVM11 is now released, changed git clone to downloads of releases
2019-11-02 - Updated some PEBKAC’s. So that is should be possible to cut’n paste where how the “code” is formatted.
2019-11-03 - Updated with some startupscripts. Trying some more with ospd-openvas…
2019-11-06 - MOAR of the same - Updated with some startupscripts. Trying some more with ospd-openvas…
2019-11-06 - Cleaned up text, tried to follow on a clean vm.. Worked :D, thou there are a few caveats still.
2019-12-21 - Debian installation here » Install Greenbone Vulnerability Manager 11 on Debian 10.2 from source

Caveats:

Update IANA Service names doesn’t work here.
Could not make OSPD-Openvas default scanner.
Still a bug if with ospd-openvas-1.0.0 that shows if a non ascii character exists in the downloaded NVT.
Not so much testing made yet, If you test it out and you find something. Leave a message in the comments :)

GSA-9

First (well, not anymore) try at GVM 11 on Ubuntu 19.04 from git source.

Like the last guides:
This installation is not made for public facing servers, there is no built in security in this setup.

I take no responsibility if this guide bork you server, burn your house down to ashes or makes your cat to leave you..
It’s under the “it worked for me[tm]” clause.

This is as always a work in progress.

preparation is key

Preparation is key…

Prepare a can of coffee and (if at home) put on your best comfy clothes.
When writhing this, The Umbrella Academy soundtrack was playing in the headphones..

Fire up an vanilla Ubuntu Server 19.04 on a vm for some testing and start the installation.

install requirements

sudo apt install software-properties-common ;\
sudo add-apt-repository universe ;\
sudo apt install -y cmake pkg-config libglib2.0-dev libgpgme11-dev libgnutls28-dev uuid-dev libssh-gcrypt-dev \
libldap2-dev doxygen graphviz libradcli-dev libhiredis-dev libpcap-dev bison libksba-dev libsnmp-dev \
gcc-mingw-w64 heimdal-dev libpopt-dev xmltoman redis-server xsltproc libical2-dev postgresql \
postgresql-contrib postgresql-server-dev-all gnutls-bin nmap rpm nsis curl wget fakeroot gnupg \
sshpass socat snmp smbclient libmicrohttpd-dev libxml2-dev python-polib gettext \
python3-paramiko python3-lxml python3-defusedxml python3-pip python3-psutil virtualenv ;\
sudo apt install -y texlive-latex-extra --no-install-recommends ;\
sudo apt install -y texlive-fonts-recommended ;\
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - ;\
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list ;\
sudo apt update ;\
sudo apt -y install yarn

create user.

cp /etc/environment ~/environment.bak ;\
sudo sed -i 's|PATH="|PATH="/opt/gvm/bin:/opt/gvm/sbin:/opt/gvm/.local/bin:|g' /etc/environment ;\
sudo bash -c 'cat << EOF > /etc/ld.so.conf.d/gvm.conf
# gmv libs location
/opt/gvm/lib
EOF'

sudo mkdir /opt/gvm ;\
sudo adduser gvm --disabled-password --home /opt/gvm/ --no-create-home --gecos '' ;\
sudo usermod -aG redis gvm  # This is for ospd-openvas can connect to redis.sock.. If you have a better idea here, pls write in the comments :) ;\
sudo chown gvm:gvm /opt/gvm/ ;\
sudo su - gvm

mkdir src ;\
cd src ;\
export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH

Download stuff

wget -O gvm-libs-11.0.0.tar.gz  https://github.com/greenbone/gvm-libs/archive/v11.0.0.tar.gz ;\
wget -O openvas-7.0.0.tar.gz https://github.com/greenbone/openvas/archive/v7.0.0.tar.gz ;\
wget -O gvmd-9.0.0.tar.gz https://github.com/greenbone/gvmd/archive/v9.0.0.tar.gz ;\
wget -O openvas-smb-1.0.5.tar.gz https://github.com/greenbone/openvas-smb/archive/v1.0.5.tar.gz ;\
wget -O gsa-9.0.0.tar.gz https://github.com/greenbone/gsa/archive/v9.0.0.tar.gz ;\
wget -O ospd-openvas-1.0.0.tar.gz https://github.com/greenbone/ospd-openvas/archive/v1.0.0.tar.gz ;\
wget -O ospd-2.0.0.tar.gz https://github.com/greenbone/ospd/archive/v2.0.0.tar.gz

unpack

find . -name \*.gz -exec tar zxvfp {} \;

install gvm-libs

cd gvm-libs-11.0.0 ;\
 export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH ;\
 mkdir build ;\
 cd build ;\
 cmake -DCMAKE_INSTALL_PREFIX=/opt/gvm .. ;\
 make ;\
 make doc ;\
 make install ;\
 cd /opt/gvm/src

config and build openvas-smb

cd openvas-smb-1.0.5 ;\
 export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH ;\
 mkdir build ;\
 cd build/ ;\
 cmake -DCMAKE_INSTALL_PREFIX=/opt/gvm .. ;\
 make ;\
 make install ;\
 cd /opt/gvm/src

config and build scanner

cd openvas-7.0.0 ;\
 export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH ;\
 mkdir build ;\
 cd build/ ;\
 cmake -DCMAKE_INSTALL_PREFIX=/opt/gvm .. ;\
 make ;\
 make doc ;\
 make install ;\
 cd /opt/gvm/src

Fix redis for default openvas install

For this become almight root. Preferably log in to another ssh session as your normal user with sudo rights.

sudo su

ldconfig ;\
cp /etc/redis/redis.conf /etc/redis/redis.orig ;\
cp /opt/gvm/src/openvas-7.0.0/config/redis-openvas.conf /etc/redis/ ;\
chown redis:redis /etc/redis/redis-openvas.conf ;\
echo "db_address = /run/redis-openvas/redis.sock" > /opt/gvm/etc/openvas/openvas.conf ;\
systemctl enable redis-server@openvas.service ;\
systemctl start redis-server@openvas.service

sysctl -w net.core.somaxconn=1024
sysctl vm.overcommit_memory=1

echo "net.core.somaxconn=1024"  >> /etc/sysctl.conf
echo "vm.overcommit_memory=1" >> /etc/sysctl.conf

cat << EOF > /etc/systemd/system/disable-thp.service
[Unit]
Description=Disable Transparent Huge Pages (THP)

[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload ;\
systemctl start disable-thp ;\
systemctl enable disable-thp ;\
systemctl restart redis-server

As openvas will be launched from an ospd-openvas process with sudo, the next configuration is required in the sudoers file:

visudo

Edit the secure_path line to this.

Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/opt/gvm/sbin"

Add this line to allow the created gvm user to launch openvas with root permissions.

### Allow the user running ospd-openvas, to launch openvas with root permissions
gvm ALL = NOPASSWD: /opt/gvm/sbin/openvas
gvm ALL = NOPASSWD: /opt/gvm/sbin/gsad

Then exit from the root shell, and go back to to the gvm user

update nvt

greenbone-nvt-sync

upload plugins in redis with openvas

To do this you have to exit and login again as gvm So as gvm

sudo openvas -u

config and build manager

cd gvmd-9.0.0 ;\
 export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH ;\
 mkdir build ;\
 cd build/ ;\
 cmake -DCMAKE_INSTALL_PREFIX=/opt/gvm .. ;\
 make ;\
 make doc ;\
 make install ;\
 cd /opt/gvm/src

Configure PostgreSQL

For this we will use the sudo enabled user.

Then setup permissions

sudo -u postgres bash
createuser -DRS gvm
createdb -O gvm gvmd

psql gvmd
create role dba with superuser noinherit;
grant dba to gvm;
create extension "uuid-ossp";
exit
exit

Exit the sudo user shell.

fix certs

gvm-manage-certs -a

create admin user

gvmd --create-user=admin --password=admin

update feeds

greenbone-certdata-sync ;\
greenbone-scapdata-sync

update IANA Service names

This is fixed in https://github.com/greenbone/gvmd/pull/802

ToDo update this:

mkdir iana_service_ports ;\
cd iana_service_ports ;\
wget https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xml ;\
gvm-portnames-update service-names-port-numbers.xml
cd /opt/gvm/src

configure and install gsa

cd gsa-9.0.0 ;\
 export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH ;\
 mkdir build ;\
 cd build/ ;\
 cmake -DCMAKE_INSTALL_PREFIX=/opt/gvm .. ;\
 make ;\
 make doc ;\
 make install ;\
 touch /opt/gvm/var/log/gvm/gsad.log ;\
 cd /opt/gvm/src

OSPD-OPENVAS

install the virtualenv

cd src ;\
export PKG_CONFIG_PATH=/opt/gvm/lib/pkgconfig:$PKG_CONFIG_PATH ;\
virtualenv --python python3.7  /opt/gvm/bin/ospd-scanner/ ;\
source /opt/gvm/bin/ospd-scanner/bin/activate

install ospd

cd ospd-2.0.0 ;\
pip3 install . ;\
cd /opt/gvm/src

install ospd-openvas

cd ospd-openvas-1.0.0 ;\
pip3 install . ;\
cd /opt/gvm/src

create startupscripts

As root run these

cat << EOF > /etc/systemd/system/gvmd.service
[Unit]
Description=Job that runs the gvm daemon
Documentation=man:gvm
After=postgresql.service

[Service]
Type=forking
User=gvm
Group=gvm
PIDFile=/opt/gvm/var/run/gvmd.pid
WorkingDirectory=/opt/gvm
ExecStart=/opt/gvm/sbin/gvmd  --osp-vt-update=/opt/gvm/var/run/ospd.sock
Restart=on-failure
RestartSec=2min
KillMode=process
KillSignal=SIGINT
GuessMainPID=no
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

cat << EOF > /etc/systemd/system/gsad.service
[Unit]
Description=Job that runs the gsa daemon
Documentation=man:gsa
After=postgresql.service

[Service]
Type=forking
PIDFile=/opt/gvm/var/run/gsad.pid
WorkingDirectory=/opt/gvm
ExecStart=/opt/gvm/sbin/gsad --drop-privileges=gvm
Restart=on-failure
RestartSec=2min
KillMode=process
KillSignal=SIGINT
GuessMainPID=no
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

cat << EOF > /etc/systemd/system/ospd-openvas.service 
[Unit]
Description=Job that runs the ospd-openvas daemon
Documentation=man:gvm
After=postgresql.service

[Service]
Environment=PATH=/opt/gvm/bin/ospd-scanner/bin:/opt/gvm/bin:/opt/gvm/sbin:/opt/gvm/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Type=simple
User=gvm
Group=gvm
WorkingDirectory=/opt/gvm
PIDFile=/opt/gvm/var/run/ospd-openvas.pid
ExecStart=/opt/gvm/bin/ospd-scanner/bin/python /opt/gvm/bin/ospd-scanner/bin/ospd-openvas --pid-file /opt/gvm/var/run/ospd-openvas.pid --unix-socket=/opt/gvm/var/run/ospd.sock --log-file /opt/gvm/var/log/gvm/ospd-scanner.log --log-level DEBUG --lock-file-dir /opt/gvm/var/run/ospd/
Restart=on-failure
RestartSec=2min
KillMode=process
KillSignal=SIGINT
GuessMainPID=no
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload ;\
systemctl enable gvmd ;\
systemctl enable gsad ;\
systemctl enable ospd-openvas ;\
systemctl start gvmd ;\
systemctl start gsad ;\
systemctl start ospd-openvas

register your new scanner

gvmd --create-scanner="TEST OPENVAS Scanner" --scanner-type="OpenVas" --scanner-host=/opt/gvm/var/run/ospd.sock

Verify scanner.

gvm@localhost:~$ gvmd --get-scanners
08b69003-5fc2-4037-a479-93b440211c73  OpenVAS Default
6acd0832-df90-11e4-b9d5-28d24461215b  CVE
a4ac7622-9137-4783-8bb3-632e076f21b5  TEST OPENVAS Scanner ««««« THIS UUID
gvm@localhost:~$ gvmd --verify-scanner=a4ac7622-9137-4783-8bb3-632e076f21b5
Scanner version: OpenVAS 7.0.0.

Now you can point your browser to GSA https://ipnumberofyourserver and login :)
And when you try to create a new test scan, remeber to change to “TEST OPENVAS Scanner”

If you have any questions, comment below or join slack and see if someone knows the answer you seek :)
» Slack invite here

Troubleshooting. And debugging

For lab I ususally use tmux, so I can start the services in the frontend. Tmux cheatsheet in a gist courtesy of henrik » tmux_cheatsheet.markdown

gvmd

As gvm user.

tmux new -s gvmd
gvmd -f --osp-vt-update=/opt/gvm/var/run/ospd.sock

gsad

And then as a sudo user. (perhaps this is fixable with editing the sudoers file later.)

tmux new -s gsad
sudo gsad --drop-privileges=gvm  -f

ospd-openvas

ospd-openvas -f --pid-file /opt/gvm/var/run/ospd-openvas.pid \
             --unix-socket=/opt/gvm/var/run/ospd.sock \
             --log-file /opt/gvm/var/log/gvm/ospd-scanner.log \
             --log-level DEBUG

After some Coffee, it’s time for sleep…