Setup¶

Requirements¶

Assuming Debian GNU/Linux 11+
for LDAP: libldap2-dev, libsasl2-dev, python3-dev
python 3.9+
pip, preferably in a virtual environment
npm
see Dockerfile for a list of required packages
Podman/Docker (for clamav)

Bundle static assets¶

npm install && npm run build

Set environment via .env file¶

Copy env.template to .env and set your environment variables.

Start wagtail-api-forms¶

source path/to/your/venv/bin/activate
(venv) pip install -r requirements.txt
(venv) ./manage.py createsuperuser
(venv) ./manage.py runserver

Site & Branding¶

Set site config via: http://fqdn/admin/sites/
(In case of multilingualism) Setup root pages for every language version: http://fqdn/admin/pages/
Set brand name, logo etc via: http://fqdn/admin/settings/home/brandingsettings/1/
Set footer links (Privacy policy etc.) via: http://fqdn/admin/snippets/home/footerlinks/

Misc¶

API¶

Endpoints¶

http://fqdn/api/formsubmission/v2/ (paginated)

Add API users¶

from django.contrib.auth import get_user_model
from rest_framework.authtoken.models import Token

# Add django user without password:
user = get_user_model().objects.create_user("username")
user.save()

# Generate token for that user:
Token.objects.create(user=user)

# or via management command:
# ./manage.py drf_create_token <username>

# or via django-admin:
# http://fqdn/django-admin/authtoken/tokenproxy/

…and set this user in wagtail admin for the desired form page in page settings. For translated forms (e.g. two formpages if EN+DE), the api user must be set for both form pages.

Documentation¶

Documentation files are build with Sphinx and served via WhiteNoise from /docs/.

Build the documentation with

make --directory docs/ html

Backup¶

.env - your configured environment
_data/[media|attachments|db] - your sqlite database and uploaded files

Restore the database from _data/db/db.snapshot.sqlite3, not from the live _data/db/db.sqlite3.

The live database runs in WAL mode. Copying db.sqlite3 while the app is running does not give a usable backup, even with the db.sqlite3-wal sidecar copied alongside it: the copy is not atomic, so a checkpoint running between the two reads can leave the pair missing committed transactions. This was already true before WAL, for a single-file copy. The huey consumer therefore writes a consistent snapshot to db.snapshot.sqlite3 every hour, using VACUUM INTO.

Note

The snapshot is only refreshed while the huey consumer is running. If huey is down the file goes stale silently, so monitor its modification time.

Virus scanning¶

Virus scanning is implemented in a ClamAV docker container and a django task queue (huey).

Note

Virus scanning could be disabled via .env setting FORMBUILDER_USE_ANTIVIR_SERVICE=false.

docker-compose up  

# or rootful via podman, for now:
podman-compose --podman-run-args='--replace' up

manage.py run_huey

Localization¶

./manage.py makemessages --locale de --ignore=assets/* --ignore=node_modules/* --ignore=staticfiles/* --ignore=.venv/*
./manage.py compilemessages --ignore=assets/* --ignore=node_modules/* --ignore=staticfiles/* --ignore=.venv/*

Production environment¶

Apache+mod_wsgi

If using apache2 and mod_wsgi, do not store huey database in global /tmp/ as apache2 is running with PrivateTmp and the mod_wsgi process could not put its tasks on the queue (https://stackoverflow.com/questions/68185057/huey-db-task-successfully-registered-by-consumer-but-does-not-receive-execut).
Apache/mod_wsgi: WSGIPassAuthorization On (https://www.django-rest-framework.org/api-guide/authentication/#apache-mod_wsgi-specific-configuration)

Configuration examples¶

Webserver and media /attachments directory

Do not allow the webserver to serve _data/attachments/ files - these files are served by Django to check for various criteria (is authenticated, from whitelisted remote ip, virus checked etc.).

Webserver (Apache)¶

<VirtualHost *:443>
    ServerName fqdn
    ServerAdmin mail@fqdn

    Alias /static /path/to/wagtail_api_forms/staticfiles
    <Directory /path/to/wagtail_api_forms/staticfiles>
            Require all granted
    </Directory>

    Alias /media /path/to/wagtail_api_forms/media
    <Directory /path/to/wagtail_api_forms/media>
            Require all granted
    </Directory>

    <Directory /path/to/wagtail_api_forms/wagtail_api_forms>
            <Files wsgi.py>
                    Require all granted
            </Files>
    </Directory>

    WSGIDaemonProcess fqdn \
        home=/path/to/wagtail_api_forms \
        user=username \
        group=groupname \
        python-path=/path/to/wagtail_api_forms \
        python-home=/path/to/venv \
        processes=2 \
        threads=2 \
        maximum-requests=10000

    WSGIProcessGroup fqdn
    WSGIPassAuthorization On
    WSGIScriptAlias / /path/to/wagtail_api_forms/wagtail_api_forms/wsgi.py process-group=fqdn

</VirtualHost>

Task Queue (huey)¶

# user$ ~./config/systemd/user/wagtailapiforms-huey.service
#
# Prerequisites
#
# In ~/.bashrc (?)
# export XDG_RUNTIME_DIR="/run/user/$UID"
# export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"
#
# root$ loginctl enable-linger $username
# user$ systemctl enable --user wagtailapiforms-huey.service
# user$ systemctl start --user wagtailapiforms-huey.service
# user$ systemd-analyze  --user security  wagtailapiforms-huey.service

[Unit]
Description=Huey Service for wagtail_api_forms
After=network.target

[Service]
Restart=always
RestartSec=30
WorkingDirectory=/path/to/wagtail_api_forms
ExecStart=/path/to/venv/bin/python3 \
    /path/to/wagtail_api_forms/manage.py run_huey \
    --logfile=/path/to/wagtailapiforms-huey.log
ExecReload=/bin/kill -s SIGHUP $MAINPID
ExecStop=/bin/kill -s SIGINT $MAINPID

[Install]
WantedBy=default.target

Production deployment steps (eg. git post receive hook)¶

#!/bin/bash

set -o errexit

# Derived at runtime — assumes:
#   - bare repo is named <project_name>.git (CWD when this hook fires)
#   - work tree lives at $HOME/<project_name>
#   - Django package name = project_name with hyphens converted to underscores
project_name="$(basename "$PWD" .git)"
project_pkg="${project_name//-/_}"
worktree="$HOME/$project_name"

GIT_WORK_TREE="$worktree" git checkout -f main

echo "Change directory to: $worktree ..."
cd "$worktree"

echo "==================================================="
echo "Environment"
echo "==================================================="
echo "user:         $(whoami)"
echo "home:         $HOME"
echo "pwd:          $PWD"
echo "project_name: $project_name"
echo "project_pkg:  $project_pkg"
echo "worktree:     $worktree"

echo "==================================================="
echo "Ensure docker-compse is up"
echo "==================================================="

docker-compose down
docker-compose up --build --force-recreate --no-deps --detach

echo "==================================================="
echo "Deploy django backend"
echo "==================================================="

echo "Activate virtual environment..."
source "$HOME/venv/bin/activate"

echo "Ensure pip and wheel are uptodate..."
pip install --upgrade pip wheel setuptools

echo "Install pip requirements..."
pip install -r requirements.txt

echo "[npm] Installing npm requirements..."
npm install --quiet

echo "Running app-level deploy steps (scripts/deploy.sh)..."
./scripts/deploy.sh

echo "==================================================="
echo "Restart huey"
echo "==================================================="

# Required for `systemctl --user` from a non-login shell.
export XDG_RUNTIME_DIR="/run/user/$UID"
export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"
systemctl --user restart "${project_pkg}_huey.service"
echo "Huey restarted!"

echo "Reload app..."
touch "$worktree/$project_pkg/wsgi.py"
echo "Reloaded app!"

echo "Run check --deploy..."
python3 manage.py check --deploy
echo "Done running check --deploy"