Docker is an application that treats a whole Linux machine, including its operating system and installed applications, as a computer-within-a-computer, called a “container.” “Containers” are similar to a virtual machine in many respects. They are typically used for “shipping” applications. Instead of installing an application on a server directly, you can run the application in a “container.” This way, the application runs bundled with all of the operating system software that it needs. Installing applications is quicker, simpler, and less error-prone. There is virtually no performance degredation.
Docker is a good platform for trying out docassemble for the first time. It is also ideal in a production environment.
Since the docassemble application depends on so many different component parts, including a web server, SQL server, Redis server, distributed task queue, background task system, scheduled task system, and other components, running it inside of a Docker container is convenient. When all of these components are running inside of a “container,” you don’t have to do the work of installing and maintaining these components.
As much as Docker simplifies the process of installing docassemble, it takes some time to understand the concepts behind “running,” “stopping,” and “starting” containers.
If you are not familiar with Docker or with hosting web applications, and you want to get up and running fast, you may want to use one of the third party providers that provide docassemble-based interview development platforms.
Docker can also be used to deploy even the most complex docassemble installations. For example, Amazon’s EC2 Container Service can be used to maintain a cluster of docassemble web server instances, created from Docker images, that communicate with a central server. For information about how to install docassemble in a multi-server arrangement on EC2 Container Service (“ECS”), see the scalability section.
Docker is a complex and powerful tool, and the docassemble documentation is not a substitute for Docker documentation. If you are new to Docker, you should learn about Docker by reading tutorials or watching videos. Here is a brief cheat sheet based on loose real-world analogies:
docker runis analogous to getting a Windows installation DVD, installing it on a computer with an empty hard drive, and then booting the computer for the first time.
docker pullis analogous to going to a store and obtaining a Windows installation DVD.
docker stopis analogous to turning off a computer (and forcibly unplugging it after a certain number of seconds after you initiate the shutdown from the Windows “start” menu).
docker startis analogous to turning on a computer.
docker rmis analogous to tossing a computer into a trash incinerator.
docker rmiis analogous to tossing a Windows installation DVD into a trash incinerator.
docker execis analogous to sitting down at your computer and opening up PowerShell.
docker psis analogous to walking around your house and making a list of your computers.
docker volumeis analogous to doing things with USB drives.
docker buildis analogous to creating a Windows installation DVD based on the Windows source code.
In these analogies, a Docker “image” is analogous to a Windows installation DVD, a Docker “container” is analogous to a particular computer that runs Windows, and a Docker “volume” is (very loosely) analogous to a USB drive.
Docker can be run on a Windows PC, a Mac, an on-site Linux machine, or a Linux-based virtual machine in the cloud. Since docassemble is a web application, the ideal platform is a Linux virtual machine in the cloud.
You can test out docassemble on a PC or a Mac, but for serious, long-term deployment, it is worthwhile to run it in the cloud, or on a dedicated on-premises server. Running Docker on a machine that shuts down or restarts frequently could lead to database corruption.
If you have never deployed a Linux-based virtual machine in the cloud before, this might be a good opportunity to learn. The ability to use virtual machines on a cloud provider like Amazon Web Services or Microsoft Azure is a valuable and transferable skill. Learning how to do cloud computing is beyond the scope of this documentation, but there are many guides on the internet. The basic steps of running Docker in the cloud are:
- Create an account with a cloud computing provider.
- Start a sufficiently powerful virtual machine that runs some flavor of Linux.
- Connect to the virtual machine using SSH in order to control it using a command line. This can be a complicated step because most providers use certificates rather than passwords for authentication.
- Install Docker on the virtual machine.
There are also methods of controlling cloud computing resources from a local command line, where you type special commands to deploy Docker containers. These can be very useful, but they tend to be more complicated to use than the basic Docker command line.
First, make sure you are running Docker on a computer or virtual computer with at least 4GB of memory and 20GB of hard drive space. The docassemble installation will use up about 10GB of space, and you should always have at least 10GB free when you are running docassemble.
If you have a Windows PC, follow the Docker installation instructions for Windows. You will need administrator access on your PC in order to install (or upgrade) Docker.
If you have a Mac, follow the Docker installation instructions for OS X.
On Ubuntu (assuming username
On Amazon Linux (assuming the username
usermod line allows the non-root user to run Docker. You may
need to log out and log back in again for this new user permission to
Docker will probably start automatically after it is installed. On
Linux, you many need to do
sudo /etc/init.d/docker start,
systemctl start docker, or
sudo service docker start.
If you just want to test out docassemble for the first time, follow the instructions in this section, and you’ll get docassemble up and running quickly in a Docker container, whether you are using a laptop or AWS.
However, you should think of this as an educational exercise; don’t start using the container for serious development work. For a serious implementation, you will want to go through additional setup steps, such as configuring HTTPS for encryption and data storage for the safe, long-term storage of development data and user data.
Once Docker is installed, you can install and run docassemble from the command line.
To get a command line on Windows, run Windows PowerShell.
To get a command line on a Mac, launch the Terminal application.
To get a command line on a virtual machine in the cloud, follow your provider’s instructions for using SSH to connect to your machine.
From the command line, simply type in:
docker run command will download and run docassemble,
making the application available on the standard HTTP port (port 80)
of your machine.
It will take several minutes for docassemble to download, and once
docker run command finishes, docassemble will start to
run. After a few minutes, you can point your web browser to the
hostname of the machine that is running Docker. If you are running
Docker on your own computer, this address is probably
The number of minutes docassemble takes to start up depends on the speed of the network connection. A number of things happen during the boot process, but the main bottleneck is the updating of software packages. On some systems it may take up to 20 minutes for docassemble to boot. If you want to investigate what is happening on the server, see the troubleshooting section.
If you are running Docker on AWS, the address will be something
your EC2 configuration for the hostname). On AWS, you will need a
Security Group that opens HTTP (port 80) to the outside world in
order to allow web browsers to connect to your EC2 instance.
Using the web browser, you can log in using the default username (“firstname.lastname@example.org”) and password (“password”), and make changes to the configuration from the menu.
docker run command, the
-d flag means that the container
will run in the background.
-p flag maps a port on the host machine to a port on the
Docker container. In this example, port 80 on the host machine will
map to port 80 within the Docker container. If you are already
using port 80 on the host machine, you could use
-p 8080:80, and
then port 8080 on the host machine would be passed through to port 80
on the Docker container.
jhpyle/docassemble tag refers to a Docker image that is
hosted on Docker Hub. The image is about 2GB in size, and when it
runs, the container uses about 10GB of hard drive space. The
jhpyle/docassemble image is based on the “master” branch of the
docassemble repository on GitHub. It is rebuilt every time the
minor version of docassemble increases.
You can shut down the container by running:
By default, Docker gives containers ten seconds to shut down before
forcibly shutting them down. Usually, ten seconds is plenty of time,
but if the server is slow, docassemble might take a little longer
than ten seconds to shut down. To be on the safe side, give the
container plenty of time to shut down gracefully. The
-t 600 means
that Docker will wait up to ten minutes before forcibly shutting
down the container. It will probably take no more than 15 seconds for
docker stop command to complete, although it can take as long
as a minute to stop a container if you are using Azure Blob
It is very important to avoid a forced shutdown of docassemble. The container runs a PostgreSQL server, and the data files of the server may become corrupted if PostgreSQL is not gracefully shut down. To facilitate data storage (more on this later), docassemble backs up your data during the shutdown process and restores from that backup during the initialization process. If the shutdown process is interrupted, your data may be left in an inconsistent state and there may be errors during later initialization.
To see a list of stopped containers, run
docker ps -a. To remove a
docker rm <containerid>.
When you run
docker run on the “image”
Docker will go onto the internet, download (“pull”) the
jhpyle/docassemble image, create a new container using that image,
and then “start” that container. However, first it will check to see
if a copy of the
jhpyle/docassemble image has already been
downloaded, and if there is a copy already downloaded, it will create
the container using that copy. This is important to keep in mind;
when you run
docker run, you might be thinking you will always get
the most recent version, but that is not the case. (See upgrading,
below, for more information.)
When the docassemble container starts, it runs one command:
(This is specified in the
Dockerfile, if you are curious.)
- A web server, NGINX, which is called
nginxwithin the Supervisor configuration.
- A application server, uWSGI, called
- A background task system, Celery, called
- A scheduled task runner, called
- A SQL server, PostgreSQL, called
- A distributed task queue system, RabbitMQ, called
- An in-memory data structure store, Redis, called
- A watchdog daemon that looks for out-of-control processes and
kills them, called
- A WebSocket server that supports the live help functionality,
In addition to starting background tasks, Supervisor coordinates the running of ad-hoc tasks, including:
- A script called
syncthat consolidates log files in one place, to support the Logs interface.
- A script called
resetthat restarts the server.
- A script called
updatethat installs and upgrades the Python packages on the system.
There is also a Supervisor service called
syslogng, which is
dormant on a single-server system. (The syslog-ng application is
used in the multi-server arrangement to consolidate log files from
Finally, there is a service called
initialize, which runs
automatically when Supervisor starts. This is a shell script that
initializes the server and starts the other services in the correct
You should not need to access the running container in order to get docassemble to work, and all the log files you need will hopefully be available from Logs in the web browser. However, you might want to gain access to the running container for some reason.
To do so, find out the ID of the running container by doing
docker ps. You will see output like the following:
The ID is in the first column. Then run:
using your own ID in place of
e4fa52ba540e. This will give you a
command prompt within the running container.
The first thing to check when you connect to a container is:
The output should be something like:
If you are running docassemble in a single-server arrangement, the
processes that should be “RUNNING” include
Log files on the container that you might wish to check include:
/usr/share/docassemble/log/docassemble.log(log for the web application)
/usr/share/docassemble/log/worker.log(log for background processes)
/usr/share/docassemble/log/uwsgi.log(log for the core of the web application)
/var/log/nginx/error.log(log for the web server)
/var/log/supervisor/initialize-stderr---supervisor-*.log(log for the startup process)
/var/log/supervisor/postgres-stderr---supervisor-*.log(log for the SQL server)
- Other files in
/var/log/supervisor/(logs for other services)
/usr/share/docassemble/log/websockets.log(log for parts of the live help feature)
/var/mail/mail(log for scheduled tasks, generated by
/tmp/flask.log(log used by Flask in rare situations)
less program, you can type spacebar to go to the next
G to go to the end of the file,
1G to go to the start of the
q to quit.
exit to leave the container and get back to your standard
supervisorctl status shows that the
initialize service is in
FAILED status, then there should be an error message in
indicating what went wrong that prevented docassemble from
initializing. You will need to fix that problem, then type
leave the container, and then restart your container by doing
stop -t 600 <containerid> followed by
docker start <containerid>.
If you are get a “server error” in your web browser when trying to
access docassemble, there should be an error message in
/usr/share/docassemble/log/uwsgi.log. If you see a message about a
“blueprint’s name collision,” this is almost always not the real
error; you need to scroll up through several error messages to find
the actual error. When the web application crashes, the error that
initiated the crash causes other errors inside of the code of the
Flask framework, and a “blueprint’s name collision” error is
typically the last error to be recorded in the error log.
If you encounter a problem with upgrading or installing packages,
/usr/share/docassemble/log/worker.log. This is the error log
for the Celery background process system. A Celery background
task controls the upgrading and installation of packages, so if you
get an error during upgrading or installation of packages, make sure
to check here first.
If you need to change the Configuration but you cannot use the web
interface to do so because your container failed to start, or the web
application does not work, you can edit the Configuration manually.
The main configuration file is located at
Because of the way that data storage works, however, you need to be
careful about editing the Configuration file directly. If you are
using S3 or Azure Blob Storage,
then during the container initialization process, the file will be
overwritten with the copy of
config.yml that is stored in the cloud.
If you are not using cloud-based data storage, then when a container
safely shuts down,
/usr/share/docassemble/config/config.yml will be
/usr/share/docassemble/backup/config.yml, and when a
container starts up,
be copied to
the existing contents. This is part of the operation of the data
storage feature; it makes it possible for you to remove a container
docker run a new one while retaining all of your data.
If you are using S3 or Azure Blob
Storage, then you should
docker stop -t 600 the
container, then edit the
config.yml file through the cloud service
web interface (usually by downloading, editing, and uploading), and
docker start the container again.
If you are not using S3 or Azure Blob
Storage, then you can edit the Configuration
file using an editor like
nano. If the status of
/usr/share/docassemble/config/config.yml file, and
supervisorctl start reset to restart the docassemble
services so that they use the new Configuration. When the container
stops, it will safely shut down, and
/usr/share/docassemble/config/config.yml will be backed up to
/usr/share/docassemble/backup/config.yml. If you are using
persistent volumes, the
backup folder will be in the Docker volume
that will persist even if you
docker rm the container. If the
EXITED, then this backup
process will not take place; in that case, you should make your
/usr/share/docassemble/backup/config.yml, and then
restart your container by doing
docker stop -t 600 followed by
If you need to make manual changes to the installation of Python
packages, note that docassemble’s Python code is installed in a
Python virtual environment in which all of the files are readable
and writable by the
www-data user. If you are using Python 2.7, the
virtual environment is located at
if you are using Python 3.6, it is located at
/usr/share/docassemble/local3.6/. Thus, installing Python
packages through Debian’s
apt-get utility will not actually make
that Python code available to docassemble. Before using pip,
you need to first change the user to
www-data, and then switch into
the appropriate Python virtual environment.
If using Python 2.7, do:
If using Python 3.6, do:
Note that if you want to install a new version of a Python package
that may already be installed, you will want to use the
To stop using the Python virtual environment, type the command
deactivate. To stop being the
www-data user, type the command
Services other than NGINX and uWSGI are an important part of
docassemble’s operations. For example, the upgrading and
installation of Python packages takes place in a background process
operated by the
celery service. In addition, the live help
feature uses a service called
websockets services all need to be restarted every
time there is a change to the Configuration or a change to Python
code. To restart all of the services at once, you can do:
However, if the
uwsgi process has crashed, then you need to do:
You need to manually restart the
uwsgi process here because the
reset process uses an optimized method of refreshing the application
server. This usually works well when you make Configuration and
Python code changes, but if uWSGI has crashed,
start reset will not bring it back to life.
If you want to access the Redis data, do
docker exec to get
inside the container and then run
redis-cli (assuming that your
Redis server is the default local Redis server). Note that
docassemble uses several of the Redis databases. If you do
redis-cli -n 1 (the default), you will access the database used on a
system level. If you do
redis-cli -n 2, you will access the
database used by
Unless you specify a different SQL server, the PostgreSQL data for
your docassemble server is inside the
running on the Docker container. The default username is
docassemble and the default password is
abc123. After doing
docker exec to get inside the container, run:
When prompted, enter password
In the example above, we started
docker run -d -p 80:80 --env DAPYTHONVERSION=3 jhpyle/docassemble.
This command will cause docassemble to use default values for all
configuration options. You can also communicate specific
configuration options to the container.
The recommended way to do this is to create a text file called
env.list in the current working directory containing environment
variable definitions in standard shell script format. For example:
Then, you can pass these environment variables to the container using
docker run command:
These configuration options will cause NGINX to use
docassemble.example.com as the server name and use HTTPS with
certificates hosted on Let’s Encrypt. (The flag
-p 443:443 is
included so that the HTTPS port is exposed.)
If you want your server to be able to accept incoming e-mails, you
will need to add
-p 25:25 in order to open port 25. See the
e-mailing the interview section for information about configuring
your server to receive e-mails.
A template for the
env.list file is included in distribution.
When running docassemble in ECS, environment variables like these are specified in JSON text that is entered into the web interface. (See the scalability section for more information about using ECS.)
env.list file, you can set a variety of options. These
options are case specific, so you need to literally specify
False will not work.
The following two options are specific to the particular server being started (which, in a multi-server arrangement, will vary from server to server).
allor a colon-separated list of services (e.g.
sql:log:redis, etc.) that should be started by the server. It is only necessary to set a
CONTAINERROLEif you are using a multi-server arrangement. The available options are:
all: the Docker container will run all of the services of docassemble on a single container.
web: The Docker container will serve as a web server.
celery: The Docker container will serve as a Celery node.
sql: The Docker container will run the central PostgreSQL service.
cron: The Docker container will run scheduled tasks and other necessary tasks, such as updating SQL tables.
redis: The Docker container will run the central Redis service.
rabbitmq: The Docker container will run the central RabbitMQ service.
log: The Docker container will run the central log aggregation service.
SERVERHOSTNAME: In a multi-server arrangement, all docassemble application servers need to be able to communicate with each other using port 9001 (the supervisor port). All application servers “register” with the central SQL server. When they register, they each provide their hostname; that is, the hostname at which the specific individual application server can be found. Then, when an application server wants to send a message to the other application servers, the application server can query the central SQL server to get a list of hostnames of other application servers. This is necessary so that any one application server can send a signal to the other application servers to install a new package or a new version of a package, so that all servers are running the same software. If you are running docassemble in a multi-server arrangement, and you are starting an application server, set
SERVERHOSTNAMEto the hostname with which other application servers can find that server. Note that you do not need to worry about setting
SERVERHOSTNAMEif you are using EC2, because Docker containers running on EC2 can discover their actual hostnames by querying a specific IP address.
The other options you can set in
env.list are global for your entire
docassemble installation, rather than specific to the server being
The following eight options indicate where an existing configuration file can be found on S3 or Azure blob storage. If a configuration file exists in the cloud at the indicated location, that configuration file will be used to set the configuration of your docassemble installation. If no configuration file yet exists in the cloud at the indicated location, docassemble will create an initial configuration file and store it in the indicated location.
S3ENABLE: Set this to
trueif you are using S3 as a repository for uploaded files, Playground files, the configuration file, and other information. This environment variable, along with others that begin with
S3, populates values in
s3section of the initial configuration file. If this is unset, but
S3BUCKETis set, it will be assumed to be
S3BUCKET: If you are using S3, set this to the bucket name. Note that docassemble will not create the bucket for you. You will need to create it for yourself beforehand. The bucket should be empty.
S3ACCESSKEY: If you are using S3, set this to the S3 access key. You can ignore this environment variable if you are using EC2 with an IAM role that allows access to your S3 bucket.
S3SECRETACCESSKEY: If you are using S3, set this to the S3 access secret. You can ignore this environment variable if you are using EC2 with an IAM role that allows access to your S3 bucket.
S3REGION: If you are using S3, set this to the region you are using (e.g.,
S3ENDPOINTURL: If you are using an S3-compatible data storage service, set
S3ENDPOINTURLto the URL of the service (e.g.,
AZUREENABLE: Set this to
trueif you are using Azure blob storage as a repository for uploaded files, Playground files, the configuration file, and other information. This environment variable, along with others that begin with
AZURE, populates values in
azuresection of the configuration file. If this is unset, but
AZURECONTAINERare set, it will be assumed to be
AZURECONTAINER: If you are using Azure blob storage, set this to the container name. Note that docassemble will not create the container for you. You will need to create it for yourself beforehand.
AZUREACCOUNTNAME: If you are using Azure blob storage, set this to the account name.
AZUREACCOUNTKEY: If you are using Azure blob storage, set this to the account key.
The following options are “setup” parameters that are useful for
pre-populating a fresh configuration with particular values. These
environment variables are effective only during an initial
the Docker container, when a configuration file does not already
Note: if you are using persistent volumes, or you have set the
options above for
S3/Azure blob storage
and a configuration file exists in your cloud storage, the values
in that configuration file will take precedence over any values you
DAWEBSERVER: This can be set either to
nginx(the default) or
apache. See the
web serverconfiguration directive.
DBHOST: The hostname of the PostgreSQL server. Keep undefined or set to
nullin order to use the PostgreSQL server on the same host. This environment variable, along with others that begin with
DB, populates values in
dbsection of the configuration file.
DBNAME: The name of the PostgreSQL database. The default is
DBUSER: The username for connecting to the PostgreSQL server. The default is
DBPASSWORD: The password for connecting to the SQL server. The default is
abc123. The password cannot contain the character
DBPREFIX: This sets the prefix for the database specifier. The default is
DBPORT: This sets the port that docassemble will use to access the SQL server. If you are using the default port for your database backend, you do not need to set this.
DBTABLEPREFIX: This allows multiple separate docassemble implementations to share the same SQL database. The value is a prefix to be added to each table in the database.
EC2: Set this to
trueif you are running Docker on EC2. This tells docassemble that it can use an EC2-specific method of determining the hostname of the server on which it is running. See the
USEHTTPS: Set this to
trueif you would like docassemble to communicate with the browser using encryption. Read the HTTPS section for more information. Defaults to
false. See the
use httpsconfiguration directive. Do not set this to
trueif you are using a load balancer that forwards non-encrypted HTTP to your server.
DAHOSTNAME: Set this to the hostname by which web browsers can find docassemble. This is necessary for HTTPS to function. See the
external hostnameconfiguration directive.
USELETSENCRYPT: Set this to
trueif you are using Let’s Encrypt. The default is
false. See the
use lets encryptconfiguration directive.
LETSENCRYPTEMAIL: Set this to the e-mail address you use with Let’s Encrypt. See the
lets encrypt emailconfiguration directive.
LOGSERVER: This is used in the multi-server arrangement where there is a separate server for collecting log messages. The default is
none, which causes the server to run Syslog-ng. See the
log serverconfiguration directive.
REDIS: If you are running docassemble in a multi-server arrangement, set this to the host name at which the Redis server can be accessed. See the
RABBITMQ: If you are running docassemble in a multi-server arrangement, set this to the host name at which the RabbitMQ server can be accessed. Note that RabbitMQ is very particular about hostnames. If the RabbitMQ server is running on a machine on which the
hostnamecommand evaluates to
abc, then your application servers will need to set
abcand nothing else. It is up to you to make sure that
abcresolves to an IP address. Note that if you run docassemble using the instructions in the scalability section, you do not need to worry about this. See the
SERVERADMIN: If your docassemble web server generates an error, the error message will contain an e-mail address that the user can contact for help. This e-mail address defaults to
webmaster@localhost. You can set this e-mail address by setting the
SERVERADMINenvironment variable to the e-mail address you want to use. See the
server administrator emailconfiguration directive.
POSTURLROOT: If users access docassemble at https://docassemble.example.com/da, set
/da/. The trailing slash is important. If users access docassemble at https://docassemble.example.com, you can ignore this. The default value is
/. See the
BEHINDHTTPSLOADBALANCER: Set this to
trueif a load balancer is in use and the load balancer accepts connections in HTTPS but forwards them to web servers as HTTP. This lets docassemble know that when it forms URLs, it should use the
httpsscheme even though requests appear to be coming in as HTTP requests. See the
behind https load balancerconfiguration directive.
XSENDFILE: Set this to
falseif the X-Sendfile header is not functional in your configuration for whatever reason. See the
DAUPDATEONSTART: Set this to
falseif you do not want the container to update its software using
pipwhen it starts up. Set
initialif you want the container to update its software during the first
docker run, but not on every
TIMEZONE: You can use this to set the time zone of the server. The value of the variable is stored in
dpkg-reconfigure -f noninteractive tzdatais run in order to set the system time zone. The default is
America/New_York. See the
LOCALE: You can use this to enable a locale on the server. When the server starts, the value of
LOCALEis appended to
update-localeare run. The default is
en_US.UTF-8 UTF-8. See the
os localeconfiguration directive.
OTHERLOCALES: You can use this to set up other locales on the system besides the default locale. Set this to a comma separated list of locales. The values need to match entries in Debian’s
/etc/locale.gen. See the
other os localesconfiguration directive.
PACKAGES: If your interviews use code that depends on certain Debian packages being installed, you can provide a comma-separated list of Debian packages in the
PACKAGESenvironment variable. The packages will be installed when the container is started. See the
debian packagesconfiguration directive.
PYTHONPACKAGES: If you want to install certain Python packages during the container start process, you can provide a comma-separated list of packages in the
PYTHONPACKAGESenvironment variable. See the
python packagesconfiguration directive.
DASECRETKEY: The secret key for protecting against cross-site forgery. See the
secretkeyconfiguration directive. If
DASECRETKEYis not set, a random secret key will be generated.
DABACKUPDAYS: The number of days backups should be kept. The default is 14. See the
backup daysconfiguration directive.
DAEXPOSEWEBSOCKETS: You may need to set this to
trueif you are operating a Docker container behind a reverse proxy and you want to use the WebSocket-based live help features. See the
expose websocketsconfiguration directive.
DAWEBSOCKETSIP: You can set this if you need to manually specify the address on which the
websocketsservice runs. See the
websockets ipconfiguration directive.
DAWEBSOCKETSPORT: You can set this if you need to manually specify the port on which the
websocketsservice runs. See the
websockets portconfiguration directive.
DAPYTHONVERSION: Set this to
3in order to run docassemble using Python 3.6. If
DAPYTHONVERSIONis not set, docassemble will be run using Python 2.7. If you have an existing docassemble instance that uses Python 2.7 (which is the default unless
DAPYTHONVERSIONis set to
3), you will not want to change this until you have done testing. See migration from Python 2.7 to Python 3.6 for more information.
If you already have an existing docassemble installation and you
run a new Docker container using it, but you want to
change the configuration of the container, there are some things you
will need to keep in mind.
When docassemble starts up on a Docker container, it:
- Creates a configuration file from a template, using environment variables for initial values, if a configuration file does not already exist.
- Initializes a PostgreSQL database, if one is not already initialized.
- Configures the NGINX configuration, if one is not already configured.
- Runs Let’s Encrypt if the configuration indicates that Let’s Encrypt should be used, and Let’s Encrypt has not yet been configured.
When docassemble stops, it saves the configuration file, a backup of the PostgreSQL database, and backups of the Let’s Encrypt configuration. If you are using persistent volumes, the information will be stored there. If you are using S3 or Azure blob storage, the information will be stored in the cloud.
When docassemble starts again, it will retrieve the configuration file, the backup of the PostgreSQL database, and backups of the Let’s Encrypt configuration from storage and use them for the container.
Suppose you have an existing installation that uses HTTPS and Let’s
Encrypt, but you want to change the
DAHOSTNAME. You will need to
delete the saved configuration before running a new container. First,
shut down the machine with
docker stop -t 600. Then, if you are using
S3, you can go to the S3 Console and delete the
“letsencrypt.tar.gz” file. If you are using Azure blob
storage, you can go to the Azure Portal and
delete the “letsencrypt.tar.gz” file.
Also, if a configuration file exists on S3/Azure blob storage (
config.yml) or in a
persistent volume, then the values in that configuration will take
precedence over the corresponding environment variables that are
passed to Docker. Once a configuration file exists, you should make
changes to the configuration file rather than passing environment
variables to Docker. However, if your configuration is on
S3/Azure blob storage, you will
at least need to pass sufficient access keys (e.g.,
AZURECONTAINER, etc.) to access that storage; otherwise your
container will not know where to find the configuration.
Also, there are some environment variables that do not exist in the
configuration file because they are specific to the individual server
being started. These include the
SERVERHOSTNAME environment variables.
Docker containers are volatile. They are designed to be run, turned off, and destroyed. When using Docker, the best way to upgrade docassemble to a new version is to destroy and rebuild your containers.
But what about your data? If you run docassemble, you are
accumulating valuable data in SQL, in files, and in Redis. If your
data are stored on the Docker container, they will be destroyed by
There are two ways around this problem. The first, and most
preferable solution, is to get an account on Amazon Web Services
(AWS) or Microsoft Azure. If you use AWS, create an S3 bucket
for your data, and then when you launch your container, set the
S3BUCKET and associated environment variables.
If you use Microsoft Azure, create an Azure blob storage resource,
and a blob storage container within it, and then when you launch
your container, set the
AZURECONTAINER environment variables.
docker stop -t 600 is run, docassemble will backup the SQL
database, the Redis database, the configuration, and your uploaded
files to your S3 bucket or blob storage container. Then, when you
docker run command with environment variables pointing
docassemble to your S3 bucket/Azure blob storage resource,
docassemble will make restore from the backup. You can
rm your container and your data will persist in the cloud.
The second way is to use persistent volumes, which is a feature of Docker. This will store the data in directories on the Docker host, so that when you destroy the container, these directories will be untouched, and when you start up a new container, it will use the saved directories.
These two options are explained in the following subsections.
To use S3 for persistent storage, sign up with
Amazon Web Services, go to the S3 Console, click “Create Bucket,”
and pick a name. If your site is at docassemble.example.com, a good
name for the bucket is
docassemble-example-com. (Client software
will have trouble accessing your bucket if it contains
characters.) Under “Region,” pick the region nearest you.
Then you need to obtain an access key and a secret access key for
S3. To obtain these credentials, go to IAM Console and create a
user with “programmatic access.” Under “Attach existing policies
directly,” find the policy called
AmazonS3FullAccess and attach it
to the user.
Note that if you run docassemble on EC2, you can launch your
EC2 instances with an IAM role that allows docassemble to
access to an S3 bucket without the necessity of setting
S3SECRETACCESSKEY. In this case, the only
environment variable you need to pass is
These secret access keys will become available to all developers who
use your docassemble server, since they are in the configuration
file. If you want to limit access to a particular bucket, you do not
have to use the
AmazonS3FullAccess policy when obtaining S3
credentials. Instead, you can create your own policy with the
docassemble-example-com in the above text with the name of
your S3 bucket.
Using Microsoft Azure is very similar to using S3. From the Azure Portal dashboard, search for “Storage accounts” in the “Resources.” Click “Add” to create a new storage account. Under “Account kind,” choose “BlobStorage.” Under “Access tier,” you can choose either “Cool” or “Hot,” but you may have to pay more for “Hot.”
Once the storage account is created, go into your “Blobs” service in
the storage account and click “+ Container” to add a new container.
Set the “Access type” to “Private.” The name of the container
corresponds with the
AZURECONTAINER environment variable. Back at
the storage account, click “Access keys.” The “Storage account name”
corresponds with the environment variable
“key1” corresponds with the
AZUREACCOUNTKEY environment variable.
(You can also use “key2.”). For example, you might use an
file such as:
To run docassemble in a single-server arrangement in such a way that the configuration, the Playground files, the uploaded files, and other data persist after the Docker container is removed or updated, run the image as follows:
--env-file=env.list is an optional parameter that refers to a
env.list containing environment variables for the
configuration. A template for the
env.list file is included in
An advantage of using persistent volumes is that you can completely
replace the docassemble container and rebuild it from scratch, and
jhpyle/docassemble image again, docassemble will
keep running where it left off.
If you are using HTTPS with your own certificates (as opposed to
using Let’s Encrypt), you can use a persistent volume to provide the
certificates to the Docker container. Just add
dacerts:/usr/share/docassemble/certs to your
docker run command.
To see what volumes exist on your Docker system, you can run:
For example, if you are using HTTPS with your own certificates, and
you need to update the certificates your server should use, you can
find the path where the
dacerts volume lives (
docker volume inspect
dacerts), copy your certificates to that path (
/var/lib/docker/volumes/dacerts/data/docassemble.key), and then stop
the container (
docker stop -t 600 <containerid>) and start it again
docker start <containerid>).
To delete all of the volumes, do:
- S3 and Azure blob storage make scaling easier. They are the “cloud” way of storing persistent data, at least until cloud-based network file systems become more robust.
- It is easier to upgrade your virtual machines to the latest software and operating system if you can just destroy them and recreate them, rather than running update scripts. If your persistent data is stored in the cloud, you can destroy and recreate virtual machines at will, without ever having to worry about copying your data on and off the machines.
However, you can get around the second problem by using
volume create to put your Docker volume on a separate drive. That
way, you could remove the virtual machine that runs the application,
along with its primary drive, without affecting the drive with the
Services on different machines
The docassemble application consists of several services, some of which are singular and some of which can be plural.
The singular services include:
- RabbitMQ for coordinating background processes
- The docassemble log message aggregator
- A cron service that runs scheduled tasks and housekeeping functions
The (potentially) plural services include:
- Web servers
- Celery nodes
The docassemble Docker container will run any subset of these
six services, depending on the value of the environment variable
CONTAINERROLE, which is passed to the container at startup. In a
single-server arrangement (
all, or left
undefined), the container runs all of the services (except the log
message aggregator, which is not necessary in the case of a
In a multi-server arrangement, you can have one machine run SQL, another machine run Redis and RabbitMQ, and any number of machines run web servers and Celery nodes. You can decide how to allocate services to different machines. For example, you might want to run central tasks on a powerful server, while running many web servers on less powerful machines.
Since the SQL, Redis, and RabbitMQ services are standard services, they do not have to be run from docassemble Docker containers. For example, if you are already running a SQL server, a Redis server, and a RabbitMQ server, you could just point docassemble to those resources.
- Regardless of the
CONTAINERROLE, port 9001 needs to be forwarded so that the container can be controlled via supervisor.
sql: forward port 5432 (Postgresql)
web: forward ports 80 (HTTP) and 443 (HTTPS)
log: forward ports 514 (Syslog-ng) and 8080 (custom web server)
redis: forward port 6379 (Redis)
rabbitmq: forward ports 4369, 5671, 5672, and 25672 (RabbitMQ).
Note that Docker will fail if any of these ports is already in use.
For example, many Linux distributions run a mail transport agent on
port 25 by default; you will have to stop that service in order to
start Docker with
-p 25:25. For example, on Amazon Linux you
may need to run:
If you run multiple docassemble Docker containers on different machines, the containers will need to have a way to share files with one another.
One way to share files among containers is to make
/usr/share/docassemble/ a persistent volume on a network file
system. This directory contains the configuration, SSL certificates,
Python virtual environment, and uploaded files. However, network
file systems present problems.
Note that when you use the cloud (S3 or
Azure blob storage) for data storage,
docassemble will copy the
config.yml file out of the cloud on
startup, and save
config.yml to the cloud whenever the configuration
This means that as long as there is a
config.yml file in the cloud
with the configuration you want, you can start docassemble
containers without specifying a lot of configuration options; you
simply have to refer to your cloud storage bucket/container, and
docassemble will take it from there. For example, to run a
central server, you can do:
To run an application server, you can do:
If you are running docassemble on EC2, the easiest way to enable HTTPS support is to set up an Application Load Balancer that accepts connections in HTTPS format and forwards them to the web servers in HTTP format. In this configuration Amazon takes care of creating and hosting the necessary SSL certificates.
If you are running docassemble in a single-server arrangement, or in a multi-server arrangement with only one web server, you can use Let’s Encrypt to enable HTTPS. If you have more than one web server, you can enable encryption without Let’s Encrypt by installing your own certificates.
To use Let’s Encrypt, set the following environment variables in
your task definition or
USELETSENCRYPT: set this to
LETSENCRYPTEMAIL: Let’s Encrypt requires an e-mail address, which it will use to get in touch with you about renewing the SSL certificates.
DAHOSTNAME: set this to the hostname that users will use to get to the web application. Let’s Encrypt needs this in order to verify that you have access to the host.
USEHTTPS: set this to
For example, your
env.list may look like:
The first time the server is started, the
letsencrypt utility will
be run, which will change the NGINX configuration in order to use the
appropriate SSL certificates. When the server is later restarted,
letsencrypt renew command will be run, which will refresh the
certificates if they are within 30 days of expiring.
In addition, a script will run on a weekly basis to attempt to renew the certificates.
If you are using a multi-server arrangement with a single web
server, you need to run the
cron role on the same server that runs
web role. If you use the e-mail receiving feature with
TLS encryption, the
Using your own SSL certificates with Docker requires that your SSL certificates reside within each container. There are several ways to accomplish this:
- Use S3 or Azure blob storage and upload the certificates to your bucket/container.
- Build your own private image in which your SSL certificates are
Docker/ssl/nginx.ca.pem. During the build process, these files will be copied into
- Use persistent volumes and copy the SSL certificate files
nginx.ca.pem) into the volume for
/usr/share/docassemble/certsbefore starting the container.
The default NGINX configuration file expects SSL certificates to be located in the following files:
The meaning of these files is as follows:
nginx.crt: this file is generated by your certificate authority when you submit a certificate signing request.
nginx.key: this file is generated at the time you create your certificate signing request.
In order to make sure that these files are replicated on every web
server, the supervisor will run the
docassemble.webapp.install_certs module before starting the web
If you are using S3 or Azure blob storage,
this module will copy the files from the
certs/ prefix in your
/etc/ssl/docassemble. You can use the S3
Console or the Azure Portal to create a folder called
upload your certificate files into that folder.
There are two ways that you can put your own certificate files into
/usr/share/docassemble/certs directory. The first way is to
create your own Docker image of docassemble and put your
certificates into the
Docker/ssl directory. The contents of this
directory are copied into
/usr/share/docassemble/certs during the
The second way is to use persistent volumes. If you have a
persistent volume called
certs for the directory
/usr/share/docassemble/certs, then you can run
inspect certs to get the directory on the Docker host corresponding
to this directory, and you can copy the SSL certificate files into
that directory before starting the container.
Note that the files need to be called
because this is what the standard web server configuration expects.
If you want to use different filesystem or cloud locations, the
docassemble.webapp.install_certs can be configured to use different
locations. See the configuration variables
cert install directory.
If you use the e-mail receiving feature, you can use TLS to
encrypt incoming e-mail communications. By default, docassemble
will install self-signed certificates into the Exim configuration,
but for best results you should use certificates that match your
incoming mail domain.
However, if you are running your
web, you will need to create
and install your own certificates. In addition, if your
incoming mail domain is different from your
DAHOSTNAME), then you will also need to install your own
If you are using S3 or
Azure blob storage, copy your certificate and
private key to the
certs folder of your S3 bucket or
Azure blob storage container, using the filenames
docassemble.webapp.install_certs will copy these files
into the appropriate location (
/etc/exim4) with the appropriate
ownership and permissions.
Then download docassemble:
To make changes to the configuration of the docassemble application that will be installed in the image, edit the following files:
docassemble/Dockerfile: you may want to change the locale and the Debian mirror; the standard “httpredir” mirror can lead to random packages not being downloaded, depending on which mirrors it chooses to use.
docassemble/Docker/config/config.yml.dist: you probably do not need to change this; it is a template that is updated based on the contents of the environment variables passed to
docker run. Once your server is up and running you can change the rest of the configuration in the web application.
docassemble/Docker/initialize.sh: this script updates
config.ymlbased on the environment variables; retrieves a new version of
config.ymlfrom S3/Azure blob storage, if available; if
CONTAINERROLEis not set to
web, starts the PostgreSQL server and initializes the database if it does not exist; creates the tables in the database if they do not already exist; copies SSL certificates from S3/Azure blob storage or
/usr/share/docassemble/certsif S3/Azure blob storage is not enabled; runs the Let’s Encrypt utility if
trueand the utility has not been run yet; and starts NGINX and other background tasks.
docassemble/Docker/config/nginx-http.dist: NGINX configuration file for handling HTTP requests.
docassemble/Docker/config/nginx-ssl.dist: NGINX configuration file for handling HTTPS requests.
docassemble/Docker/config/nginx-log.dist: NGINX configuration file for handling requests on port 8080. This is enabled if the
docassemble/Docker/ssl/nginx.crt.orig: default SSL certificate for NGINX.
docassemble/Docker/ssl/nginx.key.orig: default SSL certificate for NGINX.
docassemble/Docker/ssl/exim.crt.orig: default SSL certificate for Exim.
docassemble/Docker/ssl/exim.key.orig: default SSL certificate for Exim.
docassemble/Docker/docassemble-supervisor.conf: supervisor configuration file.
docassemble/Docker/docassemble-syslog-ng.conf: Syslog-ng configuration file used when
CONTAINERROLEdoes not include
docassemble/Docker/syslog-ng.conf: Syslog-ng configuration file used when
docassemble/Docker/rabbitmq.config: RabbitMQ configuration file.
docassemble/Docker/docassemble.logrotate: This file will be copied into
/etc/logrotate.dand will control the rotation of the docassemble log file in
docassemble/Docker/nginx.logrotate: This replaces the standard nginx logrotate configuration. It does not compress old log files, so that it is easier to view them in the web application.
docassemble/Docker/process-email.sh: This is a script that is run when an e-mail is received, if the e-mail receiving feature is configured.
docassemble/Docker/run-nginx.sh: This is a script that is run by supervisor to start the NGINX server.
docassemble/Docker/run-uwsgi.sh: This is a script that is run by supervisor to start the uWSGI server.
docassemble/Docker/run-celery.sh: This is a script that is run by supervisor to start the Celery server.
docassemble/Docker/run-cron.sh: This is a script that is run by supervisor to start the cron daemon.
docassemble/Docker/run-postgresql.sh: This is a script that is run by supervisor to start the PostgreSQL server.
docassemble/Docker/run-rabbitmq.sh: This is a script that is run by supervisor to start the RabbitMQ server.
docassemble/Docker/run-redis.sh: This is a script that is run by supervisor to start the Redis server.
docassemble/Docker/run-syslogng.sh: This is a script that is run by supervisor to start the Syslog-ng daemon.
docassemble/Docker/run-websockets.sh: This is a script that is run by supervisor to start the WebSocket server.
docassemble/Docker/reset.sh: This is a script that is run by supervisor to restart the web server, the Celery server, and the WebSocket server on a signal from a peer server.
docassemble/Docker/sync.sh: This is a script that is run by supervisor to synchronize log files.
docassemble/Docker/update.sh: This is a script that is run by supervisor to update the software on the container.
To build the image, run:
You can then run your image:
Or push it to Docker Hub:
New versions of the docassemble software are published frequently. Most changes only affect the Python code. You can upgrade the docassemble Python packages by going to “Package Management” from the menu and clicking the “Upgrade” button.
However, sometimes a “system upgrade” is necessary. This can happen
when changes are made to docassemble’s underlying operating system
files. When it is time for a “system upgrade,” you will see a message
on the Configuration screen that says “A new docassemble system
version is available. If you are using Docker, install a new Docker
image.” Performing a “system upgrade” requires retrieving a new
docassemble Docker image and running
docker run to start a new
The first time you use
docker run to start a container, Docker
will download the image from Docker Hub, store it on your system,
and then create a new container from that image. However, subsequent
docker run commands will always use the version of the image that
is stored on your system, even if a new version is available on
You can download the latest version of docassemble to your system by running:
docker run commands will use the latest
docassemble image. This means that when you are using Docker,
you can upgrade docassemble to the newest version by running
docker stop -t 600 on your existing docassemble container,
docker rm, followed by
jhpyle/docassemble-os, followed by
docker pull jhpyle/docassemble,
and then running whatever
docker run command you use to start a
Note, however, that
docker rm will delete all of the data on the
server. This is not a problem if your
docker run command
instructs docassemble to use a data storage system; in that case,
when your new container starts up, it will use the SQL server, files,
and other information that were backed up when you did
Note also that
docker pull may use up a lot of disk space. This
is because Docker does not automatically delete old versions of
images, and docassemble images are very large. So if your disk
space is limited, you probably don’t want to run
docker pull until
you get rid of the old images. (See the next section.)
Thus, so long as you are using data storage, and you aren’t running any applications other than docassemble using Docker, it is recommended that you perform a system upgrade by running:
Then, run whatever
docker run command you use to launch
When you do
docker run or
docker pull, the only image available on
Docker Hub is the “latest” image. To install a version based on an
earlier version of docassemble, you can make your own image using
Starting with version 0.5, the docassemble image is split into two
jhpyle/docassemble image uses
a base image. The
jhpyle/docassemble-os image consists of the
underlying Debian operating system with required Debian packages
jhpyle/docassemble-os image is updated much less
frequently than the
jhpyle/docassemble image. If you want to build
your own version of
jhpyle/docassemble-os, you can do so by running:
jhpyle/docassemble image incorporates by reference the
jhpyle/docassemble-os base image. The
docker build command
above overwrites the
jhpyle/docassemble-os image that is stored on
your local machine. If you want, you can edit the Dockerfile before
building your custom
jhpyle/docassemble version so that it
references a different base image.
The versioning of the docassemble-os repository on GitHub follows that of the docassemble repository. The two repositories are maintained together. However, the latest version of the docassemble-os repository is usually several versions behind that of the docassemble repository.
If you run
docker pull to retrieve new versions of docassemble,
or you build your own docassemble images more than once, you may
find your disk space being used up. The full docassemble image is
about 4GB in size, and whenever you run
docker pull or build a new
image, a new image is created – the old image is not overwritten.
The following three lines will stop all containers, remove all containers, and then remove all of the images that Docker created during the build process.
The last line, which deletes images, frees up the most disk space. It
is usually necessary to remove the containers first (the
line), as the containers depend on the images.