Starting with version 0.4.0, docassemble is compatible with both
Python 3.6 and Python 2.7. By default, docassemble will continue
to use Python 2.7, but if you wish to run docassemble using Python
3.6, you can set the Docker environment variable
Users who are getting started for the first time with docassemble
should launch their instances with
DAPYTHONVERSION set to
On or around January 1, 2020, the default in docassemble will switch from Python 2.7 to Python 3.6.
However, there are a few things you should be aware of:
- Interview answers created using Python 3.6 will not be readable by Python 2.7. This is because the way the pickle library works changed in a backwards-incompatible way between Python 2.7 and Python 3.
- Interview answers created using Python 2.7 will be readable by
Python 3.6, so long as your interviews use standard docassemble
features. Date objects based on
datetime.datetimewill migrate. However, if you used objects from third-party libraries, you should test to see whether interview answers created in Python 2.7 are readable in Python 3.6. There are no known issues yet, but there might be issues.
- This applies not just to interview answers, but also to any objects that are pickled, such as machine learning data and objects stored in Redis or SQL.
- You can go back and forth between Python 2.7 and 3.6 by changing the
docker pulland a new
docker run. However, you may get errors in Python 2.7 if you try to access Python objects created using Python 3.6.
- If you have written Python code in your interviews, or you have
written Python modules, the code that you wrote that worked in
Python 2.7 might not work in Python 3.6. The way text is encoded
and decoded changed, the
unicode()function was removed, the
itervalues()methods were removed, the
longdata type was removed, the
basestringclass was removed, the
xrange()function was removed, the
apply()functions were removed, the
fileclass was removed, and much more.
If you do not have any active users, you could simply do a
docker pull, and
docker run with
DAPYTHONVERSION to 3, and then test out your interviews and fix
any errors that appear.
If you have active users, however, an immediate upgrade might result in downtime if you find that your interview doesn’t work in Python 3.6. And if interview answers or other data created by your Python 2.7 interviews cannot be unpickled in Python 3.6, your users who have active sessions at the time you upgrade the server may lose their data. The following procedure will avoid these problems:
- Continue to operate your production server with Python 2.7 (the default).
- Run two development servers: one that runs Python 2.7 (the default)
and one that runs Python 3.6 (setting
DAPYTHONVERSIONto 3). Both should be running docassemble version 0.4.0 or above, and both should use some form of data storage.
- Using your two development servers, rewrite your interview code so
that it is compatible both with Python 2.7 and Python 3.6. See
writing Python 2-3 compatible code for tips. For example, if you
unicode()function, replace it with the
text_type()function. Keep upgrading your code on your production server, using your normal process, after verifying that your changes to the code do not break functionality on 2.7.
- If your code uses any special Python dependencies, read their documentation to see if they are compatible with Python 3. If they are not, replace them with versions that are Python 3 compatible or remove them from your code.
Once you have tested your interviews to make sure they work in both Python 2.7 and Python 3.6, test whether data pickled by Python 2.7 will be readable in Python 3.6.
- Log into your Python 2.7 development server and create a number of test sessions in your interviews. Try to get the interview answers as “full” as possible, particularly if your interview answers contain objects from third party libraries. If you have multiple interviews, create a session in each one.
- If you use machine learning, create some machine learning data.
- If you use Redis object storage or SQL object storage, store some objects.
- Take a screenshot of the My Interviews page. You will check later that all of the sessions successfully migrate.
- Now, you need to move the SQL and Redis data from the Python 2.7 development server (“server A”) to a Python 3.6 development server (“server B”). You can use your existing Python 3.6 development server for this purpose, but if you care about the data that exists on it, make sure to back up its SQL and Redis data.
- To move the SQL and Redis data between server A and server B, do
docker stopon both server A and server B.
- Then, locate the locate the PostgreSQL and Redis backup files on
both machines. If you are using S3 or Azure Blob Storage, the
PostgreSQL backup is a file called
postgresfolder, while the Redis backup is in the top-level file called
redis.rdb. If you are using persistent volumes, use
docker volume inspectto find the path on the host system where the backup volume is stored, and look in that directory. The PostgreSQL backup is a file called
postgressubdirectory, while the Redis backup is in the backup directory in a file called
- Copy the
redis.rdbfiles from Server A to Server B, replacing the existing files (which you might want to back up).
- Start server B, your Python 3.6 server, with
- When the server starts, log in and visit the My Interviews page.
- Verify that all of the interviews are still there. Then try resuming each session and see if you get any errors.
- If you get any errors, report them on Slack or GitHub.
Internally, docassemble uses a function called
recursive_fix_pickle()for converting Python 2.7 pickled objects into Python 3.6 objects. There may be a way to adapt this function so that it can make an exception for whatever object types are causing problems in your interview.
- Once all the tests pass, you can upgrade your production server
in-place by installing the latest versions of your interviews, then
docker pull, and