Story on how to use Sqlite as a database when deploying Django with Zappa
Prerequisites
I'm working on an Ubuntu 20.04, Django 3.1.1 and zappa 0.5.2 and django-extensions 3.0.9 for good measure.
I'm also assuming that you are setup with an AWS account and have a aws configuration file on you local machine.
This will not be a complete from zero to hero tutorial, rather I'll assume that you know enough basic Django to be able to apply the shown yourself.
Setup
Create a virtualenv and install the necessary packages:
virtualenv venv -p python3
source venv/bin/activate
pip install django zappa django_extensions
django-admin startproject config .
zappa init
Answer the questions from zappa to create the settings file (zappa_settings.json):
{
"dev": {
"django_settings": "config.settings",
"profile_name": "default",
"project_name": "django-zappa-sq",
"runtime": "python3.8",
"s3_bucket": "zappa-sqlite-story"
}
}
Lets try and run it locally:
python manage.py migrate
python manage.py runserver
And it should work just fine.
First deploy to AWS
Lets just try and deploy the system as is to AWS:
zappa deploy dev
You might get this error:
...
botocore.exceptions.NoRegionError: You must specify a region.
so we will have to specify a region in our settings file. Update to:
{
"dev": {
"django_settings": "config.settings",
"profile_name": "default",
"project_name": "django-zappa-sq",
"runtime": "python3.8",
"s3_bucket": "zappa-sqlite-story",
"aws_region": "eu-central-1"
}
}
Lets try and deploy again:
(venv) $:~/github/django_zappa_sqlite_blogstory$ zappa deploy dev
Calling deploy for stage dev..
Creating django-zappa-sq-dev-ZappaLambdaExecutionRole IAM Role..
Creating zappa-permissions policy on django-zappa-sq-dev-ZappaLambdaExecutionRole IAM Role.
Downloading and installing dependencies..
- pyyaml==5.4.1: Downloading
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 662k/662k [00:00<00:00, 6.79MB/s]
Packaging project as zip.
Uploading django-zappa-sq-dev-1616928259.zip (16.4MiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 17.1M/17.1M [00:01<00:00, 9.69MB/s]
Scheduling..
Scheduled django-zappa-sq-dev-zappa-keep-warm-handler.keep_warm_callback with expression rate(4 minutes)!
Uploading django-zappa-sq-dev-template-1616928273.json (1.6KiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.66k/1.66k [00:00<00:00, 25.3kB/s]
Waiting for stack django-zappa-sq-dev to create (this can take a bit)..
75%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 3/4 [00:09<00:03, 3.12s/res]
Deploying API Gateway..
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
(venv) $:~/github/django_zappa_sqlite_blogstory$
So, we got a little further, but still no luck. The problem is that sqlite doesn't work with aws lambda.
Sqlite with lambda
We need to install another package, django-s3-sqlite:
pip install django-s3-sqlite
and add it to our installed apps in config/settings.py and update the DATABASES settings as well:
ALLOWED_HOSTS = ["*"]
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
INSTALLED_APPS += ["django_s3_sqlite"]
...
...
...
DATABASES = {
"default": {
"ENGINE": "django_s3_sqlite",
"NAME": "sqlite_zappa.db",
"BUCKET": "zappa-sqlite-story",
}
}
Additionally, we need to download a binary to our root django folder (the project folder where our manage.py file lives). The file is called _sqlite.so and is found in the https://github.com/FlipperPA/django-s3-sqlite/tree/master/shared-objects directory. So, copy the file to your project folder.
Additionally, add this line to your zappa settings file: "use_precompiled_packages": false,, so the file becomes:
{
"dev": {
"django_settings": "config.settings",
"profile_name": "default",
"project_name": "django-zappa-sq",
"runtime": "python3.8",
"s3_bucket": "zappa-sqlite-story",
"aws_region": "eu-central-1",
"use_precompiled_packages": false
}
}
Update again
Lets try and update our dev environment on AWS:
zappa update dev
Still getting an error though:
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
The problem here is, that we haven't added our database to the S3 bucket. Lets rename our database that lives in our local dir and upload it to the S3 bucket.
mv db.sqlite3 sqlite_zappa.db
Update again to lambda:
(venv) $:~/github/django_zappa_sqlite_blogstory$ zappa update dev
Calling update for stage dev..
Packaging project as zip.
Uploading django-zappa-sq-dev-1616929230.zip (18.8MiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 19.7M/19.7M [00:01<00:00, 9.95MB/s]
Updating Lambda function code..
Updating Lambda function configuration..
Uploading django-zappa-sq-dev-template-1616929237.json (1.6KiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.66k/1.66k [00:00<00:00, 26.3kB/s]
Deploying API Gateway..
Scheduling..
Unscheduled django-zappa-sq-dev-zappa-keep-warm-handler.keep_warm_callback.
Scheduled django-zappa-sq-dev-zappa-keep-warm-handler.keep_warm_callback with expression rate(4 minutes)!
Your updated Zappa deployment is live!: https://nexusuynl2.execute-api.eu-central-1.amazonaws.com/dev
(venv) $:~/github/django_zappa_sqlite_blogstory$
And now, it just works.