So, you've been burning the midnight oil, crafting an exquisite Flask app that you're now itching to show off to your friends, your dog, and maybe even the entire world? But the thought of deploying it gives you jitters, doesn't it? You've been bombarded with various methods and opinions on how to deploy your application that it feels overwhelming to even start to learn each part of these numerous tools and workflows and pipelines that most of these guides assures you are the "best" way to do it.
But fear not, this topic tries to take another approach. The main goal here is to give you a basic deployment guide/reference that will allow you to deploy your application on the web without the need for any deep knowledge on any single of the various tools. This is a hands-on, step-by-step guide where we will start with a simple flask application on your local device and end up with a scalable ,easily manageable deployed web app. But also,once you pass this daunting task of deploying you first application, its rather easy to keep learning,improving and adding many layers to your deployment process or even to change it completely altogether.
Lets get started!
Introduction
Let me introduce you to the Flask application we'll be deploying, Shelfie:
Shelfie is a Flask application designed to simplify book cataloging. With Shelfie, you can easily add, update, delete, and list books in your collection. It uses a database to store important details about books such as title, author, publication year, and genre. Below is the file structure for our application:
One of the great things about Flask is its versatility, which means that your application may not necessarily follow the same structure as above, and that's okay. However, the structure shown above is a common approach used by many Flask applications. By packaging our app with the __init__.py file and dividing the code into separate files, scaling the application and adding new features or debugging becomes easier.
Most files inside the app package have self-explanatory names, but let's take a closer look at our initialization script:
#Shelfie/app/__init__.py
#...
from config import Config
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
app = Flask(__name__)
app.config.from_object(Config)
#...
db = SQLAlchemy(app)
migrate = Migrate(app,db)
from app import routes,models,utils
#...This single file addresses many of the common mistakes made when deploying Flask applications. Let's unpack it one by one:
Configuration
Initially, when learning Flask, you might overlook the importance of configuration. However, during deployment and as your application grows more complex, having a clear and secure configuration can save you significant time and effort. In this case, we're using a straightforward general configuration script called config.py that is located outside our app package.
#Shelfie/config.py
import os
from dotenv import load_dotenv
basedir = os.path.abspath(os.path.dirname(__file__)) #1
load_dotenv(os.path.join(basedir,".env")) #2
class Config:
SECRET_KEY = os.environ.get("SECRET_KEY") or "super-secret-key" #3
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') or \ #4
'sqlite:///' + os.path.join(basedir, 'app.db')
SQLALCHEMY_TRACK_MODIFICATIONS = False
#....#1: We are storing the location of our main directory in a variable called basedir, which will be used to locate other files. It's important to note that we are not hardcoding this value because it may change over time as we move our code for deployment.
#2: We load environment variables using the python-dotenv package. These variables store key-value pairs like secret keys and database URLs separately, outside the app package. This means you can easily edit and modify them while reducing the risk of accidentally uploading or sharing sensitive data they might contain. Currently, our .env file consists of one line, but we will add more as necessary.
SECRET_KEY=xZspkFCxM3x0RHW48iU6HPcylVBU2W2y59wcw8sVgYou can generate random keys with this command: python -c 'import secrets; print(secrets.token_urlsafe(16))'
#3, #4: These are typical use cases for environmental variables. The secret key is used by Flask for security, while SQLALCHEMY_DATABASE_URI determines which type of SQL Database to use (set to sqlite by default), which we will configure later.
Flask-Migrate
The other thing you may have noticed from the __init__.py script might be the use of a Flask extension called Flask-Migrate, which is responsible for the migration folder. This is a wrapper for Alembic that greatly simplifies database management when deploying Flask applications.
As your application grows and evolves, you'll likely need to modify your database structure by adding, editing, or removing elements. However, this can be challenging because relational databases rely on structured data. When the structure changes, the existing data must be migrated to the new structure. This typically requires deleting and recreating the entire database, which can be time-consuming and risky. Alembic/Flask-Migrate provides a solution to this problem by automating the process of migrating data between different database structures. If you use version control systems such as git, the process might seem similar.
Take our Shelfie app as an example. After the first deployment, we quickly realize that in addition to the existing books, authors, and genres tables, we need to add a new database table consisting of book publishers, the books they have published so far (relationship), their location, etc. To achieve this, we would have to delete the old db (sqlite:///app.db) and create a new one incorporating the changes we've made. But with Flask-Migrate,
flask db initcreates the migration repository you have seen initially at the file structure of Shelfie app. this should be done initially, before changing the the database in any format.flask db migraterecords any changes to the database structure made after initializing the repo.flask db upgradeexecutes those changes and updates the database. That is it!
Next, we move on to a fun part. SQLite databases might be satisfactory for testing our application during its initial stage, but when deploying, we need to use a production-level robust database management system. For example, let's consider PostgreSQL:
After installing and creating the database on your local machine, you can connect it using the DATABASE_URL variable we defined in config.py at the start. Let's extend our .env
SECRET_KEY=xZspkFCxM3x0RHW48iU6HPcylVBU2W2y59wcw8sVg
DATABASE_URL= postgresql://username:password123@localhost:5432/shelfiewhere the username and password are created when configuring Postgres and Shelfie is the name of the database.
Flask-Migrate makes populating this database with structures that were already set up with the previous SQLite db super easy. The migration scripts have already been recorded before, so simply run flask db upgrade.
The migration folder will be treated as part of our application from now on. When deploying, we just copy it to the server and perform flask db upgrade to recreate the database in the format we expect.
boot.sh
Shell scripts are a vital part of any deployment process as they help avoid repetitive executions of commands by grouping them into a single executable script. Let's take a look at boot.sh
#!/bin/bash
exec gunicorn run:app -b 0.0.0.0:5000 The exec command in this shell script starts the Gunicorn server. Gunicorn is a WSGI ('wiz-gee') HTTP server that we use to run our Flask application in a production environment. The syntax run:app means running the Flask object named app from the run.py script which imports it from the app package.
This script might seem minimal and even unnecessary at first, but it forms the basis for more complex operations. As we progress, we will add more commands to it.
Before executing a shell script, you need to make it executable. You can do this using the chmod command. For our boot.sh file, we use sudo chmod +x boot.sh. Once the script is executable, you can run it using bash boot.sh or ./boot.sh.
Dockerizing Shelfie
It's ideal if you can run your code on any server. Dockerizing your app helps achieve that. Dockerization is a process that allows your application to run on any server, regardless of its Python version or other system specs. By containerizing your app with Docker, you ensure its consistent operation across different environments.
First, let's create a Dockerfile inside our Shelfie folder to create an image of our Flask app:
#Shelfie/Dockerfile
#slim,why? : lightweight image that contains only the minimum set of packages needed
FROM python:slim
#create user called shelfie
RUN useradd shelfie
WORKDIR /home/shelfie
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY app app
COPY migrations migrations
#no .env
COPY run.py config.py boot.sh ./
RUN chmod +x boot.sh
ENV FLASK_APP run.py
RUN chown -R shelfie:shelfie ./
USER shelfie
EXPOSE 5000
ENTRYPOINT ["./boot.sh"]We have already created a migration folder in our local machine earlier, and we just copied it to the Docker image. So we only have to run flask db upgrade to create the database and bring it up to date. Let's revise our boot.sh file:
#!/bin/bash
flask db upgrade
exec gunicorn run:app -b 0.0.0.0:5000Now we can build our Docker file with:
docker build -t shelifie:latest .And create a container with:
docker run --name shelifie -p 5000:5000 --rm shelfie:latestThe .env file is not included in the Docker image because it contains runtime variables that are not necessary during the build process. These variables are typically set when running the container using docker run. By excluding the .env file from the Docker image, we make the Dockerfile more portable and also it allows us to customize runtime variables as needed.
The following command adds our .env file at the top as env_file when running a container named shelifie.
docker run --name shelifie -p 5000:5000 --env-file=.env --rm shelfie:latest
But you might have noticed that,even without even running it, there is still an issue . Because, the variable DATABASE_URL in .env is set to a Postgres DB server , which doesn't exist in our DB in the container, yet!
There are two approaches we can take here, depending on the deployment server being used. The first approach is ,if it exists,to use a dedicated database service provided by the server. For instance, Render - the service we are going to use to deploy our app - provides its own managed PostgreSQL service, which can be easily set up and connect to the flask application docker image.
The second approach is to include the database within your Docker image. This involves creating a Dockerfile that installs and configures the database, linking it to our flask application's image and then deploying it. While this approach offers greater control over the database configuration and enhances portability across different servers, it is generally not recommended for production environments due to the need for constant monitoring and reliability issues.
Using external database servers or our first option -choosing a deployment service that provides one for you- is often the preferred choice. This approach can simplify your deployment process and reduce the complexities of maintaining a database within your Docker image.
Deploying your flask application
Finally we are ready to deploy Shelfie . We are going to deploy our app in https://render.com/ which offers a pretty generous free tier service. Go to the sight and create an account before proceeding with this topic. After signing in you will be met with a list of services render offers, click in each of them, play around and try to read the Docs tab as well. But when you want to try a service, you simply press on the blue "+" key and select the service you want.
PostgreSQL
Lets first start our deployement by creating a new PostgreSQL service. Render provides a managed Postgres service which is super easy to set up.
Click on the "+" button and select PostgreSQL . A form will be presented to you for some configurations before creating the database and its environment.
The first form asks for the name of the instance. You can name it as you prefer, but it's recommended to give meaningful names. For this example, we will name it "postgres".
Next, you need to provide the Database name. You can name it after your app, for example, "shelfie". Similarly, for the user, you can use "shelfie".
Leave everything else as it is, scroll down and hit "Create Database".
That is it! After a short wait, your PostgreSQL database will be ready!
To get the connection details, open the dashboard tab and select your database. Scroll down to find the Internal URL. This URL will be used as the
DATABASE_URLvariable in the Flask app.
Flask app
Now that we have a database, lets deploy our dockerized application and connect it with the database.
Before going ahead with this, there are few things we need to change from our dockerfile.
By default, Render exposes port number 10000. We could adjust this, but for our purposes, we'll stick with the default one. So, let's go ahead and expose port 10000 in our Dockerfile. At the same time, we'll need to modify the port number for our gunicorn WSGI server to match this.
If you followed the flask-migration example above then only change the port on the
execline to 10000 from the boot script and skip this step, if not then you are creating a new database and not migrating from an old one, therefore remove theCOPY migrations migrationspart from the dockerfile and you can change the booting script boot.sh to:#!/bin/bash flask db init flask db migrate flask db upgrade exec gunicorn run:app -b 0.0.0.0:10000When using the PostgreSQL database with SQLAlchemy, you'll need to include the
psycopg2-binarypackage, which enables SQLAlchemy to interact with PostgreSQL. Please ensure it's listed in yourrequirements.txtfile for your project dependencies.
Making the above changes, our new Dockerfile looks as follows (delete the COPY migrations migrations part according to step 2)
#Shelfie/Dockerfile
FROM python:slim
RUN useradd shelfie
WORKDIR /home/shelfie
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY app app
COPY migrations migrations
COPY run.py config.py boot.sh ./
RUN chmod +x boot.sh
ENV FLASK_APP run.py
RUN chown -R shelfie:shelfie ./
USER shelfie
EXPOSE 10000
ENTRYPOINT ["./boot.sh"]We are now ready to deploy this to render. There are two options here, github or docker-hub. While uploading our source code together with the Dockerfile and connecting that with our render account gives us a better control over the the installation, for now we will use docker-hub for its ease.
Lets first rebuild an image from this file with :
docker build -t shelfie:latest .
We need to push this image to docker-hub , to do so, first sign up to docker-hub if you haven't already . Next run the following
docker loginand input your credential.Tag the image:
docker tag shelfie:latest <your-docker-hub-username>/shelfie:latestPush it to docker-hub:
docker push <your-docker-account-username>/shelfie:latest
Lets open render now, and yup you guessed it , click on the "+" sign and then Web Service. Choose the Deploy existing image option:
Next you will be asked for the image url and just pass the <your-docker-hub-username>/shelfie:latest part from when you pushed the image to docker hub , if you can't press the 'Next' button then make sure that your docker image is indeed pushed to docker hub by visiting from your account.
On the next page, give your service a name, Shelfie for our example and scroll down to the Advanced part and click on it. Here we set our environment variables our docker container will need:
Note the DATABASE_URL is taken from the Postgres Internal Database URL we set up earlier, but we need to change the initials postgres to postgresql.
Thats all, go ahead and push Create Web Service. Voila, your app is living and breathing.
On the next page you will see the deployment progress as your image is deployed on the web, as well a newly generated link you can use to access your application
Or maybe their was some sort of error in your app when deploying, which almost always happens, in that case rebuild and push your corrected docker image to docker hub and press on the Manual Deploy bottom shown above and then Deploy latest reference.
Conclusion
Deploying an application is a daunting task, especially if its your first time. While there are many tools that will make your production level application as well as the deployment process more robust, its better to start small and learn things as you go. We have seen how to do that in this lesson.
While you might be tempted to focus on packages, tools and external databases at first, we have seen that most services comes up configured with some featured services that makes deploying much easier. Hence its better to work backwards. Try services and choose the one you like, from that refactor your tools and configuration to be suitable for that service.