Structuring the application

As a project's code base grows, navigating and maintaining lines of code becomes increasingly difficult. Even in small personal projects, fixing bugs or adding new features without breaking existing code can be a difficult task. To combat these challenges, it's essential to structure code and files effectively.

There's no single solution that works in every scenario. A universal, bullet-proof method of arranging files can cause more issues than it solves. However, an effective code structure should be scalable, easy to navigate, and divided into manageable parts. Researching similar projects can provide valuable insights. Keep in mind that the structure may need to change as complexity increases.

Flask project layout

Flask is a micro-framework, and it is often listed as a downside. However, it is actually its biggest strength because it puts you in control. Unlike other complete frameworks, Flask allows you to decide how to use and extend it, and it lets you organize your code however you see it. If you have experienced the headache of modifying the imposed structure of other complete frameworks in the past, you will appreciate how awesome Flask is. With this freedom, you can organize your code in various ways while still following the above principles, making it easier to maintain a large codebase. This topic will explore one commonly used approach: package-based structuring.

To make everything clear and also to avoid making a big jump in the layout from what you have been doing so far, we will follow up the development of a dummy Flask application my_app from a single file system to a full-fledged flask package that is easy to scale and manage.

• Version-0.1(v0.1):

This is in the format we have been using so far, a single app.py file that handles all the application logic, a templates folder for all the HTML, and a static folder for the javascript and CSS files:

There are several reasons why it's important to modify v0.1. Firstly, having everything in a single module makes it difficult to scale and maintain the code, violating the "separation of concerns" principle. Secondly, the configuration variables for Flask and its extensions should be kept separate and managed using various methods (for more on this, check out the reference links at the end of the topic), which is not possible with the current layout. Finally, this approach is not a good practice in general.

• v0.2:

Here we are starting to branch out and make some simple changes.

We first created utils.py and put away all the functions we created to manipulate data and for other helpful purposes.
This might include:

• functions we wrote that fetch from lists and dictionaries;

• custom Jinja filters we have created;

• any other helper function that isn't part of the Flask module.

model.py is also created, and we put all our database models here from the app.py:
We easily connect these files by using from ... import ... at the top of our main app.py.
But, even at this early stage, this might lead to one of the most common errors we encounter when we try to divide a single module into multiple ones.

Circular imports

As the name suggests, this happens when two or more modules try to import one another before even one of them is fully loaded. Let's take a look at what happens inside two of our files when we run app.py directly. This is not a precise yet useful way to think about importing modules.
Note that:
1. Every time we import something from a module, Python tries to run the whole module first!
2. When we run a Python file with python file_name, Python will denote this file as __main__.

Don't worry if you haven't fully grasped how the error came about; the solution below is much easier to follow. But this kind of circular import error is very hard to track down and correct as the number of modules increases. Also, the errors aren't usually raised at the place you expect; therefore, it's always good to follow the solution outlined below from the start.
We can change our main app.py by rearranging the order of the import and the variable db declaration as follows:

#app.py
#...
db = SQLAlchemy(app)
from models import User

#..

and rename app to __main__ in our models.py:

#models.py
#...
from __main__ import db

#...
class User(db):
  #...

Even though this solves our problem, it's actually an ugly solution. We will also have yet another issue if we don't run our application in a way that makes sure the app module isn't named as __main__.

• v0.3:

Now we will make the last big change: convert our files into a package that will help us improve upon our last version.

Python packages

If you want Python to treat a folder as a package, you simply create the __init__.py file inside the folder. This file is invoked whenever the package or any module in the package is imported, making __init__.py an ideal place to define objects that will be imported frequently into other files. Here is an overview of our __init__.py:

#__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)

db = SQLAlchemy(app)

from my_app import app

if __name__=="__main__":
  app.run()

We added the last line so that when we run our package, it will find and run app.py as well. As you can also see, we are still following the solution to the circular import proposed in the previous version by importing the app.py module below our app variable definition.
When we want to import the variables initialized in __init__.py onto the other files of the package, we will use the format of thepackage_name.variable, and the modules are imported as package_name.module. let's see what our app.py will look like:

#__init__.py
#...
from my_app import app
from my_app.models import User

#...

Here, as we need the app variable, we imported it from our packages, and we imported the User class from the model's module using the dot notation.
And now we go to the final version of our dummy app.

• v0.4:

We have completed the main improvement in the last version, now we will just try to apply the principle of separation of concerns even further.

The app.py file has been separated into two files: routes.py and forms.py. These files contain all the routes and web forms that were previously in one file. Additionally, we have added a new file called run.py, which is located at the same level as the package. This file will be used to run our package and avoid confusion related to the __main__ file.

#run.py
from my_app import app
if __name__ == "__main__":
  app.run()

The other additional file is the config.py file, which contains the Flask configuration of our app using a class-based layout. Take a look at the additional references section for more information about this.
We have also divided our utils.py into a separate folder, a sub-package, containing error_handlers.py and custom.py. The __init__.py file in this folder is empty. It's there to indicate to Python that this is a sub-package.
Our main my_app/__init__.py file now looks as follows:

#my_app/__init__.py

from flask import Flask
from flask_sqlalchemy import SQLAlchemy


app = Flask(__name__)
app.config.from_object("config.DevelopmentConfig") #or any config of our choice

db = SQLAlchemy(app)

from my_app import routes

To avoid circular import errors, you can import additional modules after initializing the necessary variables and objects. Although this is our final version of the dummy application, you can still divide the files further. It's a good practice to split long lines of code or multiple concepts and ideas into smaller chunks, or even separate folders or sub-packages within the main package.

Conclusion

• Flask gives us the freedom to choose, among other things, the appropriate layout for our project based on our needs. While this freedom might seem daunting at first, it actually leads to a better understanding of our code as well as in the end an efficient, custom-made structure.

• It's always a good practice to start building your Flask application with a good structure from the beginning. Don't wait until your code's complexity has increased to start applying these layouts.

• While there are many ways to structure flask applications, we have mentioned and explained here the most common one, package-based layout. It's always best to use this structure while building any Flask-related applications. Afterward, you can customize and tweak anything you want.