Flask Test client

Any real-world software development project includes translating requirements to code based on domain knowledge. Testing is the process of evaluating the accuracy and functional correctness of the code to prevent bugs and improve performance. Well-written tests also serve as good documentation for the existing code to new people in the project.

In this topic, we will look at what testing infra is available in Flask. For this, we will use a very simple Flask application and then see how best to test it.

Simple flask application

In order to understand the testing aspect better, we will create a simple Flask application. In this application, we have an in-memory list called books where we add books as Python dictionaries and read books out of the list by id. This is to simulate a READ PATH (where we read a book by id, and raise an exception if the id to read is not present in the list) and a CREATE PATH (where we write to the books and raise an exception if the book to write is mal-formed). Look at the diagram below.

Process Flow

And now, here is the code that implements this flow.

from flask import Flask, request, jsonify, abort, make_response
app = Flask(__name__)

books = [{ "id":0, "name":"A tale of two cities", "author": "Charles Dickens", # book data
    "publisher": "Astrabound Publishers", "price": 60}]

@app.route("/get/book/<int:id_to_query>", methods=["GET"])  # READ WORKFLOW
def get_book(id_to_query):
    if id_to_query >= len(books):
        msg = "There are no books with id: {}".format(id_to_query)
        abort(make_response(jsonify(message=msg), 404))
    return jsonify(books[id_to_query])


@app.route("/post/book", methods=["POST"]) # CREATE WORKFLOW
def post_book():
    new_id = len(books)
    book = request.get_json()

    fields_to_check = ["name", "author", "publisher", "price"]
    if any(f not in book for f in fields_to_check):
        msg = "The book should have {} defined, not all found".format(','.join(fields_to_check))
        abort(make_response(jsonify(message=msg), 404))

    book["id"] = new_id
    books.append(book)
    return "Book added successfully"

if __name__ == '__main__':
    app.run(port=8081)

Using unittest to setup testing

Unittest Recap: Unittest is a module available in Python for testing. Using this, we create a class and define methods for testing specific logic as instance methods in the class. We also define two special methods setUp and tearDown in the class, that are called before and after every test instance method is executed. An example of testing using Unittest is as follows

import unittest


class DummyUnitTest(unittest.TestCase):

    def setUp(self) -> None: 
        # Setup of test happens here
        self.a = 2 
        self.b = 3

    def test_add(self): 
        # Here we create an instance method that runs the test
        self.assertEqual(self.a + self.b, 5)
        
if __name__ == '__main__':
    unittest.main()

In the above Flask app workflow diagram, we see there are four distinct paths to test

• READ WORKFLOW SUCCESS: You read for an id and find it

• CREATE WORKFLOW SUCCESS: You create a book object, and it succeeds

• READ WORKFLOW FAILURE: You read for an id that does not exist, and error handling kicks in from get_book

• CREATE WORKFLOW FAILURE: You try to create a book, but it fails as all fields are not present from post_book

Additionally, we will create a test for CREATE WORKFLOW followed by READ WORKFLOW, which will be like an integration test.

But first, we need to set up the Unit test, with the following code

import unittest
from app import app, post_book, get_book


class AppTester(unittest.TestCase):

    def setUp(self) -> None:
        # TEST DATA FOR READ SUCCESS
        self.expected = {
            "id": 0,
            "name": "A tale of two cities",
            "author": "Charles Dickens",
            "publisher": "Astrabound Publishers",
            "price": 60
        }
        # TEST DATA FOR CREATE SUCCESS
        self.new_book = {
            "name": "Rich dad poor dad",
            "author": "Robert Kiyosaki",
            "publisher": "Hawkins publishers",
            "price": 87
        }
        # TEST DATA FOR CREATE FAILURE
        self.faulty_book = {
            "publisher": "Daemon publishers",
            "author": "Roman Edward"
        }
        # FLASK APP
        self.app = app

self.expected this variable is used to verify the output of a READ SUCCESS workflow, note in the code this is the variable inside books list. self.new_book this variable is used to create a new book and thus verify the CREATE SUCCESS workflow. Lastly self.faulty_book variable shows a request body for a wrong book creation, as it does not have fields like name and price, all of which are required to create the book, this is used to test the CREATE FAILURE workflow. A word of caution, here we would be comparing output dictionaries (like self.new_book ) as objects, but the best practice to compare collections of data like dictionaries is to do a key-by-key comparison.

Flask infrastructure for testing

Flask provides two ways to test a backend logic.

1. Test client: Flask Test client extends the Werkzeug client, and it is used to send requests to the application without running a live server. This directly hits on the URL defined in the @app.route('/url,' ) for a view function i.e. the function that is decorated with @app.route

2. Test Request Context: A request context keeps track of request level data during a request like body, cookies, header, etc. It is accessed by view functions, error handlers, and other middleware rather than being passed to each. A request context that is created for testing is used to directly test the view functions.

As a note, Test Client uses the URL exposed via the view function, but the Test Request Context uses the view function directly. And it is more suitable for testing the view function more thoroughly, thus ideal for unit testing. Both the test client and test request context can be created via the Flask app object that we have saved in self.app of the AppTester class.

Testing READ SUCCESS workflow

In the READ workflow, we are going to check for the book with id 0, /get/book/0. First with test client.

 def test_read_success_with_client(self):
        response = self.app.test_client().get("/get/book/0")
        assert response.get_json() == self.expected

Here self refers to the AppTester class instance, and this method is defined in the same class. In this example, we assert the response has the same dictionary information as that of self.expected. An example to do the same test using test request context is

def test_read_success_with_request_context(self):
    with self.app.test_request_context("/get/book/0", method="GET"):
       res = get_book(0) # get_book is view function for endpoint /get/book/id
       assert res.get_json() == self.expected

Hopefully, you can see the similarity in testing, we are calling the same URL and asserting on same variables on the JSON output.

Testing CREATE SUCCESS workflow

In the CREATE workflow, we are going to create a book using the self.new_book variable and assert we get a success message.

Example using Test Client

    def test_create_success_with_client(self):
        response = self.app.test_client().post("/post/book", json=self.new_book)
        assert response.get_data(as_text=True) == "Book added successfully"

Example using Test Request Client

    def test_create_success_with_request_context(self):
        with self.app.test_request_context("/post/book", method="POST", json=self.new_book):
            res = post_book() # post_book is view function for endpoint /post/book
            assert res == "Book added successfully"

Here we see the success message Book added successfully is returned in both cases

Testing READ FAILURE workflow

In the READ failure workflow, we are going to read an id that does not exist in the backend. This will be a large number, like 100, to make sure the request fails. An example with the test client

    def test_read_failure_with_client(self):
        response = self.app.test_client().get("/get/book/100")
        assert response.get_json()["message"] == 'There are no books with id: 100'

Example with test request context

    def test_read_failure_with_request_context(self):
        from werkzeug.exceptions import HTTPException
        with self.app.test_request_context("/get/book/100", method="GET"):
            with self.assertRaises(HTTPException) as e:
                res = get_book(100)
            assert e.exception.response.get_json()["message"] == 'There are no books with id: 100'

For the request context example, we get an HTTPException raised at the backend as per the backend logic, and we assert it expecting an exception.

Testing CREATE FAILURE

For CREATE failure workflow, we are going to use an incorrect book dictionary self.faulty_book which does not have all fields to create the book and pass it to the backend. The backend is expected to raise an exception stating which fields are mandatorily required.

An example with the test client

    def test_create_failure_with_client(self):
        response = app.test_client().post("/post/book", json=self.faulty_book)
        msg = "The book should have name,author,publisher,price defined, not all found"
        assert response.get_json()["message"] == msg

An example with the test request context

    def test_create_failure_with_request_context(self):
        from werkzeug.exceptions import HTTPException
        with self.app.test_request_context("/post/book", method="POST", json=self.faulty_book):
            with self.assertRaises(HTTPException) as e:
                res = post_book()
            msg = 'The book should have name,author,publisher,price defined, not all found'
            assert e.exception.response.get_json()["message"] == msg

Similar to READ failure, we raise an HTTPException in case of a CREATE failure, and the test asserts this behavior.

Testing CREATE and READ workflow combined

In this scenario, we create a new book using self.new_book variable and read it back using GET /get/book/1. We assert the book is the same as self.new_book.

    def test_create_and_read_with_client(self):
        resp_of_create = app.test_client().post("/post/book", json=self.new_book)
        assert resp_of_create.status_code == 200 # check the status code for success
        resp_of_read = app.test_client().get("/get/book/1")
        assert resp_of_read.status_code == 200 # check the status code for success
        assert resp_of_read.get_json() == self.new_book

For this example, we do not have the equivalent test request context example because here, we have to call two URLS (/post/book and /get/book). And for creating a test request context, we need to specify one URL, and in the context of one URL, we should not test two URL calls. This establishes the fact that using the test request context, we should not test integration scenarios (multiple endpoints).

Running a session-based test

To make the use case more real-world, we can see how we can test a session-based flask view function. Sessions are used in web applications to store user-related data in temporary storage. Using sessions, we can modify our get_book code to check if the user is logged in or not before processing the request. The code in get_book will look like this. (We check flask.session to ensure the user is logged in or not).

@app.route("/get/book/<int:id_to_query>", methods=["GET"])
def get_book(id_to_query):
    from flask import session
    if 'logged' not in session:
        abort(make_response(jsonify(message="You are not authorized to use this endpoint")), 404)
    if id_to_query >= len(books):
        msg = "There are no books with id: {}".format(id_to_query)
        abort(make_response(jsonify(message=msg), 404))

    return jsonify(books[id_to_query])

The test for this function looks as follows, we create a test_session to simulate a logged-in user.

def test_read_with_session(self):
   with self.app.test_client() as c:
      with c.session_transaction() as test_session:
         test_session["logged"] = True
      response = c.get("/get_auth/book/0") # This should be defined outside the session creating with block
      assert response.get_json() == self.expected    

Pay attention to the placing of different with blocks. The call to the server with a session defined using the test client should happen outside the session-defining with block.

Running all tests

For running multiple unit tests from the editor, we need the following code at the end.

if __name__ == '__main__':
    unittest.main()

Once you run all the tests, you should see something as follows.

Conclusion

• In this topic, we created a very simple Flask application and identified different workflows that can be tested in the happy path and the error path.

• We created tests using Flask infrastructure and used unitTest module to write our tests for all scenarios

• We looked at the difference in the execution of tests using a Test client vs a Test Request context.

Testing Flask applications gives an insight into how the application is structured internally, good tests help you understand the framework better and write better application code.