Mocking

The Django REST framework offers many tools for creating web services, but how can you achieve reliable testing without directly interacting with the actual database and external services? The answer is mocking. Mocking is a method to replace part of the code making it easier to control and test. Mocking simplifies logic testing, lets you test with external services, and create mocked versions of external tools.

Instead of genuinely going to the database and processing the object, we can establish a fake database request that merely returns an object as a dictionary. Instead of creating a complex object with many attributes and methods, we can design a simple mock object that only has the necessary attributes and methods for testing.

Mocking speeds up your tests and focuses on the logic you want to test, making it a valuable tool for writing tests. Let's dig into it!

Simplifying logic

To verify that your new code doesn't hamper existing functionality, you need to test your APIs. You also need to confirm that your object processing logic performs as anticipated in various scenarios. That's why mocking is a handy technique to simulate numerous conditions and inputs.

Creating mock objects with similar APIs is a common task in testing, so Python provides some built-in tools for this. When testing web service logic in DRF, unittest.mock is a vital tool at your disposal. This library, integrated into the Python standard library, proves to be a priceless asset for devising effective test cases.

To streamline your code testing, the unittest.mock library, a part of the Python standard library, offers the ability to create dummy objects (mocks) that substitute real dependencies in your code when running tests. This efficiency not only enhances your tests but also allows for more controlled and focused evaluations of your web service logic.

Let's say you have a myapp application with a Product model:

from django.db import models

class Product(models.Model):
    name = models.CharField(max_length=100)
    price = models.DecimalField(max_digits=10, decimal_places=2)
    description = models.TextField()

And you have a view that uses ProductListView to show a list of products:

from django.views.generic import ListView
from myapp.models import Product

class ProductListView(ListView):
    model = Product
    template_name = 'product_list.html'
    context_object_name = 'products'

We want to test how this view handles queries without actually accessing the database. You can create a mock object using unittest.mock.patch decorator, which needs an argument that specifies what exactly you want to mock it isolates the code you want to test from the influence of other system elements and can tweak the behavior of functions, methods, or objects during tests to ensure they yield expected results. Now let's create a test file where we use unittest.mock.patch to mock database access and test our ProductListView:

from django.test import TestCase
from unittest.mock import patch
from myapp.views import ProductListView
from myapp.models import Product

class ProductListViewTestCase(TestCase):
    @patch('myapp.models.Product.objects.all')
    def test_product_list_view(self, mock_query):
        # creating mock product objects
        mock_query.return_value = [
            Product(name='Product 1', price=10.0, description='Description 1'),
            Product(name='Product 2', price=15.0, description='Description 2')
        ]
        response = self.client.get('/products/')  # intended URL for product listing
        self.assertEqual(response.status_code, 200)

        # check that the response contains product data
        self.assertContains(response, 'Product 1')
        self.assertContains(response, 'Product 2')
        self.assertContains(response, 'Description 1')
        self.assertContains(response, 'Description 2')

        # make sure that the database query has been called
        mock_query.assert_called_once()

unittest.mock.patch is a valuable tool for mocking objects in Python tests. It allows you to temporarily replace real objects with mocks to isolate the code under test from external dependencies. This is especially useful in testing when you want to avoid truly interacting with real resources like databases or third-party services and create predictable scenarios for your tests.

Another critical tool in the unittest.mock module is stub classes - Mock and MagicMock. These classes allow you to substitute parts of your system you are testing with stub objects and make statements about how they were used. Mock removes the need to create multiple stubs throughout your test suite:

from unittest.mock import Mock

mock = Mock()
mock.some_method.return_value = 'hello'
print(mock.some_method())

mock.some_method.assert_called_once() # testing that the method was called exactly once

Here we create a Mock object and set the return value for the some_method method. Then we call the method and check if it was called.

MagicMock is a subclass of Mock. Besides the methods and properties of the Mock class, the MagicMock class has all magic method implementations (__str__, __len__, etc.):

from unittest.mock import MagicMock

magic_mock = MagicMock()
magic_mock.__str__.return_value = 'Custom String Representation'
magic_mock.__len__.return_value = 42

str_result = str(magic_mock)
len_result = len(magic_mock)

print(str_result)       # Output: 'Custom String Representation'
print(len_result)       # Output: 42

We create a MagicMock object similar to the previous Mock example. Magic methods of the MagicMock class can be called as if they were regular methods. The printed results showcase the customized behavior of these magic methods in the context of the MagicMock object.

Simplifying tests through mocking opens new horizons in ensuring code reliability. If you didn't use mocking, you would have to either create your own objects and insert them into a temporary database or perform some other tricks to test your code. This would be more complex and time-consuming. However, when your application interacts with external services, it's crucial to broaden the testing limits. Testing against external services not only ensures that internal mechanisms are correct but also that your application works flawlessly with external components to create a reliable and resilient web application.

Testing against external services

Imagine you have a RESTful API, and you want to ensure that when your business logic changes, your API continues to function correctly. You will have to connect to the actual database each time to check that changes do not disrupt existing functionality. Not only is this time-consuming, but it can also make your tests contingent on external factors like database availability or external APIs.

The easiest way to do it is to use the APITestCase class. It provides many useful methods for testing web services built using DRF. It makes it easy to make HTTP requests to your API and validate responses, making the API testing process more efficient and convenient.

Imagine you have a simple API for managing a todo list, and you need to test it. With APITestCase you can comfortably create HTTP requests for your API, including GET, POST, PUT, PATCH, and DELETE requests. For instance, you can use self.client.get(), self.client.post(), self.client.put() and other methods to make requests.

from rest_framework.test import APITestCase
from rest_framework import status
from django.urls import reverse

class TaskAPITest(APITestCase):
    def test_get_tasks(self):
        response = self.client.get('/todo/task-list')
        self.assertEqual(response.status_code, 200)

    def test_create_task(self):
        data = {'title': 'First task', 'completed': False}
        response = self.client.post('/todo/task-list', data, format='json')
        self.assertEqual(response.status_code, 201)

In the test_get_tasks test we use self.client.get() to make a GET request to the 'task-list' URL pattern and then verify that the response status is 200. You can also check the response content to ensure that the data is correct.

In the test_create_task test we use self.client.post() to make a POST request to create a new task. Here we check that the response status is 201, which indicates that the resource was created successfully.

When testing views directly using a request factory, it's often useful to authenticate the request directly rather than crafting the correct authentication credentials. You can use the force_authenticate() method to authenticate users in your tests. This allows you to authenticate a request directly, without needing to create the correct authentication credentials:

from rest_framework.test import APITestCase
from django.contrib.auth.models import User

class AuthTestCase(APITestCase):
    def setUp(self):
        self.user = User.objects.create_user(username='testuser', password='testpass')

    def test_example(self):
        self.client.force_authenticate(user=self.user)
        response = self.client.get('/api/example/')
        self.assertEqual(response.status_code, 200)

Here we create a User instance in the setUp() method, which is invoked before each test method. Then in the test_example() method we use force_authenticate() to force authentication of that user before making a GET request to the '/api/example/' endpoint. We then check that the response status is 200, which indicates the request was successful.

While internal logic is essential, testing against external services adds another layer of complexity. Mocking proves to be instrumental in these scenarios, enabling developers to emulate responses from external services without making actual requests. To illustrate this, let's consider an example where an APITestCase tests an external service by mocking the returned response:

from rest_framework.test import APITestCase
from unittest.mock import patch

class ExternalServiceTest(APITestCase):
    @patch('requests.get')
    def test_external_service_interaction(self, mock_get):
        # define the expected response from the external service
        expected_response = {'key': 'value'}

        # configure the mock to return the expected response
        mock_get.return_value.json.return_value = expected_response

        # perform the API request that interacts with the external service
        response = self.client.get('/api/external-service/')

        # assert that the API response matches the expected response
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json(), expected_response)

Here, the requests.get method is patched using the @patch decorator, allowing the test to simulate an interaction with an external service. The test then configures the mock to return the expected response, and the subsequent API request is validated against this mock response. This approach ensures robust testing of your application's resilience in the context of varied service interactions.

Testing becomes challenging when dealing with external services that we can't control. Nevertheless, it's crucial to ensure that our code functions accurately on our end despite these external dependencies. Moving forward, we'll cover how this technique enables robust testing against different external tools, ensuring your application's resilience in the face of diverse service interactions.

Simulating Redis server

It's often necessary to provide an isolated test loop for code that depends on external services and to create predictable scenarios for working with these tools. One such service is Redis. To test how your API interacts with it, you can use the fakeredis library. This library is a useful tool for simulating interactions with Redis in tests. fakeredis lets you simulate having a Redis instance by using a Python object, thus eliminating the need to interact with a separate database for management purposes.

First, you need to install fakeredis. You can do this using the pip package manager:

pip install fakeredis

The next step is to prepare a test environment where fakeredis will emulate working with Redis. Let's examine how to use this library as part of unit testing based on unittest.

Imagine that you have a class that interacts with Redis:

import redis

class RedisManager:
    def __init__(self, host='localhost', port=6379):
        self.redis_client = redis.StrictRedis(host=host, port=port)

    def set_value(self, key, value):
        self.redis_client.set(key, value)

    def get_value(self, key):
        return self.redis_client.get(key)

Now let's write a test using fakeredis:

import unittest
from unittest.mock import patch
from fakeredis import FakeStrictRedis
from redis_module import RedisManager

class TestRedisManager(unittest.TestCase):
    @patch('redis.StrictRedis', FakeStrictRedis)
    def setUp(self):
        # replacing real Redis with FakeStrictRedis
        self.redis_manager = RedisManager()

    def test_set_value_and_get_value(self):
        key = 'test_key'
        value = 'test_value'

        # testing receiving value
        self.redis_manager.set_value(key, value)
        result = self.redis_manager.get_value(key)

        self.assertEqual(result, value)

We use the FakeStrictRedis class to replace the real Redis when instantiating a RedisManager in tests. This way, testing occurs without actually interacting with Redis, making it isolated and predictable.

Another example of using fakeredis is a scenario where the RedisManager class is used to query user properties and save them in Redis for future use. In the test, we'll verify that calling a specific function results in the creation of a new key that did not exist before the function call:

import unittest
from unittest.mock import patch
from fakeredis import FakeStrictRedis
from redis_module import RedisManager

class TestRedisManager(unittest.TestCase):
    @patch('redis.StrictRedis', FakeStrictRedis)
    def setUp(self):
        # replace real Redis with FakeStrictRedis
        self.redis_manager = RedisManager()

    def test_save_user_properties_in_redis(self):
        # initial user data
        user_id = 'user123'
        user_properties = {'name': 'John Doe', 'age': 25, 'city': 'Paris'}

        # ensure the key doesn't exist initially
        initial_result = self.redis_manager.get_value(user_id)
        self.assertIsNone(initial_result)

        # save user properties in Redis
        self.redis_manager.save_user_properties(user_id, user_properties)

        # retrieve user properties from Redis
        saved_properties = self.redis_manager.get_user_properties(user_id)

        # check if the saved properties match the original user data
        self.assertEqual(saved_properties, user_properties)

        # ensure the key now exists in Redis after saving user properties
        final_result = self.redis_manager.get_value(user_id)
        self.assertIsNotNone(final_result)

        # check that the saved key is different from the initial key
        self.assertNotEqual(final_result, initial_result)

In this example, we added two new methods to the RedisManager class: save_user_properties and get_user_properties. The test checks that calling save_user_properties creates a new key in Redis, and that calling get_user_properties retrieves the correct user properties.

fakeredis greatly simplifies the process of testing Redis-dependent code by providing tools to emulate various scenarios of working with this external tool. With this library, testing becomes a predictable and controlled process that does not require actual interaction with the Redis server while running tests.

While we focused on Redis in this example, it's common for various databases or services to have dedicated mock libraries tailored for simulation. It's a good idea to check for such tools when dealing with a specific problem.

Conclusion

Mocking simplifies logic testing by allowing you to run tests against external services and create stubs for external tools. We looked at how mocking simplifies unit testing by allowing you to create mock HTTP requests and responses, data models, and clients to isolate your code from real dependencies.

unittest.mock provides the ability to create mock objects that replace real dependencies in the code when running tests. Mocking allows you to isolate code from external influences, change the behavior of functions, methods or objects during tests to guarantee the expected results.

The APITestCase subclass provides many convenient methods for testing web services built using the DRF. This makes API testing more efficient and convenient, allowing you to make HTTP requests and test responses.

In order to simulate interactions with external tools, it's advisable to initially explore existing libraries tailored for testing the specific tool you're working with. For instance, the fakeredis library proves beneficial when testing Redis-related functionality. This library enables you to conduct tests on Redis without the need to set up an actual Redis cluster solely for testing purposes.

Ultimately, mocking provides us with the means to create reliable, isolated, and predictable test cases for code that interacts with external services and tools. Thanks to these tools, testing becomes an efficient process that does not require active interaction with real resources, making it more controllable and predictable.

It may take some time to master mocking and patching initially, but once you become familiar with these techniques, incorporating them into your test writing process will become second nature.