Caching concepts

When working with databases in Django, many tools are available to perform various types of queries. However, as databases and the number of users grows, it can become expensive to respond to every user request similarly. For example, a news site that retrieves the latest news from its database every time the page is refreshed may take a lot of time and put a significant load on the server. To avoid this, data caching can be utilized. Data caching can substantially speed up page loading and reduce the server load. This article will cover the basic concepts of caching and show how they can be utilized to improve your application's performance.

Least Recently Used

Least Recently Used (LRU) is a cache eviction algorithm that removes the least recently used items when the cache is full and space needs to be made for new data. LRU works on the intuitive principle that data that has been used recently will likely be used again soon. Therefore, when the cache is full and space for new data is required, LRU removes those items that have not been used the longest. This allows you to store the most frequently used data in the cache and increase caching efficiency. This algorithm optimizes data storage so that operations on the data are performed more efficiently than at the source.

The advantage of such an algorithm is its ease of implementation, making integrating it into various systems and applications accessible. Caching using LRU can also significantly speed up performance and reduce the load on computing resources. This is especially important when working with large amounts of data or performing complex calculations.

Despite its simplicity and efficiency, the LRU caching algorithm has several disadvantages. One is that it may be ineffective when dealing with data that is frequently requested but rarely used. In such cases, other caching algorithms may be more appropriate.

Additionally, LRU may not be efficient when dealing with large amounts of data. This is because the algorithm requires storing information about the time of the last request for each value. This can lead to increased memory consumption and slowdown of the algorithm.

It's also worth noting that LRU does not consider the frequency of data usage. This means that values used infrequently but regularly may be pushed out of the cache by values used frequently but irregularly.

That's why there are other techniques besides LRU. For example, instead of limiting the number of items in the cache, you can limit the time the value is cached before deletion. This time is usually referred to as Time To Live.

Time To Live

Time To Live (TTL) is a parameter used in caching to specify the lifetime of data in the cache. This means that after the specified time has elapsed, the data will be automatically deleted from the cache and downloaded again the next time it is requested. TTL allows you to control the data relevance in the cache and ensure its timely updating. TTL can be configured individually: for each value in the cache or the entire cache. TTL can be very useful when the data is updated regularly and frequently (for example, in the case of a trending news page on a news site).

This allows you to flexibly configure caching settings depending on your data freshness requirements. However, setting the correct TTL value can be a complex task, as many factors must be considered, such as data update frequency, system load, etc.

In addition, using TTL requires constant updating of the cache data. This may increase system load and slow down the caching algorithm.

TTL also does not guarantee that the data in the cache will always be current and consistent with the data source. Other mechanisms, such as denormalization, must be used to do this.

Denormalization

Denormalization is a process in which data from multiple database tables is combined into a single table to speed up data retrieval. This is achieved by reducing the number of table joins required when executing queries since the query is sent to only one table rather than multiple ones. Let's assume we have an online store with many products and orders. Information about products and orders is stored in different database tables: Product and Order, respectively.

from django.db import models

class Product(models.Model):
    name = models.CharField(max_length=30)
    price = models.DecimalField(max_digits=10, decimal_places=2)

class Order(models.Model):
    customer_name = models.CharField(max_length=30)
    total_price = models.DecimalField(max_digits=10, decimal_places=2)

Running database queries to obtain information about orders and related products requires joining multiple tables, which can take time. A solution might be to create an OrderItem table:

class OrderItem(models.Model):
    order = models.ForeignKey(Order, on_delete=models.CASCADE)
    product = models.ForeignKey(Product, on_delete=models.CASCADE)
    product_name = models.CharField(max_length=30)
    product_price = models.DecimalField(max_digits=10, decimal_places=2)
    quantity = models.PositiveIntegerField()

    def save(self, *args, **kwargs):
        self.product_name = self.product.name
        self.product_price = self.product.price
        super().save(*args, **kwargs)

We have overridden the save method for the OrderItem model to automatically populate the product_name and product_price fields with the name and price of the product from the associated Product model when saving an order item. This way, when selecting order items, we will not need to make an additional query to the database to get the name and price of the product – they will already be stored in the OrderItem table.

However, when using denormalization, the volume of stored data may be unreasonably large due to its redundancy. Additionally, denormalization can make it difficult to update information. Because the data is stored in a single table, updating information may require updating many rows. In our example, if we change the name or price of a product in the Product model, we will need to update all associated order items to update the product_name and product_price fields. This may result in longer information refresh times.

Additionally, denormalization is unsuitable for functions requiring frequent calculations or recursive calls. For such cases, there is another way to optimize work with data – memoization.

Memoization

Memoization means caching the output of a function so that subsequent calls can use the cached result without extra computation. This can be useful, for example, when working with expensive calculations or database queries. Instead of performing a calculation or query each time, you can return the stored result from the cache if it has already been calculated.

For example, memoization can help calculate the elements of the familiar Fibonacci sequence. Without memoization, the recursive algorithm for computing Fibonacci numbers has exponential complexity, making it very slow for large values of n. Memoization allows you to save the results of a function calculation for different values of n to avoid repeated calculations. This reduces the algorithm's complexity to linear, making it much faster. For example, this is what a function that calculates the n element of a series using the memo dictionary might look like:

def fibonacci(n, memo = {}):
    if n in memo:
        return memo[n]
    if n <= 0:
        return 0
    if n == 1:
        return 1
    memo[n] = fibonacci(n - 1, memo) + fibonacci(n - 2, memo)
    return memo[n]

An example of using the LRU concept and memoization is the functools.lru_cach decorator, which provides the ability to cache function results using the Least Recently Used strategy.

from functools import lru_cache

@lru_cache(maxsize=100)
def expensive_function(arg1, arg2):
    # some expensive computation going on here.
    result = ...
    return result

The maxsize parameter specifies the maximum number of arguments whose results can be stored in the cache. When this number is filled, when the function is called with new arguments, the least recently used element will be removed. This can save time when an expensive function is called periodically with the same arguments. The default value of maxsize is 128. You can use @lru_cache(max_size=None) to have an unlimited cache. It would mean that the cache would not evict items and grow indefinitely. Sometimes, it might make sense for some use cases (like short-lived scripts), but in such cases, it's better to use the functools.cache decorator that does the same as @lru_cache(max_size=None) but does it faster as it does not contain any logic related to size checks.

Caching delays

Finally, it's worth talking about delays when caching data. In all the examples above, we considered the reduction in operating speed as a disadvantage. However, in some cases, caching latency is a necessary element for the optimal functioning of the application.

For example, in distributed systems where data is stored across multiple servers, it may be necessary to set a cache latency to ensure data consistency across servers.

Latency caching can also manage server load and reduce maintenance costs. Additionally, caching latency can be used to manage server load or prevent abuse, such as DDoS attacks. Caching delays can also be used for non-critical permanent storage to store values between script restarts.

Another vital thing about caching delays is the hot and cold cache.

A storage area known as a hot cache is typically used to store frequently accessed data or data that needs to be accessed quickly. It is commonly located in RAM or on an SSD, allowing fast read and write speeds. A short and intentional caching delay is necessary for a hot cache to ensure the stored data is always up-to-date and fresh. However, this may cause the cache to be frequently updated by calling the data source, which can increase the system load.

A cold cache typically contains data that is rarely accessed or does not require high access speed. Cold cache is usually located on an HDD or cloud storage to save space and reduce cost. Therefore, cold caches can have long, intentional cache latencies to reduce the number of hits to the data source and reduce storage costs. However, this may cause the cache's data to become outdated.

Conclusion

Caching is essential to optimize your application's performance, particularly when managing extensive data or intricate calculations. While the LRU algorithm is a commonly used and relatively simple cache eviction strategy, it may not always be the best choice based on your application's specific requirements. TTL and denormalization are other techniques that can enhance caching efficiency and guarantee data relevance.

Ultimately, the caching strategy you select will depend on your application's specific needs and the trade-offs you will make regarding memory usage, processing time, and data consistency.

Caching is a valuable tool that can significantly speed up dynamic websites by reducing the overhead of processing requests. Setting up caching correctly using the right concepts and algorithms can make your site faster and more responsive, improving user experience and increasing performance.