Marshalling data

In this topic, we will discuss what marshaling stands for. It is a crucial topic because data transmission largely determines the work of a program. When we marshal data, we sort and structurize the data for end-users.

Basic marshaling

In broad terms, marshaling is a process of converting an object in computer memory into a form that can be sent as a message; for example, a response message from your application.

Our journey into the depth of information formatting starts with the Flask built-in marshal_with decorator. It transforms data, depending on the predefined scheme we've developed. But first, the standard ritual — create the default app and API objects:

from flask import Flask
from flask_restful import Resource, Api, marshal_with, fields

app = Flask('main')
api = Api(app)

Then, let's create a class named TodoDao. This is the object we'll be working with:

class TodoDao(object):
    def __init__(self, task, description):
        self.task = task
        self.description = description

Usually, we don't want to limit ourselves by an app. We also want to share objects with the client, whether it's a frontend application or another machine. Let's create a dictionary that will define the rules for marshaling. For this task, we will use the fields module that contains different types of data: string, integer, DateTime, and others.

resource_fields = {
    'task':   fields.String,
    'description':    fields.String
}

The marshaling architecture will be quite simple: one object with data fields (the TodoDao class), one dictionary with data types for each field (resource_fields), and Resource. The only thing missing, for now, is the Resource that is called for the TodoDao object information:

class Todo(Resource):
    @marshal_with(resource_fields)
    def get(self, **kwargs):
        return TodoDao(task="I've become so numb", description="I can't feel you there")

As you can see, we use the marshal_with decorator and specify the data types of our dictionary in brackets. After that, we return the TodoDao object. Because of the decorator, the function will return an automatically generated response object with the 200 status code and the resulting dictionary as a message.

Now, it's time to add Resource to the API and launch the app!

api.add_resource(Todo, "/")
app.run()

When we execute the curl 127.0.0.1:5000/ command, the app outputs the following:

This means that everything is correct. What is the difference, though? Let's see another example. We will replace the value of the task string with an integer in the code:

class Todo(Resource):
    @marshal_with(resource_fields)
    def get(self, **kwargs):
        return TodoDao(task=20072017, description="I can't feel you there")

Now, let's restart the application and execute another request. We will see how the data is updated:

If you look carefully, you will see that we've specified an int value in the task argument, but as a JSON answer, we've got an str value. This is due to the marshal_with decorator that helped us transform an integer into a string.

Marshmallow for marshaling

We've figured out how to create common datatype rules, but now it's time for serious business. Let's talk about the marshmallow module that can be very useful in complex and bulky APIs. It allows us to create schemas that we will use as frames for Resources. For example, we know that our task resource has a name and a description. Previously, we've defined its fields and their datatypes using a simple dictionary. This is inconvenient when we use dictionaries when working with databases. So, we need another method. This is where Object-oriented programming can help us, namely, the marshmallow module. Let's define the default app and API:

from flask import Flask
from flask_restful import Resource, Api, reqparse
from marshmallow import Schema, fields


app = Flask('main')
api = Api(app)

The next thing we should do is to create an argument parser for our API. Say, we need three arguments for our task resource: name, description, and severity. The parser will look like this:

parser = reqparse.RequestParser()
parser.add_argument('name')
parser.add_argument('description')
parser.add_argument('severity')

It's pretty simple to create the parser and add three arguments.

Now, a new thing: schema. We can create a class of TaskSchema and add some fields there:

class TaskSchema(Schema):
    name = fields.String()
    description = fields.String()
    severity = fields.Integer()

The attributes of this class will repeat the parser arguments, so if you change the parser, make sure that you change the schema as well.

The final steps are to create Resource:

class TaskResource(Resource):
    def get(self):
        schema = TaskSchema()
        args = parser.parse_args()
        return schema.dump(args)

In this snippet, you can see an instance of TaskSchema and the arguments. We obtain these arguments from the parser to create a response combining schemas opportunities and the data from users. And this is it! Let's add the resource to our API and start the app.

api.add_resource(TaskResource, '/')

app.run()

curl "http://127.0.0.1:5000/?name=The%20End&description=We%20tried%20so%20hard%20and%20get%20so%20far&severity=5"

This curl command is pretty long, and you can spot %20 inside. What are they? When we send requests with curl, we should replace all spaces with the %20: this is an encoded analog of the space bar. The result is as follows:

Basic marshaling VS marshmallow

Up to this point, marshmallow looks pretty similar to marshal_with. What's the difference, and why would we want to use marshmallow instead?

Sometimes our objects are more complex than just a few fields. Sometimes it processes database objects with relations, nested objects. Marshmallow allows us to build nested schemas as well. Also, it is possible to set a default or a missing value for each field in a schema for deserializing and serializing data. It could look like this:

class TaskSchema(Schema):
    name = fields.String()
    description = fields.String()
    severity = fields.Integer()

class LessonSchema(Schema):
    name = fields.String()
    description = fields.String()
    level = fields.Integer()
    tasks = fields.Nested(TaskSchema, default=[], missing=[])

Marshaling often comes together with unmarshaling. This is the opposite process of converting readable data (a dictionary) into an object. We use the load function for this purpose.

And it's only the beginning! If you want to discover more interesting features of that module, we recommend you visit the Marshmallow documentation.

Conclusion

In this topic, we've touched upon the concept of data marshaling. Now, we know a few things about RESTful Flask: we can create RESTful APIs, marshal data, and how it can help us. Great job, don't slow down!