HTTP protocols for testers

We've talked a lot about HTTP, but now, let's dive deeper. Let's discover why this protocol is key and beneficial, how it works, and, most importantly, why as a tester, you need it.

Structure of requests and responses

Let's study the structure of Requests and Responses taking a restaurant's website as an example. When accessing a restaurant's website, your browser sends a request to the server hosting the site using the HTTP protocol. This request informs the server of what you aim to do on the site.

An HTTP request consists of three main elements:

• The request line, which has three parts - the method, the target URL, and the HTTP version. The method indicates the action you want to perform on the server, such as GET (to retrieve a resource) or POST (to submit data). For instance, if you want to view the restaurant menu, your request line might look like this:

  GET /menu HTTP/1.1
  

• Request headers provide additional information about the request. They can include things like the browser type, accepted languages, cookies, etc. Headers enable the server to better understand your request and respond suitably. Here's an example request:

  GET /api/products/12345 HTTP/1.1
  Host: example.com
  Accept: application/json
  

• Request body (optional): This element isn't always included in every request. It comes in handy when you need to send data, like when filling out a form or submitting a comment. The request body follows the headers and is separated from them by an empty line.

Imagine you want to make a restaurant reservation and submit the details with a POST request. Your request body might look like this:

POST /api/reservation HTTP/1.1
Host: restaurant-website.com
Content-Type: application/json
{
   "guest_name": "John Doe",
   "email": "[email protected]",
   "phone": "+123456789",
   "party_size": 4,
   "date": "2023-03-01",
   "time": "19:00",
   "special_requests": "Window seat preferred"
}

Once the server receives your request, it generates an HTTP response containing the requested resource (e.g., a webpage) and other information. A response usually involves:

1. Status code: This is a three-digit number showing the request status. For instance, the status code "200 OK" implies a successful request, while "404 Not Found" indicates the requested resource couldn't be found on the server.

2. Headers: Much like request headers, response headers supply extra details about the response. They can hold data like content type, server information, caching instructions, and more.

3. Body: This part contains the actual content. For instance, if you request a webpage on a restaurant's site, the 'Body' has different functions. It contains the HTML code, advising your browser on how to display the webpage. It includes the text, headings, links, buttons, and other elements you can see on the page.

If you successfully book a table at a restaurant, the server might send this response:

HTTP/1.1 201 Created
Content-Type: application/json
{
   "reservation_id": "123456",
   "message": "Reservation successful. Your reservation ID is 123456. We look forward to welcoming you!"
}

Methods of requests

Methods serve as directives you give to the server when interacting with a website. There are six fundamental query methods.

• The GET method is handy if you're viewing a restaurant's website and want to see the restaurant's menu. GET sends a request to a server that responds with the menu.

• The HEAD method is similar to the GET method but is used to request information about the headers or metadata of a server resource, without actually retrieving the resource's full content.

• The POST method lets you send additional data to the server with the request. For instance, if you want to write a restaurant review after visiting it, POST will help you send the information to the server, and the server will process it accordingly.

• The PUT method is planned to send new information to the server. For instance, suppose you made a booking error. This method allows you to update or replace existing server information. You can send an amended version of the order so that the server can make the necessary corrections. The PATCH method serves the same purpose. The difference is that PUT is used to entirely replace the data with new data, while PATCH is used to partially update the data.

• The DELETE method is useful if you wish to cancel the reservation. It sends a server request to delete a particular piece of information from the server's memory.

• The OPTIONS method in HTTP is used to request information about the communication options available for a given resource. When a client sends an OPTIONS request to a server, the server responds with information about the supported HTTP methods, headers, and other possible capabilities for the specified resource.

Two important questions often asked in job interviews are linked to security and idempotency, crucial concepts when handling HTTP requests.

Security in the context of requests refers to the steps taken to protect data and systems from unauthorized access, attacks, and other possible threats. It involves implementing safeguards to ensure data confidentiality, integrity, and availability. Here are some crucial aspects of security in requests:

• Authentication. Confirming the identities of users or systems making requests, frequently using tokens, API keys, or other authentication measures.

• Authorization. Deciding whether the authenticated user or system has the permissions needed to perform the requested action. This ensures that users can only access or change data they've been allowed to.

• Encryption. Ensuring secure data transmission between the client and server by using protocols like HTTPS (HTTP Secure) to encrypt the communication.

• Input Validation. Confirming that input data provided in requests is valid and safe to use, aiding in the prevention of common security vulnerabilities such as injection attacks.

• Cross-Site Request Forgery (CSRF) protection. Implementing measures to fend off unauthorized requests initiated by malicious actors on behalf of an authenticated user.

• Cross-Origin Resource Sharing (CORS). Defining and enforcing policies that dictate which domains can access resources on a server.

Idempotency is a property of an operation where repeating it multiple times yields the same result as performing it once. In the context of HTTP requests, idempotency ensures that making the same request multiple times has the same effect as making it once. This property is crucial in securing predictable and safe interactions, particularly in scenarios where requests might need to be retried due to network issues or other failures.

Some common HTTP methods are considered idempotent:

1. GET. Retrieving data should be idempotent because making the same GET request multiple times yields the same result.

2. PUT. Updating a resource with a PUT request is idempotent because if you perform the same update multiple times, the result is identical to performing it once.

3. DELETE. Deleting a resource with a DELETE request is also idempotent. Deleting a resource multiple times doesn't change the fact that the resource is no longer there.

By contrast, some methods like POST aren't inherently idempotent since repeating the same POST request can lead to the creation of multiple instances of a resource. Guaranteeing idempotency is crucial in distributed systems and when dealing with unreliable networks, as it helps in avoiding unintended side effects or data inconsistencies caused by duplicate requests.

Basic data transfer formats

Data transfer formats are like the language that the client and server use to understand each other. They dictate how data is structured and organized before transmission.

JSON (JavaScript Object Notation) is widely used due to its simplicity and easy readability. It's like a common tongue that both the client and server can understand. JSON presents data in key-value pairs, similar to a dictionary. It's easy to work with and nearly all programming languages support it. It's lightweight, making it suitable for transmitting small to medium-sized data. For example, say we want to send some information about a user:

{
"Name": "John Doe",
"age": 30,
"email": "[email protected]"
}

In this example, "name" would be the key, and "John Doe" would be its value.

XML (eXtensible Markup Language) is another common format used for data transfer. It's somewhat more detailed compared to JSON but is excellent for representing complex hierarchical data structures. XML defines custom tags to describe the structure and content of data.

Here's an example of how the same user information would look in XML:

<user>
<name>John Doe</name>
<age>30</age>
<email>[email protected]</email>
</user>

In this example, the opening <user> tag signals the start of the user data block. The </user> tag closes this block. Likewise, the <name>, <age>, and <email> tags.

Both JSON and XML have their strengths and weaknesses, so it's essential to select the right format based on specific requirements. While JSON is generally preferred for its simplicity and compatibility, XML provides more flexibility for complex data.

CSV (Comma-Separated Values) is a lightweight data format commonly used for transferring tabular data. Imagine it as a way of storing information in plain text, where each line represents a row of data, and the values in each line are separated by commas.

Here's an example of a fruit list in CSV format, where each row represents a fruit record and the values are separated by commas:

Fruit, Color, Taste
Apple, Red, Sweet
Pear, Yellow, Juicy
Banana, Yellow, Sweet
Grape, Purple, Sweet

In this example:

• The first line contains the column headings: 'Fruit', 'Color', 'Taste'.

• Each subsequent row is a record of a fruit, where the values are separated by commas.

• For instance, the first entry is about an apple that is red and tastes sweet.

CSV is often used to transfer data between a client's computer and a server using the HTTP protocol. It is supported by nearly every programming language out there. Suppose there's a web application that needs to send a large quantity of data to a server. Instead of using a more convoluted data format, like JSON, you could opt for a simple CSV. Sending the data in CSV format simplifies the payload, making it easier for the server to parse and process the information.

YAML (Yet Another Markup Language) is a human-readable data serialization format. Unlike complex formats like XML or JSON, YAML is designed to be minimalistic and concise, making it more accessible for humans to read and write.

Benefits of YAML:

1. Readability: YAML's readability enables developers to quickly understand the structure of the data.

2. Conciseness: YAML allows data to be represented tersely, reducing the clutter and noise common in other formats.

3. Flexibility: YAML supports complex data structures, including lists, dictionaries, and nested objects, making it a versatile choice for representing data.

To present data in YAML, we use a combination of indentation and key-value pairs. Here's a basic employee record in YAML:

employee: 
"name": "John Doe"
"age": 25
"position": "tester"

In the example above, we have an employee record represented using YAML syntax. The "employee" key serves as the main container, and the indented lines that follow it represent the key-value pairs of the employee's attributes. In this case, we have three attributes: name, age, and position. Note how the indentations aid in visually representing the structure and hierarchy of the data.

Error codes

As mentioned earlier, when a response is received from the server, getting error codes is common. As a tester, it's essential to be familiar with these to understand the problem and properly report the error to a web developer. These codes serve as handy signals that indicate an error during the request-response cycle.

Errors starting with 1xx are informative:

• 100 Continue: The server confirms that everything's fine and the client can continue sending the rest of the request.

• 101 Switching Protocols: The server asks the client to switch to a different protocol.

• 102 Processing: This is an intermediate server response, letting the client know that it has received the request and is processing it.

Codes starting with 2xx indicate successful processes:

• 200 OK: This indicates that your request was processed successfully.

• 201 Created: This suggests that the request resulted in a new resource's creation.

• 202 Accepted: This indicates that your request has been accepted by the server but is yet to be processed.

• 203 Non-Authoritative: This indicates that the server isn't the original source of the requested data.

Codes starting with 3xx are about redirection:

• 300 Multiple Choices: This occurs when the server has multiple options available in response to a client's request. It tells the browser that there's more than one URL to follow.

• 301 Moved Permanently: The browser is informed that the requested URL has been permanently moved to a new place.

• 302 Found: This error indicates that the requested URL has been temporarily moved.

Client errors are marked with 4xx:

• 400 Bad Request: The server can't understand the request due to incorrect syntax or missing information.

• 401 Unauthorized: This appears when the client tries to access a restricted resource without providing valid authentication credentials.

• 403 Forbidden: This indicates that the client lacks permission to access the requested resource.

Server errors are highlighted by 5xx:

• 500 Internal Server Error

• 501 Not Implemented

Conclusion

In practice, testers can use tools to analyze HTTP traffic, such as browser-based developer tools, Postman, or Fiddler. Testers with a deep understanding of HTTP protocols can identify and analyze data transfer issues, ensure compatibility across browsers and platforms, and improve overall application performance.