Working with filesystem

Interaction with a server usually involves working with directories and files. Imagine you are building an Instagram clone where users can upload their photos and videos. You may want to save these files in a specific folder, move, update or even delete them sometime in the future. Node.js has an inbuilt fs module that allows us to do these operations with ease.

In this topic, we will take a closer look at the fs module in general, cover common uses to work with directories, and learn about asynchronous vs synchronous methods.

About the fs module

The fs module is one of the core features of Node.js. It helps us store, manage, and access data on our operating system. It comes with Node.js, meaning we don't need to install it, as it's already included and available for us. To start using it in your project, simply initiate a new constant and require the fs module:

const fs = require('node:fs');

Now we can access plenty of methods. Some of the most common methods that the fs offers are the following:

• fs.mkdr to create a new directory
• fs.rmdr or fs.rm to delete a directory
• fs.readFile to read data from a file
• fs.appendFile to add data to a file
• fs.unlink to delete a file

You can learn about other methods in the Node.js Documentation.

All methods are divided into two groups: asynchronous (readFile, writeFile, and so on) and synchronous (readFileSync, writeFileSync, and so on). Asynchronous operations do not block the execution of the programs, while with synchronous ones the program waits for the current operation to finish before it can move on to the next one. Below you can see two images representing each method. Here, fs.writeFileSync calls OS to write to the file, then hangs on for some time, and finally finishes the operation. During this "waiting" period, we can not run other operations.

Here's another scenario with the asynchronous method fs.writeFile. As in the previous example, it also calls OS to write to file, but while doing this operation, the program can still process other work. It can be writing to another file, reading a file, calling an API, and many more.

Well, which method to choose? It's safe to say that asynchronous methods should be preferred over synchronous ones, as the former allows us to execute multiple parallel actions at a time.

This advantage is easy to notice with our Instagram clone app. Say, several users upload photos at the same time, so our app will receive more than one request at a moment. To handle them all, asynchronous methods would be our best choice. Our server will start to execute one command, then switch to the second one even if the first one is still in progress. The previous command will run in the background and load the result once it finishes processing.

In some rare cases, however, you might want to perform a specific action first and block the rest of the code until the action is finished. Just once on startup, you may want to read a configuration file and only then run the remaining code:

const fs = require('node:fs');

const config = JSON.parse(fs.readFileSync('./configFile.json');

// run the code below after config is available

To sum up, sync methods are good to use in the above-like cases or for debugging purposes. Otherwise, always use async methods when working with the file systems.

Creating and deleting directories

There exist a lot of methods to work with directories, so let's study some of them.

Back to our Instagram clone app. In our backend, we want to create a new folder where we will save all the media files that users send to us. Before we do this, take a look at our starter app: it has an index.js file and package.json, pretty straightforward:

Now let's use the sync method mkdirSync to accomplish our goal. It takes a path as the first parameter and options as the second. The second parameter is optional, but we can use it to make chained folders (with a second argument as an object with recursive property set to true).

The async method mkdir also takes a path as the first parameter and a callback as the last parameter. It will fire off when our directory is successfully created. We included err as an argument in the callback in case something goes wrong. For example, if the avatars folder already existed, we could not have made it and an error would have popped up.

You might ask in what order these operations will run. Good question! The first two methods are synchronous, so they will fire one after another, and once they are done, the program will execute the third asynchronous method. So, first, we will see the media folder, then the assets/videos/stories folders, and, finally, the avatars folder will be created.

const fs = require('node:fs');

// create 'media' folder
fs.mkdirSync('media');

//create assets/videos/stories folders recursively (chained example)
fs.mkdirSync('assets/videos/stories', {recursive: true});

// create avatars folder with async method
fs.mkdir('avatars', (err) => {
    if (err) {
        console.log(err);
    }
    console.log('Avatars folder is created!');
});

Voilà! Our new folders are in place:

Now, what if we want to delete some folders? Well, it's pretty easy, just write fs.rmdirSync(path) or fs.rmdir(path, callback).

Reading and renaming directories

Sometimes we need to see what is inside a folder without opening it as we normally do. Then we might want to manipulate this data somehow (to sort, filter, and so on). To read a directory, we can utilize the following methods:

fs.readdir(path, callback);

fs.readdirSync(path);

There are three pictures in the avatars folder just for demonstration purposes (kate.jpeg, mike.jpeg, cole.png). Feel free to use any files of your preference. In this example, you'll see the content of the avatars folder, as well as filter the users whose avatars have .png format. As you can see, the content is saved in the second parameter in our callback, it is an array of strings containing our folder/file names.

const fs = require('node:fs');

fs.readdir('avatars', (err, content) => {
    if (err) {
        console.log(err);
    }
    const pngFiles = content.filter(file => file.includes('.png'));
    console.log(`All files: ${content}, png images: ${pngFiles}`);
});

Here is what we get after running node index.js:

Last but not least, one more method that allows us to rename our folder names, fs.rename and fs.renameSync. The first parameter is the current path, the second is the new path. Let's change avatars to user-pictures name:

const fs = require('node:fs');

fs.rename('avatars', 'user-pictures', (err) => {
    if (err) {
        console.log(err);
    }
});

It's magic! In less than a second, we got our folder renamed. Good job!

In this tutorial, we mainly went more in-depth with the directory-related methods. File manipulation is covered in other topics.

Before we jump to a conclusion, it is worth mentioning that Node.js implemented a promises-based API that provides all key methods that we have already gone through. This approach works with async/await functions. It is more readable, convenient, and, most importantly, lets us work with a mix of async and sync operations.

Conclusion

The fs module offers many methods to work with directories and files. You can create, read, delete, and rename folders. The methods are divided into two main groups: sync and async. While they do the same work, it is important to differentiate between them and utilize them according to use cases. No need to learn all of them at once, just reference the documentation when needed.