The difficulties of asynchronous programming

Losing control of asynchronous code in JavaScript is undoubtedly easy. Closures and in-place definitions of anonymous functions allow for a smooth programming experience that doesn’t require the developer to jump to other points in the codebase.

This is perfectly in line with the KISS principle (Keep It Simple, Stupid); it’s simple, it keeps the code flowing, and we get it working in less time. Unfortunately, sacrificing qualities such as modularity, reusability, and maintainability will, sooner or later, lead to the uncontrolled proliferation of callback nesting, functions growing in size, and poor code organization.

Most of the time, creating callbacks as in-place functions is not strictly required, so it’s more a matter of discipline than a problem related to asynchronous programming. Recognizing that our code is becoming unwieldy or, even better, knowing in advance that it might become unwieldy and then acting accordingly with the most adequate solution, is what differentiates a novice from an expert.

To explain this problem, we will create a little web spider, a command-line application that takes in a web URL as input and downloads its contents locally into a file. In the code presented in this article, we are going to use a couple of npm dependencies:

The core functionality of our application is contained inside a module named spider.js . Let’s see how it looks. To start with, let’s load all the dependencies that we are going to use:

Next, let’s create a new function named spider() , which takes in the URL to download and a callback function that will be invoked when the download process completes:

There is a lot going on here, so let’s discuss in more detail what happens in every step:

To complete our web spider application, we just need to invoke the spider() function by providing a URL as an input (in our case, we read it from the command-line arguments). The spider() function is exported from the file we defined previously. Let’s now create a new file called spider-cli.js that can be directly invoked from the command line:

Now, we are ready to try our web spider application, but first, make sure you have the utils.js module and the package.json file containing the full list of dependencies in your project directory. Then, install all the dependencies by running the following command:

Now, let’s execute the spider-cli.js module to download the contents of a web page with a command like this:

In the next section, you will learn how to improve the readability of this code and, in general, how to keep callback-based code as clean and readable as possible.

Looking at the spider() function we defined earlier, you will likely notice that even though the algorithm we implemented is really straightforward, the resulting code has several levels of indentation and is very hard to read. Implementing a similar function with a direct style blocking API would be straightforward, and most likely, the code would be much more readable. However, using asynchronous CPS is another story, and making bad use of in-place callback definitions can lead to incredibly bad code.

The situation where the abundance of closures and inplace callback definitions transforms the code into an unreadable and unmanageable blob is known as callback hell. It’s one of the most widely recognized and severe anti-patterns in Node.js and JavaScript in general. The typical structure of code affected by this problem looks as follows:

You can see how code written in this way assumes the shape of a pyramid due to deep nesting, and that’s why it is also colloquially known as the pyramid of doom.

The most evident problem with code such as the preceding snippet is its poor readability. Due to the nesting being so deep, it’s almost impossible to keep track of where a function ends and where another one begins.

Another issue is caused by the overlapping of the variable names used in each scope. Often, we have to use similar or even identical names to describe the content of a variable. The best example is the error argument received by each callback. Some people often try to use variations of the same name to differentiate the object in each scope, for example, err , error , err1 , err2 , and so on. Others prefer to just shadow the variable defined in the upper scope by always using the same name, for example, err . Both alternatives are far from perfect, and cause confusion and increase the probability of introducing defects.

Also, we have to keep in mind that closures come at a small price in terms of performance and memory consumption. In addition, they can create memory leaks that are not very easy to identify. In fact, we shouldn’t forget that any context referenced by an active closure is retained from garbage collection.

If you look at our spider() function, you will notice that it clearly represents a callback hell situation and has all the problems just described. That’s exactly what we are going to fix with the patterns and techniques that are covered in the following sections of next articles.

Related posts:

Not very long ago in the history of man’s foray into the interwebs, life was simple, and so were applications. They had simple linear flows and so few screens that all of them could be stacked…

If we want uncover what makes us unique, then we must do the work that others are unwilling to do: the rigorous task of constant self-actualization. Raising our own self-awareness will deepen our…

Entrepreneurship still proves to be one of the most innovative solution to many socio-economic challenges faced in Africa’s underprivileged communities. Entrepreneurial development is still a…