Faster Deployment and Dependency Management of Node.js apps with PNPM

As software engineers, we all know that fast and efficient deployment of our applications is crucial to delivering value to our users. However, managing dependencies can be a time-consuming and frustrating process, especially when dealing with large codebases and complex frameworks.

In this blog, we will explore the benefits and best practices of using PNPM for faster deployment and more efficient dependency management. We will share our own experiences and insights at Halodoc, as well as drawing on the latest research and thinking in this field.

Problem to Solve

Previously Halodoc using NPM for dependency management often faced issues with slow deployment times, especially when making frequent changes to the codebase. NPM has a tendency to install multiple copies of the same dependencies in different folders, which can lead to redundant storage and longer installation times. As a result, developers were forced to spend valuable time waiting for dependencies to install, which in turn slowed down the deployment process and hindered productivity. Furthermore, large node_modules folders would bloat the repository size and slow down Git operations, making the codebase difficult to maintain and collaborate on. This is where PNPM comes in, providing a solution to these problems by improving installation speed and disk space utilization, and allowing developers to focus on delivering value to their users.

How PNPM is different from NPM and YARN?

PNPM, NPM, and Yarn are all package managers for Node.js applications, but they have some key differences in their approach to dependency management.

One of the main differences between PNPM and NPM/Yarn is that PNPM uses a unique approach to dependency management called "linking." Instead of installing each package separately in a node_modules directory for each project, PNPM creates a single virtual store of packages on the file system and links to them from each project. This approach can save a significant amount of disk space and reduces the time required for package installations and updates, especially in large codebases with many shared dependencies.

Another difference is that PNPM can use multiple versions of the same package simultaneously, which can be useful in cases where different dependencies require different versions of the same package. NPM and Yarn, on the other hand, use a flat dependency tree and only allow a single version of each package to be installed in the project.

Finally, PNPM also supports parallel package installation and improved caching, which can lead to faster and more efficient dependency management compared to NPM and Yarn.

Overall, while NPM and Yarn are popular and widely-used package managers, PNPM offers some unique advantages in terms of disk space usage, installation speed, and handling of multiple package versions.

Installation in Halodoc Repository

To ensure that PNPM operates efficiently, we can make improvements in two areas. The first step is to install it in our codebase, which will allow us to run it locally. By doing so, we can reduce the time it takes to install dependencies, making the process faster and more secure. Additionally, PNPM will help us better manage our dependencies.

The second step is to install PNPM in our Jenkins pipeline, allowing it to be run during deployment. By doing so, we can speed up the installation and testing modules, making the deployment process faster.

The flow after the installation, will be like this:

PNPM Codebase Installation

• Run the following command to install PNPM globally on your system:

• Since Halodoc is using Angular, we need to make some configuration changes in the angular.json file.

• Replace all of the NPM command script to PNPM in package.json
• Add this script to package.json to prevent other developers from using NPM instead of PNPM.

PNPM Jenkins Pipeline Installation

When deploying with our Jenkins pipeline, we should consider changing the package manager to PNPM. Doing so will enable us to leverage PNPM's capabilities for module installation, testing, and other processes within Jenkins. This change can help streamline the deployment process and ensure that our modules are managed effectively.

In one of Halodoc's repositories, we have a Jenkins pipeline that includes a module installation and testing process. Previously, this process used the npm install and npm run test:ci commands. However, after migrating to PNPM, we have updated the commands to pnpm install and pnpm run test:ci. This change has enabled us to take advantage of PNPM's capabilities for faster and more secure module installation and management.

The Challenges We Found

During our installation, we encountered some challenges:

• Dependency Mismatches
  PNPM enforces strict dependency management, which can result in errors if dependencies are not managed correctly. We faced this issue due to some dependencies not being the same version as their corresponding peer dependency. To resolve this, we updated both the main dependency and peer dependency to the same version and updated the pnpm-lock.yaml file.
• Unsupported Angular Version
  We have encountered compatibility issues with PNPM and Angular version 12 in some of our repositories. This is due to PNPM being incompatible with ngcc, which is the Ivy compatibility compiler used in Angular 12. As a result, we need to upgrade the Angular version to at least version 13. In Angular v13, the framework libraries will transition to partial compilation, which will enable a build experience that does not require ngcc. By upgrading to the latest compatible version of Angular, we can ensure compatibility with PNPM and avoid further issues.
• Obsolete Dependencies
  When PNPM encounters an obsolete dependency, it may throw an error because it cannot properly install or manage that dependency. Additionally, obsolete dependencies may pose security risks or prevent the project from functioning as intended. To avoid such issues, it is recommended to regularly update and manage dependencies to ensure they are up-to-date and compatible with each other.

After conducting thorough testing and addressing any issues that arose during the installation process, we were able to successfully integrate PNPM into our codebases. With PNPM now running smoothly, we conducted benchmarking tests to measure its performance and evaluate its impact on our development process. These tests provided us with valuable insights, allowing us to make informed decisions regarding its continued use in our workflows.

The Benchmark

Since we made changes in two areas, both locally and in the Jenkins pipeline, we would like to share the resulting outcome with you.

Local Testing

Before: run npm install
The Total time is 1 minute and 37 seconds

After: run pnpm install
The total time is 21 seconds

In the local testing scenario, switching from npm install to pnpm install reduced the total time from 1 minute and 37 seconds to just 21 seconds. This represents a reduction of over 75% in installation time, a significant improvement that can help developers be more productive and efficient.

Jenkins Pipeline Testing

• Before:
  Run npm install for 1 minute and 35 seconds
  Run npm run test:ci for 5 minutes and 51 seconds

• After:
  Run pnpm install for 40 seconds
  Run pnpm run test:ci for 4 minutes and 18 seconds

Based on the benchmark, it is clear that using PNPM instead of NPM in the Jenkins pipeline has resulted in significant improvements in module installation times, while test execution times remain similar.

In the scenario where NPM was used, the installation time was 1 minute and 35 seconds, while the test execution time was 5 minutes and 51 seconds. In contrast, using PNPM reduced the installation time to just 40 seconds, representing a reduction of approximately 58%. The test execution time remained similar at 4 minutes and 18 seconds.

Overall, the changes made to the local environment and Jenkins pipeline have resulted in faster and more secure module installation and management, making it easier and more efficient to deploy code changes.

Conclusion

Using PNPM as a package manager for Node.js applications has proven to be a game-changer for faster deployment and more efficient dependency management. By installing PNPM in both the local codebase and Jenkins pipeline, developers can save significant time and improve productivity. The benchmark results clearly show that PNPM can reduce installation time by over 75% and streamline the deployment process without compromising on security or compatibility with the NPM registry. At Halodoc, we have experienced these benefits and recommend PNPM to all software engineers looking for a better way to manage dependencies.

References:

• PNPM documentation:  https://pnpm.io/installation
• PNPM Source Code: https://github.com/pnpm/pnpm

Join us

Scalability, reliability and maintainability are the three pillars that govern what we build at Halodoc Tech. We are actively looking for engineers at all levels and if solving hard problems with challenging requirements is your forte, please reach out to us with your resumé at careers.india@halodoc.com.

About Halodoc

Halodoc is the number 1 all around Healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke. We connect 20,000+ doctors with patients in need through our Tele-consultation service. We partner with 3500+ pharmacies in 100+ cities to bring medicine to your doorstep. We've also partnered with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allow patients to book a doctor appointment inside our application. We are extremely fortunate to be trusted by our investors, such as the Bill & Melinda Gates Foundation, Singtel, UOB Ventures, Allianz, GoJek, Astra, Temasek and many more. We recently closed our Series C round and In total have raised around USD$180 million for our mission. Our team works tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia.