Effective compression technique for web applications using Brotli
Performance plays a vital role in user experience impacting engagement. Web apps need to ensure they are performant, as most of the times they are accessed on mobile devices with flaky networks.
Web apps usually comprise of static assets such as .js .css and .json text files with redundancy and role of compression playing a vital part. Compression helps reduce the bandwidth requirement, speeding up the website rendering.
Web Supported formats
There are two commonly used HTTP compression schemes on the web today: DEFLATE, and GZIP.
DEFLATE is a very popular compression algorithm which generally wraps up data using the LZ77, algorithm and Huffman coding. GZIP is a file format that uses DEFLATE internally, along with some interesting blocking, filtering heuristics, a header and a checksum. In general, the additional blocking and heuristics that GZIP uses give it better compression ratios than DEFLATE alone.
What is Brotli?
Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression. In short, Brotli is based on the same foundations as GZIP (LZ77 and Huffman encoding) which improves with
- Static dictionary
- Larger matching windows, and
- Extended context modeling for better compression ratios.
We used the highest possible compression settings, as compression time didn't matter to our use case.
An additional advantage is that it is supported by all major browsers.
Gzip vs Brotli:
The advantage for Brotli over gzip is that it makes use of a dictionary and thus it only needs to send keys instead of full keywords.
- with Javascript files typically, the compression is better by 14% compared to GZip.
- for HTML files the compression is better by 21%.
- and with CSS files the compression is better by 17%.
How does it work?
The browser sends an Accept-Encoding header with the algorithm it supports and its order of precedence, the server picks one, uses it to compress the body of the response and uses the Content-Encoding header to tell the browser the algorithm it has chosen. As content negotiation has been used to choose a representation based on its encoding, the server must send a Vary header containing at least Accept-Encoding alongside this header in the response; that way, caches will be able to cache the different representations of the resource.
How to apply?
There were two ways by which we can deliver Brotli compressed assets:
- Enabling Brotli on our web-server
- Enabling Brotli on CDNs
Configuring with nginx:
Google has provided a module for Brotli which needs nginx to be installed from source. Once installed, the following settings need to be put in the nginx conf file:
After this, all content types which are mentioned in brotli_types setting will be brotli compressed.
Configuring with node server:
We used Node.js compression middleware shrink-ray-current npm module to apply Brotli compression.
Configuring with AWS CloudFront distribution:
You can use AWS CloudFront to automatically compress files of certain types and serve the compressed files when viewers support them (viewers indicate their support for compressed files with the Accept-Encoding HTTP header). CloudFront can compress files using the Gzip and Brotli compression formats. When the viewer supports both formats, CloudFront uses Brotli.
CloudFront's Brotli Edge compression delivers up to 24% smaller file sizes as compared to Gzip. Brotli is available for immediate use at no additional cost. This feature can be enabled via the CloudFront Console, SDK, and CLI.
Step 1:
Create a cache policy to specify caching settings, and make sure the Gzip and Brotli settings are both enabled.
Step 2:
Apply the created cache policy in step #1 to the Cloudfront distribution behaviour and whitelist the Accept-Encoding header.
What is the outcome?
When we enabled Brotli compression with all the static (JS, CSS, JSON and SVG) files and server side text content (HTML response) to our website Halodoc.com, we observed a positive result that there is approximately 15% reduction in js files and 20% reduction in HTML ones.
Conclusion
Brotli is a more powerful compression algorithm than gzip. By using Brotli for static content, we see impressive site speed wins, especially for low-bandwidth scenarios. We believe Brotli has a lot of potential and we encourage every website to invest in enabling Brotli for better user experience.
Interesting in exploring with us
We are always looking out for top engineering talent across all roles for our tech team. If challenging problems that drive big impact enthral you, do reach out to us at careers.india@halodoc.com
About Halodoc
Halodoc is the number 1 all around Healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke.
We connect 20,000+ doctors with patients in need through our Tele-consultation service. We partner with 2500+ pharmacies in 100+ cities to bring medicine to your doorstep. We've also partnered with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allows patients to book a doctor appointment inside our application.
We are extremely fortunate to be trusted by our investors, such as the Bill & Melinda Gates foundation, Singtel, UOB Ventures, Allianz, Gojek and many more. We recently closed our Series B round and In total have raised USD$100million for our mission.
Our team work tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia.