Caching Analysis: In-Process Cache at Halodoc
As with most of our technology choices, we evaluated different in-process caching libraries available for Java. Our objective was to select a suitable caching library that fits our requirement.
Our requirement criteria were as follows:
• GC Overhead - Low GC overhead
• Cost (Maintenance and Monitoring)
• RAM Overhead - while this is impacted from GC overhead, we were interested to select caching libraries with less memory usage
• Eviction Policy
• Community Support
• Datatype Support
• Language Support
• Thread Safety
For our initial analysis, we started out considering all the possible in-process caching libraries available in the market. Libraries such as:
• Cache2K
• Caffeine
• Guava Cache
• Varnish
• ConcurrentLinkedHashMap
• EhCache3
• JackRabbit
• ElasticSearch
Out of these, we zeroed in on Cache2K, Caffeine and Guava Cache for our final analysis. Among others, Varnish, we figured, was primarily a good caching library for HTTP static assets. ConcurrentLinkedHashMap with accessOrder (as opposed to insertionOrder) was slower than all the other libraries to implement our cache. Hence, it was dropped after the initial analysis.
Performance Benchmark Results
We ran Ben Manes’ benchmark code on the Mac Book Pro 2019 (https://github.com/benmanes/caffeine/tree/master/caffeine/src/jmh/java/com/github/benmanes/caffeine). We found that Cache2K and Caffeine perform the best of all the different caching algorithms for the randomly generated dataset. The labels for the below datasets with regards to benchmarking are LinkedHashMap_lru, Caffeine, ConcurrentLinkedHashMap, Guava, ElasticSearch, JackRabbit, Cache2K, ExpiringMap_Fifo, Expiring Map_Lru, TCache_lfu, TCache_lru
Evaluation
Then, we evaluated the three libraries: Caffeine, Guava and Cache2K for our required criteria.
Results
Based on these analyses, we decided to use Caffeine for the in-memory cache for server side and Guava Cache as the in-memory cache for Android development. Here is why:
• Guava Cache defaults to LRU eviction policy or manual overrides. While it's better than legacy caching libraries such as EhCache or a ConcurrentLinkedHashMap (with accessOrder), the LRU algorithm does not account for any criteria other than "recency" for its eviction evaluation. Therefore, it's recommended to consider Caffeine or Cache2K for the server side applications
• Both Caffeine and Cache2K have very comparable performance according to many benchmarks and the benchmark that was run locally on the machine. Both the libraries use advanced eviction algorithms such as Window TinyLfu and Clock Pro. These algorithms account for frequency of access to data along with recency, and therefore result in better cache hit rate. We decided to go with Caffeine as it gave marginally better performance than Cache2K.
• For Android, since Caffeine works only with Java 8 and above, we had to consider Guava Cache or Cache2K. Since Caffeine is a rewrite of Guava Cache, both of them have a similar coding pattern. When compared to Guava or Caffeine, cache2k has slightly more static memory overhead.As Guava Cache has a better community support and a familiar pattern of usage thus we recommend internally Guava Cache over Cache2K for Android
We are always looking out to hire for all roles in our tech team. If challenging problems that drive big impact excite you, do reach out to us at careers.india@halodoc.com
References:
• https://cruftex.net/2017/09/01/Java-Caching-Benchmarks-Part-3.html#memory-consumption
• https://stackoverflow.com/questions/55494488/caffeine-versus-guava-cache
• Memory Overhead between Guava and Caffeine - https://github.com/ben-manes/caffeine/wiki/Memory-overhead
• Cache2K User Guide: https://cache2k.org/docs/latest/user-guide.html
• Guava User Guide: https://github.com/google/guava/wiki/CachesExplained
• Caffeine: http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html
• https://github.com/ben-manes/caffeine/issues/161
• https://github.com/ben-manes/caffeine/wiki/Benchmarks
About Halodoc
Halodoc is the number 1 all around Healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke. We connect 20,000+ doctors with patients in need through our teleconsultation service, we partner with 1500+ pharmacies in 50 cities to bring medicine to your doorstep, we partner with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allows patients to book a doctor appointment inside our application.We are extremely fortunate to be trusted by our investors, such as the Billy & Melinda Gates foundation, Singtel, UOB Ventures, Allianz, Gojek and many more. We recently closed our Series B round and In total have raised USD$100million for our mission.Our team work tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia.