Unlocking The Power Of JDK 21

Introduction

In the ever-evolving software development landscape, keeping our frameworks and technologies up-to-date is crucial for maintaining peak performance, security, and compatibility. Migrating to JDK 21 involves a comprehensive analysis and rigorous validation to ensure smooth integration and compatibility within the application ecosystem. This upgrade brings many new features that promise to enhance our Java applications significantly. In this blog, we will delve into the key benefits of upgrading to JDK 21, explore its new features, and discuss the benefits and potential challenges you might encounter.

Key New Features In JDK 21

The following are the major new features supported in JDK 21 which we'll look at in detail in the sections below

1. Generational ZGC
2. Virtual Threads
3. Sequenced Collections

Generational ZGC

Z Garbage Collector (ZGC)is designed to minimize latency and manage large heaps with minimal impact on application performance. Its primary strength lies in its ability to perform garbage collection with very low pause times, even for applications with substantial memory requirements. Traditionally, ZGC operates by collecting all objects in the heap together, without distinction between young and old objects. This approach simplifies the garbage collection process but can be less efficient because it doesn’t leverage the typical memory behaviour patterns observed in applications.

The weak generational hypothesis is a well-established principle in garbage collection, which says that most objects have a short lifespan, meaning they die young. Because of this, collecting these young objects (which are short-lived) requires fewer resources and can quickly free up memory. In contrast, older objects, which have survived multiple garbage collection cycles, generally demand more resources to collect, and reclaiming them provides less additional free memory.

With JDK 21, ZGC has been enhanced to support generational garbage collection. This means ZGC now organizes objects into separate generations young and old. This generational approach allows ZGC to focus on collecting young objects more frequently, where most of the garbage is expected to be, thus improving the efficiency of memory reclamation. Older objects are collected less frequently, reducing the resource overhead associated with their collection. As a result, the Generational ZGC improves overall performance by efficiently managing different types of objects according to their age, significantly enhancing application performance and reducing garbage collection overhead.

To enable Generational ZGC in JVM configuration, we can add this -XX:+UseZGC -XX:+ZGenerational in the run file

Virtual Threads

Virtual Threads introduced by Project Loom, offer a lightweight, user-space concurrency mechanism for running concurrent tasks with minimal overhead. They are highly scalable, enabling the creation of millions of threads without consuming excessive system resources. Unlike traditional threads, which consume significant memory and have high context-switching overhead, virtual threads are efficient, allowing developers to manage many concurrent tasks without depleting system resources. This scalability is especially beneficial for applications requiring extensive concurrency. These are best for tasks with high latency, such as I/O operations, waiting for locks, or any other operation where the thread would spend much of its time waiting.

Dropwizard Jetty Virtual Threads

Most of our micro-services are built using the DropWizard framework, where we implemented virtual threads to handle HTTP requests through Jetty. However, our current services use DropWizard 2.1.x, which embeds Jetty 9.4.x, and virtual threads are only supported in Jetty 10 and later, available from DropWizard 3. x onwards.

Before Jetty 10, the default thread pool used in Jetty for handling incoming HTTP requests was typically the QueuedThreadPool. The QueuedThreadPool is a traditional thread pool implementation where each incoming request is assigned to a dedicated thread from the pool to be processed. With the QueuedThreadPool, a new thread is created for each incoming request and if all threads in the pool are busy processing requests, additional incoming requests are queued until a thread becomes available to handle them.

In Jetty versions 10 and later, the default thread pool implementation for handling HTTP requests remains the QueuedThreadPool. However, with the introduction of virtual threads in Java, Jetty can use virtual threads within the QueuedThreadPool to handle incoming requests more efficiently. This approach combines the benefits of virtual threads for concurrency and resource efficiency with the established reliability and flexibility of the QueuedThreadPool.

To implement Dropwizard Jetty Virtual Threads

• We need to upgrade the Dropwizard version from older versions to 3.0.7  (the latest version) in the service pom.xml file.
• We need to add a new property enableVirtualThreads under the server in the service yml file, with the value set to true.

Virtual Thread Executor Pool  

Creating or destroying a virtual thread is economical. Those were designed with the idea of using a different virtual thread for each request and there is never a need to pool them. It might not always be very helpful to use a thread pool or an executor service to create virtual threads. Pooling virtual threads to limit concurrency is not of any use. Instead, we can use constructs specifically designed for that purpose, such as semaphores.

While virtual threads support high throughput with many threads, assigning an expensive resource like a database connection to each thread can degrade performance and increase memory usage. To address this, it's recommended to use alternative caching strategies to share resources efficiently among virtual threads. This can be achieved by implementing a class using the decorator pattern to make a virtual thread executor behave like a fixed-size thread pool.

Creating Virtual Threads

1. Using Thread Class

2. Using Executor Service

We use the Executors.newVirtualThreadPerTaskExecutor() to create an executor service. This virtual thread executor executes each task on a new virtual thread. The number of threads created by the VirtualThreadPerTaskExecutor is unbounded.

3. Using Existing Executors With Thread Factory

Existing executors can be used with virtual threads by providing a virtual thread factory. However, since virtual threads are inexpensive to create, pooling them, as done with platform threads, misses their key advantage. Therefore, it's unnecessary to pool virtual threads because they are designed to be created cheaply and efficiently.

Sequenced Collections

JDK 21 introduces Sequenced Collections, updating Java’s collection framework. This feature adds new interfaces that simplify accessing the first and last elements of a collection, offer a reversed view, and provide a unified way to handle collections with a defined order. These improvements lead to more efficient, intuitive, and readable code with clearer structure and consistent behaviour.

To demonstrate the inconsistency, let’s make a comparison of accessing the first and last elements of different collection types before sequenced collections.

New interfaces into the existing hierarchy of collections framework:

Example of using sequence collections :

Implementation and observation at Halodoc for below features

1. Generational ZGC
2. Virtual Threads

Generational ZGC

After integrating Generational ZGC, we observed that while the used heap was lower compared to G1GC, the committed heap was significantly higher. This increase became more pronounced during peak loads, sometimes leading to application restarts.

How can we prevent a significant increase in committed heap memory when implementing ZGC?

ZGC | Using -XX:SoftMaxHeapSize

As the name suggests, this new option establishes a soft upper limit for the Java heap size. When set, the Garbage Collector (GC) will attempt to keep the heap from exceeding this threshold during its operations. However, this limit is not strict the GC retains the flexibility to expand the heap beyond the specified size if needed, particularly in situations where the alternatives would involve stalling an allocation request or throwing an OutOfMemoryError. This mechanism balances between controlling memory usage and ensuring the system functions smoothly under memory pressure.

There are different use cases where setting a soft max heap size can be useful. For example:

• When you want to keep the heap footprint down while maintaining the capability to handle a temporary increase in heap space requirement.
• When you want to play it safe, to increase confidence that you will not run into an allocation stall or have an OutOfMemoryError because of an unforeseen increase in allocation rate or live-set size.

Using -XX:SoftMaxHeapSize=2G -Xmx5G will tell ZGC to keep the max heap usage at 2GB, but it’s allowed to grow to 5GB if it otherwise would have resulted in an allocation stall or an OutOfMemoryError. This is useful when you want to keep the memory footprint down while maintaining the capability to handle a temporary increase in heap space requirement.

Memory usage Without SoftMaxHeapSize

Memory usage with SoftMaxHeapSize

With this implementation, we have achieved an overall  15-20% reduction in overall heap usage across the services.

Virtual Threads

We integrated virtual threads for HTTP requests in one of our services. After deployment, we observed a few restarts(once a day), the reason for those restarts was health check failures. We could not correlate it to the virtual thread initially but these restarts were happening once every day after the virtual thread deployment.

Upon analysis, we discovered that concurrent updates were causing application restarts. This issue happened due to the MySQL connector client. This library includes a class called ReadAheadInputStream, which has a synchronized method. The problem occurs specifically with the Query API's query.executeUpdate()  for updates. This leads to contention and application hangs when virtual threads execute these synchronized methods concurrently. This issue is known as virtual thread pinning. This does not occur when using EntityManager for updates, as it avoids synchronized blocks.

Below is the code snippet which shows the synchronized method inside the ReadAheadInputStream class which made a thread to be pinned.

In the below stack trace, we can see virtual thread got pinned inside that synchronized block while executing.

What is virtual threads pinning ?

Virtual Thread Pinning refers to a situation in Java where a virtual thread becomes stuck or pinned to a specific platform thread (carrier thread) due to certain operations or conditions. Virtual threads are mounted and unmounted from a carrier thread by the JVM. A virtual thread is mounted when it is active and unmounted when it is waiting. This pinning defeats the primary advantage of virtual threads, which is their ability to be suspended and resumed freely, allowing the JVM to manage them with high efficiency and low resource usage.

A virtual thread may become pinned to its carrier thread when,

1. It runs code in a synchronized block or method.
2. It runs a native method or Foreign Function

When a virtual thread enters a synchronized block or performs a native I/O operation (like reading from a socket), it cannot be suspended by the JVM until it exits that block or completes the operation. This pinning prevents the JVM from managing the virtual thread as efficiently as it would under normal circumstances.

In our use case as mentioned above the virtual thread is pinned while attempting to read data from the mysql server, which is a blocking I/O operation. This leads to the thread being stuck in a pinned state, unable to process other requests, and consequently, the service's ability to handle concurrent requests diminishes, leading to degraded performance or even failure under load.

Virtual thread pinning in an application is not necessarily incorrect but can be suboptimal. A pinned virtual thread monopolizes a carrier thread, making it unavailable for other virtual threads. If this happens frequently, it negates the benefits of using virtual threads, as carrier threads become overused.

Detecting pinned threads via logging

To monitor this, add -Djdk.tracePinnedThreads=full to the command line when running your Java program, which will cause the JVM to log virtual thread pinning to standard output.

How can pinning be avoided ?

To avoid pinning virtual threads to carrier threads, avoid using synchronized methods or blocks, as these can cause pinning. Instead, use ReentrantLock to manage concurrency, which does not lead to pinning:  For example, we can implement a lock like this

Virtual thread pinning can also occur due to native method calls or foreign function calls, and these cases are more challenging to address. There's no simple solution for these scenarios, so we need to manage the pinning individually.

Conclusion

The JDK 21 upgrade has brought tangible improvements to performance and maintainability at Halodoc, specifically through features like Generational ZGC with Soft Limit, Virtual Threads, and Sequenced Collections. However, their true potential can only be realised through careful consideration, thorough testing, and strategic integration into existing systems. At Halodoc, JDK 21 has proven to be a valuable upgrade, enhancing both performance and resource efficiency.

Generational ZGC with Soft Limit - This has optimized our memory usage by efficiently managing heap memory, leading to approximately 15-20% reduction in memory consumption across most services. Additionally, garbage collection pause times have decreased, enhancing overall system performance.

Virtual Threads - By integrating virtual threads to make parallel calls to external providers, we've achieved a 40-50% faster response time and better resource utilization. The ability to handle a large number of concurrent tasks with minimal overhead has significantly improved our application's performance.

Sequenced Collections - This feature has simplified code by offering a cleaner, more readable way to handle sequences of elements. Improving code maintainability ensures smoother development processes and easier long-term system management.

References

• Enhancing Java Application Performance: Transitioning from G1GC to ZGC at Halodoc
• ZGC Garbaage Collector
• Virtual Thread Pinning  Due to MySql connector

Join Us

Scalability, reliability and maintainability are the three pillars that govern what we build at Halodoc Tech. We are actively looking for engineers at all levels, and if solving complex problems with challenging requirements is your forte, please reach out to us with your resumé at careers.india@halodoc.com.

About Halodoc

Halodoc is the number 1 Healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke. We connect 20,000+ doctors with patients in need through our Tele-consultation service. We partner with 3500+ pharmacies in 100+ cities to bring medicine to your doorstep. We've also partnered with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allow patients to book a doctor appointment inside our application. We are extremely fortunate to be trusted by our investors, such as the Bill & Melinda Gates Foundation, Singtel, UOB Ventures, Allianz, GoJek, Astra, Temasek, and many more. We recently closed our Series D round and In total have raised around USD$100+ million for our mission. Our team works tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia