JDK 21 Virtual Thread, lightweight threading

Project Loom is now part of JDK 21. Starting JDK 21, we can use what are called Virtual Threads. In this blog we will talk about virtual threads, what they are, when to use them and what advantage these give us.

What are O/S threads?

Threads in Operating Systems are the basic units that can execute code. They are used for concurrently running multiple tasks and are normally spawned from what are called processes. Each process can in turn manage one or more threads, however they share the same set of resources that belong to the process. When we say resources it can mean file handles, memory, register and flag values etc.

Operating System kernels manage threads. A task scheduler manages the processes and threads, and each thread is given a slice of time to complete a task.

Before starting on more details, let’s talk about user space and kernel space. Kernel is the core of an operating system. Any modern operating system segregates the total available resource to be segregated between what is available for the kernel to use and what can be used by the user. Anything running on user space cannot access resources allocated to kernel space directly. This helps preventing malicious programs from running codes that can compromise the integrity of the system.

Kernel manages its memory and processes, and all hardware interfaces that it needs will normally be accessed using specialized softwares called device drivers.

Any user level application will have to submit a request (program) for executing it’s own code. Application requests thus submitted to the kernel will be executed through processes and threads (for concurrency). Kernel will maintain a process table and provide a time slice to each process to execute. There are various algorithms how this time slice is managed by the kernel. That covers on a very high level how operating system manages kernel spaces.

With that 50,000 feet overview in the back of our mind, let’s start talking about Java Platform threads vs Virtual threads, and when to use each of them.

Platform Threads

A platform thread, or the threads that we were habituated to in Java, were made as wrappers over the operating system thread. That means they were always limited to the total number of threads that the operating system exposed.

Platform threads need a lot of resources to be managed by the operating system. Also, when a thread is blocked, the corresponding OS thread is also unavailable.

Let’s see an example of platform thread.

public void platformThread(final int count, final Duration sleepTime) {

        Thread thread[] = new Thread[count];
        // Create all threads
        System.out.println("Starting Platform task..");
        for (int i=0; i<count; i++) {
            thread[i] = Thread.ofPlatform().start(() -> {
                try {
                    Thread.sleep(sleepTime);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            });
        }

        // Wait for them
        for (int i=0; i<count; i++) {
            try {
                thread[i].join();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        System.out.println("Task done...");
    }

This is an example of creating platform threads in Java. This is a basic example that does not use any kind of pooling. It starts ‘count’ number of threads and waits for them to complete before returning. This uses the builder pattern implementation for calling threads.

Virtual Threads

Virtual threads are not directly tied to OS threads. They are managed by JVM and even though they eventually use OS threads to execute tasks, but the time slicing for each of these threads is managed by JVM. That means when one of these threads is waiting for I/O operation, JVM can pause that thread and let a different thread start its processing. That way the same number of OS threads can be used more efficiently.

The implementation for virtual threads, following the same builder pattern as above, becomes very simple.

public void virtualThread(final int count, final Duration sleepTime) {

        Thread thread[] = new Thread[count];
        // Create all threads
        System.out.println("Starting Virtual task..");
        for (int i=0; i<count; i++) {
            thread[i] = Thread.ofVirtual().start(() -> {
                try {
                    Thread.sleep(sleepTime);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            });
        }

        // Wait for them
        for (int i=0; i<count; i++) {
            try {
                thread[i].join();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        System.out.println("Task done...");
    }

The only difference in this case is that we are creating threads using Thread.ofVirtual() instead of Thread.ofPlatform().

So, when do we use virtual threads?

When to use?

Since thread switching by JVM can only be worthwhile when the threads go into wait, examples being, waiting for I/O, waiting for a lock to release etc, those are the tasks where virtual threads would be useful. However, if we have CPU intensive task, it would probably make more sense to continue using platform thread. We will talk about an exception when we talk about the intent for creating virtual threads.

Since threading is very expensive operation, normally in traditional Java programming we will use a pool of threads. Let’s put the features of platform vs. virtual threads side by side to compare.

Why Virtual Threads?

The most talked about aspect of virtual threads is performance in terms of speed. A platform thread may take 10 seconds to execute something that a virtual thread may execute in a second. However, that is more of a side effect of the design. Java threads were very inefficient when it came to memory management. Project Loom targeted creation of more efficient threading models for Java. As a result, thread management was moved to JVM instead to O/S thus making them more efficient. Let’s consider the following example. We will try to call the above functions with count: 10,000 and sleepTime: 10. Let’s see what happens with Platform thread.

Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
	at java.base/java.lang.Thread.start0(Native Method)
	at java.base/java.lang.Thread.start(Thread.java:1526)
	at java.base/java.lang.ThreadBuilders$PlatformThreadBuilder.start(ThreadBuilders.java:195)

Oops, we crashed with an OOM error. That’s a little too many threads for JDK to create. Let’s see if we can create the same number of Virtual threads.

Starting Virtual task..
Task done...
Time Elapsed: 10115

That ran fine. That is because these threads are much lighter weight on resources and as such a lot more can be created without any issues.

So, that shows us one more exception for platform thread as well. If we need to create a large number of threads, even if they are CPU bound, it is better to just go with virtual threads.

Conclusion

In this blog we discussed about Virtual threads. It is one of the game changing features that has been introduced and I see a lot of use for this in future. Hope you found this blog useful. Ciao for now!