Add EchoServer example demonstrating VirtualMultithreadIoEventLoopGroup usage

franz1981 · franz1981 · commit f4d52e3dceeb · 2025-11-18T16:00:33.000+01:00
diff --git a/README.md b/README.md
@@ -1,76 +1,98 @@
 # Netty VirtualThread Scheduler
 
 ## Introduction
-This project provides an advanced integration between Java Virtual Threads (Project Loom) and Netty's event loop, enabling seamless execution of blocking operations within the Netty event loop itself. Unlike standard approaches, which require offloading blocking tasks to external thread pools (incurring multiple thread hand-offs), this scheduler allows blocking code to run directly on the event loop's carrier thread, leveraging the virtual thread execution model.
+This project provides an integration between Java Virtual Threads (Project Loom) and Netty's event loop, enabling low-overhead execution of blocking or CPU-bound work started from the Netty event loop without the usual extra thread hand-offs.
 
-## Motivation
-In traditional Netty applications, blocking operations (e.g., JDBC, file I/O) must not run on the event loop. The usual workaround is:
-1. Offload the blocking task to an external thread pool (often using the default virtual thread scheduler).
-2. Once complete, hand control back to the Netty event loop to continue processing (e.g., send a response).
+At a high level:
+- Each Netty event loop is backed by a virtual thread (the "event loop virtual thread").
+- Each such virtual thread is executed on a dedicated carrier platform thread.
+- Virtual threads created from the event-loop-specific ThreadFactory returned by the group will be associated with the same EventLoopScheduler and, when possible, will run on the same carrier platform thread as the event loop.
 
-This process involves at least two thread hand-offs, which not only increases latency and complexity, but also wastes CPU cycles due to waking up both the external thread pool and the event loop again. Additionally, moving data between threads harms cache locality, reducing cache friendliness and overall performance.
+This allows code that must block (for example, blocking I/O or synchronous library calls) to be executed without moving work between unrelated threads, reducing wake-ups and improving cache locality compared to offloading to an external thread pool.
 
-## Technical Approach
-- The Netty event loop runs as a special, long-lived virtual thread.
-- Blocking operations issued from the event loop are executed as continuations, scheduled to run on the same platform thread (the "carrier")—unless work-stealing occurs (not implemented at the moment).
-- Any virtual thread created from the event loop will, by default, run on the same carrier platform thread (again, unless work-stealing is introduced).
-- This enables blocking libraries to be used transparently, without extra thread pools or hand-offs.
+## Key behavior (user-facing)
+- Create a `VirtualMultithreadIoEventLoopGroup` like any other Netty `EventLoopGroup`. It behaves like a `MultiThreadIoEventLoopGroup` from the Netty API, but the event loops are driven by virtual threads and coordinated with carrier platform threads by a per-event-loop `EventLoopScheduler`.
+- Call `group.vThreadFactory()` to obtain a `ThreadFactory` that creates virtual threads tied to an `EventLoopScheduler` of the group. When those virtual threads block, the scheduler parks them and resumes them later using the carrier thread.
+- Virtual threads created via the group's `vThreadFactory()` will attempt to inherit the scheduler and run with low-overhead handoffs back to the event loop when continuing work.
+- Virtual threads created with `Thread.ofVirtual().factory()` (the JVM default factory) do NOT automatically inherit the group's scheduler.
+- The library requires a custom global virtual thread scheduler implementation to be installed: set `-Djdk.virtualThreadScheduler.implClass=io.netty.loom.NettyScheduler` (or use the convenience helpers in the code/tests that set up the scheduler).
+- Blocking I/O support that relies on per-carrier pollers currently depends on the JVM poller mode (the code checks `jdk.pollerMode==3`). See notes below.
 
-## Comparison Table
+## When to use this
+- You have code that must perform blocking operations from a Netty handler and you want to avoid an extra thread hand-off.
+- You want fewer CPU wake-ups and better cache locality by keeping related work on the same carrier platform thread.
 
-| Aspect                | Standard Netty + Loom (Default)         | Netty with VirtualThread Scheduler (This Project) |
-|-----------------------|-----------------------------------------|--------------------------------------------------|
-| Blocking Operation    | Offloaded to external thread pool       | Runs as continuation on event loop carrier        |
-| Thread Hand-offs      | 2+                                      | 0                                                |
-| Cache Friendliness    | Poor (data moves between threads)        | High (data stays on carrier thread)               |
-| CPU Wakeups           | More (wakes both pools)                  | Fewer (single carrier thread)                     |
+Caveats:
+- This project leverages experimental JVM features (Project Loom) and assumes recent Java versions (Java 21+ recommended).
+- You must install the Netty-specific scheduler (see above) for `VirtualMultithreadIoEventLoopGroup` to be usable.
 
-## Architecture Diagrams
+## Usage example (simple)
 
+See the runnable example and step-by-step instructions in the example module:
+
+- example-echo/README.md
+
+(That README contains the minimal build and run commands, including how to start the example server and run a quick curl smoke test.)
+
+## About the Loom build used
+
+This work targets Project Loom features and was developed and tested against very recent OpenJDK / Loom builds. If you don't want to build OpenJDK yourself, a convenient set of prebuilt Loom-enabled JDK images is available from Shipilev's builds:
+
+- https://builds.shipilev.net/openjdk-jdk-loom/
+
+Follow that site to download a suitable JDK image for your platform and point `JAVA_HOME` to the JDK image before running the project. Example:
+
+```sh
+export JAVA_HOME=/path/to/loom/build/linux-x86_64-server-release/jdk/
 ```
-Standard Netty + Loom (Default):
-
-┌──────────────┐   offload   ┌────────────────────────────┐   callback   ┌──────────────┐
-│  EventLoop   │───────────▶│ Virtual Thread (Scheduler) │────────────▶│  EventLoop   │
-│ (OS Thread)  │             │ (External Thread Pool)     │              │ (OS Thread)  │
-└──────────────┘             └────────────────────────────┘              └──────────────┘
-
-Netty with VirtualThread Scheduler:
-
-┌────────────────────────────────────────────┐
-│         Platform Thread (Carrier)          │
-│  ┌──────────────────────────────────────┐  │
-│  │   EventLoop (Virtual Thread)         │  │
-│  │   (runs on carrier platform thread)  │  │
-│  │   ────────────────────────────────   │  │
-│  │   Blocking Operation                 │  │
-│  │   (Continuation, same platform)      │  │
-│  └──────────────────────────────────────┘  │
-└────────────────────────────────────────────┘
+
+If you prefer to build from the OpenJDK `loom` repository yourself, the upstream source is:
+
+- https://github.com/openjdk/loom
+
+If you have a local build of the latest Loom-enabled OpenJDK, point `JAVA_HOME` to that build before running tests and benchmarks. Example:
+
+```sh
+export JAVA_HOME=/path/to/your/loom/build/linux-x86_64-server-release/jdk/
+mvn clean install
 ```
 
-## Build and Run
+## Integration tips and runtime flags
+- Install the Netty scheduler as the JVM virtual-thread scheduler with:
+  -Djdk.virtualThreadScheduler.implClass=io.netty.loom.NettyScheduler
+
+- Some blocking I/O integrations rely on per-carrier pollers. The code checks `jdk.pollerMode` and expects value `3` for per-carrier pollers. This can be controlled via JVM flags or defaults depending on your JVM version.
+
+- Use `group.vThreadFactory()` from inside Netty handlers if you want the spawned virtual thread to be associated with the same event loop scheduler as the handler's event loop.
 
+- If you need the virtual thread to inherit the scheduler when forking tasks via StructuredTaskScope, pass the group's `vThreadFactory()` to the scope's `withThreadFactory(...)`.
+
+## Build and Run
 This project uses Maven for build and dependency management.
 
-1. **Build the project:**
-   ```sh
-   mvn clean install
-   ```
-2. **Run Benchmarks:**
-   ```sh
-   cd benchmarks
-   mvn clean install
-   java -jar target/benchmarks.jar
-   ```
+1. Build the project:
+
+```sh
+mvn clean install
+```
+
+2. Run the benchmarks (optional):
+
+```sh
+cd benchmarks
+mvn clean install
+java -jar target/benchmarks.jar
+```
 
 ## Prerequisites
 - Java 21 or newer (with Loom support)
 - Maven 3.6+
+- To use the `VirtualMultithreadIoEventLoopGroup` set the JVM property to install the Netty scheduler:
+  -Djdk.virtualThreadScheduler.implClass=io.netty.loom.NettyScheduler
 
 ## References
-- [Project Loom (OpenJDK)](https://openjdk.org/projects/loom/)
-- [Netty Project](https://netty.io/)
+- Project Loom (OpenJDK): https://openjdk.org/projects/loom/
+- Netty Project: https://netty.io/
 
 ---
 For more details, see the source code and benchmark results in the respective modules.
diff --git a/core/src/main/java/io/netty/loom/NettyScheduler.java b/core/src/main/java/io/netty/loom/NettyScheduler.java
@@ -7,21 +7,25 @@
 /**
  * Global Netty scheduler proxy for virtual threads.
  *
- * <p>Inheritance rule (exact): a newly started virtual thread inherits the
+ * <p>
+ * Inheritance rule (exact): a newly started virtual thread inherits the
  * caller's {@code EventLoopScheduler} only when both conditions are true:
  * <ol>
- *   <li>{@code jdk.pollerMode} is {@code 3} (per-carrier pollers); and</li>
- *   <li>the thread performing the start/poller I/O is itself running under an
- *       {@code EventLoopScheduler} (i.e. {@code EventLoopScheduler.currentThreadSchedulerContext().scheduler()}
- *       returns a non-null {@code SharedRef}).</li>
+ * <li>{@code jdk.pollerMode} is {@code 3} (per-carrier pollers); and</li>
+ * <li>the thread performing the start/poller I/O is itself running under an
+ * {@code EventLoopScheduler} (i.e.
+ * {@code EventLoopScheduler.currentThreadSchedulerContext().scheduler()}
+ * returns a non-null {@code SharedRef}).</li>
  * </ol>
  *
- * <p>The current implementation only attempts scheduler inheritance for
- * poller-created virtual threads (recognized by the {@code "-Read-Poller"}
- * name suffix). If either condition above is not met (or the thread kind is
+ * <p>
+ * The current implementation only attempts scheduler inheritance for
+ * poller-created virtual threads (recognized by the {@code "-Read-Poller"} name
+ * suffix). If either condition above is not met (or the thread kind is
  * unrecognized) the virtual thread falls back to the default JDK scheduler.
  *
- * <p>This class is a proxy/dispatcher and does not implement a standalone
+ * <p>
+ * This class is a proxy/dispatcher and does not implement a standalone
  * scheduling policy. See {@link EventLoopScheduler} for details about scheduler
  * attachment and execution.
  */
diff --git a/example-echo/README.md b/example-echo/README.md
@@ -0,0 +1,56 @@
+# example-echo
+
+A tiny HTTP example that shows how to use VirtualMultithreadIoEventLoopGroup and its
+`vThreadFactory()` to spawn virtual threads from Netty handlers.
+
+Prerequisites
+- A recent Loom-enabled JDK (set `JAVA_HOME` to it).
+- Maven (for build and runtime classpath).
+- curl for the smoke test; jbang if you want to run the optional wrk/Hyperfoil test.
+
+How to build and run (minimal)
+
+1) Build (from repository root):
+
+```bash
+# build quickly (skip tests)
+mvn -DskipTests package
+```
+
+2) Start the server (adjust `JAVA_HOME`):
+
+```bash
+# compute runtime classpath for example-echo
+CP_LINE=$(mvn -q -pl example-echo dependency:build-classpath -DincludeScope=runtime -DskipTests | tail -n1)
+CP="core/target/netty-virtualthread-core-1.0-SNAPSHOT.jar:example-echo/target/example-echo-1.0-SNAPSHOT.jar:${CP_LINE}"
+
+# start server in background (use your Loom JDK)
+export JAVA_HOME=/path/to/loom/build/linux-x86_64-server-release/jdk/
+"$JAVA_HOME/bin/java" --enable-preview \
+  -Djdk.virtualThreadScheduler.implClass=io.netty.loom.NettyScheduler \
+  -Djdk.pollerMode=3 \
+  --add-opens=java.base/java.lang=ALL-UNNAMED \
+  -cp "$CP" io.netty.loom.example.EchoServer &
+
+# note the PID in $!
+```
+
+Quick smoke & load tests
+
+- Simple curl smoke test:
+
+```bash
+curl -v http://localhost:8080/
+```
+
+- Short wrk/Hyperfoil test via jbang (uses the Hyperfoil catalog):
+
+```bash
+# short 5s test
+jbang wrk@hyperfoil -t1 -c10 -d5s -R10 --latency http://localhost:8080/
+```
+
+Notes
+- `-Djdk.pollerMode=3` enables per-carrier pollers (useful for blocking I/O tests). Remove or change if you don't need it.
+- Use a Loom-enabled JDK and set `JAVA_HOME` accordingly.
+- If anything fails, paste the exact output here and I'll help debug.
diff --git a/example-echo/pom.xml b/example-echo/pom.xml
@@ -0,0 +1,75 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+
+    <parent>
+        <groupId>io.netty.loom</groupId>
+        <artifactId>netty-virtualthread-parent</artifactId>
+        <version>1.0-SNAPSHOT</version>
+    </parent>
+
+    <artifactId>example-echo</artifactId>
+    <packaging>jar</packaging>
+
+    <dependencies>
+        <dependency>
+            <groupId>io.netty.loom</groupId>
+            <artifactId>netty-virtualthread-core</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>io.netty</groupId>
+            <artifactId>netty-all</artifactId>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-compiler-plugin</artifactId>
+            </plugin>
+            <plugin>
+                <groupId>org.codehaus.mojo</groupId>
+                <artifactId>exec-maven-plugin</artifactId>
+                <version>3.1.0</version>
+                <configuration>
+                    <mainClass>io.netty.loom.example.EchoServer</mainClass>
+                    <arguments/>
+                    <systemProperties>
+                        <systemProperty>
+                            <key>jdk.virtualThreadScheduler.implClass</key>
+                            <value>io.netty.loom.NettyScheduler</value>
+                        </systemProperty>
+                    </systemProperties>
+                    <cleanupDaemonThreads>false</cleanupDaemonThreads>
+                </configuration>
+            </plugin>
+
+            <!-- Shade plugin to produce an executable fat jar -->
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-shade-plugin</artifactId>
+                <version>3.5.0</version>
+                <executions>
+                    <execution>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>shade</goal>
+                        </goals>
+                        <configuration>
+                            <createDependencyReducedPom>false</createDependencyReducedPom>
+                            <transformers>
+                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+                                    <mainClass>io.netty.loom.example.EchoServer</mainClass>
+                                </transformer>
+                            </transformers>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
+
+        </plugins>
+    </build>
+</project>
diff --git a/example-echo/src/main/java/io/netty/loom/example/EchoServer.java b/example-echo/src/main/java/io/netty/loom/example/EchoServer.java
@@ -0,0 +1,74 @@
+package io.netty.loom.example;
+
+import io.netty.bootstrap.ServerBootstrap;
+import io.netty.channel.Channel;
+import io.netty.channel.ChannelInboundHandlerAdapter;
+import io.netty.channel.ChannelInitializer;
+import io.netty.channel.nio.NioIoHandler;
+import io.netty.channel.socket.SocketChannel;
+import io.netty.channel.socket.nio.NioServerSocketChannel;
+import io.netty.handler.codec.http.DefaultFullHttpResponse;
+import io.netty.handler.codec.http.DefaultHttpRequest;
+import io.netty.handler.codec.http.HttpHeaderNames;
+import io.netty.handler.codec.http.HttpHeaderValues;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.netty.handler.codec.http.HttpServerCodec;
+import io.netty.handler.codec.http.HttpVersion;
+import io.netty.util.CharsetUtil;
+import io.netty.util.ReferenceCountUtil;
+import io.netty.loom.VirtualMultithreadIoEventLoopGroup;
+
+import java.util.concurrent.ThreadFactory;
+
+public class EchoServer {
+
+	public static void main(String[] args) throws Exception {
+		// simple echo server on port 8080
+		try (var group = new VirtualMultithreadIoEventLoopGroup(1, NioIoHandler.newFactory())) {
+			var bootstrap = new ServerBootstrap().group(group).channel(NioServerSocketChannel.class)
+					.childHandler(new ChannelInitializer<SocketChannel>() {
+						@Override
+						protected void initChannel(SocketChannel ch) {
+							ch.pipeline().addLast(new HttpServerCodec());
+							ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
+								@Override
+								public void channelRead(io.netty.channel.ChannelHandlerContext ctx, Object msg) {
+									if (msg instanceof DefaultHttpRequest) {
+										ThreadFactory vtf = group.vThreadFactory();
+										vtf.newThread(() -> {
+											try {
+												// blocking work placeholder
+												Thread.sleep(50);
+												var content = ctx.alloc().directBuffer("HELLO".length());
+												content.writeCharSequence("HELLO", CharsetUtil.US_ASCII);
+												var response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,
+														HttpResponseStatus.OK, content);
+												response.headers()
+														.set(HttpHeaderNames.CONTENT_TYPE, HttpHeaderValues.TEXT_PLAIN)
+														.set(HttpHeaderNames.CONTENT_LENGTH, content.readableBytes())
+														.set(HttpHeaderNames.CONNECTION, HttpHeaderValues.CLOSE);
+												// write the response; the inbound request is not needed by the virtual
+												// thread, so it is safe to release it immediately after scheduling.
+												ctx.writeAndFlush(response);
+											} catch (InterruptedException e) {
+												Thread.currentThread().interrupt();
+											}
+										}).start();
+										// release the inbound message immediately: the virtual thread does not use it
+										ReferenceCountUtil.release(msg);
+									} else {
+										ReferenceCountUtil.release(msg);
+									}
+								}
+							});
+						}
+					});
+
+			Channel ch = bootstrap.bind(8080).sync().channel();
+			System.out.println("Echo server started on http://localhost:8080/");
+
+			// keep server running until the channel is closed (by external shutdown)
+			ch.closeFuture().sync();
+		}
+	}
+}
diff --git a/pom.xml b/pom.xml
