Threading Model
Applications that connect to the platform receive data on the ks thread created by the KeySession. Applications may perform business logic on that thread if they are able to keep up with the rate of incoming data. The thread may also be used to publish data back to the platform.
Post Processing
The API works on a duty cycle where it receives messages off the wire, sends these to the application and then calls KeySessionListener.postProcess(boolean hasWork).
This mechanism can be used to bulk process messages that have been seen in that cycle.
The hasWork flag is true when at least one message was received during the cycle.
Session Thread
Message processors, subscription listeners, unsubscription listeners and postProcess(...) are called on the session thread.
Keep session-thread callbacks short. If work is slow or blocking, hand it to another thread and keep only the platform-facing state changes on the session thread.
The ks thread can be core pinned to an isolated CPU for low latency applications. See Session for eventLimit, idleStrategy, cpus and concurrencyMode.
Why Keep Work on the Session Thread
Low latency applications are normally easier to make predictable when the platform-facing business logic stays on the session thread.
This approach:
- avoids thread contention and synchronization overhead
- keeps callback ordering deterministic
- reduces context switching
- improves CPU cache locality
- makes race conditions and deadlocks less likely
This does not mean every application must do all work on the session thread. Slow, blocking or high allocation work should be handed to another thread. The session thread should keep the state needed to receive, route, process and publish data.
Idle Strategy
The idle strategy controls what the session thread does when there is no immediate work available.
Busy strategies can reduce latency because the thread keeps polling for work. They also consume a CPU core. Sleeping strategies use fewer CPU resources, but increase the time taken to react to new data.
| Transport | Idle Strategy | Pinned | 99.999% | Over Base |
|---|---|---|---|---|
IPC_UDP | org.agrona.concurrent.NoOpIdleStrategy | Yes | 11.5us | 1.00x |
IPC_UDP | org.agrona.concurrent.SleepingMillisIdleStrategy | Yes | 1.11ms | 96.52x |
The second row is much slower because the session thread sleeps between polls. The trade-off is CPU usage: a busy idle strategy is expected to show high CPU usage because it is actively polling for work.
Core Pinning
For low latency applications, pin the session thread to an isolated CPU core when the host has been configured for that style of deployment.
Core pinning is configured on the session context:
context.setCpus(new int[] { 8 });
where 8 is the isolated CPU core to use.
See Platform Best Practices for isolating cores at the platform level.