Serialization

The platform has support for a number of formats:

Serialization	Has Schema	Performance	Description
Simple Binary Encoding	✔️	🟩🟩🟩	Simple Binary Encoding (SBE) is the native form of serialization on the platform.
Data Transfer Objects	✔️	🟩🟩⬜	Generated domain objects that are easier to use than raw SBE while remaining fully SBE-compatible.
Flexible Messaging	❌	🟩⬜⬜	Objects such as Java POJOs, Maps or other data types can be published without a predefined schema. Suited to rapid prototyping or applications where latency is not critical.
Custom Serialization	-	🟩🟩🟩	Custom serialization formats can be registered into the API.
Raw	-	🟩🟩🟩	Raw binary payloads are supported.

Simple Binary Encoding

Simple Binary Encoding is a high-performance binary encoding protocol designed for low-latency messaging. Messages have a compact binary representation and a predictable memory layout, with generated encoders and decoders that read and write fields directly from a buffer.

The performance benefits of SBE over other serialization formats, such as Protocol Buffers, are well established. However, those benefits come with trade-offs:

• Developer productivity - SBE is lower-level than typical object serialization and requires familiarity with generated encoders, decoders and buffers.
• Idiosyncratic behaviour - Streaming reads are performant but require linear access.
• Operational visibility - binary messages are not self-describing at the byte level, so debugging, logging, and data inspection require schema-aware tooling.

KeySquare provides features to help address these deficiencies:

• Data schemas are automatically published by applications and KeySquare applications like the Data Viewer do not need to be upgraded to support viewing new schemas.
• DTO generation provides a simple Java object model while preserving compatibility with the underlying SBE schema. DTOs hide some of the lower-level SBE details, including split encoder/decoder usage and repeating group handling.
• Flexible Messaging supports Objects, Java POJOs, Maps and dynamically structured payloads for cases where flexibility and rapid development matter more than latency.
• Custom Serialization provides an extension point for teams that need to integrate with existing formats such as Protocol Buffers, Avro, FlatBuffers, etc.
• Raw payload support remains available for advanced use cases where the application wants full control over the bytes on the wire.

info

KeySquare strongly recommend using DTOs for most cases because of the simplicity they provide over SBE. Use pure SBE for high volume use cases such as bridging market data from a gateway to the platform.

Canonical Models

KeySquare provides a canonical model that defines the business models required for Fixed Income. Applications can use these models to will allow:

• Interoperability with KeySquare eTrading Applications
• Interoperability with KeySquare Data Analytics

Defining SBE Models

Models can be defined using a SBE schema file. The client-model sample project shows an example of how this can be done. In particular, examine:

• pom.xml - this contains build targets to support code generation from schema.
• src/main/resources/sbe-model-client.xml - the schema file follows the following format:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<sbe:messageSchema xmlns:sbe="http://fixprotocol.io/2016/sbe" xmlns:xi="http://www.w3.org/2001/XInclude"
  package="<package>"
  id="<id>"
  version="<version>"
  semanticVersion="<semanticVersion>"
  description="<description>"
  byteOrder="littleEndian">

  ... type definitions ...

</sbe:messageSchema>

In the schema example above the following placeholders should be replaced as appropriate:

Tag	Description
<package>	The Java package name where generated encoders / decoders should be placed.
<id>	A unique numerical identifier for the schema. (Note: 900 - 905 are reserved for platform use)
<version>	The version of the schema.
<semanticVersion>	The friendly schema name which will be used for display purposes.
<description>	A description of the schema.

tip

Automatic Code Generation - Speed up the development workflow by using IDE integration: Every time you edit and save a schema file, a build target can be automatically triggered in common IDEs like IntelliJ and Eclipse to auto-generate SBE encoders / decoders and DTO classes.

IntelliJ setup click here for details.

Eclipse setup click here for details.

Data Transfer Objects

KeySquare provides a code generator that produces Data Transfer Objects (DTOs) from SBE schemas. DTOs expose a familiar Java object model while remaining fully interoperable with the underlying SBE messages, allowing developers to work with schema-backed data without using raw encoders, decoders, buffers, or repeating groups directly.

DTOs are a good fit when developer productivity matters more than the lowest possible latency. They are also useful for fast prototyping: teams can build and validate business logic against the DTO API, then migrate performance-sensitive code paths to pure SBE once the model and business rules have stabilised.

DTOs and raw SBE both start from the same schema and remain interoperable with the same SBE message format:

The client-model sample project shows a working example of how DTOs can be automatically generated using an SBE schema.

See the Automatic Code Generation tip in the SBE section above which also applies to DTOs.

Flexible Messaging

Flexible Messaging allows applications to publish ordinary Object such as Java POJOs, Maps and similar data structures without first defining an SBE schema. The API encodes / decodes these and uses the Flexible message type, which supplies the platform header information needed to route the data.

This is useful when the shape of the data is still changing or when teams want to prototype quickly. It also supports workflows where schema flexibility matters such as Excel-driven data entry.

Flexible Messaging uses runtime payload serialization rather than a generated schema codec, so it prioritises convenience and schema flexibility over low-latency performance.

Custom Serialization

Custom Serialization allows teams to register their own serialization format with the API. This is useful when an application already has an established wire format, or when it needs to interoperate with systems that use formats such as Protocol Buffers, Avro, FlatBuffers or a proprietary binary encoding.

This option keeps the platform open to non-SBE data while still allowing applications to publish and subscribe through the KeySquare messaging APIs. It is best suited to integration boundaries or specialised performance requirements where the default SBE, DTO, and Flexible Messaging options are not the right fit.

When using Custom Serialization the application team owns the serializer, deserializer and schema compatibility rules for that format.

Serializers can work directly with transport buffers to avoid unnecessary intermediate copies. This is useful for low-allocation formats, wrapping existing binary payloads, or reusing codec-owned buffers on hot paths. Each custom codec can choose its own payload model, e.g. retainable objects, reusable flyweights or direct views over received bytes. This lets teams balance ease of use against latency and allocation requirements.

Raw

Raw serialization allows applications to publish and subscribe to binary payloads directly. The platform carries the bytes without interpreting the payload, which gives the application complete control over the message format.

This is appropriate for advanced use cases such as bridging legacy feeds, carrying already-encoded messages, or integrating with a specialised protocol where the application wants to manage the wire representation itself.

The trade-off is visibility and convenience. Raw payloads are opaque to the platform, so consumers must already know how to decode them and developers do not get schema-aware inspection, generated accessors or automatic object mapping.