Integration With Producer Code
The KPL runs in a separate process, and communicates with your parent user process using IPC. This architecture is sometimes called a microservice, and is chosen for two main reasons:
1) Your user process will not crash even if the KPL crashes
Your process could have tasks unrelated to Streams, and may be able to continue operation even if the KPL crashes. It is also possible for your parent user process to restart the KPL and recover to a fully working state (this functionality is in the official wrappers).
An example is a web server that sends metrics to Streams; the server can continue serving pages even if the Streams part has stopped working. Crashing the whole server because of a bug in the KPL would therefore cause an unnecessary outage.
2) Arbitrary clients can be supported
There are always customers who use languages other than the ones officially supported. These customers should also be able to use the KPL easily.
Recommended Usage Matrix
The following usage matrix enumerates the recommended settings for different users and advises you about whether and how you should use the KPL. Keep in mind that if aggregation is enabled, de-aggregation must also be used to extract your records on the consumer side.
|Producer side language||
|KCL Version||Checkpoint logic||Can you use the KPL?||Caveats|
|Anything but Java||*||*||*||No||N/A|
|Java||Java||Uses Java SDK directly||N/A||Yes||If aggregation is used, you have to use the provided de-aggregation library
|Java||Anything but Java||Uses SDK directly||N/A||Yes||Must disable aggregation.|
|Java||Java||1.3.x||N/A||Yes||Must disable aggregation|
|Java||Java||1.4.x||Calls checkpoint without any arguments||Yes||None|
|Java||Java||1.4.x||Calls checkpoint with an explicit sequence number||Yes||Either disable aggregation, or change the code to use extended sequence numbers for checkpointing|
|Java||Anything but Java||1.3.x + Multilanguage daemon + language-specific wrapper||N/A||Yes||Must disable aggregation|