debeziumFlow

The debeziumFlow function creates a Kotlin Flow that consumes records from a Debezium engine and emits them as CommittableRecords.

Each CommittableRecord carries the original record and a CommittableOffset that allows the caller to commit the record's offset to the Debezium engine once it has been successfully processed downstream.

Once the CommittableRecord is processed, the markProcessed function must be called.

The capacity of the internal buffer that stores the unconsumed records is defined by maxRecordsInFlight.

If the buffer fills up, the flow will start to suspend producers until more capacity becomes available again.

Example usage:

val debeziumProperties: Properties = TODO("provide the debezium properties here")

val flow = debeziumFlow { create(Json::class.java).using(debeziumProperties) }

flow.collect { record ->
    println(record)
    record.markProcessed()
}