Exactly Once, Large-Scale Stream Processing in Action

Notes from the FOSDEM 2017 presentation "Not Less, Not More: Exactly Once, Large-Scale Stream Processing in Action" (video).

 
On guarantees:

 
On processing guarantees vs output (aka delivery, end-to-end) guarantees:

 
On the types of 'exactly once processing' approaches:

  • fine grained
    • maintain a log for each operation
  • coarse grained
    • discrete execution (micro-batch)
    • long running synchronous
    • long running pipelined

 
On the semantics of the coarse-grained approach: