Formalization of Timely Dataflow's Progress Tracking Protocol

Matthias Brun, Sára Decova, Andrea Lattuada and Dmitriy Traytel

13 April 2021

Abstract

Large-scale stream processing systems often follow the dataflow paradigm, which enforces a program structure that exposes a high degree of parallelism. The Timely Dataflow distributed system supports expressive cyclic dataflows for which it offers low-latency data- and pipeline-parallel stream processing. To achieve high expressiveness and performance, Timely Dataflow uses an intricate distributed protocol for tracking the computation’s progress. We formalize this progress tracking protocol and verify its safety. Our formalization is described in detail in our forthcoming ITP'21 paper.
BSD License

Depends On

Topics

Theories

Cite

×
@article{Progress_Tracking-AFP,
  author  = {Matthias Brun, Sára Decova, Andrea Lattuada and Dmitriy Traytel},
  title   = {Formalization of Timely Dataflow's Progress Tracking Protocol},
  journal = {Archive of Formal Proofs},
  month   = April,
  year    = 2021,
  note    = {\url{http://isa-afp.org/entries/Progress_Tracking.html},
            Formal proof development},
  ISSN    = {2150-914x},
}
Download

Download

× Download latest

Older releases: