https://dagster.io/ logo
n

nicocti

07/16/2019, 1:29 PM
Nice. For now we have very low latency requirements (10k messages / minutes), however I'd try to avoid having a big part of my computation taken by the generation of the graph. The input flow is just a json payload with storage path to one or more images and a custom DAG definition (that could actually be directly replace by Dagster configuration files however) for the processings to be performed.