Can Dagster-cloud be used for Near-Real-Time applications? Does it support sub-second latency if we can trigger materializations with Sensors? My use case is to use Dagster for a ML feature store. In a feature store, there is an offline storage (which dagster I believe can support) and an online store, where if a change happen (changes happen in milliseconds) it updates an asset and other assets depending on it.

Hey Anmar, we don’t have a great story for that kind of low latency data transfer yet. It’s something we’re thinking about, given how regularly it comes up but the best recommendation we have for these kinds of problems is more like a schedule that runs extremely regularly to capture small batches of data, but not at the sub second level. Would love to chat with you about this use case if you have time in the next few days

Hey Jarred, yeah I have time. I think it would pretty cool if dagster can be a feature store