Does dagster have anything similar to Databricks Auto Loader S3 ingestion, using file notifications instead of ticks that continually re-list the same prefix?
d
daniel
07/29/2022, 3:26 AM
Hey Matt - right now Dagster sensors only work by polling. I could imagine using something using file notifications and then launching runs via the dagster graphql api, but there isn't a built-in feature that does not.