Hello, I have 2 questions: - What are differences ...
# ask-community
r
Hello, I have 2 questions: • What are differences between each available Dagster storage options? I've noticed SQLite option does not allow run retry from
dagster.yml
. What are other limitation, pros & cons? Will other storage option be available in the future? • I have noticed that Step duration graphs seem incomplete (see uploaded image). Duration seems properly rendered when step is completed successfully at first try. If a step is regenerated with a retry run, its completion duration is not displayed and cause missing data from Step and Run duration. Is it normal or should we expect a correction for this ? Thank you
y
1. it’d be just SQLite that hits the special case (Because for SQLite’s run storage is run-sharded but SQLite’s event storage isn’t run-aware, it doesn’t support run retry which operates on event storage). Other storage options should be no limitation. Is there a specific storage option you’re looking for?
r
Hello Yuhan, Thank you for the clarification. The only other option I had in mind was a Azure Blob in order to test portable/unexpensive deployment relying only on a Azure Storage account to work. Any idea about my second question ?
y
hey sorry, forgot to mention. i’ve flagged your second question to our UI team. will keep you posted.
regarding the storage, to clarify, the dagster storage options you listed are used for storing dagster’s metadata which do require to be structured. in other words, when we get to azure support, it won’t be using azure blob, instead, it might be Azure Database. but in your testing case, i’d recommend deploying dagster to a single node docker on your azure container-based system. here’s a guide: https://docs.dagster.io/deployment/guides/docker