https://dagster.io/ logo
#ask-community
Title
# ask-community
m

Matt Clarke

03/16/2023, 9:00 PM
When using
dagster dev
, is the temporary directory path available as an environment variable? I want to use a duck db database for local testing, but the database file is created in the project root so persists after exit the process
m

Matt Clarke

03/16/2023, 9:26 PM
I understand that setting $DAGSTER_HOME gives it a permanent location, but if you leave it unset then the output is redirected to a temp dir. That's good, but unless I've just done something stupid then the DAGSTER_HOME variable remains empty so you can't point at this for the outputs. I'm currently using
next(pathlib.Path('.').glob("tmp*")) / 'storage/temp.duckdb'
, which works but doesn't seem super robust
t

Tim Castillo

03/16/2023, 9:51 PM
Hi! What do you mean by:
the DAGSTER_HOME variable remains empty so you can't point at this for the outputs.
y

yuhan

03/16/2023, 9:53 PM
once you have the $DAGSTER_HOME set, it would first create an empty
dagster.yaml
and then you can configure the storage location there: https://docs.dagster.io/deployment/dagster-instance#dagster-storage
m

Matt Clarke

03/16/2023, 10:08 PM
Sorry, to clarify, I want the database to be ephemeral. When I run dagster dev, the temp directory being created is really useful, but any files I instantiate (like a new database file) need to be under that directory if I want them to be cleaned up when dagster dev exits
s

sri raghavan

03/17/2023, 10:16 AM
@Matt Clarke i think you could probably write a small shell script that uses
mktemp
and sets
DAGSTER_HOME
inline with running
dagster dev
?
m

Matt Clarke

03/17/2023, 10:22 AM
Ooh, that's a shout. Cheers!
👍🏾 1
18 Views