Is there a way to access the `DagsterInstance` out...
# ask-community
Is there a way to access the
outside of a job/asset/sensor/etc. and without setting
? Alternatively: Is there a way to initialize a
with a static set of partitions (so that I don't need the instance at all)?
dagster bot responded by community 1
My usecase: When testing my assets locally with
dagster dev
I would like to ensure a
is always initialized with at least one partition so that I actually have something to materialize. I want dagster to be ephemeral (so don't want to set DAGSTER_HOME) and would prefer not to create a
or similar because I just need to add the partition as a one-off (there's nothing to sense -- don't need it to run on a 30s interval).
If I use
I get an exception saying DAGSTER_HOME isn't set. If I use
I believe I'm getting a separate dagsterinstance from the one that was created by
dagster dev
? My hacky workaround for now is:
Copy code
# Hacky way to get the current dagster instance without setting DAGSTER_HOME
tmp_dirs = [d for d in os.listdir('.') if d.startswith('tmp')]
latest_dir = max(tmp_dirs, key=lambda d: os.path.getmtime(d))
instance = DagsterInstance.from_config(latest_dir)
I think you could just insert this line:
Copy code
instance.add_dynamic_partitions(name, keys)
in the same script which starts Dagster for you. I haven't tested this but it should work, you just need to get the correct