https://dagster.io/ logo
#ask-community
Title
# ask-community
p

Peter Law

09/01/2023, 2:15 PM
If I have a large pipeline DAG is there a way I can locally develop on a particular asset using input assets from production?
dagster bot answered by content 1
For example my pipeline contains an initial step of data extraction from another system. This step takes a long time whether run locally or otherwise, so I'd like to not have to repeat it locally.
Ideally I'd be able to download those assets from production and then run the rest of the pipeline locally to develop the pipeline tasks.
Is this something which Dagster supports?
(assume the pipeline has already anonymised etc. the data before the step I'm talking about)
j

jamie

09/01/2023, 3:27 PM
hey @Peter Law there’s an experimental feature that can help with this. I wrote a github discussion about it here https://github.com/dagster-io/dagster/discussions/16145
p

Peter Law

09/01/2023, 3:43 PM
Thanks! I'll have a look at that