Join Slack
Powered by
Let's say I want to download multiple zip files, t...
# ask-community
e
Etienne
06/19/2023, 11:17 PM
Let's say I want to download multiple zip files, then extract them and each of them have a different amount of files. each file inside the zip is an excel file that I would want to parse using pandas. what would be the best way to do that in a job?
y
yuhan
06/20/2023, 11:04 PM
there’s two ways to do it. 1. if you’re mainly using assets, you can model it as dynamically partitioned assets:
https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#dynamically-partitioned-assets
2. if you’re mainly using ops, you can model it as dynamic graph:
https://docs.dagster.io/concepts/ops-jobs-graphs/dynamic-graphs#a-dynamic-job
🌈 1
146
Views
Open in Slack
Previous
Next