https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jon Erik Kemi Warghed

03/12/2023, 8:15 PM
We have about 100 time series we want to use, along with some new ones that might be created by analysts. How should we think about this, create 100 individual assets and all new ones needs to be defined in code as well sounds like the correct approach. But is there another way or methodology i am missing?
dagster bot responded by community 1
🤖 1
e

Expoze Dummy

03/13/2023, 2:02 AM
If how they are being extracted and outputted are just the same, you can create an asset factory. See this pattern here: https://docs.dagster.io/concepts/ops-jobs-graphs/ops#op-factory (just the same as op factory)
👍 1
And yes, they do need to be defined in the code. What you can do is have some config file, read that config file that specifies your asset locations, and create an asset through an asset factory
y

yuhan

03/14/2023, 12:14 AM
if they are the similar business logic but vary based on time, i’d also recommend trying out partitions, especially if it’s time based partitions: https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#partitioned-assets
👍 1
j

Jon Erik Kemi Warghed

03/14/2023, 7:12 PM
Great will take a deeper look, been looking at partioning as well so both seems like great options to evaluate.