https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Shane Keller

03/09/2024, 6:30 PM
Is it possible to run an asset job with either a dynamic upstream asset or a hardcoded asset config
s

Scout

03/09/2024, 6:30 PM
Yes, it is possible to run an asset job with either a dynamic upstream asset or a hardcoded asset config. The documentation provides guidance on how to configure assets and jobs, including the use of dynamic configurations. For dynamic upstream assets, you can use dynamic partitioning strategies to handle assets that change over time, such as assets that represent datasets that are updated on a schedule. The documentation on dynamic partitioning and the use of sensors to trigger jobs based on events can help in setting up such a system. For hardcoded asset configs, you can provide a dictionary of configurations when defining a job, which will be used as the job’s run config whenever the job is executed. This allows you to specify configurations that are static and known ahead of time. The documentation also mentions that the values provided in the hardcoded config will be viewable and editable in the Dagster UI, so it's important to be cautious with sensitive information. In both cases, the selection of assets to be materialized or observed when the job is run must be included in the assets that are passed to the assets argument of the Definitions object that the job is included on. If you are using dynamic configurations, you may also leverage environment variables to configure assets and jobs, as described in the documentation. Remember that if you are using dynamic configurations, you need to ensure that the dynamic aspects of your configuration are handled correctly, such as by using sensors or schedules to trigger jobs based on the presence of new data or other conditions.