Joseph Marcus
07/18/2023, 12:14 PMcontext.dagster_run.asset_selection
correctly returns the corresponding assets. However, when the same assets are materialized by a scheduled job, context.dagster_run.asset_selection
unexpectedly returns None instead of the asset keys.
Our understanding is that context.dagster_run.asset_selection
should behave consistently, regardless of whether the assets are materialized manually or by a scheduled job. We have ensured that there are no differences in code or environment configurations between the manual and scheduled runs that could account for this disparity.
Is this difference in behavior expected? If not, could someone please guide us towards potential solutions?rex
07/18/2023, 6:56 PMowen
07/18/2023, 7:10 PMasset_selection
is not a public property of the DagsterRun
object, it's more of an internal implementation detail. There's not a hard-set reason for this difference in behavior, but the basic idea is that the asset_selection generally represents a subselection of the total set of assets in a job (so if you're materializing all assets in a job, there is no subselection, it's just all the assets).
What's your usecase for accessing this? There's likely another way of doing this that would be more consistentJoseph Marcus
07/19/2023, 10:59 AMDane Linssen
07/19/2023, 11:19 AMrex
07/19/2023, 12:03 PMDane Linssen
07/19/2023, 12:04 PMJoseph Marcus
07/19/2023, 12:06 PMDane Linssen
07/26/2023, 9:11 AMrun_results.json
, perfect suggestion! We have it implemented locally.
In our cluster though, our user code deployment spins up a new ephemeral pod for every run. How can we capture the run_results.json
from this pod before it gets spun down?