Andras Somi
01/18/2023, 11:57 AMobservable_source_asset
, but if I click Observe sources (either with selecting my new asset or not) I get this not very helpful error in Dagit. What’s __ASSET___JOB
?chris
01/18/2023, 7:21 PM__ASSET _JOB_
is an underlying job that houses all the assets in your deployment - but the user should never need to interact with it. paging @sean who i believe has worked more directly with this featuresean
01/18/2023, 10:04 PMAndras Somi
01/19/2023, 12:52 PM@observable_source_asset(
name="company",
key_prefix=["xxx"],
description="Main Company table",
io_manager_key="pandas_io_manager",
)
def company(context: OpExecutionContext):
engine: Engine = context.resources.db
with engine.connect() as con:
last_update: Result = con.execute("select max(updated) from xxx.company;")
return LogicalVersion(str(last_update.scalar()))
but it happens with a bare-bone asset, too, like this:
@observable_source_asset
def company(_):
return LogicalVersion("a")
sean
02/02/2023, 8:47 PMAndras Somi
02/02/2023, 9:02 PMsean
02/02/2023, 9:37 PMpandas_io_manager
-- does it have required config?What’s __ASSET_JOB ?This is an internal implementation detail that should not leak, so thank you for identifying this-- essentially
__ASSET_JOB
is a master “hidden” job that includes all assets. When you are selecting assets for materialization or observation from the global asset graph, this is the job that is used to execute them.Andras Somi
02/03/2023, 9:26 AMpandas_io_manager
takes config indeed. But even if I change it to the simplest possible version without any parameters and no logic inside the function I still get the error. So I guess it might be clashing with something else in my setup (perhaps with the asset reconciliation sensor?).perhaps with the asset reconciliation sensor?Nope. I removed that from the code, even replaced the
observable_source_asset
with a bare-bone one, the issue is still there.