Hey! I'm trying to create the following, Shell script asset that starts an Airbyte asset which calls a dbt asset. Like in the screenshot. When I materialize the asset, the shell scripts complete successfully, it creates a CSV file and places it within

/tmp/airbyte_local/

folder. But, the following asset materialization step fails, complaining the file is missing. However, when I run the Airbyte sync job manually from Airbyte UI it works. According to the error, I am missing io manager, but I'm not sure what kind I should add and where. Also, why it would require to use io manager in this case?

Hi Ohad,

load_assets_from_airbyte_instance

produces a list of assets, so no need to wrap it in

attendance_shell_asset

Hi @ben, I might be wrong but I think you missed my point. I need to connect a shell script (this asset extracts some attendance data to CSV) to an Airbyte asset. If I do as you suggested, I will lose the data extraction node, see screenshot.

I see, this makes sense. I think that this is a gap in functionality right now - we don’t have a way to add an upstream asset of the Airbyte generated assets. This is something we could add pretty easily though

Good idea, I will give it a go. Thanks!

Yes, that’d be great

done, https://github.com/dagster-io/dagster/issues/12261

Hi Ohad, `upstream_assets`should just take the asset key for

extractStudentsAttendance

, which in this case is just the string representation of the function name:

upstream_assets={AssetKey("extractStudentsAttendance")}

Thanks @ben, I tried your suggestion, but I'm getting the attached error.

Hi Ohad, can you show me where you’re using

attendance_shell_asset

?

Sure, I'm using it in the list of assets within the definition

I think you’ll need a

*

in front of

attendance_shell_asset

here, since

with_resources

returns a list

Okay, that worked! Thanks for that! The Airbyte asset is working now! However, one issue, how can I re-connect the Aitbye asset to the downstream dbt asset now?

If you click on the asset above the dbt asset there, can you show me what the sidebar shows? That is the “asset key” that dbt is looking for, and it looks like it doesn’t match the asset key for our Airbyte asset

Sorry, but it doesn't open any sidebar when I click on it..

There should be a right-hand bar clicking on

src_webattend_attendance

, like the following - are you not seeing that?

I can see this menu within the Deployment tab, like in this example.

If you click on the upstream asset there, what does it show?

I get this

aha, perfect so we will want to add this to the `build_airbyte_assets`:

asset_key_prefix=["sentral_sources"]

oh okay! let me try that

Amazing! Thanks for your patience 🙌