Hey all. I'm trying to get my job working with par...
# integration-airbyte
l
Hey all. I'm trying to get my job working with partitions. I'm using DailyPartitionsDefinition and everything downstream of the airbyte assets play nice, but if I do a backfill the airbyte sync is called many times leading to a timeout and failure of backfill for the specific day. Am I doing something wrong with the airbyte assets? I'm importing them using load_assets_from_airbyte_instance This is the error:
Copy code
2023-06-16 13:34:42 +0100 - dagster - ERROR - my_upstream_job - 55a4ed2c-611b-4ccf-a52e-95e607c292aa - airbyte_sync_2db70 - Request to Airbyte API failed: 502 Server Error: Bad Gateway for url: <http://localhost:8000/api/v1/jobs/cancel>
2023-06-16 13:34:42 +0100 - dagster - ERROR - my_upstream_job - 55a4ed2c-611b-4ccf-a52e-95e607c292aa - 6099 - airbyte_sync_2db70 - STEP_FAILURE - Execution of step "airbyte_sync_2db70" failed.

dagster._core.definitions.events.Failure: Max retries (3) exceeded with url: <http://localhost:8000/api/v1/jobs/cancel>.
a
The airbyte asset is partitioned, or only the assets downstream of airbyte? I don’t see how you could partition an airbyte asset, but you can have partitioned assets downstream of unpartitioned ones
l
I think only the assets downstream are, but in the dagster UI the airbyte partitions don't seem to be
How would I specify the airbyte assets to use a partitions def?
r
Also curious about this question @Adam Bloom wondering how parititions play with airbyte syncs
a
I don't see how you can have a partitioned airbyte asset, since airbyte has no concept of partitions.