@Scout I’m using airbyte for data ingestion. Say that, for one airbyte connection, I define 5 table assets in Dagster. In reality the airbyte connection refreshes 20 tables. When I trigger the airbyte sync from Dagster, how many credits are consumed ?

When you trigger an Airbyte sync from Dagster, the number of credits consumed is not determined by the number of assets you define in Dagster, but rather by the operation performed by Airbyte itself. According to the documentation, when you use Dagster's integration with Airbyte, you can build a set of assets representing the tables created by an Airbyte sync operation using the

build_airbyte_assets

function. This function allows you to specify a connection ID and a list of destination tables that you expect to produce, and it returns a set of assets that represent those tables in code. However, the documentation does not provide specific details on how Airbyte's credit consumption works within the context of Dagster's integration. Airbyte's credit consumption is typically a function of the Airbyte platform itself and is based on the operations performed during the sync, such as the number of records processed, the number of API calls made, or other factors as defined by Airbyte's pricing model. Therefore, to determine how many credits are consumed when you trigger an Airbyte sync from Dagster, you would need to refer to Airbyte's own documentation or pricing information to understand how they calculate credit usage for sync operations. Dagster's role is to orchestrate the sync and represent the resulting tables as assets, but it does not dictate how Airbyte's credit system operates.

@Scout I’m talking about Dagster credit consumption, not airbyte’s one. For one airbyte sync triggered from Dagster, how many Dagster credit do I consume ? Is it dependent on the number of table assets I define ?

I apologize for the confusion. Regarding Dagster credit consumption, the documentation does not explicitly state how credits are consumed in relation to the number of table assets defined for an Airbyte sync. However, it does mention that while refreshing these assets will still just require a single operation (running the Airbyte sync), representing them as assets allows you to visualize them separately in the Dagster UI. From this, we can infer that triggering an Airbyte sync from Dagster to refresh the defined table assets is considered a single operation, regardless of the number of table assets you have defined in Dagster for that connection. Therefore, the credit consumption in Dagster for triggering an Airbyte sync should not be directly dependent on the number of table assets you define, but rather on the operation itself. If you need a precise answer on credit consumption for this specific scenario, I would recommend checking the latest Dagster documentation or reaching out to Dagster support for the most accurate and up-to-date information on credit usage.