https://dagster.io/ logo
#ask-ai
Title
# ask-ai
j

Josh

02/23/2024, 10:22 AM
@Scout Give me an example on how to create a SourceAsset with a function that retrieves the data from an external api?
s

Scout

02/23/2024, 10:22 AM
The documentation provides an example of how to define a
SourceAsset
and use it within an asset function, but it does not include a direct example of retrieving data from an external API within the
SourceAsset
definition. However, I can guide you through creating a
SourceAsset
and then using it in an asset function that could retrieve data from an external API. First, you define a
SourceAsset
with a specific key:
Copy code
python from dagster import SourceAsset, AssetKey  my_source_asset = SourceAsset(key=AssetKey("a_source_asset"))
Then, you can use this
SourceAsset
in an asset function. The asset function can include the logic to retrieve data from an external API:
Copy code
python from dagster import asset  @asset def my_asset_function(a_source_asset):     # Here you would include the logic to call your external API and retrieve data     # For example, you might use the `requests` library to make an HTTP GET request     # data = requests.get("<http://your-api.com/data>").json()     # return data     return a_source_asset + [4]  # This is a placeholder for the actual data retrieval logic
Note: The example above is a placeholder and does not include the actual implementation of the API call. You would need to replace the placeholder logic with the actual code to call your external API and process the response. Remember that the
SourceAsset
itself does not contain the logic to retrieve data; it is simply a way to represent external data within the Dagster asset graph. The actual data retrieval logic should be implemented in the asset function that depends on the
SourceAsset
. If you need to retrieve data from an external API, you would typically use a resource or an IO manager that encapsulates the API logic, and then use that resource within your asset function to fetch the data. The
SourceAsset
would then be used to represent the external data dependency in the asset graph.