Joram Sbrisny
03/04/2024, 7:52 PMScout
03/04/2024, 7:52 PMAssetSelection
API to define which assets you want to include in your job. The AssetSelection
API allows you to create queries over a set of assets and asset checks, and you can use operators like “|”, “&”, and “-” to create unions, intersections, and differences of selections, respectively. Here are some examples from the documentation on how to use `AssetSelection`: - To select all assets in a group "marketing": python AssetSelection.groups("marketing")
- To select all assets in group "marketing" and also include a specific asset with key "promotion": python AssetSelection.groups("marketing") | AssetSelection.keys("promotion")
- To select all assets in group "marketing" that are downstream of asset "leads": python AssetSelection.groups("marketing") & AssetSelection.keys("leads").downstream()
- To select a specific list of assets: python my_assets_list = [...] AssetSelection.assets(*my_assets_list)
- To exclude all assets in group "marketing" from the selection: python AssetSelection.all() - AssetSelection.groups("marketing")
Once you have defined your AssetSelection
, you can use it to define a job that targets the selected assets. For example: python from dagster import define_asset_job, AssetSelection # Define your asset selection based on your criteria my_asset_selection = AssetSelection.groups("marketing") | AssetSelection.keys("promotion") # Define a job that uses the asset selection my_job = define_asset_job("my_filtered_job", selection=my_asset_selection)
After defining your job with the desired asset selection, you can execute this job from the UI, and it will materialize or observe only the assets that match your selection criteria. Please note that the ability to filter assets directly in the UI based on asset groups or other criteria may depend on the features available in the version of Dagster you are using. The examples provided here are based on defining the selection programmatically. If there are UI-specific features for filtering assets in the version of Dagster you are using, you would find them documented in the UI section of the Dagster documentation or release notes.Joram Sbrisny
03/04/2024, 7:57 PMScout
03/04/2024, 7:57 PMAssetSelection
API in code. However, there are some filtering capabilities available in Dagit that you can use to view subsets of assets: 1. Asset Group Filtering: If your assets are organized into groups, you can filter by these groups in the Dagit UI. Asset groups show up in the left navigation pane, and you can click on one to see the graph of all assets in the group. 2. Search Bar: Dagit provides a search bar that allows you to search for assets by name. You can use this to quickly find and filter assets. 3. Asset Graph: When viewing the asset graph, you can use the search functionality to filter the graph view by asset name or asset group. 4. List View: If you are in the list view of assets, you can use the search functionality to filter the list by asset name. It's important to note that there have been feature requests and discussions around enhancing the filtering capabilities in Dagit, such as allowing the use of a query filter language in list view and enabling filtering based on asset groups or tags. These discussions suggest that there is ongoing work to improve the filtering experience in the UI, but the specific features and their availability would depend on the version of Dagster you are using and any updates that have been made since my knowledge cutoff date. If you are looking for more advanced filtering capabilities in the UI, I recommend checking the latest Dagster documentation or the GitHub repository for any updates or newly introduced features that may allow for more granular filtering in Dagit.