Hello, beautiful community! I use software asset g...
# ask-community
Hello, beautiful community! I use software asset graphs as a set of operations that I only want to perform on a manually picked artifacts. Say there are hundreds of draft artifacts produced by some pipelines, then I select a handful of candidates and enable the asset graphs for them. Enablement is done through a pipeline that creates an asset materialization-marker based on ids of the selected artifacts. My repository looks like this in pseudo-code:
Copy code
def ....
    result = []    
    for enabled_artifact_asset in get_enabled_artifact_assets()
        new_asset_graph = get_asset_graph(enabled_artifact_asset.artifact_id)
    return result
Then, in order for the asset graph to appear in Dagit, I reloaded the code location through the GraphQL client. The problem: The approach stopped working after I migrated Dagster to k8s ( not by myself so there my knowledge is very limited ). Neither manual reload via
Dagit->Workspace->Code location->Reload
work - I can only see old asset graphs. How do I trigger a @repository update and thus a new asset graph to appear at k8s Dagster? Note: I'm currently running 0.15.0 and I don't really want to upgrade in the following couple of months. Many thanks for your attention.
Been there. 2 options - 1. Try GraphQL Repository Shutdown. I was doing this until I upgraded to Dagster 1.3.6. Dagster shipped an update which allows code location reload to fetch new definitions instead of reading it from cache. 2. Redeploy so that a new docker image gets deployed. Hectic.
Thanks for the reply! I tried Shutdown, yet I didn't like it at all - got some nasty glitches in return 🙂 I guess redeployment is a way to go for the time being... Anyway, it's great that there's a built-in functionality these days, I'll try to speed up the migration. Thank you again for sharing your experience!
Here's more information about how to turn on the experimental feature in 1.3.6 that Abhishek mentioned that addresses this: https://github.com/dagster-io/dagster/issues/12581#issuecomment-1563309129
@Ilya Tyutin you're welcome. Just wanted to comment that shutdown shouldn't introduce inconsistencies though. It is essentially what will happen when you redeploy. The code location will be down for a couple minutes or so (depending on how long it takes to load your definitions) but it should be back up and running with latest definitions. https://docs.dagster.io/concepts/dagit/graphql-client#shutting-down-a-repository-location-server
👍 1
Thanks for extra info! I happened to miss the notifications. I avoided the migration for the time being. I solved the issue by triggering a GitLab pipeline that reloads the code location 😃 Next stop is to upgrade Dagster.