Hello, beautiful community! I use software asset g...
# ask-community
i
Hello, beautiful community! I use software asset graphs as a set of operations that I only want to perform on a manually picked artifacts. Say there are hundreds of draft artifacts produced by some pipelines, then I select a handful of candidates and enable the asset graphs for them. Enablement is done through a pipeline that creates an asset materialization-marker based on ids of the selected artifacts. My repository looks like this in pseudo-code:
Copy code
@repository
def ....
    result = []    
    for enabled_artifact_asset in get_enabled_artifact_assets()
        new_asset_graph = get_asset_graph(enabled_artifact_asset.artifact_id)
        result.append(new_asset_graph)
    return result
Then, in order for the asset graph to appear in Dagit, I reloaded the code location through the GraphQL client. The problem: The approach stopped working after I migrated Dagster to k8s ( not by myself so there my knowledge is very limited ). Neither manual reload via
Dagit->Workspace->Code location->Reload
nor
GraphQL.reload_repository_location()
work - I can only see old asset graphs. How do I trigger a @repository update and thus a new asset graph to appear at k8s Dagster? Note: I'm currently running 0.15.0 and I don't really want to upgrade in the following couple of months. Many thanks for your attention.
a
Been there. 2 options - 1. Try GraphQL Repository Shutdown. I was doing this until I upgraded to Dagster 1.3.6. Dagster shipped an update which allows code location reload to fetch new definitions instead of reading it from cache. 2. Redeploy so that a new docker image gets deployed. Hectic.
i
Thanks for the reply! I tried Shutdown, yet I didn't like it at all - got some nasty glitches in return 🙂 I guess redeployment is a way to go for the time being... Anyway, it's great that there's a built-in functionality these days, I'll try to speed up the migration. Thank you again for sharing your experience!
d
Here's more information about how to turn on the experimental feature in 1.3.6 that Abhishek mentioned that addresses this: https://github.com/dagster-io/dagster/issues/12581#issuecomment-1563309129
a
@Ilya Tyutin you're welcome. Just wanted to comment that shutdown shouldn't introduce inconsistencies though. It is essentially what will happen when you redeploy. The code location will be down for a couple minutes or so (depending on how long it takes to load your definitions) but it should be back up and running with latest definitions. https://docs.dagster.io/concepts/dagit/graphql-client#shutting-down-a-repository-location-server
👍 1
i
Thanks for extra info! I happened to miss the notifications. I avoided the migration for the time being. I solved the issue by triggering a GitLab pipeline that reloads the code location 😃 Next stop is to upgrade Dagster.