Hello everyone I want to write a job that allows me to chain

Question

Hello everyone I want to write a job that allows me to chain a graph or op and a JobDefinition using define asset job function Or maybe design a graph that uses AssetSelection all instead of define asset job Is there anyway to achieve this

jamie · Answer

Hi < BC A> can you explain your use case a bit more If you want a job that will materialize all of your assets you can use `define asset job` with no selection parameter ```define asset job name= all assets job ```

BC A · Answer

Hi < jamie> Thing is I do have a couple of ops wrapped into a graph that needs to run before materializing my defined assets job ```define asset job name= all assets job ``` and the assets that I want to materialize are not defined as dependency to this graph even though they should at some point but the graph dosen t produce any materialization but just filling bigquery tables which are used as data source entry point for my elt the graph i m talking about is triggering google data transfer amp bigquery transfer jobs

jamie · Answer

I see In that case i think you ll want to write a run status sensor that triggers whenever your first graph succeeds Within the sensor you can start a run of the asset job It would look something like this ```all asset job = define asset job name= all assets job big query job = the job that populates bigquery with your data run status sensor run status=DagsterRunStatus SUCCESS monitored jobs=big query job request job=all asset job def report status sensor context return RunRequest run key=None ```

jamie · Answer

relevant docs <https docs dagster io concepts partitions schedules sensors sensors run status sensors>

BC A · Answer

Thank you very much I initially started implementing a sensor but it looks like I missed this run status feature

BC A · Answer

Hey < jamie> I ve got another blocking point which is I use define asset job to define my asset materialization job but this function is returning an UnresolvedAssetJobDefinition which is not supported by the run status sensor for the request job param it s expecting either a JobDefinition or a GraphDefinition I managed to overcome the issue by using ```from dagster core definitions assets job import build assets job build assets job myjob assets= dbt assets resource defs=resources definitions ``` But this is causing mess in my code and I don t know if I am supposed to use this build assets job function anyway so I was wondering if there was an alternative to turn the result of define asset job very handy with the asset selector feature into a resolved asset

jamie · Answer

are you getting an error passing the asset job to the sensor

BC A · Answer

The error I was getting before launching dagit ```Argument of type UnresolvedAssetJobDefinition cannot be assigned to parameter request job of type GraphDefinition | JobDefinition | None in function run status sensor Type UnresolvedAssetJobDefinition cannot be assigned to type GraphDefinition | JobDefinition | None UnresolvedAssetJobDefinition is incompatible with GraphDefinition UnresolvedAssetJobDefinition is incompatible with JobDefinition ``` and the code ``` run status sensor run status= DagsterRunStatus SUCCESS monitored jobs= update upstream assets job request job= define asset job elt job selection= ``` But actually when launching with dagit this works just fine I don t seem to really get it but wrong call from my side Thanks again I d take an explanation if anything

jamie · Answer

Yeah i think this is a type checking error on our side we have other sensors that work with UnresolvedAssetJobs but the runstatussensor has a type check for only jobs and graphs I don t see a reason we can t expand that to include the unresolved asset jobs i ll put a PR up and hopefully get a fix in soon

jamie · Answer

hey just looping back on this i wasn t able to replicate the issue in my own setup Can you try running this code snippet to see if you get the same error as before ```from dagster import asset run status sensor define asset job Definitions RunRequest DagsterRunStatus op job asset def one return 1 asset def two return 2 all assets job = define asset job all assets op def succeeds return True job def good job succeeds run status sensor run status=DagsterRunStatus SUCCESS request job=all assets job monitored jobs= good job def my run status sensor context return RunRequest run key=None defs = Definitions assets= one two jobs= good job all assets job sensors= my run status sensor ``` I still think we should put up a PR to fix the type signatures but in the meantime i want to make sure there isn t an additional bug to investigate

BC A · Answer

Hey seems like I m not able to reproduce it either for a reason I ignore but it was just a Pylance error message

jamie · Answer

ok I think the pr to change the type signature should remove the error for good then it ll come out in next week s release