Deividas Jodogalvis
02/06/2024, 1:09 PMreconstructable
methodScout
02/06/2024, 1:09 PMreconstructable
method, you need to follow a specific pattern as outlined in the documentation. The reconstructable
method is used when your job must cross process boundaries, such as for execution on multiple nodes or in different systems. It requires a function that returns a JobDefinition
or is decorated with @job
. Here's a step-by-step guide based on the documentation: 1. Define your job within a module-scoped function. This function should return a JobDefinition
object, which can be created by calling to_job()
on a GraphDefinition
or by using the @job
decorator. 2. Ensure that the function you define is at the module level and not nested within another function or scope. This is necessary because reconstructable
uses a conservative strategy for reconstruction that does not support jobs defined by lambdas, in nested scopes, or in interactive environments like Jupyter notebooks. 3. Use the reconstructable
method to create a ReconstructableJob
from the function you defined. Here's an example from the documentation: python from dagster import graph, reconstructable @graph def my_graph(): ... def define_my_job(): return my_graph.to_job() reconstructable(define_my_job)
In this example, define_my_job
is a module-scoped function that returns a JobDefinition
created by calling to_job()
on a GraphDefinition
. The reconstructable
method is then used to create a ReconstructableJob
from define_my_job
. If you have a job name and want to make it reconstructable, you would follow a similar pattern, ensuring that the job is defined in a way that is compatible with the reconstructable
method. If the job is already defined with the @job
decorator, you can directly pass it to `reconstructable`: python from dagster import job, reconstructable @job def my_named_job(): ... reconstructable_my_named_job = reconstructable(my_named_job)
In this case, my_named_job
is a job defined with the @job
decorator, and reconstructable_my_named_job
is the resulting ReconstructableJob
.