https://dagster.io/ logo
Join Slack
Powered by
This message was deleted.
# ask-community
s
This message was deleted.
z
I would start small, with a subset of your script. The 
@op
concept is usually more simple to start out with than 
@asset
if you're not sure where to get started. So I'd start by just taking 
func1
and trying to execute it in a graph as an op:
Copy code
@op
def func1():
  ...

@graph
def main():
  func1()

main_job = main.to_job()
defs = Definitions(jobs=[main_job])
Then add your functions as ops one at a time. One thing to keep in mind is that the code defined within 
@graph
should only call ops and should not contain general python syntax like looping or conditional statements. This is because the 
@graph
is executed at the time your code is loaded by Dagster, in order to construct the dependency graph, not when you try to run your job.
Is it an option to change the logic of the main function over time?
Often using Dagster will require you to modify your logic somewhat to adapt to the framework. One way to start could be to have a single op in your graph that wraps your main function, and then split out other ops over time as your gain more familiarity with the framework and understanding of how you want to use it for your code
You often don't want all your functions to be ops / assets, and will usually be calling your business logic (defined in vanilla / basic python functions and classes) from ops / assets, using ops / assets as higher-level units of organization
So you might be better served just starting out like
Copy code
@op
def main():
  ...

@graph
def run_main():
  main()

run_main_job = run_main.to_job()
or even using an asset if the output of your 
main()
function represents a data asset that you want to track
Copy code
@asset
def main():
  ...
g
@Zach I really appreciate it. Thank you so much for that. Although I'm working on the data inside the main() function, which has looping and condictional statements, I am unable to alter the logic inside the function. I'll consider your suggestions and get back to you, then. Regards.
16 Views
Open in Slack
PreviousNext
Powered by