https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jack Zeitoun

03/08/2023, 8:21 PM
Hi all. Does anyone know how to serialize an op definition into JSON?
c

chris

03/08/2023, 10:17 PM
What’s the use case?
j

Jack Zeitoun

03/08/2023, 10:50 PM
We’re building an application on top of Dagster and would like a way to get a list of available op definitions without having to pre-load dagster/dagit with dummy jobs. Currently we create a dummy job with all ops and then query the GraphQL api for the available ops, but it would be more convenient if we could just serialize the ops directly and send them to our client.
c

chris

03/09/2023, 12:10 AM
• If all OpDefinition objects are available within the `repo`/`definitions` object, via various jobs, you can just use the existing functionality to iterate over them and retrieve their information:
Copy code
defs = Definitions(jobs=...)

available_ops = set()
for job in defs.get_all_job_defs():
    available_ops.extend(node_def.name for node_def in job.graph.node_defs)
j

Jack Zeitoun

03/09/2023, 12:13 AM
That would still require pre-populating the repo with jobs. We are building an app that will allow you to assemble jobs from the ops using a DAG editor. The dummy jobs works ok for now but it would be good to know if there was an option to just call
op.to_dict()
or something similar. Does anything like that exist?
c

chris

03/09/2023, 12:17 AM
Hmmm.. We do have internal snapshotting functionality to do this but I wouldn’t recommend using it as it can change at any point / isn’t considered a public API. How about the following things: • A function
get_op_metadata
which, given an
OpDefinition
, outputs the dictionary of information you seek (can help you retrieve whatever that information might be hopefully) • Have a module where you export all ops to. A la in some
__init__.py
file:
Copy code
from somewhere import some_op
from somewhere_else import other_op
then you can iterate over everything exported from that module, if it’s an
OpDefinition
, run
get_op_metadata
j

Jack Zeitoun

03/09/2023, 12:19 AM
Yes that's exactly what I have in mind.
A function like
get_op_metadat
does not exist yet does it?
c

chris

03/09/2023, 12:20 AM
It does exist but is used only internally and we have no plans to export it. That is to say, you’re better off writing your own
j

Jack Zeitoun

03/09/2023, 12:20 AM
Got it. Can you point me to the internal metho? I'll use it as a reference
c

chris

03/09/2023, 12:23 AM
https://github.com/dagster-io/dagster/blob/73eede0bc1667508445a70d559f0d80ba2fafd94/python_modules/dagster/dagster/_core/snap/solid.py#L386 - we use some serializable NamedTuples instead of dicts, and things are called
solid
rather than
op
for backcompat purposes, but you could imagine building something similar with just dictionaries
j

Jack Zeitoun

03/09/2023, 12:23 AM
Perfect. Thanks, Chris
yay 1