Hi team - Do we have Dagster custom I/O manager fo...
# ask-community
a
Hi team - Do we have Dagster custom I/O manager for elasticsearch or opensearch?
🤖 1
s
Hi Abhishek, There is no official dagster integration for either of these, so no we do not provide an IO manager.
t
If you'd like to write one for yourself, here's an example of how that can be done: https://docs.dagster.io/concepts/io-management/io-managers#a-custom-io-manager-that-stores-pandas-dataframes-in-tables You would just need to define how you want the data to be read into memory and persisted into storage with those methods.
a
Thanks @sean and @Tim Castillo I'll take a look..
s
@sean Is this by any chance on the roadmap? This would be a great addition as ES/OS is widely used in the data space
t
Hey Sebastian! It's not on the roadmap. What're you looking for in an integration with ES/OS? Because you can access and use the Python client in Dagster by passing it in as a resource. Typically, first-class integrations are primarily reserved for managing services that compute their own assets, ex. we expand a dbt project to turn each model into an asset or turn every synced table in a fivetran/airbyte/stitch/meltano run into an individual asset. For storage layers, I've found that most cases can be satisfied by having the ability to write to and from it, which can be done with its Python SDK/client. If you have an idea for what a tighter integration with ES/OS might look like, you're welcome to raise a github issue and we'll assess it during our roadmapping! https://github.com/dagster-io/dagster/issues/new/choose