set the channel description: A channel for all things Dagster+Geo Spacial analysis.

set the channel description: A channel for all things Dagster+Geospatial analysis.

… and while I am here, if anybody has recommendations for improving this page, let me know. The example is beyond basic! https://dagster.io/glossary/geospatial-analysis

ha ha - thanks for fixing my typo!

Hi, I am using dagster in conjunction with OSM and postGIS

I’m getting my feet wet with dagster, writing a quick and dirty dagster PostGISGeoPandasIOManager. Feedback is welcome.

One thing I have been wondering after working with Dagster and Dagit for a while is how to manage a huge amount of assets in a way that they are easily queriable through the UI being used by other members of our team dealing not with data processing, but with monitoring of a running project. For instance, both for pointcloud data and for aerial imagery it is quite common to have tens of thousands of tiles composing a dataset. A project can span multiple datasets and so on... This problem isn't related just to our domain, but I was wondering if others were having issues as well in modelling their assets and workflows. I'm not sure if my reasoning is correct but I have pushed for an initiative of having partitions within partitions to better organize our data: https://github.com/dagster-io/dagster/issues/14228

I'm wondering if anyone has done anything with storing & processing of geo-spatial data with a time series dimension? This is something I am interested in doing at scale but I'm not sure where to start.

Generally using Apache Arrow with PostGIS and general-purpose types in ClickHouse. I like Clojure's philosophy of sticking to plain data types as much as possible. Also keeping an eye on polars and duckdb

We're setting up GDAL to enable more spatial data manipulation. I'm thinking of just using the GDAL base container image then setting up Dagster from there. Are others doing it this way?

Hi y'all. Anyone played with Dagster + DuckDB spatial extension yet or know of some good examples of similar Dagster pipelines? I'm very early on in trying to scope out a Dagster + dbt + <insert non-Arc spatial tools> pipeline to pair relational data and spatial data. A bit overwhelmed by all the tooling (largely new to me) and hunting for some examples 😅. Thanks in advance!