Has anyone worked on pipelines in Dagster that execute R Code/Scripts? The analytics team I’m working with primarily develop their models in R. I’ve done this in the past in Airflow both by using Airflow’s BashOperator and running the Rscript, as well as stuffing the script into a container and using Airflow’s DockerOperator. I’m wondering if Dagster currently provides any support for R Script execution or if the aforementioned methods are the same kinds of approaches I would need to take? I saw in the 0.8.0 launch demo you highlighted the Host/User process separation and that container support of environments would be coming soon… is R one of the languages on the roadmap? Also love the work done on 0.8.0 and really looking forward to 0.9.0 and beyond 😄

hey @John Mav, yes, for now your best bet is shelling out to Rscript via something like this, but it’s certainly on our radar for the future

Thanks @nate I’ll take a look at this sounds like it’s similar to what I’ve done with A in the past, looking forward to seeing how y’all tackle this in the future 😄

yeah! we’ve got https://github.com/dagster-io/dagster/issues/1585 tracking this w.r.t. support for polyglot jupyter notebooks, which may or may not be a fit for your use case—would you mind adding some details on your use case either to that issue or file a new one if it’s not a good match?

Polyglot jupyter notebooks would be great but I was thinking of a more a more direct way to make R an available environment in which a Dagster Pipeline could run. I understand that the latter is certainly no simple task. My team currently doesn’t really lean into Jupyter/dagstermill (although I’m trying to get them to convert 🙂 ) and have primarily been using RMarkdown and Shiny Apps for prototyping/data modeling.

yup, totally makes sense! Would love an issue with any detail you can provide on the behavior you’d like to see

I may add a bit more context later but got a rough idea up: https://github.com/dagster-io/dagster/issues/2624

perfect, thanks!

I'm a few years late here, but I've been exploring Dagster and R integration using the experimental Dagster Pipes feature. Please see this repo for more details: https://github.com/philiporlando/dagster-and-r This GitHub discussion also contains some more info: https://github.com/dagster-io/dagster/discussions/18972