Hi, I'm very new to Dagster and I'm evaluating it to add orchestration to a custom data stack (designed in the 90's). It loads, parses and aggregates lots of gzipped text files from an archive (folder hierarchy) into another folder hierarchy of months and weeks with delimited text index files at each level. Custom report interface on top. All the tools are either C++ or Perl at the moment. It supports backfilling and parallel processing. Can Dagster be used to run custom programs with arguments (like datetime)? Will it help if those programs could read/write to stdin/stdout?

In general, dagster simply runs python code, so if you can run it from python, you can run it from dagster.

This might be helpful: https://github.com/dagster-io/dagster/discussions/15694