https://dagster.io/ logo
#announcements
Title
# announcements
s

sephi

06/21/2020, 7:57 AM
Hi, We want to run a R script with in a
bash_command_solid
inside a
composite_solid
- and need to set a dependency between the output of a
solid
and the
bash_command_solid
. The pseudo code is as follows:
Copy code
@composite_solid()
def func(): 
  path_to_file = save_file_solid()
  res = bash_command_solid(f"Rscript run_process.R {path_to_file}")
  return res
The error we receive is as follows:
Copy code
dagster.core.definitions.events.Failure: Bash command execution failed with output: /tmp/tmpxxxxx line 1: syntax error near unexpected token `newline'\n/tmp/tmpxxxxx: `Rscirpt run_process.R <dagster.core.definitions.composition.InvokedSolidOutputHandle object at 0x7f....>'\n", "label": "intentional-failure", "metadata_entries":[]}
From what we understand the
composite_solid
is running within a pipeline and generating a temp path string output from the solid (without running the solid itself). Running the bash command in a terminal runs flawless. In other
composite_solid
s we are able to create dependencies so I'm guessing that it is related to
bash_command_solid
. What would be the correct approach for such task?
s

schrockn

06/21/2020, 2:02 PM
Hey sephi. The body of the @composite_solid function is run during pipeline construction time, not at runtime. So the “path_to_file” variable is not a the actual, path, but an object that represents the path. (It’s an instance of InvokedSolidOutputHandle, which explains the error string).
shell_solid
is the closest thing to what you want You may need to make your own slightly different version if you want use save_file_solid unmodified, but it is a lightweight wrapper around the workhorse
execute
function in dagster-shell which you are free to use directly
23 Views