Hi all, I am still learning and this may not be po...
# announcements
a
Hi all, I am still learning and this may not be possible or may be super simple but I am trying to understand how to properly link two solids where the internal solid uses a
Selector
. Because this solid is an internal to the dag (non a start node) I am not able to (or just don't know how to) pass in the solid config because the value is outputted by the previous solid. I have downloaded a file from s3 and the output is file path string to a csv file. Once I have this, I would then like to use the example
sum_solid
solid used in the airline_demo code that takes in this file path and using the dagster_pandas.Dataframe type. I was looking at the source code and see the Dataframe type uses a
Selector
to determine how to handle the data. Is there a way to properly do this?
Should I define the output of the download_from_s3 function to be something different?
a
So
Selector
is a tool used for materializing an instance of a value from configuration. If you are flowing information solid to solid using inputs and outputs - it shouldn't come in to play.
Thats one option - though might be best to leave
download_from_s3
generic
another option is to add a solid to go from
path
->
DataFrame
a
Thanks @alex ! I will add in a new solid then. I was thinking that but that makes sense
a
if you want to share code and have an interactive place to discuss it, collabedit has been useful in the past
cool, let me know if you run in to further issues
a
@alex That would be great if we can discuss it tomorrow if you don't mind.
a
sounds good - im usually online around 730a PST
a
Ok, I am available at that time as well. I will check in to see if you're available.