Tadas Barzdžius
06/15/2022, 5:45 PM@op(
out=DynamicOut(FileInfo),
)
def split_file(path: list[str], split_length: int = 10000) -> DynamicOutput[FileInfo]:
with open(path[0], encoding="utf-8") as file_handler:
content = [int(line.strip()) for line in file_handler]
for chunk in np.array_split(content, math.ceil(len(content) / split_length)):
filename_hash = hashlib.md5(str(chunk).encode("utf-8")).hexdigest()
filename = f"{filename_hash}.txt"
yield DynamicOutput(FileInfo(chunk, filename), mapping_key=filename.replace(".", "_"))
But I still get mypy error:
*error:* The return type of a generator function should be *"Generator"* or one of its supertypes
claire
06/15/2022, 7:28 PM@op
def my_op_yields_output() -> Generator[DynamicOutput, None, None]:
for i in range(5):
yield DynamicOutput(i, mapping_key=str(i))
Generator
object in the example above is from the typing
module:
https://docs.python.org/3/library/typing.html#typing.GeneratorTadas Barzdžius
06/16/2022, 3:56 AM