William
08/15/2022, 3:24 AMsnowflake/activity_analytics/story_daily_stats
in full_featured_example) I have to provide run_config for all other assets even they are neither up/down-stream of story_daily_stats
. In the full_featured example all `op`s do not have a config_schema
. However, once any one op
has a schema, then you'll have to enter it regardless of what asset you want to materialize.
This is likely not a desired behavior. <!subteam^S02T7CMRSE4|@dagster-cloud-support>William
08/15/2022, 3:36 AMcereals
example in tutorial:
See below right side is asset definition and left side is launchpad for `nabisco_cereals`:
• cereal_protein_fractions
requires a config named must_have_field
• nabisco_cereals
does not depend on cereal_protein_fractions
at all
• However, materializing nabisco_cereals
complains about missing required configWilliam
08/15/2022, 3:37 AMcereal_protein_fractions
config then at run time it complains it as an unexpected configsandy
08/16/2022, 1:20 AMimport csv
import requests
from dagster import asset, repository, load_assets_from_current_module, define_asset_job
@asset
def cereals():
response = requests.get("<https://docs.dagster.io/assets/cereal.csv>")
lines = response.text.split("\n")
return [row for row in csv.DictReader(lines)]
@asset
def nabisco_cereals(cereals):
"""Cereals manufactured by Nabisco"""
return [row for row in cereals if row["mfr"] == "N"]
@asset(config_schema={"a": str})
def cereal_protein_fractions(cereals):
"""
For each cereal, records its protein content as a fraction of its total mass.
"""
result = {}
for cereal in cereals:
total_grams = float(cereal["weight"]) * 28.35
result[cereal["name"]] = float(cereal["protein"]) / total_grams
return result
@asset
def highest_protein_nabisco_cereal(nabisco_cereals, cereal_protein_fractions):
"""
The name of the nabisco cereal that has the highest protein content.
"""
sorted_by_protein = sorted(
nabisco_cereals, key=lambda cereal: cereal_protein_fractions[cereal["name"]]
)
return sorted_by_protein[-1]["name"]
@repository
def repo():
return [load_assets_from_current_module(), define_asset_job("yoyo")]
would you be able to share the code you're using?