airflow.providers.openlineage.utils.spark¶

Attributes¶

`inject_parent_job_information_into_spark_properties`(...)	Inject parent job information into Spark properties if not already present.
`inject_transport_information_into_spark_properties`(...)	Inject transport information into Spark properties if not already present.

airflow.providers.openlineage.utils.spark.inject_parent_job_information_into_spark_properties(properties, context)[source]¶

Inject parent job information into Spark properties if not already present.

Args:: properties: Spark properties. context: The context containing task instance information.
Returns:: Modified Spark properties with OpenLineage parent job information properties injected, if applicable.

airflow.providers.openlineage.utils.spark.inject_transport_information_into_spark_properties(properties, context)[source]¶

Inject transport information into Spark properties if not already present.

Args:: properties: Spark properties. context: The context containing task instance information.
Returns:: Modified Spark properties with OpenLineage transport information properties injected, if applicable.