pyspark.sql.streaming.StreamingQuery.explain

StreamingQuery.explain(extended=False)

Prints the (logical and physical) plans to the console for debugging purpose.

New in version 2.1.0.

Parameters

extendedbool, optional

default False. If False, prints only the physical plan.

Examples

>>> sq = sdf.writeStream.format('memory').queryName('query_explain').start()
>>> sq.processAllAvailable() # Wait a bit to generate the runtime plans.
>>> sq.explain()
== Physical Plan ==
...
>>> sq.explain(True)
== Parsed Logical Plan ==
...
== Analyzed Logical Plan ==
...
== Optimized Logical Plan ==
...
== Physical Plan ==
...
>>> sq.stop()

pyspark.sql.streaming.StreamingQuery.exception pyspark.sql.streaming.StreamingQuery.id