You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems the Common Subexpression Elimination rule may eliminate also WindowFunctions by moving them into a Project node, which then causes physical planning to fail since a Project cannot actually execute those WindowFunctions.
To Reproduce
The following test fails on current main:
#[tokio::test]
async fn window() -> Result<()> {
let wexpr = row_number_udwf().call(vec![]);
let plan = LogicalPlanBuilder::empty(true)
.window(vec![wexpr.clone(), wexpr.alias("aliased")])?
.build()?;
println!("plan: {}", plan.display_indent_schema());
let ctx = SessionStateBuilder::new().build();
let plan = ctx.optimize(&plan)?;
println!("optimized plan: {}", plan.display_indent_schema());
DataFrame::new(ctx, plan).collect().await?;
Ok(())
}
with
plan: WindowAggr: windowExpr=[[row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING, row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING AS aliased]] [row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING:UInt64, aliased:UInt64]
EmptyRelation []
optimized plan: Projection: row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING, aliased [row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING:UInt64, aliased:UInt64]
WindowAggr: windowExpr=[[__common_expr_1 AS row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING, __common_expr_1 AS aliased]] [__common_expr_1:UInt64, row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING:UInt64, aliased:UInt64]
Projection: row_number() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING AS __common_expr_1 [__common_expr_1:UInt64]
EmptyRelation []
Error: NotImplemented("Physical plan does not support logical expression WindowFunction(WindowFunction { fun: WindowUDF(WindowUDF { inner: RowNumber { signature: Signature { type_signature: Nullary, volatility: Immutable } } }), params: WindowFunctionParams { args: [], partition_by: [], order_by: [], window_frame: WindowFrame { units: Rows, start_bound: Preceding(UInt64(NULL)), end_bound: Following(UInt64(NULL)), is_causal: false }, null_treatment: None } })")
Note how the optimize() has introduced a Project node which contains the row_number() call.
Removing either expr from the window() call makes the test pass.
Expected behavior
Test succeeds. Preferrably the Window node notices the common subexpression and deduplicates it, but that's a bonus, as long as the plan runs.
Additional context
No response
The text was updated successfully, but these errors were encountered:
Describe the bug
It seems the Common Subexpression Elimination rule may eliminate also WindowFunctions by moving them into a Project node, which then causes physical planning to fail since a Project cannot actually execute those WindowFunctions.
To Reproduce
The following test fails on current main:
with
Note how the optimize() has introduced a Project node which contains the row_number() call.
Removing either expr from the
window()
call makes the test pass.Expected behavior
Test succeeds. Preferrably the Window node notices the common subexpression and deduplicates it, but that's a bonus, as long as the plan runs.
Additional context
No response
The text was updated successfully, but these errors were encountered: