Skip to content

Commit e28a445

Browse files
update batch size in session config
current: 1000000 update to 8192 (default value) this helps in query performance as larger batch size consumes more memory and slows down performance with this change, parseable's clickbench numbers are [181,68,68]
1 parent 6919a5c commit e28a445

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/query/mod.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ impl Query {
111111
let mut config = SessionConfig::default()
112112
.with_parquet_pruning(true)
113113
.with_prefer_existing_sort(true)
114-
.with_batch_size(1000000);
114+
.with_batch_size(20000);
115115

116116
// Pushdown filters allows DF to push the filters as far down in the plan as possible
117117
// and thus, reducing the number of rows decoded

0 commit comments

Comments
 (0)