refactor + test: query response serialization #1165
Conversation
WalkthroughThe changes update the query handler and response generation logic. In the query module, a new error variant for Arrow errors is added, and error handling is expanded to cover both JSON and Arrow errors, with improved logging via a renamed timing variable. In the response module, the method for producing client responses has shifted from HTTP to direct JSON conversion using an Arrow Changes
Sequence Diagram(s)sequenceDiagram
participant C as Client
participant QH as HTTP Query Handler
participant QE as QueryError Handler
participant LOG as Logger
participant HTTP as HTTP Response
C->>QH: Send query request
QH->>QH: Process query
QH->>QE: Encounter error (JSON/Arrow error)
QE-->>QH: Return error with appropriate status code
QH->>LOG: Log debug info (using 'start' time)
QH->>HTTP: Respond with error status and message
sequenceDiagram
participant C as Client
participant QR as QueryResponse
participant AW as ArrayWriter
participant JSON as JSON Converter
C->>QR: Request query result conversion
QR->>AW: Write record batch data
AW->>JSON: Convert written data buffer to JSON
JSON-->>QR: Return JSON data
QR->>C: Send JSON response
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Tip ⚡🧪 Multi-step agentic review comment chat (experimental)
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (10)
🔇 Additional comments (8)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
src/response.rs (2)
31-59: Potential performance improvements
Writing entireRecordBatchdata into a single in-memory buffer witharrow_json::ArrayWritermay become memory-intensive for large result sets. Consider a streaming or chunk-based approach if you anticipate very large batches. Also note that settingfill_nulltotruewill require iterating over all objects to find missing fields, which can increase overhead.
62-219: Great test coverage
These tests thoroughly cover multiple scenarios, including handling of null values, empty batches, and field inclusion. However, there is repeated code for creating record batches. Consider refactoring common setup steps into a helper function to reduce duplication.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
src/handlers/http/query.rs(6 hunks)src/response.rs(2 hunks)src/utils/arrow/mod.rs(0 hunks)
💤 Files with no reviewable changes (1)
- src/utils/arrow/mod.rs
⏰ Context from checks skipped due to timeout of 90000ms (10)
- GitHub Check: Build Default x86_64-pc-windows-msvc
- GitHub Check: Build Default aarch64-apple-darwin
- GitHub Check: Build Default x86_64-apple-darwin
- GitHub Check: Build Default aarch64-unknown-linux-gnu
- GitHub Check: Build Kafka aarch64-apple-darwin
- GitHub Check: Build Default x86_64-unknown-linux-gnu
- GitHub Check: Quest Smoke and Load Tests for Distributed deployments
- GitHub Check: Build Kafka x86_64-unknown-linux-gnu
- GitHub Check: Quest Smoke and Load Tests for Standalone deployments
- GitHub Check: coverage
🔇 Additional comments (8)
src/response.rs (2)
19-19: No issues
The newly added import forQueryErrorlooks correct.
21-21: Imports for JSON are appropriate
No concerns with the addedserde_jsonimports.src/handlers/http/query.rs (6)
22-22: Explicit import
IntroducingArrowErroris consistent with the newArrowerror variant.
35-35: New debug logging
No concerns with addingerroranddebugfromtracing.
105-105: Accurate timing logs for quick count queries
Recording and logging elapsed time here is beneficial for performance monitoring.Also applies to: 126-132
143-143: End-to-end timing for standard queries
Consistent usage of timing metrics here as well ensures comprehensive performance tracking.Also applies to: 145-145, 150-152
318-321: Enhanced error handling
AddingArrowandJsonvariants toQueryErrorclearly distinguishes error origins and improves debuggability.
327-329: Status code alignment
TreatingQueryError::JsonandQueryError::Arrowas internal server errors follows the existing pattern forExecuteerrors and keeps error handling consistent.
Fixes #XXXX.
Description
Improves the testability of
QueryResponseThis PR has:
Summary by CodeRabbit