Skip to content

Enhance Function<List<T>, List<R>> to Publish Individual Messages with splitMode=true #3097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

2tsumo-hitori
Copy link
Contributor

@2tsumo-hitori 2tsumo-hitori commented Mar 14, 2025

Currently, when registering a Function<List, List> as a bean in Spring Cloud Function, Spring Cloud Stream publishes the entire list as a single message instead of splitting it into individual messages, even when the producer sets splitMode=true.

I have opened a related issue in Spring Cloud Function: Issue #1242. If this feature is improved in Spring Cloud Function and subsequently applied to Spring Cloud Stream, it would enhance message processing efficiency.

With this enhancement:

  • Before:
    • Consumed messages were aggregated into a single list and published as a single message.
  • After:
    • By enabling splitMode=true, each element in the list can be published as an individual message automatically, eliminating the need for additional handling via StreamBridge.

This is a test in a local Kafka setup.

🖼️ Before applying splitMode:
image

🖼️ After applying splitMode:
image

Spring Boot Server Configuration
(This setup references locally modified versions of Spring Cloud Function and Spring Cloud Stream, which I updated and installed in the .m2 folder.)

spring:
  cloud:
    function:
      definition: eventStream
    stream:
      bindings:
        eventStream-in-0:
          destination: prev-topic
          group: test-group-unique
          consumer:
            batch-mode: true
        eventStream-out-0:
          destination: next-topic
          producer:
            split-mode: true
      kafka:
        binder:
          brokers: localhost:9092
          auto-create-topics: true
@Bean
public Function<List<String>, List<String>> eventStream() {
      return messages -> messages
            .stream()
            .map(String::toUpperCase)
            .toList();
}

…g Cloud Stream

- Extract splitMode property from producer bindings and apply it to the function
- Introduce `setEnableSplitting(boolean)` to allow automatic message splitting
- Ensure individual messages are published instead of a single aggregated list when splitMode=true
- This change depends on Spring Cloud Function PR(spring-cloud#1243)

Signed-off-by: 2tsumo-hitori <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant