Skip to content

Commit 3bdd4e2

Browse files
jaceklaskowskijaceklaskowski
committed
Stateful Stream Processing (and Page renames + Menu reorg)
1 parent ea069ef commit 3bdd4e2

11 files changed

+49
-45
lines changed

‎SUMMARY.adoc

+29-28
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,29 @@
99

1010
. link:kafka-streams-exactly-once-support-eos.adoc[Exactly-Once Support (EOS)]
1111

12-
== Developing Stream Processing Applications with Kafka Streams
12+
== Stateful Stream Processing
13+
14+
. link:kafka-streams-stateful-stream-processing.adoc[Stateful Stream Processing]
15+
16+
. link:kafka-streams-StateStore.adoc[StateStore]
17+
.. link:kafka-streams-StateStore-KeyValueStore.adoc[KeyValueStore]
18+
.. link:kafka-streams-StateStore-SessionStore.adoc[SessionStore]
19+
.. link:kafka-streams-StateStore-WindowStore.adoc[WindowStore]
20+
21+
. link:kafka-streams-Stores.adoc[Stores]
22+
23+
. link:kafka-streams-Materialized.adoc[Materialized]
24+
25+
. link:kafka-streams-StoreSupplier.adoc[StoreSupplier]
26+
.. link:kafka-streams-KeyValueBytesStoreSupplier.adoc[KeyValueBytesStoreSupplier]
27+
.. link:kafka-streams-SessionBytesStoreSupplier.adoc[SessionBytesStoreSupplier]
28+
.. link:kafka-streams-WindowBytesStoreSupplier.adoc[WindowBytesStoreSupplier]
29+
30+
. link:kafka-streams-StoreBuilder.adoc[StoreBuilder]
31+
32+
. link:kafka-streams-ReadOnlyKeyValueStore.adoc[ReadOnlyKeyValueStore]
33+
34+
== Stream Processing with Kafka Streams
1335

1436
. link:kafka-streams-KafkaStreams.adoc[KafkaStreams]
1537
. link:kafka-streams-Topology.adoc[Topology]
@@ -111,33 +133,6 @@
111133
. link:kafka-streams-PartitionGrouper.adoc[PartitionGrouper Contract]
112134
.. link:kafka-streams-DefaultPartitionGrouper.adoc[DefaultPartitionGrouper]
113135

114-
== Developing Stateful Stream Processing Applications with Kafka Streams
115-
116-
. link:kafka-streams-StateStore.adoc[StateStore]
117-
.. link:kafka-streams-StateStore-SessionStore.adoc[SessionStore]
118-
.. link:kafka-streams-StateStore-WindowStore.adoc[WindowStore]
119-
120-
. link:kafka-streams-Stores.adoc[Stores]
121-
122-
. link:kafka-streams-Materialized.adoc[Materialized]
123-
124-
. link:kafka-streams-StoreSupplier.adoc[StoreSupplier]
125-
.. link:kafka-streams-KeyValueBytesStoreSupplier.adoc[KeyValueBytesStoreSupplier]
126-
.. link:kafka-streams-SessionBytesStoreSupplier.adoc[SessionBytesStoreSupplier]
127-
.. link:kafka-streams-WindowBytesStoreSupplier.adoc[WindowBytesStoreSupplier]
128-
129-
. link:kafka-streams-StoreBuilder.adoc[StoreBuilder]
130-
131-
. link:kafka-streams-KTableValueGetter.adoc[KTableValueGetter]
132-
133-
. link:kafka-streams-KTableValueGetterSupplier.adoc[KTableValueGetterSupplier]
134-
.. link:kafka-streams-KTableMaterializedValueGetterSupplier.adoc[KTableMaterializedValueGetterSupplier]
135-
.. link:kafka-streams-KTableSourceValueGetterSupplier.adoc[KTableSourceValueGetterSupplier]
136-
.. link:kafka-streams-KTableKTableAbstractJoinValueGetterSupplier.adoc[KTableKTableAbstractJoinValueGetterSupplier]
137-
138-
. link:kafka-streams-ReadOnlyKeyValueStore.adoc[ReadOnlyKeyValueStore]
139-
.. link:kafka-streams-StateStore-KeyValueStore.adoc[KeyValueStore]
140-
141136
== Monitoring Kafka Streams Applications
142137

143138
. link:kafka-streams-StateListener.adoc[StateListener -- KafkaStreams State Listener]
@@ -211,6 +206,12 @@
211206

212207
. link:kafka-streams-internals-KTableSuppressProcessor.adoc[KTableSuppressProcessor]
213208

209+
. link:kafka-streams-internals-KTableValueGetter.adoc[KTableValueGetter]
210+
. link:kafka-streams-internals-KTableValueGetterSupplier.adoc[KTableValueGetterSupplier]
211+
.. link:kafka-streams-internals-KTableMaterializedValueGetterSupplier.adoc[KTableMaterializedValueGetterSupplier]
212+
.. link:kafka-streams-internals-KTableSourceValueGetterSupplier.adoc[KTableSourceValueGetterSupplier]
213+
.. link:kafka-streams-internals-KTableKTableAbstractJoinValueGetterSupplier.adoc[KTableKTableAbstractJoinValueGetterSupplier]
214+
214215
=== Internals of State Stores
215216

216217
. link:kafka-streams-internals-WrappedStateStore.adoc[WrappedStateStore]

‎kafka-streams-GlobalKTableImpl.adoc

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ null
2222

2323
`GlobalKTableImpl` takes the following when created:
2424

25-
* [[valueGetterSupplier]] link:kafka-streams-KTableValueGetterSupplier.adoc[KTableValueGetterSupplier] (of `K` primary keys and `V` value changes)
25+
* [[valueGetterSupplier]] link:kafka-streams-internals-KTableValueGetterSupplier.adoc[KTableValueGetterSupplier] (of `K` primary keys and `V` value changes)
2626
* [[queryable]] `queryable` flag
2727

2828
=== [[queryableStoreName]] `queryableStoreName` Method
@@ -34,6 +34,6 @@ String queryableStoreName()
3434

3535
NOTE: `queryableStoreName` is part of link:kafka-streams-GlobalKTable.adoc#queryableStoreName[GlobalKTable Contract] to...FIXME.
3636

37-
Only when <<queryable, queryable>> is enabled, `queryableStoreName` requests the <<valueGetterSupplier, KTableValueGetterSupplier>> for link:kafka-streams-KTableValueGetterSupplier.adoc#storeNames[storeNames] and takes the very first one.
37+
Only when <<queryable, queryable>> is enabled, `queryableStoreName` requests the <<valueGetterSupplier, KTableValueGetterSupplier>> for link:kafka-streams-internals-KTableValueGetterSupplier.adoc#storeNames[storeNames] and takes the very first one.
3838

3939
Otherwise, `queryableStoreName` returns `null`.

‎kafka-streams-ProcessorContext.adoc

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
`ProcessorContext` is used when:
66

7-
* <<kafka-streams-Processor.adoc#init, Processor>>, <<kafka-streams-StateStore.adoc#init, StateStore>>, <<kafka-streams-Transformer.adoc#init, Transformer>>, <<kafka-streams-ValueTransformer.adoc#init, ValueTransformer>>, <<kafka-streams-ValueTransformerWithKey.adoc#init, ValueTransformerWithKey>> and <<kafka-streams-KTableValueGetter.adoc#init, KTableValueGetter>> are requested to initialize
7+
* <<kafka-streams-Processor.adoc#init, Processor>>, <<kafka-streams-StateStore.adoc#init, StateStore>>, <<kafka-streams-Transformer.adoc#init, Transformer>>, <<kafka-streams-ValueTransformer.adoc#init, ValueTransformer>>, <<kafka-streams-ValueTransformerWithKey.adoc#init, ValueTransformerWithKey>> and <<kafka-streams-internals-KTableValueGetter.adoc#init, KTableValueGetter>> are requested to initialize
88
99
* `Task` is requested for the <<kafka-streams-internals-Task.adoc#context, ProcessorContext>>
1010

‎kafka-streams-StateStore.adoc

+1-9
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
== [[StateStore]] StateStore Contract -- State Storage Engines
22

3-
`StateStore` is the <<contract, contract>> of <<implementations, state storage engines>> (_state stores_) that manage a state.
3+
`StateStore` is the <<contract, contract>> of <<implementations, state storage engines>> (_state stores_) that store a state.
44

55
`StateStore` can be <<persistent, persistent>> or not (i.e. *in-memory*).
66

7-
NOTE: A `StateStore` can be *local* or *global* (although it is a property of a <<kafka-streams-internals-ProcessorTopology.adoc#, ProcessorTopology>>).
8-
9-
Kafka Streams developers use <<kafka-streams-Stores.adoc#, Stores>> utility for creating state stores.
10-
11-
NOTE: A `StateStore` can be backed by a changelog topic in the Kafka cluster for fault-tolerance, i.e. with logging enabled (although it is an implementation detail, and not part of the `StateStore` contract). Kafka Streams developers use <<kafka-streams-Materialized.adoc#, Materialized>> or <<kafka-streams-StoreBuilder.adoc#, StoreBuilder>> APIs to control logging.
12-
13-
NOTE: A `StateStore` can be cached for performance, i.e. with caching enabled (although it is an implementation detail, and not part of the `StateStore` contract). Kafka Streams developers use <<kafka-streams-Materialized.adoc#, Materialized>> or <<kafka-streams-StoreBuilder.adoc#, StoreBuilder>> APIs to control logging.
14-
157
[[contract]]
168
.StateStore Contract
179
[cols="1m,3",options="header",width="100%"]

‎kafka-streams-internals-InternalStreamsBuilder.adoc

+1-1
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ NOTE: `GlobalKTables` use state stores with logging disabled.
150150

151151
`globalTable` <<addGraphNode, adds>> the `TableSourceNode` to the <<root, root>> node.
152152

153-
In the end, `globalTable` creates a <<kafka-streams-GlobalKTableImpl.adoc#, GlobalKTableImpl>> (with a new <<kafka-streams-KTableSourceValueGetterSupplier.adoc#, KTableSourceValueGetterSupplier>> and the queryable flag of the `MaterializedInternal`).
153+
In the end, `globalTable` creates a <<kafka-streams-GlobalKTableImpl.adoc#, GlobalKTableImpl>> (with a new <<kafka-streams-internals-KTableSourceValueGetterSupplier.adoc#, KTableSourceValueGetterSupplier>> and the queryable flag of the `MaterializedInternal`).
154154

155155
NOTE: `globalTable` is used exclusively when `StreamsBuilder` is requested to <<kafka-streams-StreamsBuilder.adoc#globalTable, add a GlobalKTable to the topology>>.
156156

‎kafka-streams-KTableKTableAbstractJoinValueGetterSupplier.adoc ‎kafka-streams-internals-KTableKTableAbstractJoinValueGetterSupplier.adoc

File renamed without changes.

‎kafka-streams-KTableMaterializedValueGetterSupplier.adoc ‎kafka-streams-internals-KTableMaterializedValueGetterSupplier.adoc

File renamed without changes.

‎kafka-streams-KTableSourceValueGetterSupplier.adoc ‎kafka-streams-internals-KTableSourceValueGetterSupplier.adoc

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
== [[KTableSourceValueGetterSupplier]] KTableSourceValueGetterSupplier
22

3-
`KTableSourceValueGetterSupplier` is a link:kafka-streams-KTableValueGetterSupplier.adoc[KTableValueGetterSupplier] that...FIXME
3+
`KTableSourceValueGetterSupplier` is a link:kafka-streams-internals-KTableValueGetterSupplier.adoc[KTableValueGetterSupplier] that...FIXME
44

55
`KTableSourceValueGetterSupplier` is <<creating-instance, created>> when...FIXME
66

‎kafka-streams-KTableValueGetter.adoc ‎kafka-streams-internals-KTableValueGetter.adoc

File renamed without changes.

‎kafka-streams-KTableValueGetterSupplier.adoc ‎kafka-streams-internals-KTableValueGetterSupplier.adoc

+3-3
Original file line numberDiff line numberDiff line change
@@ -33,12 +33,12 @@ interface KTableValueGetterSupplier<K, V> {
3333
| KTableValueGetterSupplier
3434
| Description
3535

36-
| link:kafka-streams-KTableMaterializedValueGetterSupplier.adoc[KTableMaterializedValueGetterSupplier]
36+
| link:kafka-streams-internals-KTableMaterializedValueGetterSupplier.adoc[KTableMaterializedValueGetterSupplier]
3737
| [[KTableMaterializedValueGetterSupplier]]
3838

39-
| link:kafka-streams-KTableSourceValueGetterSupplier.adoc[KTableSourceValueGetterSupplier]
39+
| link:kafka-streams-internals-KTableSourceValueGetterSupplier.adoc[KTableSourceValueGetterSupplier]
4040
| [[KTableSourceValueGetterSupplier]]
4141

42-
| link:kafka-streams-KTableKTableAbstractJoinValueGetterSupplier.adoc[KTableKTableAbstractJoinValueGetterSupplier]
42+
| link:kafka-streams-internals-KTableKTableAbstractJoinValueGetterSupplier.adoc[KTableKTableAbstractJoinValueGetterSupplier]
4343
| [[KTableKTableAbstractJoinValueGetterSupplier]]
4444
|===

‎kafka-streams-stateful-stream-processing.adoc

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
== Stateful Stream Processing
2+
3+
*Stateful Stream Processing* is...FIXME
4+
5+
NOTE: A `StateStore` can be *local* or *global* (although it is a property of a <<kafka-streams-internals-ProcessorTopology.adoc#, ProcessorTopology>>).
6+
7+
Kafka Streams developers use <<kafka-streams-Stores.adoc#, Stores>> utility for creating state stores.
8+
9+
NOTE: A `StateStore` can be backed by a changelog topic in the Kafka cluster for fault-tolerance, i.e. with logging enabled (although it is an implementation detail, and not part of the `StateStore` contract). Kafka Streams developers use <<kafka-streams-Materialized.adoc#, Materialized>> or <<kafka-streams-StoreBuilder.adoc#, StoreBuilder>> APIs to control logging.
10+
11+
NOTE: A `StateStore` can be cached for performance, i.e. with caching enabled (although it is an implementation detail, and not part of the `StateStore` contract). Kafka Streams developers use <<kafka-streams-Materialized.adoc#, Materialized>> or <<kafka-streams-StoreBuilder.adoc#, StoreBuilder>> APIs to control logging.

0 commit comments

Comments
 (0)