Apache Kafka ships with many Single Message Transformations included - but the great thing about it being an open API is that people can, and do, write their own transformations. Many of these are shared with the wider community, and in this final installment of the series I’m going to look at some of the transformations written by Jeremy Custenborder and available in kafka-connect-transform-common which can be downloaded and installed from Confluent Hub (or built from source, if you like that kind of thing). Also check out the XML transformation by the same author.
* Change topic case
* Use the timestamp of a field as the message timestamp
* Add the current timestamp to the message payload
* Using `SimulatorSinkConnector` (and Single Message Transform `TRACE` logging)
---
👾 Demo code and details: [ Ссылка ]
Learn more about Kafka Connect here:
🏃♂️ Quick: [ Ссылка ]
🚶 More detail: [ Ссылка ]
🗒️ SMT reference: [ Ссылка ]
---
⏱️ Time codes:
00:00:00 Single Message Transform introduction
00:01:18 Finding other Single Message Transforms on GitHub
00:01:36 A little side note about how cool Debezium is 😎
00:03:08 Transforms from Jeremy Custenborder
00:03:59 Generating test data with Kafka Connect and Voluble
00:04:34 Changing the case of a Kafka topic name
00:07:10 Using the timestamp in a message field to set the topic name
00:08:15 Chaining together three transformations
00:11:19 Adding the processing time into a field of a message in Kafka Connect
00:13:01 Kafka Connect SInk Simulator
00:14:35 Examining the output of the Simulator Sink Connector
00:15:23 Using Dynamic Logging in Kafka Connect to see detail of Single Message Transforms
00:16:50 Recap and conclusion
---
☁️ Confluent Cloud: [ Ссылка ]
💾Download Confluent Platform: [ Ссылка ]
📺 Kafka Connect connector deep-dives: [ Ссылка ]
✍️Kafka Connect documentation: [ Ссылка ]
🧩Confluent Hub: [ Ссылка ]
