Kafka Connect Basics

Hi, this article is about Kafka connect! Introduction Kafka connect is a tool for streaming data between Kafka and other systems. It is distributed and scalable by default and since it's a standardized tool there are lots of connectors already available. Connectors connect Kafka to a system or vice versa. There are two types of connectors Source: Source connectors grab data from an existing system e.g: MariaDB, PostgreSQL, S3, Jira, and others, and stream the data into one or more Kafka topics. Sink: Sink connectors grab the data from the topics and ingests it to a new system, eg: MongoDB, Snowflake, S3. If you want to stream change data capture events from your databases, the Debezium provides connectors that allow you to do just that. CDC is an append only log that identifies changes in databases, using a cdc stream you can replicate or reconstruct a database, additionally you can react on events by processing them in an external system. Kafka connect can be deployed in standalone mo