Kafka API

Olga Semme

Contents

Overview
Set dependencies
Producer API
Consumer API
Streams API
Connect API
Admin API

Overview

Kafka includes five core APIs:

Producer API
Consumer API
Streams API
Connect API
Admin API

APIs in Kafka are used to implement client applications that consume and transmit data to Kafka, manage Kafka brokers and topics, and perform stream processing.

Clients for Java applications are maintained as part of the main Kafka project, others are available as independent open source projects.

NOTE

For a list of non-Java clients, see the Kafka clients article.

Set dependencies

To create an API in Kafka, Apache Maven is used. It is a tool based on the concept of the POM (Project Object Model) that manages the build of the project, loading the appropriate versions of packages and libraries. To download the required package, you need to specify the dependency in the file pom.xml. Based on the given dependency, Maven will download and add the JAR file (Java archive) to the Java path. Java will then be able to find and use the classes defined in the JAR file.

The GAV parameters (groupId, artifactId, version) are used to specify the dependency:

groupId — ID of the object’s producer. Often the scheme adopted in the designation of Java packages is used.
artifactId — object identifier. This is usually the name of the module or application being created.
version — the version of the described object.

To use the API, the Maven dependencies in the pom.xml file are set as follows:

<dependency>
	<groupId>org.apache.kafka</groupId>
	<artifactId>kafka-clients</artifactId>
	<version>3.5.0</version>
</dependency>

Producer API

The Producer API allows applications to send streams of data to topics in a Kafka cluster.

Consumer API

The Consumer API allows applications to read data streams from topics in a Kafka cluster.

Streams API

The Streams API allows you to convert data streams from input topics to output topics.

NOTE

For more information about Kafka Streams, see the KAFKA STREAMS article.
For information on creating a Kafka Streams application using an ADS cluster, see the Write a Kafka Streams application article.

Connect API

The Connect API allows you to implement connectors that continuously fetch data from some source data system in Kafka or pass data from Kafka to some sink data system. Many Connect users do not need to use this API directly, but they can use pre-built connectors without having to write any code.

NOTE

For more information about Kafka Connect, see the KAFKA CONNECT article.
For information on working with Kafka Connect using the ADS Control interface tools, see the articles:

Admin API

The Admin API supports the management and validation of topics, brokers, ACLs, and other Kafka objects.

NOTE

For a complete list of available APIs, see the kafka 2.8.1 API article.

Found a mistake? Seleсt text and press Ctrl+Enter to report it