Kafka API
Overview
Kafka includes five core APIs:
APIs in Kafka are used to implement client applications that consume and transmit data to Kafka, manage Kafka brokers and topics, and perform stream processing.
Clients for Java applications are maintained as part of the main Kafka project, others are available as independent open source projects.
NOTE
For a list of non-Java clients, see the Kafka clients article. |
Set dependencies
To create an API in Kafka, Apache Maven is used. It is a tool based on the concept of the POM (Project Object Model) that manages the build of the project, loading the appropriate versions of packages and libraries. To download the required package, you need to specify the dependency in the file pom.xml. Based on the given dependency, Maven will download and add the JAR file (Java archive) to the Java path. Java will then be able to find and use the classes defined in the JAR file.
The GAV parameters (groupId, artifactId, version) are used to specify the dependency:
-
groupId
— ID of the object’s producer. Often the scheme adopted in the designation of Java packages is used. -
artifactId
— object identifier. This is usually the name of the module or application being created. -
version
— the version of the described object.
To use the API, the Maven dependencies in the pom.xml file are set as follows:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>3.5.0</version>
</dependency>
Producer API
The Producer API allows applications to send streams of data to topics in a Kafka cluster.
Consumer API
The Consumer API allows applications to read data streams from topics in a Kafka cluster.
Streams API
The Streams API allows you to convert data streams from input topics to output topics.
NOTE
|
Connect API
The Connect API allows you to implement connectors that continuously fetch data from some source data system in Kafka or pass data from Kafka to some sink data system. Many Connect users do not need to use this API directly, but they can use pre-built connectors without having to write any code.
NOTE
|
Admin API
The Admin API supports the management and validation of topics, brokers, ACLs, and other Kafka objects.
NOTE
For a complete list of available APIs, see the kafka 2.8.1 API article. |