Kafka сlients

Clients for working with Apache Kafka are available in many programming languages.

ADS provides the ability to install client libraries for various languages ​​that provide both low-level access to Kafka and higher-level stream processing.

Kafka clients allow you to create distributed applications and microservices that read, write, and process event streams in parallel, at scale, and in a fault-tolerant manner, even in the event of network problems or machine failures.

Kafka comes with some of these clients, which are complemented by dozens of clients provided by the Kafka community.

NOTE

Information about Kafka clients is provided as a reference. Featured clients are independent open source projects.

Below is a list of currently known and supported Kafka clients.

Implementation language Client Description

C/C++

librdkafka

A robust, high-performance C/C++ library with full protocol support, built on top of many other language bindings, including Haskell, Node.js, OCaml, PHP, Python, Ruby, C#/.NET

libkafka

Native C++ library with protocol support for metadata, production, sampling, and offset

libkafka-asio

Kafka client library for C++ headers only using Boost Asio

csi-kafka

C++ 11 asynchronous producer/consumer library for Apache Kafka based on Boost Asio

libasynckafkaclient

Single-threaded asynchronous library based on C++

kafka-cpp

C++ client for Apache Kafka. A library that allows you to create messages for a distributed Kafka publish/subscribe messaging service

Python

High performance Python client based on librdkafka with full protocol support

kafka-python

Pure Python implementation with full protocol support; сonsumer and producer implementations included; GZIP, LZ4 and Snappy compression supported

pykafka

Python driver with full protocol support, Pythonic API, and managed balanced consumer implementation

pykafka

Allows you to create messages for the distributed Kafka publish/subscribe messaging service

pykafka

A programmer-friendly Kafka client for Python. It includes implementations of Kafka producers and consumers in Python, which are additionally supported by an extension built on top of librdkafka

pykafkap

Urban Airship Python client --simple Kafka producer client for Python

brod

Allows you to create messages for the Kafka publish/subscribe distributed messaging service (producer, simple consumer, ZK-consumer)

Go (AKA golang)

sarama

Kafka client library Go licensed by MIT; сlean implementation with full protocol supported; consumer and producer implementations included; GZIP and Snappy compression supported

The Kafka Confluent client for Golang includes the librdkafka library, providing full Kafka protocol support with high performance and reliability

Pure Go implementation provides full protocol support, high performance, and support for all client-facing KIPs

Erlang

brod

Erlang Kafka client library. Full consumer protocol support, very efficient manufacturer implementation

ekaf

Minimalist high-performance Kafka client in Erlang

erlkafka

kafka-erlang

Kafka client written in Erlang

PowerShell

PowerShell client with producer, consumer, admin support. Additionally, you can run all available tools from PowerShell. For a complete list of available commands for using KNet PowerShell, see KNet: PowerShell Module

.NET

KNet

KNet is a comprehensive .NET package for Apache Kafka providing all the features: producer, consumer, admin, streams, connect, backends (ZooKeeper and Kafka)

confluent-kafka-dotnet

Full featured .NET client for Apache Kafka based on librdkafka

kafka-net

Pure C# client with full protocol support. Includes consumer, producer, lower-level components, and gzip support (no fastload)

kafka-net

This .NET client implementation for Kafka using C# covers most of the core functionality, including a simple producer and consumer

CSharpClient-for-Kafka

A .Net implementation of the Apache Kafka protocol that provides basic functionality through the producer/consumer classes. The project also offers a balanced consumer implementation. The project is a fork of the ExactTarget Kafka-net client

kafka4net

C# client, asynchronous, compression supported, transparently tracking changes to the leading partition

kafka-sharp

High performance .NET Kafka driver

kafunk

Full featured native F# client running on .NET Core

Clojure

kafka-fast

Fast Kafka API for JVM languages ​​implemented in Сlojure

clj-kafka

Wrapper for Java API to interact with Kafka

Kafka client library for clojure

Provides a producer and consumer that support the underlying sampling API as well as a controlled sequencing interface

Ruby

Karafka

Multi-threaded efficient Kafka processing framework on Ruby and Rails. Based on librdkafka

WaterDrop

Standalone library, part of Karafka, for creating Kafka messages

Racecar

A simple framework for writing Kafka consumers in Ruby that integrates well with Rails, based on ruby-kafka

DeliveryBoy

An easy way to publish messages to Kafka from Ruby applications, based on ruby-kafka

EM-Kafka

EventMachine Driver for Kafka

ruby-kafka

Ruby client library for Apache Kafka

Node.js

node-rdkafka

A high-performance NodeJS client for Apache Kafka that wraps a native library librdkafka

Kafka-node

NodeJS client with ZooKeeper integration

node-kafka-0.8-plus

JavaScript implementation for NodeJS Server with Vagrant and Docker support

Nodejs wrapper for the JAVA Kafka API

Low-level protocol support in Node.js

Alternative node client from Tagged

Proxy (HTTP REST, etc)

Dory

Producer daemon for Apache Kafka that simplifies the work of clients sending messages to Kafka, freeing them from the complexity of directly interacting with the Kafka cluster

dropwizard-kafka-http

This project creates an Apache Kafka HTTP endpoint for creating and consuming messages. It is built on the Dropwizard platform which makes Java REST services operator friendly and easy to deploy

Kafka HTTP

The Rest endpoint provides access to Scala’s native high-level consumer and producer API

Provides a RESTful interface to a Kafka cluster

Kafka-Pixy

Dual API (gRPC and REST) ​​proxy server for Kafka with automatic consumer group control. It is designed to hide the complexity of the Kafka client protocol and provide a simple API that is trivial to implement in any language

Kastle

Efficient Kafka REST Proxy for message producers

KastleX

Full featured HTTP REST Proxy for Kafka in Elixir with JWT-based authentication

Perl

Pure Perl implementation that supports producer and consumer, ZooKeeper integration, GZIP and Snappy compression

stdin/stdout

kcat

Generic producer and consumer without JVM for Kafka

PHP

phpkafka

PHP extension built on librdkafka

PHP Kafka client

Stable, ready to go and fast Kafka client for PHP based on librdkafka

PHP library with consumer (simple and based on ZooKeeper), producer and compression support

Rust

Pure Rust implementation with support for Metadata, produce, fetch and offset queries. Supports Gzip and Snappy compression

Alternative Java

Krackle

A Kafka client designed to minimize the number of objects created and therefore reduce garbage collection overhead

Storm

storm-kafka-0.8-plus

Kafka client that is a part of Apache Storm

Scala DSL

scala-kafka

DSL for Scala developers to create and consume messages with Kafka

Swift

Franz

Kafka client for iOS and macOS that includes swift producer implementation, and simple and high-Level consumer

Found a mistake? Seleсt text and press Ctrl+Enter to report it