Work with Kafka Connect in ADS Control
Kafka Connects page overview
The Clusters → <cluster name> → Kafka Connects page in the ADS Control web interface is designed to display information about available instances of the Kafka Connect service in the ADS cluster. The Kafka Connects page becomes available after selecting a cluster in the cluster management section and going to the desired tab on the General page.
The Kafka Connects page contains a table with information about instances of the Kafka Connect service. The table columns are described below.
| Field | Description |
|---|---|
Kafka Connects |
Name of the Kafka Connect service instance. By clicking on the name, you can go to the Kafka Connect instance overview page |
Running |
Number of running connectors |
Paused |
Number of stopped connectors |
Degraded |
Number of connectors in |
Failed |
Number of connectors with error |
Total |
Total number of connectors |
Kafka Connect instance Overview
After clicking on the name, the Kafka Connect instance overview page Clusters → <cluster name> → Kafka Connects → <cluster name> connector opens.
The Kafka Connects service overview page contains:
-
Search and filtering section:
-
field for searching a connector by name;
-
Type field for filtering connectors by type:
SourceorSink; -
Status field for filtering connectors by status:
-
Running -
Stopped -
Paused -
Unassigned -
Failed -
Degraded
-
-
button to reset the search and filtering results; -
Create Connector button for creation new connectors.
-
-
Table with information about created Kafka Connect connectors. The table columns are described below.
| Field | Description |
|---|---|
Name |
Connector name |
Type |
Connector type |
Plugin |
Name of the connector plugin |
Topics |
Topics related to the operation of the connector |
Broker |
Broker on which the connector is running |
Tasks |
Number of connector tasks |
Actions |
The column contains icons for managing the connector:
All actions require confirmation |
Create a connector
To create a connector, run:
-
Click Create Connector on the Kafka Connect instance overview page. After clicking Create Connector, the window for selecting the connector plugin Clusters → <cluster name> → Kafka Connects → <cluster name> connector → Kafka connector plugins opens.
Connector plugin selection window
Connector plugin selection window -
Select the required connector plugin.
The following connectors are available for use: MirrorCheckpointConnector, MirrorHeartbeatConnector, MirrorSourceConnector — connectors designed to replicate topics using the Mirror Maker 2 mechanism. For more information on working with these connectors in ADS Control, see Mirror Maker 2 in ADS Control.
-
Next, fill in the connector configuration in the window that opens. You can use populating the config as a JSON file. To do this, turn on the corresponding switch.
Connector configuration
Connector configuration
Сonnector JSON configuration file
Сonnector JSON configuration file -
After filling, click Save and get a message about the successful creation of the connector.
Message about the successful creation of the connector
Message about the successful creation of the connector -
Make sure that the page of the created connector is opened with the display of tasks.
Created connector
Created connector -
Check that the Connectors for ads page displays the created connectors in the working status. The status is determined depending on the indicator in front of the connector name:
-
— the connector/task is running; -
— the connector/task has been administratively paused; -
— the connector/task has failed (usually by raising an exception, which is reported in the status output); -
— the connector/task has not yet been assigned to a worker.
Created connectors
Created connectors
-