Monitoring cluster based on VictoriaMetrics

Elena Dvoryadkina

Overview

The Enterprise edition of ADQM supports monitoring based on VictoriaMetrics starting with 26.8.2.7.1.b1.

To set up monitoring, use:

  • An ADM cluster (version 5.0.0 or higher). An external VictoriaMetrics server can also be used.

  • The VMAgent component of an ADQM cluster’s Monitoring service — available starting with the ADQM version 26.8.2.7.1.b1.

VMAgent collects metrics from an ADQM cluster and sends them to an ADM cluster. If the Grafana service is enabled in ADM, dashboards with ADQM service and host metrics will be available in the Grafana web interface once monitoring is set up.

The following steps describe how to install the VMAgent component and set up monitoring.

NOTE

This article describes installation for ADQM clusters where the Monitoring service has not been previously installed. If the Monitoring service is already installed, follow these steps:

  1. Import configuration of an ADM cluster or external VictoriaMetrics server.

  2. Install the VMAgent component of the Monitoring service using the Add/Remove components service action after upgrading the cluster to 26.8.2.7.1.b1. Components that have already been installed in the Monitoring service (Prometheus Server, Node Exporter, Grafana) do not need to be removed.

  3. Configure the VMAgent component if necessary. After changing the configuration, restart the Monitoring service.

  4. Check the results of the VMAgent component installation.

Step 1. Install ADM

Install an ADM cluster (see the Installation guide) to be used as the base for import of the VictoriaMetrics server configuration.

The ADM cluster can be installed either before or after an ADQM cluster, but before installing the VMAgent component of the Monitoring service.

Step 2. Add the Monitoring service

  1. In the ADCM interface, open the Clusters page and click your ADQM cluster name. On the cluster page that opens, switch to the Services tab and click Add services.

    Switch to adding services
    Switch to adding services

  2. In the opened dialog, select the Monitoring service and click Add.

    Select the service
    Select the service

    As a result, the added service is displayed on the Services tab.

    Result of successful adding the service to the cluster
    Result of successful adding the service to the cluster

Step 3. Add the VMAgent component

  1. On the cluster page, open the Mapping tab to proceed to mapping instances of the VMAgent component to cluster hosts.

    Switch to mapping service components
    Switch to mapping service components

  2. Assign a host to the VMAgent component of the Monitoring service — click Add hosts and select the desired host in the pop-up window.

    Select a host for a component
    Select a host for a component
    CAUTION
    It is not recommended to install the VMAgent component on a host with ADQM — use a separate host for it. Otherwise, if an ADQM host fails and/or experiences critically high load, information about the corresponding problems will not be saved.

  3. After the distribution of components is completed, click Save.

Step 4. Configure VMAgent

  1. Open the Services tab on the cluster page and click the Monitoring service name in the Name column.

    Switch to the service configuration
    Switch to the service configuration

  2. On the page that opens, fill in or change the service configuration parameter values in the VMAgent section as needed — see parameter descriptions in the Configuration parameters article. Also, fill in the fields highlighted in red, as they are mandatory, even if the corresponding components (Prometheus Server, Grafana) are not added to the service.

    Configure the Monitoring service
    Configure the Monitoring service

  3. After specifying all necessary parameters, click Save.

Step 5. Import

Import ADM configuration

  1. Open the Import tab on the ADQM cluster page.

  2. Enable the Cluster configuration option in a row with the ADM cluster.

  3. Click Import.

Integration with ADM Cluster
Integration with ADM Cluster

Import external VictoriaMetrics server configuration

When using an external VictoriaMetrics server, import its configuration into ADQM. To do this, go to the Monitoring service settings and in the VMAgent section for the Custom remote_write parameter, use the Add property field to set the value url: "http://<vm-central>:8428/api/v1/write", where <vm-central> is the IP address of the external VictoriaMetrics server.

Step 6. Install the Monitoring service

  1. On the Services tab, click the icon actions default dark actions default light for the Monitoring service in the Actions column and run the Install action.

    Switch to the service installation
    Switch to the service installation

  2. Confirm the action in the opened window.

    Confirm the action
    Confirm the action

  3. Wait until the installation is completed. Then check that the service state has changed from created to installed.

    Installation is complete
    Installation is complete

    To view the service installation process and analyze errors if they occur, select Jobs in the left navigation menu and click the Install job name in the Jobs list.

    Install service job page
    Install service job page

Step 7. Check the results

After monitoring is configured, ADQM cluster metrics will be displayed in the VictoriaMetrics web interfaces. If the Grafana service is enabled in the ADM cluster, the Grafana web interface will contain dashboards with ADQM metrics (expand the ADQM → <your_ADQM_cluster> section on the Home → Dashboards page). Dashboards supported for ADQM versions earlier than 26.8.2.7 are marked with the legacy tag.

ADQM dashboards in the Grafana web interface
ADQM dashboards in the Grafana web interface
Found a mistake? Seleсt text and press Ctrl+Enter to report it