ADCM AIO Client usage examples

Adelya Zvyagintseva

CAUTION

Currently, the Alpha version of ADCM AIO Client is implemented. If necessary, backward compatibility with future versions can be broken. Release will be assigned the Alpha status as long as the major version number is zero (0.<minor>.<patch>). The ADCM AIO Client releases are presented in the GitHub repository.

Asynchronous HTTP Client for ADCM (ADCM AIO Client) allows you to perform multiple requests simultaneously without blocking the main thread which is provided by the asynchronous architecture.

Due to asynchronous request handling, ADCM AIO Client enables you to manage tasks more efficiently, reducing latency and improving throughput. The implemented advanced error handling mechanisms are designed to manage failures gracefully in an asynchronous environment.

The object model of ADCM AIO Client reflects the ADCM architecture and the structure of its objects, such as clusters, hosts, services, components, bundles, etc.

This article contains examples of using ADCM AIO Client to manage the ADCM objects and configurations.

Quick start with ADCM AIO Client

Follow the steps below to set up and use ADCM AIO Client.

CAUTION

Before installation, make sure that Python 3.12.0 or later and pip are available within your environment.

Install ADCM AIO Client using the following command:
```
$ pip install adcm_aio_client
```

Configure the connection to ADCM. Below is an example of getting the list of all clusters to describe how to configure the connection:

import asyncio
from adcm_aio_client import ADCMSession, Credentials

# User credentials to connect to the ADCM web interface
credentials = Credentials(username="<username>", password="<password>")

# Configure the request retry
kwargs = {"verify": False, "timeout": 10, "retry_interval": 1, "retry_attempts": 1}

async def get_clusters():

    # Create the connection to ADCM
    async with ADCMSession(url="http://<ip_address>:<port>", credentials=credentials, **kwargs) as client:
        # Get clusters
        clusters = await client.clusters.all()

asyncio.run(get_clusters())

where:

timeout — the ADCM connection timeout in seconds (the default value is 600);
retry_interval — the interval between ADCM connection attempts (the default value is 3);
retry_attempts — the number of retry attempts for the ADCM connection (the default value is 1);
<username> — the user account name;
<password> — the user account password;
<ip_address> — the IP address of the host where the ADCM container is deployed;
<port> — the port number where the ADCM container is running.

To establish an HTTPS connection to ADCM (using the SSL certificate), pass the verify parameter. This parameter determines the full path to the file (including the file name) that contains the certificate verified through a certificate authority (CA).

NOTE

Make sure that the URL begins with https and port 8443 is entered as the <port> parameter value.

import asyncio
from adcm_aio_client import ADCMSession, Credentials

credentials = Credentials(username="<username>", password="<password>")

verify = "/<file_path>/<certificate_file_name>.pem"

async def get_clusters():
    # Create the connection to ADCM using SSL
    async with ADCMSession(url="https://<ip_address>:8443", credentials=credentials, verify=verify) as client:
        # Get clusters
        clusters = await client.clusters.all()

asyncio.run(get_clusters())

Upload a bundle

# Upload a bundle with the subsequent acceptance of the license agreement
await client.bundles.create("https://downloads.arenadata.io/ADB/5.25.0_arenadata6/bundles/adcm_cluster_adb_v5.25.0_arenadata6_b1-1_enterprise.tgz", accept_license=True)

Create a hostprovider

# Determine the uploaded bundle based on which the hostprovider will be created
bundle = await client.bundles.get(name__eq="SSH Common")

# Create the hostprovider
hostprovider = await client.hostproviders.create(bundle=bundle, name="SSH")

Create hosts

# Determine the hostprovider
hostprovider=await client.hostproviders.get(name__eq="SSH")

# Create the host
host = await client.hosts.create(name="zar-adb-master", hostprovider=hostprovider))

Create a cluster

# Determine the uploaded bundle based on which the cluster will be created
bundle = await client.bundles.get(name__eq="ADB", version__eq="5.25.0_arenadata6_b1-1")

# Create the cluster
cluster = await client.clusters.create(bundle=bundle, name="Test_ADB_cluster")

Add services

# Specify the cluster and service to be added
cluster = await client.clusters.get(name__eq="Test_ADB_cluster")
service_name_conditions = Filter(attr="display_name", op="ieq", value="ADB")

# Add the service with the subsequent acceptance of the license agreement and definition of dependencies between services
await cluster.services.add(service_name_conditions, accept_license=True, with_dependencies=False)

Add hosts to a cluster

# Specify the required cluster and host to be added
cluster = await client.clusters.get(name__eq = "Test_ADB_cluster")
host = await client.hosts.get(name__ieq = "zar-adb-master")

# Add the host to the cluster
added_host = await cluster.hosts.add(host=host)

Manage mapping

ADCM AIO Client provides the mapping.add(component: Component | List[Component], host: Host| List[Host]) method to configure host-component mapping.

NOTE

Before the mapping.save() method is called, the host-component mapping is saved locally.

"""
Mapping components to hosts locally
"""
# Get hosts and components for mapping
components = await mapping.components.filter(display_name__iin=component_name_list)
hosts = await mapping.hosts.filter(name__iin=host_name_list)

# Pass obtained hosts and components as an input to the function
await mapping.add(component=components, host=hosts)

# Define host-component mapping
master_node = ["zar-adb-master"]
standby_node = ["zar-adb-standby"]
segment_node = ["zar-adb-segment1", "zar-adb-segment2"]

master_component = ["ADB Master"]
standby_component = ["ADB Standby"]
segment_component = ["ADB Segment"]

cluster = await client.clusters.get(name__eq="Test_ADB_cluster")
mapping = await cluster.mapping

await map_components_to_hosts(mapping, master_component, master_node)
await map_components_to_hosts(mapping, standby_component, standby_node)
await map_components_to_hosts(mapping, segment_component, segment_node)

# Save host-component mapping
await mapping.save()

Also, ADCM AIO Client allows you to manage mapping using the following methods:

mapping.all() — returns the current mapping pairs.
mapping.empty() — resets local changes in the host-component mapping, leaving them empty.
mapping.refresh() — actualizes current local changes in mapping taking into account remote changes. In case of conflicts, it applies one of the following strategies:
- apply_local_changes — local changes have higher priority than remote changes.
- apply_remote_changes — remote changes have higher priority than local changes.
The given example shows how to use the mapping.refresh() method. In case of conflicts, local changes will be applied because strategy takes the apply_local_changes value.
cluster = await client.clusters.get(name__eq="Test_ADB_cluster") mapping = await cluster.mapping component_2 = await mapping.components.filter(display_name__eq="ADBM server") host_1 = await mapping.hosts.filter(name__eq="zar-adb-master") host_2 = await mapping.hosts.filter(name__eq="zar-adb-control") await mapping.refresh(strategy=apply_local_changes) await mapping.remove(component_2, host_1) await mapping.add(component_2, host_2) await mapping.save()

Manage an object configuration

ADCM AIO Client allows you to get and change the value of an object (hostprovider, cluster, service, and component) and ADCM configuration parameter.

Local changes of the configuration parameter values can also be reset by using the config.reset() method if required.

Below are examples of configuration settings for a cluster.

NOTE

Pay attention to how to specify the path to a configuration parameter in the tree of configuration settings.

Get a configuration parameter value

# Specify the required cluster
cluster = await client.clusters.get(name__eq="Test_ADB_cluster")

# Get the object that controls the cluster configuration
config = await cluster.config

# Get the configuration parameter value
parameter = config['Vault integration']['Address of the Vault server'].value

Change a configuration parameter value

# Specify the required cluster
cluster = await client.clusters.get(name__eq="Test_ADB_cluster")

# Get the object that controls the cluster configuration
config = await cluster.config

# Get the configuration parameter
parameter = config['Vault integration']['Address of the Vault server']

# Set a new configuration parameter value
parameter.set('192.168.1.120')

# Save a new configuration
await config.save()

Manage an object config group

ADCM AIO Client provides methods described below to perform the following operations with config groups:

Create a cluster config group
List cluster config groups
Add hosts to a cluster config group
Get a value of a service config group parameter
Change the value of a service config group parameter

Also, ADCM AIO Client allows you to call the following methods for managing object config groups:

configuration.reset() — resets local changes in the config group configuration.
config_group.hosts.remove(host: Host| List[Host] | Filter) — removes hosts from the config group.
config_group.delete() — removes the config group from the cluster.

Create a cluster config group

# Specify the required cluster
cluster = await client.clusters.get(name__eq="Test_ADH_cluster")

# Specify the object that controls the host group of the cluster
config_group = cluster.config_host_groups

# Get host objects from the list of hosts
hosts_for_add = await cluster.hosts.filter(name__iin=["zar-adh-segment1", "zar-adh-segment2"])

# Create the config group
await config_group.create(name="config-group-1", hosts=hosts_for_add)

List cluster config groups

# Specify the required cluster
cluster = await client.clusters.get(name__eq="Test_ADH_cluster")

# Get the list of config groups
config_group_list = await cluster.config_host_groups.all()

Add hosts to a cluster config group

# Specify the required cluster
cluster = await client.clusters.get(name__eq="Test_ADH_cluster")

# Specify the config group to add a host
config_group = await cluster.config_host_groups.get(name__eq="config-group-2")

# Get the added host
added_host = await cluster.hosts.filter(name__iin="zar-adh-master")

# Add the host
host_add = await config_group.hosts.add(host=added_host)

Get the value of a service config group parameter

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="Test_ADH_cluster")
service = await cluster.services.get(name__eq="hbase")

# Specify the service config group
config_group = await service.config_host_groups.get(name__eq="service-group-1")

# Get the object that controls the config group configuration
configuration = await config_group.config

parameter = configuration["hbase-site.xml"]["hbase.client.max.perregion.tasks"].value

Change the value of a service config group parameter

NOTE

The configuration["<Path to parameter>"].sync() method allows you to turn on synchronization of the parameter value with the object primary configuration. To turn off synchronization and change the parameter value, use the configuration["<Path to parameter>"].desync() method.

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="Test_ADH_cluster")
service = await cluster.services.get(name__eq="hbase")

# Specify the service config group
config_group = await service.config_host_groups.get(name__eq="service-group-1")
configuration = await config_group.config

# Enable parameter synchronization with primary configurations
configuration["hbase-site.xml"]["hbase.client.max.perregion.tasks"].desync()

# Set a new config group parameter value
configuration["hbase-site.xml"]["hbase.client.max.perregion.tasks"].set(50)

# Save a new configuration
await configuration.save()

Manage action host groups

ADCM AIO Client provides methods that allow you to perform the following operations with action host groups:

Create an action host group
List action host groups
Add a host to an action host group
Launch an action for an action host group

Also, ADCM AIO Client allows you to call the following methods for managing action host groups:

action_host_group.hosts.remove() — removes hosts from the action host group.
action_host_group.delete() — removes the action host group from the ADCM object.

Create an action host group

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="Test ADQM cluster")
service = await cluster.services.get(name__eq="adqmdb")

# Specify the object that controls the action host group
action_group = service.action_host_groups

# Identify host objects from the list of hosts
hosts_for_add = await cluster.hosts.filter(name__iin=["dev-adqm-01.ru-central1.internal", "dev-adqm-04.ru-central1.internal"])

# Create the action host group
action_group_create = await action_group.create(name="hosts_1-4", hosts=hosts_for_add)

List action host groups

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="Test_ADQM_cluster")
service = await cluster.services.get(name__eq="adqmdb")

# Get the list of action host groups
action_group_list = await service.action_host_groups.all()

Add a host to an action host group

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="Test ADQM cluster")
service = await cluster.services.get(name__eq="adqmdb")

# Specify the action host group
action_host_group = await service.action_host_groups.get(name__eq="hosts_1-3")

# Get the added hosts
added_host = await cluster.hosts.filter(
    name__iin=["dev-adqm-02.ru-central1.internal", "dev-adqm-04.ru-central1.internal"])

# Add the host
host_add = await action_host_group.hosts.add(host=added_host)

Launch an action for an action host group

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="Test_ADQM_cluster")
service = await cluster.services.get(name__eq="adqmdb")

# Specify the action host group
action_host_group = await service.action_host_groups.get(name__eq="hosts_1-3")

# Specify the action
action = await action_host_group.actions.get(display_name__eq="Restart")

task = await action.run()

Import

ADCM AIO Client provides the cluster_import.add(*sources: Cluster | Service) method to transfer specific service settings to a cluster.

# Specify the cluster in which the service settings will be imported
cluster = await client.clusters.get(name__eq="Test_ADB_cluster")

# Get the object that controls import in the cluster
cluster_import = await cluster.imports

# Specify the cluster that contains service settings to be imported
cluster_for_import = await client.clusters.get(name__eq="Monitoring")

# Get the service settings to be imported
imported_service = await monitoring_cluster.services.get(name__eq="grafana")

# Import
imports = await cluster_import.add([imported_service])

Also, ADCM AIO Client allows you to manage import using the following methods:

cluster_import.remove(*sources: Cluster | Service) — removes imported settings of one service from a cluster.
cluster_import.set() — removes imported settings of all services from a cluster.

Perform object actions

ADCM AIO Client provides methods that allow you to perform object actions, including the following:

List actions.
Prepare an action.
Launch an action:
- without blocking an action execution thread;
- with blocking an action execution thread.
Get a job status.

As an example, consider the process of launching the Expand action.

List actions

# Specify the required cluster and service
cluster = await client.clusters.get(name__eq="ADB_cluster")
service = await cluster.services.get(name__eq="adb")

# Get all service actions
actions = await service.actions.all()

Prepare an action

Because the Expand action allows you to add new hosts to a cluster component, you need to prepare the action for the launch by specifying the following settings:

Set object configuration parameters.
Map cluster hosts to components.

Also, ADCM AIO Client allows you to run actions as follows:

in the non-blocking mode;
with the option to get detailed logs.

Set configuration parameters

NOTE

Methods for managing an object configuration are also applicable to actions containing object configuration parameters.

# Specify the required cluster and service
cluster = await client.clusters.get(name__ieq="Test_ADB_cluster")
service = await cluster.services.get(name__eq="adb")

# Specify the action
expand_action = await cluster.actions.get(name__eq="Expand")

# Get the object of the action containing configuration parameters
configuration = await expand_action.config

# Determine the parameter and its new value
configuration["Reboot timeout, sec"].set(605)

Map hosts to components

# Specify the required cluster and service
cluster = await client.clusters.get(name__ieq="Test_ADB_cluster")
service = await cluster.services.get(name__eq="adb")

# Specify the action
expand_action = await cluster.actions.get(name__eq="Expand")

# Specify hosts and components for mapping
components = await service.components.filter(name__iin=["ADB Segment", "PXF"])
hosts = await cluster.hosts.get(name__eq="test-host")

# Get the object of the action including host-component mapping
mapping = await expand_action.mapping

# Add the local mapping changes
await mapping.add(components, hosts)

Write detailed logs

# Specify the required cluster and service
cluster = await client.clusters.get(name__ieq="Test_ADB_cluster")
service = await cluster.services.get(name__eq="adb")

# Specify the action
expand_action = await cluster.actions.get(name__eq="Expand")

expand_action.verbose = True

Set the non-blocking mode

# Specify the required cluster and service
cluster = await client.clusters.get(name__ieq="Test_ADB_cluster")
service = await cluster.services.get(name__eq="adb")

# Specify the action
expand_action = await cluster.actions.get(name__eq="Expand")

expand_action.non_blocking = True

Launch an action

Without blocking the action execution thread

job = await expand_action.run()

With blocking an action execution thread

NOTE

A custom condition can be used to exit on timeout.

job = await expand_action.run()
await job.wait(exit_condition=default_exit_condition)

Get a job status

When launching actions, ADCM starts jobs. To follow the action execution, get the job status using the job.get_status() method.

job_status = await job.get_status()

Work with object collections

ADCM AIO Client provides methods described below to perform the following operations:

all() — returns a list of all objects;
list() — returns a paginated list of objects satisfying request parameters;
Example of using pagination
clusters = await client.clusters.list(query={"limit": 4, "offset": 10})

where:

limit — the maximum number of objects in a list;

offset — the number of objects to skip.
get() — returns one object satisfying the specified conditions;
filter() — returns the list of objects that match the given conditions.

To filter object collections, use the filter() method specifying filter parameters and field lookups described below.

Field lookups
Field lookup	Description
eq	Equal
ieq	Case-insensitive equal
ne	Not equal
ine	Case-insensitive not equal
contains	Substring search
icontains	Case-insensitive substring search
in	Match of one of the given comma-separated values in the `list` format
iin	Case-insensitive match of one of the given comma-separated values in the `list` format
exclude	Not equal to the given comma-separated values in the `list` format
iexclude	Case-insensitive not equal to the given comma-separated values in the `list` format

Note that different objects support different filter parameters.

Bundle
Cluster
Service
Component
Hostprovider
Host
Action
Job
Config group, action host group

name
display_name
version
edition

name
bundle
status

name
display_name
status

name
display_name

name
bundle

name
status

name
display_name

status
name
display_name
object
action

name

The bundle and action parameters can be used only with lookups eq, ne, in, and exclude.

clusters = await client.clusters.filter(name__iin=['monitoring', 'adb'])

clusters = await client.clusters.filter(name__icontains="test_adb_cluster")

Errors

The table below lists errors that may occur during processing requests.

Errors
Error code	Description
WaitTimeoutError	Execution timeout has exceeded the threshold
ClientInitError	Failed to connect to ADCM at specified URL
NotSupportedVersionError	Failed to detect ADCM version at the specified URL. Most likely, the ADCM version is lower than the minimum supported ADCM version. The ADCM version is not supported.
NoCredentialsError	Unable to locate credentials
AuthenticationError	Login to ADCM at the specified URL has failed. Possible cause is incorrect credentials
LoginError	Login to ADCM at the specified URL has failed
LogoutError	Logout from ADCM at the specified URL has failed
RetryRequestError	Request failed during the specified retry interval
ResponseDataConversionError	Invalid return type. Supported types: `dict` and `list`. Input data is not in valid JSON format.
BadRequestError	Request cannot be processed due to user error
UnauthorizedError	Request failed due to the lack of valid authentication credentials
PermissionDeniedError	Current user does not have permission to perform a request
NotFoundError	The object cannot be found
ConflictError	Request could not be completed due to a conflict with the current state of the server
ServerError	Server was unable to process the request
BadGatewayError	While processing the request, server received an invalid response from upstream server
ServiceUnavailableError	Server is unable to handle the request at this time due to a temporary overload or scheduled maintenance
AccessorError	Unspecified accessor error
MultipleObjectsReturnedError	More than one object found
ObjectDoesNotExistError	No objects found with the given filter
OperationError	Failed to download the bundle from the specified URL. HTTP error occurred.
HostNotInClusterError	Host is not added to the cluster, or it is not upgraded
ConfigComparisonError	The previous version of the schema.yaml specification file does not match the current file. Cannot refresh configuration after cluster upgrade: schema.yaml is different for local and remote servers.
ConfigNoParameterError	No configuration parameter with specified name in the tree of configuration settings
NoMappingInActionError	Selected action does not allow making changes to the mapping
NoConfigInActionError	Selected action does not allow making changes to configuration parameters
InvalidFilterError	Incorrect use of the `filter()` method. For example, invalid inline filter format or specifying an empty object collection

Found a mistake? Seleсt text and press Ctrl+Enter to report it