Prototypes overview

Mikhail Serov, Adelya Zvyagintseva

A prototype is a description of a single ADCM object. Several prototypes that belong to the corresponding bundle type can be described in one bundle.

Each prototype contains the following mandatory properties:

  • name — internal prototype name that is used in an inventory file of a product or infrastructure bundle.

  • type — type of the prototype or object. Possible values:

    • for a product bundle:

      • cluster

      • service

    • for an infrastructure bundle:

      • provider

      • host

  • version — prototype version (cluster, service, or hostprovider). Not applicable to components and hosts.

  • display_name — object name displayed in the web interface (for example, Hadoop). The difference of this property from the name property is that display_name is used to integrate with ADCM, update objects, and more.

  • description — detailed description of the prototype.

Thus, the simple prototype is as follows:

---
- type: cluster
  name: control
  version: 1
  display_name: "Controlling Software"
  description: |
     This software is intended to monitor and control
     you cluster and does it by ...

Also, depending on the object type, the prototype may contain optional properties described below.

Common properties

Each prototype of a cluster, service, component, hostprovider, or host can include the following properties:

  • actions and config

  • flag_autogeneration

  • venv

actions

The actions property defines the actions on the object that are available to the user. Executing an action creates a set of jobs designed to run the corresponding Ansible playbook with specified parameters and conditions. Detailed description of the property is given in the actions article.

config

The config property represents the description of an object configuration parameters. Description of this property, as well as the available types of configuration parameters, is provided in the config article.

flag_autogeneration

NOTE
Some of this attribute’s functionality was represented by the allow_flags property prior to the 2.2.0 release. The allow_flags property is no longer supported.

This property is designed to control (enable or disable) automatic creation of the embedded flags. The example of using the property to describe a cluster prototype is given below.

If flag_autogeneration is set for the parent and child object, then the state of the child object property will have a higher priority.

The property can be enabled or disabled for the prototype of any object. This means that flag_autogeneration can be disabled for a specific service even if the property is enabled at the cluster level (therefore all services of the cluster inherit the cluster property state).

Currently, only the adcm_outdated_config embedded flag is supported. This flag is raised automatically when the object configuration is changed. To enable automatic creation of this embedded flag when the object configuration is modified, set the enable_outdated_config property to True.

---
- type: cluster
  name: adpg
  flag_autogeneration:
    enable_outdated_config: True

venv

The venv property defines the Ansible version that will be used when launching an action of the corresponding prototype.

Possible values:

  • default — use Ansible 2.8 when launching an action (contract version 1.0 only);

  • 2.9 — use Ansible 2.9 when launching an action;

  • 2.16 — use Ansible 2.16 when launching an action (starting with ADCM 2.8.0).

The default value is default.

NOTE
With the release of ADCM 2.8.0, Ansible 2.8 is deprecated and will be removed from ADCM in Q3 2026. 
  ---
  - type: cluster # service component hostprovider host
    name: control
    version: 1

    venv: "2.9"

Cluster prototype

A cluster is a set of services logically combined into one product.

Features of working with a cluster:

  • A cluster has a defined upgrade procedure that allows transition of a cluster from one version to another.

  • Configuration can be defined for a cluster.

  • The import mechanism is designed to transfer metadata between clusters (services), namely specially marked configuration. This approach allows clusters (services) to be described for reuse in other clusters (services).

    IMPORTANT
    Importing services and clusters does not involve their physical transfer.

A cluster prototype can be defined in a bundle as a single instance.

In addition to the mandatory and common properties, each cluster prototype may also include the following optional properties:

adcm_min_version

The adcm_min_version property defines the minimal version of ADCM that is required for this bundle to work correctly. If the installed ADCM instance version is less than adcm_min_version (defined in a bundle), then uploading of such bundle will cause an error.

allow_maintenance_mode

The allow_maintenance_mode property enables the maintenance mode for all prototypes in a bundle. The default value is false.

Note that this property supports the inheritance mechanism. This means that when it is defined at the cluster prototype level, the corresponding value is automatically applied to the services and components included in the bundle. For a specific child object (service or component), the property value can be overridden — in this case, the value set at the child object level has a higher priority than the value inherited from the cluster.

Consider that when a service, component, or host is in maintenance mode, the set of actions available to the user for that object is limited. Therefore, this property should be considered together with the allow_in_maintenance_mode property.

---
- type: cluster
  name: control
  version: 1
  adcm_min_version: 2019.05.01.00
  allow_maintenance_mode: true

config_group_customization

The config_group_customization property enables support for configuration parameters at the config group level. By default, this functionality is disabled. The default value is false.

NOTE
The property can be defined for cluster, service, component, and hostprovider prototypes. This property is not supported for host prototypes.

The config_group_customization property allows enabling config group functionality for all configuration parameters of a prototype.

The property supports an inheritance mechanism: if it is set at the cluster level, the corresponding value is automatically applied to the services and components of that bundle. For a specific service or component, the property value can be overridden — in this case, value specified for the child object has higher priority.

---
- type: cluster # service component
  name: control
  version: 1

config_group_customization: true

If you want to change the behavior of a certain parameter, you can use the group_customization option of the corresponding configuration parameter.

contract_version

The contract_version property defines the bundle contract version. A contract is understood as the versioned config.y(a)ml configuration file that defines the available functions, behavior specifics, and compatibility guarantees required for the bundle to operate correctly.

---
- type: cluster
  name: ado
  display_name: ADO
  description: Arenadata Orchestrator
  version: "{{ cluster_version }}"
  contract_version: 2.0
  edition: community
  license: EULA.txt
  adcm_min_version: 2.5.0
  env: 2.9
  allow_maintenance_mode: true
NOTE
If the contract_version property is not specified in config.y(a)ml, the 1.0 value is applied automatically.

The contract version is used to ensure compatibility between bundles and ADCM and serves the following purposes:

  • validating installed and uploaded bundles before and during the ADCM update;

  • preventing the ADCM updates that could disrupt the operation of already installed bundles;

  • proactively notifying users about bundles that are being deprecated.

The contract version follows the semantic versioning policy:

  • major version — introduces incompatible changes (for example, removal of deprecated features or execution mechanisms);

  • minor version — contains changes to the bundle loading behavior while maintaining backward compatibility (for example, adding inheritance of a property from a cluster to its services).

Each ADCM release defines the following:

  • the list of supported contract versions;

  • the list of deprecated contract versions.

Bundle compatibility checks based on the contract version are performed during ADCM startup and upgrade. If incompatible bundles are detected, the operation is aborted.

edition

The property corresponding to the bundle type defines the object edition. The default value is community. The edition property can also be used to specify rules that determine the availability of upgrade operations for clusters and hostproviders.

---
- type: cluster
  name: adh
  display_name: Arenadata Hadoop
  version: 1
  edition: enterprise

export

Export is a way to specify settings that a cluster or service is able to transfer to other clusters.

If a user binds a cluster to another cluster or service using the web interface, they will appear in the inventory file, which means you can use the exported configuration parameters in an Ansible playbook as follows:

---
- type: service
  name: Hadoop
  version: 2.1

  config:
     core-site:
        param1:
           type: string
        param2:
           type: integer
     quorum:
        type: integer
        default: 3

  export:
      - core-site
      - quorum

import

Import is a way to specify service settings and versions for a cluster or service to be able to receive them from other clusters.

To import parts of config exported from another cluster, use the import property. You can specify the name and allowed versions of the exported cluster (or service within the cluster):

---
-
  type: cluster
  name: Hive
  version: 4

  import:
     Hadoop:
        versions:
           min: 1.8
           max: 2.5

If a user binds a cluster to another cluster or service using the web interface, you can use the exported configuration parameters in an Ansible playbook as follows:

{{  cluster.import.Hadoop.core-site.param2  }}
{{  cluster.import.Hadoop.quorum  }}

Keep in mind that the user may not import the required cluster or service, so you need to handle scenarios where the imported parameters are missing in the inventory file. As an alternative, you can use the default import property or make the import mandatory by applying the appropriate settings shown in the table below.

Property Default value Description

required

false

If set to true, the cluster or service will create an issue and block actions until the user explicitly imports the required settings of the cluster or service

default

 — 

An array of names from the cluster or service config group. If the cluster or service is not bound to an import section for the exported cluster or service, the Ansible inventory file will be filled with values from the specified config group

multibind

false

If set to true, the user can import multiple clusters or services into the current cluster or service

license

The license property is specified when it is necessary to display a user agreement when using a cluster, hostprovider, or service. The value specifies the path to the license file relative to the root directory of the bundle.

Below are two examples of usage:

  • for a cluster prototype:

    ---
- type: cluster
  name: adh
  display_name: Arenadata Hadoop
  version: 1
  license: misc/license.txt

  • for a service prototype:

    ---
- type: service
  name: hdfs
  display_name: HDFS
  version: 1
  license: misc/license.txt

If a cluster or service prototype contains the license property, the user will be prompted to accept the license agreement after the bundle is uploaded or before adding the service. Until the license agreement is accepted, all further actions with the bundle or service will be blocked.

NOTE

You can use a special dot syntax to specify a path to a license file. In this case, a license file will be searched in the directory that contains config.yaml:

license: ./license.txt

upgrade

The upgrade property allows you to initiate a cluster or hostprovider upgrade, during which the object metadata is updated and the object is transitioned to a new state. A detailed description of this property is provided in the upgrade article.

Service prototype

A service represents a set of components that are defined in the product bundle and cannot function in isolation.

In addition to the mandatory and common properties, each service prototype may also include the following optional properties:

Descriptions of the allow_maintenance_mode, config_group_customization, export, import, and license properties are provided in the Cluster prototype section.

components

A service prototype might have components. A component itself is a prototype, so the component description can also contain sections such as config and action.

components is a way to place service components on hosts in a cluster.

---
- type: service
  name: server
  version: 1

  components:
    master:
      display_name: "Master Node"
      description: "This node control all data nodes (see below)"
      constraint: [1,2]
    node:
      display_name: "Data Node"
      constraint: [+]
    client:
      monitoring: passive

constraint

Every component might have an optional constraint property. It describes how many instances of this component should be installed in one cluster:

  • [1] — exactly one component should be installed.

  • [0,1] — one or zero components should be installed.

  • [1,2] — one or two components should be installed.

  • [0,+] — zero or more components should be installed (default value).

  • [1,odd] — one or more components should be installed, and the total amount should be odd.

  • [0,odd] — zero or more components should be installed; if the number of components is greater than zero, the total should be odd.

  • [odd] — same as [1,odd].

  • [1,+] — one or more components should be installed.

  • [+] — component should be installed on all hosts of a cluster.

NOTE
Using the [+] constraint may lead to incorrect cluster behavior when adding a new host to an already deployed cluster, since a component with this constraint should be installed on all cluster hosts.

monitoring

Every service or component might have an optional monitoring property. This property determines whether the heartbeat of a service or component should be taken into account when calculating aggregated status. The default value is active. To exclude a service or component from the status calculation, set the value to passive.

---
- type: service
  name: server
  version: 1

  components:
    client:
      monitoring: passive

requires

Components might have an optional requires property. This property describes dependencies between the components — which components need other components or services to be installed simultaneously.

Each component might require one or more components of the same or another service in this cluster. Also, a component may depend on one or more services. If a component depends on another component that is not present in the host-component map, you cannot add this component to the host-component mapping either. In some cases, all required components should be installed simultaneously. Dependencies defined at the component level affect validation of the host-component map.

Dependencies should be considered as another type of constraint or meta-constraint for a component.

The example below considers the Hive service with three components: HiveServer, Metastore, and TezUI. The HiveServer component depends on the HDFS service without requiring information about the components of the HDFS service. The TezUI component requires the TimeLineServer component of the YARN service.

---
- type: service
  name: Hive
  version: 1.0

  components:
    HiveServer:
      requires:
        - service: HDFS
    Metastore:
      constraint: [0,1]
    TezUI:
      requires:
        - service: YARN
          component: TimeLineServer
      constraint: [0,1]

bound_to

IMPORTANT
The bound_to property is not supported since the 2.0.0 version.

monitoring

Every service or component prototype might have an optional monitoring property. This property determines whether the heartbeat of a service or component should be taken into account when calculating aggregated status. The default value is active. To exclude a service or component from the status calculation, set the value to passive.

required

The required property is used to indicate whether a service is mandatory for the correct operation of a cluster. The default value is false. If the cluster cannot operate without this service, set the value to true. In this case, the absence of the service in the cluster will result in an error.

requires

The requires property is used to specify mandatory service dependencies — it indicates which other services or components should be installed with this service for proper operation.

By default, a service is considered independent and does not require any other services or components. If a service is specified in requires as a dependency of the current service, adding it in ADCM without the required dependent service will generate an issue. So dependencies between services are validated when the service is added.

Dependencies between a service and its components are validated independently during host-component mapping.

---
- type: service
  name: Hive
  version: 1.0
  requires:
    - service: HDFS
    - service: YARN
      component: TimeLineServer

Hostprovider prototype

In addition to the mandatory and common properties, each hostprovider prototype may also include the following optional properties:

Descriptions of the properties are provided in the Cluster prototype section.

Found a mistake? Seleсt text and press Ctrl+Enter to report it