Bundle DSL

Mikhail Serov

This article describes the prototypes and the domain-specific language (DSL) that are used in configuration files for creating ADCM bundles. The format of configuration (config) file is YAML.

NOTE
The config file should be named config.yaml or config.yml. Multiple config.yaml files in a bundle are allowed. The name of the config file is reserved; other files should not be called by that name.

ADCM objects are created from prototypes. A prototype is a description of a single ADCM object.

Every prototype has the following mandatory properties:

name

name is a short name designed for use in Ansible scripts.

type

Every prototype has one of the following types:

  • cluster

  • service

  • provider (stands for hostprovider)

  • host

version

version is a version of the prototype.

So the minimal working prototype should look like this:

---
- type: cluster
  name: control
  version: 1

A prototype also might have the optional properties:

actions

An action is an entity that can be used to describe an action on an ADCM object. Such an action is ultimately an Ansible playbook run with certain parameters under certain conditions.

You can describe special actions to bind their invocation to specific endpoints and UI buttons. You can choose from one of the four special actions names:

  • adcm_turn_on_maintenance_mode

  • adcm_turn_off_maintenance_mode

  • adcm_host_turn_on_maintenance_mode

  • adcm_host_turn_off_maintenance_mode

Special actions are specified as follows:

  actions:
    adcm_turn_on_maintenance_mode

Special actions should be described in the matching prototypes.

Name Type of the prototype Description

adcm_turn_on_maintenance_mode

Service/component

Turns on maintenance mode on an object via the corresponding UI element

adcm_turn_off_maintenance_mode

Service/component

Turns off maintenance mode on an object via the corresponding UI element

adcm_host_turn_on_maintenance_mode

Cluster

Turns on maintenance mode on a cluster host via the corresponding UI element

adcm_host_turn_off_maintenance_mode

Cluster

Turns off maintenance mode on a cluster host via the corresponding UI element

adcm_delete_service

Service

Deletes service via the corresponding UI element

Special actions that are described in a cluster prototype (adcm_host_turn_on_maintenance_mode, adcm_host_turn_off_maintenance_mode) should be host actions (the value of the host_action parameter should be true). Special actions are not supported for config, hc_acl, and ui_options.

Example

 

Every prototype might have actions. Actions should be described as follows:

---
- type: cluster
  name: control
  version: 1
  description: "Monitoring and Control Software"

  actions:
    install:
      display_name: "Install Monitor Server"
      description: |
         By click on this button you install monitoring and controlling server ...
      type: job
      script_type: ansible
      script: ansible/site.yaml
      allow_to_terminate: true
      params:
        ansible_tags: install
        jinja2_native: true
      states:
        available:
          - created
        on_success: installed
        on_fail: created
      config:
        quorum:
          type: integer
      log_files:
        - check

The install action from the example above has a job type and its script property specifies an Ansible playbook that resides in the <bundle_root>/ansible/site.yaml directory.

You can use a special dot syntax to point out a path to a script file. In this case a script file will be searched in the same directory as config.yaml.

script: ./site.yaml
name

 

name is a short name that is unique for the given prototype.

display_name

 

display_name is a short human-readable name.

type

 

Action has a type statement. It defines the job and task clauses.

job is the type of action that starts a single script. Job is specified via the following statements:

  • script

  • script_type

  • params

task is the type of action that is a chain of jobs. The sequence of jobs to be executed is specified via the scripts parameter. The jobs sequence is executed according to how it is written in scripts. If any script fails, the execution is stopped and the state of object is set to the value of on_fail.

actions:
  install:
    type: task
    scripts:
      -
          name: prepare
          display_name: Prepare to install
          script_type: ansible
          script: ansible/prepare.yaml
          on_fail: created
      -
          name: install
          display_name: Actual install
          script_type: ansible
          script: ansible/install.yaml
          on_fail: prepare
    states:
      available:
        - created
        - prepare
      on_success: installed
script

 

script is a parameter that takes it value depending on the script_type value.

In case of script_type = ansible, the script parameter denotes the path to the script file.

In case of script_type = internal, the script parameter can take the value from one of the three ADCM functions:

  • bundle_downgrade — this function returns the state of the cluster as it was before the upgrade. Cluster state includes its configuration, the set of all cluster services, hosts, and components, as well as the host component mapping.

  • hc_apply — this function is useful when action includes hc_acl. In this case, the hc_apply function specifies the moment when the host component map received from hc_acl should be applied during cluster expansion or reduction operations.

  • bundle_switch — this function upgrades all prototypes to their new versions. The function can be used only for upgrade, it cannot be used in the common actions.

script is a mandatory parameter if type = job.

script_type

 

script_type is a parameter that denotes the type of the script, either ansible or internal.

script_type is a mandatory parameter if type = job.

scripts

 

scripts is a parameter that denotes the sequence of jobs to be executed.

scripts is a mandatory parameter if type = task.

params

 

You can also pass any additional parameter to the Ansible call. They will be used in ansible-playbook calls.

One special case is parameter ansible_tags. That parameter turns to be --tags parameter of an ansible-playbook call.

You can also pass the jinja2_native parameter. This parameter will be written to the ansible.cfg file. This option preserves variable types during template operations.

states

 

Action might have a states optional statement. It defines three clauses:

  • available — list of states of object for which action will be available;

  • on_success — state in which object will be set in case of actions success;

  • on_fail — state in which object will be set in case of actions fail.

The on_success and on_fail states are optional. If they are missed, the state of an object is not changed after actions success or/and fail, respectively.

The states statement also has two reserved values:

  • created — the primary state in which objects are immediately after creation;

  • ugrading —  the state to which the cluster can be put into. In this state, actions such as removing a host or service from the cluster are not available to the user.

masking

 

masking is a new DSL for state and multistate operation.

IMPORTANT
DSL masking is not compatible with the states DSL provided above. 
---
actions:
  install:
  display_name: "Install Monitor Server"
  type: job
  masking:
      # Action will be shown if both condition under state and multi_state are met.
      state:
          available:
              # If the state equal to any of this, then condition is true
              - "state_value1"
              - "state_value2"
          unavailable:
              # If you place unavailable, then no available should be there
              - "state_value1"
              - "state_value2"
      multi_state:
          available:
              # If we have any of this multistate, then condition is true
              - "multi_state_1"
              - "multi_state_2"
          unavailable:
              # If you place unavailable, then no available should be there
              - "multi_state_3"
              - "multi_state_4"
  on_fail:
      state: "new_sate_value"
      multi_state:
          set:
              - "multi_state3"
          unset:
              - "multi_state4"
  on_success:
      state: "new_sate_value"
      multi_state:
          set:
              - "multi_state3"
          unset:
              - "multi_state4"

To simplify the rule description, you could use the any scalar that selects all possible values.

---
  masking:
  state:
      available: "any"
      unavailable: "any"
  multi_state:
      available: "any"
      unavailable: "any"

If the masking state is omitted, than we think that the action is available on every state or multi_state. Hence the following variants are equal:

---
  masking:
---
  masking:
      state:
---
  masking:
      state:
          available: "any"

The following variants are equal too:

---
  masking:
---
  masking:
      multi_state:
---
  masking:
      multi_state:
          available: "any"
config

 

Action might have a config optional statement. It can define a list of values as in config parameters. When the action is run via UI, all those values will be prompted by the user before the action starts.

You can later refer those values in the Ansible script:

{{ job.config.quorum }}

The config statement works with static configuration parameters that are displayed the same way every time an action is started. If you want to work with dynamic configuration parameters, use the config_jinja statement.

config_jinja

 

Action might have a config_jinja optional statement. This statement points to the file of the jinja format (ADCM uses the 2.xx version).

config_jinja is designed to work with dynamic configuration parameters. In this case, dynamism is understood as the ability to display (or hide) a certain set of parameters depending on the conditions. Before the action starts, a file of the jinja format (jinja template) is rendered. The values generated as a result are similar to the values that are included in the inventory file.

---
actions:
  - name: Restart and Reboot
    type: job
    script_type: ansible
    script: ansible/restart.yaml
    config_jinja: config_jinja/restart.j2

The restart.j2 file:

- reboot_servers:
  type: boolean
  default: false
  display_name: Reboot servers
{%  if cluster.state == 'created' %}
- configure_etc:
  type: boolean
  default: false
  display_name: Configure /etc/hosts
{% end %}
log_files

 

Action might have a log_files optional statement. It defines the list of log tags. Currently, a check log tag is supported, and you can use the adcm_check Ansible module with it.

allow_to_terminate

 

Action might have an allow_to_terminate optional statement. This is a Boolean value. If the value is not explicitly set, then the default value is false. If allow_to_terminate is true, then it is possible to cancel the running Ansible playbook and stop the action, otherwise it is not possible.

The allow_to_terminate statement is also applied to all jobs the action consists of. When one of such jobs is stopped the action continues to run, starting with the job that follows the stopped one. Essentially, allow_to_terminate being specified for an action means allow_to_terminate being specified for all jobs of that action.

allow_in_maintenance_mode

 

Action might have an allow_in_maintenance_mode optional statement. This is a Boolean value. If the value is not explicitly set, then the default value is false. If allow_in_maintenance_mode is true and allow_maintenance_mode in the prototype is also true, then it is possible to run an action even in case of object or just one of its hosts being in maintenance mode.

ui_options

 

This block allows you to tweak how UI displays and performs an action. When you hover the cursor over an action in the UI, the popup message specified in the disclaimer field is displayed.

actions:
  install:
    type: job
    script_type: ansible
    script: ansible/site.yaml
    ui_options:
       disclaimer: "Enter your disclaimer"
host_action

 

Action might have a host_action optional statement. This is a Boolean value. If the value is not explicitly set, the default value is false. If the value is true, then the action is performed on the selected host and a target group that includes the one selected host is created.

actions:
  install:
    type: job
    script_type: ansible
    script: action.yaml
    host_action: true
hc_acl

 

Action might have a hc_acl optional statement. If hc_acl is present when an action is run via UI, then a new host component map will be asked from the user before the action starts.

actions:
  expand:
    type: job
    script_type: ansible
    script: ansible/site.yaml
    states:
      available: all
    hc_acl:
      -
        service: hadoop
        component: datanode
        action: add
      -
        service: hadoop
        component: server
        action: remove

hc_acl defines the list of 3 clauses that specify operations to modify the components distribution at hosts:

  • service — name of service to work with.

  • component — name of component to work with.

  • action — operation type. Can be add or remove.

venv

 

There is a way to specify the Ansible version for the ansible script type actions via the venv optional statement.

Valid values:

  • default — launch action in Ansible 2.8 environment;

  • 2.9 — launch action in Ansible 2.9 environment.

The default value is default.

Change the default environment for all actions on an object:

  ---
  - type: cluster # service component hostprovider host
    name: control
    version: 1

    venv: "2.9"

You also have a way to tweak venv for any actions:

actions:
  install:
    type: job
    script_type: ansible
    script: action.yaml
    venv: "2.9"

adcm_min_version

Any cluster or hostprovider prototype might have an optional adcm_min_version property. This field defines the minimal version of ADCM that is required for this bundle to work correctly. If the installed ADCM instance version is less than adcm_min_version (defined in bundle), then the load of this bundle will cause an error.

allow_flags

Every cluster prototype might have an optional allow_flags property. This option enables support of the flags for all prototypes in a bundle. The default value is false.

---
- type: cluster
  name: adpg
  allow_flags: true

allow_maintenance_mode

Every cluster prototype might have an optional allow_maintenance_mode property. This option enables support of the maintenance mode for all prototypes in a bundle. The default value is false.

---
- type: cluster
  name: control
  version: 1
  adcm_min_version: 2019.05.01.00
  allow_maintenance_mode: true

components

Service prototype might have components. Component itself is a prototype, so a component description also can contain config or action sections.

Example

 

components is a way of placing service components on hosts in a cluster.

---
- type: service
  name: server
  version: 1

  components:
    master:
      display_name: "Master Node"
      description: "This node control all data nodes (see below)"
      constraint: [1,2]
    node:
      display_name: "Data Node"
      constraint: [+]
    client:
      monitoring: passive
constraint

 

Every component might have an optional constraint property. It describes how many instances of this component should be installed in one cluster:

  • [1] — exactly one component should be installed.

  • [0,1] — one or zero components should be installed.

  • [1,2] — one or two components should be installed.

  • [0,+] — zero or any more components should be installed (default value).

  • [1,odd] — one or more components should be installed; the total amount should be odd.

  • [0,odd] — zero or more components should be installed; if more than zero, the total amount should be odd.

  • [odd] — same as [1,odd].

  • [1,+] — one or more components should be installed.

  • [+] — component should be installed on all hosts of a cluster.

monitoring

 

Every service/component might have an optional monitoring property. It defines if the service/component should be monitored. The default value is active. If you want to exclude a service/component from the monitoring system, set it to passive.

---
- type: service
  name: server
  version: 1

  components:
    client:
      monitoring: passive
requires

 

Components might have an optional requires property that describes dependencies between the components, i.e. which components need another components or services to be installed simultaneously.

Each component might require one or more components in the same service or in another service of this cluster. Also a component may need one or more services. If your component requires another component that is absent from a host component map, you cannot add your component to a host component map as well. In some cases all required components should be installed simultaneously. Dependencies defined at the component level affect the validation of the host component map.

Dependencies can be viewed as another type of constraints (or meta constraints) for a component.

The example below considers the Hive service with three components: HiveServer, Metastore, and TezUI. The HiveServer component needs the HDFS service, while the information about the HDFS service components is not required. The TezUI component needs the TimeLineServer component of the YARN service.

---
- type: service
  name: Hive
  version: 1.0

  components:
    HiveServer:
      requires:
        - service: HDFS
    Metastore:
      constraint: [0,1]
    TezUI:
      requires:
        - service: YARN
          component: TimeLineServer
      constraint: [0,1]
bound_to

 

Components might have an optional bound_to property. Those components should be located on the same hosts as the components they are bound to.

For example, the server component of the pxf service should be installed only on all those hosts where the segment components of the gpdb service are installed. It means the server component requires the segment component and cannot be installed separately.

Hence bound_to is yet another type of constraints (or meta constraints) for a component.

---
- type: service
  name: GPDB
  version: 1.0

  components:
    Segment:

- type: service
  name: PXF
  version: 2.0

  components:
    Server:
      bound_to:
        service: GPDB
        component: Segment

config

Config is a general description of object configuration parameters. Configuration parameters can be combined into groups.

Example

 

There is a number of user-editable values that are related to every object and can be represented in UI.

---
- type: cluster
  name: control
  version: 1
  description: "Monitoring and Control Software"

  config:
    - name: repos
      type: group
      subs:
         - name: ads
           display_name: "ADS repository URL"
           type: string
           default: https://ci.arenadata.io/artifactory/list/ads-centos7-x64-ads/
           required: no

In the above example you can see the ads parameter in the repos group. The ads parameter is not required. This means that it is possible for the user to leave it empty.

Group is not mandatory, so you can place your parameter at the highest level:

---
- type: cluster
  name: control
  version: 1
  description: "Monitoring and Control Software"

  config:
    - name: ads
      type: string
      default: https://ci.arenadata.io/artifactory/list/ads-centos7-x64-ads/
      required: no

Group is just a visual representation in UI.

Configuration parameters properties

 

Name Mandatory Values Default Description

type

Yes

See below

 — 

Type of a configuration parameter. For more information, see the separate Types of configuration parameters section below

display_name

No

User defined

 — 

A readable name that is displayed in UI

description

No

User defined

 — 

Allows to give the parameter a description

default

No

User defined

 — 

Allows to set a default value

required

No

yes/no

Yes

If set to yes, then it is required to have a non-empty and valid value for the variable

read_only

No

See below

No

Makes a parameter read-only (protected from change) under certain conditions

writable

No

See below

Yes

Makes a parameter writable (available for changes) under certain conditions

ui_options

No

See below

No

Allows to adjust the visual representation of a parameter in UI

group_customization

No

yes/no

No

Makes a parameter available to be changed via config groups

The ui_options parameter allows you to tweak how UI depicts config.

The following options are available:

  • Invisible

    Add invisible: true to your config to hide the config from UI.

    This option doesn’t hide information from API and doesn’t disable read_only/writable logic. This is simply an UI tweak.

    int:
  type: integer
  default: 1
  required: no
  ui_options:
    invisible: true

  • Advanced

    There is a new checkbox that allows you to hide some parameters via labeling them advanced.

    Parameter labeled as advanced is not searchable.

    float:
  type: float
  default: 1.0
  required: no
  ui_options:
    advanced: true

The group_customization parameter allows you to customize a parameter at the configuration group level.

---
 - name: ads
   type: string
   group_customization: true/false

The read_only parameter allows you to make any config parameter read-only (protected from change by the user or API) for the specified cluster/host/service states. The writable parameter removes the write protection.

The quorum parameter is read-only if a service state is installed or running:

config:
  quorum:
    type: integer
    read_only: [installed, running]

The quorum parameter is read-only for all service states except created:

config:
  quorum:
    type: integer
    writable: [created]

The quorum parameter is read-only for all service states:

config:
  quorum:
    type: integer
    default: 4
    read_only: any
Types of configuration parameters

 

  • boolean — boolean value: True or False.

  • file — the content of this property is stored as a file on a file system. When you reference this variable in an Ansible script, it returns a full path to that file.

    If the default value is set, it should be the path to the file relative to the bundle root directory.

    User can edit the content of this file via UI. The file is updated every time after config is saved.

    It can be used to store SSH private keys for an Ansible script:

    - type: host
  name: ssh
  version: 1

  config:
    ansible_private_key_file:
      type: file
      default: host/ssh.key

    You can use the special dot syntax to point out a path to the default file. In this case the default file is expected to be in the same directory as the config.yaml file:

    config:
  ansible_private_key_file:
    type: file
    default: ./ssh.key

  • float — float number. If the default value is not explicitly set, then it will be 0.0. You can use the optional min and max parameters to limit the values:

    config:
  average_load:
    type: float
    default: 1.0
    min: 0.5
    max: 3.5

  • integer — integer number. If the default value is not explicitly set, then it will be 0. You can use the optional min and max parameters to limit the values:

    config:
  memory_size:
    type: integer
    default: 16
    min: 2
    max: 64

  • json — this type stores arbitrary JSON data. The JSON structure is not preserved and can be modified during every config update:

    config:
  population:
    type: json
    default:
      -
        country: Russia
        cities:
          -
             name: Moscow
             population: 9
      -
        country: USA
        cities:
          -
             name: LA
             population: 5

  • list — dynamic array of arbitrary length. Array elements should be strings. Any element can be added, deleted, or modified on every config update:

    config:
  mount_points:
    type: list
    default:
      - /dev/rdisk0s1
      - /dev/rdisk0s2
      - /dev/rdisk0s3

  • map — dynamic key/value dictionary of arbitrary length. Keys and values should be strings. Any key or value can be added, deleted, or modified on every config update:

    config:
  person:
    type: map
    default:
      name: Joe
      age: "24"
      sex: m

  • option — key/value dictionary type. The keys are showed in UI while the corresponding values are stored in DB:

    config:
  protocol:
    type: option
    option: {http: 80, https: 443}
    default: 80

  • password — same as the string type, yet displayed as a password field in UI.

  • secretfile — obfuscates parameter and its value in UI.

  • secretmap — obfuscates the value that parameter takes in UI.

  • string — arbitrary string. The size is not limited. If the default value is not explicitly set, then it will be an empty string "".

  • structure — this type is intended for describing multi-level configurations. The number of levels is unlimited. Configurations are described according to the specification outlined below. Upon each configuration update, new data is compared against the specification. If the input data does not match the specification, ADCM issues an error.

    The root configuration element, which should be the first one described in the specification, is named root. For further attribute description, the match element is used, which can be equated to a data type. Possible values for match at this level include dict and list. Depending on the match value, attributes of the next level are added — items (for dict) or item (for list). Then, the composition of elements included in item or items is listed.

    You can see available attributes in the table below.

    Name Type Required Description

    match

    string

    Yes

    The rule according to which every other rule performs its checks

    item

    string

    Yes, for list type

    The rule according to which every object is checked in case of list type

    items

    key/value pairs

    Yes, for dict type

    The rule according to which every object is checked in case of dict type

    required_items

    list

    No

    List of mandatory object elements. Only for dict type

    invisible_items

    list

    No

    Contains elements that can be used for operational purposes and cannot be seen by the user

    default_item

    string

    No

    The rule according to which the user-created element in the object is additionally checked. Only for dict type

    You can view the available match values (element types) in the table below.

    Type Description

    string

    Simple type. Designed for strings

    boolean

    Simple type. Designed for boolean operations

    integer

    Simple type. Designed for integers

    float

    Simple type. Designed for floats

    list

    Recursive type. Checks itself first according to item, then applies rules to all its elements

    dict

    Recursive type. Checks itself first, then applies rules to its values according to default_item and required_items

    Below is a configuration example.

    config:
  - name: country_codes:
    type: structure
    yspec: codes/schema.yaml
    default:
      - country: Greece
        code: 30
      - country: France
        code: 33
      - country: Spain
        code: 34
        regions:
          - Andalusia
          - Andalusia
          - Cantabria
          - Catalonia
          - Valencia

    The schema.yaml specification file:

    ---
root:
  match: list
  item: country_code

country_code:
  match: dict
  items:
    country: some_string
    code: some_integer
    regions: list_of_string
    comment: some_string
    uid: some_string
  required_items:
    - code
    - country
  invisible_items:
    - uid

list_of_string:
  match: list
  item: some_string

some_string:
  match: string

some_integer:
  match: int

    You can use the dot syntax to point out a path to the specification file. In this case the specification file is expected to be in the same directory as the config.yaml file:

    yspec: ./schema.yaml

  • text — same as string, yet is displayed as multiline text in UI.

  • variant — a type where the user can select a single value from the list of options in UI. The sources for that can be different. You can get a list of variants from another config parameter (the type of this config parameter should be list):

    config:
  - name: mount
    type: variant
    source:
      type: config
      name: mount_points

  - name: mount_points
    type: list
    default:
      - /dev/rdisk0s1
      - /dev/rdisk0s2
      - /dev/rdisk0s3
    read_only: any
    ui_options:
      invisible: true

    Also you can get a list of variants from the built-in ADCM functions, such as cluster_hosts, that returns the list of all hosts in a current cluster:

    config:
  - name: cluster_host
    type: variant
    source:
      type: builtin
      name: host_in_cluster

    Built-in ADCM functions:

    • host_in_cluster — returns the list of all hosts in current cluster.

    • host_not_in_clusters — returns the list of all hosts not included in any cluster.

    • service_in_cluster — returns the list of all services installed in current cluster.

    • service_to_add — returns the list of all services not installed in current cluster.

    • host — receives arguments as predicate and args: predicate is a name of internal host function; args are arguments for this function. All internal functions return the list of hosts (the list can be empty):

      config:
  - name: vhost
    type: variant
    source:
       type: builtin
       name: host
       args:
         predicate: and
         args:
           - predicate: in_service
             args:
               service: UBER
           - predicate: or
             args:
               - predicate: in_component
                 args:
                   service: UBER
                   component: UBER_SERVER
               - predicate: in_cluster
                 args:

      List of internal host functions:

    • in_cluster — returns the list of all hosts in the current cluster;

    • in_service — returns the list of all hosts in the specified service;

    • not_in_service — returns the list of all hosts that are not in the specified service;

    • in_component — returns the list of all hosts in the specified component;

    • not_in_component — returns the list of all hosts that are not in the specified component;

    • in_hc — returns the list of all hosts in a host component map of this cluster;

    • not_in_hc — returns the list of all hosts that are not in a host component map of this cluster;

    • and — returns the union of sets of its arguments;

    • or — returns the intersection of sets of its arguments.

      You can list the variants simply in a bundle:

      config:
  - name: city
    type: variant
    source:
      type: inline
      strict: False
      value:
        - Moscow
        - Voronez
        - Ufa
    default: Samara

      You can also specify an optional argument strict for the variants source. If strict is set to False, then users can not only select the value from the predefined list, but also enter their own values in UI.

  • group — a type that allows you to combine configuration parameters into groups. Group can be activatable. This means that you can switch that group on and off via UI. When a group is switched off (active = false), you don’t need to fill in its subvalues in UI. Group value in the Ansible inventory file would be null as well.

    ---
- type: cluster
  name: Kafka
  version: 1

  config:
     - name: grafana
       type: group
       activatable: true
       active: false
       subs:
          - name: endpoint
            type: string
          - name: port
            type: integer

config_group_customization

Every cluster prototype might have an optional config_group_customization property. This property allows the user to customize all cluster (service, component) parameters on some group of hosts. This functionality is disabled by default. The default value is false.

---
- type: cluster # service component
  name: control
  version: 1

config_group_customization: true

If you want to change the behaviour of a certain parameter, you can use the group_customization option of the corresponding configuration parameter.

description

description is a detailed description of your prototype.

---
- type: cluster
  name: control
  version: 1
  display_name: "Controlling Software"
  description: |
     This software is intended for monitoring and controlling
     you cluster and does it by ...

display_name

display_name is a short human-readable name.

---
- type: cluster
  name: control
  version: 1
  display_name: "Controlling Software"
  description: |
     This software is intended for monitoring and controlling
     you cluster and does it by ...

edition

Every cluster or hostprovider prototype might have an optional edition property that corresponds to the bundle type (cluster or host, respectively).

Default value of edition is community. Value of edition can be used in upgrade definition.

---
- type: cluster
  name: adh
  display_name: Arenadata Hadoop
  version: 1
  edition: enterprise

export

Export is a way to specify settings that a cluster or service is able to transfer to other clusters.

Example

 

If you want to share some parts of your config with other clusters, you can use the export statement. In export you can specify any number of the first level config entries. You can export a config in a cluster or/and in a service:

---
- type: service
  name: Hadoop
  version: 2.1

  config:
     core-site:
        param1:
           type: string
        param2:
           type: integer
     quorum:
        type: integer
        default: 3

  export:
      - core-site
      - quorum

import

Import is a way to specify service settings and versions for a cluster or service to be able to receive them from other clusters.

Example

 

To use the config parts exported from other cluster, you should use the import statement in your cluster. You should specify the name and acceptable versions of an exported cluster (or a service in a cluster):

---
-
  type: cluster
  name: Hive
  version: 4

  import:
     Hadoop:
        versions:
           min: 1.8
           max: 2.5

When you bind your cluster and another cluster or service via UI, you can use the exported config parameters in your Ansible scripts like this:

{{  cluster.import.Hadoop.core-site.param2  }}
{{  cluster.import.Hadoop.quorum  }}
Optional import parameters

 

  • required — the import statement might have an optional required parameter. If required is set to true the cluster or service will raise issue and block any action before the required import is properly binded with export cluster or service. The default value for required is false.

  • default — the import statement might have an optional default parameter. The default parameter should be an array of config group names of this cluster or service. If this cluster/service is not binded with any export cluster/service import section, the Ansible inventory file will be filled with values from the specified config group.

    It’s recommended to make default group names the same as export group names.

  • multibind — import statement might have an optional multibind parameter. If it is set to true, you can bind many instances of a cluster or a service to the same cluster/service. In Ansible inventory file the import values are represented as an array. The default value of multibind is false.

license

Every cluster, service, or hostprovider prototype might have an optional license property that corresponds to the bundle type. The value of a license property is a path to the licence file relative to a bundle root.

If a prototype of a cluster or a service has a license property, a user of ADCM will be asked to accept a license after the bundle is loaded or the service is added. Before the user accepts a license any action with the bundle or service is blocked.

---
- type: cluster
  name: adh
  display_name: Arenadata Hadoop
  version: 1
  license: misc/license.txt
---
- type: service
  name: hdfs
  display_name: HDFS
  version: 1
  license: misc/license.txt
NOTE

You can use a special dot syntax to point out a path to a license file. In this case a license file will be searched in the directory that contains config.yaml:

license: ./license.txt

monitoring

Every service or component prototype might have an optional monitoring property. It defines if the service/component should be monitored. Default value is active. If you want to exclude service/component from monitoring system, set it to passive.

required

Every service prototype might have an optional required property. Default value is false. If cluster cannot work without this service, set it to true. When required service is not added to cluster, cluster will have an issue.

requires

Every service prototype might have an optional requires property.

This parameter defines dependencies between the services, i.e. which services need other services or components to be installed simultaneously.

By default a service does not depend on any other service or component. However, if a dependent service is specified for a service, then when adding that service without the dependent service, an issue is generated in ADCM. Thus the dependency of a service on another service is checked at the stage of adding it. The dependency of a service on a component is checked at the stage of the host-component mapping being saved.

---
- type: service
  name: Hive
  version: 1.0
  requires:
    - service: HDFS
    - service: YARN
      component: TimeLineServer

upgrade

Upgrade is an operation available from a particular state of an object (cluster or hostprovider) that updates the meta-information about that object and moves it to another state. There are two types of upgrades:

  • Switching the prototypes of the current bundle to the prototypes of the new one.

  • Performing some kind of action that may consist of several subactions (including switching the prototypes).

Example

 

Cluster or host prototype might have an upgrade statement. This statement describes how to perform an upgrade from one version of bundle to another. A prototype might have a few upgrade items to describe upgrades from different versions.

---
- type: cluster
  name: control
  version: 2.4
  description: "Monitoring and Control Software"

  upgrade:
     -
        name: Upgrade 1
        versions:
           min: 0.4
           max: 2.0
        description: New cool upgrade
        states:
           available: any
           on_success: upgradable
     -
        name: Upgrade 2
        versions:
           min: 1.0
           max_strict: 3.0
        scripts:
          - name: pre check
            display_name: Some pre-check before upgrade prototypes
            script_type: ansible
            script: ansible/pre-check.yaml
          - name: bundle upgrade
            display_name: Upgrade bundle
            script_type: internal
            script: bundle_switch
          - name: post tasks
            display_name: Some operation after prototype upgrades
            script_type: ansible
            script: ansible/post-task.yaml
        from_edition: community
        states:
           available:
	     - installed
           on_success: upgradable
versions

 

upgrade should have a versions property. versions is the interval of the versions appropriate for upgrade.

upgrade:
   -
      name: Upgrade
      versions:
         min: 0.4
         max: 2.0
      states:
         available: any
         on_success: upgradable
NOTE
min and max use non-strict comparison (i.e. boundaries include the version of cluster/host). If you want the comparison to be strict, you can use the min_strict and/or the max_strict instead of the min/max.
name

 

upgrade should have a name property. name is the object name.

display_name

 

upgrade should have a display_name property. display_name is the object name that is shown to user via UI.

description

 

upgrade might have an optional description property. description allows you to write description for an upgrade.

states

 

upgrade should have a states property. states defines two clauses:

  • available — lists the cluster states that allow upgrade. Special value any can be used for upgrade that is available in any state.

  • on_success — cluster state after upgrade.

upgrade:
   -
      name: Upgrade 1
      versions:
         min: 0.4
         max: 2.0
      description: New cool upgrade
      states:
         available: [created, installed]
         on_success: upgradable
from_edition

 

upgrade might have an optional from_edition property. from_edition is the list of editions that allow upgrades.

Special value any can be used to specify the permission to upgrade from bundles with any edition. The default value of from_edition is community.

upgrade:
   -
      name: Upgrade 1
      versions:
         min: 0.4
         max: 2.0
      description: New cool upgrade
      states:
         available: [created, installed]
         on_success: upgradable
      from_edition:
         - community
         - enterprise
Found a mistake? Seleсt text and press Ctrl+Enter to report it