API ### Getting Start ============= There are two wasy to interact with ADSS Core API: * Use official Python Client * Genereate your our client based on OpenApi Schema v3 (also known as Swagger) Official Python Client ---------------------- TBD Client generation ----------------- Install ADSS, and go to /api/v1/schema/. On this endpoint you will find a JSON with schema description which could be used for client generation by your tools for your language. Find a tool for you at the following link: https://openapi.tools/ API Standarts ============= Structure and operation ----------------------- API should be unified in look from endpoint to enpoint. API should be flat. All endpoints should be on the same level. API backend accepts data in "application/json" format; "multipart/form-data" format also supported by used framework, but is not recommended to use, neither supported by development team. Endpoint ~~~~~~~~ Every endpoint should have following suburls and operations (classic REST) * /endpoint/ * GET - List all objects * POST - Create new object * /endpoint// * GET - Read object according to pk_id * PATCH - Update object partitialy * PUT - Replace object * DELETE - Delete object HTTP Methods ~~~~~~~~~~~~ .. _http-list: GET on /endpoint/ ^^^^^^^^^^^^^^^^^ Also known as :term:`LIST`. This operation return all objects of same kind. For example all *Clusters* or something else. Result of this request is a JSON of following format: .. code-block:: json { "count": 2, "next": "http://127.0.0.1:8006/api/v1/cluster-type/?limit=1&offset=1", "previous": null, "results": [ { "id": 1, "name": "dummy-foxpro", "display_name": "Dummy FoxPro", "description": "FoxPro is a text-based procedurally oriented DBMS", "short_description": "FoxPro database", "url": "http://127.0.0.1:8006/api/v1/cluster-type/1/" } ] } There is a two distinct but related blocks here. .. _`DRF Pagination`: https://www.django-rest-framework.org/api-guide/pagination/ First one is a standart `DRF Pagination`_ block. .. code-block:: json { "count": 2, "next": "http://127.0.0.1:8006/api/v1/cluster-type/?limit=1&offset=1", "previous": null } Second one is a list of objects: .. code-block:: json { "results": [ { "id": 1, "name": "dummy-foxpro", "display_name": "Dummy FoxPro", "description": "FoxPro is a text-based procedurally oriented DBMS", "short_description": "FoxPro database", "url": "http://127.0.0.1:8006/api/v1/cluster-type/1/" } ] } Every object in the list should conform following logic: * Every object has *id* field which is unique, positive, integer primary key (PK). * Every object has *url* field to full object. See :ref:`http-read` * There are **no** :ref:`api-huge-attrs` here * There are **no** links to related objects. .. _http-read: GET on /endpoint// ^^^^^^^^^^^^^^^^^^^^^^ Also known as :term:`READ`. This one returns one object in form of JSON: .. code-block:: json { "id": 1, "name": "Some cluster", "display_name": "sdfa", "connection": {}, "cluster_type": { "id": 2, "url": "http://127.0.0.1:8006/api/v1/cluster-type/2/" } } Respons should be conform to following rules: * Every object has *id* field which is unique, positive, integer primary key (PK). * Every object has *url* field to full object. See :ref:`http-read` * There are :ref:`api-huge-attrs` here * There are links to related objects. .. _http-post: POST on /endpoint/ ^^^^^^^^^^^^^^^^^^ This one creates object. Content should conform following rules: * There is no id, or url parts here. * All related objects (Foreign keys) are sent as nested object with the only required field *id* (other fields are ignored, so, nested object could be GETed from API as-is). * It is allowed to send no field if you want to apply default value. See :ref:`field-property-default` * If POSTable for field is *False* then it is not allowed to pass this to POST operation. See :ref:`field-property-postable` In case of operation success we should have the same output as for :ref:`http-read` on the new object. .. _http-put: PUT on /endpoint// ^^^^^^^^^^^^^^^^^^^^^^ This one to update whole object. Content should conform following rules: * All :ref:`field-property-required` fields should be sent in request body * id should not be changed * All related objects (Foreign keys) are sent as nested object with the only required field *id* (other fields are ignored). * It is allowed to send fields with *Changeable* == *False*, but they should be send without changes. See :ref:`field-property-changeable` for more information. In case of operation success we should have the same output as for :ref:`http-read` on this object. .. _http-patch: PATCH on /endpoint// ^^^^^^^^^^^^^^^^^^^^^^^^ This one to update some fields in object. Content should conform following rules: * id should not be changed, or should not be sent * All related objects (Foreign keys) are sent as nested object with the only required field *id* (other fields are ignored). * It is allowed to send fields with *Changeable* == *False*, but they should be send without changes. See :ref:`field-property-changeable` for more information. In case of operation success we should have the same output as for :ref:`http-read` on this object. .. _flex-fields: FlexFields ~~~~~~~~~~ API supports retrieving fields absent in default object representation through extending url with query strings. For example, you can list clusters by requesting URL http://127.0.0.1/api/v1/cluster/ and get bunch of objects like this: .. code-block:: json { "id": 1, "url": "http://127.0.0.1/api/v1/cluster/1/", "name": "cluster_name", "display_name": "Cluster Name" } For more info you could retrieve details for each object by its url (here it's http://127.0.0.1/api/v1/cluster/1/): .. code-block:: json { "id": 1, "name": "cluster_name", "display_name": "Cluster Name", "connection": { "some_data": "..." }, "cluster_type": { "id": 3, "url": "http://127.0.0.1/api/v1/cluster-type/3/" }, "cluster_capacity": [ { "id": 1, "url": "http://127.0.0.1/api/v1/cluster-capacity/1/" }, { "id": 2, "url": "http://127.0.0.1/api/v1/cluster-capacity/2/" }, { "id": 3, "url": "http://127.0.0.1/api/v1/cluster-capacity/3/" }, { "id": 4, "url": "http://127.0.0.1/api/v1/cluster-capacity/4/" } ] } If list representation has not enough information and requesting details for each list item seems cumbersome, you may expand returned field-set like that - http://127.0.0.1/api/v1/cluster/?expand=type and also get info about cluster's type: .. code-block:: json { "id": 1, "url": "http://127.0.0.1/api/v1/cluster/1/", "name": "cluster_name", "display_name": "Cluster Name", "cluster_type": { "id": 3, "url": "http://127.0.0.1/api/v1/cluster-type/3/", "name": "ADB", "display_name": "Arenadata DB cluster" } } Expanded fields obeys same rules as parent objects, see :ref:`http-list`, :ref:`http-read` There is even more possibilities: - Expanding available for any depth of nesting (e.g. http://127.0.0.1/api/v1/cron-line/1/?expand=cluster.capacity.type). - Expanding available for any number of fields (e.g. http://127.0.0.1/api/v1/cron-line/1/?expand=cluster,handler). - You can select fields to include in response, omitting others (e.g. http://127.0.0.1/api/v1/cluster/?fields=id,name). - You can select fields to exclude from response (e.g. http://127.0.0.1/api/v1/cluster/?omit=url). - You can combine all above (e.g. http://127.0.0.1/api/v1/cluster/?fields=name,type&expand=type). - Although, omitting explicitly included or expanding not included fields is pointless, you allowed to do that anyway without receiving errors, the same way as including/expanding/omitting non-existing fields in query. Filtering ~~~~~~~~~ API supports filtering through extending url with query strings. Filtering is supported for all fields except :ref:`api-huge-attrs`. For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/ and get bunch of objects like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster_name_1", "display_name": "Cluster name 1" }, { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster_name_2", "display_name": "Cluster name 2" } ] To filter by the **name** field, use the request: http://127.0.0.1:8000/api/v1/cluster/?name=cluster_name_1 As a result, you will receive a response like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster_name_1", "display_name": "Cluster name 1" } ] You can also filter by nested objects, use :ref:`flex-fields`. For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/?expand=type and get bunch of objects like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster 1", "display_name": "Cluster name 1", "cluster_type": { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster-type/1/", "name": "dummy-foxpro", "display_name": "Dummy FoxPro" } }, { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster 2", "display_name": "Cluster name 2", "cluster_type": { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster-type/2/", "name": "dummy-dbase", "display_name": "Dummy dBase" } } ] To filter by the **id** field of the nested object **type** use the query: http://127.0.0.1:8000/api/v1/cluster/?expand=type&type__id=1 As a result, you will receive a response like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster 1", "display_name": "Cluster name 1", "cluster_type": { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster-type/1/", "name": "dummy-foxpro", "display_name": "Dummy FoxPro" } } ] Supported lookups: +---------+-----------------------+-------------------+ | lookups | format | description | +=========+=======================+===================+ | exact | ?= | exact match check | +---------+-----------------------+-------------------+ Types that support lookups: +---------+---------+ | type | lookups | +=========+=========+ | string | exact | +---------+---------+ | integer | exact | +---------+---------+ Ordering ~~~~~~~~ API supports ordering through extending url with query strings. Ordering is supported for all fields except :ref:`api-huge-attrs`. For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/ and get bunch of objects like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster_name_1", "display_name": "Cluster name 1" }, { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster_name_2", "display_name": "Cluster name 2" } ] To order in ascending order **name** field, use the query: http://127.0.0.1:8000/api/v1/cluster/?ordering=name As a result, you will receive a response like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster_name_1", "display_name": "Cluster name 1" }, { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster_name_2", "display_name": "Cluster name 2" } ] To order in descending order **name** field, use the query: http://127.0.0.1:8000/api/v1/cluster/?ordering=-name As a result, you will receive a response like this: .. code-block:: json [ { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster_name_2", "display_name": "Cluster name 2" }, { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster_name_1", "display_name": "Cluster name 1" } ] You can also ordering by nested objects, use :ref:`flex-fields`. For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/?expand=type and get bunch of objects like this: .. code-block:: json [ { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster 1", "display_name": "Cluster name 1", "cluster_type": { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster-type/1/", "name": "dummy-foxpro", "display_name": "Dummy FoxPro" } }, { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster 2", "display_name": "Cluster name 2", "cluster_type": { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster-type/2/", "name": "dummy-dbase", "display_name": "Dummy dBase" } } ] To order in descending order **id** field of the nested object **type**, use the query: http://127.0.0.1:8000/api/v1/cluster/?expand=type&ordering=type__id As a result, you will receive a response like this: .. code-block:: json [ { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster/2/", "name": "cluster 2", "display_name": "Cluster name 2", "cluster_type": { "id": 2, "url": "http://127.0.0.1:8000/api/v1/cluster-type/2/", "name": "dummy-dbase", "display_name": "Dummy dBase" } }, { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster/1/", "name": "cluster 1", "display_name": "Cluster name 1", "cluster_type": { "id": 1, "url": "http://127.0.0.1:8000/api/v1/cluster-type/1/", "name": "dummy-foxpro", "display_name": "Dummy FoxPro" } } ] Ordering format: +------------+-------------------------+ | ordering | format | +============+=========================+ | ascending | ?ordering= | +------------+-------------------------+ | descending | ?ordering=- | +------------+-------------------------+ .. _errors: Errors ~~~~~~ All errors, returned by back-end has different status codes as usual and JSON bodies like this: .. code-block:: json { "code": "status code description", "level": "warning, error or critical", "desc": "Short error description", "details": { "more": "error details" } } If request to API breaks more than one rule (e.g. *invalid field value* and *inconsistent combination of values* and *changing non-changeable field*), API returns some *single* error, which one - framework decides by its internal logic. Object's Fields --------------- This block of documentation describes the way end user interact with fields over API in depends of field's properties and types. Field Types ~~~~~~~~~~~ There are a number of data types allowed for Fields. Most of them are simple, obvious from their description, but some of them have to be explained. .. _`Many-to-many relation`: https://docs.djangoproject.com/en/3.1/topics/db/examples/many_to_many/ .. _`Many-to-one relation`: https://docs.djangoproject.com/en/3.1/topics/db/examples/many_to_one/ .. _`One-to-one relation`: https://docs.djangoproject.com/en/3.1/topics/db/examples/one_to_one/ Full list of field types: ========== =========== Name Description ========== =========== id primary key for object. url url of object in API. See :ref:`http-read`. integer positive unsigned integer, max 2^63 - 1. datetime ISO format string representation of DateTime string No \\n allowed. Object names limited to 255 chars, other fields - 1024, if not specified other. text 1MB max length string with \\n allowed json 1MB max json field FK Simple foreign key for `Many-to-one relation`_. O2O FK Foreign key for `One-to-one relation`_. M2M FK Foreign key `Many-to-many relation`_. ========== =========== .. _api-huge-attrs: Huge Fields ^^^^^^^^^^^ Some fields of models could be comparatively huge, so operation with such attrs should be limited. List of huge attrs * Texts (multiline strings, which could be a huge) * JSONFields Display names ^^^^^^^^^^^^^ There is two conflicting ideas behind a property "name" for various objects: * name is short alphanum string which is useful for internal use. * name is a human readable information about object to be shown in UI or logs or somewhere else. That is why we have display_name in every object which has name. If for some reason display_name is absent we should have present name instead of display_name. As it should be done everywhere, we could do that trick close to DB in ORM layer. To diminish mistakes when using GUI we apply unique constraint on "name" and "display_name" fields. Descriptions ^^^^^^^^^^^^ There are two kinds of description: * short description which is useful for hints in UI * full description which could be rendered as a page in UI. Full description is optional but can be useful FK representation ^^^^^^^^^^^^^^^^^ O2O FK and simple FK should be represented in backend responses as nested object with following information: * id - id of related object * url - url of related object M2M FK should be represented as a list of id, url pairs. For example: .. code-block:: json { "id": 1, "name": "cluster", "url": "http://127.0.0.1/api/v1/cluster/1/", "frog": [ { "id": 2, "url": "http://127.0.0.1/api/v1/frog/2/" }, { "id": 8, "url": "http://127.0.0.1/api/v1/frog/8/" } ] } .. note:: Cannot create or update nested objects. Fields properties ~~~~~~~~~~~~~~~~~ .. _field-property-postable: POSTable ^^^^^^^^ if *POSTable* == *True*, then field allowed to be set by end user over API with POST method. If *POSTable* == *False*, then field is not allowed to be set by user during object creation with POST method. If send in POST body it will be ignored with no error. .. _field-property-changeable: Changeable ^^^^^^^^^^ If *Changeable* == *True*, then field value could be changed over API with PUT or PATCH operation. If *Changeable* == *False*, there is no way to change field value over API with PUT or PATCH operations. If *Changeable* == *False*, and field value has been sent over API with PUT or PATCH, then it should be checked if new value (from request) is equal to old value (from DB). If new value != old value error should be spawned. .. _field-property-nullable: Nullable ^^^^^^^^ If *Nullable* == *False*, then field value could not be set over API with POST, PUT or PATCH to *null*. API should return error, when someone tries this. .. _field-property-default: Default ^^^^^^^ If field was not sent over API with POST, then *Default* value should be applied on backend side. .. _field-property-required: Required ^^^^^^^^ If *Required* == *True*, then the field value should be set via API using POST and PUT. The API should return an error when someone tries to skip the field. If *Required* == *False*, then the field value is not required to be set via API using POST and PUT. If field is omitted in POST than Default value will be applied. If field is omitted in PUT than current field value will not be changed. .. note:: *Default* and *Nullable* are related. If we have field with *Default* == *null* and *Nullable* == *False* there is no way to pass no field in POST, because we will have error about *null* while we sent nothing. Base Objects ============ Fields and methods ------------------ .. _object-base-resource-type: Resource Type ~~~~~~~~~~~~~ List of distinguished resource capacity types (e.g. simultaneous backup process count) Only Read and List operation allowed for this object Endpoint: */resource-type* ================= ======= ======= ======== =========== Name Type Default Nullable Description ================= ======= ======= ======== =========== id integer auto False Object id. name string null False Name of type. Constant. Internal Use. Alphanum only display_name string name False Human readable name to display on UI. short_description text null False Human readable text to show in UI. For hints. description text null False Human readable text (html may be) with helpfull information about object. ================= ======= ======= ======== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= .. _object-base-cluster-type: Cluster Type ~~~~~~~~~~~~ List of supported cluster types. It just a stab right now but can be useful in future for interaction with ADCM. Only Read and List operation allowed for this object Endpoint: */cluster-type* ================= ======= ======= ======== =========== Name Type Default Nullable Description ================= ======= ======= ======== =========== id integer auto False Object id. name string null False Name of type. Constant. Internal Use. Alphanum only display_name string name False Human readable name to display on UI. short_description text null False Human readable text to show in UI. For hints. description text null False Human readable text (html may be) with helpfull information about object. connection_schema json null False JSON schema for Cluster connection ================= ======= ======= ======== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= .. _object-base-cluster: Cluster ~~~~~~~ Cluster is container to hold connection information about cluster. Connection information should be enough to make operations without ADCM intervention. Endpoint: */cluster* ================ ======= ======= ======== ================ ================== ================== ============ Name Type Default Nullable :term:`Required` :term:`POSTable` :term:`Changeable` Description ================ ======= ======= ======== ================ ================== ================== ============ id integer auto False False False False Object id. name string null False True True True Name of cluster. display_name string name False False True True Human readable name to display on UI. Fake one, because there is no display_name in a ADCM for cluster. connection json null False True True True Data required to run commands on cluster; validated with ClusterType.connection_schema. cluster_type FK null False True True False FK to :ref:`object-base-cluster-type` cluster_capacity FK auto False False False False Back-reference from :ref:`object-base-cluster-capacity` mount_point FK auto False False False False Back-reference from :ref:`object-base-mount-point` ================ ======= ======= ======== ================ ================== ================== ============ API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST True PUT True PATCH True DELETE True ============= ======= .. _object-base-cluster-capacity: Cluster Capacity ~~~~~~~~~~~~~~~~ List of resources, available to cluster Endpoint: */cluster-capacity* ============= ======= ======= ======== ================ ================== =========== Name Type Default Nullable :term:`Required` :term:`Changeable` Description ============= ======= ======= ======== ================ ================== =========== id integer auto False False False Object id. resource_type FK null False True False FK to :ref:`object-base-resource-type`. value integer 1 False False True Resource capacity of specified type. label enum null False True False Label of priority queue to use (cron or on-demand). cluster FK null False True False FK to :ref:`object-base-cluster`. ============= ======= ======= ======== ================ ================== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT True PATCH True DELETE False ============= ======= .. _object-base-fs-type: File System Type ~~~~~~~~~~~~~~~~ Supported file system types. There is no big sense for it in current moment, but it is a stab for future. Only Read and List operation allowed for this object Endpoint: */filesystem-type* ================== ======= ======= ======== =========== Name Type Default Nullable Description ================== ======= ======= ======== =========== id integer auto False Object id. name string null False Name of type. Constant. Internal Use. Alphanum only display_name string name False Human readable name to display on UI. short_description text null False Human readable text to show in UI. For hints. description text null False Human readable text (html may be) with helpful information about object. connection_schema json null False JSON schema for Filesystem connection mount_point_schema json null False JSON schema for MountPint connection ================== ======= ======= ======== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= .. _object-base-fs: File System ~~~~~~~~~~~ Exact file system placed somewhere. Metainformation about connection to this FS should come from ADCM or created manually by end user. Note, that FS and cluster not alway equal by hosts. For example MuseFS installation could be large than ADB installation. Endpoint: */filesystem* =================== ======= ======= ======== ================ ================== ================== =========== Name Type Default Nullable :term:`Required` :term:`POSTable` :term:`Changeable` Description =================== ======= ======= ======== ================ ================== ================== =========== id integer auto False False False False Object id. name string null False True True True Name of object. display_name string name False False True True Human readable name to display on UI. connection json null False True True True Data required to work with FS on cluster; validated with FilesystemType.connection_schema. filesystem_type FK null False True True False FK to :ref:`object-base-fs-type` filesystem_capacity FK auto False False False False Back-reference from :ref:`object-base-fs-capacity` mount_point FK auto False False False False Back-reference from :ref:`object-base-mount-point` =================== ======= ======= ======== ================ ================== ================== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST True PUT True PATCH True DELETE True ============= ======= .. _object-base-fs-capacity: File System Capacity ~~~~~~~~~~~~~~~~~~~~ List of resources, available to file system Endpoint: */filesystem-capacity* ============= ======= ======= ======== ================ ================== =========== Name Type Default Nullable :term:`Required` :term:`Changeable` Description ============= ======= ======= ======== ================ ================== =========== id integer auto False False False Object id. resource_type FK null False True False FK to :ref:`object-base-resource-type`. value integer 1 False False True Resource capacity of specified type. label enum null False False False Label of priority queue to use (cron or on-demand). filesystem FK null False True False FK to :ref:`object-base-fs`. ============= ======= ======= ======== ================ ================== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT True PATCH True DELETE False ============= ======= .. _object-base-mount-point: Mount Point ~~~~~~~~~~~ Link between File System and Cluster, also contains info where exactly filesystem mounted onto cluster hosts. There could by only one mount point for specified pair of File System and Cluster. Endpoint: */mount-point* ========== ======= ======= ======== ================ ================ ================== =========== Name Type Default Nullable :term:`Required` :term:`POSTable` :term:`Changeable` Description ========== ======= ======= ======== ================ ================ ================== =========== id integer auto False False False False Object id. cluster FK null False True True False FK to Resource Type. filesystem FK null False True True False FK to File System. connection JSON {} False False [*]_ True True Data required to access FS on cluster; validated with FilesystemType.mount_point_schema. ========== ======= ======= ======== ================ ================ ================== =========== .. [*] Depends on applied FilesystemType.mount_point_schema API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST True PUT True PATCH True DELETE True ============= ======= .. _object-base-handler: Handler ~~~~~~~ Peace of code which is able to do some operation on some cluster type on some fs type. Only Read and List operation allowed for this object Endpoint: */handler* ====================== ======= ======= ======== =========== Name Type Default Nullable Description ====================== ======= ======= ======== =========== id integer auto False Object id. name string null False Name of type. Constant. Internal Use. Alphanum only display_name string name False Human readable name to display on UI. short_description text null False Human readable text to show in UI. For hints. description text null False Human readable text (html may be) with helpfull information about object. type enum null False Handler capability (backup or restore). handler_class json null False Information about code to run. cluster_type M2M null False Many2Many constraint to :ref:`object-base-cluster-type` filesystem_type M2M null False Many2Many constraint to :ref:`object-base-fs-type` cluster_consumption FK auto False Back-reference from :ref:`object-base-cluster-consumption` filesystem_consumption FK auto False Back-reference from :ref:`object-base-fs-consumption` config_schema json null False JSON schema for CronLine and JobQueue configs ====================== ======= ======= ======== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= .. _object-base-cluster-consumption: Cluster Consumption ~~~~~~~~~~~~~~~~~~~ Handler's cluster resource consumption; created along with Handler Endpoint: */cluster-consumption* ============= ======= ======= ======== =========== Name Type Default Nullable Description ============= ======= ======= ======== =========== id integer auto False Object id. resource_type FK null False FK to :ref:`object-base-resource-type`. value integer null False Resource consumption of specified type. handler FK null False FK to :ref:`object-base-handler`. ============= ======= ======= ======== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= .. _object-base-fs-consumption: File System Consumption ~~~~~~~~~~~~~~~~~~~~~~~ Handler's file system resource consumption; created along with Handler Endpoint: */filesystem-consumption* ============= ======= ======= ======== =========== Name Type Default Nullable Description ============= ======= ======= ======== =========== id integer auto False Object id. resource_type FK null False FK to :ref:`object-base-resource-type`. value integer null False Resource consumption of specified type. handler FK null False FK to :ref:`object-base-handler` ============= ======= ======= ======== =========== API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= Scheduler Objects ================= .. _object-scheduler-cronline: Cron Line ~~~~~~~~~ Endpoint: */cron-line* ============== ======== ======= ========= ================ ================== ================== =========== Name Type Default Nullable :term:`Required` :term:`POSTable` :term:`Changeable` Description ============== ======== ======= ========= ================ ================== ================== =========== id integer auto False False False False Object id. name string null False True True True Name of cron line. display_name string name False False True True Human readable name to display on UI. handler FK null False True True False M2O FK to :ref:`object-base-handler` cluster FK null False True True False M2O FK to :ref:`object-base-cluster` filesystem FK null False True True False M2O FK to :ref:`object-base-fs` config json null True [*]_ False True True Parameters for handler provided by User. cron_line string null False True True True Line in classic crontab format with seconds, 50 chars. create_date datetime now() False False False False When job was created. last_run_date datetime null True False False False Last spawned job datetime. state enum active False False True True Is it allowed to use? ============== ======== ======= ========= ================ ================== ================== =========== .. [*] Depends on applied handler.config_schema .. note:: We need both create_date and last_run date because we need to show this information to user. State Description ^^^^^^^^^^^^^^^^^ ========== ========= State Descreption ========== ========= active Job **is** allowed to run from this cronline inactive Job **is not** allowed to run from this cronline ========== ========= API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST True PUT True PATCH True DELETE True ============= ======= .. _object-scheduler-queue: Job Queue ~~~~~~~~~ Endpoint: */job-queue* ============= ======== ========= ========= ================ ================== ================== =========== Name Type Default Nullable :term:`Required` :term:`POSTable` :term:`Changeable` Description ============= ======== ========= ========= ================ ================== ================== =========== id integer auto False False False False Object id. name string null False True True False Name of job. display_name string name False False True False Human readable name to display on UI. handler FK null False True True False M2O FK to :ref:`object-base-handler` cluster FK null False True True False M2O FK to :ref:`object-base-cluster` filesystem FK null False True True False M2O FK to :ref:`object-base-fs` cron_line FK null True False False False M2O FK to :ref:`object-scheduler-cronline` config json null True [*]_ False True False Parameters for handler provided by User. pid int null True False False False Pid of :ref:`scheduler-module-jproc` state enum created False False False True State of job label enum on-demand False False False False Is this job created by cron or User? create_date datetime now() False False False False When job was created. start_date datetime null True False False False When job was switched to "running" state. end_date datetime null True False False False When job was switched to one of terminal states. See :ref:`scheduler-queue-state`. ============= ======== ========= ========= ================ ================== ================== =========== .. [*] Depends on applied handler.config_schema .. note:: While fild state marked as allowed to be changed, there is a restriction on. It is allowed to transite from "created" to "to kill" or from "running" to "to kill". All other combinations should be forbidden. .. _scheduler-queue-state: State Description ^^^^^^^^^^^^^^^^^ ========== ============== ========= State Terminal? Description ========== ============== ========= created False Initial state of job after initial entry creation. runnning False Job is currently running. It means that there is a process with "pid" in system. successed True Job is done with success. failed True Job is done with fail. to kill False User asks to kill the job. killed True Job was killed by user request. aborted True Job has crashed by unknown reason (external killing, for example). ========== ============== ========= .. _scheduler-queue-label: Label Description ^^^^^^^^^^^^^^^^^ ========== ========= Label Description ========== ========= on-demand Job has been run by User over API/UI cron Job has been run by :ref:`scheduler-module-cron` ========== ========= .. note:: While we have Fk to :ref:`object-scheduler-cronline` it seems that we could find out on-demand/cron label by value of FK. But it is impossible because we have :ref:`scheduler-operation-delete-cronline` operation which forbit us to find label value from FK value. We need this to show user two queue instead of one, because it is required by user stories. API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST True PUT True PATCH True DELETE False ============= ======= .. _object-scheduler-queue-hist: Job History ~~~~~~~~~~~ That is a historical data from :ref:`object-scheduler-queue` Endpoint: */job-history* ============= ======== ======= ======== =========== Name Type Default Nullable Description ============= ======== ======= ======== =========== id integer auto False Object id. name string null False Name of job. display_name string name False Human readable name to display on UI. handler FK null False M2O FK to :ref:`object-base-handler` cluster FK null False M2O FK to :ref:`object-base-cluster` filesystem FK null False M2O FK to :ref:`object-base-fs` cron_line FK null True M2O FK to :ref:`object-scheduler-cronline` config json null True Parameters for handler provided by User. pid int null True Pid of :ref:`scheduler-module-jproc` state enum created False State of job label enum null False Is this job created by cron or User? create_date datetime null False When job was created. start_date datetime null True When job was switched to "running" state. end_date datetime null True When job was switched to one of terminal states. See :ref:`scheduler-queue-state`. ============= ======== ======= ======== =========== For information about enums in this object see :ref:`object-scheduler-queue` description API Calls Allowed ^^^^^^^^^^^^^^^^^ ============= ======= Operation Allowed ============= ======= GET True LIST True POST False PUT False PATCH False DELETE False ============= ======= Job Life Cycle -------------- State transitions ~~~~~~~~~~~~~~~~~ All possible state in state transitions completely described by this table ========== ================================ ============== ========= From State Module To State Condition ========== ================================ ============== ========= None :ref:`scheduler-module-cron` Created Job created from cronline None API/UI created On-demand run by User created :ref:`scheduler-module-queue` running Resources to run job has been found. :ref:`scheduler-module-queue` starts :ref:`scheduler-module-jproc`. running :ref:`scheduler-module-jproc` successed Job is done with success according to Handler code. running :ref:`scheduler-module-jproc` failed Job is done with fail according to Handler code. running :ref:`scheduler-module-jclean` aborted According to Job Queue job is running, but no pid exists. running API/UI to kill Job is marked to be killed by User. created API/UI to kill Job is marked to be killed by User. to kill :ref:`scheduler-module-jclean` killed JOb should marked to be killed. ========== ================================ ============== =========