API¶
Getting Start¶
There are two wasy to interact with ADSS Core API:
- Use official Python Client
- Genereate your our client based on OpenApi Schema v3 (also known as Swagger)
Official Python Client¶
TBD
Client generation¶
Install ADSS, and go to <your_adss_core_ui_url>/api/v1/schema/. On this endpoint you will find a JSON with schema description which could be used for client generation by your tools for your language.
Find a tool for you at the following link: https://openapi.tools/
API Standarts¶
Structure and operation¶
API should be unified in look from endpoint to enpoint.
API should be flat. All endpoints should be on the same level.
API backend accepts data in “application/json” format; “multipart/form-data” format also supported by used framework, but is not recommended to use, neither supported by development team.
Endpoint¶
Every endpoint should have following suburls and operations (classic REST)
- /endpoint/
- GET - List all objects
- POST - Create new object
- /endpoint/<pk_id>/
- GET - Read object according to pk_id
- PATCH - Update object partitialy
- PUT - Replace object
- DELETE - Delete object
HTTP Methods¶
GET on /endpoint/¶
Also known as LIST.
This operation return all objects of same kind. For example all Clusters or something else.
Result of this request is a JSON of following format:
{
"count": 2,
"next": "http://127.0.0.1:8006/api/v1/cluster-type/?limit=1&offset=1",
"previous": null,
"results": [
{
"id": 1,
"name": "dummy-foxpro",
"display_name": "Dummy FoxPro",
"description": "FoxPro is a text-based procedurally oriented DBMS",
"short_description": "FoxPro database",
"url": "http://127.0.0.1:8006/api/v1/cluster-type/1/"
}
]
}
There is a two distinct but related blocks here.
First one is a standart DRF Pagination block.
{
"count": 2,
"next": "http://127.0.0.1:8006/api/v1/cluster-type/?limit=1&offset=1",
"previous": null
}
Second one is a list of objects:
{
"results": [
{
"id": 1,
"name": "dummy-foxpro",
"display_name": "Dummy FoxPro",
"description": "FoxPro is a text-based procedurally oriented DBMS",
"short_description": "FoxPro database",
"url": "http://127.0.0.1:8006/api/v1/cluster-type/1/"
}
]
}
Every object in the list should conform following logic:
- Every object has id field which is unique, positive, integer primary key (PK).
- Every object has url field to full object. See GET on /endpoint/<pk>/
- There are no Huge Fields here
- There are no links to related objects.
GET on /endpoint/<pk>/¶
Also known as READ.
This one returns one object in form of JSON:
{
"id": 1,
"name": "Some cluster",
"display_name": "sdfa",
"connection": {},
"cluster_type": {
"id": 2,
"url": "http://127.0.0.1:8006/api/v1/cluster-type/2/"
}
}
Respons should be conform to following rules:
- Every object has id field which is unique, positive, integer primary key (PK).
- Every object has url field to full object. See GET on /endpoint/<pk>/
- There are Huge Fields here
- There are links to related objects.
POST on /endpoint/¶
This one creates object.
Content should conform following rules:
- There is no id, or url parts here.
- All related objects (Foreign keys) are sent as nested object with the only required field id (other fields are ignored, so, nested object could be GETed from API as-is).
- It is allowed to send no field if you want to apply default value. See Default
- If POSTable for field is False then it is not allowed to pass this to POST operation. See POSTable
In case of operation success we should have the same output as for GET on /endpoint/<pk>/ on the new object.
PUT on /endpoint/<pk>/¶
This one to update whole object.
Content should conform following rules:
- All Required fields should be sent in request body
- id should not be changed
- All related objects (Foreign keys) are sent as nested object with the only required field id (other fields are ignored).
- It is allowed to send fields with Changeable == False, but they should be send without changes. See Changeable for more information.
In case of operation success we should have the same output as for GET on /endpoint/<pk>/ on this object.
PATCH on /endpoint/<pk>/¶
This one to update some fields in object.
Content should conform following rules:
- id should not be changed, or should not be sent
- All related objects (Foreign keys) are sent as nested object with the only required field id (other fields are ignored).
- It is allowed to send fields with Changeable == False, but they should be send without changes. See Changeable for more information.
In case of operation success we should have the same output as for GET on /endpoint/<pk>/ on this object.
FlexFields¶
API supports retrieving fields absent in default object representation through extending url with query strings. For example, you can list clusters by requesting URL http://127.0.0.1/api/v1/cluster/ and get bunch of objects like this:
{
"id": 1,
"url": "http://127.0.0.1/api/v1/cluster/1/",
"name": "cluster_name",
"display_name": "Cluster Name"
}
For more info you could retrieve details for each object by its url (here it’s http://127.0.0.1/api/v1/cluster/1/):
{
"id": 1,
"name": "cluster_name",
"display_name": "Cluster Name",
"connection": {
"some_data": "..."
},
"cluster_type": {
"id": 3,
"url": "http://127.0.0.1/api/v1/cluster-type/3/"
},
"cluster_capacity": [
{
"id": 1,
"url": "http://127.0.0.1/api/v1/cluster-capacity/1/"
},
{
"id": 2,
"url": "http://127.0.0.1/api/v1/cluster-capacity/2/"
},
{
"id": 3,
"url": "http://127.0.0.1/api/v1/cluster-capacity/3/"
},
{
"id": 4,
"url": "http://127.0.0.1/api/v1/cluster-capacity/4/"
}
]
}
If list representation has not enough information and requesting details for each list item seems cumbersome, you may expand returned field-set like that - http://127.0.0.1/api/v1/cluster/?expand=type and also get info about cluster’s type:
{
"id": 1,
"url": "http://127.0.0.1/api/v1/cluster/1/",
"name": "cluster_name",
"display_name": "Cluster Name",
"cluster_type": {
"id": 3,
"url": "http://127.0.0.1/api/v1/cluster-type/3/",
"name": "ADB",
"display_name": "Arenadata DB cluster"
}
}
Expanded fields obeys same rules as parent objects, see GET on /endpoint/, GET on /endpoint/<pk>/
There is even more possibilities:
- Expanding available for any depth of nesting (e.g. http://127.0.0.1/api/v1/cron-line/1/?expand=cluster.capacity.type).
- Expanding available for any number of fields (e.g. http://127.0.0.1/api/v1/cron-line/1/?expand=cluster,handler).
- You can select fields to include in response, omitting others (e.g. http://127.0.0.1/api/v1/cluster/?fields=id,name).
- You can select fields to exclude from response (e.g. http://127.0.0.1/api/v1/cluster/?omit=url).
- You can combine all above (e.g. http://127.0.0.1/api/v1/cluster/?fields=name,type&expand=type).
- Although, omitting explicitly included or expanding not included fields is pointless, you allowed to do that anyway without receiving errors, the same way as including/expanding/omitting non-existing fields in query.
Filtering¶
API supports filtering through extending url with query strings. Filtering is supported for all fields except Huge Fields.
For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/ and get bunch of objects like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster_name_1",
"display_name": "Cluster name 1"
},
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster_name_2",
"display_name": "Cluster name 2"
}
]
To filter by the name field, use the request: http://127.0.0.1:8000/api/v1/cluster/?name=cluster_name_1
As a result, you will receive a response like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster_name_1",
"display_name": "Cluster name 1"
}
]
You can also filter by nested objects, use FlexFields.
For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/?expand=type and get bunch of objects like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster 1",
"display_name": "Cluster name 1",
"cluster_type": {
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/1/",
"name": "dummy-foxpro",
"display_name": "Dummy FoxPro"
}
},
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster 2",
"display_name": "Cluster name 2",
"cluster_type": {
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/2/",
"name": "dummy-dbase",
"display_name": "Dummy dBase"
}
}
]
To filter by the id field of the nested object type use the query: http://127.0.0.1:8000/api/v1/cluster/?expand=type&type__id=1
As a result, you will receive a response like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster 1",
"display_name": "Cluster name 1",
"cluster_type": {
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/1/",
"name": "dummy-foxpro",
"display_name": "Dummy FoxPro"
}
}
]
Supported lookups:
lookups | format | description |
---|---|---|
exact | ?<field_name>=<value> | exact match check |
Types that support lookups:
type | lookups |
---|---|
string | exact |
integer | exact |
Ordering¶
API supports ordering through extending url with query strings. Ordering is supported for all fields except Huge Fields.
For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/ and get bunch of objects like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster_name_1",
"display_name": "Cluster name 1"
},
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster_name_2",
"display_name": "Cluster name 2"
}
]
To order in ascending order name field, use the query: http://127.0.0.1:8000/api/v1/cluster/?ordering=name
As a result, you will receive a response like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster_name_1",
"display_name": "Cluster name 1"
},
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster_name_2",
"display_name": "Cluster name 2"
}
]
To order in descending order name field, use the query: http://127.0.0.1:8000/api/v1/cluster/?ordering=-name
As a result, you will receive a response like this:
[
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster_name_2",
"display_name": "Cluster name 2"
},
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster_name_1",
"display_name": "Cluster name 1"
}
]
You can also ordering by nested objects, use FlexFields.
For example, you can list clusters by requesting URL http://127.0.0.1:8000/api/v1/cluster/?expand=type and get bunch of objects like this:
[
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster 1",
"display_name": "Cluster name 1",
"cluster_type": {
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/1/",
"name": "dummy-foxpro",
"display_name": "Dummy FoxPro"
}
},
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster 2",
"display_name": "Cluster name 2",
"cluster_type": {
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/2/",
"name": "dummy-dbase",
"display_name": "Dummy dBase"
}
}
]
To order in descending order id field of the nested object type, use the query: http://127.0.0.1:8000/api/v1/cluster/?expand=type&ordering=type__id
As a result, you will receive a response like this:
[
{
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster/2/",
"name": "cluster 2",
"display_name": "Cluster name 2",
"cluster_type": {
"id": 2,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/2/",
"name": "dummy-dbase",
"display_name": "Dummy dBase"
}
},
{
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster/1/",
"name": "cluster 1",
"display_name": "Cluster name 1",
"cluster_type": {
"id": 1,
"url": "http://127.0.0.1:8000/api/v1/cluster-type/1/",
"name": "dummy-foxpro",
"display_name": "Dummy FoxPro"
}
}
]
Ordering format:
ordering | format |
---|---|
ascending | ?ordering=<field_name> |
descending | ?ordering=-<field_name> |
Errors¶
All errors, returned by back-end has different status codes as usual and JSON bodies like this:
{
"code": "status code description",
"level": "warning, error or critical",
"desc": "Short error description",
"details": { "more": "error details" }
}
If request to API breaks more than one rule (e.g. invalid field value and inconsistent combination of values and changing non-changeable field), API returns some single error, which one - framework decides by its internal logic.
Object’s Fields¶
This block of documentation describes the way end user interact with fields over API in depends of field’s properties and types.
Field Types¶
There are a number of data types allowed for Fields. Most of them are simple, obvious from their description, but some of them have to be explained.
Full list of field types:
Name | Description |
---|---|
id | primary key for object. |
url | url of object in API. See GET on /endpoint/<pk>/. |
integer | positive unsigned integer, max 2^63 - 1. |
datetime | ISO format string representation of DateTime |
string | No \n allowed. Object names limited to 255 chars, other fields - 1024, if not specified other. |
text | 1MB max length string with \n allowed |
json | 1MB max json field |
FK | Simple foreign key for Many-to-one relation. |
O2O FK | Foreign key for One-to-one relation. |
M2M FK | Foreign key Many-to-many relation. |
Huge Fields¶
Some fields of models could be comparatively huge, so operation with such attrs should be limited.
List of huge attrs
- Texts (multiline strings, which could be a huge)
- JSONFields
Display names¶
There is two conflicting ideas behind a property “name” for various objects:
- name is short alphanum string which is useful for internal use.
- name is a human readable information about object to be shown in UI or logs or somewhere else.
That is why we have display_name in every object which has name.
If for some reason display_name is absent we should have present name instead of display_name. As it should be done everywhere, we could do that trick close to DB in ORM layer.
To diminish mistakes when using GUI we apply unique constraint on “name” and “display_name” fields.
Descriptions¶
There are two kinds of description:
- short description which is useful for hints in UI
- full description which could be rendered as a page in UI.
Full description is optional but can be useful
FK representation¶
O2O FK and simple FK should be represented in backend responses as nested object with following information:
- id - id of related object
- url - url of related object
M2M FK should be represented as a list of id, url pairs. For example:
{
"id": 1,
"name": "cluster",
"url": "http://127.0.0.1/api/v1/cluster/1/",
"frog": [
{
"id": 2,
"url": "http://127.0.0.1/api/v1/frog/2/"
},
{
"id": 8,
"url": "http://127.0.0.1/api/v1/frog/8/"
}
]
}
Note
Cannot create or update nested objects.
Fields properties¶
POSTable¶
if POSTable == True, then field allowed to be set by end user over API with POST method.
If POSTable == False, then field is not allowed to be set by user during object creation with POST method. If send in POST body it will be ignored with no error.
Changeable¶
If Changeable == True, then field value could be changed over API with PUT or PATCH operation.
If Changeable == False, there is no way to change field value over API with PUT or PATCH operations.
If Changeable == False, and field value has been sent over API with PUT or PATCH, then it should be checked if new value (from request) is equal to old value (from DB). If new value != old value error should be spawned.
Nullable¶
If Nullable == False, then field value could not be set over API with POST, PUT or PATCH to null. API should return error, when someone tries this.
Default¶
If field was not sent over API with POST, then Default value should be applied on backend side.
Required¶
If Required == True, then the field value should be set via API using POST and PUT. The API should return an error when someone tries to skip the field.
If Required == False, then the field value is not required to be set via API using POST and PUT. If field is omitted in POST than Default value will be applied. If field is omitted in PUT than current field value will not be changed.
Note
Default and Nullable are related. If we have field with Default == null and Nullable == False there is no way to pass no field in POST, because we will have error about null while we sent nothing.
Base Objects¶
Fields and methods¶
Resource Type¶
List of distinguished resource capacity types (e.g. simultaneous backup process count)
Only Read and List operation allowed for this object
Endpoint: /resource-type
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
name | string | null | False | Name of type. Constant. Internal Use. Alphanum only |
display_name | string | name | False | Human readable name to display on UI. |
short_description | text | null | False | Human readable text to show in UI. For hints. |
description | text | null | False | Human readable text (html may be) with helpfull information about object. |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
Cluster Type¶
List of supported cluster types. It just a stab right now but can be useful in future for interaction with ADCM.
Only Read and List operation allowed for this object
Endpoint: /cluster-type
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
name | string | null | False | Name of type. Constant. Internal Use. Alphanum only |
display_name | string | name | False | Human readable name to display on UI. |
short_description | text | null | False | Human readable text to show in UI. For hints. |
description | text | null | False | Human readable text (html may be) with helpfull information about object. |
connection_schema | json | null | False | JSON schema for Cluster connection |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
Cluster¶
Cluster is container to hold connection information about cluster. Connection information should be enough to make operations without ADCM intervention.
Endpoint: /cluster
Name | Type | Default | Nullable | Required | POSTable | Changeable | Description |
---|---|---|---|---|---|---|---|
id | integer | auto | False | False | False | False | Object id. |
name | string | null | False | True | True | True | Name of cluster. |
display_name | string | name | False | False | True | True | Human readable name to display on UI. Fake one, because there is no display_name in a ADCM for cluster. |
connection | json | null | False | True | True | True | Data required to run commands on cluster; validated with ClusterType.connection_schema. |
cluster_type | FK | null | False | True | True | False | FK to Cluster Type |
cluster_capacity | FK | auto | False | False | False | False | Back-reference from Cluster Capacity |
mount_point | FK | auto | False | False | False | False | Back-reference from Mount Point |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | True |
PUT | True |
PATCH | True |
DELETE | True |
Cluster Capacity¶
List of resources, available to cluster
Endpoint: /cluster-capacity
Name | Type | Default | Nullable | Required | Changeable | Description |
---|---|---|---|---|---|---|
id | integer | auto | False | False | False | Object id. |
resource_type | FK | null | False | True | False | FK to Resource Type. |
value | integer | 1 | False | False | True | Resource capacity of specified type. |
label | enum | null | False | True | False | Label of priority queue to use (cron or on-demand). |
cluster | FK | null | False | True | False | FK to Cluster. |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | True |
PATCH | True |
DELETE | False |
File System Type¶
Supported file system types. There is no big sense for it in current moment, but it is a stab for future.
Only Read and List operation allowed for this object
Endpoint: /filesystem-type
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
name | string | null | False | Name of type. Constant. Internal Use. Alphanum only |
display_name | string | name | False | Human readable name to display on UI. |
short_description | text | null | False | Human readable text to show in UI. For hints. |
description | text | null | False | Human readable text (html may be) with helpful information about object. |
connection_schema | json | null | False | JSON schema for Filesystem connection |
mount_point_schema | json | null | False | JSON schema for MountPint connection |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
File System¶
Exact file system placed somewhere. Metainformation about connection to this FS should come from ADCM or created manually by end user.
Note, that FS and cluster not alway equal by hosts. For example MuseFS installation could be large than ADB installation.
Endpoint: /filesystem
Name | Type | Default | Nullable | Required | POSTable | Changeable | Description |
---|---|---|---|---|---|---|---|
id | integer | auto | False | False | False | False | Object id. |
name | string | null | False | True | True | True | Name of object. |
display_name | string | name | False | False | True | True | Human readable name to display on UI. |
connection | json | null | False | True | True | True | Data required to work with FS on cluster; validated with FilesystemType.connection_schema. |
filesystem_type | FK | null | False | True | True | False | FK to File System Type |
filesystem_capacity | FK | auto | False | False | False | False | Back-reference from File System Capacity |
mount_point | FK | auto | False | False | False | False | Back-reference from Mount Point |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | True |
PUT | True |
PATCH | True |
DELETE | True |
File System Capacity¶
List of resources, available to file system
Endpoint: /filesystem-capacity
Name | Type | Default | Nullable | Required | Changeable | Description |
---|---|---|---|---|---|---|
id | integer | auto | False | False | False | Object id. |
resource_type | FK | null | False | True | False | FK to Resource Type. |
value | integer | 1 | False | False | True | Resource capacity of specified type. |
label | enum | null | False | False | False | Label of priority queue to use (cron or on-demand). |
filesystem | FK | null | False | True | False | FK to File System. |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | True |
PATCH | True |
DELETE | False |
Mount Point¶
Link between File System and Cluster, also contains info where exactly filesystem mounted onto cluster hosts. There could by only one mount point for specified pair of File System and Cluster.
Endpoint: /mount-point
Name | Type | Default | Nullable | Required | POSTable | Changeable | Description |
---|---|---|---|---|---|---|---|
id | integer | auto | False | False | False | False | Object id. |
cluster | FK | null | False | True | True | False | FK to Resource Type. |
filesystem | FK | null | False | True | True | False | FK to File System. |
connection | JSON | {} | False | False [*] | True | True | Data required to access FS on cluster; validated with FilesystemType.mount_point_schema. |
[*] | Depends on applied FilesystemType.mount_point_schema |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | True |
PUT | True |
PATCH | True |
DELETE | True |
Handler¶
Peace of code which is able to do some operation on some cluster type on some fs type.
Only Read and List operation allowed for this object
Endpoint: /handler
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
name | string | null | False | Name of type. Constant. Internal Use. Alphanum only |
display_name | string | name | False | Human readable name to display on UI. |
short_description | text | null | False | Human readable text to show in UI. For hints. |
description | text | null | False | Human readable text (html may be) with helpfull information about object. |
type | enum | null | False | Handler capability (backup or restore). |
handler_class | json | null | False | Information about code to run. |
cluster_type | M2M | null | False | Many2Many constraint to Cluster Type |
filesystem_type | M2M | null | False | Many2Many constraint to File System Type |
cluster_consumption | FK | auto | False | Back-reference from Cluster Consumption |
filesystem_consumption | FK | auto | False | Back-reference from File System Consumption |
config_schema | json | null | False | JSON schema for CronLine and JobQueue configs |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
Cluster Consumption¶
Handler’s cluster resource consumption; created along with Handler
Endpoint: /cluster-consumption
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
resource_type | FK | null | False | FK to Resource Type. |
value | integer | null | False | Resource consumption of specified type. |
handler | FK | null | False | FK to Handler. |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
File System Consumption¶
Handler’s file system resource consumption; created along with Handler
Endpoint: /filesystem-consumption
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
resource_type | FK | null | False | FK to Resource Type. |
value | integer | null | False | Resource consumption of specified type. |
handler | FK | null | False | FK to Handler |
API Calls Allowed¶
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
Scheduler Objects¶
Endpoint: /cron-line
Name | Type | Default | Nullable | Required | POSTable | Changeable | Description |
---|---|---|---|---|---|---|---|
id | integer | auto | False | False | False | False | Object id. |
name | string | null | False | True | True | True | Name of cron line. |
display_name | string | name | False | False | True | True | Human readable name to display on UI. |
handler | FK | null | False | True | True | False | M2O FK to Handler |
cluster | FK | null | False | True | True | False | M2O FK to Cluster |
filesystem | FK | null | False | True | True | False | M2O FK to File System |
config | json | null | True [†] | False | True | True | Parameters for handler provided by User. |
cron_line | string | null | False | True | True | True | Line in classic crontab format with seconds, 50 chars. |
create_date | datetime | now() | False | False | False | False | When job was created. |
last_run_date | datetime | null | True | False | False | False | Last spawned job datetime. |
state | enum | active | False | False | True | True | Is it allowed to use? |
[†] | Depends on applied handler.config_schema |
Note
We need both create_date and last_run date because we need to show this information to user.
State | Descreption |
---|---|
active | Job is allowed to run from this cronline |
inactive | Job is not allowed to run from this cronline |
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | True |
PUT | True |
PATCH | True |
DELETE | True |
Endpoint: /job-queue
Name | Type | Default | Nullable | Required | POSTable | Changeable | Description |
---|---|---|---|---|---|---|---|
id | integer | auto | False | False | False | False | Object id. |
name | string | null | False | True | True | False | Name of job. |
display_name | string | name | False | False | True | False | Human readable name to display on UI. |
handler | FK | null | False | True | True | False | M2O FK to Handler |
cluster | FK | null | False | True | True | False | M2O FK to Cluster |
filesystem | FK | null | False | True | True | False | M2O FK to File System |
cron_line | FK | null | True | False | False | False | M2O FK to object-scheduler-cronline |
config | json | null | True [‡] | False | True | False | Parameters for handler provided by User. |
pid | int | null | True | False | False | False | Pid of scheduler-module-jproc |
state | enum | created | False | False | False | True | State of job |
label | enum | on-demand | False | False | False | False | Is this job created by cron or User? |
create_date | datetime | now() | False | False | False | False | When job was created. |
start_date | datetime | null | True | False | False | False | When job was switched to “running” state. |
end_date | datetime | null | True | False | False | False | When job was switched to one of terminal states. See scheduler-queue-state. |
[‡] | Depends on applied handler.config_schema |
Note
While fild state marked as allowed to be changed, there is a restriction on. It is allowed to transite from “created” to “to kill” or from “running” to “to kill”. All other combinations should be forbidden.
State | Terminal? | Description |
---|---|---|
created | False | Initial state of job after initial entry creation. |
runnning | False | Job is currently running. It means that there is a process with “pid” in system. |
successed | True | Job is done with success. |
failed | True | Job is done with fail. |
to kill | False | User asks to kill the job. |
killed | True | Job was killed by user request. |
aborted | True | Job has crashed by unknown reason (external killing, for example). |
Label | Description |
---|---|
on-demand | Job has been run by User over API/UI |
cron | Job has been run by scheduler-module-cron |
Note
While we have Fk to object-scheduler-cronline it seems that we could find out on-demand/cron label by value of FK.
But it is impossible because we have scheduler-operation-delete-cronline operation which forbit us to find label value from FK value.
We need this to show user two queue instead of one, because it is required by user stories.
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | True |
PUT | True |
PATCH | True |
DELETE | False |
That is a historical data from object-scheduler-queue
Endpoint: /job-history
Name | Type | Default | Nullable | Description |
---|---|---|---|---|
id | integer | auto | False | Object id. |
name | string | null | False | Name of job. |
display_name | string | name | False | Human readable name to display on UI. |
handler | FK | null | False | M2O FK to Handler |
cluster | FK | null | False | M2O FK to Cluster |
filesystem | FK | null | False | M2O FK to File System |
cron_line | FK | null | True | M2O FK to object-scheduler-cronline |
config | json | null | True | Parameters for handler provided by User. |
pid | int | null | True | Pid of scheduler-module-jproc |
state | enum | created | False | State of job |
label | enum | null | False | Is this job created by cron or User? |
create_date | datetime | null | False | When job was created. |
start_date | datetime | null | True | When job was switched to “running” state. |
end_date | datetime | null | True | When job was switched to one of terminal states. See scheduler-queue-state. |
For information about enums in this object see object-scheduler-queue description
Operation | Allowed |
---|---|
GET | True |
LIST | True |
POST | False |
PUT | False |
PATCH | False |
DELETE | False |
Job Life Cycle¶
State transitions¶
All possible state in state transitions completely described by this table
From State | Module | To State | Condition |
---|---|---|---|
None | scheduler-module-cron | Created | Job created from cronline |
None | API/UI | created | On-demand run by User |
created | scheduler-module-queue | running | Resources to run job has been found. scheduler-module-queue starts scheduler-module-jproc. |
running | scheduler-module-jproc | successed | Job is done with success according to Handler code. |
running | scheduler-module-jproc | failed | Job is done with fail according to Handler code. |
running | scheduler-module-jclean | aborted | According to Job Queue job is running, but no pid exists. |
running | API/UI | to kill | Job is marked to be killed by User. |
created | API/UI | to kill | Job is marked to be killed by User. |
to kill | scheduler-module-jclean | killed | JOb should marked to be killed. |