Provide resources access to an ETL Entity

There are 2 ways of providing resources access to an ETL job.

Resource access through API

Use the following API to provide datasets (read or write), parameters and shared libraries access to an ETL Entity in large number.

resource_type: jobs, glueendpoints or notebooks

resource_id: unique id (uuid) associated with the jobs, glueendpoints or notebooks

API Path: /{resource_type}/{resource_id}/resourceaccess

HTTP Path: POST

Input Body:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
    "DatasetAccess": {
        "Owner": [
            {
                "DatasetId": "uuid",
                "DatasetName": "dataset_name"
            },
            {
                "DatasetId": "uuid2",
                "DatasetName": "dataset_name2"
            }, ...
        ],
        "ReadOnly": [
            {
                "DatasetId": "uuid3",
                "DatasetName": "dataset_name3"
            },
            {
                "DatasetId": "uuid4",
                "DatasetName": "dataset_name4"
            }, ....
        ]
    },
    "ParameterAccess": ["param1", "param2"],
    "SharedLibraries": ["uuid_lib1", "uuid_lib2"]
}

To retrieve the status of the latest resource update, use the GET HTTP method without input body. Output retrieved would be

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
{
    "Status": "in_progress"/"failed"/"updated",
    "Message": "update failed because of some reason", # When it fails this will appear
    "ParameterAccess": ["param1", "param2"],
    "SharedLibraries": ["uuid_lib1", "uuid_lib2"]
    "DatasetAccess": {
        "ReadOnly": [{
            "DatasetId" : "uuid1",
            "DatasetName": "name1"
        }, ...],
        "Owner": [
        {
            "DatasetId" : "uuid2",
            "DatasetName": "name2"
        }
        ]
    }
}

Below image shows how the extra resources access provided are shown in User Interface

Job details image

Resource access through UI

Provide datasets (read or write), parameters and shared libraries access to an ETL Entity in large number using the ‘Update Extra Resource Access’ button on the top right side of the ETL job details page.

Click on the ‘Update Extra Resource Access’ button and select the required resources from the dropdown and click ‘Submit’ to update the resource access.

Once the resource access is submitted, the extra resources will be updated and displayed in the ETL job details page eventually.

The following picture depicts how to update extra resources:

Job extra resource access

If the extra resources are not getting displayed then refresh the job details page to view the extra resources.

Remove all extra resource access

User can also update the extra resources or remove them all again through the same process.

Note

All the extra resources added to the job till that time will be displayed in the UI popup every time the popup is opened so that user can either remove or add accordingly.

Resource Limits

Below are the hard limits on the resource allocation for an ETL entity

Type Limit
Datasets Read 72
Datasets Write 144
Shared Libraries 62
Parameters 35