Files

Upload

Based on the Update Method selected at the time of dataset creation there are different ways to add files to dataset. There are three types of update methods available for datasets with target location AuroraMySQL and S3-Athena and two types of update methods for datasets with target location S3

Reload

When reload is selected as update method, every time a new files are added the old files get deleted. Once files are uploaded, user has options to process the files or discard them.

Below images shows the file upload mechanism

Reload File Upload
Append / Lastest Record

When Append or LatestRecord is selected as update method, the newly added files are appended to the already present files in the dataset

Below image shows the file upload mechanism for dataset with update method Latest Record and Append

Append/LatestRecord File Upload

Truncate

All the files in the dataset will be deleted when truncate is selected. This functionality is applicable only to datasets of target location S3, S3-Athena.

Below images shows the functionality of truncate dataset

Truncate Dataset

Delete

This option allows the user to delete set of files from a dataset. Files deleted can be restored using Restore functionality. This functionality is applicable only to dataset of target location S3, S3-Athena.

Below images shows the functionality of delete dataset files

Delete Dataset Files

Restore

This functionality allows the user to restore the files that were deleted from the dataset. Applicable to dataset of target location S3, S3-Athena.

Below image shows the functionality of restoring deleted files

Restore Dataset Files

Permanent Delete

This option allows the user to delete files from dataset permanently. Applicable to dataset of target location S3, S3-Athena

Below image shows the functionality of deleting files permanently

Premanent Delete Files