Skip to main content

S3 Data Source

S3 data sources are similar to the Git data sources. Instead of automatically reading the JSON out of Git, they read the data from a bundle stored in a S3 bucket. With the Git data sources, the GUI will not show the files in that bundle, but the remainder of the DAS functionality will work properly.

Mount JSON Files

For the required layout of the files stored in a bundle, see how to mount JSON files with the Git Data Sources. The root of the bundle should have the same contents as the directory pointed by the Git data source path configuration.

To mount the S3 bundle within a system:

1. Create a DAS secret with credentials that will access the S3 bucket. To create a secret named alice/buckets/data, run the following curl command. You must provide your bearer token for XXX and your AWS credentials.

curl  -H 'Authorization: Bearer XXX' -H 'Content-Type: application/json'
-X PUT -d '{
"name": "Insert AWS access key id",
"secret": "Insert AWS secret access key"

2. Mount the S3 bundle using the DAS secret. Again, you must provide your bearer token for XXX and the system ID for YYY. The system ID is located on your Systems >> Settings >> General page. To mount the files within the bundle repository to the mount point systems/<systemid>/myroot, run the following curl command.

curl  -H 'Authorization: Bearer XXX' -H 'Content-Type: application/json'
-X PUT -d '{
"category": "bundle/s3",
"type": "pull",
"bucket": "insert s3 bucket here",
"path": "insert s3 object path to bundle here",
"region": "insert S3 bucket region",
"credentials": "alice/bundles/data",
"endpoint": "leave empty unless S3 API endpoint needs to be set"

3. Check the STATUS of that data source to make sure everything is working. For example, the STATUS of the data source should have the code set to Finished if everything is functioning properly.

curl  -H 'Authorization: Bearer XXX' -H 'Content-Type: application/json'