S3 Datasource Configuration

Styra Load's support for pulling in data from any S3-compatible blob store lets you use a common storage option to push data to, and have that data available and up to date for policy evaluations in Styra Load.

Example Configuration

The S3 integration is provided via the data plugin, and needs to be enabled in Load's configuration.

load-conf.yaml (minimal)

plugins:
  data:
    all.users:
      type: s3
      url: s3://databucket/users.json
      access_id: "${AWS_ACCESS_KEY_ID}"
      secret: "${AWS_SECRET_ACCESS_KEY}"

With this minimal configuration, Styra Load will pull in the users.json file from the databucket bucket on Amazon S3 every 5 minutes.

All of this, and various more settings for S3-compatible APIs can be configured using an advanced configuration:

load-conf-advanced.yaml

plugins:
  data:
    all.users:
      type: s3
      url: s3://databucket/users.json
      endpoint: https://s3-api.internal:9000/
      access_id: "${AWS_ACCESS_KEY_ID}"
      secret: "${AWS_SECRET_ACCESS_KEY}"
      region: us-west-1
      polling_interval: 10s # minimum: 10s

With a config like this, Styra Load will retrieve the file from the specified bucket location, and attempt to parse it as any of the following formats:

• XML
• YAML
• JSON

The result will then be available to all policy evaluations under data.all.users.

Example Call

If the referenced S3 bucket contains a users.json file with this content,

[
    {"username": "alice", "roles": ["admin"]},
    {"username": "bob", "roles": []},
    {"username": "catherine", "roles": ["viewer"]}
]

then Styra Load's data.all.users will look like this:

$ curl 'http://127.0.0.1:8181/v1/data/all/users?pretty'
{
  "result": [
    {
      "roles": [
        "admin"
      ],
      "username": "alice"
    },
    {
      "roles": [],
      "username": "bob"
    },
    {
      "roles": [
        "viewer"
      ],
      "username": "catherine"
    }
  ]
}

note

The key below data in the configuration (all.users in the example) can be anything you want, and determines where the retrieved document will be found in Styra Load's data hierarchy.