Skip to main content

Querying DynamoDB with dynamodb.get and dynamodb.query

Enterprise OPA provides the dynamodb.get and dynamodb.query built-in functions for querying DynamoDB during policy evaluation.

The built-ins currently support the GetItem and the Query operations.

Overview

In this tutorial, we'll turn the usual order around and dive right in on using the built-in functions! If you'd like to run the examples on your own machine later, the instructions for doing so are provided further down in the article.

Using dynamodb.get

The dynamodb.get function works similarly to the other database-related built-in functions of Enterprise OPA, or the http.send function in OPA/EOPA.

package servers

# Get a server by ID
response := dynamodb.get({
"cache": true,
"cache_duration": "10s",
"credentials": {
"access_key": "foo",
"secret_key": "bar"
},
"endpoint": "http://localhost:8000",
"region": "eu-north-1",
"table": "Servers",
"key": {
"ID": {"S": "d28eef3e-b78a-4f68-a5ed-b30479f27cdb"},
}
})

is_web_server if response.row.Function == "Web"

# Response would look something like this
# {
# "row": {
# "ID": "d28eef3e-b78a-4f68-a5ed-b30479f27cdb",
# "Name": "Megalodon",
# "Function": "Storage"
# }
# }

Project setup

If you'd like to try the example yourself, the following steps will get you started.

  1. For the sake of simplicity, we'll be using the dynamodb-local Docker image in the examples below. This image is a local version of DynamoDB that can be used for development and testing.

    docker run -p 8000:8000 amazon/dynamodb-local
  2. In order to not use real credentials for this demo, we'll be using a fake AWS access key and secret key. You can provide those using the ~/.aws/credentials file. Obviously, you'll want to save your real credentials in a backup file before doing this, should you have any!

    [default]
    aws_access_key_id = foo
    aws_secret_access_key = bar
    region = eu-north-1
  3. We'll also use the AWS CLI appplication:

    brew install awscli

    Or any other download option. Refer to the documentation for more information on how to use the AWS CLI tool with DynamoDB.

  4. Some data to query. We'll use a list of servers in a DynamoDB table called Servers. Create a file called servers.json with the following contents:

    {
    "Servers": [
    {
    "PutRequest": {
    "Item": {
    "ID": {"S": "2de2f522-32f2-4d24-bfec-1b7b86ff8944"},
    "Name": {"S": "T-Rex"},
    "Function": {"S": "Web"}
    }
    }
    },
    {
    "PutRequest": {
    "Item": {
    "ID": {"S": "50acb25e-1e9b-4a01-bc99-aa1707e8227a"},
    "Name": {"S": "Velicraptor"},
    "Function": {"S": "Web"}
    }
    }
    },
    {
    "PutRequest": {
    "Item": {
    "ID": {"S": "d28eef3e-b78a-4f68-a5ed-b30479f27cdb"},
    "Name": {"S": "Megalodon"},
    "Function": {"S": "Storage"}
    }
    }
    },
    {
    "PutRequest": {
    "Item": {
    "ID": {"S": "29bc37fe-2b6d-4812-ad6e-6e603c7c9cbc"},
    "Name": {"S": "Allosaurus"},
    "Function": {"S": "Database"}
    }
    }
    }
    ]
    }

    We'll keep the list short here, but feel free to populate with as many items as you want. Next, let's create a table for our data, containing a primary key index for the ID attribute:

    aws dynamodb create-table \
    --endpoint-url http://localhost:8000 \
    --table-name Servers \
    --attribute-definitions \
    AttributeName=ID,AttributeType=S \
    --key-schema AttributeName=ID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 \
    --table-class STANDARD

    With a table in place, we can go ahead an populate it with the data from servers.json:

    aws dynamodb batch-write-item \
    --endpoint-url http://localhost:8000 \
    --request-items file://servers.json \
    --return-consumed-capacity INDEXES \
    --return-item-collection-metrics SIZE

You can now use dynamodb.get to query the Servers table like in the example provided above.

Tip: If you're using eopa eval to evaluate your policy, the --strict-builtin-errors can help catch mistakes like missing required attributes in the dynamodb.get request object. See the example below for its usage.

Using dynamodb.query

What if we wanted to query the Servers table for all servers with a specific function rather than one-by-one using the ID key? The dynamodb.query built-in can be used to retrieve multiple items from a DynamoDB table. The built-in requires a table containing a global secondary index (GSI). We could either create one at the same time as we create the table, or we may add one later. Let's see what adding a new index for Function looks like. To avoid typing in JSON in the shell, we'll store our index definition in a file called index.json:

[
{
"Create": {
"IndexName": "FunctionIndex",
"KeySchema": [{
"AttributeName":"Function", "KeyType": "HASH"
}],
"Projection":{
"ProjectionType": "ALL"
},
"ProvisionedThroughput": {
"ReadCapacityUnits": 10,
"WriteCapacityUnits": 5
}
}
}
]

With that in place, we may go ahead and update our table to include the new index:

aws dynamodb update-table \
--endpoint-url http://localhost:8000 \
--table-name Servers \
--attribute-definitions AttributeName=Function,AttributeType=S \
--global-secondary-index-updates file://index.json

We're now ready to query our database using the new index. Let's try and get a list of all Web servers in the list:

dynamodb.rego

package dynamodb

response := dynamodb.query({
"endpoint": "http://localhost:8000",
"table": "Servers",
"index_name": "FunctionIndex",
"key_condition_expression": "#f = :value",
"expression_attribute_values": {":value": {"S": "Web"}},
"expression_attribute_names": {"#f": "Function"},
"credentials": {"access_key": "foo", "secret_key": "bar"},
"region": "eu-north-1",
})

The response is now a list of items rather than a single one. Using eopa eval to query the response rule:

eopa eval --format pretty --strict-builtin-errors --data dynamodb.rego data.dynamodb.response

We should see something like this in the output:

{
"rows": [
{
"Function": "Web",
"ID": "50acb25e-1e9b-4a01-bc99-aa1707e8227a",
"Name": "Velicraptor"
},
{
"Function": "Web",
"ID": "2de2f522-32f2-4d24-bfec-1b7b86ff8944",
"Name": "T-Rex"
}
]
}

Configuration Reference

To see all options available for the dynamodb.get and dynamodb.query built-in functions, refer to the built-in function reference.