Skip to main content

Using Bundle Registry

The DAS distributes policy and data to OPA using OPA's upstream Bundle API. DAS supports the following two types of distribution in a feature called the Bundle Registry.

  • Automatic distribution: Every time a policy or its dependencies changes those changes are immediately distributed to all the relevant OPAs. This is the default behavior.

  • Manual distribution: Policy changes are only deployed after they are approved for distribution. You can approve a bundle either through the DAS GUI or the DAS API.

These two modes of distribution are applicable regardless of the source of the bundle change. The following will cause bundle changes that will need to be distributed to OPA.

  • A policy stored in the DAS-native backend storage could be changed through the DAS GUI or API.

  • A policy stored in Git could be changed by a merge into the main branch.

  • A data source could receive an update when it is pulled or pushed.

  • DAS system labels are changed (which could influence which systems a Stack applies to and therefore the policies that are relevant to that DAS system).

  • A policy library imported by a system policy could be updated.

Moreover, the Bundle Registry allows you to roll back to previously deployed bundles and examine the history of deployments.

The Bundle Registry is a feature that you can leverage for each DAS system. You can decide that some DAS systems should have bundles automatically distributed, while other systems require manual approval.

Besides enabling you to control automatic or manual distribution, the Bundle Registry allows you to choose where you want OPA to download bundles from.

  • DAS: OPA (or the SLP) downloads bundles from the DAS directly. This is the easiest way to get started.

  • AWS S3: OPA (or the SLP) downloads bundles from S3. DAS when it builds bundles will push them to S3 so that OPA or the SLP can download them. This option is targeted at high-scale users that want the reliability of cloud-scale storage for serving bundles.

Storing Bundles in AWS S3

This section describes the secondary functionality of Bundle Registry and how it works when you switch the storage location from DAS to AWS S3 (and vice versa).

Besides giving you the ability to manually choose what bundle to deploy to OPA, the Bundle Registry also allows you to control where OPA downloads those bundles from: DAS or S3. When you choose S3, you provide additional details about which bucket to use, what the credentials are to access it, and so on. DAS will then start building bundles and pushing them to S3. DAS will also change the configuration that OPA downloads via discovery to pull those bundles from S3.

OPA's discovery protocol also uses bundles, so when OPA asks for discovery, DAS responds with a bundle. These discovery bundles can also be served out of S3. However, if you have already deployed OPA and configured it to use DAS for bundles, and then switch the settings to use S3 for bundles, discovery bundles will still be served from DAS, but regular bundles will be served from S3. OPA is configured to use DAS to download discovery, so while DAS can redirect OPA to pull regular bundles from S3, DAS cannot change OPA's original configuration. If you want both the discovery and regular bundles served out of S3, you need to select S3 for bundle storage and then deploy OPA with the configuration that tells it to pull discovery bundles out of S3.

Operations

By default, all bundles are deployed automatically and hosted by DAS. To change this, you must configure the Bundle Registry, understand how to deploy bundles, learn how to Roll them back, and so on.

Use the Bundle Registry to perform the following operations:

  1. Configure the Bundle Registry

  2. Deploy Bundles

  3. Expand Policy Bundles

  4. Deploy the Latest Bundle Update

  5. View Older Bundles

  6. Rollback to a Previously Deployed Bundle

  7. Rollback to an Older Bundle

  8. Publish the Bundles

Configure the Bundle Registry

The Bundle Registry displays the information about the contents of the bundle for you to understand which one of several bundles to deploy. You must be able to configure a S3 bucket as an alternative for serving bundles. When this option is configured, the OPA installation instructions must be changed accordingly.

Using the GUI

To configure the Bundle Registry through the Styra DAS GUI:

  1. Navigate to Your System >> Settings >> Bundle Registry dialog.

  2. The Bundle Registry page has the following fields:

  • Policy bundle deployment: Select one of the following modes to deploy the policy bundles.

    • Automatic: When you publish or push and merge policies, the system builds and automatically deploys its policy bundle (policies and data) that OPA synchronizes with.

    • Manual: When you publish or push and merge policies, the system builds its policy bundle (policies and data), but you must manually deploy the policy bundle that OPA synchronizes with.

  • Max deployed policy bundles to keep: Maximum number of deployed policy bundles to keep when deploying manually and this field is useful for rollback. The default is 10.

  • Max policy bundles to keep: Maximum number of built and deployed policy bundles to keep. The default is 100.

  • Policy bundle registry: Select one of the following from the drop down list.

    - **DAS**: DAS stores policy bundles and the history of past bundles. OPA downloads bundles from DAS. Click **Save changes** to save your changes or **Reset** button to reset the changes.

    - **Amazon S3**: DAS stores policy bundles and the history of past bundles in the Amazon S3 bucket you specify. This option does not rely on DAS high availability since OPA downloads bundles from the S3 bucket. Enter the details required for the following fields.

    - **Amazon region (required)**: A string representing the AWS region.

    - **S3 bucket (required)**: A string representing the bucket name.

    - **S3 endpoint**: A gateway endpoint.

    - **Discovery bundle path (required)**: A string representing the discovery bundle path.

    - **Policy bundle path (required)**: A string representing the policy bundle path.

    - **DAS access keys for S3 bucket**: This DAS secret is required if you are using a S3 bucket within your own AWS account.

    - **OPA credentials for S3 bucket**: Select one of the following credentials from the drop down list.

    - **Environment credentials**: SLP/OPA expects to find environment variables for `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_REGION`.

    - **Metadata credentials**: OPA will use the AWS metadata services for EC2 or ECS to obtain the necessary credentials when running within a supported virtual machine or container. Default is **IAM role**.
    note

    For OPA pods to retrieve credentials from EC2 metadata API, the --http-put-response-hop-limit on the EKS worker nodes must be set to 2

          - **Web identity credentials**: SLP/OPA expects to find environment variables for `AWS_ROLE_ARN` and `AWS_WEB_IDENTITY_TOKEN_FILE`, in accordance with the convention used by the AWS EKS IAM Roles for Service Accounts. The default session name is `open-policy-agent`.
  1. Check if the Automatic mode is selected in the Policy bundle deployment drop down list.

  2. Click the Save changes button.

Using the APIs

Bundle Registry maintains an internal archive of bundles. Its contents may be inspected with GET /v1/systems/<system id>/bundles and GET /v1/systems/<system id>/bundles/<bundle id>/<version>/details calls.

For example,

  • GET /v1/systems/b87975caa0084ea49f7e763107929706/bundles: Lists all the bundles in the archive.

  • ?past: Option for the GET lists only the bundles that have been deployed in the past.

  • ?version=NNN: A parameter that paginates through larger archive.

You can decide whether to use the Bundle Registry for each DAS system. To configure the Bundle Registry, you must provide the configuration as a part of the system configuration, by setting the bundle_registry field of a system.

For example:

  1. Download the system configuration with: GET /v1/systems/b87975caa0084ea49f7e763107929706.

  2. Modify the system configuration in the result.

  3. Upload it back after adding the bundle_registry field to the system configuration object.

note

The uploaded system configuration is actually in the result field of the GET call and the outer most object wrapping the system configuration should be dropped.

The configuration for the Bundle Registry comprises of the fields listed in Using the GUI section.

For example, the following JSON snippet is a valid bundle registry configuration that instructs DAS to use an external S3 bucket (styra-example-storage) with the configured file names for the discovery and policy bundles. It also instructs the SLP/OPA to use AWS environment variables in obtaining the S3 credentials. It configures a short history for the bundle archives with at most two bundles will be archived (in total), but at least two bundles deployed in the past are kept in the archive.

"bundle_registry": {
"max_bundles": 2,
"max_deployed_bundles": 2,
"distribution_s3": {
"access_keys": "styra-example-storage-secret",
"region": "us-east-1",
"bucket": "styra-example-storage",
"discovery_path": "discovery-p1.tgz",
"policy_path": "policy-p2.tgz",
"opa_credentials": {
"environment_credentials": {
"aws_default_region": "us-east-1"
}
}
}
},

To configure the bundle registry using APIs:

  1. Create a System.

  2. Create an API token.

  3. Create S3 secret with following curl command:

    $ curl --request PUT 'https://styra-das-id.styra.com/v1/secrets/<secretname>' --header 'Content-Type: application/json' --header 'Authorization: Bearer <DAS API Token>'
    --data-raw '{
    "description": "S3 export key",
    "name": "<aws_access_key_id>",
    "secret": "<aws_secret_access_key>"
    }'
  4. Use the following instructions to update the system created with bundle_registry configuration:

    • Download the system configuration file using the following curl command:

      $ curl -o system-config.json styra-das-id.styra.com/v1/systems/SYSTEM_ID

    • Edit the system-config.json file removing the remove outer level: "request_id" and "result"; then add the bundle_registry field.

      "bundle_registry": {
      "distribution_s3": {
      "access_keys": "<secretname>",
      "bucket": "styra-onprem-test01",
      "discovery_path": "discovery-p1.tgz",
      "opa_credentials": {
      "environment_credentials": {
      "aws_default_region": "us-east-1"
      }
      },
      "policy_path": "policy-p2.tgz",
      "region": "us-east-1"
      },
      "manual_deployment": true,
      "max_bundles": 2,
      "max_deployed_bundles": 2
      }
    • Upload the updated version by using the following curl command:

      $ curl -X PUT styra-das-id.styra.com/v1/systems/SYSTEM_ID -d @system-config.json

  5. Trigger bundle_deploy using API with the bundle version and revision obtained through getbundles API by using the following command:

    $ curl -XPUT -H "Content-Type: application/json" styra-das-id.styra.com/v1/systems/SYSTEM_ID/bundle-deploy -d '{"primary": {"id": "policy", "version" : VERSION}}'

Deploy Bundles

To deploy a bundle:

  1. Navigate to Latest built bundle and click the Deploy to OPA button.

  2. A Deploy bundle dialog appears with a Deploy to OPA and Cancel button.

  3. Click the Deploy to OPA button to deploy the bundle. Now, the Latest built bundle is changed to Currently deployed bundle which looks similar to the Automatic mode. Also, the previous Currently deployed bundle is changed to Previously deployed bundle and everything else is hidden.

  4. To redeploy the bundle to OPA, click the Redeploy to OPA button.

  5. To download the bundle, click the Download policy bundle button located at the top right of the Deployments dialog.

Expand Policy Bundles

In the Bundle Registry, expand the policy bundles to see the latest bundle updates, policy built updates, currently deployed bundle, and previously deployed bundle.

To expand the policy bundles:

  1. Navigate to Your System and click on the Deployments tab to view the policy bundle and OPA instances

  2. Click the expand icon > near the Policy bundles to see the following:

  • Latest bundle update. This row has Deploy to OPA button to deploy the bundle to OPA.

  • Updates built between 4 and 10 hours ago. To view the list of policies built during this time frame, click the expand icon > near Updates built between 4 and 10 hours ago.

  • Currently deployed bundle. This row has the currently deployed bundle.

  • Previously deployed bundle: This row has Redeploy to OPA button to redeploy the bundle.

    tip

    Navigate to Your System and see the Deployments tab badged with a number. For example, If 4 is badged in the Deployments tab, it implies there are 4 updates including the latest bundle update.

  1. Click the expand icon > near the OPA instances tab to view the number of active OPA instances which synchronizes with the deployed bundle from DAS (10 active OPA instances sync’d 2 mins ago) and the number of erroneous OPA instances (10 active OPA instances sync’d 2 mins ago).
note
  1. If OPA fails to compile a recently pulled bundle, the original bundle stays active until it gets restarted.

  2. When OPA restarts, it may lose the original bundle and have only the new bundle which will fail to activate.

  3. Similarly, if a new OPA gets created it will only be able to download the new, broken bundle, and again fail to activate.

Deploy the Latest Bundle Update

To deploy the latest bundle update, OPA downloads and compiles that currently deployed bundle successfully.

  1. Navigate and click on the Deployments tab.

  2. Click the expand icon > near the Policy bundles to see the Latest bundle update.

  3. In this row, click the Deploy to OPA button.

  4. A Deploy Bundle dialog appears with the following fields:

    • Deploy Bundle: Displays the built time.

    • Replaces bundle: Displays the built time of the Currently deployed bundle.

    • Summary (required): Enter the summary of the bundle to be deployed.

    • Details: Enter the details to deploy the latest bundle update.

5. Click the Deploy to OPA button to deploy the bundle to OPA.

Now, the Currently deployed bundle becomes the latest bundle and waits for OPA to synchronize with the deployed bundle from DAS. The Previously deployed bundle becomes the Currently deployed bundle. This row shows the number of OPAs that are in synchronization with the deployed bundle and the number of replayed decisions.

Finally, the latest bundle update is deployed. OPA downloads and compiles the currently deployed bundle successfully.

View Older Bundles

The Policy Builder allows you to view older bundles by clicking the expand icon > near the policy built during the following time or days:

  1. A specific time (Built 16 min ago).

  2. A given time frame (**Built between 4 and 10 hours ago).

  3. Couple of days before (Built 5 days ago).

  4. Between a number of days (Built between 26 and 45 days ago).

Rollback to a Previously Deployed Bundle

When you make a mistake, you can go back to an older bundle by clicking the Other bundles >> Built bundle >> deploy to OPA button.

To rollback to previously deployed bundle:

  1. Navigate to Your System and click on the Deployments tab.

  2. Click the expand icon > near the Policy bundles to see the Previously deployed bundle. In this row, click the Redeploy to OPA button.

  3. A Redeploy Bundle dialog pops up with the following fields:

    • Deploy Bundle: Displays the built time.

    • Replaces bundle: Displays the built time of the Previously deployed bundle.

    • Summary (required): Enter the summary of the bundle to be deployed.

    • Details: Enter the details to deploy the latest bundle update.

4. Click the Deploy to OPA button to deploy the bundle.

Now, the Policy Builder rolls back Previously deployed bundle to display in the first row and the Currently deployed bundle in the second row. This helps you to move from an erroneous bundle to a working bundle.

Rollback to an Older Bundle

To rollback to an older bundle, click the expand icon > near the policy built during one of the following time/days:

  • A specific time (Built 16 min ago).

  • A given time frame (**Built between 4 and 10 hours ago).

  • Couple of days before (Built 5 days ago).

  • Between a number of days (Built between 26 and 45 days ago).

For example:

  1. Scroll down to the row which shows Built 5 days ago and click the three vertical dots () to see the following options:

    • Download: To download the bundle.

    • Edit info: To edit the bundle information.

    • Redeploy to OPA: To redeploy the bundle to OPA.

  2. Now, click the Redeploy to OPA button.

  3. A Redeploy Bundle dialog pops up with the following fields:

    • Deploy Bundle: Displays the built time.

      For example:

      Built 46 days ago at 2021-01-15 16:45

      Deployed 46 days ago at 2021-01-15 17:00

    • Replaces bundle: Displays the built time of the Previously deployed bundle.

      For example:

      Built 1 day ago at 2021-03-02 13:45

      Deployed 1 day ago at 2021-03-02 14:00

    • Summary (required): Enter the summary of the bundle to be deployed.

      For example:

      Rolled back to an older bundle

    • Details: Enter the details to deploy the latest bundle update.

4. From the Deploy Bundle field, select and copy-paste or append the timestamp 2021-01-15 16:45 to the Summary (required) field. For example: Rolled back to an older bundle 2021-01-15 16:45.

5.Click the Deploy to OPA button.

Now, the Policy Builder displays the previous Currently deployed bundle as the Previously deployed bundle and rolls back the older bundle with timestamp 2021-01-15 16:45 to display as the Currently deployed bundle.

Publish the Bundles

If you have a policy in Your System >> validating >> rules >>rules.rego and if you want to publish this policy, but you don't want to deploy to OPA yet. Then you have to publish the policies, and control when to deploy to the OPA. When you click a publish on the policy it creates a new bundle and the bundle registry will help you to control the deployment.

To publish the bundles:

  1. Navigate to Your System >> Rules >> rules.rego.

  2. In the rules.rego window, hover over the System to see information on Git state and deployment state.

  3. Click the Push button.

The Push changes to review branch dialog appears. After you create a PR and merge this review branch, your changes will be published to the system. Now, go to the Deployments tab to manually deploy the bundle.

tip

The warning icon from the systems tells you to go to the Deployment tab to manually deploy the bundle with the latest bundle update.