Use Git as Storage for Systems

Styra DAS supports using Git as a storage mechanism for policies. Each system can be configured with its own Git repository as storage.

git-storage is an internal functionality that co-ordinates all the activities between the Git repository and Styra DAS.

Using Git as storage also enables the following features:

• Embrace the policy-as-code philosophy, allowing you to leverage the same source of truth for your policies that you have for your code: A Git repository.

• Rollback policy changes using the same machinery you do for your other code: Update the tip of your chosen branch to a previous workable state.

• Implement change control via peer-review using the Git-workflow provided by your organization.

Once you configure Git, you can use Styra the same way you always have, except when you want to promote policies for distribution to OPA.

The following shows the policy-authoring flow without configuring git-storage:

1. Make a policy change in the Styra editor.
2. Promote the change. Styra stores the new policy in its backend.
3. Styra distributes new policy to OPAs.

The following shows the policy-authoring flow when git-storage is configured.

1. Make a policy change in the Styra editor.
2. Promote the change. Styra pushes this policy to your GitHub repository in a special branch. This branch is named styra-{repo-path}-{basebranch}-system-{user@domain.com}, where {repo-path} is the path configured for git, {basebranch} is the base branch name (typically "main"), and {user@domain.com} is the email of the user who created the branch.
3. The team can review, test, and merge this special branch into your main branch. Styra does not open a pull request or similar -- you will need to do this manually.
4. Styra picks up the change in the main branch and distributes the new policy to OPAs.

Figure 1 - Architecture with Git as a Policy Backend

Policy Versions

The policy editor shows the following three different versions of policy with git-storage:

• Your chosen branch (called System).
• The private branch (called Branch).
• A draft (called Draft).

The following shows the restrictions on the policy versions:

• The Enforced and Branch versions are read-only. The read-only permission allows you to see the policy versions as they exist in the outside world. If required, you can also compare the policy versions. This acts as a starting point to write new policies.

• Before you have a Draft, if you look at either the Enforced or Branch versions, and you try to make changes, then you must create a Draft with your changes.

• When you already have a Draft, you must either publish it to your private branch or discard it before creating a new draft.

Transition to-and-from git-storage

If you either enable git-storage or disable git-storage, there will be no disruption to the policies being distributed to the OPAs.

The following section shows you how to transition to-and-from git-storage.

Turn On git-storage

1. If you have already written, published, and distributed policies to OPA without git-storage.
2. Enable git-storage on Styra.
3. Styra fetches the tip of your chosen branch. If policies exist, then it copies them to the (non-Git) Styra policy backend. For Kubernetes, Styra copies the admission_control.rego and test.rego files.
4. Styra always distributes the policy in the Styra backend to all OPAs. If your policy is already in your chosen branch, then that policy is distributed to the OPAs. If those policies are not already in your chosen branch, then your existing policy continues to be distributed to the OPAs.

note

When enabling git-storage, Styra does not automatically push the current policy into Git. But, you can use the Styra policy editor to push your current policy into your private branch, and then use the standard Git workflow to merge the private branch into your chosen branch. The Styra backend will then pick up these changes and distribute them to OPA.

important

Styra DAS synchronizes the state from the user Git repeatedly in one minute intervals. In case of a Git outage, the DAS caches the latest synchronized state and continues to poll the user's Git until it is resumed. When Git is resumed, DAS continues to synchronize the state.

Turn Off git-storage

1. Disable git-storage on Styra.
2. Styra will no longer pull changes from your Git repository.
3. If you make any more policy changes, then they are promoted directly to the non-Git policy backend, which makes them the version of policy distributed to the OPAs.

Git Configuration Options

The following shows how to configure Git for each system individually.

• Under SYSTEMS, click the system name for which you want to save policies in Git.
• Click Settings tab.
• Click Git Repository.

Configure Git Authentication

Styra DAS supports either HTTPS or SSH authentication for Git. You can navigate to the Git settings dialog and select the Git authentication mode HTTPS or SSH. The Git settings may only have one authentication mode selected per system level.

HTTPS

• Git username (required): Your Git username.

• Git secret (required): The secret corresponding to your Git username.

• Git repository (required): A Git HTTPS URL to your Git repository. For example: https://github.com/hooli/foo.git.

• Git reference: Specify a tag or branch reference (defaults to refs/heads/main—the main branch).

• Git commit SHA: Specify a commit SHA.

• Repository path: (Optional) The subdirectory where you want to save the policies.

SSH

• SSH key (required): A private SSH key. For example: The contents of ~/.ssh/id_rsa.

• SSH key passphrase: (Optional) The passphrase specified at the time the private SSH key was created.

• Git repository (required): A Git SSH URL to your Git repository. For example: git@github.com:hooli/foo.git.

• Git reference: Specify a tag or branch reference (defaults to refs/heads/main—the main branch).

• Git commit SHA: Specify a commit SHA.

• Repository path: (Optional) The subdirectory where you want to save the policies.

Click the Save changes button.

important

• Git reference and Git commit SHA are mutually exclusive, only one can be submitted per Git configuration.
• For SSH configuration, the corresponding SSH public key must be configured on the Git service (GitHub, Bitbucket, and so on) in order for authentication to work.

Configure GitHub Repositories

Use the following tips to find the required configuration for a GitHub repository.

• Git secret: Styra recommends to use a GitHub Personal access token. You can generate a token at github.com/settings/token or by clicking through Your-picture and navigate to Settings >> Developer Settings >> Personal access tokens.

• Git repository: To create a new Git repository, navigate to your GitHub account and click the + button on the top-right corner of the page.

  • To add a new repository, click the New repository button.

  • In the new repository, enter the name of your new repository in the Repository name field, the Description field is optional.

  • Select Public (anyone on the internet can see this repository. You can choose who can commit.) or Private (You choose who can see and commit to this repository.) based on your requirement.

  • Click the Create repository button.

• SSH key: To create a SSH key and SSH key passphrase, see the GitHub SSH Key documentation page.

tip

You can navigate to your repository on GitHub and click the Code button to see the clone options, such as HTTPS and SSH. Copy the HTTPS or SSH link to clone the Git repository.

Directory Layout in Git

To add a directory layout in Git to a system:

• Configure git-storage for each system-type.
• Save the files.
• Adhere to the directory structure.

The package names for the Rego files mirror the directories to which they belong. For a given system, the package inside of rules/rules.rego is called package rules, and the package inside of test/test.rego is called package test.

The synchronization mechanism for git-storage inspects the linked Git repository to detect any *.rego files present under preset paths. The path presets vary based on the type of each individual system.

The following shows the directory layout in Git for git-storage to work in a system:

caution

The synchronization mechanism ignores any files or packages present in the Git repository outside the well-known paths for the configured system.

To view the directory layout in Git for your system-type:

1. Go to your terminal window and navigate to your repository.

2. Run the tree command to view the directory layout in Git.

In your workspace name >> Settings >> Git Repository, when you create a policy directory for the Repository path field it is important to understand that Styra DAS will create the full directory structure and files for the following systems.

Custom System

├── rules
│   └── rules.rego
└── test
    └── test.rego

​ Envoy System

├── policy
│   ├── com.styra.envoy.app
│   │   ├── rules
│   │   │   └── rules
│   │   │       └── app.rego
│   │   └── test
│   │       └── test
│   │           └── test.rego
│   ├── com.styra.envoy.egress
│   │   ├── rules
│   │   │   └── rules
│   │   │       └── egress.rego
│   │   └── test
│   │       └── test
│   │           └── test.rego
│   └── com.styra.envoy.ingress
│       ├── rules
│       │   └── rules
│       │     └── ingress.rego
│       └── test
│           └── test
│               └── test.rego
└── system
    └── log
        └── mask.rego

​ Kubernetes System

├── policy
│   ├── com.styra.kubernetes.mutating
│   │   ├── rules
│   │   │   └── rules
│   │   │       └── mutating.rego
│   │   └── test
│   │       └── test
│   │           └── test.rego
│   └── com.styra.kubernetes.validating
│       ├── rules
│       │   └── rules
│       │       └── validating.rego
│       └── test
│           └── test
│               └── test.rego
└── system
    └── log
        └── mask.rego

Security-critical policies

The top-level policies for Kubernetes or Envoy systems (Labels, Features, Notifications) are not shown in the directories above because they are stored in the Workspace-level Git repository. Splitting the security-critical policies from the system ensures that for example Stacks cannot be disabled on a system without administrative approval because the Labels policy can only be merged into the Workspace-level Git repository.

The following shows the directory layout in Git for git-storage for security-critical policies.

# Within the Workspace-level git repo
├── metadata
│   ├── cf489de0bdcd4747bdc41b15125fd553
│   │   └── features
│   │       └── features.rego
│   │   └── labels
│   │       └── labels.rego
│   │   └── notifications
│   │       └── notifications.rego
│   ├── ea778774e2b44e58899cc478edfed9d8
│   │   └── features
│   │       └── features.rego
│   │   └── labels
│   │       └── labels.rego
│   │   └── notifications
│   │       └── notifications.rego

Create a Git-backed System through the API

To create a Git-backed system through the API:

1. Setup a secret that allows Styra to read and write to your Git repository.

   PUT https://<das-id>.styra.com/v1/secrets/git/policy-repo-access

   {
       "name": "git-user-name",
       "secret": "token value"
   }

   Where,

   • git/policy-repo-access is the id of the secret. You can change the id as you see fit and use this id in the next step.

   • name is the Git user id.

   • secret is a password or token for that user. The credentials given here must have read and write access to the Git repository.

2. Create the system with the following API.

   POST https://<das-id>.styra.com/v1/systems

   {
       "name": "test kubernetes cluster",
       "description": "aws cluster in region us-east-1",
       "type": "kubernetes:v2",
       "read_only": false,
       "source_control": {
           "origin": {
               "url": "https://github.com/user/repo",
               "credentials": "git/policy-repo-access",
               "reference": "refs/heads/main",
               "path": "path/within/repo"
           }
       }
   }

   Where,

   • Within the source_control object, the field url is the URL of the Git repository.

   • credentials is a reference to the secret created in the first step.

   • reference is the release branch (contents of this branch will be synchronized to Styra DAS).

   • path is a path in the repository where the Rego policy files will reside.