Preparing a Project for Import

One thing to be aware of when importing systems from Git repositories is Styra DAS's requirements on the directory structure where policies reside. More specifically, Styra DAS expects policy package names to be mirrored in the directory structure.

For example, if you have a policy declaring its package as internal.authorization.policies, that policy must reside under a directory tree like internal/authorization/policies (relative to the bundle root, or the project root directory if none is provided). Failing to comply with this requirement will result in an error when the repository is imported.

Directory / package mismatch error on import of project from git

Manually tracking down files that violate this requirement can be a tedious task, especially in large projects. Having directory structure mirror policy packages isn't just a Styra DAS requirement, it's generally considered a best practice! Luckily, there's a tool to help both enforce best practices as well as to help remediate cases where they're not met. That tool is Regal, our linter and language server for Rego. See Regal's documentation to learn more about the tool and how to integrate it into your development workflow and builds. For the scope of this documentation, we'll simply focus on how to use Regal to make our policy repo's structure ready for Styra DAS.

TL;DR

In case you've done this before and just need a quick reminder, the steps outlined in this document are provided here. If this is the first time you're using Regal to ensure your directories mirror your Rego packages, we recommend reading the full document to understand more about the process.

1. Download the latest Regal binary from the Regal download page.
2. Create a .regal/config.yaml file in your project's root directory with the following content: .regal/config.yaml

rules:
  default: # remove this later to use other rules
    level: ignore
  idiomatic:
    directory-package-mismatch:
      level: error
      exclude-test-suffix: false

1. Run regal lint . to identify policy files that violate the directory-package-mismatch rule.
2. Run regal fix --dry-run . to see what changes regal fix would make.
3. Run regal fix . to actually move the policy files to directories matching their package path.

Identifying Violations with regal lint

First off, make sure to download the latest Regal binary. At least version v0.26.2 is required for the directory-package-mismatch rule to be available. Since we're currently focusing on finding cases of this particular violation, we'll create a Regal configuration file to enable the directory-package-mismatch rule only.

.regal/config.yaml

rules:
  default: # remove this later to use other rules
    level: ignore
  idiomatic:
    directory-package-mismatch:
      level: error
      exclude-test-suffix: false

In your project's root directory, create a directory named .regal and place this file there as config.yaml.

This configuration should speak for itself, but to summarize, we first set all rule levels to ignore, and then override the default level for directory-package-mismatch to error. The exclude-test-suffix option is set to false, which means that Regal will require that also test packages, like internal.authorization.policies_test, follow this convention, and reside in their own directory (e.g., internal/authorization/policies_test) rather than being placed next to the policies they test.

A simple example of a project layout where policies are stored directly under the project root directory might look like this:

.
├── .regal
│   └── config.yaml
├── authz.rego       # package authz
├── authz_test.rego  # package authz_test
├── lib.rego         # package lib.common
└── opa-conf.yaml

You can now run Regal to find policy files where the package declaration isn't mirrored in the directory structure. From the project's root directory, run:

regal lint .

If we were to run this in our example project, we'd see something like this in the output:

Rule:         	directory-package-mismatch
Description:  	Directory structure should mirror package
Category:     	idiomatic
Location:     	lib.rego:1:9
Text:         	package lib.common
Documentation:	https://docs.styra.com/regal/rules/idiomatic/directory-package-mismatch

Rule:         	directory-package-mismatch
Description:  	Directory structure should mirror package
Category:     	idiomatic
Location:     	authz.rego:1:9
Text:         	package authz
Documentation:	https://docs.styra.com/regal/rules/idiomatic/directory-package-mismatch

Rule:         	directory-package-mismatch
Description:  	Directory structure should mirror package
Category:     	idiomatic
Location:     	authz_test.rego:1:9
Text:         	package authz_test
Documentation:	https://docs.styra.com/regal/rules/idiomatic/directory-package-mismatch

3 files linted. 3 violations found in 3 files.

Hint: 3/3 violations can be automatically fixed (directory-package-mismatch)
      Run regal fix --help for more details.

Now that we've identified the policies that aren't not compliant with Styra DAS's requirements, we can proceed to fix that! If you do not see some or all of the files you were hoping to move, please check:

• that the Regal command was run from the correct directory and of version v0.26.2 or newer.
• that the configuration file has the correct contents and path.
• that the files are not already in the correct location!

Remediation with regal fix

In the example above, Regal has found three policy files that violate the directory-package-mismatch rule. In other words, their respective package paths (as seen in the value of the Text field) does not match directory they're in (as seen in the value of the Location field). We can now either move these files ourself, or have Regal help us with that. To have Regal move the files to the correct location for us, we can use the fix command.

It's advisable to run regal fix with the --dry-run flag first to confirm the files will be moved as expected.

regal fix --dry-run .

3 fixes to apply:
In project root: /Users/johndoe/workspace
authz.rego -> authz/authz.rego:
- directory-package-mismatch
authz_test.rego -> authz_test/authz_test.rego:
- directory-package-mismatch
lib.rego -> lib/common/lib.rego:
- directory-package-mismatch

As seen above, Regal would create a authz directory, an authz_test directory, and a lib/common directory, all in accordance with the package declarations in the respective policy files. The authz_test directory is created since our configuration says exclude-test-suffix: false, meaning that test packages should be mirrored in directories of their own rather than be placed next to the policies they test.

Before moving the files with regal fix, we need to make sure that none of the files to be moved have changes in Git. Run git status to confirm this and commit or stage any changes you might have before continuing.

We're now ready to move the files, and can do so simply by removing the --dry-run flag from the previous command. All the policy files in your workspace are now under directories matching their package path, which means both that your project is following best practices for organizing policy files, and that it's ready to be imported into Styra DAS!

Project Roots

In the examples above, we've assumed that the project root directory is the directory where the .regal directory is located. This may not always be the case. Sometimes you have multiple bundle roots in the same project, or you may want to keep your policies in subdirectories nested deeper than the project root. If you have existing project / bundle roots that you want regal fix to consider when moving files, you can provide Regal with a list of roots in the configuration file. Use the top-level project key to specify the roots:

.regal/config.yaml

project:
  roots:
    - src
    - lib/common

Policies found under either src or lib/common will now be moved to directories relative to their roots rather than the workspace root. If you for example have a policy in lib/common with its package path declared as authz.rbac, the regal fix command would move it under the lib/common/authz/rbac/ directory.

Editor Integration

While this document covers using regal lint and regal fix from the command-line, both the linter rule directory-package-mismatch and the fix feature is provided also by Regal's language server, which editors like VS Code use to bring you the best possible development experience for Rego.

In supported editors, and when the directory-package-mismatch rule is enabled, the following features are also available from within your editor:

• Diagnostics: Your editor will show violations against the rule right where they happen (i.e. in the package declaration)
• Code Action: Press the lightbulb icon ("Quick Fix") button to automatically move policy files to directories matching their package path
• New Policy Template: When a new policy is created in a directory, a package declaration matching the directory will automatically be added to the file.