Skip to main content

Cloud Agnostic Infrastructure

Cloud agnostic infrastructure provides guidance on configuring cloud-agnostic software components in Styra DAS.

Elasticsearch​

Styra DAS requires an Elasticsearch cluster to provide decision search and log-replay functionality. Styra bundles an in-cluster Elasticsearch deployment with our self-hosted Helm charts to provide easy installation. The in-cluster installation is generally sufficient for most workloads, but Styra DAS can also work with managed Elasticsearch offerings to provide greater resiliency for decision functionality, as long as the offerings maintain full API compatibility with open-source Elasticsearch.

In-Cluster Elasticsearch Operator​

Self-Hosted Styra DAS comes pre-bundled with manifests for deploying an Elasticsearch cluster. If elasticsearch.enabled is set to true in values.yaml, the Self-Hosted Styra DAS Helm chart deploys an Elasticsearch cluster to the same namespace used to run the DAS microservices. elasticsearch.enabled is set to true by default.

Managed Elasticsearch​

Styra DAS can work with some managed Elasticsearch offerings. Using a managed Elasticsearch offering can provide greater resiliency for decision log browsing and replay. The managed Elasticsearch offering must offer full API compatibility with the open-source version of Elasticsearch to guarantee full decision log functionality.

With a managed Elasticsearch offering, Styra DAS currently only supports username and password authentication. Styra DAS can be configured to authenticate against the endpoint by setting the relevant fields in the managedElastic section of values.yaml.

note

Self-Hosted Styra DAS does not currently support using Amazon’s managed Elasticsearch service, OpenSearch for decision log functionality. OpenSearch does not currently guarantee API compatibility with open-source Elasticsearch.

Sizing Elasticsearch​​

The following section provides guidance on sizing Elasticsearch.

The approximate formula to estimate the total available disk space required to maintain the index is as follows:

hours of decisions indexed x decisions per hour x average size of decision x overhead factor

Where:

  • Hours of decisions indexed: The number of hours of decisions stored in the Styra DAS decision log. Set to 72 hours (3 days) by default.
  • Decisions per hours: The average number of decisions produced by all OPAs connected to the Self-Hosted Styra DAS within a given hour.
  • Average size of decision: The average size of a given decision reported back to DAS. One test method is to run a sample system long enough to generate a representative sample of decisions, and then download individual decisions from the log to check their size.
  • Overhead factor: Elasticsearch needs additional disk space to handle share replication and indexing. The amount of space required varies based on Elasticsearch replication settings, but we generally use 1.66 as an estimate for most deployments.

The following shows an example of Elasticsearch sizing. This example assumes all of Styra DAS Systems are Kubernetes. This makes the decision size easier to estimate, as Kubernetes decision data tend to be relatively similar in size.

  • Hours of decisions indexed: use the default value of 72 hours.
  • Decisions per hour: assume the connected OPAs produce 100 decisions a second, for a total of 360,000 an hour.
  • Average size of decision: use the default of 1.66.

In this example, the total space required to maintain the indices is approximately: 72 x 360,000 x 8192 x 1.66 = 352478822400 bytes. Converting to GB, 328.27 GB is the minimum safe Elasticsearch storage size.

SMTP Server​

Styra DAS uses a Simple Mail Transfer Protocol (SMTP) server to invite users to Styra DAS or to reset passwords for existing users. Styra DAS does not come bundled with an in-cluster SMTP and uses an existing SMTP server.

Styra DAS can work with any SMTP server that accepts a username and password as authentication. These values and the URL of the server are used during Configuration and Installation.

Styra Docker Images​

Running Styra DAS requires access to Styra DAS microservice Docker images. Styra hosts all of our microservice images for Self-Hosted installations at registry.styra.com. You can pull images directly from registry.styra.com or pull the images and upload them to an organization-internal registry mirror. Both options require a valid Styra DAS API Token to authenticate and fetch the original images.

Configuring Access​

Create an API Token provides information on creating an image pull token. Once in the “New API Token” creation window:

  • Put any name in the “Pathname” field that will help you remember the purpose of the token.
  • Use .* for the “Allowed API paths” field.
  • Either set a long TTL, or leave the TTL field blank for a non-expiring token.
  • Click “Add Permission” and select “WorkspaceViewer”.

The settings above will create a token that has the correct permissions to pull Styra Docker images for your Self-Hosted installation. Store the generated token value somewhere secure, as it will be necessary during Configuration and Installation.

(Optional) Internal Registry Mirror​

All Styra DAS Self-Hosted images are available at registry.styra.com. Some customers, however, prefer to mirror the images internally. Some reasons for doing so might include network policies that forbid pulling external images, or a desire for fewer network hops during installation.

Transferring Styra DAS Images​

The following instructions provide guidelines for downloading Styra’s microservice images and re-uploading them to an internal registry.

  1. Use the Styra DAS API Token to login to registry.styra.com with the Docker CLI.

    docker login registry.styra.com
    Username: <styra-das-tenant>.styra.com
    Password: <DAS API token>

  2. Login into your internal registry.

    export INTERNAL_REPOSITORY_URL="my.registry.internal"
    docker login $INTERNAL_REPOSITORY_URL

  3. Once authenticated, use the following command to pull the Styra images to your local Docker registry.

    export SELF_HOSTED_VERSION=0.12.0
    export STYRA_SERVICES=("activity" "agentbundle" "agentloader" "agentstatus" "agentstatusstore" \
    "analysis-api" "blueprints" "bundleregistry" "coordinator" "datasources" "environment-configurator" \
    "fetchdb" "gateway" "inspector" "job-bundles" "job-compliance" "logreplay" "metrics-exporter" "mock-opa" \
    "policies" "stacks" "systems" "tenants" "timeseries" "ui" "web")

    for SERVICE in "${STYRA_SERVICES[@]}"
    do
       docker pull registry.styra.com/styra/$SERVICE:on-prem-${SELF_HOSTED_VERSION}
    done

  4. Use the following command to tag the Styra images.

    for SERVICE in "${STYRA_SERVICES[@]}"
    do
        docker tag registry.styra.com/styra/$SERVICE:on-prem-${SELF_HOSTED_VERSION} $INTERNAL_REPOSITORY_URL/styra/$SERVICE:on-prem-${SELF_HOSTED_VERSION}
    done

  5. Use the following command to push the Styra images to your internal registry.

    for SERVICE in "${STYRA_SERVICES[@]}"
    do
       docker push $INTERNAL_REPOSITORY_URL/styra/$SERVICE:on-prem-${SELF_HOSTED_VERSION}
    done

Transferring Third-Party Images​

The Styra microservice images are uploaded to your internal repository. You can also upload the images for Elasticsearch and Postgres, which allows you to use a single source repository for all deployed images.

  1. Download and re-upload Elasticsearch images through the following command: 
    docker pull elasticsearch:7.16.5
    docker tag elasticsearch:7.16.5 $INTERNAL_REPOSITORY_URL/elasticsearch:7.16.5
    docker push $INTERNAL_REPOSITORY_URL/elasticsearch:7.16.5
  2. Download and re-upload Postgres image using the following command: 
    docker pull postgres:14.4
    docker tag postgres:14.4 $INTERNAL_REPOSITORY_URL/postgres:14.4
    docker push $INTERNAL_REPOSITORY_URL/postgres:14.4

Istio​

Self-Hosted Styra DAS does not require a service mesh to operate, but can still work if one is installed on the cluster. This section provides guidance on using Istio with Styra DAS.

The primary benefit to running Styra DAS alongside Istio, or other service meshes, is that doing so enables in-cluster HTTPS connections between Styra DAS services. By default, Styra DAS microservices communicate over HTTP connections. Enabling Istio sidecars on the styra-das namespace is a quick and reliable way of configuring inter-service encryption for customers already familiar with running Istio.

All Styra DAS microservices can be deployed with Istio proxy sidecars without issue. There are, however, two elements that need to be configured to work properly in an Istio-enabled cluster; the inbound gateway, and Elasticsearch (if running Elasticsearch in-cluster).

Prerequisites​

Running Styra DAS on an Istio-enabled cluster requires following:

note

When choosing which Istio Ingress to use, it is important to consider where TLS termination will happen in the inbound network path. If using Load Balancer TLS termination, for example, a basic Istio Ingress will be sufficient. Terminating TLS at the Kubernetes Ingress will require a TLS-capable Istio Ingress approach, such as Secure Gateways.

To configure Styra DAS for deployment on an Istio-enabled cluster, modify the settings in the istio section of values.yaml.

The available istio settings are:

  • istio.enabled: Setting this to true will cause the Helm chart to create a VirtualService and add Istio-specific annotations to any Elasticsearch pods.
  • istio.host: Corresponds to the hosts field in the created Configuration of the created VirtualService.
  • istio.gateway: Corresponds to the gateways field in the created Configuration of the created VirtualService.

The following configuration example would configure Styra DAS to run in an Istio-enabled cluster, if the cluster were running an Istio Ingress in the istio-system namespace:

istio:
 enabled: true
 host: "istio-system/<istio-gateway-name>"
 gateway: "<gateway-address>.<mydomain.com>"