Ringmaster is a cloud infrastructure orchestration tool. It runs through directories of scripts in alphabetical order so you have exact control over when each script is processed.

Ringmaster focuses on two main areas:

  1. Running cloudformation scripts
  2. Templating k8s configuration files

The overall philosophy is one command to bring up:

ringmaster stack up

or tear down:

ringmaster stack down

whole systems.

Why not Terraform?

Ringmaster doesn’t try to replace or use Terraform. For a declarative DSL on AWS ringmaster uses regular cloudformation yaml files.

Terraform has a state file and resources must not be modified outside of Terraform.

Terraform can manage K8s as native Terraform resources. but this would prevent using the kubectl command to load K8s configuration files.

Why not Helm?

Helm charts work with ringmaster as part of a stack. Usually you need several independent helm charts for a whole stack, eg:

  • external secrets
  • load balancer
  • ingress controller

Ringmaster can run helm charts in the right order and with any associated cloudformation.

Scripts and stacks

Directories of scripts are called stacks in Ringmaster. A stack should contain subdirectories of scripts to run, in alphabetical order. Prepending a 4-digit number is an easy way to get consistent results:

stack
├── 0110-eks-cluster
│   └── cluster.eksctl.yaml
├── 0510-microservices
│   ├── api.kubectl.yaml
│   ├── cdn.kubectl.yaml
│   ├── demo.kubectl.yaml
│   └── website.kubectl.yaml
└── 0520-ingresses
    ├── api.kubectl.yaml
    ├── cdn.kubectl.yaml
    ├── demo.kubectl.yaml
    └── www.kubectl.yaml

The stack name is taken from the directory name which is just stack in this example.

What else does ringmaster do?

Ringmaster has several handlers which are triggered when a file matches a pattern. eg *.sh activates the BASH handler:

  • BASH scripts
  • Python scripts
  • Snowflake (query and scripts)
  • helm
  • IAM policies and roles
  • EKSCTL
  • Kustomizer
  • Kubectl

There is a minimal DSL to support:

  • AWS Secrets Manager
  • Remote cloudformation scripts
  • Cloudflare

Databags

Ringmaster uses yaml files called databags to share key-value pairs of data between steps. Steps are able to add to the databag as needed while a stack is going up. Databags are accessed in:

  • cloudformation templates via parameters
  • BASH scripts via environment variables
  • Python scripts as a dict
  • K8s configuration files and most other integrations as jinja2 template
  • variables.

Environments

Ringmaster supports environments such as dev/prod clusters by using different AWS credentials and databags. Environments are specified via the --environment ENVIRONMENT_NAME argument when running ringmaster. Files are kept in the .env directory. You can specify common variables in .env/databag.yaml and these will be merged with the selected environment databag:

.env
├── databag.yaml
├── dev
│   ├── connections.yaml
│   ├── databag.yaml
│   └── output_databag.yaml
└── prod
    ├── connections.yaml
    ├── databag.yaml
    └── output_databag.yaml

Processed files

Ringmaster processes jinja2 templates during a run and saves the output of any to the .processed/ENVIRONMENT_NAME directory, eg:

.processed/dev/stack/
├── 0110-eks-cluster
│   └── cluster.eksctl.yaml
├── 0510-microservices
│   ├── api.kubectl.yaml
│   ├── cdn.kubectl.yaml
│   ├── demo.kubectl.yaml
│   └── website.kubectl.yaml
└── 0520-ingresses
    ├── api.kubectl.yaml
    ├── cdn.kubectl.yaml
    ├── demo.kubectl.yaml
    └── www.kubectl.yaml

By keeping the processed files around or by committing them to git they can be used with kubectl or eksctl directly if required.

Connections and credentials

AWS

There is a rudimentary support for selecting the AWS profile to use in .env/ENVIRONMENT_NAME/connections.yaml:

aws:
  profile: "dev"

However this support is incomplete. Run with AWS_PROFILE=xxx ringmaster ... as a workaround.

EKSCTL

Not currently configured directly, run with AWS_PROFILE=xxx ringmaster ... as a workaround.

Kubectl/helm

Ringmaster runs kubectl and helm with a --context or --kube-context argument respectively. The value is sourced from .env/ENVIRONMENT_NAME/connections.yaml , eg:

k8s:
  profile: "[email protected]"

Snowflake

Ringmaster has built-in support to query and execute scripts in snowflake. Authentication is configured in two places:

  1. Select which credentials to use in .env/ENVIRONMENT_NAME/connections.yaml
snowflake:
  profile: "dev"
  1. Credentials are stored in: ~/.ringmaster/snowflake.yaml
dev:
  credentials:
    user: "someuser"
    password: "topsecret"
    account: "accountnumber"

By keeping the credentials in the home directory and out of the source tree they are protected from ever being committed to git.