Dynamic Secrets [Vault]

6 min readMar 13, 2021

AWS Access and Secrets

In AWS , best practices is to always create an IAM user for managing infrastructure(don’t use root account) , and IAM always provides us two keys: access and secret key to login as a non-root user which has its own permissions or policies. But here we do the whole process of creating IAM user is manually .

Dynamic Secrets

The above process which we have done manually, now can be achieve by a single click….!!

It means, we can create multiple IAM users with their access and secrets, and also have the capability to revoke the IAM user within a specific time. Now we don’t have to always look for the IAM credentials and security.

How we can achieve this….?

Vault tightly controls access to secrets and encryption keys by authenticating against trusted sources of identity such as Active Directory, LDAP, Kubernetes, Cloud Foundry, and cloud platforms. Vault enables fine grained authorization of which users and applications are permitted access to secrets and keys.

It is mainly designed to manage our secrets such as databases and can also secure the credentials for cloud platforms as well i.e. AWS, azure…….

It also provides his own web-UI or the dashboard for better understanding .

Lets start the Vault server !!

Here I setup vault for windows, you can also use Linux.

Download

Unzip and add the folder to the path

Vault Installed successfully..!!

vault --version
Vault v1.6.3 (b540be4b7ec48d0dd7512c8d8df9399d6bf84d76)

Start the Vault server

Note: Don’t use ‘-dev’ for the production, it’s just for demo or for practice.

vault server -dev

Set the Vault address

set VAULT_ADDR=http://127.0.0.1:8200

Open the browser with Vault address

Use the root token to sign-in Vault.

Now configure AWS secret engine

vault secrets enable aws
Success! Enabled the aws secrets engine at: aws/

Configure Access and Secret key

In Linux

vault write aws/config/root \
    access_key=<Your AWS Access key ID> \
    secret_key=<Your AWS Secret access key> \
    region=ap-south-1

Add policy/permissions to the IAM user

In Linux

vault write aws/roles/myrole \
        credential_type=iam_user \
        policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*"
    }
  ]
}
EOF

Here I added the admin permissions to the IAM user.

Set the Lease time

vault write aws/config/lease lease=300s lease_max=300s

It will revoke the lease at the given time you provided.

Generate the Access and Secret keys

vault read aws/creds/myrole

Output

vault read aws/creds/myrole
Key                Value
---                -----
lease_id           aws/creds/myrole/s1TNP7R4kzmVo8CaZw0x4Fv3
lease_duration     5m
lease_renewable    true
access_key         AKIA3ABYZUQJ6WK27QWT
secret_key         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
security_token     <nil>

Every time you read the access and secret key , it will always be different from previous and all IAM user will have the same lease time after it’s generated.

Go to AWS console and check if there is any new user has been created……..

Finally Vault is configured….!!

Terraform state file

Terraform is use to create infrastructure as a code, so to maintain the versioning of the infrastructure as a code terraform.tfstate file records all the metadata . So we can see in future that what we have created had what needs to be update in future.

As soon we plan/apply terraform script the terraform.tfstate file gets generated automatically.

Best practices is to save the state file in high availability and encrypted for security reasons. So terraform helps us to save the state file in various platforms such as in clouds: Azure, AWS, GCP.

I already done a practice where I save the state file in AWS S3 bucket. So if you want to learn how to save your state file in S3 click here.

Here we will use the Hashicorp Consul to store our state file.

Lets start the Consul

Download

Unzip and add the folder to the path

Consul installed successfully..!!

consul --version
Consul v1.9.4
Revision 10bb6cb3b
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

Start the Consul server

consul agent -dev

It will start on localhosts of 8500 port number

So now we have both the servers running up, Vault and Consul.

The scenario here is, we will create the dynamic secrets from Vault which helps us to provision AWS and to deploy our infrastructure. The state file, we will be saving remotely on Consul.

Terraform Time….!!

terraform {required_version = ">=0.11.8"}provider "vault" {address = "http://127.0.0.1:8200"}data "vault_aws_access_credentials" "aws_creds" {backend = "aws"role = "myrole"}provider "aws" {access_key  =   "${data.vault_aws_access_credentials.aws_creds.access_key}"secret_key  =   "${data.vault_aws_access_credentials.aws_creds.secret_key}"region      =   "ap-south-1"}terraform {backend "consul" {address = "http://127.0.0.1:8500"path    = "tf/state"}}locals {rules = [{description = "allow http port for all",port = 80,cidr_blocks = ["0.0.0.0/0"],},{description = "allow ssh port for all",port = 22,cidr_blocks = ["0.0.0.0/0"],}]}resource "aws_security_group" "vault_sg" {name        = "vault"description = "vault"dynamic "ingress" {for_each = local.rulescontent {description = ingress.value.descriptionfrom_port   = ingress.value.portto_port     = ingress.value.portprotocol    = "tcp"cidr_blocks = ingress.value.cidr_blocks}}tags  = {Name = "vault_sg"}}resource "aws_instance" "vault_inst" {ami = "ami-08f63db601b82ff5f"instance_type = "t2.micro"subnet_id = "subnet-05a9ffd7b7df35128"key_name    = "acceptor"security_groups = [aws_security_group.vault_sg.id]tags    =   {Name    = "vault_inst"}}

Terraform init

It will initialize the backend for consul.

Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.