Vault Integration

Using the Vault Integration allows connecting to an external vault service and retrieve secrets without storing them in Cloudomation.

Use Cases

The Vault Integration offers the possibility to authenticate to external services:

Authenticate yourself towards services integrated in Cloudomation
Use secrets (passwords, usernames, keys, ...) in flows without "hard-coding" them
Interact with your Vault service (e.g. write secrets)

Concept

Cloudomation offers the integration of a HashiCorp Vault (for more information see https://www.vaultproject.io/).

The user can interact with a Vault using:

Vault Configuration: configure access to your Vault service in Cloudomation.
Connections: configure which secrets will be used from your Vault service for a particular connection. When the saved connection is used, it will retrieve the secrets.
TaskVAULT: provides an interface for exhaustive interaction with your Vault (read, write, update, versioning of secrets, change secret-metadata, ...).

Configuration

To use the Vault integration you have to

Create a Vault Configuration

You can create a Vault configuration in the User Interface by pressing the "Create" button and selecting "Vault config":

The buttons to create a Vault configuration

The new Vault configuration is opened and you can directly modify its fields.

Please see the table below for fields used in the Vault configuration and their meanings:

Field	Description	Example
Enabled	If unset, Cloudomation will not use this Vault configuration.
Auto-renew	If set, Cloudomation will try to renew the token before it expires. Renewal will only succeed if the MAX_TTL of the token is not reached. Please refer to token renew for details.
Vault URL	The URL to your vault installation	https://vault.example.com:8200
CA certificate	A certificate to verify the identity of the vault. Only needed if the Vault installation uses a self-signed certificate.
Token	A Vault access token which is used to fetch secrets.

Attach References to Secrets in Connections

You can attach references to secrets to connections. Open a connection and press the "Add vault secret" button:

The add vault secret button

Please see the table below for the fields you'll have to specify and their meanings:

Field	Description	Example
Vault config ID	From which Vault configurartion to fetch the secret
Engine path	The path at which secrets of a specific vault engine are stored	`my-kv-engine`
Secret path	Path to a secret within an engine type; can be prefixed with e.g. a method	`method/my-secret`

Depending on the secret engine being used, different paths and prefixes are available and required. Please refer to Secrets Engines for details.

Required prefix for a secret in the key-value engine using version 2 is data/!

After a vault secret is attached to a connection, its values can be used in the connection value. You can map the value of a vault secret key to any value of the connection:

Example

Map vault secrets to connection values.

Let's assume there is a vault secret stored in a key-value version 2 secret engine below secret in the path oracle which contains two keys:

user: my-user
password: my-secret-password

Create a connection for the SQLORACLE task type, call it "oracle" and enter the following into the value field:

host: my-oracle-server
service_name: xe
user: vault.secret(user)
password: vault.secret(password)

Click on Add vault secret, choose the Vault configuration, enter the engine path "secret", prefix and secret path "data/oracle".

The example above assumes that the secret store being used in vault is of type key-value version 2. You can find more information about vault secret engines at https://www.vaultproject.io/api-docs/secret

Use the Vault Integration

The preferred method to read secrets is the use of a Connection.

The preferred method to write secrets is the use of VaultConfig.write_secret() method.

Read Secrets with Connections

A connection which has Vault secrets attached and mapped will automatically fetch and use the secrets every time the connection is used. The secret values are not stored in Cloudomation.

Example

Use a connection which has a vault secret attached.

import flow_api
def handler(system: flow_api.System, this: flow_api.Execution):
    oracle_db_version = this.connect(
        'oracle',
        execute='SELECT * FROM v$version',
    ).get('output_value')['result']
    this.log(oracle_db_version)
    return this.success('all done')

Using vault secrets in connections is a safe way where secrets will not be stored or exposed within Cloudomation.

Write Secrets

Cloudomation can write secrets to a key-value Vault engine using the Flow-API method VaultConfig.write_secret().

Example

Write a secret to vault.

import flow_api
def handler(system: flow_api.System, this: flow_api.Execution):
    system.vault_config('my-vault').write_secret(
        engine_path='my-kv-engine',
        secret_path='data/my-new-secret',
        data={
            'username': 'cloudomation',
            'password': 'super-secret',
        },
    )

Use the Vault Connector Type

To use the Vault connector type it is not required to have a Vault configuration set up. All connection parameters to the Vault have to specified. It is possible to create a connection of the Vault connector type.

If not properly used secrets could become exposed within your flow or executions. Use this method with caution!

Example

Use the Vault connector type

import flow_api
def handler(system: flow_api.System, this: flow_api.Execution):
    # create a secret
    this.task(
        'VAULT',
        host='https://my-vault-host:8200',
        engine_path='kv',
        secret_path='data/my-secret',
        data={
            'secret-key': 'secret-value',
        },
        token='my-vault-token',
    )
    # read a secret
    secret_value = this.task(
        'VAULT',
        host='https://my-vault-host:8200',
        engine_path='kv',
        secret_path='data/my-secret',
        version=None,  # read latest version
        token='my-vault-token',
    ).get('output_value')['result']['data']['data']
    assert secret_value == {'secret-key': 'secret-value'}
    # destroy all versions of secret
    this.task(
        'VAULT',
        host='https://my-vault-host:8200',
        engine_path='kv',
        secret_path='my-secret',
        mode='delete_metadata',
        token='my-vault-token',
    )
    return this.success('all done')

Knowledge Base — Previous

Third-Party Python Modules

Next — Knowledge Base

Webhooks