Tutorial Docker Dashboard Reading From Gcp Bucket

Berglas

Build Status GoDoc

Berglas Logo

Berglas is a command line tool and library for storing and retrieving secrets on Google Cloud. Secrets are encrypted with Deject KMS and stored in Cloud Storage. An interoperable layer too exists with Secret Manager.

As a CLI, berglas automates the procedure of encrypting, decrypting, and storing data on Google Cloud.
As a library, berglas automates the inclusion of secrets into diverse Google Cloud runtimes.

Berglas is not an officially supported Google product.

Setup

Prerequisites

Install the Deject SDK for your operating arrangement. Alternatively, you tin can run these commands from Cloud Shell, which has the SDK and other pop tools pre-installed.

If y'all are running from your local machine, you as well need Default Awarding Credentials:
```
                  gcloud auth application-default login                                  
```
This will open a web browser and prompt for a login to your Google business relationship. On headless devices, yous will demand to create a service account. For more information, please see the authentication section.
Install the berglas CLI using one of the following methods:
- Install a pre-compiled binary for your operating system:
  - darwin/amd64
  - linux/amd64
  - windows/amd64
  This will download the latest version built against the main branch. To download a specific version, specify a git tag in place of "master" in the URL.
  
  Depending on your operating arrangement, you may demand to marking the downloaded binary as executable:
```
                      chmod +x /path/to/berglas                                          
```
- If y'all use Homebrew on macOS, you can install like this:
- Employ an official Docker container:
```
                      asia-docker.pkg.dev/berglas/berglas/berglas europe-docker.pkg.dev/berglas/berglas/berglas us-docker.pkg.dev/berglas/berglas/berglas                                          
```
  This volition pull the latest version built against the chief branch. To use a specific version, specify a git tag in place of "latest" in the URL.
  
  Note: the gcr.io/berglas/berglas:latest epitome remains for backwards compatibility, but new versions are not published there.
- Install from source (requires a working Go installation):
```
                      go install github.com/GoogleCloudPlatform/berglas@latest                                          
```
Export your projection ID equally an environment variable. The residue of this setup guide assumes this environment variable is gear up:
```
                  export PROJECT_ID=my-gcp-project-id                                  
```
Delight notation, this is the projection ID, not the projection name or projection number. You can discover the project ID by running gcloud projects listing or in the web UI.

Secret Manager Storage

Enable required services on the project:

                  gcloud services enable --project ${PROJECT_ID} \   secretmanager.googleapis.com

Cloud Storage Storage

Export your desired Cloud Storage bucket proper name. The rest of this setup guide assumes this environs variable is gear up:
```
                  export BUCKET_ID=my-secrets                                  
```
Replace my-secrets with the proper noun of your bucket. Set merely the name, without the gs:// prefix. This bucket should not be nonetheless!

Enable required services on the project:

                  gcloud services enable --project ${PROJECT_ID} \   cloudkms.googleapis.com \   storage-api.googleapis.com \   storage-component.googleapis.com

Bootstrap a Berglas surroundings. This will create a new Cloud Storage bucket for storing secrets and a Deject KMS key for encrypting data.
```
                  berglas bootstrap --project $PROJECT_ID --bucket $BUCKET_ID                                  
```
This command uses the default values. You tin customize the storage bucket and KMS key configuration using the optional flags. Run berglas bootstrap -h for more than details.

If you want total control over the creation of the Cloud Storage and Cloud KMS keys, delight see the custom setup documentation.
(Optional) Bootstrap a Berglas environment specifying a bucket location. By default the berglas bucket is created in the multi-regional location Us. You can specify your location by using the following control. Please see the list of supported locations in the GCP bucket location documentation page
```
                  export BUCKET_LOCATION=europe-west1 berglas bootstrap \   --project $PROJECT_ID \   --bucket $BUCKET_ID \   --bucket-location $BUCKET_LOCATION                                  
```
This control uses the default values. You tin can customize the storage bucket and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

If y'all want full control over the creation of the Cloud Storage and Cloud KMS keys, please run across the custom setup documentation.

(Optional) Enable Cloud Inspect logging on the saucepan:

Please notation this volition enable audit logging on all Deject KMS keys and all Cloud Storage buckets in the project, which may incur additional costs.

Download the exiting projection IAM policy:

                      gcloud projects get-iam-policy ${PROJECT_ID} > policy.yaml

Add together Deject Audit logging for Cloud KMS and Deject Storage:

                      cat <<EOF >> policy.yaml auditConfigs: - auditLogConfigs:   - logType: DATA_READ   - logType: ADMIN_READ   - logType: DATA_WRITE   service: cloudkms.googleapis.com - auditLogConfigs:   - logType: ADMIN_READ   - logType: DATA_READ   - logType: DATA_WRITE   service: storage.googleapis.com EOF

Submit the new policy:

                      gcloud projects set-iam-policy ${PROJECT_ID} policy.yaml

Remove the updated policy from local deejay:

CLI Usage

Create a cloak-and-dagger:

Using Hush-hush Manager storage:

                  berglas create sm://${PROJECT_ID}/foo my-hugger-mugger-data

Using Cloud Storage storage:

                  berglas create ${BUCKET_ID}/foo my-secret-data \   --central projects/${PROJECT_ID}/locations/global/keyRings/berglas/cryptoKeys/berglas-key

Grant access to a secret:

Using Hush-hush Manager storage:

                  berglas grant sm://${PROJECT_ID}/foo --member user:user@mydomain.com

Using Cloud Storage storage:

                  berglas grant ${BUCKET_ID}/foo --member user:user@mydomain.com

Access a secret'southward data:

Using Hugger-mugger Manager storage:

                  berglas admission sm://${PROJECT_ID}/foo my-secret-information

Using Deject Storage storage:

                  berglas access ${BUCKET_ID}/foo my-hush-hush-data

Spawn a child procedure with secrets populated in the child'south surroundings:

                  berglas exec -- myapp --flag-a --flag-b

This will spawn myapp with an environment parsed past berglas.

Access data from a specific version/generation of a secret:

Using Secret Manager storage:

                  berglas access sm://${PROJECT_ID}/foo#1 my-previous-clandestine-data

Using Cloud Storage storage:

                  berglas access ${BUCKET_ID}/foo#1563925940580201 my-previous-secret-data

Revoke admission to a secret:

Using Hole-and-corner Manager storage:

                  berglas revoke sm://${PROJECT_ID}/foo --fellow member user:user@mydomain.com my-previous-underground-data

Using Cloud Storage storage:

                  berglas revoke ${BUCKET_ID}/foo --fellow member user:user@mydomain.com

Delete a secret:

Using Hugger-mugger Managing director storage:

                  berglas delete sm://${PROJECT_ID}/foo

Using Cloud Storage storage:

                  berglas delete ${BUCKET_ID}/foo

In add-on to standard Unix exit codes, if the CLI exits with a known error, Berglas will exit with one of the following:

lx - API mistake. Berglas got a bad response when communicating with an upstream API.
61 - Misuse error. You gave unexpected input or behavior. Please read the mistake message. Open an outcome if you think this is a error.

The but exception is berglas exec, which will exit with the exit condition of its child command, if one was provided.

Integrations

App Engine (Flex) - When invoked via App Engine Flex, Berglas resolves environment variables to their plaintext values using the `berglas://reference syntax. This integration works with any language runtime because berglas serves every bit the entrypoint to the Docker container. Come across examples/appengineflex for examples and invocations.
App Engine (Standard) - When invoked via App Engine, Berglas resolves surroundings variables to their plaintext values using the berglas://reference syntax. This integration but works with the Become language runtime considering it requires importing the auto/ parcel. See examples/appengine for examples and invocations.
Cloud Run - When invoked via Cloud Run, Berglas resolves environs variables to their plaintext values using the berglas:// reference syntax. This integration works with any language runtime because berglas serves as the entrypoint to the Docker container. Encounter examples/cloudrun for examples and invocations.
Cloud Functions - When invoked via Deject Functions, Berglas resolves environment variables to their plaintext values using the berglas:// reference syntax. This integration simply works with the Go language runtime because it requires importing the auto/ packet. See examples/cloudfunctions for examples and invocations.
Cloud Build - When invoked via Deject Build, Berglas resolves environs variables to plaintext values using the berglas:// reference syntax. This integration only works with volume mounts, so all Berglas secrets need to specify the ?destination parameter. See examples/cloudbuild for examples and invocations.
Kubernetes - Kubernetes pods tin can consume Berglas secrets by installing a MutatingWebhook. This webhook mutates incoming pods with the berglas:// reference syntax in surround references to resolve at runtime. This integration works with any container, but all pods requesting berglas secrets must set an command in their Kubernetes manifests. See examples/kubernetes for samples and installation instructions.
Anything - Wrap whatsoever process with berglas exec -- and Berglas will parse any local environment variables with the berglas:// reference syntax and spawn your app as a subprocess with the plaintext environment replaced.

Logging

Both the berglas CLI and berglas library back up debug-style logging. This logging is off by default because information technology adds additional overhead and logs information that may exist security-sensitive.

The default logging beliefs for the berglas CLI is "text" (it can exist inverse with the --log-format flag). The default logging beliefs for the berglas library is structured JSON which integrates well with Deject Logging (it can be inverse to whatever valid formatter and yous can even inject your own logger).

Examples

Examples are bachelor in the examples/ binder.

Library Usage

Berglas is also a Go library that tin be imported in Go projects:

              import              ( 	_              "github.com/GoogleCloudPlatform/berglas/pkg/machine"              )

When imported, the berglas package will:

Download and decrypt any secrets that match the Berglas environment variable reference syntax in the environment.
Supersede the value for the environment variable with the decrypted secret.

Yous can likewise opt out of auto-parsing and call the library yourself instead:

              import              (              "context"              "log"              "os"              "github.com/GoogleCloudPlatform/berglas/pkg/berglas"              )              func              chief() {              ctx              :=              context.Background()              // This higher-level API parses the secret reference at the specified              // environment variable, downloads and decrypts the secret, and replaces the              // contents of the given environment variable with the secret upshot.              if              err              :=              berglas.Replace(ctx,              "MY_SECRET");              err              !=              nil              {              log.Fatal(err) 	}              // This lower-level API parses the secret reference, downloads and decrypts              // the underground, and returns the result. This is useful if you need to mutate              // the result.              if              v              :=              os.Getenv("MY_SECRET");              v              !=              ""              {              plaintext,              err              :=              berglas.Resolve(ctx,              v)              if              err              !=              nil              {              log.Fatal(err) 		}              os.Unsetenv("MY_SECRET")              os.Setenv("MY_OTHER_SECRET",              string(plaintext)) 	} }

For more examples and documentation, please see the godoc.

Authentication

Past default, Berglas uses Google Deject Default Application Credentials. If you lot have gcloud installed locally, ensure you have awarding default credentials:

              gcloud auth application-default login

On GCP services (like Cloud Build, Compute, etc), it will use the service business relationship fastened to the resource.

To use a specific service account, set up the GOOGLE_APPLICATION_CREDENTIALS environs variable to the filepath to the JSON file where your credentials reside on disk:

              export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/credentials.json

To learn more than, delight see the Google Deject Service Account documentation.

Authorisation

To control who or what has admission to a secret, use berglas grant and berglas revoke commands. These methods employ Cloud IAM internally. Whatsoever service account or entity using Berglas will need to authorize using the cloud-platform scope.

Secret Manager Storage

Creating a secret requires roles/secretmanager.admin on Surreptitious Director in the project.

Accessing a secret requires roles/secretmanager.secretAccessor on the secret in Undercover Manager.

Deleting a underground requires roles/secretmanager.admin on Secret Managing director in the projection.

Cloud Storage Storage

Creating a surreptitious requires roles/storage.objectCreator on the Cloud Storage bucket and roles/cloudkms.cryptoKeyEncrypter on the Cloud KMS primal.

Accessing a secret requires roles/storage.objectViewer on the Cloud Storage saucepan and roles/cloudkms.cryptoKeyDecrypter on the Cloud KMS cardinal.

Deleting a hush-hush requires roles/storage.objectAdmin on the Cloud Storage saucepan.

Implementation

Secret Managing director Storage

This department describes the Secret Manager implementation. This noesis is not required to use Berglas, merely it is included for security-conscious/curious users who want to larn about how Berglas works internally to build a threat model.

Berglas calls the Secret Manager API direct for all operations.

Cloud Storage Storage

This section describes the Cloud Storage implementation. This knowledge is not required to utilize Berglas, merely it is included for security-witting/curious users who want to learn about how Berglas works internally to build a threat model.

When encrypting a secret:

Berglas generates an AES-256-GCM data encryption cardinal (DEK) using Go's crypto package for each secret. (N.B. each clandestine has its own, unique DEK).
Berglas encrypts the plaintext data using the locally-generated DEK, producing encrypted ciphertext, prepended with the AES-GCM nonce.
Berglas encrypts the DEK using the specified Cloud KMS key, likewise known as a key encryption key (KEK). This process is called envelope encryption.
Berglas stores the Cloud KMS central name, encrypted DEK, and encrypted ciphertext as a single blob in Cloud Storage.

When decrypting a hole-and-corner:

Berglas downloads the blob from Deject Storage and separates the Deject KMS key name, encrypted DEK, and ciphertext out of the blob.
Berglas decrypts the DEK using Cloud KMS. This is part of envelope encryption.
Berglas decrypts the ciphertext data locally using the decrypted DEK.

Security & Threat Model

Run across the security and threat model.

FAQ

Q: Should I use Berglas or Secret Manager?
Berglas is compatible with Secret Manager and offers convenience wrappers around managing secrets regardless of whether they reside in Cloud Storage or Secret Manager. New projects should investigate using Clandestine Manager directly as it has less operational overhead and complexity, but Berglas will continue to support Cloud Storage + Deject KMS secrets.

Q: Is there a size limit on the data I tin encrypt?
Berglas is targeted at application secrets like certificates, passwords, and API keys. While its possible to encrypt larger binary files like PDFs or images, Berglas uses a a GCM cipher mode to encrypt data, pregnant the data must fit in memory and is limited to 64GiB.

Q: Why do you use envelope encryption instead of encrypting the data straight with Cloud KMS?
Envelope encryption allows for encrypting the data at the application layer, and information technology enables encryption of larger payloads, since Cloud KMS has a limit on the size of the payload information technology tin encrypt. By using envelope encryption, Cloud KMS always encrypts a fixed size information (the AES-256-GCM cardinal). This saves bandwidth (since large payloads are encrypted locally) and increases the size of the data which can be encrypted.

Q: Why does Berglas need permission to view my GCP resource?
Berglas communicates with the API to read the environment variables that were assail the resource at deploy time. Otherwise, a package could inject arbitrary surround variables in the Berglas format during application kicking.

Q: I renamed a underground in Cloud Storage and now information technology fails to decrypt - why?
Berglas encrypts secrets with boosted authenticated data including the proper noun of the secret. This reduces the take a chance an attacker tin can escalate privilege by disarming someone to rename a secret so they can gain access.

Q: Why is information technology named Berglas?
Berglas is a famous wizard who is best known for his secrets.

Contributing

Please see the contributing guidelines.

License

This library is licensed nether Apache two.0. Full license text is available in LICENSE.

Tutorial Docker Dashboard Reading From Gcp Bucket

Berglas

Setup

Prerequisites

Secret Manager Storage

Cloud Storage Storage

CLI Usage

Integrations

Logging

Examples

Library Usage

Authentication

Authorisation

Secret Manager Storage

Cloud Storage Storage

Implementation

Secret Managing director Storage

Cloud Storage Storage

Security & Threat Model

FAQ

Contributing

License

0 Response to "Tutorial Docker Dashboard Reading From Gcp Bucket"

Post a Comment

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel