Managing Secrets

When tasks or container image builds require sensitive credentials—such as API tokens, database passwords, or private registry pull keys—storing them in plaintext inside source code or configuration files exposes them to security risks. In flyte-sdk, secret management is handled through flyte.remote.Secret, which communicates with the backend SecretService to securely create, inspect, list, and delete secrets scoped to your organization, project, and domain.

Managing Secrets with the Python SDK

To create and manage secrets on your cluster, initialize the Flyte remote client and use the flyte.remote.Secret class methods.

import flyte
from flyte.remote import Secret

# Initialize the Flyte client with cluster credentials
flyte.init(
    endpoint="dns:///localhost:8080",
    insecure=True,
    project="flytesnacks",
    domain="development",
)

# 1. Create a generic secret
Secret.create(name="openai-api-key", value="sk-example-token-12345")

# 2. Create a private container registry image pull secret
Secret.create(
    name="docker-hub-credentials",
    value=b'{"auths":{"https://index.docker.io/v1/":{"auth":"ZXhhbXBsZTpwYXNzd29yZA=="}}}',
    type="image_pull",
)

# 3. Retrieve secret metadata
secret_info = Secret.get("openai-api-key")
print(f"Secret name: {secret_info.name}, Type: {secret_info.type}")

# 4. List all secrets in the current project/domain
for s in Secret.listall():
    print(f"Found secret: {s.name} ({s.type})")

# 5. Delete a secret
Secret.delete("openai-api-key")

Key Components

flyte.remote.Secret.create(name, value, type="regular"): Creates a secret on the remote control plane. value accepts either str (stored in string_value of SecretSpec) or bytes (stored in binary_value). The type argument accepts "regular" (which maps to protobuf SecretType.SECRET_TYPE_GENERIC) or "image_pull" (which maps to SecretType.SECRET_TYPE_IMAGE_PULL_SECRET).
flyte.remote.Secret.get(name): Fetches metadata for an existing secret, returning a Secret wrapper containing protobuf metadata such as creation time, overall status, and cluster presence status.
flyte.remote.Secret.listall(limit=100): Fetches all secrets registered under the active organization, project, and domain. Handles gRPC pagination continuation tokens automatically.
flyte.remote.Secret.delete(name): Sends a DeleteSecretRequest for the specified secret name within the configured project and domain.

Variations & Common Scenarios

Creating Image Pull Secrets from Binary Data or Files

Image pull secrets authenticate Kubernetes clusters with private container registries (e.g., Docker Hub, GitHub Container Registry, AWS ECR). Pass a binary .dockerconfigjson string or bytes directly:

from pathlib import Path
from flyte.remote import Secret

docker_config = Path("~/.docker/config.json").expanduser().read_bytes()

Secret.create(
    name="ghcr-pull-secret",
    value=docker_config,
    type="image_pull",
)

Asynchronous Remote Operations

All classmethods on flyte.remote.Secret are wrapped with @syncify. In asynchronous applications or services, access the underlying async coroutines and async iterators using the .aio property:

import asyncio
from flyte.remote import Secret

async def manage_secrets_async():
    # Asynchronous secret creation
    await Secret.create.aio(name="async-api-key", value="secret-val")

    # Asynchronous secret retrieval
    secret_obj = await Secret.get.aio("async-api-key")
    print("Fetched:", secret_obj.name)

    # Asynchronous pagination with async for
    async for s in Secret.listall.aio(limit=50):
        print("Secret:", s.name, s.type)

    # Asynchronous deletion
    await Secret.delete.aio("async-api-key")

asyncio.run(manage_secrets_async())

Inspecting Secret Metadata and Cluster Distribution

For security, Secret.get() and Secret.listall() do not expose the raw secret value. Instead, they return a Secret dataclass wrapping definition_pb2.Secret, which provides metadata and multi-cluster synchronization status:

from flyte.remote import Secret

secret = Secret.get("my-db-password")

# Access basic properties
print("Name:", secret.name)
print("Type:", secret.type)  # "regular" or "image_pull"

# Access underlying protobuf metadata
meta = secret.pb2.secret_metadata
print("Created:", meta.created_time.ToDatetime().isoformat())
print("Overall Status:", meta.secret_status.overall_status)

# Check cluster presence status across connected Kubernetes clusters
for cluster_status in meta.secret_status.cluster_status:
    print(f"Cluster: {cluster_status.cluster.name}, Status: {cluster_status.presence_status}")

# Convert to JSON-compatible dictionary via ToJSONMixin
secret_dict = secret.to_dict()