Secret Manager

The FlexAI Secret Manager provides a secure, encrypted storage solution for managing sensitive information such as API keys, access tokens, credentials, and configuration data. All secrets are encrypted at rest and can be safely referenced in your training and inference workloads.

The Secret Manager enables you to:

• Securely store credentials for cloud storage providers (AWS S3, Google Cloud Storage, etc.)
• Manage API tokens for services like Hugging Face Hub
• Store configuration files as secrets for easy deployment
• Reference secrets in workloads without exposing sensitive data
• Inject environment variables automatically during training and inference

Key Features

Encrypted Storage

All secrets are encrypted at rest using industry-standard encryption protocols

Flexible Content Types

Store strings, tokens, or entire files as secrets for maximum flexibility

Workload Integration

Automatically inject secrets as environment variables in Training and Inference jobs

Access Control

Secure access patterns ensure secrets are only available to authorized workloads

Common Use Cases

Cloud Storage Authentication

Store access credentials for Remote Storage Connections to enable seamless Dataset uploads and Checkpoint exports:

• AWS S3: Store Secret Access Keys for S3 bucket access
• Google Cloud Storage: Store Service Account JSON files
• Azure Blob Storage: Store connection strings and access keys

Model Repository Access

Securely authenticate to Hugging Face to pull Models and Datasets.

Training Configuration

Store sensitive configuration data and credentials:

• Database connections: Store connection strings and credentials
• External APIs: Store API keys for third-party services
• Certificates: Store SSL/TLS certificates and private keys
• Experiment Tracking: Store API keys for experiment tracking services, such as Weights & Biases, Neptune AI, or MLflow

Security Best Practices

• Never include secrets in code - Always use the Secret Manager instead of hardcoding credentials
• Use descriptive names - Name secrets clearly to identify their purpose and scope
• Rotate credentials regularly - FlexAI Secret Manager allows you to update Secrets periodically and remove unused ones

Getting Started

Using the FlexAI Console

Visit the Secret Manager section 🔗 of the FlexAI Console to create, update, or delete Secrets through an intuitive web interface.

Using the FlexAI CLI

Creating a Secret

1. Create a Secret by using the flexai secret create command, which will receive the name of the secret as its only argument. In this case we will use hf_token to store a Hugging Face Access Token.

   flexai secret create hf_token

2. You will be prompted to enter the value for the secret. You can either type in the value directly or paste it. In any case, the value will not be displayed in the terminal.

   flexai secret create hf_token
   Secret Value: █

Example

Passing a value from Standard input (stdin)

You can pass the value of a Secret directly from standard input (stdin) by piping in the value. This is particularly useful for automated scripts, when you want to avoid interactive prompts, or when you want to store the contents of a file as a Secret —like a Google Cloud Service Account File.

Piping in the value and using the -p/--value-stdin flag:

cat gcp-service-account.json | flexai secret create --value-stdin secret-from-sa-file

Example

Updating a Secret

An existing Secret can be updated by using the flexai secret update command, which will receive the name of the secret as its only argument.

1. Let’s update hf_token to modify the Hugging Face Access Token’s value.

   flexai secret update hf_token

2. You will be prompted to enter the new value for the Secret, just like when creating it.

   flexai secret update hf_token
   Secret Value: █

Example

Using Secrets

Secrets can be passed to Workloads in different ways: some commands flags receive Secret names as their value, and others explicitly require the name of a Secret and a name to associate it with, such as the -S/--secret flag of the flexai training run and flexai training debug-ssh commands.

Training or Fine-tuning Workloads

Creating a Training Job

flexai training run test-123 \
  --dataset open_web \
  --repository-url https://github.com/flexaihq/nanoGPT/ \
  --secret HF_TOKEN=hf_token \
  --secret WANDB_API_KEY=wandb-key \
  -- train.py ...

debug-ssh Sessions

Starting a debug-ssh session

flexai training debug-ssh \
  --secret HF_TOKEN=hf_token_dev \
  --secret SA_FILE=secret-from-sa-file \
  -S DB_API_KEY=db-api-key-staging \
  --repository-url https://github.com/flexaihq/nanoGPT/ \
  --vscode

Note

Note that Training and Fine-tuning Workloads, as well as a debug-ssh sessions can receive multiple instances of the -S/--secret flag.

Inference Endpoints

The flexai inference serve command’s --hf-token-secret and --api-key-secret flags expect the name of a Secret:

Creating an Inference Endpoint

flexai inference serve llm-text-inference-prod \
  --hf-token-secrethf_token_prod \
  --api-key-secret api-key-prod \
  -- --model=mistralai/Mistral-7B-Instruct-v0.1 ...