dataset push

Pushes a new dataset to a FlexAI. Multiple files can be uploaded at once by using multiple instances of the —file flag, or by pointing the —file flag to a directory containing dataset files. The Datasets Manager Service page provides more information on how to upload (or push) both local files and files from a Remote Storage Provider.

Usage

flexai dataset push <dataset_name> [(--file <path_on_filesystem>=<fcs_dataset_path> ...) | (--storage-provider <storage_provider_name> --source-path <source_path>)] [flags]

Arguments

Argument	Type	Required	Description
`dataset_name`	string	Yes	The name of the Dataset you want to create. Dataset names must be unique within a given FlexAI organization and must follow the Resource Naming conventions.

Flags

Flag	Short	Type	Description
`--file`	`-f`	string/key-value	The local path to the dataset files on the host machine running the FlexAI CLI. The Dataset will be mounted and made available to your code under the `/input/<dataset_name>` path. Can also be a mapping between a local path and the path where the dataset file will be stored in the Dataset. If no target nested path is provided, the file will be stored at the root of the Dataset (`/input/<dataset_name>/<file_name>`). If a target directory path is provided, the file will be stored under that path (`/input/<dataset_name>/<target_nested_path>/<file_name>`). If the source path is a directory, the entire directory will be uploaded recursively to the Dataset. To learn more about Dataset Organization and their File Structure, visit the Datasets Manager Service page.
`--help`	`-h`	boolean	Displays this help page.
`--source-path`		string	The path to the bucket and file or directory to be pushed. Syntax: `<bucket_name>/<path>`. Used in together with the `--storage-provider` flag.
`--storage-provider`		string	The name of the Remote Storage Provider Connection to use for the Dataset upload. You can list the available Remote Storage Provider Connections using the `flexai storage list` command.
`--verbose`	`-v`	boolean	Provides more detailed output when pushing a Dataset.

Examples

Push local files to a dataset

flexai dataset push my-dataset —file /path/to/file1.csv —file /path/to/file2.txt

flexai dataset push my-dataset --file /path/to/file1.csv --file /path/to/file2.txt

Push files with custom paths

flexai dataset push my-dataset —file /path/to/file1.csv=train/data/file1.csv —file /path/to/file2.txt=eval/data/file2.txt

flexai dataset push my-dataset --file /path/to/file1.csv=train/data/file1.csv --file /path/to/file2.txt=eval/data/file2.txt

Push from remote storage provider

flexai dataset push my-dataset —storage-provider my-s3-provider —source-path bucket-name/path/to/data/

flexai dataset push my-dataset --storage-provider my-s3-provider --source-path bucket-name/path/to/data/

Getting Started

Inference

Fine-tuning

Training

Platform Services

Interactive Development

CLI

Console

Best Practices

FAQ

Blueprints

Usage

Arguments

Flags

Examples

Push local files to a dataset

Push files with custom paths

Push from remote storage provider

​Usage

​Arguments

​Flags

​Examples

​Push local files to a dataset

​Push files with custom paths

​Push from remote storage provider

Usage

Arguments

Flags

Examples

Push local files to a dataset

Push files with custom paths

Push from remote storage provider