Skip to main content

​
Dataset Lifecycle

​
Creation and Upload

  1. Prepare data - Organize files locally or identify cloud storage locations
  2. Upload Dataset - Use either the FlexAI Console or CLI to upload the Dataset
  3. Verify completion - Check upload status progress until it reaches the available status
  4. Ready for training - Dataset becomes available for Training or Fine jobs

​
Statuses

Monitor Dataset lifecycle with clear status indicators:
StatusSourceDescription
uploadingHostDataset files are being transferred from a host machine
syncingRemote Storage ProviderDataset files are being synchronized from a Remote Storage Provider
pendingHost or Remote Storage ProviderThe Dataset is being processed by FlexAI
availableHost or Remote Storage ProviderThe Dataset is ready for use in Training or Fine-tuning Jobs
errorHost or Remote Storage ProviderThe upload process failed. Inspecting its details can provide more information: flexai dataset inspect <dataset_name>

​
Dataset lifecycle

You will notice that the Dataset lifecycle follows a clear progression through its statuses, with the only difference being the source of the upload: 
"uploading (Host)" -> pending {style.animated: true}
"uploading (Host)" -> error
"syncing (Remote Storage Provider)" -> pending {style.animated: true}
"syncing (Remote Storage Provider)" -> error
pending -> available {style.animated: true}
pending -> error