Managing Clusters

Efficiently manage your Clusters using the Tensor One CLI. This guide covers starting, stopping, terminating, and listing your Clusters.

Start a Cluster

To start a single Cluster, use the following command:

tensoronecli start cluster $TENSORONE_CLUSTER_ID

To start multiple Clusters simultaneously (bulk start), run:

tensoronecli create clusters \
--name bulk-task \
--gpuType "NVIDIA GeForce RTX 3070,NVIDIA GeForce RTX 3080" \
--imageName "tensorone/your-image-name" \
--containerDiskSize 10 \
--volumeSize 0 \
--mem 1 \
--args "bash -c 'your-bash-command'"

This example starts up to 10 Clusters with specified GPUs and executes a bash command.

Stop a Cluster

To stop a single Cluster:

tensoronecli stop cluster $TENSORONE_CLUSTER_ID

Stop a Cluster After a Specific Duration

You can schedule a Cluster to stop after a certain period. For example, to stop a Cluster after 2 hours:

sleep 2h; tensoronecli stop cluster $TENSORONE_CLUSTER_ID &

This command waits for 2 hours before automatically stopping the Cluster. The & runs the process in the background.

Warning: Idle Clusters incur storage charges. Terminate Clusters if persistent storage isn't required.

Terminate a Cluster

Danger: Terminating a Cluster permanently deletes all data not stored on your network volume. Ensure you've saved all critical data beforehand.

To terminate a single Cluster:

tensoronecli remove cluster $TENSORONE_CLUSTER_ID

To terminate multiple Clusters simultaneously, for example, up to 40 Clusters named my-bulk-task:

tensoronecli remove clusters my-bulk-task --clusterCount 40

List Clusters

To list all your active Clusters, use:

tensoronecli get cluster

This provides a summary of currently active or available Clusters.