Manage Clusters

Start a Cluster

To start a single Cluster, enter the command tensoronecli start cluster. You can pass the environment variable TENSORONE_CLUSTER_ID to identify each Cluster.

tensoronecli start cluster $TENSORONE_CLUSTER_ID

To start Clusters in bulk, enter the command tensoronecli create clusters.

For example, to start up to 10 Clusters with the name bulk-task on 3070 and 3080 GPUs and run a bash command, enter the following command.

tensoronecli create clusters \
--name bulk-task \
--gpuType "NVIDIA GeForce RTX 3070,NVIDIA GeForce RTX 3080" \
--imageName "tensorone/your-image-name" \
--containerDiskSize 10 \
--volumeSize 0 \
--mem 1 \
--args "bash -c 'your-bash-command'"

Stop a Cluster

To stop a Cluster, enter the following command.

tensoronecli  stop cluster$TENSORONE_CLUSTER_ID

Stop a Cluster after a specific time

You can also stop a Cluster after a specific amount of time. For example, the following command sleeps for 2 hours, and then stops the Cluster.

Use the following command to stop a Cluster after 2 hours:

sleep 2h; tensoronecli stop cluster $TENSORONE_CLUSTER_ID &

This command uses sleep to wait for 2 hours before executing the tensoronecli stop cluster command to stop the Cluster. The & at the end runs the command in the background, allowing you to continue using the SSH session.

Warning You are charged for storing idle Clusters. If you do not need to store your Cluster, be sure to terminate it next.

Terminate a Cluster

Danger Terminating a PClusterd permanently deletes all data outside your network volume. Be sure you've saved any data you want to access again.

To remove a single Cluster, enter the following command.

tensoronecli remove cluster $TENSORONE_CLUSTER_ID

You can also remove Clusters sin bulk. For example, the following command terminates up to 40 Clusters with the name my-bulk-task.

tensoronecli remove clusters my-bulk-task --cluster Count 40

List Clusters

If you're using the command line, enter the following command to list your Clusters.

tensoronecli get cluster