Usage
Options
string
template hash (required, but Note: if you use this field, you can skip search_params, as they are automatically inferred from the template)
integer
template id (optional)
boolean
Disable default search param query args (alias:
--no-default)string
launch args string for create instance ex: “
--onstart onstart_wget.sh --env ‘-e ONSTART_PATH=https://s3.amazonaws.com/vast.ai/onstart_OOBA.sh’ --image atinoda/text-generation-webui:default-nightly --disk 64”string
deployment endpoint name (allows multiple workergroups to share same deployment endpoint)
integer
deployment endpoint id (allows multiple workergroups to share same deployment endpoint)
integer
default:"3"
number of workers to create to get an performance estimate for while initializing workergroup (default 3)
number
estimated GPU RAM req (independent of search string)
string
search param string for search offers ex: “gpu_ram>=23 num_gpus=2 gpu_name=RTX_4090 inet_down>200 direct_port_count>2 disk_space>=64”
number
[NOTE: this field isn’t currently used at the workergroup level] minimum floor load in perf units/s (token/s for LLms)
number
[NOTE: this field isn’t currently used at the workergroup level] target capacity utilization (fraction, max 1.0, default 0.9)
number
[NOTE: this field isn’t currently used at the workergroup level]cold/stopped instance capacity target as multiple of hot capacity target (default 2.0)
integer
min number of workers to keep ‘cold’ for this workergroup
Description
Create a new autoscaling group to manage a pool of worker instances. Example: vastai create workergroup--template_hash HASH --endpoint_name “LLama” --test_workers 5