Usage
Arguments
integer
required
id of autoscale group to update
Options
number
minimum floor load in perf units/s (token/s for LLms)
number
target capacity utilization (fraction, max 1.0, default 0.9)
number
cold/stopped instance capacity target as multiple of hot capacity target (default 2.5)
integer
min number of workers to keep ‘cold’ for this workergroup
integer
number of workers to create to get an performance estimate for while initializing workergroup (default 3)
number
estimated GPU RAM req (independent of search string)
string
template hash (Note: if you use this field, you can skip search_params, as they are automatically inferred from the template)
integer
template id
string
search param string for search offers ex: “gpu_ram>=23 num_gpus=2 gpu_name=RTX_4090 inet_down>200 direct_port_count>2 disk_space>=64”
boolean
Disable default search param query args (alias:
--no-default)string
launch args string for create instance ex: “
--onstart onstart_wget.sh --env ‘-e ONSTART_PATH=https://s3.amazonaws.com/public.vast.ai/onstart_OOBA.sh’ --image atinoda/text-generation-webui:default-nightly --disk 64”string
deployment endpoint name (allows multiple workergroups to share same deployment endpoint)
integer
deployment endpoint id (allows multiple workergroups to share same deployment endpoint)
Description
Example: vastai update workergroup 4242--min_load 100 --target_util 0.9 --cold_mult 2.0 --search_params “gpu_ram>=23 num_gpus=2 gpu_name=RTX_4090 inet_down>200 direct_port_count>2 disk_space>=64” --launch_args “--onstart onstart_wget.sh --env ‘-e ONSTART_PATH=https://s3.amazonaws.com/public.vast.ai/onstart_OOBA.sh’ --image atinoda/text-generation-webui:default-nightly --disk 64” --gpu_ram 32.0 --endpoint_name “LLama” --endpoint_id 2