Usage
Arguments
integer
required
id of endpoint group to update
Options
number
minimum floor load in perf units/s (token/s for LLms)
number
minimum floor load in perf units/s (token/s for LLms), but allow handling with cold workers
string
active, suspended, or stopped
number
target capacity utilization (fraction, max 1.0, default 0.9)
number
cold/stopped instance capacity target as multiple of hot capacity target (default 2.5)
integer
min number of workers to keep ‘cold’ when you have no load (default 5)
integer
max number of workers your endpoint group can have (default 20)
string
deployment endpoint name (allows multiple workergroups to share same deployment endpoint)
number
maximum seconds requests may be queued on each worker (default 30.0)
number
target seconds for the queue to be cleared (default 10.0)
integer
seconds of no traffic before the endpoint can scale to zero active workers
Description
Example: vastai update endpoint 4242--min_load 100 --target_util 0.9 --cold_mult 2.0 --endpoint_name “LLama”