Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support passing variables in config.pbtxt #7530

Open
riZZZhik opened this issue Aug 15, 2024 · 0 comments
Open

Support passing variables in config.pbtxt #7530

riZZZhik opened this issue Aug 15, 2024 · 0 comments

Comments

@riZZZhik
Copy link

riZZZhik commented Aug 15, 2024

Is your feature request related to a problem? Please describe.
Variables like max_batch_size, dynamic_batching, etc. need to be changed based on the hardware environment and third-party claims every run
Right now there isn't a convenient way to do so

Describe the solution you'd like
Support environment variables inside config

example config.pbtxt:

...
max_batch_size: ${MAX_BATCH_SIZE} or $$MAX_BATCH_SIZE
...

and/or pass them to tritonserver command:

tritonserver --model-repository models --custom_variables '{"var1": "value1"}'

Describe alternatives you've considered
I wrote shell script to be runned before tritonserver with sed command to replace based on env
It works, but it isn't convenient and poses issues inside Kubernetes

for config in $(find $MODELS_PATH -type f -name 'config.pbtxt'); do
    sed -i -e "s/max_batch_size: [0-9]\{1,\}/max_batch_size: ${max_batch_size}/g" $config
done
@riZZZhik riZZZhik changed the title Support env variables in config.pbtxt Support passing variables in config.pbtxt Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant