Skip to content

[UPDATE][vllmgemma412bitawqint4v3][1.0.12] #3422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Terminus-Gitbot merged 1 commit into from
Aug 12, 2026
Merged

[UPDATE][vllmgemma412bitawqint4v3][1.0.12] #3422

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion vllmgemma412bitawqint4v3/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ appVersion: 0.24.0
description: Gemma-4-12B (AWQ-INT4) served with vLLM + llm-init (GPU only)
name: vllmgemma412bitawqint4v3
type: application
version: 1.0.11
version: 1.0.12
8 changes: 5 additions & 3 deletions vllmgemma412bitawqint4v3/OlaresManifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ metadata:
description: "Gemma-4-12B (AWQ-INT4) served with vLLM."
appid: vllmgemma412bitawqint4v3
title: Gemma-4-12B (vLLM)
version: '1.0.11'
version: '1.0.12'
categories:
- AI
sharedEntrances:
Expand Down Expand Up @@ -35,6 +35,8 @@ spec:
onlyAdmin: true
versionName: 'cyankiwi/gemma-4-12B-it-AWQ-INT4'
upgradeDescription: |
v1.0.12: ENGINE_ARGS uses the gemma4 tool-call and reasoning parsers.

v1.0.11: bump llm-init to v1.3.5.
v1.0.10: Move `HF_TOKEN` into Kubernetes Secret (`secretKeyRef`); map HF env from `OLARES_SYSTEM_HUGGINGFACE_*`.

Expand All @@ -52,7 +54,7 @@ spec:
**Preset Configuration**
- Model: `cyankiwi/gemma-4-12B-it-AWQ-INT4`
- Capabilities: thinking, tools, vision
- Engine args: `--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser qwen3_coder --reasoning-parser qwen3 --enable-prefix-caching --enable-auto-tool-choice`
- Engine args: `--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser gemma4 --reasoning-parser gemma4 --enable-prefix-caching --enable-auto-tool-choice`
- Required GPU memory: `23Gi`

**Accelerator modes**
Expand Down Expand Up @@ -166,7 +168,7 @@ envs:
description: "The model capabilities. Pick what your model supports. If no extra capabilities, pick None."
- envName: ENGINE_ARGS # stability-first example for quantized Ornith 9B on 24GB: --max-model-len 131072 --gpu-memory-utilization 0.92 --tensor-parallel-size 1 --max-num-batched-tokens 4096 --kv-cache-dtype fp8 --quantization compressed-tensors --language-model-only --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml --trust-remote-code --enforce-eager
required: true
default: "--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser qwen3_coder --reasoning-parser qwen3 --enable-prefix-caching --enable-auto-tool-choice"
default: "--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser gemma4 --reasoning-parser gemma4 --enable-prefix-caching --enable-auto-tool-choice"
type: string
editable: true
applyOnChange: true
Expand Down
2 changes: 1 addition & 1 deletion vllmgemma412bitawqint4v3/i18n/en-US/OlaresManifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ spec:
**Preset Configuration**
- Model: `cyankiwi/gemma-4-12B-it-AWQ-INT4`
- Capabilities: thinking, tools, vision
- Engine args: `--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser qwen3_coder --reasoning-parser qwen3 --enable-prefix-caching --enable-auto-tool-choice`
- Engine args: `--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser gemma4 --reasoning-parser gemma4 --enable-prefix-caching --enable-auto-tool-choice`
- Required GPU memory: `23Gi`

**Accelerator modes**
Expand Down
2 changes: 1 addition & 1 deletion vllmgemma412bitawqint4v3/i18n/zh-CN/OlaresManifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ spec:
**预设配置**
- 模型：`cyankiwi/gemma-4-12B-it-AWQ-INT4`
- 能力：思考、工具、视觉
- 引擎参数：`--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser qwen3_coder --reasoning-parser qwen3 --enable-prefix-caching --enable-auto-tool-choice`
- 引擎参数：`--max-model-len 65536 --gpu-memory-utilization 0.9 --tensor-parallel-size 1 --max-num-batched-tokens 8192 --tool-call-parser gemma4 --reasoning-parser gemma4 --enable-prefix-caching --enable-auto-tool-choice`
- 所需显存：`23Gi`

**加速器模式**
Expand Down