diff --git a/llamacppllmbasev3/Chart.yaml b/llamacppllmbasev3/Chart.yaml index 0834a7c82..141917f88 100644 --- a/llamacppllmbasev3/Chart.yaml +++ b/llamacppllmbasev3/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v2 -appVersion: b10362 +appVersion: b10454 description: Generic llama.cpp + llm-init base; the GGUF model is supplied at install time via env name: llamacppllmbasev3 type: application -version: 1.2.30 +version: 1.2.33 \ No newline at end of file diff --git a/llamacppllmbasev3/OlaresManifest.yaml b/llamacppllmbasev3/OlaresManifest.yaml index c82e47de6..d14f45661 100644 --- a/llamacppllmbasev3/OlaresManifest.yaml +++ b/llamacppllmbasev3/OlaresManifest.yaml @@ -7,9 +7,11 @@ metadata: description: "Generic llama.cpp engine base. Pick any GGUF model at install via env." appid: llamacppllmbasev3 title: llama.cpp Engine Base - version: '1.2.30' + version: '1.2.33' categories: - AI + tags: + - engine sharedEntrances: - name: llamacppllmbasev3 host: sharedentrances-api @@ -33,17 +35,25 @@ workloadReplicas: llminit: 1 spec: onlyAdmin: true - versionName: 'b10362' + versionName: 'b10454' upgradeDescription: | - Chart 1.2.30 — clarify upgrade notes for engine `server-cuda12-b10362` (unchanged from 1.2.29). + Chart 1.2.33 — bump engine to `server-cuda12-b10454` and llm-init to `v1.3.8`. - **Muse Glimmer 30B (Meta)** — requires **b10353+**; **b10362 includes** the `muse-glimmer` architecture ([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Older builds (e.g. b10331) fail with `unknown model architecture: 'muse-glimmer'`. + **Images** + - llama.cpp: `docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362` → `server-cuda12-b10454` (latest published beclab CUDA 12 server tag; amd64 + arm64). Upstream GitHub is already at b10456; no beclab b10455/b10456 image yet. + - llm-init: `docker.io/beclab/llm-init:v1.3.5` → `v1.3.8`. + + **Muse Glimmer 30B (Meta)** — requires **b10353+**; **b10454 includes** the `muse-glimmer` architecture ([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Older builds (e.g. b10331) fail with `unknown model architecture: 'muse-glimmer'`. Clone this base and point env at [meta-models/Muse-Glimmer-30B-GGUF](https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF) or community quants (e.g. unsloth). Typical layout: - Main GGUF + `mmproj` in `MODEL_SOURCE` (comma-separated) for vision - DFlash drafter (`dflash-kquant.gguf`) + `--spec-type draft-dflash` or `draft-dspark` in `ENGINE_ARGS` for speculative decode - **Also in b10331 → b10362:** DSpark / Eagle3-v3 / MTP spec-decode, DeepSeek V4 & multimodal parsers, CUDA Q2_0 / Mamba-2 SSD prefill, server router & metrics improvements. + **Also in b10362 → b10454:** + - Chat templates receive OpenAI `reasoning_effort` ([llama.cpp #26941](https://github.com/ggml-org/llama.cpp/pull/26941)); Qwen3.8 uses `xhigh` / `medium` / `low` via `--chat-template-kwargs` or the request body + - MTP draft type auto-detect; DFlash / DSpark backend sampling + - MiniMax Text01 / M1 conversion support + - Server `/metrics` and `/slots` stay available during decode; mtmd and jinja fixes See https://github.com/ggml-org/llama.cpp/releases for full upstream release notes. fullDescription: | diff --git a/llamacppllmbasev3/i18n/en-US/OlaresManifest.yaml b/llamacppllmbasev3/i18n/en-US/OlaresManifest.yaml index 3708663da..835e2e842 100644 --- a/llamacppllmbasev3/i18n/en-US/OlaresManifest.yaml +++ b/llamacppllmbasev3/i18n/en-US/OlaresManifest.yaml @@ -3,6 +3,26 @@ metadata: title: llama.cpp Engine Base spec: + upgradeDescription: | + Chart 1.2.33 — bump engine to `server-cuda12-b10454` and llm-init to `v1.3.8`. + + **Images** + - llama.cpp: `docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362` → `server-cuda12-b10454` (latest published beclab CUDA 12 server tag; amd64 + arm64). Upstream GitHub is already at b10456; no beclab b10455/b10456 image yet. + - llm-init: `docker.io/beclab/llm-init:v1.3.5` → `v1.3.8`. + + **Muse Glimmer 30B (Meta)** — requires **b10353+**; **b10454 includes** the `muse-glimmer` architecture ([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Older builds (e.g. b10331) fail with `unknown model architecture: 'muse-glimmer'`. + + Clone this base and point env at [meta-models/Muse-Glimmer-30B-GGUF](https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF) or community quants (e.g. unsloth). Typical layout: + - Main GGUF + `mmproj` in `MODEL_SOURCE` (comma-separated) for vision + - DFlash drafter (`dflash-kquant.gguf`) + `--spec-type draft-dflash` or `draft-dspark` in `ENGINE_ARGS` for speculative decode + + **Also in b10362 → b10454:** + - Chat templates receive OpenAI `reasoning_effort` ([llama.cpp #26941](https://github.com/ggml-org/llama.cpp/pull/26941)); Qwen3.8 uses `xhigh` / `medium` / `low` via `--chat-template-kwargs` or the request body + - MTP draft type auto-detect; DFlash / DSpark backend sampling + - MiniMax Text01 / M1 conversion support + - Server `/metrics` and `/slots` stay available during decode; mtmd and jinja fixes + + See https://github.com/ggml-org/llama.cpp/releases for full upstream release notes. fullDescription: | **IMPORTANT NOTE** This app is a template and cannot be used on its own. To use it, set the environment variables below to connect a specific model. diff --git a/llamacppllmbasev3/i18n/zh-CN/OlaresManifest.yaml b/llamacppllmbasev3/i18n/zh-CN/OlaresManifest.yaml index ea7d512e2..666700a95 100644 --- a/llamacppllmbasev3/i18n/zh-CN/OlaresManifest.yaml +++ b/llamacppllmbasev3/i18n/zh-CN/OlaresManifest.yaml @@ -3,6 +3,26 @@ metadata: title: llama.cpp 引擎基座 spec: + upgradeDescription: | + Chart 1.2.33 — 引擎升级到 `server-cuda12-b10454`,llm-init 升级到 `v1.3.8`。 + + **镜像** + - llama.cpp:`docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362` → `server-cuda12-b10454`(当前已发布的最新 beclab CUDA 12 server 标签;amd64 + arm64)。上游 GitHub 已到 b10456;尚无 beclab 的 b10455/b10456 镜像。 + - llm-init:`docker.io/beclab/llm-init:v1.3.5` → `v1.3.8`。 + + **Muse Glimmer 30B(Meta)** — 需要 **b10353+**;**b10454 已包含** `muse-glimmer` 架构([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841))。更旧的构建(如 b10331)会报 `unknown model architecture: 'muse-glimmer'`。 + + 克隆此基座后,将环境变量指向 [meta-models/Muse-Glimmer-30B-GGUF](https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF) 或社区量化(如 unsloth)。常见布局: + - 主 GGUF + `mmproj` 写入 `MODEL_SOURCE`(逗号分隔)以支持视觉 + - DFlash 草稿模型(`dflash-kquant.gguf`)并在 `ENGINE_ARGS` 中加 `--spec-type draft-dflash` 或 `draft-dspark` 以开启投机解码 + + **b10362 → b10454 其它变化:** + - 聊天模板可接收 OpenAI `reasoning_effort`([llama.cpp #26941](https://github.com/ggml-org/llama.cpp/pull/26941));Qwen3.8 可用 `--chat-template-kwargs` 或请求体设置 `xhigh` / `medium` / `low` + - MTP 草稿类型自动检测;DFlash / DSpark 后端采样 + - 支持 MiniMax Text01 / M1 转换 + - 解码过程中 `/metrics` 与 `/slots` 仍可用;mtmd 与 jinja 修复 + + 完整上游发布说明见 https://github.com/ggml-org/llama.cpp/releases fullDescription: | **重要提示** 此应用是一个模板,不能单独使用。请按下方环境变量设置模型后,方可使用本应用提供模型服务。 diff --git a/llamacppllmbasev3/templates/llamacpp.yaml b/llamacppllmbasev3/templates/llamacpp.yaml index b70a06e93..10c95802b 100644 --- a/llamacppllmbasev3/templates/llamacpp.yaml +++ b/llamacppllmbasev3/templates/llamacpp.yaml @@ -73,7 +73,7 @@ spec: defaultMode: 0555 containers: - name: llamacpp - image: docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362 + image: docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10454 imagePullPolicy: IfNotPresent command: ["/bin/sh", "/llm-init/wrappers/llamacpp.sh"] env: diff --git a/llamacppllmbasev3/templates/llm-init.yaml b/llamacppllmbasev3/templates/llm-init.yaml index 794bc4dec..2ec8b8931 100644 --- a/llamacppllmbasev3/templates/llm-init.yaml +++ b/llamacppllmbasev3/templates/llm-init.yaml @@ -109,7 +109,7 @@ spec: # v1.2.4 defaults to the stable LFS download path (hf_xet disabled # unless HF_ENABLE_XET=true); LFS streams each file to disk with a # bounded RAM footprint, avoiding the hf_xet OOM (huggingface_hub#3300). - image: docker.io/beclab/llm-init:v1.3.5 + image: docker.io/beclab/llm-init:v1.3.8 imagePullPolicy: IfNotPresent env: - name: ENGINE_KIND