Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions llamacppllmbasev3/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
appVersion: b10362
appVersion: b10454
description: Generic llama.cpp + llm-init base; the GGUF model is supplied at install time via env
name: llamacppllmbasev3
type: application
version: 1.2.30
version: 1.2.33
20 changes: 15 additions & 5 deletions llamacppllmbasev3/OlaresManifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@ metadata:
description: "Generic llama.cpp engine base. Pick any GGUF model at install via env."
appid: llamacppllmbasev3
title: llama.cpp Engine Base
version: '1.2.30'
version: '1.2.33'
categories:
- AI
tags:
- engine
sharedEntrances:
- name: llamacppllmbasev3
host: sharedentrances-api
Expand All @@ -33,17 +35,25 @@ workloadReplicas:
llminit: 1
spec:
onlyAdmin: true
versionName: 'b10362'
versionName: 'b10454'
upgradeDescription: |
Chart 1.2.30clarify upgrade notes for engine `server-cuda12-b10362` (unchanged from 1.2.29).
Chart 1.2.33bump engine to `server-cuda12-b10454` and llm-init to `v1.3.8`.

**Muse Glimmer 30B (Meta)** — requires **b10353+**; **b10362 includes** the `muse-glimmer` architecture ([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Older builds (e.g. b10331) fail with `unknown model architecture: 'muse-glimmer'`.
**Images**
- llama.cpp: `docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362` → `server-cuda12-b10454` (latest published beclab CUDA 12 server tag; amd64 + arm64). Upstream GitHub is already at b10456; no beclab b10455/b10456 image yet.
- llm-init: `docker.io/beclab/llm-init:v1.3.5` → `v1.3.8`.

**Muse Glimmer 30B (Meta)** — requires **b10353+**; **b10454 includes** the `muse-glimmer` architecture ([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Older builds (e.g. b10331) fail with `unknown model architecture: 'muse-glimmer'`.

Clone this base and point env at [meta-models/Muse-Glimmer-30B-GGUF](https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF) or community quants (e.g. unsloth). Typical layout:
- Main GGUF + `mmproj` in `MODEL_SOURCE` (comma-separated) for vision
- DFlash drafter (`dflash-kquant.gguf`) + `--spec-type draft-dflash` or `draft-dspark` in `ENGINE_ARGS` for speculative decode

**Also in b10331 → b10362:** DSpark / Eagle3-v3 / MTP spec-decode, DeepSeek V4 & multimodal parsers, CUDA Q2_0 / Mamba-2 SSD prefill, server router & metrics improvements.
**Also in b10362 → b10454:**
- Chat templates receive OpenAI `reasoning_effort` ([llama.cpp #26941](https://github.com/ggml-org/llama.cpp/pull/26941)); Qwen3.8 uses `xhigh` / `medium` / `low` via `--chat-template-kwargs` or the request body
- MTP draft type auto-detect; DFlash / DSpark backend sampling
- MiniMax Text01 / M1 conversion support
- Server `/metrics` and `/slots` stay available during decode; mtmd and jinja fixes

See https://github.com/ggml-org/llama.cpp/releases for full upstream release notes.
fullDescription: |
Expand Down
20 changes: 20 additions & 0 deletions llamacppllmbasev3/i18n/en-US/OlaresManifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,26 @@ metadata:
title: llama.cpp Engine Base

spec:
upgradeDescription: |
Chart 1.2.33 — bump engine to `server-cuda12-b10454` and llm-init to `v1.3.8`.

**Images**
- llama.cpp: `docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362` → `server-cuda12-b10454` (latest published beclab CUDA 12 server tag; amd64 + arm64). Upstream GitHub is already at b10456; no beclab b10455/b10456 image yet.
- llm-init: `docker.io/beclab/llm-init:v1.3.5` → `v1.3.8`.

**Muse Glimmer 30B (Meta)** — requires **b10353+**; **b10454 includes** the `muse-glimmer` architecture ([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841)). Older builds (e.g. b10331) fail with `unknown model architecture: 'muse-glimmer'`.

Clone this base and point env at [meta-models/Muse-Glimmer-30B-GGUF](https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF) or community quants (e.g. unsloth). Typical layout:
- Main GGUF + `mmproj` in `MODEL_SOURCE` (comma-separated) for vision
- DFlash drafter (`dflash-kquant.gguf`) + `--spec-type draft-dflash` or `draft-dspark` in `ENGINE_ARGS` for speculative decode

**Also in b10362 → b10454:**
- Chat templates receive OpenAI `reasoning_effort` ([llama.cpp #26941](https://github.com/ggml-org/llama.cpp/pull/26941)); Qwen3.8 uses `xhigh` / `medium` / `low` via `--chat-template-kwargs` or the request body
- MTP draft type auto-detect; DFlash / DSpark backend sampling
- MiniMax Text01 / M1 conversion support
- Server `/metrics` and `/slots` stay available during decode; mtmd and jinja fixes

See https://github.com/ggml-org/llama.cpp/releases for full upstream release notes.
fullDescription: |
**IMPORTANT NOTE**
This app is a template and cannot be used on its own. To use it, set the environment variables below to connect a specific model.
Expand Down
20 changes: 20 additions & 0 deletions llamacppllmbasev3/i18n/zh-CN/OlaresManifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,26 @@ metadata:
title: llama.cpp 引擎基座

spec:
upgradeDescription: |
Chart 1.2.33 — 引擎升级到 `server-cuda12-b10454`,llm-init 升级到 `v1.3.8`。

**镜像**
- llama.cpp:`docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362` → `server-cuda12-b10454`(当前已发布的最新 beclab CUDA 12 server 标签;amd64 + arm64)。上游 GitHub 已到 b10456;尚无 beclab 的 b10455/b10456 镜像。
- llm-init:`docker.io/beclab/llm-init:v1.3.5` → `v1.3.8`。

**Muse Glimmer 30B(Meta)** — 需要 **b10353+**;**b10454 已包含** `muse-glimmer` 架构([llama.cpp #26841](https://github.com/ggml-org/llama.cpp/pull/26841))。更旧的构建(如 b10331)会报 `unknown model architecture: 'muse-glimmer'`。

克隆此基座后,将环境变量指向 [meta-models/Muse-Glimmer-30B-GGUF](https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF) 或社区量化(如 unsloth)。常见布局:
- 主 GGUF + `mmproj` 写入 `MODEL_SOURCE`(逗号分隔)以支持视觉
- DFlash 草稿模型(`dflash-kquant.gguf`)并在 `ENGINE_ARGS` 中加 `--spec-type draft-dflash` 或 `draft-dspark` 以开启投机解码

**b10362 → b10454 其它变化:**
- 聊天模板可接收 OpenAI `reasoning_effort`([llama.cpp #26941](https://github.com/ggml-org/llama.cpp/pull/26941));Qwen3.8 可用 `--chat-template-kwargs` 或请求体设置 `xhigh` / `medium` / `low`
- MTP 草稿类型自动检测;DFlash / DSpark 后端采样
- 支持 MiniMax Text01 / M1 转换
- 解码过程中 `/metrics` 与 `/slots` 仍可用;mtmd 与 jinja 修复

完整上游发布说明见 https://github.com/ggml-org/llama.cpp/releases
fullDescription: |
**重要提示**
此应用是一个模板,不能单独使用。请按下方环境变量设置模型后,方可使用本应用提供模型服务。
Expand Down
2 changes: 1 addition & 1 deletion llamacppllmbasev3/templates/llamacpp.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ spec:
defaultMode: 0555
containers:
- name: llamacpp
image: docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10362
image: docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10454
imagePullPolicy: IfNotPresent
command: ["/bin/sh", "/llm-init/wrappers/llamacpp.sh"]
env:
Expand Down
2 changes: 1 addition & 1 deletion llamacppllmbasev3/templates/llm-init.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ spec:
# v1.2.4 defaults to the stable LFS download path (hf_xet disabled
# unless HF_ENABLE_XET=true); LFS streams each file to disk with a
# bounded RAM footprint, avoiding the hf_xet OOM (huggingface_hub#3300).
image: docker.io/beclab/llm-init:v1.3.5
image: docker.io/beclab/llm-init:v1.3.8
imagePullPolicy: IfNotPresent
env:
- name: ENGINE_KIND
Expand Down