Skip to content

feat(memory): add Hindsight conversation memory#16

Open
hotwa wants to merge 7 commits into
Lynpoint:featuresfrom
hotwa:features
Open

feat(memory): add Hindsight conversation memory#16
hotwa wants to merge 7 commits into
Lynpoint:featuresfrom
hotwa:features

Conversation

@hotwa

@hotwa hotwa commented May 14, 2026

Copy link
Copy Markdown

中文说明

概要

本 PR 为 CyberVerse 增加 Hindsight 长期记忆接入,覆盖 PersonaAgent 数字人对话路径和 Go standard conversation path。

主要变化:

  • 增加 Hindsight retain/recall 客户端配置
  • 支持动态 bank id:cv:user:{user_id}:character:{character_id}
  • 为记忆写入增加 tags、metadata、document_id
  • PersonaAgent 在回复前自动 recall,并通过 response_instructions 注入长期记忆上下文
  • PersonaAgent 在最终回复后异步 retain 用户/助手对话
  • Go standard pipeline 也接入同一套 Hindsight conversation memory
  • 更新 README、.env.example、GPU 部署配置和开发计划文档
  • 增加 Python/Go 单元测试覆盖 Hindsight payload 和 PersonaAgent memory flow

当前验证状态

CyberVerse 侧接入和写入链路已验证通过:

  • PersonaAgent 能正常生成回复
  • retain 能写入 Hindsight bank/document
  • 写入的 document 包含预期的 transcript、tags、metadata、document_id
  • 本地单元测试和 Go orchestrator 测试通过

但端到端“长期记忆可 recall”尚未完全成功。

远程 Hindsight 服务 http://w4ce.jmsu.top:8888 当前 retain/reprocess operation 能完成,但生成结果为:

  • memory_unit_count=0
  • total_nodes=0
  • usage.total_tokens=0
  • recall 返回空结果

这说明 CyberVerse 的写入 payload 已进入 Hindsight,但 Hindsight 后端抽取/LLM worker 目前没有生成 memory units。该问题需要在 Hindsight 服务侧继续排查。

测试

已执行:

git diff --check
pixi run -e macos-arm pytest tests/unit/test_hindsight_memory.py tests/unit/test_persona_agent_plugin.py -q
pixi run -e macos-arm go test ./internal/orchestrator
pixi run -e macos-arm python - <<'PY'
import yaml
from pathlib import Path
for path in ["infra/cyberverse_config.example.yaml", "deploy/gpu/cyberverse_config.gpu.yaml"]:
    yaml.safe_load(Path(path).read_text())
PY

结果:

  • Python focused tests: 26 passed
  • Go orchestrator tests: passed
  • YAML parse: passed
  • git diff --check: passed

English

Summary

This PR adds Hindsight long-term conversation memory support to CyberVerse, covering both the PersonaAgent digital-human path and the Go standard conversation path.

Main changes:

  • Add Hindsight retain/recall client configuration
  • Support dynamic bank IDs: cv:user:{user_id}:character:{character_id}
  • Add tags, metadata, and document IDs to retained memory items
  • Automatically recall Hindsight memory before PersonaAgent responses and inject it through response_instructions
  • Retain user/assistant turns asynchronously after final PersonaAgent responses
  • Wire the Go standard pipeline to the same Hindsight conversation memory client
  • Update README, .env.example, GPU deployment config, and the development plan
  • Add Python/Go tests for Hindsight payloads and PersonaAgent memory flow

Current Verification Status

The CyberVerse integration and write path have been verified:

  • PersonaAgent can respond normally
  • retain writes documents into the expected Hindsight bank
  • stored documents contain the expected transcript, tags, metadata, and document ID
  • local Python and Go tests pass

However, full end-to-end durable recall is not yet successful.

Against the remote Hindsight service at http://w4ce.jmsu.top:8888, retain/reprocess operations complete, but extraction currently produces:

  • memory_unit_count=0
  • total_nodes=0
  • usage.total_tokens=0
  • empty recall results

This indicates the CyberVerse payload is reaching Hindsight correctly, but the Hindsight backend extraction/LLM worker is not generating memory units yet. That needs follow-up on the Hindsight service side.

Tests

Executed:

git diff --check
pixi run -e macos-arm pytest tests/unit/test_hindsight_memory.py tests/unit/test_persona_agent_plugin.py -q
pixi run -e macos-arm go test ./internal/orchestrator
pixi run -e macos-arm python - <<'PY'
import yaml
from pathlib import Path
for path in ["infra/cyberverse_config.example.yaml", "deploy/gpu/cyberverse_config.gpu.yaml"]:
    yaml.safe_load(Path(path).read_text())
PY

Results:

  • Python focused tests: 26 passed
  • Go orchestrator tests: passed
  • YAML parse: passed
  • git diff --check: passed

hotwa added 7 commits May 13, 2026 14:38
Add reproducible Pixi environments for macOS ARM and Linux x86, document the setup across README variants, and add remote GPU docker-compose deployment files for FlashHead on host GPU 1.\n\nAdd a POST /sessions/{id}/close compatibility route and switch the frontend close call to it so remote deployments can avoid networks that block HTTP DELETE.\n\nVerification:\n- pixi run go-test\n- npm run --prefix frontend build\n- remote health + session create/close over http://122.205.95.186:8080
Configure Pixi Go tasks with GOPROXY and GOSUMDB so linux-x86 environments can download Go modules on the remote Ubuntu host.\n\nVerification:\n- pixi install -e macos-arm\n- remote pixi install -e linux-x86\n- remote pixi run -e linux-x86 python-agent-test\n- remote pixi run -e linux-x86 go-test\n- pixi run -e macos-arm go-test
Connect Hindsight retain and recall into PersonaAgent and the standard digital-human pipeline. Add local shadow fallback, text-input TTS/avatar rendering, deploy configuration, frontend task i18n, and focused Go/Python tests.
… features

# Conflicts:
#	frontend/src/pages/LaunchConfigPage.vue
#	frontend/src/pages/SessionPage.vue
#	inference/plugins/voice_llm/persona_agent.py
#	server/internal/orchestrator/orchestrator.go
#	tests/unit/test_persona_agent_plugin.py
Implement Hindsight retain/recall wiring for PersonaAgent and the standard Go conversation path with dynamic user/character scoped bank IDs, tags, document IDs, metadata, and nonblocking retain behavior.

Update examples, deployment docs, and focused tests for the Hindsight payloads and PersonaAgent recall injection flow.

Remote smoke against http://w4ce.jmsu.top:8888 confirms CyberVerse writes documents into the expected bank, but durable recall remains blocked because the Hindsight backend currently completes extraction with zero memory units.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant