[pull] develop from bfenetworks:develop - #320
Merged
Merged
Conversation
Signed-off-by: xuleiming <leiming.xu@163.com>
Signed-off-by: xuleiming <leiming.xu@163.com>
Signed-off-by: xuleiming <leiming.xu@163.com>
Signed-off-by: xuleiming <leiming.xu@163.com>
body rewind when fallback
Add AI gateway mod_ai_route system design docs under docs/zh_cn/sys_design: - mod_ai_route.md: module system design - mod_ai_route_bfe_changes.md: BFE main program changes for AI gateway
…rocess and add TC08-TC10 - Replace in-process tests under tests/integration/mod_ai_route with real BFE process tests under tests/integration/implementation/scenario-SC01-*. - Add common harness (ProcessEnv, BFEConfigBuilder, MockBackend) for building config and running real bfe binary. - Add TC08: fallback preserves complete body after primary disconnects mid-stream. - Add TC09: fallback is aborted when body exceeds accessibleBodySize. - Add TC10: fallback is disabled when totalBodyBufferSize limit is reached. - Add test design documents for SC01.
- Extend AIConf with Keys array, KeyPolicy, Provider and ModelTable - Implement weighted random key selection, 429 rotation, 401/403 death, and 5xx same-key retry with exponential backoff in reverse proxy - Remove legacy single-key AI handling from ServeHTTP; AI requests now flow exclusively through ServeHTTPForAI / aiClusterInvoke - Fix 4xx early-return bug and ensure 5xx retries keep the same key - Add SC02 integration tests for multi-key selection, rotation and retry - Extend test helpers: MockBackend auth header capture, per-cluster AIConf - Update zh_cn and en_us cluster_conf.data.md documentation
- Extend AIConf.ModelTable with fixed-point RMB price conversion and price index. - Add UsedCost to TokenUsage and Unit/Currency to QuotaPlan. - Implement RMB quota deduction via single-key fixed-point Lua script. - Cache SvrDataConf in TokenAuthContext for cost calculation at response stage. - Add design doc docs/zh_cn/sys_design/rmb_quota.md. - Add SC03 integration tests for RMB quota deduction scenarios. - Bump go-lib to v0.0.2 for shared quota fixed-point helpers.
Modifications of V1.8.5
- Add MatchPrefix/StripPrefix to AIConf and validation in AIConfCheck - Implement prefix stripping in reverseproxy.doSingleAIForward - Add unit tests for AIConf validation and prefix stripping - Add SC04 integration tests and design docs for provider/model prefix strip - Update cluster_conf.data docs for MatchPrefix/StripPrefix fields
feat(bfe): provider/model prefix routing support
…aming responses - Move UsedCost calculation from HandleReadResponse to HandleRequestFinish so streaming responses (ContentLength=-1) are billed correctly. - Enable mod_body_process in SC03 integration tests. - Add streaming RMB quota deduction test case. - Update rmb_quota.md design doc and AGENTS.md. Fixes #1316
fix(bfe): 4xx status codes trigger cluster-level fallback
- Upgrade bfe-access-pb from v0.1.0 to v0.2.0 - Rename access log fields: ai_apikey->ai_apikey_id, ai_mapped_model->ai_target_model, ai_prompt_tokens->ai_input_tokens - Add AiBasicInfo fields: Provider, RetryCount, CostCurrency, ClusterKeyNames - Add AiAuthInfo.HitQuotaPlans - Populate provider/currency/retry/cluster-key in reverseproxy.go - Record HitQuotaPlans in mod_ai_token_auth - Update request_log.go and request_log_test.go - Add SC05 integration tests covering all 20 AI fields (701-900) - Add b2log parser helper in tests/integration/common - Add sys_design/ai_access_log_fields.md and modifications design doc
- Refactor stripProviderPrefix to a pure string transform function. - In doSingleAIForward, compute final model (override -> strip -> mapping) and call ReqBodyJsonSet at most once. - Reset ContentLength for both outreq and basicReq.HttpRequest in one place. - Update related unit tests and sys_design docs.
…tempts - In doSingleAIForward, compute final model from ClientModel each attempt instead of inheriting TargetModel from previous attempts. - Copy request body before rewriting only when a rewrite is actually required, avoiding shared buffer mutation on fallback clusters. - Add SC04 TC07 to verify fallback cluster without rewrite rules receives the original client model. - Document the fix and its test design.
V1.8.5 fix problem, and adapt bfe-access-pb v0.2.0
…uster attempts" This reverts commit 1ae02e4.
Revert "fix(bfe_server): prevent model body rewrite leaking across cluster
The integration test TestTC10_TotalBodyBufferSizeExceedsLimit could hang for 10 minutes if waitForTotalBytesBodyBuffer timed out, because the holder mock backend was blocked on HoldBeforeRead and httptest.Server.Close() would wait indefinitely. Changes: - Use context.Context for HoldBeforeRead so cleanup cancels it. - Increase metric wait timeout from 5s to 30s. - Include last metric value in failure message.
fix(test): prevent TestTC10 from hanging on cleanup
…ary cache - Use context.Context for HoldBeforeRead so cleanup cancels it and does not hang when waitForTotalBytesBodyBuffer times out. - Increase metric wait timeout from 5s to 30s for slow CI environments. - Include git commit hash in integration test binary path and check source mtime so that stale cached binaries are rebuilt when source changes. - Add holder request result logging for diagnostics.
fix(test): prevent TestTC10 from hanging on cleanup and fix stale bin…
The holder request intentionally blocks until the backend is released. Under race detector / slow CI the default 30s client timeout fires before BFE finishes wrapping the body, causing total_bytes_body_buffer to stay at 0 and the test to fail. Increase the holder client timeout to 2 minutes and the metric wait timeout to 60 seconds.
fix(test): increase holder timeout for TestTC10 in slow CI
…backend The original TestTC10 relied on a holder request with a blocking backend to keep a 2 MB bytes_body buffer allocated. This was fragile in CI: under race detector / load the client or backend timeouts fired before the body was wrapped, causing total_bytes_body_buffer to stay at 0. Redesign: - Add BFE_TEST_INITIAL_TOTAL_BYTES_BODY_BUFFER env var to let integration tests pre-seed the global body buffer counter in the BFE process. - TestTC10 now sets the counter to the limit, starts BFE, verifies the monitor endpoint reports the expected value, sends the test request, and asserts that fallback is disabled. - Remove the holder request, blocking backend, and polling logic. This makes the test deterministic and independent of request timing.
refactor(test): redesign TestTC10 to avoid timing-sensitive blocking …
Release/v1.8.5
Merge pull request #1328 from bfenetworks/release/v1.8.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )