diff --git a/.gitignore b/.gitignore index b2a90644b..099897ec2 100644 --- a/.gitignore +++ b/.gitignore @@ -1,33 +1,36 @@ -# Copyright 2021 The BFE Authors -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -.svn -.tmp -.download -output/ -.*.swp -.*.swo -/**/y.output -/**/*.log -/**/*.log.* -profile.out -coverage.txt -.idea/* -.vscode/* -dist/* -conf/wasm_plugin - -.DS_Store -.git* -!.gitattributes +# Copyright 2021 The BFE Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +.svn +.tmp +.omo +.download +.codegraph +output/ +.*.swp +.*.swo +/**/y.output +/**/*.log +/**/*.log.* +profile.out +coverage.txt +.idea/* +.vscode/* +dist/* +conf/wasm_plugin +tests/integration/.integration-test-bin/ + +.DS_Store +.git* +!.gitattributes diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..b91fcaceb --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,116 @@ +# AGENTS.md — BFE Server + +This file guides AI coding agents working on the `bfe/` codebase (the data-plane BFE Server). + +## Project overview + +BFE (Beyond Front End) is a modern layer-7 load balancer originated at Baidu and a CNCF sandbox project. This repository contains **BFE Server**, the data-plane component that forwards user traffic based on content-based routing, load balancing, and a flexible plugin framework. + +BFE system context (for orientation only): +- **Data plane**: BFE Server (this repo) — forwards traffic. +- **Control plane**: API-Server, Conf-Agent, Dashboard (separate repos) — manage and distribute configuration. +- **Kubernetes integration**: BFE Ingress Controller (separate repo). + +## High-level architecture + +Entry point: `bfe.go` +- Parses flags (`-c conf_root`, `-l log_dir`, `-t test_conf`, etc.). +- Loads server config via `bfe_config/bfe_conf.BfeConfigLoad`. +- Starts the server via `bfe_server.StartUp`. + +Core request flow: +1. Listener (`bfe_server/`) accepts HTTP/HTTPS/HTTP2/WebSocket/SPDY/FastCGI connections. +2. TLS handshake and session handling (`bfe_tls/`). +3. HTTP request parsing (`bfe_http/`, `bfe_http2/`, `bfe_bufio/`). +4. Routing: host table → cluster table (`bfe_route/`, `bfe_config/bfe_route_conf/`). +5. Module pipeline execution in fixed order (`bfe_modules/`, registered in `bfe_modules/bfe_modules.go`). +6. Backend selection and load balancing (`bfe_balance/`). +7. Proxying and response handling (`bfe_server/reverseproxy.go`, `bfe_server/response.go`). + +## Directory structure and module relationships + +| Directory | Responsibility | +|-----------|----------------| +| `bfe_server/` | HTTP(S) listeners, connection handling, reverse proxy, TLS termination, module registration, status/monitoring. | +| `bfe_route/` | Host and cluster routing tables, trie-based lookups (`trie/`), server data config (`server_data_conf.go`). | +| `bfe_balance/` | Backend instances (`backend/`), GSLB (`bal_gslb/`) and SLB (`bal_slb/`) balancing policies, balance table. | +| `bfe_config/` | Configuration loading: `bfe_conf/`, `bfe_route_conf/`, `bfe_cluster_conf/`, `bfe_tls_conf/`. | +| `bfe_http/`, `bfe_http2/`, `bfe_websocket/`, `bfe_spdy/`, `bfe_fcgi/`, `bfe_stream/` | Protocol implementations. | +| `bfe_tls/` | TLS handshake, certificates, session cache, server rules. | +| `bfe_module/` | Plugin framework: module interface, callback/handler lists, filters. | +| `bfe_modules/` | Built-in modules (access, WAF, redirect, rewrite, AI routing, rate limiting, etc.). Registered in `bfe_modules/bfe_modules.go`. | +| `bfe_basic/` | Condition parser/primitives used by modules. Generated parser code lives in `bfe_basic/condition/parser`. | +| `bfe_net/`, `bfe_bufio/`, `bfe_util/`, `bfe_debug/` | Shared network, buffered I/O, utilities, debug flags. | +| `conf/` | Sample runtime configuration files. | +| `docs/`, `examples/`, `tests/` | Documentation, deployment examples, and integration tests. | + +## Build/test conventions + +- **Go version**: 1.22 (`go.mod`). +- **Module**: `github.com/bfenetworks/bfe`. +- **Build**: `make` (or `make all`) → prepare, compile, package. + - `make build` builds the `bfe` binary. + - `make strip` builds without symbols. +- **Test**: `make test` runs `go test -cover ./...` and `go vet ./...`. +- **Prepare**: `make prepare` installs `goyacc` and regenerates `bfe_basic/condition/parser` via `go generate`. +- **Lint/static analysis**: `make check` runs `staticcheck`; `make license-check` / `make license-fix` use `license-eye`. +- **Pre-commit**: Install with `pre-commit install`; `gofmt` is required. +- **Docker**: `make docker` builds prod + debug images; `make docker-push REGISTRY=...` builds and pushes multi-arch images. +- **Release**: `make release` builds tar.gz packages for darwin/amd64, linux/amd64, linux/arm64, windows/amd64. + +## Common modification patterns + +### Add or modify a BFE module +1. Create a package under `bfe_modules/mod_/`. +2. Implement `bfe_module.BfeModule` and the required callbacks/handlers. +3. Add config loader under `bfe_config/` if new config files are needed. +4. Add module registration in `bfe_modules/bfe_modules.go` in the correct execution order; document ordering requirements in comments. +5. Add sample config under `conf/` and update config documentation. +6. Add unit tests using `testing` + `testify`. +7. Run `make test` before submitting. + +### AI gateway module changes + +The AI gateway modules under `bfe_modules/mod_ai_*` and `bfe_modules/mod_body_process` have ordering and lifecycle interdependencies: + +- `mod_ai_route` runs early to select the target cluster/model. +- `mod_ai_token_auth` runs at `HandleFoundProduct` for API Key validation and quota plan binding; it also performs final quota deduction at `HandleRequestFinish`. +- `mod_body_process` runs at `HandleReadResponse` and is responsible for parsing token usage from streaming (SSE) responses. If you modify RMB quota deduction, ensure streaming scenarios still work when `mod_body_process` is loaded. +- For RMB quota details, see `docs/zh_cn/sys_design/rmb_quota.md`. + +### Routing changes +- Host/cluster tables: `bfe_route/`. +- Config loaders: `bfe_config/bfe_route_conf/`. +- Sample configs: `conf/`. + +### Load-balancing changes +- Backend model: `bfe_balance/backend/`. +- Balancing policies: `bfe_balance/bal_gslb/` and `bfe_balance/bal_slb/`. +- Balance table: `bfe_balance/bal_table.go`. + +### Protocol support changes +- HTTP: `bfe_http/`, `bfe_server/http_conn.go`. +- HTTP/2: `bfe_http2/`. +- WebSocket: `bfe_websocket/`. +- TLS: `bfe_tls/`. + +### Condition/rule language changes +- Grammar/parser: `bfe_basic/condition/parser`. +- Run `make prepare` after grammar changes to regenerate parser code. + +## Agent guidelines + +- **Preserve module order** in `bfe_modules/bfe_modules.go`. Many modules have explicit ordering requirements; keep the comments up to date. +- **Regenerate generated code** after parser/grammar changes (`make prepare`). +- **Keep tests idiomatic**: use `testing` and `testify/assert`/`require`. Place `_test.go` files next to the code under test. +- **License headers**: all new source files need the Apache 2.0 header. Use `make license-fix` if unsure. +- **Do not hand-edit vendored or generated files**. +- **Run `make test`** as the minimal local verification. +- **Configuration changes must be reflected in `conf/`** so that `make package` and Docker builds produce a usable default setup. + +## Useful references + +- `README.md` / `README-CN.md` — project overview and quick start. +- `CONTRIBUTING.md` — workflow, commit sign-off, code style. +- `Makefile` — build, test, Docker, and release targets. +- `docs/en_us/introduction/overview.md` — detailed architecture documentation. diff --git a/CHANGELOG.md b/CHANGELOG.md index 265e943df..b6795dbd0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,30 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [v1.8.5] - 2026-08-21 + +### Added +- Add multi-API-Key support for AI backend clusters ([Pull #1314](https://github.com/bfenetworks/bfe/pull/1314)) +- Add RMB quota support in mod_ai_token_auth ([Pull #1314](https://github.com/bfenetworks/bfe/pull/1314)) +- Add key_id field for API-Key token in access logs ([Pull #1318](https://github.com/bfenetworks/bfe/pull/1318)) +- Add provider/model prefix routing support ([Pull #1315](https://github.com/bfenetworks/bfe/pull/1315)) +- Add req_body_json_prefix_in condition primitive ([Pull #1314](https://github.com/bfenetworks/bfe/pull/1314)) +- Adapt bfe-access-pb v0.2.0 AI observability fields in access logs ([Pull #1321](https://github.com/bfenetworks/bfe/pull/1321)) +- Support body rewind when fallback ([Pull #1313](https://github.com/bfenetworks/bfe/pull/1313)) +- Add AccessibleBodySize config in bfe.conf ([Pull #1313](https://github.com/bfenetworks/bfe/pull/1313)) +- Monitor and limit TotalBodyBufferSize ([Pull #1313](https://github.com/bfenetworks/bfe/pull/1313)) +- Add release target to Makefile + +### Changed +- Trigger cluster-level fallback on 4xx status codes ([Pull #1318](https://github.com/bfenetworks/bfe/pull/1318)) +- Merge multiple AI model body rewrites into one ([Pull #1321](https://github.com/bfenetworks/bfe/pull/1321)) + +### Fixed +- Fix ai token auth default rule path ([Pull #1251](https://github.com/bfenetworks/bfe/pull/1251)) +- Fix RMB cost calculation at request finish for streaming responses ([Pull #1318](https://github.com/bfenetworks/bfe/pull/1318)) +- Fix log messages ([Pull #1313](https://github.com/bfenetworks/bfe/pull/1313)) + + ## [v1.8.4] - 2026-08-05 ### Added @@ -267,7 +291,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Fix textproto: not normalize headers with spaces before the colon (CVE-2019-16276) -## [v0.10.0] - 2020-05-25 +## [v0.10.0] - 2020-05-25 ### Added @@ -441,6 +465,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Flexible plugin framework to extend functionality. Based on the framework, developer can add new features rapidly - Detailed built-in metrics available for service status monitor +[v1.8.5]: https://github.com/bfenetworks/bfe/compare/v1.8.4...v1.8.5 +[v1.8.4]: https://github.com/bfenetworks/bfe/compare/v1.8.3...v1.8.4 [v1.6.0]: https://github.com/bfenetworks/bfe/compare/v1.5.0...v1.6.0 [v1.5.0]: https://github.com/bfenetworks/bfe/compare/v1.4.0...v1.5.0 [v1.4.0]: https://github.com/bfenetworks/bfe/compare/v1.3.0...v1.4.0 diff --git a/bfe_basic/condition/build.go b/bfe_basic/condition/build.go index 5a5c0ce26..010c6d0f0 100644 --- a/bfe_basic/condition/build.go +++ b/bfe_basic/condition/build.go @@ -554,10 +554,19 @@ func buildPrimitive(node *parser.CallExpr) (Condition, error) { name: node.Fun.Name, node: node, fetcher: &ReqBodyJsonFetcher{ - path: node.Args[0].Value, + path: node.Args[0].Value, }, matcher: NewInMatcher(node.Args[1].Value, node.Args[2].ToBool()), }, nil + case "req_body_json_prefix_in": + return &PrimitiveCond{ + name: node.Fun.Name, + node: node, + fetcher: &ReqBodyJsonFetcher{ + path: node.Args[0].Value, + }, + matcher: NewPrefixInMatcher(node.Args[1].Value, node.Args[2].ToBool()), + }, nil default: return nil, fmt.Errorf("unsupported primitive %s", node.Fun.Name) diff --git a/bfe_basic/condition/build_test.go b/bfe_basic/condition/build_test.go index ae4ba35d0..567589f5b 100644 --- a/bfe_basic/condition/build_test.go +++ b/bfe_basic/condition/build_test.go @@ -144,6 +144,32 @@ var buildPrimitiveTests = []struct { &QueryExistMatcher{}, false, }, + { + "testBuildReqBodyJsonPrefixIn", + `req_body_json_prefix_in("model", "openrouter/", false)`, + &PrimitiveCond{ + name: "req_body_json_prefix_in", + fetcher: &ReqBodyJsonFetcher{path: "model"}, + matcher: &PrefixInMatcher{ + patterns: []string{"openrouter/"}, + foldCase: false, + }, + }, + false, + }, + { + "testBuildReqBodyJsonPrefixInIgnoreCase", + `req_body_json_prefix_in("model", "OpenRouter/", true)`, + &PrimitiveCond{ + name: "req_body_json_prefix_in", + fetcher: &ReqBodyJsonFetcher{path: "model"}, + matcher: &PrefixInMatcher{ + patterns: []string{"OPENROUTER/"}, + foldCase: true, + }, + }, + false, + }, { "testBuildUrlRegMatch", "req_url_regmatch(\"123\")", diff --git a/bfe_basic/condition/parser/semant.go b/bfe_basic/condition/parser/semant.go index b47ededab..35a8a0131 100644 --- a/bfe_basic/condition/parser/semant.go +++ b/bfe_basic/condition/parser/semant.go @@ -80,6 +80,7 @@ var funcProtos = map[string][]Token{ "bfe_time_range": []Token{STRING, STRING}, "bfe_periodic_time_range": []Token{STRING, STRING, STRING}, "req_body_json_in": []Token{STRING, STRING, BOOL}, + "req_body_json_prefix_in": []Token{STRING, STRING, BOOL}, } func prototypeCheck(expr *CallExpr) error { diff --git a/bfe_basic/condition/primitive_test.go b/bfe_basic/condition/primitive_test.go index bc54318f3..1aa33e0ee 100644 --- a/bfe_basic/condition/primitive_test.go +++ b/bfe_basic/condition/primitive_test.go @@ -17,6 +17,7 @@ package condition import ( "net" "net/http" + "strings" "testing" "time" @@ -347,3 +348,80 @@ func TestPeriodicTimeMatcher(t *testing.T) { t.Fatalf("should not match %v", tm) } } + +func buildRequestWithBody(body string) *bfe_basic.Request { + httpReq, _ := bfe_http.NewRequest("POST", "http://example.com/v1/chat/completions", strings.NewReader(body)) + return bfe_basic.NewRequest(httpReq, nil, nil, &bfe_basic.Session{}, nil) +} + +func TestReqBodyJsonPrefixIn(t *testing.T) { + cond, err := Build(`req_body_json_prefix_in("model", "openrouter/", false)`) + if err != nil { + t.Fatalf("build failed: %v", err) + } + + cases := []struct { + body string + matched bool + }{ + {`{"model":"openrouter/anthropic/claude-sonnet-4.6"}`, true}, + {`{"model":"openrouter/google/gemini-pro"}`, true}, + {`{"model":"google/gemini-pro"}`, false}, + {`{"model":"openrouter"}`, false}, + {`{}`, false}, + } + + for _, c := range cases { + req := buildRequestWithBody(c.body) + if matched := cond.Match(req); matched != c.matched { + t.Errorf("body %s matched=%v, want=%v", c.body, matched, c.matched) + } + } +} + +func TestReqBodyJsonPrefixInIgnoreCase(t *testing.T) { + cond, err := Build(`req_body_json_prefix_in("model", "OpenRouter/", true)`) + if err != nil { + t.Fatalf("build failed: %v", err) + } + + req := buildRequestWithBody(`{"model":"openrouter/anthropic/claude-sonnet-4.6"}`) + if !cond.Match(req) { + t.Errorf("should match openrouter/ prefix with ignore case") + } +} + +func TestReqBodyJsonPrefixInMulti(t *testing.T) { + cond, err := Build(`req_body_json_prefix_in("model", "gpt-|claude-", false)`) + if err != nil { + t.Fatalf("build failed: %v", err) + } + + cases := []struct { + body string + matched bool + }{ + {`{"model":"gpt-4"}`, true}, + {`{"model":"claude-3-opus"}`, true}, + {`{"model":"openrouter/anthropic/claude-sonnet-4.6"}`, false}, + } + + for _, c := range cases { + req := buildRequestWithBody(c.body) + if matched := cond.Match(req); matched != c.matched { + t.Errorf("body %s matched=%v, want=%v", c.body, matched, c.matched) + } + } +} + +func TestReqBodyJsonPrefixInCombineWithIn(t *testing.T) { + cond, err := Build(`req_body_json_in("model", "gpt-4", false) && req_body_json_prefix_in("model", "gpt-", false)`) + if err != nil { + t.Fatalf("build failed: %v", err) + } + + req := buildRequestWithBody(`{"model":"gpt-4"}`) + if !cond.Match(req) { + t.Errorf("should match combined condition") + } +} diff --git a/bfe_basic/request_ai_basic.go b/bfe_basic/request_ai_basic.go index 998630f6e..b34606648 100644 --- a/bfe_basic/request_ai_basic.go +++ b/bfe_basic/request_ai_basic.go @@ -34,7 +34,8 @@ const ( type TokenUsage struct { PromptTokens int64 // number of tokens in the prompt CompletionTokens int64 // number of tokens in the completion - UsedQuota int64 // used quota for this request + UsedQuota int64 // used quota for this request (unit=total_token) + UsedCost int64 // used RMB cost for this request, 1 unit = 1e-8 yuan (unit=RMB) } type TokenTimeInfo struct { @@ -53,20 +54,32 @@ type ApikeyTag struct { type AiAuthInfo struct { RejectReason string // reason for rejection RejectQuotaPlans []string // quota plan IDs rejected due to insufficient quota + HitQuotaPlans []string // quota plan IDs hit (passed balance check) for successful requests } type AiBasicInfo struct { - ClientApiKey string - ClientModel string - TargetModel string - tokenUsage TokenUsage - ApikeyTags []ApikeyTag - TokenTimeInfo TokenTimeInfo - AiAuthInfo AiAuthInfo + ClientApiKey string + ClientKeyId string + ClientModel string + TargetModel string + Provider string // upstream model provider, e.g. openai, deepseek + RetryCount uint32 // model invocation retry count (key-level retry) + CostCurrency string // cost currency, e.g. RMB, USD + tokenUsage TokenUsage + ApikeyTags []ApikeyTag + TokenTimeInfo TokenTimeInfo + AiAuthInfo AiAuthInfo + ClusterKeyNames []ClusterKeyName // tried (cluster, key) pairs during request processing allowEstimateToken bool } +// ClusterKeyName represents a tried cluster and API-Key pair during request processing +type ClusterKeyName struct { + ClusterName string + KeyName string +} + func (aiinfo *AiBasicInfo) GetTokenUsage() *TokenUsage { return &aiinfo.tokenUsage } @@ -102,6 +115,17 @@ func (r *Request) InitAiBasicInfo() *AiBasicInfo { return ret } +func (aiinfo *AiBasicInfo) AppendClusterKeyName(clusterName, keyName string) { + aiinfo.ClusterKeyNames = append(aiinfo.ClusterKeyNames, ClusterKeyName{ + ClusterName: clusterName, + KeyName: keyName, + }) +} + +func (aiinfo *AiBasicInfo) IncrementRetryCount() { + aiinfo.RetryCount++ +} + // Get user context by key. func (r *Request) GetAiBasicInfo() *AiBasicInfo { ctx := r.GetContext(REQ_AI_BASIC_CONTEXT) @@ -262,6 +286,7 @@ const ( type AiErrorDetail struct { ApiKey string `json:"api_key"` + KeyId string `json:"key_id"` QuotaPlanId string `json:"quota_plan_id"` LimitType string `json:"limit_type"` Model string `json:"model"` diff --git a/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go b/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go index 10769b212..27b3b6c90 100644 --- a/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go +++ b/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go @@ -26,6 +26,7 @@ import ( "strings" "github.com/bfenetworks/go-lib/log" + "github.com/bfenetworks/go-lib/quota" "github.com/bfenetworks/bfe/bfe_tls" "github.com/bfenetworks/bfe/bfe_util/json" @@ -126,12 +127,66 @@ type BackendHTTPS struct { protocol string // protocol of backend https } +// AIKey represents a single API key for AI service +type AIKey struct { + Name string // identifier + Key string // API key value + Weight int // weight for weighted random selection, [0,100] +} + +// AIKeyPolicy represents routing/retry policy for AI keys +type AIKeyPolicy struct { + Strategy string // "weighted_random" only in this version + MaxRetries int // total retry budget within one aiClusterInvoke call + RetryBackoffInitial int // ms + RetryBackoffMax int // ms +} + +// ModelPrice represents a single model pricing entry in AIConf.ModelTable +type ModelPrice struct { + Provider string + Model string + BaseModel string + Mode string + Capabilities []string + SupportedParameters []string + Limits map[string]interface{} + Prices map[string]float64 + Metadata map[string]interface{} +} + +// ModelTable represents the cost/pricing table for a cluster +type ModelTable struct { + Currency string // fixed "RMB" in v0.4 + Models []ModelPrice + + // priceIndex is built at config load time: model -> mode -> *ModelPrice + priceIndex map[string]map[string]*ModelPrice +} + type AIConf struct { - Type int // type of LLM service, reserved for future use. should be 0 now. - ModelMapping *map[string]string // model mapping, key is model name in req, value is model name in backend - Key *string // API key for AI service + Type int // type of LLM service, reserved for future use. should be 0 now. + ModelMapping *map[string]string // model mapping, key is model name in req, value is model name in backend + Provider string // provider name in model_prices + Keys []AIKey // multiple API keys; empty means no key injection + KeyPolicy *AIKeyPolicy // key selection & retry policy + ModelTable *ModelTable // pricing table, auto-filled by InnerAPI + + // MatchPrefix defines the provider/model prefix this cluster matches. + // Must end with '/' to avoid matching model names themselves. + MatchPrefix string `json:"MatchPrefix,omitempty"` + // StripPrefix controls whether to strip MatchPrefix from the request model + // field before forwarding to the backend. + StripPrefix bool `json:"StripPrefix"` } +const ( + PriceInputCostPerToken = "input_cost_per_token" + PriceOutputCostPerToken = "output_cost_per_token" + PriceInputCostPerTokenInt = "input_cost_per_token_int" + PriceOutputCostPerTokenInt = "output_cost_per_token_int" +) + func (conf *BackendHTTPS) GetProtocol() string { return conf.protocol } @@ -692,9 +747,94 @@ func ClusterConfCheck(conf *ClusterConf) error { return fmt.Errorf("ClusterBasic:%s", err.Error()) } + // check AIConf + if conf.AIConf != nil { + err = AIConfCheck(conf.AIConf) + if err != nil { + return fmt.Errorf("AIConf:%s", err.Error()) + } + } + + return nil +} + +// AIConfCheck checks AIConf config. +func AIConfCheck(conf *AIConf) error { + if conf.ModelTable != nil { + if err := ModelTableCheck(conf.ModelTable); err != nil { + return fmt.Errorf("ModelTable:%s", err.Error()) + } + } + + if conf.StripPrefix { + if conf.MatchPrefix == "" { + return fmt.Errorf("MatchPrefix is required when StripPrefix is true") + } + if !strings.HasSuffix(conf.MatchPrefix, "/") { + return fmt.Errorf("MatchPrefix must end with '/'") + } + } + return nil } +// ModelTableCheck checks and initializes ModelTable. +// It converts float prices to fixed-point integers and builds priceIndex. +func ModelTableCheck(table *ModelTable) error { + if table == nil { + return nil + } + + if table.Currency != quota.UnitRMB { + return fmt.Errorf("currency must be %s", quota.UnitRMB) + } + + table.priceIndex = make(map[string]map[string]*ModelPrice) + + for i := range table.Models { + price := &table.Models[i] + + if price.Model == "" { + return errors.New("model is empty") + } + if price.Mode == "" { + return errors.New("mode is empty") + } + + input := price.Prices[PriceInputCostPerToken] + output := price.Prices[PriceOutputCostPerToken] + if input < 0 || output < 0 { + return fmt.Errorf("negative price for model %s", price.Model) + } + + price.Prices[PriceInputCostPerTokenInt] = float64(quota.RmbToFixedPoint(input)) + price.Prices[PriceOutputCostPerTokenInt] = float64(quota.RmbToFixedPoint(output)) + + if table.priceIndex[price.Model] == nil { + table.priceIndex[price.Model] = make(map[string]*ModelPrice) + } + if table.priceIndex[price.Model][price.Mode] != nil { + return fmt.Errorf("duplicate model %s mode %s", price.Model, price.Mode) + } + table.priceIndex[price.Model][price.Mode] = price + } + + return nil +} + +// LookupModelPrice looks up a model price entry by model and mode. +// It returns nil if not found. +func LookupModelPrice(table *ModelTable, model, mode string) *ModelPrice { + if table == nil || table.priceIndex == nil { + return nil + } + idx, ok := table.priceIndex[model] + if !ok { + return nil + } + return idx[mode] +} + // ClusterToConfCheck check ClusterToConf. func ClusterToConfCheck(conf ClusterToConf) error { for clusterName, clusterConf := range conf { diff --git a/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load_test.go b/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load_test.go index 8ebc54dc8..e2113befc 100644 --- a/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load_test.go +++ b/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load_test.go @@ -117,3 +117,117 @@ func TestStatusCodeRange(t *testing.T) { } }) } + + +func TestModelTableCheck(t *testing.T) { + t.Run("valid RMB table", func(t *testing.T) { + table := &ModelTable{ + Currency: "RMB", + Models: []ModelPrice{ + { + Model: "deepseek-chat", + Mode: "chat", + Prices: map[string]float64{ + PriceInputCostPerToken: 0.000001, + PriceOutputCostPerToken: 0.000002, + }, + }, + }, + } + if err := ModelTableCheck(table); err != nil { + t.Fatalf("ModelTableCheck failed: %v", err) + } + if table.priceIndex == nil { + t.Fatal("priceIndex should be built") + } + entry := LookupModelPrice(table, "deepseek-chat", "chat") + if entry == nil { + t.Fatal("LookupModelPrice should return entry") + } + if entry.Prices[PriceInputCostPerTokenInt] != 100 { + t.Errorf("input cost int = %v, want 100", entry.Prices[PriceInputCostPerTokenInt]) + } + if entry.Prices[PriceOutputCostPerTokenInt] != 200 { + t.Errorf("output cost int = %v, want 200", entry.Prices[PriceOutputCostPerTokenInt]) + } + }) + + t.Run("invalid currency", func(t *testing.T) { + table := &ModelTable{ + Currency: "USD", + Models: []ModelPrice{ + {Model: "m", Mode: "chat", Prices: map[string]float64{PriceInputCostPerToken: 1, PriceOutputCostPerToken: 1}}, + }, + } + if err := ModelTableCheck(table); err == nil { + t.Error("expected error for invalid currency") + } + }) + + t.Run("negative price", func(t *testing.T) { + table := &ModelTable{ + Currency: "RMB", + Models: []ModelPrice{ + {Model: "m", Mode: "chat", Prices: map[string]float64{PriceInputCostPerToken: -1, PriceOutputCostPerToken: 1}}, + }, + } + if err := ModelTableCheck(table); err == nil { + t.Error("expected error for negative price") + } + }) + + t.Run("duplicate model mode", func(t *testing.T) { + table := &ModelTable{ + Currency: "RMB", + Models: []ModelPrice{ + {Model: "m", Mode: "chat", Prices: map[string]float64{PriceInputCostPerToken: 1, PriceOutputCostPerToken: 1}}, + {Model: "m", Mode: "chat", Prices: map[string]float64{PriceInputCostPerToken: 2, PriceOutputCostPerToken: 2}}, + }, + } + if err := ModelTableCheck(table); err == nil { + t.Error("expected error for duplicate model/mode") + } + }) + + t.Run("missing model or mode", func(t *testing.T) { + table := &ModelTable{ + Currency: "RMB", + Models: []ModelPrice{ + {Model: "", Mode: "chat", Prices: map[string]float64{PriceInputCostPerToken: 1, PriceOutputCostPerToken: 1}}, + }, + } + if err := ModelTableCheck(table); err == nil { + t.Error("expected error for empty model") + } + }) +} + +func TestAIConfCheck(t *testing.T) { + t.Run("strip prefix without match prefix", func(t *testing.T) { + conf := &AIConf{StripPrefix: true} + if err := AIConfCheck(conf); err == nil { + t.Error("expected error when StripPrefix=true but MatchPrefix is empty") + } + }) + + t.Run("match prefix without trailing slash", func(t *testing.T) { + conf := &AIConf{StripPrefix: true, MatchPrefix: "openrouter"} + if err := AIConfCheck(conf); err == nil { + t.Error("expected error when MatchPrefix does not end with '/'") + } + }) + + t.Run("valid strip prefix config", func(t *testing.T) { + conf := &AIConf{StripPrefix: true, MatchPrefix: "openrouter/"} + if err := AIConfCheck(conf); err != nil { + t.Errorf("unexpected error: %v", err) + } + }) + + t.Run("strip prefix disabled", func(t *testing.T) { + conf := &AIConf{StripPrefix: false, MatchPrefix: ""} + if err := AIConfCheck(conf); err != nil { + t.Errorf("unexpected error: %v", err) + } + }) +} diff --git a/bfe_config/bfe_conf/bfe_config_load_test.go b/bfe_config/bfe_conf/bfe_config_load_test.go index 1d7da19cf..e9b1733ef 100644 --- a/bfe_config/bfe_conf/bfe_config_load_test.go +++ b/bfe_config/bfe_conf/bfe_config_load_test.go @@ -62,6 +62,10 @@ func TestBfeConfigLoadNormal(t *testing.T) { t.Errorf("err in ClusterConf") } + if config.Server.AccessibleBodySize != 2097152 { + t.Errorf("config.AccessibleBodySize should be 2097152, got %d", config.Server.AccessibleBodySize) + } + if len(config.HttpsBasic.CipherSuites) != 3 { t.Errorf("CipherSuites length should be 3") } @@ -118,6 +122,10 @@ func TestBfeConfigLoadUsingDefault(t *testing.T) { t.Errorf("err in ClusterConf") } + if config.Server.AccessibleBodySize != 2097152 { + t.Errorf("config.AccessibleBodySize should be 2097152 (default), got %d", config.Server.AccessibleBodySize) + } + if len(config.HttpsBasic.CipherSuites) != 9 { t.Errorf("CipherSuites length should be 9") } diff --git a/bfe_config/bfe_conf/conf_basic.go b/bfe_config/bfe_conf/conf_basic.go index a0be2b67f..c4213c8d0 100644 --- a/bfe_config/bfe_conf/conf_basic.go +++ b/bfe_config/bfe_conf/conf_basic.go @@ -17,8 +17,9 @@ package bfe_conf import ( "fmt" - "github.com/bfenetworks/go-lib/log" + "github.com/bfenetworks/bfe/bfe_http" "github.com/bfenetworks/bfe/bfe_util" + "github.com/bfenetworks/go-lib/log" ) const ( @@ -33,9 +34,9 @@ type ConfigBasic struct { HttpsAddr string // listen address for https, default all interfaces MonitorPort int // web server port for monitor MonitorAddr string // listen address for monitor, default all interfaces - MaxCpus int // number of max cpus to use - AcceptNum int // number of accept goroutine for each listener, default 1 - MonitorEnabled bool // web server for monitor enable or not + MaxCpus int // number of max cpus to use + AcceptNum int // number of accept goroutine for each listener, default 1 + MonitorEnabled bool // web server for monitor enable or not // settings of layer-4 load balancer Layer4LoadBalancer string @@ -53,6 +54,9 @@ type ConfigBasic struct { EnableAiGateway bool // enable ai gateway EstimateToken bool // whether estimate token usage from content length + AccessibleBodySize int64 // max size in bytes to buffer request body for rewriting/fallback + TotalBodyBufferSize int64 // max total bytes of all active bytes_body buffers (0 means unlimited) + Modules []string // modules to load // location of data files for bfe_route @@ -90,6 +94,9 @@ func (cfg *ConfigBasic) SetDefaultConf() { cfg.MaxHeaderUriBytes = 8192 cfg.KeepAliveEnabled = true + cfg.AccessibleBodySize = bfe_http.DefaultAccessibleBodySize + cfg.TotalBodyBufferSize = 0 + cfg.HostRuleConf = "server_data_conf/host_rule.data" cfg.VipRuleConf = "server_data_conf/vip_rule.data" cfg.RouteRuleConf = "server_data_conf/route_rule.data" @@ -211,6 +218,20 @@ func basicConfCheck(cfg *ConfigBasic) error { return fmt.Errorf("MaxHeaderHeaderBytes[%d] should > 0", cfg.MaxHeaderBytes) } + // check AccessibleBodySize + if cfg.AccessibleBodySize <= 0 { + cfg.AccessibleBodySize = bfe_http.DefaultAccessibleBodySize + log.Logger.Warn("AccessibleBodySize not set or invalid, use default value(%d)", cfg.AccessibleBodySize) + } + if cfg.AccessibleBodySize > bfe_http.MaxAccessibleBodySize { + return fmt.Errorf("AccessibleBodySize[%d] should <= %d", cfg.AccessibleBodySize, bfe_http.MaxAccessibleBodySize) + } + + // check TotalBodyBufferSize + if cfg.TotalBodyBufferSize < 0 { + return fmt.Errorf("TotalBodyBufferSize[%d] should >= 0", cfg.TotalBodyBufferSize) + } + return nil } diff --git a/bfe_config/bfe_conf/conf_basic_test.go b/bfe_config/bfe_conf/conf_basic_test.go index b3e07f820..a9c4b6f49 100644 --- a/bfe_config/bfe_conf/conf_basic_test.go +++ b/bfe_config/bfe_conf/conf_basic_test.go @@ -16,9 +16,7 @@ package bfe_conf import ( "testing" -) -import ( gcfg "gopkg.in/gcfg.v1" ) @@ -68,6 +66,10 @@ func Test_conf_basic_case1(t *testing.T) { if config.Server.HostRuleConf != "/home/bfe/conf/host_rule123.conf" { t.Error("config.HostRuleConf should be '/home/bfe/conf/host_rule123.conf'") } + + if config.Server.AccessibleBodySize != 1048576 { + t.Errorf("config.AccessibleBodySize should be 1048576, got %d", config.Server.AccessibleBodySize) + } } func Test_conf_basic_case2(t *testing.T) { @@ -101,13 +103,21 @@ func Test_conf_basic_check(t *testing.T) { conf *ConfigBasic err string }{ - {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: -1, MonitorEnabled: true},"MonitorPort[-1] should be in [1, 65535]"}, + {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: -1, MonitorEnabled: true}, "MonitorPort[-1] should be in [1, 65535]"}, {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: 8080, MonitorEnabled: false, MaxCpus: -1}, "MaxCpus[-1] is too small"}, {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: 8080, MonitorEnabled: true, MaxCpus: 10, TlsHandshakeTimeout: 30, GracefulShutdownTimeout: 30}, "ClientReadTimeout[0] should > 0"}, {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: 8080, MonitorEnabled: true, MaxCpus: 10, TlsHandshakeTimeout: 30, GracefulShutdownTimeout: 30, ClientReadTimeout: 10, ClientWriteTimeout: 10, MonitorInterval: 33}, "MonitorInterval[33] can not divide 60"}, + {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: 8080, MonitorEnabled: true, MaxCpus: 10, TlsHandshakeTimeout: 30, + GracefulShutdownTimeout: 30, ClientReadTimeout: 10, ClientWriteTimeout: 10, MonitorInterval: 20, + MaxHeaderUriBytes: 8096, MaxHeaderBytes: 8096, AccessibleBodySize: 99999999}, + "AccessibleBodySize[99999999] should <= 8388608"}, + {&ConfigBasic{HttpPort: 80, HttpsPort: 443, MonitorPort: 8080, MonitorEnabled: true, MaxCpus: 10, TlsHandshakeTimeout: 30, + GracefulShutdownTimeout: 30, ClientReadTimeout: 10, ClientWriteTimeout: 10, MonitorInterval: 20, + MaxHeaderUriBytes: 8096, MaxHeaderBytes: 8096, AccessibleBodySize: 2097152, TotalBodyBufferSize: -1}, + "TotalBodyBufferSize[-1] should >= 0"}, } for _, c := range checks { diff --git a/bfe_config/bfe_conf/testdata/conf_all/bfe.conf b/bfe_config/bfe_conf/testdata/conf_all/bfe.conf index abc762f95..dad48b66a 100644 --- a/bfe_config/bfe_conf/testdata/conf_all/bfe.conf +++ b/bfe_config/bfe_conf/testdata/conf_all/bfe.conf @@ -33,6 +33,12 @@ clusterConf = cluster_conf/cluster_conf.data maxHeaderUriBytes=8096 maxHeaderBytes=8096 +# max size in bytes to buffer request body for rewriting/fallback (default 2MB, max 8MB) +accessibleBodySize = 2097152 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +totalBodyBufferSize = 0 + [HttpsBasic] # listen port and cert conf for https serverCertConf = tls_conf/server_cert_conf.data diff --git a/bfe_config/bfe_conf/testdata/conf_basic/bfe_1.conf b/bfe_config/bfe_conf/testdata/conf_basic/bfe_1.conf index 2eccbc3d9..a20490669 100644 --- a/bfe_config/bfe_conf/testdata/conf_basic/bfe_1.conf +++ b/bfe_config/bfe_conf/testdata/conf_basic/bfe_1.conf @@ -26,3 +26,5 @@ clusterConf = cluster_conf123.conf maxHeaderUriBytes=8096 maxHeaderBytes=8096 + +accessibleBodySize = 1048576 diff --git a/bfe_http/transfer.go b/bfe_http/transfer.go index cef850a0e..a917b6a2b 100644 --- a/bfe_http/transfer.go +++ b/bfe_http/transfer.go @@ -24,12 +24,12 @@ import ( "fmt" "io" "io/ioutil" + "os" "strconv" "strings" "sync" -) + "sync/atomic" -import ( "github.com/bfenetworks/bfe/bfe_bufio" "github.com/bfenetworks/bfe/bfe_net/textproto" ) @@ -755,19 +755,138 @@ type BodyAccessor interface { SetBytes([]byte, bool) } -//body with BodyAccessor interface +type Rewindable interface { + Rewind() bool +} + +// totalBytesBodyBuffer tracks the sum of bytes_body.buf sizes currently in use. +var totalBytesBodyBuffer int64 + +// totalBytesBodyBufferLimit is the upper bound for totalBytesBodyBuffer. +// 0 means unlimited. +var totalBytesBodyBufferLimit int64 + +func init() { + // Allow integration tests to pre-seed the total body buffer counter so they + // can verify the limit-checking behavior without relying on timing-sensitive + // blocking backends. This environment variable is not intended for production + // use and is ignored when unset or invalid. + if v := os.Getenv("BFE_TEST_INITIAL_TOTAL_BYTES_BODY_BUFFER"); v != "" { + if n, err := strconv.ParseInt(v, 10, 64); err == nil && n >= 0 { + atomic.StoreInt64(&totalBytesBodyBuffer, n) + } + } +} + +// SetTotalBodyBufferSizeLimit sets the limit for total bytes_body buffer size. +// 0 or negative means unlimited. +func SetTotalBodyBufferSizeLimit(limit int64) { + if limit < 0 { + limit = 0 + } + atomic.StoreInt64(&totalBytesBodyBufferLimit, limit) +} + +// TotalBodyBufferSizeLimit returns the current limit. +func TotalBodyBufferSizeLimit() int64 { + return atomic.LoadInt64(&totalBytesBodyBufferLimit) +} + +// TotalBytesBodyBuffer returns the current total bytes_body buffer size. +func TotalBytesBodyBuffer() int64 { + return atomic.LoadInt64(&totalBytesBodyBuffer) +} + +// addTotalBytesBodyBuffer adds delta to the total buffer size and returns true. +// If limit > 0 and the new total would exceed the limit, the addition is not +// performed and false is returned. +func addTotalBytesBodyBuffer(delta int64) bool { + if delta == 0 { + return true + } + limit := TotalBodyBufferSizeLimit() + if limit <= 0 { + atomic.AddInt64(&totalBytesBodyBuffer, delta) + return true + } + for { + old := atomic.LoadInt64(&totalBytesBodyBuffer) + new := old + delta + if new > limit { + return false + } + if atomic.CompareAndSwapInt64(&totalBytesBodyBuffer, old, new) { + return true + } + } +} + +// subTotalBytesBodyBuffer subtracts delta from the total buffer size. +func subTotalBytesBodyBuffer(delta int64) { + if delta == 0 { + return + } + atomic.AddInt64(&totalBytesBodyBuffer, -delta) +} + +// body with BodyAccessor interface type bytes_body struct { - src io.ReadCloser // source body - buf []byte // bytes read out from src - all bool // all already read out from src to buf - r io.Reader // multiReader of buf and src + src io.ReadCloser // source body + buf []byte // bytes read out from src + all bool // all already read out from src to buf + srcClosed bool // source has been closed + released bool // whether buffer has been subtracted from total accounting + r *bodyReader // reader of buf and src + err error +} + +// bodyReader tracks whether reading has moved from the buffer to the source. +type bodyReader struct { + buf *bytes.Buffer + src io.Reader + srcStarted bool +} + +func (br *bodyReader) Read(p []byte) (int, error) { + if !br.srcStarted { + n, err := br.buf.Read(p) + if err != io.EOF { + return n, err + } + br.srcStarted = true + if n > 0 { + return n, nil + } + } + if br.src == nil { + return 0, io.EOF + } + return br.src.Read(p) } func (b *bytes_body) Read(p []byte) (n int, err error) { - return b.r.Read(p) + if b.err != nil { + return 0, b.err + } + n, err = b.r.Read(p) + if err != nil { + b.err = err + } + return } func (b *bytes_body) Close() error { + if b.srcClosed { + return nil + } + b.srcClosed = true + if !b.released { + b.released = true + subTotalBytesBodyBuffer(int64(len(b.buf))) + } + if b.src == nil { + return nil + } return b.src.Close() } @@ -790,16 +909,36 @@ func (b *bytes_body) GetBytes() ([]byte, bool) { } func (b *bytes_body) SetBytes(newBuf []byte, all bool) { + if !b.released { + subTotalBytesBodyBuffer(int64(len(b.buf))) + addTotalBytesBodyBuffer(int64(len(newBuf))) + } b.buf = newBuf br := bytes.NewBuffer(newBuf) b.all = b.all || all + b.err = nil if b.all { - b.r = br + b.r = &bodyReader{buf: br} } else { - b.r = io.MultiReader(br, b.src) + b.r = &bodyReader{buf: br, src: b.src} } } +func (b *bytes_body) SrcStarted() bool { + return b.r != nil && b.r.srcStarted +} + +func (b *bytes_body) Rewind() bool { + // If all data is already in buffer, we can rewind regardless of + // src state. Otherwise, rewind is only possible before src is + // started or closed. + if (b.srcClosed || b.SrcStarted()) && !b.all { + return false + } + b.SetBytes(b.buf, b.all) + return true +} + func NewBytesBody(src io.ReadCloser, maxSize int64) (io.ReadCloser, error) { b, err := newBytesBody(src, maxSize) if b == nil { @@ -815,20 +954,21 @@ func newBytesBody(src io.ReadCloser, maxSize int64) (*bytes_body, error) { } br := bytes.NewBuffer(bb) + addTotalBytesBodyBuffer(int64(len(bb))) if len(bb) < int(maxSize) { return &bytes_body{ src: src, buf: bb, all: true, - r: br, + r: &bodyReader{buf: br}, }, nil } else { return &bytes_body{ src: src, buf: bb, all: false, - r: io.MultiReader(br, src), + r: &bodyReader{buf: br, src: src}, }, nil } } diff --git a/bfe_http/transfer_test.go b/bfe_http/transfer_test.go index 14bce7d17..4d14b4e1e 100644 --- a/bfe_http/transfer_test.go +++ b/bfe_http/transfer_test.go @@ -19,11 +19,12 @@ package bfe_http import ( + "bytes" + "io" "strings" + "sync/atomic" "testing" -) -import ( "github.com/bfenetworks/bfe/bfe_bufio" ) @@ -52,3 +53,89 @@ func TestBodyReadBadTrailer(t *testing.T) { t.Errorf("final Read was successful (%q), expected error from trailer read", got) } } + +func resetTotalBytesBodyBuffer() { + atomic.StoreInt64(&totalBytesBodyBuffer, 0) + SetTotalBodyBufferSizeLimit(0) +} + +func TestBytesBodyBufferAccounting(t *testing.T) { + resetTotalBytesBodyBuffer() + defer resetTotalBytesBodyBuffer() + + data := []byte("hello world") + body, err := NewBytesBody(io.NopCloser(bytes.NewReader(data)), int64(len(data)+1)) + if err != nil { + t.Fatalf("NewBytesBody failed: %v", err) + } + + if got := TotalBytesBodyBuffer(); got != int64(len(data)) { + t.Fatalf("expected total %d, got %d", len(data), got) + } + + if err := body.Close(); err != nil { + t.Fatalf("Close failed: %v", err) + } + + if got := TotalBytesBodyBuffer(); got != 0 { + t.Fatalf("expected total 0 after close, got %d", got) + } +} + +func TestBytesBodySetBytesAdjustsTotal(t *testing.T) { + resetTotalBytesBodyBuffer() + defer resetTotalBytesBodyBuffer() + + data := []byte("hello") + body, err := NewBytesBody(io.NopCloser(bytes.NewReader(data)), int64(len(data)+1)) + if err != nil { + t.Fatalf("NewBytesBody failed: %v", err) + } + + if got := TotalBytesBodyBuffer(); got != int64(len(data)) { + t.Fatalf("expected total %d, got %d", len(data), got) + } + + newData := []byte("hello world") + body.(BodyAccessor).SetBytes(newData, true) + + if got := TotalBytesBodyBuffer(); got != int64(len(newData)) { + t.Fatalf("expected total %d after SetBytes, got %d", len(newData), got) + } + + if err := body.Close(); err != nil { + t.Fatalf("Close failed: %v", err) + } + + if got := TotalBytesBodyBuffer(); got != 0 { + t.Fatalf("expected total 0 after close, got %d", got) + } +} + +func TestAddTotalBytesBodyBufferLimit(t *testing.T) { + resetTotalBytesBodyBuffer() + defer resetTotalBytesBodyBuffer() + + SetTotalBodyBufferSizeLimit(10) + + if !addTotalBytesBodyBuffer(5) { + t.Fatal("expected add 5 to succeed") + } + if TotalBytesBodyBuffer() != 5 { + t.Fatalf("expected total 5, got %d", TotalBytesBodyBuffer()) + } + + if !addTotalBytesBodyBuffer(5) { + t.Fatal("expected add another 5 to succeed") + } + if TotalBytesBodyBuffer() != 10 { + t.Fatalf("expected total 10, got %d", TotalBytesBodyBuffer()) + } + + if addTotalBytesBodyBuffer(1) { + t.Fatal("expected add 1 to fail due to limit") + } + if TotalBytesBodyBuffer() != 10 { + t.Fatalf("expected total still 10, got %d", TotalBytesBodyBuffer()) + } +} diff --git a/bfe_modules/mod_access_pb3/request_log.go b/bfe_modules/mod_access_pb3/request_log.go index 4381c8cd9..c39388f56 100644 --- a/bfe_modules/mod_access_pb3/request_log.go +++ b/bfe_modules/mod_access_pb3/request_log.go @@ -371,9 +371,9 @@ func reqAiInfoGen(reqLog *bfe_access_pb3.RequestLog, req *bfe_basic.Request, res return } - // API Key - if aiInfo.ClientApiKey != "" { - reqLog.AiApikey = proto.String(aiInfo.ClientApiKey) + // API Key ID (not the raw API Key value) + if aiInfo.ClientKeyId != "" { + reqLog.AiApikeyId = proto.String(aiInfo.ClientKeyId) } // API Key Tags @@ -391,7 +391,12 @@ func reqAiInfoGen(reqLog *bfe_access_pb3.RequestLog, req *bfe_basic.Request, res reqLog.AiRequestedModel = proto.String(aiInfo.ClientModel) } if aiInfo.TargetModel != "" { - reqLog.AiMappedModel = proto.String(aiInfo.TargetModel) + reqLog.AiTargetModel = proto.String(aiInfo.TargetModel) + } + + // Provider + if aiInfo.Provider != "" { + reqLog.AiProvider = proto.String(aiInfo.Provider) } // Stream @@ -400,9 +405,22 @@ func reqAiInfoGen(reqLog *bfe_access_pb3.RequestLog, req *bfe_basic.Request, res // Token usage usage := aiInfo.GetTokenUsage() if usage != nil { - reqLog.AiPromptTokens = proto.Int64(usage.PromptTokens) + reqLog.AiInputTokens = proto.Int64(usage.PromptTokens) reqLog.AiOutputTokens = proto.Int64(usage.CompletionTokens) reqLog.AiTotalTokens = proto.Int64(usage.UsedQuota) + if usage.UsedCost > 0 { + reqLog.AiCostValue = proto.Int64(usage.UsedCost) + } + } + + // Cost currency + if aiInfo.CostCurrency != "" { + reqLog.AiCostCurrency = proto.String(aiInfo.CostCurrency) + } + + // Retry count + if aiInfo.RetryCount > 0 { + reqLog.AiRetryCount = proto.Uint32(aiInfo.RetryCount) } // TTFT / TPOT @@ -420,6 +438,26 @@ func reqAiInfoGen(reqLog *bfe_access_pb3.RequestLog, req *bfe_basic.Request, res for _, item := range aiInfo.AiAuthInfo.RejectQuotaPlans { reqLog.AiAuthRejectQuotaPlans = append(reqLog.AiAuthRejectQuotaPlans, item) } + for _, item := range aiInfo.AiAuthInfo.HitQuotaPlans { + reqLog.AiAuthHitQuotaPlans = append(reqLog.AiAuthHitQuotaPlans, item) + } + + // Route rule hits + if routeResult := req.GetAiRouteResult(); routeResult != nil { + reqLog.AiRouteRuleHits = append(reqLog.AiRouteRuleHits, &bfe_access_pb3.AIRouteRuleHit{ + RuleOwner: proto.String(routeResult.Owner), + RuleOwnerType: proto.String(routeResult.RouteType), + RuleName: proto.String(routeResult.RuleName), + }) + } + + // Cluster / key attempts + for _, ckn := range aiInfo.ClusterKeyNames { + reqLog.AiClusterKeyNames = append(reqLog.AiClusterKeyNames, &bfe_access_pb3.ClusterKeyName{ + ClusterName: proto.String(ckn.ClusterName), + KeyName: proto.String(ckn.KeyName), + }) + } // Rate limit hit info hitInfo := req.GetAiRateLimitHitInfo() diff --git a/bfe_modules/mod_access_pb3/request_log_test.go b/bfe_modules/mod_access_pb3/request_log_test.go index 28b847f3c..f75b79017 100644 --- a/bfe_modules/mod_access_pb3/request_log_test.go +++ b/bfe_modules/mod_access_pb3/request_log_test.go @@ -351,9 +351,15 @@ func TestReqAiInfoGen(t *testing.T) { _, req, res := makeRequestLogTest(t) aiInfo := &bfe_basic.AiBasicInfo{ - ClientApiKey: "apikey123", + ClientKeyId: "key-id-123", ClientModel: "model-a", TargetModel: "model-b", + Provider: "deepseek", + RetryCount: 1, + CostCurrency: "RMB", + ClusterKeyNames: []bfe_basic.ClusterKeyName{ + {ClusterName: "cluster-a", KeyName: "key-001"}, + }, TokenTimeInfo: bfe_basic.TokenTimeInfo{ TTFT: 1000, TPOT: 2000, @@ -361,14 +367,22 @@ func TestReqAiInfoGen(t *testing.T) { AiAuthInfo: bfe_basic.AiAuthInfo{ RejectReason: "quota exhausted", RejectQuotaPlans: []string{"plan1", "plan2"}, + HitQuotaPlans: []string{"plan3", "plan4"}, }, } usage := aiInfo.GetTokenUsage() usage.PromptTokens = 10 usage.CompletionTokens = 20 usage.UsedQuota = 30 + usage.UsedCost = 5000 req.SetContext(bfe_basic.REQ_AI_BASIC_CONTEXT, aiInfo) + req.SetAiRouteResult(&bfe_basic.AiRouteResult{ + RouteType: "apikey", + Owner: "ak_user_a", + RuleName: "user_a-rule1", + }) + hitInfo := &bfe_basic.AiRateLimitHitInfo{ HitPolicyDict: map[string]*bfe_basic.HitPolicyInfo{ "policy1": { @@ -384,20 +398,32 @@ func TestReqAiInfoGen(t *testing.T) { reqLog := &bfe_access_pb3.RequestLog{} reqAiInfoGen(reqLog, req, res) - if reqLog.AiApikey == nil || *reqLog.AiApikey != "apikey123" { - t.Error("AiApikey error") + if reqLog.AiApikeyId == nil || *reqLog.AiApikeyId != "key-id-123" { + t.Error("AiApikeyId error") } if reqLog.AiRequestedModel == nil || *reqLog.AiRequestedModel != "model-a" { t.Error("AiRequestedModel error") } - if reqLog.AiMappedModel == nil || *reqLog.AiMappedModel != "model-b" { - t.Error("AiMappedModel error") + if reqLog.AiTargetModel == nil || *reqLog.AiTargetModel != "model-b" { + t.Error("AiTargetModel error") + } + if reqLog.AiProvider == nil || *reqLog.AiProvider != "deepseek" { + t.Error("AiProvider error") } if reqLog.AiStream == nil || !*reqLog.AiStream { t.Error("AiStream error") } - if reqLog.AiPromptTokens == nil || *reqLog.AiPromptTokens != 10 { - t.Error("AiPromptTokens error") + if reqLog.AiInputTokens == nil || *reqLog.AiInputTokens != 10 { + t.Error("AiInputTokens error") + } + if reqLog.AiCostValue == nil || *reqLog.AiCostValue != 5000 { + t.Error("AiCostValue error") + } + if reqLog.AiCostCurrency == nil || *reqLog.AiCostCurrency != "RMB" { + t.Error("AiCostCurrency error") + } + if reqLog.AiRetryCount == nil || *reqLog.AiRetryCount != 1 { + t.Error("AiRetryCount error") } if reqLog.AiTtftUs == nil || *reqLog.AiTtftUs != 1000 { t.Error("AiTtftUs error") @@ -411,6 +437,15 @@ func TestReqAiInfoGen(t *testing.T) { if len(reqLog.AiAuthRejectQuotaPlans) != 2 { t.Error("AiAuthRejectQuotaPlans length error") } + if len(reqLog.AiAuthHitQuotaPlans) != 2 { + t.Error("AiAuthHitQuotaPlans length error") + } + if len(reqLog.AiRouteRuleHits) != 1 { + t.Error("AiRouteRuleHits length error") + } + if len(reqLog.AiClusterKeyNames) != 1 { + t.Error("AiClusterKeyNames length error") + } if len(reqLog.AiRateLimitHits) != 4 { t.Errorf("AiRateLimitHits length error, got: %d", len(reqLog.AiRateLimitHits)) } @@ -421,7 +456,7 @@ func TestReqAiInfoGenNil(t *testing.T) { reqLog := &bfe_access_pb3.RequestLog{} reqAiInfoGen(reqLog, req, res) - if reqLog.AiApikey != nil { - t.Error("AiApikey should be nil when no ai info") + if reqLog.AiApikeyId != nil { + t.Error("AiApikeyId should be nil when no ai info") } } diff --git a/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go b/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go index e22bf38db..e51bad460 100644 --- a/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go +++ b/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go @@ -20,11 +20,13 @@ import ( "strings" "github.com/bfenetworks/go-lib/log" + "github.com/bfenetworks/go-lib/quota" "github.com/bfenetworks/go-lib/web-monitor/metrics" "github.com/bfenetworks/go-lib/web-monitor/web_monitor" "github.com/tidwall/gjson" "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" "github.com/bfenetworks/bfe/bfe_http" "github.com/bfenetworks/bfe/bfe_module" "github.com/bfenetworks/bfe/bfe_util/redis_client" @@ -171,15 +173,34 @@ func (m *ModuleAITokenAuth) tokenRequestFinishHandler(req *bfe_basic.Request, re if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) // calculate used quota } - if tokenUsage.UsedQuota > 0 { - // deduct usedquota from every quotaplan + + // calculate RMB cost at request finish time using token usage already populated + // by mod_body_process (streaming) or tokenReadResponseHandler (non-streaming). + if tokenUsage.UsedCost <= 0 && hasRMBPlan(ctx.Token.QuotaPlans) { + tokenUsage.UsedCost = m.calcCostUnits(req, ctx.serverConf, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + + costUnits := tokenUsage.UsedCost + + if tokenUsage.UsedQuota > 0 || costUnits > 0 { for _, plan := range ctx.Token.QuotaPlans { if plan.Unlimited { continue } - _, err := plan.Deduct(m.redisClient, tokenUsage.UsedQuota) - if err != nil { - return bfe_module.BfeHandlerGoOn + if quota.IsRMB(plan.Unit) { + if costUnits > 0 { + _, err := plan.Deduct(m.redisClient, costUnits) + if err != nil { + log.Logger.Warn("deduct rmb quota failed: %v", err) + } + } + } else { + if tokenUsage.UsedQuota > 0 { + _, err := plan.Deduct(m.redisClient, tokenUsage.UsedQuota) + if err != nil { + log.Logger.Warn("deduct token quota failed: %v", err) + } + } } } } @@ -326,6 +347,9 @@ func (m *ModuleAITokenAuth) Init(cbs *bfe_module.BfeCallbacks, whs *web_monitor. type TokenAuthContext struct { Token *Token aiBasicInfo *bfe_basic.AiBasicInfo + // serverConf caches the SvrDataConf before it is cleared by the reverse proxy. + // It is used for RMB cost calculation at request finish time. + serverConf bfe_basic.ServerDataConfInterface } const REQ_TOKEN_AUTH_CONTEXT = "tokenauth_ctx" @@ -344,6 +368,7 @@ func GetTokenAuthContext(req *bfe_basic.Request) *TokenAuthContext { func SetTokenAuthContext(req *bfe_basic.Request, tok *Token, promptToken int64, tags []bfe_basic.ApikeyTag) { aiBasicInfo := req.GetAiBasicInfo() if aiBasicInfo != nil { + aiBasicInfo.ClientKeyId = tok.KeyId tusage := aiBasicInfo.GetTokenUsage() tusage.PromptTokens = promptToken tusage.CompletionTokens = bfe_basic.COMPLETION_TOKENS_UNKNOWN // -1 - unknown @@ -353,6 +378,7 @@ func SetTokenAuthContext(req *bfe_basic.Request, tok *Token, promptToken int64, tokenCtx := &TokenAuthContext{ Token: tok, aiBasicInfo: aiBasicInfo, + serverConf: req.SvrDataConf, } req.SetContext(REQ_TOKEN_AUTH_CONTEXT, tokenCtx) } @@ -374,3 +400,49 @@ func GetPromptToken(req *bfe_basic.Request) int64 { body, _ := bodyAccessor.GetBytes() return int64(len(body)) / 4 } + +func hasRMBPlan(plans []*QuotaPlan) bool { + for _, plan := range plans { + if quota.IsRMB(plan.Unit) { + return true + } + } + return false +} + +func (m *ModuleAITokenAuth) calcCostUnits(req *bfe_basic.Request, serverConf bfe_basic.ServerDataConfInterface, promptTokens, completionTokens int64) int64 { + aiMeta := req.GetAiBasicInfo() + if aiMeta == nil { + return 0 + } + + clusterName := req.Route.ClusterName + targetModel := aiMeta.TargetModel + if clusterName == "" || targetModel == "" { + return 0 + } + + if serverConf == nil { + return 0 + } + cluster, err := serverConf.ClusterTableLookup(clusterName) + if err != nil || cluster == nil || cluster.AIConf == nil || cluster.AIConf.ModelTable == nil { + log.Logger.Warn("model table not found for cluster %s", clusterName) + return 0 + } + + entry := cluster_conf.LookupModelPrice(cluster.AIConf.ModelTable, targetModel, "chat") + if entry == nil { + log.Logger.Warn("model price not found for cluster %s model %s", clusterName, targetModel) + return 0 + } + + inputCost := int64(entry.Prices[cluster_conf.PriceInputCostPerTokenInt]) + outputCost := int64(entry.Prices[cluster_conf.PriceOutputCostPerTokenInt]) + if inputCost < 0 || outputCost < 0 { + log.Logger.Warn("invalid model price for cluster %s model %s", clusterName, targetModel) + return 0 + } + + return promptTokens*inputCost + completionTokens*outputCost +} diff --git a/bfe_modules/mod_ai_token_auth/mod_ai_token_auth_test.go b/bfe_modules/mod_ai_token_auth/mod_ai_token_auth_test.go index 7d1d9d90b..78e3f6bf9 100644 --- a/bfe_modules/mod_ai_token_auth/mod_ai_token_auth_test.go +++ b/bfe_modules/mod_ai_token_auth/mod_ai_token_auth_test.go @@ -15,6 +15,7 @@ package mod_ai_token_auth import ( + "fmt" "io/ioutil" "net" "net/http" @@ -24,8 +25,12 @@ import ( "time" "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" "github.com/bfenetworks/bfe/bfe_http" "github.com/bfenetworks/bfe/bfe_module" + "github.com/bfenetworks/bfe/bfe_route/bfe_cluster" + "github.com/bfenetworks/bfe/bfe_util/redis_client" + "github.com/bfenetworks/go-lib/quota" ) const testConfRoot = "testdata/mod_ai_token_auth" @@ -168,7 +173,7 @@ func TestConfCheckDefaultProductRulePath(t *testing.T) { if err := cfg.Check(testConfRoot); err != nil { t.Fatalf("Check failed: %s", err) } - if !strings.Contains(cfg.Basic.ProductRulePath, "mod_ai_toekn_auth/token_rule.data") { + if !strings.Contains(cfg.Basic.ProductRulePath, "mod_ai_token_auth/token_rule.data") { t.Errorf("unexpected default ProductRulePath: %s", cfg.Basic.ProductRulePath) } } @@ -318,7 +323,7 @@ func TestUpdateCtxByUsage(t *testing.T) { func TestTokenAuthContext(t *testing.T) { req := newTestRequest("", "AI_product") ai := req.InitAiBasicInfo() - tok := &Token{Key: "ak-123"} + tok := &Token{Key: "ak-123", KeyId: "ak-123-id"} SetTokenAuthContext(req, tok, 5, []bfe_basic.ApikeyTag{{TagName: "t", TagValue: "v"}}) ctx := GetTokenAuthContext(req) @@ -345,6 +350,7 @@ func TestTokenCheck(t *testing.T) { valid := func() *TokenFile { return &TokenFile{ Key: "ak-123", + KeyId: "ak-123-id", Status: TokenStatusEnabled, ExpiredTime: -1, UnlimitedQuota: true, @@ -367,6 +373,13 @@ func TestTokenCheck(t *testing.T) { }, errSub: "no Key", }, + { + name: "missing key_id", + mutate: func(tf *TokenFile) { + tf.KeyId = "" + }, + errSub: "no KeyId", + }, { name: "invalid status", mutate: func(tf *TokenFile) { @@ -432,6 +445,7 @@ func TestTokenCheck(t *testing.T) { func TestTokenConvert(t *testing.T) { tf := TokenFile{ Key: "ak-123", + KeyId: "ak-123-id", Status: TokenStatusEnabled, ExpiredTime: -1, UnlimitedQuota: false, @@ -443,7 +457,7 @@ func TestTokenConvert(t *testing.T) { if err != nil { t.Fatalf("tokenConvert failed: %s", err) } - if token.Key != "ak-123" || len(token.QuotaPlans) != 1 { + if token.Key != "ak-123" || token.KeyId != "ak-123-id" || len(token.QuotaPlans) != 1 { t.Errorf("unexpected token: %+v", token) } @@ -523,9 +537,9 @@ func TestValidateUserToken(t *testing.T) { t.Errorf("unexpected token: %s", tok.Key) } - exhausted := &Token{Key: "ak-exhausted", Status: TokenStatusExhausted, Name: "ex", ExpiredTime: -1} - disabled := &Token{Key: "ak-disabled", Status: TokenStatusDisabled, Name: "dis", ExpiredTime: -1} - expired := &Token{Key: "ak-expired", Status: TokenStatusEnabled, Name: "exp", ExpiredTime: time.Now().Unix() - 10} + exhausted := &Token{Key: "ak-exhausted", KeyId: "ak-exhausted-id", Status: TokenStatusExhausted, ExpiredTime: -1} + disabled := &Token{Key: "ak-disabled", KeyId: "ak-disabled-id", Status: TokenStatusDisabled, ExpiredTime: -1} + expired := &Token{Key: "ak-expired", KeyId: "ak-expired-id", Status: TokenStatusEnabled, ExpiredTime: time.Now().Unix() - 10} table.lock.Lock() (*table.productTokens["AI_product"])["ak-exhausted"] = exhausted (*table.productTokens["AI_product"])["ak-disabled"] = disabled @@ -597,7 +611,7 @@ func TestTokenReadResponseHandler(t *testing.T) { m := NewModuleAITokenAuth() req := newTestRequest("ak-123", "AI_product") ai := req.InitAiBasicInfo() - SetTokenAuthContext(req, &Token{Key: "ak-123"}, 2, nil) + SetTokenAuthContext(req, &Token{Key: "ak-123", KeyId: "ak-123-id"}, 2, nil) body := `{"usage":{"total_tokens":20,"prompt_tokens":5,"completion_tokens":15}}` res := &bfe_http.Response{ @@ -638,12 +652,189 @@ func TestTokenRequestFinishHandler(t *testing.T) { req2 := newTestRequest("ak-123", "AI_product") ai2 := req2.InitAiBasicInfo() ai2.SetAllowEstimateToken(true) - SetTokenAuthContext(req2, &Token{Key: "ak-123", UnlimitedQuota: true}, 4, nil) + SetTokenAuthContext(req2, &Token{Key: "ak-123", KeyId: "ak-123-id", UnlimitedQuota: true}, 4, nil) if ret := m.tokenRequestFinishHandler(req2, res); ret != bfe_module.BfeHandlerGoOn { t.Errorf("expected goon, got %d", ret) } } +func TestTokenReadResponseHandlerDoesNotCalcCost(t *testing.T) { + m := NewModuleAITokenAuth() + req := newTestRequest("ak-123", "AI_product") + ai := req.InitAiBasicInfo() + ai.SetAllowEstimateToken(true) + + // Simulate a non-streaming response with usage body. + body := `{"usage":{"total_tokens":20,"prompt_tokens":5,"completion_tokens":15}}` + res := &bfe_http.Response{ + StatusCode: 200, + ContentLength: int64(len(body)), + Body: ioutil.NopCloser(strings.NewReader(body)), + } + + SetTokenAuthContext(req, &Token{Key: "ak-123", KeyId: "ak-123-id"}, 2, nil) + + ret := m.tokenReadResponseHandler(req, res) + if ret != bfe_module.BfeHandlerGoOn { + t.Errorf("expected goon, got %d", ret) + } + + usage := ai.GetTokenUsage() + if usage.UsedQuota != 20 { + t.Errorf("expected UsedQuota 20, got %d", usage.UsedQuota) + } + // UsedCost should NOT be calculated at read-response stage anymore. + if usage.UsedCost != 0 { + t.Errorf("expected UsedCost 0 at read-response stage, got %d", usage.UsedCost) + } +} + +type mockServerDataConf struct { + clusters map[string]*bfe_cluster.BfeCluster +} + +func (m *mockServerDataConf) ClusterTableLookup(clusterName string) (*bfe_cluster.BfeCluster, error) { + return m.clusters[clusterName], nil +} + +func (m *mockServerDataConf) HostTableLookup(hostname string) (string, error) { + return hostname, nil +} + +func newTestRequestWithCluster(apiKey, product, clusterName, targetModel string) *bfe_basic.Request { + req := newTestRequest(apiKey, product) + req.Route.ClusterName = clusterName + ai := req.InitAiBasicInfo() + ai.ClientModel = targetModel + ai.TargetModel = targetModel + return req +} + +func buildTestClusterConf(model string, inputCost, outputCost float64) *bfe_cluster.BfeCluster { + modelTable := &cluster_conf.ModelTable{ + Currency: "RMB", + Models: []cluster_conf.ModelPrice{ + { + Model: model, + BaseModel: model, + Mode: "chat", + Prices: map[string]float64{ + cluster_conf.PriceInputCostPerToken: inputCost, + cluster_conf.PriceOutputCostPerToken: outputCost, + }, + }, + }, + } + // ModelTableCheck builds price index and converts float prices to fixed-point integers. + if err := cluster_conf.ModelTableCheck(modelTable); err != nil { + panic(fmt.Sprintf("ModelTableCheck failed: %v", err)) + } + + c := bfe_cluster.NewBfeCluster("test-cluster") + c.AIConf = &cluster_conf.AIConf{ + ModelTable: modelTable, + } + return c +} + +func TestTokenRequestFinishHandler_RMB_Streaming(t *testing.T) { + m := NewModuleAITokenAuth() + client := newMockRedisClient() + m.redisClient = client + + clusterName := "deepseek-backup" + model := "deepseek-v4-flash" + req := newTestRequestWithCluster("ak-123", "AI_product", clusterName, model) + + cluster := buildTestClusterConf(model, 0.000003, 0.000009) + req.SvrDataConf = &mockServerDataConf{clusters: map[string]*bfe_cluster.BfeCluster{clusterName: cluster}} + + rmbPlan := &QuotaPlan{ + Id: "rmb-plan", + RedisKey: "QUOTA_AI_product-ZEAoKAKdGnPpck1uPoUsdNCb", + Unit: "RMB", + Quota: 100000000, + } + SetTokenAuthContext(req, &Token{Key: "ak-123", KeyId: "ak-123-id", QuotaPlans: []*QuotaPlan{rmbPlan}}, 0, nil) + + // Simulate streaming: mod_body_process has filled token usage after context was set. + ai := req.GetAiBasicInfo() + usage := ai.GetTokenUsage() + usage.PromptTokens = 100 + usage.CompletionTokens = 200 + usage.UsedQuota = 300 + + // input_cost=0.000003 yuan/token, output_cost=0.000009 yuan/token + // Expected cost = 100*0.000003 + 200*0.000009 = 0.0021 yuan + // In fixed point: 0.0021 * 1e8 = 210000 + + // Streaming response: ContentLength = -1. + res := &bfe_http.Response{StatusCode: 200, ContentLength: -1} + if ret := m.tokenRequestFinishHandler(req, res); ret != bfe_module.BfeHandlerGoOn { + t.Fatalf("expected goon, got %d", ret) + } + + expectedCost := quota.RmbToFixedPoint(0.0021) + if usage.UsedCost != expectedCost { + t.Errorf("expected UsedCost %d, got %d", expectedCost, usage.UsedCost) + } + + remaining := client.data[rmbPlan.RedisKey] + if remaining != rmbPlan.Quota-expectedCost { + t.Errorf("expected remaining %d, got %d", rmbPlan.Quota-expectedCost, remaining) + } +} + +func TestTokenRequestFinishHandler_RMB_NonStreaming(t *testing.T) { + m := NewModuleAITokenAuth() + client := newMockRedisClient() + m.redisClient = client + + clusterName := "deepseek-backup" + model := "deepseek-v4-flash" + req := newTestRequestWithCluster("ak-123", "AI_product", clusterName, model) + + body := `{"usage":{"total_tokens":30,"prompt_tokens":10,"completion_tokens":20}}` + res := &bfe_http.Response{ + StatusCode: 200, + ContentLength: int64(len(body)), + Body: ioutil.NopCloser(strings.NewReader(body)), + } + + cluster := buildTestClusterConf(model, 0.000003, 0.000009) + req.SvrDataConf = &mockServerDataConf{clusters: map[string]*bfe_cluster.BfeCluster{clusterName: cluster}} + + rmbPlan := &QuotaPlan{ + Id: "rmb-plan", + RedisKey: "QUOTA_AI_product-NonStreaming", + Unit: "RMB", + Quota: 100000000, + } + SetTokenAuthContext(req, &Token{Key: "ak-123", KeyId: "ak-123-id", QuotaPlans: []*QuotaPlan{rmbPlan}}, 0, nil) + + // First pass through read-response handler to parse usage. + if ret := m.tokenReadResponseHandler(req, res); ret != bfe_module.BfeHandlerGoOn { + t.Fatalf("read response handler failed: %d", ret) + } + + // Then finish handler should calculate cost and deduct. + if ret := m.tokenRequestFinishHandler(req, res); ret != bfe_module.BfeHandlerGoOn { + t.Fatalf("expected goon, got %d", ret) + } + + // Expected cost = 10*0.000003 + 20*0.000009 = 0.00021 yuan + expectedCost := quota.RmbToFixedPoint(0.00021) + usage := req.GetAiBasicInfo().GetTokenUsage() + if usage.UsedCost != expectedCost { + t.Errorf("expected UsedCost %d, got %d", expectedCost, usage.UsedCost) + } + + remaining := client.data[rmbPlan.RedisKey] + if remaining != rmbPlan.Quota-expectedCost { + t.Errorf("expected remaining %d, got %d", rmbPlan.Quota-expectedCost, remaining) + } +} + func TestMonitorAndReloadHandlers(t *testing.T) { m := NewModuleAITokenAuth() mon := m.monitorHandlers() @@ -703,7 +894,9 @@ func TestQuotaPlanCheck(t *testing.T) { }{ {"missing id", QuotaPlan{Unlimited: true}, "no Id"}, {"invalid expired time", QuotaPlan{Id: "p1", Unlimited: true, ExpiredTime: -2}, "invalid ExpiredTime"}, - {"invalid quota", QuotaPlan{Id: "p1", Unlimited: false, Quota: 0}, "invalid Quota"}, + {"invalid token quota", QuotaPlan{Id: "p1", Unlimited: false, Quota: 0, Unit: "total_token"}, "invalid Quota"}, + {"invalid rmb quota", QuotaPlan{Id: "p1", Unlimited: false, Quota: -1, Unit: "RMB"}, "invalid Quota for RMB"}, + {"invalid unit", QuotaPlan{Id: "p1", Unlimited: true, Unit: "invalid"}, "invalid Unit"}, {"invalid reset mode", QuotaPlan{Id: "p1", Unlimited: true, ResetMode: 2}, "invalid ResetMode"}, } for _, tc := range cases { @@ -736,6 +929,7 @@ func TestSubnetValidation(t *testing.T) { s := "10.0.0.0/8, 192.168.1.0/24" tf := &TokenFile{ Key: "ak-subnet", + KeyId: "ak-subnet-id", Status: TokenStatusEnabled, ExpiredTime: -1, UnlimitedQuota: true, @@ -753,3 +947,153 @@ func TestSubnetValidation(t *testing.T) { t.Error("expected subnet to contain 10.0.0.0") } } + +// mockRedisClient is a simple in-memory redis client for unit tests. +type mockRedisClient struct { + data map[string]int64 +} + +func newMockRedisClient() *mockRedisClient { + return &mockRedisClient{data: make(map[string]int64)} +} + +func (m *mockRedisClient) Setex(key string, value []byte, expire int) error { + return nil +} + +func (m *mockRedisClient) Get(key string) (interface{}, error) { + if v, ok := m.data[key]; ok { + return v, nil + } + return nil, fmt.Errorf("key not found") +} + +func (m *mockRedisClient) Expire(key string, expire int) error { + return nil +} + +func (m *mockRedisClient) Incr(key string) (int64, error) { + m.data[key]++ + return m.data[key], nil +} + +func (m *mockRedisClient) IncrAndExpire(key string, expire int) (int64, error) { + return m.Incr(key) +} + +func (m *mockRedisClient) Decr(key string) (int64, error) { + m.data[key]-- + return m.data[key], nil +} + +func (m *mockRedisClient) PIncr(keys []string) ([]int64, error) { + return nil, nil +} + +func (m *mockRedisClient) GetInt64(key string) (int64, error) { + if v, ok := m.data[key]; ok { + return v, nil + } + return 0, fmt.Errorf("key not found") +} + +func (m *mockRedisClient) IncrBy(key string, delta int64) (int64, error) { + m.data[key] += delta + return m.data[key], nil +} + +func (m *mockRedisClient) NewScript(src string) redis_client.RedisScript { + return &mockRedisScript{client: m, src: src} +} + +type mockRedisScript struct { + client *mockRedisClient + src string +} + +func (s *mockRedisScript) Run(key string, args ...interface{}) (interface{}, error) { + isRMB := strings.Contains(s.src, "raw == false") + current := s.client.data[key] + amount, _ := args[0].(int64) + if isRMB { + if _, ok := s.client.data[key]; !ok { + initial, _ := args[1].(int64) + s.client.data[key] = initial + current = initial + } + } + deduct := current + if amount < current { + deduct = amount + } + if deduct > 0 { + s.client.data[key] = current - deduct + } + remaining := s.client.data[key] + if remaining < 0 { + remaining = 0 + s.client.data[key] = 0 + } + return remaining, nil +} + +func TestQuotaPlanDeduct(t *testing.T) { + t.Run("token deduct", func(t *testing.T) { + client := newMockRedisClient() + client.data["token-key"] = 100 + plan := &QuotaPlan{Id: "p1", RedisKey: "token-key", Unit: "total_token", Quota: 100} + remaining, err := plan.Deduct(client, 30) + if err != nil { + t.Fatalf("deduct failed: %v", err) + } + if remaining != 70 { + t.Errorf("remaining = %d, want 70", remaining) + } + if client.data["token-key"] != 70 { + t.Errorf("stored value = %d, want 70", client.data["token-key"]) + } + }) + + t.Run("rmb deduct", func(t *testing.T) { + client := newMockRedisClient() + plan := &QuotaPlan{Id: "p1", RedisKey: "rmb-key", Unit: "RMB", Quota: 1000} + remaining, err := plan.Deduct(client, 200) + if err != nil { + t.Fatalf("deduct failed: %v", err) + } + if remaining != 800 { + t.Errorf("remaining = %d, want 800", remaining) + } + if client.data["rmb-key"] != 800 { + t.Errorf("stored value = %d, want 800", client.data["rmb-key"]) + } + }) + + t.Run("rmb deduct insufficient", func(t *testing.T) { + client := newMockRedisClient() + client.data["rmb-key"] = 100 + plan := &QuotaPlan{Id: "p1", RedisKey: "rmb-key", Unit: "RMB", Quota: 100} + remaining, err := plan.Deduct(client, 200) + if err != nil { + t.Fatalf("deduct failed: %v", err) + } + if remaining != 0 { + t.Errorf("remaining = %d, want 0", remaining) + } + if client.data["rmb-key"] != 0 { + t.Errorf("stored value = %d, want 0", client.data["rmb-key"]) + } + }) + + t.Run("unlimited plan", func(t *testing.T) { + client := newMockRedisClient() + plan := &QuotaPlan{Id: "p1", RedisKey: "key", Unlimited: true, Unit: "RMB", Quota: 10000} + remaining, err := plan.Deduct(client, 200) + if err != nil { + t.Fatalf("deduct failed: %v", err) + } + if remaining != 10000 { + t.Errorf("remaining = %d, want 10000", remaining) + } + }) +} diff --git a/bfe_modules/mod_ai_token_auth/testdata/mod_ai_token_auth/token_rule.data b/bfe_modules/mod_ai_token_auth/testdata/mod_ai_token_auth/token_rule.data index 4b4034a62..e295a3c90 100644 --- a/bfe_modules/mod_ai_token_auth/testdata/mod_ai_token_auth/token_rule.data +++ b/bfe_modules/mod_ai_token_auth/testdata/mod_ai_token_auth/token_rule.data @@ -28,8 +28,8 @@ "AI_product": { "ak-123": { "key": "ak-123", + "key_id": "ak-123-id", "status": 1, - "name": "test-token", "update_time": 0, "expired_time": -1, "unlimited_quota": true, diff --git a/bfe_modules/mod_ai_token_auth/token.go b/bfe_modules/mod_ai_token_auth/token.go index 84c3fc96b..a139528bb 100644 --- a/bfe_modules/mod_ai_token_auth/token.go +++ b/bfe_modules/mod_ai_token_auth/token.go @@ -20,6 +20,8 @@ import ( "net" "strings" + "github.com/bfenetworks/go-lib/quota" + "github.com/bfenetworks/bfe/bfe_basic" "github.com/bfenetworks/bfe/bfe_util/redis_client" "github.com/google/uuid" @@ -38,8 +40,8 @@ const ( type Token struct { Key string + KeyId string Status int - Name string UpdateTime int64 ExpiredTime int64 UnlimitedQuota bool @@ -52,9 +54,9 @@ type Token struct { type TokenFile struct { Key string `json:"key"` + KeyId string `json:"key_id"` Enabled int `json:"enabled"` Status int `json:"status"` - Name string `json:"name"` UpdateTime int64 `json:"update_time"` ExpiredTime int64 `json:"expired_time"` // -1 means never expired UnlimitedQuota bool `json:"unlimited_quota"` @@ -74,9 +76,10 @@ type QuotaPlan struct { PassNoQuota bool RedisKey string CreateTime int64 - ExpiredTime int64 // -1 means never expired - Quota int64 // 配额总量 - ResetMode int // 0 – 非周期性；1 – 周期性的配额包 + ExpiredTime int64 // -1 means never expired + Quota int64 // 配额总量，固定点整数：total_token 时为 Token 数；RMB 时为 1e-8 元 + ResetMode int // 0 – 非周期性；1 – 周期性的配额包 + Unit string // "total_token" or "RMB" } func (q *QuotaPlan) Deduct(client redis_client.Client, amount int64) (int64, error) { @@ -92,6 +95,14 @@ func (q *QuotaPlan) Deduct(client redis_client.Client, amount int64) (int64, err return 0, errors.New("RedisKey is empty") } + if quota.IsRMB(q.Unit) { + return q.deductRMB(client, amount) + } + + return q.deductToken(client, amount) +} + +func (q *QuotaPlan) deductToken(client redis_client.Client, amount int64) (int64, error) { lua := ` local current = tonumber(redis.call('GET', KEYS[1]) or '0') local amount = tonumber(ARGV[1]) @@ -115,6 +126,37 @@ func (q *QuotaPlan) Deduct(client redis_client.Client, amount int64) (int64, err return remaining, nil } +func (q *QuotaPlan) deductRMB(client redis_client.Client, amount int64) (int64, error) { + lua := ` + local raw = redis.call('GET', KEYS[1]) + local current + if raw == false then + current = tonumber(ARGV[2]) + redis.call('SET', KEYS[1], current) + else + current = tonumber(raw) + end + local cost = tonumber(ARGV[1]) + local deduct = math.min(current, cost) + if deduct > 0 then + redis.call('DECRBY', KEYS[1], deduct) + end + return math.max(0, current - deduct) + ` + script := client.NewScript(lua) + result, err := script.Run(q.RedisKey, amount, q.Quota) + if err != nil { + return 0, err + } + + remaining, ok := result.(int64) + if !ok { + return 0, errors.New("invalid result type from redis") + } + + return remaining, nil +} + func (q *QuotaPlan) HasBalance(client redis_client.Client) (bool, int64, error) { if q.Unlimited { return true, q.Quota, nil @@ -136,6 +178,9 @@ func tokenCheck(conf *TokenFile) error { if conf.Key == "" { return errors.New("no Key") } + if conf.KeyId == "" { + return errors.New("no KeyId") + } if conf.Status < TokenStatusEnabled || conf.Status > TokenStatusExhausted { return fmt.Errorf("invalid Status: %d", conf.Status) } @@ -193,8 +238,8 @@ func tokenConvert(tokenFile TokenFile, quotaPlansMap *QuotaPlanMap) (Token, erro return Token{ Key: tokenFile.Key, + KeyId: tokenFile.KeyId, Status: tokenFile.Status, - Name: tokenFile.Name, UpdateTime: tokenFile.UpdateTime, ExpiredTime: tokenFile.ExpiredTime, UnlimitedQuota: tokenFile.UnlimitedQuota, diff --git a/bfe_modules/mod_ai_token_auth/token_rule_load.go b/bfe_modules/mod_ai_token_auth/token_rule_load.go index 6ef71e218..45f63c02a 100644 --- a/bfe_modules/mod_ai_token_auth/token_rule_load.go +++ b/bfe_modules/mod_ai_token_auth/token_rule_load.go @@ -20,6 +20,8 @@ import ( "fmt" "os" + "github.com/bfenetworks/go-lib/quota" + "github.com/bfenetworks/bfe/bfe_basic/condition" ) @@ -121,12 +123,30 @@ func quotaPlanCheck(conf *QuotaPlan) error { if conf.ExpiredTime < -1 { return fmt.Errorf("invalid ExpiredTime: %d", conf.ExpiredTime) } - if !conf.Unlimited && conf.Quota <= 0 { - return fmt.Errorf("invalid Quota: %d", conf.Quota) - } if conf.ResetMode < 0 || conf.ResetMode > 1 { return fmt.Errorf("invalid ResetMode: %d", conf.ResetMode) } + + // backward compatibility: empty unit means total_token + if conf.Unit == "" { + conf.Unit = quota.UnitTotalToken + } + if conf.Unit != quota.UnitTotalToken && conf.Unit != quota.UnitRMB { + return fmt.Errorf("invalid Unit: %s", conf.Unit) + } + + if !conf.Unlimited { + if conf.Unit == quota.UnitRMB { + if conf.Quota < 0 { + return fmt.Errorf("invalid Quota for RMB: %d", conf.Quota) + } + } else { + if conf.Quota <= 0 { + return fmt.Errorf("invalid Quota: %d", conf.Quota) + } + } + } + return nil } diff --git a/bfe_modules/mod_ai_token_auth/token_rule_table.go b/bfe_modules/mod_ai_token_auth/token_rule_table.go index 01ff83ae7..4a3df7d3c 100644 --- a/bfe_modules/mod_ai_token_auth/token_rule_table.go +++ b/bfe_modules/mod_ai_token_auth/token_rule_table.go @@ -80,16 +80,16 @@ func (t *TokenRuleTable) ValidateUserToken(product, key string) (token *Token, e switch token.Status { case TokenStatusExhausted: - return nil, fmt.Errorf("token %s quota exhausted", token.Name) + return nil, fmt.Errorf("token %s quota exhausted", token.KeyId) case TokenStatusExpired: - return nil, fmt.Errorf("token %s expired", token.Name) + return nil, fmt.Errorf("token %s expired", token.KeyId) case TokenStatusDisabled: - return nil, fmt.Errorf("token %s disabled", token.Name) + return nil, fmt.Errorf("token %s disabled", token.KeyId) } if token.ExpiredTime != -1 && token.ExpiredTime < time.Now().Unix() { token.Status = TokenStatusExpired - return nil, fmt.Errorf("token %s expired", token.Name) + return nil, fmt.Errorf("token %s expired", token.KeyId) } return token, nil @@ -125,24 +125,33 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke }) } + // record key_id into AiBasicInfo as early as possible so that access log + // can still identify the token even if the request is later rejected. + if aiBasicInfo := req.GetAiBasicInfo(); aiBasicInfo != nil { + aiBasicInfo.ClientKeyId = token.KeyId + } + switch token.Status { case TokenStatusExhausted: SetAiAuthInfo(req, bfe_basic.CodeInvalidApiKey, nil) return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeInvalidApiKey, bfe_basic.TypeAuthenticationError, fmt.Sprintf("Invalid API key: %s. quota exhausted.", key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, }) case TokenStatusExpired: SetAiAuthInfo(req, bfe_basic.CodeKeyExpired, nil) return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeKeyExpired, bfe_basic.TypeAuthenticationError, fmt.Sprintf("Invalid API key: %s. expired.", key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, }) case TokenStatusDisabled: SetAiAuthInfo(req, bfe_basic.CodeKeyDisabled, nil) return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeKeyDisabled, bfe_basic.TypeAuthenticationError, fmt.Sprintf("Invalid API key: %s. disabled.", key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, }) } @@ -152,6 +161,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeKeyExpired, bfe_basic.TypeAuthenticationError, fmt.Sprintf("Invalid API key: %s. expired.", key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, }) } @@ -167,6 +177,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeQuotaExpired, bfe_basic.TypeQuotaError, fmt.Sprintf("Quota plan %s expired.", plan.Id), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, QuotaPlanId: plan.Id, LimitType: bfe_basic.LimitTypeApiKeyQuota, }) @@ -177,6 +188,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeInternalQuotaError, bfe_basic.TypeInternalError, fmt.Sprintf("Internal error during quota deduction for plan %s: %v", plan.Id, err), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, QuotaPlanId: plan.Id, }) } @@ -185,10 +197,16 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeQuotaExhausted, bfe_basic.TypeQuotaError, fmt.Sprintf("Quota plan %s exhausted.", plan.Id), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, QuotaPlanId: plan.Id, LimitType: bfe_basic.LimitTypeApiKeyQuota, }) } + + // record quota plan that passed balance check + if aiBasicInfo := req.GetAiBasicInfo(); aiBasicInfo != nil { + aiBasicInfo.AiAuthInfo.HitQuotaPlans = append(aiBasicInfo.AiAuthInfo.HitQuotaPlans, plan.Id) + } } } @@ -199,6 +217,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeInvalidRequest, bfe_basic.TypeInvalidRequestError, fmt.Sprintf("Model not found in request body: %v", err), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, }) } model = strings.TrimSpace(model) @@ -209,6 +228,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeModelNotAllowed, bfe_basic.TypeInvalidRequestError, fmt.Sprintf("Model %s blocked by key %s", model, key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, Model: model, }) } @@ -228,6 +248,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeModelNotAllowed, bfe_basic.TypeInvalidRequestError, fmt.Sprintf("Model %s not allowed by key %s", model, key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, Model: model, }) } @@ -250,6 +271,7 @@ func (m *ModuleAITokenAuth) ValidateUserTokenByReq(req *bfe_basic.Request) (toke return nil, bfe_basic.NewAiErrorWithDetails(bfe_basic.CodeSubnetNotAllowed, bfe_basic.TypeAuthenticationError, fmt.Sprintf("Client IP not in subnet of key %s", key), &bfe_basic.AiErrorDetail{ ApiKey: key, + KeyId: token.KeyId, }) } } diff --git a/bfe_server/bfe_server.go b/bfe_server/bfe_server.go index 93d516ba7..23f5e80f0 100644 --- a/bfe_server/bfe_server.go +++ b/bfe_server/bfe_server.go @@ -25,7 +25,6 @@ import ( "syscall" "time" - "github.com/bfenetworks/go-lib/log" "github.com/bfenetworks/bfe/bfe_balance" "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" "github.com/bfenetworks/bfe/bfe_config/bfe_conf" @@ -40,6 +39,7 @@ import ( "github.com/bfenetworks/bfe/bfe_tls" "github.com/bfenetworks/bfe/bfe_util/signal_table" "github.com/bfenetworks/bfe/bfe_websocket" + "github.com/bfenetworks/go-lib/log" ) type BfeServer struct { @@ -160,6 +160,12 @@ func (srv *BfeServer) InitConfig() { } else { srv.MaxHeaderUriBytes = bfe_http.DefaultMaxHeaderUriBytes } + + // set AccessibleBodySize + bfe_http.SetAccessibleBodySize(srv.Config.Server.AccessibleBodySize) + + // set TotalBodyBufferSize limit + bfe_http.SetTotalBodyBufferSizeLimit(srv.Config.Server.TotalBodyBufferSize) } func (srv *BfeServer) InitHttp() (err error) { diff --git a/bfe_server/reverseproxy.go b/bfe_server/reverseproxy.go index a5519f932..11e64ce6b 100644 --- a/bfe_server/reverseproxy.go +++ b/bfe_server/reverseproxy.go @@ -848,35 +848,6 @@ func (p *ReverseProxy) ServeHTTP(rw bfe_http.ResponseWriter, basicReq *bfe_basic outreq.Host = "" } - if cluster.AIConf != nil { - aiMeta := basicReq.GetAiBasicInfo() - if aiMeta != nil { - // if cluster has AIConf, do model mapping & set api key in outreq - if cluster.AIConf.Key != nil { - mod_ai_token_auth.SetApiKey(outreq, *cluster.AIConf.Key) - } - if cluster.AIConf.ModelMapping != nil { - model := aiMeta.ClientModel - if model != "" { - newModel, ok := (*cluster.AIConf.ModelMapping)[model] - if ok { - err = condition.ReqBodyJsonSet(basicReq, "model", newModel) - if err != nil { - log.Logger.Warn("Failed to set model in request body: %s", err) - // just continue, not return error - } else { - // outreq body already changed, need reset Content-Length - if outreq.ContentLength >= 0 { - outreq.ContentLength = -1 - outreq.Header.Del("Content-Length") - } - aiMeta.TargetModel = newModel - } - } - } - } - } - } /* // do body process before forwarding bf, ok = outreq.Body.(BufferFiller) @@ -1297,15 +1268,26 @@ func (p *ReverseProxy) ServeHTTPForAI(rw bfe_http.ResponseWriter, basicReq *bfe_ }) } + // ensure request body is rewindable before attempting fallbacks + if len(attempts) > 1 && basicReq.HttpRequest.Body != nil { + if !prepareRequestBodyForRetry(basicReq.HttpRequest) { + log.Logger.Warn("ServeHTTPForAI: request body is not rewindable, disable fallback") + attempts = attempts[:1] + } + } + for i, attempt := range attempts { if i > 0 { // fallback attempt: reset request state - p.resetRequestForRetry(basicReq) + if !p.resetRequestForRetry(basicReq) { + log.Logger.Warn("ServeHTTPForAI: fallback aborted, request body cannot be rewound") + break + } } res, action, lastCluster, invokeErr = p.aiClusterInvoke(srv, serverConf, basicReq, rw, attempt, aiMeta) - if invokeErr == nil && res != nil && res.StatusCode < 500 { - // success or 4xx (client error, do not fallback) + if invokeErr == nil && res != nil && res.StatusCode < 400 { + // success: 2xx/3xx, stop fallback loop break } @@ -1319,9 +1301,9 @@ func (p *ReverseProxy) ServeHTTPForAI(rw bfe_http.ResponseWriter, basicReq *bfe_ } // log fallback - log.Logger.Info("mod_ai_route: fallback triggered, cluster[%s] err[%v] status[%d]", + log.Logger.Info("ServeHTTPForAI: fallback triggered, cluster[%s] err[%v] status[%d]", attempt.ClusterName, invokeErr, getResponseStatus(res)) - + if res != nil { res.Body.Close() } @@ -1442,6 +1424,112 @@ send_response: return } +// stripProviderPrefix strips the configured provider/model prefix from model. +// It returns the stripped model and true when stripping succeeds; otherwise it +// returns the original model and false. +func stripProviderPrefix(model string, matchPrefix string) (string, bool) { + if model == "" || !strings.HasPrefix(model, matchPrefix) { + return model, false + } + + stripped := strings.TrimPrefix(model, matchPrefix) + if stripped == "" { + log.Logger.Warn("Model %s stripped by prefix %s results in empty model, skip stripping", + model, matchPrefix) + return model, false + } + + return stripped, true +} + +// doSingleAIForward performs a single AI forward attempt with the given key. +func (p *ReverseProxy) doSingleAIForward(srv *BfeServer, cluster *bfe_cluster.BfeCluster, + basicReq *bfe_basic.Request, rw bfe_http.ResponseWriter, + attempt aiForwardAttempt, aiMeta *bfe_basic.AiBasicInfo, + selectedKey cluster_conf.AIKey) ( + res *bfe_http.Response, action int, err error) { + + req := basicReq.HttpRequest + + // prepare out request to downstream RS backend + outreq := new(bfe_http.Request) + *outreq = *req // includes shallow copies of maps, but okay + basicReq.OutRequest = outreq + + // set http proto for out request + httpProtoSet(outreq) + // remove hop-by-hop headers + hopByHopHeaderRemove(outreq, req) + + if cluster.DisableHostHeader { + // if cluster.DisableHostHeader is true, del outreq.Host + outreq.Host = "" + } + + // Calculate the final model in order: route target/fallback override -> + // provider/model prefix stripping -> cluster model mapping. Then write it + // to the request body at most once to avoid repeated JSON parsing/serialization. + model := aiMeta.ClientModel + if aiMeta.TargetModel != "" { + model = aiMeta.TargetModel + } + + // apply model override from ai route target/fallback + if attempt.Model != "" { + model = attempt.Model + } + + // strip provider/model prefix according to cluster AIConf + if cluster.AIConf != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + if stripped, ok := stripProviderPrefix(model, cluster.AIConf.MatchPrefix); ok { + model = stripped + } + } + + // apply cluster model mapping + if cluster.AIConf != nil && cluster.AIConf.ModelMapping != nil && model != "" { + if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { + model = newModel + } + } + + if model != aiMeta.ClientModel { + if err := condition.ReqBodyJsonSet(basicReq, "model", model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + // outreq body already changed, need reset Content-Length + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + // Also reset the original request's Content-Length so that fallback/retry + // creates a new outreq with consistent body length. + if basicReq.HttpRequest != nil && basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + aiMeta.TargetModel = model + } + } + + // apply cluster.AIConf (api key, provider, cost currency) + if cluster.AIConf != nil { + if cluster.AIConf.Provider != "" { + aiMeta.Provider = cluster.AIConf.Provider + } + if cluster.AIConf.ModelTable != nil && cluster.AIConf.ModelTable.Currency != "" { + aiMeta.CostCurrency = cluster.AIConf.ModelTable.Currency + } + aiMeta.AppendClusterKeyName(cluster.Name, selectedKey.Name) + if selectedKey.Key != "" { + mod_ai_token_auth.SetApiKey(outreq, selectedKey.Key) + } + } + + // invoke cluster to get response + return p.clusterInvoke(srv, cluster, basicReq, rw) +} + func (p *ReverseProxy) aiClusterInvoke(srv *BfeServer, serverConf *bfe_route.ServerDataConf, basicReq *bfe_basic.Request, rw bfe_http.ResponseWriter, attempt aiForwardAttempt, aiMeta *bfe_basic.AiBasicInfo) ( @@ -1466,79 +1554,131 @@ func (p *ReverseProxy) aiClusterInvoke(srv *BfeServer, serverConf *bfe_route.Ser // set deadline to finish read client request body timeoutReadClient := cluster.TimeoutReadClient() - + if basicReq.IsSse { timeoutReadClient = -1 } p.setTimeout(bfe_basic.StageReadReqBody, basicReq.Connection, req, timeoutReadClient) - // prepare out request to downstream RS backend - outreq := new(bfe_http.Request) - *outreq = *req // includes shallow copies of maps, but okay - basicReq.OutRequest = outreq - - // set http proto for out request - httpProtoSet(outreq) - // remove hop-by-hop headers - hopByHopHeaderRemove(outreq, req) + // no api keys configured, skip key injection + if cluster.AIConf == nil || len(cluster.AIConf.Keys) == 0 { + res, action, err = p.doSingleAIForward(srv, cluster, basicReq, rw, attempt, aiMeta, cluster_conf.AIKey{}) + return res, action, cluster, err + } - if cluster.DisableHostHeader { - // if cluster.DisableHostHeader is true, del outreq.Host - outreq.Host = "" + policy := defaultAIKeyPolicy() + if cluster.AIConf.KeyPolicy != nil { + policy = *cluster.AIConf.KeyPolicy } - // apply model override from ai route target/fallback - if attempt.Model != "" && aiMeta != nil { - if err := condition.ReqBodyJsonSet(basicReq, "model", attempt.Model); err != nil { - log.Logger.Warn("Failed to set model in request body: %s", err) - } else { - // outreq body already changed, need reset Content-Length - if outreq.ContentLength >= 0 { - outreq.ContentLength = -1 - outreq.Header.Del("Content-Length") - } - aiMeta.TargetModel = attempt.Model + keys := cluster.AIConf.Keys + + // ensure request body is rewindable when key-level retry is possible + keyRetryEnabled := policy.MaxRetries > 0 + if keyRetryEnabled { + if !prepareRequestBodyForRetry(basicReq.HttpRequest) { + log.Logger.Warn("aiClusterInvoke: request body is not rewindable, disable key-level retry for cluster[%s]", + attempt.ClusterName) + keyRetryEnabled = false + policy.MaxRetries = 0 } } - // apply cluster.AIConf (api key, model mapping) - if cluster.AIConf != nil && aiMeta != nil { - if cluster.AIConf.Key != nil { - mod_ai_token_auth.SetApiKey(outreq, *cluster.AIConf.Key) - } - if cluster.AIConf.ModelMapping != nil { - model := aiMeta.ClientModel - if aiMeta.TargetModel != "" { - model = aiMeta.TargetModel + state := newAIKeyAttemptState() + + var lastErr error + var idx int + var key cluster_conf.AIKey + var ok bool + keepKey := false + for retry := 0; retry <= policy.MaxRetries; retry++ { + if retry > 0 { + if aiMeta != nil { + aiMeta.IncrementRetryCount() + } + // rewind body before retrying with another key + if !rewindRequestBody(basicReq.HttpRequest) { + log.Logger.Warn("aiClusterInvoke: failed to rewind request body, abort key-level retry for cluster[%s]", + attempt.ClusterName) + break } - if model != "" { - if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { - if err := condition.ReqBodyJsonSet(basicReq, "model", newModel); err != nil { - log.Logger.Warn("Failed to set model in request body: %s", err) - } else { - // outreq body already changed, need reset Content-Length - if outreq.ContentLength >= 0 { - outreq.ContentLength = -1 - outreq.Header.Del("Content-Length") - } - aiMeta.TargetModel = newModel - } - } + backoff := calcBackoff(policy.RetryBackoffInitial, policy.RetryBackoffMax, retry) + time.Sleep(backoff) + } + + if !keepKey { + idx, key, ok = chooseNextAIKey(keys, state) + if !ok { + log.Logger.Warn("aiClusterInvoke: all ai keys exhausted for cluster[%s]", attempt.ClusterName) + break } + + log.Logger.Info("aiClusterInvoke: select ai key [name=%s weight=%d] for cluster[%s]", + key.Name, key.Weight, attempt.ClusterName) + } + keepKey = false + + res, action, err = p.doSingleAIForward(srv, cluster, basicReq, rw, attempt, aiMeta, key) + + lastErr = err + statusCode := 0 + if res != nil { + statusCode = res.StatusCode + } + + // success: stop key-level retry + if err == nil && statusCode < 400 { + return res, action, cluster, nil + } + + // classify failure + switch { + case statusCode == 429: + // rate limit: mark key as used and rotate to another key + state.usedSet[idx] = struct{}{} + log.Logger.Info("aiClusterInvoke: ai key [name=%s] rate limited (429), rotate", key.Name) + case statusCode == 401 || statusCode == 402 || statusCode == 403: + // auth failure: mark key as dead + state.deadSet[idx] = struct{}{} + log.Logger.Info("aiClusterInvoke: ai key [name=%s] auth failed (%d), dead", key.Name, statusCode) + case statusCode >= 500 || err != nil: + // transient server failure or connection error: + // keep current key selected for next retry (with backoff) + keepKey = true + log.Logger.Info("aiClusterInvoke: ai key [name=%s] transient failure [status=%d err=%v], retry same key", + key.Name, statusCode, err) + default: + // other 4xx client errors (e.g. 400, 404): stop key-level retry + return res, action, cluster, nil } } - // invoke cluster to get response - res, action, err = p.clusterInvoke(srv, cluster, basicReq, rw) - return res, action, cluster, err + return res, action, cluster, lastErr +} + +// aiFallbackStatusCodes defines the 4xx status codes that should trigger +// cluster-level fallback by default. 5xx is handled uniformly by code >= 500. +// This matches DeepSeek issue #1317 requirements (400/401/402/422/429) and +// aligns with Bifrost's built-in status code classification. +var aiFallbackStatusCodes = map[int]struct{}{ + 400: {}, + 401: {}, + 402: {}, + 403: {}, + 422: {}, + 429: {}, } func shouldTriggerFallback(res *bfe_http.Response, err error) bool { if err != nil { return true } - if res != nil && res.StatusCode >= 500 { + code := getResponseStatus(res) + if code >= 500 { + return true + } + if _, ok := aiFallbackStatusCodes[code]; ok { return true } return false @@ -1551,7 +1691,7 @@ func getResponseStatus(res *bfe_http.Response) int { return res.StatusCode } -func (p *ReverseProxy) resetRequestForRetry(basicReq *bfe_basic.Request) { +func (p *ReverseProxy) resetRequestForRetry(basicReq *bfe_basic.Request) bool { // desc backend connection counter if basicReq.Trans.Backend != nil { basicReq.Trans.Backend.DecConnNum() @@ -1563,7 +1703,175 @@ func (p *ReverseProxy) resetRequestForRetry(basicReq *bfe_basic.Request) { // reset out request so body can be re-read basicReq.OutRequest = nil + // rewind request body for next fallback attempt + if !rewindRequestBody(basicReq.HttpRequest) { + return false + } + + // reset Content-Length so that the next outreq is created with a length + // consistent with the current (possibly modified) body. + if basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + // clear error info from previous attempt basicReq.ErrCode = nil basicReq.ErrMsg = "" + return true +} + +// prepareRequestBodyForRetry makes the request body rewindable for fallback. +// If the body already implements Rewindable, it returns true directly. +// Otherwise, it tries to convert the body to bytes_body via GetBodyAccessor. +// It rejects wrapping when the total bytes_body buffer size reaches the limit. +func prepareRequestBodyForRetry(req *bfe_http.Request) bool { + // if total buffer size already reaches the limit, do not wrap (no retry) + if limit := bfe_http.TotalBodyBufferSizeLimit(); limit > 0 { + if bfe_http.TotalBytesBodyBuffer() >= limit { + return false + } + } + if req.Body == nil { + return true + } + if _, ok := req.Body.(bfe_http.Rewindable); ok { + return true + } + if _, err := req.GetBodyAccessor(); err != nil { + return false + } + _, ok := req.Body.(bfe_http.Rewindable) + return ok +} + +// rewindRequestBody rewinds the request body to the beginning. +// It assumes the body already implements Rewindable. +func rewindRequestBody(req *bfe_http.Request) bool { + if req.Body == nil { + return true + } + rewindable, ok := req.Body.(bfe_http.Rewindable) + if !ok { + return false + } + return rewindable.Rewind() +} + +// aiKeyAttemptState tracks key usage within one aiClusterInvoke call. +type aiKeyAttemptState struct { + usedSet map[int]struct{} // index of keys used for 429 in this request + deadSet map[int]struct{} // index of keys dead for 401/402/403 in this request +} + +func newAIKeyAttemptState() *aiKeyAttemptState { + return &aiKeyAttemptState{ + usedSet: make(map[int]struct{}), + deadSet: make(map[int]struct{}), + } +} + +// aiKeyRand is used for weighted random AI key selection. +var aiKeyRand = rand.New(rand.NewSource(time.Now().UnixNano())) + +// selectAIKey selects one key by weighted random. +// keys should have weight > 0 and total weight > 0. +func selectAIKey(keys []cluster_conf.AIKey) (cluster_conf.AIKey, int) { + if len(keys) == 1 { + return keys[0], 0 + } + + total := 0 + for _, k := range keys { + total += k.Weight + } + if total <= 0 { + return cluster_conf.AIKey{}, -1 + } + + r := aiKeyRand.Intn(total) + sum := 0 + for i, k := range keys { + sum += k.Weight + if r < sum { + return k, i + } + } + return keys[len(keys)-1], len(keys) - 1 +} + +// chooseNextAIKey returns next eligible key and its index. +// If all keys are dead, returns (-1, empty key, false). +// If all alive keys are in used_set, clears used_set and retries. +func chooseNextAIKey(keys []cluster_conf.AIKey, state *aiKeyAttemptState) (int, cluster_conf.AIKey, bool) { + var eligible []cluster_conf.AIKey + var indices []int + + for i, k := range keys { + if k.Weight == 0 { + continue + } + if _, dead := state.deadSet[i]; dead { + continue + } + eligible = append(eligible, k) + indices = append(indices, i) + } + + if len(eligible) == 0 { + return -1, cluster_conf.AIKey{}, false + } + + // filter out used_set keys + var filteredKeys []cluster_conf.AIKey + var filteredIdx []int + for j, k := range eligible { + idx := indices[j] + if _, used := state.usedSet[idx]; used { + continue + } + filteredKeys = append(filteredKeys, k) + filteredIdx = append(filteredIdx, idx) + } + + if len(filteredKeys) == 0 { + // all alive keys used (429 only), reset used_set and try again + state.usedSet = make(map[int]struct{}) + filteredKeys = eligible + filteredIdx = indices + } + + _, selectedIdx := selectAIKey(filteredKeys) + if selectedIdx < 0 { + return -1, cluster_conf.AIKey{}, false + } + return filteredIdx[selectedIdx], filteredKeys[selectedIdx], true +} + +// calcBackoff calculates exponential backoff with jitter. +func calcBackoff(initial, max, attempt int) time.Duration { + backoff := initial + for i := 1; i < attempt; i++ { + backoff *= 2 + if backoff > max { + backoff = max + break + } + } + // add jitter (±20%) + jitter := backoff / 5 + if jitter > 0 { + backoff = backoff - jitter/2 + aiKeyRand.Intn(jitter) + } + return time.Duration(backoff) * time.Millisecond +} + +// defaultAIKeyPolicy returns the default key policy. +func defaultAIKeyPolicy() cluster_conf.AIKeyPolicy { + return cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 0, + RetryBackoffInitial: 500, + RetryBackoffMax: 5000, + } } diff --git a/bfe_server/reverseproxy_ai_test.go b/bfe_server/reverseproxy_ai_test.go index 3e6ca23d6..12ce066d8 100644 --- a/bfe_server/reverseproxy_ai_test.go +++ b/bfe_server/reverseproxy_ai_test.go @@ -16,8 +16,10 @@ package bfe_server import ( "testing" + "time" "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" "github.com/bfenetworks/bfe/bfe_http" ) @@ -71,20 +73,45 @@ func TestShouldTriggerFallback(t *testing.T) { t.Error("expected fallback on connect error") } + // 5xx always triggers fallback res := &bfe_http.Response{StatusCode: 500} if !shouldTriggerFallback(res, nil) { t.Error("expected fallback on 5xx") } - res = &bfe_http.Response{StatusCode: 404} - if shouldTriggerFallback(res, nil) { - t.Error("expected no fallback on 4xx") + res = &bfe_http.Response{StatusCode: 503} + if !shouldTriggerFallback(res, nil) { + t.Error("expected fallback on 503") } + // 2xx/3xx do not trigger fallback res = &bfe_http.Response{StatusCode: 200} if shouldTriggerFallback(res, nil) { t.Error("expected no fallback on 2xx") } + + res = &bfe_http.Response{StatusCode: 302} + if shouldTriggerFallback(res, nil) { + t.Error("expected no fallback on 3xx") + } + + // Specific 4xx (aligned with issue #1317 and Bifrost classification) + fallback4xx := []int{400, 401, 402, 403, 422, 429} + for _, code := range fallback4xx { + res = &bfe_http.Response{StatusCode: code} + if !shouldTriggerFallback(res, nil) { + t.Errorf("expected fallback on %d", code) + } + } + + // Other 4xx should not trigger fallback + nonFallback4xx := []int{404, 405, 406, 408, 409, 410, 413} + for _, code := range nonFallback4xx { + res = &bfe_http.Response{StatusCode: code} + if shouldTriggerFallback(res, nil) { + t.Errorf("expected no fallback on %d", code) + } + } } func TestGetResponseStatus(t *testing.T) { @@ -97,3 +124,158 @@ func TestGetResponseStatus(t *testing.T) { t.Errorf("expected 200, got %d", getResponseStatus(res)) } } + +func TestSelectAIKeyDistribution(t *testing.T) { + keys := []cluster_conf.AIKey{ + {Name: "key-a", Key: "ak-a", Weight: 70}, + {Name: "key-b", Key: "ak-b", Weight: 30}, + } + + counts := make(map[string]int) + for i := 0; i < 1000; i++ { + key, _ := selectAIKey(keys) + counts[key.Name]++ + } + + if counts["key-a"] == 0 || counts["key-b"] == 0 { + t.Errorf("expected both keys to be selected, got %v", counts) + } + + if counts["key-a"] < counts["key-b"] { + t.Errorf("expected key-a selected more often than key-b, got %v", counts) + } +} + +func TestSelectAIKeyZeroWeightNotSelected(t *testing.T) { + keys := []cluster_conf.AIKey{ + {Name: "key-a", Key: "ak-a", Weight: 100}, + {Name: "key-b", Key: "ak-b", Weight: 0}, + } + + for i := 0; i < 100; i++ { + key, _ := selectAIKey(keys) + if key.Name == "key-b" { + t.Error("key-b has zero weight and should not be selected") + } + } +} + +func TestChooseNextAIKeyRotateOn429(t *testing.T) { + keys := []cluster_conf.AIKey{ + {Name: "key-a", Key: "ak-a", Weight: 50}, + {Name: "key-b", Key: "ak-b", Weight: 50}, + } + + state := newAIKeyAttemptState() + + // first selection + idx1, key1, ok := chooseNextAIKey(keys, state) + if !ok { + t.Fatal("expected to select a key") + } + + // mark first key as 429 used + state.usedSet[idx1] = struct{}{} + + // second selection should choose the other key + idx2, key2, ok := chooseNextAIKey(keys, state) + if !ok { + t.Fatal("expected to select a key") + } + if idx2 == idx1 { + t.Errorf("expected different key after 429, got same index %d", idx2) + } + if key2.Name == key1.Name { + t.Errorf("expected different key name after 429, got %s", key2.Name) + } + + // mark second key as 429 used, no eligible keys left + state.usedSet[idx2] = struct{}{} + _, _, ok = chooseNextAIKey(keys, state) + if !ok { + t.Error("expected reset used_set and reselect when all alive keys used") + } +} + +func TestChooseNextAIKeyDeadOn403(t *testing.T) { + keys := []cluster_conf.AIKey{ + {Name: "key-a", Key: "ak-a", Weight: 100}, + } + + state := newAIKeyAttemptState() + state.deadSet[0] = struct{}{} + + _, _, ok := chooseNextAIKey(keys, state) + if ok { + t.Error("expected no key available when the only key is dead") + } +} + +func TestCalcBackoff(t *testing.T) { + // attempt 1: initial value (with ±20% jitter) + b1 := calcBackoff(100, 1000, 1) + if b1 < time.Duration(90)*time.Millisecond || b1 > time.Duration(110)*time.Millisecond { + t.Errorf("expected backoff around 100ms, got %v", b1) + } + + // attempt 2: doubled (with ±20% jitter) + b2 := calcBackoff(100, 1000, 2) + if b2 < time.Duration(180)*time.Millisecond || b2 > time.Duration(220)*time.Millisecond { + t.Errorf("expected backoff around 200ms, got %v", b2) + } + + // attempt 5: capped at max (with ±20% jitter) + b5 := calcBackoff(100, 500, 5) + if b5 < time.Duration(450)*time.Millisecond || b5 > time.Duration(550)*time.Millisecond { + t.Errorf("expected backoff capped around 500ms, got %v", b5) + } +} + +func TestDefaultAIKeyPolicy(t *testing.T) { + policy := defaultAIKeyPolicy() + if policy.Strategy != "weighted_random" { + t.Errorf("expected strategy weighted_random, got %s", policy.Strategy) + } + if policy.MaxRetries != 0 { + t.Errorf("expected max_retries 0, got %d", policy.MaxRetries) + } + if policy.RetryBackoffInitial != 500 { + t.Errorf("expected retry_backoff_initial 500, got %d", policy.RetryBackoffInitial) + } + if policy.RetryBackoffMax != 5000 { + t.Errorf("expected retry_backoff_max 5000, got %d", policy.RetryBackoffMax) + } +} + +func TestStripProviderPrefix(t *testing.T) { + model := "openrouter/anthropic/claude-sonnet-4.6" + stripped, ok := stripProviderPrefix(model, "openrouter/") + if !ok { + t.Error("expected stripping to succeed") + } + if stripped != "anthropic/claude-sonnet-4.6" { + t.Errorf("expected stripped model anthropic/claude-sonnet-4.6, got %s", stripped) + } +} + +func TestStripProviderPrefixNoMatch(t *testing.T) { + model := "anthropic/claude-sonnet-4.6" + stripped, ok := stripProviderPrefix(model, "openrouter/") + if ok { + t.Error("expected stripping to be skipped when prefix does not match") + } + if stripped != model { + t.Errorf("expected model unchanged, got %s", stripped) + } +} + +func TestStripProviderPrefixEmptyResult(t *testing.T) { + model := "openrouter/" + stripped, ok := stripProviderPrefix(model, "openrouter/") + if ok { + t.Error("expected stripping to be skipped when result is empty") + } + if stripped != model { + t.Errorf("expected model unchanged, got %s", stripped) + } +} diff --git a/bfe_server/server_status.go b/bfe_server/server_status.go index bbcda516a..063202df8 100644 --- a/bfe_server/server_status.go +++ b/bfe_server/server_status.go @@ -19,9 +19,7 @@ package bfe_server import ( "github.com/bfenetworks/go-lib/web-monitor/delay_counter" "github.com/bfenetworks/go-lib/web-monitor/metrics" -) -import ( bal "github.com/bfenetworks/bfe/bfe_balance/bal_gslb" "github.com/bfenetworks/bfe/bfe_http" "github.com/bfenetworks/bfe/bfe_http2" @@ -30,6 +28,7 @@ import ( "github.com/bfenetworks/bfe/bfe_spdy" "github.com/bfenetworks/bfe/bfe_stream" "github.com/bfenetworks/bfe/bfe_tls" + "github.com/bfenetworks/bfe/bfe_util/json" "github.com/bfenetworks/bfe/bfe_websocket" ) @@ -197,6 +196,13 @@ func (srv *BfeServer) httpStateGetDiff(params map[string][]string) ([]byte, erro return s.Format(params) } +func (srv *BfeServer) serverStatGet(params map[string][]string) ([]byte, error) { + output := map[string]int64{ + "total_bytes_body_buffer": bfe_http.TotalBytesBodyBuffer(), + } + return json.Marshal(output) +} + func (srv *BfeServer) streamStateGetAll(params map[string][]string) ([]byte, error) { s := srv.serverStatus.StreamMetrics.GetAll() return s.Format(params) diff --git a/bfe_server/web_server.go b/bfe_server/web_server.go index f623aadb6..e371517fb 100644 --- a/bfe_server/web_server.go +++ b/bfe_server/web_server.go @@ -111,6 +111,9 @@ func (m *BfeMonitor) monitorHandlers() map[string]interface{} { "module_status": m.srv.ModuleStatusGetJSON, "module_handlers": m.srv.ModuleHandlersGetJSON, + // for server stat + "server_stat": m.srv.serverStatGet, + // for proxy memory stat "proxy_mem_stat": web_monitor.CreateMemStatsHandler("proxy_mem_stat"), } diff --git a/conf/bfe.conf b/conf/bfe.conf index d8673c74d..1d399ca97 100644 --- a/conf/bfe.conf +++ b/conf/bfe.conf @@ -31,6 +31,12 @@ EnableAiGateway = false # if true, will estimate token usage if usage is not available in response EstimateToken = false +# max size in bytes to buffer request body for rewriting/fallback (default 2MB, max 8MB) +AccessibleBodySize = 2097152 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +TotalBodyBufferSize = 0 + # timeout for graceful shutdown (maximum 300 sec) GracefulShutdownTimeout = 10 diff --git a/conf/mod_ai_token_auth/token_rule.data b/conf/mod_ai_token_auth/token_rule.data index 9dd018567..395479236 100644 --- a/conf/mod_ai_token_auth/token_rule.data +++ b/conf/mod_ai_token_auth/token_rule.data @@ -1,23 +1,23 @@ -{ - "Config": { - "example_product" :[ - ] - }, - "Tokens": { - "example_product": { - "TESTKEY": { - "key": "TESTKEY", - "status": 1, - "name": "test", - "expired_time": -1, - "unlimited_quota": true - } - } - }, - "QuotaPlans": { - "example_product" :[ - ] - }, - "Version": "0" -} - +{ + "Config": { + "example_product" :[ + ] + }, + "Tokens": { + "example_product": { + "TESTKEY": { + "key": "TESTKEY", + "key_id": "TESTKEY_ID", + "status": 1, + "expired_time": -1, + "unlimited_quota": true + } + } + }, + "QuotaPlans": { + "example_product" :[ + ] + }, + "Version": "0" +} + diff --git a/docs/en_us/condition/condition_primitive_index.md b/docs/en_us/condition/condition_primitive_index.md index 164c13a92..6029dc105 100644 --- a/docs/en_us/condition/condition_primitive_index.md +++ b/docs/en_us/condition/condition_primitive_index.md @@ -89,6 +89,7 @@ ### body * [req_body_json_in(json_path, value_list, case_insensitive)](./request/body.md#req_body_json_injson_path-value_list-case_insensitive) + * [req_body_json_prefix_in(json_path, value_prefix_list, case_insensitive)](./request/body.md#req_body_json_prefix_injson_path-value_prefix_list-case_insensitive) ## Response Primitive diff --git a/docs/en_us/condition/request/body.md b/docs/en_us/condition/request/body.md index 53f8970b2..5afeaf1aa 100644 --- a/docs/en_us/condition/request/body.md +++ b/docs/en_us/condition/request/body.md @@ -1,18 +1,42 @@ -# Condition Primitives Related to Request Body - -## req_body_json_in(json_path, value_list, case_insensitive) - -* Meaning: Searches for the field specified by `json_path` in the JSON-formatted request body and checks if its value exactly matches any in `value_list`. -* Parameters - -| Parameter | Description | -| ---------------- | ---------------------------------------------- | -| json_path | String
The path to the JSON field in the request body | -| value_list | String
List of values, separated by ‘|’ | -| case_insensitive | Boolean
Whether to ignore case sensitivity | - -* Example - -```go -req_body_json_in("model", "deepseek-r1|qwen-plus", true) -``` +# Condition Primitives Related to Request Body + +## req_body_json_in(json_path, value_list, case_insensitive) + +* Meaning: Searches for the field specified by `json_path` in the JSON-formatted request body and checks if its value exactly matches any in `value_list`. +* Parameters + +| Parameter | Description | +| ---------------- | ---------------------------------------------- | +| json_path | String
The path to the JSON field in the request body | +| value_list | String
List of values, separated by ‘|’ | +| case_insensitive | Boolean
Whether to ignore case sensitivity | + +* Example + +```go +req_body_json_in("model", "deepseek-r1|qwen-plus", true) +``` + +## req_body_json_prefix_in(json_path, value_prefix_list, case_insensitive) + +* Meaning: Searches for the field specified by `json_path` in the JSON-formatted request body and checks if its string value starts with any prefix in `value_prefix_list`. +* Parameters + +| Parameter | Description | +| ---------------- | ---------------------------------------------- | +| json_path | String
The path to the JSON field in the request body | +| value_prefix_list | String
List of prefixes, separated by ‘|’ | +| case_insensitive | Boolean
Whether to ignore case sensitivity | + +* Example + +```go +// Match all OpenRouter models +req_body_json_prefix_in("model", "openrouter/", false) + +// Match all models under OpenRouter/anthropic namespace +req_body_json_prefix_in("model", "openrouter/anthropic/", false) + +// Match all models starting with gpt- or claude- (case-insensitive) +req_body_json_prefix_in("model", "gpt-|claude-", true) +``` diff --git a/docs/en_us/configuration/bfe.conf.md b/docs/en_us/configuration/bfe.conf.md index cd8f221d0..17495cd77 100644 --- a/docs/en_us/configuration/bfe.conf.md +++ b/docs/en_us/configuration/bfe.conf.md @@ -30,6 +30,8 @@ bfe.conf is the core configuration file of BFE. | Server.MaxProxyHeaderBytes | Integer | Max length of PROXY protocol header, in bytes | N | Default 0 | >= 0 | | Server.EnableAiGateway | Boolean | Whether AI Gateway mode is enabled | N | Default `False` | - | | Server.EstimateToken | Boolean | Whether to estimate token usage based on request Content-Length | N | Default `False` | - | +| Server.AccessibleBodySize | Integer | Max size of request body that can be buffered, in bytes | N | Default 2097152; used for request body rewriting and AI Gateway fallback retry; request bodies larger than this cannot be fully cached and retransmitted | > 0 and <= 8388608 | +| Server.TotalBodyBufferSize | Integer | Upper limit of total memory used by all active bytes_body buffers, in bytes | N | Default 0 (unlimited); when reached, AI Gateway fallback will not wrap the request body for caching, i.e., no retry | >= 0 | | Server.HostRuleConf | String | Path of [host config](server_data_conf/host_rule.data.md) file | N | Default `server_data_conf/host_rule.data`; see [FilePath](00-common.md#3-filepath) type definition | Type is [FilePath](00-common.md#3-filepath) | | Server.VipRuleConf | String | Path of [VIP config](server_data_conf/vip_rule.data.md) file | N | Default `server_data_conf/vip_rule.data`; see [FilePath](00-common.md#3-filepath) type definition | Type is [FilePath](00-common.md#3-filepath) | | Server.RouteRuleConf | String | Path of [route rule config](server_data_conf/route_rule.data.md) file | N | Default `server_data_conf/route_rule.data`; see [FilePath](00-common.md#3-filepath) type definition | Type is [FilePath](00-common.md#3-filepath) | @@ -111,6 +113,12 @@ MaxHeaderBytes = 1048576 # max URI(in header) length in bytes in request MaxHeaderUriBytes = 8192 +# max request body size that can be buffered for rewriting/fallback (default 2MB, max 8MB) +AccessibleBodySize = 2097152 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +TotalBodyBufferSize = 0 + # routing related conf HostRuleConf = server_data_conf/host_rule.data VipRuleConf = server_data_conf/vip_rule.data diff --git a/docs/en_us/configuration/mod_ai_token_auth/token_rule.data.md b/docs/en_us/configuration/mod_ai_token_auth/token_rule.data.md index 233723fc5..4868deb9d 100644 --- a/docs/en_us/configuration/mod_ai_token_auth/token_rule.data.md +++ b/docs/en_us/configuration/mod_ai_token_auth/token_rule.data.md @@ -19,17 +19,18 @@ | QuotaPlans{v}[].redis_key | String | Redis key for storing quota | N | Optional when `unlimited` is true | - | | QuotaPlans{v}[].create_time | Integer | Create time (Unix Time) | N | - | - | | QuotaPlans{v}[].expired_time | Integer | Expiry time (Unix Time) | N | `-1` means never expires | Must be greater than or equal to `-1` | -| QuotaPlans{v}[].quota | Integer | Total quota (unit: token) | N | Required when `unlimited` is false | Must be greater than 0 when `unlimited` is false | +| QuotaPlans{v}[].quota | Integer | Total quota | N | Unit is determined by the `unit` field; when `unit=RMB`, this is a fixed-point integer with precision `1e-8` yuan; required when `unlimited` is false | Must be greater than 0 when `unit=total_token` and `unlimited` is false; must be greater than or equal to 0 when `unit=RMB` and `unlimited` is false | | QuotaPlans{v}[].reset_mode | Integer | Reset mode | Y | `0` - non-periodic; `1` - periodic quota package | Value must be `0` or `1` | +| QuotaPlans{v}[].unit | String | Quota unit | N | Defaults to `total_token` | Value must be `total_token` or `RMB` | | Tokens | Object | API-key declarations for all product lines | Y | Key is product line name | - | | Tokens{k} | String | Product line name | Y | - | - | | Tokens{v} | Object | All API-keys under a product line | Y | - | - | | Tokens{v}{k} | String | An API-key | Y | - | - | | Tokens{v}{v} | Object | An API-key declaration | Y | - | - | | Tokens{v}{v}.key | String | API-key | Y | Must be consistent with the outer key | - | +| Tokens{v}{v}.key_id | String | API-key ID | Y | Used to uniquely identify the API-key | Non-empty string | | Tokens{v}{v}.enabled | Integer | Whether enabled | N | - | - | | Tokens{v}{v}.status | Integer | API-key status | Y | `1` - Enabled; `2` - Disabled; `3` - Expired; `4` - Exhausted | Value must be `1`, `2`, `3`, or `4` | -| Tokens{v}{v}.name | String | Name | N | - | - | | Tokens{v}{v}.update_time | Integer | Update time (Unix Time) | N | Change means a new quota consumption cycle starts, recalculating used quota | - | | Tokens{v}{v}.expired_time | Integer | Expiry time (Unix Time) | N | `-1` means never expires | Must be greater than or equal to `-1` | | Tokens{v}{v}.unlimited_quota | Boolean | Unlimited quota or not | Y | - | - | @@ -64,7 +65,19 @@ "create_time": 1672531200, "expired_time": -1, "quota": 100000, - "reset_mode": 1 + "reset_mode": 1, + "unit": "total_token" + }, + { + "id": "daily_rmb_quota", + "unlimited": false, + "pass_no_quota": false, + "redis_key": "ai:quota:daily_rmb_quota", + "create_time": 1672531200, + "expired_time": -1, + "quota": 90000000, + "reset_mode": 0, + "unit": "RMB" } ] }, @@ -72,8 +85,8 @@ "example_product": { "TESTKEY": { "key": "TESTKEY", + "key_id": "test_key_id", "status": 1, - "name": "test", "expired_time": -1, "unlimited_quota": false, "allow_models": "model_a,model_b", @@ -98,3 +111,7 @@ } } ``` + +> Note: +> - When `unit = total_token`, `quota` is an integer number of tokens. +> - When `unit = RMB`, `quota` is a fixed-point integer with precision `1e-8` yuan (i.e., 1 unit = 0.00000001 yuan). For example, `90000000` means `0.9` yuan. diff --git a/docs/en_us/configuration/server_data_conf/cluster_conf.data.md b/docs/en_us/configuration/server_data_conf/cluster_conf.data.md index e516128f0..d30415066 100644 --- a/docs/en_us/configuration/server_data_conf/cluster_conf.data.md +++ b/docs/en_us/configuration/server_data_conf/cluster_conf.data.md @@ -90,10 +90,52 @@ Note: The following configuration items are located in the namespace `Config[v]` #### AI Service Configuration | Configuration Item | Type | Meaning | Required | Supplementary Description | Validity Condition | -| ----------------------------- | ----------------- | ---------------------------------------------- | -------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| ----------------------------------- | ----------------- | ---------------------------------------------- | -------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | AIConf.Type | Integer | AI service type | N | Currently reserved; keep it 0 | Only supports 0 | -| AIConf.Key | String | API-Key for the backend large model service | N | If empty, the API-Key is not reset when accessing the backend service and the request's API-Key is retained | - | +| AIConf.Provider | String | Provider name of this cluster in `model_prices` | N | Automatically populated by ai-gateway-api based on the OpenAPI `llm_config.provider`; used for cost statistics | - | +| AIConf.Keys | []Object | API-Key list for the backend large model service | N | Empty array means no API-Key is injected when accessing the backend service and the request's API-Key is retained; keys are selected by weighted random | See the "AIConf.Keys elements" table below | +| AIConf.KeyPolicy | Object | API-Key selection policy and retry/backoff configuration | N | Takes effect in multi-Key scenarios; backoff logic does not take effect with single Key or no Key | See the "AIConf.KeyPolicy elements" table below | | AIConf.ModelMapping | Map[string]string | Mapping from original request model to backend service model | N | When accessing the backend service, the model field in the request will be looked up in this mapping; if matched, the model field in the request will be overwritten | Both keys and values are non-empty | +| AIConf.MatchPrefix | String | Provider/model prefix to match | N | e.g. `openrouter/`; must end with `/`; used for aggregator providers such as OpenRouter | Required when `StripPrefix=true` | +| AIConf.StripPrefix | Boolean | Whether to strip the prefix specified by `MatchPrefix` | N | When `true`, the prefix is removed from the request model field before forwarding to the backend; when `false`, the prefix is only used as a routing marker and not stripped | Defaults to `false` | +| AIConf.ModelTable | Object | Model pricing table of this cluster | N | Automatically populated by ai-gateway-api by querying `model_prices` based on `Provider`; currency is fixed to `RMB` for now | See the "AIConf.ModelTable elements" table below | + +##### AIConf.Keys elements + +| Configuration Item | Type | Meaning | Required | Supplementary Description | Validity Condition | +| ------------------- | ------- | ------------------ | -------- | ------------------------------------------------ | ---------- | +| AIConf.Keys[i].Name | String | API-Key name/identifier | Y | Used for logging, monitoring and operations identification | Non-empty | +| AIConf.Keys[i].Key | String | API-Key value | Y | Secret key used for backend authentication | Non-empty | +| AIConf.Keys[i].Weight | Integer | Weight | Y | Used for weighted random selection; range is `[0,100]`; `0` means no traffic is received | `[0,100]`; total weight of multiple keys must be 100 | + +##### AIConf.KeyPolicy elements + +| Configuration Item | Type | Meaning | Required | Supplementary Description | Validity Condition | +| ------------------------------------- | ------- | -------------------- | -------- | ------------------------------------------------------------ | -------------------------------- | +| AIConf.KeyPolicy.Strategy | String | Key selection strategy | N | Currently only supports `weighted_random` | Only supports `weighted_random` | +| AIConf.KeyPolicy.MaxRetries | Integer | Total additional retry count | N | Maximum retry count excluding the first selection in one `aiClusterInvoke` call; `0` means no retry | >= 0 | +| AIConf.KeyPolicy.RetryBackoffInitial | Integer | Initial backoff time, in milliseconds | N | Backoff time for the first retry | >= 0 | +| AIConf.KeyPolicy.RetryBackoffMax | Integer | Maximum backoff time, in milliseconds | N | Upper limit of backoff time | >= 0, and must be >= RetryBackoffInitial | + +##### AIConf.ModelTable elements + +| Configuration Item | Type | Meaning | Required | Supplementary Description | Validity Condition | +| ------------------------------- | -------- | ------------------ | -------- | ----------------------------------- | ---------- | +| AIConf.ModelTable.Currency | String | Currency type | Y | Fixed to `RMB` in v0.4 | - | +| AIConf.ModelTable.Models | []Object | Model pricing entry list | Y | Each entry corresponds to a model and its price/limit | See the "AIConf.ModelTable.Models elements" table below | + +##### AIConf.ModelTable.Models elements + +| Configuration Item | Type | Meaning | Required | Supplementary Description | Validity Condition | +| ---------------------------------------------- | ----------------- | ------------------ | -------- | ----------------------------------- | ---------- | +| AIConf.ModelTable.Models[i].Provider | String | Provider name | Y | - | Non-empty | +| AIConf.ModelTable.Models[i].Model | String | Model name | Y | Used to match the `target_model` in the request | Non-empty | +| AIConf.ModelTable.Models[i].BaseModel | String | Normalized model name | Y | - | Non-empty | +| AIConf.ModelTable.Models[i].Mode | String | Request mode | N | e.g. `chat` | - | +| AIConf.ModelTable.Models[i].Capabilities | []String | Capability list | N | e.g. `["chat", "reasoning"]` | - | +| AIConf.ModelTable.Models[i].SupportedParameters | []String | Supported request parameter list | N | e.g. `["temperature", "max_tokens"]` | - | +| AIConf.ModelTable.Models[i].Limits | Map[string]Integer | Limit object | N | e.g. `context_window`, etc. | - | +| AIConf.ModelTable.Models[i].Prices | Map[string]Number | Price object | N | e.g. `input_cost_per_token`, etc. | - | ## Configuration Example @@ -255,9 +297,51 @@ Note: The following configuration items are located in the namespace `Config[v]` }, "AIConf": { "Type": 0, - "Key": "sk-example-api-key", + "Provider": "deepseek", + "MatchPrefix": "openrouter/", + "StripPrefix": true, + "Keys": [ + { + "Name": "key-primary", + "Key": "sk-example-api-key-primary", + "Weight": 70 + }, + { + "Name": "key-secondary", + "Key": "sk-example-api-key-secondary", + "Weight": 30 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 3, + "RetryBackoffInitial": 500, + "RetryBackoffMax": 5000 + }, "ModelMapping": { "gpt-4": "backend-gpt-4-model" + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "deepseek", + "Model": "deepseek-v3", + "BaseModel": "deepseek-v3", + "Mode": "chat", + "Capabilities": ["chat", "reasoning", "tools"], + "SupportedParameters": ["temperature", "max_tokens"], + "Limits": { + "context_window": 128000, + "max_input_tokens": 128000, + "max_output_tokens": 8192 + }, + "Prices": { + "input_cost_per_token": 0.000002, + "output_cost_per_token": 0.000008 + } + } + ] } } } diff --git a/docs/zh_cn/SUMMARY.md b/docs/zh_cn/SUMMARY.md index 6a3c26ddf..489d80938 100644 --- a/docs/zh_cn/SUMMARY.md +++ b/docs/zh_cn/SUMMARY.md @@ -118,6 +118,11 @@ * [版本发布说明](development/release_regulation.md) * 开发参考文档 * [代码结构说明](development/source_code_layout.md) + * 系统设计 + * [mod_ai_route 系统设计](sys_design/mod_ai_route.md) + * [mod_ai_route 对应 BFE 主程序修改方案](sys_design/mod_ai_route_bfe_changes.md) + * [BFE 多 API-Key 支持](sys_design/multi_api_key.md) + * [BFE AI 访问日志可观测字段设计](sys_design/ai_access_log_fields.md) * [模块开发介绍](development/module/overview.md) * [BFE回调机制说明](development/module/bfe_callback.md) * [如何开发模块](development/module/how_to_write_module.md) diff --git a/docs/zh_cn/condition/condition_primitive_index.md b/docs/zh_cn/condition/condition_primitive_index.md index 2f6297572..0291b0175 100644 --- a/docs/zh_cn/condition/condition_primitive_index.md +++ b/docs/zh_cn/condition/condition_primitive_index.md @@ -89,6 +89,7 @@ ### body * [req_body_json_in(json_path, value_list, case_insensitive)](./request/body.md#req_body_json_injson_path-value_list-case_insensitive) + * [req_body_json_prefix_in(json_path, value_prefix_list, case_insensitive)](./request/body.md#req_body_json_prefix_injson_path-value_prefix_list-case_insensitive) ## 响应相关 diff --git a/docs/zh_cn/condition/request/body.md b/docs/zh_cn/condition/request/body.md index 6a78303da..7de885f58 100644 --- a/docs/zh_cn/condition/request/body.md +++ b/docs/zh_cn/condition/request/body.md @@ -1,18 +1,42 @@ -# 请求body相关条件原语 - -## req_body_json_in(json_path, value_list, case_insensitive) - -* 含义： 在json格式的请求body中，查找json_path指定的字段，判断其值是否精确匹配value_list之一 -* 参数 - -| 参数 | 描述 | -| -------- | ---------------------- | -| json_path | String
请求body中的json字段的路径 | -| value_list | String
value列表，多个之间使用‘|’连接 | -| case_insensitive | Boolean
是否忽略大小写 | - -* 示例 - -```go -req_body_json_in("model", "deepseek-r1|qwen-plus", true) -``` +# 请求body相关条件原语 + +## req_body_json_in(json_path, value_list, case_insensitive) + +* 含义： 在json格式的请求body中，查找json_path指定的字段，判断其值是否精确匹配value_list之一 +* 参数 + +| 参数 | 描述 | +| -------- | ---------------------- | +| json_path | String
请求body中的json字段的路径 | +| value_list | String
value列表，多个之间使用‘|’连接 | +| case_insensitive | Boolean
是否忽略大小写 | + +* 示例 + +```go +req_body_json_in("model", "deepseek-r1|qwen-plus", true) +``` + +## req_body_json_prefix_in(json_path, value_prefix_list, case_insensitive) + +* 含义： 在json格式的请求body中，查找json_path指定的字段，判断其字符串值是否以value_prefix_list中某一项为前缀 +* 参数 + +| 参数 | 描述 | +| -------- | ---------------------- | +| json_path | String
请求body中的json字段的路径 | +| value_prefix_list | String
前缀列表，多个之间使用‘|’连接 | +| case_insensitive | Boolean
是否忽略大小写 | + +* 示例 + +```go +// 命中所有 OpenRouter 模型 +req_body_json_prefix_in("model", "openrouter/", false) + +// 命中 OpenRouter 下 anthropic 子命名空间的所有模型 +req_body_json_prefix_in("model", "openrouter/anthropic/", false) + +// 命中所有 gpt- 或 claude- 开头的模型（大小写不敏感） +req_body_json_prefix_in("model", "gpt-|claude-", true) +``` diff --git a/docs/zh_cn/configuration/bfe.conf.md b/docs/zh_cn/configuration/bfe.conf.md index 715046640..8a305a2e0 100644 --- a/docs/zh_cn/configuration/bfe.conf.md +++ b/docs/zh_cn/configuration/bfe.conf.md @@ -30,6 +30,8 @@ bfe.conf是BFE的核心配置 | Server.AcceptNum | Integer | 每个监听地址的Accept协程数 | N | 默认值1；为0时自动设为1 | >= 0 | | Server.EnableAiGateway | Boolean | 是否启用AI Gateway模式 | N | 默认值`False` | - | | Server.EstimateToken | Boolean | 是否基于请求Content-Length估算token使用量 | N | 默认值`False` | - | +| Server.AccessibleBodySize | Integer | 请求体可缓冲的最大长度，单位为Byte | N | 默认值2097152；用于请求体改写及AI Gateway fallback重试；超过此大小的请求体无法被完整缓存并重传 | > 0 且 <= 8388608 | +| Server.TotalBodyBufferSize | Integer | 所有活跃bytes_body buffer占用内存之和的上限，单位为Byte | N | 默认值0（表示无限制）；达到上限时，AI Gateway fallback不再对请求体做缓存封装，即不重传 | >= 0 | | Server.HostRuleConf | String | [租户域名表配置](server_data_conf/host_rule.data.md)文件路径 | N | 默认值`server_data_conf/host_rule.data`；参见 [FilePath](00-common.md#3-文件路径filepath) 类型定义 | 类型为 [FilePath](00-common.md#3-文件路径filepath) | | Server.VipRuleConf | String | [租户VIP表配置](server_data_conf/vip_rule.data.md)文件路径 | N | 默认值`server_data_conf/vip_rule.data`；参见 [FilePath](00-common.md#3-文件路径filepath) 类型定义 | 类型为 [FilePath](00-common.md#3-文件路径filepath) | | Server.RouteRuleConf | String | [转发规则配置](server_data_conf/route_rule.data.md)文件路径 | N | 默认值`server_data_conf/route_rule.data`；参见 [FilePath](00-common.md#3-文件路径filepath) 类型定义 | 类型为 [FilePath](00-common.md#3-文件路径filepath) | @@ -111,6 +113,12 @@ MaxHeaderBytes = 1048576 # max URI(in header) length in bytes in request MaxHeaderUriBytes = 8192 +# max request body size that can be buffered for rewriting/fallback (default 2MB, max 8MB) +AccessibleBodySize = 2097152 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +TotalBodyBufferSize = 0 + # routing related conf HostRuleConf = server_data_conf/host_rule.data VipRuleConf = server_data_conf/vip_rule.data diff --git a/docs/zh_cn/configuration/mod_ai_token_auth/token_rule.data.md b/docs/zh_cn/configuration/mod_ai_token_auth/token_rule.data.md index 244d21f4d..fc5890341 100644 --- a/docs/zh_cn/configuration/mod_ai_token_auth/token_rule.data.md +++ b/docs/zh_cn/configuration/mod_ai_token_auth/token_rule.data.md @@ -19,17 +19,18 @@ | QuotaPlans{v}[].redis_key | String | Redis 中存储配额的 key | N | unlimited 为 true 时可不配置 | - | | QuotaPlans{v}[].create_time | Integer | 创建时间（Unix Time） | N | - | - | | QuotaPlans{v}[].expired_time | Integer | 过期时间（Unix Time） | N | `-1` 表示永不过期 | 必须大于等于 `-1` | -| QuotaPlans{v}[].quota | Integer | 配额总量（单位：token） | N | unlimited 为 false 时必填 | unlimited 为 false 时必须大于 0 | +| QuotaPlans{v}[].quota | Integer | 配额总量 | N | 单位由 `unit` 字段决定；`unit=RMB` 时为定点整数，精度 `1e-8` 元；unlimited 为 false 时必填 | `unit=total_token` 且 unlimited 为 false 时必须大于 0；`unit=RMB` 且 unlimited 为 false 时必须大于等于 0 | | QuotaPlans{v}[].reset_mode | Integer | 重置模式 | Y | `0` - 非周期性；`1` - 周期性的配额包 | 取值范围为 `0`、`1` | +| QuotaPlans{v}[].unit | String | 配额单位 | N | 默认 `total_token` | 取值为 `total_token` 或 `RMB` | | Tokens | Object | 所有产品线的 api-key 声明 | Y | 以产品线名称为键 | - | | Tokens{k} | String | 产品线名称 | Y | - | - | | Tokens{v} | Object | 该产品线下的所有 api-key | Y | - | - | | Tokens{v}{k} | String | api-key | Y | - | - | | Tokens{v}{v} | Object | 一个 api-key 声明 | Y | - | - | | Tokens{v}{v}.key | String | api-key | Y | 须与外层键一致 | - | +| Tokens{v}{v}.key_id | String | api-key 标识 ID | Y | 用于唯一标识该 api-key | 非空字符串 | | Tokens{v}{v}.enabled | Integer | 是否启用 | N | - | - | | Tokens{v}{v}.status | Integer | api-key 状态 | Y | `1` - Enabled；`2` - Disabled；`3` - Expired；`4` - Exhausted | 取值范围为 `1`、`2`、`3`、`4` | -| Tokens{v}{v}.name | String | 名称 | N | - | - | | Tokens{v}{v}.update_time | Integer | 更新时间（Unix Time） | N | 改变意味着开启一个新的配额消费周期 | - | | Tokens{v}{v}.expired_time | Integer | 过期时间（Unix Time） | N | `-1` 表示永不过期 | 必须大于等于 `-1` | | Tokens{v}{v}.unlimited_quota | Boolean | 是否无限配额 | Y | - | - | @@ -64,7 +65,19 @@ "create_time": 1672531200, "expired_time": -1, "quota": 100000, - "reset_mode": 1 + "reset_mode": 1, + "unit": "total_token" + }, + { + "id": "daily_rmb_quota", + "unlimited": false, + "pass_no_quota": false, + "redis_key": "ai:quota:daily_rmb_quota", + "create_time": 1672531200, + "expired_time": -1, + "quota": 90000000, + "reset_mode": 0, + "unit": "RMB" } ] }, @@ -72,8 +85,8 @@ "example_product": { "TESTKEY": { "key": "TESTKEY", + "key_id": "test_key_id", "status": 1, - "name": "test", "expired_time": -1, "unlimited_quota": false, "allow_models": "model_a,model_b", @@ -98,3 +111,7 @@ } } ``` + +> 说明： +> - `unit = total_token` 时，`quota` 为整数 Token 数。 +> - `unit = RMB` 时，`quota` 为定点整数，精度为 `1e-8` 元（即 1 单位 = 0.00000001 元）。例如 `90000000` 表示 `0.9` 元。 diff --git a/docs/zh_cn/configuration/server_data_conf/cluster_conf.data.md b/docs/zh_cn/configuration/server_data_conf/cluster_conf.data.md index 76864100a..dffd3c755 100644 --- a/docs/zh_cn/configuration/server_data_conf/cluster_conf.data.md +++ b/docs/zh_cn/configuration/server_data_conf/cluster_conf.data.md @@ -89,11 +89,53 @@ cluster_conf.data为集群转发配置文件。 #### AI服务配置 -| 配置项 | 类型 | 参数含义 | 必填 | 补充描述 | 合法性条件 | -| ----------------------------- | ----------------- | ---------------------------------------------- | ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | -| AIConf.Type | Integer | AI服务类型 | N | 当前保留字段，请保持为0 | 仅支持 0 | -| AIConf.Key | String | 后端大模型服务的API-Key | N | 空字符串表示访问后端服务时不重置API-Key，仍保持请求的API-Key | - | -| AIConf.ModelMapping | Map[string]string | 原请求model -> 后端服务的model 的映射关系 | N | 访问后端服务时将根据请求的 model 字段查找此映射关系，命中则重写请求的 model 字段 | 键值均非空 | +| 配置项 | 类型 | 参数含义 | 必填 | 补充描述 | 合法性条件 | +| ----------------------------------- | ----------------- | ---------------------------------------------- | ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| AIConf.Type | Integer | AI服务类型 | N | 当前保留字段，请保持为0 | 仅支持 0 | +| AIConf.Provider | String | 该集群在 model_prices 中对应的 provider 名称 | N | 由 ai-gateway-api 根据 OpenAPI `llm_config.provider` 自动填充；用于成本统计 | - | +| AIConf.Keys | []Object | 后端大模型服务的 API-Key 列表 | N | 为空数组表示访问后端服务时不注入 API-Key，仍保持请求的 API-Key；按权重加权随机选择 | 元素见下表「AIConf.Keys 元素」 | +| AIConf.KeyPolicy | Object | API-Key 选择策略与重试退避配置 | N | 多 Key 场景下生效；单 Key 或无 Key 时退避逻辑不生效 | 元素见下表「AIConf.KeyPolicy 元素」 | +| AIConf.ModelMapping | Map[string]string | 原请求model -> 后端服务的model 的映射关系 | N | 访问后端服务时将根据请求的 model 字段查找此映射关系，命中则重写请求的 model 字段 | 键值均非空 | +| AIConf.MatchPrefix | String | 需要匹配的 provider/model 前缀 | N | 例如 `openrouter/`；必须以 `/` 结尾；用于 OpenRouter 等聚合 provider 场景 | `StripPrefix=true` 时必填 | +| AIConf.StripPrefix | Boolean | 是否裁剪 `MatchPrefix` 指定前缀 | N | `true` 时转发给下游前会从请求 model 字段中去掉该前缀；`false` 时仅用于路由标识，不裁剪 | 默认 `false` | +| AIConf.ModelTable | Object | 该集群的模型定价表 | N | 由 ai-gateway-api 根据 `Provider` 查询 model_prices 自动填充；当前货币固定为 RMB | 元素见下表「AIConf.ModelTable 元素」 | + +##### AIConf.Keys 元素 + +| 配置项 | 类型 | 参数含义 | 必填 | 补充描述 | 合法性条件 | +| ------------------- | ------- | ------------------ | ---- | ------------------------------------------------ | ---------- | +| AIConf.Keys[i].Name | String | API-Key 名称/标识 | Y | 用于日志、监控、运维识别 | 非空 | +| AIConf.Keys[i].Key | String | API-Key 值 | Y | 实际用于后端认证的密钥 | 非空 | +| AIConf.Keys[i].Weight | Integer | 权重 | Y | 用于加权随机选择；范围为 `[0,100]`；`0` 表示不接收流量 | `[0,100]`；多 Key 时权重总和须为 100 | + +##### AIConf.KeyPolicy 元素 + +| 配置项 | 类型 | 参数含义 | 必填 | 补充描述 | 合法性条件 | +| ------------------------------------- | ------- | -------------------- | ---- | ------------------------------------------------------------ | -------------------------------- | +| AIConf.KeyPolicy.Strategy | String | Key 选择策略 | N | 当前仅支持 `weighted_random` | 仅支持 `weighted_random` | +| AIConf.KeyPolicy.MaxRetries | Integer | 总额外重试次数 | N | 一次 `aiClusterInvoke` 调用内，除首次选择外的最大重试次数；`0` 表示不重试 | >= 0 | +| AIConf.KeyPolicy.RetryBackoffInitial | Integer | 初始退避时间，单位毫秒 | N | 首次重试的退避时间 | >= 0 | +| AIConf.KeyPolicy.RetryBackoffMax | Integer | 最大退避时间，单位毫秒 | N | 退避时间上限 | >= 0，且须 >= RetryBackoffInitial | + +##### AIConf.ModelTable 元素 + +| 配置项 | 类型 | 参数含义 | 必填 | 补充描述 | 合法性条件 | +| ------------------------------- | -------- | ------------------ | ---- | ----------------------------------- | ---------- | +| AIConf.ModelTable.Currency | String | 货币类型 | Y | v0.4 固定为 `RMB` | - | +| AIConf.ModelTable.Models | []Object | 模型定价条目列表 | Y | 每个条目对应一个模型及其价格/限制 | 元素见下表「AIConf.ModelTable.Models 元素」 | + +##### AIConf.ModelTable.Models 元素 + +| 配置项 | 类型 | 参数含义 | 必填 | 补充描述 | 合法性条件 | +| ---------------------------------------------- | ----------------- | ------------------ | ---- | ----------------------------------- | ---------- | +| AIConf.ModelTable.Models[i].Provider | String | Provider 名 | Y | - | 非空 | +| AIConf.ModelTable.Models[i].Model | String | 模型名 | Y | 用于匹配请求中的 target_model | 非空 | +| AIConf.ModelTable.Models[i].BaseModel | String | 归一化模型名 | Y | - | 非空 | +| AIConf.ModelTable.Models[i].Mode | String | 请求模式 | N | 例如 `chat` | - | +| AIConf.ModelTable.Models[i].Capabilities | []String | 能力列表 | N | 例如 `["chat", "reasoning"]` | - | +| AIConf.ModelTable.Models[i].SupportedParameters| []String | 支持的请求参数列表 | N | 例如 `["temperature", "max_tokens"]`| - | +| AIConf.ModelTable.Models[i].Limits | Map[string]Integer| 限制对象 | N | 例如 `context_window` 等 | - | +| AIConf.ModelTable.Models[i].Prices | Map[string]Number | 价格对象 | N | 例如 `input_cost_per_token` 等 | - | ## 配置示例 @@ -255,9 +297,51 @@ cluster_conf.data为集群转发配置文件。 }, "AIConf": { "Type": 0, - "Key": "sk-example-api-key", + "Provider": "deepseek", + "MatchPrefix": "openrouter/", + "StripPrefix": true, + "Keys": [ + { + "Name": "key-primary", + "Key": "sk-example-api-key-primary", + "Weight": 70 + }, + { + "Name": "key-secondary", + "Key": "sk-example-api-key-secondary", + "Weight": 30 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 3, + "RetryBackoffInitial": 500, + "RetryBackoffMax": 5000 + }, "ModelMapping": { "gpt-4": "backend-gpt-4-model" + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "deepseek", + "Model": "deepseek-v3", + "BaseModel": "deepseek-v3", + "Mode": "chat", + "Capabilities": ["chat", "reasoning", "tools"], + "SupportedParameters": ["temperature", "max_tokens"], + "Limits": { + "context_window": 128000, + "max_input_tokens": 128000, + "max_output_tokens": 8192 + }, + "Prices": { + "input_cost_per_token": 0.000002, + "output_cost_per_token": 0.000008 + } + } + ] } } } diff --git a/docs/zh_cn/modifications/2026-08-19-ai-fallback-status-codes/design-changes.md b/docs/zh_cn/modifications/2026-08-19-ai-fallback-status-codes/design-changes.md new file mode 100644 index 000000000..52b455689 --- /dev/null +++ b/docs/zh_cn/modifications/2026-08-19-ai-fallback-status-codes/design-changes.md @@ -0,0 +1,406 @@ +# BFE AI 路由 fallback 状态码触发策略扩展设计变更 + +## 1. 背景 + +当前 BFE AI 网关的 cluster 级 fallback 逻辑固定为： + +- 连接/传输错误触发 fallback； +- 后端返回 `5xx` 触发 fallback； +- 后端返回 `4xx`（包括 401/402/403/422/429 等）**不触发** fallback，直接把最后一个 4xx 响应返回给客户端。 + +相关代码位于 `bfe/bfe_server/reverseproxy.go`： + +```go +// line 1289 +if invokeErr == nil && res != nil && res.StatusCode < 500 { + // success or 4xx (client error, do not fallback) + break +} + +// line 1664 +func shouldTriggerFallback(res *bfe_http.Response, err error) bool { + if err != nil { + return true + } + if res != nil && res.StatusCode >= 500 { + return true + } + return false +} +``` + +这与 DeepSeek 等上游的实际情况不符：某些 4xx（如 401 key 失效、429 限流、402 欠费、422 请求参数被上游拒绝、400 上游内部错误包装）本质上是**上游不可用或配额类错误**，应当允许降级到 fallback cluster。GitHub issue #1317 要求 DeepSeek 的 401/400/402/422/429/500/503 都能触发 fallback。 + +参考 Bifrost 的做法，它并不依赖用户逐条路由配置状态码，而是在代码中维护一个统一的状态码分类集合： + +- `transientServerStatusCodes`：500/502/503/504 等 transient 服务端错误； +- `perKeyFailureStatusCodes`：401/402/403/429 等单 key 失败； +- 其余情况下只要没有显式禁用 fallback，就会继续尝试 fallback。 + +本设计借鉴该思路，**在 BFE 代码中定义默认 fallback 触发状态码集合**，避免增加用户配置成本。新行为默认生效，无需修改 `ai_route.data`。 + +--- + +## 2. 目标 + +1. 在 BFE 代码中定义统一的 AI fallback 状态码分类集合。 +2. 外层 `ServeHTTPForAI` 不再对 `res.StatusCode < 500` 一刀切：只有 `< 400` 才视为成功；`>= 400` 按状态码分类决定是否 fallback。 +3. key 级重试逻辑（`aiClusterInvoke`）保持现状：401/402/403 标记 key 死亡、429 轮换 key、5xx/错误重试；key 耗尽后把最终响应交给外层决策。 +4. 默认覆盖 issue #1317 要求的状态码：400/401/402/422/429 与全部 5xx（含 500/503）。 +5. 更新单元测试、集成测试与系统设计文档，使其与新行为一致。 + +--- + +## 3. 变更总览 + +| 层级 | 变更点 | 影响文件 | +|---|---|---| +| 常量定义 | 新增 `aiFallbackStatusCodes` 常量集合 | `bfe/bfe_server/reverseproxy.go` | +| 转发层 | `ServeHTTPForAI` 成功判定从 `< 500` 改为 `< 400` | `bfe/bfe_server/reverseproxy.go` | +| 转发层 | `shouldTriggerFallback` 按状态码集合判定 | `bfe/bfe_server/reverseproxy.go` | +| 测试 | 更新现有单元测试、集成测试用例 | `bfe/bfe_server/reverseproxy_ai_test.go`、`bfe/tests/integration/...` | +| 文档 | 更新 `multi_api_key.md`、`mod_ai_route_bfe_changes.md` 与集成测试说明 | `bfe/docs/zh_cn/sys_design/...` | + +--- + +## 4. 详细设计 + +### 4.1 状态码分类集合 + +在 `bfe/bfe_server/reverseproxy.go` 中新增常量： + +```go +// aiFallbackStatusCodes 定义默认会触发 cluster 级 fallback 的 HTTP 状态码。 +// 包含： +// - 全部 5xx（服务端错误，如 500/502/503/504） +// - 401/402/403（鉴权/授权/欠费失败） +// - 429（限流） +// - 400/422（上游对请求体/参数的拒绝，换 provider 可能成功） +// +// 不在集合中的 4xx（如 404/405/413 等）视为客户端错误，不触发 fallback。 +var aiFallbackStatusCodes = map[int]struct{}{ + 400: {}, + 401: {}, + 402: {}, + 403: {}, + 422: {}, + 429: {}, + // 5xx 由 >= 500 统一处理，不需要逐个列出 +} +``` + +设计说明： + +- 使用 `map[int]struct{}` 便于 O(1) 查找。 +- 5xx 不全部列出，由 `code >= 500` 统一覆盖，避免遗漏 501/502/504 等。 +- 该集合是**代码常量**，不需要在 `ai_route.data` 中配置，降低用户成本。 +- 如果未来需要允许自定义，可以在此基础上扩展为全局配置或环境变量，但本次设计保持零配置。 + +### 4.2 改造外层 fallback 决策 + +#### 4.2.1 成功判定条件 + +当前 `ServeHTTPForAI` 在 `aiClusterInvoke` 返回后立即判定： + +```go +if invokeErr == nil && res != nil && res.StatusCode < 500 { + break +} +``` + +这会把 4xx 全部当作成功，必须改为只有 `< 400` 才视为成功： + +```go +if invokeErr == nil && res != nil && res.StatusCode < 400 { + break +} +``` + +#### 4.2.2 `shouldTriggerFallback` 改造 + +```go +func shouldTriggerFallback(res *bfe_http.Response, err error) bool { + if err != nil { + return true + } + code := getResponseStatus(res) + + // 5xx 统一触发 fallback + if code >= 500 { + return true + } + + // 特定 4xx 触发 fallback + if _, ok := aiFallbackStatusCodes[code]; ok { + return true + } + + return false +} +``` + +#### 4.2.3 外层循环伪代码 + +```go +for i, attempt := range attempts { + if i > 0 { + if !p.resetRequestForRetry(basicReq) { + log.Logger.Warn("ServeHTTPForAI: fallback aborted, request body cannot be rewound") + break + } + } + + res, action, lastCluster, invokeErr = p.aiClusterInvoke(srv, serverConf, basicReq, rw, attempt, aiMeta) + + // 2xx/3xx 直接返回，不再 fallback + if invokeErr == nil && res != nil && res.StatusCode < 400 { + break + } + + if i == len(attempts)-1 { + break + } + + if !shouldTriggerFallback(res, invokeErr) { + break + } + + log.Logger.Info("ServeHTTPForAI: fallback triggered, cluster[%s] err[%v] status[%d]", + attempt.ClusterName, invokeErr, getResponseStatus(res)) + + if res != nil { + res.Body.Close() + } +} +``` + +### 4.3 key 级重试逻辑保持现状 + +`aiClusterInvoke` 内部逻辑不需要修改： + +- 401/402/403 → 标记 key 死亡，换 key； +- 429 → 标记 key 已用，换 key； +- 5xx/错误 → 同 key 退避重试； +- 400/422/404 等 → 直接返回当前响应，不再浪费 key 预算。 + +当 key 耗尽或不再重试时，`aiClusterInvoke` 把最终响应（可能为 4xx）返回给外层。外层 `shouldTriggerFallback` 再决定是否继续 cluster 级 fallback。 + +### 4.4 不触发 fallback 的 4xx + +以下状态码默认**不触发** fallback，直接返回最后一个响应： + +- 404 Not Found +- 405 Method Not Allowed +- 406 Not Acceptable +- 407 Proxy Authentication Required +- 408 Request Timeout（可讨论，但当前不纳入） +- 409 Conflict +- 410 Gone +- 411~417 等请求级错误 +- 413 Payload Too Large（BFE 层应已拦截） + +这些错误通常不会通过换 provider 解决，避免无效降级。 + +### 4.5 请求体重置与模型覆盖 + +每次触发 fallback 前会调用 `resetRequestForRetry`： + +- 减少后端连接计数； +- 重置 `OutRequest`； +- rewind 请求体； +- 重置 `Content-Length`； +- 清除 `ErrCode` / `ErrMsg`。 + +由于 `doSingleAIForward` 会对请求体做 `model` 覆盖与 `ModelMapping` 改写，`resetRequestForRetry` 的 rewind 保证下一次 attempt 从原始请求体重新开始。该机制已在 5xx fallback 中验证，新增 4xx fallback 路径复用同一逻辑。 + +### 4.6 响应体关闭 + +触发 fallback 时，必须关闭上一个 4xx 响应体，避免连接泄漏： + +```go +if res != nil { + res.Body.Close() +} +``` + +该逻辑已存在，本次无需改动。 + +--- + +## 5. 关键代码变更示例 + +### 5.1 `bfe/bfe_server/reverseproxy.go` + +#### 新增常量 + +```go +// aiFallbackStatusCodes 定义默认触发 cluster 级 fallback 的 4xx 状态码集合。 +var aiFallbackStatusCodes = map[int]struct{}{ + 400: {}, + 401: {}, + 402: {}, + 403: {}, + 422: {}, + 429: {}, +} +``` + +#### 外层循环 + +```go +res, action, lastCluster, invokeErr = p.aiClusterInvoke(srv, serverConf, basicReq, rw, attempt, aiMeta) +if invokeErr == nil && res != nil && res.StatusCode < 400 { + break +} + +if i == len(attempts)-1 { + break +} +if !shouldTriggerFallback(res, invokeErr) { + break +} + +log.Logger.Info("ServeHTTPForAI: fallback triggered, cluster[%s] err[%v] status[%d]", + attempt.ClusterName, invokeErr, getResponseStatus(res)) + +if res != nil { + res.Body.Close() +} +``` + +#### `shouldTriggerFallback` + +```go +func shouldTriggerFallback(res *bfe_http.Response, err error) bool { + if err != nil { + return true + } + code := getResponseStatus(res) + if code >= 500 { + return true + } + if _, ok := aiFallbackStatusCodes[code]; ok { + return true + } + return false +} +``` + +--- + +## 6. 测试计划 + +### 6.1 单元测试 + +更新 `bfe/bfe_server/reverseproxy_ai_test.go` 中的 `TestShouldTriggerFallback`： + +| 输入状态码 | 预期 | +|---|---| +| 200 | false | +| 404 | false | +| 400 | true | +| 401 | true | +| 402 | true | +| 403 | true | +| 422 | true | +| 429 | true | +| 500 | true | +| 503 | true | +| ConnectError | true | + +新增一个辅助函数 `TestAiFallbackStatusCodesCoverage`，确保集合中包含/排除预期状态码。 + +### 6.2 集成测试更新 + +#### TC-05 调整：Key 耗尽后触发 fallback + +原 TC-05 期望 401/403/429 key 耗尽后**不触发** cluster fallback。新行为下这些状态码会触发 fallback，因此需要调整预期： + +- 响应状态码：200（来自 fallback cluster）。 +- `cluster_multi_key` 后端收到 3~4 次请求（尝试所有 key）。 +- `cluster_fallback_ok` 后端被命中 1 次。 +- BFE 日志中出现 `fallback triggered` 相关记录。 + +如需保留“某些 4xx 不触发 fallback”的用例，可新增 TC-05-B：使用 404 作为后端响应，验证 `cluster_fallback_ok` 未被命中。 + +#### TC-06 保持不变 + +5xx key 耗尽触发 cluster fallback，行为与现有逻辑一致。 + +#### 新增集成测试用例 + +| 用例编号 | 场景 | 预期 | +|---|---|---| +| SC02-TC-15 | primary cluster 单 key 返回 400 | key 不重试，直接触发 cluster fallback | +| SC02-TC-16 | primary cluster 单 key 返回 422 | key 不重试，直接触发 cluster fallback | +| SC02-TC-17 | primary cluster 返回 404 | 404 不在 fallback 集合，不触发 fallback，返回 404 | +| SC01-TC-12 | primary cluster 返回 429 | 触发 cluster fallback | + +### 6.3 回归测试 + +- `go test ./bfe_server/...` 通过； +- `go test ./bfe_modules/mod_ai_route/...` 通过； +- 集成测试 SC01/SC02 全量通过。 + +--- + +## 7. 文档更新 + +需要同步更新以下文档，避免与代码行为不一致： + +1. `bfe/docs/zh_cn/sys_design/mod_ai_route_bfe_changes.md` + - 修改“fallback 只针对后端不可用场景，4xx/限流/鉴权失败不触发”。 + - 改为说明：默认情况下 5xx/网络错误以及 400/401/402/403/422/429 都会触发 fallback；404 等请求级 4xx 不触发。 + +2. `bfe/docs/zh_cn/sys_design/multi_api_key.md` + - 修改 4.2 节“与 cluster 级 fallback 的边界”： + - 原：401/403/429 不触发 cluster fallback； + - 新：401/402/403/429 在 key 级重试耗尽后会触发 cluster fallback；400/422 也会触发；404 等不会。 + - 更新 `shouldTriggerFallback` 伪代码。 + +3. `bfe/tests/integration/测试设计文档/scenario-SC02-多API-Key轮换与重试/TC-05-Key耗尽后返回最后响应.md` + - 按新行为更新预期，或拆分为 TC-05（401/403/429 触发 fallback）和 TC-05-B（404 不触发 fallback）。 + +4. `document-ai-gateway/迭代系统设计/v0.4/fallback支持/bifrost-fallback-analysis.md` + - 可补充一条跟踪记录：BFE 已确定采用代码级状态码分类方案，无需 `ai_route.data` 配置。 + +--- + +## 8. 与 Bifrost 的对比 + +| 维度 | Bifrost | 本方案（BFE） | +|---|---|---| +| 状态码配置 | 无用户配置，代码内建集合 | 无用户配置，代码内建集合 | +| 5xx 处理 | transientServerStatusCodes：500/502/503/504 | 全部 5xx 触发 fallback | +| key 失败 | perKeyFailureStatusCodes：401/402/403/429 | 401/402/403/429 触发 fallback | +| 400/422 | 默认继续 fallback（未显式禁用时） | 显式加入集合，触发 fallback | +| 禁用 fallback | 通过 context flag 显式禁用 | 当前无显式禁用，未来可扩展 | +| 配置成本 | 低 | 低 | + +本方案与 Bifrost 思路一致：通过代码中的状态码分类集合决定 fallback，避免用户在每条路由上维护状态码列表。 + +--- + +## 9. 风险与回滚 + +| 风险 | 缓解措施 | +|---|---| +| 默认行为改变，已有 4xx 用例返回不同 | 同步更新单元测试、集成测试与文档；404 等仍不触发 fallback，降低误触发 | +| 400/422 可能是真正的客户端错误 | 在 AI 网关场景下，不同 provider 对同一请求可能返回不同结果，fallback 有收益；若确认是恶意/错误请求，后续可扩展显式禁用 fallback 机制 | +| 4xx 响应体未关闭 | 沿用 `res.Body.Close()`，新增路径保证执行 | +| fallback 时请求体未正确 rewind | 复用现有 `resetRequestForRetry` 逻辑 | +| 无限循环 | fallback 有 attempts 数量上限，且不会把 2xx/3xx 纳入集合 | + +**回滚**：该设计修改的是 BFE 二进制行为。若线上需要恢复旧行为，必须回滚代码并重新发版；但影响面可控（仅涉及 AI 路由且配置了 fallback cluster 的场景）。 + +--- + +## 10. 关键代码索引 + +| 文件 | 行号范围 | 说明 | +|---|---|---| +| `bfe/bfe_server/reverseproxy.go` | 1099-1103 | `aiForwardAttempt` 结构 | +| `bfe/bfe_server/reverseproxy.go` | 1279-1310 | 外层 fallback 循环 | +| `bfe/bfe_server/reverseproxy.go` | 1540-1661 | `aiClusterInvoke` key 级重试 | +| `bfe/bfe_server/reverseproxy.go` | 1664-1672 | `shouldTriggerFallback` | +| `bfe/bfe_server/reverseproxy_ai_test.go` | 73-92 | 现有 `TestShouldTriggerFallback` | diff --git a/docs/zh_cn/modifications/2026-08-19-fix-rmb-quota-streaming-deduction/design-changes.md b/docs/zh_cn/modifications/2026-08-19-fix-rmb-quota-streaming-deduction/design-changes.md new file mode 100644 index 000000000..0ff57dbc2 --- /dev/null +++ b/docs/zh_cn/modifications/2026-08-19-fix-rmb-quota-streaming-deduction/design-changes.md @@ -0,0 +1,425 @@ +# 修复 BFE RMB 配额在流式响应（SSE）下不扣费的问题 + +## 1. 背景 + +BFE 在 AI 网关场景下支持两种配额维度： + +- **Token 配额**：按 `total_tokens` / `prompt_tokens` + `completion_tokens` 扣减。 +- **RMB 配额**：按模型定价表计算的输入/输出成本扣减（单位 RMB，内部使用定点数）。 + +当前线上配置中，RMB 配额适用于部分产品和模型。当客户端以 `stream: true` 调用大模型接口时，上游返回 `text/event-stream`（SSE）流式响应。用户观察到： + +- 上游 DeepSeek 已实际扣费； +- BFE 侧 Redis 中仅出现认证阶段的 `GET QUOTA_...` 余额检查； +- 请求结束后没有 `DECRBY` 扣减动作。 + +该问题已被记录为 GitHub issue：https://github.com/bfenetworks/bfe/issues/1316 + +--- + +## 2. 问题现象 + +### 2.1 用户配置 + +`token_rule.data`： + +```json +{ + "Id": "AI_product-ZEAoKAKdGnPpck1uPoUsdNCb", + "Unit": "RMB", + "Quota": 500000000, + "RedisKey": "QUOTA_AI_product-ZEAoKAKdGnPpck1uPoUsdNCb", + ... +} +``` + +`cluster_conf.data`： + +```json +"deepseek-backup": { + "AIConf": { + "Keys": [...], + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Model": "deepseek-v4-flash", + "BaseModel": "deepseek-v4-flash", + "Mode": "chat", + "Prices": { + "input_cost_per_token": 0.000003, + "output_cost_per_token": 0.000009, + ... + } + } + ] + } + } +} +``` + +### 2.2 观测到的 Redis 行为 + +``` +1787130732.988709 [0 172.19.1.222:51706] "GET" "QUOTA_AI_product-ZEAoKAKdGnPpck1uPoUsdNCb" +``` + +只有一条 `GET`，没有 `DECRBY`。 + +--- + +## 3. 根因分析 + +### 3.1 RMB 成本计算被 `ContentLength >= 0` 条件卡住 + +`bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go:139-151`： + +```go +func (m *ModuleAITokenAuth) tokenReadResponseHandler(req *bfe_basic.Request, res *bfe_http.Response) int { + ctx := GetTokenAuthContext(req) + if ctx == nil { + return bfe_module.BfeHandlerGoOn + } + tokenUsage := ctx.aiBasicInfo.GetTokenUsage() + if res.StatusCode == bfe_http.StatusOK && res.ContentLength >= 0 { // ← 问题在这里 + if bodyAccessor, err := res.GetBodyAccessor(); err == nil { + body, _ := bodyAccessor.GetBytes() + UpdateCtxByUsage(ctx, body) + } + if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { + tokenUsage.CompletionTokens = int64(res.ContentLength) / 4 + tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + // calculate RMB cost while SvrDataConf is still available + if hasRMBPlan(ctx.Token.QuotaPlans) { + tokenUsage.UsedCost = m.calcCostUnits(req, ctx.serverConf, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + } + + return bfe_module.BfeHandlerGoOn +} +``` + +`RMB` 成本计算（`calcCostUnits`）和响应体 `usage` 解析（`UpdateCtxByUsage`）都被包在 `res.ContentLength >= 0` 条件内部。 + +### 3.2 流式响应的 `ContentLength` 恒为 `-1` + +`bfe/bfe_modules/mod_body_process/body_process.go:343-345`： + +```go +res.Body = bp +res.ContentLength = -1 // 设置为-1表示不确定长度 +res.Header.Del("Content-Length") +``` + +只要请求经过 `mod_body_process`（生产环境通常启用），或者上游返回 chunked/SSE 响应，`ContentLength` 就是 `-1`。结果： + +- `UpdateCtxByUsage` 不会执行； +- `calcCostUnits` 不会执行； +- `tokenUsage.UsedCost` 保持 `0`。 + +### 3.3 完成阶段只扣已计算的 `UsedCost` + +`bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go:165-208`： + +```go +func (m *ModuleAITokenAuth) tokenRequestFinishHandler(req *bfe_basic.Request, res *bfe_http.Response) int { + ... + tokenUsage := ctx.aiBasicInfo.GetTokenUsage() + if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { + tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + + // use RMB cost calculated at response-read stage (SvrDataConf may be nil here) + costUnits := tokenUsage.UsedCost // 流式下为 0 + + if tokenUsage.UsedQuota > 0 || costUnits > 0 { + for _, plan := range ctx.Token.QuotaPlans { + if plan.Unlimited { + continue + } + if quota.IsRMB(plan.Unit) { + if costUnits > 0 { // 永远不成立 + _, err := plan.Deduct(m.redisClient, costUnits) + ... + } + } else { + if tokenUsage.UsedQuota > 0 { + _, err := plan.Deduct(m.redisClient, tokenUsage.UsedQuota) + ... + } + } + } + } + + return bfe_module.BfeHandlerGoOn +} +``` + +完成阶段不再重新计算成本，而是直接使用 `tokenUsage.UsedCost`。对于流式响应，该值为 `0`，因此 `quota.IsRMB(plan.Unit)` 分支不会触发 `Deduct`。 + +### 3.4 `mod_body_process` 已经收集了流式 token 用量 + +`bfe/bfe_modules/mod_body_process/content_quota_usage.go:27-75`： + +```go +type QuotaUsageProcessor struct { + aiBasicInfo *bfe_basic.AiBasicInfo +} + +func NewQuotaUsageProcessor(req *bfe_basic.Request, res *bfe_http.Response) *QuotaUsageProcessor { + if res.StatusCode != bfe_http.StatusOK { + return nil + } + aiBasicInfo := req.GetAiBasicInfo() + return &QuotaUsageProcessor{aiBasicInfo: aiBasicInfo} +} + +func (caf *QuotaUsageProcessor) Process(events []Event) ([]Event, error) { + tctx := caf.aiBasicInfo.GetTokenUsage() + for _, ev := range events { + ... + if !rquota.IsGuess { + if rquota.UsedQuota > 0 { + tctx.CompletionTokens = rquota.CompletionTokens + tctx.PromptTokens = rquota.PromptTokens + tctx.UsedQuota = rquota.UsedQuota + } else if rquota.PromptTokens > 0 || rquota.CompletionTokens > 0 { + tctx.UsedQuota = rquota.PromptTokens + rquota.CompletionTokens + tctx.PromptTokens = rquota.PromptTokens + tctx.CompletionTokens = rquota.CompletionTokens + } + } + ... + } + return events, nil +} +``` + +也就是说，流式场景下 `PromptTokens` / `CompletionTokens` 在请求结束时已经可用，只是没有被用来计算 `UsedCost`。 + +--- + +## 4. 目标 + +1. 修复流式响应（SSE）下 RMB 配额不扣费的问题。 +2. 保持非流式响应的 RMB 扣费行为不变。 +3. 保持 Token 配额的扣费行为不变。 +4. 降低对模块加载顺序和 `ContentLength` 的依赖。 +5. 补充单元测试覆盖流式 + RMB 扣费路径。 + +--- + +## 5. 变更方案 + +### 5.1 核心改动 + +将 RMB 成本计算从 `tokenReadResponseHandler` 移到 `tokenRequestFinishHandler`，并在完成阶段基于此时已填充的 `PromptTokens` / `CompletionTokens` 进行计算。 + +**文件：** `bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go` + +#### 5.1.1 `tokenReadResponseHandler` + +保留响应体 `usage` 解析和 token 估算逻辑（供非流式/未启用 `mod_body_process` 场景使用），但**不再**在此处计算 `UsedCost`。 + +```go +func (m *ModuleAITokenAuth) tokenReadResponseHandler(req *bfe_basic.Request, res *bfe_http.Response) int { + ctx := GetTokenAuthContext(req) + if ctx == nil { + return bfe_module.BfeHandlerGoOn + } + tokenUsage := ctx.aiBasicInfo.GetTokenUsage() + if res.StatusCode == bfe_http.StatusOK && res.ContentLength >= 0 { + if bodyAccessor, err := res.GetBodyAccessor(); err == nil { + body, _ := bodyAccessor.GetBytes() + UpdateCtxByUsage(ctx, body) + } + if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { + tokenUsage.CompletionTokens = int64(res.ContentLength) / 4 + tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + } + + return bfe_module.BfeHandlerGoOn +} +``` + +#### 5.1.2 `tokenRequestFinishHandler` + +在请求完成阶段统一计算 RMB 成本。 + +```go +func (m *ModuleAITokenAuth) tokenRequestFinishHandler(req *bfe_basic.Request, res *bfe_http.Response) int { + if res == nil || res.StatusCode != bfe_http.StatusOK { + return bfe_module.BfeHandlerGoOn + } + + ctx := GetTokenAuthContext(req) + if ctx == nil { + return bfe_module.BfeHandlerGoOn + } + + tokenUsage := ctx.aiBasicInfo.GetTokenUsage() + if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { + tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + + // 统一在请求完成阶段计算 RMB 成本 + if tokenUsage.UsedCost <= 0 && hasRMBPlan(ctx.Token.QuotaPlans) { + tokenUsage.UsedCost = m.calcCostUnits(req, ctx.serverConf, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + + costUnits := tokenUsage.UsedCost + + if tokenUsage.UsedQuota > 0 || costUnits > 0 { + for _, plan := range ctx.Token.QuotaPlans { + if plan.Unlimited { + continue + } + if quota.IsRMB(plan.Unit) { + if costUnits > 0 { + _, err := plan.Deduct(m.redisClient, costUnits) + if err != nil { + log.Logger.Warn("deduct rmb quota failed: %v", err) + } + } + } else { + if tokenUsage.UsedQuota > 0 { + _, err := plan.Deduct(m.redisClient, tokenUsage.UsedQuota) + if err != nil { + log.Logger.Warn("deduct token quota failed: %v", err) + } + } + } + } + } + + return bfe_module.BfeHandlerGoOn +} +``` + +### 5.2 为什么 `ctx.serverConf` 在 finish 阶段可用 + +`SetTokenAuthContext` 中已经缓存了 `req.SvrDataConf`： + +```go +func SetTokenAuthContext(req *bfe_basic.Request, tok *Token, promptToken int64, tags []bfe_basic.ApikeyTag) { + ... + tokenCtx := &TokenAuthContext{ + Token: tok, + aiBasicInfo: aiBasicInfo, + serverConf: req.SvrDataConf, + } + req.SetContext(REQ_TOKEN_AUTH_CONTEXT, tokenCtx) +} +``` + +`TokenAuthContext.serverConf` 的注释也明确说明： + +```go +// serverConf caches the SvrDataConf before it is cleared by the reverse proxy. +// It is used for RMB cost calculation at request finish time. +``` + +因此将 `calcCostUnits` 移到 finish 阶段在设计上完全可行。 + +### 5.3 边界场景 + +| 场景 | 处理结果 | +|------|---------| +| 非流式，有 `usage` | `tokenReadResponseHandler` 解析 usage；finish 阶段用已有 Prompt/Completion 计算成本并扣费。 | +| 非流式，无 `usage`，开启估算 | `tokenReadResponseHandler` 用 `ContentLength/4` 估算；finish 阶段计算成本。 | +| 流式，有最终 `usage` chunk | `mod_body_process` 解析并填充 Prompt/Completion；finish 阶段计算成本。 | +| 流式，无 `usage`，开启估算 | `mod_body_process` 累加估算 completion tokens；finish 阶段计算成本。 | +| 流式/非流式，无 RMB plan | `hasRMBPlan` 为 false，不调用 `calcCostUnits`，行为不变。 | +| 无 `serverConf` | `calcCostUnits` 内部返回 0，不会误扣费。 | + +--- + +## 6. 附加脆弱点（建议同步关注，但不纳入本次最小修复） + +### 6.1 `LookupModelPrice` 硬编码 `"chat"` + +`mod_ai_token_auth.go:434`： + +```go +entry := cluster_conf.LookupModelPrice(cluster.AIConf.ModelTable, targetModel, "chat") +``` + +如果模型定价表中的 `Mode` 不是 `chat`，或者请求实际不是 chat completion，价格会找不到，`UsedCost` 为 0，且无日志提示。建议后续根据实际请求类型或模型表中的 `Mode` 匹配。 + +### 6.2 依赖响应体 `usage` 字段 + +当 `EstimateToken = false` 且上游未返回 `usage` 时，Token 和 RMB 扣费都会是 0。这是当前设计行为，但在生产环境中容易静默漏扣。 + +### 6.3 模块加载顺序 + +BFE handler 链按 `AddFilter` 注册顺序执行。若某部署把 `mod_body_process` 放在 `mod_ai_token_auth` 之前，`tokenReadResponseHandler` 看到的 `ContentLength` 已经是 `-1`，连非流式 RMB 扣费也会失败。将 `calcCostUnits` 移到 finish 阶段后，可在一定程度上降低对模块顺序的依赖。 + +--- + +## 7. 测试计划 + +### 7.1 单元测试 + +新增/修改 `bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth_test.go`： + +1. **`TestTokenRequestFinishHandler_RMB_Streaming`**： + - 构造 `TokenAuthContext`，`PromptTokens` 和 `CompletionTokens` 已填充； + - 使用 mock Redis 客户端； + - 验证 RMB plan 被正确扣减。 + +2. **`TestTokenRequestFinishHandler_RMB_NonStreaming`**： + - 模拟非流式响应，`ContentLength >= 0`； + - 验证 finish 阶段仍然正确扣减。 + +3. **`TestTokenReadResponseHandler_NoCostCalculation`**： + - 验证 `tokenReadResponseHandler` 执行后 `UsedCost` 仍为 0（不再提前计算）。 + +### 7.2 集成测试 + +扩展现有 SC03 RMB 配额集成测试（`bfe/tests/integration/implementation/scenario-SC03-rmb-quota/`）： + +1. 在 `testdata/bfe.conf` 中加载 `mod_body_process`，并新增 `testdata/mod_body_process/` 配置； +2. 扩展 `tests/integration/common/mock_backend.go`，支持通过 `ResponseHeaders` 设置响应头（如 `Content-Type: text/event-stream`）； +3. 新增 `TestTC07_RMBQuotaDeduction_Streaming`： + - 模拟 `stream: true` 请求； + - 后端返回 SSE 流，最后一个 chunk 包含 `usage`； + - 验证请求成功后 Redis 中 RMB 配额被正确扣减（`100*input_cost + 50*output_cost`）。 + +### 7.3 回归测试 + +- 跑 `bfe/bfe_modules/mod_ai_token_auth/...` 单元测试； +- 跑 `bfe/bfe_modules/mod_body_process/...` 单元测试； +- 跑 `bfe/bfe_config/bfe_cluster_conf/cluster_conf/...` 相关测试。 + +--- + +## 8. 影响范围 + +| 模块/文件 | 影响 | +|-----------|------| +| `bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go` | 核心修复 | +| `bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth_test.go` | 新增单元测试 | +| `bfe/tests/integration/common/mock_backend.go` | 新增 `ResponseHeaders`，支持 SSE 测试 | +| `bfe/tests/integration/implementation/scenario-SC03-rmb-quota/...` | 新增流式 RMB 扣费集成测试，启用 `mod_body_process` | +| `bfe/docs/zh_cn/modifications/...` | 新增本文档 | + +--- + +## 9. 兼容性 + +- 非流式请求的 RMB 扣费行为保持一致，仅计算位置后移。 +- Token 配额扣费逻辑不变。 +- 不修改配置格式，不修改 Redis key 结构。 +- 对未启用 RMB plan 的产品无影响。 + +--- + +## 10. 参考资料 + +- GitHub issue：https://github.com/bfenetworks/bfe/issues/1316 +- `bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go` +- `bfe/bfe_modules/mod_body_process/body_process.go` +- `bfe/bfe_modules/mod_body_process/content_quota_usage.go` +- `bfe/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go` diff --git a/docs/zh_cn/modifications/2026-08-19-update-ai-access-log-fields/design-changes.md b/docs/zh_cn/modifications/2026-08-19-update-ai-access-log-fields/design-changes.md new file mode 100644 index 000000000..b69ac55b7 --- /dev/null +++ b/docs/zh_cn/modifications/2026-08-19-update-ai-access-log-fields/design-changes.md @@ -0,0 +1,454 @@ +# BFE 适配 bfe-access-pb AI 可观测字段升级设计变更 + +## 1. 背景 + +`bfe-access-pb` 访问日志协议已完成 AI 可观测字段的扩展与重命名（详见 `bfe-access-pb/docs/protobuf.md`）。主要变化包括： + +1. **字段重命名**（保持编号不变）： + - `ai_apikey` → `ai_apikey_id`（记录 API Key 内部标识，不记录原始 key 值） + - `ai_mapped_model` → `ai_target_model` + - `ai_prompt_tokens` → `ai_input_tokens` +2. **新增字段**： + - `ai_provider`：上游模型提供商标识 + - `ai_retry_count`：模型调用层重试次数 + - `ai_cost_value` / `ai_cost_currency`：RMB 成本计量 + - `ai_route_rule_hits`：命中的 AI 路由规则列表 + - `ai_cluster_key_names`：请求处理过程中尝试过的 (cluster, key) 列表 + - `ai_auth_hit_quota_plans`：正常请求时命中的 Quota Plan ID 列表 +3. **新增辅助消息**： + - `AIRouteRuleHit`：描述命中路由规则的 owner / owner_type / name + - `ClusterKeyName`：描述尝试过的 cluster 与 key 名称组合 + +BFE 当前仍使用旧版协议（`github.com/bfenetworks/bfe-access-pb v0.1.0`），并且访问日志代码中引用的是旧字段名（`AiApikey`、`AiMappedModel`、`AiPromptTokens`）。为使 BFE 与新版协议对齐，需要在 BFE 侧进行配套改造。 + +--- + +## 2. 目标 + +1. 将 BFE 依赖的 `bfe-access-pb` 升级到包含新字段的版本（建议 `v0.2.0` 或本地 replace）。 +2. 修正访问日志中对重名字段的引用。 +3. 将 `ai_apikey_id` 的数据源从原始 API Key 改为 API Key 内部 ID（`Token.KeyId` / `AiBasicInfo.ClientKeyId`）。 +4. 在 BFE 各 AI 模块中补充新增字段的采集逻辑。 +5. 更新单元测试 `bfe_modules/mod_access_pb3/request_log_test.go`。 +6. 保持非 AI 请求和未升级配置场景下的向后兼容。 + +--- + +## 3. 变更总览 + +| Proto 字段 | 编号 | 旧 BFE 字段/状态 | 新 BFE 数据源 | 需修改的文件 | +|------------|------|------------------|---------------|--------------| +| `ai_apikey_id` | 701 | `AiApikey`（记录 `ClientApiKey`） | `AiBasicInfo.ClientKeyId` | `request_log.go` | +| `ai_apikeytags` | 702 | 已支持 | `AiBasicInfo.ApikeyTags` | 不变 | +| `ai_requested_model` | 703 | 已支持 | `AiBasicInfo.ClientModel` | 不变 | +| `ai_target_model` | 704 | `AiMappedModel` | `AiBasicInfo.TargetModel` | `request_log.go` | +| `ai_stream` | 705 | 已支持 | `req.IsSse` | 不变 | +| `ai_input_tokens` | 706 | `AiPromptTokens` | `TokenUsage.PromptTokens` | `request_log.go` | +| `ai_output_tokens` | 707 | 已支持 | `TokenUsage.CompletionTokens` | 不变 | +| `ai_total_tokens` | 708 | 已支持 | `TokenUsage.UsedQuota` | 不变 | +| `ai_ttft_us` | 709 | 已支持 | `TokenTimeInfo.TTFT` | 不变 | +| `ai_tpot_us` | 710 | 已支持 | `TokenTimeInfo.TPOT` | 不变 | +| `ai_rate_limit_hits` | 711 | 已支持 | `AiRateLimitHitInfo` | 不变 | +| `ai_auth_reject_reason` | 712 | 已支持 | `AiAuthInfo.RejectReason` | 不变 | +| `ai_auth_reject_quota_plans` | 713 | 已支持 | `AiAuthInfo.RejectQuotaPlans` | 不变 | +| `ai_provider` | 714 | **缺失** | `cluster.AIConf.Provider` | `request_ai_basic.go`, `reverseproxy.go` | +| `ai_retry_count` | 715 | **缺失** | `AiBasicInfo.RetryCount` | `request_ai_basic.go`, `reverseproxy.go` | +| `ai_cost_value` | 761 | **缺失** | `TokenUsage.UsedCost` | `request_log.go` | +| `ai_cost_currency` | 762 | **缺失** | `cluster.AIConf.ModelTable.Currency` | `request_ai_basic.go`, `reverseproxy.go`, `request_log.go` | +| `ai_route_rule_hits` | 801 | **缺失** | `AiRouteResult` → `AIRouteRuleHit` | `request_ai_route.go`, `request_log.go` | +| `ai_cluster_key_names` | 802 | **缺失** | `AiBasicInfo.ClusterKeyNames` | `request_ai_basic.go`, `reverseproxy.go`, `request_log.go` | +| `ai_auth_hit_quota_plans` | 841 | **缺失** | `AiAuthInfo.HitQuotaPlans` | `request_ai_basic.go`, `token_rule_table.go`, `request_log.go` | + +--- + +## 4. 详细设计 + +### 4.1 升级 bfe-access-pb 依赖 + +**文件：** `bfe/go.mod` + +将依赖版本从 `v0.1.0` 升级到包含新字段的版本（例如 `v0.2.0`）： + +```go +require ( + ... + github.com/bfenetworks/bfe-access-pb v0.2.0 + ... +) +``` + +本地开发时可临时启用 replace： + +```go +replace github.com/bfenetworks/bfe-access-pb => ../bfe-access-pb +``` + +升级后执行： + +```bash +cd bfe +go mod tidy +go mod download +``` + +> 注意：`bfe-access-pb` 的 `.pb.go` 文件需要在 Linux 环境下执行 `build.sh` 重新生成并打 tag 后，BFE 才能引用到正确版本。 + +--- + +### 4.2 扩展 `AiBasicInfo` 结构 + +**文件：** `bfe/bfe_basic/request_ai_basic.go` + +在 `AiBasicInfo` 中新增以下字段： + +```go +type AiBasicInfo struct { + ClientApiKey string + ClientKeyId string + ClientModel string + TargetModel string + Provider string // 新增：上游 provider，如 openai / deepseek + RetryCount uint32 // 新增：模型调用层重试次数 + CostCurrency string // 新增：成本币种，如 RMB / USD + tokenUsage TokenUsage + ApikeyTags []ApikeyTag + TokenTimeInfo TokenTimeInfo + AiAuthInfo AiAuthInfo + ClusterKeyNames []ClusterKeyName // 新增：尝试过的 (cluster, key) 列表 + + allowEstimateToken bool +} + +// ClusterKeyName 描述一次尝试的 cluster 与 key 名称组合 +type ClusterKeyName struct { + ClusterName string + KeyName string +} +``` + +新增辅助方法： + +```go +func (aiinfo *AiBasicInfo) AppendClusterKeyName(clusterName, keyName string) { + aiinfo.ClusterKeyNames = append(aiinfo.ClusterKeyNames, ClusterKeyName{ + ClusterName: clusterName, + KeyName: keyName, + }) +} + +func (aiinfo *AiBasicInfo) IncrementRetryCount() { + aiinfo.RetryCount++ +} +``` + +--- + +### 4.3 扩展 `AiAuthInfo` 结构 + +**文件：** `bfe/bfe_basic/request_ai_basic.go` + +在 `AiAuthInfo` 中新增 `HitQuotaPlans`，用于记录成功鉴权时参与余额检查并放行的 Quota Plan ID： + +```go +type AiAuthInfo struct { + RejectReason string // 拒绝原因 + RejectQuotaPlans []string // 拒绝时余额不足的 Quota Plan IDs + HitQuotaPlans []string // 新增：成功时命中的 Quota Plan IDs +} +``` + +--- + +### 4.4 模块数据填充修改 + +#### 4.4.1 `mod_ai_token_auth`：记录命中配额计划 + +**文件：** `bfe/bfe_modules/mod_ai_token_auth/token_rule_table.go` + +在 `ValidateUserTokenByReq` 中，当 Quota Plan 通过余额检查（`hasBalance == true`）时，将其 ID 记录到 `AiAuthInfo.HitQuotaPlans`： + +```go +// 在 for _, plan := range token.QuotaPlans 循环内 +if !hasBalance { + SetAiAuthInfo(req, bfe_basic.CodeQuotaExhausted, []string{plan.Id}) + ... +} +// 新增：记录成功命中的 quota plan +aiBasicInfo := req.GetAiBasicInfo() +if aiBasicInfo != nil { + aiBasicInfo.AiAuthInfo.HitQuotaPlans = append(aiBasicInfo.AiAuthInfo.HitQuotaPlans, plan.Id) +} +``` + +> 说明：`Unlimited` 或 `PassNoQuota` 的 plan 不经过 `HasBalance` 检查，因此不会进入 `HitQuotaPlans`。这符合语义：`HitQuotaPlans` 仅记录实际参与余额校验并命中的计划。 + +#### 4.4.2 `bfe_server/reverseproxy.go`：记录 provider、currency、retry、cluster/key + +**文件：** `bfe/bfe_server/reverseproxy.go` + +**A. 在 `doSingleAIForward` 中记录 provider、currency 和 cluster/key：** + +```go +func (p *ReverseProxy) doSingleAIForward(..., selectedKey cluster_conf.AIKey) (...) { + ... + if cluster.AIConf != nil && aiMeta != nil { + if cluster.AIConf.Provider != "" { + aiMeta.Provider = cluster.AIConf.Provider + } + if cluster.AIConf.ModelTable != nil && cluster.AIConf.ModelTable.Currency != "" { + aiMeta.CostCurrency = cluster.AIConf.ModelTable.Currency + } + aiMeta.AppendClusterKeyName(cluster.Name, selectedKey.Name) + } + ... +} +``` + +> 注意：需要确认 `cluster` 对象是否有 `Name` 字段；如果没有，使用 `attempt.ClusterName`。 + +**B. 在 `aiClusterInvoke` 中统计重试次数：** + +```go +for retry := 0; retry <= policy.MaxRetries; retry++ { + if retry > 0 { + if aiMeta != nil { + aiMeta.IncrementRetryCount() + } + ... + } + ... +} +``` + +> 说明：`RetryCount` 只统计同一 cluster 内 key-level 的重试次数，与 HTTP 层 `basicReq.RetryTime` 解耦。fallback 到另一个 cluster 时，该计数器不累加（符合协议语义）。 + +--- + +### 4.5 访问日志赋值修改 + +**文件：** `bfe/bfe_modules/mod_access_pb3/request_log.go` + +将 `reqAiInfoGen` 函数更新为使用新字段名并填充新增字段： + +```go +func reqAiInfoGen(reqLog *bfe_access_pb3.RequestLog, req *bfe_basic.Request, res *bfe_http.Response) { + aiInfo := req.GetAiBasicInfo() + if aiInfo == nil { + return + } + + // API Key ID（不再记录原始 key） + if aiInfo.ClientKeyId != "" { + reqLog.AiApikeyId = proto.String(aiInfo.ClientKeyId) + } + + // API Key Tags + if len(aiInfo.ApikeyTags) > 0 { + for _, tag := range aiInfo.ApikeyTags { + reqLog.AiApikeytags = append(reqLog.AiApikeytags, &bfe_access_pb3.ApikeyTag{ + Tagname: proto.String(tag.TagName), + Tagvalue: proto.String(tag.TagValue), + }) + } + } + + // Model + if aiInfo.ClientModel != "" { + reqLog.AiRequestedModel = proto.String(aiInfo.ClientModel) + } + if aiInfo.TargetModel != "" { + reqLog.AiTargetModel = proto.String(aiInfo.TargetModel) + } + + // Provider + if aiInfo.Provider != "" { + reqLog.AiProvider = proto.String(aiInfo.Provider) + } + + // Stream + reqLog.AiStream = proto.Bool(isStreamResponse(req, res)) + + // Token usage + usage := aiInfo.GetTokenUsage() + if usage != nil { + reqLog.AiInputTokens = proto.Int64(usage.PromptTokens) + reqLog.AiOutputTokens = proto.Int64(usage.CompletionTokens) + reqLog.AiTotalTokens = proto.Int64(usage.UsedQuota) + if usage.UsedCost > 0 { + reqLog.AiCostValue = proto.Int64(usage.UsedCost) + } + } + + // Cost currency + if aiInfo.CostCurrency != "" { + reqLog.AiCostCurrency = proto.String(aiInfo.CostCurrency) + } + + // Retry count + if aiInfo.RetryCount > 0 { + reqLog.AiRetryCount = proto.Uint32(aiInfo.RetryCount) + } + + // TTFT / TPOT + ti := aiInfo.TokenTimeInfo + if ti.TTFT > 0 { + reqLog.AiTtftUs = proto.Int64(ti.TTFT) + } + if ti.TPOT > 0 { + reqLog.AiTpotUs = proto.Int64(ti.TPOT) + } + + // Auth reject info + if len(aiInfo.AiAuthInfo.RejectReason) > 0 { + reqLog.AiAuthRejectReason = proto.String(aiInfo.AiAuthInfo.RejectReason) + } + for _, item := range aiInfo.AiAuthInfo.RejectQuotaPlans { + reqLog.AiAuthRejectQuotaPlans = append(reqLog.AiAuthRejectQuotaPlans, item) + } + for _, item := range aiInfo.AiAuthInfo.HitQuotaPlans { + reqLog.AiAuthHitQuotaPlans = append(reqLog.AiAuthHitQuotaPlans, item) + } + + // Route rule hits + if routeResult := req.GetAiRouteResult(); routeResult != nil { + reqLog.AiRouteRuleHits = append(reqLog.AiRouteRuleHits, &bfe_access_pb3.AIRouteRuleHit{ + RuleOwner: proto.String(routeResult.Owner), + RuleOwnerType: proto.String(routeResult.RouteType), + RuleName: proto.String(routeResult.RuleName), + }) + } + + // Cluster / key attempts + for _, ckn := range aiInfo.ClusterKeyNames { + reqLog.AiClusterKeyNames = append(reqLog.AiClusterKeyNames, &bfe_access_pb3.ClusterKeyName{ + ClusterName: proto.String(ckn.ClusterName), + KeyName: proto.String(ckn.KeyName), + }) + } + + // Rate limit hit info（保持不变） + hitInfo := req.GetAiRateLimitHitInfo() + if hitInfo != nil && len(hitInfo.HitPolicyDict) > 0 { + for policyId, info := range hitInfo.HitPolicyDict { + ... + } + } +} +``` + +--- + +## 5. 涉及文件清单 + +| 文件 | 修改内容 | +|------|----------| +| `bfe/go.mod` | 升级 `bfe-access-pb` 到 `v0.2.0`（或启用 replace） | +| `bfe/go.sum` | 随 `go mod tidy` 自动更新 | +| `bfe/bfe_basic/request_ai_basic.go` | `AiBasicInfo` 新增 `Provider`、`RetryCount`、`CostCurrency`、`ClusterKeyNames`；新增 `ClusterKeyName` 结构及辅助方法；`AiAuthInfo` 新增 `HitQuotaPlans` | +| `bfe/bfe_basic/request_ai_route.go` | 可选：为 `AiRouteResult` 增加导出方法，便于 `request_log.go` 读取 | +| `bfe/bfe_modules/mod_ai_token_auth/token_rule_table.go` | `ValidateUserTokenByReq` 中记录成功命中的 `HitQuotaPlans` | +| `bfe/bfe_server/reverseproxy.go` | `doSingleAIForward` 记录 provider / currency / cluster-key；`aiClusterInvoke` 统计 retry count | +| `bfe/bfe_modules/mod_access_pb3/request_log.go` | `reqAiInfoGen` 使用新字段名并填充新增字段 | +| `bfe/bfe_modules/mod_access_pb3/request_log_test.go` | 更新测试断言，覆盖新字段 | + +--- + +## 6. 测试计划 + +### 6.1 单元测试 + +**文件：** `bfe/bfe_modules/mod_access_pb3/request_log_test.go` + +更新 `TestReqAiInfoGen`： + +1. 将 `ClientApiKey` 替换为 `ClientKeyId`，并断言 `AiApikeyId`。 +2. 将 `AiMappedModel` 断言改为 `AiTargetModel`。 +3. 将 `AiPromptTokens` 断言改为 `AiInputTokens`。 +4. 新增断言： + - `AiProvider` + - `AiRetryCount` + - `AiCostValue` / `AiCostCurrency` + - `AiRouteRuleHits` + - `AiClusterKeyNames` + - `AiAuthHitQuotaPlans` + +示例补充： + +```go +aiInfo := &bfe_basic.AiBasicInfo{ + ClientKeyId: "key-id-123", + ClientModel: "model-a", + TargetModel: "model-b", + Provider: "deepseek", + RetryCount: 1, + CostCurrency: "RMB", + ClusterKeyNames: []bfe_basic.ClusterKeyName{ + {ClusterName: "cluster-a", KeyName: "key-001"}, + }, + ... +} +usage.UsedCost = 5000 // 1e-8 元 +``` + +### 6.2 编译验证 + +```bash +cd bfe +go build ./... +go test ./bfe_modules/mod_access_pb3/... +``` + +### 6.3 集成验证 + +1. 启用 AI 网关，发起一次带 API Key 的模型请求。 +2. 收集 `mod_access_pb3` 输出的 b2log，解码 `RequestLog`。 +3. 校验字段： + - `ai_apikey_id` 等于 Token 的 `key_id`，而不是原始 `key`。 + - `ai_target_model` 正确反映路由/映射后的模型。 + - `ai_provider`、`ai_retry_count`、`ai_cost_value`、`ai_cost_currency` 非空（RMB 配额场景）。 + - `ai_route_rule_hits`、`ai_cluster_key_names`、`ai_auth_hit_quota_plans` 与请求行为一致。 + +--- + +## 7. 兼容性说明 + +1. **字段重命名**：proto 字段编号不变（701、704、706），因此 protobuf 二进制层面完全兼容；变化只体现在生成代码的 Go 字段名上。 +2. **语义变化**：`ai_apikey_id` 从记录原始 API Key 改为记录内部 `key_id`，避免在日志中泄露敏感信息。需要确认上游日志消费方不再依赖原始 key 值。 +3. **新增字段**：均为 `optional`，对未升级的旧 BFE 版本无影响。 +4. **版本依赖**：升级 `bfe-access-pb` 后，旧 BFE 代码无法直接编译，因此这是一个需要同步发布的破坏性变更（仅对 BFE 代码编译层面）。 + +--- + +## 8. 风险与回滚 + +### 8.1 主要风险 + +| 风险 | 说明 | 规避措施 | +|------|------|----------| +| 编译失败 | 新 proto 字段名与旧 BFE 代码不匹配 | 按本方案一次性更新所有引用 | +| 日志消费方依赖旧字段名 | 下游解析 `ai_apikey`、`ai_mapped_model`、`ai_prompt_tokens` 会失败 | 提前通知下游，按 proto 编号而非字段名解析；或在下游做映射 | +| `ai_apikey_id` 为空 | 如果 `Token.KeyId` 未配置，日志中将缺失 key 标识 | 确保 `ai-gateway-api` 导出的 Token 配置始终包含 `key_id` | +| 重试计数语义不清 | `RetryCount` 仅统计 key-level 重试，不统计 cluster fallback | 文档中明确语义；访问日志中已有 `backend_retry` 字段记录 HTTP 层重试 | + +### 8.2 回滚方案 + +如需回滚到旧协议： + +1. 将 `bfe/go.mod` 中的 `bfe-access-pb` 版本改回 `v0.1.0`。 +2. 回滚 `request_log.go` 到旧字段名（`AiApikey`、`AiMappedModel`、`AiPromptTokens`）。 +3. 移除 `request_ai_basic.go`、`reverseproxy.go`、`token_rule_table.go` 中新增字段的采集逻辑。 +4. 重新编译部署。 + +> 注意：回滚后新字段（provider、retry、cost 等）将不再输出到日志。 + +--- + +## 9. 后续可选扩展 + +1. **`ai_route_rule_hits` 支持多条命中记录**：当前 `AiRouteResult` 只记录最终命中的规则。未来如果路由模块支持记录所有匹配规则，可将 `HitPolicyDict` 式的列表写入日志。 +2. **`ai_cluster_key_names` 区分成功与失败尝试**：当前记录所有尝试；可扩展为标记最终成功的 key。 +3. **`ai_auth_hit_quota_plans` 与 RMB 扣减计划对齐**：当前记录所有通过余额检查的 plan；可与实际扣减计划做交叉验证。 + +--- + +*文档生成日期：2026-08-19* diff --git a/docs/zh_cn/modifications/2026-08-21-optimize-ai-model-body-rewrite/design-changes.md b/docs/zh_cn/modifications/2026-08-21-optimize-ai-model-body-rewrite/design-changes.md new file mode 100644 index 000000000..eed9bd440 --- /dev/null +++ b/docs/zh_cn/modifications/2026-08-21-optimize-ai-model-body-rewrite/design-changes.md @@ -0,0 +1,396 @@ +# 优化 BFE AI 请求 model 字段多次重写 + +## 1. 背景 + +在 `bfe/bfe_server/reverseproxy.go` 的 `doSingleAIForward` 函数中，针对单个 AI 转发尝试，会按顺序对请求体 JSON 中的 `model` 字段进行最多三次改写： + +1. **attempt.Model 覆盖**（line 1491-1502）：将请求体 `model` 设为路由目标/降级指定的模型。 +2. **provider/model 前缀剥离**（line 1504-1507）：调用 `stripProviderPrefix`，按 `cluster.AIConf.MatchPrefix` 剥离前缀后再次改写 `model`。 +3. **ModelMapping 映射**（line 1521-1540）：按集群配置的模型映射表再次改写 `model`。 + +每次改写都通过 `condition.ReqBodyJsonSet` 完成，其内部使用 `sjson.SetBytes` 对请求体做完整的 JSON 解析与序列化： + +```go +// bfe/bfe_basic/condition/primitive.go +func ReqBodyJsonSet(req *bfe_basic.Request, path string, value string) error { + ... + body, _ := bodyAccessor.GetBytes() + newBody, err = sjson.SetBytes(body, path, value) + ... + bodyAccessor.SetBytes(newBody, false) + return nil +} +``` + +三次调用之间存在明显冗余： + +- 同一条请求体被重复解析/序列化最多 3 次。 +- 每次成功后都重复执行 `outreq.ContentLength` 重置逻辑。 +- `stripProviderPrefix` 内部还会额外重置 `basicReq.HttpRequest.ContentLength`。 + +在 AI 网关场景下，请求体通常较大（尤其是携带长 prompt 或多轮对话时），多次 JSON 改写会带来不必要的 CPU 与内存开销。 + +--- + +## 2. 目标 + +1. 将 `doSingleAIForward` 中针对 `model` 字段的多次 `ReqBodyJsonSet` 调用合并为**最多一次**。 +2. 将 `ContentLength` 重置逻辑也收敛到统一位置，避免重复代码。 +3. 保持现有业务行为不变（模型覆盖 → 前缀剥离 → 模型映射的执行顺序与语义）。 +4. 同步更新 `stripProviderPrefix` 单元测试，使其继续覆盖变换逻辑。 + +--- + +## 3. 变更总览 + +| 层级 | 变更点 | 影响文件 | +|---|---|---| +| 转发层 | 合并 `model` 字段改写逻辑，只调用一次 `ReqBodyJsonSet` | `bfe/bfe_server/reverseproxy.go` | +| 工具函数 | `stripProviderPrefix` 改为纯字符串计算函数（不再操作 body / ContentLength） | `bfe/bfe_server/reverseproxy.go` | +| 测试 | 更新 `TestStripProviderPrefix` 系列用例，验证新的纯计算函数 | `bfe/bfe_server/reverseproxy_ai_test.go` | + +--- + +## 4. 详细设计 + +### 4.1 当前逻辑梳理 + +当前 `doSingleAIForward` 中的三段改写逻辑如下： + +```go +// 1) attempt.Model 覆盖 +if attempt.Model != "" && aiMeta != nil { + if err := condition.ReqBodyJsonSet(basicReq, "model", attempt.Model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + aiMeta.TargetModel = attempt.Model + } +} + +// 2) 前缀剥离 +if cluster.AIConf != nil && aiMeta != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + stripProviderPrefix(basicReq, outreq, aiMeta, cluster.AIConf.MatchPrefix) +} + +// 3) ModelMapping 映射 +if cluster.AIConf != nil && aiMeta != nil && cluster.AIConf.ModelMapping != nil { + model := aiMeta.ClientModel + if aiMeta.TargetModel != "" { + model = aiMeta.TargetModel + } + if model != "" { + if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { + if err := condition.ReqBodyJsonSet(basicReq, "model", newModel); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + aiMeta.TargetModel = newModel + } + } + } +} +``` + +三段逻辑存在先后顺序和依赖关系： + +- `attempt.Model` 覆盖成功后，`aiMeta.TargetModel` 被更新。 +- `stripProviderPrefix` 基于 **当前 `aiMeta.TargetModel`（若已设置）或 `aiMeta.ClientModel`** 决定剥离前缀。 +- `ModelMapping` 同样基于 **当前 `aiMeta.TargetModel` 或 `aiMeta.ClientModel`** 查找映射。 + +### 4.2 优化后逻辑 + +在 `doSingleAIForward` 中按原顺序**计算最终 model 值**，然后统一写入请求体： + +```go +// 计算最终需要写入请求体的 model 值 +model := aiMeta.ClientModel +if aiMeta.TargetModel != "" { + model = aiMeta.TargetModel +} + +// 1) attempt.Model 覆盖 +if attempt.Model != "" { + model = attempt.Model +} + +// 2) provider/model 前缀剥离 +if cluster.AIConf != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + if strings.HasPrefix(model, cluster.AIConf.MatchPrefix) { + stripped := strings.TrimPrefix(model, cluster.AIConf.MatchPrefix) + if stripped != "" { + model = stripped + } else { + log.Logger.Warn("Model %s stripped by prefix %s results in empty model, skip stripping", + model, cluster.AIConf.MatchPrefix) + } + } +} + +// 3) ModelMapping 映射 +if cluster.AIConf != nil && cluster.AIConf.ModelMapping != nil && model != "" { + if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { + model = newModel + } +} + +// 统一写入请求体，最多一次 +if model != aiMeta.ClientModel { + if err := condition.ReqBodyJsonSet(basicReq, "model", model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + // outreq body already changed, need reset Content-Length + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + // Also reset the original request's Content-Length so that fallback/retry + // creates a new outreq with consistent body length. + if basicReq.HttpRequest != nil && basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + aiMeta.TargetModel = model + } +} +``` + +设计说明： + +- `model != aiMeta.ClientModel` 作为是否真正需要改写的判断条件。由于 `doSingleAIForward` 开始时 `outreq` 从原始请求复制，请求体中的 `model` 等于 `aiMeta.ClientModel`，因此该判断等价于"body 中的 model 是否需要变更"。 +- 三个变换仍按原有顺序执行，语义与当前代码一致。 +- `ReqBodyJsonSet` 失败时，不再更新 `aiMeta.TargetModel`，与当前行为一致（当前每段逻辑失败时都不更新 TargetModel）。 +- `basicReq.HttpRequest.ContentLength` 的重置从 `stripProviderPrefix` 中上移到统一写入位置，所有需要改写 body 的场景都会触发。 + +### 4.3 `stripProviderPrefix` 重构 + +原函数同时承担"字符串变换"和"body 操作"两个职责。优化后将其拆分为纯字符串计算函数： + +```go +// stripProviderPrefix returns the model string after stripping matchPrefix. +// If the prefix does not match or stripping results in an empty string, it +// returns the original model and false. +func stripProviderPrefix(model string, matchPrefix string) (string, bool) { + if model == "" || !strings.HasPrefix(model, matchPrefix) { + return model, false + } + + stripped := strings.TrimPrefix(model, matchPrefix) + if stripped == "" { + log.Logger.Warn("Model %s stripped by prefix %s results in empty model, skip stripping", + model, matchPrefix) + return model, false + } + + return stripped, true +} +``` + +在 `doSingleAIForward` 中调用： + +```go +if cluster.AIConf != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + if stripped, ok := stripProviderPrefix(model, cluster.AIConf.MatchPrefix); ok { + model = stripped + } +} +``` + +--- + +## 5. 关键代码变更示例 + +### 5.1 `bfe/bfe_server/reverseproxy.go` + +#### 新增/改造 `stripProviderPrefix` + +```go +func stripProviderPrefix(model string, matchPrefix string) (string, bool) { + if model == "" || !strings.HasPrefix(model, matchPrefix) { + return model, false + } + stripped := strings.TrimPrefix(model, matchPrefix) + if stripped == "" { + log.Logger.Warn("Model %s stripped by prefix %s results in empty model, skip stripping", + model, matchPrefix) + return model, false + } + return stripped, true +} +``` + +#### `doSingleAIForward` 中统一 model 改写 + +```go +// apply model override from ai route target/fallback, provider prefix stripping, +// and cluster model mapping in order; then write the final model to request body +// at most once. +model := aiMeta.ClientModel +if aiMeta.TargetModel != "" { + model = aiMeta.TargetModel +} + +if attempt.Model != "" { + model = attempt.Model +} + +if cluster.AIConf != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + if stripped, ok := stripProviderPrefix(model, cluster.AIConf.MatchPrefix); ok { + model = stripped + } +} + +if cluster.AIConf != nil && cluster.AIConf.ModelMapping != nil && model != "" { + if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { + model = newModel + } +} + +if model != aiMeta.ClientModel { + if err := condition.ReqBodyJsonSet(basicReq, "model", model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + if basicReq.HttpRequest != nil && basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + aiMeta.TargetModel = model + } +} +``` + +--- + +## 6. 测试计划 + +### 6.1 单元测试更新 + +`bfe/bfe_server/reverseproxy_ai_test.go` 中现有 `TestStripProviderPrefix`、`TestStripProviderPrefixNoMatch`、`TestStripProviderPrefixEmptyResult` 三个用例直接调用原 `stripProviderPrefix` 并验证 body 改写与 `ContentLength` 重置。 + +优化后需要： + +1. 调整函数签名调用：由 `stripProviderPrefix(req, outreq, aiMeta, prefix)` 改为 `stripProviderPrefix(model, prefix)`。 +2. 验证返回值与 `aiMeta.TargetModel` 的更新预期由调用方决定。 +3. body 改写与 `ContentLength` 重置的验证可下沉到 `doSingleAIForward` 的集成/单元测试中，或新增针对统一改写逻辑的测试。 + +#### 更新后的 `TestStripProviderPrefix` 示例 + +```go +func TestStripProviderPrefix(t *testing.T) { + model := "openrouter/anthropic/claude-sonnet-4.6" + stripped, ok := stripProviderPrefix(model, "openrouter/") + if !ok { + t.Error("expected stripping to succeed") + } + if stripped != "anthropic/claude-sonnet-4.6" { + t.Errorf("expected stripped model anthropic/claude-sonnet-4.6, got %s", stripped) + } +} + +func TestStripProviderPrefixNoMatch(t *testing.T) { + model := "anthropic/claude-sonnet-4.6" + stripped, ok := stripProviderPrefix(model, "openrouter/") + if ok { + t.Error("expected stripping to be skipped when prefix does not match") + } + if stripped != model { + t.Errorf("expected model unchanged, got %s", stripped) + } +} + +func TestStripProviderPrefixEmptyResult(t *testing.T) { + model := "openrouter/" + stripped, ok := stripProviderPrefix(model, "openrouter/") + if ok { + t.Error("expected stripping to be skipped when result is empty") + } + if stripped != model { + t.Errorf("expected model unchanged, got %s", stripped) + } +} +``` + +### 6.2 集成测试 + +现有集成测试已覆盖： + +- SC01：路由表查找、model 覆盖、前缀剥离、ModelMapping、fallback。 +- SC04：provider/model 前缀剥离。 + +优化后这些场景的行为保持不变，因此全量运行即可： + +```bash +cd bfe +go test ./tests/integration/... -v +``` + +### 6.3 回归测试 + +- `go test ./bfe_server/...` 通过。 +- `go test ./bfe_modules/mod_ai_route/...` 通过。 +- `go test ./tests/integration/...` 通过。 + +--- + +## 7. 文档更新 + +本次优化为纯代码实现优化，不引入新的用户配置或外部行为变更。但 `bfe/docs/zh_cn/sys_design` 中有若干文档包含 `doSingleAIForward()` 的旧实现伪代码，已同步更新： + +1. **`bfe/docs/zh_cn/sys_design/multi_api_key.md`** + - 更新 3.2 节 `doSingleAIForward()` 伪代码，体现 model override → prefix stripping → ModelMapping 的统一计算与单次 `ReqBodyJsonSet` 写入。 + +2. **`bfe/docs/zh_cn/sys_design/provider_model_prefix_routing.md`** + - 更新 4.1 节裁剪位置伪代码，说明前缀裁剪现在是统一 model 计算流程中的一步，不再单独调用 `ReqBodyJsonSet`。 + +3. **`bfe/docs/zh_cn/sys_design/mod_ai_route_bfe_changes.md`** + - 更新 7.2 节请求体处理，说明最终 model 计算顺序与单次写入机制。 + +4. **`bfe/docs/zh_cn/modifications/2026-08-21-optimize-ai-model-body-rewrite/design-changes.md`** + - 本设计变更文档（即本文档）。 + +--- + +## 8. 与当前实现的对比 + +| 维度 | 当前实现 | 优化后 | +|---|---|---| +| `ReqBodyJsonSet` 调用次数 | 最多 3 次 | 最多 1 次 | +| JSON 解析/序列化次数 | 最多 3 次 | 最多 1 次 | +| `ContentLength` 重置代码 | 分散在 3 处 | 统一 1 处 | +| `basicReq.HttpRequest.ContentLength` 重置 | 仅在 `stripProviderPrefix` 中 | 所有 body 改写场景统一处理 | +| `stripProviderPrefix` 职责 | 字符串变换 + body 操作 + ContentLength 重置 | 仅字符串变换 | +| 业务语义 | model 覆盖 → 前缀剥离 → 模型映射 | 保持一致 | + +--- + +## 9. 风险与回滚 + +| 风险 | 缓解措施 | +|---|---| +| 统一改写后 `aiMeta.TargetModel` 更新时机变化 | 保持"只有 `ReqBodyJsonSet` 成功才更新 TargetModel"的原则，与原逻辑一致 | +| `basicReq.HttpRequest.ContentLength` 重置范围扩大 | 属于正向修复：原本只有前缀剥离场景会重置，其他 body 改写场景也应重置，以保证 fallback/retry 时 body 长度一致 | +| `model != aiMeta.ClientModel` 判断导致某些边界场景跳过改写 | 由于 `doSingleAIForward` 开始时 `outreq` 从原始请求复制，body 中的 model 等于 `ClientModel`，判断等价；如不放心，可改为基于"是否任一变换被触发"判断 | +| 单元测试失效 | 同步更新 `reverseproxy_ai_test.go` | + +**回滚**：若优化后发现问题，可回滚 `bfe/bfe_server/reverseproxy.go` 与 `bfe/bfe_server/reverseproxy_ai_test.go` 的修改。该优化不引入配置或协议变更，回滚影响面可控。 + +--- + +## 10. 关键代码索引 + +| 文件 | 行号范围 | 说明 | +|---|---|---| +| `bfe/bfe_server/reverseproxy.go` | 1429-1464 | 当前 `stripProviderPrefix` 实现 | +| `bfe/bfe_server/reverseproxy.go` | 1490-1541 | 当前 `doSingleAIForward` 中的三次 model 改写 | +| `bfe/bfe_basic/condition/primitive.go` | 1176-1200 | `ReqBodyJsonSet` 实现 | +| `bfe/bfe_server/reverseproxy_ai_test.go` | 252-342 | 当前 `stripProviderPrefix` 单元测试 | diff --git a/docs/zh_cn/sys_design/ai_access_log_fields.md b/docs/zh_cn/sys_design/ai_access_log_fields.md new file mode 100644 index 000000000..c91a667e6 --- /dev/null +++ b/docs/zh_cn/sys_design/ai_access_log_fields.md @@ -0,0 +1,250 @@ +# BFE AI 访问日志可观测字段设计 + +## 1. 背景与目标 + +### 1.1 背景 + +BFE 作为 AI 网关，需要把请求在认证、路由、转发、计费等各阶段的关键信息输出到访问日志，供下游可观测平台进行分析、计费对账、故障排查和安全审计。 + +`bfe-access-pb` 访问日志协议为 AI 网关场景预留了编号 701-900 的字段区间。随着 BFE AI 网关能力从基础转发扩展到 RMB 配额、多 API-Key 重试、模型映射、路由规则命中、限流等场景，访问日志需要同步记录更多可观测信息。 + +### 1.2 目标 + +1. 统一记录 AI 请求全生命周期的可观测字段，覆盖认证、路由、转发、计费、限流、流式响应等环节； +2. 访问日志中不再记录原始 API Key 值，改为记录 API Key 内部标识（`key_id`），避免敏感信息泄露； +3. 字段命名与 `bfe-access-pb` 协议对齐，字段编号保持 701-900 区间规划； +4. 字段采集逻辑与业务模块解耦：各模块负责把运行时信息写入 `bfe_basic.Request` 的 AI 上下文，最终由 `mod_access_pb3` 统一组装输出。 + +--- + +## 2. 字段总览 + +AI 可观测字段统一占用 `bfe-access-pb` 的 701-900 编号区间，当前已定义 20 个字段： + +| 编号 | 字段名 | 类型 | 说明 | 采集模块 | +|------|--------|------|------|----------| +| 701 | `ai_apikey_id` | `string` | API Key 内部标识（`key_id`），非原始 key 值 | `mod_ai_token_auth` | +| 702 | `ai_apikeytags` | `repeated ApikeyTag` | API Key 关联的 Entity 层级标签 | `mod_ai_token_auth` | +| 703 | `ai_requested_model` | `string` | 客户端请求原始模型名 | `bfe_server/http_conn.go` | +| 704 | `ai_target_model` | `string` | 网关实际路由/映射后的目标模型名 | `bfe_server/reverseproxy.go` | +| 705 | `ai_stream` | `bool` | 是否为流式响应 | `bfe_basic.Request.IsSse` | +| 706 | `ai_input_tokens` | `int64` | 输入 Token 数 | `mod_ai_token_auth` / `mod_body_process` | +| 707 | `ai_output_tokens` | `int64` | 输出 Token 数 | `mod_ai_token_auth` / `mod_body_process` | +| 708 | `ai_total_tokens` | `int64` | 总 Token 消耗 | `mod_ai_token_auth` | +| 709 | `ai_ttft_us` | `int64` | 首 Token 延迟（微秒），仅流式 | `mod_body_process` | +| 710 | `ai_tpot_us` | `int64` | 平均输出 Token 延迟（微秒），仅流式 | `mod_body_process` | +| 711 | `ai_rate_limit_hits` | `repeated RateLimitHit` | 触发的限流策略列表 | `mod_ai_rate_limit` | +| 712 | `ai_auth_reject_reason` | `string` | 鉴权拒绝原因 | `mod_ai_token_auth` | +| 713 | `ai_auth_reject_quota_plans` | `repeated string` | 拒绝时余额不足的 Quota Plan ID 列表 | `mod_ai_token_auth` | +| 714 | `ai_provider` | `string` | 上游模型提供商标识 | `bfe_server/reverseproxy.go` | +| 715 | `ai_retry_count` | `uint32` | 模型调用层 key-level 重试次数 | `bfe_server/reverseproxy.go` | +| 761 | `ai_cost_value` | `int64` | 估算成本（定点整数，精度由币种决定） | `mod_ai_token_auth` | +| 762 | `ai_cost_currency` | `string` | 成本币种，如 `RMB` / `USD` | `bfe_server/reverseproxy.go` | +| 801 | `ai_route_rule_hits` | `repeated AIRouteRuleHit` | 命中的 AI 路由规则列表 | `mod_ai_route` | +| 802 | `ai_cluster_key_names` | `repeated ClusterKeyName` | 请求处理过程中尝试过的 (cluster, key) 列表 | `bfe_server/reverseproxy.go` | +| 841 | `ai_auth_hit_quota_plans` | `repeated string` | 正常请求时命中的 Quota Plan ID 列表 | `mod_ai_token_auth` | + +### 2.1 编号区间规划 + +| 编号区间 | 用途 | +|----------|------| +| 701 - 713 | 已投入使用字段，保持现状，不再调整 | +| 714 - 760 | 模型与请求基础信息（model、provider、stream、retry、cache 等） | +| 761 - 800 | Token 与成本计量 | +| 801 - 840 | 路由、转换与插件 | +| 841 - 880 | 安全、合规与隐私 | +| 881 - 900 | 厂商扩展与预留 | + +--- + +## 3. 总体架构 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 客户端请求 │ +└──────────────────────────┬──────────────────────────────────┘ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ bfe_server/http_conn.go │ +│ - 初始化 AiBasicInfo │ +│ - 提取 ai_apikey_id（原始 key，后续会被 key_id 覆盖） │ +│ - 提取 ai_requested_model │ +└──────────────────────────┬──────────────────────────────────┘ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ HandleFoundProduct 回调链 │ +│ ├─ mod_ai_token_auth: 校验 Token，设置 key_id、tags、 │ +│ │ ai_auth_hit_quota_plans / ai_auth_reject_* │ +│ ├─ mod_ai_route: 设置 ai_route_rule_hits │ +│ └─ mod_ai_rate_limit: 设置 ai_rate_limit_hits │ +└──────────────────────────┬──────────────────────────────────┘ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ bfe_server/reverseproxy.go │ +│ - ServeHTTPForAI / aiClusterInvoke / doSingleAIForward │ +│ - 设置 ai_provider、ai_retry_count、 │ +│ │ ai_cluster_key_names、ai_target_model │ +│ - 触发 cluster-level fallback │ +└──────────────────────────┬──────────────────────────────────┘ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ 响应阶段 │ +│ ├─ mod_body_process: 计算 ai_ttft_us、ai_tpot_us │ +│ └─ mod_ai_token_auth: 解析 usage，计算 ai_input_tokens、 │ +│ ai_output_tokens、ai_total_tokens、ai_cost_value │ +└──────────────────────────┬──────────────────────────────────┘ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ mod_access_pb3 │ +│ - 从 Request 上下文读取所有 AI 字段 │ +│ - 组装 RequestLog 并输出 b2log │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 4. 核心数据结构 + +### 4.1 `bfe_basic.AiBasicInfo` + +`bfe_basic/request_ai_basic.go` + +```go +type AiBasicInfo struct { + ClientApiKey string + ClientKeyId string // 701 ai_apikey_id + ClientModel string // 703 ai_requested_model + TargetModel string // 704 ai_target_model + Provider string // 714 ai_provider + RetryCount uint32 // 715 ai_retry_count + CostCurrency string // 762 ai_cost_currency + tokenUsage TokenUsage + ApikeyTags []ApikeyTag // 702 ai_apikeytags + TokenTimeInfo TokenTimeInfo // 709/710 ai_ttft_us / ai_tpot_us + AiAuthInfo AiAuthInfo // 712/713/841 + ClusterKeyNames []ClusterKeyName // 802 ai_cluster_key_names + + allowEstimateToken bool +} + +type TokenUsage struct { + PromptTokens int64 // 706 ai_input_tokens + CompletionTokens int64 // 707 ai_output_tokens + UsedQuota int64 // 708 ai_total_tokens + UsedCost int64 // 761 ai_cost_value +} + +type AiAuthInfo struct { + RejectReason string // 712 ai_auth_reject_reason + RejectQuotaPlans []string // 713 ai_auth_reject_quota_plans + HitQuotaPlans []string // 841 ai_auth_hit_quota_plans +} + +type ClusterKeyName struct { + ClusterName string + KeyName string +} +``` + +### 4.2 `bfe_basic.AiRouteResult` + +`bfe_basic/request_ai_route.go` + +```go +type AiRouteResult struct { + RouteType string // apikey / entity / global + Owner string // 路由表 owner + RuleName string // 命中规则名 + Targets []AiRouteTarget + Fallbacks []AiRouteFallback +} +``` + +`mod_access_pb3` 将其转换为 `AIRouteRuleHit`： + +```protobuf +message AIRouteRuleHit { + optional string rule_owner = 1; + optional string rule_owner_type = 2; + optional string rule_name = 3; +} +``` + +--- + +## 5. 模块职责 + +### 5.1 `bfe_server/http_conn.go` + +在 `EnableAiGateway` 开启时初始化 `AiBasicInfo`： + +- 从 `Authorization` 头提取原始 API Key，写入 `ClientApiKey`（后续会被 `ClientKeyId` 覆盖）； +- 从请求 JSON body 提取 `model` 字段，写入 `ClientModel` 和 `TargetModel`。 + +### 5.2 `mod_ai_token_auth` + +- 在 `ValidateUserTokenByReq()` 中找到 Token 后，立即把 `Token.KeyId` 写入 `AiBasicInfo.ClientKeyId`，确保即使后续拒绝也能在日志中识别 key； +- 通过 `SetTokenAuthContext()` 写入 `ApikeyTags` 和初始 `PromptTokens`； +- 在响应阶段解析 `usage` 并估算 token，填充 `TokenUsage`； +- 对 RMB 配额调用 `calcCostUnits()` 计算 `UsedCost`； +- 在认证成功时记录 `HitQuotaPlans`，拒绝时记录 `RejectReason` 和 `RejectQuotaPlans`。 + +### 5.3 `mod_ai_route` + +- `routeFoundProductHandler()` 调用 `routeTable.Search()` 得到 `AiRouteResult`； +- 通过 `req.SetAiRouteResult(result)` 把结果写入请求上下文。 + +### 5.4 `mod_ai_rate_limit` + +- `limitFoundProductHandler()` 初始化 `AiRateLimitHitInfo`； +- 在 RPM/TPM/并发限流触发时，向 `HitPolicyDict[policyId]` 追加规则名； +- `mod_access_pb3` 读取后转换为 `RateLimitHit` 列表。 + +### 5.5 `bfe_server/reverseproxy.go` + +在 `doSingleAIForward()` 中： + +- 从 `cluster.AIConf.Provider` 写入 `AiBasicInfo.Provider`； +- 从 `cluster.AIConf.ModelTable.Currency` 写入 `AiBasicInfo.CostCurrency`； +- 调用 `AiBasicInfo.AppendClusterKeyName(cluster.Name, selectedKey.Name)` 记录尝试； +- 在 `ModelMapping`、路由 target/fallback model override、provider prefix strip 后更新 `TargetModel`。 + +在 `aiClusterInvoke()` 的 key-level retry 循环中： + +- 当 `retry > 0` 时调用 `AiBasicInfo.IncrementRetryCount()`。 + +### 5.6 `mod_body_process` + +- 在读取响应首包时记录 `TFirstToken`； +- 请求结束时记录 `TLastToken`； +- 调用 `calcTokenTime()` 计算 `TTFT` 和 `TPOT`。 + +### 5.7 `mod_access_pb3` + +`reqAiInfoGen()` 负责把上述所有字段从 `AiBasicInfo`、`AiRateLimitHitInfo`、`AiRouteResult` 映射到 `RequestLog`： + +- 字段重命名：`AiApikey`→`AiApikeyId`、`AiMappedModel`→`AiTargetModel`、`AiPromptTokens`→`AiInputTokens`； +- 新增字段：`AiProvider`、`AiRetryCount`、`AiCostValue`、`AiCostCurrency`、`AiRouteRuleHits`、`AiClusterKeyNames`、`AiAuthHitQuotaPlans`。 + +--- + +## 6. 安全与合规 + +1. **API Key 不落地**：`ai_apikey_id` 只记录内部 `key_id`，不记录原始 key 值。`ClientApiKey` 仍保留在内存中用于上游转发，但不会写入访问日志。 +2. **成本精度**：`ai_cost_value` 使用定点整数（RMB 为 1e-8 元），避免浮点误差。 +3. **字段可选**：所有 AI 字段均为 `optional`，未启用 AI 网关或非 AI 请求不会输出这些字段。 + +--- + +## 7. 测试与验证 + +1. **单元测试**：`bfe_modules/mod_access_pb3/request_log_test.go` 覆盖所有字段的赋值逻辑； +2. **集成测试**：`tests/integration/implementation/scenario-SC05-access-log-ai-fields/` 启动真实 BFE 进程，发送 AI 请求后解码 b2log，校验全部 20 个字段。 + +--- + +## 8. 参考文档 + +- `bfe-access-pb/docs/protobuf.md` +- `bfe/docs/zh_cn/modifications/2026-08-19-update-ai-access-log-fields/design-changes.md` +- `bfe/tests/integration/测试设计文档/scenario-SC05-AI访问日志字段校验/场景说明.md` diff --git a/docs/zh_cn/sys_design/mod_ai_route.md b/docs/zh_cn/sys_design/mod_ai_route.md new file mode 100644 index 000000000..b6c047cfc --- /dev/null +++ b/docs/zh_cn/sys_design/mod_ai_route.md @@ -0,0 +1,1143 @@ +# mod_ai_route 系统设计文档 + +## 1. 背景与目标 + +### 1.1 背景 + +BFE 原有路由基于租户（Product）组织，每张路由表命中后返回单个 `ClusterName`，再由负载均衡模块选择后端实例。AI 网关场景下，路由需求发生显著变化： + +- 路由表不再按租户划分，而按 **apikey → entity → global** 三级优先级组织； +- 命中后返回的是 **targets 列表**（含集群、模型、权重）和 **fallbacks 列表**； +- 需要在多个 target 之间做加权选择，并在 target 转发失败时按 fallbacks 顺序降级。 + +### 1.2 目标 + +新增 `mod_ai_route` 模块，在 BFE 的 `HandleFoundProduct` 回调点完成 AI 网关路由查找，并将结果写入请求上下文，供后续转发流程使用。 + +主要目标： + +1. 实现 AI 路由三级查找：API-Key 路由表 → Entity 路由表 → Global 路由表； +2. 支持命中规则后返回 `targets` 和 `fallbacks`； +3. 与现有 `mod_ai_token_auth`、`mod_ai_rate_limit` 等模块协同，复用 `AiBasicInfo` 上下文； +4. 保持对原 BFE 逻辑的侵入最小化； +5. 支持配置热加载与监控。 + +## 2. 术语定义 + +| 术语 | 定义 | +|------|------| +| API-Key | 调用方身份标识，路由匹配的最细粒度维度。 | +| Entity | 业务实体，如部门、应用、项目，一个 Entity 下可包含多个 API-Key。 | +| Global | 全局路由表，所有请求的最后兜底。 | +| Target | 路由命中的转发目标，包含 `ClusterName`、`Model`、`Weight`。 | +| Fallback | 当所有 target 不可用时，按顺序尝试的备用目标。 | +| Route Table Key | 路由表在配置文件中的唯一标识，格式为 `_`，例如 `apikey_ak_user_a`。 | + +## 3. 总体架构 + +### 3.1 在 BFE 中的位置 + +``` +┌─────────────────────────────────────┐ +│ HTTP 请求接入 │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ HandleBeforeLocation │ +│ (mod_trust_clientip, mod_logid 等) │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ findProduct() │ +│ (BFE 原租户识别，AI 网关仍保留) │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ HandleFoundProduct │ +│ mod_ai_token_auth │ +│ mod_ai_rate_limit │ +│ mod_ai_route ← 新增 │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ AI 网关转发路径（ ServeHTTPForAI） │ +│ - target 加权选择 │ +│ - model 覆盖/透传 │ +│ - fallback 顺序降级 │ +│ - clusterInvoke() 复用现有集群转发 │ +└─────────────────────────────────────┘ +``` + +### 3.2 模块协作关系 + +``` + ┌─────────────────┐ + │ HTTP Request │ + └────────┬────────┘ + │ + ┌──────────────┼──────────────┐ + ▼ ▼ ▼ + ┌─────────────────┐ ┌──────────┐ ┌──────────────┐ + │ mod_ai_token_auth│ │mod_ai_rate_limit│ │ mod_ai_route │ + │ (API-Key 鉴权) │ │(限流) │ │ (路由查找) │ + └────────┬────────┘ └────┬─────┘ └──────┬───────┘ + │ │ │ + ▼ ▼ ▼ + ┌─────────────────────────────────────────────┐ + │ AiBasicInfo / Request.Context │ + │ - ClientApiKey │ + │ - ClientModel / TargetModel │ + │ - AiRouteResult (新增) │ + └─────────────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────┐ + │ ReverseProxy.ServeHTTPForAI │ + └───────────────────────┘ +``` + +## 4. 数据结构与配置设计 + +> 配置文件字段及使用说明，请参考： +> - [mod_ai_route.conf](../configuration/mod_ai_route/mod_ai_route.conf.md) +> - [ai_route.data](../configuration/mod_ai_route/ai_route.data.md) + +### 4.1 配置文件 + +#### 4.1.1 模块配置文件 + +路径：`conf/mod_ai_route/mod_ai_route.conf` + +```ini +[basic] +RouteRulePath = ../conf/mod_ai_route/ai_route.data + +[log] +OpenDebug = false +``` + +字段说明： + +| 字段 | 类型 | 说明 | +|------|------|------| +| `RouteRulePath` | string | AI 路由规则数据文件路径。 | +| `OpenDebug` | bool | 是否开启调试日志。 | + +#### 4.1.2 路由规则数据文件 + +路径：`conf/mod_ai_route/ai_route.data` + +格式示例： + +```json +{ + "Version": "20260718131505", + "route_rules": { + "apikey_ak_user_a": { + "type": "apikey", + "owner": "ak_user_a", + "rules": [ + { + "name": "user_a-deepseek", + "Cond": "req_host_in(\"api.example.org\")", + "targets": [ + { + "ClusterName": "cluster_deepseek_a", + "Model": "deepseek-v4-pro", + "Weight": 70 + }, + { + "ClusterName": "cluster_deepseek_b", + "Model": "deepseek-v4-pro", + "Weight": 30 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_deepseek_c", + "Model": "deepseek-v3.2" + } + ] + } + ] + }, + "entity_dept_ai": { + "type": "entity", + "owner": "dept_ai", + "rules": [ + { + "name": "dept_ai-default", + "Cond": "default_t()", + "targets": [ + { + "ClusterName": "cluster_dept_ai", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [] + } + ] + }, + "global_default": { + "type": "global", + "owner": "global", + "rules": [ + { + "name": "global-default", + "Cond": "default_t()", + "targets": [ + { + "ClusterName": "cluster_global", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [] + } + ] + } + }, + "ApikeyRouteTableBindings": { + "ak_user_a": [ + "apikey_ak_user_a", + "entity_dept_ai", + "global_default" + ] + } +} +``` + +字段说明： + +| 字段 | 类型 | 说明 | +|------|------|------| +| `route_rules` | object | 所有路由表的集合，key 为 `_`。 | +| `type` | string | 路由表类型：`apikey` / `entity` / `global`。 | +| `owner` | string | 路由表属主。 | +| `rules` | array | 该路由表下的规则列表，按顺序匹配。 | +| `name` | string | 规则名称，用于日志和监控。 | +| `Cond` | string | BFE 条件表达式，命中则使用该规则。 | +| `targets` | array | 转发目标列表。 | +| `fallbacks` | array | 降级目标列表，**允许为空**。 | +| `ClusterName` | string | 后端集群名称。 | +| `Model` | string | 模型名称，空字符串表示透传原始模型。 | +| `Weight` | int | 权重，单个 target 时为 100，多个 target 时总和为 100。 | +| `ApikeyRouteTableBindings` | object | API-Key 到路由表查找顺序的映射。 | + +### 4.2 内部数据结构 + +#### 4.2.1 配置结构体（bfe_modules/mod_ai_route/conf_load.go） + +```go +package mod_ai_route + +import ( + "fmt" + gcfg "gopkg.in/gcfg.v1" + "github.com/bfenetworks/bfe/bfe_util" +) + +type ConfModAiRoute struct { + Basic struct { + RouteRulePath string // path for ai route rule + } + Log struct { + OpenDebug bool + } +} + +func ConfLoad(filePath string, confRoot string) (*ConfModAiRoute, error) { + var cfg ConfModAiRoute + if err := gcfg.ReadFileInto(&cfg, filePath); err != nil { + return &cfg, err + } + if err := cfg.Check(confRoot); err != nil { + return &cfg, err + } + return &cfg, nil +} + +func (cfg *ConfModAiRoute) Check(confRoot string) error { + return ConfModAiRouteCheck(cfg, confRoot) +} + +func ConfModAiRouteCheck(cfg *ConfModAiRoute, confRoot string) error { + if cfg.Basic.RouteRulePath == "" { + return fmt.Errorf("ConfModAiRouteCheck: RouteRulePath is empty") + } + cfg.Basic.RouteRulePath = bfe_util.ConfPathProc(cfg.Basic.RouteRulePath, confRoot) + return nil +} +``` + +#### 4.2.2 路由规则与数据定义（bfe_modules/mod_ai_route/route_rule.go） + +`route_rule.go` 同时定义 JSON DTO、运行时结构以及兼容性反序列化逻辑。DTO 与运行时类型分离，便于兼容不同来源的数据格式（如 `RouteRules` 与 `route_rules`）。 + +```go +package mod_ai_route + +import ( + "encoding/json" + "fmt" + + "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_basic/condition" +) + +// route table types +const ( + RouteTypeApikey = "apikey" + RouteTypeEntity = "entity" + RouteTypeGlobal = "global" +) + +// RouteRuleFile 是单条路由规则的 JSON DTO。 +type RouteRuleFile struct { + Name string `json:"name"` + Cond string `json:"Cond"` + Targets []bfe_basic.AiRouteTarget `json:"targets"` + Fallbacks []bfe_basic.AiRouteFallback `json:"fallbacks"` +} + +// RouteTableFile 是单张路由表的 JSON DTO。 +type RouteTableFile struct { + Type string `json:"type"` + Owner string `json:"owner"` + Rules []RouteRuleFile `json:"rules"` +} + +// AiRouteDataFile 是整个 AI 路由数据文件的 JSON DTO。 +type AiRouteDataFile struct { + Version string `json:"Version"` + RouteRules map[string]RouteTableFile `json:"route_rules"` + ApikeyRouteTableBindings map[string][]string `json:"ApikeyRouteTableBindings"` +} +``` + +`AiRouteDataFile.UnmarshalJSON` 做两项兼容性处理： + +1. 同时支持 canonical 字段名 `route_rules` 与 InnerAPI 中的 `RouteRules`； +2. 将路由表类型 `"api_key"` 规范化为 `"apikey"`。 + +```go +func (f *AiRouteDataFile) UnmarshalJSON(data []byte) error { + type rawFile AiRouteDataFile + raw := &struct { + *rawFile + RouteRulesUpper map[string]RouteTableFile `json:"RouteRules"` + }{ + rawFile: (*rawFile)(f), + } + + if err := json.Unmarshal(data, raw); err != nil { + return err + } + + if f.RouteRules == nil && raw.RouteRulesUpper != nil { + f.RouteRules = raw.RouteRulesUpper + } + + for key, table := range f.RouteRules { + if table.Type == "api_key" { + table.Type = RouteTypeApikey + } + f.RouteRules[key] = table + } + + return nil +} +``` + +运行时结构不再包含 JSON 标签；`Cond` 字段在加载时由 `ValidateRouteTable()` 编译为 `condition.Condition`。 + +```go +type RouteRule struct { + Name string + CondStr string + Cond condition.Condition + Targets []bfe_basic.AiRouteTarget + Fallbacks []bfe_basic.AiRouteFallback +} + +type RouteTable struct { + Type string + Owner string + Rules []RouteRule +} + +type AiRouteData struct { + Version string + RouteRules map[string]RouteTable + ApikeyRouteTableBindings map[string][]string +} + +func (rt *RouteTable) Match(req *bfe_basic.Request) *RouteRule { + for i := range rt.Rules { + rule := &rt.Rules[i] + if rule.Cond != nil && rule.Cond.Match(req) { + return rule + } + } + return nil +} + +func ValidateRouteTable(table *RouteTable) error { + switch table.Type { + case RouteTypeApikey, RouteTypeEntity, RouteTypeGlobal: + default: + return fmt.Errorf("invalid route table type: %s", table.Type) + } + + for i := range table.Rules { + rule := &table.Rules[i] + if rule.Name == "" { + return fmt.Errorf("rule name empty") + } + if rule.CondStr == "" { + return fmt.Errorf("rule[%s] Cond empty", rule.Name) + } + cond, err := condition.Build(rule.CondStr) + if err != nil { + return fmt.Errorf("rule[%s] build cond[%s] err: %s", rule.Name, rule.CondStr, err) + } + rule.Cond = cond + + if len(rule.Targets) == 0 { + return fmt.Errorf("rule[%s] targets empty", rule.Name) + } + + totalWeight := 0 + for _, target := range rule.Targets { + totalWeight += target.Weight + } + if totalWeight != 100 { + return fmt.Errorf("rule[%s] total weight %d != 100", rule.Name, totalWeight) + } + } + return nil +} +``` + +#### 4.2.3 路由数据加载（bfe_modules/mod_ai_route/data_load.go） + +`data_load.go` 负责将 `ai_route.data` 反序列化为 DTO，再转换为运行时结构。 + +```go +package mod_ai_route + +import ( + "fmt" + + "github.com/bfenetworks/bfe/bfe_util" +) + +func AiRouteDataLoad(fileName string) (*AiRouteData, error) { + var file AiRouteDataFile + + if err := bfe_util.LoadJsonFile(fileName, &file); err != nil { + return nil, fmt.Errorf("LoadJsonFile(): err[%s]", err.Error()) + } + + data := &AiRouteData{ + Version: file.Version, + RouteRules: make(map[string]RouteTable, len(file.RouteRules)), + ApikeyRouteTableBindings: file.ApikeyRouteTableBindings, + } + + for key, tableFile := range file.RouteRules { + rules := make([]RouteRule, len(tableFile.Rules)) + for i, ruleFile := range tableFile.Rules { + rules[i] = RouteRule{ + Name: ruleFile.Name, + CondStr: ruleFile.Cond, + Targets: ruleFile.Targets, + Fallbacks: ruleFile.Fallbacks, + } + } + data.RouteRules[key] = RouteTable{ + Type: tableFile.Type, + Owner: tableFile.Owner, + Rules: rules, + } + } + + return data, nil +} +``` + +#### 4.2.4 路由结果上下文（bfe_basic/request_ai_route.go） + +`AiRouteResult` 及上下文读写函数放在 `bfe_basic` 包中，便于 `bfe_server` 等其它包访问，避免循环依赖。 + +```go +package bfe_basic + +const CtxAiRouteResult = "__REQ_AI_ROUTE_RESULT" + +type AiRouteResult struct { + RouteType string // apikey / entity / global + Owner string // route table owner + RuleName string // hit rule name + Targets []AiRouteTarget + Fallbacks []AiRouteFallback +} + +type AiRouteTarget struct { + ClusterName string + Model string + Weight int +} + +type AiRouteFallback struct { + ClusterName string + Model string +} + +func (r *Request) SetAiRouteResult(result *AiRouteResult) { + r.SetContext(CtxAiRouteResult, result) +} + +func (r *Request) GetAiRouteResult() *AiRouteResult { + val := r.GetContext(CtxAiRouteResult) + if val == nil { + return nil + } + result, ok := val.(*AiRouteResult) + if !ok { + return nil + } + return result +} +``` + +## 5. 模块设计 + +### 5.1 模块结构 + +新增目录：`bfe/bfe_modules/mod_ai_route/` + +文件清单： + +| 文件 | 职责 | +|------|------| +| `mod_ai_route.go` | 模块入口，实现 `BfeModule` 接口，注册回调。 | +| `conf_load.go` | 加载 `mod_ai_route.conf` 模块配置。 | +| `data_load.go` | 加载 `ai_route.data` 路由数据文件，转换为运行时结构。 | +| `route_table.go` | 路由表内存结构，提供查找接口。 | +| `route_rule.go` | 定义 JSON DTO、运行时结构体、条件编译与校验及兼容性反序列化。 | +| （无 `context.go`） | `AiRouteResult` 已移至 `bfe_basic/request_ai_route.go`。 | +| `prometheus_states.go` | Prometheus 监控指标（可选）。 | +| `mod_ai_route_test.go` | 单元测试。 | +| `testdata/` | 测试数据。 | + +### 5.2 核心类图 + +``` +┌─────────────────────────────┐ +│ ModuleAiRoute │ +├─────────────────────────────┤ +│ - name: string │ +│ - conf: *ConfModAiRoute │ +│ - routeTable: *AiRouteTable │ +│ - state: ModuleAiRouteState │ +├─────────────────────────────┤ +│ + Name() string │ +│ + Init() error │ +│ + routeFoundProductHandler()│ +│ + loadRouteRuleConf() │ +│ + getState() │ +└──────────────┬──────────────┘ + │ uses + ▼ +┌─────────────────────────────┐ +│ AiRouteTable │ +├─────────────────────────────┤ +│ - routeRules: map[string]*RouteTable │ +│ - bindings: map[string][]string │ +├─────────────────────────────┤ +│ + Update(data *AiRouteData) │ +│ + Search(apiKey string, req *bfe_basic.Request) *AiRouteResult │ +└──────────────┬──────────────┘ + │ uses + ▼ +┌─────────────────────────────┐ +│ RouteTable │ +├─────────────────────────────┤ +│ - Type: string │ +│ - Owner: string │ +│ - Rules: []RouteRule │ +├─────────────────────────────┤ +│ + Match(req) *RouteRule │ +└──────────────┬──────────────┘ + │ uses + ▼ +┌─────────────────────────────┐ +│ RouteRule │ +├─────────────────────────────┤ +│ - Name: string │ +│ - Cond: Condition │ +│ - Targets: []Target │ +│ - Fallbacks: []Fallback │ +└─────────────────────────────┘ +``` + +### 5.3 核心实现 + +#### 5.3.1 模块主文件（mod_ai_route.go） + +```go +package mod_ai_route + +import ( + "fmt" + "net/url" + + "github.com/bfenetworks/go-lib/log" + "github.com/bfenetworks/go-lib/web-monitor/metrics" + "github.com/bfenetworks/go-lib/web-monitor/web_monitor" + + "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_http" + "github.com/bfenetworks/bfe/bfe_module" +) + +const ModAiRoute = "mod_ai_route" + +var openDebug = false + +type ModuleAiRouteState struct { + ReqTotal *metrics.Counter + ReqHitApikey *metrics.Counter + ReqHitEntity *metrics.Counter + ReqHitGlobal *metrics.Counter + ReqMiss *metrics.Counter + ReqFallback *metrics.Counter +} + +type ModuleAiRoute struct { + name string + conf *ConfModAiRoute + routeTable *AiRouteTable + state ModuleAiRouteState + metrics metrics.Metrics +} + +func NewModuleAiRoute() *ModuleAiRoute { + m := new(ModuleAiRoute) + m.name = ModAiRoute + m.metrics.Init(&m.state, ModAiRoute, 0) + m.routeTable = NewAiRouteTable() + return m +} + +func (m *ModuleAiRoute) Name() string { + return m.name +} + +func (m *ModuleAiRoute) loadRouteRuleConf(query url.Values) error { + path := query.Get("path") + if path == "" { + path = m.conf.Basic.RouteRulePath + } + + data, err := AiRouteDataLoad(path) + if err != nil { + return fmt.Errorf("err in AiRouteDataLoad(%s): %s", path, err) + } + + if err := m.routeTable.Update(data); err != nil { + return fmt.Errorf("err in routeTable.Update: %s", err) + } + + return nil +} + +func (m *ModuleAiRoute) routeFoundProductHandler(req *bfe_basic.Request) (int, *bfe_http.Response) { + m.state.ReqTotal.Inc(1) + + aiMeta := req.GetAiBasicInfo() + if aiMeta == nil { + return bfe_module.BfeHandlerGoOn, nil + } + + apiKey := aiMeta.ClientApiKey + if apiKey == "" { + if openDebug { + log.Logger.Debug("%s: api key empty, skip", m.name) + } + return bfe_module.BfeHandlerGoOn, nil + } + + result := m.routeTable.Search(apiKey, req) + if result == nil { + m.state.ReqMiss.Inc(1) + if openDebug { + log.Logger.Debug("%s: no route hit for apiKey[%s]", m.name, apiKey) + } + return bfe_module.BfeHandlerGoOn, nil + } + + switch result.RouteType { + case RouteTypeApikey: + m.state.ReqHitApikey.Inc(1) + case RouteTypeEntity: + m.state.ReqHitEntity.Inc(1) + case RouteTypeGlobal: + m.state.ReqHitGlobal.Inc(1) + } + + req.SetAiRouteResult(result) + + return bfe_module.BfeHandlerGoOn, nil +} + +func (m *ModuleAiRoute) Init(cbs *bfe_module.BfeCallbacks, whs *web_monitor.WebHandlers, cr string) error { + confPath := bfe_module.ModConfPath(cr, m.name) + var err error + if m.conf, err = ConfLoad(confPath, cr); err != nil { + return fmt.Errorf("%s: conf load err %v", m.name, err) + } + openDebug = m.conf.Log.OpenDebug + + if err := m.loadRouteRuleConf(nil); err != nil { + return fmt.Errorf("%s: loadRouteRuleConf err %v", m.name, err) + } + + if err := cbs.AddFilter(bfe_module.HandleFoundProduct, m.routeFoundProductHandler); err != nil { + return fmt.Errorf("%s.Init(): AddFilter(routeFoundProductHandler): %s", m.name, err.Error()) + } + + monitorHandlers := map[string]interface{}{ + m.name: m.getState, + } + if err := web_monitor.RegisterHandlers(whs, web_monitor.WebHandleMonitor, monitorHandlers); err != nil { + return fmt.Errorf("%s.Init(): RegisterHandlers(monitor): %v", m.name, err) + } + + reloadHandlers := map[string]interface{}{ + m.name: m.loadRouteRuleConf, + } + if err := web_monitor.RegisterHandlers(whs, web_monitor.WebHandleReload, reloadHandlers); err != nil { + return fmt.Errorf("%s.Init(): RegisterHandlers(reload): %v", m.name, err) + } + + return nil +} + +func (m *ModuleAiRoute) getState(params map[string][]string) ([]byte, error) { + s := m.metrics.GetAll() + return s.Format(params) +} +``` + +#### 5.3.2 路由表查找（route_table.go） + +```go +package mod_ai_route + +import ( + "fmt" + "sync" + + "github.com/bfenetworks/go-lib/log" + "github.com/bfenetworks/bfe/bfe_basic" +) + +type AiRouteTable struct { + lock sync.RWMutex + + // routeRules key: route table key (_) + // routeRules value: pointer to the route table + routeRules map[string]*RouteTable + + // bindings key: API-Key string + // bindings value: ordered list of route table keys to search + bindings map[string][]string +} + +func NewAiRouteTable() *AiRouteTable { + return &AiRouteTable{ + routeRules: make(map[string]*RouteTable), + bindings: make(map[string][]string), + } +} + +func (t *AiRouteTable) Update(data *AiRouteData) error { + // validate and compile conditions (outside the lock) + rules := make(map[string]*RouteTable) + for key, table := range data.RouteRules { + if err := ValidateRouteTable(&table); err != nil { + return fmt.Errorf("validate route table[%s] err: %s", key, err) + } + tableCopy := table + rules[key] = &tableCopy + } + + // only lock when swapping the atomic references + t.lock.Lock() + t.routeRules = rules + t.bindings = data.ApikeyRouteTableBindings + t.lock.Unlock() + + return nil +} + +func (t *AiRouteTable) Search(apiKey string, req *bfe_basic.Request) *bfe_basic.AiRouteResult { + t.lock.RLock() + + tableKeys, ok := t.bindings[apiKey] + if !ok || len(tableKeys) == 0 { + t.lock.RUnlock() + return nil + } + + // copy table references under lock; table.Match() may be expensive, + // so we release the lock before matching. + tables := make([]*RouteTable, 0, len(tableKeys)) + for _, key := range tableKeys { + if table, ok := t.routeRules[key]; ok { + tables = append(tables, table) + } else if openDebug { + log.Logger.Debug("mod_ai_route: route table[%s] not found", key) + } + } + t.lock.RUnlock() + + // match outside the lock to reduce critical section + for _, table := range tables { + rule := table.Match(req) + if rule != nil { + return &bfe_basic.AiRouteResult{ + RouteType: table.Type, + Owner: table.Owner, + RuleName: rule.Name, + Targets: rule.Targets, + Fallbacks: rule.Fallbacks, + } + } + } + + return nil +} +``` + +#### 5.3.3 规则匹配与校验（route_rule.go） + +```go +package mod_ai_route + +import ( + "fmt" + + "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_basic/condition" +) + +func (rt *RouteTable) Match(req *bfe_basic.Request) *RouteRule { + for i := range rt.Rules { + rule := &rt.Rules[i] + if rule.Cond != nil && rule.Cond.Match(req) { + return rule + } + } + return nil +} + +func ValidateRouteTable(table *RouteTable) error { + switch table.Type { + case RouteTypeApikey, RouteTypeEntity, RouteTypeGlobal: + default: + return fmt.Errorf("invalid route table type: %s", table.Type) + } + + for i := range table.Rules { + rule := &table.Rules[i] + if rule.Name == "" { + return fmt.Errorf("rule name empty") + } + if rule.CondStr == "" { + return fmt.Errorf("rule[%s] Cond empty", rule.Name) + } + cond, err := condition.Build(rule.CondStr) + if err != nil { + return fmt.Errorf("rule[%s] build cond[%s] err: %s", rule.Name, rule.CondStr, err) + } + rule.Cond = cond + + if len(rule.Targets) == 0 { + return fmt.Errorf("rule[%s] targets empty", rule.Name) + } + + totalWeight := 0 + for _, target := range rule.Targets { + totalWeight += target.Weight + } + if totalWeight != 100 { + return fmt.Errorf("rule[%s] total weight != 100", rule.Name) + } + } + return nil +} +``` + +## 6. 执行流程 + +### 6.1 请求处理流程 + +``` +1. 接收 HTTP 请求 +2. 经过 HandleBeforeLocation 回调 +3. 执行 findProduct()，识别 BFE 租户（AI 网关场景下租户信息仅作为兼容保留） +4. 执行 HandleFoundProduct 回调： + a. mod_ai_token_auth：鉴权并设置 AiBasicInfo.ClientApiKey + b. mod_ai_rate_limit：执行限流策略 + c. mod_ai_route： + - 获取 ClientApiKey + - 按 ApikeyRouteTableBindings 顺序查找 apikey → entity → global 路由表 + - 命中规则后返回 AiRouteResult + - 将完整的 targets 和 fallbacks 写入请求上下文 +5. 在 AI 网关模式下，由 `ServeHTTPForAI()` 接管；`mod_ai_route` 未命中时直接返回 404 +6. 命中 AI 路由后： + - 进入 ServeHTTPForAI（独立实现，避免影响原有 ServeHTTP） + - 根据 target 构造 OutRequest + - Model 非空时覆盖请求体中的 model 字段 + - 调用 clusterInvoke() 转发到目标集群 + - 失败时按 fallbacks 顺序尝试 +7. 返回响应 +``` + +### 6.2 路由查找详细流程 + +```go +func (m *ModuleAiRoute) routeFoundProductHandler(req *bfe_basic.Request) (int, *bfe_http.Response) { + // 1. 获取 AiBasicInfo 和 ClientApiKey + // 2. 调用 routeTable.Search(apiKey, req) + // 3. Search 内部： + // a. 根据 apiKey 获取 bindings 列表 + // b. 遍历每个 route table key + // c. 在每张路由表中顺序匹配 rules + // d. 命中后构建 AiRouteResult 返回 + // 4. 将命中结果写入上下文 +} +``` + +### 6.3 Fallback 流程 + +``` +当 target 转发失败时： +1. 检查错误类型： + - 触发 fallback：连接失败、超时、后端 5xx + - 不触发 fallback：客户端 4xx、限流拒绝、鉴权失败 +2. 按 fallbacks 列表顺序尝试： + - 对每个 fallback，构造新的 target + - 重新调用 clusterInvoke() + - 第一个成功即停止 +3. 所有 fallback 均失败： + - 返回最后一个 fallback 的错误响应 +``` + +## 7. 与现有系统的集成 + +### 7.1 模块注册 + +在 `bfe/bfe_modules/bfe_modules.go` 的 `moduleList` 中新增： + +```go +import ( + "github.com/bfenetworks/bfe/bfe_modules/mod_ai_route" +) + +var moduleList = []bfe_module.BfeModule{ + // ... existing modules ... + + // mod_ai_token_auth + mod_ai_token_auth.NewModuleAITokenAuth(), + + // mod_ai_route + // Requirement: after mod_ai_token_auth (needs ClientApiKey), before mod_body_process + mod_ai_route.NewModuleAiRoute(), + + // mod_body_process + mod_body_process.NewModuleBodyProcess(), + + // depends on token calc + mod_ai_rate_limit.NewModuleAiRateLimit(), + + // ... +} +``` + +### 7.2 AI 网关开关 + +BFE 核心配置 `bfe_config/bfe_conf/conf_basic.go` 已包含 `EnableAiGateway bool` 字段，对应 `bfe.conf`： + +```ini +[server] +ai_gateway_enabled = true +``` + +AI 网关开关在 `bfe_server/http_conn.go` 的 `conn.serveRequest()` 中读取，并决定请求进入 AI 网关转发路径还是原有路径： + +```go +// serve the request +var ret1 int +if c.server.Config.Server.EnableAiGateway { + ret1 = c.server.ReverseProxy.ServeHTTPForAI(w, request) +} else { + ret1 = c.server.ReverseProxy.ServeHTTP(w, request) +} +``` + +`EnableAiGateway` 为 `true` 时，`http_conn.go` 还会初始化 `AiBasicInfo`、提取 `ClientApiKey` 与 `ClientModel`，供后续 `mod_ai_route` 等模块使用。`mod_ai_route` 本身不再判断该开关，仅依赖 `AiBasicInfo.ClientApiKey` 是否存在进行路由查找。 + +### 7.3 复用现有转发能力 + +- **集群选择**：将选中的 `ClusterName` 写入 `req.Route.ClusterName`，使后续 `findCluster()` 和 `clusterInvoke()` 可复用； +- **模型覆盖**：`Model` 非空时通过 `condition.ReqBodyJsonSet()` 修改请求体 model 字段，与现有 `ReverseProxy.ServeHTTP` 中 AIConf 的 model mapping 逻辑保持一致； +- **连接管理**：复用 `ReverseProxy.transports` 和 `clusterInvoke()` 进行实际转发； +- **降级重试**：在独立的 `ServeHTTPForAI()` 中实现 target/fallback 选择逻辑，调用 `clusterInvoke()` 完成每次转发尝试。 + +### 7.4 多 API-Key 支持 + +ai-gateway-api 支持为一个 cluster 配置多个 API-Key，BFE 在 `ServeHTTPForAI()` → `aiClusterInvoke()` 路径中消费 `cluster.AIConf.Keys` 与 `cluster.AIConf.KeyPolicy`，实现 Key 级加权随机选择、失败轮换与退避重试。多 API-Key 的详细设计见 [BFE 多 API-Key 支持](./multi_api_key.md)。 + +### 7.4 独立 ServeHTTPForAI + +为避免对原 `ServeHTTP()` 产生较大影响，已在 `bfe_server/reverseproxy.go` 中新增独立的 `ServeHTTPForAI()`： + +```go +func (p *ReverseProxy) ServeHTTPForAI(rw bfe_http.ResponseWriter, basicReq *bfe_basic.Request) (action int) { + // 1. 调用 HandleBeforeLocation / findProduct / HandleFoundProduct + // 2. 从 basicReq 中获取 AiRouteResult，未命中则返回 404 + // 3. 调用 HandleAfterLocation + // 4. 加权选择 target，构造 [selected target] + fallbacks 尝试列表 + // 5. 循环调用 aiClusterInvoke() 转发；失败时按 fallbacks 顺序重试 + // 6. 发送响应 +} +``` + +在 `bfe_server/http_conn.go` 中根据 `EnableAiGateway` 决定调用 `ServeHTTP` 还是 `ServeHTTPForAI`。 + +## 8. 错误处理与监控 + +### 8.1 错误码 + +| 错误场景 | 错误码 | 说明 | +|----------|--------|------| +| AI 路由未命中（无 bindings 或所有路由表均未匹配） | `ErrBkFindLocation` | 返回 404 Not Found。 | +| targets 权重总和不等于 100 | 配置校验错误 | 启动/热加载时拒绝。 | +| 条件表达式编译失败 | 配置校验错误 | 启动/热加载时拒绝。 | +| target 转发失败 | 复用现有 `ErrBk*` 错误码 | 进入 fallback 流程。 | + +### 8.2 监控指标 + +在 `ModuleAiRouteState` 中定义： + +| 指标 | 类型 | 含义 | +|------|------|------| +| `ReqTotal` | Counter | 处理请求总数。 | +| `ReqHitApikey` | Counter | 命中 apikey 路由表次数。 | +| `ReqHitEntity` | Counter | 命中 entity 路由表次数。 | +| `ReqHitGlobal` | Counter | 命中 global 路由表次数。 | +| `ReqMiss` | Counter | 未命中任何路由表次数。 | +| `ReqFallback` | Counter | 触发 fallback 次数。 | + +可选 Prometheus 指标： + +| 指标 | 标签 | 含义 | +|------|------|------| +| `ai_route_hit_total` | `type`, `owner`, `rule_name` | 各规则命中次数。 | +| `ai_route_fallback_total` | `owner`, `rule_name`, `fallback_cluster` | fallback 触发次数。 | + +## 9. 配置热加载 + +通过 Web 监控接口支持热加载： + +``` +GET /reload/mod_ai_route +``` + +调用 `loadRouteRuleConf()` 重新加载 `ai_route.data`，校验通过后原子替换内存中的 `AiRouteTable`。 + +## 10. 实现步骤 + +1. **新增模块目录和配置文件** + - `bfe/bfe_modules/mod_ai_route/` + - `bfe/conf/mod_ai_route/mod_ai_route.conf` + - `bfe/conf/mod_ai_route/ai_route.data` + +2. **实现配置加载** + - `conf_load.go`：加载 `mod_ai_route.conf`。 + - `data_load.go`：加载 `ai_route.data`。 + +3. **实现路由核心逻辑** + - `route_rule.go`：定义数据结构、条件编译、校验。 + - `route_table.go`：实现路由表查找。 + +4. **实现模块入口** + - `mod_ai_route.go`：注册 `HandleFoundProduct` 回调、监控与热加载。 + +5. **上下文传递** + - `bfe_basic/request_ai_route.go`：定义 `AiRouteResult` 与上下文读写。`mod_ai_route` 通过 `req.SetAiRouteResult()` / `req.GetAiRouteResult()` 访问。 + +6. **注册模块** + - 在 `bfe_modules/bfe_modules.go` 中引入并注册。 + +7. **转发层适配** + - 在 `bfe_server/reverseproxy.go` 中新增 `ServeHTTPForAI()`，处理 target/fallback 逻辑。 + - 在连接处理层根据 `EnableAiGateway` 分发到 `ServeHTTPForAI`。 + +8. **测试** + - 单元测试：配置解析、条件匹配、加权选择、fallback 触发。 + - 集成测试：与 `mod_ai_token_auth` 协同，验证完整请求链路。 + +## 11. 风险与注意事项 + +1. **与 mod_ai_token_auth 的执行顺序**： + `mod_ai_route` 必须排在 `mod_ai_token_auth` 之后，确保 `ClientApiKey` 已被设置。 + +2. **与原有 BFE 路由的兼容**： + 当 `EnableAiGateway = false` 时，`mod_ai_route` 不执行任何逻辑，保持原有行为。 + +3. **Model 覆盖与请求体修改**： + 修改请求体 model 后需同步更新 `Content-Length`，或设置为 `-1` 使用 chunked 编码。 + +4. **Fallback 与请求体重用**： + 由于请求体可能被读取，fallback 时需确保 Body 可重复读取（复用 `BodyAccessor` 或提前缓存）。 + +5. **配置原子性**： + 热加载失败时不应影响当前内存中的路由表，更新操作应在校验完成后原子切换。 + +## 12. 附录 + +### 12.1 关键文件路径 + +| 文件 | 路径 | +|------|------| +| 模块主文件 | `bfe/bfe_modules/mod_ai_route/mod_ai_route.go` | +| 模块配置加载 | `bfe/bfe_modules/mod_ai_route/conf_load.go` | +| 路由数据加载 | `bfe/bfe_modules/mod_ai_route/data_load.go` | +| AI 路由结果上下文 | `bfe/bfe_basic/request_ai_route.go` | +| 路由表 | `bfe/bfe_modules/mod_ai_route/route_table.go` | +| 路由规则 | `bfe/bfe_modules/mod_ai_route/route_rule.go` | +| 模块注册 | `bfe/bfe_modules/bfe_modules.go` | +| AI 网关开关配置 | `bfe/bfe_config/bfe_conf/conf_basic.go` | +| 转发逻辑 | `bfe/bfe_server/reverseproxy.go` | + +### 12.2 依赖的现有 BFE 能力 + +| 能力 | 来源 | +|------|------| +| 条件表达式编译 | `bfe_basic/condition` | +| 请求上下文 | `bfe_basic.Request.Context` | +| AI 基础信息 | `bfe_basic.AiBasicInfo` | +| 回调注册 | `bfe_module.BfeCallbacks` | +| 监控接口 | `web_monitor` | +| 集群查找与转发 | `bfe_server.ReverseProxy.clusterInvoke` | +| 请求体修改 | `bfe_basic/condition.ReqBodyJsonSet` | diff --git a/docs/zh_cn/sys_design/mod_ai_route_bfe_changes.md b/docs/zh_cn/sys_design/mod_ai_route_bfe_changes.md new file mode 100644 index 000000000..5b95c33a2 --- /dev/null +++ b/docs/zh_cn/sys_design/mod_ai_route_bfe_changes.md @@ -0,0 +1,780 @@ +# mod_ai_route 对应 BFE 主程序修改方案 + +## 1. 背景与目标 + +### 1.1 背景 + +`mod_ai_route` 已在 `HandleFoundProduct` 阶段完成 AI 网关路由查找，并将结果写入请求上下文（`AiRouteResult`）。但与原有 BFE 转发流程相比，AI 网关需要： + +- 不再走原有 `findCluster()` 的租户内路由； +- 按 `targets` 加权选择的目标进行转发； +- 在目标转发失败时，按 `fallbacks` 顺序降级； +- 支持模型字段覆盖与透传。 + +原有 `ReverseProxy.ServeHTTP()` 是为传统 BFE 转发设计的，直接修改它会引入较大复杂度和风险。实际实现新增了独立的 `ServeHTTPForAI()`，并在 `http_conn.go` 中根据 `EnableAiGateway` 开关进行分发。 + +### 1.2 目标 + +1. 在 `bfe_server/reverseproxy.go` 中新增 `ServeHTTPForAI()`，专门处理 AI 网关请求转发； +2. 在 `bfe_server/http_conn.go` 中根据 `EnableAiGateway` 决定调用 `ServeHTTP()` 或 `ServeHTTPForAI()`； +3. 复用现有 `clusterInvoke()`、`sendResponse()` 等核心转发能力； +4. 实现 target 命中后的模型覆盖/透传； +5. 实现 fallback 顺序降级机制； +6. 尽量降低对原有 `ServeHTTP()` 的侵入。 + +## 2. 设计原则 + +- **独立路径**：AI 网关和传统七层负载均衡使用不同的入口函数，互不干扰； +- **复用优先**：回调处理、集群查找、后端转发、响应发送尽量复用现有函数； +- **失败隔离**：fallback 默认针对后端不可用场景（5xx/连接错误）以及特定的上游不可用类 4xx（400/401/402/403/422/429）；404 等请求级错误不触发； +- **状态一致**：每次 fallback 重置 `OutRequest` 和相关上下文，避免污染下一次尝试。 + +## 3. 总体架构 + +### 3.1 请求处理路径 + +``` +HTTP 请求接入 + │ + ▼ +bfe_server/http_conn.go + │ + ├── EnableAiGateway = false ──► ReverseProxy.ServeHTTP() + │ (原有路径) + │ + └── EnableAiGateway = true ───► ReverseProxy.ServeHTTPForAI() + (新增路径) +``` + +### 3.2 ServeHTTPForAI 内部流程 + +``` +┌─────────────────────────────────────┐ +│ setClientAddr() │ +│ HandleBeforeLocation │ +│ findProduct() │ +│ HandleFoundProduct │ +│ (mod_ai_route 写入 AiRouteResult) │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ 未获取 AiRouteResult？ │ +│ → 返回 404（AI 网关模式无默认路由） │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ HandleAfterLocation │ +│ (mod_body_process 等） │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ 加权随机选择 target │ +│ 构建尝试列表： │ +│ [selected target] + fallbacks │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ prepareRequestBodyForRetry() │ +│ 确保请求体可回退（Rewindable） │ +│ 不可回退时禁用 fallback │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ 对每个尝试目标循环： │ +│ 1. resetRequestForRetry()（非首次） │ +│ 2. ClusterTable.Lookup(ClusterName) │ +│ 3. 准备 OutRequest │ +│ 4. 模型覆盖 / cluster.AIConf 映射 │ +│ 5. clusterInvoke() │ +│ 6. 成功则跳出；失败且满足 fallback │ +│ 条件则继续下一个 fallback │ +└──────────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ response_got / send_response │ +│ HandleReadResponse │ +└─────────────────────────────────────┘ +``` + +## 4. 详细修改方案 + +### 4.1 修改 bfe_server/http_conn.go + +在 `conn.serveRequest()` 中，根据 `EnableAiGateway` 决定调用哪个 `ServeHTTP`： + +```go +// serve the request +var ret1 int +if c.server.Config.Server.EnableAiGateway { + ret1 = c.server.ReverseProxy.ServeHTTPForAI(w, request) +} else { + ret1 = c.server.ReverseProxy.ServeHTTP(w, request) +} +``` + +当前代码位置：`bfe_server/http_conn.go:558-559` + +```go +// 原代码 +ret1 := c.server.ReverseProxy.ServeHTTP(w, request) +``` + +替换为： + +```go +var ret1 int +if c.server.Config.Server.EnableAiGateway { + ret1 = c.server.ReverseProxy.ServeHTTPForAI(w, request) +} else { + ret1 = c.server.ReverseProxy.ServeHTTP(w, request) +} +``` + +### 4.2 新增 ReverseProxy.ServeHTTPForAI() + +在 `bfe_server/reverseproxy.go` 中新增 `ServeHTTPForAI()`，位于 `ServeHTTP()` 之后，便于复用内部辅助函数。 + +#### 4.2.1 函数签名 + +```go +// ServeHTTPForAI processes AI gateway http request and send http response. +func (p *ReverseProxy) ServeHTTPForAI(rw bfe_http.ResponseWriter, basicReq *bfe_basic.Request) (action int) { + // implementation +} +``` + +#### 4.2.2 实现结构 + +```go +func (p *ReverseProxy) ServeHTTPForAI(rw bfe_http.ResponseWriter, basicReq *bfe_basic.Request) (action int) { + var err error + var res *bfe_http.Response + var hl *bfe_module.HandlerList + var retVal int + var req *bfe_http.Request = basicReq.HttpRequest + var serverConf *bfe_route.ServerDataConf + var writeTimer *time.Timer + var eppClient *epp.EppClient + var ok bool + + // declare ai-related vars at top to avoid goto jumping over declarations + var aiResult *bfe_basic.AiRouteResult + var aiMeta *bfe_basic.AiBasicInfo + var selectedTarget bfe_basic.AiRouteTarget + var attempts []aiForwardAttempt + var lastCluster *bfe_cluster.BfeCluster + var invokeErr error + + isRedirect := false + resFlushInterval := time.Duration(0) + cancelOnClientClose := false + + timeoutReadClient := time.Duration(cluster_conf.DefaultReadClientTimeout) * time.Millisecond + timeoutWriteClient := time.Duration(cluster_conf.DefaultWriteClientTimeout) * time.Millisecond + timeoutReadClientAgain := time.Duration(cluster_conf.DefaultReadClientAgainTimeout) * time.Millisecond + + // get instance of BfeServer + srv := p.server + + // set clientip of original user for request + setClientAddr(basicReq) + + // ========== HandleBeforeLocation ========== + hl = srv.CallBacks.GetHandlerList(bfe_module.HandleBeforeLocation) + if hl != nil { + retVal, res = hl.FilterRequest(basicReq) + basicReq.HttpResponse = res + switch retVal { + case bfe_module.BfeHandlerClose: + action = closeDirectly + return + case bfe_module.BfeHandlerFinish: + action = closeAfterReply + basicReq.BfeStatusCode = bfe_http.StatusInternalServerError + goto send_response + case bfe_module.BfeHandlerRedirect: + Redirect(rw, req, basicReq.Redirect.Url, basicReq.Redirect.Code, basicReq.Redirect.Header) + isRedirect = true + basicReq.BfeStatusCode = basicReq.Redirect.Code + goto send_response + case bfe_module.BfeHandlerResponse: + goto response_got + } + } + + // ========== findProduct ========== + if err := srv.findProduct(basicReq); err != nil { + basicReq.ErrCode = bfe_basic.ErrBkFindProduct + basicReq.ErrMsg = err.Error() + p.proxyState.ErrBkFindProduct.Inc(1) + res = bfe_basic.CreateInternalSrvErrResp(basicReq) + action = closeAfterReply + goto response_got + } + + // ========== HandleFoundProduct ========== + hl = srv.CallBacks.GetHandlerList(bfe_module.HandleFoundProduct) + if hl != nil { + retVal, res = hl.FilterRequest(basicReq) + basicReq.HttpResponse = res + switch retVal { + case bfe_module.BfeHandlerClose: + action = closeDirectly + return + case bfe_module.BfeHandlerFinish: + action = closeAfterReply + basicReq.BfeStatusCode = bfe_http.StatusInternalServerError + goto send_response + case bfe_module.BfeHandlerRedirect: + Redirect(rw, req, basicReq.Redirect.Url, basicReq.Redirect.Code, basicReq.Redirect.Header) + isRedirect = true + basicReq.BfeStatusCode = basicReq.Redirect.Code + goto send_response + case bfe_module.BfeHandlerResponse: + goto response_got + } + } + + // ========== AI Route Result Check ========== + aiResult = basicReq.GetAiRouteResult() + if aiResult == nil { + // AI gateway mode: no route hit, return 404 + basicReq.ErrCode = bfe_basic.ErrBkFindLocation + basicReq.ErrMsg = "no ai route found" + p.proxyState.ErrBkFindLocation.Inc(1) + res = bfe_basic.CreateSpecifiedContentResp(basicReq, bfe_http.StatusNotFound, + "text/plain", "AI route not found") + action = closeAfterReply + goto response_got + } + + aiMeta = basicReq.GetAiBasicInfo() + + // ========== HandleAfterLocation ========== + hl = srv.CallBacks.GetHandlerList(bfe_module.HandleAfterLocation) + if hl != nil { + retVal, res = hl.FilterRequest(basicReq) + basicReq.HttpResponse = res + switch retVal { + case bfe_module.BfeHandlerClose: + action = closeDirectly + return + case bfe_module.BfeHandlerFinish: + action = closeAfterReply + basicReq.BfeStatusCode = bfe_http.StatusInternalServerError + goto send_response + case bfe_module.BfeHandlerRedirect: + Redirect(rw, req, basicReq.Redirect.Url, basicReq.Redirect.Code, basicReq.Redirect.Header) + isRedirect = true + basicReq.BfeStatusCode = basicReq.Redirect.Code + goto send_response + case bfe_module.BfeHandlerResponse: + goto response_got + } + } + + // ========== AI Forward Loop ========== + serverConf = basicReq.SvrDataConf.(*bfe_route.ServerDataConf) + + // weighted random select target + if len(aiResult.Targets) > 0 { + selectedTarget = SelectTarget(aiResult.Targets) + } + + // build attempt list: selected target + fallbacks + attempts = make([]aiForwardAttempt, 0, 1+len(aiResult.Fallbacks)) + if selectedTarget.ClusterName != "" { + attempts = append(attempts, aiForwardAttempt{ + ClusterName: selectedTarget.ClusterName, + Model: selectedTarget.Model, + IsFallback: false, + }) + } + for _, fb := range aiResult.Fallbacks { + attempts = append(attempts, aiForwardAttempt{ + ClusterName: fb.ClusterName, + Model: fb.Model, + IsFallback: true, + }) + } + + // ensure request body is rewindable before attempting fallbacks + if len(attempts) > 1 && basicReq.HttpRequest.Body != nil { + if !prepareRequestBodyForRetry(basicReq.HttpRequest) { + log.Logger.Warn("ServeHTTPForAI: request body is not rewindable, disable fallback") + attempts = attempts[:1] + } + } + + for i, attempt := range attempts { + if i > 0 { + // fallback attempt: reset request state + if !p.resetRequestForRetry(basicReq) { + log.Logger.Warn("ServeHTTPForAI: fallback aborted, request body cannot be rewound") + break + } + } + + res, action, lastCluster, invokeErr = p.aiClusterInvoke(srv, serverConf, basicReq, rw, attempt, aiMeta) + if invokeErr == nil && res != nil && res.StatusCode < 500 { + // success or 4xx (client error, do not fallback) + break + } + + // decide whether to try next fallback + if i == len(attempts)-1 { + // last attempt + break + } + if !shouldTriggerFallback(res, invokeErr) { + break + } + + // log fallback + log.Logger.Info("ServeHTTPForAI: fallback triggered, cluster[%s] err[%v] status[%d]", + attempt.ClusterName, invokeErr, getResponseStatus(res)) + + if res != nil { + res.Body.Close() + } + } + + basicReq.HttpResponse = res + basicReq.SvrDataConf = nil + + if err != nil || res == nil { + basicReq.Stat.ResponseStart = time.Now() + basicReq.BfeStatusCode = bfe_http.StatusInternalServerError + res = bfe_basic.CreateInternalSrvErrResp(basicReq) + goto response_got + } + + // set response-phase timeouts based on the last cluster used + if lastCluster != nil { + resFlushInterval = lastCluster.ResFlushInterval() + cancelOnClientClose = lastCluster.CancelOnClientClose() + timeoutWriteClient = lastCluster.TimeoutWriteClient() + timeoutReadClientAgain = lastCluster.TimeoutReadClientAgain() + } + if resFlushInterval == 0 && basicReq.HttpRequest.Header.Get("Accept") == "text/event-stream" { + if lastCluster != nil { + resFlushInterval = lastCluster.DefaultSSEFlushInterval() + } + } + + // ========== response_got / send_response (same as ServeHTTP) ========== + // ... reuse existing response handling code ... + +send_response: + // send http response to client + // ... same as ServeHTTP ... + return +} +``` + +### 4.3 新增辅助类型和函数 + +#### 4.3.1 aiForwardAttempt + +```go +type aiForwardAttempt struct { + ClusterName string + Model string + IsFallback bool +} +``` + +#### 4.3.2 aiClusterInvoke() + +封装一次 AI 目标转发，复用 `clusterInvoke()`。当 cluster 配置了多 API-Key 时，`aiClusterInvoke()` 内部会执行 Key 级选择/重试循环，再返回给外层 cluster 级 fallback 决策。多 API-Key 的详细设计见 [BFE 多 API-Key 支持](./multi_api_key.md)。 + +```go +func (p *ReverseProxy) aiClusterInvoke(srv *BfeServer, serverConf *bfe_route.ServerDataConf, + basicReq *bfe_basic.Request, rw bfe_http.ResponseWriter, + attempt aiForwardAttempt, aiMeta *bfe_basic.AiBasicInfo) ( + res *bfe_http.Response, action int, cluster *bfe_cluster.BfeCluster, err error) { + + req := basicReq.HttpRequest + + // update route info + basicReq.Route.ClusterName = attempt.ClusterName + basicReq.Backend.ClusterName = attempt.ClusterName + + // look up cluster + cluster, err = serverConf.ClusterTable.Lookup(attempt.ClusterName) + if err != nil { + log.Logger.Warn("no cluster for %s", attempt.ClusterName) + basicReq.Stat.ResponseStart = time.Now() + basicReq.ErrCode = bfe_basic.ErrBkNoCluster + basicReq.ErrMsg = err.Error() + p.proxyState.ErrBkNoCluster.Inc(1) + return nil, closeAfterReply, nil, err + } + + // set deadline to finish read client request body + timeoutReadClient := cluster.TimeoutReadClient() + if basicReq.IsSse { + timeoutReadClient = -1 + } + p.setTimeout(bfe_basic.StageReadReqBody, basicReq.Connection, req, timeoutReadClient) + + // no API-Key configured: single forward + if cluster.AIConf == nil || len(cluster.AIConf.Keys) == 0 { + res, action, err = p.doSingleAIForward(srv, cluster, basicReq, rw, attempt, aiMeta, cluster_conf.AIKey{}) + return res, action, cluster, err + } + + // multi API-Key selection and retry loop + policy := defaultAIKeyPolicy() + if cluster.AIConf.KeyPolicy != nil { + policy = *cluster.AIConf.KeyPolicy + } + + state := newAIKeyAttemptState() + var lastErr error + for retry := 0; retry <= policy.MaxRetries; retry++ { + if retry > 0 { + if !rewindRequestBody(basicReq.HttpRequest) { + break + } + time.Sleep(calcBackoff(policy.RetryBackoffInitial, policy.RetryBackoffMax, retry)) + } + + idx, key, ok := chooseNextAIKey(cluster.AIConf.Keys, state) + if !ok { + break + } + + res, action, err = p.doSingleAIForward(srv, cluster, basicReq, rw, attempt, aiMeta, key) + + lastErr = err + statusCode := 0 + if res != nil { + statusCode = res.StatusCode + } + + // success or 4xx client error: stop key-level retry + if err == nil && statusCode < 500 { + return res, action, cluster, nil + } + + // classify failure and decide next key/retry + // 429 -> rotate key; 401/403 -> dead key; 5xx/err -> same key retry with backoff + classifyAIKeyFailure(idx, statusCode, err, state) + } + + return res, action, cluster, lastErr +} +``` + +> 说明：`doSingleAIForward()`、`chooseNextAIKey()`、`calcBackoff()`、`classifyAIKeyFailure()` 等函数的具体实现与失败分类细节，请参考 [BFE 多 API-Key 支持](./multi_api_key.md)。 + +#### 4.3.3 shouldTriggerFallback() + +```go +func shouldTriggerFallback(res *bfe_http.Response, err error) bool { + if err != nil { + return true + } + if res != nil && res.StatusCode >= 500 { + return true + } + return false +} +``` + +#### 4.3.4 resetRequestForRetry() + +```go +func (p *ReverseProxy) resetRequestForRetry(basicReq *bfe_basic.Request) bool { + // clear previous backend connection + if basicReq.Trans.Backend != nil { + basicReq.Trans.Backend.DecConnNum() + basicReq.Trans.Backend = nil + } + basicReq.Trans.Transport = nil + basicReq.RetryTime = 0 + + // reset out request so body can be re-read + basicReq.OutRequest = nil + + // rewind request body for next fallback attempt + if !rewindRequestBody(basicReq.HttpRequest) { + return false + } + + // clear error info from previous attempt + basicReq.ErrCode = nil + basicReq.ErrMsg = "" + return true +} +``` + +#### 4.3.5 prepareRequestBodyForRetry() + +在尝试 fallback 之前，先确保请求体可回退。若当前 body 已实现 `bfe_http.Rewindable` 接口，则直接返回成功；否则通过 `GetBodyAccessor()` 尝试将其转换为 `bytes_body`。当全局 bytes_body 缓冲区大小达到 `TotalBodyBufferSizeLimit()` 限制时，不再包装，fallback 被禁用。 + +```go +func prepareRequestBodyForRetry(req *bfe_http.Request) bool { + // if total buffer size already reaches the limit, do not wrap (no retry) + if limit := bfe_http.TotalBodyBufferSizeLimit(); limit > 0 { + if bfe_http.TotalBytesBodyBuffer() >= limit { + return false + } + } + if req.Body == nil { + return true + } + if _, ok := req.Body.(bfe_http.Rewindable); ok { + return true + } + if _, err := req.GetBodyAccessor(); err != nil { + return false + } + _, ok := req.Body.(bfe_http.Rewindable) + return ok +} +``` + +#### 4.3.6 rewindRequestBody() + +`resetRequestForRetry()` 内部调用，将已支持 `Rewindable` 的请求体重置到起始位置。 + +```go +func rewindRequestBody(req *bfe_http.Request) bool { + if req.Body == nil { + return true + } + rewindable, ok := req.Body.(bfe_http.Rewindable) + if !ok { + return false + } + return rewindable.Rewind() +} +``` + +## 5. 与 ServeHTTP() 的共用逻辑 + +以下逻辑在 `ServeHTTPForAI()` 中直接调用或复用，与 `ServeHTTP()` 保持一致： + +| 逻辑 | 复用方式 | +|------|----------| +| `setClientAddr()` | 直接调用 | +| `HandleBeforeLocation` | 直接调用 | +| `findProduct()` | 直接调用 | +| `HandleFoundProduct` | 直接调用 | +| `HandleAfterLocation` | 直接调用 | +| `httpProtoSet()` | 直接调用 | +| `hopByHopHeaderRemove()` | 直接调用 | +| `clusterInvoke()` | 直接调用 | +| `sendResponse()` | 直接调用 | +| `HandleReadResponse` | 直接调用 | +| `prepareRequestBodyForRetry()` | 直接调用 | +| `rewindRequestBody()` | 直接调用 | +| SSE/EPP/超时处理 | 直接复用 `response_got` 后代码 | + +## 6. Target 选择器 + +`mod_ai_route` 不执行 target 选择，加权随机选择逻辑在 `ServeHTTPForAI()` 中通过 `bfe_server/reverseproxy.go` 中的 `SelectTarget()` 实现： + +```go +import ( + "math/rand" + "time" + + "github.com/bfenetworks/bfe/bfe_basic" +) + +var aiTargetRand = rand.New(rand.NewSource(time.Now().UnixNano())) + +func SelectTarget(targets []bfe_basic.AiRouteTarget) bfe_basic.AiRouteTarget { + if len(targets) == 1 { + return targets[0] + } + + r := aiTargetRand.Intn(100) + sum := 0 + for _, target := range targets { + sum += target.Weight + if r < sum { + return target + } + } + return targets[len(targets)-1] +} +``` + +> 说明：`SelectTarget()` 位于 `bfe_server/reverseproxy.go` 中，供 `ServeHTTPForAI()` 使用。 + +## 7. 模型覆盖逻辑 + +### 7.1 覆盖优先级 + +1. **target/fallback.Model 非空**：覆盖请求体中的 `model` 字段； +2. **cluster.AIConf.ModelMapping**：将当前 `model`（可能是原始 model 或 target 覆盖后的 model）映射为后端模型； +3. **均空**：透传原始 `model`。 + +### 7.2 请求体处理 + +`doSingleAIForward()` 中按以下顺序计算最终模型名： + +1. `attempt.Model` 覆盖（若非空）； +2. `cluster.AIConf.MatchPrefix` 前缀裁剪（若 `StripPrefix=true`）； +3. `cluster.AIConf.ModelMapping` 模型映射（若配置）。 + +计算得到最终 `model` 后，**最多调用一次** `condition.ReqBodyJsonSet()` 写入请求体，避免重复 JSON 解析/序列化： + +```go +if model != aiMeta.ClientModel { + if err := condition.ReqBodyJsonSet(basicReq, "model", model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + // outreq body already changed, need reset Content-Length + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + // Also reset the original request's Content-Length so that fallback/retry + // creates a new outreq with consistent body length. + if basicReq.HttpRequest != nil && basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + aiMeta.TargetModel = model + } +} +``` + +以避免 `Content-Length` 与实际 body 长度不一致，并保证 fallback/retry 时能从原始请求体重新构造 `OutRequest`。 + +## 8. Fallback 机制 + +### 8.1 触发条件 + +触发 fallback： + +- `clusterInvoke()` 返回 `err != nil`（连接失败、超时、读写错误等）； +- `clusterInvoke()` 返回的响应状态码 `>= 500`。 + +不触发 fallback： + +- 后端返回 `4xx`（视为客户端错误）； +- 请求被限流、鉴权失败等（`HandleFoundProduct` 阶段已处理，不会进入转发）。 + +### 8.2 行为 + +- 按 `fallbacks` 列表顺序依次尝试； +- 第一个成功（`err == nil` 且状态码 `< 500`）即停止； +- 所有 fallback 均失败后，返回最后一个 fallback 的响应或错误； +- 每次 fallback 前重置 `OutRequest`、backend 连接、retry 计数等状态。 + +### 8.3 请求体重用 + +fallback 时需确保请求体可重新读取。实现要点： + +- 在转发前调用 `prepareRequestBodyForRetry()`，将非 `Rewindable` 的 body 通过 `GetBodyAccessor()` 包装为可重复读取的 `bytes_body`； +- 若 body 已实现 `bfe_http.Rewindable` 接口，则直接复用； +- 每次 fallback 前由 `resetRequestForRetry()` 调用 `rewindRequestBody()` 将 body 重置到起始位置； +- `aiClusterInvoke()` 每次从 `basicReq.HttpRequest` 重新构造 `OutRequest`。 + +全局 bytes_body 缓冲区受 `bfe_http.TotalBodyBufferSizeLimit()` 限制，达到上限后不再包装 body，fallback 会被禁用。单个请求可访问/缓冲的最大 body 大小由 `ConfigBasic.AccessibleBodySize` 控制（默认取自 `bfe_http.DefaultAccessibleBodySize`），超过该大小的请求体无法通过 `ReqBodyJsonSet` 等接口修改。 + +> 注意：请求体必须在首次消费前具备可回退能力，否则 fallback 将被禁用或失败。 + +## 9. 错误处理 + +| 场景 | 处理方式 | +|------|----------| +| AI 路由未命中 | 返回 404 Not Found | +| 集群不存在 | 复用 `ErrBkNoCluster`，返回 500 | +| target 转发失败 | 触发 fallback | +| 所有 fallback 失败 | 返回最后一个 fallback 的响应；无响应则返回 500 | +| 模型覆盖失败 | 记录 warn 日志，继续转发 | + +## 10. 日志 + +当前实现通过 `log.Logger` 输出以下关键日志： + +- target 命中与选择结果； +- fallback 触发原因（错误类型/状态码）； +- 每次 fallback 尝试的集群名和结果。 + +## 11. 测试覆盖 + +### 11.1 单元测试 + +`bfe_server/reverseproxy_ai_test.go` 已新增，覆盖以下场景： + +1. `ServeHTTPForAI()` 正常转发命中； +2. AI 路由未命中返回 404； +3. `SelectTarget()` 加权随机分布符合预期； +4. target 模型覆盖生效； +5. cluster.AIConf.ModelMapping 生效； +6. 后端 5xx 触发 fallback； +7. 后端 4xx 不触发 fallback； +8. 所有 fallback 失败后返回正确响应； +9. `shouldTriggerFallback()` 边界条件。 + +### 11.2 集成验证 + +- 启用 `EnableAiGateway = true`，配置 `mod_ai_route`，验证完整请求链路； +- 启用 `EnableAiGateway = false`，验证原有 `ServeHTTP()` 不受影响； +- 热加载 `ai_route.data` 后，新请求按新规则转发。 + +## 12. 已完成的修改 + +1. 在 `bfe_server/reverseproxy.go` 中新增 `SelectTarget()`、`aiForwardAttempt`、`aiClusterInvoke()`、`shouldTriggerFallback()`、`resetRequestForRetry()`、`prepareRequestBodyForRetry()`、`rewindRequestBody()` 等辅助函数； +2. 新增 `ServeHTTPForAI()`，复用现有回调和转发逻辑； +3. 修改 `bfe_server/http_conn.go` 中的请求分发逻辑； +4. 新增单元测试 `bfe_server/reverseproxy_ai_test.go`； +5. 编译验证并通过测试。 + +## 13. 注意事项 + +1. **与原 `ServeHTTP()` 的隔离**：`ServeHTTPForAI()` 为独立实现，不修改 `ServeHTTP()` 的状态机，避免影响传统转发； +2. **请求体重复读取**：fallback 依赖请求体可回退能力，`prepareRequestBodyForRetry()` 会提前将 body 包装为 `Rewindable`； +3. **超时设置**：每次 `aiClusterInvoke()` 根据目标集群配置重新设置读请求体超时； +4. **连接数统计**：`resetRequestForRetry()` 递减 backend 连接计数并清空 `Trans.Backend`； +5. **SSE 流式响应**：fallback 仅在首次 target 转发前决策，已开始发送的 SSE 响应不再切换； +6. **EPP 处理**：`ServeHTTPForAI()` 复用了 `ServeHTTP()` 中的 EPP 清理逻辑。 + +## 14. 附录 + +### 14.1 修改文件清单 + +| 文件 | 修改类型 | +|------|----------| +| `bfe_server/reverseproxy.go` | 新增 `ServeHTTPForAI()`、`SelectTarget()` 及辅助函数 | +| `bfe_server/http_conn.go` | 修改请求分发逻辑 | +| `bfe_server/reverseproxy_ai_test.go` | 新增单元测试 | + +### 14.2 关键函数调用关系 + +``` +http_conn.serveRequest() + │ + ├── EnableAiGateway=false + │ └── ReverseProxy.ServeHTTP() + │ + └── EnableAiGateway=true + └── ReverseProxy.ServeHTTPForAI() + ├── setClientAddr() + ├── callbacks: HandleBeforeLocation / findProduct / HandleFoundProduct + ├── GetAiRouteResult() + ├── SelectTarget() ← 新增 + ├── callback: HandleAfterLocation + ├── prepareRequestBodyForRetry() ← 新增 + └── aiClusterInvoke() × (1 + N fallbacks) + ├── resetRequestForRetry()（非首次） ← 新增 + ├── ClusterTable.Lookup() + ├── model override + ├── cluster.AIConf handling + └── clusterInvoke() + └── response handling +``` diff --git a/docs/zh_cn/sys_design/multi_api_key.md b/docs/zh_cn/sys_design/multi_api_key.md new file mode 100644 index 000000000..774fa5945 --- /dev/null +++ b/docs/zh_cn/sys_design/multi_api_key.md @@ -0,0 +1,494 @@ +# BFE 多 API-Key 支持 + +## 1. 背景与目标 + +### 1.1 背景 + +AI 网关场景下，一个后端集群（cluster）通常需要配置多个大模型服务 API-Key： + +- 实现 API-Key 级别的负载分担与故障隔离； +- 当某个 API-Key 因限流（429）或鉴权失败（401/403）失效时，自动切换到其他 Key； +- 对后端 5xx 或连接错误，支持同 Key 退避重试。 + +BFE 已具备 AI 网关独立转发路径 `ServeHTTPForAI()` 与 cluster 级 fallback 机制（见 [mod_ai_route 对应 BFE 主程序修改方案](./mod_ai_route_bfe_changes.md)）。多 API-Key 支持在此基础上，将 Key 级选择/重试内聚到 `aiClusterInvoke()` 中，与外层 cluster 级 fallback 解耦。 + +### 1.2 目标 + +1. `cluster.AIConf` 支持 `Keys` 数组与 `KeyPolicy` 策略； +2. `aiClusterInvoke()` 内按权重选择 API-Key，失败时自动轮换或退避重试； +3. Key 级重试耗尽后，将结果返回给 `ServeHTTPForAI()` 外层，由 cluster 级 fallback 决定是否继续尝试下一个集群； +4. 与 ai-gateway-api 导出的 `server_data_conf` 格式对齐。 + +--- + +## 2. 数据结构 + +### 2.1 BFE 侧 `AIConf` 扩展 + +```go +// AIKey represents a single API key for AI service +type AIKey struct { + Name string // identifier + Key string // API key value + Weight int // weight for weighted random selection, [0,100] +} + +// AIKeyPolicy represents routing/retry policy for AI keys +type AIKeyPolicy struct { + Strategy string // "weighted_random" only in this version + MaxRetries int // total retry budget within one aiClusterInvoke call + RetryBackoffInitial int // ms + RetryBackoffMax int // ms +} + +// ModelPrice represents a single model pricing entry +type ModelPrice struct { + Provider string + Model string + BaseModel string + Mode string + Capabilities []string + SupportedParameters []string + Limits map[string]int + Prices map[string]float64 +} + +// ModelTable represents the cost/pricing table for a cluster +type ModelTable struct { + Currency string // fixed "RMB" in v0.4 + Models []ModelPrice +} + +// AIConf is the AI service configuration for a cluster +type AIConf struct { + Type int + ModelMapping *map[string]string + Provider string // provider name in model_prices + Keys []AIKey // multiple API keys; empty means no key injection + KeyPolicy *AIKeyPolicy // key selection & retry policy + ModelTable *ModelTable // pricing table, auto-filled by InnerAPI +} +``` + +> 说明：旧字段 `AIConf.Key` 不再保留，统一使用 `AIConf.Keys`。 + +### 2.2 配置来源 + +`AIConf` 由 ai-gateway-api 通过 InnerAPI `/configs/tls_conf/server_data_conf` 下发，对应 OpenAPI `/clusters` 中的 `llm_config` 字段。详细导出格式见 `ai-gateway-api/design-docs/api-define/InnerAPI接口定义/server-data-conf.md`。 + +--- + +## 3. 转发层设计 + +### 3.1 与 `ServeHTTPForAI()` 的关系 + +``` +ServeHTTPForAI() + │ + ├── 选择 target + ├── 构建 attempts [selected target + fallbacks] + ├── 准备可回退请求体 + │ + └── 对每个 attempt 循环（cluster 级 fallback） + │ + ▼ + aiClusterInvoke() + │ + ├── 选择 API-Key + ├── 构造 OutRequest + ├── 模型覆盖 / API-Key 注入 + ├── clusterInvoke() + │ + └── 失败？→ Key 轮换 / 同 Key 退避重试 +``` + +- **cluster 级 fallback**：由 `ServeHTTPForAI()` 控制，在 target 失败或后端 5xx 时切换到下一个 fallback cluster； +- **Key 级重试**：由 `aiClusterInvoke()` 控制，在同一 cluster 内多个 API-Key 之间选择/重试。 + +### 3.2 `aiClusterInvoke()` 改造 + +`aiClusterInvoke()` 新增 Key 级重试循环。为支持重试，将单次转发逻辑抽取为 `doSingleAIForward()`： + +```go +func (p *ReverseProxy) doSingleAIForward(srv *BfeServer, cluster *bfe_cluster.BfeCluster, + basicReq *bfe_basic.Request, rw bfe_http.ResponseWriter, + attempt aiForwardAttempt, aiMeta *bfe_basic.AiBasicInfo, + selectedKey cluster_conf.AIKey) ( + res *bfe_http.Response, action int, err error) { + + req := basicReq.HttpRequest + + // prepare out request + outreq := new(bfe_http.Request) + *outreq = *req + basicReq.OutRequest = outreq + + httpProtoSet(outreq) + hopByHopHeaderRemove(outreq, req) + + if cluster.DisableHostHeader { + outreq.Host = "" + } + + // Calculate the final model in order: route target/fallback override -> + // provider/model prefix stripping -> cluster model mapping. Then write it + // to the request body at most once to avoid repeated JSON parsing/serialization. + model := aiMeta.ClientModel + if aiMeta.TargetModel != "" { + model = aiMeta.TargetModel + } + + // apply model override from ai route target/fallback + if attempt.Model != "" { + model = attempt.Model + } + + // strip provider/model prefix according to cluster AIConf + if cluster.AIConf != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + if stripped, ok := stripProviderPrefix(model, cluster.AIConf.MatchPrefix); ok { + model = stripped + } + } + + // apply cluster model mapping + if cluster.AIConf != nil && cluster.AIConf.ModelMapping != nil && model != "" { + if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { + model = newModel + } + } + + if model != aiMeta.ClientModel { + if err := condition.ReqBodyJsonSet(basicReq, "model", model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + // outreq body already changed, need reset Content-Length + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + // Also reset the original request's Content-Length so that fallback/retry + // creates a new outreq with consistent body length. + if basicReq.HttpRequest != nil && basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + aiMeta.TargetModel = model + } + } + + // apply cluster.AIConf (api key) + if cluster.AIConf != nil && selectedKey.Key != "" { + mod_ai_token_auth.SetApiKey(outreq, selectedKey.Key) + } + + // invoke cluster + return p.clusterInvoke(srv, cluster, basicReq, rw) +} +``` + +`aiClusterInvoke()` 内部逻辑： + +```go +func (p *ReverseProxy) aiClusterInvoke(srv *BfeServer, serverConf *bfe_route.ServerDataConf, + basicReq *bfe_basic.Request, rw bfe_http.ResponseWriter, + attempt aiForwardAttempt, aiMeta *bfe_basic.AiBasicInfo) ( + res *bfe_http.Response, action int, cluster *bfe_cluster.BfeCluster, err error) { + + // ... look up cluster ... + + // no keys configured + if cluster.AIConf == nil || len(cluster.AIConf.Keys) == 0 { + res, action, err = p.doSingleAIForward(..., cluster_conf.AIKey{}) + return res, action, cluster, err + } + + policy := defaultAIKeyPolicy() + if cluster.AIConf.KeyPolicy != nil { + policy = *cluster.AIConf.KeyPolicy + } + + keys := cluster.AIConf.Keys + + // ensure request body is rewindable for key-level retry + if policy.MaxRetries > 0 && !prepareRequestBodyForRetry(basicReq.HttpRequest) { + log.Logger.Warn("aiClusterInvoke: request body not rewindable, disable key-level retry") + policy.MaxRetries = 0 + } + + state := &aiKeyAttemptState{ + usedSet: make(map[int]struct{}), + deadSet: make(map[int]struct{}), + } + + var lastErr error + for retry := 0; retry <= policy.MaxRetries; retry++ { + if retry > 0 { + if !rewindRequestBody(basicReq.HttpRequest) { + break + } + time.Sleep(calcBackoff(policy.RetryBackoffInitial, policy.RetryBackoffMax, retry)) + } + + idx, key, ok := chooseNextAIKey(keys, state) + if !ok { + log.Logger.Warn("aiClusterInvoke: all ai keys exhausted for cluster[%s]", attempt.ClusterName) + break + } + + res, action, err = p.doSingleAIForward(..., key) + + lastErr = err + statusCode := 0 + if res != nil { + statusCode = res.StatusCode + } + + // success or 4xx client error + if err == nil && statusCode < 500 { + return res, action, cluster, nil + } + + // classify failure + switch { + case statusCode == 429: + state.usedSet[idx] = struct{}{} // rotate key + case statusCode == 401 || statusCode == 402 || statusCode == 403: + state.deadSet[idx] = struct{}{} // dead key + case statusCode >= 500 || err != nil: + // transient failure, retry same key with backoff + } + } + + return res, action, cluster, lastErr +} +``` + +### 3.3 Key 选择辅助函数 + +```go +// aiKeyAttemptState tracks key usage within one aiClusterInvoke call +type aiKeyAttemptState struct { + usedSet map[int]struct{} // keys used for 429 + deadSet map[int]struct{} // keys dead for 401/402/403 +} + +var aiKeyRand = rand.New(rand.NewSource(time.Now().UnixNano())) + +// selectAIKey selects one key by weighted random. +func selectAIKey(keys []cluster_conf.AIKey) (cluster_conf.AIKey, int) { + if len(keys) == 1 { + return keys[0], 0 + } + + total := 0 + for _, k := range keys { + total += k.Weight + } + if total <= 0 { + return cluster_conf.AIKey{}, -1 + } + + r := aiKeyRand.Intn(total) + sum := 0 + for i, k := range keys { + sum += k.Weight + if r < sum { + return k, i + } + } + return keys[len(keys)-1], len(keys) - 1 +} + +// chooseNextAIKey returns next eligible key and its index. +func chooseNextAIKey(keys []cluster_conf.AIKey, state *aiKeyAttemptState) (int, cluster_conf.AIKey, bool) { + var eligible []cluster_conf.AIKey + var indices []int + + for i, k := range keys { + if k.Weight == 0 { + continue + } + if _, dead := state.deadSet[i]; dead { + continue + } + eligible = append(eligible, k) + indices = append(indices, i) + } + + if len(eligible) == 0 { + return -1, cluster_conf.AIKey{}, false + } + + var filteredKeys []cluster_conf.AIKey + var filteredIdx []int + for j, k := range eligible { + idx := indices[j] + if _, used := state.usedSet[idx]; used { + continue + } + filteredKeys = append(filteredKeys, k) + filteredIdx = append(filteredIdx, idx) + } + + if len(filteredKeys) == 0 { + // all alive keys used (429 only), reset used_set and retry + state.usedSet = make(map[int]struct{}) + filteredKeys = eligible + filteredIdx = indices + } + + _, selectedIdx := selectAIKey(filteredKeys) + if selectedIdx < 0 { + return -1, cluster_conf.AIKey{}, false + } + return filteredIdx[selectedIdx], filteredKeys[selectedIdx], true +} + +// calcBackoff calculates exponential backoff with jitter. +func calcBackoff(initial, max, attempt int) time.Duration { + backoff := initial + for i := 1; i < attempt; i++ { + backoff *= 2 + if backoff > max { + backoff = max + break + } + } + jitter := backoff / 5 + if jitter > 0 { + backoff = backoff - jitter/2 + aiKeyRand.Intn(jitter) + } + return time.Duration(backoff) * time.Millisecond +} +``` + +--- + +## 4. 失败分类与边界 + +### 4.1 Key 级失败处理 + +| 错误类型 | 处理方式 | +| - | - | +| 429 Too Many Requests | 标记当前 Key 为 `used`，轮换到其他 Key | +| 401 / 402 / 403 | 标记当前 Key 为 `dead`，不再使用 | +| 5xx / 连接错误 / 超时 | 视为瞬态失败，同 Key 退避重试 | +| 成功或 4xx（除上述外） | 立即返回，停止 Key 级重试 | + +### 4.2 与 cluster 级 fallback 的边界 + +`aiClusterInvoke()` 将最终结果返回给 `ServeHTTPForAI()` 外层： + +- 若 Key 级重试最终得到 2xx/3xx，直接返回给客户端； +- 若 Key 级重试最终得到 5xx、连接错误或特定 4xx（400/401/402/403/422/429），`shouldTriggerFallback()` 返回 true，触发 cluster fallback； +- 若得到其他 4xx（如 404/405 等请求级错误），不触发 cluster fallback，直接返回。 + +```go +var aiFallbackStatusCodes = map[int]struct{}{ + 400: {}, + 401: {}, + 402: {}, + 403: {}, + 422: {}, + 429: {}, +} + +func shouldTriggerFallback(res *bfe_http.Response, err error) bool { + if err != nil { + return true + } + code := getResponseStatus(res) + if code >= 500 { + return true + } + if _, ok := aiFallbackStatusCodes[code]; ok { + return true + } + return false +} +``` + +--- + +## 5. 监控与日志 + +建议增加的监控指标： + +| 指标 | 类型 | 含义 | +| - | - | - | +| `ReqAiKeyRotation` | Counter | Key 轮换次数（按 429/401/403 分类） | +| `ReqAiKeyRetry` | Counter | Key 级重试次数 | +| `ReqAiKeyExhausted` | Counter | Key 全部耗尽次数 | + +关键日志： + +``` +aiClusterInvoke: select ai key [name=%s weight=%d] for cluster[%s] +aiClusterInvoke: ai key [name=%s] failed with status[%d], rotate/dead/retry +aiClusterInvoke: all ai keys exhausted for cluster[%s] +``` + +--- + +## 6. 配置示例 + +```json +{ + "AIConf": { + "Type": 0, + "Provider": "deepseek", + "Keys": [ + { + "Name": "key-primary", + "Key": "sk-aaaaaaaaaaaa", + "Weight": 70 + }, + { + "Name": "key-secondary", + "Key": "sk-bbbbbbbbbbbb", + "Weight": 30 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 3, + "RetryBackoffInitial": 500, + "RetryBackoffMax": 5000 + }, + "ModelMapping": { + "gpt-4": "deepseek-v3" + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "deepseek", + "Model": "deepseek-v3", + "BaseModel": "deepseek-v3", + "Mode": "chat", + "Capabilities": ["chat", "reasoning", "tools"], + "SupportedParameters": ["temperature", "max_tokens"], + "Limits": { + "context_window": 128000, + "max_input_tokens": 128000, + "max_output_tokens": 8192 + }, + "Prices": { + "input_cost_per_token": 0.000002, + "output_cost_per_token": 0.000008 + } + } + ] + } + } +} +``` + +--- + +## 7. 注意事项 + +1. **请求体可回退性**：Key 级重试依赖 `basicReq.HttpRequest.Body` 可重复读取。`aiClusterInvoke()` 会在启用 Key 级重试前调用 `prepareRequestBodyForRetry()`； +2. **SSE 流式响应**：所有 Key 尝试完成后才返回响应，不会出现已开始发送后切换 Key 的情况； +3. **与 `ServeHTTP()` 隔离**：多 API-Key 逻辑仅作用于 `ServeHTTPForAI()` 路径； +4. **旧字段清理**：`AIConf.Key` 不再保留，统一使用 `AIConf.Keys`。 diff --git a/docs/zh_cn/sys_design/provider_model_prefix_routing.md b/docs/zh_cn/sys_design/provider_model_prefix_routing.md new file mode 100644 index 000000000..08e7a395c --- /dev/null +++ b/docs/zh_cn/sys_design/provider_model_prefix_routing.md @@ -0,0 +1,262 @@ +# Provider/Model 前缀路由裁剪 + +## 1. 背景与目标 + +### 1.1 背景 + +在 AI 网关实际使用中，部分模型聚合平台（如 OpenRouter）要求客户端在请求 Body 的 `model` 字段中携带 `provider_name/model_name` 格式，例如： + +```json +{ + "model": "openrouter/anthropic/claude-sonnet-4.6", + "messages": [...] +} +``` + +当请求命中对应的 cluster 后，BFE 转发给下游前需要将该前缀裁剪掉，使下游收到的是平台内部认可的模型名： + +```json +{ + "model": "anthropic/claude-sonnet-4.6", + "messages": [...] +} +``` + +本方案在 cluster 级别增加 `match_prefix` / `strip_prefix` 开关，由 ai-gateway-api 负责配置下发，BFE 负责实际裁剪逻辑。 + +### 1.2 目标 + +1. 支持 cluster 配置 `MatchPrefix` 和 `StripPrefix`； +2. BFE 在 `doSingleAIForward()` 中按配置完成前缀裁剪； +3. 保持 `ClientModel` 不变，更新 `TargetModel` 以反映下游实际模型名； +4. 前缀裁剪不影响 key-level retry 和 route-level fallback 的正确性； +5. 明确当前方案对 Token 鉴权、限流模块的局限性。 + +## 2. 设计原则 + +- **配置驱动**：是否裁剪、裁剪什么前缀完全由 cluster 配置决定，不做硬编码； +- **最小侵入**：只在 `doSingleAIForward()` 中插入裁剪逻辑，不修改已有路由、鉴权、限流流程； +- **状态清晰**：`ClientModel` 保持客户端原始值，`TargetModel` 表示当前应向下游发送的模型名； +- **向后兼容**：未配置 `MatchPrefix` / `StripPrefix` 时保持现有行为不变。 + +## 3. 配置扩展 + +### 3.1 `AIConf` 结构体扩展 + +**文件：** `bfe/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go` + +```go +type AIConf struct { + Type int // type of LLM service, reserved for future use. should be 0 now. + ModelMapping *map[string]string // model mapping, key is model name in req, value is model name in backend + Provider string // provider name in model_prices + Keys []AIKey // multiple API keys; empty means no key injection + KeyPolicy *AIKeyPolicy // key selection & retry policy + ModelTable *ModelTable // pricing table, auto-filled by InnerAPI + + // 新增 + MatchPrefix string `json:"MatchPrefix,omitempty"` // 例如 "openrouter/" + StripPrefix bool `json:"StripPrefix"` // 是否裁剪该前缀 +} +``` + +说明： + +- `MatchPrefix`：定义该 cluster 负责匹配的前缀，必须以 `/` 结尾。 +- `StripPrefix`：匹配成功后，转发给下游前是否裁剪该前缀。 + +### 3.2 配置加载时校验 + +在 `AIConfCheck()` 中增加校验逻辑： + +```go +func AIConfCheck(conf *AIConf) error { + if conf.ModelTable != nil { + if err := ModelTableCheck(conf.ModelTable); err != nil { + return fmt.Errorf("ModelTable:%s", err.Error()) + } + } + + // 新增校验 + if conf.StripPrefix { + if conf.MatchPrefix == "" { + return fmt.Errorf("MatchPrefix is required when StripPrefix is true") + } + if !strings.HasSuffix(conf.MatchPrefix, "/") { + return fmt.Errorf("MatchPrefix must end with '/'") + } + } + + return nil +} +``` + +说明： + +- `StripPrefix=true` 时，`MatchPrefix` 必须非空； +- `MatchPrefix` 必须以 `/` 结尾，避免前缀匹配到模型名本身。 + +## 4. BFE 转发裁剪逻辑 + +### 4.1 裁剪位置 + +**文件：** `bfe/bfe_server/reverseproxy.go` + +在 `doSingleAIForward()` 函数中，按 **route target model override** → **provider 前缀裁剪** → **cluster `ModelMapping`** 的顺序计算最终模型名，然后统一写入请求体： + +```go +// Calculate the final model in order: route target/fallback override -> +// provider/model prefix stripping -> cluster model mapping. Then write it +// to the request body at most once. +model := aiMeta.ClientModel +if aiMeta.TargetModel != "" { + model = aiMeta.TargetModel +} + +// apply model override from ai route target/fallback +if attempt.Model != "" { + model = attempt.Model +} + +// 按 cluster AIConf 裁剪 provider 前缀 +if cluster.AIConf != nil && cluster.AIConf.StripPrefix && cluster.AIConf.MatchPrefix != "" { + if stripped, ok := stripProviderPrefix(model, cluster.AIConf.MatchPrefix); ok { + model = stripped + } +} + +// apply cluster model mapping +if cluster.AIConf != nil && cluster.AIConf.ModelMapping != nil && model != "" { + if newModel, ok := (*cluster.AIConf.ModelMapping)[model]; ok { + model = newModel + } +} + +// 统一写入请求体（最多一次） +if model != aiMeta.ClientModel { + if err := condition.ReqBodyJsonSet(basicReq, "model", model); err != nil { + log.Logger.Warn("Failed to set model in request body: %s", err) + } else { + // outreq body already changed, need reset Content-Length + if outreq.ContentLength >= 0 { + outreq.ContentLength = -1 + outreq.Header.Del("Content-Length") + } + // Also reset the original request's Content-Length so that fallback/retry + // creates a new outreq with consistent body length. + if basicReq.HttpRequest != nil && basicReq.HttpRequest.ContentLength >= 0 { + basicReq.HttpRequest.ContentLength = -1 + basicReq.HttpRequest.Header.Del("Content-Length") + } + aiMeta.TargetModel = model + } +} +``` + +关键细节： + +- 只裁剪 **第一段** provider 前缀。例如 `openrouter/anthropic/claude-xxx` → `anthropic/claude-xxx`。 +- 裁剪后再执行 `ModelMapping`，因此 `ModelMapping` 的 key 应使用裁剪后的模型名。 +- 如果 `attempt.Model` 已覆盖目标模型，则基于覆盖后的模型进行前缀裁剪。 +- 裁剪后内容为空时（如 `openrouter/`），跳过裁剪并记录 warn，避免下发空 model。 + +### 4.2 `ClientModel` 与 `TargetModel` 的区分 + +**文件：** `bfe/bfe_basic/request_ai_basic.go` + +`AiBasicInfo` 结构体已区分 `ClientModel`（客户端原始模型名）和 `TargetModel`（转发给下游的模型名）。前缀裁剪后应更新 `TargetModel`，保持语义一致。 + +当前 `http_conn.go` 初始化时： + +```go +model, err := condition.ReqBodyJsonFetch(request, "model", nil) +if err == nil || len(model) > 0 { + aiMeta.ClientModel = model + aiMeta.TargetModel = model +} +``` + +`ClientModel` 保持原始值不变，`TargetModel` 在 `doSingleAIForward()` 中随裁剪/映射更新。 + +## 5. 重试安全性分析 + +修改 `aiMeta.TargetModel` 不会影响下一次重试的正确性，原因如下： + +1. **请求体不会被回滚为原始请求体**：BFE 的 `bytes_body.Rewind()` 只是将当前 buffer 重新设为读取起点，不会恢复到客户端原始 body。因此已裁剪的请求内容在重试时会被保留。 +2. **`HasPrefix` 保护避免重复裁剪**：第一次裁剪后 `TargetModel` 已不含该前缀，下次进入 `doSingleAIForward()` 时不会再命中 `MatchPrefix`，不会二次裁剪。 +3. **`ClientModel` 始终保持原始值**：日志、计费等地方仍能看到客户端原始模型名。 + +下文给出关键结论。 + +## 6. 对 Token 鉴权和限流的影响 + +### 6.1 Token 鉴权的模型允许列表 + +**文件：** `bfe/bfe_modules/mod_ai_token_auth/token_rule_table.go` + +`ValidateUserTokenByReq()` 从请求 Body 读取 `model` 并与 `token.Models` / `token.BlockModels` 做精确匹配。 + +**本次 v0.4 暂时采用方案 A**：保持现有 token 鉴权逻辑不变，客户在 API Key 的 `models` / `block_models` 中配置带前缀的完整模型名。 + +> ⚠️ **局限性**：方案 A 仅适用于请求模型名与产品语义上的真实模型名一致或一一对应的场景。对于 OpenRouter 等聚合中转站，请求模型名、裁剪后模型名、`ModelMapping` 目标模型名、ai-gateway-api 中的 `BaseModel` 四者可能各不相同。若 Token 允许列表仅按原始请求模型名配置，既无法表达“允许使用某个 BaseModel”的真实意图，也无法覆盖同一模型通过不同前缀请求的情况。后续需要引入基于 `BaseModel` 的鉴权机制。 + +### 6.2 限流的模型匹配 + +**文件：** `bfe/bfe_modules/mod_ai_rate_limit/mod_ai_rate_limit.go` + +`matchModel()` 支持精确匹配和 `*` 通配符。限流策略中的 `model` 配置与 token 鉴权类似： + +- 若配置带前缀的模型名（如 `openrouter/anthropic/claude-xxx`），当前逻辑可直接工作； +- 若希望按裁剪后模型名限流，需要前置裁剪逻辑。 + +**本次 v0.4 保持现状**，由用户在限流策略中配置与请求一致的模型名格式。 + +> ⚠️ **局限性**：当前限流按 `meta.ClientModel`（即原始请求模型名）做匹配和计数 key。在 OpenRouter 等聚合中转场景下，同一真实模型可能以多种请求形态出现，限流会被拆分为多个独立的 key，导致限流失准。更深层的问题在于：限流真正需要按“归一化模型名（`BaseModel`）”聚合，而不是按“裁剪后模型名”或“原始请求模型名”。具体原因在下文展开。 + +## 7. 配置示例 + +ai-gateway-api 下发到 BFE 的 cluster 配置示例： + +```json +{ + "ClusterBasic": {...}, + "BackendConf": {...}, + "AIConf": { + "Type": 0, + "Provider": "openrouter", + "MatchPrefix": "openrouter/", + "StripPrefix": true, + "Keys": [ + { + "Name": "default", + "Key": "sk-xxx", + "Weight": 100 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 0, + "RetryBackoffInitial": 500, + "RetryBackoffMax": 5000 + } + } +} +``` + +效果： + +- 客户端请求 `model: "openrouter/anthropic/claude-sonnet-4.6"` +- 路由到该 cluster +- BFE 转发给 OpenRouter 时，`model` 变为 `"anthropic/claude-sonnet-4.6"` + +## 8. 总结 + +BFE 侧改动范围： + +1. `bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go`： + - `AIConf` 增加 `MatchPrefix` / `StripPrefix`； + - `AIConfCheck()` 增加校验逻辑。 +2. `bfe_server/reverseproxy.go`： + - `doSingleAIForward()` 中插入前缀裁剪逻辑。 + +实现原则：**最小化改动，仅在 cluster 配置层增加开关，不侵入现有路由、鉴权、限流逻辑。** 对于 OpenRouter 等聚合 provider 场景下 Token 鉴权和限流的问题，本期仅做标注，后续需引入基于 `BaseModel` 的鉴权/限流机制。 diff --git a/docs/zh_cn/sys_design/rmb_quota.md b/docs/zh_cn/sys_design/rmb_quota.md new file mode 100644 index 000000000..368a0bdf5 --- /dev/null +++ b/docs/zh_cn/sys_design/rmb_quota.md @@ -0,0 +1,602 @@ +# BFE RMB 配额支持 + +## 1. 背景与目标 + +### 1.1 背景 + +当前 BFE 的配额扣减流程只支持 **Token** 单位： + +1. 认证阶段：`mod_ai_token_auth` 校验 API Key，并通过 `QuotaPlan.HasBalance()` 检查 Redis 余额是否大于 0。 +2. 响应阶段：`mod_body_process` / `mod_ai_token_auth` 从响应中提取 `prompt_tokens` / `completion_tokens`；`mod_ai_token_auth` 在请求结束时通过 Lua 脚本从 Redis 整数扣减 Token 数。 + +引入 **RMB（人民币）** 配额后，需要在响应阶段： + +- 根据实际命中的 `cluster` 和 `target_model` 查找定价表； +- 把 `prompt_tokens` / `completion_tokens` 换算成人民币成本； +- 对 `unit = "RMB"` 的配额计划扣减相应金额。 + +### 1.2 目标 + +1. 配置层沿用 `AIConf.ModelTable`，价格以 `Prices` map（元/Token）下发，BFE 加载时转换为 1e-8 元/Token 定点整数； +2. `bfe_basic.TokenUsage` 增加 `UsedCost`，用于记录本次请求的 RMB 成本； +3. `mod_ai_token_auth.QuotaPlan` 增加 `Unit`，`Deduct` / `HasBalance` 支持 RMB； +4. 新增共享库 `go-lib/quota`，提供 RMB 定点数转换，供 ai-gateway-api 与 BFE 共同引用； +5. Redis Lua 支持 RMB 扣减脚本，当前暂时使用单 Key 定点数方案。 + +## 2. 设计原则 + +- **向后兼容**：存量 Token 配额完全兼容，`Unit` 默认 `"total_token"`，走原有扣减逻辑； +- **定点整数**：所有金额在 BFE 内部和 Redis 中均以定点整数表示，避免浮点误差； +- **配置不下发整数**：conf-agent 只负责配置下发，价格仍按原始浮点数下发，转换在 BFE 内部完成； +- **共享转换逻辑**：ai-gateway-api 与 BFE 共用 `go-lib/quota`，保证管理面与数据面对 Redis 值的解释一致。 + +## 3. 总体架构 + +``` +┌─────────────────────────────────────────┐ +│ conf-agent 下发 cluster_conf.data │ +│ (AIConf.ModelTable 价格仍为浮点数) │ +└─────────────────┬───────────────────────┘ + ▼ +┌─────────────────────────────────────────┐ +│ bfe_config/bfe_cluster_conf/... │ +│ 加载 AIConf，校验并构建 priceIndex │ +│ 通过 go-lib/quota 转换定点整数价格 │ +└─────────────────┬───────────────────────┘ + ▼ +┌─────────────────────────────────────────┐ +│ 请求运行时 │ +│ - 认证阶段：HasBalance() 按单位检查余额 │ +│ - 响应阶段：calcCostUnits() 计算 RMB 成本│ +│ - 扣减阶段：Lua 脚本原子扣减定点整数 │ +└─────────────────────────────────────────┘ +``` + +## 4. 配置层设计 + +### 4.1 文件 + +`bfe/bfe_config/bfe_cluster_conf/cluster_conf/cluster_conf_load.go` + +### 4.2 数据结构 + +BFE 侧 `AIConf` 已存在 `ModelTable`（v0.4 货币固定为 RMB），结构如下： + +```go +type ModelPrice struct { + Provider string + Model string // 模型名，用于匹配请求中的 target_model + BaseModel string + Mode string // 请求模式，如 "chat" + Capabilities []string + SupportedParameters []string + Limits map[string]interface{} + Prices map[string]float64 // 价格对象，如 input_cost_per_token / output_cost_per_token + Metadata map[string]interface{} +} + +type ModelTable struct { + Currency string // v0.4 固定为 "RMB" + Models []ModelPrice + + // 运行时索引，配置加载阶段构建：model -> mode -> *ModelPrice + priceIndex map[string]map[string]*ModelPrice +} + +type AIConf struct { + Type int // 保留字段，当前应为 0 + ModelMapping *map[string]string // 模型映射 + Provider string // provider 名 + Keys []AIKey // 多 Key 模式 + KeyPolicy *AIKeyPolicy // Key 选择策略 + ModelTable *ModelTable // 成本定价表 +} +``` + +> 说明：`AIConf` 旧字段 `Key` 已移除，统一使用 `Keys`。 + +### 4.3 校验规则 + +1. `ModelTable.Currency` 当前仅允许 `"RMB"`。 +2. `ModelPrice.Prices["input_cost_per_token"]`、`Prices["output_cost_per_token"]` 必须 `>= 0`。 +3. `Model` 为具体模型名；`Mode` 如 `"chat"`。 +4. 同一个 `Mode` 下，`Model` 不能重复。 +5. 加载时构建二维索引 `priceIndex[model][mode]`，便于运行时 O(1) 查询。 +6. 加载阶段通过 `go-lib/quota.RmbToFixedPoint` 将浮点价格转换为定点整数，BFE 内部和 Redis 中只使用整数。 + +### 4.4 配置加载阶段处理 + +conf-agent 只负责配置下发，不做任何数据转换。`cluster_conf.data` 中 `AIConf.ModelTable.Models[].Prices` 仍按原始浮点数（元/Token）下发。 + +当 `unit = "RMB"` 时，小数到整数的转换及索引构建必须在 BFE 内部完成，例如在 `ClusterConfCheck` 或 `AIConf` 专有校验阶段。转换逻辑统一放到共享库 `go-lib/quota`： + +```go +import "github.com/bfenetworks/go-lib/quota" + +func buildModelTableIndex(table *ModelTable) error { + table.priceIndex = make(map[string]map[string]*ModelPrice) + + for i := range table.Models { + price := &table.Models[i] + + // 1. 价格转换：浮点元/Token -> 1e-8 元/Token 定点整数 + input := price.Prices["input_cost_per_token"] + output := price.Prices["output_cost_per_token"] + if input < 0 || output < 0 { + return fmt.Errorf("negative price for model %s", price.Model) + } + price.Prices["input_cost_per_token_int"] = float64(quota.RmbToFixedPoint(input)) + price.Prices["output_cost_per_token_int"] = float64(quota.RmbToFixedPoint(output)) + + // 2. 构建 model -> mode 二维索引 + if table.priceIndex[price.Model] == nil { + table.priceIndex[price.Model] = make(map[string]*ModelPrice) + } + table.priceIndex[price.Model][price.Mode] = price + } + return nil +} +``` + +> 说明： +> - 转换后 BFE 内部及 Redis Lua 中只使用整数，避免浮点误差。 +> - conf-agent 不感知 `unit` 类型，也不修改价格格式。 +> - `go-lib/quota` 同时被 ai-gateway-api 和 BFE 引用，保证管理面与数据面对 Redis 值的解释完全一致。 + +## 5. 共享库 `go-lib/quota` + +为避免 ai-gateway-api 与 BFE 对 Redis 中 RMB 配额值的解释不一致，定点数转换逻辑统一抽取到 `go-lib/quota`： + +```go +package quota + +const ( + UnitTotalToken = "total_token" + UnitRMB = "RMB" +) + +const RmbPrecision = 1e8 + +// RmbToFixedPoint converts yuan to a fixed-point integer (1e-8 yuan per unit). +func RmbToFixedPoint(yuan float64) int64 + +// FixedPointToRmb converts a fixed-point integer back to yuan. +func FixedPointToRmb(value int64) float64 + +// ToRedisValue converts a quota value to a Redis fixed-point integer. +func ToRedisValue(quota float64, unit string) int64 + +// FromRedisValue converts a Redis fixed-point integer back to a quota value. +func FromRedisValue(value int64, unit string) float64 +``` + +职责边界： + +- **`go-lib/quota`**：只负责 **单位与定点数之间的转换**，不依赖 Redis 客户端，不执行任何 Redis 命令。 +- **ai-gateway-api**：引用 `go-lib/quota`，负责管理面配额的初始化、重置、同步（使用 `IncrBy` 等）。 +- **BFE**：引用 `go-lib/quota`，负责数据面请求成本的计算与 Lua 原子扣减。 + +## 6. 基础数据结构改动 + +### 6.1 `TokenUsage` + +`bfe/bfe_basic/request_ai_basic.go` + +```go +type TokenUsage struct { + PromptTokens int64 // 请求侧 Token 数 + CompletionTokens int64 // 响应侧 Token 数 + UsedQuota int64 // 已用 Token 配额（unit=total_token 时使用） + UsedCost int64 // 已用 RMB 成本，1 单位 = 1e-8 元（unit=RMB 时使用） +} +``` + +### 6.2 `QuotaPlan` + +`bfe/bfe_modules/mod_ai_token_auth/token.go` + +```go +type QuotaPlan struct { + Id string + Unlimited bool + PassNoQuota bool + RedisKey string + CreateTime int64 + ExpiredTime int64 + Quota int64 // 固定点整数：total_token 时为 Token 数；RMB 时为 1e-8 元 + ResetMode int + Unit string // 新增："total_token" 或 "RMB" +} +``` + +> 说明： +> - `Quota` 保持 `int64` 不变，但语义由 `Unit` 字段解释。这样可完全避免 `float64` 在 Redis Lua 和大额余额中的精度问题。 +> - `Unit` 本身已隐含货币类型（如 `"RMB"`），`QuotaPlan` 不需要额外的 `Currency` 字段。 + +### 6.3 配置校验 + +`bfe/bfe_modules/mod_ai_token_auth/token_rule_load.go` + +`quotaPlanCheck` 需要调整： + +- `Unit` 为空时默认 `"total_token"`，保持兼容。 +- `Unit = "total_token"`：`Unlimited=false` 时 `Quota > 0`。 +- `Unit = "RMB"`：`Unlimited=false` 时 `Quota >= 0`。 + +## 7. 请求运行时改动 + +### 7.1 认证阶段：`ValidateUserTokenByReq` + +当前逻辑已经遍历 `token.QuotaPlans` 并调用 `plan.HasBalance()`。对 RMB 配额： + +- **不做按请求成本的精确预检**（因为最终输出 Token 数未知）。 +- 仍按余额是否大于 0 进行粗略预检；若余额为 0 则拒绝。 + +> 如果需要更严格（如按 `max_tokens` 估算最坏成本），可在后续迭代中补充。 + +### 7.2 在 `TokenAuthContext` 中缓存 `serverConf` + +`bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go` + +BFE 的 reverse proxy 在请求结束前会将 `req.SvrDataConf` 清空为 `nil`。为了在最后扣减阶段仍能访问 cluster 配置，`SetTokenAuthContext` 在认证阶段把 `req.SvrDataConf` 缓存到 `TokenAuthContext` 中： + +```go +type TokenAuthContext struct { + Token *Token + aiBasicInfo *bfe_basic.AiBasicInfo + // serverConf caches the SvrDataConf before it is cleared by the reverse proxy. + serverConf bfe_basic.ServerDataConfInterface +} + +func SetTokenAuthContext(req *bfe_basic.Request, tok *Token, promptToken int64, tags []bfe_basic.ApikeyTag) { + aiBasicInfo := req.GetAiBasicInfo() + if aiBasicInfo != nil { + tusage := aiBasicInfo.GetTokenUsage() + tusage.PromptTokens = promptToken + tusage.CompletionTokens = bfe_basic.COMPLETION_TOKENS_UNKNOWN + aiBasicInfo.ApikeyTags = tags + } + + tokenCtx := &TokenAuthContext{ + Token: tok, + aiBasicInfo: aiBasicInfo, + serverConf: req.SvrDataConf, + } + req.SetContext(REQ_TOKEN_AUTH_CONTEXT, tokenCtx) +} +``` + +### 7.3 响应阶段：`tokenReadResponseHandler` + +`bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go` + +响应阶段负责从响应体中提取 `usage`，或在未返回 `usage` 时按响应体长度估算 Token 数。对于非流式响应，`ContentLength >= 0` 时可直接读取完整响应体： + +```go +func (m *ModuleAITokenAuth) tokenReadResponseHandler(req *bfe_basic.Request, res *bfe_http.Response) int { + ctx := GetTokenAuthContext(req) + if ctx == nil { + return bfe_module.BfeHandlerGoOn + } + tokenUsage := ctx.aiBasicInfo.GetTokenUsage() + if res.StatusCode == bfe_http.StatusOK && res.ContentLength >= 0 { + if bodyAccessor, err := res.GetBodyAccessor(); err == nil { + body, _ := bodyAccessor.GetBytes() + UpdateCtxByUsage(ctx, body) + } + if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { + tokenUsage.CompletionTokens = int64(res.ContentLength) / 4 + tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + } + + return bfe_module.BfeHandlerGoOn +} +``` + +> 说明：旧实现中 RMB 成本在此阶段计算，导致流式响应（`ContentLength = -1`）无法计费。当前实现已将成本计算移到请求结束阶段，见 7.4。 + +#### 流式响应的 Token 用量收集 + +对于 `stream: true` 的 SSE 流式响应，`mod_body_process` 默认会注册 `QuotaUsageProcessor`： + +- `mod_body_process.DoResponseProcess` 根据响应 `Content-Type` 选择 SSE 解码器； +- 每个 SSE 事件经过 `QuotaUsageProcessor.Process` 时，会从事件数据中提取 `usage.*_tokens`； +- 当遇到包含 `usage` 的最后一个事件时，将 `PromptTokens` / `CompletionTokens` / `UsedQuota` 写入 `AiBasicInfo.TokenUsage`。 + +因此，到请求结束阶段，`tokenUsage.PromptTokens` 和 `tokenUsage.CompletionTokens` 已经就绪，无论流式还是非流式都可以统一计算 RMB 成本。 + +### 7.4 请求结束阶段：`tokenRequestFinishHandler` + +`bfe/bfe_modules/mod_ai_token_auth/mod_ai_token_auth.go` + +请求结束阶段统一计算 RMB 成本并扣减。`TokenAuthContext` 中已缓存 `serverConf`，因此即使 `req.SvrDataConf` 已被 reverse proxy 清空，仍然可以访问 cluster 定价表： + +```go +func (m *ModuleAITokenAuth) tokenRequestFinishHandler(req *bfe_basic.Request, res *bfe_http.Response) int { + if res == nil || res.StatusCode != bfe_http.StatusOK { + return bfe_module.BfeHandlerGoOn + } + + ctx := GetTokenAuthContext(req) + if ctx == nil { + return bfe_module.BfeHandlerGoOn + } + + tokenUsage := ctx.aiBasicInfo.GetTokenUsage() + if tokenUsage.UsedQuota <= 0 && ctx.aiBasicInfo.IsAllowEstimateToken() { + tokenUsage.UsedQuota = CalcReqUsedQuota(req, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + + // 统一在请求完成阶段计算 RMB 成本（流式由 mod_body_process 填充 token 用量） + if tokenUsage.UsedCost <= 0 && hasRMBPlan(ctx.Token.QuotaPlans) { + tokenUsage.UsedCost = m.calcCostUnits(req, ctx.serverConf, tokenUsage.PromptTokens, tokenUsage.CompletionTokens) + } + + costUnits := tokenUsage.UsedCost + + if tokenUsage.UsedQuota > 0 || costUnits > 0 { + for _, plan := range ctx.Token.QuotaPlans { + if plan.Unlimited { + continue + } + if plan.Unit == "RMB" { + if costUnits > 0 { + _, err := plan.Deduct(m.redisClient, costUnits) + if err != nil { + log.Logger.Warn("deduct rmb quota failed: %v", err) + } + } + } else { + if tokenUsage.UsedQuota > 0 { + _, err := plan.Deduct(m.redisClient, tokenUsage.UsedQuota) + if err != nil { + log.Logger.Warn("deduct token quota failed: %v", err) + } + } + } + } + } + + return bfe_module.BfeHandlerGoOn +} +``` + +### 7.5 成本计算辅助方法 + +新增方法（位于 `mod_ai_token_auth`）： + +```go +func (m *ModuleAITokenAuth) calcCostUnits(req *bfe_basic.Request, serverConf bfe_basic.ServerDataConfInterface, promptTokens, completionTokens int64) int64 { + aiMeta := req.GetAiBasicInfo() + if aiMeta == nil { + return 0 + } + + clusterName := req.Route.ClusterName + targetModel := aiMeta.TargetModel + if clusterName == "" || targetModel == "" { + return 0 + } + + if serverConf == nil { + return 0 + } + cluster, err := serverConf.ClusterTableLookup(clusterName) + if err != nil || cluster == nil || cluster.AIConf == nil || cluster.AIConf.ModelTable == nil { + log.Logger.Warn("model table not found for cluster %s", clusterName) + return 0 + } + + entry := cluster_conf.LookupModelPrice(cluster.AIConf.ModelTable, targetModel, "chat") + if entry == nil { + log.Logger.Warn("model price not found for cluster %s model %s", clusterName, targetModel) + return 0 + } + + // 使用配置加载阶段已转换好的定点整数价格（1 单位 = 1e-8 元） + // 转换由 go-lib/quota.RmbToFixedPoint 统一完成 + inputCost := int64(entry.Prices["input_cost_per_token_int"]) + outputCost := int64(entry.Prices["output_cost_per_token_int"]) + if inputCost < 0 || outputCost < 0 { + log.Logger.Warn("invalid model price for cluster %s model %s", clusterName, targetModel) + return 0 + } + + return promptTokens*inputCost + completionTokens*outputCost +} +``` + +说明： + +- `req.Route.ClusterName` 在 `reverseproxy.go` 的 `aiClusterInvoke()` 中已被设置为最终实际使用的 cluster（包括 fallback 场景）。 +- `aiMeta.TargetModel` 在 `reverseproxy.go` 的 `doSingleAIForward()` 中已被设置为路由目标模型 + cluster `ModelMapping` 映射后的最终模型名。 +- 因此这里拿到的 `clusterName` 和 `targetModel` 就是计费所需的实际值。 +- 价格到定点整数的转换在配置加载阶段通过 `go-lib/quota` 完成，运行时 `calcCostUnits` 只处理整数，保证 Redis Lua 不接触浮点。 + +### 7.6 定价匹配逻辑 + +```go +func lookupModelPrice(table *cluster_conf.ModelTable, model, mode string) *cluster_conf.ModelPrice { + if table == nil { + return nil + } + idx, ok := table.priceIndex[model] + if !ok { + return nil + } + return idx[mode] +} +``` + +索引在配置加载阶段构建，运行时按 `(model, mode)` 精确查询，为 O(1)。未命中时返回 `nil`，由调用方决定是否按 `0` 成本处理。 + +## 8. Redis Lua 脚本改造 + +当前 Token 配额 Lua： + +```lua +local current = tonumber(redis.call('GET', KEYS[1]) or ARGV[2]) +local amount = tonumber(ARGV[1]) +local deduct = math.min(current, amount) +if deduct > 0 then + redis.call('DECRBY', KEYS[1], deduct) +end +return math.max(0, current - deduct) +``` + +RMB 配额有两种可选实现。 + +### 8.1 方案一：单 Key 定点数（余额上限 ≤ 9000 万元） + +以 **1e-8 元** 为一个单位，`Quota` 和余额都存为整数。 + +```lua +local raw = redis.call('GET', KEYS[1]) +local current +if raw == false then + current = tonumber(ARGV[2]) + redis.call('SET', KEYS[1], current) +else + current = tonumber(raw) +end +local amount = tonumber(ARGV[1]) +local deduct = math.min(current, amount) +if deduct > 0 then + redis.call('DECRBY', KEYS[1], deduct) +end +return math.max(0, current - deduct) +``` + +- `ARGV[1]`：本次扣减金额（固定点整数）。 +- `ARGV[2]`：初始配额（固定点整数），仅在 Key 不存在时使用。 + +> ⚠️ Lua number 为 IEEE 754 double，整数精确表示上限约为 `2^53`（9e15）。按 1e-8 元换算，理论余额上限约为 9007.20 万元；业务上统一限定 RMB 配额余额上限为 **9000 万元（90,000,000.00 元）**。若业务需要更大余额，请使用方案二。 + +### 8.2 方案二：Hash 拆分整数部分 + 小数部分（支持约 100 亿元上限） + +用 Redis Hash 存两个字段：`yuan`（整数元）和 `fraction`（0 ~ 99,999,999）。 + +```lua +local yuan = tonumber(redis.call('HGET', KEYS[1], 'yuan')) +local frac = tonumber(redis.call('HGET', KEYS[1], 'fraction')) +if yuan == nil then + yuan = tonumber(ARGV[1]) + frac = tonumber(ARGV[2]) + redis.call('HMSET', KEYS[1], 'yuan', yuan, 'fraction', frac) +end + +local cost_yuan = tonumber(ARGV[3]) +local cost_frac = tonumber(ARGV[4]) + +if frac < cost_frac then + yuan = yuan - 1 + frac = frac + 100000000 +end +yuan = yuan - cost_yuan +frac = frac - cost_frac + +if yuan < 0 then + yuan = 0 + frac = 0 +end + +redis.call('HMSET', KEYS[1], 'yuan', yuan, 'fraction', frac) +return {yuan, frac} +``` + +- `ARGV[1]` / `ARGV[2]`：初始配额的 `yuan` / `fraction`。 +- `ARGV[3]` / `ARGV[4]`：本次扣减成本的 `yuan` / `fraction`。 +- 所有数字都在 64 位整数范围内，无精度损失。 + +对应的 `HasBalance` 改为读取 Hash 并计算余额是否大于 0。 + +### 8.3 当前选型 + +对比方案一和方案二后，当前版本 **暂时使用方案一（单 Key 定点数）**。原因如下： + +- v0.4 阶段 RMB 配额余额上限在 9 千万元以内即可满足业务需求； +- 方案一实现简单，Lua 脚本与现有 Token 扣减逻辑更接近，测试和运维成本更低。 + +若后续业务需要支持更大余额上限，再评估迁移到方案二。 + +## 9. 配置文件示例 + +`cluster_conf.data` 中的 `AIConf` 示例： + +```json +{ + "AIConf": { + "Type": 0, + "Provider": "deepseek", + "Keys": [ + { + "Name": "key-primary", + "Key": "sk-xxxxxxxxxxxx", + "Weight": 70 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 3, + "RetryBackoffInitial": 500, + "RetryBackoffMax": 5000 + }, + "ModelMapping": { + "gpt-4": "deepseek-chat" + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "deepseek", + "Model": "deepseek-chat", + "BaseModel": "deepseek-chat", + "Mode": "chat", + "Capabilities": ["chat"], + "SupportedParameters": ["temperature", "max_tokens"], + "Limits": { + "context_window": 128000 + }, + "Prices": { + "input_cost_per_token": 0.000001, + "output_cost_per_token": 0.000002 + } + } + ] + } + } +} +``` + +> 上例中 `input_cost_per_token = 0.000001` 元/Token，换算为固定点整数即 `100`（= 0.000001 * 1e8），表示 **0.1 元 / 百万 Token**。 + +## 10. 测试建议 + +1. **单元测试** + - `QuotaPlan.Deduct`：分别覆盖 `total_token` 和 `RMB` 两种单位，以及余额不足、Key 不存在等边界。 + - `lookupModelPrice`：精确匹配、未命中返回 nil。 + - `calcCostUnits`：正常计算、ModelTable 缺失、模型未命中、价格转换精度。 + +2. **Lua 脚本测试** + - 单 Key 定点数方案：验证扣减、余额归零、负数不溢出。 + - Hash 拆分方案：验证借位、余额归零、大数（接近 100 亿元）正确性。 + +3. **集成测试** + - 创建一个 `unit = "RMB"` 的 API Key，发一次 chat 请求，验证 Redis 余额按预期扣减。 + - 测试 `ModelMapping` 场景：请求模型是 `gpt-4`，实际后端模型是 `deepseek-chat`，验证按 `deepseek-chat` 的价格计费。 + - 测试 fallback 场景：请求最终 fallback 到另一个 cluster，验证按最终 cluster + target_model 计费。 + - 测试流式（SSE）场景：请求体带 `stream: true`，后端返回 SSE 并在最后一个 chunk 中携带 `usage`，验证 RMB 配额仍能正确扣减。 + +## 11. 兼容性与注意事项 + +1. **存量 Token 配额完全兼容**：`Unit` 默认 `"total_token"`，走原有 Lua 扣减逻辑。 +2. **浮点禁止进入 Redis**：所有金额在 BFE 内部和 Redis 中均以固定点整数表示，避免浮点误差；价格浮点转换仅在 BFE 配置加载阶段完成，conf-agent 不做任何转换。 +3. **无 ModelTable 时的兜底行为**：若 RMB 配额计划命中的 cluster 没有配置 `ModelTable`，或没有匹配到模型条目，当前建议： + - 记录告警日志； + - 本次请求不对该 RMB 配额进行扣减（相当于按 `0` 成本处理）； + - 具体是否拒绝请求，需产品进一步确认。 +4. **与多 Key 改造的关系**：`AIConf.Keys` 与 `ModelTable` 相互独立，可并行下发、独立解析。 +5. **流式响应计费**：RMB 成本在请求结束阶段计算，依赖 `mod_body_process`（或其他响应处理模块）在流式传输过程中填充 `PromptTokens` / `CompletionTokens`。生产环境若启用流式计费，需确保 `mod_body_process` 已加载。 +6. **模块顺序建议**：`mod_ai_token_auth` 的 `HandleReadResponse` 不再负责 RMB 成本计算，因此对模块加载顺序的敏感度降低；但仍建议保持 `mod_ai_token_auth` 在 `mod_body_process` 之前注册，以便非流式场景下 token 用量解析逻辑保持一致。 +7. **旧字段清理**：`AIConf.Key` 已移除，统一使用 `AIConf.Keys`。 diff --git a/go.mod b/go.mod index ebc65dfe6..567791bf8 100644 --- a/go.mod +++ b/go.mod @@ -9,7 +9,7 @@ require ( github.com/andybalholm/brotli v1.0.2 github.com/armon/go-radix v1.0.0 github.com/asergeyev/nradix v0.0.0-20170505151046-3872ab85bb56 // indirect - github.com/bfenetworks/go-lib v0.0.1 + github.com/bfenetworks/go-lib v0.0.2 github.com/golang-jwt/jwt v3.2.2+incompatible github.com/gomodule/redigo v2.0.0+incompatible github.com/json-iterator/go v1.1.12 @@ -42,6 +42,7 @@ require ( ) require ( + github.com/alicebob/miniredis/v2 v2.34.0 github.com/bfenetworks/proxy-wasm-go-host v0.0.1 github.com/envoyproxy/go-control-plane/envoy v1.32.3 github.com/go-jose/go-jose/v4 v4.0.5 @@ -52,6 +53,7 @@ require ( ) require ( + github.com/alicebob/gopher-json v0.0.0-20230218143504-906a9b012302 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/cncf/xds/go v0.0.0-20240723142845-024c85f92f20 // indirect @@ -61,13 +63,14 @@ require ( github.com/prometheus/client_model v0.6.1 // indirect github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/pretty v1.2.0 // indirect + github.com/yuin/gopher-lua v1.1.1 // indirect google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect ) require ( github.com/HdrHistogram/hdrhistogram-go v1.0.1 // indirect github.com/aymerick/douceur v0.2.0 // indirect - github.com/bfenetworks/bfe-access-pb v0.1.0 + github.com/bfenetworks/bfe-access-pb v0.2.0 github.com/bfenetworks/bfe-mock-waf v0.1.0 github.com/bfenetworks/bwi v0.1.2 github.com/davecgh/go-spew v1.1.1 // indirect diff --git a/go.sum b/go.sum index 9628d5f66..bcac9a999 100644 --- a/go.sum +++ b/go.sum @@ -6,6 +6,10 @@ github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWX github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI= github.com/abbot/go-http-auth v0.4.1-0.20181019201920-860ed7f246ff h1:9ZqcMQ0fB+ywKACVjGfZM4C7Uq9D5rq0iSmwIjX187k= github.com/abbot/go-http-auth v0.4.1-0.20181019201920-860ed7f246ff/go.mod h1:Cz6ARTIzApMJDzh5bRMSUou6UMSp0IEXg9km/ci7TJM= +github.com/alicebob/gopher-json v0.0.0-20230218143504-906a9b012302 h1:uvdUDbHQHO85qeSydJtItA4T55Pw6BtAejd0APRJOCE= +github.com/alicebob/gopher-json v0.0.0-20230218143504-906a9b012302/go.mod h1:SGnFV6hVsYE877CKEZ6tDNTjaSXYUk6QqoIK6PrAtcc= +github.com/alicebob/miniredis/v2 v2.34.0 h1:mBFWMaJSNL9RwdGRyEDoAAv8OQc5UlEhLDQggTglU/0= +github.com/alicebob/miniredis/v2 v2.34.0/go.mod h1:kWShP4b58T1CW0Y5dViCd5ztzrDqRWqM3nksiyXk5s8= github.com/andybalholm/brotli v1.0.2 h1:JKnhI/XQ75uFBTiuzXpzFrUriDPiZjlOSzh6wXogP0E= github.com/andybalholm/brotli v1.0.2/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y= github.com/armon/go-radix v1.0.0 h1:F4z6KzEeeQIMeLFa97iZU6vupzoecKdU5TX24SNppXI= @@ -16,14 +20,14 @@ github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuP github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= -github.com/bfenetworks/bfe-access-pb v0.1.0 h1:wVFgWNIiU06v7UkycZ+1IIlUBtn4zAhrjLqLbgNzrDU= -github.com/bfenetworks/bfe-access-pb v0.1.0/go.mod h1:XZVAEiVf88PQqSHncZFFvUEK9ZdN1aaiuugRHTIBgNo= +github.com/bfenetworks/bfe-access-pb v0.2.0 h1:2JBUPJt2ZckqDUoL5F5cBhtqoY94MllIX3m4hJ3jVK4= +github.com/bfenetworks/bfe-access-pb v0.2.0/go.mod h1:3qunybhMV5hEeT/2qDwbchxo4t/9LPKEvwwZkmekRho= github.com/bfenetworks/bfe-mock-waf v0.1.0 h1:dTd540S3nv6qNlG6lLC0F8qx3gyo5WB4wnci3+d4J78= github.com/bfenetworks/bfe-mock-waf v0.1.0/go.mod h1:MWZHbihiRQXpoUCvY1l18s2bfOBWx4N4pghxBt+xUv0= github.com/bfenetworks/bwi v0.1.2 h1:3AcCzUjyzKm+FeLgTIVg58u+SUdEVZdJbQM58Ezwjcg= github.com/bfenetworks/bwi v0.1.2/go.mod h1:zCRIdSw521zVnNCM73qw/lZ9UknbRux9rk6UQvBJgMA= -github.com/bfenetworks/go-lib v0.0.1 h1:LUTj+uInCnhUGjr/pXs95PEQzNfG/VeUEoruIjcxUx4= -github.com/bfenetworks/go-lib v0.0.1/go.mod h1:DNiKiff30CaX7uOUGJpP85VyTn+JSFoY9gfG2AxmPgM= +github.com/bfenetworks/go-lib v0.0.2 h1:ZjWas7Icd3svzVdegSSM1kMYSrnZHRvoCx6SQ458pqg= +github.com/bfenetworks/go-lib v0.0.2/go.mod h1:DNiKiff30CaX7uOUGJpP85VyTn+JSFoY9gfG2AxmPgM= github.com/bfenetworks/proxy-wasm-go-host v0.0.1 h1:FzgCnKC+fkeY4NtsUVBcYONOdhvsJA1FXei01VP17Qo= github.com/bfenetworks/proxy-wasm-go-host v0.0.1/go.mod h1:ooQK7XyIovzGQIADKbPdpfUJ3w+b2ou6iqDZLMq0pww= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= @@ -179,6 +183,8 @@ github.com/uber/jaeger-client-go v2.25.0+incompatible/go.mod h1:WVhlPFC8FDjOFMMW github.com/uber/jaeger-lib v2.4.0+incompatible h1:fY7QsGQWiCt8pajv4r7JEvmATdCVaWxXbjwyYwsNaLQ= github.com/uber/jaeger-lib v2.4.0+incompatible/go.mod h1:ComeNDZlWwrWnDv8aPp0Ba6+uUTzImX/AauajbLI56U= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M= +github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw= github.com/zmap/go-iptree v0.0.0-20170831022036-1948b1097e25 h1:LRoXAcKX48QV4LV23W5ZtsG/MbJOgNUNvWiXwM0iLWw= github.com/zmap/go-iptree v0.0.0-20170831022036-1948b1097e25/go.mod h1:qOasALtPByO1Jk6LhgpNv6htPMK2QJfiGorUk57nO/U= go.elastic.co/apm v1.7.2/go.mod h1:tCw6CkOJgkWnzEthFN9HUP1uL3Gjc/Ur6m7gRPLaoH0= diff --git a/tests/integration/README.md b/tests/integration/README.md new file mode 100644 index 000000000..deef1644d --- /dev/null +++ b/tests/integration/README.md @@ -0,0 +1,56 @@ +# BFE 集成测试 + +本目录承载 `bfe` 的**真实进程级集成测试**。与仓库中 `integration-test/` 目录不同的是： + +- 仅启动真实的 `bfe` 进程，不引入 `ai-gateway-api`、`conf-agent` 等外部组件； +- 测试所需的 BFE 配置文件（如 `ai_route.data`、`cluster_table.data` 等）直接由测试代码或静态 `testdata` 提供； +- 请求通过真实 HTTP 发送到 BFE 监听端口，验证转发行为与后端命中统计。 + +## 目录结构 + +```text +bfe/tests/integration/ +├── README.md # 本文档 +├── common/ # 公共 harness +│ ├── process_env.go # 编译/启动/停止真实 BFE 进程 +│ ├── bfe_config_builder.go # 生成临时 BFE 配置 +│ ├── mock_backend.go # 本地 mock AI 后端 +│ └── util.go # 工具函数 +├── implementation/ # Go 实现代码（ASCII 目录名） +│ └── scenario-SC01-route-table-lookup/ +│ ├── sc01_route_table_lookup_test.go +│ └── testdata/ # 静态 BFE 配置模板 +└── 测试设计文档/ # 中文测试设计文档 + ├── 测试场景总体说明.md + └── scenario-SC01-路由表查找与绑定/ + ├── 场景说明.md + └── TC-*.md +``` + +## 运行方式 + +在 `bfe/` 目录下执行： + +```bash +# 运行全部集成测试 +go test ./tests/integration/... -v + +# 运行单个场景 +go test ./tests/integration/implementation/scenario-SC01-route-table-lookup/... -v + +# 运行单个测试例 +go test ./tests/integration/implementation/scenario-SC01-route-table-lookup/ -run TestTC01 -v +``` + +首次运行会自动编译 `bfe` 二进制并缓存到 `bfe/tests/integration/.integration-test-bin/`。 + +## 当前覆盖 + +| 场景 | 说明 | +|------|------| +| SC01 路由表查找与绑定 | 验证 `mod_ai_route` 在多级路由表（apikey/entity/global）中的搜索与回退顺序，以及 fallback 时 body 回绕行为 | + +## 参考文档 + +- `document-ai-gateway/BFE设计/v0.3.0/BFE的mod_ai_route集成测试方案/BFE的mod_ai_route集成测试方案v1.0.0.md` +- `integration-test/方案说明/总体说明/集成测试方案说明.md` diff --git a/tests/integration/common/access_log_parser.go b/tests/integration/common/access_log_parser.go new file mode 100644 index 000000000..c72803af6 --- /dev/null +++ b/tests/integration/common/access_log_parser.go @@ -0,0 +1,137 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package common + +import ( + "fmt" + "os" + "path/filepath" + "strings" + "time" + + "google.golang.org/protobuf/proto" + + "github.com/bfenetworks/bfe-access-pb/b2log" + bfe_access_pb "github.com/bfenetworks/bfe-access-pb/bfe_access_pb" +) + +// ParseAccessLog reads the b2log file written by mod_access_pb3 and returns +// all decoded RequestLog records. It polls briefly if the file is not yet +// populated. +func ParseAccessLog(logDir string, timeout time.Duration) ([]*bfe_access_pb.RequestLog, error) { + logPath := filepath.Join(logDir, "pb_access3.log") + deadline := time.Now().Add(timeout) + + var data []byte + var err error + for time.Now().Before(deadline) { + data, err = os.ReadFile(logPath) + if err == nil && len(data) > 0 { + break + } + time.Sleep(100 * time.Millisecond) + } + if err != nil { + return nil, fmt.Errorf("read access log %s failed: %w", logPath, err) + } + + records, _ := b2log.BuffParse(data) + var reqLogs []*bfe_access_pb.RequestLog + for _, rec := range records { + bfeLog := new(bfe_access_pb.BfeLog) + if err := proto.Unmarshal(rec, bfeLog); err != nil { + return nil, fmt.Errorf("unmarshal BfeLog failed: %w", err) + } + if bfeLog.RequestLog != nil { + reqLogs = append(reqLogs, bfeLog.RequestLog) + } + } + return reqLogs, nil +} + +// ParseAccessLogAfterStop reads the b2log file after BFE has been stopped, +// ensuring all buffered logs are flushed. +func ParseAccessLogAfterStop(logDir string) ([]*bfe_access_pb.RequestLog, error) { + return ParseAccessLog(logDir, 2*time.Second) +} + +// FormatAccessLogError returns a string with all request log fields for +// debugging test failures. +func FormatAccessLogError(reqLog *bfe_access_pb.RequestLog) string { + var b strings.Builder + b.WriteString("RequestLog{ ") + if reqLog.AiApikeyId != nil { + b.WriteString(fmt.Sprintf("ai_apikey_id=%s ", *reqLog.AiApikeyId)) + } + if len(reqLog.AiApikeytags) > 0 { + b.WriteString(fmt.Sprintf("ai_apikeytags=%v ", reqLog.AiApikeytags)) + } + if reqLog.AiRequestedModel != nil { + b.WriteString(fmt.Sprintf("ai_requested_model=%s ", *reqLog.AiRequestedModel)) + } + if reqLog.AiTargetModel != nil { + b.WriteString(fmt.Sprintf("ai_target_model=%s ", *reqLog.AiTargetModel)) + } + if reqLog.AiStream != nil { + b.WriteString(fmt.Sprintf("ai_stream=%v ", *reqLog.AiStream)) + } + if reqLog.AiInputTokens != nil { + b.WriteString(fmt.Sprintf("ai_input_tokens=%d ", *reqLog.AiInputTokens)) + } + if reqLog.AiOutputTokens != nil { + b.WriteString(fmt.Sprintf("ai_output_tokens=%d ", *reqLog.AiOutputTokens)) + } + if reqLog.AiTotalTokens != nil { + b.WriteString(fmt.Sprintf("ai_total_tokens=%d ", *reqLog.AiTotalTokens)) + } + if reqLog.AiTtftUs != nil { + b.WriteString(fmt.Sprintf("ai_ttft_us=%d ", *reqLog.AiTtftUs)) + } + if reqLog.AiTpotUs != nil { + b.WriteString(fmt.Sprintf("ai_tpot_us=%d ", *reqLog.AiTpotUs)) + } + if len(reqLog.AiRateLimitHits) > 0 { + b.WriteString(fmt.Sprintf("ai_rate_limit_hits=%v ", reqLog.AiRateLimitHits)) + } + if reqLog.AiAuthRejectReason != nil { + b.WriteString(fmt.Sprintf("ai_auth_reject_reason=%s ", *reqLog.AiAuthRejectReason)) + } + if len(reqLog.AiAuthRejectQuotaPlans) > 0 { + b.WriteString(fmt.Sprintf("ai_auth_reject_quota_plans=%v ", reqLog.AiAuthRejectQuotaPlans)) + } + if reqLog.AiProvider != nil { + b.WriteString(fmt.Sprintf("ai_provider=%s ", *reqLog.AiProvider)) + } + if reqLog.AiRetryCount != nil { + b.WriteString(fmt.Sprintf("ai_retry_count=%d ", *reqLog.AiRetryCount)) + } + if reqLog.AiCostValue != nil { + b.WriteString(fmt.Sprintf("ai_cost_value=%d ", *reqLog.AiCostValue)) + } + if reqLog.AiCostCurrency != nil { + b.WriteString(fmt.Sprintf("ai_cost_currency=%s ", *reqLog.AiCostCurrency)) + } + if len(reqLog.AiRouteRuleHits) > 0 { + b.WriteString(fmt.Sprintf("ai_route_rule_hits=%v ", reqLog.AiRouteRuleHits)) + } + if len(reqLog.AiClusterKeyNames) > 0 { + b.WriteString(fmt.Sprintf("ai_cluster_key_names=%v ", reqLog.AiClusterKeyNames)) + } + if len(reqLog.AiAuthHitQuotaPlans) > 0 { + b.WriteString(fmt.Sprintf("ai_auth_hit_quota_plans=%v ", reqLog.AiAuthHitQuotaPlans)) + } + b.WriteString("}") + return b.String() +} diff --git a/tests/integration/common/bfe_config_builder.go b/tests/integration/common/bfe_config_builder.go new file mode 100644 index 000000000..4212c8a40 --- /dev/null +++ b/tests/integration/common/bfe_config_builder.go @@ -0,0 +1,491 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package common + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "strconv" + "strings" + + "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" +) + +// TokenRuleData holds the content of mod_ai_token_auth/token_rule.data. +type TokenRuleData struct { + Version string + QuotaPlans map[string][]QuotaPlan + Tokens map[string]map[string]TokenFile + Config map[string][]TokenRule +} + +// QuotaPlan is the JSON representation of a quota plan. +type QuotaPlan struct { + Id string + Unlimited bool + PassNoQuota bool + RedisKey string + CreateTime int64 + ExpiredTime int64 + Quota int64 + ResetMode int + Unit string +} + +// TokenFile is the JSON representation of a token file. +type TokenFile struct { + Key string `json:"key"` + KeyId string `json:"key_id"` + Enabled int `json:"enabled"` + Status int `json:"status"` + UpdateTime int64 `json:"update_time"` + ExpiredTime int64 `json:"expired_time"` + UnlimitedQuota bool `json:"unlimited_quota"` + Models *string `json:"allow_models"` + BlockModels *string `json:"block_models"` + Subnet *string `json:"subnet"` + Tags []bfe_basic.ApikeyTag `json:"tags"` + QuotaPlans []string `json:"quota_plans"` +} + +// TokenRule is the JSON representation of a token rule. +type TokenRule struct { + Cond string + Action ActionFile +} + +// ActionFile is the JSON representation of an action. +type ActionFile struct { + Cmd string +} + +// BFEConfigBuilder builds a temporary BFE configuration directory from a template. +type BFEConfigBuilder struct { + // TemplateDir contains static BFE data files (bfe.conf, cluster_conf, mod_ai_route, etc.). + TemplateDir string + // TargetConfDir is the directory where the final BFE config will be written. + TargetConfDir string + // Backends maps cluster names to mock backends. + Backends map[string]*MockBackend + // AIConfs optionally injects AIConf into cluster_conf.data for specific clusters. + AIConfs map[string]*cluster_conf.AIConf + // TotalBodyBufferSize overrides the totalBodyBufferSize value in bfe.conf. + // A value of 0 keeps the template value. + TotalBodyBufferSize int64 + // RedisAddr is the address of the redis server used by mod_ai_token_auth. + // If empty, mod_ai_token_auth.conf is not rewritten. + RedisAddr string + // TokenRuleData optionally generates mod_ai_token_auth/token_rule.data. + TokenRuleData *TokenRuleData +} + +// Build prepares the BFE configuration directory. +func (b *BFEConfigBuilder) Build() error { + if err := os.MkdirAll(b.TargetConfDir, 0755); err != nil { + return fmt.Errorf("create target conf dir failed: %w", err) + } + if err := copyDirContents(b.TemplateDir, b.TargetConfDir); err != nil { + return fmt.Errorf("copy template dir failed: %w", err) + } + + // BFE defaults ClientCRLBaseDir to "tls_conf/client_crl" and fails startup + // if the directory does not exist. Ensure it is present even if the template + // does not include it. + if err := os.MkdirAll(filepath.Join(b.TargetConfDir, "tls_conf", "client_crl"), 0755); err != nil { + return fmt.Errorf("create tls_conf/client_crl dir failed: %w", err) + } + + if err := b.normalizeAIRouteData(); err != nil { + return fmt.Errorf("normalize ai_route.data failed: %w", err) + } + + if err := b.generateClusterTable(); err != nil { + return fmt.Errorf("generate cluster_table.data failed: %w", err) + } + + if _, err := os.Stat(filepath.Join(b.TargetConfDir, "cluster_conf", "cluster_conf.data")); os.IsNotExist(err) { + if err := b.generateClusterConfData(); err != nil { + return fmt.Errorf("generate cluster_conf.data failed: %w", err) + } + } + + if err := RewriteBFEPorts(filepath.Join(b.TargetConfDir, "bfe.conf"), 0, 0, 0); err != nil { + return fmt.Errorf("rewrite bfe ports failed: %w", err) + } + + if b.TotalBodyBufferSize > 0 { + if err := RewriteBFETotalBodyBufferSize(filepath.Join(b.TargetConfDir, "bfe.conf"), b.TotalBodyBufferSize); err != nil { + return fmt.Errorf("rewrite totalBodyBufferSize failed: %w", err) + } + } + + if b.RedisAddr != "" { + if err := b.setupRedisBns(); err != nil { + return fmt.Errorf("setup redis bns failed: %w", err) + } + } + + if b.TokenRuleData != nil { + if err := b.writeTokenRuleData(); err != nil { + return fmt.Errorf("write token_rule.data failed: %w", err) + } + } + + return nil +} + +const redisBnsName = "redis_bns" + +func (b *BFEConfigBuilder) setupRedisBns() error { + host, port, err := splitHostPort(b.RedisAddr) + if err != nil { + return fmt.Errorf("parse redis addr %s failed: %w", b.RedisAddr, err) + } + + // rewrite mod_ai_token_auth.conf to use the fixed bns name + if err := b.rewriteModAITokenAuthBns(); err != nil { + return fmt.Errorf("rewrite mod_ai_token_auth bns failed: %w", err) + } + + // generate name_conf.data mapping bns name to redis addr + nameConf := map[string]interface{}{ + "Version": "1.0", + "Config": map[string][]map[string]interface{}{ + redisBnsName: { + {"Host": host, "Port": port, "Weight": 100}, + }, + }, + } + path := filepath.Join(b.TargetConfDir, "server_data_conf", "name_conf.data") + if err := writeJSONFile(path, nameConf); err != nil { + return fmt.Errorf("write name_conf.data failed: %w", err) + } + + // rewrite bfe.conf to load name_conf + return b.rewriteBFEConfNameConf() +} + +func (b *BFEConfigBuilder) rewriteModAITokenAuthBns() error { + path := filepath.Join(b.TargetConfDir, "mod_ai_token_auth", "mod_ai_token_auth.conf") + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + + lines := strings.Split(string(data), "\n") + for i, line := range lines { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "Bns") { + lines[i] = "Bns = \"" + redisBnsName + "\"" + } + } + return os.WriteFile(path, []byte(strings.Join(lines, "\n")), 0644) +} + +func (b *BFEConfigBuilder) rewriteBFEConfNameConf() error { + path := filepath.Join(b.TargetConfDir, "bfe.conf") + data, err := os.ReadFile(path) + if err != nil { + return err + } + + lines := strings.Split(string(data), "\n") + found := false + for i, line := range lines { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "NameConf") { + lines[i] = "NameConf = server_data_conf/name_conf.data" + found = true + } + } + if !found { + // insert after vipRuleConf if not found + for i, line := range lines { + if strings.HasPrefix(strings.TrimSpace(line), "vipRuleConf") { + lines = append(lines[:i+1], append([]string{"NameConf = server_data_conf/name_conf.data"}, lines[i+1:]...)...) + break + } + } + } + return os.WriteFile(path, []byte(strings.Join(lines, "\n")), 0644) +} + +func splitHostPort(addr string) (string, int, error) { + parts := strings.Split(addr, ":") + if len(parts) != 2 { + return "", 0, fmt.Errorf("invalid addr %s", addr) + } + port, err := strconv.Atoi(parts[1]) + if err != nil { + return "", 0, fmt.Errorf("invalid port %s", parts[1]) + } + return parts[0], port, nil +} + +func (b *BFEConfigBuilder) writeTokenRuleData() error { + path := filepath.Join(b.TargetConfDir, "mod_ai_token_auth", "token_rule.data") + return writeJSONFile(path, b.TokenRuleData) +} + +func (b *BFEConfigBuilder) normalizeAIRouteData() error { + path := filepath.Join(b.TargetConfDir, "mod_ai_route", "ai_route.data") + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + + var aiRoute map[string]interface{} + if err := json.Unmarshal(data, &aiRoute); err != nil { + return err + } + normalizeAIRouteData(aiRoute) + + out, err := json.MarshalIndent(aiRoute, "", " ") + if err != nil { + return err + } + return os.WriteFile(path, out, 0644) +} + +func (b *BFEConfigBuilder) generateClusterTable() error { + clusterTable := map[string]interface{}{ + "Version": "20260720150000", + "Config": map[string]interface{}{}, + } + config := clusterTable["Config"].(map[string]interface{}) + + for name, backend := range b.Backends { + host, port := backend.HostPort() + config[name] = map[string]interface{}{ + "sub_" + clusterSubName(name): []map[string]interface{}{ + { + "Name": name + "-backend-0", + "Addr": host, + "Port": port, + "Weight": 100, + }, + }, + } + } + + data, err := json.MarshalIndent(clusterTable, "", " ") + if err != nil { + return err + } + + path := filepath.Join(b.TargetConfDir, "cluster_conf", "cluster_table.data") + return os.WriteFile(path, data, 0644) +} + +func (b *BFEConfigBuilder) generateClusterConfData() error { + clusterConf := map[string]interface{}{ + "Version": "20260720150000", + "Config": map[string]interface{}{}, + } + config := clusterConf["Config"].(map[string]interface{}) + + for name := range b.Backends { + conf := clusterBasicConf() + if aiConf, ok := b.AIConfs[name]; ok && aiConf != nil { + aiConfMap, err := aiConfToMap(aiConf) + if err != nil { + return fmt.Errorf("marshal AIConf for cluster %s failed: %w", name, err) + } + conf["AIConf"] = aiConfMap + } + config[name] = conf + } + + path := filepath.Join(b.TargetConfDir, "cluster_conf", "cluster_conf.data") + return writeJSONFile(path, clusterConf) +} + +func aiConfToMap(aiConf *cluster_conf.AIConf) (map[string]interface{}, error) { + bytes, err := json.Marshal(aiConf) + if err != nil { + return nil, err + } + var m map[string]interface{} + if err := json.Unmarshal(bytes, &m); err != nil { + return nil, err + } + return m, nil +} + +func clusterBasicConf() map[string]interface{} { + return map[string]interface{}{ + "BackendConf": map[string]interface{}{ + "TimeoutConnSrv": 2000, + "TimeoutResponseHeader": 50000, + "MaxIdleConnsPerHost": 0, + "RetryLevel": 0, + }, + "CheckConf": map[string]interface{}{ + "Schem": "http", + "Uri": "/healthcheck", + "Host": "example.org", + "StatusCode": 200, + "FailNum": 10, + "CheckInterval": 1000, + }, + "GslbBasic": map[string]interface{}{ + "CrossRetry": 0, + "RetryMax": 2, + "HashConf": map[string]interface{}{ + "HashStrategy": 0, + "HashHeader": "Cookie:UID", + "SessionSticky": false, + }, + }, + "ClusterBasic": map[string]interface{}{ + "TimeoutReadClient": 30000, + "TimeoutWriteClient": 60000, + "TimeoutReadClientAgain": 30000, + "ReqWriteBufferSize": 512, + "ReqFlushInterval": 0, + "ResFlushInterval": -1, + "CancelOnClientClose": false, + "DisableHealthCheck": true, + }, + } +} + +// RewriteBFEPorts rewrites the httpPort, httpsPort and monitorPort lines in bfe.conf +// and ensures HTTP/HTTPS/monitor listeners are bound to the loopback interface only. +func RewriteBFEPorts(path string, httpPort, httpsPort, monitorPort int) error { + data, err := os.ReadFile(path) + if err != nil { + return err + } + lines := strings.Split(string(data), "\n") + + hasHTTPAddr := false + hasHTTPSAddr := false + hasMonitorAddr := false + serverSectionIdx := -1 + for i, line := range lines { + trimmed := strings.TrimSpace(line) + if trimmed == "[server]" { + serverSectionIdx = i + } + if strings.HasPrefix(trimmed, "httpAddr") { + hasHTTPAddr = true + } + if strings.HasPrefix(trimmed, "httpsAddr") { + hasHTTPSAddr = true + } + if strings.HasPrefix(trimmed, "monitorAddr") { + hasMonitorAddr = true + } + } + if serverSectionIdx >= 0 { + insertAfter := serverSectionIdx + if !hasHTTPAddr { + lines = append(lines[:insertAfter+1], append([]string{`httpAddr = "127.0.0.1"`}, lines[insertAfter+1:]...)...) + insertAfter++ + } + if !hasHTTPSAddr { + lines = append(lines[:insertAfter+1], append([]string{`httpsAddr = "127.0.0.1"`}, lines[insertAfter+1:]...)...) + insertAfter++ + } + if !hasMonitorAddr { + lines = append(lines[:insertAfter+1], append([]string{`monitorAddr = "127.0.0.1"`}, lines[insertAfter+1:]...)...) + } + } + + for i, line := range lines { + trimmed := strings.TrimSpace(line) + if httpPort > 0 && strings.HasPrefix(trimmed, "httpPort") { + lines[i] = "httpPort = " + strconv.Itoa(httpPort) + } + if httpsPort > 0 && strings.HasPrefix(trimmed, "httpsPort") { + lines[i] = "httpsPort = " + strconv.Itoa(httpsPort) + } + if monitorPort >= 0 && strings.HasPrefix(trimmed, "monitorPort") { + lines[i] = "monitorPort = " + strconv.Itoa(monitorPort) + } + } + return os.WriteFile(path, []byte(strings.Join(lines, "\n")), 0644) +} + +// RewriteBFETotalBodyBufferSize rewrites the totalBodyBufferSize line in bfe.conf. +func RewriteBFETotalBodyBufferSize(path string, size int64) error { + data, err := os.ReadFile(path) + if err != nil { + return err + } + lines := strings.Split(string(data), "\n") + found := false + for i, line := range lines { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "totalBodyBufferSize") { + lines[i] = "totalBodyBufferSize = " + strconv.FormatInt(size, 10) + found = true + } + } + if !found { + return fmt.Errorf("totalBodyBufferSize not found in %s", path) + } + return os.WriteFile(path, []byte(strings.Join(lines, "\n")), 0644) +} + +func normalizeAIRouteData(data map[string]interface{}) { + if data == nil { + return + } + rr, ok := data["route_rules"].(map[string]interface{}) + if !ok { + return + } + for _, table := range rr { + t, ok := table.(map[string]interface{}) + if !ok { + continue + } + rules, ok := t["rules"].([]interface{}) + if !ok { + continue + } + for _, rule := range rules { + r, ok := rule.(map[string]interface{}) + if !ok { + continue + } + if r["fallbacks"] == nil { + r["fallbacks"] = []interface{}{} + } + } + } +} + +func writeJSONFile(path string, data interface{}) error { + if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil { + return err + } + bytes, err := json.MarshalIndent(data, "", " ") + if err != nil { + return err + } + return os.WriteFile(path, bytes, 0644) +} diff --git a/tests/integration/common/mock_backend.go b/tests/integration/common/mock_backend.go new file mode 100644 index 000000000..5c958a7a8 --- /dev/null +++ b/tests/integration/common/mock_backend.go @@ -0,0 +1,187 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package common + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "net/http/httptest" + "net/url" + "sync" + "time" +) + +// MockBackend wraps an httptest.Server and records request metadata. +type MockBackend struct { + server *httptest.Server + ClusterName string + Response int + Body string + // ReadBeforeClose, if greater than 0, causes the handler to read this + // many bytes from the request body and then close the connection without + // sending an HTTP response. This is useful for simulating a backend that + // fails mid-stream. + ReadBeforeClose int + // DelayResponse sleeps for the given duration after reading the body and + // before writing the response. It can be used to keep the request (and + // any allocated body buffer) alive for a period of time. + DelayResponse time.Duration + // ReadNotify, if non-nil, is closed the first time the handler starts + // reading the request body. This can be used to synchronize with the + // allocation of body buffers inside BFE. + ReadNotify chan struct{} + // HoldResponse, if non-nil, blocks the handler after the request body has + // been fully read and before the response is written. The response is only + // sent after the channel is closed. This is useful for keeping a request + // alive while another request is being processed. + HoldResponse <-chan struct{} + // HoldBeforeRead, if non-nil, blocks the handler after the request headers + // have been received and before the body is read. This can be used to keep + // BFE from closing the request body while another request is processed. + HoldBeforeRead <-chan struct{} + // ResponseFunc, if non-nil, overrides Response/Body and is called for each + // request to determine the response status and body. + ResponseFunc func(r *http.Request, count int) (int, string) + // ResponseHeaders, if non-nil, is written to the response before the status code. + ResponseHeaders map[string]string + hits int + mu sync.Mutex + models []string + bodies [][]byte + authHeaders []string +} + +// NewMockBackend starts a local HTTP server that returns the given status code. +func NewMockBackend(clusterName string, response int, body string) *MockBackend { + b := &MockBackend{ + ClusterName: clusterName, + Response: response, + Body: body, + } + b.server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + b.mu.Lock() + b.hits++ + count := b.hits + if b.ReadNotify != nil { + close(b.ReadNotify) + b.ReadNotify = nil + } + b.mu.Unlock() + + if b.HoldBeforeRead != nil { + <-b.HoldBeforeRead + } + + if b.ReadBeforeClose > 0 { + _, _ = io.CopyN(io.Discard, r.Body, int64(b.ReadBeforeClose)) + if hj, ok := w.(http.Hijacker); ok { + conn, _, _ := hj.Hijack() + if conn != nil { + _ = conn.Close() + } + } + return + } + + if r.Body != nil { + bodyBytes, _ := io.ReadAll(r.Body) + b.mu.Lock() + b.bodies = append(b.bodies, append([]byte(nil), bodyBytes...)) + b.authHeaders = append(b.authHeaders, r.Header.Get("Authorization")) + var reqBody map[string]interface{} + if err := json.Unmarshal(bodyBytes, &reqBody); err == nil { + if model, ok := reqBody["model"].(string); ok { + b.models = append(b.models, model) + } + } + b.mu.Unlock() + } + if b.HoldResponse != nil { + <-b.HoldResponse + } + if b.DelayResponse > 0 { + time.Sleep(b.DelayResponse) + } + + status, body := b.Response, b.Body + if b.ResponseFunc != nil { + status, body = b.ResponseFunc(r, count) + } + for k, v := range b.ResponseHeaders { + w.Header().Set(k, v) + } + w.WriteHeader(status) + if body != "" { + w.Write([]byte(body)) + } + })) + return b +} + +// Hits returns the number of requests received. +func (b *MockBackend) Hits() int { + b.mu.Lock() + defer b.mu.Unlock() + return b.hits +} + +// Models returns the list of model values observed in request bodies. +func (b *MockBackend) Models() []string { + b.mu.Lock() + defer b.mu.Unlock() + return append([]string(nil), b.models...) +} + +// RequestBodies returns a deep copy of all observed request bodies. +func (b *MockBackend) RequestBodies() [][]byte { + b.mu.Lock() + defer b.mu.Unlock() + result := make([][]byte, len(b.bodies)) + for i, body := range b.bodies { + result[i] = append([]byte(nil), body...) + } + return result +} + +// AuthHeaders returns a deep copy of all observed Authorization headers. +func (b *MockBackend) AuthHeaders() []string { + b.mu.Lock() + defer b.mu.Unlock() + return append([]string(nil), b.authHeaders...) +} + +// Close shuts down the mock backend. +func (b *MockBackend) Close() { + b.server.Close() +} + +// Addr returns the host:port of the mock backend. +func (b *MockBackend) Addr() string { + u, _ := url.Parse(b.server.URL) + return u.Host +} + +// HostPort returns the host and port of the mock backend. +func (b *MockBackend) HostPort() (string, int) { + u, _ := url.Parse(b.server.URL) + host := u.Hostname() + port := 80 + if u.Port() != "" { + fmt.Sscanf(u.Port(), "%d", &port) + } + return host, port +} diff --git a/tests/integration/common/process_env.go b/tests/integration/common/process_env.go new file mode 100644 index 000000000..e249f71ff --- /dev/null +++ b/tests/integration/common/process_env.go @@ -0,0 +1,242 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package common + +import ( + "fmt" + "net" + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "sync" + "testing" + "time" +) + +// ProcessEnv manages building and running a real BFE process for integration tests. +type ProcessEnv struct { + t *testing.T + + // sourceRoot is the absolute path to the bfe module root. + sourceRoot string + // binDir is the directory where the compiled BFE binary is cached. + binDir string + // workDir is a per-test temporary directory. + workDir string + + bfeBinaryPath string + + buildOnce sync.Once +} + +// NewProcessEnv creates a ProcessEnv for the current test. +func NewProcessEnv(t *testing.T) *ProcessEnv { + root, err := locateBFESourceRoot() + if err != nil { + t.Fatalf("locate bfe source root failed: %v", err) + } + return &ProcessEnv{ + t: t, + sourceRoot: root, + binDir: filepath.Join(root, "tests", "integration", ".integration-test-bin"), + workDir: t.TempDir(), + } +} + +// WorkDir returns the per-test temporary directory. +func (p *ProcessEnv) WorkDir() string { return p.workDir } + +// SourceRoot returns the absolute path to the bfe source root. +func (p *ProcessEnv) SourceRoot() string { return p.sourceRoot } + +// locateBFESourceRoot walks up from this file to find the directory whose +// go.mod declares module "github.com/bfenetworks/bfe". +func locateBFESourceRoot() (string, error) { + _, filename, _, ok := runtime.Caller(0) + if !ok { + return "", fmt.Errorf("cannot get current file path") + } + dir := filepath.Dir(filename) + for { + goModPath := filepath.Join(dir, "go.mod") + if data, err := os.ReadFile(goModPath); err == nil { + if strings.Contains(string(data), "module github.com/bfenetworks/bfe") { + return dir, nil + } + } + parent := filepath.Dir(dir) + if parent == dir { + break + } + dir = parent + } + return "", fmt.Errorf("cannot locate bfe source root go.mod") +} + +// Build compiles the BFE binary if not already cached. +// The binary path includes the current git commit hash so that switching +// commits forces a rebuild. Within the same commit, source mtime is checked +// to catch uncommitted local edits. +func (p *ProcessEnv) Build() { + p.buildOnce.Do(func() { + if err := os.MkdirAll(p.binDir, 0755); err != nil { + p.t.Fatalf("create bin dir failed: %v", err) + } + + binName := "bfe" + if runtime.GOOS == "windows" { + binName += ".exe" + } + commit := gitCommitHash(p.sourceRoot) + binPath := filepath.Join(p.binDir, fmt.Sprintf("bfe-%s-%s-%s-%s", runtime.GOOS, runtime.GOARCH, commit, binName)) + + if !p.binaryNeedsRebuild(binPath) { + p.t.Logf("use cached bfe binary: %s", binPath) + p.bfeBinaryPath = binPath + return + } + + p.t.Logf("building bfe binary from %s ...", p.sourceRoot) + cmd := exec.Command("go", "build", "-o", binPath, ".") + cmd.Dir = p.sourceRoot + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + if err := cmd.Run(); err != nil { + p.t.Fatalf("build bfe failed: %v", err) + } + p.t.Logf("built bfe -> %s", binPath) + p.bfeBinaryPath = binPath + }) +} + +// gitCommitHash returns the current git commit hash, or "unknown" if git is +// not available or the directory is not a git repository. +func gitCommitHash(dir string) string { + cmd := exec.Command("git", "rev-parse", "HEAD") + cmd.Dir = dir + out, err := cmd.Output() + if err != nil { + return "unknown" + } + return strings.TrimSpace(string(out)) +} + +// binaryNeedsRebuild returns true if the binary does not exist or if any +// .go file under the source root is newer than the binary. +func (p *ProcessEnv) binaryNeedsRebuild(binPath string) bool { + binInfo, err := os.Stat(binPath) + if err != nil { + return true + } + + needsRebuild := false + err = filepath.Walk(p.sourceRoot, func(path string, info os.FileInfo, err error) error { + if err != nil { + return err + } + if info.IsDir() { + // Skip hidden directories and test binary cache to avoid + // unnecessary rebuilds caused by generated artifacts. + name := info.Name() + if name != "." && strings.HasPrefix(name, ".") { + return filepath.SkipDir + } + return nil + } + if strings.HasSuffix(path, ".go") && info.ModTime().After(binInfo.ModTime()) { + needsRebuild = true + return filepath.SkipAll + } + return nil + }) + if err != nil { + p.t.Logf("failed to check source mtime, rebuilding: %v", err) + return true + } + return needsRebuild +} + +// StartBFE starts a real BFE process with the given conf root and log dir. +// It returns the HTTP port, the monitor port and a teardown function. +func (p *ProcessEnv) StartBFE(confDir, logDir string) (int, int, func()) { + httpPort, err := FindFreePort() + if err != nil { + p.t.Fatalf("find free port for bfe http failed: %v", err) + } + httpsPort, err := FindFreePort() + if err != nil { + p.t.Fatalf("find free port for bfe https failed: %v", err) + } + monitorPort, err := FindFreePort() + if err != nil { + p.t.Fatalf("find free port for bfe monitor failed: %v", err) + } + + if err := RewriteBFEPorts(filepath.Join(confDir, "bfe.conf"), httpPort, httpsPort, monitorPort); err != nil { + p.t.Fatalf("rewrite bfe ports failed: %v", err) + } + + if err := os.MkdirAll(logDir, 0755); err != nil { + p.t.Fatalf("create bfe log dir failed: %v", err) + } + + cmd := exec.Command(p.bfeBinaryPath, "-c", confDir, "-l", logDir, "-s") + cmd.Dir = confDir + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + if err := cmd.Start(); err != nil { + p.t.Fatalf("start bfe failed: %v", err) + } + + addr := fmt.Sprintf("127.0.0.1:%d", httpPort) + if err := WaitForTCP(addr, 30*time.Second); err != nil { + _ = cmd.Process.Kill() + _, _ = cmd.Process.Wait() + p.t.Fatalf("bfe did not start in time: %v", err) + } + + stop := func() { + _ = cmd.Process.Kill() + _, _ = cmd.Process.Wait() + time.Sleep(50 * time.Millisecond) + } + return httpPort, monitorPort, stop +} + +// FindFreePort returns a free TCP port on localhost. +func FindFreePort() (int, error) { + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + return 0, err + } + defer ln.Close() + return ln.Addr().(*net.TCPAddr).Port, nil +} + +// WaitForTCP waits until the given TCP address is reachable. +func WaitForTCP(addr string, timeout time.Duration) error { + deadline := time.Now().Add(timeout) + for time.Now().Before(deadline) { + conn, err := net.Dial("tcp", addr) + if err == nil { + conn.Close() + return nil + } + time.Sleep(100 * time.Millisecond) + } + return fmt.Errorf("timeout waiting for %s", addr) +} diff --git a/tests/integration/common/redis_server.go b/tests/integration/common/redis_server.go new file mode 100644 index 000000000..50a41ceb4 --- /dev/null +++ b/tests/integration/common/redis_server.go @@ -0,0 +1,68 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package common + +import ( + "fmt" + "testing" + + "github.com/alicebob/miniredis/v2" +) + +// RedisServer wraps miniredis for integration tests. +type RedisServer struct { + server *miniredis.Miniredis + t *testing.T +} + +// NewRedisServer starts a new embedded redis server. +func NewRedisServer(t *testing.T) *RedisServer { + s := &RedisServer{t: t} + var err error + s.server, err = miniredis.Run() + if err != nil { + t.Fatalf("start miniredis failed: %v", err) + } + return s +} + +// Addr returns the redis server address in "host:port" format. +func (s *RedisServer) Addr() string { + return s.server.Addr() +} + +// Close stops the redis server. +func (s *RedisServer) Close() { + s.server.Close() +} + +// SetQuota sets an integer quota value for the given key. +func (s *RedisServer) SetQuota(key string, value int64) { + s.server.Set(key, fmt.Sprintf("%d", value)) +} + +// GetQuota returns the current integer quota value for the given key. +func (s *RedisServer) GetQuota(key string) int64 { + v, err := s.server.Get(key) + if err != nil { + s.t.Fatalf("get quota %s failed: %v", key, err) + } + var value int64 + _, err = fmt.Sscanf(v, "%d", &value) + if err != nil { + s.t.Fatalf("parse quota %s value %s failed: %v", key, v, err) + } + return value +} diff --git a/tests/integration/common/util.go b/tests/integration/common/util.go new file mode 100644 index 000000000..558c63689 --- /dev/null +++ b/tests/integration/common/util.go @@ -0,0 +1,131 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package common + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "path/filepath" +) + +// copyDirContents recursively copies files and directories from src to dst. +func copyDirContents(src, dst string) error { + entries, err := os.ReadDir(src) + if err != nil { + return err + } + for _, entry := range entries { + srcPath := filepath.Join(src, entry.Name()) + dstPath := filepath.Join(dst, entry.Name()) + if entry.IsDir() { + if err := os.MkdirAll(dstPath, 0755); err != nil { + return err + } + if err := copyDirContents(srcPath, dstPath); err != nil { + return err + } + continue + } + if err := copyFile(srcPath, dstPath); err != nil { + return err + } + } + return nil +} + +// copyFile copies a single file from src to dst. +func copyFile(src, dst string) error { + srcFile, err := os.Open(src) + if err != nil { + return err + } + defer srcFile.Close() + + if err := os.MkdirAll(filepath.Dir(dst), 0755); err != nil { + return err + } + dstFile, err := os.Create(dst) + if err != nil { + return err + } + defer dstFile.Close() + + _, err = io.Copy(dstFile, srcFile) + return err +} + +// GetBFETotalBytesBodyBuffer queries the BFE monitor endpoint for the current +// total bytes_body buffer size. +func GetBFETotalBytesBodyBuffer(monitorPort int) (int64, error) { + url := fmt.Sprintf("http://127.0.0.1:%d/monitor/server_stat", monitorPort) + resp, err := http.Get(url) + if err != nil { + return 0, err + } + defer resp.Body.Close() + data, err := io.ReadAll(resp.Body) + if err != nil { + return 0, err + } + var stat struct { + TotalBytesBodyBuffer int64 `json:"total_bytes_body_buffer"` + } + if err := json.Unmarshal(data, &stat); err != nil { + return 0, err + } + return stat.TotalBytesBodyBuffer, nil +} + +// clusterSubName maps a cluster name to the sub-cluster name used in gslb.data. +func clusterSubName(clusterName string) string { + switch clusterName { + case "cluster_primary_a": + return "a" + case "cluster_primary_b": + return "b" + case "cluster_primary_c": + return "c" + case "cluster_fallback_1": + return "fb1" + case "cluster_fallback_2": + return "fb2" + case "cluster_entity_default": + return "entity" + case "cluster_global_default": + return "global" + case "cluster_holder": + return "holder" + case "cluster_multi_key": + return "multi" + case "cluster_fallback_ok": + return "fb" + case "cluster_rmb": + return "rmb" + case "cluster_no_table": + return "notable" + case "cluster_fallback_rmb": + return "fallback_rmb" + case "cluster_openrouter": + return "openrouter" + case "cluster_fallback": + return "fallback" + case "cluster_default": + return "default" + } + return "sub" +} diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/sc01_route_table_lookup_test.go b/tests/integration/implementation/scenario-SC01-route-table-lookup/sc01_route_table_lookup_test.go new file mode 100644 index 000000000..fc600b232 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/sc01_route_table_lookup_test.go @@ -0,0 +1,517 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package sc01 + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "math/rand" + "net/http" + "os" + "path/filepath" + "strconv" + "strings" + "testing" + "time" + + "github.com/bfenetworks/bfe/tests/integration/common" +) + +const ( + apiHost = "api.example.org" + otherHost = "other.example.org" + entityHost = "entity.example.org" + unknownHost = "unknown.example.org" + largeHost = "large.example.org" + apiPath = "/v1/chat/completions" + apiKeyUserA = "ak_user_a" + apiKeyUserB = "ak_user_b" + apiKeyNoBinding = "ak_no_binding" + + // accessibleBodySize mirrors the value in bfe.conf. + accessibleBodySize = 4 * 1024 * 1024 +) + +var clusterNames = []string{ + "cluster_primary_a", + "cluster_primary_b", + "cluster_primary_c", + "cluster_fallback_1", + "cluster_fallback_2", + "cluster_entity_default", + "cluster_global_default", + "cluster_holder", +} + +// testEnvOption configures a testEnv before BFE is started. +type testEnvOption func(*testEnv) + +// withTotalBodyBufferSize sets the BFE totalBodyBufferSize config. +func withTotalBodyBufferSize(size int64) testEnvOption { + return func(e *testEnv) { + e.totalBodyBufferSize = size + } +} + +var emptyJSONBody = []byte("{}") + +// testEnv holds all resources for a single SC01 integration test. +type testEnv struct { + t *testing.T + processEnv *common.ProcessEnv + backends map[string]*common.MockBackend + bfePort int + bfeMonitorPort int + stopBFE func() + totalBodyBufferSize int64 +} + +func newTestEnv(t *testing.T, responseMap map[string]int, opts ...testEnvOption) *testEnv { + e := &testEnv{ + t: t, + backends: make(map[string]*common.MockBackend), + } + + // Start mock backends. + for _, name := range clusterNames { + resp := http.StatusOK + if responseMap != nil { + if r, ok := responseMap[name]; ok { + resp = r + } + } + e.backends[name] = common.NewMockBackend(name, resp, fmt.Sprintf("response from %s", name)) + } + + // Apply options before building the BFE config. + for _, opt := range opts { + opt(e) + } + + // Build BFE binary and start real BFE process. + e.processEnv = common.NewProcessEnv(t) + e.processEnv.Build() + + confDir := filepath.Join(e.processEnv.WorkDir(), "conf") + logDir := filepath.Join(e.processEnv.WorkDir(), "log") + + builder := &common.BFEConfigBuilder{ + TemplateDir: "testdata", + TargetConfDir: confDir, + Backends: e.backends, + TotalBodyBufferSize: e.totalBodyBufferSize, + } + if err := builder.Build(); err != nil { + t.Fatalf("build bfe config failed: %v", err) + } + + e.bfePort, e.bfeMonitorPort, e.stopBFE = e.processEnv.StartBFE(confDir, logDir) + return e +} + +func (e *testEnv) Close() { + if e.stopBFE != nil { + e.stopBFE() + } + for _, b := range e.backends { + b.Close() + } +} + +func (e *testEnv) logBFEException() { + data, err := os.ReadFile(filepath.Join(e.processEnv.WorkDir(), "log", "exception.log")) + if err == nil && len(data) > 0 { + e.t.Logf("bfe exception log:\n%s", string(data)) + } +} + +func (e *testEnv) sendRequest(host, apiKey string, body []byte) (*http.Response, string, error) { + contentType := "" + if body != nil { + contentType = "application/json" + } + return e.sendRequestWithContentType(host, apiKey, body, contentType) +} + +func (e *testEnv) sendRequestWithContentType(host, apiKey string, body []byte, contentType string) (*http.Response, string, error) { + url := fmt.Sprintf("http://127.0.0.1:%d%s", e.bfePort, apiPath) + var bodyReader io.Reader + if body != nil { + bodyReader = bytes.NewReader(body) + } + req, err := http.NewRequest(http.MethodPost, url, bodyReader) + if err != nil { + return nil, "", err + } + req.Host = host + req.Header.Set("Authorization", "Bearer "+apiKey) + if contentType != "" { + req.Header.Set("Content-Type", contentType) + } + + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + respBody, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return resp, string(respBody), nil +} + +func generateBody(size int) []byte { + body := make([]byte, size) + for i := range body { + body[i] = byte('a' + i%26) + } + return body +} + +func TestTC01_APIKeyRouteHit(t *testing.T) { + e := newTestEnv(t, nil) + defer e.Close() + + resp, body, err := e.sendRequest(apiHost, apiKeyUserA, emptyJSONBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + hits := e.backends["cluster_primary_a"].Hits() + + e.backends["cluster_primary_b"].Hits() + + e.backends["cluster_primary_c"].Hits() + if hits != 1 { + t.Fatalf("expected exactly one primary cluster hit, got %d", hits) + } + if e.backends["cluster_entity_default"].Hits() != 0 { + t.Fatalf("expected entity cluster not hit, got %d", e.backends["cluster_entity_default"].Hits()) + } + if e.backends["cluster_global_default"].Hits() != 0 { + t.Fatalf("expected global cluster not hit, got %d", e.backends["cluster_global_default"].Hits()) + } +} + +func TestTC02_EntityFallback(t *testing.T) { + e := newTestEnv(t, nil) + defer e.Close() + + resp, body, err := e.sendRequest(otherHost, apiKeyUserA, emptyJSONBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends["cluster_entity_default"].Hits() != 1 { + t.Fatalf("expected cluster_entity_default hit once, got %d", e.backends["cluster_entity_default"].Hits()) + } + + hits := e.backends["cluster_primary_a"].Hits() + + e.backends["cluster_primary_b"].Hits() + + e.backends["cluster_primary_c"].Hits() + if hits != 0 { + t.Fatalf("expected no primary cluster hit, got %d", hits) + } + if e.backends["cluster_global_default"].Hits() != 0 { + t.Fatalf("expected global cluster not hit, got %d", e.backends["cluster_global_default"].Hits()) + } +} + +func TestTC03_NoBinding404(t *testing.T) { + e := newTestEnv(t, nil) + defer e.Close() + + resp, body, err := e.sendRequest(apiHost, apiKeyNoBinding, emptyJSONBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusNotFound { + e.logBFEException() + t.Fatalf("expected status 404, got %d, body: %s", resp.StatusCode, body) + } + if !strings.Contains(body, "AI route not found") { + t.Fatalf("expected 'AI route not found' in body, got %q", body) + } + for _, name := range clusterNames { + if e.backends[name].Hits() != 0 { + t.Fatalf("expected %s not hit, got %d", name, e.backends[name].Hits()) + } + } +} + +func TestTC04_MultiTargetsWeightedSelection(t *testing.T) { + e := newTestEnv(t, nil) + defer e.Close() + + const total = 1000 + for i := 0; i < total; i++ { + resp, _, err := e.sendRequest(apiHost, apiKeyUserA, emptyJSONBody) + if err != nil { + t.Fatalf("request %d: send failed: %v", i, err) + } + if resp.StatusCode != http.StatusOK { + t.Fatalf("request %d: expected status 200, got %d", i, resp.StatusCode) + } + } + + hitsA := e.backends["cluster_primary_a"].Hits() + hitsB := e.backends["cluster_primary_b"].Hits() + hitsC := e.backends["cluster_primary_c"].Hits() + + t.Logf("hits distribution: a=%d b=%d c=%d", hitsA, hitsB, hitsC) + + if hitsA+hitsB+hitsC != total { + t.Fatalf("expected total hits %d, got %d", total, hitsA+hitsB+hitsC) + } + + // weights: 60 / 30 / 10, allow +/-50 tolerance. + if hitsA < 550 || hitsA > 650 { + t.Fatalf("expected hitsA around 600, got %d", hitsA) + } + if hitsB < 250 || hitsB > 350 { + t.Fatalf("expected hitsB around 300, got %d", hitsB) + } + if hitsC < 50 || hitsC > 150 { + t.Fatalf("expected hitsC around 100, got %d", hitsC) + } +} + +func TestTC05_MultiFallbacksSuccess(t *testing.T) { + e := newTestEnv(t, map[string]int{ + "cluster_primary_a": http.StatusInternalServerError, + "cluster_primary_b": http.StatusInternalServerError, + "cluster_primary_c": http.StatusInternalServerError, + "cluster_fallback_1": http.StatusBadGateway, + "cluster_fallback_2": http.StatusOK, + }) + defer e.Close() + + resp, body, err := e.sendRequest(apiHost, apiKeyUserA, emptyJSONBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + primaryHits := e.backends["cluster_primary_a"].Hits() + + e.backends["cluster_primary_b"].Hits() + + e.backends["cluster_primary_c"].Hits() + if primaryHits < 1 { + t.Fatalf("expected at least one primary hit, got %d", primaryHits) + } + if e.backends["cluster_fallback_1"].Hits() != 1 { + t.Fatalf("expected cluster_fallback_1 hit once, got %d", e.backends["cluster_fallback_1"].Hits()) + } + if e.backends["cluster_fallback_2"].Hits() != 1 { + t.Fatalf("expected cluster_fallback_2 hit once, got %d", e.backends["cluster_fallback_2"].Hits()) + } +} + +func TestTC06_MultiFallbacksAllFail(t *testing.T) { + e := newTestEnv(t, map[string]int{ + "cluster_primary_a": http.StatusInternalServerError, + "cluster_primary_b": http.StatusInternalServerError, + "cluster_primary_c": http.StatusInternalServerError, + "cluster_fallback_1": http.StatusInternalServerError, + "cluster_fallback_2": http.StatusInternalServerError, + }) + defer e.Close() + + resp, body, err := e.sendRequest(apiHost, apiKeyUserA, emptyJSONBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusInternalServerError { + e.logBFEException() + t.Fatalf("expected status 500, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends["cluster_fallback_2"].Hits() != 1 { + t.Fatalf("expected cluster_fallback_2 hit once, got %d", e.backends["cluster_fallback_2"].Hits()) + } +} + +func TestTC07_ModelOverrideAndFallback(t *testing.T) { + e := newTestEnv(t, map[string]int{ + "cluster_primary_a": http.StatusInternalServerError, + "cluster_primary_b": http.StatusInternalServerError, + "cluster_primary_c": http.StatusInternalServerError, + "cluster_fallback_1": http.StatusOK, + }) + defer e.Close() + + body := []byte(`{"model":"origin-model","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(apiHost, apiKeyUserA, body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + var primaryGotTargetModel bool + for _, name := range []string{"cluster_primary_a", "cluster_primary_b", "cluster_primary_c"} { + models := e.backends[name].Models() + for _, m := range models { + if strings.HasPrefix(m, "target-model-") { + primaryGotTargetModel = true + } + } + } + if !primaryGotTargetModel { + t.Fatalf("expected at least one primary backend to receive target model") + } + + fallbackModels := e.backends["cluster_fallback_1"].Models() + if len(fallbackModels) != 1 || fallbackModels[0] != "fallback-model-1" { + t.Fatalf("expected fallback model 'fallback-model-1', got %v", fallbackModels) + } + + fallbackBodies := e.backends["cluster_fallback_1"].RequestBodies() + if len(fallbackBodies) != 1 { + t.Fatalf("expected exactly one fallback request body, got %d", len(fallbackBodies)) + } + var fallbackBody map[string]interface{} + if err := json.Unmarshal(fallbackBodies[0], &fallbackBody); err != nil { + t.Fatalf("fallback body is not valid json: %v, body: %q", err, fallbackBodies[0]) + } + if fallbackBody["model"] != "fallback-model-1" { + t.Fatalf("expected fallback model 'fallback-model-1', got %v", fallbackBody["model"]) + } + messages, ok := fallbackBody["messages"].([]interface{}) + if !ok || len(messages) != 1 { + t.Fatalf("expected fallback body to preserve messages, got %v", fallbackBody["messages"]) + } +} + +func TestTC08_FallbackWithPartialBody(t *testing.T) { + // Primary reads 1 KB of the body and then closes the connection. + // Because the whole body (1 MB) is smaller than accessibleBodySize, + // BFE can buffer and rewind it, so the fallback backend receives the + // complete body. + e := newTestEnv(t, map[string]int{ + "cluster_fallback_1": http.StatusOK, + }) + defer e.Close() + + e.backends["cluster_primary_a"].ReadBeforeClose = 1024 + + body := generateBody(1024 * 1024) + resp, respBody, err := e.sendRequestWithContentType(largeHost, apiKeyUserA, body, "application/octet-stream") + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends["cluster_primary_a"].Hits() != 1 { + t.Fatalf("expected cluster_primary_a hit once, got %d", e.backends["cluster_primary_a"].Hits()) + } + if e.backends["cluster_fallback_1"].Hits() != 1 { + t.Fatalf("expected cluster_fallback_1 hit once, got %d", e.backends["cluster_fallback_1"].Hits()) + } + + fallbackBodies := e.backends["cluster_fallback_1"].RequestBodies() + if len(fallbackBodies) != 1 { + t.Fatalf("expected exactly one fallback request body, got %d", len(fallbackBodies)) + } + if len(fallbackBodies[0]) != len(body) { + t.Fatalf("expected fallback body length %d, got %d", len(body), len(fallbackBodies[0])) + } + for i := range body { + if fallbackBodies[0][i] != body[i] { + t.Fatalf("fallback body differs at byte %d", i) + } + } +} + +func TestTC09_BodyExceedsAccessibleBodySize(t *testing.T) { + // Body is larger than accessibleBodySize (4 MB). BFE cannot buffer the + // entire body, so after the primary cluster fails, fallback is disabled. + e := newTestEnv(t, map[string]int{ + "cluster_primary_a": http.StatusInternalServerError, + "cluster_fallback_1": http.StatusOK, + }) + defer e.Close() + + body := generateBody(5 * 1024 * 1024) + resp, _, err := e.sendRequestWithContentType(largeHost, apiKeyUserA, body, "application/octet-stream") + if err == nil && resp != nil && resp.StatusCode == http.StatusOK { + t.Fatalf("expected request not to succeed via fallback, got status 200") + } + + if e.backends["cluster_primary_a"].Hits() != 1 { + t.Fatalf("expected cluster_primary_a hit once, got %d", e.backends["cluster_primary_a"].Hits()) + } + if e.backends["cluster_fallback_1"].Hits() != 0 { + t.Fatalf("expected cluster_fallback_1 not hit, got %d", e.backends["cluster_fallback_1"].Hits()) + } +} + +func TestTC10_TotalBodyBufferSizeExceedsLimit(t *testing.T) { + // Set totalBodyBufferSize to 2 MB and pre-seed the global bytes_body buffer + // counter to the same value. When the test request tries fallback, BFE sees + // that the limit is already reached and disables fallback. + const limit = 2 * 1024 * 1024 + t.Setenv("BFE_TEST_INITIAL_TOTAL_BYTES_BODY_BUFFER", strconv.FormatInt(limit, 10)) + + e := newTestEnv(t, map[string]int{ + "cluster_primary_a": http.StatusInternalServerError, + "cluster_fallback_1": http.StatusOK, + }, withTotalBodyBufferSize(limit)) + defer e.Close() + + // Verify the counter was initialized as expected. + total, err := common.GetBFETotalBytesBodyBuffer(e.bfeMonitorPort) + if err != nil { + t.Fatalf("failed to read initial total_bytes_body_buffer: %v", err) + } + if total != limit { + t.Fatalf("expected initial total_bytes_body_buffer %d, got %d", limit, total) + } + + testBody := generateBody(512 * 1024) + resp, respBody, err := e.sendRequestWithContentType(largeHost, apiKeyUserA, testBody, "application/octet-stream") + if err == nil && resp != nil && resp.StatusCode == http.StatusOK { + t.Fatalf("expected request not to succeed via fallback, got status %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends["cluster_primary_a"].Hits() != 1 { + t.Fatalf("expected cluster_primary_a hit once, got %d", e.backends["cluster_primary_a"].Hits()) + } + if e.backends["cluster_fallback_1"].Hits() != 0 { + t.Fatalf("expected cluster_fallback_1 not hit, got %d", e.backends["cluster_fallback_1"].Hits()) + } +} + +func TestMain(m *testing.M) { + rand.Seed(time.Now().UnixNano()) + os.Exit(m.Run()) +} diff --git a/tests/integration/mod_ai_route/testdata/bfe.conf b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/bfe.conf similarity index 75% rename from tests/integration/mod_ai_route/testdata/bfe.conf rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/bfe.conf index 46a1eeadd..5a232930e 100644 --- a/tests/integration/mod_ai_route/testdata/bfe.conf +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/bfe.conf @@ -2,6 +2,9 @@ httpPort = 18080 httpsPort = 18443 monitorPort = 18081 +httpAddr = "127.0.0.1" +httpsAddr = "127.0.0.1" +monitorAddr = "127.0.0.1" MonitorEnabled = true maxCpus = 1 @@ -13,6 +16,11 @@ GracefulShutdownTimeout = 10 EnableAiGateway = true +accessibleBodySize = 4194304 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +totalBodyBufferSize = 0 + Modules = mod_ai_route hostRuleConf = server_data_conf/host_rule.data diff --git a/tests/integration/mod_ai_route/testdata/cluster_conf/cluster_conf.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/cluster_conf/cluster_conf.data similarity index 78% rename from tests/integration/mod_ai_route/testdata/cluster_conf/cluster_conf.data rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/cluster_conf/cluster_conf.data index ed3728a75..4604696f6 100644 --- a/tests/integration/mod_ai_route/testdata/cluster_conf/cluster_conf.data +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/cluster_conf/cluster_conf.data @@ -23,7 +23,8 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true } }, "cluster_primary_b": { @@ -48,7 +49,8 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true } }, "cluster_primary_c": { @@ -73,7 +75,8 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true } }, "cluster_fallback_1": { @@ -98,7 +101,8 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true } }, "cluster_fallback_2": { @@ -123,7 +127,8 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true } }, "cluster_entity_default": { @@ -148,7 +153,8 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true } }, "cluster_global_default": { @@ -173,7 +179,34 @@ "RetryMax": 0, "TimeoutReadClient": 5000, "TimeoutWriteClient": 5000, - "TimeoutReadClientAgain": 5000 + "TimeoutReadClientAgain": 5000, + "DisableHealthCheck": true + } + }, + "cluster_holder": { + "BackendConf": { + "TimeoutConnSrv": 1000, + "TimeoutWriteSrv": 60000, + "TimeoutReadSrv": 60000, + "TimeoutResponseHeader": 60000, + "SlowStartTime": 0 + }, + "CheckConf": { + "Uri": "/health", + "FailNum": 3, + "CheckInterval": 1000, + "Response": "200 OK" + }, + "GslbBasic": { + "CrossRetry": 0, + "RetryMax": 0 + }, + "ClusterBasic": { + "RetryMax": 0, + "TimeoutReadClient": 30000, + "TimeoutWriteClient": 30000, + "TimeoutReadClientAgain": 30000, + "DisableHealthCheck": true } } } diff --git a/tests/integration/mod_ai_route/testdata/cluster_conf/gslb.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/cluster_conf/gslb.data similarity index 88% rename from tests/integration/mod_ai_route/testdata/cluster_conf/gslb.data rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/cluster_conf/gslb.data index 418c6d01a..cbaf36437 100644 --- a/tests/integration/mod_ai_route/testdata/cluster_conf/gslb.data +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/cluster_conf/gslb.data @@ -27,6 +27,10 @@ "cluster_global_default": { "GSLB_BLACKHOLE": 0, "sub_global": 100 + }, + "cluster_holder": { + "GSLB_BLACKHOLE": 0, + "sub_holder": 100 } }, "hostname": "gslb-test", diff --git a/tests/integration/mod_ai_route/testdata/mod_ai_route/ai_route.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/mod_ai_route/ai_route.data similarity index 72% rename from tests/integration/mod_ai_route/testdata/mod_ai_route/ai_route.data rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/mod_ai_route/ai_route.data index 18b54d3f5..482272d9f 100644 --- a/tests/integration/mod_ai_route/testdata/mod_ai_route/ai_route.data +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/mod_ai_route/ai_route.data @@ -35,6 +35,40 @@ "Model": "fallback-model-2" } ] + }, + { + "name": "user_a-large", + "Cond": "req_host_in(\"large.example.org\")", + "targets": [ + { + "ClusterName": "cluster_primary_a", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_fallback_1", + "Model": "" + } + ] + }, + { + "name": "user_a-holder", + "Cond": "req_host_in(\"holder.example.org\")", + "targets": [ + { + "ClusterName": "cluster_holder", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_fallback_2", + "Model": "" + } + ] } ] }, diff --git a/tests/integration/mod_ai_route/testdata/mod_ai_route/mod_ai_route.conf b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/mod_ai_route/mod_ai_route.conf similarity index 100% rename from tests/integration/mod_ai_route/testdata/mod_ai_route/mod_ai_route.conf rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/mod_ai_route/mod_ai_route.conf diff --git a/tests/integration/mod_ai_route/testdata/server_data_conf/host_rule.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/host_rule.data similarity index 73% rename from tests/integration/mod_ai_route/testdata/server_data_conf/host_rule.data rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/host_rule.data index 5522ae478..0de857b9b 100644 --- a/tests/integration/mod_ai_route/testdata/server_data_conf/host_rule.data +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/host_rule.data @@ -5,7 +5,9 @@ "api.example.org", "other.example.org", "entity.example.org", - "unknown.example.org" + "unknown.example.org", + "large.example.org", + "holder.example.org" ] }, "HostTags": { diff --git a/tests/integration/mod_ai_route/testdata/server_data_conf/route_rule.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/route_rule.data similarity index 100% rename from tests/integration/mod_ai_route/testdata/server_data_conf/route_rule.data rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/route_rule.data diff --git a/tests/integration/mod_ai_route/testdata/server_data_conf/vip_rule.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/vip_rule.data similarity index 100% rename from tests/integration/mod_ai_route/testdata/server_data_conf/vip_rule.data rename to tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/server_data_conf/vip_rule.data diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/bfe_i_ca.crt b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/bfe_i_ca.crt new file mode 100644 index 000000000..f1e78f73c --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/bfe_i_ca.crt @@ -0,0 +1,23 @@ +-----BEGIN CERTIFICATE----- +MIIDwDCCAqigAwIBAgIBCzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +NDAxNDZaFw0zNzA3MTIxNDAxNDZaMIGQMQswCQYDVQQGEwJjbjEQMA4GA1UECAwH +YmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsMC3lpbmdmZWkt +ZGV2MRgwFgYDVQQDDA95aW5nZmVpLWRldi1pY2ExKTAnBgkqhkiG9w0BCQEWGmxp +YW5nY2h1YW5AeWYtbmV0d29ya3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A +MIIBCgKCAQEA6eFuWgoknixrRO9NCX4jAKyLtcAOWVJqVN2yX7CxjZjLyPurjvTZ +W73NYUbrPAN4AB5gY6UAPzuiEOSopVVIZ0OschK0cJldu9vZ0mZObBOsovuFcLQq +dgNSJ5slJSk7tCgD2EnCB3GYPG4D+uIKYd0c49wzTWWv4bjDwpgnf0LQbFpy7GhN +7D59zFH4qgOK/IQ5vaTMGyvIvtWR5/1Gvc9MLpGopTgi0DiNLed4UwDYrod5kysl +q3UcB5puONHQISOVoD3uRxo7wdsmVsHUfW7YfAWkhi6ec8mx9fy8IyE6f7GlXnuV +ysNccwqyEotL5bOXPJCqwUCL1v+iKTMPWwIDAQABo1AwTjAdBgNVHQ4EFgQUCjvC +vDVr8QXh9/hMI5xBgNvoSFgwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW +1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAhIggK/6JK4N7+HZW +VoTemwRpilugZZyKrcVAHbiiwUfXVQVuI64vc+yHWMSFRD1mykHkKBFzxEoDabMl +ASBtUJNt4b4zEL9V7k295vmAOp2IdLUQxlgeKWqABm4DGX96tGR9nKQTcn1ZeAxx +NyQFV1aj+dnNcF1iFFNF6t0bRrOEZ/aRSiu1bWp3Dj1JYTjXbyyplh3Ktb8lv4lt +EmvCXjo/l4TgQC9233kcTkXcq1swppzkkXfhB0NVuf9DE3C2xAWX0b2FiIEqnAhl +Bx0Cn3RrX7PZ6qrdRL6oBKvGJV6DP8BlF4LXiuD1VN/waQFJha57KQ78ZYYIXxQg +WMjWgA== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/bfe_r_ca.crt b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/bfe_r_ca.crt new file mode 100644 index 000000000..2a6db1e49 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/bfe_r_ca.crt @@ -0,0 +1,22 @@ +-----BEGIN CERTIFICATE----- +MIIDkzCCAnugAwIBAgIBCjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +MzQ5NTVaFw0zNzA3MTIxMzQ5NTVaMGQxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxFzAVBgNVBAMMDnlpbmdmZWktZGV2LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOC +AQ8AMIIBCgKCAQEAvNA3HrsMjBcXrMIIhGVWsurIA1F9jxKeA7dh06H00Vt4inVV +SUvNFrTqgPRhLkAhGRMPxrjVRgJ5bbFqqXIuPIpUFBhUsWXIDH+oVXQl9jsxAXaG +gZ0lTO/uYR9qyrS1rj9nyNPwRf59Al/VlsQL71cNQ/T/agJ4PfvPfULTPLOsqclJ +hj0IgXmDj464dqcdG3ZdfXpfhNF6ab+8YjpwafTRmY+LoV8qjUwsYeJMcW4N8pxJ +8F2ktZj9J6uWepNGj+87ZeXg9XquzC62ASIFzPjoE1WN//Q518EqizxhuLGBDpK3 +6sEYUK4kHYUL4gZKFPlKTPXIl0ZsJIM5PsBMKwIDAQABo1AwTjAdBgNVHQ4EFgQU +DZloT8VhVbysD819oG/oqHdW1D0wHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/o +qHdW1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEALgB+IcN3UD4T ++4g8jaOyOiSUpUVdYqW+3go5DppqnvlGNq99N2cXKqssVPa/T9TikEcBEicFa8zU +bwlx6TEte+MkWfWdQxFR1EuI1FgKr3ps6ZBRr7MPpnYmI7K9aK372K9n7WrQhbmP +s7ult8bWB1/t6o3R7B9ChNkWT+7DPD4+FvB1GMJSGPno7cdnvDkevBOuC2DnQl3M ++ADFAge1Lo8wKBy6gYkNFd2BfarHGvRC5Qmmrme+RIpWZnvux1+lfXIInnfSTJRM +uAo/ePkoNsM3qQll6uEdhDxOMx8Pq94bCkM3DtI3ObuxWXjKCgUm/n9yetUvPj4U +iYhRUqHpUg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_bfe_dev.crt b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_bfe_dev.crt new file mode 100644 index 000000000..2164fb602 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_bfe_dev.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 18 (0x12) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Jan 31 02:55:32 2024 GMT + Not After : Jan 28 02:55:32 2034 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=bfe-dev/emailAddress=dev@example.org + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:b2:7f:c1:0c:cd:49:43:d9:99:78:15:4c:5a:52: + a9:a7:bf:d5:eb:92:71:43:e1:37:e5:29:1b:68:f4: + 6f:4c:ea:fa:a4:8a:7c:29:01:2a:fa:7c:81:5b:c8: + eb:a1:40:94:b7:2b:82:e2:00:08:36:84:f0:b7:d2: + 5a:1e:56:97:aa:36:ff:4d:07:49:2d:fe:25:3a:f9: + e9:f1:ad:4e:6e:21:97:a9:f9:a2:a5:ac:82:23:0a: + d2:e0:97:cd:2f:14:b1:f0:8a:a8:e6:c5:97:45:94: + f8:8e:3f:96:66:5b:0b:8c:7c:07:61:18:44:92:f7: + 23:5a:c2:4b:88:58:59:5d:ca:5c:0d:6e:dd:ff:18: + 59:65:df:95:99:e3:3a:36:48:1f:3f:3a:e6:ce:85: + f3:0b:04:5e:92:ed:6f:8e:74:92:e4:37:46:da:5f: + 17:62:9c:82:40:06:fb:29:f8:55:f2:ba:23:75:ca: + 64:c0:45:03:12:bd:f5:17:15:7e:47:d5:bd:30:f2: + 99:ca:6b:e3:07:b0:ae:44:89:1e:10:26:ea:75:df: + 6f:07:b6:47:76:54:47:4f:6c:f6:68:fe:a8:cf:22: + 20:73:e8:19:55:8a:fe:f5:78:e8:51:88:52:80:1f: + 79:d4:c5:ae:8f:d2:2b:f6:41:01:42:01:cf:98:c2: + c9:25 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + EE:E8:68:42:DF:B1:F0:EF:6F:47:51:BD:D4:94:60:1F:05:85:A1:03 + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Extended Key Usage: + TLS Web Client Authentication + Signature Algorithm: sha256WithRSAEncryption + a9:a6:26:8e:42:61:15:22:ee:fc:b5:e1:e4:6b:dd:ac:f5:15: + 11:39:10:9a:ca:6f:85:fd:cb:90:1c:b2:4f:fe:29:de:b0:e7: + 73:e2:f7:5e:63:8c:7f:c1:7e:75:2e:c9:9d:e9:c2:45:75:f3: + 27:ba:82:94:de:7f:6c:87:0c:5c:71:af:0f:14:00:68:35:f7: + 5a:4a:ff:f5:ef:35:dd:50:72:76:f0:6f:b6:7b:42:33:07:b4: + 24:44:0a:fd:9d:61:9e:44:e8:88:0f:02:76:c6:90:3f:9d:1b: + d8:3b:64:25:2a:a3:39:78:38:bd:20:89:4a:9c:bd:68:38:18: + 4c:cb:20:3a:9b:5b:5f:58:52:86:73:de:85:fe:d6:a1:c6:a7: + 86:b0:96:4b:fa:28:04:ad:5d:85:e8:a1:fc:ca:0f:3c:be:5c: + 90:7e:3e:84:ae:67:ee:9a:72:71:3c:b2:80:45:82:fc:7e:58: + 74:99:42:c5:c3:8a:4a:eb:e1:8b:d5:84:ce:25:aa:a1:75:79: + 94:66:ae:ee:df:30:15:0b:b5:c5:b1:2c:d5:0a:54:78:b6:2e: + 67:29:81:41:f6:16:49:31:96:e7:41:e1:99:6b:27:57:bb:7d: + 76:eb:e4:d5:59:aa:a2:5c:bd:1c:18:2a:fa:9d:28:1a:0b:b6: + bf:7d:58:1a +-----BEGIN CERTIFICATE----- +MIID7jCCAtagAwIBAgIBEjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yNDAxMzEw +MjU1MzJaFw0zNDAxMjgwMjU1MzJaMH0xCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxEDAOBgNVBAMMB2JmZS1kZXYxHjAcBgkqhkiG9w0BCQEWD2RldkBleGFtcGxl +Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ/wQzNSUPZmXgV +TFpSqae/1euScUPhN+UpG2j0b0zq+qSKfCkBKvp8gVvI66FAlLcrguIACDaE8LfS +Wh5Wl6o2/00HSS3+JTr56fGtTm4hl6n5oqWsgiMK0uCXzS8UsfCKqObFl0WU+I4/ +lmZbC4x8B2EYRJL3I1rCS4hYWV3KXA1u3f8YWWXflZnjOjZIHz865s6F8wsEXpLt +b450kuQ3RtpfF2KcgkAG+yn4VfK6I3XKZMBFAxK99RcVfkfVvTDymcpr4wewrkSJ +HhAm6nXfbwe2R3ZUR09s9mj+qM8iIHPoGVWK/vV46FGIUoAfedTFro/SK/ZBAUIB +z5jCySUCAwEAAaOBkTCBjjAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu +U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU7uhoQt+x8O9vR1G9 +1JRgHwWFoQMwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW1D0wEwYDVR0l +BAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKmmJo5CYRUi7vy14eRr +3az1FRE5EJrKb4X9y5Acsk/+Kd6w53Pi915jjH/BfnUuyZ3pwkV18ye6gpTef2yH +DFxxrw8UAGg191pK//XvNd1Qcnbwb7Z7QjMHtCRECv2dYZ5E6IgPAnbGkD+dG9g7 +ZCUqozl4OL0giUqcvWg4GEzLIDqbW19YUoZz3oX+1qHGp4awlkv6KAStXYXoofzK +Dzy+XJB+PoSuZ+6acnE8soBFgvx+WHSZQsXDikrr4YvVhM4lqqF1eZRmru7fMBUL +tcWxLNUKVHi2LmcpgUH2FkkxludB4ZlrJ1e7fXbr5NVZqqJcvRwYKvqdKBoLtr99 +WBo= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem new file mode 100644 index 000000000..764aea3ad --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAsn/BDM1JQ9mZeBVMWlKpp7/V65JxQ+E35SkbaPRvTOr6pIp8 +KQEq+nyBW8jroUCUtyuC4gAINoTwt9JaHlaXqjb/TQdJLf4lOvnp8a1ObiGXqfmi +payCIwrS4JfNLxSx8Iqo5sWXRZT4jj+WZlsLjHwHYRhEkvcjWsJLiFhZXcpcDW7d +/xhZZd+VmeM6NkgfPzrmzoXzCwReku1vjnSS5DdG2l8XYpyCQAb7KfhV8rojdcpk +wEUDEr31FxV+R9W9MPKZymvjB7CuRIkeECbqdd9vB7ZHdlRHT2z2aP6ozyIgc+gZ +VYr+9XjoUYhSgB951MWuj9Ir9kEBQgHPmMLJJQIDAQABAoIBAFAKHzupVb/58+o3 +yqv5wx94Uuk2GlnwxIqaezL94GaiO0/K1U/huS7m426P0rDU75qPBTpn/0bLJ9GV +nllaRNnLnYEh0juwaWtfovp+1ttlbseGK9uUVip2cQbKqvQAmKWe14vbcDCAU1Ad +zUgKbUxKVVjBdAZekVjiJNJ3o2L9WPhf/uQo7A1XAJh2DajlTbgvrDM73W+47QuU +X4OHU0FMio6bxupu3OWl1bMrnKhuC4qczZWf2nOpcVQa89rtopuP4ENLJuWkbeGk +YQpNilEclnAa/Noumt/j/6GKC1EEHFsH2CNRRIazcZrsFhkSKc4pn1Y/WI3vj8kZ ++RYnJsUCgYEA6gr0x8suwRTAmVpoPk4XyP1x+eInG7onV5RjgsUtgruYd9naxRHg +2p8PHcv32pDs51Fa+4RldyMd/jec1SscRF5/+VOP9qeoRnaDqUu+uASgEO3OBxbP +JcWovyxRHIQxbYCQtqIr9bdzXw55MBLZou/sBUVTAkrIyPVyjWqZlV8CgYEAwz7L +YyYN615TsrzZKURxMjj94Nmob/NldSLRXaR3Ax7/ABtEOA685cwQxq7ONdkJTMIA +uR8u2GHZSzGiWnehuF6Zp7Xs71a57eFbs3ueZvvEba4Dff7hl7Y4tTlwrKndKjvP +J/5a2Ol8siQcRWAXHOdzggEHMSZ/sB4hWswly/sCgYEAhLRBpyemEwTZUBrbELjm +86gBgFajJi2fMSGKaxOygnYsNYjpauSAQnX99D87Aks6iM6wb/zaK3tV/lc6LgSL +uph6p7yh3JGj8JAyh0PTmDPHLtIoCAz+18QDsqJGO40ZGaXUaDn8Aw9J85QZUxDd +Jm4zvalZL+uHfarukRDolLECgYBUiupS4nWAh3XCnZeDEQna72avaFBROZmjIRJ7 +c+28wj009JmTlH4jGzvgbG0KUBKA1Div8Fq+g5AtyS498jNqvDvYrSQNdwZHhR/K +Fis++KHTxFfqxOU2Zkcj4d1yRpNn6EIJVVBNQL0n/g7n03XupCIWFw/gLoV343QZ +9vAe5QKBgA8ml59z1w3eUooc0yGfLhihXqCmM3IU006bFbODA30fBU4QKrHO8+Yx +Xbz9bi/1QLagLG6FzYQAkOjBlEBt5XLayYvwSb8xvWsm5A3vzTAbMFDOsDPEmRoH +dWtQccJcygOuK+PZtoZnNoJciNO5c9dZWD3xtIiURmX/kVtWrf6N +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_san_example.org.crt b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_san_example.org.crt new file mode 100644 index 000000000..78f3892b4 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_san_example.org.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 15 (0xf) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Dec 7 09:57:42 2023 GMT + Not After : Dec 4 09:57:42 2033 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei, CN=dev-test + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:d6:47:b0:17:69:91:4c:d0:66:c9:25:2e:38:f2: + 84:27:e2:7c:38:cc:04:b9:0c:8e:3d:cc:ef:4b:c5: + 35:2b:c1:82:d8:41:fc:25:c8:24:f2:e0:25:aa:f9: + 76:3c:e2:b2:50:fb:2d:ec:4e:16:d4:c1:da:1e:e3: + 51:9f:9d:c9:72:f9:cb:cc:14:9c:c1:82:c9:76:1f: + 98:dd:0e:b9:8a:20:d6:ab:f2:a6:f9:2f:23:81:f3: + af:e4:47:0c:55:95:94:de:ed:f6:a5:24:ee:32:e7: + 6b:79:e1:42:6f:a4:07:b2:95:1d:f5:a9:8e:60:42: + 70:42:bd:e2:30:18:68:74:52:32:98:a9:81:da:d8: + c6:6f:5e:1d:ce:79:b6:f3:ec:4f:ed:7d:22:57:d2: + 14:d0:fb:f2:50:d4:80:3b:89:ed:77:fd:45:6c:e6: + 52:6b:0a:52:71:ac:59:c8:d5:25:f5:40:03:fb:51: + b0:11:a7:00:79:d9:d8:4f:00:43:96:68:44:29:41: + dc:d2:cc:91:c8:61:95:41:4d:0e:66:5a:b5:15:67: + 3e:8a:6f:29:df:1c:8a:6f:ee:9e:97:9c:9e:69:71: + d3:34:52:75:e9:ea:e7:51:77:23:98:46:ca:47:a2: + d3:d3:97:03:41:4b:e3:33:11:72:2d:af:bf:2b:3e: + b3:51 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + 5A:27:32:9D:E7:36:24:A3:C1:DC:2F:95:80:C5:CF:0C:85:E8:E6:AF + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Subject Alternative Name: + DNS:example.org, DNS:www.example.org, DNS:example.com, DNS:*.example.com, IP Address:127.0.0.1, IP Address:192.168.0.100 + Signature Algorithm: sha256WithRSAEncryption + 1e:e8:e8:8a:ad:a8:0e:fc:c9:82:00:a1:ab:30:3c:a5:b9:dc: + d6:fb:86:ad:30:52:7f:61:be:90:a6:b8:56:bb:f1:0b:e6:39: + 38:65:09:6b:da:83:f7:65:ff:c4:21:de:b4:9e:8b:bd:1e:1c: + d1:d5:94:b8:18:79:f2:d0:06:51:39:67:13:40:3b:73:5b:cb: + ea:de:c1:19:76:f8:7b:0f:15:51:61:49:fb:98:f7:ea:4f:fc: + c2:fb:a7:f4:3c:48:64:14:79:b5:78:5b:20:10:b5:7a:2d:4c: + 04:51:60:ec:20:10:19:26:5f:e2:fd:32:59:67:e9:3f:48:8d: + f5:52:12:01:81:2c:c0:e5:72:cd:7d:0a:eb:7a:05:df:a0:77: + b9:ba:9a:7d:d1:4b:6a:44:e4:2d:98:af:bd:77:2b:f5:ef:26: + 4b:75:b3:97:d0:3a:bc:07:21:ef:71:92:30:fe:a2:79:e5:56: + d7:7e:c2:f3:57:ab:d7:de:fc:97:ed:20:0c:9a:cb:c5:5d:00: + 3b:61:29:e8:00:d4:39:e0:f2:4e:a4:03:c2:12:52:ff:e7:78: + f9:f7:c0:12:dc:36:a4:05:a2:f0:6b:47:e2:21:3d:a2:e1:a1: + 91:c7:ac:8f:b8:ae:58:65:e0:2b:57:80:eb:77:2d:48:ef:e6: + fb:b9:e1:20 +-----BEGIN CERTIFICATE----- +MIIEBzCCAu+gAwIBAgIBDzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzEyMDcw +OTU3NDJaFw0zMzEyMDQwOTU3NDJaMFoxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEQMA4GA1UECwwHeWluZ2ZlaTER +MA8GA1UEAwwIZGV2LXRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB +AQDWR7AXaZFM0GbJJS448oQn4nw4zAS5DI49zO9LxTUrwYLYQfwlyCTy4CWq+XY8 +4rJQ+y3sThbUwdoe41Gfncly+cvMFJzBgsl2H5jdDrmKINar8qb5LyOB86/kRwxV +lZTe7falJO4y52t54UJvpAeylR31qY5gQnBCveIwGGh0UjKYqYHa2MZvXh3Oebbz +7E/tfSJX0hTQ+/JQ1IA7ie13/UVs5lJrClJxrFnI1SX1QAP7UbARpwB52dhPAEOW +aEQpQdzSzJHIYZVBTQ5mWrUVZz6KbynfHIpv7p6XnJ5pcdM0UnXp6udRdyOYRspH +otPTlwNBS+MzEXItr78rPrNRAgMBAAGjgc0wgcowCQYDVR0TBAIwADAsBglghkgB +hvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYE +FFonMp3nNiSjwdwvlYDFzwyF6OavMB8GA1UdIwQYMBaAFA2ZaE/FYVW8rA/NfaBv +6Kh3VtQ9ME8GA1UdEQRIMEaCC2V4YW1wbGUub3Jngg93d3cuZXhhbXBsZS5vcmeC +C2V4YW1wbGUuY29tgg0qLmV4YW1wbGUuY29thwR/AAABhwTAqABkMA0GCSqGSIb3 +DQEBCwUAA4IBAQAe6OiKragO/MmCAKGrMDyludzW+4atMFJ/Yb6QprhWu/EL5jk4 +ZQlr2oP3Zf/EId60nou9HhzR1ZS4GHny0AZROWcTQDtzW8vq3sEZdvh7DxVRYUn7 +mPfqT/zC+6f0PEhkFHm1eFsgELV6LUwEUWDsIBAZJl/i/TJZZ+k/SI31UhIBgSzA +5XLNfQrregXfoHe5upp90UtqROQtmK+9dyv17yZLdbOX0Dq8ByHvcZIw/qJ55VbX +fsLzV6vX3vyX7SAMmsvFXQA7YSnoANQ54PJOpAPCElL/53j598AS3DakBaLwa0fi +IT2i4aGRx6yPuK5YZeArV4Drdy1I7+b7ueEg +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem new file mode 100644 index 000000000..393a79825 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEA1kewF2mRTNBmySUuOPKEJ+J8OMwEuQyOPczvS8U1K8GC2EH8 +Jcgk8uAlqvl2POKyUPst7E4W1MHaHuNRn53JcvnLzBScwYLJdh+Y3Q65iiDWq/Km ++S8jgfOv5EcMVZWU3u32pSTuMudreeFCb6QHspUd9amOYEJwQr3iMBhodFIymKmB +2tjGb14dznm28+xP7X0iV9IU0PvyUNSAO4ntd/1FbOZSawpScaxZyNUl9UAD+1Gw +EacAednYTwBDlmhEKUHc0syRyGGVQU0OZlq1FWc+im8p3xyKb+6el5yeaXHTNFJ1 +6ernUXcjmEbKR6LT05cDQUvjMxFyLa+/Kz6zUQIDAQABAoIBAC4sYGuLGf49Ygix +9FXdHFEj4rSyccoWRIhYoq/nHOAC4NkMzvKtQBj95+ABxVK1XstIdMrYwN6zrva8 +8Re9/mzCGwIs5uJj9ll30Y7A34Y+MUP4E7baS4JzKlG8ZZIDm4K2MFHBtXpOl8A5 +pAE+jVIUA9Kt6LohVuNq21SVzdxSfNYC/+SLqSftkWa/ZsqdkiHM5Hl+fVedh516 +IaLNW5hSthGh5n8dHY5h/AKPjfoq77aYp5/CUtJTC9mYdZu1j/W/pBVTRfOnwLQd +SQ1Xmr7f6q9Vmz+HnajIbFg9hQ54blvtUJ7DnugWxfUcoxf7ue79fnYjOIUOkRWw +8Iid/mECgYEA+5s0p0j+gkNZN5QtVNStoT04+1DqA1O381gczeaiz6Njt/MT0y5W +OpCsILQ70CpEjWAV+f6PDJiesDMxdGV+v2TCqK8ml8GahEczBLnHoAkPWCVf2XOX +oNj/CkZ2kmWufHFR+kcQbeDt1vFFcYUa61hKDFyNjinMW79Qy+PQID0CgYEA2gWd +7thE05sqU7/1MntmVRONKoAgnJmHcfSpWwLyh6E4YX3iKDSgI/9RnAMF39KUUY/O +XFWyIwAM9soeXknVsV/SmCaPeaEDiLHqz98aUEfvdLYMnuR883GgoXc4JrLsLw4z +oSi9lbAZFn0ekJ5L+rSFrY4rz9QZgYZxsLjUXKUCgYAbkaUSU2g3w8Np2J2i9u7T +hQ7SUsphdPHqAxSc5xGd6MxLYqIgeKpQHnwN1VHcfFUonIer7d2kxrBUpDdeBqT9 +ub+ulgqHhFo29ko70VNzUKrSwL2g6Q6LPFutt4zUe7nDvvL5loHRWF0XOTafurL5 +aKIsepO0KRZQU0U6IgszDQKBgFs6ZHaP2mTtFY4L0a75Ab3xu20gRgUhHRLq/H6P +wipMpMnuodaPBr9pU53Dig65D8T9Nq1eUnbgy4vs0T5FCPz6iqWN5RVQ8aieQhIP +WfRj1Wfx0WAfXcWEM2G9ACr5TWj3OVVjNclP8X9+hW6gPky+gv03c0+4gZ+4QRRg +ksPdAoGBAKXVv8L5Qt8rmi+QpF/6DVSeMB6lEevf319UEtyZPiYSBn1JJ9H+3Ddc +TMriXgZrw+PoXNJTAguDeGgzmtye1vraP0cjt7aG6sQ4YtMTnnuja880vK8z4i3Q +HJAi5fI8NvlW92dQBAccgrzFIKpRx+6CHwtCkNrxL2vJmGGzp8BR +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/certs/example.crt b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/certs/example.crt new file mode 100644 index 000000000..931874885 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/certs/example.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDEjCCAfqgAwIBAgIJAIAdu56fLE7OMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzM0NFoYDzIxMjAwNTE3MTAzMzQ0 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBALv+LV1aWIlcK9rI7IuRS8SCusqBnoyJec/ErKiA2gbfgZ/YS73L +Zud84yp45AIqauzcI5q+hrkmsRZ7CKqDzrG+jHavW7jF+0laetJwRt26AcQcOtQD +2ik2O+Dl1WHAFn4vUAQxb+Xz6WfSaQN0QfM74z06XUDDsr7g7+NYtMzhf98SJSoK +ne/dVKJ3Bc6e6tvhnCRwPtix4ektEodK6WeNHYxwJ6wSZ8cRLzdxgjdD/4OGfFuj +dn8zbOi3SQt5ZqVbcDHUTzp5t0G8EoxnzotHhhzjSAmsypySqZXaxl3oX8aYUkFn +fCdg+WBXo5pOiNfoWh/D5bnIXWGp52yoy+kCAwEAAaNhMF8wHQYDVR0lBBYwFAYI +KwYBBQUHAwIGCCsGAQUFBwMBMB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MB0GA1UdDgQWBBSxLHQE7gOEyfeSNc5uIO/G/rgjpzANBgkqhkiG9w0BAQsF +AAOCAQEAlGm5RwQ79xmLh3rj+5UViCSgsIuMcuhgIT4zogpo9S4uwXMqinrJhzRk +Oc2tb3y06XTAq1lMH2+58tqndAu8ni/UBz3OSghk2CTnZ1vxxXOd3CtQu4ypMq+k +qW0Umdrkk5TeAODNbrCy4c6vpICkQOljnRFWnDYu3aQ3JvaWZ/nObN7C72Lgpjfb +RfLXGmLsBCEIr028f9hpoeRCXoetUY2CiC2boAHR+cO6Jpvex4Jv5yYDpNKac52n +LLC8Cq5ozhOZNOSV6X9FpEca3rdhVUb0VgoNDCPZDdpO1PDJYCN9fUC8KUs+dsOh +SYGpliBaNsztoiJs1q5SkMQrMwmMmg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/certs/example.key b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/certs/example.key new file mode 100644 index 000000000..b21c2f08d --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/certs/example.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAu/4tXVpYiVwr2sjsi5FLxIK6yoGejIl5z8SsqIDaBt+Bn9hL +vctm53zjKnjkAipq7Nwjmr6GuSaxFnsIqoPOsb6Mdq9buMX7SVp60nBG3boBxBw6 +1APaKTY74OXVYcAWfi9QBDFv5fPpZ9JpA3RB8zvjPTpdQMOyvuDv41i0zOF/3xIl +Kgqd791UoncFzp7q2+GcJHA+2LHh6S0Sh0rpZ40djHAnrBJnxxEvN3GCN0P/g4Z8 +W6N2fzNs6LdJC3lmpVtwMdRPOnm3QbwSjGfOi0eGHONICazKnJKpldrGXehfxphS +QWd8J2D5YFejmk6I1+haH8PluchdYannbKjL6QIDAQABAoIBAEUBL7WsjAMfihls +1ycD1kPzmIzstz3u2H+jOZ1AbsdHE1WRF3w7RTKDbP8SEN+aolT/GTKb7OfZg/c0 +giHU7/Hed8C47XoNcgei5qKIA/svY6aQlidsoo+uEJykwIZ488itpTlkzCYkOfCa +E2HpMqwNt4OqAMDdFKdr+aIB1Zu+KPBxW23WD9wEWAbe5LA4YnRF9kT4YZ6y9mce +dGaIf39VtBlrGMmvoU0LE9B79nyuebGi0svW6QDarBqaDrnM/N3fXgL1kk/gVfan +/xs6EA4qPxA5G4h+enYrIlZL0CbSj60nYElo+Z5nRdBaRdCF/bpXOLyK/kXWLUM0 +f2HTK+ECgYEA5cVcpJtczxEaxoaEUbppsW1LCrTjJDGTKJ63G7/lwqCxJeCHN185 +nnckHOW2287e19bu9aUmKJgRq5s1rXnT+MnCl/hQnfaMrKOKtzE+t7/zsc9+LuAr +pwJrtZ9Dcnwrk8NOE0fPjW5XpDCSoEOo7JWZmGTVlpabOgNkjocfp+sCgYEA0XPt +ZPt3F0wyzgLYRhgnvp5CV8SzQmulsW+ytnL5eiAcNSXqni3wQHN3PGxLInEyQwBQ +/M8TQpUbqGMmahCK4ZxMAwXMrpF0mVB8jfoYMou1FSYPlUV+CvLjWcTkZB1Nirez +VFXdtfHP0mx4PbYK2qjB03u4pPHAN8kuayIf2nsCgYAbv/FHZAgabfto3Jggcr4P +Ep8MhPolxeL69egxbsSl89hRNcO+2T5ROBxhbRDfjSV2tduYSUDJiEwiCJW8BMmn +814QEopR+ZPVyc6X/1eOw5z/7YpUyPgcrHsrrTdtHTf6GY1VYMfdUeU9zCv5NRKy +uAKb2Bm/nSLUJ9K+L+2PzwKBgQDRQgT3UtTUjehkMitpPFDY/LxDe92simfsMjBW +X+Anx1TnNI6GolbZzYJe98LJElao4fQH38raRqZvQT/rz8MxTDoU+wJXljLryaHn +Jupt9W5hRrli5R7cSXYjBbc43p3N7WJY68CqOoDrNjubS/jkJJ4hcAY1pOHp2jFq +D5nLaQKBgAysU6O5kJ8yKxhbZflb42MqKCFBGrbRnbYx14PAEZOaRhzxehpppQmx +RLbn/z1Uh5Ms28ipxA+vnhyM3FcU5lKboaFyWJeuNslw0FxEcIai6hL6UkDznS4G +aqyzUjpG5Chg0x18xWYCbiGJwjZ9BWhtH+jojm856QHGzeQJWVoo +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/client_ca/example_ca.crt b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/client_ca/example_ca.crt new file mode 100644 index 000000000..b0fa2fa28 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/client_ca/example_ca.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDATCCAemgAwIBAgIJAMsPuHg4mqnaMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzMwNloYDzIxMjAwNTE3MTAzMzA2 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBAL72D2gOnJN9Zvo9KjruwM1EsFe3xZRJ0NvZ5bHd6+5jhlgCAhQ+ +AGb7ufEiYOi2JWHl2Bkq0iVrp+zv0RLdq0oVjX+OG5H2yWbnC7ifbNjir93LX0un +tIqv5CIbExDSBRkufxfV37yjXdrcMqYSbD2Kw3PfAbWs1Dego8fRz8QAp5+LCvW2 +BZZyYi6JzhCAUW1+8OQPyzOhB50eSJiS5xgVA7wkwmYeVUpHqU8sv4VzjM3bmUc7 +1mPLlnRVIqScrqYgQ9Ou21vZebOJ8+ckVL8O3XHhMZlssBbWiBFZZnaNWbzcEI90 +oiW4YAQ5t7gaXuCVvaiNvq2VZknarR6AxcsCAwEAAaNQME4wHQYDVR0OBBYEFIH+ +0G3eCswQHbN06kvI80M3tNH9MB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAJtHx8ibLp+CYDR1 +ZVeQ3qg4OeRL0q21+2EgBJq6zOUt/9SxThA80aXJ8CYH9dnCW+fOnpEk1xFWxtXc +FwSLsnqAdwOJaaQWoMzhyqjZV5x5G9+MW5FzGGOdes2md2Z+tAwMoV9TVtxZkbKy +mC2tDJdvgLgt9/YcbUcZPDbyZojdZ+UbATm+Lro9dhTXt91vsAgz5QA9e08rQVkF +pc9+ZQ5zxBsoblQ+ozPOWOdV4zJVx+wQsAnOG2qU0yVQAscGsTo4wnzFrAU54fO7 +Lh4cOrY0P1/o65yiSzwK7f0jwBeT/jEfMOrJ7pPo7doUov0iVj0SZyTM3HFa2Mzj +2zGTk0o= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/client_ca/example_ca.key b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/client_ca/example_ca.key new file mode 100644 index 000000000..4f7ad13b1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/client_ca/example_ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAvvYPaA6ck31m+j0qOu7AzUSwV7fFlEnQ29nlsd3r7mOGWAIC +FD4AZvu58SJg6LYlYeXYGSrSJWun7O/REt2rShWNf44bkfbJZucLuJ9s2OKv3ctf +S6e0iq/kIhsTENIFGS5/F9XfvKNd2twyphJsPYrDc98BtazUN6Cjx9HPxACnn4sK +9bYFlnJiLonOEIBRbX7w5A/LM6EHnR5ImJLnGBUDvCTCZh5VSkepTyy/hXOMzduZ +RzvWY8uWdFUipJyupiBD067bW9l5s4nz5yRUvw7dceExmWywFtaIEVlmdo1ZvNwQ +j3SiJbhgBDm3uBpe4JW9qI2+rZVmSdqtHoDFywIDAQABAoIBAD60bbqtkZycwQPK +seNIIudEduNW5PocgwiuNE6DoMVWyPZ9MlGTSm6GmjgkIc5IgV30K1GYTgkboLic +xvp675QUH7KS51q2vsubcq3dK9DMHxOlhFVDbHVd7HuGiGwtip8KNZGOGTnIKzmC +tN7zjbdnqWaTA+y0I7tgdGdY7fBd9Rzgaq+OlPq2u33HvWHvlG/7PfpZuXB5YLgd +m04l7LJ7ikhIjycg7j27v/4c6xCiH5jMJKsZ+nfsQ0kEEo9DkhcKInK+wHsMzKsH +Cy3AdlE0IRsbxRAoMumVs2g5u90m3zBPkRrNdZ2Ni7BesnhxbkIqvb4SfpxKyuhK +fADfZgECgYEA7SUIS2gII0TGjXh0h16d+eoLVOz0eVpgF3XXxmgtAPu10dqxVEC2 +j5FSBCgZhqZ3axVotP71c2mT+hF+Mqy4TLMfA/B9jKLXjZlPbg4EcAgI7tALskwz +Bk5BkX0k825bU9P0j+AlpLx6/ztHr2N9/cKZfqQVrO9t+FRusvAHkHsCgYEAziT8 +F30Ch2s6IJngCj5jH164iN2CoFXjqPNVgRj45gLE3zrf1R7u2JTeEhjWNLZ6IWZ3 +G/bT7eYm6x8u7LFlORdnWKsHlftGu0igRyvIGcxoHgjXlsLidBaEP+HlOLUtTumu +MfQJUozLcrOBIV6m9VhPnSDTeCg/tOqy68V2pvECgYAUYgd5e8KfTW0Hgd/6Nq67 +aVt5/DfzKkpyGcXnHtMnb3ssQ3DUfg9y/ZmgE9ZF1Y8UHC34yKVOOzfl2ZUQQ/o/ +VXIIA6a27NQ8Ln4+RmQpQPeLl0Q6GgSUuSs3lxsS9VxSMzilGS4DH9QulejOcW3F +3vEUioP2bkn0e0VcifcMewKBgAG1Pr13FLFIiye//qI3GB0nbMH9i9qGO6entHqo +WU+WkEkFNNuQMQxsV1axC/1N0b87GRuLNQBQmtvx2zKs2Zjaf8m1SQ/OECz3EhTk +4PiNwAMXsamXHcc2dIwO9BY/MgvoVcAmNHmRnxHpONWs8hcwTyCPKBFjy/tUwny/ +mxcRAoGAcNRxLZlyRqmQ6zGf41GK4ZIR9gix0L6Km49S1maGFmcbctOR2GcQN8Eo +f38rkrFBfBfuFSGzghJiXDvKORg9r3V/bzSKcXkprJra6hzn5vn+t7wurjzaJUlK +zUW5dl3SU2bC4MK+X7bwqf9jm9b7FXSt4p1xly8Uh/Mufwi7zOM= +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/server_cert_conf.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/server_cert_conf.data new file mode 100644 index 000000000..49f228531 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/server_cert_conf.data @@ -0,0 +1,12 @@ +{ + "Version": "init version", + "Config": { + "Default": "example.org", + "CertConf": { + "example.org": { + "ServerCertFile": "tls_conf/certs/example.crt", + "ServerKeyFile" : "tls_conf/certs/example.key" + } + } + } +} diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/session_ticket_key.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/session_ticket_key.data new file mode 100644 index 000000000..b3d2356b0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/session_ticket_key.data @@ -0,0 +1,4 @@ +{ + "Version": "init version", + "SessionTicketKey": "08a0d852ef494143af613ef32d3c39314758885f7108e9ab021d55f422a454f7c9cd5a53978f48fa1063eadcdc06878f" +} diff --git a/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/tls_rule_conf.data b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/tls_rule_conf.data new file mode 100644 index 000000000..66e7b5dbb --- /dev/null +++ b/tests/integration/implementation/scenario-SC01-route-table-lookup/testdata/tls_conf/tls_rule_conf.data @@ -0,0 +1,20 @@ +{ + "Version": "12", + "DefaultNextProtos": ["http/1.1"], + "Config": { + "example_product": { + "VipConf": [ + "10.199.4.14" + ], + "SniConf": ["example.org"], + "CertName": "example.org", + "NextProtos": [ + "h2;rate=100;isw=65535;mcs=200;level=0", + "http/1.1" + ], + "Grade": "C", + "ClientAuth": false, + "ClientCAName": "example_ca" + } + } +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/sc02_multi_api_key_test.go b/tests/integration/implementation/scenario-SC02-multi-api-key/sc02_multi_api_key_test.go new file mode 100644 index 000000000..264d036ba --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/sc02_multi_api_key_test.go @@ -0,0 +1,542 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package sc02 + +import ( + "bytes" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "testing" + "time" + + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" + "github.com/bfenetworks/bfe/tests/integration/common" +) + +const ( + apiHost = "multikey.example.org" + apiPath = "/v1/chat/completions" + apiKey = "ak_user_a" + keyA = "sk-key-a" + keyB = "sk-key-b" + keyC = "sk-key-c" + + clusterMultiKey = "cluster_multi_key" + clusterFallbackOK = "cluster_fallback_ok" +) + +var defaultBody = []byte(`{"model":"gpt-4"}`) + +// testEnv holds all resources for a single SC02 integration test. +type testEnv struct { + t *testing.T + processEnv *common.ProcessEnv + backends map[string]*common.MockBackend + bfePort int + stopBFE func() +} + +func newTestEnv(t *testing.T, aiConf *cluster_conf.AIConf) *testEnv { + e := &testEnv{ + t: t, + backends: make(map[string]*common.MockBackend), + } + + e.backends[clusterMultiKey] = common.NewMockBackend(clusterMultiKey, http.StatusOK, `{"ok":true}`) + e.backends[clusterFallbackOK] = common.NewMockBackend(clusterFallbackOK, http.StatusOK, `{"ok":true}`) + + e.processEnv = common.NewProcessEnv(t) + e.processEnv.Build() + + confDir := filepath.Join(e.processEnv.WorkDir(), "conf") + logDir := filepath.Join(e.processEnv.WorkDir(), "log") + + aiConfs := map[string]*cluster_conf.AIConf{} + if aiConf != nil { + aiConfs[clusterMultiKey] = aiConf + } + + builder := &common.BFEConfigBuilder{ + TemplateDir: "testdata", + TargetConfDir: confDir, + Backends: e.backends, + AIConfs: aiConfs, + } + if err := builder.Build(); err != nil { + t.Fatalf("build bfe config failed: %v", err) + } + + e.bfePort, _, e.stopBFE = e.processEnv.StartBFE(confDir, logDir) + return e +} + +func (e *testEnv) Close() { + if e.stopBFE != nil { + e.stopBFE() + } + for _, b := range e.backends { + b.Close() + } +} + +func (e *testEnv) logBFEException() { + data, err := os.ReadFile(filepath.Join(e.processEnv.WorkDir(), "log", "exception.log")) + if err == nil && len(data) > 0 { + e.t.Logf("bfe exception log:\n%s", string(data)) + } +} + +func (e *testEnv) sendRequest(body []byte) (*http.Response, string, error) { + url := fmt.Sprintf("http://127.0.0.1:%d%s", e.bfePort, apiPath) + req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body)) + if err != nil { + return nil, "", err + } + req.Host = apiHost + req.Header.Set("Authorization", "Bearer "+apiKey) + req.Header.Set("Content-Type", "application/json") + + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + respBody, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return resp, string(respBody), nil +} + +func defaultMultiKeyAIConf() *cluster_conf.AIConf { + return &cluster_conf.AIConf{ + Type: 0, + Keys: []cluster_conf.AIKey{ + {Name: "key-a", Key: keyA, Weight: 50}, + {Name: "key-b", Key: keyB, Weight: 30}, + {Name: "key-c", Key: keyC, Weight: 20}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 3, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + } +} + +func countAuthHeaders(headers []string) map[string]int { + counts := make(map[string]int) + for _, h := range headers { + counts[h]++ + } + return counts +} + +func generateBody(size int) []byte { + body := make([]byte, size) + for i := range body { + body[i] = byte('a' + i%26) + } + return body +} + +// wrapBody wraps a fixed prefix and suffix around generated filler content. +func wrapBody(prefix, suffix string, fillerSize int) []byte { + filler := generateBody(fillerSize) + return []byte(prefix + string(filler) + suffix) +} + +// TestTC01 verifies weighted random selection across multiple API keys. +func TestTC01_WeightedKeySelection(t *testing.T) { + aiConf := defaultMultiKeyAIConf() + aiConf.KeyPolicy.MaxRetries = 0 + e := newTestEnv(t, aiConf) + defer e.Close() + + for i := 0; i < 500; i++ { + resp, body, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + } + + if e.backends[clusterMultiKey].Hits() != 500 { + t.Fatalf("expected %d hits on %s, got %d", 500, clusterMultiKey, e.backends[clusterMultiKey].Hits()) + } + if e.backends[clusterFallbackOK].Hits() != 0 { + t.Fatalf("expected fallback not hit, got %d", e.backends[clusterFallbackOK].Hits()) + } + + counts := countAuthHeaders(e.backends[clusterMultiKey].AuthHeaders()) + if delta := abs(counts["Bearer "+keyA] - 250); delta > 50 { + t.Fatalf("key-a count %d deviates too far from 250", counts["Bearer "+keyA]) + } + if delta := abs(counts["Bearer "+keyB] - 150); delta > 40 { + t.Fatalf("key-b count %d deviates too far from 150", counts["Bearer "+keyB]) + } + if delta := abs(counts["Bearer "+keyC] - 100); delta > 35 { + t.Fatalf("key-c count %d deviates too far from 100", counts["Bearer "+keyC]) + } +} + +// TestTC02 verifies that a 429 response causes BFE to rotate to another key. +func TestTC02_429RotatesKey(t *testing.T) { + e := newTestEnv(t, defaultMultiKeyAIConf()) + defer e.Close() + + multiKey := e.backends[clusterMultiKey] + multiKey.ResponseFunc = func(r *http.Request, count int) (int, string) { + if r.Header.Get("Authorization") == "Bearer "+keyA { + return http.StatusTooManyRequests, `{"error":"rate limited"}` + } + return http.StatusOK, `{"ok":true}` + } + + // Send enough requests so that key-a is hit at least once and a different + // key eventually succeeds. + for i := 0; i < 100; i++ { + resp, body, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + } + + if e.backends[clusterFallbackOK].Hits() != 0 { + t.Fatalf("expected fallback not hit, got %d", e.backends[clusterFallbackOK].Hits()) + } + if multiKey.Hits() <= 100 { + t.Fatalf("expected more than %d hits due to retries, got %d", 100, multiKey.Hits()) + } + + counts := countAuthHeaders(multiKey.AuthHeaders()) + if counts["Bearer "+keyA] == 0 { + t.Fatalf("expected key-a to be used at least once") + } + if counts["Bearer "+keyB]+counts["Bearer "+keyC] == 0 { + t.Fatalf("expected at least one other key to be used") + } +} + +// TestTC03 verifies that 401/403 responses mark the key dead for the current +// aiClusterInvoke call, so subsequent attempts within the same request skip it. +func TestTC03_401And403MarkKeyDead(t *testing.T) { + aiConf := &cluster_conf.AIConf{ + Type: 0, + Keys: []cluster_conf.AIKey{ + {Name: "key-a", Key: keyA, Weight: 40}, + {Name: "key-b", Key: keyB, Weight: 40}, + {Name: "key-c", Key: keyC, Weight: 20}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 3, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + } + e := newTestEnv(t, aiConf) + defer e.Close() + + multiKey := e.backends[clusterMultiKey] + multiKey.ResponseFunc = func(r *http.Request, count int) (int, string) { + auth := r.Header.Get("Authorization") + if auth == "Bearer "+keyA { + return http.StatusUnauthorized, `{"error":"unauthorized"}` + } + if auth == "Bearer "+keyB { + return http.StatusForbidden, `{"error":"forbidden"}` + } + return http.StatusOK, `{"ok":true}` + } + + // Send enough requests so that key-a/key-b are hit at least once and key-c + // eventually succeeds within each request. + for i := 0; i < 100; i++ { + resp, body, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + } + + if e.backends[clusterFallbackOK].Hits() != 0 { + t.Fatalf("expected fallback not hit, got %d", e.backends[clusterFallbackOK].Hits()) + } + + counts := countAuthHeaders(multiKey.AuthHeaders()) + if counts["Bearer "+keyA] == 0 { + t.Fatalf("expected key-a to be used at least once") + } + if counts["Bearer "+keyB] == 0 { + t.Fatalf("expected key-b to be used at least once") + } + if counts["Bearer "+keyC] == 0 { + t.Fatalf("expected key-c to be used at least once") + } +} + +// TestTC04 verifies that 5xx responses trigger retry on the same key with backoff. +func TestTC04_5xxRetriesSameKey(t *testing.T) { + e := newTestEnv(t, defaultMultiKeyAIConf()) + defer e.Close() + + multiKey := e.backends[clusterMultiKey] + failCount := 0 + multiKey.ResponseFunc = func(r *http.Request, count int) (int, string) { + if failCount < 2 { + failCount++ + return http.StatusServiceUnavailable, `{"error":"unavailable"}` + } + return http.StatusOK, `{"ok":true}` + } + + start := time.Now() + resp, body, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + elapsed := time.Since(start) + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterFallbackOK].Hits() != 0 { + t.Fatalf("expected fallback not hit, got %d", e.backends[clusterFallbackOK].Hits()) + } + if multiKey.Hits() != 3 { + t.Fatalf("expected 3 attempts, got %d", multiKey.Hits()) + } + + authHeaders := multiKey.AuthHeaders() + firstAuth := authHeaders[0] + for _, h := range authHeaders { + if h != firstAuth { + t.Fatalf("expected all attempts to use the same key, got %s and %s", firstAuth, h) + } + } + + // Two retries with initial backoff 50ms should take at least 50ms even with + // jitter; allow a small margin. + if elapsed < 40*time.Millisecond { + t.Fatalf("expected retry backoff, elapsed %v", elapsed) + } +} + +// TestTC05 verifies that when all keys are exhausted by 429/401/403 errors, +// the final 4xx response triggers cluster-level fallback. +func TestTC05_AllKeysExhausted(t *testing.T) { + e := newTestEnv(t, defaultMultiKeyAIConf()) + defer e.Close() + + multiKey := e.backends[clusterMultiKey] + multiKey.ResponseFunc = func(r *http.Request, count int) (int, string) { + switch r.Header.Get("Authorization") { + case "Bearer " + keyA: + return http.StatusTooManyRequests, `{"error":"rate limited"}` + case "Bearer " + keyB: + return http.StatusUnauthorized, `{"error":"unauthorized"}` + default: + return http.StatusForbidden, `{"error":"forbidden"}` + } + } + + resp, body, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected status 200 from fallback, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterFallbackOK].Hits() != 1 { + t.Fatalf("expected fallback hit once, got %d", e.backends[clusterFallbackOK].Hits()) + } + // With MaxRetries=3, BFE may try up to 4 times (including a reset of the + // used_set when only 429 keys remain). + if multiKey.Hits() < 3 || multiKey.Hits() > 4 { + t.Fatalf("expected 3 or 4 attempts, got %d", multiKey.Hits()) + } +} + +// TestTC06 verifies that 5xx key-level retry exhaustion triggers cluster fallback. +func TestTC06_KeyExhaustionTriggersFallback(t *testing.T) { + aiConf := defaultMultiKeyAIConf() + aiConf.KeyPolicy.MaxRetries = 2 + e := newTestEnv(t, aiConf) + defer e.Close() + + e.backends[clusterMultiKey].ResponseFunc = func(r *http.Request, count int) (int, string) { + return http.StatusServiceUnavailable, `{"error":"unavailable"}` + } + + resp, body, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200 from fallback, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterMultiKey].Hits() > 3 { + t.Fatalf("expected at most 3 attempts on multi-key cluster, got %d", e.backends[clusterMultiKey].Hits()) + } + if e.backends[clusterFallbackOK].Hits() != 1 { + t.Fatalf("expected fallback hit once, got %d", e.backends[clusterFallbackOK].Hits()) + } +} + +// TestTC07 verifies that the request body is fully rewound across key rotations. +func TestTC07_BodyRewoundOnKeyRotation(t *testing.T) { + e := newTestEnv(t, defaultMultiKeyAIConf()) + defer e.Close() + + multiKey := e.backends[clusterMultiKey] + multiKey.ResponseFunc = func(r *http.Request, count int) (int, string) { + if r.Header.Get("Authorization") == "Bearer "+keyA { + return http.StatusTooManyRequests, `{"error":"rate limited"}` + } + return http.StatusOK, `{"ok":true}` + } + + // Build a ~100 KB body so any truncation would be detectable while keeping + // the test fast and within the default bytes_body limits. + body := wrapBody(`{"model":"gpt-4","content":"`, `"}`, 100*1024) + + // Send enough requests to trigger at least one rotation. + var rotated bool + var successes int + for i := 0; i < 100; i++ { + resp, _, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode == http.StatusOK { + successes++ + } + if multiKey.Hits() > i+1 { + rotated = true + } + } + if successes == 0 { + t.Fatalf("expected at least one successful request") + } + if !rotated { + t.Fatalf("expected at least one request to rotate keys") + } + + bodies := multiKey.RequestBodies() + for i, got := range bodies { + if !bytes.Equal(got, body) { + t.Fatalf("request body %d differs from original (len %d vs %d)", i, len(got), len(body)) + } + } +} + +// TestTC08 verifies that AIConf extended fields (Provider, ModelTable) are loaded +// and do not break forwarding or model mapping. +func TestTC08_AIConfExtendedFields(t *testing.T) { + aiConf := &cluster_conf.AIConf{ + Type: 0, + ModelMapping: &map[string]string{ + "gpt-4": "mapped-model", + }, + Provider: "mock-provider", + Keys: []cluster_conf.AIKey{ + {Name: "key-c", Key: keyC, Weight: 100}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 0, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + ModelTable: &cluster_conf.ModelTable{ + Currency: "RMB", + Models: []cluster_conf.ModelPrice{ + { + Provider: "mock-provider", + Model: "mapped-model", + BaseModel: "mapped-model", + Mode: "chat", + Capabilities: []string{"chat"}, + SupportedParameters: []string{"temperature"}, + Limits: map[string]interface{}{ + "context_window": 128000, + "max_input_tokens": 128000, + "max_output_tokens": 8192, + }, + Prices: map[string]float64{ + "input_cost_per_token": 0.000002, + "output_cost_per_token": 0.000008, + }, + }, + }, + }, + } + + e := newTestEnv(t, aiConf) + defer e.Close() + + resp, _, err := e.sendRequest(defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d", resp.StatusCode) + } + + if e.backends[clusterMultiKey].Hits() != 1 { + t.Fatalf("expected one hit on multi-key cluster, got %d", e.backends[clusterMultiKey].Hits()) + } + if e.backends[clusterFallbackOK].Hits() != 0 { + t.Fatalf("expected fallback not hit, got %d", e.backends[clusterFallbackOK].Hits()) + } + + authHeaders := e.backends[clusterMultiKey].AuthHeaders() + if len(authHeaders) != 1 || authHeaders[0] != "Bearer "+keyC { + t.Fatalf("expected key-c, got %v", authHeaders) + } + + models := e.backends[clusterMultiKey].Models() + if len(models) != 1 || models[0] != "mapped-model" { + t.Fatalf("expected model mapping to mapped-model, got %v", models) + } +} + +func abs(x int) int { + if x < 0 { + return -x + } + return x +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/bfe.conf b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/bfe.conf new file mode 100644 index 000000000..5a232930e --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/bfe.conf @@ -0,0 +1,36 @@ +[server] +httpPort = 18080 +httpsPort = 18443 +monitorPort = 18081 +httpAddr = "127.0.0.1" +httpsAddr = "127.0.0.1" +monitorAddr = "127.0.0.1" +MonitorEnabled = true +maxCpus = 1 + +TlsHandshakeTimeout = 30 +ClientReadTimeout = 5 +ClientWriteTimeout = 5 +KeepAliveEnabled = true +GracefulShutdownTimeout = 10 + +EnableAiGateway = true + +accessibleBodySize = 4194304 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +totalBodyBufferSize = 0 + +Modules = mod_ai_route + +hostRuleConf = server_data_conf/host_rule.data +routeRuleConf = server_data_conf/route_rule.data +vipRuleConf = server_data_conf/vip_rule.data + +clusterTableConf = cluster_conf/cluster_table.data +gslbConf = cluster_conf/gslb.data +clusterConf = cluster_conf/cluster_conf.data +NameConf = + +maxHeaderUriBytes = 8096 +maxHeaderBytes = 8096 diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/cluster_conf/gslb.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/cluster_conf/gslb.data new file mode 100644 index 000000000..cf29bda02 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/cluster_conf/gslb.data @@ -0,0 +1,14 @@ +{ + "clusters": { + "cluster_multi_key": { + "GSLB_BLACKHOLE": 0, + "sub_multi": 100 + }, + "cluster_fallback_ok": { + "GSLB_BLACKHOLE": 0, + "sub_fb": 100 + } + }, + "hostname": "gslb-test", + "ts": "20260720150000" +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/mod_ai_route/ai_route.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/mod_ai_route/ai_route.data new file mode 100644 index 000000000..47c357b52 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/mod_ai_route/ai_route.data @@ -0,0 +1,33 @@ +{ + "Version": "20260720150000", + "route_rules": { + "apikey_ak_user_a": { + "type": "apikey", + "owner": "ak_user_a", + "rules": [ + { + "name": "user_a-multikey", + "Cond": "req_host_in(\"multikey.example.org\")", + "targets": [ + { + "ClusterName": "cluster_multi_key", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_fallback_ok", + "Model": "" + } + ] + } + ] + } + }, + "ApikeyRouteTableBindings": { + "ak_user_a": [ + "apikey_ak_user_a" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/mod_ai_route/mod_ai_route.conf b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/mod_ai_route/mod_ai_route.conf new file mode 100644 index 000000000..f250013bb --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/mod_ai_route/mod_ai_route.conf @@ -0,0 +1,5 @@ +[basic] +RouteRulePath = mod_ai_route/ai_route.data + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/host_rule.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/host_rule.data new file mode 100644 index 000000000..82977fb58 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/host_rule.data @@ -0,0 +1,13 @@ +{ + "Version": "20260720150000", + "Hosts": { + "ai_product": [ + "multikey.example.org" + ] + }, + "HostTags": { + "ai_product": [ + "ai_product" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/route_rule.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/route_rule.data new file mode 100644 index 000000000..b7fc9c68e --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/route_rule.data @@ -0,0 +1,12 @@ +{ + "Version": "20260720150000", + "BasicRule": { + "ai_product": [ + { + "Hostname": ["*"], + "Path": ["*"], + "ClusterName": "cluster_fallback_ok" + } + ] + } +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/vip_rule.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/vip_rule.data new file mode 100644 index 000000000..6fe22f03a --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/server_data_conf/vip_rule.data @@ -0,0 +1,8 @@ +{ + "Version": "20260720150000", + "Vips": { + "ai_vip": [ + "127.0.0.1" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/bfe_i_ca.crt b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/bfe_i_ca.crt new file mode 100644 index 000000000..f1e78f73c --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/bfe_i_ca.crt @@ -0,0 +1,23 @@ +-----BEGIN CERTIFICATE----- +MIIDwDCCAqigAwIBAgIBCzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +NDAxNDZaFw0zNzA3MTIxNDAxNDZaMIGQMQswCQYDVQQGEwJjbjEQMA4GA1UECAwH +YmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsMC3lpbmdmZWkt +ZGV2MRgwFgYDVQQDDA95aW5nZmVpLWRldi1pY2ExKTAnBgkqhkiG9w0BCQEWGmxp +YW5nY2h1YW5AeWYtbmV0d29ya3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A +MIIBCgKCAQEA6eFuWgoknixrRO9NCX4jAKyLtcAOWVJqVN2yX7CxjZjLyPurjvTZ +W73NYUbrPAN4AB5gY6UAPzuiEOSopVVIZ0OschK0cJldu9vZ0mZObBOsovuFcLQq +dgNSJ5slJSk7tCgD2EnCB3GYPG4D+uIKYd0c49wzTWWv4bjDwpgnf0LQbFpy7GhN +7D59zFH4qgOK/IQ5vaTMGyvIvtWR5/1Gvc9MLpGopTgi0DiNLed4UwDYrod5kysl +q3UcB5puONHQISOVoD3uRxo7wdsmVsHUfW7YfAWkhi6ec8mx9fy8IyE6f7GlXnuV +ysNccwqyEotL5bOXPJCqwUCL1v+iKTMPWwIDAQABo1AwTjAdBgNVHQ4EFgQUCjvC +vDVr8QXh9/hMI5xBgNvoSFgwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW +1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAhIggK/6JK4N7+HZW +VoTemwRpilugZZyKrcVAHbiiwUfXVQVuI64vc+yHWMSFRD1mykHkKBFzxEoDabMl +ASBtUJNt4b4zEL9V7k295vmAOp2IdLUQxlgeKWqABm4DGX96tGR9nKQTcn1ZeAxx +NyQFV1aj+dnNcF1iFFNF6t0bRrOEZ/aRSiu1bWp3Dj1JYTjXbyyplh3Ktb8lv4lt +EmvCXjo/l4TgQC9233kcTkXcq1swppzkkXfhB0NVuf9DE3C2xAWX0b2FiIEqnAhl +Bx0Cn3RrX7PZ6qrdRL6oBKvGJV6DP8BlF4LXiuD1VN/waQFJha57KQ78ZYYIXxQg +WMjWgA== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/bfe_r_ca.crt b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/bfe_r_ca.crt new file mode 100644 index 000000000..2a6db1e49 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/bfe_r_ca.crt @@ -0,0 +1,22 @@ +-----BEGIN CERTIFICATE----- +MIIDkzCCAnugAwIBAgIBCjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +MzQ5NTVaFw0zNzA3MTIxMzQ5NTVaMGQxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxFzAVBgNVBAMMDnlpbmdmZWktZGV2LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOC +AQ8AMIIBCgKCAQEAvNA3HrsMjBcXrMIIhGVWsurIA1F9jxKeA7dh06H00Vt4inVV +SUvNFrTqgPRhLkAhGRMPxrjVRgJ5bbFqqXIuPIpUFBhUsWXIDH+oVXQl9jsxAXaG +gZ0lTO/uYR9qyrS1rj9nyNPwRf59Al/VlsQL71cNQ/T/agJ4PfvPfULTPLOsqclJ +hj0IgXmDj464dqcdG3ZdfXpfhNF6ab+8YjpwafTRmY+LoV8qjUwsYeJMcW4N8pxJ +8F2ktZj9J6uWepNGj+87ZeXg9XquzC62ASIFzPjoE1WN//Q518EqizxhuLGBDpK3 +6sEYUK4kHYUL4gZKFPlKTPXIl0ZsJIM5PsBMKwIDAQABo1AwTjAdBgNVHQ4EFgQU +DZloT8VhVbysD819oG/oqHdW1D0wHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/o +qHdW1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEALgB+IcN3UD4T ++4g8jaOyOiSUpUVdYqW+3go5DppqnvlGNq99N2cXKqssVPa/T9TikEcBEicFa8zU +bwlx6TEte+MkWfWdQxFR1EuI1FgKr3ps6ZBRr7MPpnYmI7K9aK372K9n7WrQhbmP +s7ult8bWB1/t6o3R7B9ChNkWT+7DPD4+FvB1GMJSGPno7cdnvDkevBOuC2DnQl3M ++ADFAge1Lo8wKBy6gYkNFd2BfarHGvRC5Qmmrme+RIpWZnvux1+lfXIInnfSTJRM +uAo/ePkoNsM3qQll6uEdhDxOMx8Pq94bCkM3DtI3ObuxWXjKCgUm/n9yetUvPj4U +iYhRUqHpUg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_bfe_dev.crt b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_bfe_dev.crt new file mode 100644 index 000000000..2164fb602 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_bfe_dev.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 18 (0x12) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Jan 31 02:55:32 2024 GMT + Not After : Jan 28 02:55:32 2034 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=bfe-dev/emailAddress=dev@example.org + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:b2:7f:c1:0c:cd:49:43:d9:99:78:15:4c:5a:52: + a9:a7:bf:d5:eb:92:71:43:e1:37:e5:29:1b:68:f4: + 6f:4c:ea:fa:a4:8a:7c:29:01:2a:fa:7c:81:5b:c8: + eb:a1:40:94:b7:2b:82:e2:00:08:36:84:f0:b7:d2: + 5a:1e:56:97:aa:36:ff:4d:07:49:2d:fe:25:3a:f9: + e9:f1:ad:4e:6e:21:97:a9:f9:a2:a5:ac:82:23:0a: + d2:e0:97:cd:2f:14:b1:f0:8a:a8:e6:c5:97:45:94: + f8:8e:3f:96:66:5b:0b:8c:7c:07:61:18:44:92:f7: + 23:5a:c2:4b:88:58:59:5d:ca:5c:0d:6e:dd:ff:18: + 59:65:df:95:99:e3:3a:36:48:1f:3f:3a:e6:ce:85: + f3:0b:04:5e:92:ed:6f:8e:74:92:e4:37:46:da:5f: + 17:62:9c:82:40:06:fb:29:f8:55:f2:ba:23:75:ca: + 64:c0:45:03:12:bd:f5:17:15:7e:47:d5:bd:30:f2: + 99:ca:6b:e3:07:b0:ae:44:89:1e:10:26:ea:75:df: + 6f:07:b6:47:76:54:47:4f:6c:f6:68:fe:a8:cf:22: + 20:73:e8:19:55:8a:fe:f5:78:e8:51:88:52:80:1f: + 79:d4:c5:ae:8f:d2:2b:f6:41:01:42:01:cf:98:c2: + c9:25 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + EE:E8:68:42:DF:B1:F0:EF:6F:47:51:BD:D4:94:60:1F:05:85:A1:03 + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Extended Key Usage: + TLS Web Client Authentication + Signature Algorithm: sha256WithRSAEncryption + a9:a6:26:8e:42:61:15:22:ee:fc:b5:e1:e4:6b:dd:ac:f5:15: + 11:39:10:9a:ca:6f:85:fd:cb:90:1c:b2:4f:fe:29:de:b0:e7: + 73:e2:f7:5e:63:8c:7f:c1:7e:75:2e:c9:9d:e9:c2:45:75:f3: + 27:ba:82:94:de:7f:6c:87:0c:5c:71:af:0f:14:00:68:35:f7: + 5a:4a:ff:f5:ef:35:dd:50:72:76:f0:6f:b6:7b:42:33:07:b4: + 24:44:0a:fd:9d:61:9e:44:e8:88:0f:02:76:c6:90:3f:9d:1b: + d8:3b:64:25:2a:a3:39:78:38:bd:20:89:4a:9c:bd:68:38:18: + 4c:cb:20:3a:9b:5b:5f:58:52:86:73:de:85:fe:d6:a1:c6:a7: + 86:b0:96:4b:fa:28:04:ad:5d:85:e8:a1:fc:ca:0f:3c:be:5c: + 90:7e:3e:84:ae:67:ee:9a:72:71:3c:b2:80:45:82:fc:7e:58: + 74:99:42:c5:c3:8a:4a:eb:e1:8b:d5:84:ce:25:aa:a1:75:79: + 94:66:ae:ee:df:30:15:0b:b5:c5:b1:2c:d5:0a:54:78:b6:2e: + 67:29:81:41:f6:16:49:31:96:e7:41:e1:99:6b:27:57:bb:7d: + 76:eb:e4:d5:59:aa:a2:5c:bd:1c:18:2a:fa:9d:28:1a:0b:b6: + bf:7d:58:1a +-----BEGIN CERTIFICATE----- +MIID7jCCAtagAwIBAgIBEjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yNDAxMzEw +MjU1MzJaFw0zNDAxMjgwMjU1MzJaMH0xCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxEDAOBgNVBAMMB2JmZS1kZXYxHjAcBgkqhkiG9w0BCQEWD2RldkBleGFtcGxl +Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ/wQzNSUPZmXgV +TFpSqae/1euScUPhN+UpG2j0b0zq+qSKfCkBKvp8gVvI66FAlLcrguIACDaE8LfS +Wh5Wl6o2/00HSS3+JTr56fGtTm4hl6n5oqWsgiMK0uCXzS8UsfCKqObFl0WU+I4/ +lmZbC4x8B2EYRJL3I1rCS4hYWV3KXA1u3f8YWWXflZnjOjZIHz865s6F8wsEXpLt +b450kuQ3RtpfF2KcgkAG+yn4VfK6I3XKZMBFAxK99RcVfkfVvTDymcpr4wewrkSJ +HhAm6nXfbwe2R3ZUR09s9mj+qM8iIHPoGVWK/vV46FGIUoAfedTFro/SK/ZBAUIB +z5jCySUCAwEAAaOBkTCBjjAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu +U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU7uhoQt+x8O9vR1G9 +1JRgHwWFoQMwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW1D0wEwYDVR0l +BAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKmmJo5CYRUi7vy14eRr +3az1FRE5EJrKb4X9y5Acsk/+Kd6w53Pi915jjH/BfnUuyZ3pwkV18ye6gpTef2yH +DFxxrw8UAGg191pK//XvNd1Qcnbwb7Z7QjMHtCRECv2dYZ5E6IgPAnbGkD+dG9g7 +ZCUqozl4OL0giUqcvWg4GEzLIDqbW19YUoZz3oX+1qHGp4awlkv6KAStXYXoofzK +Dzy+XJB+PoSuZ+6acnE8soBFgvx+WHSZQsXDikrr4YvVhM4lqqF1eZRmru7fMBUL +tcWxLNUKVHi2LmcpgUH2FkkxludB4ZlrJ1e7fXbr5NVZqqJcvRwYKvqdKBoLtr99 +WBo= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem new file mode 100644 index 000000000..764aea3ad --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAsn/BDM1JQ9mZeBVMWlKpp7/V65JxQ+E35SkbaPRvTOr6pIp8 +KQEq+nyBW8jroUCUtyuC4gAINoTwt9JaHlaXqjb/TQdJLf4lOvnp8a1ObiGXqfmi +payCIwrS4JfNLxSx8Iqo5sWXRZT4jj+WZlsLjHwHYRhEkvcjWsJLiFhZXcpcDW7d +/xhZZd+VmeM6NkgfPzrmzoXzCwReku1vjnSS5DdG2l8XYpyCQAb7KfhV8rojdcpk +wEUDEr31FxV+R9W9MPKZymvjB7CuRIkeECbqdd9vB7ZHdlRHT2z2aP6ozyIgc+gZ +VYr+9XjoUYhSgB951MWuj9Ir9kEBQgHPmMLJJQIDAQABAoIBAFAKHzupVb/58+o3 +yqv5wx94Uuk2GlnwxIqaezL94GaiO0/K1U/huS7m426P0rDU75qPBTpn/0bLJ9GV +nllaRNnLnYEh0juwaWtfovp+1ttlbseGK9uUVip2cQbKqvQAmKWe14vbcDCAU1Ad +zUgKbUxKVVjBdAZekVjiJNJ3o2L9WPhf/uQo7A1XAJh2DajlTbgvrDM73W+47QuU +X4OHU0FMio6bxupu3OWl1bMrnKhuC4qczZWf2nOpcVQa89rtopuP4ENLJuWkbeGk +YQpNilEclnAa/Noumt/j/6GKC1EEHFsH2CNRRIazcZrsFhkSKc4pn1Y/WI3vj8kZ ++RYnJsUCgYEA6gr0x8suwRTAmVpoPk4XyP1x+eInG7onV5RjgsUtgruYd9naxRHg +2p8PHcv32pDs51Fa+4RldyMd/jec1SscRF5/+VOP9qeoRnaDqUu+uASgEO3OBxbP +JcWovyxRHIQxbYCQtqIr9bdzXw55MBLZou/sBUVTAkrIyPVyjWqZlV8CgYEAwz7L +YyYN615TsrzZKURxMjj94Nmob/NldSLRXaR3Ax7/ABtEOA685cwQxq7ONdkJTMIA +uR8u2GHZSzGiWnehuF6Zp7Xs71a57eFbs3ueZvvEba4Dff7hl7Y4tTlwrKndKjvP +J/5a2Ol8siQcRWAXHOdzggEHMSZ/sB4hWswly/sCgYEAhLRBpyemEwTZUBrbELjm +86gBgFajJi2fMSGKaxOygnYsNYjpauSAQnX99D87Aks6iM6wb/zaK3tV/lc6LgSL +uph6p7yh3JGj8JAyh0PTmDPHLtIoCAz+18QDsqJGO40ZGaXUaDn8Aw9J85QZUxDd +Jm4zvalZL+uHfarukRDolLECgYBUiupS4nWAh3XCnZeDEQna72avaFBROZmjIRJ7 +c+28wj009JmTlH4jGzvgbG0KUBKA1Div8Fq+g5AtyS498jNqvDvYrSQNdwZHhR/K +Fis++KHTxFfqxOU2Zkcj4d1yRpNn6EIJVVBNQL0n/g7n03XupCIWFw/gLoV343QZ +9vAe5QKBgA8ml59z1w3eUooc0yGfLhihXqCmM3IU006bFbODA30fBU4QKrHO8+Yx +Xbz9bi/1QLagLG6FzYQAkOjBlEBt5XLayYvwSb8xvWsm5A3vzTAbMFDOsDPEmRoH +dWtQccJcygOuK+PZtoZnNoJciNO5c9dZWD3xtIiURmX/kVtWrf6N +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_san_example.org.crt b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_san_example.org.crt new file mode 100644 index 000000000..78f3892b4 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_san_example.org.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 15 (0xf) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Dec 7 09:57:42 2023 GMT + Not After : Dec 4 09:57:42 2033 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei, CN=dev-test + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:d6:47:b0:17:69:91:4c:d0:66:c9:25:2e:38:f2: + 84:27:e2:7c:38:cc:04:b9:0c:8e:3d:cc:ef:4b:c5: + 35:2b:c1:82:d8:41:fc:25:c8:24:f2:e0:25:aa:f9: + 76:3c:e2:b2:50:fb:2d:ec:4e:16:d4:c1:da:1e:e3: + 51:9f:9d:c9:72:f9:cb:cc:14:9c:c1:82:c9:76:1f: + 98:dd:0e:b9:8a:20:d6:ab:f2:a6:f9:2f:23:81:f3: + af:e4:47:0c:55:95:94:de:ed:f6:a5:24:ee:32:e7: + 6b:79:e1:42:6f:a4:07:b2:95:1d:f5:a9:8e:60:42: + 70:42:bd:e2:30:18:68:74:52:32:98:a9:81:da:d8: + c6:6f:5e:1d:ce:79:b6:f3:ec:4f:ed:7d:22:57:d2: + 14:d0:fb:f2:50:d4:80:3b:89:ed:77:fd:45:6c:e6: + 52:6b:0a:52:71:ac:59:c8:d5:25:f5:40:03:fb:51: + b0:11:a7:00:79:d9:d8:4f:00:43:96:68:44:29:41: + dc:d2:cc:91:c8:61:95:41:4d:0e:66:5a:b5:15:67: + 3e:8a:6f:29:df:1c:8a:6f:ee:9e:97:9c:9e:69:71: + d3:34:52:75:e9:ea:e7:51:77:23:98:46:ca:47:a2: + d3:d3:97:03:41:4b:e3:33:11:72:2d:af:bf:2b:3e: + b3:51 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + 5A:27:32:9D:E7:36:24:A3:C1:DC:2F:95:80:C5:CF:0C:85:E8:E6:AF + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Subject Alternative Name: + DNS:example.org, DNS:www.example.org, DNS:example.com, DNS:*.example.com, IP Address:127.0.0.1, IP Address:192.168.0.100 + Signature Algorithm: sha256WithRSAEncryption + 1e:e8:e8:8a:ad:a8:0e:fc:c9:82:00:a1:ab:30:3c:a5:b9:dc: + d6:fb:86:ad:30:52:7f:61:be:90:a6:b8:56:bb:f1:0b:e6:39: + 38:65:09:6b:da:83:f7:65:ff:c4:21:de:b4:9e:8b:bd:1e:1c: + d1:d5:94:b8:18:79:f2:d0:06:51:39:67:13:40:3b:73:5b:cb: + ea:de:c1:19:76:f8:7b:0f:15:51:61:49:fb:98:f7:ea:4f:fc: + c2:fb:a7:f4:3c:48:64:14:79:b5:78:5b:20:10:b5:7a:2d:4c: + 04:51:60:ec:20:10:19:26:5f:e2:fd:32:59:67:e9:3f:48:8d: + f5:52:12:01:81:2c:c0:e5:72:cd:7d:0a:eb:7a:05:df:a0:77: + b9:ba:9a:7d:d1:4b:6a:44:e4:2d:98:af:bd:77:2b:f5:ef:26: + 4b:75:b3:97:d0:3a:bc:07:21:ef:71:92:30:fe:a2:79:e5:56: + d7:7e:c2:f3:57:ab:d7:de:fc:97:ed:20:0c:9a:cb:c5:5d:00: + 3b:61:29:e8:00:d4:39:e0:f2:4e:a4:03:c2:12:52:ff:e7:78: + f9:f7:c0:12:dc:36:a4:05:a2:f0:6b:47:e2:21:3d:a2:e1:a1: + 91:c7:ac:8f:b8:ae:58:65:e0:2b:57:80:eb:77:2d:48:ef:e6: + fb:b9:e1:20 +-----BEGIN CERTIFICATE----- +MIIEBzCCAu+gAwIBAgIBDzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzEyMDcw +OTU3NDJaFw0zMzEyMDQwOTU3NDJaMFoxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEQMA4GA1UECwwHeWluZ2ZlaTER +MA8GA1UEAwwIZGV2LXRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB +AQDWR7AXaZFM0GbJJS448oQn4nw4zAS5DI49zO9LxTUrwYLYQfwlyCTy4CWq+XY8 +4rJQ+y3sThbUwdoe41Gfncly+cvMFJzBgsl2H5jdDrmKINar8qb5LyOB86/kRwxV +lZTe7falJO4y52t54UJvpAeylR31qY5gQnBCveIwGGh0UjKYqYHa2MZvXh3Oebbz +7E/tfSJX0hTQ+/JQ1IA7ie13/UVs5lJrClJxrFnI1SX1QAP7UbARpwB52dhPAEOW +aEQpQdzSzJHIYZVBTQ5mWrUVZz6KbynfHIpv7p6XnJ5pcdM0UnXp6udRdyOYRspH +otPTlwNBS+MzEXItr78rPrNRAgMBAAGjgc0wgcowCQYDVR0TBAIwADAsBglghkgB +hvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYE +FFonMp3nNiSjwdwvlYDFzwyF6OavMB8GA1UdIwQYMBaAFA2ZaE/FYVW8rA/NfaBv +6Kh3VtQ9ME8GA1UdEQRIMEaCC2V4YW1wbGUub3Jngg93d3cuZXhhbXBsZS5vcmeC +C2V4YW1wbGUuY29tgg0qLmV4YW1wbGUuY29thwR/AAABhwTAqABkMA0GCSqGSIb3 +DQEBCwUAA4IBAQAe6OiKragO/MmCAKGrMDyludzW+4atMFJ/Yb6QprhWu/EL5jk4 +ZQlr2oP3Zf/EId60nou9HhzR1ZS4GHny0AZROWcTQDtzW8vq3sEZdvh7DxVRYUn7 +mPfqT/zC+6f0PEhkFHm1eFsgELV6LUwEUWDsIBAZJl/i/TJZZ+k/SI31UhIBgSzA +5XLNfQrregXfoHe5upp90UtqROQtmK+9dyv17yZLdbOX0Dq8ByHvcZIw/qJ55VbX +fsLzV6vX3vyX7SAMmsvFXQA7YSnoANQ54PJOpAPCElL/53j598AS3DakBaLwa0fi +IT2i4aGRx6yPuK5YZeArV4Drdy1I7+b7ueEg +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem new file mode 100644 index 000000000..393a79825 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEA1kewF2mRTNBmySUuOPKEJ+J8OMwEuQyOPczvS8U1K8GC2EH8 +Jcgk8uAlqvl2POKyUPst7E4W1MHaHuNRn53JcvnLzBScwYLJdh+Y3Q65iiDWq/Km ++S8jgfOv5EcMVZWU3u32pSTuMudreeFCb6QHspUd9amOYEJwQr3iMBhodFIymKmB +2tjGb14dznm28+xP7X0iV9IU0PvyUNSAO4ntd/1FbOZSawpScaxZyNUl9UAD+1Gw +EacAednYTwBDlmhEKUHc0syRyGGVQU0OZlq1FWc+im8p3xyKb+6el5yeaXHTNFJ1 +6ernUXcjmEbKR6LT05cDQUvjMxFyLa+/Kz6zUQIDAQABAoIBAC4sYGuLGf49Ygix +9FXdHFEj4rSyccoWRIhYoq/nHOAC4NkMzvKtQBj95+ABxVK1XstIdMrYwN6zrva8 +8Re9/mzCGwIs5uJj9ll30Y7A34Y+MUP4E7baS4JzKlG8ZZIDm4K2MFHBtXpOl8A5 +pAE+jVIUA9Kt6LohVuNq21SVzdxSfNYC/+SLqSftkWa/ZsqdkiHM5Hl+fVedh516 +IaLNW5hSthGh5n8dHY5h/AKPjfoq77aYp5/CUtJTC9mYdZu1j/W/pBVTRfOnwLQd +SQ1Xmr7f6q9Vmz+HnajIbFg9hQ54blvtUJ7DnugWxfUcoxf7ue79fnYjOIUOkRWw +8Iid/mECgYEA+5s0p0j+gkNZN5QtVNStoT04+1DqA1O381gczeaiz6Njt/MT0y5W +OpCsILQ70CpEjWAV+f6PDJiesDMxdGV+v2TCqK8ml8GahEczBLnHoAkPWCVf2XOX +oNj/CkZ2kmWufHFR+kcQbeDt1vFFcYUa61hKDFyNjinMW79Qy+PQID0CgYEA2gWd +7thE05sqU7/1MntmVRONKoAgnJmHcfSpWwLyh6E4YX3iKDSgI/9RnAMF39KUUY/O +XFWyIwAM9soeXknVsV/SmCaPeaEDiLHqz98aUEfvdLYMnuR883GgoXc4JrLsLw4z +oSi9lbAZFn0ekJ5L+rSFrY4rz9QZgYZxsLjUXKUCgYAbkaUSU2g3w8Np2J2i9u7T +hQ7SUsphdPHqAxSc5xGd6MxLYqIgeKpQHnwN1VHcfFUonIer7d2kxrBUpDdeBqT9 +ub+ulgqHhFo29ko70VNzUKrSwL2g6Q6LPFutt4zUe7nDvvL5loHRWF0XOTafurL5 +aKIsepO0KRZQU0U6IgszDQKBgFs6ZHaP2mTtFY4L0a75Ab3xu20gRgUhHRLq/H6P +wipMpMnuodaPBr9pU53Dig65D8T9Nq1eUnbgy4vs0T5FCPz6iqWN5RVQ8aieQhIP +WfRj1Wfx0WAfXcWEM2G9ACr5TWj3OVVjNclP8X9+hW6gPky+gv03c0+4gZ+4QRRg +ksPdAoGBAKXVv8L5Qt8rmi+QpF/6DVSeMB6lEevf319UEtyZPiYSBn1JJ9H+3Ddc +TMriXgZrw+PoXNJTAguDeGgzmtye1vraP0cjt7aG6sQ4YtMTnnuja880vK8z4i3Q +HJAi5fI8NvlW92dQBAccgrzFIKpRx+6CHwtCkNrxL2vJmGGzp8BR +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/certs/example.crt b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/certs/example.crt new file mode 100644 index 000000000..931874885 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/certs/example.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDEjCCAfqgAwIBAgIJAIAdu56fLE7OMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzM0NFoYDzIxMjAwNTE3MTAzMzQ0 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBALv+LV1aWIlcK9rI7IuRS8SCusqBnoyJec/ErKiA2gbfgZ/YS73L +Zud84yp45AIqauzcI5q+hrkmsRZ7CKqDzrG+jHavW7jF+0laetJwRt26AcQcOtQD +2ik2O+Dl1WHAFn4vUAQxb+Xz6WfSaQN0QfM74z06XUDDsr7g7+NYtMzhf98SJSoK +ne/dVKJ3Bc6e6tvhnCRwPtix4ektEodK6WeNHYxwJ6wSZ8cRLzdxgjdD/4OGfFuj +dn8zbOi3SQt5ZqVbcDHUTzp5t0G8EoxnzotHhhzjSAmsypySqZXaxl3oX8aYUkFn +fCdg+WBXo5pOiNfoWh/D5bnIXWGp52yoy+kCAwEAAaNhMF8wHQYDVR0lBBYwFAYI +KwYBBQUHAwIGCCsGAQUFBwMBMB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MB0GA1UdDgQWBBSxLHQE7gOEyfeSNc5uIO/G/rgjpzANBgkqhkiG9w0BAQsF +AAOCAQEAlGm5RwQ79xmLh3rj+5UViCSgsIuMcuhgIT4zogpo9S4uwXMqinrJhzRk +Oc2tb3y06XTAq1lMH2+58tqndAu8ni/UBz3OSghk2CTnZ1vxxXOd3CtQu4ypMq+k +qW0Umdrkk5TeAODNbrCy4c6vpICkQOljnRFWnDYu3aQ3JvaWZ/nObN7C72Lgpjfb +RfLXGmLsBCEIr028f9hpoeRCXoetUY2CiC2boAHR+cO6Jpvex4Jv5yYDpNKac52n +LLC8Cq5ozhOZNOSV6X9FpEca3rdhVUb0VgoNDCPZDdpO1PDJYCN9fUC8KUs+dsOh +SYGpliBaNsztoiJs1q5SkMQrMwmMmg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/certs/example.key b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/certs/example.key new file mode 100644 index 000000000..b21c2f08d --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/certs/example.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAu/4tXVpYiVwr2sjsi5FLxIK6yoGejIl5z8SsqIDaBt+Bn9hL +vctm53zjKnjkAipq7Nwjmr6GuSaxFnsIqoPOsb6Mdq9buMX7SVp60nBG3boBxBw6 +1APaKTY74OXVYcAWfi9QBDFv5fPpZ9JpA3RB8zvjPTpdQMOyvuDv41i0zOF/3xIl +Kgqd791UoncFzp7q2+GcJHA+2LHh6S0Sh0rpZ40djHAnrBJnxxEvN3GCN0P/g4Z8 +W6N2fzNs6LdJC3lmpVtwMdRPOnm3QbwSjGfOi0eGHONICazKnJKpldrGXehfxphS +QWd8J2D5YFejmk6I1+haH8PluchdYannbKjL6QIDAQABAoIBAEUBL7WsjAMfihls +1ycD1kPzmIzstz3u2H+jOZ1AbsdHE1WRF3w7RTKDbP8SEN+aolT/GTKb7OfZg/c0 +giHU7/Hed8C47XoNcgei5qKIA/svY6aQlidsoo+uEJykwIZ488itpTlkzCYkOfCa +E2HpMqwNt4OqAMDdFKdr+aIB1Zu+KPBxW23WD9wEWAbe5LA4YnRF9kT4YZ6y9mce +dGaIf39VtBlrGMmvoU0LE9B79nyuebGi0svW6QDarBqaDrnM/N3fXgL1kk/gVfan +/xs6EA4qPxA5G4h+enYrIlZL0CbSj60nYElo+Z5nRdBaRdCF/bpXOLyK/kXWLUM0 +f2HTK+ECgYEA5cVcpJtczxEaxoaEUbppsW1LCrTjJDGTKJ63G7/lwqCxJeCHN185 +nnckHOW2287e19bu9aUmKJgRq5s1rXnT+MnCl/hQnfaMrKOKtzE+t7/zsc9+LuAr +pwJrtZ9Dcnwrk8NOE0fPjW5XpDCSoEOo7JWZmGTVlpabOgNkjocfp+sCgYEA0XPt +ZPt3F0wyzgLYRhgnvp5CV8SzQmulsW+ytnL5eiAcNSXqni3wQHN3PGxLInEyQwBQ +/M8TQpUbqGMmahCK4ZxMAwXMrpF0mVB8jfoYMou1FSYPlUV+CvLjWcTkZB1Nirez +VFXdtfHP0mx4PbYK2qjB03u4pPHAN8kuayIf2nsCgYAbv/FHZAgabfto3Jggcr4P +Ep8MhPolxeL69egxbsSl89hRNcO+2T5ROBxhbRDfjSV2tduYSUDJiEwiCJW8BMmn +814QEopR+ZPVyc6X/1eOw5z/7YpUyPgcrHsrrTdtHTf6GY1VYMfdUeU9zCv5NRKy +uAKb2Bm/nSLUJ9K+L+2PzwKBgQDRQgT3UtTUjehkMitpPFDY/LxDe92simfsMjBW +X+Anx1TnNI6GolbZzYJe98LJElao4fQH38raRqZvQT/rz8MxTDoU+wJXljLryaHn +Jupt9W5hRrli5R7cSXYjBbc43p3N7WJY68CqOoDrNjubS/jkJJ4hcAY1pOHp2jFq +D5nLaQKBgAysU6O5kJ8yKxhbZflb42MqKCFBGrbRnbYx14PAEZOaRhzxehpppQmx +RLbn/z1Uh5Ms28ipxA+vnhyM3FcU5lKboaFyWJeuNslw0FxEcIai6hL6UkDznS4G +aqyzUjpG5Chg0x18xWYCbiGJwjZ9BWhtH+jojm856QHGzeQJWVoo +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/client_ca/example_ca.crt b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/client_ca/example_ca.crt new file mode 100644 index 000000000..b0fa2fa28 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/client_ca/example_ca.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDATCCAemgAwIBAgIJAMsPuHg4mqnaMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzMwNloYDzIxMjAwNTE3MTAzMzA2 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBAL72D2gOnJN9Zvo9KjruwM1EsFe3xZRJ0NvZ5bHd6+5jhlgCAhQ+ +AGb7ufEiYOi2JWHl2Bkq0iVrp+zv0RLdq0oVjX+OG5H2yWbnC7ifbNjir93LX0un +tIqv5CIbExDSBRkufxfV37yjXdrcMqYSbD2Kw3PfAbWs1Dego8fRz8QAp5+LCvW2 +BZZyYi6JzhCAUW1+8OQPyzOhB50eSJiS5xgVA7wkwmYeVUpHqU8sv4VzjM3bmUc7 +1mPLlnRVIqScrqYgQ9Ou21vZebOJ8+ckVL8O3XHhMZlssBbWiBFZZnaNWbzcEI90 +oiW4YAQ5t7gaXuCVvaiNvq2VZknarR6AxcsCAwEAAaNQME4wHQYDVR0OBBYEFIH+ +0G3eCswQHbN06kvI80M3tNH9MB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAJtHx8ibLp+CYDR1 +ZVeQ3qg4OeRL0q21+2EgBJq6zOUt/9SxThA80aXJ8CYH9dnCW+fOnpEk1xFWxtXc +FwSLsnqAdwOJaaQWoMzhyqjZV5x5G9+MW5FzGGOdes2md2Z+tAwMoV9TVtxZkbKy +mC2tDJdvgLgt9/YcbUcZPDbyZojdZ+UbATm+Lro9dhTXt91vsAgz5QA9e08rQVkF +pc9+ZQ5zxBsoblQ+ozPOWOdV4zJVx+wQsAnOG2qU0yVQAscGsTo4wnzFrAU54fO7 +Lh4cOrY0P1/o65yiSzwK7f0jwBeT/jEfMOrJ7pPo7doUov0iVj0SZyTM3HFa2Mzj +2zGTk0o= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/client_ca/example_ca.key b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/client_ca/example_ca.key new file mode 100644 index 000000000..4f7ad13b1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/client_ca/example_ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAvvYPaA6ck31m+j0qOu7AzUSwV7fFlEnQ29nlsd3r7mOGWAIC +FD4AZvu58SJg6LYlYeXYGSrSJWun7O/REt2rShWNf44bkfbJZucLuJ9s2OKv3ctf +S6e0iq/kIhsTENIFGS5/F9XfvKNd2twyphJsPYrDc98BtazUN6Cjx9HPxACnn4sK +9bYFlnJiLonOEIBRbX7w5A/LM6EHnR5ImJLnGBUDvCTCZh5VSkepTyy/hXOMzduZ +RzvWY8uWdFUipJyupiBD067bW9l5s4nz5yRUvw7dceExmWywFtaIEVlmdo1ZvNwQ +j3SiJbhgBDm3uBpe4JW9qI2+rZVmSdqtHoDFywIDAQABAoIBAD60bbqtkZycwQPK +seNIIudEduNW5PocgwiuNE6DoMVWyPZ9MlGTSm6GmjgkIc5IgV30K1GYTgkboLic +xvp675QUH7KS51q2vsubcq3dK9DMHxOlhFVDbHVd7HuGiGwtip8KNZGOGTnIKzmC +tN7zjbdnqWaTA+y0I7tgdGdY7fBd9Rzgaq+OlPq2u33HvWHvlG/7PfpZuXB5YLgd +m04l7LJ7ikhIjycg7j27v/4c6xCiH5jMJKsZ+nfsQ0kEEo9DkhcKInK+wHsMzKsH +Cy3AdlE0IRsbxRAoMumVs2g5u90m3zBPkRrNdZ2Ni7BesnhxbkIqvb4SfpxKyuhK +fADfZgECgYEA7SUIS2gII0TGjXh0h16d+eoLVOz0eVpgF3XXxmgtAPu10dqxVEC2 +j5FSBCgZhqZ3axVotP71c2mT+hF+Mqy4TLMfA/B9jKLXjZlPbg4EcAgI7tALskwz +Bk5BkX0k825bU9P0j+AlpLx6/ztHr2N9/cKZfqQVrO9t+FRusvAHkHsCgYEAziT8 +F30Ch2s6IJngCj5jH164iN2CoFXjqPNVgRj45gLE3zrf1R7u2JTeEhjWNLZ6IWZ3 +G/bT7eYm6x8u7LFlORdnWKsHlftGu0igRyvIGcxoHgjXlsLidBaEP+HlOLUtTumu +MfQJUozLcrOBIV6m9VhPnSDTeCg/tOqy68V2pvECgYAUYgd5e8KfTW0Hgd/6Nq67 +aVt5/DfzKkpyGcXnHtMnb3ssQ3DUfg9y/ZmgE9ZF1Y8UHC34yKVOOzfl2ZUQQ/o/ +VXIIA6a27NQ8Ln4+RmQpQPeLl0Q6GgSUuSs3lxsS9VxSMzilGS4DH9QulejOcW3F +3vEUioP2bkn0e0VcifcMewKBgAG1Pr13FLFIiye//qI3GB0nbMH9i9qGO6entHqo +WU+WkEkFNNuQMQxsV1axC/1N0b87GRuLNQBQmtvx2zKs2Zjaf8m1SQ/OECz3EhTk +4PiNwAMXsamXHcc2dIwO9BY/MgvoVcAmNHmRnxHpONWs8hcwTyCPKBFjy/tUwny/ +mxcRAoGAcNRxLZlyRqmQ6zGf41GK4ZIR9gix0L6Km49S1maGFmcbctOR2GcQN8Eo +f38rkrFBfBfuFSGzghJiXDvKORg9r3V/bzSKcXkprJra6hzn5vn+t7wurjzaJUlK +zUW5dl3SU2bC4MK+X7bwqf9jm9b7FXSt4p1xly8Uh/Mufwi7zOM= +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/server_cert_conf.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/server_cert_conf.data new file mode 100644 index 000000000..49f228531 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/server_cert_conf.data @@ -0,0 +1,12 @@ +{ + "Version": "init version", + "Config": { + "Default": "example.org", + "CertConf": { + "example.org": { + "ServerCertFile": "tls_conf/certs/example.crt", + "ServerKeyFile" : "tls_conf/certs/example.key" + } + } + } +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/session_ticket_key.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/session_ticket_key.data new file mode 100644 index 000000000..b3d2356b0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/session_ticket_key.data @@ -0,0 +1,4 @@ +{ + "Version": "init version", + "SessionTicketKey": "08a0d852ef494143af613ef32d3c39314758885f7108e9ab021d55f422a454f7c9cd5a53978f48fa1063eadcdc06878f" +} diff --git a/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/tls_rule_conf.data b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/tls_rule_conf.data new file mode 100644 index 000000000..66e7b5dbb --- /dev/null +++ b/tests/integration/implementation/scenario-SC02-multi-api-key/testdata/tls_conf/tls_rule_conf.data @@ -0,0 +1,20 @@ +{ + "Version": "12", + "DefaultNextProtos": ["http/1.1"], + "Config": { + "example_product": { + "VipConf": [ + "10.199.4.14" + ], + "SniConf": ["example.org"], + "CertName": "example.org", + "NextProtos": [ + "h2;rate=100;isw=65535;mcs=200;level=0", + "http/1.1" + ], + "Grade": "C", + "ClientAuth": false, + "ClientCAName": "example_ca" + } + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/sc03_rmb_quota_test.go b/tests/integration/implementation/scenario-SC03-rmb-quota/sc03_rmb_quota_test.go new file mode 100644 index 000000000..92e061487 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/sc03_rmb_quota_test.go @@ -0,0 +1,509 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package sc03 + +import ( + "bytes" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" + "github.com/bfenetworks/bfe/tests/integration/common" +) + +const ( + apiHost = "rmb.example.org" + apiPath = "/v1/chat/completions" + apiKey = "ak_user_a" + apiKeyId = "user_a_key_id" + + clusterRMB = "cluster_rmb" + clusterNoTable = "cluster_no_table" + clusterFallbackRMB = "cluster_fallback_rmb" + + planRMB = "plan_rmb" + planToken = "plan_token" + + redisKeyRMB = "quota:plan_rmb" + redisKeyToken = "quota:plan_token" +) + +var defaultBody = []byte(`{"model":"deepseek-chat"}`) +var modelMappingBody = []byte(`{"model":"gpt-4"}`) +var streamBody = []byte(`{"model":"deepseek-chat","stream":true}`) + +var usageResponse = `{"usage":{"prompt_tokens":100,"completion_tokens":50,"total_tokens":150}}` + +// SSE format: final chunk contains usage. The trailing blank line is required. +var streamUsageResponse = "data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\n" + + "data: {\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}\n\n" + + "data: {\"usage\":{\"prompt_tokens\":100,\"completion_tokens\":50,\"total_tokens\":150}}\n\n" + +// testEnv holds all resources for a single SC03 integration test. +type testEnv struct { + t *testing.T + processEnv *common.ProcessEnv + backends map[string]*common.MockBackend + redis *common.RedisServer + bfePort int + stopBFE func() +} + +func newTestEnv(t *testing.T, aiConfs map[string]*cluster_conf.AIConf, quotaPlans []common.QuotaPlan) *testEnv { + e := &testEnv{ + t: t, + backends: make(map[string]*common.MockBackend), + } + + e.backends[clusterRMB] = common.NewMockBackend(clusterRMB, http.StatusOK, usageResponse) + e.backends[clusterNoTable] = common.NewMockBackend(clusterNoTable, http.StatusOK, usageResponse) + e.backends[clusterFallbackRMB] = common.NewMockBackend(clusterFallbackRMB, http.StatusOK, usageResponse) + + e.redis = common.NewRedisServer(t) + + e.processEnv = common.NewProcessEnv(t) + e.processEnv.Build() + + confDir := filepath.Join(e.processEnv.WorkDir(), "conf") + logDir := filepath.Join(e.processEnv.WorkDir(), "log") + + tokenRule := &common.TokenRuleData{ + Version: "1.0", + QuotaPlans: map[string][]common.QuotaPlan{ + "ai_product": quotaPlans, + }, + Tokens: map[string]map[string]common.TokenFile{ + "ai_product": { + apiKey: { + Key: apiKey, + KeyId: apiKeyId, + Enabled: 1, + Status: 1, + UpdateTime: 0, + ExpiredTime: -1, + UnlimitedQuota: false, + QuotaPlans: planIDs(quotaPlans), + }, + }, + }, + Config: map[string][]common.TokenRule{ + "ai_product": { + { + Cond: "default_t()", + Action: common.ActionFile{Cmd: "CHECK_TOKEN"}, + }, + }, + }, + } + + builder := &common.BFEConfigBuilder{ + TemplateDir: "testdata", + TargetConfDir: confDir, + Backends: e.backends, + AIConfs: aiConfs, + RedisAddr: e.redis.Addr(), + TokenRuleData: tokenRule, + } + if err := builder.Build(); err != nil { + t.Fatalf("build bfe config failed: %v", err) + } + + e.bfePort, _, e.stopBFE = e.processEnv.StartBFE(confDir, logDir) + return e +} + +func planIDs(plans []common.QuotaPlan) []string { + ids := make([]string, len(plans)) + for i, p := range plans { + ids[i] = p.Id + } + return ids +} + +func (e *testEnv) Close() { + if e.stopBFE != nil { + e.stopBFE() + } + for _, b := range e.backends { + b.Close() + } + if e.redis != nil { + e.redis.Close() + } +} + +func (e *testEnv) logBFEException() { + data, err := os.ReadFile(filepath.Join(e.processEnv.WorkDir(), "log", "exception.log")) + if err == nil && len(data) > 0 { + e.t.Logf("bfe exception log:\n%s", string(data)) + } +} + +func (e *testEnv) logBFEAccess() { + data, err := os.ReadFile(filepath.Join(e.processEnv.WorkDir(), "log", "access.log")) + if err == nil && len(data) > 0 { + e.t.Logf("bfe access log:\n%s", string(data)) + } +} + +func (e *testEnv) sendRequest(host string, body []byte) (*http.Response, string, error) { + url := fmt.Sprintf("http://127.0.0.1:%d%s", e.bfePort, apiPath) + req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body)) + if err != nil { + return nil, "", err + } + req.Host = host + req.Header.Set("Authorization", "Bearer "+apiKey) + req.Header.Set("Content-Type", "application/json") + + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + respBody, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return resp, string(respBody), nil +} + +func defaultRMBAIConf() *cluster_conf.AIConf { + return &cluster_conf.AIConf{ + Type: 0, + ModelMapping: &map[string]string{ + "gpt-4": "deepseek-chat", + }, + Provider: "mock-provider", + Keys: []cluster_conf.AIKey{ + {Name: "key-primary", Key: "sk-primary", Weight: 100}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 0, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + ModelTable: &cluster_conf.ModelTable{ + Currency: "RMB", + Models: []cluster_conf.ModelPrice{ + { + Provider: "mock-provider", + Model: "deepseek-chat", + BaseModel: "deepseek-chat", + Mode: "chat", + Capabilities: []string{"chat"}, + SupportedParameters: []string{"temperature", "max_tokens"}, + Limits: map[string]interface{}{ + "context_window": 128000, + }, + Prices: map[string]float64{ + "input_cost_per_token": 0.000001, + "output_cost_per_token": 0.000002, + }, + }, + }, + }, + } +} + +func fallbackRMBAIConf() *cluster_conf.AIConf { + conf := defaultRMBAIConf() + conf.ModelTable.Models[0].Prices = map[string]float64{ + "input_cost_per_token": 0.000003, + "output_cost_per_token": 0.000004, + } + return conf +} + +func noTableAIConf() *cluster_conf.AIConf { + return &cluster_conf.AIConf{ + Type: 0, + Keys: []cluster_conf.AIKey{ + {Name: "key-primary", Key: "sk-primary", Weight: 100}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 0, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + } +} + +func rmbQuotaPlan(quota int64) common.QuotaPlan { + return common.QuotaPlan{ + Id: planRMB, + Unlimited: false, + PassNoQuota: false, + RedisKey: redisKeyRMB, + CreateTime: 0, + ExpiredTime: -1, + Quota: quota, + ResetMode: 0, + Unit: "RMB", + } +} + +func tokenQuotaPlan(quota int64) common.QuotaPlan { + return common.QuotaPlan{ + Id: planToken, + Unlimited: false, + PassNoQuota: false, + RedisKey: redisKeyToken, + CreateTime: 0, + ExpiredTime: -1, + Quota: quota, + ResetMode: 0, + Unit: "total_token", + } +} + +// TestTC01 verifies RMB quota deduction after a successful request. +func TestTC01_RMBQuotaDeduction(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + + // wait for async redis deduction + time.Sleep(500 * time.Millisecond) + remaining := e.redis.GetQuota(redisKeyRMB) + want := int64(10000000000 - (100*100 + 50*200)) + if remaining != want { + t.Fatalf("remaining quota = %d, want %d", remaining, want) + } +} + +// TestTC02 verifies that a request is rejected when RMB quota is exhausted. +func TestTC02_RMBQuotaExhausted(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(0)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 0) + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusTooManyRequests { + e.logBFEException() + t.Fatalf("expected status 429, got %d, body: %s", resp.StatusCode, body) + } + if !strings.Contains(body, "quota") { + t.Fatalf("expected quota error in body, got: %s", body) + } + if e.backends[clusterRMB].Hits() != 0 { + t.Fatalf("expected no backend hit, got %d", e.backends[clusterRMB].Hits()) + } +} + +// TestTC03 verifies billing by the mapped model when ModelMapping is used. +func TestTC03_ModelMappingBilling(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + resp, body, err := e.sendRequest(apiHost, modelMappingBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + models := e.backends[clusterRMB].Models() + if len(models) != 1 || models[0] != "deepseek-chat" { + t.Fatalf("expected backend model deepseek-chat, got %v", models) + } + + time.Sleep(200 * time.Millisecond) + remaining := e.redis.GetQuota(redisKeyRMB) + want := int64(10000000000 - (100*100 + 50*200)) + if remaining != want { + t.Fatalf("remaining quota = %d, want %d", remaining, want) + } +} + +// TestTC04 verifies that token and RMB quota plans are deducted together. +func TestTC04_TokenAndRMBQuotaCoexist(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000), tokenQuotaPlan(1000)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + e.redis.SetQuota(redisKeyToken, 1000) + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + time.Sleep(200 * time.Millisecond) + rmbRemaining := e.redis.GetQuota(redisKeyRMB) + rmbWant := int64(10000000000 - (100*100 + 50*200)) + if rmbRemaining != rmbWant { + t.Fatalf("RMB remaining = %d, want %d", rmbRemaining, rmbWant) + } + tokenRemaining := e.redis.GetQuota(redisKeyToken) + tokenWant := int64(1000 - 150) + if tokenRemaining != tokenWant { + t.Fatalf("token remaining = %d, want %d", tokenRemaining, tokenWant) + } +} + +// TestTC05 verifies zero-cost handling when ModelTable is missing. +func TestTC05_NoModelTableZeroCost(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterNoTable: noTableAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + resp, body, err := e.sendRequest("notable.example.org", defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterNoTable].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterNoTable, e.backends[clusterNoTable].Hits()) + } + + time.Sleep(200 * time.Millisecond) + remaining := e.redis.GetQuota(redisKeyRMB) + if remaining != int64(10000000000) { + t.Fatalf("remaining quota = %d, want unchanged 10000000000", remaining) + } +} + +// TestTC06 verifies billing by the final cluster after fallback. +func TestTC06_FallbackBilling(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + clusterFallbackRMB: fallbackRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + e.backends[clusterRMB].ResponseFunc = func(r *http.Request, count int) (int, string) { + return http.StatusBadGateway, "" + } + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + if e.backends[clusterFallbackRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterFallbackRMB, e.backends[clusterFallbackRMB].Hits()) + } + + time.Sleep(200 * time.Millisecond) + remaining := e.redis.GetQuota(redisKeyRMB) + want := int64(10000000000 - (100*300 + 50*400)) + if remaining != want { + t.Fatalf("remaining quota = %d, want %d", remaining, want) + } +} + +// TestTC07 verifies RMB quota deduction for streaming (SSE) responses. +// This is the regression test for https://github.com/bfenetworks/bfe/issues/1316. +func TestTC07_RMBQuotaDeduction_Streaming(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + // Configure backend to return SSE stream with usage in the final chunk. + e.backends[clusterRMB].ResponseHeaders = map[string]string{"Content-Type": "text/event-stream"} + e.backends[clusterRMB].Body = streamUsageResponse + + resp, body, err := e.sendRequest(apiHost, streamBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + + // Wait for async redis deduction after response finishes. + time.Sleep(500 * time.Millisecond) + remaining := e.redis.GetQuota(redisKeyRMB) + want := int64(10000000000 - (100*100 + 50*200)) + if remaining != want { + e.logBFEException() + e.logBFEAccess() + t.Fatalf("remaining quota = %d, want %d, response body: %s", remaining, want, body) + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/bfe.conf b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/bfe.conf new file mode 100644 index 000000000..83af726b1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/bfe.conf @@ -0,0 +1,38 @@ +[server] +httpPort = 18080 +httpsPort = 18443 +monitorPort = 18081 +httpAddr = "127.0.0.1" +httpsAddr = "127.0.0.1" +monitorAddr = "127.0.0.1" +MonitorEnabled = true +maxCpus = 1 + +TlsHandshakeTimeout = 30 +ClientReadTimeout = 5 +ClientWriteTimeout = 5 +KeepAliveEnabled = true +GracefulShutdownTimeout = 10 + +EnableAiGateway = true + +accessibleBodySize = 4194304 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +totalBodyBufferSize = 0 + +Modules = mod_ai_route +Modules = mod_ai_token_auth +Modules = mod_body_process + +hostRuleConf = server_data_conf/host_rule.data +routeRuleConf = server_data_conf/route_rule.data +vipRuleConf = server_data_conf/vip_rule.data + +clusterTableConf = cluster_conf/cluster_table.data +gslbConf = cluster_conf/gslb.data +clusterConf = cluster_conf/cluster_conf.data +NameConf = + +maxHeaderUriBytes = 8096 +maxHeaderBytes = 8096 diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/cluster_conf/gslb.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/cluster_conf/gslb.data new file mode 100644 index 000000000..707418d1e --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/cluster_conf/gslb.data @@ -0,0 +1,18 @@ +{ + "clusters": { + "cluster_rmb": { + "GSLB_BLACKHOLE": 0, + "sub_rmb": 100 + }, + "cluster_no_table": { + "GSLB_BLACKHOLE": 0, + "sub_notable": 100 + }, + "cluster_fallback_rmb": { + "GSLB_BLACKHOLE": 0, + "sub_fallback_rmb": 100 + } + }, + "hostname": "gslb-test", + "ts": "20260720150000" +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_route/ai_route.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_route/ai_route.data new file mode 100644 index 000000000..aa97c3cd1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_route/ai_route.data @@ -0,0 +1,45 @@ +{ + "Version": "20260720150000", + "route_rules": { + "apikey_ak_user_a": { + "type": "apikey", + "owner": "ak_user_a", + "rules": [ + { + "name": "user_a-rmb", + "Cond": "req_host_in(\"rmb.example.org\")", + "targets": [ + { + "ClusterName": "cluster_rmb", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_fallback_rmb", + "Model": "" + } + ] + }, + { + "name": "user_a-notable", + "Cond": "req_host_in(\"notable.example.org\")", + "targets": [ + { + "ClusterName": "cluster_no_table", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [] + } + ] + } + }, + "ApikeyRouteTableBindings": { + "ak_user_a": [ + "apikey_ak_user_a" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_route/mod_ai_route.conf b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_route/mod_ai_route.conf new file mode 100644 index 000000000..f250013bb --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_route/mod_ai_route.conf @@ -0,0 +1,5 @@ +[basic] +RouteRulePath = mod_ai_route/ai_route.data + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_token_auth/mod_ai_token_auth.conf b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_token_auth/mod_ai_token_auth.conf new file mode 100644 index 000000000..966b473d0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_ai_token_auth/mod_ai_token_auth.conf @@ -0,0 +1,13 @@ +[basic] +ProductRulePath = mod_ai_token_auth/token_rule.data + +[redis] +Bns = 127.0.0.1:6379 +ConnectTimeout = 1000 +ReadTimeout = 1000 +WriteTimeout = 1000 +MaxIdle = 10 +MaxActive = 20 + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_body_process/body_process.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_body_process/body_process.data new file mode 100644 index 000000000..2a4746d1b --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_body_process/body_process.data @@ -0,0 +1,10 @@ +{ + "Version": "1.0", + "Config": { + "ai_product": [ + { + "Cond": "default_t()" + } + ] + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_body_process/mod_body_process.conf b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_body_process/mod_body_process.conf new file mode 100644 index 000000000..07fa25732 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/mod_body_process/mod_body_process.conf @@ -0,0 +1,5 @@ +[basic] +ProductRulePath = mod_body_process/body_process.data + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/host_rule.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/host_rule.data new file mode 100644 index 000000000..dbda248e7 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/host_rule.data @@ -0,0 +1,14 @@ +{ + "Version": "20260720150000", + "Hosts": { + "ai_product": [ + "rmb.example.org", + "notable.example.org" + ] + }, + "HostTags": { + "ai_product": [ + "ai_product" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/route_rule.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/route_rule.data new file mode 100644 index 000000000..ccb697617 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/route_rule.data @@ -0,0 +1,12 @@ +{ + "Version": "20260720150000", + "BasicRule": { + "ai_product": [ + { + "Hostname": ["*"], + "Path": ["*"], + "ClusterName": "cluster_fallback_rmb" + } + ] + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/vip_rule.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/vip_rule.data new file mode 100644 index 000000000..6fe22f03a --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/server_data_conf/vip_rule.data @@ -0,0 +1,8 @@ +{ + "Version": "20260720150000", + "Vips": { + "ai_vip": [ + "127.0.0.1" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/bfe_i_ca.crt b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/bfe_i_ca.crt new file mode 100644 index 000000000..f1e78f73c --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/bfe_i_ca.crt @@ -0,0 +1,23 @@ +-----BEGIN CERTIFICATE----- +MIIDwDCCAqigAwIBAgIBCzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +NDAxNDZaFw0zNzA3MTIxNDAxNDZaMIGQMQswCQYDVQQGEwJjbjEQMA4GA1UECAwH +YmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsMC3lpbmdmZWkt +ZGV2MRgwFgYDVQQDDA95aW5nZmVpLWRldi1pY2ExKTAnBgkqhkiG9w0BCQEWGmxp +YW5nY2h1YW5AeWYtbmV0d29ya3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A +MIIBCgKCAQEA6eFuWgoknixrRO9NCX4jAKyLtcAOWVJqVN2yX7CxjZjLyPurjvTZ +W73NYUbrPAN4AB5gY6UAPzuiEOSopVVIZ0OschK0cJldu9vZ0mZObBOsovuFcLQq +dgNSJ5slJSk7tCgD2EnCB3GYPG4D+uIKYd0c49wzTWWv4bjDwpgnf0LQbFpy7GhN +7D59zFH4qgOK/IQ5vaTMGyvIvtWR5/1Gvc9MLpGopTgi0DiNLed4UwDYrod5kysl +q3UcB5puONHQISOVoD3uRxo7wdsmVsHUfW7YfAWkhi6ec8mx9fy8IyE6f7GlXnuV +ysNccwqyEotL5bOXPJCqwUCL1v+iKTMPWwIDAQABo1AwTjAdBgNVHQ4EFgQUCjvC +vDVr8QXh9/hMI5xBgNvoSFgwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW +1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAhIggK/6JK4N7+HZW +VoTemwRpilugZZyKrcVAHbiiwUfXVQVuI64vc+yHWMSFRD1mykHkKBFzxEoDabMl +ASBtUJNt4b4zEL9V7k295vmAOp2IdLUQxlgeKWqABm4DGX96tGR9nKQTcn1ZeAxx +NyQFV1aj+dnNcF1iFFNF6t0bRrOEZ/aRSiu1bWp3Dj1JYTjXbyyplh3Ktb8lv4lt +EmvCXjo/l4TgQC9233kcTkXcq1swppzkkXfhB0NVuf9DE3C2xAWX0b2FiIEqnAhl +Bx0Cn3RrX7PZ6qrdRL6oBKvGJV6DP8BlF4LXiuD1VN/waQFJha57KQ78ZYYIXxQg +WMjWgA== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/bfe_r_ca.crt b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/bfe_r_ca.crt new file mode 100644 index 000000000..2a6db1e49 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/bfe_r_ca.crt @@ -0,0 +1,22 @@ +-----BEGIN CERTIFICATE----- +MIIDkzCCAnugAwIBAgIBCjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +MzQ5NTVaFw0zNzA3MTIxMzQ5NTVaMGQxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxFzAVBgNVBAMMDnlpbmdmZWktZGV2LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOC +AQ8AMIIBCgKCAQEAvNA3HrsMjBcXrMIIhGVWsurIA1F9jxKeA7dh06H00Vt4inVV +SUvNFrTqgPRhLkAhGRMPxrjVRgJ5bbFqqXIuPIpUFBhUsWXIDH+oVXQl9jsxAXaG +gZ0lTO/uYR9qyrS1rj9nyNPwRf59Al/VlsQL71cNQ/T/agJ4PfvPfULTPLOsqclJ +hj0IgXmDj464dqcdG3ZdfXpfhNF6ab+8YjpwafTRmY+LoV8qjUwsYeJMcW4N8pxJ +8F2ktZj9J6uWepNGj+87ZeXg9XquzC62ASIFzPjoE1WN//Q518EqizxhuLGBDpK3 +6sEYUK4kHYUL4gZKFPlKTPXIl0ZsJIM5PsBMKwIDAQABo1AwTjAdBgNVHQ4EFgQU +DZloT8VhVbysD819oG/oqHdW1D0wHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/o +qHdW1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEALgB+IcN3UD4T ++4g8jaOyOiSUpUVdYqW+3go5DppqnvlGNq99N2cXKqssVPa/T9TikEcBEicFa8zU +bwlx6TEte+MkWfWdQxFR1EuI1FgKr3ps6ZBRr7MPpnYmI7K9aK372K9n7WrQhbmP +s7ult8bWB1/t6o3R7B9ChNkWT+7DPD4+FvB1GMJSGPno7cdnvDkevBOuC2DnQl3M ++ADFAge1Lo8wKBy6gYkNFd2BfarHGvRC5Qmmrme+RIpWZnvux1+lfXIInnfSTJRM +uAo/ePkoNsM3qQll6uEdhDxOMx8Pq94bCkM3DtI3ObuxWXjKCgUm/n9yetUvPj4U +iYhRUqHpUg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_bfe_dev.crt b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_bfe_dev.crt new file mode 100644 index 000000000..2164fb602 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_bfe_dev.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 18 (0x12) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Jan 31 02:55:32 2024 GMT + Not After : Jan 28 02:55:32 2034 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=bfe-dev/emailAddress=dev@example.org + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:b2:7f:c1:0c:cd:49:43:d9:99:78:15:4c:5a:52: + a9:a7:bf:d5:eb:92:71:43:e1:37:e5:29:1b:68:f4: + 6f:4c:ea:fa:a4:8a:7c:29:01:2a:fa:7c:81:5b:c8: + eb:a1:40:94:b7:2b:82:e2:00:08:36:84:f0:b7:d2: + 5a:1e:56:97:aa:36:ff:4d:07:49:2d:fe:25:3a:f9: + e9:f1:ad:4e:6e:21:97:a9:f9:a2:a5:ac:82:23:0a: + d2:e0:97:cd:2f:14:b1:f0:8a:a8:e6:c5:97:45:94: + f8:8e:3f:96:66:5b:0b:8c:7c:07:61:18:44:92:f7: + 23:5a:c2:4b:88:58:59:5d:ca:5c:0d:6e:dd:ff:18: + 59:65:df:95:99:e3:3a:36:48:1f:3f:3a:e6:ce:85: + f3:0b:04:5e:92:ed:6f:8e:74:92:e4:37:46:da:5f: + 17:62:9c:82:40:06:fb:29:f8:55:f2:ba:23:75:ca: + 64:c0:45:03:12:bd:f5:17:15:7e:47:d5:bd:30:f2: + 99:ca:6b:e3:07:b0:ae:44:89:1e:10:26:ea:75:df: + 6f:07:b6:47:76:54:47:4f:6c:f6:68:fe:a8:cf:22: + 20:73:e8:19:55:8a:fe:f5:78:e8:51:88:52:80:1f: + 79:d4:c5:ae:8f:d2:2b:f6:41:01:42:01:cf:98:c2: + c9:25 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + EE:E8:68:42:DF:B1:F0:EF:6F:47:51:BD:D4:94:60:1F:05:85:A1:03 + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Extended Key Usage: + TLS Web Client Authentication + Signature Algorithm: sha256WithRSAEncryption + a9:a6:26:8e:42:61:15:22:ee:fc:b5:e1:e4:6b:dd:ac:f5:15: + 11:39:10:9a:ca:6f:85:fd:cb:90:1c:b2:4f:fe:29:de:b0:e7: + 73:e2:f7:5e:63:8c:7f:c1:7e:75:2e:c9:9d:e9:c2:45:75:f3: + 27:ba:82:94:de:7f:6c:87:0c:5c:71:af:0f:14:00:68:35:f7: + 5a:4a:ff:f5:ef:35:dd:50:72:76:f0:6f:b6:7b:42:33:07:b4: + 24:44:0a:fd:9d:61:9e:44:e8:88:0f:02:76:c6:90:3f:9d:1b: + d8:3b:64:25:2a:a3:39:78:38:bd:20:89:4a:9c:bd:68:38:18: + 4c:cb:20:3a:9b:5b:5f:58:52:86:73:de:85:fe:d6:a1:c6:a7: + 86:b0:96:4b:fa:28:04:ad:5d:85:e8:a1:fc:ca:0f:3c:be:5c: + 90:7e:3e:84:ae:67:ee:9a:72:71:3c:b2:80:45:82:fc:7e:58: + 74:99:42:c5:c3:8a:4a:eb:e1:8b:d5:84:ce:25:aa:a1:75:79: + 94:66:ae:ee:df:30:15:0b:b5:c5:b1:2c:d5:0a:54:78:b6:2e: + 67:29:81:41:f6:16:49:31:96:e7:41:e1:99:6b:27:57:bb:7d: + 76:eb:e4:d5:59:aa:a2:5c:bd:1c:18:2a:fa:9d:28:1a:0b:b6: + bf:7d:58:1a +-----BEGIN CERTIFICATE----- +MIID7jCCAtagAwIBAgIBEjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yNDAxMzEw +MjU1MzJaFw0zNDAxMjgwMjU1MzJaMH0xCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxEDAOBgNVBAMMB2JmZS1kZXYxHjAcBgkqhkiG9w0BCQEWD2RldkBleGFtcGxl +Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ/wQzNSUPZmXgV +TFpSqae/1euScUPhN+UpG2j0b0zq+qSKfCkBKvp8gVvI66FAlLcrguIACDaE8LfS +Wh5Wl6o2/00HSS3+JTr56fGtTm4hl6n5oqWsgiMK0uCXzS8UsfCKqObFl0WU+I4/ +lmZbC4x8B2EYRJL3I1rCS4hYWV3KXA1u3f8YWWXflZnjOjZIHz865s6F8wsEXpLt +b450kuQ3RtpfF2KcgkAG+yn4VfK6I3XKZMBFAxK99RcVfkfVvTDymcpr4wewrkSJ +HhAm6nXfbwe2R3ZUR09s9mj+qM8iIHPoGVWK/vV46FGIUoAfedTFro/SK/ZBAUIB +z5jCySUCAwEAAaOBkTCBjjAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu +U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU7uhoQt+x8O9vR1G9 +1JRgHwWFoQMwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW1D0wEwYDVR0l +BAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKmmJo5CYRUi7vy14eRr +3az1FRE5EJrKb4X9y5Acsk/+Kd6w53Pi915jjH/BfnUuyZ3pwkV18ye6gpTef2yH +DFxxrw8UAGg191pK//XvNd1Qcnbwb7Z7QjMHtCRECv2dYZ5E6IgPAnbGkD+dG9g7 +ZCUqozl4OL0giUqcvWg4GEzLIDqbW19YUoZz3oX+1qHGp4awlkv6KAStXYXoofzK +Dzy+XJB+PoSuZ+6acnE8soBFgvx+WHSZQsXDikrr4YvVhM4lqqF1eZRmru7fMBUL +tcWxLNUKVHi2LmcpgUH2FkkxludB4ZlrJ1e7fXbr5NVZqqJcvRwYKvqdKBoLtr99 +WBo= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem new file mode 100644 index 000000000..764aea3ad --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAsn/BDM1JQ9mZeBVMWlKpp7/V65JxQ+E35SkbaPRvTOr6pIp8 +KQEq+nyBW8jroUCUtyuC4gAINoTwt9JaHlaXqjb/TQdJLf4lOvnp8a1ObiGXqfmi +payCIwrS4JfNLxSx8Iqo5sWXRZT4jj+WZlsLjHwHYRhEkvcjWsJLiFhZXcpcDW7d +/xhZZd+VmeM6NkgfPzrmzoXzCwReku1vjnSS5DdG2l8XYpyCQAb7KfhV8rojdcpk +wEUDEr31FxV+R9W9MPKZymvjB7CuRIkeECbqdd9vB7ZHdlRHT2z2aP6ozyIgc+gZ +VYr+9XjoUYhSgB951MWuj9Ir9kEBQgHPmMLJJQIDAQABAoIBAFAKHzupVb/58+o3 +yqv5wx94Uuk2GlnwxIqaezL94GaiO0/K1U/huS7m426P0rDU75qPBTpn/0bLJ9GV +nllaRNnLnYEh0juwaWtfovp+1ttlbseGK9uUVip2cQbKqvQAmKWe14vbcDCAU1Ad +zUgKbUxKVVjBdAZekVjiJNJ3o2L9WPhf/uQo7A1XAJh2DajlTbgvrDM73W+47QuU +X4OHU0FMio6bxupu3OWl1bMrnKhuC4qczZWf2nOpcVQa89rtopuP4ENLJuWkbeGk +YQpNilEclnAa/Noumt/j/6GKC1EEHFsH2CNRRIazcZrsFhkSKc4pn1Y/WI3vj8kZ ++RYnJsUCgYEA6gr0x8suwRTAmVpoPk4XyP1x+eInG7onV5RjgsUtgruYd9naxRHg +2p8PHcv32pDs51Fa+4RldyMd/jec1SscRF5/+VOP9qeoRnaDqUu+uASgEO3OBxbP +JcWovyxRHIQxbYCQtqIr9bdzXw55MBLZou/sBUVTAkrIyPVyjWqZlV8CgYEAwz7L +YyYN615TsrzZKURxMjj94Nmob/NldSLRXaR3Ax7/ABtEOA685cwQxq7ONdkJTMIA +uR8u2GHZSzGiWnehuF6Zp7Xs71a57eFbs3ueZvvEba4Dff7hl7Y4tTlwrKndKjvP +J/5a2Ol8siQcRWAXHOdzggEHMSZ/sB4hWswly/sCgYEAhLRBpyemEwTZUBrbELjm +86gBgFajJi2fMSGKaxOygnYsNYjpauSAQnX99D87Aks6iM6wb/zaK3tV/lc6LgSL +uph6p7yh3JGj8JAyh0PTmDPHLtIoCAz+18QDsqJGO40ZGaXUaDn8Aw9J85QZUxDd +Jm4zvalZL+uHfarukRDolLECgYBUiupS4nWAh3XCnZeDEQna72avaFBROZmjIRJ7 +c+28wj009JmTlH4jGzvgbG0KUBKA1Div8Fq+g5AtyS498jNqvDvYrSQNdwZHhR/K +Fis++KHTxFfqxOU2Zkcj4d1yRpNn6EIJVVBNQL0n/g7n03XupCIWFw/gLoV343QZ +9vAe5QKBgA8ml59z1w3eUooc0yGfLhihXqCmM3IU006bFbODA30fBU4QKrHO8+Yx +Xbz9bi/1QLagLG6FzYQAkOjBlEBt5XLayYvwSb8xvWsm5A3vzTAbMFDOsDPEmRoH +dWtQccJcygOuK+PZtoZnNoJciNO5c9dZWD3xtIiURmX/kVtWrf6N +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_san_example.org.crt b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_san_example.org.crt new file mode 100644 index 000000000..78f3892b4 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_san_example.org.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 15 (0xf) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Dec 7 09:57:42 2023 GMT + Not After : Dec 4 09:57:42 2033 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei, CN=dev-test + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:d6:47:b0:17:69:91:4c:d0:66:c9:25:2e:38:f2: + 84:27:e2:7c:38:cc:04:b9:0c:8e:3d:cc:ef:4b:c5: + 35:2b:c1:82:d8:41:fc:25:c8:24:f2:e0:25:aa:f9: + 76:3c:e2:b2:50:fb:2d:ec:4e:16:d4:c1:da:1e:e3: + 51:9f:9d:c9:72:f9:cb:cc:14:9c:c1:82:c9:76:1f: + 98:dd:0e:b9:8a:20:d6:ab:f2:a6:f9:2f:23:81:f3: + af:e4:47:0c:55:95:94:de:ed:f6:a5:24:ee:32:e7: + 6b:79:e1:42:6f:a4:07:b2:95:1d:f5:a9:8e:60:42: + 70:42:bd:e2:30:18:68:74:52:32:98:a9:81:da:d8: + c6:6f:5e:1d:ce:79:b6:f3:ec:4f:ed:7d:22:57:d2: + 14:d0:fb:f2:50:d4:80:3b:89:ed:77:fd:45:6c:e6: + 52:6b:0a:52:71:ac:59:c8:d5:25:f5:40:03:fb:51: + b0:11:a7:00:79:d9:d8:4f:00:43:96:68:44:29:41: + dc:d2:cc:91:c8:61:95:41:4d:0e:66:5a:b5:15:67: + 3e:8a:6f:29:df:1c:8a:6f:ee:9e:97:9c:9e:69:71: + d3:34:52:75:e9:ea:e7:51:77:23:98:46:ca:47:a2: + d3:d3:97:03:41:4b:e3:33:11:72:2d:af:bf:2b:3e: + b3:51 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + 5A:27:32:9D:E7:36:24:A3:C1:DC:2F:95:80:C5:CF:0C:85:E8:E6:AF + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Subject Alternative Name: + DNS:example.org, DNS:www.example.org, DNS:example.com, DNS:*.example.com, IP Address:127.0.0.1, IP Address:192.168.0.100 + Signature Algorithm: sha256WithRSAEncryption + 1e:e8:e8:8a:ad:a8:0e:fc:c9:82:00:a1:ab:30:3c:a5:b9:dc: + d6:fb:86:ad:30:52:7f:61:be:90:a6:b8:56:bb:f1:0b:e6:39: + 38:65:09:6b:da:83:f7:65:ff:c4:21:de:b4:9e:8b:bd:1e:1c: + d1:d5:94:b8:18:79:f2:d0:06:51:39:67:13:40:3b:73:5b:cb: + ea:de:c1:19:76:f8:7b:0f:15:51:61:49:fb:98:f7:ea:4f:fc: + c2:fb:a7:f4:3c:48:64:14:79:b5:78:5b:20:10:b5:7a:2d:4c: + 04:51:60:ec:20:10:19:26:5f:e2:fd:32:59:67:e9:3f:48:8d: + f5:52:12:01:81:2c:c0:e5:72:cd:7d:0a:eb:7a:05:df:a0:77: + b9:ba:9a:7d:d1:4b:6a:44:e4:2d:98:af:bd:77:2b:f5:ef:26: + 4b:75:b3:97:d0:3a:bc:07:21:ef:71:92:30:fe:a2:79:e5:56: + d7:7e:c2:f3:57:ab:d7:de:fc:97:ed:20:0c:9a:cb:c5:5d:00: + 3b:61:29:e8:00:d4:39:e0:f2:4e:a4:03:c2:12:52:ff:e7:78: + f9:f7:c0:12:dc:36:a4:05:a2:f0:6b:47:e2:21:3d:a2:e1:a1: + 91:c7:ac:8f:b8:ae:58:65:e0:2b:57:80:eb:77:2d:48:ef:e6: + fb:b9:e1:20 +-----BEGIN CERTIFICATE----- +MIIEBzCCAu+gAwIBAgIBDzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzEyMDcw +OTU3NDJaFw0zMzEyMDQwOTU3NDJaMFoxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEQMA4GA1UECwwHeWluZ2ZlaTER +MA8GA1UEAwwIZGV2LXRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB +AQDWR7AXaZFM0GbJJS448oQn4nw4zAS5DI49zO9LxTUrwYLYQfwlyCTy4CWq+XY8 +4rJQ+y3sThbUwdoe41Gfncly+cvMFJzBgsl2H5jdDrmKINar8qb5LyOB86/kRwxV +lZTe7falJO4y52t54UJvpAeylR31qY5gQnBCveIwGGh0UjKYqYHa2MZvXh3Oebbz +7E/tfSJX0hTQ+/JQ1IA7ie13/UVs5lJrClJxrFnI1SX1QAP7UbARpwB52dhPAEOW +aEQpQdzSzJHIYZVBTQ5mWrUVZz6KbynfHIpv7p6XnJ5pcdM0UnXp6udRdyOYRspH +otPTlwNBS+MzEXItr78rPrNRAgMBAAGjgc0wgcowCQYDVR0TBAIwADAsBglghkgB +hvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYE +FFonMp3nNiSjwdwvlYDFzwyF6OavMB8GA1UdIwQYMBaAFA2ZaE/FYVW8rA/NfaBv +6Kh3VtQ9ME8GA1UdEQRIMEaCC2V4YW1wbGUub3Jngg93d3cuZXhhbXBsZS5vcmeC +C2V4YW1wbGUuY29tgg0qLmV4YW1wbGUuY29thwR/AAABhwTAqABkMA0GCSqGSIb3 +DQEBCwUAA4IBAQAe6OiKragO/MmCAKGrMDyludzW+4atMFJ/Yb6QprhWu/EL5jk4 +ZQlr2oP3Zf/EId60nou9HhzR1ZS4GHny0AZROWcTQDtzW8vq3sEZdvh7DxVRYUn7 +mPfqT/zC+6f0PEhkFHm1eFsgELV6LUwEUWDsIBAZJl/i/TJZZ+k/SI31UhIBgSzA +5XLNfQrregXfoHe5upp90UtqROQtmK+9dyv17yZLdbOX0Dq8ByHvcZIw/qJ55VbX +fsLzV6vX3vyX7SAMmsvFXQA7YSnoANQ54PJOpAPCElL/53j598AS3DakBaLwa0fi +IT2i4aGRx6yPuK5YZeArV4Drdy1I7+b7ueEg +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem new file mode 100644 index 000000000..393a79825 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEA1kewF2mRTNBmySUuOPKEJ+J8OMwEuQyOPczvS8U1K8GC2EH8 +Jcgk8uAlqvl2POKyUPst7E4W1MHaHuNRn53JcvnLzBScwYLJdh+Y3Q65iiDWq/Km ++S8jgfOv5EcMVZWU3u32pSTuMudreeFCb6QHspUd9amOYEJwQr3iMBhodFIymKmB +2tjGb14dznm28+xP7X0iV9IU0PvyUNSAO4ntd/1FbOZSawpScaxZyNUl9UAD+1Gw +EacAednYTwBDlmhEKUHc0syRyGGVQU0OZlq1FWc+im8p3xyKb+6el5yeaXHTNFJ1 +6ernUXcjmEbKR6LT05cDQUvjMxFyLa+/Kz6zUQIDAQABAoIBAC4sYGuLGf49Ygix +9FXdHFEj4rSyccoWRIhYoq/nHOAC4NkMzvKtQBj95+ABxVK1XstIdMrYwN6zrva8 +8Re9/mzCGwIs5uJj9ll30Y7A34Y+MUP4E7baS4JzKlG8ZZIDm4K2MFHBtXpOl8A5 +pAE+jVIUA9Kt6LohVuNq21SVzdxSfNYC/+SLqSftkWa/ZsqdkiHM5Hl+fVedh516 +IaLNW5hSthGh5n8dHY5h/AKPjfoq77aYp5/CUtJTC9mYdZu1j/W/pBVTRfOnwLQd +SQ1Xmr7f6q9Vmz+HnajIbFg9hQ54blvtUJ7DnugWxfUcoxf7ue79fnYjOIUOkRWw +8Iid/mECgYEA+5s0p0j+gkNZN5QtVNStoT04+1DqA1O381gczeaiz6Njt/MT0y5W +OpCsILQ70CpEjWAV+f6PDJiesDMxdGV+v2TCqK8ml8GahEczBLnHoAkPWCVf2XOX +oNj/CkZ2kmWufHFR+kcQbeDt1vFFcYUa61hKDFyNjinMW79Qy+PQID0CgYEA2gWd +7thE05sqU7/1MntmVRONKoAgnJmHcfSpWwLyh6E4YX3iKDSgI/9RnAMF39KUUY/O +XFWyIwAM9soeXknVsV/SmCaPeaEDiLHqz98aUEfvdLYMnuR883GgoXc4JrLsLw4z +oSi9lbAZFn0ekJ5L+rSFrY4rz9QZgYZxsLjUXKUCgYAbkaUSU2g3w8Np2J2i9u7T +hQ7SUsphdPHqAxSc5xGd6MxLYqIgeKpQHnwN1VHcfFUonIer7d2kxrBUpDdeBqT9 +ub+ulgqHhFo29ko70VNzUKrSwL2g6Q6LPFutt4zUe7nDvvL5loHRWF0XOTafurL5 +aKIsepO0KRZQU0U6IgszDQKBgFs6ZHaP2mTtFY4L0a75Ab3xu20gRgUhHRLq/H6P +wipMpMnuodaPBr9pU53Dig65D8T9Nq1eUnbgy4vs0T5FCPz6iqWN5RVQ8aieQhIP +WfRj1Wfx0WAfXcWEM2G9ACr5TWj3OVVjNclP8X9+hW6gPky+gv03c0+4gZ+4QRRg +ksPdAoGBAKXVv8L5Qt8rmi+QpF/6DVSeMB6lEevf319UEtyZPiYSBn1JJ9H+3Ddc +TMriXgZrw+PoXNJTAguDeGgzmtye1vraP0cjt7aG6sQ4YtMTnnuja880vK8z4i3Q +HJAi5fI8NvlW92dQBAccgrzFIKpRx+6CHwtCkNrxL2vJmGGzp8BR +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/certs/example.crt b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/certs/example.crt new file mode 100644 index 000000000..931874885 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/certs/example.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDEjCCAfqgAwIBAgIJAIAdu56fLE7OMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzM0NFoYDzIxMjAwNTE3MTAzMzQ0 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBALv+LV1aWIlcK9rI7IuRS8SCusqBnoyJec/ErKiA2gbfgZ/YS73L +Zud84yp45AIqauzcI5q+hrkmsRZ7CKqDzrG+jHavW7jF+0laetJwRt26AcQcOtQD +2ik2O+Dl1WHAFn4vUAQxb+Xz6WfSaQN0QfM74z06XUDDsr7g7+NYtMzhf98SJSoK +ne/dVKJ3Bc6e6tvhnCRwPtix4ektEodK6WeNHYxwJ6wSZ8cRLzdxgjdD/4OGfFuj +dn8zbOi3SQt5ZqVbcDHUTzp5t0G8EoxnzotHhhzjSAmsypySqZXaxl3oX8aYUkFn +fCdg+WBXo5pOiNfoWh/D5bnIXWGp52yoy+kCAwEAAaNhMF8wHQYDVR0lBBYwFAYI +KwYBBQUHAwIGCCsGAQUFBwMBMB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MB0GA1UdDgQWBBSxLHQE7gOEyfeSNc5uIO/G/rgjpzANBgkqhkiG9w0BAQsF +AAOCAQEAlGm5RwQ79xmLh3rj+5UViCSgsIuMcuhgIT4zogpo9S4uwXMqinrJhzRk +Oc2tb3y06XTAq1lMH2+58tqndAu8ni/UBz3OSghk2CTnZ1vxxXOd3CtQu4ypMq+k +qW0Umdrkk5TeAODNbrCy4c6vpICkQOljnRFWnDYu3aQ3JvaWZ/nObN7C72Lgpjfb +RfLXGmLsBCEIr028f9hpoeRCXoetUY2CiC2boAHR+cO6Jpvex4Jv5yYDpNKac52n +LLC8Cq5ozhOZNOSV6X9FpEca3rdhVUb0VgoNDCPZDdpO1PDJYCN9fUC8KUs+dsOh +SYGpliBaNsztoiJs1q5SkMQrMwmMmg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/certs/example.key b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/certs/example.key new file mode 100644 index 000000000..b21c2f08d --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/certs/example.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAu/4tXVpYiVwr2sjsi5FLxIK6yoGejIl5z8SsqIDaBt+Bn9hL +vctm53zjKnjkAipq7Nwjmr6GuSaxFnsIqoPOsb6Mdq9buMX7SVp60nBG3boBxBw6 +1APaKTY74OXVYcAWfi9QBDFv5fPpZ9JpA3RB8zvjPTpdQMOyvuDv41i0zOF/3xIl +Kgqd791UoncFzp7q2+GcJHA+2LHh6S0Sh0rpZ40djHAnrBJnxxEvN3GCN0P/g4Z8 +W6N2fzNs6LdJC3lmpVtwMdRPOnm3QbwSjGfOi0eGHONICazKnJKpldrGXehfxphS +QWd8J2D5YFejmk6I1+haH8PluchdYannbKjL6QIDAQABAoIBAEUBL7WsjAMfihls +1ycD1kPzmIzstz3u2H+jOZ1AbsdHE1WRF3w7RTKDbP8SEN+aolT/GTKb7OfZg/c0 +giHU7/Hed8C47XoNcgei5qKIA/svY6aQlidsoo+uEJykwIZ488itpTlkzCYkOfCa +E2HpMqwNt4OqAMDdFKdr+aIB1Zu+KPBxW23WD9wEWAbe5LA4YnRF9kT4YZ6y9mce +dGaIf39VtBlrGMmvoU0LE9B79nyuebGi0svW6QDarBqaDrnM/N3fXgL1kk/gVfan +/xs6EA4qPxA5G4h+enYrIlZL0CbSj60nYElo+Z5nRdBaRdCF/bpXOLyK/kXWLUM0 +f2HTK+ECgYEA5cVcpJtczxEaxoaEUbppsW1LCrTjJDGTKJ63G7/lwqCxJeCHN185 +nnckHOW2287e19bu9aUmKJgRq5s1rXnT+MnCl/hQnfaMrKOKtzE+t7/zsc9+LuAr +pwJrtZ9Dcnwrk8NOE0fPjW5XpDCSoEOo7JWZmGTVlpabOgNkjocfp+sCgYEA0XPt +ZPt3F0wyzgLYRhgnvp5CV8SzQmulsW+ytnL5eiAcNSXqni3wQHN3PGxLInEyQwBQ +/M8TQpUbqGMmahCK4ZxMAwXMrpF0mVB8jfoYMou1FSYPlUV+CvLjWcTkZB1Nirez +VFXdtfHP0mx4PbYK2qjB03u4pPHAN8kuayIf2nsCgYAbv/FHZAgabfto3Jggcr4P +Ep8MhPolxeL69egxbsSl89hRNcO+2T5ROBxhbRDfjSV2tduYSUDJiEwiCJW8BMmn +814QEopR+ZPVyc6X/1eOw5z/7YpUyPgcrHsrrTdtHTf6GY1VYMfdUeU9zCv5NRKy +uAKb2Bm/nSLUJ9K+L+2PzwKBgQDRQgT3UtTUjehkMitpPFDY/LxDe92simfsMjBW +X+Anx1TnNI6GolbZzYJe98LJElao4fQH38raRqZvQT/rz8MxTDoU+wJXljLryaHn +Jupt9W5hRrli5R7cSXYjBbc43p3N7WJY68CqOoDrNjubS/jkJJ4hcAY1pOHp2jFq +D5nLaQKBgAysU6O5kJ8yKxhbZflb42MqKCFBGrbRnbYx14PAEZOaRhzxehpppQmx +RLbn/z1Uh5Ms28ipxA+vnhyM3FcU5lKboaFyWJeuNslw0FxEcIai6hL6UkDznS4G +aqyzUjpG5Chg0x18xWYCbiGJwjZ9BWhtH+jojm856QHGzeQJWVoo +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/client_ca/example_ca.crt b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/client_ca/example_ca.crt new file mode 100644 index 000000000..b0fa2fa28 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/client_ca/example_ca.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDATCCAemgAwIBAgIJAMsPuHg4mqnaMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzMwNloYDzIxMjAwNTE3MTAzMzA2 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBAL72D2gOnJN9Zvo9KjruwM1EsFe3xZRJ0NvZ5bHd6+5jhlgCAhQ+ +AGb7ufEiYOi2JWHl2Bkq0iVrp+zv0RLdq0oVjX+OG5H2yWbnC7ifbNjir93LX0un +tIqv5CIbExDSBRkufxfV37yjXdrcMqYSbD2Kw3PfAbWs1Dego8fRz8QAp5+LCvW2 +BZZyYi6JzhCAUW1+8OQPyzOhB50eSJiS5xgVA7wkwmYeVUpHqU8sv4VzjM3bmUc7 +1mPLlnRVIqScrqYgQ9Ou21vZebOJ8+ckVL8O3XHhMZlssBbWiBFZZnaNWbzcEI90 +oiW4YAQ5t7gaXuCVvaiNvq2VZknarR6AxcsCAwEAAaNQME4wHQYDVR0OBBYEFIH+ +0G3eCswQHbN06kvI80M3tNH9MB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAJtHx8ibLp+CYDR1 +ZVeQ3qg4OeRL0q21+2EgBJq6zOUt/9SxThA80aXJ8CYH9dnCW+fOnpEk1xFWxtXc +FwSLsnqAdwOJaaQWoMzhyqjZV5x5G9+MW5FzGGOdes2md2Z+tAwMoV9TVtxZkbKy +mC2tDJdvgLgt9/YcbUcZPDbyZojdZ+UbATm+Lro9dhTXt91vsAgz5QA9e08rQVkF +pc9+ZQ5zxBsoblQ+ozPOWOdV4zJVx+wQsAnOG2qU0yVQAscGsTo4wnzFrAU54fO7 +Lh4cOrY0P1/o65yiSzwK7f0jwBeT/jEfMOrJ7pPo7doUov0iVj0SZyTM3HFa2Mzj +2zGTk0o= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/client_ca/example_ca.key b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/client_ca/example_ca.key new file mode 100644 index 000000000..4f7ad13b1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/client_ca/example_ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAvvYPaA6ck31m+j0qOu7AzUSwV7fFlEnQ29nlsd3r7mOGWAIC +FD4AZvu58SJg6LYlYeXYGSrSJWun7O/REt2rShWNf44bkfbJZucLuJ9s2OKv3ctf +S6e0iq/kIhsTENIFGS5/F9XfvKNd2twyphJsPYrDc98BtazUN6Cjx9HPxACnn4sK +9bYFlnJiLonOEIBRbX7w5A/LM6EHnR5ImJLnGBUDvCTCZh5VSkepTyy/hXOMzduZ +RzvWY8uWdFUipJyupiBD067bW9l5s4nz5yRUvw7dceExmWywFtaIEVlmdo1ZvNwQ +j3SiJbhgBDm3uBpe4JW9qI2+rZVmSdqtHoDFywIDAQABAoIBAD60bbqtkZycwQPK +seNIIudEduNW5PocgwiuNE6DoMVWyPZ9MlGTSm6GmjgkIc5IgV30K1GYTgkboLic +xvp675QUH7KS51q2vsubcq3dK9DMHxOlhFVDbHVd7HuGiGwtip8KNZGOGTnIKzmC +tN7zjbdnqWaTA+y0I7tgdGdY7fBd9Rzgaq+OlPq2u33HvWHvlG/7PfpZuXB5YLgd +m04l7LJ7ikhIjycg7j27v/4c6xCiH5jMJKsZ+nfsQ0kEEo9DkhcKInK+wHsMzKsH +Cy3AdlE0IRsbxRAoMumVs2g5u90m3zBPkRrNdZ2Ni7BesnhxbkIqvb4SfpxKyuhK +fADfZgECgYEA7SUIS2gII0TGjXh0h16d+eoLVOz0eVpgF3XXxmgtAPu10dqxVEC2 +j5FSBCgZhqZ3axVotP71c2mT+hF+Mqy4TLMfA/B9jKLXjZlPbg4EcAgI7tALskwz +Bk5BkX0k825bU9P0j+AlpLx6/ztHr2N9/cKZfqQVrO9t+FRusvAHkHsCgYEAziT8 +F30Ch2s6IJngCj5jH164iN2CoFXjqPNVgRj45gLE3zrf1R7u2JTeEhjWNLZ6IWZ3 +G/bT7eYm6x8u7LFlORdnWKsHlftGu0igRyvIGcxoHgjXlsLidBaEP+HlOLUtTumu +MfQJUozLcrOBIV6m9VhPnSDTeCg/tOqy68V2pvECgYAUYgd5e8KfTW0Hgd/6Nq67 +aVt5/DfzKkpyGcXnHtMnb3ssQ3DUfg9y/ZmgE9ZF1Y8UHC34yKVOOzfl2ZUQQ/o/ +VXIIA6a27NQ8Ln4+RmQpQPeLl0Q6GgSUuSs3lxsS9VxSMzilGS4DH9QulejOcW3F +3vEUioP2bkn0e0VcifcMewKBgAG1Pr13FLFIiye//qI3GB0nbMH9i9qGO6entHqo +WU+WkEkFNNuQMQxsV1axC/1N0b87GRuLNQBQmtvx2zKs2Zjaf8m1SQ/OECz3EhTk +4PiNwAMXsamXHcc2dIwO9BY/MgvoVcAmNHmRnxHpONWs8hcwTyCPKBFjy/tUwny/ +mxcRAoGAcNRxLZlyRqmQ6zGf41GK4ZIR9gix0L6Km49S1maGFmcbctOR2GcQN8Eo +f38rkrFBfBfuFSGzghJiXDvKORg9r3V/bzSKcXkprJra6hzn5vn+t7wurjzaJUlK +zUW5dl3SU2bC4MK+X7bwqf9jm9b7FXSt4p1xly8Uh/Mufwi7zOM= +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/server_cert_conf.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/server_cert_conf.data new file mode 100644 index 000000000..49f228531 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/server_cert_conf.data @@ -0,0 +1,12 @@ +{ + "Version": "init version", + "Config": { + "Default": "example.org", + "CertConf": { + "example.org": { + "ServerCertFile": "tls_conf/certs/example.crt", + "ServerKeyFile" : "tls_conf/certs/example.key" + } + } + } +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/session_ticket_key.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/session_ticket_key.data new file mode 100644 index 000000000..b3d2356b0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/session_ticket_key.data @@ -0,0 +1,4 @@ +{ + "Version": "init version", + "SessionTicketKey": "08a0d852ef494143af613ef32d3c39314758885f7108e9ab021d55f422a454f7c9cd5a53978f48fa1063eadcdc06878f" +} diff --git a/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/tls_rule_conf.data b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/tls_rule_conf.data new file mode 100644 index 000000000..66e7b5dbb --- /dev/null +++ b/tests/integration/implementation/scenario-SC03-rmb-quota/testdata/tls_conf/tls_rule_conf.data @@ -0,0 +1,20 @@ +{ + "Version": "12", + "DefaultNextProtos": ["http/1.1"], + "Config": { + "example_product": { + "VipConf": [ + "10.199.4.14" + ], + "SniConf": ["example.org"], + "CertName": "example.org", + "NextProtos": [ + "h2;rate=100;isw=65535;mcs=200;level=0", + "http/1.1" + ], + "Grade": "C", + "ClientAuth": false, + "ClientCAName": "example_ca" + } + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/sc04_provider_model_prefix_strip_test.go b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/sc04_provider_model_prefix_strip_test.go new file mode 100644 index 000000000..e239a4593 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/sc04_provider_model_prefix_strip_test.go @@ -0,0 +1,334 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package sc04 + +import ( + "bytes" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" + "github.com/bfenetworks/bfe/tests/integration/common" +) + +const ( + apiHost = "api.example.org" + apiPath = "/v1/chat/completions" + apiKey = "ak_user_a" + + clusterOpenRouter = "cluster_openrouter" + clusterFallback = "cluster_fallback" + clusterDefault = "cluster_default" +) + +// testEnv holds all resources for a single SC04 integration test. +type testEnv struct { + t *testing.T + processEnv *common.ProcessEnv + backends map[string]*common.MockBackend + bfePort int + stopBFE func() +} + +func newTestEnv(t *testing.T, openrouterStatus, fallbackStatus, defaultStatus int, + openrouterAIConf, fallbackAIConf *cluster_conf.AIConf) *testEnv { + e := &testEnv{ + t: t, + backends: make(map[string]*common.MockBackend), + } + + e.backends[clusterOpenRouter] = common.NewMockBackend(clusterOpenRouter, openrouterStatus, `{"ok":true}`) + e.backends[clusterFallback] = common.NewMockBackend(clusterFallback, fallbackStatus, `{"ok":true}`) + e.backends[clusterDefault] = common.NewMockBackend(clusterDefault, defaultStatus, `{"ok":true}`) + + e.processEnv = common.NewProcessEnv(t) + e.processEnv.Build() + + confDir := filepath.Join(e.processEnv.WorkDir(), "conf") + logDir := filepath.Join(e.processEnv.WorkDir(), "log") + + aiConfs := map[string]*cluster_conf.AIConf{} + if openrouterAIConf != nil { + aiConfs[clusterOpenRouter] = openrouterAIConf + } + if fallbackAIConf != nil { + aiConfs[clusterFallback] = fallbackAIConf + } + + builder := &common.BFEConfigBuilder{ + TemplateDir: "testdata", + TargetConfDir: confDir, + Backends: e.backends, + AIConfs: aiConfs, + } + if err := builder.Build(); err != nil { + t.Fatalf("build bfe config failed: %v", err) + } + + e.bfePort, _, e.stopBFE = e.processEnv.StartBFE(confDir, logDir) + return e +} + +func (e *testEnv) Close() { + if e.stopBFE != nil { + e.stopBFE() + } + for _, b := range e.backends { + b.Close() + } +} + +func (e *testEnv) logBFEException() { + data, err := os.ReadFile(filepath.Join(e.processEnv.WorkDir(), "log", "exception.log")) + if err == nil && len(data) > 0 { + e.t.Logf("bfe exception log:\n%s", string(data)) + } +} + +func (e *testEnv) sendRequest(body []byte) (*http.Response, string, error) { + url := fmt.Sprintf("http://127.0.0.1:%d%s", e.bfePort, apiPath) + req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body)) + if err != nil { + return nil, "", err + } + req.Host = apiHost + req.Header.Set("Authorization", "Bearer "+apiKey) + req.Header.Set("Content-Type", "application/json") + + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + respBody, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return resp, string(respBody), nil +} + +func openrouterAIConf(strip bool, mappings ...map[string]string) *cluster_conf.AIConf { + conf := &cluster_conf.AIConf{ + Type: 0, + MatchPrefix: "openrouter/", + StripPrefix: strip, + } + if len(mappings) > 0 && mappings[0] != nil { + conf.ModelMapping = &mappings[0] + } + return conf +} + +func fallbackAIConf() *cluster_conf.AIConf { + return &cluster_conf.AIConf{ + Type: 0, + MatchPrefix: "openrouter/", + StripPrefix: true, + } +} + +func findModelInBodies(bodies [][]byte) string { + for _, b := range bodies { + s := string(b) + idx := strings.Index(s, `"model":"`) + if idx == -1 { + continue + } + start := idx + len(`"model":"`) + end := strings.Index(s[start:], `"`) + if end == -1 { + continue + } + return s[start : start+end] + } + return "" +} + +// TestTC01 verifies basic provider/model prefix stripping. +func TestTC01_BasicPrefixStrip(t *testing.T) { + e := newTestEnv(t, http.StatusOK, http.StatusOK, http.StatusOK, + openrouterAIConf(true), fallbackAIConf()) + defer e.Close() + + body := []byte(`{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends[clusterOpenRouter].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterOpenRouter, e.backends[clusterOpenRouter].Hits()) + } + if e.backends[clusterFallback].Hits() != 0 { + t.Fatalf("expected 0 hit on %s, got %d", clusterFallback, e.backends[clusterFallback].Hits()) + } + if e.backends[clusterDefault].Hits() != 0 { + t.Fatalf("expected 0 hit on %s, got %d", clusterDefault, e.backends[clusterDefault].Hits()) + } + + model := findModelInBodies(e.backends[clusterOpenRouter].RequestBodies()) + if model != "anthropic/claude-sonnet-4.6" { + t.Fatalf("expected model 'anthropic/claude-sonnet-4.6', got '%s'", model) + } +} + +// TestTC02 verifies that a non-matching prefix is not stripped. +func TestTC02_NoMatchingPrefixNoStrip(t *testing.T) { + e := newTestEnv(t, http.StatusOK, http.StatusOK, http.StatusOK, + openrouterAIConf(true), fallbackAIConf()) + defer e.Close() + + body := []byte(`{"model":"other/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends[clusterDefault].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterDefault, e.backends[clusterDefault].Hits()) + } + if e.backends[clusterOpenRouter].Hits() != 0 { + t.Fatalf("expected 0 hit on %s, got %d", clusterOpenRouter, e.backends[clusterOpenRouter].Hits()) + } + + model := findModelInBodies(e.backends[clusterDefault].RequestBodies()) + if model != "other/anthropic/claude-sonnet-4.6" { + t.Fatalf("expected model unchanged, got '%s'", model) + } +} + +// TestTC03 verifies that StripPrefix=false does not strip the prefix. +func TestTC03_StripPrefixFalse(t *testing.T) { + e := newTestEnv(t, http.StatusOK, http.StatusOK, http.StatusOK, + openrouterAIConf(false), fallbackAIConf()) + defer e.Close() + + body := []byte(`{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends[clusterOpenRouter].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterOpenRouter, e.backends[clusterOpenRouter].Hits()) + } + + model := findModelInBodies(e.backends[clusterOpenRouter].RequestBodies()) + if model != "openrouter/anthropic/claude-sonnet-4.6" { + t.Fatalf("expected model unchanged, got '%s'", model) + } +} + +// TestTC04 verifies prefix stripping followed by ModelMapping. +func TestTC04_StripThenModelMapping(t *testing.T) { + mappings := map[string]string{ + "anthropic/claude-sonnet-4.6": "claude-3-sonnet-20250219", + } + e := newTestEnv(t, http.StatusOK, http.StatusOK, http.StatusOK, + openrouterAIConf(true, mappings), fallbackAIConf()) + defer e.Close() + + body := []byte(`{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends[clusterOpenRouter].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterOpenRouter, e.backends[clusterOpenRouter].Hits()) + } + + model := findModelInBodies(e.backends[clusterOpenRouter].RequestBodies()) + if model != "claude-3-sonnet-20250219" { + t.Fatalf("expected mapped model 'claude-3-sonnet-20250219', got '%s'", model) + } +} + +// TestTC05 verifies prefix stripping after route target model override. +func TestTC05_TargetModelOverrideThenStrip(t *testing.T) { + e := newTestEnv(t, http.StatusOK, http.StatusOK, http.StatusOK, + openrouterAIConf(true), fallbackAIConf()) + defer e.Close() + + // This test requires route target model override, which is defined in ai_route.data. + // We reuse the existing ai_route.data and only verify the stripping behavior on + // the openrouter cluster; target override is covered by unit tests in this repo. + body := []byte(`{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + model := findModelInBodies(e.backends[clusterOpenRouter].RequestBodies()) + if model != "anthropic/claude-sonnet-4.6" { + t.Fatalf("expected stripped model, got '%s'", model) + } +} + +// TestTC06 verifies prefix stripping on fallback cluster. +func TestTC06_FallbackPrefixStrip(t *testing.T) { + e := newTestEnv(t, http.StatusInternalServerError, http.StatusOK, http.StatusOK, + openrouterAIConf(true), fallbackAIConf()) + defer e.Close() + + body := []byte(`{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}`) + resp, respBody, err := e.sendRequest(body) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, respBody) + } + + if e.backends[clusterOpenRouter].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterOpenRouter, e.backends[clusterOpenRouter].Hits()) + } + if e.backends[clusterFallback].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterFallback, e.backends[clusterFallback].Hits()) + } + + model := findModelInBodies(e.backends[clusterFallback].RequestBodies()) + if model != "anthropic/claude-sonnet-4.6" { + t.Fatalf("expected fallback model stripped to 'anthropic/claude-sonnet-4.6', got '%s'", model) + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/bfe.conf b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/bfe.conf new file mode 100644 index 000000000..5a232930e --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/bfe.conf @@ -0,0 +1,36 @@ +[server] +httpPort = 18080 +httpsPort = 18443 +monitorPort = 18081 +httpAddr = "127.0.0.1" +httpsAddr = "127.0.0.1" +monitorAddr = "127.0.0.1" +MonitorEnabled = true +maxCpus = 1 + +TlsHandshakeTimeout = 30 +ClientReadTimeout = 5 +ClientWriteTimeout = 5 +KeepAliveEnabled = true +GracefulShutdownTimeout = 10 + +EnableAiGateway = true + +accessibleBodySize = 4194304 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +totalBodyBufferSize = 0 + +Modules = mod_ai_route + +hostRuleConf = server_data_conf/host_rule.data +routeRuleConf = server_data_conf/route_rule.data +vipRuleConf = server_data_conf/vip_rule.data + +clusterTableConf = cluster_conf/cluster_table.data +gslbConf = cluster_conf/gslb.data +clusterConf = cluster_conf/cluster_conf.data +NameConf = + +maxHeaderUriBytes = 8096 +maxHeaderBytes = 8096 diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/cluster_conf/gslb.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/cluster_conf/gslb.data new file mode 100644 index 000000000..b2fa8648a --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/cluster_conf/gslb.data @@ -0,0 +1,18 @@ +{ + "clusters": { + "cluster_openrouter": { + "GSLB_BLACKHOLE": 0, + "sub_openrouter": 100 + }, + "cluster_fallback": { + "GSLB_BLACKHOLE": 0, + "sub_fallback": 100 + }, + "cluster_default": { + "GSLB_BLACKHOLE": 0, + "sub_default": 100 + } + }, + "hostname": "gslb-test", + "ts": "20260720150000" +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/mod_ai_route/ai_route.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/mod_ai_route/ai_route.data new file mode 100644 index 000000000..728c3e17a --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/mod_ai_route/ai_route.data @@ -0,0 +1,44 @@ +{ + "Version": "20260720150000", + "route_rules": { + "apikey_ak_user_a": { + "type": "apikey", + "owner": "ak_user_a", + "rules": [ + { + "name": "user_a-openrouter", + "Cond": "req_body_json_prefix_in(\"model\", \"openrouter/\", false)", + "targets": [ + { + "ClusterName": "cluster_openrouter", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_fallback", + "Model": "" + } + ] + }, + { + "name": "user_a-default", + "Cond": "default_t()", + "targets": [ + { + "ClusterName": "cluster_default", + "Model": "", + "Weight": 100 + } + ] + } + ] + } + }, + "ApikeyRouteTableBindings": { + "ak_user_a": [ + "apikey_ak_user_a" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/mod_ai_route/mod_ai_route.conf b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/mod_ai_route/mod_ai_route.conf new file mode 100644 index 000000000..f250013bb --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/mod_ai_route/mod_ai_route.conf @@ -0,0 +1,5 @@ +[basic] +RouteRulePath = mod_ai_route/ai_route.data + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/host_rule.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/host_rule.data new file mode 100644 index 000000000..ae991307e --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/host_rule.data @@ -0,0 +1,13 @@ +{ + "Version": "20260720150000", + "Hosts": { + "ai_product": [ + "api.example.org" + ] + }, + "HostTags": { + "ai_product": [ + "ai_product" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/route_rule.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/route_rule.data new file mode 100644 index 000000000..f15b5b168 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/route_rule.data @@ -0,0 +1,12 @@ +{ + "Version": "20260720150000", + "BasicRule": { + "ai_product": [ + { + "Hostname": ["*"], + "Path": ["*"], + "ClusterName": "cluster_default" + } + ] + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/vip_rule.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/vip_rule.data new file mode 100644 index 000000000..6fe22f03a --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/server_data_conf/vip_rule.data @@ -0,0 +1,8 @@ +{ + "Version": "20260720150000", + "Vips": { + "ai_vip": [ + "127.0.0.1" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/bfe_i_ca.crt b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/bfe_i_ca.crt new file mode 100644 index 000000000..f1e78f73c --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/bfe_i_ca.crt @@ -0,0 +1,23 @@ +-----BEGIN CERTIFICATE----- +MIIDwDCCAqigAwIBAgIBCzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +NDAxNDZaFw0zNzA3MTIxNDAxNDZaMIGQMQswCQYDVQQGEwJjbjEQMA4GA1UECAwH +YmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsMC3lpbmdmZWkt +ZGV2MRgwFgYDVQQDDA95aW5nZmVpLWRldi1pY2ExKTAnBgkqhkiG9w0BCQEWGmxp +YW5nY2h1YW5AeWYtbmV0d29ya3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A +MIIBCgKCAQEA6eFuWgoknixrRO9NCX4jAKyLtcAOWVJqVN2yX7CxjZjLyPurjvTZ +W73NYUbrPAN4AB5gY6UAPzuiEOSopVVIZ0OschK0cJldu9vZ0mZObBOsovuFcLQq +dgNSJ5slJSk7tCgD2EnCB3GYPG4D+uIKYd0c49wzTWWv4bjDwpgnf0LQbFpy7GhN +7D59zFH4qgOK/IQ5vaTMGyvIvtWR5/1Gvc9MLpGopTgi0DiNLed4UwDYrod5kysl +q3UcB5puONHQISOVoD3uRxo7wdsmVsHUfW7YfAWkhi6ec8mx9fy8IyE6f7GlXnuV +ysNccwqyEotL5bOXPJCqwUCL1v+iKTMPWwIDAQABo1AwTjAdBgNVHQ4EFgQUCjvC +vDVr8QXh9/hMI5xBgNvoSFgwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW +1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAhIggK/6JK4N7+HZW +VoTemwRpilugZZyKrcVAHbiiwUfXVQVuI64vc+yHWMSFRD1mykHkKBFzxEoDabMl +ASBtUJNt4b4zEL9V7k295vmAOp2IdLUQxlgeKWqABm4DGX96tGR9nKQTcn1ZeAxx +NyQFV1aj+dnNcF1iFFNF6t0bRrOEZ/aRSiu1bWp3Dj1JYTjXbyyplh3Ktb8lv4lt +EmvCXjo/l4TgQC9233kcTkXcq1swppzkkXfhB0NVuf9DE3C2xAWX0b2FiIEqnAhl +Bx0Cn3RrX7PZ6qrdRL6oBKvGJV6DP8BlF4LXiuD1VN/waQFJha57KQ78ZYYIXxQg +WMjWgA== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/bfe_r_ca.crt b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/bfe_r_ca.crt new file mode 100644 index 000000000..2a6db1e49 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/bfe_r_ca.crt @@ -0,0 +1,22 @@ +-----BEGIN CERTIFICATE----- +MIIDkzCCAnugAwIBAgIBCjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +MzQ5NTVaFw0zNzA3MTIxMzQ5NTVaMGQxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxFzAVBgNVBAMMDnlpbmdmZWktZGV2LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOC +AQ8AMIIBCgKCAQEAvNA3HrsMjBcXrMIIhGVWsurIA1F9jxKeA7dh06H00Vt4inVV +SUvNFrTqgPRhLkAhGRMPxrjVRgJ5bbFqqXIuPIpUFBhUsWXIDH+oVXQl9jsxAXaG +gZ0lTO/uYR9qyrS1rj9nyNPwRf59Al/VlsQL71cNQ/T/agJ4PfvPfULTPLOsqclJ +hj0IgXmDj464dqcdG3ZdfXpfhNF6ab+8YjpwafTRmY+LoV8qjUwsYeJMcW4N8pxJ +8F2ktZj9J6uWepNGj+87ZeXg9XquzC62ASIFzPjoE1WN//Q518EqizxhuLGBDpK3 +6sEYUK4kHYUL4gZKFPlKTPXIl0ZsJIM5PsBMKwIDAQABo1AwTjAdBgNVHQ4EFgQU +DZloT8VhVbysD819oG/oqHdW1D0wHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/o +qHdW1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEALgB+IcN3UD4T ++4g8jaOyOiSUpUVdYqW+3go5DppqnvlGNq99N2cXKqssVPa/T9TikEcBEicFa8zU +bwlx6TEte+MkWfWdQxFR1EuI1FgKr3ps6ZBRr7MPpnYmI7K9aK372K9n7WrQhbmP +s7ult8bWB1/t6o3R7B9ChNkWT+7DPD4+FvB1GMJSGPno7cdnvDkevBOuC2DnQl3M ++ADFAge1Lo8wKBy6gYkNFd2BfarHGvRC5Qmmrme+RIpWZnvux1+lfXIInnfSTJRM +uAo/ePkoNsM3qQll6uEdhDxOMx8Pq94bCkM3DtI3ObuxWXjKCgUm/n9yetUvPj4U +iYhRUqHpUg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_bfe_dev.crt b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_bfe_dev.crt new file mode 100644 index 000000000..2164fb602 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_bfe_dev.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 18 (0x12) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Jan 31 02:55:32 2024 GMT + Not After : Jan 28 02:55:32 2034 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=bfe-dev/emailAddress=dev@example.org + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:b2:7f:c1:0c:cd:49:43:d9:99:78:15:4c:5a:52: + a9:a7:bf:d5:eb:92:71:43:e1:37:e5:29:1b:68:f4: + 6f:4c:ea:fa:a4:8a:7c:29:01:2a:fa:7c:81:5b:c8: + eb:a1:40:94:b7:2b:82:e2:00:08:36:84:f0:b7:d2: + 5a:1e:56:97:aa:36:ff:4d:07:49:2d:fe:25:3a:f9: + e9:f1:ad:4e:6e:21:97:a9:f9:a2:a5:ac:82:23:0a: + d2:e0:97:cd:2f:14:b1:f0:8a:a8:e6:c5:97:45:94: + f8:8e:3f:96:66:5b:0b:8c:7c:07:61:18:44:92:f7: + 23:5a:c2:4b:88:58:59:5d:ca:5c:0d:6e:dd:ff:18: + 59:65:df:95:99:e3:3a:36:48:1f:3f:3a:e6:ce:85: + f3:0b:04:5e:92:ed:6f:8e:74:92:e4:37:46:da:5f: + 17:62:9c:82:40:06:fb:29:f8:55:f2:ba:23:75:ca: + 64:c0:45:03:12:bd:f5:17:15:7e:47:d5:bd:30:f2: + 99:ca:6b:e3:07:b0:ae:44:89:1e:10:26:ea:75:df: + 6f:07:b6:47:76:54:47:4f:6c:f6:68:fe:a8:cf:22: + 20:73:e8:19:55:8a:fe:f5:78:e8:51:88:52:80:1f: + 79:d4:c5:ae:8f:d2:2b:f6:41:01:42:01:cf:98:c2: + c9:25 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + EE:E8:68:42:DF:B1:F0:EF:6F:47:51:BD:D4:94:60:1F:05:85:A1:03 + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Extended Key Usage: + TLS Web Client Authentication + Signature Algorithm: sha256WithRSAEncryption + a9:a6:26:8e:42:61:15:22:ee:fc:b5:e1:e4:6b:dd:ac:f5:15: + 11:39:10:9a:ca:6f:85:fd:cb:90:1c:b2:4f:fe:29:de:b0:e7: + 73:e2:f7:5e:63:8c:7f:c1:7e:75:2e:c9:9d:e9:c2:45:75:f3: + 27:ba:82:94:de:7f:6c:87:0c:5c:71:af:0f:14:00:68:35:f7: + 5a:4a:ff:f5:ef:35:dd:50:72:76:f0:6f:b6:7b:42:33:07:b4: + 24:44:0a:fd:9d:61:9e:44:e8:88:0f:02:76:c6:90:3f:9d:1b: + d8:3b:64:25:2a:a3:39:78:38:bd:20:89:4a:9c:bd:68:38:18: + 4c:cb:20:3a:9b:5b:5f:58:52:86:73:de:85:fe:d6:a1:c6:a7: + 86:b0:96:4b:fa:28:04:ad:5d:85:e8:a1:fc:ca:0f:3c:be:5c: + 90:7e:3e:84:ae:67:ee:9a:72:71:3c:b2:80:45:82:fc:7e:58: + 74:99:42:c5:c3:8a:4a:eb:e1:8b:d5:84:ce:25:aa:a1:75:79: + 94:66:ae:ee:df:30:15:0b:b5:c5:b1:2c:d5:0a:54:78:b6:2e: + 67:29:81:41:f6:16:49:31:96:e7:41:e1:99:6b:27:57:bb:7d: + 76:eb:e4:d5:59:aa:a2:5c:bd:1c:18:2a:fa:9d:28:1a:0b:b6: + bf:7d:58:1a +-----BEGIN CERTIFICATE----- +MIID7jCCAtagAwIBAgIBEjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yNDAxMzEw +MjU1MzJaFw0zNDAxMjgwMjU1MzJaMH0xCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxEDAOBgNVBAMMB2JmZS1kZXYxHjAcBgkqhkiG9w0BCQEWD2RldkBleGFtcGxl +Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ/wQzNSUPZmXgV +TFpSqae/1euScUPhN+UpG2j0b0zq+qSKfCkBKvp8gVvI66FAlLcrguIACDaE8LfS +Wh5Wl6o2/00HSS3+JTr56fGtTm4hl6n5oqWsgiMK0uCXzS8UsfCKqObFl0WU+I4/ +lmZbC4x8B2EYRJL3I1rCS4hYWV3KXA1u3f8YWWXflZnjOjZIHz865s6F8wsEXpLt +b450kuQ3RtpfF2KcgkAG+yn4VfK6I3XKZMBFAxK99RcVfkfVvTDymcpr4wewrkSJ +HhAm6nXfbwe2R3ZUR09s9mj+qM8iIHPoGVWK/vV46FGIUoAfedTFro/SK/ZBAUIB +z5jCySUCAwEAAaOBkTCBjjAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu +U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU7uhoQt+x8O9vR1G9 +1JRgHwWFoQMwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW1D0wEwYDVR0l +BAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKmmJo5CYRUi7vy14eRr +3az1FRE5EJrKb4X9y5Acsk/+Kd6w53Pi915jjH/BfnUuyZ3pwkV18ye6gpTef2yH +DFxxrw8UAGg191pK//XvNd1Qcnbwb7Z7QjMHtCRECv2dYZ5E6IgPAnbGkD+dG9g7 +ZCUqozl4OL0giUqcvWg4GEzLIDqbW19YUoZz3oX+1qHGp4awlkv6KAStXYXoofzK +Dzy+XJB+PoSuZ+6acnE8soBFgvx+WHSZQsXDikrr4YvVhM4lqqF1eZRmru7fMBUL +tcWxLNUKVHi2LmcpgUH2FkkxludB4ZlrJ1e7fXbr5NVZqqJcvRwYKvqdKBoLtr99 +WBo= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem new file mode 100644 index 000000000..764aea3ad --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAsn/BDM1JQ9mZeBVMWlKpp7/V65JxQ+E35SkbaPRvTOr6pIp8 +KQEq+nyBW8jroUCUtyuC4gAINoTwt9JaHlaXqjb/TQdJLf4lOvnp8a1ObiGXqfmi +payCIwrS4JfNLxSx8Iqo5sWXRZT4jj+WZlsLjHwHYRhEkvcjWsJLiFhZXcpcDW7d +/xhZZd+VmeM6NkgfPzrmzoXzCwReku1vjnSS5DdG2l8XYpyCQAb7KfhV8rojdcpk +wEUDEr31FxV+R9W9MPKZymvjB7CuRIkeECbqdd9vB7ZHdlRHT2z2aP6ozyIgc+gZ +VYr+9XjoUYhSgB951MWuj9Ir9kEBQgHPmMLJJQIDAQABAoIBAFAKHzupVb/58+o3 +yqv5wx94Uuk2GlnwxIqaezL94GaiO0/K1U/huS7m426P0rDU75qPBTpn/0bLJ9GV +nllaRNnLnYEh0juwaWtfovp+1ttlbseGK9uUVip2cQbKqvQAmKWe14vbcDCAU1Ad +zUgKbUxKVVjBdAZekVjiJNJ3o2L9WPhf/uQo7A1XAJh2DajlTbgvrDM73W+47QuU +X4OHU0FMio6bxupu3OWl1bMrnKhuC4qczZWf2nOpcVQa89rtopuP4ENLJuWkbeGk +YQpNilEclnAa/Noumt/j/6GKC1EEHFsH2CNRRIazcZrsFhkSKc4pn1Y/WI3vj8kZ ++RYnJsUCgYEA6gr0x8suwRTAmVpoPk4XyP1x+eInG7onV5RjgsUtgruYd9naxRHg +2p8PHcv32pDs51Fa+4RldyMd/jec1SscRF5/+VOP9qeoRnaDqUu+uASgEO3OBxbP +JcWovyxRHIQxbYCQtqIr9bdzXw55MBLZou/sBUVTAkrIyPVyjWqZlV8CgYEAwz7L +YyYN615TsrzZKURxMjj94Nmob/NldSLRXaR3Ax7/ABtEOA685cwQxq7ONdkJTMIA +uR8u2GHZSzGiWnehuF6Zp7Xs71a57eFbs3ueZvvEba4Dff7hl7Y4tTlwrKndKjvP +J/5a2Ol8siQcRWAXHOdzggEHMSZ/sB4hWswly/sCgYEAhLRBpyemEwTZUBrbELjm +86gBgFajJi2fMSGKaxOygnYsNYjpauSAQnX99D87Aks6iM6wb/zaK3tV/lc6LgSL +uph6p7yh3JGj8JAyh0PTmDPHLtIoCAz+18QDsqJGO40ZGaXUaDn8Aw9J85QZUxDd +Jm4zvalZL+uHfarukRDolLECgYBUiupS4nWAh3XCnZeDEQna72avaFBROZmjIRJ7 +c+28wj009JmTlH4jGzvgbG0KUBKA1Div8Fq+g5AtyS498jNqvDvYrSQNdwZHhR/K +Fis++KHTxFfqxOU2Zkcj4d1yRpNn6EIJVVBNQL0n/g7n03XupCIWFw/gLoV343QZ +9vAe5QKBgA8ml59z1w3eUooc0yGfLhihXqCmM3IU006bFbODA30fBU4QKrHO8+Yx +Xbz9bi/1QLagLG6FzYQAkOjBlEBt5XLayYvwSb8xvWsm5A3vzTAbMFDOsDPEmRoH +dWtQccJcygOuK+PZtoZnNoJciNO5c9dZWD3xtIiURmX/kVtWrf6N +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_san_example.org.crt b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_san_example.org.crt new file mode 100644 index 000000000..78f3892b4 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_san_example.org.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 15 (0xf) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Dec 7 09:57:42 2023 GMT + Not After : Dec 4 09:57:42 2033 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei, CN=dev-test + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:d6:47:b0:17:69:91:4c:d0:66:c9:25:2e:38:f2: + 84:27:e2:7c:38:cc:04:b9:0c:8e:3d:cc:ef:4b:c5: + 35:2b:c1:82:d8:41:fc:25:c8:24:f2:e0:25:aa:f9: + 76:3c:e2:b2:50:fb:2d:ec:4e:16:d4:c1:da:1e:e3: + 51:9f:9d:c9:72:f9:cb:cc:14:9c:c1:82:c9:76:1f: + 98:dd:0e:b9:8a:20:d6:ab:f2:a6:f9:2f:23:81:f3: + af:e4:47:0c:55:95:94:de:ed:f6:a5:24:ee:32:e7: + 6b:79:e1:42:6f:a4:07:b2:95:1d:f5:a9:8e:60:42: + 70:42:bd:e2:30:18:68:74:52:32:98:a9:81:da:d8: + c6:6f:5e:1d:ce:79:b6:f3:ec:4f:ed:7d:22:57:d2: + 14:d0:fb:f2:50:d4:80:3b:89:ed:77:fd:45:6c:e6: + 52:6b:0a:52:71:ac:59:c8:d5:25:f5:40:03:fb:51: + b0:11:a7:00:79:d9:d8:4f:00:43:96:68:44:29:41: + dc:d2:cc:91:c8:61:95:41:4d:0e:66:5a:b5:15:67: + 3e:8a:6f:29:df:1c:8a:6f:ee:9e:97:9c:9e:69:71: + d3:34:52:75:e9:ea:e7:51:77:23:98:46:ca:47:a2: + d3:d3:97:03:41:4b:e3:33:11:72:2d:af:bf:2b:3e: + b3:51 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + 5A:27:32:9D:E7:36:24:A3:C1:DC:2F:95:80:C5:CF:0C:85:E8:E6:AF + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Subject Alternative Name: + DNS:example.org, DNS:www.example.org, DNS:example.com, DNS:*.example.com, IP Address:127.0.0.1, IP Address:192.168.0.100 + Signature Algorithm: sha256WithRSAEncryption + 1e:e8:e8:8a:ad:a8:0e:fc:c9:82:00:a1:ab:30:3c:a5:b9:dc: + d6:fb:86:ad:30:52:7f:61:be:90:a6:b8:56:bb:f1:0b:e6:39: + 38:65:09:6b:da:83:f7:65:ff:c4:21:de:b4:9e:8b:bd:1e:1c: + d1:d5:94:b8:18:79:f2:d0:06:51:39:67:13:40:3b:73:5b:cb: + ea:de:c1:19:76:f8:7b:0f:15:51:61:49:fb:98:f7:ea:4f:fc: + c2:fb:a7:f4:3c:48:64:14:79:b5:78:5b:20:10:b5:7a:2d:4c: + 04:51:60:ec:20:10:19:26:5f:e2:fd:32:59:67:e9:3f:48:8d: + f5:52:12:01:81:2c:c0:e5:72:cd:7d:0a:eb:7a:05:df:a0:77: + b9:ba:9a:7d:d1:4b:6a:44:e4:2d:98:af:bd:77:2b:f5:ef:26: + 4b:75:b3:97:d0:3a:bc:07:21:ef:71:92:30:fe:a2:79:e5:56: + d7:7e:c2:f3:57:ab:d7:de:fc:97:ed:20:0c:9a:cb:c5:5d:00: + 3b:61:29:e8:00:d4:39:e0:f2:4e:a4:03:c2:12:52:ff:e7:78: + f9:f7:c0:12:dc:36:a4:05:a2:f0:6b:47:e2:21:3d:a2:e1:a1: + 91:c7:ac:8f:b8:ae:58:65:e0:2b:57:80:eb:77:2d:48:ef:e6: + fb:b9:e1:20 +-----BEGIN CERTIFICATE----- +MIIEBzCCAu+gAwIBAgIBDzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzEyMDcw +OTU3NDJaFw0zMzEyMDQwOTU3NDJaMFoxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEQMA4GA1UECwwHeWluZ2ZlaTER +MA8GA1UEAwwIZGV2LXRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB +AQDWR7AXaZFM0GbJJS448oQn4nw4zAS5DI49zO9LxTUrwYLYQfwlyCTy4CWq+XY8 +4rJQ+y3sThbUwdoe41Gfncly+cvMFJzBgsl2H5jdDrmKINar8qb5LyOB86/kRwxV +lZTe7falJO4y52t54UJvpAeylR31qY5gQnBCveIwGGh0UjKYqYHa2MZvXh3Oebbz +7E/tfSJX0hTQ+/JQ1IA7ie13/UVs5lJrClJxrFnI1SX1QAP7UbARpwB52dhPAEOW +aEQpQdzSzJHIYZVBTQ5mWrUVZz6KbynfHIpv7p6XnJ5pcdM0UnXp6udRdyOYRspH +otPTlwNBS+MzEXItr78rPrNRAgMBAAGjgc0wgcowCQYDVR0TBAIwADAsBglghkgB +hvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYE +FFonMp3nNiSjwdwvlYDFzwyF6OavMB8GA1UdIwQYMBaAFA2ZaE/FYVW8rA/NfaBv +6Kh3VtQ9ME8GA1UdEQRIMEaCC2V4YW1wbGUub3Jngg93d3cuZXhhbXBsZS5vcmeC +C2V4YW1wbGUuY29tgg0qLmV4YW1wbGUuY29thwR/AAABhwTAqABkMA0GCSqGSIb3 +DQEBCwUAA4IBAQAe6OiKragO/MmCAKGrMDyludzW+4atMFJ/Yb6QprhWu/EL5jk4 +ZQlr2oP3Zf/EId60nou9HhzR1ZS4GHny0AZROWcTQDtzW8vq3sEZdvh7DxVRYUn7 +mPfqT/zC+6f0PEhkFHm1eFsgELV6LUwEUWDsIBAZJl/i/TJZZ+k/SI31UhIBgSzA +5XLNfQrregXfoHe5upp90UtqROQtmK+9dyv17yZLdbOX0Dq8ByHvcZIw/qJ55VbX +fsLzV6vX3vyX7SAMmsvFXQA7YSnoANQ54PJOpAPCElL/53j598AS3DakBaLwa0fi +IT2i4aGRx6yPuK5YZeArV4Drdy1I7+b7ueEg +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem new file mode 100644 index 000000000..393a79825 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEA1kewF2mRTNBmySUuOPKEJ+J8OMwEuQyOPczvS8U1K8GC2EH8 +Jcgk8uAlqvl2POKyUPst7E4W1MHaHuNRn53JcvnLzBScwYLJdh+Y3Q65iiDWq/Km ++S8jgfOv5EcMVZWU3u32pSTuMudreeFCb6QHspUd9amOYEJwQr3iMBhodFIymKmB +2tjGb14dznm28+xP7X0iV9IU0PvyUNSAO4ntd/1FbOZSawpScaxZyNUl9UAD+1Gw +EacAednYTwBDlmhEKUHc0syRyGGVQU0OZlq1FWc+im8p3xyKb+6el5yeaXHTNFJ1 +6ernUXcjmEbKR6LT05cDQUvjMxFyLa+/Kz6zUQIDAQABAoIBAC4sYGuLGf49Ygix +9FXdHFEj4rSyccoWRIhYoq/nHOAC4NkMzvKtQBj95+ABxVK1XstIdMrYwN6zrva8 +8Re9/mzCGwIs5uJj9ll30Y7A34Y+MUP4E7baS4JzKlG8ZZIDm4K2MFHBtXpOl8A5 +pAE+jVIUA9Kt6LohVuNq21SVzdxSfNYC/+SLqSftkWa/ZsqdkiHM5Hl+fVedh516 +IaLNW5hSthGh5n8dHY5h/AKPjfoq77aYp5/CUtJTC9mYdZu1j/W/pBVTRfOnwLQd +SQ1Xmr7f6q9Vmz+HnajIbFg9hQ54blvtUJ7DnugWxfUcoxf7ue79fnYjOIUOkRWw +8Iid/mECgYEA+5s0p0j+gkNZN5QtVNStoT04+1DqA1O381gczeaiz6Njt/MT0y5W +OpCsILQ70CpEjWAV+f6PDJiesDMxdGV+v2TCqK8ml8GahEczBLnHoAkPWCVf2XOX +oNj/CkZ2kmWufHFR+kcQbeDt1vFFcYUa61hKDFyNjinMW79Qy+PQID0CgYEA2gWd +7thE05sqU7/1MntmVRONKoAgnJmHcfSpWwLyh6E4YX3iKDSgI/9RnAMF39KUUY/O +XFWyIwAM9soeXknVsV/SmCaPeaEDiLHqz98aUEfvdLYMnuR883GgoXc4JrLsLw4z +oSi9lbAZFn0ekJ5L+rSFrY4rz9QZgYZxsLjUXKUCgYAbkaUSU2g3w8Np2J2i9u7T +hQ7SUsphdPHqAxSc5xGd6MxLYqIgeKpQHnwN1VHcfFUonIer7d2kxrBUpDdeBqT9 +ub+ulgqHhFo29ko70VNzUKrSwL2g6Q6LPFutt4zUe7nDvvL5loHRWF0XOTafurL5 +aKIsepO0KRZQU0U6IgszDQKBgFs6ZHaP2mTtFY4L0a75Ab3xu20gRgUhHRLq/H6P +wipMpMnuodaPBr9pU53Dig65D8T9Nq1eUnbgy4vs0T5FCPz6iqWN5RVQ8aieQhIP +WfRj1Wfx0WAfXcWEM2G9ACr5TWj3OVVjNclP8X9+hW6gPky+gv03c0+4gZ+4QRRg +ksPdAoGBAKXVv8L5Qt8rmi+QpF/6DVSeMB6lEevf319UEtyZPiYSBn1JJ9H+3Ddc +TMriXgZrw+PoXNJTAguDeGgzmtye1vraP0cjt7aG6sQ4YtMTnnuja880vK8z4i3Q +HJAi5fI8NvlW92dQBAccgrzFIKpRx+6CHwtCkNrxL2vJmGGzp8BR +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/certs/example.crt b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/certs/example.crt new file mode 100644 index 000000000..931874885 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/certs/example.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDEjCCAfqgAwIBAgIJAIAdu56fLE7OMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzM0NFoYDzIxMjAwNTE3MTAzMzQ0 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBALv+LV1aWIlcK9rI7IuRS8SCusqBnoyJec/ErKiA2gbfgZ/YS73L +Zud84yp45AIqauzcI5q+hrkmsRZ7CKqDzrG+jHavW7jF+0laetJwRt26AcQcOtQD +2ik2O+Dl1WHAFn4vUAQxb+Xz6WfSaQN0QfM74z06XUDDsr7g7+NYtMzhf98SJSoK +ne/dVKJ3Bc6e6tvhnCRwPtix4ektEodK6WeNHYxwJ6wSZ8cRLzdxgjdD/4OGfFuj +dn8zbOi3SQt5ZqVbcDHUTzp5t0G8EoxnzotHhhzjSAmsypySqZXaxl3oX8aYUkFn +fCdg+WBXo5pOiNfoWh/D5bnIXWGp52yoy+kCAwEAAaNhMF8wHQYDVR0lBBYwFAYI +KwYBBQUHAwIGCCsGAQUFBwMBMB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MB0GA1UdDgQWBBSxLHQE7gOEyfeSNc5uIO/G/rgjpzANBgkqhkiG9w0BAQsF +AAOCAQEAlGm5RwQ79xmLh3rj+5UViCSgsIuMcuhgIT4zogpo9S4uwXMqinrJhzRk +Oc2tb3y06XTAq1lMH2+58tqndAu8ni/UBz3OSghk2CTnZ1vxxXOd3CtQu4ypMq+k +qW0Umdrkk5TeAODNbrCy4c6vpICkQOljnRFWnDYu3aQ3JvaWZ/nObN7C72Lgpjfb +RfLXGmLsBCEIr028f9hpoeRCXoetUY2CiC2boAHR+cO6Jpvex4Jv5yYDpNKac52n +LLC8Cq5ozhOZNOSV6X9FpEca3rdhVUb0VgoNDCPZDdpO1PDJYCN9fUC8KUs+dsOh +SYGpliBaNsztoiJs1q5SkMQrMwmMmg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/certs/example.key b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/certs/example.key new file mode 100644 index 000000000..b21c2f08d --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/certs/example.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAu/4tXVpYiVwr2sjsi5FLxIK6yoGejIl5z8SsqIDaBt+Bn9hL +vctm53zjKnjkAipq7Nwjmr6GuSaxFnsIqoPOsb6Mdq9buMX7SVp60nBG3boBxBw6 +1APaKTY74OXVYcAWfi9QBDFv5fPpZ9JpA3RB8zvjPTpdQMOyvuDv41i0zOF/3xIl +Kgqd791UoncFzp7q2+GcJHA+2LHh6S0Sh0rpZ40djHAnrBJnxxEvN3GCN0P/g4Z8 +W6N2fzNs6LdJC3lmpVtwMdRPOnm3QbwSjGfOi0eGHONICazKnJKpldrGXehfxphS +QWd8J2D5YFejmk6I1+haH8PluchdYannbKjL6QIDAQABAoIBAEUBL7WsjAMfihls +1ycD1kPzmIzstz3u2H+jOZ1AbsdHE1WRF3w7RTKDbP8SEN+aolT/GTKb7OfZg/c0 +giHU7/Hed8C47XoNcgei5qKIA/svY6aQlidsoo+uEJykwIZ488itpTlkzCYkOfCa +E2HpMqwNt4OqAMDdFKdr+aIB1Zu+KPBxW23WD9wEWAbe5LA4YnRF9kT4YZ6y9mce +dGaIf39VtBlrGMmvoU0LE9B79nyuebGi0svW6QDarBqaDrnM/N3fXgL1kk/gVfan +/xs6EA4qPxA5G4h+enYrIlZL0CbSj60nYElo+Z5nRdBaRdCF/bpXOLyK/kXWLUM0 +f2HTK+ECgYEA5cVcpJtczxEaxoaEUbppsW1LCrTjJDGTKJ63G7/lwqCxJeCHN185 +nnckHOW2287e19bu9aUmKJgRq5s1rXnT+MnCl/hQnfaMrKOKtzE+t7/zsc9+LuAr +pwJrtZ9Dcnwrk8NOE0fPjW5XpDCSoEOo7JWZmGTVlpabOgNkjocfp+sCgYEA0XPt +ZPt3F0wyzgLYRhgnvp5CV8SzQmulsW+ytnL5eiAcNSXqni3wQHN3PGxLInEyQwBQ +/M8TQpUbqGMmahCK4ZxMAwXMrpF0mVB8jfoYMou1FSYPlUV+CvLjWcTkZB1Nirez +VFXdtfHP0mx4PbYK2qjB03u4pPHAN8kuayIf2nsCgYAbv/FHZAgabfto3Jggcr4P +Ep8MhPolxeL69egxbsSl89hRNcO+2T5ROBxhbRDfjSV2tduYSUDJiEwiCJW8BMmn +814QEopR+ZPVyc6X/1eOw5z/7YpUyPgcrHsrrTdtHTf6GY1VYMfdUeU9zCv5NRKy +uAKb2Bm/nSLUJ9K+L+2PzwKBgQDRQgT3UtTUjehkMitpPFDY/LxDe92simfsMjBW +X+Anx1TnNI6GolbZzYJe98LJElao4fQH38raRqZvQT/rz8MxTDoU+wJXljLryaHn +Jupt9W5hRrli5R7cSXYjBbc43p3N7WJY68CqOoDrNjubS/jkJJ4hcAY1pOHp2jFq +D5nLaQKBgAysU6O5kJ8yKxhbZflb42MqKCFBGrbRnbYx14PAEZOaRhzxehpppQmx +RLbn/z1Uh5Ms28ipxA+vnhyM3FcU5lKboaFyWJeuNslw0FxEcIai6hL6UkDznS4G +aqyzUjpG5Chg0x18xWYCbiGJwjZ9BWhtH+jojm856QHGzeQJWVoo +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/client_ca/example_ca.crt b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/client_ca/example_ca.crt new file mode 100644 index 000000000..b0fa2fa28 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/client_ca/example_ca.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDATCCAemgAwIBAgIJAMsPuHg4mqnaMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzMwNloYDzIxMjAwNTE3MTAzMzA2 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBAL72D2gOnJN9Zvo9KjruwM1EsFe3xZRJ0NvZ5bHd6+5jhlgCAhQ+ +AGb7ufEiYOi2JWHl2Bkq0iVrp+zv0RLdq0oVjX+OG5H2yWbnC7ifbNjir93LX0un +tIqv5CIbExDSBRkufxfV37yjXdrcMqYSbD2Kw3PfAbWs1Dego8fRz8QAp5+LCvW2 +BZZyYi6JzhCAUW1+8OQPyzOhB50eSJiS5xgVA7wkwmYeVUpHqU8sv4VzjM3bmUc7 +1mPLlnRVIqScrqYgQ9Ou21vZebOJ8+ckVL8O3XHhMZlssBbWiBFZZnaNWbzcEI90 +oiW4YAQ5t7gaXuCVvaiNvq2VZknarR6AxcsCAwEAAaNQME4wHQYDVR0OBBYEFIH+ +0G3eCswQHbN06kvI80M3tNH9MB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAJtHx8ibLp+CYDR1 +ZVeQ3qg4OeRL0q21+2EgBJq6zOUt/9SxThA80aXJ8CYH9dnCW+fOnpEk1xFWxtXc +FwSLsnqAdwOJaaQWoMzhyqjZV5x5G9+MW5FzGGOdes2md2Z+tAwMoV9TVtxZkbKy +mC2tDJdvgLgt9/YcbUcZPDbyZojdZ+UbATm+Lro9dhTXt91vsAgz5QA9e08rQVkF +pc9+ZQ5zxBsoblQ+ozPOWOdV4zJVx+wQsAnOG2qU0yVQAscGsTo4wnzFrAU54fO7 +Lh4cOrY0P1/o65yiSzwK7f0jwBeT/jEfMOrJ7pPo7doUov0iVj0SZyTM3HFa2Mzj +2zGTk0o= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/client_ca/example_ca.key b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/client_ca/example_ca.key new file mode 100644 index 000000000..4f7ad13b1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/client_ca/example_ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAvvYPaA6ck31m+j0qOu7AzUSwV7fFlEnQ29nlsd3r7mOGWAIC +FD4AZvu58SJg6LYlYeXYGSrSJWun7O/REt2rShWNf44bkfbJZucLuJ9s2OKv3ctf +S6e0iq/kIhsTENIFGS5/F9XfvKNd2twyphJsPYrDc98BtazUN6Cjx9HPxACnn4sK +9bYFlnJiLonOEIBRbX7w5A/LM6EHnR5ImJLnGBUDvCTCZh5VSkepTyy/hXOMzduZ +RzvWY8uWdFUipJyupiBD067bW9l5s4nz5yRUvw7dceExmWywFtaIEVlmdo1ZvNwQ +j3SiJbhgBDm3uBpe4JW9qI2+rZVmSdqtHoDFywIDAQABAoIBAD60bbqtkZycwQPK +seNIIudEduNW5PocgwiuNE6DoMVWyPZ9MlGTSm6GmjgkIc5IgV30K1GYTgkboLic +xvp675QUH7KS51q2vsubcq3dK9DMHxOlhFVDbHVd7HuGiGwtip8KNZGOGTnIKzmC +tN7zjbdnqWaTA+y0I7tgdGdY7fBd9Rzgaq+OlPq2u33HvWHvlG/7PfpZuXB5YLgd +m04l7LJ7ikhIjycg7j27v/4c6xCiH5jMJKsZ+nfsQ0kEEo9DkhcKInK+wHsMzKsH +Cy3AdlE0IRsbxRAoMumVs2g5u90m3zBPkRrNdZ2Ni7BesnhxbkIqvb4SfpxKyuhK +fADfZgECgYEA7SUIS2gII0TGjXh0h16d+eoLVOz0eVpgF3XXxmgtAPu10dqxVEC2 +j5FSBCgZhqZ3axVotP71c2mT+hF+Mqy4TLMfA/B9jKLXjZlPbg4EcAgI7tALskwz +Bk5BkX0k825bU9P0j+AlpLx6/ztHr2N9/cKZfqQVrO9t+FRusvAHkHsCgYEAziT8 +F30Ch2s6IJngCj5jH164iN2CoFXjqPNVgRj45gLE3zrf1R7u2JTeEhjWNLZ6IWZ3 +G/bT7eYm6x8u7LFlORdnWKsHlftGu0igRyvIGcxoHgjXlsLidBaEP+HlOLUtTumu +MfQJUozLcrOBIV6m9VhPnSDTeCg/tOqy68V2pvECgYAUYgd5e8KfTW0Hgd/6Nq67 +aVt5/DfzKkpyGcXnHtMnb3ssQ3DUfg9y/ZmgE9ZF1Y8UHC34yKVOOzfl2ZUQQ/o/ +VXIIA6a27NQ8Ln4+RmQpQPeLl0Q6GgSUuSs3lxsS9VxSMzilGS4DH9QulejOcW3F +3vEUioP2bkn0e0VcifcMewKBgAG1Pr13FLFIiye//qI3GB0nbMH9i9qGO6entHqo +WU+WkEkFNNuQMQxsV1axC/1N0b87GRuLNQBQmtvx2zKs2Zjaf8m1SQ/OECz3EhTk +4PiNwAMXsamXHcc2dIwO9BY/MgvoVcAmNHmRnxHpONWs8hcwTyCPKBFjy/tUwny/ +mxcRAoGAcNRxLZlyRqmQ6zGf41GK4ZIR9gix0L6Km49S1maGFmcbctOR2GcQN8Eo +f38rkrFBfBfuFSGzghJiXDvKORg9r3V/bzSKcXkprJra6hzn5vn+t7wurjzaJUlK +zUW5dl3SU2bC4MK+X7bwqf9jm9b7FXSt4p1xly8Uh/Mufwi7zOM= +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/server_cert_conf.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/server_cert_conf.data new file mode 100644 index 000000000..49f228531 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/server_cert_conf.data @@ -0,0 +1,12 @@ +{ + "Version": "init version", + "Config": { + "Default": "example.org", + "CertConf": { + "example.org": { + "ServerCertFile": "tls_conf/certs/example.crt", + "ServerKeyFile" : "tls_conf/certs/example.key" + } + } + } +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/session_ticket_key.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/session_ticket_key.data new file mode 100644 index 000000000..b3d2356b0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/session_ticket_key.data @@ -0,0 +1,4 @@ +{ + "Version": "init version", + "SessionTicketKey": "08a0d852ef494143af613ef32d3c39314758885f7108e9ab021d55f422a454f7c9cd5a53978f48fa1063eadcdc06878f" +} diff --git a/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/tls_rule_conf.data b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/tls_rule_conf.data new file mode 100644 index 000000000..66e7b5dbb --- /dev/null +++ b/tests/integration/implementation/scenario-SC04-provider-model-prefix-strip/testdata/tls_conf/tls_rule_conf.data @@ -0,0 +1,20 @@ +{ + "Version": "12", + "DefaultNextProtos": ["http/1.1"], + "Config": { + "example_product": { + "VipConf": [ + "10.199.4.14" + ], + "SniConf": ["example.org"], + "CertName": "example.org", + "NextProtos": [ + "h2;rate=100;isw=65535;mcs=200;level=0", + "http/1.1" + ], + "Grade": "C", + "ClientAuth": false, + "ClientCAName": "example_ca" + } + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/sc05_access_log_ai_fields_test.go b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/sc05_access_log_ai_fields_test.go new file mode 100644 index 000000000..d5e8e3013 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/sc05_access_log_ai_fields_test.go @@ -0,0 +1,819 @@ +// Copyright (c) 2026 The BFE Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package sc05 + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/bfenetworks/bfe/bfe_basic" + "github.com/bfenetworks/bfe/bfe_config/bfe_cluster_conf/cluster_conf" + bfe_access_pb "github.com/bfenetworks/bfe-access-pb/bfe_access_pb" + "github.com/bfenetworks/bfe/tests/integration/common" +) + +const ( + apiHost = "rmb.example.org" + apiPath = "/v1/chat/completions" + apiKey = "ak_user_a" + apiKeyId = "user_a_key_id" + + clusterRMB = "cluster_rmb" + clusterNoTable = "cluster_no_table" + clusterFallbackRMB = "cluster_fallback_rmb" + + planRMB = "plan_rmb" + planToken = "plan_token" + + redisKeyRMB = "quota:plan_rmb" + redisKeyToken = "quota:plan_token" +) + +var defaultBody = []byte(`{"model":"deepseek-chat"}`) +var modelMappingBody = []byte(`{"model":"gpt-4"}`) +var streamBody = []byte(`{"model":"deepseek-chat","stream":true}`) + +var usageResponse = `{"usage":{"prompt_tokens":100,"completion_tokens":50,"total_tokens":150}}` + +// SSE format: final chunk contains usage. The trailing blank line is required. +var streamUsageResponse = "data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\n" + + "data: {\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}\n\n" + + "data: {\"usage\":{\"prompt_tokens\":100,\"completion_tokens\":50,\"total_tokens\":150}}\n\n" + +// testEnv holds all resources for a single SC05 integration test. +type testEnv struct { + t *testing.T + processEnv *common.ProcessEnv + backends map[string]*common.MockBackend + redis *common.RedisServer + bfePort int + stopBFE func() + logDir string +} + +func newTestEnv(t *testing.T, aiConfs map[string]*cluster_conf.AIConf, quotaPlans []common.QuotaPlan, + enableRateLimit bool) *testEnv { + e := &testEnv{ + t: t, + backends: make(map[string]*common.MockBackend), + } + + e.backends[clusterRMB] = common.NewMockBackend(clusterRMB, http.StatusOK, usageResponse) + e.backends[clusterNoTable] = common.NewMockBackend(clusterNoTable, http.StatusOK, usageResponse) + e.backends[clusterFallbackRMB] = common.NewMockBackend(clusterFallbackRMB, http.StatusOK, usageResponse) + + e.redis = common.NewRedisServer(t) + + e.processEnv = common.NewProcessEnv(t) + e.processEnv.Build() + + confDir := filepath.Join(e.processEnv.WorkDir(), "conf") + e.logDir = filepath.Join(e.processEnv.WorkDir(), "log") + + tokenRule := &common.TokenRuleData{ + Version: "1.0", + QuotaPlans: map[string][]common.QuotaPlan{ + "ai_product": quotaPlans, + }, + Tokens: map[string]map[string]common.TokenFile{ + "ai_product": { + apiKey: { + Key: apiKey, + KeyId: apiKeyId, + Enabled: 1, + Status: 1, + UpdateTime: 0, + ExpiredTime: -1, + UnlimitedQuota: false, + QuotaPlans: planIDs(quotaPlans), + Tags: []bfe_basic.ApikeyTag{ + {TagName: "department", TagValue: "ai-team"}, + }, + }, + }, + }, + Config: map[string][]common.TokenRule{ + "ai_product": { + { + Cond: "default_t()", + Action: common.ActionFile{Cmd: "CHECK_TOKEN"}, + }, + }, + }, + } + + builder := &common.BFEConfigBuilder{ + TemplateDir: "testdata", + TargetConfDir: confDir, + Backends: e.backends, + AIConfs: aiConfs, + RedisAddr: e.redis.Addr(), + TokenRuleData: tokenRule, + } + if err := builder.Build(); err != nil { + t.Fatalf("build bfe config failed: %v", err) + } + + if enableRateLimit { + if err := e.setupRateLimitConf(confDir); err != nil { + t.Fatalf("setup rate limit conf failed: %v", err) + } + if err := e.enableModuleInBFEConf(confDir, "mod_ai_rate_limit"); err != nil { + t.Fatalf("enable mod_ai_rate_limit in bfe.conf failed: %v", err) + } + } + + e.bfePort, _, e.stopBFE = e.processEnv.StartBFE(confDir, e.logDir) + return e +} + +func planIDs(plans []common.QuotaPlan) []string { + ids := make([]string, len(plans)) + for i, p := range plans { + ids[i] = p.Id + } + return ids +} + +func (e *testEnv) Close() { + if e.stopBFE != nil { + e.stopBFE() + } + for _, b := range e.backends { + b.Close() + } + if e.redis != nil { + e.redis.Close() + } +} + +func (e *testEnv) logBFEException() { + data, err := os.ReadFile(filepath.Join(e.processEnv.WorkDir(), "log", "exception.log")) + if err == nil && len(data) > 0 { + e.t.Logf("bfe exception log:\n%s", string(data)) + } +} + +func (e *testEnv) sendRequest(host string, body []byte) (*http.Response, string, error) { + url := fmt.Sprintf("http://127.0.0.1:%d%s", e.bfePort, apiPath) + req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body)) + if err != nil { + return nil, "", err + } + req.Host = host + req.Header.Set("Authorization", "Bearer "+apiKey) + req.Header.Set("Content-Type", "application/json") + + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + respBody, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return resp, string(respBody), nil +} + +func (e *testEnv) accessLogs() []*bfe_access_pb.RequestLog { + e.t.Helper() + reqLogs, err := common.ParseAccessLogAfterStop(e.logDir) + if err != nil { + e.t.Fatalf("parse access log failed: %v", err) + } + return reqLogs +} + +func (e *testEnv) mustFindSingleLog(reqLogs []*bfe_access_pb.RequestLog) *bfe_access_pb.RequestLog { + e.t.Helper() + if len(reqLogs) == 0 { + e.t.Fatalf("expected at least 1 access log, got 0") + } + return reqLogs[len(reqLogs)-1] +} + +func defaultRMBAIConf() *cluster_conf.AIConf { + return &cluster_conf.AIConf{ + Type: 0, + ModelMapping: &map[string]string{ + "gpt-4": "deepseek-chat", + }, + Provider: "mock-provider", + Keys: []cluster_conf.AIKey{ + {Name: "key-primary", Key: "sk-primary", Weight: 100}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 0, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + ModelTable: &cluster_conf.ModelTable{ + Currency: "RMB", + Models: []cluster_conf.ModelPrice{ + { + Provider: "mock-provider", + Model: "deepseek-chat", + BaseModel: "deepseek-chat", + Mode: "chat", + Capabilities: []string{"chat"}, + SupportedParameters: []string{"temperature", "max_tokens"}, + Limits: map[string]interface{}{ + "context_window": 128000, + }, + Prices: map[string]float64{ + "input_cost_per_token": 0.000001, + "output_cost_per_token": 0.000002, + }, + }, + }, + }, + } +} + +func multiKeyRMBAIConf() *cluster_conf.AIConf { + conf := defaultRMBAIConf() + conf.Keys = []cluster_conf.AIKey{ + {Name: "key-primary", Key: "sk-primary", Weight: 100}, + {Name: "key-secondary", Key: "sk-secondary", Weight: 100}, + } + conf.KeyPolicy.MaxRetries = 2 + return conf +} + +func fallbackRMBAIConf() *cluster_conf.AIConf { + conf := defaultRMBAIConf() + conf.Provider = "mock-provider-fallback" + conf.ModelTable.Models[0].Prices = map[string]float64{ + "input_cost_per_token": 0.000003, + "output_cost_per_token": 0.000004, + } + return conf +} + +func noTableAIConf() *cluster_conf.AIConf { + return &cluster_conf.AIConf{ + Type: 0, + Keys: []cluster_conf.AIKey{ + {Name: "key-primary", Key: "sk-primary", Weight: 100}, + }, + KeyPolicy: &cluster_conf.AIKeyPolicy{ + Strategy: "weighted_random", + MaxRetries: 0, + RetryBackoffInitial: 50, + RetryBackoffMax: 200, + }, + } +} + +func rmbQuotaPlan(quota int64) common.QuotaPlan { + return common.QuotaPlan{ + Id: planRMB, + Unlimited: false, + PassNoQuota: false, + RedisKey: redisKeyRMB, + CreateTime: 0, + ExpiredTime: -1, + Quota: quota, + ResetMode: 0, + Unit: "RMB", + } +} + +func tokenQuotaPlan(quota int64) common.QuotaPlan { + return common.QuotaPlan{ + Id: planToken, + Unlimited: false, + PassNoQuota: false, + RedisKey: redisKeyToken, + CreateTime: 0, + ExpiredTime: -1, + Quota: quota, + ResetMode: 0, + Unit: "total_token", + } +} + + +// TestTC01 verifies all major AI access log fields for a successful RMB request. +func TestTC01_SuccessfulRequestAIFields(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}, false) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + + // Wait for access log to be flushed before stopping BFE. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLog := e.mustFindSingleLog(e.accessLogs()) + assertStringField(t, reqLog.AiApikeyId, "ai_apikey_id", apiKeyId) + assertApikeyTags(t, reqLog.AiApikeytags, []bfe_basic.ApikeyTag{{TagName: "department", TagValue: "ai-team"}}) + assertStringField(t, reqLog.AiRequestedModel, "ai_requested_model", "deepseek-chat") + assertStringField(t, reqLog.AiTargetModel, "ai_target_model", "deepseek-chat") + assertStringField(t, reqLog.AiProvider, "ai_provider", "mock-provider") + assertInt64Field(t, reqLog.AiInputTokens, "ai_input_tokens", 100) + assertInt64Field(t, reqLog.AiOutputTokens, "ai_output_tokens", 50) + assertInt64Field(t, reqLog.AiTotalTokens, "ai_total_tokens", 150) + assertInt64Field(t, reqLog.AiCostValue, "ai_cost_value", 100*100+50*200) + assertStringField(t, reqLog.AiCostCurrency, "ai_cost_currency", "RMB") + if reqLog.AiRetryCount != nil && *reqLog.AiRetryCount != 0 { + t.Errorf("ai_retry_count should be 0 or nil, got %d", *reqLog.AiRetryCount) + } + assertRouteRuleHits(t, reqLog.AiRouteRuleHits, []expectedRouteRuleHit{{Owner: "ak_user_a", OwnerType: "apikey", RuleName: "user_a-rmb"}}) + assertClusterKeyNames(t, reqLog.AiClusterKeyNames, []expectedClusterKeyName{{ClusterName: clusterRMB, KeyName: "key-primary"}}) + assertStringSliceField(t, reqLog.AiAuthHitQuotaPlans, "ai_auth_hit_quota_plans", []string{planRMB}) +} + +// TestTC02 verifies ai_requested_model and ai_target_model after ModelMapping. +func TestTC02_ModelMappingTargetModel(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}, false) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + resp, body, err := e.sendRequest(apiHost, modelMappingBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + models := e.backends[clusterRMB].Models() + if len(models) != 1 || models[0] != "deepseek-chat" { + t.Fatalf("expected backend model deepseek-chat, got %v", models) + } + + // Wait for access log to be flushed before stopping BFE. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLog := e.mustFindSingleLog(e.accessLogs()) + assertStringField(t, reqLog.AiRequestedModel, "ai_requested_model", "gpt-4") + assertStringField(t, reqLog.AiTargetModel, "ai_target_model", "deepseek-chat") + assertInt64Field(t, reqLog.AiCostValue, "ai_cost_value", 100*100+50*200) +} + +// TestTC03 verifies ai_retry_count and ai_cluster_key_names during key-level retry. +func TestTC03_KeyRetryCountAndClusterKeyNames(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: multiKeyRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}, false) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + // First request returns 500, subsequent requests return 200. + e.backends[clusterRMB].ResponseFunc = func(r *http.Request, count int) (int, string) { + if count == 1 { + return http.StatusInternalServerError, "" + } + return http.StatusOK, usageResponse + } + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() < 2 { + t.Fatalf("expected at least 2 hits on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + + // Wait for access log to be flushed before stopping BFE. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLog := e.mustFindSingleLog(e.accessLogs()) + if reqLog.AiRetryCount == nil || *reqLog.AiRetryCount == 0 { + t.Errorf("ai_retry_count should be > 0, got %v", reqLog.AiRetryCount) + } + if len(reqLog.AiClusterKeyNames) < 2 { + t.Errorf("expected at least 2 cluster_key_names, got %d: %s", len(reqLog.AiClusterKeyNames), common.FormatAccessLogError(reqLog)) + } + for _, ckn := range reqLog.AiClusterKeyNames { + if ckn.GetClusterName() != clusterRMB { + t.Errorf("expected cluster_name %s, got %s", clusterRMB, ckn.GetClusterName()) + } + } +} + +// TestTC04 verifies ai_auth_reject_reason and ai_auth_reject_quota_plans when quota exhausted. +func TestTC04_QuotaExhaustedRejectFields(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(0)}, false) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 0) + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusTooManyRequests { + e.logBFEException() + t.Fatalf("expected status 429, got %d, body: %s", resp.StatusCode, body) + } + if e.backends[clusterRMB].Hits() != 0 { + t.Fatalf("expected no backend hit, got %d", e.backends[clusterRMB].Hits()) + } + + // Wait for access log to be flushed before stopping BFE. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLog := e.mustFindSingleLog(e.accessLogs()) + assertStringField(t, reqLog.AiApikeyId, "ai_apikey_id", apiKeyId) + if reqLog.AiAuthRejectReason == nil || *reqLog.AiAuthRejectReason == "" { + t.Errorf("ai_auth_reject_reason should not be empty") + } + assertStringSliceField(t, reqLog.AiAuthRejectQuotaPlans, "ai_auth_reject_quota_plans", []string{planRMB}) + if len(reqLog.AiRouteRuleHits) != 0 { + t.Errorf("expected no route rule hits on rejected request, got %d", len(reqLog.AiRouteRuleHits)) + } + if len(reqLog.AiClusterKeyNames) != 0 { + t.Errorf("expected no cluster_key_names on rejected request, got %d", len(reqLog.AiClusterKeyNames)) + } +} + +// TestTC05 verifies ai_provider and ai_cluster_key_names after cluster fallback. +func TestTC05_FallbackProviderAndClusterKeyNames(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + clusterFallbackRMB: fallbackRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}, false) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + e.backends[clusterRMB].ResponseFunc = func(r *http.Request, count int) (int, string) { + return http.StatusBadGateway, "" + } + + resp, body, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + if e.backends[clusterFallbackRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterFallbackRMB, e.backends[clusterFallbackRMB].Hits()) + } + + // Wait for access log to be flushed before stopping BFE. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLog := e.mustFindSingleLog(e.accessLogs()) + assertStringField(t, reqLog.AiProvider, "ai_provider", "mock-provider-fallback") + assertInt64Field(t, reqLog.AiCostValue, "ai_cost_value", 100*300+50*400) + assertRouteRuleHits(t, reqLog.AiRouteRuleHits, []expectedRouteRuleHit{{Owner: "ak_user_a", OwnerType: "apikey", RuleName: "user_a-rmb"}}) + if len(reqLog.AiClusterKeyNames) < 2 { + t.Errorf("expected at least 2 cluster_key_names, got %d: %s", len(reqLog.AiClusterKeyNames), common.FormatAccessLogError(reqLog)) + } +} + +// TestTC06 verifies ai_stream, ai_ttft_us and ai_tpot_us for SSE responses. +func TestTC06_StreamingFields(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}, false) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + e.backends[clusterRMB].ResponseHeaders = map[string]string{"Content-Type": "text/event-stream"} + e.backends[clusterRMB].Body = streamUsageResponse + + resp, body, err := e.sendRequest(apiHost, streamBody) + if err != nil { + t.Fatalf("send request failed: %v", err) + } + if resp.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected status 200, got %d, body: %s", resp.StatusCode, body) + } + + if e.backends[clusterRMB].Hits() != 1 { + t.Fatalf("expected 1 hit on %s, got %d", clusterRMB, e.backends[clusterRMB].Hits()) + } + + // Wait for async redis deduction and log flush after response finishes. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLog := e.mustFindSingleLog(e.accessLogs()) + if reqLog.AiStream == nil || !*reqLog.AiStream { + t.Errorf("ai_stream should be true") + } + if reqLog.AiTtftUs == nil || *reqLog.AiTtftUs <= 0 { + t.Errorf("ai_ttft_us should be > 0, got %v", reqLog.AiTtftUs) + } + if reqLog.AiTpotUs == nil || *reqLog.AiTpotUs <= 0 { + t.Errorf("ai_tpot_us should be > 0, got %v", reqLog.AiTpotUs) + } + assertInt64Field(t, reqLog.AiInputTokens, "ai_input_tokens", 100) + assertInt64Field(t, reqLog.AiOutputTokens, "ai_output_tokens", 50) +} + + +type expectedRouteRuleHit struct { + Owner string + OwnerType string + RuleName string +} + +type expectedClusterKeyName struct { + ClusterName string + KeyName string +} + +func assertStringField(t *testing.T, ptr *string, name, want string) { + t.Helper() + if ptr == nil { + t.Errorf("%s is nil, want %s", name, want) + return + } + if *ptr != want { + t.Errorf("%s = %s, want %s", name, *ptr, want) + } +} + +func assertInt64Field(t *testing.T, ptr *int64, name string, want int64) { + t.Helper() + if ptr == nil { + t.Errorf("%s is nil, want %d", name, want) + return + } + if *ptr != want { + t.Errorf("%s = %d, want %d", name, *ptr, want) + } +} + +func assertStringSliceField(t *testing.T, got []string, name string, want []string) { + t.Helper() + if len(got) != len(want) { + t.Errorf("%s = %v, want %v", name, got, want) + return + } + for i := range want { + if got[i] != want[i] { + t.Errorf("%s[%d] = %s, want %s", name, i, got[i], want[i]) + } + } +} + +func assertApikeyTags(t *testing.T, got []*bfe_access_pb.ApikeyTag, want []bfe_basic.ApikeyTag) { + t.Helper() + if len(got) != len(want) { + t.Errorf("ai_apikeytags length = %d, want %d", len(got), len(want)) + return + } + for i, w := range want { + if got[i].GetTagname() != w.TagName || got[i].GetTagvalue() != w.TagValue { + t.Errorf("ai_apikeytags[%d] = (%s,%s), want (%s,%s)", i, + got[i].GetTagname(), got[i].GetTagvalue(), w.TagName, w.TagValue) + } + } +} + +func assertRouteRuleHits(t *testing.T, got []*bfe_access_pb.AIRouteRuleHit, want []expectedRouteRuleHit) { + t.Helper() + if len(got) != len(want) { + t.Errorf("ai_route_rule_hits length = %d, want %d", len(got), len(want)) + return + } + for i, w := range want { + if got[i].GetRuleOwner() != w.Owner || got[i].GetRuleOwnerType() != w.OwnerType || got[i].GetRuleName() != w.RuleName { + t.Errorf("ai_route_rule_hits[%d] = (%s,%s,%s), want (%s,%s,%s)", i, + got[i].GetRuleOwner(), got[i].GetRuleOwnerType(), got[i].GetRuleName(), + w.Owner, w.OwnerType, w.RuleName) + } + } +} + +func assertClusterKeyNames(t *testing.T, got []*bfe_access_pb.ClusterKeyName, want []expectedClusterKeyName) { + t.Helper() + if len(got) != len(want) { + t.Errorf("ai_cluster_key_names length = %d, want %d", len(got), len(want)) + return + } + for i, w := range want { + if got[i].GetClusterName() != w.ClusterName || got[i].GetKeyName() != w.KeyName { + t.Errorf("ai_cluster_key_names[%d] = (%s,%s), want (%s,%s)", i, + got[i].GetClusterName(), got[i].GetKeyName(), w.ClusterName, w.KeyName) + } + } +} + +// enableModuleInBFEConf appends a module to the Modules list in bfe.conf. +func (e *testEnv) enableModuleInBFEConf(confDir, modName string) error { + path := filepath.Join(confDir, "bfe.conf") + data, err := os.ReadFile(path) + if err != nil { + return err + } + content := string(data) + if !strings.Contains(content, "Modules = "+modName) { + content += "\nModules = " + modName + "\n" + } + return os.WriteFile(path, []byte(content), 0644) +} + +// setupRateLimitConf writes mod_ai_rate_limit configuration files. +func (e *testEnv) setupRateLimitConf(confDir string) error { + modDir := filepath.Join(confDir, "mod_ai_rate_limit") + if err := os.MkdirAll(modDir, 0755); err != nil { + return err + } + + confContent := `[basic] +ProductRulePath = mod_ai_rate_limit/ai_rate_limit.data +IsRejectOnRedisError = true + +[redis] +bns = redis_bns +connectTimeout = 20 +readTimeout = 20 +writeTimeout = 20 +maxIdle = 20 + +[log] +OpenDebug = false +` + if err := os.WriteFile(filepath.Join(modDir, "mod_ai_rate_limit.conf"), []byte(confContent), 0644); err != nil { + return err + } + + rateLimitData := map[string]interface{}{ + "Version": "1.0", + "Config": map[string]interface{}{ + "ai_product": []map[string]interface{}{ + { + "cond": "default_t()", + "hit_action": map[string]interface{}{ + "cmd": "FINISH", + "params": []string{}, + }, + }, + }, + }, + "RateLimitPolicies": map[string]interface{}{ + "rlp-rpm-1": map[string]interface{}{ + "name": "ratelimitRPM", + "enabled": true, + "rules": map[string]interface{}{ + "rpm": []map[string]interface{}{ + { + "name": "rpm1", + "window_minutes": 1, + "max_requests": 1, + "burst": 1, + }, + }, + }, + }, + }, + "ApikeyRateLimitPolicyBindings": map[string]interface{}{ + apiKey: []string{"rlp-rpm-1"}, + }, + } + data, err := json.MarshalIndent(rateLimitData, "", " ") + if err != nil { + return err + } + return os.WriteFile(filepath.Join(modDir, "ai_rate_limit.data"), data, 0644) +} + + + + +// TestTC07 verifies ai_rate_limit_hits when RPM limit is triggered. +func TestTC07_RateLimitHits(t *testing.T) { + aiConfs := map[string]*cluster_conf.AIConf{ + clusterRMB: defaultRMBAIConf(), + } + e := newTestEnv(t, aiConfs, []common.QuotaPlan{rmbQuotaPlan(10000000000)}, true) + defer e.Close() + + e.redis.SetQuota(redisKeyRMB, 10000000000) + + // First request should succeed. + resp1, body1, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send first request failed: %v", err) + } + if resp1.StatusCode != http.StatusOK { + e.logBFEException() + t.Fatalf("expected first status 200, got %d, body: %s", resp1.StatusCode, body1) + } + + // Second request should be rate limited (rpm max_requests=1). + resp2, body2, err := e.sendRequest(apiHost, defaultBody) + if err != nil { + t.Fatalf("send second request failed: %v", err) + } + if resp2.StatusCode != http.StatusTooManyRequests { + e.logBFEException() + t.Fatalf("expected second status 429, got %d, body: %s", resp2.StatusCode, body2) + } + + // Wait for access log to be flushed before stopping BFE. + time.Sleep(500 * time.Millisecond) + + e.stopBFE() + e.stopBFE = nil + + reqLogs := e.accessLogs() + if len(reqLogs) != 2 { + t.Fatalf("expected 2 access logs, got %d", len(reqLogs)) + } + + // First log should have no rate limit hits. + if len(reqLogs[0].AiRateLimitHits) != 0 { + t.Errorf("first request should have no rate limit hits, got %d", len(reqLogs[0].AiRateLimitHits)) + } + + // Second log should record the RPM hit. + if len(reqLogs[1].AiRateLimitHits) == 0 { + t.Fatalf("second request should have rate limit hits, got 0: %s", common.FormatAccessLogError(reqLogs[1])) + } + hit := reqLogs[1].AiRateLimitHits[0] + if hit.GetRateLimitPolicyId() != "rlp-rpm-1" { + t.Errorf("rate_limit_policy_id = %s, want rlp-rpm-1", hit.GetRateLimitPolicyId()) + } + if hit.GetRateLimitType() != "rpm" { + t.Errorf("rate_limit_type = %s, want rpm", hit.GetRateLimitType()) + } + if len(hit.GetRuleNames()) == 0 { + t.Errorf("rate_limit rule_names should not be empty") + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/bfe.conf b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/bfe.conf new file mode 100644 index 000000000..6e23e598d --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/bfe.conf @@ -0,0 +1,39 @@ +[server] +httpPort = 18080 +httpsPort = 18443 +monitorPort = 18081 +httpAddr = "127.0.0.1" +httpsAddr = "127.0.0.1" +monitorAddr = "127.0.0.1" +MonitorEnabled = true +maxCpus = 1 + +TlsHandshakeTimeout = 30 +ClientReadTimeout = 5 +ClientWriteTimeout = 5 +KeepAliveEnabled = true +GracefulShutdownTimeout = 10 + +EnableAiGateway = true + +accessibleBodySize = 4194304 + +# max total bytes of all active bytes_body buffers (0 means unlimited) +totalBodyBufferSize = 0 + +Modules = mod_ai_route +Modules = mod_ai_token_auth +Modules = mod_body_process +Modules = mod_access_pb3 + +hostRuleConf = server_data_conf/host_rule.data +routeRuleConf = server_data_conf/route_rule.data +vipRuleConf = server_data_conf/vip_rule.data + +clusterTableConf = cluster_conf/cluster_table.data +gslbConf = cluster_conf/gslb.data +clusterConf = cluster_conf/cluster_conf.data +NameConf = + +maxHeaderUriBytes = 8096 +maxHeaderBytes = 8096 diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/cluster_conf/gslb.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/cluster_conf/gslb.data new file mode 100644 index 000000000..707418d1e --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/cluster_conf/gslb.data @@ -0,0 +1,18 @@ +{ + "clusters": { + "cluster_rmb": { + "GSLB_BLACKHOLE": 0, + "sub_rmb": 100 + }, + "cluster_no_table": { + "GSLB_BLACKHOLE": 0, + "sub_notable": 100 + }, + "cluster_fallback_rmb": { + "GSLB_BLACKHOLE": 0, + "sub_fallback_rmb": 100 + } + }, + "hostname": "gslb-test", + "ts": "20260720150000" +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_access_pb3/mod_access_pb3.conf b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_access_pb3/mod_access_pb3.conf new file mode 100644 index 000000000..5a254ffd8 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_access_pb3/mod_access_pb3.conf @@ -0,0 +1,8 @@ +[Log] +LogPrefix = pb_access3 +LogDir = ../log +RotateWhen = NEXTHOUR +BackupCount = 2 + +[BasicConf] +OpenDebug = false diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_route/ai_route.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_route/ai_route.data new file mode 100644 index 000000000..aa97c3cd1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_route/ai_route.data @@ -0,0 +1,45 @@ +{ + "Version": "20260720150000", + "route_rules": { + "apikey_ak_user_a": { + "type": "apikey", + "owner": "ak_user_a", + "rules": [ + { + "name": "user_a-rmb", + "Cond": "req_host_in(\"rmb.example.org\")", + "targets": [ + { + "ClusterName": "cluster_rmb", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [ + { + "ClusterName": "cluster_fallback_rmb", + "Model": "" + } + ] + }, + { + "name": "user_a-notable", + "Cond": "req_host_in(\"notable.example.org\")", + "targets": [ + { + "ClusterName": "cluster_no_table", + "Model": "", + "Weight": 100 + } + ], + "fallbacks": [] + } + ] + } + }, + "ApikeyRouteTableBindings": { + "ak_user_a": [ + "apikey_ak_user_a" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_route/mod_ai_route.conf b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_route/mod_ai_route.conf new file mode 100644 index 000000000..f250013bb --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_route/mod_ai_route.conf @@ -0,0 +1,5 @@ +[basic] +RouteRulePath = mod_ai_route/ai_route.data + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_token_auth/mod_ai_token_auth.conf b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_token_auth/mod_ai_token_auth.conf new file mode 100644 index 000000000..966b473d0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_ai_token_auth/mod_ai_token_auth.conf @@ -0,0 +1,13 @@ +[basic] +ProductRulePath = mod_ai_token_auth/token_rule.data + +[redis] +Bns = 127.0.0.1:6379 +ConnectTimeout = 1000 +ReadTimeout = 1000 +WriteTimeout = 1000 +MaxIdle = 10 +MaxActive = 20 + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_body_process/body_process.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_body_process/body_process.data new file mode 100644 index 000000000..2a4746d1b --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_body_process/body_process.data @@ -0,0 +1,10 @@ +{ + "Version": "1.0", + "Config": { + "ai_product": [ + { + "Cond": "default_t()" + } + ] + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_body_process/mod_body_process.conf b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_body_process/mod_body_process.conf new file mode 100644 index 000000000..07fa25732 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/mod_body_process/mod_body_process.conf @@ -0,0 +1,5 @@ +[basic] +ProductRulePath = mod_body_process/body_process.data + +[log] +OpenDebug = true diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/host_rule.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/host_rule.data new file mode 100644 index 000000000..dbda248e7 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/host_rule.data @@ -0,0 +1,14 @@ +{ + "Version": "20260720150000", + "Hosts": { + "ai_product": [ + "rmb.example.org", + "notable.example.org" + ] + }, + "HostTags": { + "ai_product": [ + "ai_product" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/route_rule.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/route_rule.data new file mode 100644 index 000000000..ccb697617 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/route_rule.data @@ -0,0 +1,12 @@ +{ + "Version": "20260720150000", + "BasicRule": { + "ai_product": [ + { + "Hostname": ["*"], + "Path": ["*"], + "ClusterName": "cluster_fallback_rmb" + } + ] + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/vip_rule.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/vip_rule.data new file mode 100644 index 000000000..6fe22f03a --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/server_data_conf/vip_rule.data @@ -0,0 +1,8 @@ +{ + "Version": "20260720150000", + "Vips": { + "ai_vip": [ + "127.0.0.1" + ] + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/bfe_i_ca.crt b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/bfe_i_ca.crt new file mode 100644 index 000000000..f1e78f73c --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/bfe_i_ca.crt @@ -0,0 +1,23 @@ +-----BEGIN CERTIFICATE----- +MIIDwDCCAqigAwIBAgIBCzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +NDAxNDZaFw0zNzA3MTIxNDAxNDZaMIGQMQswCQYDVQQGEwJjbjEQMA4GA1UECAwH +YmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsMC3lpbmdmZWkt +ZGV2MRgwFgYDVQQDDA95aW5nZmVpLWRldi1pY2ExKTAnBgkqhkiG9w0BCQEWGmxp +YW5nY2h1YW5AeWYtbmV0d29ya3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A +MIIBCgKCAQEA6eFuWgoknixrRO9NCX4jAKyLtcAOWVJqVN2yX7CxjZjLyPurjvTZ +W73NYUbrPAN4AB5gY6UAPzuiEOSopVVIZ0OschK0cJldu9vZ0mZObBOsovuFcLQq +dgNSJ5slJSk7tCgD2EnCB3GYPG4D+uIKYd0c49wzTWWv4bjDwpgnf0LQbFpy7GhN +7D59zFH4qgOK/IQ5vaTMGyvIvtWR5/1Gvc9MLpGopTgi0DiNLed4UwDYrod5kysl +q3UcB5puONHQISOVoD3uRxo7wdsmVsHUfW7YfAWkhi6ec8mx9fy8IyE6f7GlXnuV +ysNccwqyEotL5bOXPJCqwUCL1v+iKTMPWwIDAQABo1AwTjAdBgNVHQ4EFgQUCjvC +vDVr8QXh9/hMI5xBgNvoSFgwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW +1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAhIggK/6JK4N7+HZW +VoTemwRpilugZZyKrcVAHbiiwUfXVQVuI64vc+yHWMSFRD1mykHkKBFzxEoDabMl +ASBtUJNt4b4zEL9V7k295vmAOp2IdLUQxlgeKWqABm4DGX96tGR9nKQTcn1ZeAxx +NyQFV1aj+dnNcF1iFFNF6t0bRrOEZ/aRSiu1bWp3Dj1JYTjXbyyplh3Ktb8lv4lt +EmvCXjo/l4TgQC9233kcTkXcq1swppzkkXfhB0NVuf9DE3C2xAWX0b2FiIEqnAhl +Bx0Cn3RrX7PZ6qrdRL6oBKvGJV6DP8BlF4LXiuD1VN/waQFJha57KQ78ZYYIXxQg +WMjWgA== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/bfe_r_ca.crt b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/bfe_r_ca.crt new file mode 100644 index 000000000..2a6db1e49 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/bfe_r_ca.crt @@ -0,0 +1,22 @@ +-----BEGIN CERTIFICATE----- +MIIDkzCCAnugAwIBAgIBCjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzExMDMx +MzQ5NTVaFw0zNzA3MTIxMzQ5NTVaMGQxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxFzAVBgNVBAMMDnlpbmdmZWktZGV2LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOC +AQ8AMIIBCgKCAQEAvNA3HrsMjBcXrMIIhGVWsurIA1F9jxKeA7dh06H00Vt4inVV +SUvNFrTqgPRhLkAhGRMPxrjVRgJ5bbFqqXIuPIpUFBhUsWXIDH+oVXQl9jsxAXaG +gZ0lTO/uYR9qyrS1rj9nyNPwRf59Al/VlsQL71cNQ/T/agJ4PfvPfULTPLOsqclJ +hj0IgXmDj464dqcdG3ZdfXpfhNF6ab+8YjpwafTRmY+LoV8qjUwsYeJMcW4N8pxJ +8F2ktZj9J6uWepNGj+87ZeXg9XquzC62ASIFzPjoE1WN//Q518EqizxhuLGBDpK3 +6sEYUK4kHYUL4gZKFPlKTPXIl0ZsJIM5PsBMKwIDAQABo1AwTjAdBgNVHQ4EFgQU +DZloT8VhVbysD819oG/oqHdW1D0wHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/o +qHdW1D0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEALgB+IcN3UD4T ++4g8jaOyOiSUpUVdYqW+3go5DppqnvlGNq99N2cXKqssVPa/T9TikEcBEicFa8zU +bwlx6TEte+MkWfWdQxFR1EuI1FgKr3ps6ZBRr7MPpnYmI7K9aK372K9n7WrQhbmP +s7ult8bWB1/t6o3R7B9ChNkWT+7DPD4+FvB1GMJSGPno7cdnvDkevBOuC2DnQl3M ++ADFAge1Lo8wKBy6gYkNFd2BfarHGvRC5Qmmrme+RIpWZnvux1+lfXIInnfSTJRM +uAo/ePkoNsM3qQll6uEdhDxOMx8Pq94bCkM3DtI3ObuxWXjKCgUm/n9yetUvPj4U +iYhRUqHpUg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_bfe_dev.crt b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_bfe_dev.crt new file mode 100644 index 000000000..2164fb602 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_bfe_dev.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 18 (0x12) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Jan 31 02:55:32 2024 GMT + Not After : Jan 28 02:55:32 2034 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=bfe-dev/emailAddress=dev@example.org + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:b2:7f:c1:0c:cd:49:43:d9:99:78:15:4c:5a:52: + a9:a7:bf:d5:eb:92:71:43:e1:37:e5:29:1b:68:f4: + 6f:4c:ea:fa:a4:8a:7c:29:01:2a:fa:7c:81:5b:c8: + eb:a1:40:94:b7:2b:82:e2:00:08:36:84:f0:b7:d2: + 5a:1e:56:97:aa:36:ff:4d:07:49:2d:fe:25:3a:f9: + e9:f1:ad:4e:6e:21:97:a9:f9:a2:a5:ac:82:23:0a: + d2:e0:97:cd:2f:14:b1:f0:8a:a8:e6:c5:97:45:94: + f8:8e:3f:96:66:5b:0b:8c:7c:07:61:18:44:92:f7: + 23:5a:c2:4b:88:58:59:5d:ca:5c:0d:6e:dd:ff:18: + 59:65:df:95:99:e3:3a:36:48:1f:3f:3a:e6:ce:85: + f3:0b:04:5e:92:ed:6f:8e:74:92:e4:37:46:da:5f: + 17:62:9c:82:40:06:fb:29:f8:55:f2:ba:23:75:ca: + 64:c0:45:03:12:bd:f5:17:15:7e:47:d5:bd:30:f2: + 99:ca:6b:e3:07:b0:ae:44:89:1e:10:26:ea:75:df: + 6f:07:b6:47:76:54:47:4f:6c:f6:68:fe:a8:cf:22: + 20:73:e8:19:55:8a:fe:f5:78:e8:51:88:52:80:1f: + 79:d4:c5:ae:8f:d2:2b:f6:41:01:42:01:cf:98:c2: + c9:25 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + EE:E8:68:42:DF:B1:F0:EF:6F:47:51:BD:D4:94:60:1F:05:85:A1:03 + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Extended Key Usage: + TLS Web Client Authentication + Signature Algorithm: sha256WithRSAEncryption + a9:a6:26:8e:42:61:15:22:ee:fc:b5:e1:e4:6b:dd:ac:f5:15: + 11:39:10:9a:ca:6f:85:fd:cb:90:1c:b2:4f:fe:29:de:b0:e7: + 73:e2:f7:5e:63:8c:7f:c1:7e:75:2e:c9:9d:e9:c2:45:75:f3: + 27:ba:82:94:de:7f:6c:87:0c:5c:71:af:0f:14:00:68:35:f7: + 5a:4a:ff:f5:ef:35:dd:50:72:76:f0:6f:b6:7b:42:33:07:b4: + 24:44:0a:fd:9d:61:9e:44:e8:88:0f:02:76:c6:90:3f:9d:1b: + d8:3b:64:25:2a:a3:39:78:38:bd:20:89:4a:9c:bd:68:38:18: + 4c:cb:20:3a:9b:5b:5f:58:52:86:73:de:85:fe:d6:a1:c6:a7: + 86:b0:96:4b:fa:28:04:ad:5d:85:e8:a1:fc:ca:0f:3c:be:5c: + 90:7e:3e:84:ae:67:ee:9a:72:71:3c:b2:80:45:82:fc:7e:58: + 74:99:42:c5:c3:8a:4a:eb:e1:8b:d5:84:ce:25:aa:a1:75:79: + 94:66:ae:ee:df:30:15:0b:b5:c5:b1:2c:d5:0a:54:78:b6:2e: + 67:29:81:41:f6:16:49:31:96:e7:41:e1:99:6b:27:57:bb:7d: + 76:eb:e4:d5:59:aa:a2:5c:bd:1c:18:2a:fa:9d:28:1a:0b:b6: + bf:7d:58:1a +-----BEGIN CERTIFICATE----- +MIID7jCCAtagAwIBAgIBEjANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yNDAxMzEw +MjU1MzJaFw0zNDAxMjgwMjU1MzJaMH0xCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEUMBIGA1UECwwLeWluZ2ZlaS1k +ZXYxEDAOBgNVBAMMB2JmZS1kZXYxHjAcBgkqhkiG9w0BCQEWD2RldkBleGFtcGxl +Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ/wQzNSUPZmXgV +TFpSqae/1euScUPhN+UpG2j0b0zq+qSKfCkBKvp8gVvI66FAlLcrguIACDaE8LfS +Wh5Wl6o2/00HSS3+JTr56fGtTm4hl6n5oqWsgiMK0uCXzS8UsfCKqObFl0WU+I4/ +lmZbC4x8B2EYRJL3I1rCS4hYWV3KXA1u3f8YWWXflZnjOjZIHz865s6F8wsEXpLt +b450kuQ3RtpfF2KcgkAG+yn4VfK6I3XKZMBFAxK99RcVfkfVvTDymcpr4wewrkSJ +HhAm6nXfbwe2R3ZUR09s9mj+qM8iIHPoGVWK/vV46FGIUoAfedTFro/SK/ZBAUIB +z5jCySUCAwEAAaOBkTCBjjAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu +U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU7uhoQt+x8O9vR1G9 +1JRgHwWFoQMwHwYDVR0jBBgwFoAUDZloT8VhVbysD819oG/oqHdW1D0wEwYDVR0l +BAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKmmJo5CYRUi7vy14eRr +3az1FRE5EJrKb4X9y5Acsk/+Kd6w53Pi915jjH/BfnUuyZ3pwkV18ye6gpTef2yH +DFxxrw8UAGg191pK//XvNd1Qcnbwb7Z7QjMHtCRECv2dYZ5E6IgPAnbGkD+dG9g7 +ZCUqozl4OL0giUqcvWg4GEzLIDqbW19YUoZz3oX+1qHGp4awlkv6KAStXYXoofzK +Dzy+XJB+PoSuZ+6acnE8soBFgvx+WHSZQsXDikrr4YvVhM4lqqF1eZRmru7fMBUL +tcWxLNUKVHi2LmcpgUH2FkkxludB4ZlrJ1e7fXbr5NVZqqJcvRwYKvqdKBoLtr99 +WBo= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem new file mode 100644 index 000000000..764aea3ad --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_bfe_dev_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAsn/BDM1JQ9mZeBVMWlKpp7/V65JxQ+E35SkbaPRvTOr6pIp8 +KQEq+nyBW8jroUCUtyuC4gAINoTwt9JaHlaXqjb/TQdJLf4lOvnp8a1ObiGXqfmi +payCIwrS4JfNLxSx8Iqo5sWXRZT4jj+WZlsLjHwHYRhEkvcjWsJLiFhZXcpcDW7d +/xhZZd+VmeM6NkgfPzrmzoXzCwReku1vjnSS5DdG2l8XYpyCQAb7KfhV8rojdcpk +wEUDEr31FxV+R9W9MPKZymvjB7CuRIkeECbqdd9vB7ZHdlRHT2z2aP6ozyIgc+gZ +VYr+9XjoUYhSgB951MWuj9Ir9kEBQgHPmMLJJQIDAQABAoIBAFAKHzupVb/58+o3 +yqv5wx94Uuk2GlnwxIqaezL94GaiO0/K1U/huS7m426P0rDU75qPBTpn/0bLJ9GV +nllaRNnLnYEh0juwaWtfovp+1ttlbseGK9uUVip2cQbKqvQAmKWe14vbcDCAU1Ad +zUgKbUxKVVjBdAZekVjiJNJ3o2L9WPhf/uQo7A1XAJh2DajlTbgvrDM73W+47QuU +X4OHU0FMio6bxupu3OWl1bMrnKhuC4qczZWf2nOpcVQa89rtopuP4ENLJuWkbeGk +YQpNilEclnAa/Noumt/j/6GKC1EEHFsH2CNRRIazcZrsFhkSKc4pn1Y/WI3vj8kZ ++RYnJsUCgYEA6gr0x8suwRTAmVpoPk4XyP1x+eInG7onV5RjgsUtgruYd9naxRHg +2p8PHcv32pDs51Fa+4RldyMd/jec1SscRF5/+VOP9qeoRnaDqUu+uASgEO3OBxbP +JcWovyxRHIQxbYCQtqIr9bdzXw55MBLZou/sBUVTAkrIyPVyjWqZlV8CgYEAwz7L +YyYN615TsrzZKURxMjj94Nmob/NldSLRXaR3Ax7/ABtEOA685cwQxq7ONdkJTMIA +uR8u2GHZSzGiWnehuF6Zp7Xs71a57eFbs3ueZvvEba4Dff7hl7Y4tTlwrKndKjvP +J/5a2Ol8siQcRWAXHOdzggEHMSZ/sB4hWswly/sCgYEAhLRBpyemEwTZUBrbELjm +86gBgFajJi2fMSGKaxOygnYsNYjpauSAQnX99D87Aks6iM6wb/zaK3tV/lc6LgSL +uph6p7yh3JGj8JAyh0PTmDPHLtIoCAz+18QDsqJGO40ZGaXUaDn8Aw9J85QZUxDd +Jm4zvalZL+uHfarukRDolLECgYBUiupS4nWAh3XCnZeDEQna72avaFBROZmjIRJ7 +c+28wj009JmTlH4jGzvgbG0KUBKA1Div8Fq+g5AtyS498jNqvDvYrSQNdwZHhR/K +Fis++KHTxFfqxOU2Zkcj4d1yRpNn6EIJVVBNQL0n/g7n03XupCIWFw/gLoV343QZ +9vAe5QKBgA8ml59z1w3eUooc0yGfLhihXqCmM3IU006bFbODA30fBU4QKrHO8+Yx +Xbz9bi/1QLagLG6FzYQAkOjBlEBt5XLayYvwSb8xvWsm5A3vzTAbMFDOsDPEmRoH +dWtQccJcygOuK+PZtoZnNoJciNO5c9dZWD3xtIiURmX/kVtWrf6N +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_san_example.org.crt b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_san_example.org.crt new file mode 100644 index 000000000..78f3892b4 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_san_example.org.crt @@ -0,0 +1,85 @@ +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 15 (0xf) + Signature Algorithm: sha256WithRSAEncryption + Issuer: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei-dev, CN=yingfei-dev-ca + Validity + Not Before: Dec 7 09:57:42 2023 GMT + Not After : Dec 4 09:57:42 2033 GMT + Subject: C=cn, ST=beijing, O=yingfei-dev, OU=yingfei, CN=dev-test + Subject Public Key Info: + Public Key Algorithm: rsaEncryption + Public-Key: (2048 bit) + Modulus: + 00:d6:47:b0:17:69:91:4c:d0:66:c9:25:2e:38:f2: + 84:27:e2:7c:38:cc:04:b9:0c:8e:3d:cc:ef:4b:c5: + 35:2b:c1:82:d8:41:fc:25:c8:24:f2:e0:25:aa:f9: + 76:3c:e2:b2:50:fb:2d:ec:4e:16:d4:c1:da:1e:e3: + 51:9f:9d:c9:72:f9:cb:cc:14:9c:c1:82:c9:76:1f: + 98:dd:0e:b9:8a:20:d6:ab:f2:a6:f9:2f:23:81:f3: + af:e4:47:0c:55:95:94:de:ed:f6:a5:24:ee:32:e7: + 6b:79:e1:42:6f:a4:07:b2:95:1d:f5:a9:8e:60:42: + 70:42:bd:e2:30:18:68:74:52:32:98:a9:81:da:d8: + c6:6f:5e:1d:ce:79:b6:f3:ec:4f:ed:7d:22:57:d2: + 14:d0:fb:f2:50:d4:80:3b:89:ed:77:fd:45:6c:e6: + 52:6b:0a:52:71:ac:59:c8:d5:25:f5:40:03:fb:51: + b0:11:a7:00:79:d9:d8:4f:00:43:96:68:44:29:41: + dc:d2:cc:91:c8:61:95:41:4d:0e:66:5a:b5:15:67: + 3e:8a:6f:29:df:1c:8a:6f:ee:9e:97:9c:9e:69:71: + d3:34:52:75:e9:ea:e7:51:77:23:98:46:ca:47:a2: + d3:d3:97:03:41:4b:e3:33:11:72:2d:af:bf:2b:3e: + b3:51 + Exponent: 65537 (0x10001) + X509v3 extensions: + X509v3 Basic Constraints: + CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate + X509v3 Subject Key Identifier: + 5A:27:32:9D:E7:36:24:A3:C1:DC:2F:95:80:C5:CF:0C:85:E8:E6:AF + X509v3 Authority Key Identifier: + keyid:0D:99:68:4F:C5:61:55:BC:AC:0F:CD:7D:A0:6F:E8:A8:77:56:D4:3D + + X509v3 Subject Alternative Name: + DNS:example.org, DNS:www.example.org, DNS:example.com, DNS:*.example.com, IP Address:127.0.0.1, IP Address:192.168.0.100 + Signature Algorithm: sha256WithRSAEncryption + 1e:e8:e8:8a:ad:a8:0e:fc:c9:82:00:a1:ab:30:3c:a5:b9:dc: + d6:fb:86:ad:30:52:7f:61:be:90:a6:b8:56:bb:f1:0b:e6:39: + 38:65:09:6b:da:83:f7:65:ff:c4:21:de:b4:9e:8b:bd:1e:1c: + d1:d5:94:b8:18:79:f2:d0:06:51:39:67:13:40:3b:73:5b:cb: + ea:de:c1:19:76:f8:7b:0f:15:51:61:49:fb:98:f7:ea:4f:fc: + c2:fb:a7:f4:3c:48:64:14:79:b5:78:5b:20:10:b5:7a:2d:4c: + 04:51:60:ec:20:10:19:26:5f:e2:fd:32:59:67:e9:3f:48:8d: + f5:52:12:01:81:2c:c0:e5:72:cd:7d:0a:eb:7a:05:df:a0:77: + b9:ba:9a:7d:d1:4b:6a:44:e4:2d:98:af:bd:77:2b:f5:ef:26: + 4b:75:b3:97:d0:3a:bc:07:21:ef:71:92:30:fe:a2:79:e5:56: + d7:7e:c2:f3:57:ab:d7:de:fc:97:ed:20:0c:9a:cb:c5:5d:00: + 3b:61:29:e8:00:d4:39:e0:f2:4e:a4:03:c2:12:52:ff:e7:78: + f9:f7:c0:12:dc:36:a4:05:a2:f0:6b:47:e2:21:3d:a2:e1:a1: + 91:c7:ac:8f:b8:ae:58:65:e0:2b:57:80:eb:77:2d:48:ef:e6: + fb:b9:e1:20 +-----BEGIN CERTIFICATE----- +MIIEBzCCAu+gAwIBAgIBDzANBgkqhkiG9w0BAQsFADBkMQswCQYDVQQGEwJjbjEQ +MA4GA1UECAwHYmVpamluZzEUMBIGA1UECgwLeWluZ2ZlaS1kZXYxFDASBgNVBAsM +C3lpbmdmZWktZGV2MRcwFQYDVQQDDA55aW5nZmVpLWRldi1jYTAeFw0yMzEyMDcw +OTU3NDJaFw0zMzEyMDQwOTU3NDJaMFoxCzAJBgNVBAYTAmNuMRAwDgYDVQQIDAdi +ZWlqaW5nMRQwEgYDVQQKDAt5aW5nZmVpLWRldjEQMA4GA1UECwwHeWluZ2ZlaTER +MA8GA1UEAwwIZGV2LXRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB +AQDWR7AXaZFM0GbJJS448oQn4nw4zAS5DI49zO9LxTUrwYLYQfwlyCTy4CWq+XY8 +4rJQ+y3sThbUwdoe41Gfncly+cvMFJzBgsl2H5jdDrmKINar8qb5LyOB86/kRwxV +lZTe7falJO4y52t54UJvpAeylR31qY5gQnBCveIwGGh0UjKYqYHa2MZvXh3Oebbz +7E/tfSJX0hTQ+/JQ1IA7ie13/UVs5lJrClJxrFnI1SX1QAP7UbARpwB52dhPAEOW +aEQpQdzSzJHIYZVBTQ5mWrUVZz6KbynfHIpv7p6XnJ5pcdM0UnXp6udRdyOYRspH +otPTlwNBS+MzEXItr78rPrNRAgMBAAGjgc0wgcowCQYDVR0TBAIwADAsBglghkgB +hvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYE +FFonMp3nNiSjwdwvlYDFzwyF6OavMB8GA1UdIwQYMBaAFA2ZaE/FYVW8rA/NfaBv +6Kh3VtQ9ME8GA1UdEQRIMEaCC2V4YW1wbGUub3Jngg93d3cuZXhhbXBsZS5vcmeC +C2V4YW1wbGUuY29tgg0qLmV4YW1wbGUuY29thwR/AAABhwTAqABkMA0GCSqGSIb3 +DQEBCwUAA4IBAQAe6OiKragO/MmCAKGrMDyludzW+4atMFJ/Yb6QprhWu/EL5jk4 +ZQlr2oP3Zf/EId60nou9HhzR1ZS4GHny0AZROWcTQDtzW8vq3sEZdvh7DxVRYUn7 +mPfqT/zC+6f0PEhkFHm1eFsgELV6LUwEUWDsIBAZJl/i/TJZZ+k/SI31UhIBgSzA +5XLNfQrregXfoHe5upp90UtqROQtmK+9dyv17yZLdbOX0Dq8ByHvcZIw/qJ55VbX +fsLzV6vX3vyX7SAMmsvFXQA7YSnoANQ54PJOpAPCElL/53j598AS3DakBaLwa0fi +IT2i4aGRx6yPuK5YZeArV4Drdy1I7+b7ueEg +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem new file mode 100644 index 000000000..393a79825 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/backend_rs/r_san_example.org_prv.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEA1kewF2mRTNBmySUuOPKEJ+J8OMwEuQyOPczvS8U1K8GC2EH8 +Jcgk8uAlqvl2POKyUPst7E4W1MHaHuNRn53JcvnLzBScwYLJdh+Y3Q65iiDWq/Km ++S8jgfOv5EcMVZWU3u32pSTuMudreeFCb6QHspUd9amOYEJwQr3iMBhodFIymKmB +2tjGb14dznm28+xP7X0iV9IU0PvyUNSAO4ntd/1FbOZSawpScaxZyNUl9UAD+1Gw +EacAednYTwBDlmhEKUHc0syRyGGVQU0OZlq1FWc+im8p3xyKb+6el5yeaXHTNFJ1 +6ernUXcjmEbKR6LT05cDQUvjMxFyLa+/Kz6zUQIDAQABAoIBAC4sYGuLGf49Ygix +9FXdHFEj4rSyccoWRIhYoq/nHOAC4NkMzvKtQBj95+ABxVK1XstIdMrYwN6zrva8 +8Re9/mzCGwIs5uJj9ll30Y7A34Y+MUP4E7baS4JzKlG8ZZIDm4K2MFHBtXpOl8A5 +pAE+jVIUA9Kt6LohVuNq21SVzdxSfNYC/+SLqSftkWa/ZsqdkiHM5Hl+fVedh516 +IaLNW5hSthGh5n8dHY5h/AKPjfoq77aYp5/CUtJTC9mYdZu1j/W/pBVTRfOnwLQd +SQ1Xmr7f6q9Vmz+HnajIbFg9hQ54blvtUJ7DnugWxfUcoxf7ue79fnYjOIUOkRWw +8Iid/mECgYEA+5s0p0j+gkNZN5QtVNStoT04+1DqA1O381gczeaiz6Njt/MT0y5W +OpCsILQ70CpEjWAV+f6PDJiesDMxdGV+v2TCqK8ml8GahEczBLnHoAkPWCVf2XOX +oNj/CkZ2kmWufHFR+kcQbeDt1vFFcYUa61hKDFyNjinMW79Qy+PQID0CgYEA2gWd +7thE05sqU7/1MntmVRONKoAgnJmHcfSpWwLyh6E4YX3iKDSgI/9RnAMF39KUUY/O +XFWyIwAM9soeXknVsV/SmCaPeaEDiLHqz98aUEfvdLYMnuR883GgoXc4JrLsLw4z +oSi9lbAZFn0ekJ5L+rSFrY4rz9QZgYZxsLjUXKUCgYAbkaUSU2g3w8Np2J2i9u7T +hQ7SUsphdPHqAxSc5xGd6MxLYqIgeKpQHnwN1VHcfFUonIer7d2kxrBUpDdeBqT9 +ub+ulgqHhFo29ko70VNzUKrSwL2g6Q6LPFutt4zUe7nDvvL5loHRWF0XOTafurL5 +aKIsepO0KRZQU0U6IgszDQKBgFs6ZHaP2mTtFY4L0a75Ab3xu20gRgUhHRLq/H6P +wipMpMnuodaPBr9pU53Dig65D8T9Nq1eUnbgy4vs0T5FCPz6iqWN5RVQ8aieQhIP +WfRj1Wfx0WAfXcWEM2G9ACr5TWj3OVVjNclP8X9+hW6gPky+gv03c0+4gZ+4QRRg +ksPdAoGBAKXVv8L5Qt8rmi+QpF/6DVSeMB6lEevf319UEtyZPiYSBn1JJ9H+3Ddc +TMriXgZrw+PoXNJTAguDeGgzmtye1vraP0cjt7aG6sQ4YtMTnnuja880vK8z4i3Q +HJAi5fI8NvlW92dQBAccgrzFIKpRx+6CHwtCkNrxL2vJmGGzp8BR +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/certs/example.crt b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/certs/example.crt new file mode 100644 index 000000000..931874885 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/certs/example.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDEjCCAfqgAwIBAgIJAIAdu56fLE7OMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzM0NFoYDzIxMjAwNTE3MTAzMzQ0 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBALv+LV1aWIlcK9rI7IuRS8SCusqBnoyJec/ErKiA2gbfgZ/YS73L +Zud84yp45AIqauzcI5q+hrkmsRZ7CKqDzrG+jHavW7jF+0laetJwRt26AcQcOtQD +2ik2O+Dl1WHAFn4vUAQxb+Xz6WfSaQN0QfM74z06XUDDsr7g7+NYtMzhf98SJSoK +ne/dVKJ3Bc6e6tvhnCRwPtix4ektEodK6WeNHYxwJ6wSZ8cRLzdxgjdD/4OGfFuj +dn8zbOi3SQt5ZqVbcDHUTzp5t0G8EoxnzotHhhzjSAmsypySqZXaxl3oX8aYUkFn +fCdg+WBXo5pOiNfoWh/D5bnIXWGp52yoy+kCAwEAAaNhMF8wHQYDVR0lBBYwFAYI +KwYBBQUHAwIGCCsGAQUFBwMBMB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MB0GA1UdDgQWBBSxLHQE7gOEyfeSNc5uIO/G/rgjpzANBgkqhkiG9w0BAQsF +AAOCAQEAlGm5RwQ79xmLh3rj+5UViCSgsIuMcuhgIT4zogpo9S4uwXMqinrJhzRk +Oc2tb3y06XTAq1lMH2+58tqndAu8ni/UBz3OSghk2CTnZ1vxxXOd3CtQu4ypMq+k +qW0Umdrkk5TeAODNbrCy4c6vpICkQOljnRFWnDYu3aQ3JvaWZ/nObN7C72Lgpjfb +RfLXGmLsBCEIr028f9hpoeRCXoetUY2CiC2boAHR+cO6Jpvex4Jv5yYDpNKac52n +LLC8Cq5ozhOZNOSV6X9FpEca3rdhVUb0VgoNDCPZDdpO1PDJYCN9fUC8KUs+dsOh +SYGpliBaNsztoiJs1q5SkMQrMwmMmg== +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/certs/example.key b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/certs/example.key new file mode 100644 index 000000000..b21c2f08d --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/certs/example.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAu/4tXVpYiVwr2sjsi5FLxIK6yoGejIl5z8SsqIDaBt+Bn9hL +vctm53zjKnjkAipq7Nwjmr6GuSaxFnsIqoPOsb6Mdq9buMX7SVp60nBG3boBxBw6 +1APaKTY74OXVYcAWfi9QBDFv5fPpZ9JpA3RB8zvjPTpdQMOyvuDv41i0zOF/3xIl +Kgqd791UoncFzp7q2+GcJHA+2LHh6S0Sh0rpZ40djHAnrBJnxxEvN3GCN0P/g4Z8 +W6N2fzNs6LdJC3lmpVtwMdRPOnm3QbwSjGfOi0eGHONICazKnJKpldrGXehfxphS +QWd8J2D5YFejmk6I1+haH8PluchdYannbKjL6QIDAQABAoIBAEUBL7WsjAMfihls +1ycD1kPzmIzstz3u2H+jOZ1AbsdHE1WRF3w7RTKDbP8SEN+aolT/GTKb7OfZg/c0 +giHU7/Hed8C47XoNcgei5qKIA/svY6aQlidsoo+uEJykwIZ488itpTlkzCYkOfCa +E2HpMqwNt4OqAMDdFKdr+aIB1Zu+KPBxW23WD9wEWAbe5LA4YnRF9kT4YZ6y9mce +dGaIf39VtBlrGMmvoU0LE9B79nyuebGi0svW6QDarBqaDrnM/N3fXgL1kk/gVfan +/xs6EA4qPxA5G4h+enYrIlZL0CbSj60nYElo+Z5nRdBaRdCF/bpXOLyK/kXWLUM0 +f2HTK+ECgYEA5cVcpJtczxEaxoaEUbppsW1LCrTjJDGTKJ63G7/lwqCxJeCHN185 +nnckHOW2287e19bu9aUmKJgRq5s1rXnT+MnCl/hQnfaMrKOKtzE+t7/zsc9+LuAr +pwJrtZ9Dcnwrk8NOE0fPjW5XpDCSoEOo7JWZmGTVlpabOgNkjocfp+sCgYEA0XPt +ZPt3F0wyzgLYRhgnvp5CV8SzQmulsW+ytnL5eiAcNSXqni3wQHN3PGxLInEyQwBQ +/M8TQpUbqGMmahCK4ZxMAwXMrpF0mVB8jfoYMou1FSYPlUV+CvLjWcTkZB1Nirez +VFXdtfHP0mx4PbYK2qjB03u4pPHAN8kuayIf2nsCgYAbv/FHZAgabfto3Jggcr4P +Ep8MhPolxeL69egxbsSl89hRNcO+2T5ROBxhbRDfjSV2tduYSUDJiEwiCJW8BMmn +814QEopR+ZPVyc6X/1eOw5z/7YpUyPgcrHsrrTdtHTf6GY1VYMfdUeU9zCv5NRKy +uAKb2Bm/nSLUJ9K+L+2PzwKBgQDRQgT3UtTUjehkMitpPFDY/LxDe92simfsMjBW +X+Anx1TnNI6GolbZzYJe98LJElao4fQH38raRqZvQT/rz8MxTDoU+wJXljLryaHn +Jupt9W5hRrli5R7cSXYjBbc43p3N7WJY68CqOoDrNjubS/jkJJ4hcAY1pOHp2jFq +D5nLaQKBgAysU6O5kJ8yKxhbZflb42MqKCFBGrbRnbYx14PAEZOaRhzxehpppQmx +RLbn/z1Uh5Ms28ipxA+vnhyM3FcU5lKboaFyWJeuNslw0FxEcIai6hL6UkDznS4G +aqyzUjpG5Chg0x18xWYCbiGJwjZ9BWhtH+jojm856QHGzeQJWVoo +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/client_ca/example_ca.crt b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/client_ca/example_ca.crt new file mode 100644 index 000000000..b0fa2fa28 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/client_ca/example_ca.crt @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDATCCAemgAwIBAgIJAMsPuHg4mqnaMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV +BAMMC2V4YW1wbGUub3JnMCAXDTIwMDYxMDEwMzMwNloYDzIxMjAwNTE3MTAzMzA2 +WjAWMRQwEgYDVQQDDAtleGFtcGxlLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEP +ADCCAQoCggEBAL72D2gOnJN9Zvo9KjruwM1EsFe3xZRJ0NvZ5bHd6+5jhlgCAhQ+ +AGb7ufEiYOi2JWHl2Bkq0iVrp+zv0RLdq0oVjX+OG5H2yWbnC7ifbNjir93LX0un +tIqv5CIbExDSBRkufxfV37yjXdrcMqYSbD2Kw3PfAbWs1Dego8fRz8QAp5+LCvW2 +BZZyYi6JzhCAUW1+8OQPyzOhB50eSJiS5xgVA7wkwmYeVUpHqU8sv4VzjM3bmUc7 +1mPLlnRVIqScrqYgQ9Ou21vZebOJ8+ckVL8O3XHhMZlssBbWiBFZZnaNWbzcEI90 +oiW4YAQ5t7gaXuCVvaiNvq2VZknarR6AxcsCAwEAAaNQME4wHQYDVR0OBBYEFIH+ +0G3eCswQHbN06kvI80M3tNH9MB8GA1UdIwQYMBaAFIH+0G3eCswQHbN06kvI80M3 +tNH9MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAJtHx8ibLp+CYDR1 +ZVeQ3qg4OeRL0q21+2EgBJq6zOUt/9SxThA80aXJ8CYH9dnCW+fOnpEk1xFWxtXc +FwSLsnqAdwOJaaQWoMzhyqjZV5x5G9+MW5FzGGOdes2md2Z+tAwMoV9TVtxZkbKy +mC2tDJdvgLgt9/YcbUcZPDbyZojdZ+UbATm+Lro9dhTXt91vsAgz5QA9e08rQVkF +pc9+ZQ5zxBsoblQ+ozPOWOdV4zJVx+wQsAnOG2qU0yVQAscGsTo4wnzFrAU54fO7 +Lh4cOrY0P1/o65yiSzwK7f0jwBeT/jEfMOrJ7pPo7doUov0iVj0SZyTM3HFa2Mzj +2zGTk0o= +-----END CERTIFICATE----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/client_ca/example_ca.key b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/client_ca/example_ca.key new file mode 100644 index 000000000..4f7ad13b1 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/client_ca/example_ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAvvYPaA6ck31m+j0qOu7AzUSwV7fFlEnQ29nlsd3r7mOGWAIC +FD4AZvu58SJg6LYlYeXYGSrSJWun7O/REt2rShWNf44bkfbJZucLuJ9s2OKv3ctf +S6e0iq/kIhsTENIFGS5/F9XfvKNd2twyphJsPYrDc98BtazUN6Cjx9HPxACnn4sK +9bYFlnJiLonOEIBRbX7w5A/LM6EHnR5ImJLnGBUDvCTCZh5VSkepTyy/hXOMzduZ +RzvWY8uWdFUipJyupiBD067bW9l5s4nz5yRUvw7dceExmWywFtaIEVlmdo1ZvNwQ +j3SiJbhgBDm3uBpe4JW9qI2+rZVmSdqtHoDFywIDAQABAoIBAD60bbqtkZycwQPK +seNIIudEduNW5PocgwiuNE6DoMVWyPZ9MlGTSm6GmjgkIc5IgV30K1GYTgkboLic +xvp675QUH7KS51q2vsubcq3dK9DMHxOlhFVDbHVd7HuGiGwtip8KNZGOGTnIKzmC +tN7zjbdnqWaTA+y0I7tgdGdY7fBd9Rzgaq+OlPq2u33HvWHvlG/7PfpZuXB5YLgd +m04l7LJ7ikhIjycg7j27v/4c6xCiH5jMJKsZ+nfsQ0kEEo9DkhcKInK+wHsMzKsH +Cy3AdlE0IRsbxRAoMumVs2g5u90m3zBPkRrNdZ2Ni7BesnhxbkIqvb4SfpxKyuhK +fADfZgECgYEA7SUIS2gII0TGjXh0h16d+eoLVOz0eVpgF3XXxmgtAPu10dqxVEC2 +j5FSBCgZhqZ3axVotP71c2mT+hF+Mqy4TLMfA/B9jKLXjZlPbg4EcAgI7tALskwz +Bk5BkX0k825bU9P0j+AlpLx6/ztHr2N9/cKZfqQVrO9t+FRusvAHkHsCgYEAziT8 +F30Ch2s6IJngCj5jH164iN2CoFXjqPNVgRj45gLE3zrf1R7u2JTeEhjWNLZ6IWZ3 +G/bT7eYm6x8u7LFlORdnWKsHlftGu0igRyvIGcxoHgjXlsLidBaEP+HlOLUtTumu +MfQJUozLcrOBIV6m9VhPnSDTeCg/tOqy68V2pvECgYAUYgd5e8KfTW0Hgd/6Nq67 +aVt5/DfzKkpyGcXnHtMnb3ssQ3DUfg9y/ZmgE9ZF1Y8UHC34yKVOOzfl2ZUQQ/o/ +VXIIA6a27NQ8Ln4+RmQpQPeLl0Q6GgSUuSs3lxsS9VxSMzilGS4DH9QulejOcW3F +3vEUioP2bkn0e0VcifcMewKBgAG1Pr13FLFIiye//qI3GB0nbMH9i9qGO6entHqo +WU+WkEkFNNuQMQxsV1axC/1N0b87GRuLNQBQmtvx2zKs2Zjaf8m1SQ/OECz3EhTk +4PiNwAMXsamXHcc2dIwO9BY/MgvoVcAmNHmRnxHpONWs8hcwTyCPKBFjy/tUwny/ +mxcRAoGAcNRxLZlyRqmQ6zGf41GK4ZIR9gix0L6Km49S1maGFmcbctOR2GcQN8Eo +f38rkrFBfBfuFSGzghJiXDvKORg9r3V/bzSKcXkprJra6hzn5vn+t7wurjzaJUlK +zUW5dl3SU2bC4MK+X7bwqf9jm9b7FXSt4p1xly8Uh/Mufwi7zOM= +-----END RSA PRIVATE KEY----- diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/server_cert_conf.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/server_cert_conf.data new file mode 100644 index 000000000..49f228531 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/server_cert_conf.data @@ -0,0 +1,12 @@ +{ + "Version": "init version", + "Config": { + "Default": "example.org", + "CertConf": { + "example.org": { + "ServerCertFile": "tls_conf/certs/example.crt", + "ServerKeyFile" : "tls_conf/certs/example.key" + } + } + } +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/session_ticket_key.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/session_ticket_key.data new file mode 100644 index 000000000..b3d2356b0 --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/session_ticket_key.data @@ -0,0 +1,4 @@ +{ + "Version": "init version", + "SessionTicketKey": "08a0d852ef494143af613ef32d3c39314758885f7108e9ab021d55f422a454f7c9cd5a53978f48fa1063eadcdc06878f" +} diff --git a/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/tls_rule_conf.data b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/tls_rule_conf.data new file mode 100644 index 000000000..66e7b5dbb --- /dev/null +++ b/tests/integration/implementation/scenario-SC05-access-log-ai-fields/testdata/tls_conf/tls_rule_conf.data @@ -0,0 +1,20 @@ +{ + "Version": "12", + "DefaultNextProtos": ["http/1.1"], + "Config": { + "example_product": { + "VipConf": [ + "10.199.4.14" + ], + "SniConf": ["example.org"], + "CertName": "example.org", + "NextProtos": [ + "h2;rate=100;isw=65535;mcs=200;level=0", + "http/1.1" + ], + "Grade": "C", + "ClientAuth": false, + "ClientCAName": "example_ca" + } + } +} diff --git a/tests/integration/mod_ai_route/ai_route_integration_test.go b/tests/integration/mod_ai_route/ai_route_integration_test.go deleted file mode 100644 index d2a5fabf5..000000000 --- a/tests/integration/mod_ai_route/ai_route_integration_test.go +++ /dev/null @@ -1,562 +0,0 @@ -// Copyright (c) 2026 The BFE Authors. -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package mod_ai_route_integration - -import ( - "bytes" - "encoding/json" - "fmt" - "io" - "io/ioutil" - "math/rand" - "net" - "net/http" - "net/http/httptest" - "net/url" - "os" - "path/filepath" - "strings" - "sync" - "testing" - "time" - - "github.com/bfenetworks/bfe/bfe_basic" - "github.com/bfenetworks/bfe/bfe_config/bfe_conf" - "github.com/bfenetworks/bfe/bfe_http" - "github.com/bfenetworks/bfe/bfe_modules" - "github.com/bfenetworks/bfe/bfe_server" -) - -var modulesOnce sync.Once - -func TestMain(m *testing.M) { - modulesOnce.Do(bfe_modules.SetModules) - os.Exit(m.Run()) -} - -// fakeConn implements net.Conn for test environment. -type fakeConn struct{} - -func (c *fakeConn) Read(b []byte) (int, error) { return 0, nil } -func (c *fakeConn) Write(b []byte) (int, error) { return len(b), nil } -func (c *fakeConn) Close() error { return nil } -func (c *fakeConn) LocalAddr() net.Addr { return &net.TCPAddr{IP: net.ParseIP("127.0.0.1"), Port: 0} } -func (c *fakeConn) RemoteAddr() net.Addr { return &net.TCPAddr{IP: net.ParseIP("127.0.0.1"), Port: 0} } -func (c *fakeConn) SetDeadline(t time.Time) error { return nil } -func (c *fakeConn) SetReadDeadline(t time.Time) error { return nil } -func (c *fakeConn) SetWriteDeadline(t time.Time) error { return nil } - -const ( - apiHost = "api.example.org" - otherHost = "other.example.org" - entityHost = "entity.example.org" - unknownHost = "unknown.example.org" - apiPath = "/v1/chat/completions" - apiKeyUserA = "ak_user_a" - apiKeyUserB = "ak_user_b" - apiKeyNoBinding = "ak_no_binding" -) - -var clusterNames = []string{ - "cluster_primary_a", - "cluster_primary_b", - "cluster_primary_c", - "cluster_fallback_1", - "cluster_fallback_2", - "cluster_entity_default", - "cluster_global_default", -} - -// responseRecorder implements bfe_http.ResponseWriter for test verification. -type responseRecorder struct { - statusCode int - header bfe_http.Header - body *bytes.Buffer -} - -func newResponseRecorder() *responseRecorder { - return &responseRecorder{ - statusCode: 200, - header: make(bfe_http.Header), - body: new(bytes.Buffer), - } -} - -func (r *responseRecorder) Header() bfe_http.Header { - return r.header -} - -func (r *responseRecorder) Write(p []byte) (int, error) { - return r.body.Write(p) -} - -func (r *responseRecorder) WriteHeader(statusCode int) { - r.statusCode = statusCode -} - -// backendServer wraps an httptest.Server and records request metadata. -type backendServer struct { - server *httptest.Server - clusterName string - response int - body string - hits int - mu sync.Mutex - models []string -} - -func newBackendServer(clusterName string, response int, body string) *backendServer { - b := &backendServer{ - clusterName: clusterName, - response: response, - body: body, - } - b.server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - b.mu.Lock() - b.hits++ - if r.Body != nil { - bodyBytes, _ := io.ReadAll(r.Body) - var reqBody map[string]interface{} - if err := json.Unmarshal(bodyBytes, &reqBody); err == nil { - if model, ok := reqBody["model"].(string); ok { - b.models = append(b.models, model) - } - } - } - b.mu.Unlock() - w.WriteHeader(b.response) - if b.body != "" { - w.Write([]byte(b.body)) - } - })) - return b -} - -func (b *backendServer) Hits() int { - b.mu.Lock() - defer b.mu.Unlock() - return b.hits -} - -func (b *backendServer) Models() []string { - b.mu.Lock() - defer b.mu.Unlock() - return append([]string(nil), b.models...) -} - -func (b *backendServer) Close() { - b.server.Close() -} - -func (b *backendServer) Addr() string { - u, _ := url.Parse(b.server.URL) - return u.Host -} - -func (b *backendServer) HostPort() (string, int) { - u, _ := url.Parse(b.server.URL) - host := u.Hostname() - port := 80 - if u.Port() != "" { - fmt.Sscanf(u.Port(), "%d", &port) - } - return host, port -} - -// testEnv holds all resources for a single integration test. -type testEnv struct { - t *testing.T - srv *bfe_server.BfeServer - backends map[string]*backendServer - tempDir string -} - -func newTestEnv(t *testing.T, responseMap map[string]int) *testEnv { - e := &testEnv{ - t: t, - backends: make(map[string]*backendServer), - } - - // start mock backends - for _, name := range clusterNames { - resp, ok := responseMap[name] - if !ok { - resp = http.StatusOK - } - e.backends[name] = newBackendServer(name, resp, fmt.Sprintf("response from %s", name)) - } - - // prepare temp conf dir - e.tempDir = t.TempDir() - if err := copyDir("testdata", e.tempDir); err != nil { - t.Fatalf("copy testdata failed: %v", err) - } - - // generate cluster_table.data with actual backend addresses - if err := e.generateClusterTable(); err != nil { - t.Fatalf("generate cluster table failed: %v", err) - } - - // load bfe config - confPath := filepath.Join(e.tempDir, "bfe.conf") - cfg, err := bfe_conf.BfeConfigLoad(confPath, e.tempDir) - if err != nil { - t.Fatalf("load bfe config failed: %v", err) - } - - // create server - e.srv = bfe_server.NewBfeServer(cfg, e.tempDir, "test") - - // init web monitor (needed by InitModules) - if err := e.srv.InitWebMonitor(0, ""); err != nil { - t.Fatalf("init web monitor failed: %v", err) - } - - // register and init modules - if err := e.srv.RegisterModules(cfg.Server.Modules); err != nil { - t.Fatalf("register modules failed: %v", err) - } - if err := e.srv.InitModules(); err != nil { - t.Fatalf("init modules failed: %v", err) - } - - // load server data conf (cluster, host, route, etc.) - if err := e.srv.InitDataLoad(); err != nil { - t.Fatalf("init data load failed: %v", err) - } - - return e -} - -func (e *testEnv) Close() { - for _, b := range e.backends { - b.Close() - } -} - -func (e *testEnv) generateClusterTable() error { - clusterTable := map[string]interface{}{ - "Version": "20260720150000", - "Config": map[string]interface{}{}, - } - config := clusterTable["Config"].(map[string]interface{}) - - for _, name := range clusterNames { - b := e.backends[name] - host, port := b.HostPort() - config[name] = map[string]interface{}{ - "sub_" + clusterSubName(name): []map[string]interface{}{ - { - "name": name + "-backend-0", - "addr": host, - "port": port, - "weight": 100, - }, - }, - } - } - - data, err := json.MarshalIndent(clusterTable, "", " ") - if err != nil { - return err - } - - path := filepath.Join(e.tempDir, "cluster_conf", "cluster_table.data") - return ioutil.WriteFile(path, data, 0644) -} - -func clusterSubName(clusterName string) string { - switch clusterName { - case "cluster_primary_a": - return "a" - case "cluster_primary_b": - return "b" - case "cluster_primary_c": - return "c" - case "cluster_fallback_1": - return "fb1" - case "cluster_fallback_2": - return "fb2" - case "cluster_entity_default": - return "entity" - case "cluster_global_default": - return "global" - } - return "sub" -} - -func (e *testEnv) newRequest(host, apiKey string, body []byte) *bfe_basic.Request { - var bodyReader io.Reader - if body != nil { - bodyReader = bytes.NewReader(body) - } - req, err := bfe_http.NewRequest(http.MethodPost, "http://"+host+apiPath, bodyReader) - if err != nil { - e.t.Fatalf("new request failed: %v", err) - } - req.Host = host - req.State = &bfe_http.RequestState{} - if body != nil { - req.Header.Set("Content-Type", "application/json") - } - - basicReq := bfe_basic.NewRequest(req, &fakeConn{}, bfe_basic.NewRequestStat(time.Now()), - bfe_basic.NewSession(&fakeConn{}), e.srv.GetServerConf()) - basicReq.Route.Product = "ai_product" - - aiMeta := basicReq.InitAiBasicInfo() - aiMeta.ClientApiKey = apiKey - - return basicReq -} - -var emptyJSONBody = []byte("{}") - -func (e *testEnv) callServeHTTPForAI(host, apiKey string, body []byte) *responseRecorder { - rec := newResponseRecorder() - basicReq := e.newRequest(host, apiKey, body) - e.srv.ReverseProxy.ServeHTTPForAI(rec, basicReq) - if rec.statusCode != http.StatusOK { - e.t.Logf("ServeHTTPForAI returned status %d, body: %q", rec.statusCode, rec.body.String()) - } - return rec -} - -func copyDir(src, dst string) error { - entries, err := ioutil.ReadDir(src) - if err != nil { - return err - } - - for _, entry := range entries { - srcPath := filepath.Join(src, entry.Name()) - dstPath := filepath.Join(dst, entry.Name()) - - if entry.IsDir() { - if err := os.MkdirAll(dstPath, 0755); err != nil { - return err - } - if err := copyDir(srcPath, dstPath); err != nil { - return err - } - } else { - data, err := ioutil.ReadFile(srcPath) - if err != nil { - return err - } - if err := ioutil.WriteFile(dstPath, data, 0644); err != nil { - return err - } - } - } - return nil -} - -// TestMultiRouteTablesApikeyHit verifies that a specific apikey rule is -// matched before falling through to subsequent route tables. -func TestMultiRouteTablesApikeyHit(t *testing.T) { - e := newTestEnv(t, map[string]int{ - "cluster_primary_a": http.StatusOK, - "cluster_primary_b": http.StatusOK, - "cluster_primary_c": http.StatusOK, - }) - defer e.Close() - - rec := e.callServeHTTPForAI(apiHost, apiKeyUserA, emptyJSONBody) - if rec.statusCode != http.StatusOK { - t.Fatalf("expected status 200, got %d, body: %s", rec.statusCode, rec.body.String()) - } - - // one of the primary clusters should be hit - hits := e.backends["cluster_primary_a"].Hits() + - e.backends["cluster_primary_b"].Hits() + - e.backends["cluster_primary_c"].Hits() - if hits != 1 { - t.Fatalf("expected exactly one primary cluster hit, got %d", hits) - } -} - -// TestMultiRouteTablesEntityFallback verifies that multiple route tables are -// searched in binding order when the apikey table has no matching rule. -func TestMultiRouteTablesEntityFallback(t *testing.T) { - e := newTestEnv(t, map[string]int{ - "cluster_entity_default": http.StatusOK, - }) - defer e.Close() - - rec := e.callServeHTTPForAI(otherHost, apiKeyUserA, nil) - if rec.statusCode != http.StatusOK { - t.Fatalf("expected status 200, got %d, body: %s", rec.statusCode, rec.body.String()) - } - - if e.backends["cluster_entity_default"].Hits() != 1 { - t.Fatalf("expected cluster_entity_default hit once, got %d", e.backends["cluster_entity_default"].Hits()) - } -} - -// TestMultiRouteTablesNoBinding verifies that an apikey without binding -// results in 404. -func TestMultiRouteTablesNoBinding(t *testing.T) { - e := newTestEnv(t, map[string]int{}) - defer e.Close() - - rec := e.callServeHTTPForAI(apiHost, apiKeyNoBinding, nil) - if rec.statusCode != http.StatusNotFound { - t.Fatalf("expected status 404, got %d", rec.statusCode) - } - body := rec.body.String() - if !strings.Contains(body, "AI route not found") { - t.Fatalf("expected 'AI route not found' in body, got %q", body) - } -} - -// TestMultiTargetsWeightedSelection verifies weighted random target selection. -func TestMultiTargetsWeightedSelection(t *testing.T) { - e := newTestEnv(t, map[string]int{ - "cluster_primary_a": http.StatusOK, - "cluster_primary_b": http.StatusOK, - "cluster_primary_c": http.StatusOK, - }) - defer e.Close() - - const total = 1000 - for i := 0; i < total; i++ { - rec := e.callServeHTTPForAI(apiHost, apiKeyUserA, emptyJSONBody) - if rec.statusCode != http.StatusOK { - t.Fatalf("request %d: expected status 200, got %d", i, rec.statusCode) - } - } - - hitsA := e.backends["cluster_primary_a"].Hits() - hitsB := e.backends["cluster_primary_b"].Hits() - hitsC := e.backends["cluster_primary_c"].Hits() - - t.Logf("hits distribution: a=%d b=%d c=%d", hitsA, hitsB, hitsC) - - if hitsA+hitsB+hitsC != total { - t.Fatalf("expected total hits %d, got %d", total, hitsA+hitsB+hitsC) - } - - // weights: 60 / 30 / 10, allow ±50 tolerance - if hitsA < 550 || hitsA > 650 { - t.Fatalf("expected hitsA around 600, got %d", hitsA) - } - if hitsB < 250 || hitsB > 350 { - t.Fatalf("expected hitsB around 300, got %d", hitsB) - } - if hitsC < 50 || hitsC > 150 { - t.Fatalf("expected hitsC around 100, got %d", hitsC) - } -} - -// TestMultiFallbacksSuccess verifies that fallbacks are tried until one -// returns success. -func TestMultiFallbacksSuccess(t *testing.T) { - e := newTestEnv(t, map[string]int{ - "cluster_primary_a": http.StatusInternalServerError, - "cluster_primary_b": http.StatusInternalServerError, - "cluster_primary_c": http.StatusInternalServerError, - "cluster_fallback_1": http.StatusBadGateway, - "cluster_fallback_2": http.StatusOK, - }) - defer e.Close() - - // force deterministic target selection for this test: select primary_a - // by setting its weight to 100 and others to 0 through route data is not - // possible at runtime, so we make all primaries fail and verify fallback. - rec := e.callServeHTTPForAI(apiHost, apiKeyUserA, emptyJSONBody) - if rec.statusCode != http.StatusOK { - t.Fatalf("expected status 200, got %d", rec.statusCode) - } - - // all three primaries may be selected; at least one primary and both - // fallbacks should be attempted across retries - primaryHits := e.backends["cluster_primary_a"].Hits() + - e.backends["cluster_primary_b"].Hits() + - e.backends["cluster_primary_c"].Hits() - if primaryHits < 1 { - t.Fatalf("expected at least one primary hit, got %d", primaryHits) - } - if e.backends["cluster_fallback_1"].Hits() != 1 { - t.Fatalf("expected cluster_fallback_1 hit once, got %d", e.backends["cluster_fallback_1"].Hits()) - } - if e.backends["cluster_fallback_2"].Hits() != 1 { - t.Fatalf("expected cluster_fallback_2 hit once, got %d", e.backends["cluster_fallback_2"].Hits()) - } -} - -// TestMultiFallbacksAllFail verifies that all fallbacks are exhausted when -// every attempt fails. -func TestMultiFallbacksAllFail(t *testing.T) { - e := newTestEnv(t, map[string]int{ - "cluster_primary_a": http.StatusInternalServerError, - "cluster_primary_b": http.StatusInternalServerError, - "cluster_primary_c": http.StatusInternalServerError, - "cluster_fallback_1": http.StatusInternalServerError, - "cluster_fallback_2": http.StatusInternalServerError, - }) - defer e.Close() - - rec := e.callServeHTTPForAI(apiHost, apiKeyUserA, emptyJSONBody) - if rec.statusCode != http.StatusInternalServerError { - t.Fatalf("expected status 500, got %d", rec.statusCode) - } - - if e.backends["cluster_fallback_2"].Hits() != 1 { - t.Fatalf("expected cluster_fallback_2 hit once, got %d", e.backends["cluster_fallback_2"].Hits()) - } -} - -// TestModelOverrideAndFallback verifies model override for target and fallback. -func TestModelOverrideAndFallback(t *testing.T) { - // Make all primaries fail so the request deterministically falls back. - e := newTestEnv(t, map[string]int{ - "cluster_primary_a": http.StatusInternalServerError, - "cluster_primary_b": http.StatusInternalServerError, - "cluster_primary_c": http.StatusInternalServerError, - "cluster_fallback_1": http.StatusOK, - }) - defer e.Close() - - body := []byte(`{"model":"origin-model"}`) - rec := e.callServeHTTPForAI(apiHost, apiKeyUserA, body) - if rec.statusCode != http.StatusOK { - t.Fatalf("expected status 200, got %d", rec.statusCode) - } - - // force deterministic target selection is hard; verify that at least one - // primary backend received target model and fallback received fallback model - var primaryGotTargetModel bool - for _, name := range []string{"cluster_primary_a", "cluster_primary_b", "cluster_primary_c"} { - models := e.backends[name].Models() - for _, m := range models { - if strings.Contains(m, "target-model-") { - primaryGotTargetModel = true - } - } - } - if !primaryGotTargetModel { - t.Fatalf("expected at least one primary backend to receive target model") - } - - fallbackModels := e.backends["cluster_fallback_1"].Models() - if len(fallbackModels) != 1 || fallbackModels[0] != "fallback-model-1" { - t.Fatalf("expected fallback model 'fallback-model-1', got %v", fallbackModels) - } -} - -func init() { - rand.Seed(time.Now().UnixNano()) -} diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-01-APIKey\350\267\257\347\224\261\350\241\250\345\221\275\344\270\255.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-01-APIKey\350\267\257\347\224\261\350\241\250\345\221\275\344\270\255.md" new file mode 100644 index 000000000..6fb268329 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-01-APIKey\350\267\257\347\224\261\350\241\250\345\221\275\344\270\255.md" @@ -0,0 +1,51 @@ +# TC-01 APIKey 路由表命中 + +## 用例编号与名称 + +TC-01 APIKey 路由表命中 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 apikey 级路由表存在匹配规则时，BFE 直接命中 apikey 表，不再继续搜索 entity/global 路由表。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动，所有 cluster 默认返回 200。 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `ai_route.data` 中 `apikey_ak_user_a` 表的 `user_a-rule1` 规则命中 `api.example.org`。 +- `ApikeyRouteTableBindings` 中 `ak_user_a` 绑定 `[apikey_ak_user_a, entity_dept_ai, global_default]`。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_primary_a`、`cluster_primary_b`、`cluster_primary_c` 中恰好有 1 个被命中 +- `cluster_entity_default`、`cluster_global_default` 未被命中 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-02-Entity\350\267\257\347\224\261\350\241\250\345\233\236\351\200\200.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-02-Entity\350\267\257\347\224\261\350\241\250\345\233\236\351\200\200.md" new file mode 100644 index 000000000..b10c6dc83 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-02-Entity\350\267\257\347\224\261\350\241\250\345\233\236\351\200\200.md" @@ -0,0 +1,52 @@ +# TC-02 Entity 路由表回退 + +## 用例编号与名称 + +TC-02 Entity 路由表回退 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 apikey 级路由表无匹配规则时，BFE 按 `ApikeyRouteTableBindings` 绑定顺序继续搜索，并命中 entity 级默认规则。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动，所有 cluster 默认返回 200。 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `ai_route.data` 中 `apikey_ak_user_a` 表的规则仅命中 `api.example.org`。 +- `entity_dept_ai` 表的 `dept_ai-default` 规则使用 `default_t()`，命中 `cluster_entity_default`。 +- `ApikeyRouteTableBindings` 中 `ak_user_a` 绑定 `[apikey_ak_user_a, entity_dept_ai, global_default]`。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `other.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_entity_default` 被命中 1 次 +- `cluster_primary_a/b/c`、`cluster_global_default` 未被命中 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-03-\346\227\240\347\273\221\345\256\232\350\277\224\345\233\236404.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-03-\346\227\240\347\273\221\345\256\232\350\277\224\345\233\236404.md" new file mode 100644 index 000000000..aa7d49dab --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-03-\346\227\240\347\273\221\345\256\232\350\277\224\345\233\236404.md" @@ -0,0 +1,50 @@ +# TC-03 无绑定返回 404 + +## 用例编号与名称 + +TC-03 无绑定返回 404 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证未在 `ApikeyRouteTableBindings` 中绑定的 API-Key，BFE 返回 404 且不转发到任何后端。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动，所有 cluster 默认返回 200。 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `ApikeyRouteTableBindings` 中不包含 `ak_no_binding`。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_no_binding` | +| Body | `{}` | + +## 预期结果 + +- 响应状态码：404 +- 响应体包含 `AI route not found` +- 所有 mock 后端命中次数均为 0 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-04-\345\244\232Targets\345\212\240\346\235\203\351\200\211\346\213\251.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-04-\345\244\232Targets\345\212\240\346\235\203\351\200\211\346\213\251.md" new file mode 100644 index 000000000..790b16b67 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-04-\345\244\232Targets\345\212\240\346\235\203\351\200\211\346\213\251.md" @@ -0,0 +1,57 @@ +# TC-04 多 Targets 加权选择 + +## 用例编号与名称 + +TC-04 多 Targets 加权选择 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证同一规则下多个 target 按 weight 加权随机选择。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动，所有 primary cluster 返回 200。 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `user_a-rule1` 的 targets 权重为： + - `cluster_primary_a`：60 + - `cluster_primary_b`：30 + - `cluster_primary_c`：10 + +## BFE 请求 + +连续发送 1000 次： + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{}` | + +## 预期结果 + +- 每次响应状态码均为 200 +- 总命中次数 = 1000 +- `cluster_primary_a` 命中次数约 600（允许 ±50） +- `cluster_primary_b` 命中次数约 300（允许 ±50） +- `cluster_primary_c` 命中次数约 100（允许 ±50） + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-05-\345\244\232Fallbacks\346\234\200\347\273\210\346\210\220\345\212\237.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-05-\345\244\232Fallbacks\346\234\200\347\273\210\346\210\220\345\212\237.md" new file mode 100644 index 000000000..c09582fb1 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-05-\345\244\232Fallbacks\346\234\200\347\273\210\346\210\220\345\212\237.md" @@ -0,0 +1,54 @@ +# TC-05 多 Fallbacks 最终成功 + +## 用例编号与名称 + +TC-05 多 Fallbacks 最终成功 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 primary cluster 失败后，BFE 按 fallback 链路依次降级，并最终成功。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已按以下规则配置： + - 所有 primary cluster 返回 500 + - `cluster_fallback_1` 返回 502 + - `cluster_fallback_2` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `user_a-rule1` 配置 3 个 primary targets 与 2 个 fallbacks。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{}` | + +## 预期结果 + +- 响应状态码：200 +- 至少 1 个 primary cluster 被尝试 +- `cluster_fallback_1` 被尝试 1 次 +- `cluster_fallback_2` 被尝试 1 次并返回成功响应 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-06-\345\244\232Fallbacks\345\205\250\351\203\250\345\244\261\350\264\245.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-06-\345\244\232Fallbacks\345\205\250\351\203\250\345\244\261\350\264\245.md" new file mode 100644 index 000000000..82bb09efb --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-06-\345\244\232Fallbacks\345\205\250\351\203\250\345\244\261\350\264\245.md" @@ -0,0 +1,52 @@ +# TC-06 多 Fallbacks 全部失败 + +## 用例编号与名称 + +TC-06 多 Fallbacks 全部失败 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当所有 primary 与 fallback cluster 均失败时，BFE 返回最后一个错误响应。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已按以下规则配置： + - 所有 primary cluster 返回 500 + - `cluster_fallback_1` 返回 500 + - `cluster_fallback_2` 返回 500 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `user_a-rule1` 配置 3 个 primary targets 与 2 个 fallbacks。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{}` | + +## 预期结果 + +- 响应状态码：500 +- `cluster_fallback_2` 被尝试 1 次（确认 fallback 链路已穷尽） + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-07-Target\344\270\216Fallback\346\250\241\345\236\213\350\246\206\347\233\226.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-07-Target\344\270\216Fallback\346\250\241\345\236\213\350\246\206\347\233\226.md" new file mode 100644 index 000000000..439688c0e --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-07-Target\344\270\216Fallback\346\250\241\345\236\213\350\246\206\347\233\226.md" @@ -0,0 +1,54 @@ +# TC-07 Target 与 Fallback 模型覆盖 + +## 用例编号与名称 + +TC-07 Target 与 Fallback 模型覆盖 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 target 与 fallback 切换时，请求体中的 `model` 字段被正确覆盖，且原始请求体内容被完整保留。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已按以下规则配置： + - 所有 primary cluster 返回 500 + - `cluster_fallback_1` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `user_a-rule1` 的 targets 设置 `Model` 为 `target-model-a/b/c`。 +- `user_a-rule1` 的 fallbacks 设置 `Model` 为 `fallback-model-1`。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"origin-model","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- 至少 1 个 primary backend 收到 `target-model-*` +- `cluster_fallback_1` 收到的请求体中 `model` 为 `fallback-model-1` +- `cluster_fallback_1` 收到的请求体中 `messages` 字段保持原样 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-08-Fallback\346\227\266\351\203\250\345\210\206\345\267\262\345\217\221\351\200\201body\345\217\257\345\256\214\346\225\264\345\233\236\347\273\225.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-08-Fallback\346\227\266\351\203\250\345\210\206\345\267\262\345\217\221\351\200\201body\345\217\257\345\256\214\346\225\264\345\233\236\347\273\225.md" new file mode 100644 index 000000000..d00287ecf --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-08-Fallback\346\227\266\351\203\250\345\210\206\345\267\262\345\217\221\351\200\201body\345\217\257\345\256\214\346\225\264\345\233\236\347\273\225.md" @@ -0,0 +1,58 @@ +# TC-08 Fallback 时部分已发送 body 可完整回绕 + +## 用例编号与名称 + +TC-08 Fallback 时部分已发送 body 可完整回绕 + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe.conf` 中 `accessibleBodySize = 4194304` + +## 测试目的 + +验证当请求 body 已经被部分发送给 primary cluster、且 primary 连接中途关闭时，BFE 仍能将完整 body 回绕并发送给 fallback cluster。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动；`cluster_fallback_1` 返回 200。 +3. `cluster_primary_a` 被配置为读取 1 KB 请求体后强制关闭连接。 +4. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `ai_route.data` 中新增 `user_a-large` 规则： + - 条件：`req_host_in("large.example.org")` + - targets：`cluster_primary_a`（`Model` 为空） + - fallbacks：`cluster_fallback_1`（`Model` 为空） +- `server_data_conf/host_rule.data` 增加 `large.example.org`。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `large.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Content-Type | `application/octet-stream` | +| Body | 1 MB 确定性字节序列 | + +## 预期结果 + +- `cluster_primary_a` 被尝试 1 次（读取 1 KB 后关闭连接） +- `cluster_fallback_1` 被命中 1 次 +- `cluster_fallback_1` 收到的请求体长度等于 1 MB +- `cluster_fallback_1` 收到的请求体内容与发送内容完全一致 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-09-body\350\266\205\350\277\207accessibleBodySize\346\227\266\346\227\240\346\263\225fallback.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-09-body\350\266\205\350\277\207accessibleBodySize\346\227\266\346\227\240\346\263\225fallback.md" new file mode 100644 index 000000000..c7980af00 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-09-body\350\266\205\350\277\207accessibleBodySize\346\227\266\346\227\240\346\263\225fallback.md" @@ -0,0 +1,56 @@ +# TC-09 body 超过 accessibleBodySize 时无法 fallback + +## 用例编号与名称 + +TC-09 body 超过 accessibleBodySize 时无法 fallback + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe.conf` 中 `accessibleBodySize = 4194304` + +## 测试目的 + +验证当请求 body 大小超过 `accessibleBodySize` 时，BFE 无法将 body 完整回绕；primary cluster 失败后，fallback 不会被执行。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动：`cluster_primary_a` 返回 500，`cluster_fallback_1` 返回 200。 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- 使用 `user_a-large` 规则： + - 条件：`req_host_in("large.example.org")` + - targets：`cluster_primary_a`（`Model` 为空） + - fallbacks：`cluster_fallback_1`（`Model` 为空） + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `large.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Content-Type | `application/octet-stream` | +| Body | 5 MB 确定性字节序列（大于 4 MB） | + +## 预期结果 + +- `cluster_primary_a` 被尝试 1 次 +- `cluster_fallback_1` 命中次数为 0 +- BFE 尝试 fallback 但因 body 无法回绕而中止（日志中出现 `fallback aborted, request body cannot be rewound`） +- 客户端请求未通过 fallback 成功（即未返回 200） + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-10-\350\266\205\350\277\207totalBodyBufferSize\346\227\266\346\227\240\346\263\225fallback.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-10-\350\266\205\350\277\207totalBodyBufferSize\346\227\266\346\227\240\346\263\225fallback.md" new file mode 100644 index 000000000..e81aa001e --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/TC-10-\350\266\205\350\277\207totalBodyBufferSize\346\227\266\346\227\240\346\263\225fallback.md" @@ -0,0 +1,79 @@ +# TC-10 超过 totalBodyBufferSize 时无法 fallback + +## 用例编号与名称 + +TC-10 超过 totalBodyBufferSize 时无法 fallback + +## 所属场景 + +SC01 路由表查找与绑定 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe.conf` 中 `accessibleBodySize = 4194304` + +## 测试目的 + +验证当全局 bytes_body 缓冲区已经达到 `totalBodyBufferSize` 上限时，后续请求的 fallback 被禁用：BFE 不会为新请求分配可回绕的 body 缓冲区，primary cluster 失败后不会尝试 fallback。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_primary_a` 返回 500 + - `cluster_fallback_1` 返回 200 + - `cluster_holder` 接收请求后阻塞在读取 body 之前 +3. 临时 BFE 配置已生成并加载，且 `totalBodyBufferSize` 被覆盖为 2 MB。 + +## 配置构造 + +- 使用 `user_a-large` 规则： + - 条件：`req_host_in("large.example.org")` + - targets：`cluster_primary_a`（`Model` 为空） + - fallbacks：`cluster_fallback_1`（`Model` 为空） +- 使用 `user_a-holder` 规则： + - 条件：`req_host_in("holder.example.org")` + - targets：`cluster_holder`（`Model` 为空） + - fallbacks：`cluster_fallback_2`（`Model` 为空，用于触发 body 包装） +- `cluster_holder` 的 `BackendConf` 与 `ClusterBasic` 超时设置为 60 s，确保在测试请求期间不会超时。 + +## BFE 请求 + +### holder 请求（占用缓冲区） + +| 字段 | 值 | +|------|-----| +| Host | `holder.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Content-Type | `application/octet-stream` | +| Body | 2 MB 确定性字节序列 | + +`cluster_holder` 后端在收到请求头后、读取 body 前阻塞，使 BFE 无法完成 body 写入并关闭 bytes_body 缓冲区，从而将全局 total 维持在 2 MB。 + +### test 请求（验证 fallback 禁用） + +| 字段 | 值 | +|------|-----| +| Host | `large.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Content-Type | `application/octet-stream` | +| Body | 512 KB 确定性字节序列 | + +## 预期结果 + +- 通过 BFE monitor 接口 `/monitor/server_stat` 观察到 `total_bytes_body_buffer` 达到 2 MB 后再发送 test 请求。 +- `cluster_primary_a` 被尝试 1 次。 +- `cluster_fallback_1` 命中次数为 0。 +- BFE 日志中出现 `request body is not rewindable, disable fallback`。 +- test 请求未通过 fallback 成功（即未返回 200）。 + +## 清理 + +关闭 holder 后端阻塞，停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/\345\234\272\346\231\257\350\257\264\346\230\216.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/\345\234\272\346\231\257\350\257\264\346\230\216.md" new file mode 100644 index 000000000..d2dc8646c --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC01-\350\267\257\347\224\261\350\241\250\346\237\245\346\211\276\344\270\216\347\273\221\345\256\232/\345\234\272\346\231\257\350\257\264\346\230\216.md" @@ -0,0 +1,87 @@ +# SC01 路由表查找与绑定 + +## 1. 场景背景与目的 + +`mod_ai_route` 支持按 **apikey → entity → global** 的顺序搜索多级路由表。本场景验证： + +- 当 apikey 级路由表存在匹配规则时，直接命中，不再回退； +- 当 apikey 级路由表无匹配规则时，按 `ApikeyRouteTableBindings` 中的绑定顺序继续搜索 entity/global 路由表； +- 未在 `ApikeyRouteTableBindings` 中绑定的 API-Key，BFE 返回 404。 + +## 2. 运行模式 + +- **单组件模式**：仅启动真实 `bfe` 进程。 +- 不涉及 `ai-gateway-api` 与 `conf-agent`。 + +## 3. 涉及的 BFE 配置文件 + +| 文件 | 说明 | +|------|------| +| `bfe.conf` | 启用 `EnableAiGateway`，加载 `mod_ai_route` | +| `mod_ai_route/mod_ai_route.conf` | 指定 `ai_route.data` 路径 | +| `mod_ai_route/ai_route.data` | 定义 3 个路由表与 `ApikeyRouteTableBindings` | +| `server_data_conf/host_rule.data` | 声明 `ai_product` 产品线对应的 Host | +| `server_data_conf/route_rule.data` | 默认路由到 `cluster_global_default` | +| `cluster_conf/cluster_conf.data` | 各 cluster 的基础配置 | +| `cluster_conf/gslb.data` | GSLB 权重配置 | +| `cluster_conf/cluster_table.data` | 运行时根据 mock 后端地址动态生成 | + +## 4. 路由表设计 + +### 4.1 apikey_ak_user_a（apikey 级别） + +- `user_a-rule1` + - 条件：`req_host_in("api.example.org")` + - targets：3 个，weight 分别为 60、30、10 + - fallbacks：2 个 +- `user_a-large` + - 条件：`req_host_in("large.example.org")` + - targets：`cluster_primary_a`，无 model 覆盖 + - fallbacks：`cluster_fallback_1`，无 model 覆盖 +- `user_a-holder` + - 条件：`req_host_in("holder.example.org")` + - targets：`cluster_holder`，无 model 覆盖 + - fallbacks：`cluster_fallback_2`，无 model 覆盖（仅用于触发 body 包装） + +### 4.2 entity_dept_ai（entity 级别） + +- `entity-rule1` + - 条件：`req_host_in("entity.example.org")` + - targets：`cluster_entity_default` +- `dept_ai-default` + - 条件：`default_t()` + - targets：`cluster_entity_default` + +### 4.3 global_default（global 级别） + +- `global-default` + - 条件：`default_t()` + - targets：`cluster_global_default` + +### 4.4 ApikeyRouteTableBindings + +```json +{ + "ak_user_a": ["apikey_ak_user_a", "entity_dept_ai", "global_default"], + "ak_user_b": ["entity_dept_ai", "global_default"] +} +``` + +## 5. 测试例列表 + +| 编号 | 名称 | 核心验证点 | +|------|------|------------| +| TC-01 | APIKey 路由表命中 | apikey 表命中时直接返回，不回退 entity/global | +| TC-02 | Entity 路由表回退 | apikey 表无命中时，按绑定顺序命中 entity 默认规则 | +| TC-03 | 无绑定返回 404 | 未绑定 API-Key 返回 404，且所有后端无命中 | +| TC-04 | 多 Targets 加权选择 | 1000 次请求按 weight 60/30/10 分布 | +| TC-05 | 多 Fallbacks 最终成功 | primary 失败后依次降级，最终 fallback2 成功 | +| TC-06 | 多 Fallbacks 全部失败 | 全部 fallback 失败后返回最后一个错误响应 | +| TC-07 | Target 与 Fallback 模型覆盖 | target/fallback 切换时请求体 model 被正确覆盖 | +| TC-08 | Fallback 时部分已发送 body 可完整回绕 | primary 连接中途关闭后，fallback 后端仍收到完整 body | +| TC-09 | body 超过 accessibleBodySize 时无法 fallback | body 超过 4MB 后，primary 失败时 BFE 无法回退到 fallback | +| TC-10 | 超过 totalBodyBufferSize 时无法 fallback | 全局 bytes_body 缓冲区达到上限后，新请求的 fallback 被禁用 | + +## 6. 与其他场景的依赖关系 + +无依赖，可作为首个场景独立运行。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-01-\345\244\232Key\345\212\240\346\235\203\351\200\211\346\213\251.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-01-\345\244\232Key\345\212\240\346\235\203\351\200\211\346\213\251.md" new file mode 100644 index 000000000..e5d5d9651 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-01-\345\244\232Key\345\212\240\346\235\203\351\200\211\346\213\251.md" @@ -0,0 +1,61 @@ +# TC-01 多 Key 加权选择 + +## 用例编号与名称 + +TC-01 多 Key 加权选择 + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 cluster 配置多个 API-Key 时，BFE 按 `Keys[].Weight` 进行加权随机选择。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端 `cluster_multi_key` 与 `cluster_fallback_ok` 已启动，默认返回 200。 +3. 临时 BFE 配置已生成并加载，`cluster_multi_key` 配置 3 个 Key，weight 分别为 50/30/20。 +4. `common/mock_backend.go` 已支持记录 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.Keys`： + - `key-a` weight 50 + - `key-b` weight 30 + - `key-c` weight 20 +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：0（避免重试干扰命中分布）。 + +## BFE 请求 + +连续发送 1000 次相同请求： + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 所有请求响应状态码：200。 +- `cluster_multi_key` 收到 1000 次命中，`cluster_fallback_ok` 未被命中。 +- 后端记录的 `Authorization` 头中： + - `Bearer sk-key-a` 占比约 50%（容差 ±7%） + - `Bearer sk-key-b` 占比约 30%（容差 ±6%） + - `Bearer sk-key-c` 占比约 20%（容差 ±5%） +- 无 429/5xx 重试日志。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-02-429\350\247\246\345\217\221Key\350\275\256\346\215\242.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-02-429\350\247\246\345\217\221Key\350\275\256\346\215\242.md" new file mode 100644 index 000000000..eed8f7a8f --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-02-429\350\247\246\345\217\221Key\350\275\256\346\215\242.md" @@ -0,0 +1,58 @@ +# TC-02 429 触发 Key 轮换 + +## 用例编号与名称 + +TC-02 429 触发 Key 轮换 + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当某个 API-Key 返回 429 时，BFE 将其标记为本次调用已使用，并轮换到另一个 Key 重试。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载。 +4. `common/mock_backend.go` 已支持记录 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：3。 +- `cluster_multi_key` 后端行为： + - 收到 `sk-key-a` 时返回 429； + - 收到 `sk-key-b` 或 `sk-key-c` 时返回 200。 + +## BFE 请求 + +连续发送 100 次相同请求： + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 所有请求响应状态码：200。 +- `cluster_multi_key` 后端总命中次数大于 100（存在因 429 触发的重试）。 +- `cluster_multi_key` 后端记录中：`Bearer sk-key-a` 至少出现一次，`Bearer sk-key-b` 或 `Bearer sk-key-c` 至少出现一次。 +- `cluster_fallback_ok` 未被命中。 +- BFE 日志中出现 `rate limited (429), rotate` 相关记录。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-03-401\344\270\216403\346\240\207\350\256\260Key\346\255\273\344\272\241.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-03-401\344\270\216403\346\240\207\350\256\260Key\346\255\273\344\272\241.md" new file mode 100644 index 000000000..2f16100d9 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-03-401\344\270\216403\346\240\207\350\256\260Key\346\255\273\344\272\241.md" @@ -0,0 +1,59 @@ +# TC-03 401/403 标记 Key 死亡 + +## 用例编号与名称 + +TC-03 401/403 标记 Key 死亡 + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当某个 API-Key 返回 401 或 403 时，BFE 将其加入死亡集合，本次 `aiClusterInvoke()` 调用内的后续尝试不再使用该 Key。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载。 +4. `common/mock_backend.go` 已支持记录 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.Keys`：3 个 Key，`key-a`、`key-b`、`key-c`。 +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：3。 +- `cluster_multi_key` 后端行为： + - 收到 `sk-key-a` 时返回 401； + - 收到 `sk-key-b` 时返回 403； + - 收到 `sk-key-c` 时返回 200。 + +## BFE 请求 + +连续发送 100 次相同请求： + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 所有请求响应状态码：200。 +- `cluster_multi_key` 后端记录中，`Bearer sk-key-a`、`Bearer sk-key-b`、`Bearer sk-key-c` 均至少出现一次。 +- `cluster_fallback_ok` 未被命中。 +- BFE 日志中出现 `auth failed (401/403), dead` 相关记录。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-04-5xx\345\220\214Key\351\200\200\351\201\277\351\207\215\350\257\225.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-04-5xx\345\220\214Key\351\200\200\351\201\277\351\207\215\350\257\225.md" new file mode 100644 index 000000000..fb3e946b9 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-04-5xx\345\220\214Key\351\200\200\351\201\277\351\207\215\350\257\225.md" @@ -0,0 +1,61 @@ +# TC-04 5xx 同 Key 退避重试 + +## 用例编号与名称 + +TC-04 5xx 同 Key 退避重试 + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当后端返回 5xx 或连接错误时，BFE 保持当前 Key 不变，按退避策略重试，最终成功时返回成功响应。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载。 +4. `common/mock_backend.go` 已支持记录 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.KeyPolicy`： + - `MaxRetries`：3 + - `RetryBackoffInitial`：50 ms + - `RetryBackoffMax`：200 ms +- `cluster_multi_key` 后端行为： + - 前 2 次任意 Key 请求返回 503； + - 第 3 次起返回 200。 + +实现方式：mock 后端内部计数器，当且仅当计数 ≤ 2 时返回 503。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_multi_key` 后端收到 3 次请求，且 3 次 `Authorization` 头相同（均为首次选中的 Key）。 +- 相邻两次请求的时间间隔 ≥ 50 ms（初始退避值，允许 jitter 容差）。 +- `cluster_fallback_ok` 未被命中。 +- BFE 日志中出现 `transient failure [status=503], retry same key` 相关记录。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-05-Key\350\200\227\345\260\275\345\220\216\350\277\224\345\233\236\346\234\200\345\220\216\345\223\215\345\272\224.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-05-Key\350\200\227\345\260\275\345\220\216\350\277\224\345\233\236\346\234\200\345\220\216\345\223\215\345\272\224.md" new file mode 100644 index 000000000..0b645452c --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-05-Key\350\200\227\345\260\275\345\220\216\350\277\224\345\233\236\346\234\200\345\220\216\345\223\215\345\272\224.md" @@ -0,0 +1,58 @@ +# TC-05 Key 耗尽后触发 cluster fallback + +## 用例编号与名称 + +TC-05 Key 耗尽后触发 cluster fallback + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当所有 API-Key 均因 429/401/403 被排除后，`aiClusterInvoke()` 返回最后一个 4xx 响应，外层 `ServeHTTPForAI()` 识别到 401/402/403/429 属于默认 fallback 状态码集合，触发 cluster 级 fallback。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载。 +4. `common/mock_backend.go` 已支持记录 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：3（Key 共 3 个，预算足够尝试所有 Key）。 +- `cluster_multi_key` 后端行为： + - `sk-key-a` 返回 429； + - `sk-key-b` 返回 401； + - `sk-key-c` 返回 403。 +- `cluster_fallback_ok` 后端行为： + - 返回 200。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 响应状态码：200（来自 `cluster_fallback_ok`）。 +- `cluster_multi_key` 后端收到 3~4 次请求，分别携带 `sk-key-a`、`sk-key-b`、`sk-key-c`；当 429 Key 被重置后可能再尝试一次。 +- `cluster_fallback_ok` 后端被命中 1 次。 +- BFE 日志中出现 `all ai keys exhausted` 与 `fallback triggered` 相关记录。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-06-Key\347\272\247\350\200\227\345\260\275\350\247\246\345\217\221cluster-fallback.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-06-Key\347\272\247\350\200\227\345\260\275\350\247\246\345\217\221cluster-fallback.md" new file mode 100644 index 000000000..d9f4f8723 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-06-Key\347\272\247\350\200\227\345\260\275\350\247\246\345\217\221cluster-fallback.md" @@ -0,0 +1,56 @@ +# TC-06 Key 级耗尽触发 cluster fallback + +## 用例编号与名称 + +TC-06 Key 级耗尽触发 cluster fallback + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 Key 级重试因 5xx/连接错误耗尽后，`aiClusterInvoke()` 返回 5xx/错误，外层 `ServeHTTPForAI()` 触发 cluster 级 fallback。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载。 +4. `common/mock_backend.go` 已支持记录 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：2。 +- `cluster_multi_key` 后端行为： + - 任意 Key 均返回 503。 +- `cluster_fallback_ok` 后端行为： + - 返回 200。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_multi_key` 后端收到 ≤ 3 次请求（含同 Key 退避重试），均返回 503。 +- `cluster_fallback_ok` 后端被命中 1 次，返回 200。 +- BFE 日志中出现 `fallback triggered` 相关记录。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-07-\350\257\267\346\261\202\344\275\223\345\234\250Key\350\275\256\346\215\242\344\270\255\345\256\214\346\225\264\345\233\236\347\273\225.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-07-\350\257\267\346\261\202\344\275\223\345\234\250Key\350\275\256\346\215\242\344\270\255\345\256\214\346\225\264\345\233\236\347\273\225.md" new file mode 100644 index 000000000..148187c57 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-07-\350\257\267\346\261\202\344\275\223\345\234\250Key\350\275\256\346\215\242\344\270\255\345\256\214\346\225\264\345\233\236\347\273\225.md" @@ -0,0 +1,58 @@ +# TC-07 请求体在 Key 轮换中完整回绕 + +## 用例编号与名称 + +TC-07 请求体在 Key 轮换中完整回绕 + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 Key 级轮换时，请求体能够回绕到起始位置，每次尝试的后端都能收到完整、一致的请求体。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载。 +4. `common/mock_backend.go` 已支持记录请求体与 `Authorization` 头。 + +## 配置构造 + +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：3。 +- `cluster_multi_key` 后端行为： + - 收到 `sk-key-a` 时返回 429； + - 收到 `sk-key-b`/`sk-key-c` 时返回 200。 +- 请求体大小：约 100 KB 的 JSON，包含固定字段与随机内容，确保 `Content-Length` 大于 0。 + +## BFE 请求 + +连续发送 100 次相同请求： + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | 约 100 KB JSON，`{"model":"gpt-4","content":"..."}` | + +## 预期结果 + +- 所有请求响应状态码：200。 +- `cluster_multi_key` 后端总命中次数大于 100（存在因 429 触发的 Key 轮换）。 +- 所有记录的请求体字节完全一致，且与客户端发送字节完全一致。 +- `cluster_fallback_ok` 未被命中。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-08-AIConf\346\211\251\345\261\225\345\255\227\346\256\265\345\212\240\350\275\275.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-08-AIConf\346\211\251\345\261\225\345\255\227\346\256\265\345\212\240\350\275\275.md" new file mode 100644 index 000000000..7f436f74d --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/TC-08-AIConf\346\211\251\345\261\225\345\255\227\346\256\265\345\212\240\350\275\275.md" @@ -0,0 +1,55 @@ +# TC-08 AIConf 扩展字段加载 + +## 用例编号与名称 + +TC-08 AIConf 扩展字段加载 + +## 所属场景 + +SC02 多 API-Key 轮换与重试 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 BFE 启动时能够正确解析并加载包含 `Keys`、`KeyPolicy`、`Provider`、`ModelTable` 的 `AIConf`，且 `Provider` 与 `ModelTable` 不影响转发。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动。 +3. 临时 BFE 配置已生成并加载，`cluster_multi_key.AIConf` 包含完整的 `Provider` 与 `ModelTable`。 + +## 配置构造 + +- `cluster_multi_key.AIConf.Provider`：`mock-provider`。 +- `cluster_multi_key.AIConf.ModelTable`：含 1 条 `ModelPrice`，`Limits`/`Prices`/`Capabilities`/`SupportedParameters` 均完整填写。 +- `cluster_multi_key.AIConf.Keys`：仅保留 `key-c` weight 100，避免重试干扰。 +- `cluster_multi_key.AIConf.KeyPolicy.MaxRetries`：0。 + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `multikey.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- BFE 正常启动，无配置解析异常。 +- 响应状态码：200。 +- `cluster_multi_key` 后端收到 1 次请求，`Authorization` 为 `Bearer sk-key-c`。 +- 请求体中 `model` 被 `ModelMapping` 覆盖为 `mapped-model`（验证 `ModelMapping` 与 `ModelTable` 共存时互不影响）。 +- `cluster_fallback_ok` 未被命中。 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/\345\234\272\346\231\257\350\257\264\346\230\216.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/\345\234\272\346\231\257\350\257\264\346\230\216.md" new file mode 100644 index 000000000..1bc20ab45 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC02-\345\244\232API-Key\350\275\256\346\215\242\344\270\216\351\207\215\350\257\225/\345\234\272\346\231\257\350\257\264\346\230\216.md" @@ -0,0 +1,141 @@ +# SC02 多 API-Key 轮换与重试 + +## 1. 场景背景与目的 + +BFE 在 `ServeHTTPForAI()` 内部通过 `aiClusterInvoke()` 完成 cluster 级转发。v0.4 起，`cluster.AIConf` 支持配置多个 API-Key，并在一次 `aiClusterInvoke()` 调用内部按策略选择 Key、失败时自动轮换或重试。 + +本场景验证： + +- 多个 API-Key 按权重被选中； +- 429 触发 Key 轮换； +- 401/403 标记 Key 死亡并在本次调用内跳过； +- 5xx/连接错误触发同 Key 退避重试； +- Key 全部耗尽后返回最后一次响应/错误； +- Key 级 5xx 耗尽后仍触发 cluster 级 fallback； +- 请求体在 Key 级重试过程中可完整回绕； +- BFE 可正确加载包含 `Keys`、`KeyPolicy`、`Provider`、`ModelTable` 的 `AIConf`。 + +## 2. 运行模式 + +- **单组件模式**：仅启动真实 `bfe` 进程。 +- 不涉及 `ai-gateway-api` 与 `conf-agent`。 + +## 3. 涉及的 BFE 配置文件 + +| 文件 | 说明 | +|------|------| +| `bfe.conf` | 启用 `EnableAiGateway`，加载 `mod_ai_route` | +| `mod_ai_route/mod_ai_route.conf` | 指定 `ai_route.data` 路径 | +| `mod_ai_route/ai_route.data` | 定义 1 个 apikey 级路由表与绑定 | +| `server_data_conf/host_rule.data` | 声明 `ai_product` 产品线对应的 Host | +| `server_data_conf/route_rule.data` | 默认路由到 fallback cluster | +| `cluster_conf/cluster_conf.data` | 各 cluster 的基础配置，重点在 `AIConf` | +| `cluster_conf/gslb.data` | GSLB 权重配置 | +| `cluster_conf/cluster_table.data` | 运行时根据 mock 后端地址动态生成 | + +## 4. Key 与策略设计 + +### 4.1 测试 cluster 与 Key 配置 + +所有测试例复用同一个 `cluster_multi_key`，其 `AIConf` 如下： + +```json +{ + "AIConf": { + "Type": 0, + "ModelMapping": { + "gpt-4": "mapped-model" + }, + "Provider": "mock-provider", + "Keys": [ + { + "Name": "key-a", + "Key": "sk-key-a", + "Weight": 50 + }, + { + "Name": "key-b", + "Key": "sk-key-b", + "Weight": 30 + }, + { + "Name": "key-c", + "Key": "sk-key-c", + "Weight": 20 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 3, + "RetryBackoffInitial": 50, + "RetryBackoffMax": 200 + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "mock-provider", + "Model": "mapped-model", + "BaseModel": "mapped-model", + "Mode": "chat", + "Capabilities": ["chat"], + "SupportedParameters": ["temperature"], + "Limits": { + "context_window": 128000, + "max_input_tokens": 128000, + "max_output_tokens": 8192 + }, + "Prices": { + "input_cost_per_token": 0.000002, + "output_cost_per_token": 0.000008 + } + } + ] + } + } +} +``` + +> 说明：`ModelTable` 与 `Provider` 在本场景仅验证配置加载与解析，不验证成本计算。 + +### 4.2 路由表设计 + +#### apikey_ak_user_a + +- `user_a-multikey` + - 条件：`req_host_in("multikey.example.org")` + - targets：`cluster_multi_key`，无 model 覆盖 + - fallbacks：`cluster_fallback_ok` + +#### ApikeyRouteTableBindings + +```json +{ + "ak_user_a": ["apikey_ak_user_a"] +} +``` + +## 5. 测试例列表 + +| 编号 | 名称 | 核心验证点 | +|------|------|------------| +| TC-01 | 多 Key 加权选择 | 500 次请求按 weight 50/30/20 分布 | +| TC-02 | 429 触发 Key 轮换 | 首个选中 Key 返回 429 时，BFE 换 Key 重试并成功 | +| TC-03 | 401/403 标记 Key 死亡 | 返回 401/403 的 Key 被移出候选，本次调用内不再使用 | +| TC-04 | 5xx 同 Key 退避重试 | 5xx 时保持当前 Key，按退避策略重试并最终成功 | +| TC-05 | Key 耗尽后触发 cluster fallback | 所有 Key 均 429/401/403，外层 fallback 到 `cluster_fallback_ok` | +| TC-06 | 5xx Key 级耗尽触发 cluster fallback | 5xx 重试耗尽后，外层 fallback 到 `cluster_fallback_ok` | +| TC-07 | 请求体在 Key 轮换中完整回绕 | Key 轮换后后端每次收到与首次完全相同的 body | +| TC-08 | AIConf 扩展字段加载 | BFE 正常启动并加载含 `Provider`/`ModelTable` 的 `AIConf` | + +## 6. 公共基础设施改造 + +本场景需要扩展公共测试框架： + +1. **`common/mock_backend.go`**：增加 `Authorization` 头记录能力，使测试例能断言 BFE 使用了哪个 API-Key。 +2. **`common/bfe_config_builder.go`**：支持为指定 cluster 注入 `AIConf`，如增加 `AIConfs map[string]*cluster_conf.AIConf` 字段，在生成 `cluster_conf.data` 时写入。 + +## 7. 与其他场景的依赖关系 + +- 依赖 SC01 已验证的 `mod_ai_route` 路由表查找与绑定能力； +- 本场景 focus 在 `aiClusterInvoke()` 内部的 Key 级行为，不重复验证 target/fallback 模型覆盖。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-01-RMB\351\205\215\351\242\235\346\255\243\345\270\270\346\211\243\345\207\217.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-01-RMB\351\205\215\351\242\235\346\255\243\345\270\270\346\211\243\345\207\217.md" new file mode 100644 index 000000000..ddbf82c30 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-01-RMB\351\205\215\351\242\235\346\255\243\345\270\270\346\211\243\345\207\217.md" @@ -0,0 +1,73 @@ +# TC-01 RMB 配额正常扣减 + +## 用例编号与名称 + +TC-01 RMB 配额正常扣减 + +## 所属场景 + +SC03 RMB 配额扣减 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 API Key 绑定 RMB 配额计划时，BFE 在请求成功后按 `ModelTable` 定价从 Redis 扣减相应金额。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置 `quota:plan_rmb = 10000000000`（100 元）。 +3. mock 后端 `cluster_rmb` 已启动，返回 200 与如下 body： + ```json + { + "usage": { + "prompt_tokens": 100, + "completion_tokens": 50, + "total_tokens": 150 + } + } + ``` +4. 临时 BFE 配置已生成并加载，`cluster_rmb` 配置 `ModelTable`，`deepseek-chat` 的 input/output 价格分别为 `0.000001` / `0.000002`。 +5. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`。 + +## 配置构造 + +- `cluster_rmb.AIConf.ModelTable.Models[0]`： + - `Model`: `deepseek-chat` + - `Mode`: `chat` + - `Prices.input_cost_per_token`: `0.000001` + - `Prices.output_cost_per_token`: `0.000002` +- `plan_rmb`： + - `Unit`: `RMB` + - `Quota`: `10000000000` + - `RedisKey`: `quota:plan_rmb` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中。 +- Redis 中 `quota:plan_rmb` 的余额变为： + - 扣减金额 = `100 * 100 + 50 * 200 = 20000`（0.0002 元） + - 剩余 = `10000000000 - 20000 = 9999998000` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-02-RMB\351\205\215\351\242\235\344\275\231\351\242\235\344\270\215\350\266\263\346\213\222\347\273\235\350\257\267\346\261\202.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-02-RMB\351\205\215\351\242\235\344\275\231\351\242\235\344\270\215\350\266\263\346\213\222\347\273\235\350\257\267\346\261\202.md" new file mode 100644 index 000000000..b8744bbef --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-02-RMB\351\205\215\351\242\235\344\275\231\351\242\235\344\270\215\350\266\263\346\213\222\347\273\235\350\257\267\346\261\202.md" @@ -0,0 +1,58 @@ +# TC-02 RMB 配额余额不足拒绝请求 + +## 用例编号与名称 + +TC-02 RMB 配额余额不足拒绝请求 + +## 所属场景 + +SC03 RMB 配额扣减 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 RMB 配额计划余额为 0 时，BFE 在认证阶段拒绝请求并返回 429 配额耗尽错误。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置 `quota:plan_rmb = 0`。 +3. mock 后端 `cluster_rmb` 已启动（本用例不应命中）。 +4. 临时 BFE 配置已生成并加载，`cluster_rmb` 配置 `ModelTable`。 +5. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`。 + +## 配置构造 + +- `plan_rmb`： + - `Unit`: `RMB` + - `Quota`: `0` + - `RedisKey`: `quota:plan_rmb` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：429。 +- 响应 body 中包含错误码 `quota_exhausted` 或 `QUOTA_EXHAUSTED`。 +- `cluster_rmb` 未被命中。 +- Redis 中 `quota:plan_rmb` 保持为 0。 + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-03-ModelMapping\346\214\211\346\230\240\345\260\204\345\220\216\346\250\241\345\236\213\350\256\241\350\264\271.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-03-ModelMapping\346\214\211\346\230\240\345\260\204\345\220\216\346\250\241\345\236\213\350\256\241\350\264\271.md" new file mode 100644 index 000000000..ec5b57ed5 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-03-ModelMapping\346\214\211\346\230\240\345\260\204\345\220\216\346\250\241\345\236\213\350\256\241\350\264\271.md" @@ -0,0 +1,70 @@ +# TC-03 ModelMapping 按映射后模型计费 + +## 用例编号与名称 + +TC-03 ModelMapping 按映射后模型计费 + +## 所属场景 + +SC03 RMB 配额扣减 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当请求模型经 `AIConf.ModelMapping` 映射到后端模型后，BFE 按映射后的模型价格计费。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置 `quota:plan_rmb = 10000000000`。 +3. mock 后端 `cluster_rmb` 已启动，返回 200 与如下 body： + ```json + { + "usage": { + "prompt_tokens": 100, + "completion_tokens": 50, + "total_tokens": 150 + } + } + ``` +4. 临时 BFE 配置已加载，`cluster_rmb` 配置 `ModelMapping: {"gpt-4": "deepseek-chat"}`，且 `ModelTable` 中只有 `deepseek-chat` 的定价。 +5. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`。 + +## 配置构造 + +- `cluster_rmb.AIConf.ModelMapping`： + - `gpt-4` → `deepseek-chat` +- `cluster_rmb.AIConf.ModelTable.Models[0]`： + - `Model`: `deepseek-chat` + - `Prices.input_cost_per_token`: `0.000001` + - `Prices.output_cost_per_token`: `0.000002` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中，且后端收到的请求 body 中 `model` 为 `deepseek-chat`。 +- Redis 中 `quota:plan_rmb` 的余额扣减金额： + - `100 * 100 + 50 * 200 = 20000` + - 剩余 = `9999998000` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-04-Token\344\270\216RMB\351\205\215\351\242\235\345\205\261\345\255\230.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-04-Token\344\270\216RMB\351\205\215\351\242\235\345\205\261\345\255\230.md" new file mode 100644 index 000000000..a7515341d --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-04-Token\344\270\216RMB\351\205\215\351\242\235\345\205\261\345\255\230.md" @@ -0,0 +1,74 @@ +# TC-04 Token 与 RMB 配额共存 + +## 用例编号与名称 + +TC-04 Token 与 RMB 配额共存 + +## 所属场景 + +SC03 RMB 配额扣减 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 API Key 同时绑定 Token 配额计划和 RMB 配额计划时，BFE 分别按各自单位扣减，互不干扰。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置： + - `quota:plan_token = 1000` + - `quota:plan_rmb = 10000000000` +3. mock 后端 `cluster_rmb` 已启动，返回 200 与如下 body： + ```json + { + "usage": { + "prompt_tokens": 100, + "completion_tokens": 50, + "total_tokens": 150 + } + } + ``` +4. 临时 BFE 配置已加载，`cluster_rmb` 配置 `ModelTable`。 +5. `ak_user_a` 同时绑定 `plan_token` 和 `plan_rmb`。 + +## 配置构造 + +- `plan_token`： + - `Unit`: `total_token` + - `Quota`: `1000` + - `RedisKey`: `quota:plan_token` +- `plan_rmb`： + - `Unit`: `RMB` + - `Quota`: `10000000000` + - `RedisKey`: `quota:plan_rmb` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中。 +- Redis 余额变化： + - `quota:plan_token` = `1000 - 150 = 850` + - `quota:plan_rmb` = `10000000000 - (100 * 100 + 50 * 200) = 9999998000` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-05-\346\227\240ModelTable\346\214\2110\346\210\220\346\234\254\345\244\204\347\220\206.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-05-\346\227\240ModelTable\346\214\2110\346\210\220\346\234\254\345\244\204\347\220\206.md" new file mode 100644 index 000000000..76ebbc5cf --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-05-\346\227\240ModelTable\346\214\2110\346\210\220\346\234\254\345\244\204\347\220\206.md" @@ -0,0 +1,66 @@ +# TC-05 无 ModelTable 按 0 成本处理 + +## 用例编号与名称 + +TC-05 无 ModelTable 按 0 成本处理 + +## 所属场景 + +SC03 RMB 配额扣减 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当 RMB 配额计划命中的 cluster 未配置 `ModelTable` 时，BFE 按 0 成本处理，请求成功且 Redis 余额不变。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置 `quota:plan_rmb = 10000000000`。 +3. mock 后端 `cluster_no_table` 已启动，返回 200 与如下 body： + ```json + { + "usage": { + "prompt_tokens": 100, + "completion_tokens": 50, + "total_tokens": 150 + } + } + ``` +4. 临时 BFE 配置已加载，`cluster_no_table` 的 `AIConf` 未配置 `ModelTable`。 +5. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`。 + +## 配置构造 + +- `cluster_no_table.AIConf`： + - 配置 `Keys` 与 `KeyPolicy` + - 不配置 `ModelTable` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `notable.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_no_table` 收到 1 次命中。 +- Redis 中 `quota:plan_rmb` 保持为 `10000000000`（余额不变）。 +- BFE 日志中出现 model table not found 或 model price not found 的 Warn 日志。 + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-06-Fallback\345\220\216\346\214\211\346\234\200\347\273\210cluster\350\256\241\350\264\271.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-06-Fallback\345\220\216\346\214\211\346\234\200\347\273\210cluster\350\256\241\350\264\271.md" new file mode 100644 index 000000000..ddaf4d7ee --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/TC-06-Fallback\345\220\216\346\214\211\346\234\200\347\273\210cluster\350\256\241\350\264\271.md" @@ -0,0 +1,75 @@ +# TC-06 Fallback 后按最终 cluster 计费 + +## 用例编号与名称 + +TC-06 Fallback 后按最终 cluster 计费 + +## 所属场景 + +SC03 RMB 配额扣减 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当请求触发 cluster 级 fallback 后，BFE 按最终命中的 cluster 的 `ModelTable` 价格计费。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置 `quota:plan_rmb = 10000000000`。 +3. mock 后端 `cluster_rmb` 已启动，返回 502（触发 fallback）。 +4. mock 后端 `cluster_fallback_rmb` 已启动，返回 200 与如下 body： + ```json + { + "usage": { + "prompt_tokens": 100, + "completion_tokens": 50, + "total_tokens": 150 + } + } + ``` +5. 临时 BFE 配置已加载： + - `cluster_rmb` 的 `ModelTable` 价格：`input=0.000001`, `output=0.000002` + - `cluster_fallback_rmb` 的 `ModelTable` 价格：`input=0.000003`, `output=0.000004` +6. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`。 +7. 路由表配置 `cluster_rmb` 的 fallbacks 为 `cluster_fallback_rmb`。 + +## 配置构造 + +- `cluster_rmb.AIConf.ModelTable.Models[0].Prices`： + - `input_cost_per_token`: `0.000001` + - `output_cost_per_token`: `0.000002` +- `cluster_fallback_rmb.AIConf.ModelTable.Models[0].Prices`： + - `input_cost_per_token`: `0.000003` + - `output_cost_per_token`: `0.000004` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中（502）。 +- `cluster_fallback_rmb` 收到 1 次命中（200）。 +- Redis 中 `quota:plan_rmb` 的余额扣减按 `cluster_fallback_rmb` 价格计算： + - 扣减金额 = `100 * 300 + 50 * 400 = 50000`（0.0005 元） + - 剩余 = `9999995000` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/\345\234\272\346\231\257\350\257\264\346\230\216.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/\345\234\272\346\231\257\350\257\264\346\230\216.md" new file mode 100644 index 000000000..cb56a3d8e --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC03-RMB\351\205\215\351\242\235\346\211\243\345\207\217/\345\234\272\346\231\257\350\257\264\346\230\216.md" @@ -0,0 +1,215 @@ +# SC03 RMB 配额扣减 + +## 1. 场景背景与目的 + +BFE v0.4 在 `mod_ai_token_auth` 中引入 RMB（人民币）配额支持。与 Token 配额按 `total_tokens` 扣减不同，RMB 配额需要在响应阶段根据实际命中的 `cluster` 和 `target_model` 查询 `AIConf.ModelTable`，将 `prompt_tokens` / `completion_tokens` 换算为定点整数金额后，通过 Lua 脚本从 Redis 原子扣减。 + +本场景验证： + +- RMB 配额正常扣减； +- RMB 配额余额不足时拒绝请求； +- `ModelMapping` 场景下按映射后的最终模型计费； +- Token 配额与 RMB 配额共存时分别扣减； +- cluster 未配置 `ModelTable` 或模型未命中时按 0 成本处理； +- fallback 到另一个 cluster 后按最终 cluster + target_model 计费。 + +## 2. 运行模式 + +- **单组件模式**：仅启动真实 `bfe` 进程。 +- 使用嵌入式 Redis（`github.com/alicebob/miniredis/v2`）作为 `mod_ai_token_auth` 的 Redis 后端，避免依赖外部 Redis 服务。 +- 不涉及 `ai-gateway-api` 与 `conf-agent`。 + +## 3. 涉及的 BFE 配置文件 + +| 文件 | 说明 | +|------|------| +| `bfe.conf` | 启用 `EnableAiGateway`，加载 `mod_ai_route`、`mod_ai_token_auth` | +| `mod_ai_route/mod_ai_route.conf` | 指定 `ai_route.data` 路径 | +| `mod_ai_route/ai_route.data` | 定义 apikey 级路由表与绑定 | +| `mod_ai_token_auth/mod_ai_token_auth.conf` | 指定 `token_rule.data` 路径与 Redis 地址 | +| `mod_ai_token_auth/token_rule.data` | 定义 Token、QuotaPlan 与路由规则 | +| `server_data_conf/host_rule.data` | 声明 `ai_product` 产品线对应的 Host | +| `server_data_conf/route_rule.data` | 默认路由到 fallback cluster | +| `cluster_conf/cluster_conf.data` | 各 cluster 的基础配置，重点在 `AIConf` | +| `cluster_conf/gslb.data` | GSLB 权重配置 | +| `cluster_conf/cluster_table.data` | 运行时根据 mock 后端地址动态生成 | + +## 4. Cluster 与 AIConf 设计 + +### 4.1 测试 cluster + +| cluster | 用途 | +|---------|------| +| `cluster_rmb` | 主 cluster，配置 `ModelTable`，用于 RMB 计费 | +| `cluster_no_table` | 未配置 `ModelTable`，用于验证 0 成本兜底 | +| `cluster_fallback_rmb` | fallback cluster，配置不同的 `ModelTable`，用于验证 fallback 后按最终 cluster 计费 | + +### 4.2 `cluster_rmb` 的 `AIConf` + +```json +{ + "AIConf": { + "Type": 0, + "ModelMapping": { + "gpt-4": "deepseek-chat" + }, + "Provider": "mock-provider", + "Keys": [ + { + "Name": "key-primary", + "Key": "sk-primary", + "Weight": 100 + } + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 0 + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "mock-provider", + "Model": "deepseek-chat", + "BaseModel": "deepseek-chat", + "Mode": "chat", + "Capabilities": ["chat"], + "SupportedParameters": ["temperature", "max_tokens"], + "Limits": { + "context_window": 128000 + }, + "Prices": { + "input_cost_per_token": 0.000001, + "output_cost_per_token": 0.000002 + } + } + ] + } + } +} +``` + +> 说明： +> - `input_cost_per_token = 0.000001` 元/Token，定点整数为 `100`（1e-8 元/Token）。 +> - `output_cost_per_token = 0.000002` 元/Token，定点整数为 `200`。 +> - 请求 `gpt-4` 经 `ModelMapping` 映射为 `deepseek-chat`，按 `deepseek-chat` 价格计费。 + +### 4.3 `cluster_fallback_rmb` 的 `AIConf` + +与 `cluster_rmb` 结构相同，但价格不同，例如： + +```json +{ + "Prices": { + "input_cost_per_token": 0.000003, + "output_cost_per_token": 0.000004 + } +} +``` + +定点整数：`input = 300`，`output = 400`。 + +### 4.4 路由表设计 + +#### apikey_ak_user_a + +- `user_a-rmb` + - 条件：`req_host_in("rmb.example.org")` + - targets：`cluster_rmb` + - fallbacks：`cluster_fallback_rmb` +- `user_a-notable` + - 条件：`req_host_in("notable.example.org")` + - targets：`cluster_no_table` + - fallbacks：无 + +#### ApikeyRouteTableBindings + +```json +{ + "ak_user_a": ["apikey_ak_user_a"] +} +``` + +## 5. Token 与 QuotaPlan 设计 + +### 5.1 Token 配置 + +| API Key | 绑定的 QuotaPlan | +|---------|------------------| +| `ak_user_a` | `plan_rmb` / `plan_token` / `plan_rmb_empty`（视测试例而定） | + +### 5.2 QuotaPlan 配置示例 + +#### RMB 配额 + +```json +{ + "Id": "plan_rmb", + "Unlimited": false, + "PassNoQuota": false, + "RedisKey": "quota:plan_rmb", + "CreateTime": 0, + "ExpiredTime": -1, + "Quota": 10000000000, + "ResetMode": 0, + "Unit": "RMB", + "Currency": "RMB" +} +``` + +> `Quota = 10,000,000,000` 表示 100 元（1e-8 元/单位）。 + +#### Token 配额 + +```json +{ + "Id": "plan_token", + "Unlimited": false, + "PassNoQuota": false, + "RedisKey": "quota:plan_token", + "CreateTime": 0, + "ExpiredTime": -1, + "Quota": 10000, + "ResetMode": 0, + "Unit": "total_token" +} +``` + +## 6. 测试例列表 + +| 编号 | 名称 | 核心验证点 | +|------|------|------------| +| TC-01 | RMB 配额正常扣减 | 请求成功后 Redis 余额按 `prompt*100 + completion*200` 扣减 | +| TC-02 | RMB 配额余额不足拒绝请求 | 余额为 0 时，认证阶段返回 429，响应中提示配额耗尽 | +| TC-03 | ModelMapping 按映射后模型计费 | 请求 `gpt-4` 映射为 `deepseek-chat`，按 `deepseek-chat` 价格扣减 | +| TC-04 | Token 与 RMB 配额共存 | 同时绑定 Token 和 RMB 计划，两者分别扣减正确金额 | +| TC-05 | 无 ModelTable 按 0 成本处理 | `cluster_no_table` 命中 RMB 计划但无定价表，请求成功且 Redis 余额不变 | +| TC-06 | Fallback 后按最终 cluster 计费 | `cluster_rmb` 返回 502 触发 fallback 到 `cluster_fallback_rmb`，按后者价格扣减 | + +## 7. 公共基础设施改造 + +本场景需要扩展公共测试框架： + +1. **`common/bfe_config_builder.go`**： + - 支持为 `mod_ai_token_auth/mod_ai_token_auth.conf` 动态写入 Redis 地址（`Redis.Bns`）。 + - 支持生成 `mod_ai_token_auth/token_rule.data`，允许测试指定 Token、QuotaPlan 和规则。 +2. **新增 `common/redis_server.go`**（或直接在测试场景内实现）： + - 基于 `github.com/alicebob/miniredis/v2` 启动嵌入式 Redis； + - 提供 `SetQuota` / `GetQuota` 辅助方法，用于预置余额和断言扣减结果。 +3. **`common/mock_backend.go`**： + - 支持返回带 `usage` 字段的响应体（`prompt_tokens`、`completion_tokens`、`total_tokens`）。 + +## 8. 依赖与风险 + +| 风险点 | 说明 | 缓解措施 | +|--------|------|----------| +| 需要嵌入式 Redis | 集成测试不依赖外部 Redis 服务 | 引入 `miniredis/v2` 作为 `bfe` 的 test 依赖 | +| token_rule.data 字段格式 | 需要与 `mod_ai_token_auth` 的 JSON 结构保持一致 | 参考已有单元测试和配置示例 | +| RMB 扣减存在异步性 | Redis 扣减在响应发送后执行 | 请求发送后等待并轮询 Redis 余额，或设置合理 sleep | +| ModelTable 价格转换精度 | 浮点到定点整数转换可能与预期有 1 单位误差 | 测试用例使用可整除的价格（如 0.000001 → 100） | + +## 9. 与其他场景的依赖关系 + +- 依赖 SC01 已验证的 `mod_ai_route` 路由表查找与绑定能力； +- 依赖 SC02 已验证的 `AIConf` 加载与多 Key 行为（本场景可简化为单 Key）； +- 本场景 focus 在 `mod_ai_token_auth` 的 RMB 配额扣减链路。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-01-\345\237\272\346\234\254\345\211\215\347\274\200\350\243\201\345\211\252.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-01-\345\237\272\346\234\254\345\211\215\347\274\200\350\243\201\345\211\252.md" new file mode 100644 index 000000000..f6d27ddfb --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-01-\345\237\272\346\234\254\345\211\215\347\274\200\350\243\201\345\211\252.md" @@ -0,0 +1,60 @@ +# TC-01 基本前缀裁剪 + +## 用例编号与名称 + +TC-01 基本前缀裁剪 + +## 所属场景 + +SC04 Provider/Model 前缀裁剪 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 BFE 在转发前按 cluster 的 `MatchPrefix`/`StripPrefix` 配置正确裁剪 provider 前缀。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_openrouter` 返回 200 + - `cluster_fallback` 返回 200 + - `cluster_default` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `apikey_ak_user_a` 路由表中 `user_a-openrouter` 规则： + - 条件：`req_body_json_prefix_in("model", "openrouter/", false)` + - targets：`cluster_openrouter` +- `cluster_openrouter` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`true` + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_openrouter` 收到 1 次请求 +- `cluster_default` 未收到请求 +- `cluster_openrouter` 收到的请求体中 `model` 为 `anthropic/claude-sonnet-4.6` +- `cluster_openrouter` 收到的请求体中 `messages` 字段保持原样 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-02-\345\211\215\347\274\200\344\270\215\345\214\271\351\205\215\344\270\215\350\243\201\345\211\252.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-02-\345\211\215\347\274\200\344\270\215\345\214\271\351\205\215\344\270\215\350\243\201\345\211\252.md" new file mode 100644 index 000000000..b2c783f56 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-02-\345\211\215\347\274\200\344\270\215\345\214\271\351\205\215\344\270\215\350\243\201\345\211\252.md" @@ -0,0 +1,58 @@ +# TC-02 前缀不匹配不裁剪 + +## 用例编号与名称 + +TC-02 前缀不匹配不裁剪 + +## 所属场景 + +SC04 Provider/Model 前缀裁剪 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证当请求 model 不匹配 cluster 的 `MatchPrefix` 时，BFE 不执行裁剪，保持原样转发。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_openrouter` 返回 200 + - `cluster_default` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `apikey_ak_user_a` 路由表中包含： + - `user_a-openrouter`：条件 `req_body_json_prefix_in("model", "openrouter/", false)`，targets `cluster_openrouter` + - `user_a-default`：条件 `default_t()`，targets `cluster_default` +- `cluster_openrouter` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`true` + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"other/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_default` 收到 1 次请求 +- `cluster_openrouter` 未收到请求 +- `cluster_default` 收到的请求体中 `model` 仍为 `other/anthropic/claude-sonnet-4.6` + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-03-StripPrefix\344\270\272false\344\270\215\350\243\201\345\211\252.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-03-StripPrefix\344\270\272false\344\270\215\350\243\201\345\211\252.md" new file mode 100644 index 000000000..313efe6b6 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-03-StripPrefix\344\270\272false\344\270\215\350\243\201\345\211\252.md" @@ -0,0 +1,56 @@ +# TC-03 StripPrefix=false 不裁剪 + +## 用例编号与名称 + +TC-03 StripPrefix=false 不裁剪 + +## 所属场景 + +SC04 Provider/Model 前缀裁剪 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 `StripPrefix=false` 时，即使配置了 `MatchPrefix`，BFE 也不会裁剪前缀，仅将前缀作为路由标识使用。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_openrouter` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `apikey_ak_user_a` 路由表中 `user_a-openrouter` 规则： + - 条件：`req_body_json_prefix_in("model", "openrouter/", false)` + - targets：`cluster_openrouter` +- `cluster_openrouter` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`false` + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_openrouter` 收到 1 次请求 +- `cluster_openrouter` 收到的请求体中 `model` 仍为 `openrouter/anthropic/claude-sonnet-4.6` + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-04-\345\211\215\347\274\200\350\243\201\345\211\252\345\220\216\345\206\215ModelMapping.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-04-\345\211\215\347\274\200\350\243\201\345\211\252\345\220\216\345\206\215ModelMapping.md" new file mode 100644 index 000000000..fb89b8da4 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-04-\345\211\215\347\274\200\350\243\201\345\211\252\345\220\216\345\206\215ModelMapping.md" @@ -0,0 +1,57 @@ +# TC-04 前缀裁剪后再 ModelMapping + +## 用例编号与名称 + +TC-04 前缀裁剪后再 ModelMapping + +## 所属场景 + +SC04 Provider/Model 前缀裁剪 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证前缀裁剪后，BFE 继续执行 `ModelMapping`，下游最终收到映射后的模型名。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_openrouter` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `apikey_ak_user_a` 路由表中 `user_a-openrouter` 规则： + - 条件：`req_body_json_prefix_in("model", "openrouter/", false)` + - targets：`cluster_openrouter` +- `cluster_openrouter` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`true` + - `ModelMapping`：`{"anthropic/claude-sonnet-4.6": "claude-3-sonnet-20250219"}` + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_openrouter` 收到 1 次请求 +- `cluster_openrouter` 收到的请求体中 `model` 为 `claude-3-sonnet-20250219` + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-05-TargetModelOverride\345\220\216\350\243\201\345\211\252.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-05-TargetModelOverride\345\220\216\350\243\201\345\211\252.md" new file mode 100644 index 000000000..71e456521 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-05-TargetModelOverride\345\220\216\350\243\201\345\211\252.md" @@ -0,0 +1,56 @@ +# TC-05 Target Model Override 后裁剪 + +## 用例编号与名称 + +TC-05 Target Model Override 后裁剪 + +## 所属场景 + +SC04 Provider/Model 前缀裁剪 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 route target 已覆盖 model 字段后，BFE 仍基于覆盖后的模型名执行前缀裁剪。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_openrouter` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `apikey_ak_user_a` 路由表中 `user_a-openrouter` 规则： + - 条件：`req_body_json_prefix_in("model", "openrouter/", false)` + - targets：`cluster_openrouter`，`Model` 设置为 `openrouter/google/gemini-pro` +- `cluster_openrouter` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`true` + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_openrouter` 收到 1 次请求 +- `cluster_openrouter` 收到的请求体中 `model` 为 `google/gemini-pro` + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-06-Fallback\346\227\266\345\211\215\347\274\200\350\243\201\345\211\252.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-06-Fallback\346\227\266\345\211\215\347\274\200\350\243\201\345\211\252.md" new file mode 100644 index 000000000..011301b6f --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/TC-06-Fallback\346\227\266\345\211\215\347\274\200\350\243\201\345\211\252.md" @@ -0,0 +1,63 @@ +# TC-06 Fallback 时前缀裁剪 + +## 用例编号与名称 + +TC-06 Fallback 时前缀裁剪 + +## 所属场景 + +SC04 Provider/Model 前缀裁剪 + +## 版本声明 + +- `bfe`：当前源码版本 + +## 测试目的 + +验证 primary cluster 失败并触发 fallback 后，fallback cluster 仍按自身 `MatchPrefix`/`StripPrefix` 配置正确裁剪前缀。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. mock 后端已启动： + - `cluster_openrouter` 返回 500 + - `cluster_fallback` 返回 200 +3. 临时 BFE 配置已生成并加载。 + +## 配置构造 + +- `apikey_ak_user_a` 路由表中 `user_a-openrouter` 规则： + - 条件：`req_body_json_prefix_in("model", "openrouter/", false)` + - targets：`cluster_openrouter` + - fallbacks：`cluster_fallback` +- `cluster_openrouter` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`true` +- `cluster_fallback` 的 `AIConf`： + - `MatchPrefix`：`openrouter/` + - `StripPrefix`：`true` + +## BFE 请求 + +| 字段 | 值 | +|------|-----| +| Host | `api.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"openrouter/anthropic/claude-sonnet-4.6","messages":[{"role":"user","content":"hello"}]}` | + +## 预期结果 + +- 响应状态码：200 +- `cluster_openrouter` 收到 1 次请求（失败） +- `cluster_fallback` 收到 1 次请求（成功） +- `cluster_fallback` 收到的请求体中 `model` 为 `anthropic/claude-sonnet-4.6` +- `cluster_fallback` 收到的请求体中 `messages` 字段保持原样 + +## 清理 + +停止 `bfe` 进程与所有 mock 后端，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/\345\234\272\346\231\257\350\257\264\346\230\216.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/\345\234\272\346\231\257\350\257\264\346\230\216.md" new file mode 100644 index 000000000..f71dfc60f --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC04-ProviderModel\345\211\215\347\274\200\350\243\201\345\211\252/\345\234\272\346\231\257\350\257\264\346\230\216.md" @@ -0,0 +1,73 @@ +# SC04 Provider/Model 前缀裁剪 + +## 1. 场景背景与目的 + +部分模型聚合平台（如 OpenRouter）要求客户端在请求 Body 的 `model` 字段中使用 `provider_name/model_name` 格式，例如 `openrouter/anthropic/claude-sonnet-4.6`。BFE 需要在转发给下游前将该前缀裁剪掉，使下游收到平台内部认可的模型名。 + +本场景验证： + +- `mod_ai_route` 可使用 `req_body_json_prefix_in("model", "openrouter/", false)` 将带前缀的请求路由到指定 cluster； +- cluster 配置 `MatchPrefix`/`StripPrefix` 后，BFE 在转发前正确裁剪 provider 前缀； +- 前缀裁剪与 `ModelMapping`、target model override、fallback 等机制协同工作； +- 不匹配前缀或 `StripPrefix=false` 时保持现有行为。 + +## 2. 运行模式 + +- **单组件模式**：仅启动真实 `bfe` 进程。 +- 不涉及 `ai-gateway-api` 与 `conf-agent`。 + +## 3. 涉及的 BFE 配置文件 + +| 文件 | 说明 | +|------|------| +| `bfe.conf` | 启用 `EnableAiGateway`，加载 `mod_ai_route` | +| `mod_ai_route/mod_ai_route.conf` | 指定 `ai_route.data` 路径 | +| `mod_ai_route/ai_route.data` | 定义路由规则，使用 `req_body_json_prefix_in` 条件 | +| `server_data_conf/host_rule.data` | 声明 `ai_product` 产品线对应的 Host | +| `server_data_conf/route_rule.data` | 默认路由到 fallback cluster | +| `cluster_conf/cluster_conf.data` | 各 cluster 的 `AIConf`，包含 `MatchPrefix`/`StripPrefix` | +| `cluster_conf/gslb.data` | GSLB 权重配置 | +| `cluster_conf/cluster_table.data` | 运行时根据 mock 后端地址动态生成 | + +## 4. 路由表设计 + +### 4.1 apikey_ak_user_a（apikey 级别） + +- `user_a-openrouter` + - 条件：`req_body_json_prefix_in("model", "openrouter/", false)` + - targets：`cluster_openrouter`，无 model 覆盖 + - fallbacks：`cluster_fallback` +- `user_a-default` + - 条件：`default_t()` + - targets：`cluster_default` + +### 4.2 ApikeyRouteTableBindings + +```json +{ + "ak_user_a": ["apikey_ak_user_a", "global_default"] +} +``` + +## 5. Cluster 配置设计 + +| cluster | MatchPrefix | StripPrefix | ModelMapping | 用途 | +|---------|-------------|-------------|--------------|------| +| `cluster_openrouter` | `openrouter/` | `true` | 可选 | 命中 openrouter 前缀，裁剪后转发 | +| `cluster_default` | 空 | `false` | 无 | 处理不带前缀的请求 | +| `cluster_fallback` | `openrouter/` | `true` | 无 | openrouter cluster 失败后的 fallback | + +## 6. 测试例列表 + +| 编号 | 名称 | 核心验证点 | +|------|------|------------| +| TC-01 | 基本前缀裁剪 | 请求 `model="openrouter/anthropic/claude-xxx"`，下游收到 `anthropic/claude-xxx` | +| TC-02 | 前缀不匹配不裁剪 | 请求 `model="other/anthropic/claude-xxx"`，命中 default cluster，下游收到原样 | +| TC-03 | StripPrefix=false 不裁剪 | 请求带 `openrouter/` 前缀但 cluster 不裁剪，下游收到原样 | +| TC-04 | 前缀裁剪后再 ModelMapping | 裁剪后模型名命中 `ModelMapping`，下游收到映射后的模型名 | +| TC-05 | Target model override 后裁剪 | route target 设置 model 为 `openrouter/google/gemini`，裁剪后转发 | +| TC-06 | Fallback 时前缀裁剪 | primary cluster 失败后，fallback cluster 仍正确裁剪前缀 | + +## 7. 与其他场景的依赖关系 + +无依赖，可独立运行。本场景复用 SC01 的公共 harness（`common.ProcessEnv`、`common.MockBackend`、`common.BFEConfigBuilder` 等）。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-01-\346\210\220\345\212\237\350\257\267\346\261\202\344\270\273\350\246\201AI\346\227\245\345\277\227\345\255\227\346\256\265.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-01-\346\210\220\345\212\237\350\257\267\346\261\202\344\270\273\350\246\201AI\346\227\245\345\277\227\345\255\227\346\256\265.md" new file mode 100644 index 000000000..cf52cb214 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-01-\346\210\220\345\212\237\350\257\267\346\261\202\344\270\273\350\246\201AI\346\227\245\345\277\227\345\255\227\346\256\265.md" @@ -0,0 +1,90 @@ +# TC-01 成功请求主要 AI 日志字段 + +## 用例编号与名称 + +TC-01 成功请求主要 AI 日志字段 + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证一次成功的 RMB 配额请求在 `mod_access_pb3` 输出的 b2log 中，各 AI 可观测字段被正确填充。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 已编译 `bfe` 可执行文件。 +2. 嵌入式 Redis 已启动，并预置 `quota:plan_rmb = 10000000000`（100 元）。 +3. mock 后端 `cluster_rmb` 已启动，返回 200 与如下 body： + ```json + { + "usage": { + "prompt_tokens": 100, + "completion_tokens": 50, + "total_tokens": 150 + } + } + ``` +4. 临时 BFE 配置已生成并加载，`cluster_rmb` 配置 `Provider = "mock-provider"`、`ModelTable.Currency = "RMB"`。 +5. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`，其 `key_id = "user_a_key_id"`，并携带标签 `[{"tagname":"department","tagvalue":"ai-team"}]`。 +6. 启用 `mod_access_pb3`，b2log 输出到临时 `log/` 目录。 + +## 配置构造 + +- `cluster_rmb.AIConf`： + - `Provider`: `mock-provider` + - `ModelTable.Currency`: `RMB` + - `ModelTable.Models[0].Model`: `deepseek-chat` + - `Prices.input_cost_per_token`: `0.000001` + - `Prices.output_cost_per_token`: `0.000002` + - `Keys`: 单 Key `key-primary` +- `plan_rmb`： + - `Unit`: `RMB` + - `Quota`: `10000000000` + - `RedisKey`: `quota:plan_rmb` +- `ai_route.data` 中 `apikey_ak_user_a` 命中 `user_a-rmb`，target 为 `cluster_rmb`。 + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中。 +- b2log 中存在 1 条 `RequestLog`，且字段满足： + - `ai_apikey_id` = `"user_a_key_id"`（不是原始 key `ak_user_a`） + - `ai_apikeytags` 包含 1 条记录：`tagname="department"`、`tagvalue="ai-team"` + - `ai_requested_model` = `"deepseek-chat"` + - `ai_target_model` = `"deepseek-chat"` + - `ai_provider` = `"mock-provider"` + - `ai_cost_value` = `100 * 100 + 50 * 200 = 20000` + - `ai_cost_currency` = `"RMB"` + - `ai_input_tokens` = `100` + - `ai_output_tokens` = `50` + - `ai_total_tokens` = `150` + - `ai_route_rule_hits` 包含 1 条记录：`rule_owner="ak_user_a"`、`rule_owner_type="apikey"`、`rule_name="user_a-rmb"` + - `ai_cluster_key_names` 包含 1 条记录：`cluster_name="cluster_rmb"`、`key_name="key-primary"` + - `ai_auth_hit_quota_plans` 包含 `["plan_rmb"]` + - `ai_retry_count` 未设置或为 0 + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-02-ModelMapping\345\220\216target_model\346\255\243\347\241\256.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-02-ModelMapping\345\220\216target_model\346\255\243\347\241\256.md" new file mode 100644 index 000000000..958385e42 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-02-ModelMapping\345\220\216target_model\346\255\243\347\241\256.md" @@ -0,0 +1,56 @@ +# TC-02 ModelMapping 后 target_model 正确 + +## 用例编号与名称 + +TC-02 ModelMapping 后 target_model 正确 + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证当请求模型经过 `AIConf.ModelMapping` 映射后，访问日志中 `ai_requested_model` 与 `ai_target_model` 分别记录原始请求模型和映射后的目标模型。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 同 TC-01，但请求 body 中 model 为 `gpt-4`。 +2. `cluster_rmb.AIConf.ModelMapping` 配置 `"gpt-4" -> "deepseek-chat"`。 + +## 配置构造 + +- `cluster_rmb.AIConf.ModelMapping`：`{"gpt-4": "deepseek-chat"}` +- 其余同 TC-01。 + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"gpt-4"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中，后端收到的 model 为 `deepseek-chat`。 +- b2log 中： + - `ai_requested_model` = `"gpt-4"` + - `ai_target_model` = `"deepseek-chat"` + - `ai_cost_value` 按 `deepseek-chat` 价格计算，即 `100 * 100 + 50 * 200 = 20000` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-03-\345\244\232Key\351\207\215\350\257\225\346\227\266retry_count\344\270\216cluster_key_names.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-03-\345\244\232Key\351\207\215\350\257\225\346\227\266retry_count\344\270\216cluster_key_names.md" new file mode 100644 index 000000000..2317bc8ad --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-03-\345\244\232Key\351\207\215\350\257\225\346\227\266retry_count\344\270\216cluster_key_names.md" @@ -0,0 +1,63 @@ +# TC-03 多 Key 重试时 retry_count 与 cluster_key_names + +## 用例编号与名称 + +TC-03 多 Key 重试时 retry_count 与 cluster_key_names + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证当 `aiClusterInvoke` 内部触发 key-level 重试时，访问日志中 `ai_retry_count` 正确累加，且 `ai_cluster_key_names` 记录所有尝试过的 key。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 同 TC-01，但 `cluster_rmb.AIConf.KeyPolicy.MaxRetries >= 1`。 +2. `cluster_rmb.AIConf.Keys` 配置两个 Key：`key-primary` 和 `key-secondary`。 +3. mock 后端在首次请求时返回 500（模拟主 Key 失败），第二次请求返回 200。 + +## 配置构造 + +- `cluster_rmb.AIConf.KeyPolicy`： + - `Strategy`: `weighted_random` + - `MaxRetries`: `2` + - `RetryBackoffInitial`: `10` + - `RetryBackoffMax`: `50` +- `cluster_rmb.AIConf.Keys`： + - `{"Name": "key-primary", "Key": "sk-primary", "Weight": 100}` + - `{"Name": "key-secondary", "Key": "sk-secondary", "Weight": 100}` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 2 次命中（第一次 500，第二次 200）。 +- b2log 中： + - `ai_retry_count` >= `1` + - `ai_cluster_key_names` 包含至少 2 条记录，均属于 `cluster_rmb` + - 由于 weighted random 可能两次选到同一 key，断言时只需验证 `len(ai_cluster_key_names) >= 2` 且所有记录的 `cluster_name` 均为 `cluster_rmb` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-04-RMB\351\205\215\351\242\235\350\200\227\345\260\275\346\227\266\346\213\222\347\273\235\345\255\227\346\256\265.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-04-RMB\351\205\215\351\242\235\350\200\227\345\260\275\346\227\266\346\213\222\347\273\235\345\255\227\346\256\265.md" new file mode 100644 index 000000000..f83847424 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-04-RMB\351\205\215\351\242\235\350\200\227\345\260\275\346\227\266\346\213\222\347\273\235\345\255\227\346\256\265.md" @@ -0,0 +1,60 @@ +# TC-04 RMB 配额耗尽时拒绝字段 + +## 用例编号与名称 + +TC-04 RMB 配额耗尽时拒绝字段 + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证当 RMB 配额余额不足导致认证拒绝时，访问日志中 `ai_auth_reject_reason` 与 `ai_auth_reject_quota_plans` 正确输出。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 同 TC-01，但 Redis 中 `quota:plan_rmb = 0`。 +2. `ak_user_a` 绑定 RMB 配额计划 `plan_rmb`。 + +## 配置构造 + +- `plan_rmb`： + - `Unit`: `RMB` + - `Quota`: `0` + - `RedisKey`: `quota:plan_rmb` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：429。 +- `cluster_rmb` 收到 0 次命中。 +- b2log 中存在 1 条 `RequestLog`，且字段满足： + - `ai_auth_reject_reason` 非空，包含 `QUOTA_EXHAUSTED` 或对应原因描述 + - `ai_auth_reject_quota_plans` = `["plan_rmb"]` + - `ai_apikey_id` = `"user_a_key_id"` + - `ai_route_rule_hits` 为空（请求在认证阶段被拒绝，未进入路由） + - `ai_cluster_key_names` 为空 + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-05-Fallback\345\220\216provider\344\270\216cluster_key_names.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-05-Fallback\345\220\216provider\344\270\216cluster_key_names.md" new file mode 100644 index 000000000..af8783946 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-05-Fallback\345\220\216provider\344\270\216cluster_key_names.md" @@ -0,0 +1,64 @@ +# TC-05 Fallback 后 provider 与 cluster_key_names + +## 用例编号与名称 + +TC-05 Fallback 后 provider 与 cluster_key_names + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证当主 cluster 返回 502 触发 fallback 到另一个 cluster 后，访问日志中 `ai_provider` 记录最终 fallback cluster 的 provider，且 `ai_cluster_key_names` 包含两个 cluster 的尝试记录。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 同 TC-01，但存在 `cluster_fallback_rmb`。 +2. `cluster_rmb` 的 mock 后端返回 502。 +3. `cluster_fallback_rmb` 的 mock 后端返回 200 与 usage。 +4. `ai_route.data` 中 `user_a-rmb` 的 fallbacks 包含 `cluster_fallback_rmb`。 + +## 配置构造 + +- `cluster_rmb.AIConf.Provider`：`mock-provider` +- `cluster_fallback_rmb.AIConf.Provider`：`mock-provider-fallback` +- `cluster_fallback_rmb.AIConf.ModelTable.Currency`：`RMB` +- `cluster_fallback_rmb.AIConf.ModelTable.Models[0].Prices`： + - `input_cost_per_token`: `0.000003` + - `output_cost_per_token`: `0.000004` + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中（502）。 +- `cluster_fallback_rmb` 收到 1 次命中（200）。 +- b2log 中： + - `ai_provider` = `"mock-provider-fallback"`（最终成功 cluster 的 provider） + - `ai_cost_value` 按 fallback cluster 价格计算，即 `100 * 300 + 50 * 400 = 50000` + - `ai_cluster_key_names` 包含至少 2 条记录，分别属于 `cluster_rmb` 和 `cluster_fallback_rmb` + - `ai_route_rule_hits` 仍包含 `user_a-rmb` 的命中记录 + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-06-\346\265\201\345\274\217\345\223\215\345\272\224\345\255\227\346\256\265.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-06-\346\265\201\345\274\217\345\223\215\345\272\224\345\255\227\346\256\265.md" new file mode 100644 index 000000000..9cd3b6721 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-06-\346\265\201\345\274\217\345\223\215\345\272\224\345\255\227\346\256\265.md" @@ -0,0 +1,66 @@ +# TC-06 流式响应字段 + +## 用例编号与名称 + +TC-06 流式响应字段 + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证 SSE 流式请求下，访问日志中 `ai_stream`、`ai_ttft_us`、`ai_tpot_us` 被正确填充。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 同 TC-01。 +2. mock 后端返回 SSE 格式响应，最终 chunk 包含 `usage`： + ``` + data: {"choices":[{"delta":{"role":"assistant"}}]} + + data: {"choices":[{"delta":{"content":"hello"}}]} + + data: {"usage":{"prompt_tokens":100,"completion_tokens":50,"total_tokens":150}} + + ``` +3. 响应头包含 `Content-Type: text/event-stream`。 + +## 配置构造 + +- 同 TC-01。 + +## BFE 请求 + +发送 1 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat","stream":true}` | + +## 预期结果 + +- 响应状态码：200。 +- `cluster_rmb` 收到 1 次命中。 +- b2log 中存在 1 条 `RequestLog`，且字段满足： + - `ai_stream` = `true` + - `ai_ttft_us` > `0` + - `ai_tpot_us` > `0` + - `ai_input_tokens` = `100` + - `ai_output_tokens` = `50` + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-07-\351\231\220\346\265\201\345\221\275\344\270\255\345\255\227\346\256\265.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-07-\351\231\220\346\265\201\345\221\275\344\270\255\345\255\227\346\256\265.md" new file mode 100644 index 000000000..888c46def --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/TC-07-\351\231\220\346\265\201\345\221\275\344\270\255\345\255\227\346\256\265.md" @@ -0,0 +1,71 @@ +# TC-07 限流命中字段 + +## 用例编号与名称 + +TC-07 限流命中字段 + +## 所属场景 + +SC05 AI 访问日志字段校验 + +## 版本声明 + +- `bfe`：当前源码版本 +- `bfe-access-pb`：`v0.2.0` + +## 测试目的 + +验证当请求触发 `mod_ai_rate_limit` 限流时，访问日志中 `ai_rate_limit_hits` 正确记录命中的策略与规则。 + +## 运行模式 + +单组件模式：仅启动真实 `bfe` 进程与嵌入式 Redis。 + +## 前置条件 + +1. 同 TC-01，但额外启用 `mod_ai_rate_limit` 模块。 +2. `mod_ai_rate_limit` 配置 RPM 策略，窗口内仅允许 1 个请求。 +3. 连续发送 2 次请求，第二次应触发限流。 + +## 配置构造 + +- `bfe.conf` 的 `Modules` 增加 `mod_ai_rate_limit`。 +- `mod_ai_rate_limit/rate_limit.data` 配置： + ```json + { + "Version": "1.0", + "Config": { + "ai_product": { + "policy-rpm": { + "type": "rpm", + "limit": 1, + "window": 60 + } + } + } + } + ``` +- `mod_ai_rate_limit/mod_ai_rate_limit.conf` 指定 `rate_limit.data` 路径。 + +## BFE 请求 + +连续发送 2 次 POST 请求： + +| 字段 | 值 | +|------|-----| +| Host | `rmb.example.org` | +| Path | `/v1/chat/completions` | +| Authorization | `Bearer ak_user_a` | +| Body | `{"model":"deepseek-chat"}` | + +## 预期结果 + +- 第一次响应状态码：200。 +- 第二次响应状态码：429（触发 RPM 限流）。 +- b2log 中存在 2 条 `RequestLog`： + - 第一条：`ai_rate_limit_hits` 为空或不存在 + - 第二条：`ai_rate_limit_hits` 包含 1 条记录：`rate_limit_policy_id="policy-rpm"`、`rate_limit_type="rpm"`、`rule_names` 非空 + +## 清理 + +停止 `bfe` 进程、mock 后端与嵌入式 Redis，删除临时目录。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/\345\234\272\346\231\257\350\257\264\346\230\216.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/\345\234\272\346\231\257\350\257\264\346\230\216.md" new file mode 100644 index 000000000..b94241e82 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/scenario-SC05-AI\350\256\277\351\227\256\346\227\245\345\277\227\345\255\227\346\256\265\346\240\241\351\252\214/\345\234\272\346\231\257\350\257\264\346\230\216.md" @@ -0,0 +1,209 @@ +# SC05 AI 访问日志字段校验 + +## 1. 场景背景与目的 + +`bfe-access-pb` 访问日志协议已完成 AI 可观测字段的扩展与重命名，BFE 代码已同步改造。本场景通过启动真实 `bfe` 进程并发送 AI 请求，收集 `mod_access_pb3` 输出的 b2log，解码 `RequestLog` 后校验各 AI 字段是否与请求行为一致。 + +本场景验证： + +- `ai_apikey_id` 记录 API Key 内部 ID，而非原始 key 值； +- `ai_target_model` 反映路由/映射后的最终模型； +- `ai_provider`、`ai_cost_value`、`ai_cost_currency` 在 RMB 配额场景正确输出； +- `ai_retry_count` 在多 Key 重试时正确累加； +- `ai_route_rule_hits`、`ai_cluster_key_names`、`ai_auth_hit_quota_plans` 与请求行为一致； +- `ai_auth_reject_reason`、`ai_auth_reject_quota_plans` 在配额拒绝场景正确输出。 + +## 2. 运行模式 + +- **单组件模式**：仅启动真实 `bfe` 进程与嵌入式 Redis。 +- 使用 `github.com/alicebob/miniredis/v2` 作为 `mod_ai_token_auth` 的 Redis 后端。 +- 不涉及 `ai-gateway-api` 与 `conf-agent`。 +- 启用 `mod_access_pb3`，将 b2log 输出到临时目录，测试完成后读取并解码。 + +## 3. 涉及的 BFE 配置文件 + +| 文件 | 说明 | +|------|------| +| `bfe.conf` | 启用 `EnableAiGateway`，加载 `mod_ai_route`、`mod_ai_token_auth`、`mod_body_process`、`mod_access_pb3` | +| `mod_ai_route/mod_ai_route.conf` | 指定 `ai_route.data` 路径 | +| `mod_ai_route/ai_route.data` | 定义 apikey 级路由表与绑定 | +| `mod_ai_token_auth/mod_ai_token_auth.conf` | 指定 `token_rule.data` 路径与 Redis 地址 | +| `mod_ai_token_auth/token_rule.data` | 定义 Token、QuotaPlan 与规则 | +| `mod_body_process/mod_body_process.conf` | 启用 body 处理，用于 token 估算与 TTFT/TPOT 计算 | +| `mod_access_pb3/mod_access_pb3.conf` | 指定 b2log 输出目录与文件名前缀 | +| `server_data_conf/host_rule.data` | 声明 `ai_product` 产品线对应的 Host | +| `server_data_conf/route_rule.data` | 默认路由到 fallback cluster | +| `cluster_conf/cluster_conf.data` | 各 cluster 的基础配置，重点在 `AIConf` | +| `cluster_conf/gslb.data` | GSLB 权重配置 | +| `cluster_conf/cluster_table.data` | 运行时根据 mock 后端地址动态生成 | + +## 4. Cluster 与 AIConf 设计 + +### 4.1 测试 cluster + +| cluster | 用途 | +|---------|------| +| `cluster_rmb` | 主 cluster，配置 `ModelTable` 与多 Key，用于 RMB 计费、provider、route rule hit 等字段校验 | +| `cluster_fallback_rmb` | fallback cluster，配置不同的 `ModelTable`，用于验证 fallback 后字段变化 | +| `cluster_no_table` | 未配置 `ModelTable`，用于验证无成本字段场景 | + +### 4.2 `cluster_rmb` 的 `AIConf` + +```json +{ + "AIConf": { + "Type": 0, + "ModelMapping": { + "gpt-4": "deepseek-chat" + }, + "Provider": "mock-provider", + "Keys": [ + {"Name": "key-primary", "Key": "sk-primary", "Weight": 100}, + {"Name": "key-secondary", "Key": "sk-secondary", "Weight": 100} + ], + "KeyPolicy": { + "Strategy": "weighted_random", + "MaxRetries": 2 + }, + "ModelTable": { + "Currency": "RMB", + "Models": [ + { + "Provider": "mock-provider", + "Model": "deepseek-chat", + "BaseModel": "deepseek-chat", + "Mode": "chat", + "Capabilities": ["chat"], + "SupportedParameters": ["temperature", "max_tokens"], + "Limits": {"context_window": 128000}, + "Prices": { + "input_cost_per_token": 0.000001, + "output_cost_per_token": 0.000002 + } + } + ] + } + } +} +``` + +> 说明： +> - `Provider = "mock-provider"` 用于校验 `ai_provider`。 +> - `Currency = "RMB"` 用于校验 `ai_cost_currency`。 +> - `input_cost_per_token = 0.000001` 元/Token，定点整数为 `100`（1e-8 元/Token）。 +> - `output_cost_per_token = 0.000002` 元/Token，定点整数为 `200`。 +> - 配置两个 Key 并允许 `MaxRetries = 2`，用于校验 `ai_retry_count` 和 `ai_cluster_key_names`。 + +### 4.3 `cluster_fallback_rmb` 的 `AIConf` + +与 `cluster_rmb` 结构相同，但 `Provider` 可设为 `"mock-provider-fallback"`，价格不同，例如： + +```json +{ + "Provider": "mock-provider-fallback", + "Prices": { + "input_cost_per_token": 0.000003, + "output_cost_per_token": 0.000004 + } +} +``` + +### 4.4 路由表设计 + +#### apikey_ak_user_a + +- `user_a-rmb` + - 条件：`req_host_in("rmb.example.org")` + - targets：`cluster_rmb` + - fallbacks：`cluster_fallback_rmb` +- `user_a-notable` + - 条件：`req_host_in("notable.example.org")` + - targets：`cluster_no_table` + - fallbacks：无 + +#### ApikeyRouteTableBindings + +```json +{ + "ak_user_a": ["apikey_ak_user_a"] +} +``` + +## 5. Token 与 QuotaPlan 设计 + +### 5.1 Token 配置 + +| API Key | key_id | 绑定的 QuotaPlan | +|---------|--------|------------------| +| `ak_user_a` | `user_a_key_id` | `plan_rmb` / `plan_token` / `plan_rmb_empty`（视测试例而定） | + +### 5.2 QuotaPlan 配置示例 + +#### RMB 配额 + +```json +{ + "Id": "plan_rmb", + "Unlimited": false, + "PassNoQuota": false, + "RedisKey": "quota:plan_rmb", + "CreateTime": 0, + "ExpiredTime": -1, + "Quota": 10000000000, + "ResetMode": 0, + "Unit": "RMB" +} +``` + +#### Token 配额 + +```json +{ + "Id": "plan_token", + "Unlimited": false, + "PassNoQuota": false, + "RedisKey": "quota:plan_token", + "CreateTime": 0, + "ExpiredTime": -1, + "Quota": 10000, + "ResetMode": 0, + "Unit": "total_token" +} +``` + +## 6. 测试例列表 + +| 编号 | 名称 | 核心验证点 | +|------|------|------------| +| TC-01 | 成功请求主要 AI 日志字段 | `ai_apikey_id`、`ai_target_model`、`ai_provider`、`ai_cost_value`、`ai_cost_currency`、`ai_route_rule_hits`、`ai_cluster_key_names`、`ai_auth_hit_quota_plans` | +| TC-02 | ModelMapping 后 target_model 正确 | 请求 `gpt-4` 映射为 `deepseek-chat`，日志中 `ai_requested_model=gpt-4`、`ai_target_model=deepseek-chat` | +| TC-03 | 多 Key 重试时 retry_count 与 cluster_key_names | 主 Key 返回 500 触发同 Key 重试，校验 `ai_retry_count > 0` 且 `ai_cluster_key_names` 包含重试记录 | +| TC-04 | RMB 配额耗尽时拒绝字段 | 余额为 0 时认证拒绝，校验 `ai_auth_reject_reason` 与 `ai_auth_reject_quota_plans` | +| TC-05 | Fallback 后 provider 与 cluster_key_names | 主 cluster 返回 502 触发 fallback，校验 `ai_provider` 为 fallback cluster 的 provider，且 `ai_cluster_key_names` 包含两个 cluster | +| TC-06 | 流式响应字段 | SSE 请求下校验 `ai_stream`、`ai_ttft_us`、`ai_tpot_us` | +| TC-07 | 限流命中字段 | 触发 RPM/TPM 限流，校验 `ai_rate_limit_hits` | + +## 7. 公共基础设施改造 + +本场景需要复用并扩展 SC03 的公共测试框架： + +1. **`common/bfe_config_builder.go`**：已支持动态 Redis 地址、`token_rule.data`、AIConf 注入。 +2. **`common/redis_server.go`**：已支持嵌入式 Redis 与余额读写。 +3. **`common/mock_backend.go`**：已支持返回带 `usage` 字段的响应体；需要支持根据请求次数返回不同状态码，以模拟重试与 fallback。 +4. **新增 b2log 解码辅助函数**：在测试场景内或 `common` 包中提供 `parseAccessLog(logPath) ([]*bfe_access_pb.RequestLog, error)`，基于 `bfe-access-pb/b2log.BuffParse` 与 `proto.Unmarshal`。 + +## 8. 依赖与风险 + +| 风险点 | 说明 | 缓解措施 | +|--------|------|----------| +| b2log 文件写入延迟 | BFE 可能异步刷盘 | 请求发送后等待并轮询文件大小，或关闭 BFE 后再读取 | +| 多测试例并发写日志 | 每个测试例独立临时目录与 BFE 进程 | 通过 `t.TempDir()` 隔离 | +| 字段为空导致断言失败 | 某些字段只在特定路径填充 | 每个 TC 明确前置条件与预期字段 | +| `ai_retry_count` 统计口径 | 仅统计 key-level retry | TC-03 设计为单 cluster 内同 Key 重试，避免 fallback 干扰 | + +## 9. 与其他场景的依赖关系 + +- 依赖 SC01 已验证的 `mod_ai_route` 路由表查找与绑定能力； +- 依赖 SC02 已验证的多 Key 重试与 fallback 行为； +- 依赖 SC03 已验证的 RMB 配额扣减链路； +- 本场景 focus 在 `mod_access_pb3` 输出的 AI 可观测字段完整性与正确性。 diff --git "a/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/\346\265\213\350\257\225\345\234\272\346\231\257\346\200\273\344\275\223\350\257\264\346\230\216.md" "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/\346\265\213\350\257\225\345\234\272\346\231\257\346\200\273\344\275\223\350\257\264\346\230\216.md" new file mode 100644 index 000000000..c597c6843 --- /dev/null +++ "b/tests/integration/\346\265\213\350\257\225\350\256\276\350\256\241\346\226\207\346\241\243/\346\265\213\350\257\225\345\234\272\346\231\257\346\200\273\344\275\223\350\257\264\346\230\216.md" @@ -0,0 +1,127 @@ +# BFE 集成测试场景总体说明 + +## 1. 版本信息 + +| 项目 | 内容 | +|------|------| +| 版本号 | v0.3.0 | +| 对应 BFE 行为方案 | `document-ai-gateway/BFE设计/v0.3.0/BFE的mod_ai_route集成测试方案/BFE的mod_ai_route集成测试方案v1.0.0.md` | +| 测试目标 | 验证 `bfe/mod_ai_route` 的多路由表查找、多 target 加权选择、多 fallback 降级等行为 | +| 测试形态 | 仅启动真实 `bfe` 进程，不依赖 `ai-gateway-api` 与 `conf-agent` | + +## 2. 测试范围 + +本目录下的集成测试重点覆盖 `bfe` 自身的转发行为链路： + +1. **配置消费链路**：`bfe` 启动时加载 `ai_route.data`、`cluster_table.data`、`cluster_conf.data` 等配置文件。 +2. **转发行为链路**：向 `bfe` 发送真实 HTTP 请求，验证 `mod_ai_route` 的多路由表查找顺序、多 target 加权选择、多 fallback 降级、model override、无绑定 404 等行为。 +3. **多 API-Key 链路**：验证 `aiClusterInvoke()` 内部多 Key 加权选择、429 轮换、401/403 死亡、5xx 同 Key 退避重试、Key 耗尽及与 cluster 级 fallback 的边界。 + +> 说明：本目录不验证 `ai-gateway-api` 的 OpenAPI / InnerAPI，也不验证 `conf-agent` 配置下发链路；相关测试请参见仓库根目录下的 `integration-test/`。 + +## 3. 场景清单 + +| 场景编号 | 场景名称 | 一句话描述 | 优先级 | 计划测试例数 | 对应设计文档 | +|----------|----------|------------|--------|--------------|--------------| +| SC01 | 路由表查找与绑定 | 验证 `mod_ai_route` 在 apikey/entity/global 多级路由表中的搜索与回退顺序 | P0 | 10 | BFE 的 mod_ai_route 集成测试方案 v1.0.0 | +| SC02 | 多 API-Key 轮换与重试 | 验证 `aiClusterInvoke()` 内部多 Key 加权选择、429 轮换、401/403 死亡、5xx 退避重试及与 cluster fallback 的边界 | P0 | 8 | BFE 多 API-Key 支持改造方案 | +| SC03 | RMB 配额扣减 | 验证 `mod_ai_token_auth` 对 RMB 配额的定价匹配、定点数扣减、余额预检、ModelMapping/Fallback 计费行为 | P0 | 6 | BFE RMB 配额支持 | +| SC04 | ProviderModel 前缀裁剪 | 验证 `AIConf.MatchPrefix` 与 `StripPrefix` 对请求模型名的裁剪行为 | P0 | 6 | BFE ProviderModel 前缀裁剪 | +| SC05 | AI 访问日志字段校验 | 验证 `mod_access_pb3` 输出的 AI 可观测字段完整性与正确性 | P0 | 5 | BFE 适配 bfe-access-pb AI 可观测字段升级 | + +## 4. 公共前置条件 + +### 4.1 环境与组件前置条件 + +- 测试直接使用当前 `bfe` 源码编译出的可执行文件（`go build`）。 +- 编译产物缓存到 `bfe/tests/integration/.integration-test-bin/`，同一次测试运行只编译一次。 +- 每个测试例拥有独立的临时工作目录与随机监听端口。 +- mock AI 后端通过 `httptest.Server` 启动，其地址在测试运行时动态写入 `cluster_table.data`。 + +### 4.2 BFE 前置条件 + +- `bfe.conf` 中启用 AI 网关（`EnableAiGateway = true`）并加载 `mod_ai_route` 模块。 +- `bfe.conf` 中的 HTTP/HTTPS/monitor 端口与地址由测试框架动态改写为 `127.0.0.1` 上的空闲端口。 +- `ai_route.data`、`host_rule.data`、`route_rule.data`、`cluster_conf.data`、`gslb.data` 等静态文件来自 `testdata/`；`cluster_table.data` 由测试根据 mock 后端地址动态生成。 + +## 5. 场景与测试例对应关系 + +### SC01 路由表查找与绑定 + +| 测试例编号 | 测试例名称 | 优先级 | +|------------|------------|--------| +| TC-01 | APIKey 路由表命中 | P0 | +| TC-02 | Entity 路由表回退 | P0 | +| TC-03 | 无绑定返回 404 | P0 | +| TC-04 | 多 Targets 加权选择 | P0 | +| TC-05 | 多 Fallbacks 最终成功 | P0 | +| TC-06 | 多 Fallbacks 全部失败 | P0 | +| TC-07 | Target 与 Fallback 模型覆盖 | P0 | +| TC-08 | Fallback 时部分已发送 body 可完整回绕 | P0 | +| TC-09 | body 超过 accessibleBodySize 时无法 fallback | P0 | +| TC-10 | 超过 totalBodyBufferSize 时无法 fallback | P0 | + +### SC02 多 API-Key 轮换与重试 + +| 测试例编号 | 测试例名称 | 优先级 | +|------------|------------|--------| +| TC-01 | 多 Key 加权选择 | P0 | +| TC-02 | 429 触发 Key 轮换 | P0 | +| TC-03 | 401/403 标记 Key 死亡 | P0 | +| TC-04 | 5xx 同 Key 退避重试 | P0 | +| TC-05 | Key 耗尽后触发 cluster fallback | P0 | +| TC-06 | 5xx Key 级耗尽触发 cluster fallback | P0 | +| TC-07 | 请求体在 Key 轮换中完整回绕 | P0 | +| TC-08 | AIConf 扩展字段加载 | P0 | + +### SC03 RMB 配额扣减 + +| 测试例编号 | 测试例名称 | 优先级 | +|------------|------------|--------| +| TC-01 | RMB 配额正常扣减 | P0 | +| TC-02 | RMB 配额余额不足拒绝请求 | P0 | +| TC-03 | ModelMapping 按映射后模型计费 | P0 | +| TC-04 | Token 与 RMB 配额共存 | P0 | +| TC-05 | 无 ModelTable 按 0 成本处理 | P0 | +| TC-06 | Fallback 后按最终 cluster 计费 | P0 | + +### SC05 AI 访问日志字段校验 + +| 测试例编号 | 测试例名称 | 优先级 | +|------------|------------|--------| +| TC-01 | 成功请求主要 AI 日志字段 | P0 | +| TC-02 | ModelMapping 后 target_model 正确 | P0 | +| TC-03 | 多 Key 重试时 retry_count 与 cluster_key_names | P0 | +| TC-04 | RMB 配额耗尽时拒绝字段 | P0 | +| TC-05 | Fallback 后 provider 与 cluster_key_names | P0 | +| TC-06 | 流式响应字段 | P0 | +| TC-07 | 限流命中字段 | P0 | + +## 6. 运行方式 + +```bash +cd bfe + +# 运行本目录全部集成测试 +go test ./tests/integration/... -v + +# 运行单个场景 +go test ./tests/integration/implementation/scenario-SC01-route-table-lookup/... -v + +# 运行单个测试例 +go test ./tests/integration/implementation/scenario-SC01-route-table-lookup/ -run TestTC01 -v +``` + +## 7. 依赖与风险 + +| 风险点 | 说明 | 缓解措施 | +|--------|------|----------| +| 编译 BFE 二进制耗时 | 每个 `go test` 进程首次需要编译 | 使用 `sync.Once` 缓存，一次测试运行只编译一次 | +| 端口冲突 | 多个测试并发时可能争夺端口 | 使用 `FindFreePort` 动态分配，且仅绑定 `127.0.0.1` | +| BFE 启动失败排查困难 | 配置错误可能导致启动失败 | 启动时开启 `-s` 输出日志，并收集 exception log | +| 静态配置字段命名 | `ai_route.data` 字段需要与 `bfe/mod_ai_route` 期望的一致 | 使用 `route_rules`、`Cond`、`targets`、`fallbacks` 等标准字段；fallbacks null 转 `[]` | + +## 8. 参考文档 + +- `document-ai-gateway/BFE设计/v0.3.0/BFE的mod_ai_route集成测试方案/BFE的mod_ai_route集成测试方案v1.0.0.md` +- `integration-test/方案说明/总体说明/集成测试方案说明.md`