性能测试和规格测试 - #3837
Open
lijiayang619 wants to merge 4 commits into
Open
Conversation
lijiayang619
requested review from
Dallas98,
WMC001 and
jeffwu-1999
as code owners
September 1, 2026 07:30
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a standalone “scale/spec + perf” testing harness under test/scale/ to validate Nexent’s data-volume specifications and northbound API performance specs against a running instance, with resource monitoring and report generation.
Changes:
- Introduces a shared test framework (
BaseTestCase/PerfTestCase), HTTP/session utilities, resource monitoring, and markdown report generation. - Adds 12 data-volume spec tests (
spec_test.py) and 3 performance spec tests (perf_test.py) with CLI filtering/selection. - Adds rollback + report regeneration utilities, and prevents these integration tests from being collected by pytest.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
test/scale/utils.py |
Core framework utilities: login/session + HTTP helpers, resource monitoring, and final report generation. |
test/scale/spec_test.py |
Data-volume specification test suite with filter/skip CLI support. |
test/scale/perf_test.py |
Performance specification test suite hitting the northbound streaming API. |
test/scale/rollback.py |
Cleanup script to remove test-created data by prefix or via a dangerous --all mode. |
test/scale/generate_report.py |
Regenerates the final markdown report from stored spec/perf report files. |
test/scale/conftest.py |
Prevents pytest/CI from collecting these integration test scripts. |
Suppressed comments (1)
test/scale/utils.py:74
- Avoid logging any part of the newly created northbound access key; redact it completely to prevent credential leakage via logs.
key = resp.get("data", {}).get("access_key")
if not key:
raise RuntimeError("Failed to create NB key")
logger.info("Created new NB key: %s", key[:20] + "...")
return key
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+34
to
+36
| NB_BASE = os.getenv("NEXENT_NB_BASE", "http://localhost:5013") | ||
| NB_KEY = os.getenv("NEXENT_NB_KEY", "nexent-perftest-ddca790af3c2") | ||
| DP_BASE = os.getenv("NEXENT_DP_BASE", "http://localhost:5012") |
Comment on lines
+64
to
+67
| key = token_list[0].get("access_key") | ||
| if key: | ||
| logger.info("Reusing existing NB key: %s", key[:20] + "...") | ||
| return key |
Comment on lines
+123
to
+130
| current_session = _session_cache.get(email) | ||
| if current_session is not session: | ||
| session = current_session | ||
| else: | ||
| logger.warning("401 on %s %s, re-logining...", method, path) | ||
| _session_cache.pop(email, None) | ||
| session = login(email) | ||
| resp = session.request(method, url, timeout=timeout, **kwargs) |
Comment on lines
+33
to
+37
| create_skill, delete_skill, | ||
| create_memory_record, delete_memory_record, | ||
| api_get, api_post, api_delete, | ||
| collect_db_metrics, | ||
| BASE_URL, ADMIN_EMAIL, |
Comment on lines
+286
to
+287
| nb_base = "http://localhost:5013" | ||
| nb_headers = {"Authorization": "Bearer nexent-perftest-ddca790af3c2"} |
Comment on lines
+428
to
+445
| created_count = 0 | ||
| def _create(idx): | ||
| nonlocal created_count | ||
| try: | ||
| resp = create_mcp_service(f"scale-mcp-{RUN_ID}-{idx:04d}") | ||
| if isinstance(resp, dict) and resp.get("status") == "success": | ||
| created_count += 1 | ||
| return True | ||
| return None | ||
| except Exception as e: | ||
| logger.warning("Create MCP %d failed: %s", idx, e) | ||
| return None | ||
|
|
||
| with concurrent.futures.ThreadPoolExecutor(max_workers=CONCURRENCY) as pool: | ||
| futures = [pool.submit(_create, i) for i in range(self.target)] | ||
| for f in concurrent.futures.as_completed(futures): | ||
| f.result() | ||
| return list(range(created_count)) |
Comment on lines
+96
to
+98
| tasks = [] | ||
| loop = asyncio.get_event_loop() | ||
| next_time = loop.time() |
Comment on lines
+17
to
+21
| import os | ||
| import time | ||
| import logging | ||
| import concurrent.futures | ||
| from typing import List |
added 2 commits
September 1, 2026 17:24
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
文件
测试结果
用法
```bash
设环境变量
export NEXENT_TEST_EMAIL=your_email
export NEXENT_TEST_PASSWORD=your_password
跑规格测试
python test/scale/spec_test.py --filter 1-11
跑性能测试
python test/scale/perf_test.py
清理数据
python test/scale/rollback.py --all
生成报告
python test/scale/generate_report.py
```
"