Skip to content

Commit 46192a2

Browse files
jahnvi480Copilot
andcommitted
AI: Preserve original Perf Police flow with additive guidance
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 2612361 commit 46192a2

2 files changed

Lines changed: 114 additions & 67 deletions

File tree

‎.github/skills/mssql-profiler/SKILL.md‎

Lines changed: 31 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,10 @@ Read the [profiler guide](../../../profiler/README.md) and inspect the relevant
1515
1. Identify whether the request is diagnosis of one revision, a base/PR
1616
comparison, or interpretation of existing artifacts. Identify the workload,
1717
requested platform, measurement boundary, and reasonable run budget.
18+
1819
State the suspected cause, the expected observation, and what would refute it.
1920
Prefer one bounded experiment using existing workloads over a new framework.
21+
2022
2. Use the supplied checkout and its active development interpreter. Record the
2123
revision and dirty changes. For A/B work, use separate pinned checkouts/builds;
2224
do not switch source underneath one binary or alter the user's shared checkout.
@@ -203,27 +205,35 @@ enable boundary, or disabled interval can be dropped.
203205
Verify each process imports its intended package and native binary.
204206
2. Confirm the workload reaches the changed path. Include eligible repetitions,
205207
shape changes, and excluded inputs where applicable; unchanged fallback
206-
timings are not proof of cache benefits. Read the actual benchmark registry
207-
and worker, not just scenario names: ordinary SELECT fetching may never reach
208-
a catalog-specific branch. Include unchanged-path controls for shared changes.
208+
timings are not proof of cache benefits.
209+
210+
Read the actual benchmark registry and worker, not just scenario names:
211+
ordinary SELECT fetching may never reach a catalog-specific branch. Include
212+
unchanged-path controls for shared changes.
213+
209214
3. Define cold setup, warmup, and the measured sequence before running. Keep
210215
setup, input generation, readback, and cleanup outside timing unless they are
211216
explicitly part of the question. Do not invalidate a prepared statement with
212217
housekeeping SQL on the same cursor between measured repetitions.
213218
4. Run matching profiling-enabled builds for attribution. Record actual call
214219
counts and changed-value correctness, not only duration. A missing timer can
215220
mean missing instrumentation; verify expected enclosing counters before
216-
interpreting absence as zero calls. For a diagnostic experiment, vary one
217-
suspected cause while retaining a matched control and the other conditions.
218-
Check the predicted operation/phase change, not just a lower total time.
219-
Keep diagnostic probes out of the final release-latency comparison.
221+
interpreting absence as zero calls.
222+
223+
For a diagnostic experiment, vary one suspected cause while retaining a
224+
matched control and the other conditions. Check the predicted operation/phase
225+
change, not just a lower total time. Keep diagnostic probes out of the final
226+
release-latency comparison.
227+
220228
5. Separately rebuild both revisions without native profiling and time the same
221229
workload/window using an existing matching benchmark or a small scratch timing
222230
harness. `Profiler()` cannot run on an uninstrumented build; do not use the
223-
profiling runner for this control or compare different workloads. Share setup,
224-
values, timing, validation, and cleanup where possible; adapt only the
225-
instrumentation. Report unavailable counters as unavailable, not zero or
226-
results manufactured by a fake profiler.
231+
profiling runner for this control or compare different workloads.
232+
233+
Share setup, values, timing, validation, and cleanup where possible; adapt
234+
only the instrumentation. Report unavailable counters as unavailable, not
235+
zero or results manufactured by a fake profiler.
236+
227237
6. Counterbalance base/PR order over repeated rounds. Preserve individual
228238
observations and note contention. Do not run competing builds, benchmarks,
229239
or DB-heavy tests on the same machine/server during measurement.
@@ -295,13 +305,19 @@ Return a concise report with these fields, marking unavailable fields explicitly
295305

296306
| Field | Required detail |
297307
| --- | --- |
298-
| Provenance | Base/head SHA and CI merge/run/job/attempt when applicable, dirty changes, interpreter/dependencies, native binary and ODBC provider identity, effective build/profiling flags, OS/architecture, SQL Server version |
308+
| Provenance | Base/head SHA, dirty changes, interpreter, native binary identity, effective build/profiling flags, OS/architecture, SQL Server version |
299309
| Workload | Scenario/script, rows/columns/parameters, value/type distribution, expected path and observed operation counts |
300-
| Window | Setup/warmup policy, measured operations, commit/fetch/cleanup inclusion, repetitions, process reuse, predecessor workloads and within-worker order |
310+
| Window | Setup/warmup policy, measured operations, commit/fetch/cleanup inclusion, repetitions and run order |
301311
| Results | Raw artifact paths, per-round observations, aggregate units, timings, deltas and variability |
302312
| Interpretation | What was observed, what remains uncertain, failures/skips, and platform/execution limits |
303313

314+
Extend provenance with dependencies, actual ODBC provider identity, and any CI
315+
merge/run/job/attempt. Include the process reuse, predecessor workloads, and
316+
within-worker order described above in the window.
317+
304318
Do not include credentials or private connection details in that report.
305-
Preserve raw evidence and clean up only run-owned temporary resources. Verify
306-
cleanup actually completed; a timeout or stop request alone does not prove it.
319+
Preserve raw evidence and clean up only run-owned temporary resources.
307320
Do not push artifacts, edit PR descriptions, or publish findings unless requested.
321+
322+
Verify cleanup actually completed; a timeout or stop request alone does not
323+
prove it.

‎.github/skills/performance-code-review/SKILL.md‎

Lines changed: 83 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -33,49 +33,74 @@ it is not a replacement for unrelated repository checks.
3333
Use this workflow to assess a change or plan a permitted scratch experiment;
3434
it does not authorize production edits or additional runtime activity.
3535

36-
1. **Define the problem and the boundaries.** State the observable failure or
37-
slow operation, affected inputs, expected behavior, and what would count as
38-
improvement. Identify contracts and neighboring paths that must not change.
39-
2. **Establish a trustworthy baseline.** Reproduce through the supported public
40-
API using the smallest relevant existing harness and the same input on the
41-
candidate and its actual base. Reduce the workload without removing the
42-
conditions that trigger the problem. Separate pre-existing behavior and
43-
environment/setup failures from a change-specific effect.
44-
3. **Trace what actually runs.** Read changed units, callers, deleted checks,
45-
and cleanup paths. Follow dispatch rather than assuming similarly named
46-
operations share an implementation. For parameter work, distinguish
47-
`execute`, `executemany`, array binding, overrides, and DAE. Establish loop
48-
frequency, data flow, ownership, and timer boundaries before naming a cost.
49-
4. **Separate observations from explanations.** Record the raw symptom, likely
50-
causes, and the smallest check that would distinguish them. For each proposed
51-
cause, predict what should change and what should stay unchanged. A costly
52-
pattern or a correlated timing change is a hypothesis, not a root cause.
53-
5. **Test one explanation at a time.** Keep a pinned control and vary one
54-
relevant factor in an isolated experiment when execution is permitted.
55-
Hold unrelated code, workload, settings, and timing boundaries fixed.
56-
Retain adverse and inconclusive results. If a check cannot distinguish the
57-
causes, refine the hypothesis rather than enlarge the patch or discard data.
58-
6. **Assess the smallest causal fix.** Connect the proposed change to the
59-
demonstrated mechanism, not just a better aggregate score. Prefer a local
60-
removal of redundant work, a valid invariant moved out of a loop, or a
61-
narrowly eligible fast path over a broad rewrite or new cache. Treat an
62-
untested alternative as a suggestion, not an equivalent fix.
63-
7. **Trace the full impact before removing work.** Identify every caller of
64-
shared helpers and every correctness responsibility of the old path.
65-
Preserve diagnostics, error propagation, ownership, invalidation, fallback,
66-
and cleanup. Move a responsibility to a verified replacement before
67-
removing its old implementation; do not trade correctness for fewer calls.
68-
8. **Check both the mechanism and unaffected behavior.** Inspect existing tests
69-
and diff coverage. Verify the intended behavior change and unchanged
70-
contracts against the baseline, with eligible, excluded, boundary, failure,
71-
and recovery cases. Assert promised operation reductions as well as values.
72-
Retain representative neighboring-path controls; a passing target workload
73-
or coverage percentage does not establish that shared behavior is safe.
74-
9. **Decide from the evidence.** Check whether attribution supports the proposed
75-
cause and whether matched release measurements justify the added complexity.
76-
Drop disproved findings and explain corrections. Separate established
77-
mechanisms, measured effects, unresolved alternatives, and untested paths;
78-
do not force an optimization when the evidence remains inconclusive.
36+
1. Read changed units and their surrounding callers, including deleted safety
37+
checks, tests, public Python entry points, and native cleanup paths.
38+
39+
For a failure or slow operation, state the affected inputs, expected behavior,
40+
what improvement means, and neighboring contracts that must not change.
41+
42+
2. Trace the current call graph. For parameter work, distinguish `execute`,
43+
`executemany`, array binding, overrides, and DAE rather than assuming that
44+
similarly named operations share a binder.
45+
46+
Follow actual dispatch, data flow, and timer boundaries before attributing
47+
a cost.
48+
49+
3. Locate each changed loop and its input-dependent frequency. Apply the cost
50+
model and pattern checklist below; record concrete growth before zone labels.
51+
52+
Separate the observed cost from its explanation: a costly pattern or
53+
correlated timing change is not a demonstrated root cause.
54+
55+
4. Trace ownership, Python contracts, error returns, and invalidation end to end.
56+
Treat initial findings as hypotheses, not established defects.
57+
58+
For each suspected cause, identify the smallest check that could refute it
59+
and predict what should change and what should stay unchanged. Trace every
60+
shared-helper caller; move diagnostics, error propagation, ownership,
61+
invalidation, fallback, and cleanup to a verified replacement before
62+
removing old work.
63+
64+
5. Reproduce candidate defects with the smallest relevant existing harness.
65+
Compare the same input on the PR and its actual base. Check public API paths,
66+
not only direct calls into internal wrappers.
67+
68+
In a permitted isolated experiment, keep a pinned control and vary one
69+
relevant factor; hold unrelated code, workload, settings, and timing
70+
boundaries fixed. Reduce the workload without losing the trigger, and
71+
distinguish setup, fixture, and harness failures from driver behavior.
72+
73+
6. Inspect available diff coverage and existing regression tests. Prioritize
74+
uncovered failure, fallback, boundary, and teardown paths. Coverage percentage
75+
alone neither proves safety nor establishes a defect.
76+
77+
Assert promised operation reductions as well as values. Retain representative
78+
neighboring-path controls and eligible, excluded, boundary, and recovery cases;
79+
a passing target workload alone does not establish shared-path safety.
80+
81+
7. Evaluate performance evidence and simplifications using the rules below.
82+
Do not ask for a migration or dependency PR without a demonstrated call-path
83+
dependency; an exceptional DAE path does not justify rewriting all batching.
84+
85+
Connect the smallest proposed fix to the demonstrated mechanism, not only
86+
an aggregate score. Prefer removing redundant work, moving a valid invariant
87+
out of a loop, or a narrowly eligible fast path over a rewrite or new cache.
88+
Use matched release measurements to judge added complexity; an untested
89+
alternative is a suggestion, not an equivalent fix.
90+
91+
8. Challenge each finding: what evidence would disprove it, is it pre-existing,
92+
is the caller supported, and does the suggested fix preserve every invariant?
93+
Drop disproved claims and explain material corrections plainly.
94+
95+
Retain adverse and inconclusive results. If a check cannot distinguish causes,
96+
refine the hypothesis rather than enlarge the patch or discard data. Do not
97+
force an optimization without supporting evidence.
98+
99+
9. Report only actionable findings. Separate correctness, measured performance,
100+
unverified concerns, and code-organization suggestions.
101+
102+
Scope claims to established mechanisms and measured effects; identify
103+
unresolved alternatives and untested paths.
79104

80105
Before concluding, answer these questions from the evidence: does the change
81106
address the ask; is its scope justified; does it regress other supported paths;
@@ -159,9 +184,11 @@ dispatch path. Never flag it solely because its type name appears in a loop.
159184
preserve the expected exception behavior.
160185
- Keep Python code-point counts, UTF-16 code units, and encoded bytes distinct.
161186
Astral characters require two UTF-16 units. Size the final normalized/formatted
162-
value, including the chosen encoding and any required terminator. Use the same
163-
discipline in SQL fixtures: verify that boundary values fit the declared type
164-
and reach the intended driver path before interpreting a failed assertion.
187+
value, including the chosen encoding and any required terminator.
188+
189+
Use the same discipline in SQL fixtures: verify that boundary values fit the
190+
declared type and reach the intended driver path before interpreting a failed
191+
assertion.
165192
- Inspect actual consumers of new metadata. Populating an unused length field
166193
may be a correctness prerequisite, but it is not evidence of a current
167194
customer-visible crash fix or a delivered speedup.
@@ -230,9 +257,11 @@ concurrent-lifecycle design.
230257
adding another framework; more validation machinery is not performance evidence.
231258
- Require evidence for a performance optimization's payoff. Safety ownership,
232259
protocol validation, and error handling do **not** need a throughput gain to
233-
justify their existence. Classify added code by responsibility before proposing
234-
cuts; neither fewer lines nor fewer calls proves lower latency. Reject a
235-
measured slowdown rather than preserving machinery because it looks cheaper.
260+
justify their existence.
261+
262+
Classify added code by responsibility before proposing cuts; neither fewer
263+
lines nor fewer calls proves lower latency. Reject a measured slowdown rather
264+
than preserving machinery because it looks cheaper.
236265
- Before calling a guard redundant, prove the state unreachable through all
237266
supported callers. Distinguish a genuinely fixed bound from user-supplied data.
238267
- A proposed cut must preserve diagnostics, lifetime, fallback, and public API
@@ -288,9 +317,11 @@ do not start an unrelated benchmark merely because a review is running.
288317
loader can legitimately load the `.so`/`.pyd`; its own path is not sufficient
289318
provenance. Discard mislabeled or logging-contaminated runs.
290319
- Use Release/`-DNDEBUG` for comparisons. Debug-only assertion costs are not
291-
release regressions. Verify effective configuration and compiler flags as
292-
described in the [profiler skill](../mssql-profiler/SKILL.md#build-and-verify-instrumentation),
293-
not a build's label. Do not copy another PR's numbers onto a later change.
320+
release regressions. Do not copy another PR's numbers onto a later change.
321+
322+
Verify effective configuration and compiler flags as described in the
323+
[profiler skill](../mssql-profiler/SKILL.md#build-and-verify-instrumentation),
324+
not a build's label.
294325
- Counterbalance base/PR order, retain raw per-round observations, and avoid
295326
competing benchmarks or DB-heavy tests on the same machine/server. Alternation
296327
reduces order bias; it does not remove all contention or server-state effects.

0 commit comments

Comments
 (0)