di.grafana module

[Olly99999]

[Olly99999]

Summary

Extracts the .grafana namespace from TorQ's grafana.q into a standalone kdb-x module: di.grafana. The module turns a kdb+ process into a Grafana SimPod JSON datasource: it installs HTTP handlers that answer Grafana's /search, /query and connection-test requests with the JSON the plugin expects, serving the process's tables as timeseries and table panels. It satisfies the di.* module contract: dependency injection via init, a clean exported API, and no hard dependencies beyond an injected logger.

Background

TorQ's grafana.q implements the Grafana JSON-datasource adaptor as a .grafana namespace that hooks .z.pp/.z.ph through TorQ's .dotz handler manager. This PR is part of the broader TorQ → kdb-x modularisation effort: make each functional area independently loadable, testable, and injectable, removing the dependency on TorQ's global process framework.

Changes

New files

File	Description
di/grafana/grafana.q	Core implementation — init, getconfig, the .z.pp/.z.ph wrappers, and the search/query request builders
di/grafana/init.q	Module entry point — loads grafana.q and declares the export list
di/grafana/test.csv	k4unit test suite (32 tests)
di/grafana/grafana.md	Module README

Differences from TorQ original

Aspect	TorQ .grafana	di.grafana
Handlers	.dotz.set / .dotz.getcommand (TorQ handler manager)	init closure-wraps .z.pp/.z.ph directly, preserving any pre-existing handler so non-Grafana traffic falls through untouched. Follows the di.timer precedent (.z.ts); no di.handlers module exists yet to inject. Verified by a fall-through test.
Logging	None	Required injected log dependency (info/warn/error, {[ctx;msg]}). Request dispatch trapped; failures logged under the grafana context
Config	@[value; .grafana.x ;default] globals	Module-local defaults + optional config dict in init (timecol, sym, timebackdate, ticks, del)
Global state / namespace	\d .grafana	Flat module namespace; mutable state on .z.m/.z.M
Dependency validation	N/A	init validates a non-null log dep and errors with a di.grafana-prefixed message if absent
Module contract	None	kdb-x use singleton, init[deps] pattern, export: list

Exported API

grafana:use`di.grafana

grafana.init[deps]      / wire dependencies + config, install handlers; required before use
grafana.getconfig[]     / return the currently active configuration dict

Everything else is driven by inbound Grafana HTTP requests via the installed handlers, so it is intentionally not exported.

deps forms:

grafana.init[enlist[`log]!enlist logdep]              / logger only, default config
grafana.init[`log`config!(logdep;`ticks!500)]         / logger + config overrides

Query target syntax

/search emits encoded metric strings that /query parses back; del (default .) separates the arguments:

Target	Meaning
t.<table>	table panel, whole table
t.<table>.<sym>	table panel filtered to one sym
g.<table>	graph, one series per numeric column
g.<table>.<col>	graph, one series per sym for a column
o.<table>...	"other" panels (stat/gauge)
f....	target is a function call rather than a table

Test coverage

Run via k4unit: k4unit.moduletest `di.grafana

Area	Tests
Export surface	only init + getconfig exported
init — dependency validation	rejects :: and a null log dep
init — wiring	wired flag set; handlers installed
Config	defaults applied; override takes; getconfig reflects it
Parsing helpers	isfunc / istab / istype / prefix
Data fetch	finddistinctsyms, memvals, catchvals
/query table	tbfunc returns the columns/rows/table schema
/query timeseries	tsfunc returns target/datapoints numeric series
Handler routing	/search POST + GET test-connection via live handlers; non-Grafana POST falls through to the pre-existing handler

Total: 32 tests, all passing.

Notes

• di.grafana has no hard module dependencies — only the injected log dict is required.
• di.log satisfies the log contract out of the box:

logdep:`info`warn`error!(log.info;log.warn;log.error)

• The handler wrapper preserves any pre-existing .z.pp/.z.ph, so adding the module to a process that already serves HTTP does not break existing endpoints.
• The /annotations endpoint returns a "not yet implemented" marker, matching the TorQ original.

[DI-Software-Engineering]

In search, the variable rsp is built by appending string table names (string tabs), then later appending prefixed targets (prefix["t";string timetabs] etc). The initial string tabs entries have no prefix, so they appear as raw table names alongside t.*/g.* targets. This is almost certainly wrong — bare table names cannot be dispatched by the prefix-based routing in tbfunc/tsfunc, and zpp → query will fall through to annotation for them. Remove the bare string tabs line or replace it with appropriate prefixed entries.

[DI-Software-Engineering]

finddistinctsyms uses a functional select that references the module-level names timecol, sym, and timebackdate by value (not via .z.m.*). After init overrides these via .z.m.timecol etc., the local namespace variables are never updated, so finddistinctsyms (and other functions referencing timecol, sym, ticks, timebackdate, del by bare name) will always use the initial default values instead of the configured ones. The init function assigns deps values to .z.m.timecol, .z.m.sym, etc. but the rest of the code reads the unqualified names, which resolve to the module-level constants set at load time.

[DI-Software-Engineering]

memvals uses ?[x;enlist(within;i;count[x]-ticks,0);0b;()]. The withincheck`i within count[x]-ticks,0is evaluated as`i within (count[x]-ticks),0— meaning the lower bound iscount[x]-ticksand the upper bound is0, which is always an empty range for any positive count. The intended expression is likely (count[x]-ticks),count[x]-1or?[x;();0b;()]followed by a tail take. This meansmemvals` silently returns an empty table for every in-memory table.

[DI-Software-Engineering]

In tsfunc, args 1 is used as a table reference (passed to value) when the target is a plain g.table.col string. When isfunc is false, args is `$del vs targ — a symbol list. args 1 is thus a symbol like `trade, and value on a symbol executes it, which is fine for a global. But when isfunc is true, args is a 2-element list from (0;1+targ?del)cut targ:2_targ — a pair of strings, not symbols. args 1 would then be a string, and 0!value args 1 would attempt value on a string, which evaluates it as q code rather than looking up a table. This is an injection risk: a crafted Grafana target of the form f.<q expression> will be executed by the server.

[DI-Software-Engineering]

In tsfunc, when isfunc is false the branch is `$del vs targ giving a symbol list; args 0 is then a symbol like `g. The subsequent check 10h=abs type args 0 tests for a char/string — a symbol has type -11h, so tyargs will be assigned via `$1# applied to a symbol, yielding `g (symbol), not `g (symbol from char). The downstream $[(2<numargs)and\g~tyargs;...]comparisons use ``g (symbol) for `tyargs` and g `` (symbol) for the literal, so they coincidentally match — but when isfuncis trueargs 0is a string (type 10h),tyargsbecomes ``$1#"g" = g ``, so both branches produce the same result. The logic is confusing and fragile but the isfuncpath hides the mismatch. Flag for clarity: the10h=abs type args 0branch should be theisfunc`-false (symbol) branch, not the string branch; the condition is inverted relative to the comment.

[DI-Software-Engineering]

sethandlers stores the previous .z.pp/.z.ph into .z.m.prevpp/.z.m.prevph and then defines the new .z.pp/.z.ph as plain lambda closures that reference .z.m.prevpp etc. by name. However, .z.m.wired is set at the end of sethandlers, but init checks the unqualified wired variable — which is always 0b (it was set at module load and never updated via .z.m.wired). So sethandlers is called on every init invocation, re-wrapping the handlers each time and creating a chain of growing closures. The guard if[not wired;sethandlers[]] should read if[not .z.m.wired;sethandlers[]].

[DI-Software-Engineering]

In zpp, r:(0;n?" ")cut n:first x splits the raw HTTP request line at the first space. For a POST /query HTTP/1.1 request, r 0 will be the method (e.g. "POST"), not the path (e.g. "query"). The handler dispatch $["query"~r 0;...] then compares the method string to the path, and will never match, so every request falls through to annotation. The split should extract the path component (between the first and second spaces), not the method.

[DI-Software-Engineering]

In query, rqtype:raze rqt[\targets]`typeretrieves thetypefield from all targets and razes them. If multiple targets are present with different types (mixed timeserie/table), the raze produces a concatenated string and the comparisonrqtype~"timeserie"will be false, silently sending a mixed request totbfunc`. The dispatch logic does not handle multi-target requests correctly.

[DI-Software-Engineering]

In tbfunc, rqt:raze rqt[\targets]`targetretrieves all targets and razes them. Iftargetscontains multiple entries (Grafana can send multiple targets per query), the strings are concatenated byraze, producing a garbled target name. All of tsfuncandtbfunc` assume a single target; multi-target requests are silently corrupted rather than rejected or iterated.

[DI-Software-Engineering]

In query, rqtype:raze rqt[targets]type extracts the type field from the targets list. When multiple targets are present, raze concatenates all their type strings together (e.g. "timeserietimeserie" or "timetimeserie"), so the subsequent ~"timeserie" test will silently fail for more than one target. Each target should be dispatched individually, or the type should be taken from a single/first target rather than razing all.

[DI-Software-Engineering]

query dispatches the entire rqt (with all targets) to either tsfunc or tbfunc, but those functions extract raze rqt[targets]target which collapses all targets into one string. Grafana /query requests regularly contain multiple targets; only the first (or a garbled concatenation) will be processed, silently dropping the others. The response should loop over targets.

[DI-Software-Engineering]

memvals uses `i as a virtual column in a functional select: ?[x;enlist(within;i;count[x]-ticks,0);0b;()]. The lower bound count[x]-tickscan be negative when the table has fewer rows thanticks, producing an invalid range (e.g. -500to0). This will return zero rows instead of the whole table. The lower bound should be clamped to 0, e.g. 0|count[x]-ticks`.

[DI-Software-Engineering]

In tsfunc, rqt:0!value args 1 evaluates args 1 as a kdb+ value. When isfunc is false the target is split by del and args is a symbol list (`$del vs targ), so args 1 is a plain symbol and value on it looks up a variable — that is acceptable. But when isfunc is true the target is a string (the function-call string is kept as a character list), so args 1 is a character-list substring that value will execute as code. A Grafana user (or attacker) controlling the target field can inject arbitrary q code. The function path should use a safe lookup rather than value on a user-supplied string.

[DI-Software-Engineering]

tsfunc converts the time column with rqt:@[rqt;timecol;+;.z.D] when the column type is not "p". .z.D is a date, not a timespan/timestamp. Adding a date to most time types (e.g. time which is a timespan) uses kdb+ temporal arithmetic, but the result type and correctness depend on the original column type. If the column is already a datetime or timestamp-like type other than "p", this coercion may produce unexpected results or a type error.

[DI-Software-Engineering]

range:"P"$-1_'x[range]fromtodrops the last character of the Grafana ISO-8601 datetime strings (theZ) before casting to timestamp. This fragile manual trim assumes the strings always end in exactly one character to drop. A malformed or differently formatted timestamp from Grafana (e.g. with milliseconds "2024-01-01T00:00:00.000Z") will still have its last character dropped but may then fail to parse or parse incorrectly. Use a robust parse or strip the trailing Zexplicitly withssr`.

[DI-Software-Engineering]

buildnosym uses value each ?[x;();0b;z!z] to extract column values, where z is a two-element list (colname;msec). Selecting two columns returns a table with two columns; value eachon a table returns a list of two column vectors (one per row ofvalue). The datapoints structure Grafana expects is a list of (value;milliseconds)pairs.flip value flip(as used inothersym/graphsym) would be correct here; value each` on the table produces per-column vectors, not per-row pairs.

[DI-Software-Engineering]

In tbfunc, when the target is a plain table name (not prefixed with f. or t.), the code falls into the istab branch anyway because istab rqt is tested last and returns false, so rqt is passed through unchanged as a string. The final else branch rqt would be the raw target string. Then 0!value rqt attempts to evaluate the raw string. A target like "trade" (no prefix) would not match isfunc or istab, so rqt remains the original string and value executes it — which is another code-injection vector for non-prefixed targets. The else/fallthrough case should reject unknown prefixes rather than calling value on the raw input.

[DI-Software-Engineering]

init guards against re-installing handlers with if[not wired;sethandlers[]], reading the module-level wired variable. But sethandlers writes .z.m.wired (the namespaced copy) while the guard reads the bare wired (the top-level copy). After the first init call .z.m.wired is 1b but wired remains 0b, so sethandlers will be called on every subsequent init, re-wrapping .z.pp/.z.ph each time and creating a growing chain of nested closures.

[DI-Software-Engineering]

In sethandlers, the .z.pp handler detects Grafana requests by checking ($"X-Grafana-Org-Id") in key last x. The kdb+ HTTP POST handler receives a two-element list: the body string and the header dictionary. If the request arrives with no headers (or xis not a two-element list),last xis the body string andkey on a string returns indices (0 1 2 ...), so the symbol will never be found — this is benign. However, if xis a plain string (e.g. non-HTTP invocation),last xis a character andkeyerrors. More critically, a plainGETfrom.z.phwhere headers are absent will cause.z.ppto error rather than fall through. The handler should guardtype last xbefore callingkey`.

[DI-Software-Engineering]

DIReview Summary

2 critical | 8 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.