Conversation
pgtypes.Array and pgtypes.EnumArray now implement pgtype.ArrayGetter and pgtype.ArraySetter, and their Scan method decodes pgx's binary array format as well as the text literal. bobgen-psql generates pgtypes.Array[T] for every array column instead of the lib/pq array types, which only understand the text format. Fixes stephenafamo#90, stephenafamo#739 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A pgtype.Map is not safe for concurrent use. Sharing one across all array scans only worked because current pgx versions do not memoize scan plans, which is an undocumented internal. Take a map from a sync.Pool per scan instead; creating one is cheap since it defers to pgx's default registry. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Scanning the text format and producing a driver.Value for the seven primitive element types is about twice as fast through lib/pq's typed arrays than through pq.GenericArray and the pgx text parser, and it produces the same bytes those columns produced before. Anything lib/pq cannot parse, such as explicit bounds, still falls back to pgx. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Both types had identical method bodies. A generic type alias keeps the exported name and method set while removing the duplication. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bobgen-psql no longer emits the lib/pq array types, so their built-in definitions were dead code. Users who want them can still register them under the types configuration. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Remove a mislabeled duplicate case, share the unknown-OID fixture helper between the array and enum tests, and simplify the generated JSON comparison helper. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@stephenafamo hope that you would not be angry that this code is AI generated. We are trying to move in our codebase to pgx - but we are blocked by #90 - so I am trying to solve it. If you have any feedback please share I would try to address. |
Only the native pgx driver now generates pgtypes.Array for every array column; lib/pq and pgx/v5/stdlib keep generating the same types as before and the golden files are unchanged. The native driver is enabled as experimental and tested against a dedicated schema of array columns. The runtime drops the lib/pq fast paths that database/sql users no longer reach, adds translator unit tests, a nullable-parameter round-trip case and documentation for replacing array types under native pgx. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@jalaziz looking on your codebase I decided to have a change to which you inspred me. #771 tackles the same problem by generating bare |
|
Hi! I'd like to chime in and support this effort. In our team, this issue is also blocking the full transition to the native |
Summary
Makes PostgreSQL array columns work with native pgx, without changing anything for existing
lib/pqandpgx/v5/stdlibusers.Fixes #90
Fixes #739
The problem
Both issues are the same bug. Every Go type Bob generates for an array column is a
sql.Scannerthat only understands the Postgres text array literal ({a,b}):pq.StringArrayand friends for primitive elements, andpgtypes.Array[T](a thinpq.GenericArrayshim) for everything else.Native pgx tries
pgtype.ArraySetterfirst and, for anything else, falls back to thesql.Scannerpath, where it hands the Scanner the raw binary wire format. That is thepq: unable to parse array; expected '{' at offset 0in#90. A
null.Val[...]wrapper is itself a Scanner, so it receives the same binary bytes and forwards them to the inner type, which is theunsupported Scan, storing driver.Value type []uint8in #739.pgx stdlib only works because it forces the text format for array OIDs.
On the Go 1.26/1.27
driver.RowsColumnScannerquestion raised in #90: it does not help here. Native pgx never goes throughdatabase/sql, and pgx's stdlib implementation checkssql.ScannerbeforeArraySetter, so any Scanner typestill receives the text value. This PR needs no Go or pgx version bump.
What changed
Runtime (
types/pgtypes)pgtypes.Array[T]implementspgtype.ArrayGetterandpgtype.ArraySetter(modelled on pgx'spgtype.FlatArray), so native pgx scans and encodes it directly. It stays asql.Scanner/driver.Valuerfordatabase/sqldrivers.Scanalso decodes pgx's binary array format, using pgx's ownArrayCodecwith the element OID from the header. This is what makes the wrapped cases work:null.Val,sql.Nullandorm.NullTypeConverter. Nullable arrays keepnull.Valsemantics, identical across drivers.pgtype.Mapis not safe for concurrent use, so each scan takes one from async.Pool. A race-detector test covers this.pgtypes.EnumArray[T]is now an alias ofpgtypes.Array[T]; the two had identical method bodies.Code generation (
bobgen-psql)github.com/jackc/pgx/v5(native pgx, used withbob/drivers/pgx) is accepted asdriveragain, marked experimental.pgtypes.Array[T]. Thelib/pqandpgx/v5/stdlibdrivers generate exactly what they generated before; the golden files are unchanged.pq.BoolArray's built-in type definition was missing a compare expression; added.Not breaking
No generated type changes unless the native driver is selected. The only API-visible changes are that
EnumArray[T]is an alias ofArray[T], and thattypes/pgtypesnow importsgithub.com/jackc/pgx/v5/pgtype.Tests
types/pgtypesuse pgx itself to build binary fixtures and cover text and binary decoding for every element kind the generator emits, NULL vs{}, error cases, concurrency, and the exactnull.Val/sql.Nullshapesfrom Nullable array columns (text[], jsonb[]) fail to scan with the native pgx driver #739.
NULL. It runs for all three drivers.
(see below).
Known gaps, left for follow-ups
intervalandtsvectorare mapped tostring, which pgx cannot fill from its binary format. Documented, with a pointer to type replacements.parser/args_cols.gopasses the underscore-prefixed type name (_int4) to the array translation. Pre-existing and unrelated to scanning.pgtypes.Array[T]rather than a plain[]T; a plain slice insidenull.Valcannot be scanned for the reasons above. Documented.Commits
The history reflects the review iteration; the last commit
b5b10d5is the final state. Happy to squash or restructure on request.🤖 Generated with Claude Code