Repository navigation
feat(products): pick tags from a curated corpus in the product form - #647
Merged
Merged
Conversation
The product form, in create and edit mode, gets a Tags field: a text input that suggests tags from a new sc-{stage}-tags table (keyed by tag_id, the tag itself) and shows chosen tags as removable chips. createProduct and updateProduct save the submitted tags to metadata.tags and reject any outside the corpus, except tags the product already carries, so a tag retired from the corpus never blocks an edit to some other field.
The app only reads the corpus, so the Vercel role gets read access to the table and nothing more. scripts/seed-tags.ts seeds it from the tags products already have, and init-local creates and seeds it locally.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Claude finished @alukach's task in 22s —— View job ✅ No blocking issues — safe to merge. I read the operations, action,
Non-blocking notes:
Simplify (ponytail)
Docs
💰 Estimated review cost: $0.63 · 0m21s · 8 turns |
The browser's datalist dropdown wasn't styled by the theme and hid the corpus until you typed. Chosen tags now show as removable chips beside an "Edit tags" button, which opens a popover listing every corpus tag as a checkbox, with a box to filter them. Submission and server-side validation are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alukach
added a commit
to source-cooperative/docs.source.coop
that referenced
this pull request
Oct 8, 2026
The tag field now opens a list to tick tags from rather than suggesting them as you type (source-cooperative/source.coop#647). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
parseTags now returns either the tags or the form error naming those outside the corpus, so the one-use unknownTagsState wrapper goes away. The popover's tag list gets a note on its ceiling: it renders every tag, which is fine to a few hundred and wants virtualising past that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/clean-tags.ts replaces seed-tags.ts. Instead of copying every free-text tag into the corpus, it lowercases them, strips `#`, splits tags that hold several, maps synonyms, typos and sub-national places to one canonical tag, and drops noise. A canonical tag joins the corpus when two or more products carry it, or when it is on a keep-list (countries and a few broad topics). Each product then keeps only its cleaned tags that made the corpus. It previews by default and writes only with APPLY=1. Given the `uniq -c` counts of production's tags (--counts), it previews from those instead of DynamoDB, and TABLE=1 adds a before/after table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving a product without tags no longer reads the tags table, so a create or edit that sets no tags doesn't depend on that table existing. The picker's toggle also updates from the latest tags rather than the ones it rendered with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alukach
marked this pull request as ready for review
October 9, 2026 00:27
Contributor
Author
|
@tylere please review the tag corpus table in this PR description and the tags that we will be dropping, I'd be curious to hear your thoughts |
Contributor
|
While I am fully on board with (e.g. tag cleanup), I think there are aspects that would benefit from future discussion.
|
# Conflicts: # src/lib/actions/products.test.ts # src/lib/actions/products.ts
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
tylere
approved these changes
Oct 20, 2026
tylere
left a comment
tylere
left a comment
Contributor
There was a problem hiding this comment.
Lots of minor suggestions on aliasing, but approving overall.
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
Co-authored-by: Tyler Erickson <92862+tylere@users.noreply.github.com>
tylere
approved these changes
Oct 20, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Product owners can now set a product's tags from the product form, in both create and edit mode, choosing from a curated corpus kept in a new DynamoDB table.
What changes
sc-{stage}-tags(deploy/lib/database-construct.ts), partition keytag_id, which is the tag string itself (the same string stored inproduct.metadata.tags). Each row is just{ tag_id }. The Vercel role getsgrantReadDataonly, because the app never writes the corpus. Curation happens outside the app.TagsTable.listAll()(src/lib/clients/database/tags.ts) scans the table with pagination and returns the tags sorted.TagPicker(src/components/features/products/TagPicker.tsx) shows the chosen tags as removable chips beside an "Edit tags" button. The button opens a Radix ThemesPopoverthat lists every corpus tag as a checkbox, with a box to filter them, so people can browse the whole list without guessing what to type. Each chosen tag is submitted as its owntagsform value from a hidden input next to its chip. It is wired intoProductCreationFormas acustomfield under Description, and both pages pass the corpus in astagOptions.src/lib/operations/products.ts):createProductandupdateProductin the operations layer take an optionaltagsarray, dedupe it, and reject any tag outside the corpus with atagsfield error. A tag the product already carries is accepted even after it leaves the corpus. This follows the same legacy-drift rule as visibility: editing the title shouldn't fail on a tag nobody chose today. Because the API routes share these operations,POSTandPATCHon products accepttagstoo, documented in the generated reference. The form actions insrc/lib/actions/products.tsalways passformData.getAll("tags");Object.fromEntries(formData)alone would keep only the last tag.scripts/clean-tags.ts <stage>reads every product's free-text tags, cleans them, and builds the corpus from the result (see Tag cleanup below). It previews by default;APPLY=1rewrites each product'smetadata.tagsto its cleaned tags and puts the corpus intosc-{stage}-tags.init-localcreates the table and seeds it from the fixture products' tags.Rollout
AWS_REGION=us-west-2 npx tsx scripts/clean-tags.ts prodand review the preview: the corpus, the dropped tags, and each product whose tags change. Production is in us-west-2; the script defaults to us-east-1 like the others inscripts/. AdjustALIASESandKEEPin the script until the result is right.APPLY=1. This rewrites product tags, so take an on-demand backup ofsc-prod-productsfirst. Until then the corpus is empty, and the popover shows "No tags to choose from yet."Tag cleanup
Production's products carry 416 distinct free-text tags. Many are the same tag in another case or spelling (
PMTilesandpmtiles;ftwandfieldsoftheworld), several tags typed as one string (#overturemaps #roads #riyadh …), places below country level (14 Spanish regions, German state codes from the fiboa datasets), or noise (test,foo,--, an empty tag, andgeospatialandopen data, which are true of everything on Source).scripts/clean-tags.tslowercases each tag, strips#, splits combined tags, maps synonyms, typos and places to one canonical tag throughALIASES, and drops noise. A canonical tag joins the corpus when two or more products carry it, or when it is inKEEP(countries, pluselevation,machine learning,time series,forestsandhydrology). Each product keeps only its cleaned tags that are in the corpus.Run against the production counts (
TABLE=1 npx tsx scripts/clean-tags.ts --counts counts.txt): 416 raw tags → 196 cleaned → 136 in corpus. Counts are products per tag. They are built from per-tag counts, so a product that had two spellings of one tag counts twice and some totals run slightly high; the<stage>run counts per product exactly.The 136 tags in the corpus, by number of products
field boundariesgeoparquetpmtilesagriculturevectorfiboafields of the worldspainfieldscapessentinel-2segmentationweatherclimategermanyzarrbuilding footprintsenergyhydrologysigpaccrop typecropsland coverrasterunited statescogsatellite imageryoverture mapssaudi arabiabiodiversityforecastingparquetsentinel-1admin boundariesplacesprecipitationconservationelevationforestsindialabelsmachine learningcarboncloud nativeicechunkjapanland usesolarstactransportationvegetationembeddingsgloballandsatoceanopenstreetmapplanetscopesaraicarbon dioxide removalflatgeobuffoundation modelsgapfilledlidarmodisnatural gasnhdremote sensingtanzaniawildfireyouthmappersaerosolsatmospherebelgiumcanadacensuscerclaycloudecologyfisheriesimage classificationimagerynetherlandsradarreanalysisregressionrwandaafricaaustriabathymetrybrazilcomputer visionduckdbeurocropsexportsflood detectiongeopackagegeostationarygoogleh3hlsimportsinvekoslatviamarine debrismicrosoftmonitoringnaipnebnoaaobject detectionobservationssloveniaswitzerlandtilestime seriesunited kingdomusdawetlandswildlifeantarcticacambodiacroatiaczechiadenmarkestoniaeuropefinlandfranceirelandluxembourgnew zealandportugalslovakiaswedenvietnamDropped because only one product carries them (60)
american bollwormsattribution benchmarkbioregionalisationbrpcatalogcopernicuscorridorscsvdigitaltwindisaster riskdwree-mrioegmseiaenvironmentenvironmental dataepaequityesaexiobasefercfinancefoodsecurityfoodsystemsfudegdalgeocodinggeosgeozarrgoesgovernmenthexhistoricalhuman footprinticebergimputationjpssjusticelandscape elementsmeiji erametopnasaneonngffnstapink bollwormsplodinypngpolicypopulationprocurementproperty dataprotected areasrealtimesaharasnowsupplychainsynthetic dataterrestrialusgsKeep any of these by adding it to
KEEP, or merge it into a corpus tag inALIASES.Every tag that changes (295 rows; unchanged tags appear only in the corpus list)
ftwfields of the worldfieldsoftheworldfields of the worldgeospatialboundariesadmin boundariessatellitesatellite imageryovertureoverture mapslandsat 8landsatfoundational modelfoundation modelsnhdnhd,hydrologymlmachine learningimagerysatellite imageryusaunited statesukunited kingdomsentinel2sentinel-2opendatakarnatakaindiaforecastforecastingfieldboundariesfield boundariesfield boundaryfield boundariesfieldearthdemelevationcropcropscloud optimised geotiffcogclimate changeclimatecdrcarbon dioxide removalcaliforniaunited statesbuildingsbuilding footprintsbuilding footprintbuilding footprintsadminadmin boundariesSentinel2sentinel-2PMTilespmtilesJapanjapanCOGcogAIai#geoparquet #aois #places #dataset #riyadh #geospatial #overturemaps #saudiarabiageoparquet,places,saudi arabia,overture mapsyouthmasppersyouthmappersyearsxaimachine learningwheatcropswbdhydrologywaterwayshydrologywatershed boundarieshydrologywaterhydrologywadhwaniaivegetation covervegetationvalenciaspainusgsusunited statestropical stormweathertreesforeststopographyelevationtimeseriestime seriestiffrasterticksbiodiversitythgermanytestrepo12test unidata netcdftestterrestrialtemperatureweathertaxi zonestransportationtamsatprecipitationsynthetic dataswissswitzerlandsupplychainstreet level imageryimagerystaticstatestac-geoparquetstac,geoparquetsstoceanspecies distribution modelbiodiversitysouthern oceanoceansouth fork eel riverunited statessourcesolar mapssolarsnowslovenia fiboa field boundaries krms gerk geoparquet vector pmtilesslovenia,fiboa,field boundaries,geoparquet,vector,pmtilesslgermanysixpacsigpacshgermanysferself-supervisedmachine learningsaxsaudiarabiasaudi arabiasaharariyadhsaudi arabiaremotesensingremote sensingreanalysis datareanalysisrealtimerainfall radarprecipitation,radarpvsolarpublic-datapublicprotected areasproperty dataprocurementpopulationpolygonpolitical boundariesadmin boundariespolicypoisplacespoiplacespngpmtiles gtfspmtiles,transportationplodinyplanet fusionplanetscopepink bollwormsphotovoltaicsolarperspective imagesimagerypacificspatialoverturemapsoverture mapsosmopenstreetmapopen dataomeoil productionenergyobservationobservationsnycunited statesnwpforecasting,weathernstanrwgermanyngffnetherlands fiboa field boundaries aan geoparquet vector pmtilesnetherlands,fiboa,field boundaries,geoparquet,vector,pmtilesnet zerocarbonneonndsgermanynavarraspainnatural gas productionnatural gasnational hydrology datasetnhd,hydrologynasamvgermanymultitemporaltime seriesmrmsprecipitationmiscmetopmetadatamaizecropsmadridspainluxembourg fiboa field flik boundaries geoparquet vector pmtilesluxembourg,fiboa,field boundaries,geoparquet,vector,pmtileslulcland cover,land uselowcoderlive fuel moisturewildfirelidar 3dtiles magolidarlandscape elementslandcoverland coverland-use and land-cover changeland cover,land uselandlacunaksrsacindiakarnatakindiajusticejpssireland fiboa field boundarie geoparquet vector pmtilesireland,fiboa,field boundaries,geoparquet,vector,pmtilesinsolationsolarimputationimergprecipitationimageiceberghydrographyhydrologyhuman footprinthistoricalhexheightelevationhcathavasuunited stateshamburggermanygroundwater qualityhydrologygroundtruthedlabelsground-truthedlabelsground-truthlabelsgpmprecipitationgovernmentgoesgeozarrgeosgeocodinggeo spatialgdalgaliciaspainfudeforest carbonforests,carbonforestforestsforecastsforecastingfoodsystemsfoodsecurityfooflowlineshydrologyflandersbelgiumfishbasefisheriesfishfisheriesfirewildfirefinancefercfarmlandagricultureextremaduraspaineuhvdesaera5reanalysis,climateequityepaenvironmental dataenvironmentenergy use by fuelenergyenergy useenergyenergy transitionenergyenergy productionenergyenergy efficiencyenergyenergy demand by sectorenergyenergy demandenergyemissionscarbonembeddingembeddingselectricity generationenergyelectricityenergyeiaeatch monitoringmonitoringdwrdustaerosolsdroneimagerydisaster riskdigitaltwindeforestationforestsdeep learningmachine learningdatasetczechczechiacsvcroptypecrop typecountycountriescottoncropscorridorscontourelevationcoalenergycloud-nativecloud nativecdlcrop type,united statescataloniaspaincatalogcastillalamanchaspaincastile and leonspaincarbon emissionscarboncantabriaspaincanary islandsspainbusinessesplacesbuildingbuilding footprintsbrpbremengermanyboundary vector agriculturefield boundaries,vector,agriculturebloreindiabioregionalisationbiomassforestsbelgium wallonia fiboa field boundaries geoparquet vector pmtilesbelgium,fiboa,field boundaries,geoparquet,vector,pmtilesbeetlesbiodiversitybbgermanybasquespainbalearic islandsspainattribution benchmarkarcaragonspainaquaticsoceanaoisandalusiaspainandalucíaspainanalysisamxamerican bollwormsaerosolaerosolsTreesforestsTottorijapanTopographic MapelevationSentinel2 mosaics cloud-free global cog rastersentinel-2,global,cog,rasterPMTiles FlatGeobufpmtiles,flatgeobufMeiji eraForestforestsEXIOBASEEUeuropeEGMSEE-MRIODEMelevationCopernicusCloud-Nativecloud native3d tileslidar--#waterwayshydrology#vectorvector#tifraster#stacstac#sentinel #satellite #imagery #cog #aws #remotesensing #saudiarabia #tabaqatsentinel-2,satellite imagery,cog,remote sensing,saudi arabia#riyadh #satellite #rgb #cog #2024saudi arabia,satellite imagery,cog#rasterraster#pmtiles #cng #saudiarabia #sentinel #tabaqat #awspmtiles,cloud native,saudi arabia,sentinel-2#parquetparquet#overturemaps #roads #riyadh #osm #pmtiles #cng #tabaqatoverture maps,transportation,saudi arabia,openstreetmap,pmtiles,cloud native#overturemaps #roads #pmtiles #cng #osm #tabaqat #saudiarabiaoverture maps,transportation,pmtiles,cloud native,openstreetmap,saudi arabia#geospatial #ksa #riyadh #geoparquet #places #poissaudi arabia,geoparquet,places#cbam #climate reanalysisreanalysis,climateDecisions worth a second look
ALIASESandKEEPlists inscripts/clean-tags.tsencode it, and the tables above show the result. Calls most worth checking:boundariesandadminbecomeadmin boundaries;satelliteandimagerymerge intosatellite imagery; agencies on one product (nasa,usgs,esa,epa) are dropped while those on two or more (noaa,usda,cer,neb) stay.tagsout of an update keeps them; an empty list clears them. That suits API callers, who shouldn't wipe a product's tags by editing its title. The form always sends the list, because removing every chip submits notagsfield at all, and that has to clear them.<datalist>, but its dropdown ignored the theme and hid the list until you typed. Of the alternatives (every tag as a toggle chip, or a popover like GitHub's label picker), the popover keeps the field compact and uses only components Radix Themes already ships. The list renders every tag without virtualisation, which is fine to a few hundred tags; past that it should be virtualised (marked with aponytail:comment).Stories
On this branch's Storybook deploy:
The Filtering story, with the popover open and filtered to tags containing "c":
Docs
docs/using-source/create-a-data-product.mddescribed tags as "Comma-separated, up to 20 tags", which is now stale. Updated in docs.source.coop#39.docs/about-source/core-concepts.md("Tags and Discoverability") still holds: it says what tags are for, not how they are entered.public/llms.txtdoesn't mention tags, and this adds no public endpoint. Checked the data-proxy ADRs: tags have no bearing on authorization, credentials, or federation.Testing
npm run testforsrc/lib/actions/products.test.tsandsrc/components/features/products/TagPicker.test.tsxpasses locally (30 tests), and so doesnpm run type-check.npm run lintreports nothing in the changed files.TagPicker.test.tsxcovers removing a chip, and the popover filtering case-insensitively and toggling a tag on and off.🤖 Generated with Claude Code