Client-side media: generate animated image sub-sizes from the first frame only, matching core

[adamsilverstein]

Fixes #80266

What?

Two changes to the client-side media pipeline for animated GIFs:

1. @wordpress/vips: generate sub-sizes of animated images from the first frame only, matching WordPress core's server-side behavior. Previously, every uncropped sub-size loaded all frames ([n=-1]) and re-encoded a complete animated GIF.
2. @wordpress/video-conversion: use a 10-second key frame interval for GIF→video output (mediabunny defaults to 2 seconds).

Why?

Uploading a real-world screen-capture GIF (1517×855, 769 frames, 2.2 MB) took 91.5 seconds in local wp-env, with the upload spinner blocking the whole time. Instrumenting every worker RPC call showed the GIF→video conversion itself takes only ~3.6s; the three uncropped animated sub-size re-encodes took ~88 seconds (medium 16.5s, medium_large 27.6s, large 44s), serialized on the single vips worker. Profiling shows ~85% of each resize is gifsave palette re-quantization, not decoding or resizing.

The animated sub-sizes were also larger than the original file (the 300×169 medium was 5.5 MB from a 2.2 MB source), because downscaling destroys the GIF's inter-frame optimization.

WordPress core's server-side path creates static first-frame sub-sizes for animated GIFs: GD flattens inherently, WP_Image_Editor_Imagick::load() explicitly selects frame 1, and wp_calculate_image_srcset() prevents flattened sub-sizes and the animated full-size image from mixing at render time. That render-time guard keys off the GIF mime type, so it applies to client-generated static sub-sizes automatically. Uploads through the wp-admin Media Library still take the server path even with client-side processing active, so before this change the same GIF produced structurally different attachments depending on the upload surface. Cropped sizes (thumbnail) already skipped [n=-1] and flattened, so client-side behavior was internally inconsistent too.

Animation is preserved where it matters: the full-size original GIF is untouched, and the GIF→video flow attaches an animated_video companion.

The key frame change: fewer key frames roughly halve the output size for long conversions (2.2 MB → 1.14 MB for the same GIF) at identical encode time and quality. These looping, autoplaying GIF replacements don't need fine seek granularity.

Measured results

Same GIF, same environment (wp-env, headless Chromium via Playwright, Apple Silicon):

Metric	Before	After
Upload wall time (queue drain)	91.5s	6.9s (13.3×)
All 4 sub-size resizes	~88s	0.17s
medium sub-size file	5.5 MB (animated)	6 KB (static)
Companion video size	2.22 MB	1.29 MB

Alternatives measured and rejected: tuning gifsave (effort: 2, interframe_maxerror: 8, interpalette_maxerror: 16) is 4-8× faster and fixes the size bloat but still costs ~35-50s for three sizes and diverges from core; skipping sub-sizes entirely is equally fast but makes the media library grid fall back to the full-size animated GIF.

How?

• resizeImage() in packages/vips no longer passes [n=-1] load options, so thumbnailBuffer operates on the first frame, as it already did for cropped sizes. The AVIF high-bit-depth detection no longer needs the animation guard.
• convertGifToVideo() in packages/video-conversion passes keyFrameInterval: 10 to mediabunny's VideoSampleSource.
• New unit tests pin both behaviors, and a new e2e test uploads the animated fixture and asserts (via ImageDecoder) that the full-size GIF stays animated while the medium sub-size has exactly one frame. The e2e test fails against trunk behavior (12 frames) and passes with this change.

Testing Instructions

1. Enable client-side media processing (requires Chromium 137+).
2. Upload a long animated GIF (hundreds of frames) into an Image block.
3. Observe the upload completes in seconds instead of minutes.
4. Verify the attachment: full-size GIF still animated, sub-sizes are static single frames, animated_video companion (MP4) and poster present.
5. npm run test:unit -- packages/vips packages/video-conversion
6. npm run test:e2e -- gif-to-video.spec

cc @swissspidy - media-experiments chose the all-frames path deliberately, so flagging the behavior change: uncropped animated sub-sizes become static stills, matching core's server output.

[github-actions]

Size Change: +117 B (0%)

Total Size: 7.72 MB

📦 View Changed

Filename	Size	Change
build/modules/video-conversion/worker.min.js	83.1 kB	+159 B (+0.19%)
build/modules/vips/worker.min.js	3.69 MB	-42 B (0%)

_{compressed-size-action}

[github-actions]

Flaky tests detected in 86397ba.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/29557543818
📝 Reported issues:

• [Flaky Test] Should navigate inner blocks with arrow keys #39607 in /test/e2e/specs/editor/various/writing-flow.spec.js

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: andrewserong <andrewserong@git.wordpress.org>
Co-authored-by: swissspidy <swissspidy@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[swissspidy]

Worth linking to https://core.trac.wordpress.org/ticket/28474 somewhere (and vice-versa).

Considering that we're ideally converting GIFs to videos anyway, this alignment change sounds reasonable to me. I'd prefer multiple video sub-sizes over animated thumbnails for performance reasons. Maybe that could be something for 7.2, or a plugin.

[adamsilverstein]

Worth linking to https://core.trac.wordpress.org/ticket/28474 somewhere (and vice-versa).

Considering that we're ideally converting GIFs to videos anyway, this alignment change sounds reasonable to me. I'd prefer multiple video sub-sizes over animated thumbnails for performance reasons. Maybe that could be something for 7.2, or a plugin.

I added a comment to that old ticket, thanks for linking to it. Since we already have the capability to generate sub-sized gifs and videos, we should probably enable that somehow if only via a developer controlled hook.

[adamsilverstein]

Additional data supporting this PR, from debugging the problem GIF reported on #80260 (2554×1676 × 800 frames, 20 MB): each animated sub-size costs 80-90 s in wasm-vips regardless of output size (150px: 84 s, 300px: 91 s; the ~80 s floor is decoding all 800 source frames), so the missing-sizes chain runs for tens of minutes and in one test crashed the tab. A first-frame-only sub-size takes ~2 s at 1024px. Full investigation in #80379.

[adamsilverstein]

@andrewserong - this PR should fix the issue you were seeing with converting large GIFs to Video.

[github-actions]

Size Change: +123 B (0%)

Total Size: 7.73 MB

📦 View Changed

Filename	Size	Change
build/modules/video-conversion/worker.min.js	83.5 kB	+160 B (+0.19%)
build/modules/vips/worker.min.js	3.69 MB	-37 B (0%)

_{compressed-size-action}

[adamsilverstein]

Worth linking to https://core.trac.wordpress.org/ticket/28474 somewhere (and vice-versa).

Considering that we're ideally converting GIFs to videos anyway, this alignment change sounds reasonable to me. I'd prefer multiple video sub-sizes over animated thumbnails for performance reasons. Maybe that could be something for 7.2, or a plugin.

@swissspidy -

I realized we could fix that Trac ticket by adding a filter to enable the full GIF output pipeline which seems like a nice win for a capability we have already.

I opened:

• Client-side media processing: add an opt-in filter to generate animated GIF sub-sizes #80383

[andrewserong]

Glad this was a simple fix in the end, very cool! Testing well for me:

✅ GIFs smaller than the big size threshold are available to run in their animated form by selecting the Full image size (matching previous core / server-side approaches)
✅ For GIFs larger than the big size threshold, their scaled version is a static GIF (again as in previous core behaviour), but the user can switch to the Video variation if they'd like to retain the animated form of the GIF
✅ With this PR the massive GIFs that previous took forever now seem to take < 30s for me on my MacBook Air, so this is feeling really good
✅ All of my test GIFs are uploading successfully now

LGTM, great work on this!

[andrewserong]

Tiny nit: because we no longer have the code that handled this, we could move this comment to the JSDoc before the function? (Not a big deal to leave it here of course)

[adamsilverstein]

Good idea, moved to the JSDoc in 1a96713.

[github-actions]

I just cherry-picked this PR to the wp/7.1 branch to get it included in the next release: 244a546

[adamsilverstein]

@t-hamano I see you added Backport to Gutenberg RC - should I be doing that with any PRs I want to make into 7.1?

[t-hamano]

I see you added Backport to Gutenberg RC - should I be doing that with any PRs I want to make into 7.1?

@adamsilverstein, Since WordPress 7.1 and Gutenberg 23.6 are expected to have essentially the same functionality, I aim to backport it to both Gutenberg 23.6 and WordPress 7.1 as much as possible.

You don't need to worry about the Backport to Gutenberg RC label specifically. I will decide which PRs to backport to GB RC and add the label as needed.

[adamsilverstein]

You don't need to worry about the Backport to Gutenberg RC label specifically. I will decide which PRs to backport to GB RC and add the label as needed.

Thanks for clarifying, I'll stop adding that label and let you manage it instead.

[t-hamano]

I just cherry-picked this PR to the release/23.6 branch to get it included in the next release: 851899b