Follow-up work unlocked once HarperFast/harper#2152 lands and reaches instances. Blocked until then.
Background
#1591 was a browser crash downloading a large application. #1617 shipped the Studio-side guardrail — the modal states the package size up front and cautions when it's large or unmeasurable — but deliberately did not change the transport, because verifying that needs a Harper build carrying #2152.
harper#2152 (closes HarperFast/harper#2150) adds two things to package_component:
stream: true — returns the tar.gz as raw bytes with content-type: application/gzip and a content-disposition filename, instead of base64 inside JSON. Constant memory, no size ceiling.estimate: true — returns { project, total_size, dangling_symlinks } from a single directory walk, packaging nothing.
The base64 shape stays the default, so nothing breaks on its own.
1. Consume the streamed archive
The reason this is worth doing: today's path makes ~5 full-size copies of the archive in the renderer (response text → JSON.parse → atob → byte array → Blob), and both strings hit V8's 512 MiB cap. Streaming removes all of it.
Unknown to settle first: whether the download currently goes direct or through the Fabric Connect proxy. The 5 MB app used to verify #1617 downloaded fine, but its base64 payload was probably under the 2 MB cap, so that run doesn't distinguish the two. This decides how prominent the proxy handling needs to be.
2. Use estimate for the include-node_modules case — ⚠️ needs a version gate
get_components omits node_modules, so with "Include Node Modules" ticked Studio can't size the package and currently just cautions unconditionally. estimate would give it a real number.
It cannot be called blindly. Harper's operation validator runs with allowUnknown: true and the pre-#2152 handler ignores fields it doesn't recognise — so on an older instance package_component { project, estimate: true } doesn't return an estimate, it packages the entire application and base64-encodes it. Asking "how big is this?" would trigger the exact expensive operation the estimate exists to avoid, on precisely the applications where that hurts most.
Note the asymmetry, because it affects sequencing: stream: true degrades harmlessly on an old instance (unknown key ignored → JSON response → the content-type branch handles it), while estimate: true degrades badly. Item 1 can ship on content-type sniffing alone; item 2 needs a real capability or version check.
3. Recalibrate the caution once streaming is live
4. Minor: the download filename
DownloadApplicationModal names the file ${project}.gz, but the archive is a tar inside a gzip. The stream path fixes itself — Harper sets filename="${project}.tar.gz" via content-disposition — so only the legacy base64 path would still be wrong.
Not planned
Multi-file download (one request per file, reassembled client-side) was considered and rejected in HarperFast/harper#2150: the browser can't write a directory tree without showDirectoryPicker(), a real application is thousands of extra round trips through get_component_file's own base64-in-JSON, it discards symlink/mode/empty-dir fidelity, and reassembling into one artifact means zipping in the browser — pushing every byte back through JS memory, which is the thing being fixed.
Follow-up work unlocked once HarperFast/harper#2152 lands and reaches instances. Blocked until then.
Background
#1591 was a browser crash downloading a large application. #1617 shipped the Studio-side guardrail — the modal states the package size up front and cautions when it's large or unmeasurable — but deliberately did not change the transport, because verifying that needs a Harper build carrying #2152.
harper#2152 (closes HarperFast/harper#2150) adds two things to
package_component:stream: true— returns the tar.gz as raw bytes withcontent-type: application/gzipand acontent-dispositionfilename, instead of base64 inside JSON. Constant memory, no size ceiling.estimate: true— returns{ project, total_size, dangling_symlinks }from a single directory walk, packaging nothing.The base64 shape stays the default, so nothing breaks on its own.
1. Consume the streamed archive
The reason this is worth doing: today's path makes ~5 full-size copies of the archive in the renderer (response text →
JSON.parse→atob→ byte array → Blob), and both strings hit V8's 512 MiB cap. Streaming removes all of it.resolveInstanceConnectionalready resolves URL + auth +modefor exactly this purpose (deployComponentStreamuses it).stream: trueand branch on the responsecontent-type—application/gzip→ stream path, JSON → the existing base64 path. One code path covers old and new instances with no version check needed.showSaveFilePicker()+response.body.pipeTo(writable)where available (Chromium), falling back toawait response.blob()elsewhere. The fallback is still a large win — Chrome backs big Blobs with disk, so it sidesteps the string cap,atob, and the per-byte decode loop even without File System Access.mode: 'proxy'explicitly. Fabric Connect is a message-passing proxy with a 2 MB body cap ("Fabric Connect body size must be less than 2MB. Utilize Direct Connect for larger payloads.") and cannot stream at all, so proxy-mode instances need a clear message rather than a hang or a truncated file.Unknown to settle first: whether the download currently goes direct or through the Fabric Connect proxy. The 5 MB app used to verify #1617 downloaded fine, but its base64 payload was probably under the 2 MB cap, so that run doesn't distinguish the two. This decides how prominent the proxy handling needs to be.
2. Use⚠️ needs a version gate
estimatefor the include-node_modules case —get_componentsomitsnode_modules, so with "Include Node Modules" ticked Studio can't size the package and currently just cautions unconditionally.estimatewould give it a real number.It cannot be called blindly. Harper's operation validator runs with
allowUnknown: trueand the pre-#2152 handler ignores fields it doesn't recognise — so on an older instancepackage_component { project, estimate: true }doesn't return an estimate, it packages the entire application and base64-encodes it. Asking "how big is this?" would trigger the exact expensive operation the estimate exists to avoid, on precisely the applications where that hurts most.Note the asymmetry, because it affects sequencing:
stream: truedegrades harmlessly on an old instance (unknown key ignored → JSON response → the content-type branch handles it), whileestimate: truedegrades badly. Item 1 can ship on content-type sniffing alone; item 2 needs a real capability or version check.3. Recalibrate the caution once streaming is live
showSaveFilePickerpath and is much weaker on the Blob path. Both the copy andLARGE_PACKAGE_BYTESshould become conditional on the transport actually chosen — otherwise we're warning about a problem we fixed.unmeasuredcaution stays useful regardless: it covers instances that report no file sizes, which are also the instances least likely to have the streamed download.4. Minor: the download filename
DownloadApplicationModalnames the file${project}.gz, but the archive is a tar inside a gzip. The stream path fixes itself — Harper setsfilename="${project}.tar.gz"viacontent-disposition— so only the legacy base64 path would still be wrong.Not planned
Multi-file download (one request per file, reassembled client-side) was considered and rejected in HarperFast/harper#2150: the browser can't write a directory tree without
showDirectoryPicker(), a real application is thousands of extra round trips throughget_component_file's own base64-in-JSON, it discards symlink/mode/empty-dir fidelity, and reassembling into one artifact means zipping in the browser — pushing every byte back through JS memory, which is the thing being fixed.