Universal Page-to-Data Extractor
Convert any URL or HTML into clean Markdown, JSON, or Text for RAG pipelines, datasets, and AI training.
Features · Install · Developers · API · Architecture · Roadmap · Trust · Changelog
| Workspace | Image resolution |
 |
 |
- High-precision extraction — Rust readability pipeline with html2md output
- JavaScript rendering — headless Chrome/Edge (Windows) or Chrome/Chromium (macOS/Linux) for SPAs
- Full-size image resolution — scroll + network capture, lightbox clicks, optional deep gallery crawl
- Extraction presets — Fast / Standard / Deep Gallery one-click profiles
- Live progress & history — SSE stage updates, local re-run history, markdown rendered preview
- Per-image & bulk download — SSRF-safe proxy; parallel bulk save from the Images tab
- Flexible exports — Markdown, JSON, Text, PDF, TXT, MD file downloads
- Native desktop app — Tauri shell with embedded API (~23 MB on Windows, no Python)
- Cross-platform — Windows, macOS (Apple Silicon), and Linux installers from CI
- Self-hosted — API on
127.0.0.1:8000, no vendor lock-in
Recommended: download pre-built installers from GitHub Releases. No Rust, Node, or terminal setup required — run the app and the embedded API starts automatically.
| Platform | Artifacts | Notes |
|---|---|---|
| Windows | Portable omniparse.exe · NSIS setup · MSI |
WebView2 required (Windows 10/11 usually has it). Chrome or Edge for JS rendering. |
| macOS | .dmg (Apple Silicon) |
Built on macos-latest CI. Install Chrome for JS rendering / image resolution. |
| Linux | .deb · .AppImage |
x86_64, built on Ubuntu 22.04 CI. Install Chrome or Chromium for JS rendering. |
After .\scripts\build-desktop.ps1:
| Artifact | Path |
|---|---|
| Portable exe | target\release\omniparse.exe |
| NSIS setup | target\release\bundle\nsis\OmniParse_*_x64-setup.exe |
| MSI | target\release\bundle\msi\OmniParse_*_x64_en-US.msi |
There is no separate portable zip — the release omniparse.exe is the portable build.
./scripts/build-desktop.sh
# Binary: target/release/omniparse
# Bundles: target/release/bundle/ (.dmg · .deb · .AppImage)CI builds for macOS and Linux run on every v* tag and via the Build Desktop workflow (workflow_dispatch).
OmniParse blocks URLs that resolve to private networks by default (SSRF protection). To extract intentional LAN pages, enable Allow Private Network URLs in the Arrangements panel or set ALLOW_PRIVATE_NETWORK_URLS=true in:
- Windows:
%LOCALAPPDATA%\OmniParse\.env - macOS:
~/Library/Application Support/omniparse/.env - Linux:
~/.config/omniparse/.env
Use this section only if you are contributing or running from source — end users should use Install above.
Prerequisites: Rust · Node.js 20+ · Chrome, Edge, or Chromium (for JS rendering)
cd frontend
npm install
npm run tauri:dev# Terminal 1 — API
cargo run --bin omniparse-server
# Terminal 2 — UI
cd frontend && npm install && npm run dev
# http://localhost:3000 (API on :8000)On Windows, start.bat opens the same two-process dev setup.
Set NEXT_PUBLIC_API_URL=http://localhost:8000 in frontend/.env.local if the API runs elsewhere.
| OS | Command |
|---|---|
| Windows | .\scripts\build-desktop.ps1 |
| macOS / Linux | ./scripts/build-desktop.sh |
| Any | cd frontend && npm run tauri:build |
omni-parse/
├── .github/
│ └── workflows/
│ ├── ci.yml # Rust + frontend lint/build on PR
│ └── build-desktop.yml # macOS + Linux release artifacts (tags)
├── .cargo/
│ └── config.toml # Shared target/ dir for workspace
├── crates/
│ └── omniparse-core/ # Axum API, extraction, browser pool, SSE
├── frontend/
│ ├── src/ # Next.js UI (workspace, history, presets)
│ └── src-tauri/ # Tauri 2 desktop shell
├── scripts/
│ ├── build-desktop.ps1 # Windows release build
│ ├── build-desktop.sh # macOS / Linux release build
│ ├── generate-sha256.ps1 # Checksum manifests (Windows)
│ ├── generate-sha256.sh # Checksum manifests (Unix)
│ └── clean.ps1
├── docs/
│ ├── architecture/ # Module & data-flow docs
│ ├── assets/ # Logo, screenshots, demo media
│ ├── vision.md # Product roadmap (v1.6 → v2.0)
│ ├── TRUST.md # Verification & VirusTotal notes
│ └── index.html # GitHub Pages landing
├── SHA256.txt # Source-tree checksums
├── SHA256-release-v*.txt # Release binary checksums (after build)
├── Cargo.toml # Rust workspace (v1.6.0)
├── CHANGELOG.md
└── start.bat # Windows dev launcher (contributors)
See docs/architecture/codebase-map.md for module-level detail.
| Method | Path | Description |
|---|---|---|
POST |
/extract |
Extract title, markdown, metadata, and images |
POST |
/extract/stream |
Same as /extract with SSE progress events |
POST |
/convert |
Convert text/markdown to PDF, TXT, or MD download |
GET |
/images/download |
Download a public image URL (SSRF-safe proxy) |
GET |
/settings |
Read server arrangement values |
PUT |
/settings |
Update arrangements (persisted to .env) |
GET |
/health |
Health check |
| Field | Default | Description |
|---|---|---|
render_js |
false |
Render with headless browser before extraction |
extract_images |
true |
Include image URLs in the response |
resolve_fullsize_images |
false |
Fast browser pass: scroll + network/lazy URLs |
resolve_deep |
false |
Slow crawl: gallery items + lightboxes |
curl -X POST http://localhost:8000/extract \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/article", "render_js": true}'| Resource | Purpose |
|---|---|
| SHA256.txt | Checksums for scripts, workflows, and workspace manifests |
SHA256-release-v1.6.0.txt |
Checksums for release binaries (after tauri:build or CI) |
| docs/TRUST.md | Verification steps, false-positive notes, VirusTotal reports |
Regenerate checksums:
powershell -File scripts\generate-sha256.ps1
powershell -File scripts\generate-sha256.ps1 -Release # after desktop build./scripts/generate-sha256.sh
./scripts/generate-sha256.sh --releaseCommunity scans for release builds. Detection counts change as vendors update signatures — open each report for the current vendor list.
| Platform | Artifact | Detections | Report |
|---|---|---|---|
| Windows | MSI installer | 0 / 62 | VirusTotal |
| Windows | Portable omniparse.exe |
2 / 72 | VirusTotal |
| Windows | NSIS setup | 3 / 70 | VirusTotal |
| macOS | .dmg (Apple Silicon) |
1 / 67 | VirusTotal |
| Linux | .deb · .AppImage |
2 / 70 | VirusTotal |
| Artifact | Detections | Report |
|---|---|---|
Portable omniparse.exe |
1 / 71 | VirusTotal |
| MSI installer | see report | VirusTotal |
| NSIS setup | 3 / 71 | VirusTotal |
A few heuristic flags on unsigned desktop tools are common (ML scores, installer packers). Verify SHA-256, build from source, or use the portable binary if you prefer. Details: docs/TRUST.md.
| Layer | Technologies |
|---|---|
| API | Rust, Axum, reqwest, chromiumoxide, readability, printpdf |
| Desktop | Tauri 2 |
| Frontend | Next.js 15, TypeScript, Tailwind CSS, shadcn/ui |
| CI | GitHub Actions — ci.yml + build-desktop.yml (macOS, Linux) |
Contributions are welcome. See CONTRIBUTING.md and the product roadmap.
MIT © OmniParse Contributors