Skip to content

perf: bypass redundant NetworkX wrapper checks to optimize graph building loops (#798)#802

Open
suhaniiz wants to merge 1 commit into
param20h:devfrom
suhaniiz:fix/graph-lookup-efficiency-798
Open

perf: bypass redundant NetworkX wrapper checks to optimize graph building loops (#798)#802
suhaniiz wants to merge 1 commit into
param20h:devfrom
suhaniiz:fix/graph-lookup-efficiency-798

Conversation

@suhaniiz

@suhaniiz suhaniiz commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

📋 PR Checklist

Thank you for contributing to PDF-Assistant-RAG! 🎉
Please fill out this template before submitting. PRs without it filled in will be closed.


🔗 Related Issue

Closes #798


📝 What does this PR do?

Optimizes performance inside the entity co-occurrence mapping loop within build_graph by removing redundant hash lookups.

Previously, the code invoked explicit wrapper methods (graph.has_node(...) and graph.has_edge(...)) sequentially before modifying attributes. This pattern forced NetworkX to execute multiple duplicate lookups across internal dictionary states on every element traversal.

This PR optimizes lookups by:

  1. Migrating node tracking to check direct dictionary presence (entity.id in graph.nodes).
  2. Streamlining edge discovery inside the tight $O(N^2)$ loop by evaluating entity adjacency directly within the pre-fetched source node mapping (right_id in graph[left_id]).

This significantly lowers lookup overhead when constructing high-density knowledge graphs from heavy multi-page payloads.


🗂️ Type of Change

  • 🐛 Bug fix
  • ✨ New feature
  • 🔧 Refactor / code cleanup
  • 📝 Documentation update
  • 🎨 UI / styling change
  • ⚙️ CI / tooling / config change
  • 🧪 Tests

🧪 How was this tested?

  • Tested the affected API endpoints manually
  • Profiled data ingestion loops locally to ensure nodes and co-occurrence edges continue to map tracking metrics (mentions, weight, chunks, pages) accurately.

📸 Screenshots (if UI change)

N/A (Performance optimization)


⚠️ Anything to flag for reviewers?

No breaking architectural changes. The approach leans on standard, idiomatic Python dictionary lookup operations, remaining completely compatible with NetworkX's core internal graph dictionary structures.


✅ Self-Review Checklist

  • My branch is based on dev, not main
  • I have not added any secrets / API keys
  • I have not modified main branch or any HuggingFace deployment config
  • My code follows the existing style (no unnecessary formatting changes)
  • I have updated relevant docs / comments if needed

@suhaniiz suhaniiz requested a review from param20h as a code owner July 6, 2026 19:05
@vercel

vercel Bot commented Jul 6, 2026

Copy link
Copy Markdown

@suhaniiz is attempting to deploy a commit to the param20h's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] : Inefficient Graph Element Verification Overhead

1 participant