Skip to content

[Suggestion]: expose the remaining walker scope knobs (skipHidden, maxDepth, include) on InitOptions #992

Closed

Description

@gaojunran

Which frontend(s)?

Core or Rust crate

What problem are you trying to solve?

extraIgnore / disableGit / disableIgnoreFiles (#988) cover what gets excluded. The remaining indexing knobs are still decided by the engine, and three of them are already available in zlob (=1.8.0, which fff now pins) with no way for an embedder to reach them.

1. Hidden files are gated on "is this a git repo" — crates/fff-core/src/walk/zlob.rs:35-37:

let use_non_git_defaults = !is_git_repo && !ignore_options.disable_git;
if use_non_git_defaults {
    flags |= WalkFlags::SKIP_HIDDEN;
}

and the same rule in the ripgrep backend (walk/ripgrep.rs:26, .hidden(!is_git_repo)). So the same directory is indexed differently depending on whether a .git happens to exist: in a git repo every dot directory is indexed, outside one none are. An embedder cannot pick either behaviour.

Measured on a real 11,332-file repo (@ff-labs/fff-node@0.11.1-nightly.b3b8ced, darwin-arm64): 602 paths (5.3%) contain a dot component — .slim/ 318, .agents/ 61, .gitignore 49, .eslintrc.js 32, .prettierrc.js 18, .github/ 11. Which of those belong in a search index is a product decision (a VS Code extension wants .gitignore and .github/ out; an agent tool may well want .agents/ in), and right now it is made for us by whether the folder is a repo.

Related: #972 is the same split showing up between the scan and the watcher on non-git roots.

2. Depth limit — zlob::walk::WalkBuilder::max_depth(Option<usize>) (rust/src/walk.rs:226) is never called. Useful for very large mono-repos where only the top levels are worth indexing.

3. An inclusion glob — zlob::walk::WalkBuilder::include() + include_flags() (rust/src/walk.rs:172-193) restrict the walk to paths matching a pattern, and directories outside the pattern's literal prefix are pruned from traversal, not merely filtered. fff already has an internal scope: Option<&Path> (walk/zlob.rs:25, used by watcher/background_watcher.rs:841-855 to re-walk a subtree while keeping root-relative rules), but nothing embedder-facing.

This is not the same as #719/#80 (scan_inclusions, re-including ignored paths — negation) or #736 (picker-side directory search): here it is purely "index only this part of the tree".

Proposed solution

Extend FilePickerIgnoreOptions / the SDK's InitOptions with the three fields, defaulting to today's behaviour:

FileFinder.create({
  basePath: process.cwd(),
  extraIgnore: ['/generated/'],
  skipHidden: true,     // undefined = current rule (!is_git_repo only)
  maxDepth: 4,          // undefined = unlimited
  include: 'src/',      // undefined = whole tree
});

Why an embedder cares beyond taste: on Linux the watch set equals the index set (background_watcher.rs takes one NonRecursive watch per indexed directory, see the numbers in #530), so anything the walk does not prune is paid for twice — memory and fs.inotify.max_user_watches. include prunes traversal, so it buys the same thing as extraIgnore for the "only my code" case.

One caveat worth calling out: the ripgrep backend currently ignores FilePickerIgnoreOptions entirely (walk/ripgrep.rs:20 is _ignore_options), while zlob honours it. Keeping parity would need .hidden(true) and an OverrideBuilder scope there, but zlob is what release/npm builds ship.

Context: I maintain a VS Code extension on top of @ff-labs/fff-node (gaojunran/vscode-fff-search) that maps search.exclude/files.exclude into extraIgnore; these three would let it match VS Code's own indexing policy instead of approximating it. Happy to test a branch.

Activity

  1. gustav-fff commented on Nov 5, 2026

    @gustav-fff
    Collaborator

    [triage-bot] DESIGN NEEDED:

    Feature request. Code on b3b8ced confirms it: hidden gating is !is_git_repo && !disable_git (crates/fff-core/src/walk/zlob.rs:27,35-37, walk/ripgrep.rs:26), zlob max_depth()/include() are never called, ripgrep backend drops _ignore_options (walk/ripgrep.rs:20). Not duplicate: #530 (done in #988) is excludes, #719 is negation.

    No PR from me. Walker part is three lines in walk/zlob.rs. The rest is not:

    • Watcher parity. IgnoreFilter (watcher/background_watcher.rs:980) consults only zlob IgnoreRules, which carry no include/depth. Without new checks, files created outside include or below maxDepth get indexed live and disappear on restart, same drift as [Bug]: on a non-git base path, the watcher indexes new files in hidden folders that the scan skips (0.11.0) #972. Linux watch registration (:519) must also skip them, otherwise include saves nothing on inotify.
    • index_new_directory walks from the new subdir unless extra_ignore is set (background_watcher.rs:841). With maxDepth it must always walk from root with scope, else depth counts from wrong base.
    • ABI: FffCreateOptions v4 (crates/fff-c/src/ffi_types.rs:16,58-65, read_versioned needs v3 prefix arm), fff.h, hand-written layouts in packages/fff-node/src/ffi.ts:93, packages/fff-bun/src/ffi.ts:421-585, packages/shared/fff-api.ts:47. feat(sdk): configure ignore rules during indexing #988 was +672/-173 over 22 files for three fields. This is same plumbing plus new watcher logic.

    Open questions for @dmtrKovalenko:

    1. skipHidden: false on non-git root also drops the $HOME guard (walk/ripgrep.rs:25). OK when set explicitly, or gate on enableHomeDirScanning?
    2. include: one pattern (zlob include() is single-slot) or list? Root-relative?
    3. Ripgrep backend: parity, or "requires zlob" like extraIgnore?
    4. Must go after fix(watcher): match the scan’s hidden-file rules outside Git worktrees #973, both change hidden handling in IgnoreFilter.

    Honk-Honk 🪿

  2. dkovalenko-oai commented on Nov 10, 2026

    @dkovalenko-oai
    Collaborator

    I think we already have skip hidden, a custom include/exclude settings and I feel unsure about maxDepth as I think this setting is very harmful and potentially breaking user workflows. what is your use case?

    i'll cose the issue in the meantime but feel free to express more interest we can add it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions