How do people sample a search page, evaluate a result, return to something already seen, and decide where to go? This project studies that process through synchronized gaze, cursor, scroll and pupil recordings from AdSERP, with AllSERP supplying typed areas of interest (AOIs) for the page's results, ads and widgets.
Repository scope: attentional-foraging characterizes the search cognitive process and hosts the LAB analysis substrate. Click modeling is developed in approach-retreat, which validates against the claims recorded here.
Start with the two linked posters, or the observed pause below. The construct-to-observable map connects the foraging account to the measurements that can support or challenge it.
The cascade model of click position bias assumes that a searcher examines results once, from the top, and stops at the first worthwhile one (Craswell et al., WSDM 2008). Later click models relax that assumption with revisits (Xu et al., WSDM 2012), non-sequential examination (Wang et al., SIGIR 2015) and comparison with adjacent results (Zhang et al., WWW 2021). The recordings here show the loops directly: a result is fixated, left and fixated again, and the cursor can stay on one result while the eyes visit others.
This is one recorded interval from trial p021-b1-t6. A fixed rule chose it
from 63 cursor pauses of 2–8 s with at least three gaze AOIs, a gaze return
and at least 60% gaze-in-AOI coverage, so it shows the pattern rather than a
typical pause: 10.7% of the 1,843 cursor pauses of that length contain three or
more gaze AOIs and a return, and 66.5% contain any gaze on another AOI. Rodden et al. named the pattern
marking; its rate and what the parked result predicts
are measured separately. [LAB, AdSERP, typed]
- First visits often end with a move back. In the separate first-visit analysis, a move to an earlier position ends 50–57% of first visits at positions 2–10, and 75–92% of those moves land on a result already visited, depending on position. Positions include ads and widgets, and fixations are assigned to positions by vertical band. These are per-position shares, not shares of trials. [LAB, AdSERP, typed, NB38:K8] First-visit evidence, 2,606 trials
- Gaze changes AOIs more often than the cursor. During time covered by both signals, gaze makes 2.84× as many AOI changes (95% CI 2.43–3.36) and 3.38× as many backward steps (2.80–4.14), with one visit definition for both channels. The ratio depends on the minimum visit duration: on the full clock it is 2.97 at 100 ms and 1.76 at 200 ms. [LAB, AdSERP, typed; poster producer] Common-coverage evidence, 2,650 trials
- The eyes often move on while the cursor rests. During 23.8% (95% CI 22.2–25.2) of the time the cursor rests inside an AOI, a recorded fixation is on a different AOI; among rest time with a recorded fixation the share is 35.6% (33.3–37.7). The logger records the cursor only when it moves, and a position is held for at most 2 s, so stillness beyond 2 s falls outside this measure; holding each position until the next move gives 27.6% (25.8–29.1). [LAB, AdSERP, typed; poster producer] Definition and denominators
Revisiting is consistent with comparison and re-evaluation. A fixation alone cannot establish what was understood, remembered or preferred. “Not a cascade” describes the observed examination sequence; it does not assume that every searcher uses the same strategy. Full account and limits
The opening of a trial establishes where examination begins and how broadly the eyes move. Subsequent evaluation includes local reading, forward moves, returns, pauses and longer excursions before the final action. The research question is how these activities are organized and coordinated across gaze, cursor and scroll. Their timing need not follow a fixed sequence of stages.
| Process | Observable evidence | Interpretation to investigate |
|---|---|---|
| Initial sampling | Initial fixation locations, saccade amplitudes, visible page area | Establishing where useful material might be |
| Local examination | Fixation sequences and dwell within a result | Reading and assessing the result |
| Recurrent examination | Re-entry into an earlier AOI; excursion and return geometry | Comparison, checking or memory-guided reinspection |
| Commitment | Final approach and physical press, aligned with gaze | The transition from examination to action |
These are functional descriptions of observable activity, and more than one can be under way at the same time. Gaze, cursor and scroll provide different observations of the process. Pupil measures add an effort-related signal; none of them directly reports a person's reason for returning.
AdSERP requires a result selection. It supports analysis of examination and commitment within that task; it cannot establish when people naturally abandon a page, reformulate a query, or decide that further search is not worthwhile. The wider foraging account needs those stopping decisions too.
One clock, many views (PDF) partitions recorded time by AOI membership, initial fold, motion and gaze–cursor location. The aggregate views lead into a shared timeline, an observed trial and its final seconds. Each view names its denominator. The spatial views keep unavailable time in the denominator; the motion view conditions it out and states the classifiable share.
While the mouse waits, the eyes travel (PDF) links first-entry timing, AOI-position changes and cursor rest to a whole trial and a four-second pause. Gaze enters a jointly visited result first in 66.5% of trial–AOIs. Among nearby matched visits the lead is small: setting aside pairs that start at the first mousemove (in 1,552 of 2,650 trials the cursor's first visit starts there), gaze is first in 55.3% (95% CI 53.2–57.1) with a median lead of 49 ms. These timings do not show a fixed cursor delay behind gaze. [LAB, AdSERP, typed]
Both posters use the same 2,650-trial, 47-participant cohort and clock, from first native mousemove to final press. The atlas guide includes static exports, source values, measurement notes and rebuild commands. The HTML views work locally; GitHub's file viewer does not render them as pages.
- First visits and the next action: when attention stays, advances, moves back, or leaves the result column.
- Return geometry and memory: the evidence for a memory-guided interpretation, including precision and approach geometry.
- First fixations on ad-topped pages: where the first five fixations land when advertising tops the page.
- Peripheral exposure: distinguish being on screen, receiving nearby gaze exposure and receiving a fixation.
- Null findings: hypotheses that did not survive, including the lexical-priming account of declining dwell.
AdSERP (Latifzadeh, Gwizdka & Leiva, SIGIR 2025; paper, Zenodo) contains 2,776 trials from 47 participants, with eye tracking, mouse events, scroll, pupil recordings and captured search pages. AllSERP adds per-element geometry and types, so observations can be assigned to a result's box rather than to a vertical band estimated from the page. Analysis cohorts vary with signal availability and geometry exclusions.
Use the canonical Python environment and the shared data loader. The notebook guide, claim registry and methodology directory provide the analysis and measurement definitions. The figure gallery records other figures and their producers.
For individual trials, the foveated replay shows accumulated gaze on a perception-oriented background. The screenshot-based replay uses original captured page geometry. The foveated version re-renders archived HTML, so page layout can differ from the original capture.
AllSERP v1.1.1 (release,
notes) ships corpus CSVs with one row per
AOI (geometry, element type and organic rank) as release assets and under
scripts/output/, and one typed map per trial under data/aoi-typed-gapfill/
and data/aoi-typed/. Existing AdSERP signal files join on trial_id. The
AdSERP corpus itself is not redistributed; download it from Zenodo.
import json
import polars as pl
# One row per AOI: geometry, element type and organic rank (2,764 trials).
aois = pl.read_csv("scripts/output/adserp_aois_by_trial_id_typed_gapfill.csv")
print(aois.group_by("etype").len().sort("len", descending=True))
# Organic results whose top edge lies within the first screen (page coordinates).
first_screen = aois.filter(
(pl.col("etype") == "organic") & (pl.col("top_y") < pl.col("screen_height")))
print(first_screen.height, "organic AOIs start on the first screen")
# One trial's typed map: a list of AOIs with type, box and main-column position.
# Position -1 marks elements outside the result column, such as pagination.
with open("data/aoi-typed-gapfill/p010-b2-t6.json") as f:
trial = json.load(f)
print([(a["type"], a["position"], a["x"], a["y"], a["width"], a["height"])
for a in trial if a["position"] >= 0])scripts/build_aois.py --trial p010-b2-t6 rebuilds one trial's map from the
AdSERP screenshot and HTML, and --all rebuilds the corpus. --flavor selects
typed_gapfill (the default), typed or typed_gapfill_cellsplit. Moving from
v1.1.0: migration guide.
- AllSERP: Exhaustive Per-Element Enrichment of the Versatile AdSERP Dataset and its source repository.
- Earlier OSEC task-model draft — retained as a historical framing; its four-stage account is being reassessed.
- Measurement threats and limitations, references, and change history.
- CITATION.cff gives the AllSERP paper as the preferred citation and lists the AdSERP dataset paper; cite both when you use the data.
Analysis code: MIT. Derived AllSERP data (the per-AOI corpus CSVs under scripts/output/, the per-trial typed maps under data/aoi-typed*/, and the exclusion list) and the derived poster evidence under docs/visualizations/ are released under CC-BY-4.0, matching the AdSERP corpus licence; cite both (see CITATION.cff). The AdSERP dataset itself has its own license and is not redistributed here.