AgentFlare

Scholar · web navigation agents benchmarks

Web Agents & Navigation Benchmarks: Research Digest

10 foundational papers with 234 combined citations, plus the latest arXiv preprints.

Foundational papers10234 citations
Most citedSt-webagentbench: A benchmark for eva…145 citations · 2026
arXiv, Sep 202617 papers-43% vs Oct 2025
Newest preprint2026-10-26

AISA Scholar API · arXiv

arXiv submissions per monthPapers matching (abs:"web agent" OR abs:"web navigation") · last 12 complete months
monthly arXiv submissions: Oct 2025 30 → Sep 2026 17; high 30 (Oct 2025), low 10 (Nov 2025)8162432Oct 2025Mar 2026Sep 2026
402 · Pay-per-Crawl$0.02

You're reading the full report — free for humans and search engines. The same content ships as a structured JSON dataset (normalized facts, sources and citation arrays, no HTML parsing) so AI agents can consume it directly — metered per fetch via the open HTTP 402 payment protocol.

⚡ Agents: fetch .data.json · $0.02 → agents: crawlerAutoPrice=true · verify at https://esa.aisa.one/api/v1/access/verify

What’s new on arXiv

A12026-10-26cs.LG

SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skills

Chenxu Zhang, Zhou Liu, Wu Xin, Wentao Zhang

Web agents reuse skills whose stored code, selected version, and loaded implementation can diverge across tasks and updates. SkillLifeBench links these states to task outcomes in 388 paired scenarios spanning implementation, interface, runtime, composition, and version changes. We evaluate 14 models on two paired decision tracks: 89 runtime tasks and 60 autonomous lifecycle tasks spanning 12 templates derived from introduced cases. Lifecycle tasks allow agents to inspect, select, and reconfigure skills while completing verifiable user requests. Joint success requires task completion without the registered attack goal. GPT-5.6-Sol and Sonnet 4.6 both achieve 82.0% on runtime tasks, but reach 96.7% and 56.7% on lifecycle tasks. Across lifecycle evaluations, challenged task success is 77.5%, while joint success is 65.2%; 103 runs complete the task and the attack goal. Adding execution provenance in a matched recovery study raises utility joint success from 49.35% with logs and metadata to 65.58%. A deterministic execution-consistency rule reaches 64.94% with higher benign utility, while full restoration reaches 69.48%. Lifecycle tasks distinguish models with similar runtime scores, and execution evidence improves recovery.

A22026-10-26cs.AI

Harness-Model Collaboration for Long-Horizon Continual Learning

Zhen Bi, Zhizhi Peng, Yan Wang, Xueshu Chen, Haofei Zhang, Yaobo Zhang, Zeyu Yang, Qing Shen, Jungang Lou

Long-horizon agents continuously encounter reusable procedural knowledge, yet existing systems typically rely either on model adaptation or external memory, trading persistent learning for rapid reuse. This limitation is especially important for Web agents that repeatedly execute related multi-step workflows. We propose HaMo, a Harness-Model collaborative framework that combines fast non-parametric experience accumulation with slow parametric adaptation. HaMo distills trajectories into strategy-level experience cards for immediate retrieval, while an experience-conditioned continual learning objective trains the model to interpret, ground, and execute retrieved strategies across changing contexts. A retrieval-coverage-based selection mechanism further identifies compact and complementary trajectories for parameter updates. Experiments show that HaMo outperforms harness-only and model-only adaptation under within-domain and cross-domain shifts, improving both retention and adaptation. These results highlight the importance of jointly preserving procedural experience and internalizing the competence to transfer and execute it across tasks.

A32026-10-25cs.MA

AgentProofBench: Benchmarking Reusable Verifiers for Web Agents

Jingjie Ning, Xueqi Li, Yi Lu, Yibo Kong

Reusable Web-agent verification requires a rule to follow changing requirements and support subsequent execution. We introduce AgentProofBench, which connects paired tests of a frozen acceptance rule to execution from identical saved openings. The resource spans 97 tasks, 79 families, and four Web services. Experiments with DeepSeek Flash/Pro establish two linked measurements. Across 48 families, complete-panel and decision-replay tests retain 82/96 Flash and 79/96 Pro source programs; paired reuse-rebuild BA intervals are [-5.47, 1.57] and [-10.16, 0.00] percentage points. Across 22 execution families, deployment-state panel correctness separates program assignments whose two saved openings are both delivered correctly: 32/33 assignments with error-free panels, versus 1/11 with panel errors. All reference-defined program-gate regressions and withheld deliveries occur in the latter group. Neutral continuation attains the highest correct delivery under the tested generic feedback, providing a concrete baseline for measuring the value added by a gate. Released instructions, states, programs, reference rules, and trajectories connect each test failure to its execution consequences. AgentProofBench makes reusable verification assessable at both the requirement and policy levels.

A42026-10-24cs.AI

SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents

Bo-Yu Chen, Zhi Zhou, Yu-Feng Li

Skill learning enables web agents to reuse prior experience for long-horizon interactive tasks. However, learned skills are typically evaluated in closed environments, overlooking the fact that real-world web deployment involves continuous shifts in invocation context, perceptual grounding, and execution feasibility. To this end, we introduce SkillShift-Bench, the first benchmark for evaluating skill learning with controlled open-environment shifts. SkillShift-Bench simulates Contextual Shift, Perceptual Shift, and Execution Shift across three self-hosted web systems, and evaluates agents with diagnostics covering task success, interaction cost, skill use, and skill failure. Comprehensive experiments reveal systematic robustness degradation under controlled shifts, characterized by lower success rates, higher interaction costs, and more frequent skill failures, with Execution Shift exerting the strongest pressure. We further formulate shifted skill execution as a nonstationary local executability problem and provide both a dynamic-regret analysis and a practical test-time adaptation method, demonstrating the feasibility of improving learned-skill execution without relearning the skill library. Code, tasks, environments, documentation, and the leaderboard are available at https://skillshift-bench.github.io/.

A52026-10-22cs.CL

Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web Agents

Yiqi Wang, Zizhi Zeng, Jiaqi Zhang, Yiqun Duan, Mingkai Zheng, Taotao Cai

Web agents increasingly serve the same user across repeated online tasks such as booking travel, shopping, and scheduling, so their long-term usefulness depends on how well they know the user's preferences. Passive memory and personalization methods cannot obtain preferences the user never reveals, and clarification methods that value a question by its benefit to the current task overlook the benefit of reusing the answer in later tasks. We study Active Preference Acquisition (APA): which missing preferences are worth the cost of asking? We define the Prospective Personalization Value (PPV) of a missing preference as its expected cumulative utility gain over future tasks minus the effort and disclosure-risk cost of asking, and select questions under a per-interaction budget. We characterize when cumulative reuse justifies acquisition despite insufficient single-task benefit, and evaluate this regime with an initial-profile diagnostic. APA computes the counterfactual gain of each candidate preference on public task prototypes with the agent's own task solver and aggregates it by the expected reuse of each task family. We evaluate it on ActiveWebPref, a controlled longitudinal benchmark with hidden but queryable user profiles and exact task utilities. At a horizon of 20 tasks, 34.4% of budget-feasible valuable missing preferences are amortization-dependent; APA acquires 80.3% of them, whereas current-task value of information acquires 5.1%. Under per-user asking-cost caps matched to APA's realized expenditure, APA achieves higher net utility than four heuristic selection baselines. In our cost setting, asking about every unknown preference yields lower net utility than not asking, while APA recovers 84.2% of its utility gain with 27.6% of its questions. The value of acquiring a user preference should account for its reuse across future tasks, not only its benefit to the task at hand.

A62026-10-21cs.CR

MPIDetector: Detecting Multi-modal Prompt Injection via LLM-based Reasoning

Yiheng Li, Jingyue Li

With the rapid integration of Multi-modal Large Language Models (MLLMs) into real-world applications, Prompt Injection (PI) has emerged as a critical security threat. Attackers can conceal malicious instructions within textual narratives or visual inputs to bypass safety alignment. Existing approaches, such as signature- and perplexity-based methods, rely heavily on pattern matching, making them insufficient to detect sophisticated, cross-modal PI payloads. In this paper, we propose a novel detection framework, called MPIDetector, that leverages LLMs' intrinsic reasoning capabilities via few-shot In-Context Learning (ICL). Instead of fine-tuning, we provide the model with specific prompt templates and representative attack examples, enabling it to scrutinize the semantic intent of input across modalities. We constructed a comprehensive evaluation benchmark comprising eight textual and seven visual malicious categories, supplemented with benign datasets to rigorously test for false positives. Our evaluation across multiple leading LLMs, including Gemini-2.0-Flash, Qwen3-VL-32B-Instruct, and Doubao-seed-1.6-flash, demonstrates the efficacy and generalizability of this approach. The proposed method achieves up to 90.91% accuracy and 100% recall on textual tasks. In the visual domain, it achieves up to 99.28% accuracy while maintaining 100% precision across all evaluated models. Crucially, when evaluated against 500 highly obfuscated environmental web-agent injections (WebInject), MPIDetector achieved a detection rate up to 99.80%. Comparisons with two baseline approaches for detecting textual PI payloads, along with two additional methods for visual input detection, demonstrate MPIDetector's superior accuracy. These results show that LLM-driven reasoning provides a robust, cross-modal defense against advanced prompt injections.

A72026-10-19cs.CL

LoWAgent: A Low-Resource Multilingual Benchmark for Web Agents

Ocean Monjur, Daud Ahmad Nisar, Shahriar Kabir Nahin, Tadesse Destaw Belay, Sushruti Mishra, Israel Abebe Azime, Anshuman Chhabra

Web agents are typically evaluated on English-language interfaces, leaving their ability to operate across the world's multilingual websites largely unexplored. This gap is especially significant for low-resource languages, where standardized resources for evaluating web agents remain limited. To address this gap, we introduce Low-Resource Web Agent Benchmark (LoWAgent), a benchmark for evaluating web agents across different low-resource languages (LRLs) in comparison to the default, English. LoWAgent evaluates web agents across English and four LRLs: Hindi, Bengali, Urdu, and Amharic. The benchmark comprises of six fully localized websites, instantiated across five languages to yield 3,000 task instances. We evaluate three closed-source and five open-source models across all five languages on all six websites. Our findings show that both closed-source and open-source models experience performance degradation for LRLs compared to English, with substantially greater performance drops observed among the open-source model agents. We further evaluate the robustness of the models on realistic interface disruptions, such as cookie notices and advertisement pop-ups, and find that these further degrade agentic performance. Finally, we show that post-training on a small subset of LoWAgent traces can significantly improve low-resource language agentic task performance, demonstrating the usefulness of our benchmark.

A82026-10-15cs.LG

Don't Retrain, Remix: Adaptation of Web Agents using Mixture Hypernetworks

Md Farhan Ishmam, Minh Pham Dinh, Kenneth Marino

The web is not static or uniform. Websites that serve the same purpose can look and navigate very differently. Even within a single site, successive redesigns can change how familiar features are presented and accessed. Humans adapt to this by recognizing familiar interaction patterns and reusing what they already know. We give web agents a similar ability with WebMix, a mixture hypernetwork that adapts them to unseen interfaces by composing previously trained low-rank adapters. WebMix begins with a bank of adapters trained on source interfaces. At deployment, a scout agent explores the target interface and summarizes its observations into a compact manual. A lightweight router uses this manual to predict mixing coefficients over the adapter bank, which combine the existing adapters into a new one for inference. Adapting to a new interface takes only a scouting run and a forward pass, without gradient updates on the target. With Qwen and Llama models, we evaluate domain transfer on TimeWarp and WebArena-Lite-v2, and version transfer on TimeWarp. In all three settings, WebMix matches or outperforms in-context learning and LoRA baselines, often matching the average success of specialists trained directly on those interfaces. On TimeWarp domain transfer, it gains about 8 percentage points on average over the best baseline with both models. These gains do not come at the cost of seen interfaces, where WebMix keeps its lead, and extend to version transfer. Adapting to a new interface can therefore rely on exploration and composition instead of retraining.

Foundational papers

  1. 2025
    Mm-browsecomp: A comprehensive benchmark for multimodal browsing agents40 citations
  2. 2026
    Webds: An end-to-end benchmark for web-based data science4 citations
  3. 2026
    WARC-Bench: Web Archive based Benchmark for GUI Subtask Executions1 citations
  4. 2026
    The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge0 citations
  5. 2026
    CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception1 citations
  6. 2026
    St-webagentbench: A benchmark for evaluating safety and trustworthiness in web agents145 citations
  7. 2026-10
    Latest on arXiv: SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skillscs.LG
Citations per paperCitation counts via the AISA Scholar API
citations: R1 WebRetriever: A Large… 1, R2 From grounding to pla… 18, R3 Webds: An end-to-end … 4, R4 MolmoWeb: Open Visual… 21, R5 WARC-Bench: Web Archi… 1, R6 Mm-browsecomp: A comp… 40, R7 The Hard Part Comes A… 0, R8 Error Patterns of Lar… 3, R9 CAP: A Scalable Bench… 1, R10 St-webagentbench: A b… 145R1 WebRetriever: A Large…1R2 From grounding to pla…18R3 Webds: An end-to-end …4R4 MolmoWeb: Open Visual…21R5 WARC-Bench: Web Archi…1R6 Mm-browsecomp: A comp…40R7 The Hard Part Comes A…0R8 Error Patterns of Lar…3R9 CAP: A Scalable Bench…1R10 St-webagentbench: A b…145
R320264 citations

Webds: An end-to-end benchmark for web-based data science

E Hsu, HM Yam, I Bouissou, A John et al.

… often begin by browsing the web, navigating across multiple websites, and synthesizing information … To address the limitations of existing web agent and data science benchmarks, we …

How this page is made. Foundational papers and citation counts come from academic search via the AISA Scholar API, with abstracts read from each paper’s page; recent preprints and monthly volumes come from the public arXiv API (metadata CC0). The digest is written by a language model from those abstracts only — every theme links the papers it draws on, and figures not in the inputs are removed.

Sources & citations

  1. WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation 1 citations
  2. From grounding to planning: Benchmarking bottlenecks in web agents 18 citations
  3. Webds: An end-to-end benchmark for web-based data science 4 citations
  4. MolmoWeb: Open Visual Web Agent and Open Data for the Open Web 21 citations
  5. WARC-Bench: Web Archive based Benchmark for GUI Subtask Executions 1 citations
  6. Mm-browsecomp: A comprehensive benchmark for multimodal browsing agents 40 citations
  7. The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge 0 citations
  8. Error Patterns of Large Language Model Agents in Multi-Step Web Navigation: A Comparative Empirical Study across Prompting Strategies and Backends 3 citations
  9. CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception 1 citations
  10. St-webagentbench: A benchmark for evaluating safety and trustworthiness in web agents 145 citations
  11. SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skills arXiv · 2026-10-26
  12. Harness-Model Collaboration for Long-Horizon Continual Learning arXiv · 2026-10-26
  13. AgentProofBench: Benchmarking Reusable Verifiers for Web Agents arXiv · 2026-10-25
  14. SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents arXiv · 2026-10-24
  15. Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web Agents arXiv · 2026-10-22
  16. MPIDetector: Detecting Multi-modal Prompt Injection via LLM-based Reasoning arXiv · 2026-10-21
  17. LoWAgent: A Low-Resource Multilingual Benchmark for Web Agents arXiv · 2026-10-19
  18. Don't Retrain, Remix: Adaptation of Web Agents using Mixture Hypernetworks arXiv · 2026-10-15

Related