AgentFlareScholar · large language model agents planning
LLM Agents & Planning: Literature Digest
10 foundational papers with 3,266 combined citations, plus the latest arXiv preprints.
AISA Scholar API · arXiv
You're reading the full report — free for humans and search engines.
The same content ships as a structured JSON dataset (normalized facts, sources and
citation arrays, no HTML parsing) so AI agents can consume it directly — metered per fetch via the open
HTTP 402 payment protocol.
crawlerAutoPrice=true · verify at https://esa.aisa.one/api/v1/access/verify
What’s new on arXiv
OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents
Humza Nusrat, Kundan Thind
Background and purpose: Radiotherapy planning agents based on large language models (LLMs) decide when to submit a plan or request review but are evaluated mainly by dose. This study developed OpenGray, a benchmark of planning tasks with known correct decisions. Materials and methods: OpenGray adds tasks with known answers to the OpenKBP-Opt head-and-neck dataset, including two contradictions with dose goals proven incompatible and a feasible control with a stored plan meeting every hard goal. Four systems, from a rule-based planning script to a full LLM agent, were evaluated on 30 patients, the LLM-based ones with a commercial (Terra) and a local (Ministral 3 8B) LLM. Decisions, achieved dose, problem recognition, and technical failures were recorded and also read from logs of 39 stereotactic radiosurgery runs of SAGE, a clinical LLM agent. Results: With Terra, the script preceded by an LLM review and the full agent had decision accuracies of 95.4% and 92.1%, respectively, yet escalated only 50% (95% confidence interval, 25-75%) and 25% (10-40%) of overlap contradictions, which required arithmetic on the anatomy. All four systems submitted every feasible control, but 24% of these plans missed a hard goal that the stored plan had met. In four SAGE runs, one goal failed at every attempt; the agent identified it in every reply but had no action for stopping. Conclusions: Favorable aggregate scores concealed specific decision failures. Planning agents should be evaluated on their decisions as well as their dose and given an explicit action for stopping and requesting review.
LM-DSB: A Lightweight Multi-agent System for Automated Daily Safety Bulletin Generation in Construction
Xianxiang Zhao, Advik Mehta, Falak Sethi, Brian Gue, Qipei Mei, Lingzi Wu
Construction safety management still struggles to produce daily safety bulletins (DSBs), structured daily communications that identify site-specific hazards, convey safety protocols, and set preventive measures for workers. Manual DSB creation takes 2-3 hours daily and yields inconsistent quality, with 25% of crews never conducting pretask planning (Olson et al., 2016), diverting safety personnel from field supervision. This study proposes the Lightweight Multi-Agent DSB System (LM-DSB), which integrates retrieval-augmented generation (RAG) with a locally deployable Large Language Model (LLM) to automate DSB generation on consumer-grade personal computers. LM-DSB coordinates specialized planner, researcher, and writer agents in a hierarchical architecture, enhanced by domain-specific fine-tuning on construction safety documentation and RAG grounding in 37 health, safety, and environment (HSE) documents. A two-phase validation assessed feasibility. Phase 1 comparative evaluation showed the multi-agent RAG system (Llama 3-8B) outperformed all alternative configurations, with strong lexical alignment and coherence (TF-IDF: 0.70, BLEU: 0.39, ROUGE-1: 0.66) and near-complete document generation (length ratio: 1.01 vs. 0.39 for single-agent RAG). Phase 2 expert evaluation (N=40) confirmed practical acceptability (mean scores 3.50-4.20 on a 5-point scale), particularly in regulatory compliance (4.20) and factual accuracy (4.10). The findings show that architectural design enables lightweight models to handle safety-critical documentation on consumer-grade hardware, advancing technology democratization in construction safety management. Limitations include knowledge base scope, language constraints, and lack of field deployment validation; future work should address multimodal integration and workflow adoption.
MAREval: A Multi-Agent Framework for Evaluating Natural Language Recommendation Explanations
Reza Yousefi Maragheh, Jayesh Uddhav Kudase, Aysenur Inan, Ramin Giahi, Kai Zhao, Parth Parikh, Jianpeng Xu, Jason Cho, Evren Korpeoglu, Su…
Evaluating natural language explanations in recommender systems is essential for fostering user trust, transparency, and engagement. However, existing evaluation approaches like human evaluations, while accurate, are resource-intensive and impractical at the scale required by modern recommendation platforms. Also, automated methods using single-agent LLMs suffer from prompt sensitivity and inconsistent outputs. To address these challenges, we propose MAREval, a structured multi-agent framework for evaluating recommendation explanations using large language models. MAREval orchestrates (i) a planner agent that uses a novel Chain of Debate (CoD) prompting strategy to coordinate agent roles and enforce logically consistent evaluation plans; (ii) a moderator agent that regulates discussions by mitigating prompt drift; and (iii) an arbitrator agent that aggregates outputs from multiple evaluation rounds and (iv) a Monte Carlo sampling method, improving robustness and alignment with human judgment. We conduct comprehensive evaluations on both public (TopicalChat) and proprietary recommendation datasets, demonstrating that MAREval outperforms state-of-the-art baselines. Comprehensive experiments on a public benchmark and a proprietary e-commerce dataset show that MAREval improves alignment with human judgments over strong singleand multi-agent baselines. Stability analyses indicate substantially lower variability across repeated trials. In a large human-annotation gate, MAREval meets production quality thresholds where prior evaluators fall short, and online A/B testing demonstrates statistically significant improvements in engagement and revenue metrics. These results establish MAREval as a scalable and reliable solution for human-aligned evaluation of recommendation explanations in real-world systems
EREBOTS 4.0: A framework for conversational assistants with A2A agents and MCP tools
Elia Pacioni, Davide Calvaresi, Andrei C. Coman, Ismael Jaggi, Valerio Crocetti, Gaetano Manzo, Michael Ignaz Schumacher
In recent years, multi-agent systems have shifted from rule-based, symbolic agents to sub-symbolic agents built on language models. However, several limitations remain unresolved. The new frontier is neuro-symbolic agents, which pair language models with deterministic tools and explicit protocols. Without a common framework to standardize and compose them, each study rebuilds the same infrastructure, and results are hard to replicate. This paper presents EREBOTS 4.0, an open-source, hybrid-AI, multi-agent framework that fills this gap for conversational assistants. Language-model agents plan and route requests, while deterministic tools compute exact results over the Model Context Protocol (MCP). An orchestrator selects one or more specialist agents per turn and calls them over the Agent2Agent protocol (A2A). A new agent, local or remote, is added by listing its URL in a configuration file. The framework also manages identity, user context with consent flags, task states and traces. Its A2A server passed 97 tracked requirements in the official A2A Technology Compatibility Kit, with no failures.
EvoRouter: Cost-Efficient Multi-Agent Routing via Insight-Guided On-Policy Distillation
Yitao Xiao, Xu Zhang, Ye Cheng, Shaoyong Guo, Yan Sun, Ziyue Zhu, Sitao Xie, Guo Tong, Haoran Luo
Serving web queries at scale with multi-agent systems requires jointly configuring large language models, collaboration modes, agent counts, interaction rounds, and skills. However, the resulting configuration space grows combinatorially, while sparse execution rewards provide limited fine-grained supervision for interdependent planning decisions. We propose EvoRouter, a self-evolving multi-agent routing framework driven by insight-guided on-policy distillation. A dedicated expert periodically synthesizes performance-cost feedback from configuration trajectories across tasks into reusable conditional routing insights about capability requirements, returns from resource scaling, and interactions among configuration choices. A privileged teacher combines these evolving insights with current execution feedback to provide dense token-level supervision for the router's own planning trajectories through on-policy distillation. Cost-aware reinforcement learning jointly optimizes task performance and computational cost. Together, these training signals internalize execution insights into the routing policy. The updated policy generates new trajectories that support subsequent insight extraction, forming an execution-driven self-improvement loop. We evaluate EvoRouter on 19 in-domain datasets spanning five domains and two out-of-domain benchmarks. EvoRouter achieves an average in-domain accuracy of 68.69%, outperforming the strongest routing baseline by 7.23 percentage points while reducing inference compute cost by 50.95%. Our code is open-sourced at https://github.com/CenYangtze/EvoRouter.
From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks
Zhenshuai Yin, Yongxin Guo, Mingda Zhang, Zhongze Luo, Haoyue Liu, Lianmin Zhang, Xiaoying Tang
Effective collaboration requires language models to understand what others know and believe, and to act accordingly. Models that assume others share their information may omit necessary communication and undermine coordination. Many theory-of-mind (ToM) benchmarks assess mental-state inference through story questions, but correct answers do not establish that models can use these judgments in subsequent tasks. We introduce a multi-level evaluation built on existing, verifiable tasks. We extend household collaboration and calendar scheduling with asymmetric information while retaining their original constraints and validators. Independent interfaces assess world understanding, information access, beliefs, decisions, and execution. Controls supply personal evidence or correct beliefs to distinguish state recovery from state use. Across six model configurations, these abilities diverge. On matched scheduling cases, Claude recovers all agent beliefs correctly but produces subjectively optimal plans in only 67.9% of cases. In household collaboration, supplying beliefs raises Qwen-27B success from 55.0% to 75.0%, yet Qwen-4B answers every fixed supplied-belief source question correctly while completing only 20.0% of the corresponding continuation condition. These results show why ToM evaluation must connect perspective inference to independently verified decisions and task consequences.
CodeMeepo: A Multi-Agent Workflow for Repository-Level Code Migration and Optimization
Farzad Habibi, Changrong Wu, Jinzhou Jiang, Guohan Lu
Repository-level code migration rewrites an entire project in a new programming language while preserving its observable behavior. It is challenging because behavior spans interdependent modules, dependencies and build infrastructure must be adapted to the target ecosystem, and the source code is often the only specification. Large language model agents can now attempt these migrations end to end, but three challenges remain. First, an agent that writes both the translation and its acceptance tests may overlook the same requirements in both, leaving missing or incorrect behavior undetected. Second, large repositories can exceed a model's context window, and dividing the work into smaller tasks does not ensure that all required behavior is migrated. Third, even behaviorally correct translations may be slower than the original implementation. We present CodeMeepo, a multi-agent pipeline for complete and efficient migration of large repositories. Each milestone is evaluated against a fixed external oracle that the agents neither author nor modify. A separate parity assessment compares source and target components in scope and adds identified gaps back into the plan as required work. Ordered milestones limit the scope of each task, while cumulative regression checks guard against breaking already-migrated behavior. A benchmark-guided optimization phase addresses performance, followed by full-suite conformance testing and repair. Eight specialized agent roles communicate through file artifacts, and a deterministic state machine governs repair, deferral, scope changes, and termination. On nine subjects in three language pairs, CodeMeepo achieves the best pass rate on every subject and passes 18% more tests than a state-of-the-art multi-agent baseline using the same model. On a production daemon, its optimization phase improves half of the benchmarks, by 25% on average.
How Humans Learn Unfamiliar Worlds: A Think-Aloud Dataset on ARC-AGI-3
Shuzhi Liu, Archana Warrier, Dat Nguyen, Chengqi Zheng, Kevin Ellis, Yewen Pu
Learning through exploration and using newly acquired knowledge to achieve goals are central capabilities of general intelligence. Understanding these capabilities requires tracing how learners build, use, and revise world models during interaction. We construct the first dataset that records how humans discover and use the mechanisms of novel environments, using ARC-AGI-3 as a testbed: 144 trajectories from 64 participants across 25 public games, each pairing time-synchronized actions with think-aloud audio, plus 250 trajectories from large language model (LLM) agents with and without harnesses on the same games. We annotate mechanism triggers and recognition, action purposes, and errors and recovery, linking changes in understanding to the observations and actions that support them. We analyze characteristics of human cognition and compare them with those of LLM agents. Both groups anticipate mechanisms and use familiar analogies. Qualitative representations are more common in successful human trajectories than in successful LLM trajectories (50.0% versus 14.3%). Humans more often explore unfamiliar situations without a concrete hypothesis, while hypothesis testing is common in both groups. We also trace how learners recover through belief revision, replanning, and execution correction. As LLMs with well-designed harnesses reach saturated performance on public ARC-AGI-3 games, our cognitive analyses motivate next-generation benchmarks that challenge familiar priors, require accumulated evidence and abstraction, and test planning through subgoals, experimentation, and recovery. Our dataset is available at [https://arc3-human-think-aloud.github.io/](https://arc3-human-think-aloud.github.io/).
Foundational papers
- 2023
- 2023Embodied task planning with large language models206 citations
- 2024
- 2025
- 2025
- 2025
- 2025
- 2026-10
Llm-planner: Few-shot grounded planning for embodied agents with large language models
CH Song, J Wu, C Washington et al.
… large language models (LLMs) as a planner for embodied agents that can follow natural language … methods hinders the development of versatile agents that are capable of many tasks …
Large language models for planning: A comprehensive and systematic survey
P Cao, T Men, W Liu, J Zhang, X Li, X Lin, D Sui et al.
… for LLM-based agent planning to enlighten and guide … agent planning, which is organized into four closely related parts, involving planning definitions, planning methods, planning …
Smart-llm: Smart multi-agent robot task planning using large language models
SS Kannan, VLN Venkatesh et al.
… natural language to articulate tasks and utilizes pre-trained language models equipped with … Similarly, in SMART-LLM, we employ large-language models to deconstruct tasks into robot …
Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning
K Rana, J Haviland, S Garg, J Abou-Chakra et al.
… is on integrating large-scale scenes into planning agents based on Language Models (LLMs) … We aim to tackle two key problems: 1) representing large-scale scenes within LLM token …
AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
M Hu, P Zhao, C Xu, Q Sun, JG Lou, Q Lin et al.
… Unlike previous papers on all agent training, AgentGen goes beyond merely generating … using Large Language Models. Instead, we utilize Large Language Models to generate agent …
Embodied task planning with large language models
Z Wu, Z Wang, X Xu, J Lu, H Yan
… In this paper, we present a task planning agent called TaPA for embodied task plan grounding in physical scenes. The unreleased SayCan cannot be applied in diverse indoor …
Large language model for participatory urban planning
Z Zhou, Y Lin, D Jin, Y Li
… participatory planning process given the large number of … planning framework through multi-agent collaboration. Specifically, the planner agent first proposes an initial land-use plan for …
Recap: Recursive context-aware reasoning and planning for large language model agents
Z Zhang, T Chen, W Xu et al.
… reasoning and dynamic re-planning remain challenging for large language models (LLMs). … and Planning), a hierarchical framework with shared context for reasoning and planning in …
Planning with multi-constraints via collaborative language agents
C Zhang, XD Goh, D Li, H Zhang et al.
… The rapid advancement of neural language models has sparked a new surge of intelligent agent research. Unlike traditional agents, large language model-based agents (LLM agents) …
Revealing the barriers of language agents in planning
J Xie, K Zhang, J Chen, S Yuan, K Zhang et al.
… The emergence of large language models (LLMs) and their powerful reasoning … -level planning abilities? Specifically, we focus on language agents’ vanilla planning as well as planning …
How this page is made. Foundational papers and citation counts come from academic search via the AISA Scholar API, with abstracts read from each paper’s page; recent preprints and monthly volumes come from the public arXiv API (metadata CC0). The digest is written by a language model from those abstracts only — every theme links the papers it draws on, and figures not in the inputs are removed.
Sources & citations
- Llm-planner: Few-shot grounded planning for embodied agents with large language models 1,292 citations
- Large language models for planning: A comprehensive and systematic survey 76 citations
- Smart-llm: Smart multi-agent robot task planning using large language models 521 citations
- Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning 789 citations
- AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation 136 citations
- Embodied task planning with large language models 206 citations
- Large language model for participatory urban planning 146 citations
- Recap: Recursive context-aware reasoning and planning for large language model agents 16 citations
- Planning with multi-constraints via collaborative language agents 54 citations
- Revealing the barriers of language agents in planning 30 citations
- OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents arXiv · 2026-10-27
- LM-DSB: A Lightweight Multi-agent System for Automated Daily Safety Bulletin Generation in Construction arXiv · 2026-10-27
- MAREval: A Multi-Agent Framework for Evaluating Natural Language Recommendation Explanations arXiv · 2026-10-26
- EREBOTS 4.0: A framework for conversational assistants with A2A agents and MCP tools arXiv · 2026-10-26
- EvoRouter: Cost-Efficient Multi-Agent Routing via Insight-Guided On-Policy Distillation arXiv · 2026-10-26
- From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks arXiv · 2026-10-26
- CodeMeepo: A Multi-Agent Workflow for Repository-Level Code Migration and Optimization arXiv · 2026-10-25
- How Humans Learn Unfamiliar Worlds: A Think-Aloud Dataset on ARC-AGI-3 arXiv · 2026-10-25