本周AI论文精选：模型与工具链、零记忆、反思方法等

1. Model or Harness

Agent evaluations mostly report system-level outcomes, so a failed run leaves the repair unassigned. The same visible failure might call for model post-training, harness engineering, environment redesign, or benchmark repair, and outcome labels cannot separate those cases.

• Every failure gets an edge: The taxonomy organizes 41 failure modes by assigning each one to an edge between two components (model, harness, user, tools, memory, environment) plus a fault side naming where the repair belongs.
• The schema is actionable by construction: Model-side failures identify post-training targets, harness-side failures point at scaffolding and tool-integration fixes, and environment or grader failures expose evaluation conditions that need redesign.
• It survives automation: Across four frontier models, the strongest judge reaches Cohen’s kappa of 0.76 against human category labels, so the labeling can run continuously over production traces instead of once per postmortem.
• Why it matters: Harness engineering became the main lever for agent builders this year while teams still lacked a shared vocabulary for where a harness bug ends and a model bug begins. This supplies that vocabulary, and it applies across coding assistants, long-horizon personal assistants, and multi-agent systems.

Paper | Tweet

Message from the Editor

We just released Build HTML Artifacts with Agents, a beginner-friendly, hands-on lab where you work alongside an AI agent to turn plain-English requests into polished HTML artifacts. Across 12 short labs, you build profile cards, data tables, charts, dashboards, and comparison grids in a live workspace, learning the request, inspect, and refine loop with no coding experience required.

Get Started

2. Zero-Mem

Production memory stacks spend extra model calls on summarizing interactions, writing records, and reranking retrievals. Each of those calls costs tokens and latency, and the generated summaries quietly discard the evidence you later need. This work asks whether structured memory access requires generation at all.

• Zero-token memory operations: No step outside final question answering invokes an LLM or consumes LLM tokens, with encoder computation accounted for separately, so the memory layer stops being a recurring inference bill.
• Two views over the original traces: Zero-Mem keeps raw interaction traces as its record and indexes them twice. An entity-context graph exposes connections across sessions while a temporal hierarchy preserves conversational locality and session state.
• Deterministic calibration before the reader: For each query it weighs both views, retrieves from both, follows their structure to recover supporting relations or surrounding context, then discards conflicting evidence so the single reader call stays grounded in retrieved traces.
• Why it matters: At matched reader and context budget, memory-operation time cost drops 57.6% against the fastest compared baseline with competitive accuracy on long-memory and long-context QA, which suggests a large share of memory-stack spend is buying structure that indexing already provides.

Paper | Tweet

3. Sample More Reflect Less

Methods that make a model criticize and rewrite its own answer nearly all generate far more text than a single chain of thought. Since generating more text raises accuracy on its own, a reported gain leaves open whether the method’s idea is what helped. This paper reruns the comparison as a designed experiment.

• Every token counted: Seven methods, open models at 1.5B, 3B, and 7B, two math benchmarks with 150 questions each, and every generated token counted including critiques, reflections, debate turns, and checking, with each method compared against repeated sampling at its own measured cost.
• No reliable win anywhere: All 36 comparisons are paired by question with bootstrap intervals and multiplicity correction, and repeated sampling holds up against every method at equal cost in every setting.
• Self-inspection is the failure mode: Ten comparisons come back reliably worse and every one of them is a method where the model inspects its own output, with all 18 self-inspection comparisons negative. Reflexion as published never triggered its own retry on the smallest model because it judged itself correct every time.
• Why it matters: Adding a critique step is the default reflex when an agent loop underperforms, and this study runs the comparison with paired bootstrap intervals and multiplicity correction, which the earlier point-estimate comparison lacked.

Paper | Tweet

4. Harness-R1

Agents accumulate interaction trajectories during deployment and then leave them unused, because their behavior stays fixed. Those trajectories can improve the harness that constructs context, mediates tools, validates actions, and recovers execution, and this work makes that editing a learned capability.

• A dedicated harness engineer: A separate 9B model converts batches of target-agent failures into validated executable patches across the runtime lifecycle, initialized with cold-start supervised fine-tuning and then trained online with group-relative policy optimization.
• The target stays frozen: Fresh same-batch reruns of the frozen target supply outcome rewards, so training updates only the engineer and the agent being repaired holds still under the reward signal.
• It works before and after tuning the target: Across WebShop, ALFWorld, and DBBench, vanilla Qwen3.5-9B goes from 44.3% to 53.6%, and after the target itself is fine-tuned a target-specific engineer lifts the average further from 59.2% to 64.2%.
• Why it matters: If you run agents in production you already have the training data, and because the gains hold on both sides of target fine-tuning, the paper points toward co-evolving the harness engineer and the agent it repairs.

Paper | Tweet

5. DataSpace

Real organizational analytics scatters evidence across databases, structured files, long documents, and video. Existing benchmarks isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic scoring untested together.

• Workspace-scale tasks: DataSpace contains 410 cross-language tasks over 7,439 artifacts totaling 15.01 GB across CSV, JSON, SQLite, Markdown, PDF, and video, and each agent receives only a question and a workspace before returning the full requested tabular result.
• Deterministic evaluation: Scoring performs header-invariant column alignment, type-aware and precision-aware normalization, and order-aware row comparison, which removes the judge model from the loop entirely.
• Harness choice is worth 15 points: Across six recently released frontier multimodal models and five widely used agent harnesses, the best accuracy reaches 66.34%, and holding the backbone fixed while swapping the harness moves accuracy by 15.36 points.
• Why it matters: Multimodal evidence integration and joins reduce accuracy across all six backbones, so the benchmark remains far from saturated. It also served as the official evaluation benchmark for the KDD Cup 2026 Data Agents for Complex Data Analysis competition.

Paper | Tweet

6. Prompt-Induced Waste

Two prompts can request the same code change and produce the same correct patch while causing a coding agent to perform radically different kinds and amounts of work. This preregistered study measures that effect across 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real harnesses.

• Wording changes where effort goes: Prompt phrasing redirects effort into different work. Asking for multiple approaches inflates reasoning by 2.4x to 7.4x across all six open models and produces roughly three elaborated but discarded solution branches, still yielding one implemented solution and no success gain.