Judging what is worth saving: how people decide the fate of defective AI agent work

• During execution. Zhou et al. [4] model when a user should be asked to confirm an agent’s work mid-task; Chen et al. [9] compare four ways of supervising an agent while it runs.
• Checking what an agent did. Grunde-McLaughlin et al. [5] tested trace displays for finding agent errors: their design helped people find errors faster and feel more confident, without improving final accuracy. They raised the possibility that process-focused presentation supplies context while also justifying overreliance — the closest existing statement of the mechanism examined here, offered as a possibility rather than a tested claim. HANSEL [6] extracts interactive evidence from web-agent trajectories and supports resuming from a prior successful state; AgentTrace [7] proposes an interface for auditing what an agent touched and changed.
• After failure. Agent-side reversibility has been formalised, with a rollback taxonomy benchmarked against full restart [11]. Developers already review agent work after the fact [8]. REX [12] studies automated repair and explanation of robot failures. Khanna et al. [13] show that explaining a failure’s history changes how successfully people resolve it.
• Choosing a recovery strategy. Robotics has gone furthest. REPAIR-Bench [15] predicts which communicative recovery strategy a user would prefer; Genua et al. [16] allocate failures to operators; Banerjee et al. [17] use a module graph, calibrated uncertainty and estimated query cost to decide what to ask a person and when, minimising a weighted sum of query cost and task cost.

Recovery choice, dependency-graph representations and expected-cost allocation are therefore not novel in themselves, and the contribution is not a system for recovering partial agent work. What remains is narrower: once a completed digital artifact is known to contain a fault, how do people judge what is still recoverable, and what does the handback interface do to that judgement? Direct behavioural evidence appears limited on this specific decision — particularly where downstream invalidation and the enacted cost of each option are independently controlled. That is a qualified claim from a targeted search, not proof of absence; a fuller search precedes publication.

“Recoverability” is used to mean two different things, and the design depends on separating them.

They come apart. An artifact can be structurally almost intact and still not worth repairing; one with wide dependency reach can be worth repairing when restart is expensive. The interface does not manipulate structural recoverability — that is fixed by the artifact’s dependency graph. It manipulates the visibility of structural-impact information.

H1 and H2a predict effects in opposite directions on different errors. That is the central claim: structural transparency may not be straightforwardly good. Grunde-McLaughlin et al. [5] and Villavicencio et al. [18] both report substantial behavioural shifts with no improvement in accuracy; in each, a single outcome measure would have reported “no effect” while the interface changed what people did.

H2b is motivated by Villavicencio et al. [18], whose node-link display reduced external verification without increasing agreement. That study concerns uncertainty granularity in advice rather than downstream damage in an artifact: it plausibly motivates H2b, but does not establish that a dependency display causes premature restart. H2a is a genuine open question.

Participants work in a functional prototype, reviewing marketing campaigns built by an AI agent. Each artifact contains one fault at a known point. Choices are carried out, not clicked:

• Continue as-is — submit the artifact unchanged. Near-zero effort; quality shortfall proportional to the fault’s downstream reach. Submission is explicit and final, so it is a deliberate choice, not a default.
• Repair — fix the faulty content within the intact artifact, then submit.
• Salvage — build a new artifact using only unaffected components, then submit.
• Restart — re-run the agent, wait, and submit the new output.

All four end in a submitted artifact scored on the same terms. Quality is defined from the task specification, not from the strategy set: compliance with the brief’s stated requirements and correctness of the required components. Total volume is not a quality criterion — a shorter artifact that satisfies the brief is as good as a longer one. Repair and salvage must be experienced as genuinely different actions, and pilot interviews check that they are.

Before any interface is designed, the full matrix of dependency-reach level × strategy is written out: expected effort, expected quality shortfall and the loss-minimising action for each cell. For each scenario and strategy:

Roughly 120 people (40 per interface condition), each completing four scenarios in a session of about 35 minutes, recruited without payment through professional community channels. The final N is set by power simulation.

Inclusion: independently performed marketing-related activities within the past three years — professionally or for a personal or project purpose — involving creation, editing, evaluation or management of marketing content or campaigns. The judgement under study is largely structural, and the brief supplies the ground truth, so participants need enough competence to assess an artifact against a brief rather than deep marketing craft. Four safeguards keep that breadth from becoming a confound:

1. Competence is verified, not self-reported, by a short screener.
2. Experience is a pre-registered covariate: recency, frequency, context and campaign scale.
3. A minimum proportion of professionally experienced participants is pre-registered.
4. Experience is an exploratory covariate, not a separate benchmark; retained scenarios must keep the same optimal strategy across effort values spanning fast and slow performers.

After each scenario, participants write one or two sentences explaining their choice. Rationales come from the whole sample, are coded inductively against categories derived from the mechanism, and explain departures from the benchmark rather than adjudicate whether an effect exists.

The endpoint hierarchy is fixed in advance:

• Sole primary confirmatory endpoint: retention-error probability (H1), by interface condition.
• Primary descriptive outcome: probability of choosing the loss-minimising strategy.
• Pre-registered exploratory: discard error (H2a), independent-evidence inspection (H2b), the reach interaction (RQ3), cost information (H4) and all mediation.

1. A behavioural account of recoverability judgement for dependent artifacts: how people decide what is worth saving from defective agent work, and where those judgements depart from measured cost.
2. Evidence on a trade-off for which direct behavioural evidence appears limited: whether structural transparency reduces unsafe retention at the price of unnecessary abandonment.
3. A design implication for comparable handback tasks, if results support it — show structural impact when a recovery decision is required, but pair it with cost information so visible damage does not push people into premature restarts. Scoped to one domain and a fixed restart delay.
4. A calibrated loss function and open stimuli, with parameters measured rather than assumed.

The study is informative if H1 fails. A dependency view that improves comprehension without changing loss-relevant decisions, or shifts attention without improving choice, would join earlier findings [5,18] that more structured interface information need not improve final performance.

Where this leads. Three follow-ups are implied by what this design holds fixed: vary restart cost, to test whether the recoverability boundary moves with the price of abandonment; vary the artifact domain — code and analytical work have different dependency topologies; and vary graph reliability, since in deployment a dependency graph would be reconstructed from an agent trace and would be imperfect.

• Marketing is the experimental setting. Whether the findings hold for code, analysis or documents is an open question this study does not settle.
• Participants do not lose money from a poor decision; consequence comes from doing the work themselves and having outcomes scored and reported back.
• Restart delay is fixed rather than varied; varying it would change which strategy minimises loss, and is the natural next study.
• The display may shift economic judgement directly, not only through what people believe survives; mediation is therefore reported as explanatory rather than causal.
• The dependency graph comes from the artifact’s known structure and has no omissions or false edges. How people respond to unreliable structural information is a separate question.
• The inclusion criterion admits a range of marketing experience, guarded by the screener and the experience covariate; the design can examine, but is not powered to settle, differences by expertise.