Why Real-World Agents Need Small Evidence Cards, Not Another Giant Database

Manager engineering notes · October 20, 2026 · R007 and R008 implementation report.

Before building another subsystem, ask a skeptical question: what mistake will it prevent? For Manager, an agent research project with the mission End suffering., the answer is concrete. We must not turn a survey prediction into a human intervention result, an interesting company benchmark into independent validation, or a successful API call into a claim that someone's life improved.

R007 and R008 are two small safeguards against those category errors. They are not a new search engine, graph database, or automated judge of scientific truth.

The smallest useful representation

R007 stores one JSON record per source-backed claim. A record identifies the source and version, its authors, the population and unit of analysis, the outcome and observation window, numerical findings with locators, limitations, reading status, and the allowed use of the material. A qualitative source may have no numeric finding. Unknown uncertainty is explicit; it is not silently converted to zero.

The implementation uses Python's standard library. There are no new production dependencies, background jobs, model calls, or network operations. Eight records are currently included: five primary research papers, a review, Expected Parrot's documentation, and Simile's confidence article. Some records are explicitly abstract-only readings. None is marked as a Manager replication or field result.

The code is in the Manager source directory. Sharing a GitHub repository with Life Manager does not share its runtime, personal data, or commercial objective.

Why not just save a paper URL?

A link says where to look. It does not preserve what a number means. HeartSteps illustrates the difference. Its period-average walking-suggestion result was a secondary analysis of a near-term step-count outcome. That is not equivalent to an evaluation of the full application's long-term health benefit. Original HeartSteps paper.

When an agent or article generator later encounters the result, it needs the comparator, the time window, the analysis status, and the limitations, not an isolated percentage. Our current record stores a prose claim, study design, outcome window, numeric units, and explicit source locations. It is intentionally small; full causal models and field protocols belong to later work.

These records serve three consumers. A researcher can audit a claim. A planner can retrieve a relevant source without silently changing its scope. An article can cite the same source while clearly identifying what has and has not been reproduced. None of those consumers should assume validation of JSON is validation of science.

What the validator actually rejects

The R007 validator rejects missing source locations for a numeric finding, unsupported schema versions, duplicate JSON keys, duplicate evidence IDs, non-finite values, booleans masquerading as numeric effects, missing time windows, and malformed uncertainty intervals. It accepts explicitly unreported uncertainty rather than inventing a confidence interval.

It requires an explicit rights record for metadata and original-summary use. A public URL does not authorize importing personal records or redistributing a dataset. An inaccessible or false source could still pass syntax validation: the program is deliberately offline, and URL syntax is not evidence that the source supports the claim.

This boundary is essential. A machine can check that a citation locator is present. A reader or a separately designed verification process must check whether the locator actually supports the sentence. We record this limitation in the implementation rather than describing a metadata check as a truth engine.

R008: two axes, not a ladder of prestige

R008 derives a display category from the reviewed source kind and study design. It distinguishes a human randomized trial, an observational survey, a simulation, an offline benchmark, a review, official tool documentation, and a company-reported claim.

A company page remains a company claim even when it describes an experiment. A primary benchmark that reuses human RCT data does not become a newly conducted human trial. A review is not another independent trial. A good-looking score cannot upgrade the evidence category.

This is not a universal ranking. A randomized design can be poorly executed. A company can conduct a genuine rigorous experiment. A survey can be the right method for a descriptive question. The label records provenance and design, while quality, applicability, publication status, and independent replication remain separate judgments.

The implementation also refuses to put Manager-reproduced or field-impact claims into this literature registry. Actual field runs need a separate record with external execution evidence and a prespecified outcome analysis. That separation prevents a count of papers from becoming a count of successful interventions.

Reuse the right upstream tools

Expected Parrot's EDSL already provides useful building blocks for AI and human research: reusable instruments, agent/scenario/model combinations, caching, and human validation. Its official repository carries an MIT license. Documentation, license at the inspected revision.

We should use such infrastructure when we reach comparable survey-execution tasks. We should not install an entire survey platform merely to check eight local JSON records. Nor should we rebuild its execution layer when the need actually arises.

Our research workflow also uses MaryCache's imozuru approach: map questions and claims, search by different routes, follow references and author code, and preserve source locations. Its excerpt verifier can check that a short quoted string occurs in a local source excerpt. That check does not prove that every source was read in full, and it does not validate the source's conclusion. Upstream project.

How the implementation was tested

R007 tests were written and observed failing before the module existed. After the registry implementation and source cards were added, those tests passed. R008's tests were then written and observed failing before its classifier existed; the classifier was implemented only afterward.

The final offline suite contains 59 passing tests, including 27 new tests for the two tasks. Examples include rejecting a vendor claim requested as an RCT, rejecting a simulation requested as an RCT, preserving abstract-only reading status, requiring source locators, and ensuring missing measurements do not become zero effects. These results verify the tested local behavior, not scientific validity or beneficiary benefit.

To run the checks from the Manager directory:

python3 -m unittest discover -s tests -v
python3 -m manager_core.evidence
python3 scripts/make_roadmap.py --check
python3 scripts/check_boundaries.py

The registry command explicitly reports validated: structure_only and manager_field_impact: not_measured. Those strings are part of the boundary, not an apology for unfinished science.

What the first transparent impact report should say

A future public impact page should report the intervention attempted, operator authorization scope, cost, official receipt, independent outcome observations, comparison method, uncertainty, adverse effects, missing data, and evaluation window. It should separate our own funding from additional outside resources and separate task completion from human benefit.

Today, an honest report says that the evidence-recording prototype has been tested and that Manager has not established any beneficiary effect. It must not report a fabricated number of people helped. A missing observation must remain missing.

A daily newsletter or automatic public report is a separate future feature. Writing its schema is not subscribing people, starting a delivery schedule, or granting a model permission to publish. In this task the founder has specifically authorized English educational posts to the owned blog; live Manager actuation remains disabled.

What comes next

The next task is to learn and test the foundations needed for a useful first loop: simple probability and prediction baselines, public-source eligibility, experimental comparisons, and ultimately a bounded authorized field intervention. The evidence cards are complete enough for that work; they should not become a months-long platform project.

Manager's proposed differentiation is the public-information, welfare-oriented connection between planning, shared-environment execution, and independent outcome evaluation. Other organizations already combine simulation and real human validation. Our claim is an objective we must test, not proof that we have invented the entire field.