- Early Years, journey 1
- Junior, journey 2
- Maker, journey 3
- Explorer, journey 4
- Creator, journey 5
- Researcher
Question the model. Find the evidence.
Researcher is planned. It will focus on testing what a model can and cannot do, explaining its answers, finding its limits and reporting what was found. The evidence tools it builds on already run inside Maker.
- Who it is for
- Learners who have built and tested models and want to check claims about them.
- What learners will build
- Experiments with a question, a method and a written finding, including its limits.
- What you need first
- Experience training and testing models in Junior or Maker.
- Available now?
- No. Researcher is planned, with no launch date. Its tools run inside Maker now.
For learners who ask “how do you know?”
Every earlier journey builds models. Researcher is about trusting them only as far as the evidence allows.
What makes this stage different
The work is not making a model better. It is finding out what a model really does: where it fails, why it answered, what its confidence means, and what the evidence does not show.
Planned The aims below are for the Researcher journey. The next section shows the evidence tools already inside Maker.
Tests a claim
Turns “the model works” into a question that an experiment can answer.
Reads the evidence
Uses scores on new examples, mix-up tables and X-Rays as evidence, not decoration.
Finds limits
Shows where a model fails, including inputs it was never taught.
Reports honestly
Writes a finding with its method and its limits, and claims no more than the evidence shows.
The evidence tools inside Maker
Real screens from Maker, which is open now.
Learning or memorising?
Every brain reports its score on held-back examples beside its score on the ones it studied. A big gap is evidence of memorising.
Where the model looked
Every brain has an X-Ray drawn from its real numbers. Picture brains add a heat map of what mattered, so a learner can see when it looked at the background.
An answer you can trace
Some models explain themselves by design. Neighbours shows the rows that voted; the decision tree writes out every question it asks.
Read what it learned
The Trial Brain learns from rewards, and its whole policy is a table a learner can read, row by row.
Question, test, evidence, report
The planned shape of every Researcher project.
Planned
Question
Turn a claim into something testable: does it work on things it was never taught?
Test
Design a fair test: new examples, one change at a time.
Evidence
Collect scores, mix-ups and X-Rays as evidence.
Report
Write what was found, how, and where the finding stops.
A real finding: shown a car, it says “Square”
- What the learner tests
- A Picture Brain trained only on circles, squares, triangles and stars.
- What happens
- Shown a toy car, it answers Square, 100% sure. It has no answer for “none of these”.
- What they must understand
- Confidence is how strongly a model leans among the answers it knows, not whether it is right.
A findings card
- Question
- Is the shape model fooled by things it was never taught?
- What we did
- Showed it a toy car, which is not one of its four shapes.
- What we found
- “Square, 100% sure.” It has no way to say “none of these”.
- Limits
- One object, one camera, one room. Try more before claiming more.
A way to write up an experiment. The finding is real: see the screen beside it.
Ideas Researcher is designed to teach
Confidence is not correctness
A model can be 100% sure and wrong, especially on things it was never taught.
Test data must stay apart
If near-identical examples leak from training into testing, the score is too good to be true.
One number hides failures
A good average can hide one class that almost always fails.
Explanations show where, not why
A heat map shows where a model looked. It does not prove the model understood.
Models reflect their data
A model trained on one kind of example can fail on people or things unlike them.
Findings have limits
Say what was tested, and do not claim more than that.
How learners test their own claims
Researchers do not improve the model first. They improve the test.
Write the claim down
“It recognises stars” becomes “it labels new star photos correctly most of the time”.
Try to break it
Show it things it was never taught, unusual angles, other lighting, other people.
Change one thing
Keep everything else the same so the result has one cause.
Report the limits
List what was not tested. That list is part of the finding.
Why this matters outside the classroom
Checking medical AI
Hospitals test AI tools on patients the tools never saw before trusting them.
Fairness checks
Teams measure whether a model works equally well for different groups of people.
Model reports
Many AI teams publish what a model was tested on and where it fails.
Where Researcher sits on the path
Researcher is the last of the six journeys. It asks the most of a learner: experience building models, and the habit of asking how we know.
- Before
Creator Planned
Language: how a computer reads and writes.
- This journey
Researcher Planned
undefined
- Next
Your own questions
After Researcher, learners take on open projects of their own, with the same tools.
Available today, and planned
We are careful here: many explainability methods exist, and only some are built.
Inside Maker today
- Honest scores
- Scores on held-back examples, beside the score on studied ones, for every brain.
- What it mixed up
- A table of which class was confused with which, on new examples.
- X-Ray and Brain views
- All 19 brains, drawn from their real numbers, and the inputs pushing hardest right now.
- Readable models
- A decision tree written out, neighbours that vote, and a practice table.
Planned
- Fairness tests
- Comparing how well a model works for different groups. Not built.
- Attribution methods such as SHAP
- Standard ways to measure each input’s share of an answer. Not built.
- Findings reports
- A template for writing up a finding, its method and its limits. Not built.
Planned, with no launch date yet
Leave your email and we will write once, when Researcher opens. The evidence tools are in Maker now.
Get told when Researcher opens
Start inside Maker now
Train any brain in Maker, then read its two scores, its mix-up table and its X-Ray. Maker is open and free during early access.
Questions about Researcher
When will Researcher open?
Does EagLearn have explainability tools such as SHAP?
Can learners do research now?
Who is Researcher for?
Trust a model as far as the evidence goes.
Researcher is planned. Its tools are inside Maker now.