All stories
5 min read

Reading homework without inventing answers

Engineering lessons from worksheet image analysis: separating reading from scoring, preserving uncertainty and checking what the model actually saw.

AIDevelopmentEducation
A cobalt magnifying glass examines abstract handwritten marks on an ivory worksheet, with an orange pencil and checking symbols on a lavender background.

A parent takes a photo of a completed worksheet and uploads it. The application reads the answers, checks the work and returns useful feedback.

That is a short description of a surprisingly awkward engineering problem.

Worksheet image analysis sits between something very ordinary — a child writing on paper — and something that needs to be dependable: feedback about their work. This story uses a generalised workflow and fictional examples to explore the engineering challenges.

A central challenge is getting the system to report exactly what is on the page. Even when the model knows a better answer.

Give reading a separate job

Consider a made-up example: a worksheet asks for 7 × 8, and the child writes 54.

A model can recognise the calculation and produce 56. That would be a perfectly good answer to the maths question and a completely unhelpful transcription of the child’s work.

A useful design separates those responsibilities. The vision step reads visible answers, working and selections. Application code then compares those readings with the worksheet’s answer key. A separate feedback step can explain an answer that needs attention.

That separation makes failures easier to investigate. Did the system misread a digit, match the answer to the wrong question, or compare it incorrectly? Each is a different problem with a different fix.

It also gives the model a narrower task: read the work, including mistakes and blanks, without repairing it.

Find the right sheet first

Before reading answers, the upload needs to be connected to the right homework record.

A submission needs enough context to identify the intended worksheet and page. The system also needs to establish its orientation. A clear photograph can still produce a useless result if it is checked against the wrong page’s answers.

Orientation needs more thought than simply finding the top edge of the paper. The worksheet content itself can be rotated within the printed page. Validating the direction of the worksheet body allows the image to be rotated before transcription.

When orientation cannot be established, the result can be held for human verification. Proceeding with a confident-looking result would hide the problem rather than resolve it.

Handwriting has its own rules

Children’s handwriting is wonderfully unconcerned with the expectations of an image-processing pipeline.

Digits have uneven spacing. Minus signs are small. An answer might sit beside the question, underneath it, or outside the intended box. A crossed-out attempt can leave several plausible readings.

For example, a gap between digits is not evidence of a decimal point. A decimal needs a visible inked dot in the appropriate place. Otherwise the reader can turn spacing into a value the child never wrote.

Worksheets also use more than written numbers. Children may circle an option, shade a region, join a question to an answer, or copy a code into a box. The reader has to follow those marks. A line selecting the wrong value is still evidence of the child’s answer.

Riddle worksheets make this especially interesting. A recognisable phrase can tempt the model to complete or correct it. Reading individual code cells helps preserve the actual letters, including copying mistakes and empty positions.

These details are why a generic instruction to “read and mark this homework” leaves too much room for interpretation.

Let uncertainty survive the pipeline

Reading output can distinguish clear, unclear and blank answers. Those states matter beyond the model response.

One approach is to take multiple readings and reconcile them. Disputed or poorly read answers can receive a targeted rescan, sometimes using a closer crop. On dense worksheets, isolating answer regions can reduce distracting context.

Two readings agreeing is useful evidence, but it is not proof. Both can make the same mistake. A third reading also needs rules about how it relates to the earlier evidence; it cannot simply become authoritative because it happened last.

Structured responses and schema validation keep the output usable by the application. They do not establish that a handwritten digit was read correctly. That uncertainty needs its own representation.

A family-facing interface needs distinct states for processing, failure and answers needing checking. Human review should also be represented separately from automated marking. An automated result should not quietly acquire the meaning of a tutor having checked the work.

Feedback deserves a smaller scope too

A feedback stage can work from the marked results, with a narrower task than reinterpreting the whole photograph.

Its job is to produce a short, useful note. Validating the response includes checking that notes refer to the requested answers. If feedback generation fails, preserving the existing results allows the application to recover without losing completed work.

That boundary matters. Generating an explanation should not silently change the evidence or the scoring decision underneath it.

Measure the mistakes that matter

Evaluation needs to separate transcription, marking and feedback. Test cases should include awkward photographs: rotated pages, low light, blur and crumpled sheets.

An overall success rate hides important differences. Reading an answer from a blank space and marking an incorrect answer as correct deserve their own failure categories. Cases that fail to run also need to be reported separately, so an incomplete evaluation cannot make the pass rate look better by dropping difficult examples.

Approaches that regress results and worksheet types with remaining reliability problems belong in the evaluation record too. More prompts and more model calls do not automatically produce a dependable result.

The boundaries around an AI feature deserve as much attention as the model itself: what evidence each step receives, what it is allowed to decide, and what happens when it cannot decide reliably.

For homework, the useful outcome is feedback that stays connected to the child’s actual work — with enough honesty about uncertainty for a person to check what still needs attention.

Cover: original AI-generated editorial illustration.

Adam Spice

Developer. Curious human.

More stories