Persist the paper's question-layout signals (A-level Section A/B, EITHER/OR choice
groups, marks confidence) onto exam_templates.extraction_meta so the setup UI can
show them. Data was carried by the contract but dropped at persistence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GruxHXxfdp4kZCgAMVFgvV
The full-replace save dropped the rich region meta (figure name/description, OMR
geometry). Add meta to ResponseAreaPayload + the replace insert so a named context
figure survives a round-trip. Pairs with the app carrying name/description through.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GruxHXxfdp4kZCgAMVFgvV
_map_service_contract_to_rows now carries the two v2 fields onto exam_questions
ghost rows (command_word on leaf parts, preamble jsonb). Requires supabase
migration 78; applied to dev .94.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GruxHXxfdp4kZCgAMVFgvV
_run_service_extract_merge now records extraction_meta {engine, slug, audit
(cover-total reconciliation), counts} on the template (migration 76) — a durable
trust signal the setup UI shows. New GET /templates/{id}/digital-text proxies the
service's /api/replica for the paper (resolved via extraction_meta.slug), returning
the digital-replica markdown. exam_extract.get_replica client added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The prior P2 merge lost the templates.py + compose changes (a git reset --hard in
the commit sequence discarded the tracked-file edits; only the two new files survived).
This re-applies: exam_extract import + _frac_box/_frac_y canvas adapters +
_extract_slug + _map_service_contract_to_rows + _run_service_extract_merge/_job +
the auto_map_template routing, and the EXAM_EXTRACT_URL compose env.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When EXAM_EXTRACT_URL is set, POST /templates/{id}/auto-map now runs the spike's
full recognition pipeline (via the P1 service) instead of the thin first-pass, as
an async job. New modules/services/exam_extract.py (HTTP client: POST paper, poll,
return the analyse contract). templates.py: _map_service_contract_to_rows adapts the
page-fraction analyse contract onto the app's 780-wide stacked canvas (fraction ×
rendered dims + page_top), re-namespaces the service's uuid5s per template (stable
re-map, no cross-template collision), and is FK-safe (drops orphan regions, de-parents
dangling parents). Rich meta (OMR boxes, select_n, unit/quantity, n_options) persists
to the new exam_response_areas.meta column (migration 75). Typed response_form + mcq
answer_type + kind:context reference links map natively (schema already allows them).
The thin path is preserved as fallback when EXAM_EXTRACT_URL is unset. Unit test
covers the mapping (coord conversion, id remap, FK-safety, meta passthrough).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GET /api/exam/corpus — read-only view over the seeded eb_specifications + eb_exams
catalogue: board → subject → spec → papers grouped by paper_code+session, each
showing which of QP/MS/ER exist AND are stored, with per-spec + global rollups
(specs, papers, sessions, QP/MS/ER counts). Surfaces the collected exam bank so
the app can display its state — e.g. AQA GCSE Physics 8463: 21 QP / 12 MS / 17 ER.
Validated against the live catalogue (60 specs, 1178 docs, 535 papers) + a unit
test (grouping, QP/MS/ER presence, stored-vs-catalogued). As-user (catalogue is
public reference data).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GET /api/exam/bank — leaf questions across the caller's institute templates
(RLS-scoped) with source-paper context + spec_ref/subject filters and facets;
the spec-planning surface for build-your-own.
POST /api/exam/custom-papers {title, subject?, question_ids[]} — copy-on-assemble:
create a new exam_template owned by the caller and insert COPIES of the selected
leaf questions (new ids, requested order, carrying marks/answer_type/spec_ref/
bounds + their response areas), then project to Neo4j. A custom paper is a normal
template, so it flows through setup/marking/results/projection unchanged. The
shared-question decouple is Phase-2 (design: ~/cc/ideas/2026-07-02-mode3-...).
Tests (4, pass in-container): bank lists leaves + facets (excludes containers),
spec_ref filter, custom-paper copies in order with fresh ids, 400/404 guards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
batches.py upsert_mark previously never advanced a submission or batch to
'complete', and never validated an award against the question's max.
- Reject (422) an awarded_marks that exceeds the question's max_marks — only when
a max is actually set (0/None = not-yet-scored AI/unmapped question, unvalidatable).
- _advance_completion: a submission with a mark for every markable (leaf) question
→ 'complete'; a batch whose every non-absent submission is complete → 'complete'.
Container questions and absent students are excluded; the helper only promotes,
never regresses, so it is safe on every upsert.
Adds test_upsert_mark_rejects_over_max and test_upsert_mark_completes_submission_and_batch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The board grammar already parses per-part marks ([N marks] / "Total for Question
N is M marks"), but they were dropped building the first-pass template and the
row mapper hardcoded max_marks=0 — so every born-digital paper came back with
zero marks for a teacher to re-key by hand.
Thread the parsed mark through: bands.py derive_bands now carries each part's
`marks`, template.build passes it into part_bands, and _map_first_pass_to_rows
reads it via a defensive _safe_marks (non-negative int; unknown/None -> 0, so
image-only OCR — which has no marks yet — is unchanged). Containers still roll up
from parts. Adds test_auto_map_surfaces_born_digital_part_marks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Only AQA GCSE Physics 8463 (4.1-4.8) was seeded, so any other board/spec — or
any spec_ref that didn't match those 8 — projected zero (:Part)-[:ASSESSES]->
(:SpecPoint) edges, silently. Seed the full top-level topic catalogue for the 6
current test specs (GCSE + A-level Physics/Chemistry/Biology, 44 SpecPoints) and
loop the Specification/SpecPoint MERGE over all of them (board created once).
Idempotent; deterministic uuid5 keys unchanged. Sub-point granularity remains a
later data task. Run: python3 -c "from run.initialization.init_exam_graph import
init; import json; print(json.dumps(init()))" in the ccapi container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Only the born-digital fast path enqueued project_template_safe; the async OCR
job (_run_auto_map_job) — where image-only papers, R3's primary target, are
routed — never projected, so those papers never reached the graph after
auto-map. Project at the end of the async job too.
Adds test_auto_map_ocr_path_projects_to_neo4j.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_refresh_ai_rows deleted only unconfirmed AI rows, then bulk-inserted freshly
generated rows keyed by a deterministic uuid5 of (template_id, semantic key).
A confirmed ghost keeps that id, so a re-run re-emitted it → primary-key
conflict that failed the whole insert batch (reachable: auto-map is blocked only
when marks exist, not when ghosts are confirmed).
Fix: after the delete, read the ids that survived (confirmed-AI + manual) and
skip re-inserting any freshly generated row whose id matches — preserving the
teacher's curated row and making the insert PK-safe.
Adds test_auto_map_rerun_after_confirm_does_not_pk_collide, which confirms a
ghost at its REAL generated id (the prior test used an arbitrary id that never
collided) and asserts the row survives exactly once with confirmed=True.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The app sets canvas.height = Math.ceil(viewport.height) per page and stacks pages by those
heights; the backend page_top used the raw float, so it fell ~1px/page short, compounding to a
visible upward shape shift on later pages (~36px over 40 pages). Ceil rendered_h to match exactly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_pdf_page_geometry left rendered_w/h in PDF points (~595x842), but the app renders each PDF
page at PAGE_WIDTH=780 with proportional height and places shapes at the raw bounds. Result:
every detected region rendered shrunk (~0.76x) and shifted up-left. Set rendered_w=780 +
rendered_h=780*aspect (matches pdfLoader + pageGeometryFromImages), and scale px/point TOPLEFT
boxes into that space (was a hardcoded 0.5). Path-2 point boxes auto-correct via rendered_w/page_pt_w.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On papers where band detection yields few/no questions but opencv/gemma still emit response
regions, those regions referenced a synthetic default_qid that was never inserted -> FK violation
(exam_response_areas/exam_boundaries -> exam_questions). Ensure the fallback container question
exists and reattach orphan child rows to it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
B1-4 live-route validation: continuation bands re-emit the same stable AI id for
response_areas/boundaries/layout (not just questions), causing duplicate-pkey insert
failures. Add _dedupe_rows_by_id applied to all four tables in _refresh_ai_rows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Eval harness for AQA A-level + GCSE-science image-only papers: finalize.py --b1-only,
RapidOCR runner (rapid_pass.py via dsync), GT fixtures (make_b1_gt.py + b1_gt_labels.json),
and fetch_b1_corpus.py to pull the eval corpus from .94 cc.examboards at runtime.
Salvaged from t_15be12ed (which timed out on iteration budget re-running OCR): exam PDFs and
generated OCR caches/reports are NOT committed (third-party copyright + reproducible) — gitignored
and fetched/generated at runtime. Baseline coverage recorded in the task evidence file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
t_d1600327 added a standalone scope=user-subset, but a full reset (scope=all)
and scope=exam-corpus still left the --user-subset cc.users storage objects
orphaned (files rows are wiped by the table clear, but the Storage API objects
are not). Call the same _clear_user_subset_files() helper in both paths so the
finding-#2 gap is fully closed: storage removed before rows, idempotent.
Closes overwatch review finding #2 (user-subset not cleaned by reset).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
/admin/reset and reset_environment.reset() act on os.environ['SUPABASE_URL'].
A platform-admin call on a prod-deployed API would wipe prod data + exam
corpus + storage. Refuse when the target matches a known prod marker
(.156 / supabase.classroomcopilot) unless RESET_ALLOW_PROD=1 is set.
Addresses overwatch review finding #1 on feature/exam-seeding-overhaul.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the seed_exam_corpus.py skeleton TODOs against the real APIs and
fills the public exam corpus from official board sources.
Loader (run/initialization/seed_exam_corpus.py):
- _resolve_source_bytes: local path | url: fetch with on-disk cache + PDF validation
- upload_file: real StorageAdmin.upload_file, skip-if-exists+sha256 unless --force
- upsert_specification/upsert_paper: real upserts on spec_code/exam_code.
Fix: QP/MS/INSERT/ER role -> eb_exams.type_code; doc_type set to 'pdf'
(doc_type is CHECK-constrained to file formats; the skeleton wrote the role there).
- copy_user_test_subset: copy a QP subset into a test user's cc.users exam space + files rows
- first_sweep: auto_map + the /auto-map row mapper over seeded QPs -> system-owned
exam_templates + questions/response_areas/boundaries/layout (idempotent)
- identity discovery via institute_memberships.profile_id
Manifest (run/initialization/manifests/):
- exam-corpus.yaml: 505 papers / 18 specs / AQA+Edexcel+OCR, every source URL HEAD-verified.
AQA sciences GCSE 8461/8462/8463/8464 + AS/A-level 7401-7408, sessions JUN18-JUN24, QP+MS+ER, F+H.
- generate_corpus_manifest.py: regenerates + re-verifies all URLs from official hosts.
seed_curriculum.py: deprecation banner -> superseded by seed_exam_corpus.py; storage_loc
standardised on cc.examboards.
Verified on dev .94: full 505-paper seed (eb_specifications=18, eb_exams=505, QP=211),
idempotent re-runs, first-sweep + user-subset, 6/6 buckets provisioned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add 'exam-corpus' INIT_MODE: docker-entrypoint.sh case -> main.py --mode
exam-corpus -> run_exam_corpus_mode() -> seed_exam_corpus.load(). Driven by
EXAM_CORPUS_MANIFEST (+ DRY_RUN/FORCE/BOARD/SPEC/USER_SUBSET/FIRST_SWEEP env).
Skips gracefully (success) when no manifest is configured, so it is safe in a
comma list like INIT_MODE=infra,seed,exam-corpus before papers are gathered.
Bucket provisioning stays in infra mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
infra (buckets.py): add cc.public / cc.institutes / cc.admin to the bucket
provisioner alongside cc.examboards; make initialize_buckets idempotent
(already-exists treated as success). Bucket provisioning stays in infra init.
new (seed_exam_corpus.py): manifest-driven loader scaffold that USES the buckets
(does not create them) — validate -> upload to cc.examboards (canonical path) ->
upsert eb_specifications/eb_exams -> optional user test subset -> optional
--first-sweep auto-map pass. TODOs marked for the gathering task to complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get_class selected a non-existent enrollment_requests.created_at column,
causing a PostgREST 42703 -> 500 on /database/timetable/classes/{id}
(class detail / ResultsWidget). The table column is requested_at.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The auto-map endpoint returns dict (sync 200) or JSONResponse (202 async OCR);
FastAPI cannot build a response model from that Union. Fixes import-time
FastAPIError introduced with the S5-2 endpoint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add synthesize_part_box() as the single authoritative S5 part-box projection
(T3 swap point): content-margin x-extent x part-band y-extent, BOTTOMLEFT
coords; label_box retained as a separate anchor. build() attaches box per part.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
