Compare commits

...
Author SHA1 Message Date
kcar d3d0639f44 Merge exam-bank corpus coverage endpoint
api-ci-deploy / test-build-deploy (push) Has been cancelled
2026-07-02 22:33:07 +00:00
kcarandClaude Opus 4.8 52801a5d34 Exam bank: corpus coverage endpoint (state of the collected exam bank)
GET /api/exam/corpus — read-only view over the seeded eb_specifications + eb_exams
catalogue: board → subject → spec → papers grouped by paper_code+session, each
showing which of QP/MS/ER exist AND are stored, with per-spec + global rollups
(specs, papers, sessions, QP/MS/ER counts). Surfaces the collected exam bank so
the app can display its state — e.g. AQA GCSE Physics 8463: 21 QP / 12 MS / 17 ER.

Validated against the live catalogue (60 specs, 1178 docs, 535 papers) + a unit
test (grouping, QP/MS/ER presence, stored-vs-catalogued). As-user (catalogue is
public reference data).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-02 22:33:07 +00:00
kcar 6a6d12875d Merge mode-3 question bank + custom-paper endpoints
api-ci-deploy / test-build-deploy (push) Has been cancelled
2026-07-02 22:05:39 +00:00
kcarandClaude Opus 4.8 df827b990c Mode-3: question bank + custom-paper assembly endpoints
GET /api/exam/bank — leaf questions across the caller's institute templates
(RLS-scoped) with source-paper context + spec_ref/subject filters and facets;
the spec-planning surface for build-your-own.

POST /api/exam/custom-papers {title, subject?, question_ids[]} — copy-on-assemble:
create a new exam_template owned by the caller and insert COPIES of the selected
leaf questions (new ids, requested order, carrying marks/answer_type/spec_ref/
bounds + their response areas), then project to Neo4j. A custom paper is a normal
template, so it flows through setup/marking/results/projection unchanged. The
shared-question decouple is Phase-2 (design: ~/cc/ideas/2026-07-02-mode3-...).

Tests (4, pass in-container): bank lists leaves + facets (excludes containers),
spec_ref filter, custom-paper copies in order with fresh ids, 400/404 guards.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-02 22:05:39 +00:00
kcar 4f66fb83ce Merge fx-3-image-only-projection
api-ci-deploy / test-build-deploy (push) Has been cancelled
2026-07-02 21:48:08 +00:00
kcar a0f77333b9 Merge fx-2-born-digital-marks 2026-07-02 21:48:08 +00:00
kcar e4b0477b77 Merge fx-1-auto-map-pk 2026-07-02 21:48:08 +00:00
kcar e95b148e0f Merge FX-7: marking completion status + max_marks validation 2026-07-02 21:48:01 +00:00
kcar b0b59077bc Merge FX-6: seed SpecPoint catalogue for all 6 test specs 2026-07-02 21:48:01 +00:00
CC WorkerandClaude Opus 4.8 931e254b93 FX-7: marking completion status + max_marks validation
batches.py upsert_mark previously never advanced a submission or batch to
'complete', and never validated an award against the question's max.

- Reject (422) an awarded_marks that exceeds the question's max_marks — only when
  a max is actually set (0/None = not-yet-scored AI/unmapped question, unvalidatable).
- _advance_completion: a submission with a mark for every markable (leaf) question
  → 'complete'; a batch whose every non-absent submission is complete → 'complete'.
  Container questions and absent students are excluded; the helper only promotes,
  never regresses, so it is safe on every upsert.

Adds test_upsert_mark_rejects_over_max and test_upsert_mark_completes_submission_and_batch.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-02 21:35:24 +00:00
CC WorkerandClaude Opus 4.8 cce46305c9 FX-2: surface born-digital per-part marks from auto-map
The board grammar already parses per-part marks ([N marks] / "Total for Question
N is M marks"), but they were dropped building the first-pass template and the
row mapper hardcoded max_marks=0 — so every born-digital paper came back with
zero marks for a teacher to re-key by hand.

Thread the parsed mark through: bands.py derive_bands now carries each part's
`marks`, template.build passes it into part_bands, and _map_first_pass_to_rows
reads it via a defensive _safe_marks (non-negative int; unknown/None -> 0, so
image-only OCR — which has no marks yet — is unchanged). Containers still roll up
from parts. Adds test_auto_map_surfaces_born_digital_part_marks.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-02 21:30:28 +00:00
CC WorkerandClaude Opus 4.8 41614b78ef FX-3: project image-only papers to Neo4j after auto-map
Only the born-digital fast path enqueued project_template_safe; the async OCR
job (_run_auto_map_job) — where image-only papers, R3's primary target, are
routed — never projected, so those papers never reached the graph after
auto-map. Project at the end of the async job too.

Adds test_auto_map_ocr_path_projects_to_neo4j.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-02 21:22:54 +00:00
CC WorkerandClaude Opus 4.8 1671518ca8 FX-1: auto-map re-run must not PK-collide with confirmed ghosts
_refresh_ai_rows deleted only unconfirmed AI rows, then bulk-inserted freshly
generated rows keyed by a deterministic uuid5 of (template_id, semantic key).
A confirmed ghost keeps that id, so a re-run re-emitted it → primary-key
conflict that failed the whole insert batch (reachable: auto-map is blocked only
when marks exist, not when ghosts are confirmed).

Fix: after the delete, read the ids that survived (confirmed-AI + manual) and
skip re-inserting any freshly generated row whose id matches — preserving the
teacher's curated row and making the insert PK-safe.

Adds test_auto_map_rerun_after_confirm_does_not_pk_collide, which confirms a
ghost at its REAL generated id (the prior test used an arbitrary id that never
collided) and asserts the row survives exactly once with confirmed=True.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-02 21:02:10 +00:00
11 changed files with 577 additions and 3 deletions
+4 -1
View File
@@ -67,11 +67,13 @@ def derive_bands(result, doc=None, rapid_glob=None):
topnum = _topnumber_boxes(docs)
# gather parts with geometry, grouped by page
by_page = defaultdict(list) # page -> [(q, label, t, b)]
part_marks = {} # (question, part label) -> parsed marks (born-digital grammar)
for q in result.get("questions", []):
for p in q["parts"]:
bb, pg = p.get("bbox"), p.get("page")
if bb and pg:
by_page[pg].append((q["question"], p["label"], bb["t"], bb["b"]))
part_marks[(q["question"], p["label"])] = p.get("marks")
# global first page each question appears on (to mark the true start vs continuation pages)
q_first_page = {}
@@ -104,7 +106,8 @@ def derive_bands(result, doc=None, rapid_glob=None):
for (q, lab), st, en, _ in _ends(part_items):
qen = main_band.get(q, (st, 0))[1] # don't run past the question end
part.append({"label": lab, "question": q,
"y_start": round(st, 1), "y_end": round(max(en, qen), 1)})
"y_start": round(st, 1), "y_end": round(max(en, qen), 1),
"marks": part_marks.get((q, lab))})
pages[pg] = {"main": main, "part": part}
return {"board": result.get("board"), "paper_code": result.get("paper_code"),
+1
View File
@@ -159,6 +159,7 @@ def build(structured, bands, furniture, pdf=None, page_roles=None):
"y_start": p["y_start"], "y_end": p["y_end"],
"label_box": part_bbox.get(p["label"]), # anchor, not the part extent
"box": synthesize_part_box(p, xband),
"marks": p.get("marks"), # parsed per-part marks (born-digital)
"source": "auto", "confirmed": False,
})
pr = page_roles.get(pgs) or page_roles.get(pg) or {}
+4
View File
@@ -8,9 +8,13 @@ from fastapi import APIRouter
from routers.exam.templates import router as templates_router
from routers.exam.batches import router as batches_router
from routers.exam.bank import router as bank_router
from routers.exam.corpus import router as corpus_router
router = APIRouter()
router.include_router(templates_router)
router.include_router(batches_router)
router.include_router(bank_router)
router.include_router(corpus_router)
__all__ = ["router"]
+152
View File
@@ -0,0 +1,152 @@
"""Question bank + custom-paper assembly (/api/exam/bank, /api/exam/custom-papers) — mode 3.
Build-your-own-from-the-spec. The BANK is a browsable index over the leaf questions across the templates
the caller can see (RLS-scoped, institute-wide); a CUSTOM PAPER is a normal exam_template whose questions
are COPIES of the selected bank questions, so it flows through the existing setup / marking / projection
pipeline unchanged. Copy-on-assemble (v1) avoids decoupling question identity from a template; a shared-
question model is Phase-2. Design: ~/cc/ideas/2026-07-02-mode3-question-bank-design.md.
All access is as-the-user (E1): the bank query returns only questions in templates RLS lets the caller see,
and a custom paper is created owned by the caller + institute-scoped (writes pass the same RLS with-check).
"""
from __future__ import annotations
import os
import uuid
from typing import Any, Dict, List, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from modules.database.services.exam_projection import project_template_safe
from modules.logger_tool import initialise_logger
from routers.exam.dependencies import ExamContext, get_exam_context
logger = initialise_logger(__name__, os.getenv("LOG_LEVEL"), os.getenv("LOG_PATH"), "default", True)
router = APIRouter()
def _rows(result: Any) -> List[Dict[str, Any]]:
data = getattr(result, "data", None)
if not data:
return []
return data if isinstance(data, list) else [data]
def _first(result: Any) -> Optional[Dict[str, Any]]:
rows = _rows(result)
return rows[0] if rows else None
@router.get("/bank")
async def list_bank(
spec_ref: Optional[str] = None,
subject: Optional[str] = None,
ctx: ExamContext = Depends(get_exam_context),
) -> Dict[str, Any]:
"""Leaf questions across the caller's institute templates, with their source-paper context.
Filter by `spec_ref` (spec point) and/or `subject`. Facets over the full visible set drive the UI
filters (so filtering by one axis doesn't hide the others' options). This is the spec-planning surface:
a teacher picks a spec point → gets every question tagged it across their papers → assembles a paper.
"""
sel = (
"id, template_id, label, max_marks, answer_type, spec_ref, bounds, page, "
"exam_templates(id, title, subject, exam_code)"
)
query = ctx.supabase.table("exam_questions").select(sel).eq("is_container", False)
if spec_ref:
query = query.eq("spec_ref", spec_ref)
rows = _rows(query.execute())
facets_spec: Dict[str, int] = {}
facets_subject: Dict[str, int] = {}
items: List[Dict[str, Any]] = []
for r in rows:
tmpl = r.get("exam_templates") or {}
subj = tmpl.get("subject")
if r.get("spec_ref"):
facets_spec[r["spec_ref"]] = facets_spec.get(r["spec_ref"], 0) + 1
if subj:
facets_subject[subj] = facets_subject.get(subj, 0) + 1
if subject and subj != subject:
continue
items.append({
"id": r["id"], "template_id": r["template_id"], "label": r.get("label"),
"max_marks": r.get("max_marks"), "answer_type": r.get("answer_type"),
"spec_ref": r.get("spec_ref"), "bounds": r.get("bounds"), "page": r.get("page"),
"paper": {"id": tmpl.get("id"), "title": tmpl.get("title"),
"subject": subj, "exam_code": tmpl.get("exam_code")},
})
return {"questions": items, "n": len(items),
"facets": {"spec_ref": facets_spec, "subject": facets_subject}}
class CustomPaperRequest(BaseModel):
title: str
subject: Optional[str] = None
institute_id: Optional[str] = None
question_ids: List[str]
@router.post("/custom-papers")
async def create_custom_paper(
body: CustomPaperRequest,
ctx: ExamContext = Depends(get_exam_context),
) -> Dict[str, Any]:
"""Assemble the selected bank questions into a new template (copy-on-assemble), then project it."""
if not body.question_ids:
raise HTTPException(status_code=400, detail="question_ids is required")
institute_id = ctx.resolve_institute(body.institute_id)
# RLS makes this return only leaf questions the caller may see; unknown/forbidden ids are silently dropped.
srcs = _rows(
ctx.supabase.table("exam_questions").select("*")
.in_("id", body.question_ids).eq("is_container", False).execute()
)
by_id = {s["id"]: s for s in srcs}
ordered = [by_id[qid] for qid in body.question_ids if qid in by_id] # preserve the caller's order
if not ordered:
raise HTTPException(status_code=404, detail="No accessible questions for the given ids")
template_id = str(uuid.uuid4())
ctx.supabase.table("exam_templates").insert({
"id": template_id, "title": body.title, "subject": body.subject,
"institute_id": institute_id, "teacher_id": ctx.user_id, "status": "draft",
}).execute()
id_map: Dict[str, str] = {}
q_rows: List[Dict[str, Any]] = []
for order, s in enumerate(ordered):
new_id = str(uuid.uuid4())
id_map[s["id"]] = new_id
q_rows.append({
"id": new_id, "template_id": template_id, "parent_id": None,
"label": s.get("label"), "order": order, "max_marks": s.get("max_marks") or 0,
"answer_type": s.get("answer_type"), "mcq_options": s.get("mcq_options"),
"mark_scheme": s.get("mark_scheme") or {}, "is_container": False,
"spec_ref": s.get("spec_ref"), "bounds": s.get("bounds"), "page": s.get("page"),
"source": "manual", "confirmed": True,
})
ctx.supabase.table("exam_questions").insert(q_rows).execute()
# Copy each selected question's response areas onto its new copy (geometry travels with the question).
ra_rows: List[Dict[str, Any]] = []
if id_map:
for ra in _rows(
ctx.supabase.table("exam_response_areas").select("*")
.in_("question_id", list(id_map.keys())).execute()
):
ra_rows.append({
"id": str(uuid.uuid4()), "question_id": id_map[ra["question_id"]], "template_id": template_id,
"page": ra.get("page"), "bounds": ra.get("bounds"), "kind": ra.get("kind"),
"response_form": ra.get("response_form"), "context_type": ra.get("context_type"),
"source": "manual", "confirmed": True,
})
if ra_rows:
ctx.supabase.table("exam_response_areas").insert(ra_rows).execute()
project_template_safe(template_id) # a custom paper is a normal paper in the graph
return {"id": template_id, "title": body.title, "subject": body.subject,
"n_questions": len(q_rows), "n_response_areas": len(ra_rows)}
+42
View File
@@ -245,6 +245,36 @@ async def batch_csv(
# ─── marks ───────────────────────────────────────────────────────────────────
def _advance_completion(ctx: ExamContext, batch_id: str, submission_id: str) -> None:
"""After a mark upsert, advance statuses: a submission with a mark for every markable (leaf)
question → complete; a batch whose every non-absent submission is complete → complete. Nothing
here regresses a status (only promotes to complete), so it is safe to run on every upsert."""
batch = _first(
ctx.supabase.table("marking_batches").select("id, template_id, status").eq("id", batch_id).limit(1).execute()
)
if not batch:
return
markable = {
q["id"] for q in _rows(
ctx.supabase.table("exam_questions").select("id, is_container").eq("template_id", batch["template_id"]).execute()
) if not q.get("is_container")
}
if not markable:
return
marked = {
m["question_id"] for m in _rows(
ctx.supabase.table("mark_entries").select("question_id").eq("submission_id", submission_id).execute()
)
}
if not markable.issubset(marked):
return
ctx.supabase.table("student_submissions").update({"status": "complete"}).eq("id", submission_id).execute()
subs = _rows(ctx.supabase.table("student_submissions").select("status").eq("batch_id", batch_id).execute())
active = [s for s in subs if s.get("status") != "absent"]
if active and all(s.get("status") == "complete" for s in active) and batch.get("status") != "complete":
ctx.supabase.table("marking_batches").update({"status": "complete"}).eq("id", batch_id).execute()
@router.put("/marks/{mark_id}")
async def upsert_mark(
mark_id: str,
@@ -259,6 +289,15 @@ async def upsert_mark(
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Reject an award that exceeds the question's max (only when a max is actually set; 0/None means
# "not scored yet" for AI/unmapped questions, so we can't validate those).
question = _first(
ctx.supabase.table("exam_questions").select("id, max_marks").eq("id", body.question_id).limit(1).execute()
)
max_marks = (question or {}).get("max_marks")
if isinstance(max_marks, (int, float)) and max_marks > 0 and body.awarded_marks is not None and body.awarded_marks > max_marks:
raise HTTPException(status_code=422, detail=f"awarded_marks {body.awarded_marks} exceeds max_marks {max_marks} for this question")
row = {
"id": mark_id,
"submission_id": body.submission_id,
@@ -285,6 +324,9 @@ async def upsert_mark(
if submission.get("status") in ("absent", "unmatched"):
ctx.supabase.table("student_submissions").update({"status": "marking"}).eq("id", body.submission_id).execute()
# Promote the submission/batch to complete once every markable question has a mark.
_advance_completion(ctx, submission["batch_id"], body.submission_id)
return upserted
+98
View File
@@ -0,0 +1,98 @@
"""Exam-bank corpus coverage (/api/exam/corpus) — the state of the collected exam bank.
Read-only view over the seeded exam-board catalogue (eb_specifications + eb_exams): board → subject →
specification → papers, with per-session QP/MS/ER coverage and rollup counts. Shows what the app has
COLLECTED (question papers, mark schemes, examiner reports) so a teacher/admin can see the bank's state
and where science coverage is complete vs thin. Catalogue data is public reference — read as-the-user.
"""
from __future__ import annotations
import os
import re
from typing import Any, Dict, List, Optional
from fastapi import APIRouter, Depends
from modules.logger_tool import initialise_logger
from routers.exam.dependencies import ExamContext, get_exam_context
logger = initialise_logger(__name__, os.getenv("LOG_LEVEL"), os.getenv("LOG_PATH"), "default", True)
router = APIRouter()
DOC_TYPES = ("QP", "MS", "ER")
def _rows(result: Any) -> List[Dict[str, Any]]:
data = getattr(result, "data", None)
if not data:
return []
return data if isinstance(data, list) else [data]
def _award_level(spec: Dict[str, Any]) -> str:
"""Best-effort GCSE / AS / A-level from the spec code (AQA GCSE = 8xxx, A-level/AS = 7xxx)."""
code = re.sub(r"\D", "", spec.get("award_code") or spec.get("spec_code") or "")
if code.startswith("8"):
return "GCSE"
if code.startswith("7"):
return "A-level"
return spec.get("award_level") or "Other"
@router.get("/corpus")
async def corpus_coverage(ctx: ExamContext = Depends(get_exam_context)) -> Dict[str, Any]:
specs = _rows(
ctx.supabase.table("eb_specifications")
.select("spec_code, exam_board_code, subject_code, award_code, first_teach").execute()
)
exams = _rows(
ctx.supabase.table("eb_exams")
.select("exam_code, spec_code, paper_code, tier, session, type_code, storage_loc").execute()
)
# group exam docs → per spec → per paper (paper_code + session) → which doc types are present
by_spec: Dict[str, Dict[str, Dict[str, Any]]] = {}
for e in exams:
sc = e.get("spec_code")
if not sc:
continue
key = f"{e.get('paper_code') or '?'}|{e.get('session') or '?'}"
paper = by_spec.setdefault(sc, {}).setdefault(key, {
"paper_code": e.get("paper_code"), "session": e.get("session"),
"tier": e.get("tier"), "docs": {}, "exam_codes": {},
})
dt = (e.get("type_code") or "").upper()
if dt in DOC_TYPES:
paper["docs"][dt] = bool(e.get("storage_loc"))
paper["exam_codes"][dt] = e.get("exam_code")
totals = {"specs": 0, "papers": 0, "sessions": 0, **{d: 0 for d in DOC_TYPES}}
boards: Dict[str, Dict[str, Any]] = {}
for s in specs:
sc = s["spec_code"]
papers_map = by_spec.get(sc, {})
if not papers_map:
continue
totals["specs"] += 1
board = s.get("exam_board_code") or "?"
level = _award_level(s)
papers = sorted(papers_map.values(), key=lambda p: (str(p["session"]), str(p["paper_code"])))
counts = {d: sum(1 for p in papers if p["docs"].get(d)) for d in DOC_TYPES}
for d in DOC_TYPES:
totals[d] += counts[d]
totals["papers"] += len(papers)
totals["sessions"] += len({p["session"] for p in papers})
spec_entry = {
"spec_code": sc, "subject": (s.get("subject_code") or "").title(), "level": level,
"board": board, "first_teach": s.get("first_teach"),
"n_papers": len(papers), "counts": counts, "papers": papers,
}
boards.setdefault(board, {"board": board, "specs": []})["specs"].append(spec_entry)
board_list = []
for board in sorted(boards):
specs_sorted = sorted(boards[board]["specs"], key=lambda x: (x["level"], x["subject"], x["spec_code"]))
board_list.append({"board": board, "n_specs": len(specs_sorted), "specs": specs_sorted})
return {"totals": totals, "boards": board_list}
+22 -2
View File
@@ -465,6 +465,17 @@ def _safe_confidence(value: Any = None) -> float:
return 0.75
def _safe_marks(value: Any = None) -> int:
"""Parsed per-part marks → a non-negative int; unknown/None → 0 (image-only OCR has no marks yet)."""
if isinstance(value, bool):
return 0
if isinstance(value, (int, float)):
return max(0, int(value))
if isinstance(value, str) and value.strip().isdigit():
return int(value.strip())
return 0
def _margin_values(first_pass: Dict[str, Any], page_number: int) -> Dict[str, Optional[float]]:
vals: Dict[str, Optional[float]] = {"left": None, "right": None, "top": None, "bottom": None}
for m in first_pass.get("margins") or []:
@@ -541,7 +552,7 @@ def _map_first_pass_to_rows(template_id: str, first_pass: Dict[str, Any], pdf_by
top = max(float(y1), float(y2)); bottom = min(float(y1), float(y2))
bounds = _box_to_canvas({"l": margins["left"], "r": margins["right"], "t": top, "b": bottom, "coord_origin": "BOTTOMLEFT"}, page_number, pages_geom)
bounds = bounds or _box_to_canvas(band.get("label_box"), page_number, pages_geom)
questions.append({"id": pid, "template_id": template_id, "parent_id": parent_id, "label": label, "order": len(questions), "max_marks": 0, "is_container": False, "bounds": bounds, "page": page_number, "source": "ai", "confirmed": False, "confidence": _safe_confidence(band.get("confidence")), "derivation": "docling-part-band-x-margins"})
questions.append({"id": pid, "template_id": template_id, "parent_id": parent_id, "label": label, "order": len(questions), "max_marks": _safe_marks(band.get("marks")), "is_container": False, "bounds": bounds, "page": page_number, "source": "ai", "confirmed": False, "confidence": _safe_confidence(band.get("confidence")), "derivation": "docling-part-band-x-margins"})
default_qid = questions[0]["id"] if questions else _ai_id(template_id, "question", "auto")
for page_key in sorted(pages_obj, key=lambda k: int(k)):
@@ -602,7 +613,13 @@ def _refresh_ai_rows(ctx: ExamContext, template_id: str, rows: Dict[str, List[Di
for table in ("exam_response_areas", "exam_boundaries", "exam_template_layout", "exam_questions"):
sb.table(table).delete().eq("template_id", template_id).eq("source", "ai").eq("confirmed", False).execute()
for table, key in (("exam_questions", "questions"), ("exam_response_areas", "response_areas"), ("exam_boundaries", "boundaries"), ("exam_template_layout", "layout")):
payload = _dedupe_rows_by_id(rows.get(key) or [])
# A row that survived the delete above is confirmed-AI or manual — the teacher's curated version.
# AI ids are a deterministic uuid5 of (template_id, semantic key), so a re-run re-emits the SAME id
# for a confirmed ghost; re-inserting it would PK-collide and fail the whole batch. Skip those ids so
# confirmed work is preserved and the insert is safe (fixes the re-map-after-confirm collision).
kept = sb.table(table).select("id").eq("template_id", template_id).execute()
kept_ids = {str(r["id"]) for r in (getattr(kept, "data", None) or []) if isinstance(r, dict) and r.get("id")}
payload = [r for r in _dedupe_rows_by_id(rows.get(key) or []) if str(r.get("id")) not in kept_ids]
if payload:
sb.table(table).insert(payload).execute()
@@ -634,6 +651,9 @@ def _run_auto_map_job(job_id: str, ctx: ExamContext, template_id: str, pdf_bytes
_set_auto_map_status(job_id, {"status": "running", "template_id": template_id})
try:
rows = _run_auto_map_merge(ctx, template_id, pdf_bytes, source_label)
# Project to Neo4j like the born-digital fast path does — otherwise image-only papers (R3's
# primary target, routed here because they need OCR) never reach the graph after auto-map.
project_template_safe(template_id)
_set_auto_map_status(job_id, {"status": "completed", "template_id": template_id, "counts": {k: len(v) for k, v in rows.items()}})
except Exception as exc:
logger.exception(f"auto-map job failed for template {template_id}: {exc}")
+134
View File
@@ -0,0 +1,134 @@
"""Tests for the question-bank + custom-paper assembly router (/api/exam/bank, /custom-papers) — mode 3.
Same FakeSupabase + dependency-override pattern as test_exam_templates.py; the as-user RLS itself is
verified against the live dev DB (.94), not here.
"""
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
import routers.exam.bank as bank_mod
from routers.exam.bank import router
from routers.exam.dependencies import ExamContext, get_exam_context
TEACHER = "00000000-0000-0000-0000-000000000001"
INST_A = "10000000-0000-0000-0000-000000000001"
@pytest.fixture(autouse=True)
def _stub_projection(monkeypatch):
monkeypatch.setattr(bank_mod, "project_template_safe", lambda tid: None)
class FakeResult:
def __init__(self, data):
self.data = data
class FakeQuery:
def __init__(self, store, table):
self.store = store
self.table = table
self.rows = list(store.get(table, []))
self._op = None
self._payload = None
def select(self, *_a, **_k):
self._op = "select"
return self
def insert(self, payload):
self._op = "insert"
self._payload = payload
return self
def eq(self, key, value):
self.rows = [r for r in self.rows if r.get(key) == value]
return self
def in_(self, key, values):
values = set(values)
self.rows = [r for r in self.rows if r.get(key) in values]
return self
def execute(self):
backing = self.store.setdefault(self.table, [])
if self._op == "insert":
payloads = self._payload if isinstance(self._payload, list) else [self._payload]
inserted = []
for p in payloads:
row = dict(p)
row.setdefault("id", f"gen-{self.table}-{len(backing)}")
backing.append(row)
inserted.append(row)
return FakeResult(inserted)
return FakeResult(self.rows)
class FakeSupabase:
def __init__(self, store):
self.store = store
def table(self, name):
return FakeQuery(self.store, name)
def make_client(store, user_id=TEACHER, institute_ids=(INST_A,)):
app = FastAPI()
app.include_router(router, prefix="/api/exam")
app.dependency_overrides[get_exam_context] = lambda: ExamContext(user_id, "tok", FakeSupabase(store), list(institute_ids))
return TestClient(app), store
def test_bank_lists_leaf_questions_with_facets_excluding_containers():
store = {"exam_questions": [
{"id": "c1", "template_id": "t1", "label": "Q1", "is_container": True, "spec_ref": None,
"exam_templates": {"id": "t1", "title": "Physics 1H", "subject": "physics", "exam_code": "AQA-PHYS-8463"}},
{"id": "q1", "template_id": "t1", "label": "01.1", "is_container": False, "max_marks": 3, "spec_ref": "4.2",
"exam_templates": {"id": "t1", "title": "Physics 1H", "subject": "physics", "exam_code": "AQA-PHYS-8463"}},
{"id": "q2", "template_id": "t1", "label": "01.2", "is_container": False, "max_marks": 2, "spec_ref": "4.2",
"exam_templates": {"id": "t1", "title": "Physics 1H", "subject": "physics", "exam_code": "AQA-PHYS-8463"}},
]}
c, _ = make_client(store)
body = c.get("/api/exam/bank").json()
labels = {q["label"] for q in body["questions"]}
assert labels == {"01.1", "01.2"} # container excluded
assert body["facets"]["spec_ref"] == {"4.2": 2}
assert body["questions"][0]["paper"]["exam_code"] == "AQA-PHYS-8463"
def test_bank_filters_by_spec_ref():
store = {"exam_questions": [
{"id": "q1", "template_id": "t1", "label": "a", "is_container": False, "spec_ref": "4.2", "exam_templates": {"subject": "physics"}},
{"id": "q2", "template_id": "t1", "label": "b", "is_container": False, "spec_ref": "4.5", "exam_templates": {"subject": "physics"}},
]}
c, _ = make_client(store)
body = c.get("/api/exam/bank?spec_ref=4.2").json()
assert [q["id"] for q in body["questions"]] == ["q1"]
def test_custom_paper_copies_selected_questions_in_order():
store = {"exam_questions": [
{"id": "q1", "template_id": "t1", "label": "01.1", "is_container": False, "max_marks": 3, "spec_ref": "4.2", "answer_type": "short"},
{"id": "q2", "template_id": "t2", "label": "05.1", "is_container": False, "max_marks": 6, "spec_ref": "4.5", "answer_type": "written"},
]}
c, store = make_client(store)
r = c.post("/api/exam/custom-papers", json={"title": "Electricity mini-test", "subject": "physics", "question_ids": ["q2", "q1"]})
assert r.status_code == 200
body = r.json()
assert body["n_questions"] == 2
# a new template row was created, owned + institute-scoped
new_tpls = [t for t in store["exam_templates"] if t["id"] == body["id"]]
assert new_tpls and new_tpls[0]["teacher_id"] == TEACHER and new_tpls[0]["institute_id"] == INST_A
# two COPIES exist under the new template, order preserving the requested [q2, q1], with fresh ids
copies = sorted([q for q in store["exam_questions"] if q["template_id"] == body["id"]], key=lambda q: q["order"])
assert [q["label"] for q in copies] == ["05.1", "01.1"]
assert all(q["id"] not in ("q1", "q2") for q in copies) # not the originals
assert copies[0]["max_marks"] == 6 and copies[0]["spec_ref"] == "4.5"
def test_custom_paper_requires_accessible_questions():
c, _ = make_client({"exam_questions": []})
assert c.post("/api/exam/custom-papers", json={"title": "x", "question_ids": ["nope"]}).status_code == 404
assert c.post("/api/exam/custom-papers", json={"title": "x", "question_ids": []}).status_code == 400
+24
View File
@@ -249,6 +249,30 @@ def test_upsert_mark_submission_404():
assert c.put("/api/exam/marks/mk-x", json={"submission_id": "nope", "question_id": "q1", "awarded_marks": 1}).status_code == 404
def test_upsert_mark_rejects_over_max():
c = make_client(_batch_with_cohort()) # q1 max_marks = 3
r = c.put("/api/exam/marks/mk-over", json={"submission_id": "sub2", "question_id": "q1", "awarded_marks": 4})
assert r.status_code == 422
def test_upsert_mark_completes_submission_and_batch():
store = base_store(
marking_batches=[{"id": "b1", "template_id": TPL, "institute_id": INST_A, "teacher_id": TEACHER, "status": "marking"}],
exam_questions=[
{"id": "q0", "template_id": TPL, "label": "Q1", "max_marks": 0, "order": 0, "is_container": True},
{"id": "q1", "template_id": TPL, "label": "01", "max_marks": 3, "order": 1, "is_container": False},
{"id": "q2", "template_id": TPL, "label": "02", "max_marks": 5, "order": 2, "is_container": False},
],
student_submissions=[{"id": "sub1", "batch_id": "b1", "student_id": "s1", "status": "marking"}],
mark_entries=[{"id": "m1", "batch_id": "b1", "submission_id": "sub1", "question_id": "q1", "awarded_marks": 2}],
)
c = make_client(store)
# marking the last leaf question completes the submission (container q0 doesn't block) and the batch
assert c.put("/api/exam/marks/m2", json={"submission_id": "sub1", "question_id": "q2", "awarded_marks": 4}).status_code == 200
assert next(s for s in store["student_submissions"] if s["id"] == "sub1")["status"] == "complete"
assert next(b for b in store["marking_batches"] if b["id"] == "b1")["status"] == "complete"
# ─── scans (E3 guards) ───────────────────────────────────────────────────────
def _batch_store():
+48
View File
@@ -0,0 +1,48 @@
"""Test the exam-bank corpus coverage grouping (/api/exam/corpus)."""
from fastapi import FastAPI
from fastapi.testclient import TestClient
from routers.exam.corpus import router
from routers.exam.dependencies import ExamContext, get_exam_context
TEACHER = "00000000-0000-0000-0000-000000000001"
INST_A = "10000000-0000-0000-0000-000000000001"
class FakeResult:
def __init__(self, data): self.data = data
class FakeQuery:
def __init__(self, store, table): self.store, self.table = store, table
def select(self, *_a, **_k): return self
def execute(self): return FakeResult(list(self.store.get(self.table, [])))
class FakeSupabase:
def __init__(self, store): self.store = store
def table(self, name): return FakeQuery(self.store, name)
def _client(store):
app = FastAPI(); app.include_router(router, prefix="/api/exam")
app.dependency_overrides[get_exam_context] = lambda: ExamContext(TEACHER, "tok", FakeSupabase(store), [INST_A])
return TestClient(app)
def test_corpus_groups_papers_by_session_with_qp_ms_er_coverage():
store = {
"eb_specifications": [{"spec_code": "AQA-PHYS-8463", "exam_board_code": "AQA", "subject_code": "PHYSICS", "award_code": "8463", "first_teach": "2016"}],
"eb_exams": [
{"exam_code": "a", "spec_code": "AQA-PHYS-8463", "paper_code": "8463/1H", "session": "2022-Jun", "type_code": "QP", "storage_loc": "cc.examboards/a.pdf"},
{"exam_code": "b", "spec_code": "AQA-PHYS-8463", "paper_code": "8463/1H", "session": "2022-Jun", "type_code": "MS", "storage_loc": "cc.examboards/b.pdf"},
{"exam_code": "c", "spec_code": "AQA-PHYS-8463", "paper_code": "8463/1H", "session": "2022-Jun", "type_code": "ER", "storage_loc": None},
{"exam_code": "d", "spec_code": "AQA-PHYS-8463", "paper_code": "8463/2H", "session": "2023-Jun", "type_code": "QP", "storage_loc": "cc.examboards/d.pdf"},
],
}
body = _client(store).get("/api/exam/corpus").json()
assert body["totals"]["specs"] == 1 and body["totals"]["papers"] == 2
spec = body["boards"][0]["specs"][0]
assert spec["level"] == "GCSE" and spec["subject"] == "Physics"
assert spec["counts"] == {"QP": 2, "MS": 1, "ER": 0} # ER present but not stored → not counted
p = next(p for p in spec["papers"] if p["paper_code"] == "8463/1H")
assert p["docs"] == {"QP": True, "MS": True, "ER": False}
+48
View File
@@ -642,6 +642,21 @@ def test_auto_map_fast_path_merges_ai_rows_and_returns_detail(monkeypatch):
assert store["exam_boundaries"] and store["exam_boundaries"][0]["derivation"] == "docling-main-band"
def test_auto_map_surfaces_born_digital_part_marks(monkeypatch):
# Regression: the born-digital grammar parses per-part marks, but the row mapper hardcoded
# max_marks=0. A part band carrying `marks` must flow through to the question row's max_marks.
store = _template_with_source()
store.update({"exam_questions": [], "exam_response_areas": [], "exam_boundaries": [], "exam_template_layout": []})
client, store = make_client(store=store)
fp = _first_pass_template()
fp["pages"]["1"]["part_bands"][0]["marks"] = 4
_patch_auto_map(monkeypatch, store, fast=True)
monkeypatch.setattr(templates_mod, "auto_map", lambda *_a, **_k: fp) # override with the marked part band
assert client.post("/api/exam/templates/t1/auto-map").status_code == 200
part = next(q for q in store["exam_questions"] if q.get("label") == "01.1")
assert part["max_marks"] == 4
def test_auto_map_deduplicates_repeated_response_area_ids(monkeypatch):
store = _template_with_source()
client, store = make_client(store=store)
@@ -674,6 +689,27 @@ def test_auto_map_preserves_manual_and_confirmed_rows_on_rerun(monkeypatch):
assert "old-ai" not in ids
def test_auto_map_rerun_after_confirm_does_not_pk_collide(monkeypatch):
# Regression: a confirmed ghost keeps its DETERMINISTIC uuid5 id, which auto-map re-emits on a
# re-run — the old code re-inserted that id and PK-collided. Use the real generated id (not an
# arbitrary one, unlike the test above) so the collision path is actually exercised.
store = _template_with_source()
store.update({"exam_questions": [], "exam_response_areas": [], "exam_boundaries": [], "exam_template_layout": []})
client, store = make_client(store=store)
_patch_auto_map(monkeypatch, store, fast=True)
assert client.post("/api/exam/templates/t1/auto-map").status_code == 200
ghost_id = next(q["id"] for q in store["exam_questions"] if q["source"] == "ai")
# teacher confirms that ghost (row kept, deterministic id unchanged)
for q in store["exam_questions"]:
if q["id"] == ghost_id:
q["confirmed"] = True
# re-run auto-map: must not re-insert the same id, and must preserve the teacher's confirmation
assert client.post("/api/exam/templates/t1/auto-map").status_code == 200
same = [r for r in store["exam_questions"] if r["id"] == ghost_id]
assert len(same) == 1, "confirmed ghost must not be duplicated (PK collision) on re-run"
assert same[0]["confirmed"] is True, "teacher's confirmation must survive a re-run"
def test_auto_map_non_owner_is_403_before_download(monkeypatch):
store = _template_with_source(owner=OTHER_TEACHER)
client, store = make_client(user_id=TEACHER, institute_ids=(INST_A,), store=store)
@@ -718,4 +754,16 @@ def test_auto_map_ocr_returns_job_id_and_status_completes(monkeypatch):
body = status.json()
assert body["status"] == "completed"
assert body["counts"]["questions"] >= 2
def test_auto_map_ocr_path_projects_to_neo4j(monkeypatch, _stub_projection):
# Image-only papers route through the async OCR job; regression: that path must project to Neo4j
# like the born-digital fast path, or the graph is never built for the primary target.
store = _template_with_source()
client, store = make_client(store=store)
_patch_auto_map(monkeypatch, store, fast=False)
resp = client.post("/api/exam/templates/t1/auto-map")
assert resp.status_code == 202
# the BackgroundTask runs after the response under TestClient
assert "t1" in _stub_projection
assert body["template"]["layout"]