feat(seed): implement exam-corpus loader + filled 505-paper manifest
Implements the seed_exam_corpus.py skeleton TODOs against the real APIs and fills the public exam corpus from official board sources. Loader (run/initialization/seed_exam_corpus.py): - _resolve_source_bytes: local path | url: fetch with on-disk cache + PDF validation - upload_file: real StorageAdmin.upload_file, skip-if-exists+sha256 unless --force - upsert_specification/upsert_paper: real upserts on spec_code/exam_code. Fix: QP/MS/INSERT/ER role -> eb_exams.type_code; doc_type set to 'pdf' (doc_type is CHECK-constrained to file formats; the skeleton wrote the role there). - copy_user_test_subset: copy a QP subset into a test user's cc.users exam space + files rows - first_sweep: auto_map + the /auto-map row mapper over seeded QPs -> system-owned exam_templates + questions/response_areas/boundaries/layout (idempotent) - identity discovery via institute_memberships.profile_id Manifest (run/initialization/manifests/): - exam-corpus.yaml: 505 papers / 18 specs / AQA+Edexcel+OCR, every source URL HEAD-verified. AQA sciences GCSE 8461/8462/8463/8464 + AS/A-level 7401-7408, sessions JUN18-JUN24, QP+MS+ER, F+H. - generate_corpus_manifest.py: regenerates + re-verifies all URLs from official hosts. seed_curriculum.py: deprecation banner -> superseded by seed_exam_corpus.py; storage_loc standardised on cc.examboards. Verified on dev .94: full 505-paper seed (eb_specifications=18, eb_exams=505, QP=211), idempotent re-runs, first-sweep + user-subset, 6/6 buckets provisioned. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
d8cf3bbc62
commit
5750413f43
@@ -1,15 +1,20 @@
|
||||
"""
|
||||
seed_curriculum.py — Create curriculum data: exam board specifications and exams.
|
||||
seed_curriculum.py — DEPRECATED hardcoded curriculum/exam seeder.
|
||||
|
||||
Seeds eb_specifications and eb_exams tables with realistic UK exam board data
|
||||
(AQA, Edexcel, OCR) for Physics, Maths, and Computer Science across both schools.
|
||||
⚠️ SUPERSEDED (2026-06-07) by the manifest-driven corpus loader:
|
||||
run/initialization/seed_exam_corpus.py (+ manifests/exam-corpus.yaml)
|
||||
|
||||
Also seeds curriculum_topics in Neo4j for the school databases.
|
||||
The exam-board parts of this file (eb_specifications / eb_exams) are now seeded from a
|
||||
verified, provenance-bearing manifest with real uploaded PDFs — not the hardcoded rows
|
||||
below. This module also had a storage_loc inconsistency the overhaul standardises away:
|
||||
exam-board files belong in the `cc.examboards` bucket at the canonical path
|
||||
`cc.examboards/{board}/{subject}/{award}/{paper}/{session}/{role}.pdf`, NOT under
|
||||
`cc.public.snapshots/curriculum/...` (the placeholder rows below still show the old path).
|
||||
|
||||
Tables: eb_specifications, eb_exams
|
||||
Neo4j: curriculum topic nodes in school databases
|
||||
KEEP ONLY for the Neo4j `curriculum_topics` seed (step [3]) which has no replacement yet.
|
||||
Do NOT use the eb_specifications/eb_exams blocks for new work — use seed_exam_corpus.py.
|
||||
|
||||
Run inside ccapi container:
|
||||
Run (Neo4j curriculum topics only is the supported remaining use):
|
||||
python3 -c "from run.initialization.seed_curriculum import seed; seed()"
|
||||
"""
|
||||
import os
|
||||
|
||||
Reference in New Issue
Block a user