feat(seed): expand corpus to 1178 papers + download-only/unseed/granular reset

PRIMARY — corpus breadth (505->1178 papers, 18->60 specs, all URLs HEAD-verified):
- AQA (enumerated): Maths, English Lang/Lit, Geography, Computer Science, Business,
  Psychology, MFL (French/Spanish/German), GCSE + A-level, on top of round-1 sciences.
- Edexcel + OCR (confirmed direct URLs via research): Maths, English, Geography, History,
  Business, Computer Science, GCSE + A-level.
- generate_corpus_manifest.py: _subj/_mfl AQA builders, Edexcel/OCR spec+URL tables,
  derived exam_code (_mk_exam_code) matching the locked convention, concurrent re-verify.
Verified on dev .94: eb_specifications=60, eb_exams=1178, QP=469, doc_type all 'pdf',
seed idempotent (uploaded=673 new, skipped=505), failed=0.

SECONDARY:
- --download-only + persistent bucket-shaped local store (manifests/_corpus_store/, gitignored):
  download-once, seed-many, offline-repeatable; --store-dir/--no-store. (_store_path/_item_bytes/
  download_corpus). Verified: store populated, seed reads offline (download_cached).
- --unseed [--board/--spec]: inverse loader — storage objects (Storage API; protect_delete blocks
  raw SQL), first-sweep seed templates, eb_exams, eb_specifications. Verified reversible on .94.
- Granular admin reset: POST /admin/reset?scope=all|exam-corpus|timetable. reset_environment.reset(scope)
  adds EXAM_CORPUS_TABLES (10) + cc.examboards storage cleanup + TIMETABLE_TABLES (13); 'all' now also
  clears the exam subsystem the legacy reset missed. No schema migration required.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
CC Worker
2026-06-07 23:33:20 +00:00
co-authored by Claude Opus 4.8
parent 5750413f43
commit cdc105ae54
6 changed files with 8958 additions and 52 deletions
@@ -126,13 +126,24 @@ async def platform_stats(
@router.post("/reset")
async def reset_environment(
scope: str = "all",
_: dict = Depends(require_platform_admin),
) -> Dict[str, Any]:
"""DESTRUCTIVE: wipe all test data. Neo4j + Supabase. Platform admin only."""
"""DESTRUCTIVE: wipe test data. Platform admin only.
scope (query param):
- all : full wipe (Neo4j + Supabase data + auth users) AND exam subsystem + storage.
- exam-corpus : ONLY the exam corpus — eb_*/exam_* tables + cc.examboards storage objects
(load/unload the public corpus without touching schools/users).
- timetable : ONLY timetable/calendar materialization tables.
"""
if scope not in ("all", "exam-corpus", "timetable"):
raise HTTPException(status_code=400, detail="scope must be one of: all, exam-corpus, timetable")
import asyncio
import functools
from run.initialization.reset_environment import reset as _reset
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, _reset)
result = await loop.run_in_executor(None, functools.partial(_reset, scope))
return {"status": "ok", **result}