feat(seed): expand corpus to 1178 papers + download-only/unseed/granular reset
PRIMARY — corpus breadth (505->1178 papers, 18->60 specs, all URLs HEAD-verified): - AQA (enumerated): Maths, English Lang/Lit, Geography, Computer Science, Business, Psychology, MFL (French/Spanish/German), GCSE + A-level, on top of round-1 sciences. - Edexcel + OCR (confirmed direct URLs via research): Maths, English, Geography, History, Business, Computer Science, GCSE + A-level. - generate_corpus_manifest.py: _subj/_mfl AQA builders, Edexcel/OCR spec+URL tables, derived exam_code (_mk_exam_code) matching the locked convention, concurrent re-verify. Verified on dev .94: eb_specifications=60, eb_exams=1178, QP=469, doc_type all 'pdf', seed idempotent (uploaded=673 new, skipped=505), failed=0. SECONDARY: - --download-only + persistent bucket-shaped local store (manifests/_corpus_store/, gitignored): download-once, seed-many, offline-repeatable; --store-dir/--no-store. (_store_path/_item_bytes/ download_corpus). Verified: store populated, seed reads offline (download_cached). - --unseed [--board/--spec]: inverse loader — storage objects (Storage API; protect_delete blocks raw SQL), first-sweep seed templates, eb_exams, eb_specifications. Verified reversible on .94. - Granular admin reset: POST /admin/reset?scope=all|exam-corpus|timetable. reset_environment.reset(scope) adds EXAM_CORPUS_TABLES (10) + cc.examboards storage cleanup + TIMETABLE_TABLES (13); 'all' now also clears the exam subsystem the legacy reset missed. No schema migration required. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
5750413f43
commit
cdc105ae54
@@ -126,13 +126,24 @@ async def platform_stats(
|
||||
|
||||
@router.post("/reset")
|
||||
async def reset_environment(
|
||||
scope: str = "all",
|
||||
_: dict = Depends(require_platform_admin),
|
||||
) -> Dict[str, Any]:
|
||||
"""DESTRUCTIVE: wipe all test data. Neo4j + Supabase. Platform admin only."""
|
||||
"""DESTRUCTIVE: wipe test data. Platform admin only.
|
||||
|
||||
scope (query param):
|
||||
- all : full wipe (Neo4j + Supabase data + auth users) AND exam subsystem + storage.
|
||||
- exam-corpus : ONLY the exam corpus — eb_*/exam_* tables + cc.examboards storage objects
|
||||
(load/unload the public corpus without touching schools/users).
|
||||
- timetable : ONLY timetable/calendar materialization tables.
|
||||
"""
|
||||
if scope not in ("all", "exam-corpus", "timetable"):
|
||||
raise HTTPException(status_code=400, detail="scope must be one of: all, exam-corpus, timetable")
|
||||
import asyncio
|
||||
import functools
|
||||
from run.initialization.reset_environment import reset as _reset
|
||||
loop = asyncio.get_event_loop()
|
||||
result = await loop.run_in_executor(None, _reset)
|
||||
result = await loop.run_in_executor(None, functools.partial(_reset, scope))
|
||||
return {"status": "ok", **result}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user