exam-ingest

Purpose

Convert scattered prep materials into the fixed workspace structure that exam-cram depends on. Build the knowledge base only; do not teach or grade. Produce references/wiki/, references/quiz_bank.json, study_plan.md, and study_progress.md, then hand control back to exam-cram.

Activation

Activate when the workspace is missing — that is, any of references/wiki/, references/quiz_bank.json, or study_progress.md is absent. Also activate when the user has just uploaded courseware/syllabus/highlights/past exams, or explicitly requests 「初始化 / 建库 / 开始备考」(initialize / build the bank / start prepping).

Inputs

• Student-uploaded materials: text, textbook page images, teacher-marked highlights, past exam papers, lecture audio transcripts.
• Target workspace directory (default: current workspace root).

Workflow

0. Dependency preflight (BEFORE touching the materials — never let a student discover a missing library as a mid-ingest crash). Run python "${CLAUDE_SKILL_DIR}/scripts/check_deps.py" --materials <dir>: it is the single manifest of every optional dependency (PDF text backends / PyMuPDF page rendering / local browser for the cheatsheet PDF), probes what is installed, and prints the exact pip install … command for anything the student's materials actually NEED (exit 5 = a needed backend is missing). When something needed is missing, ask the student ONCE with the one-line consent phrasing from the language pack, install on yes, then re-run the preflight; on no, proceed only for the non-PDF part of the materials and say plainly what will be skipped. Never install silently, and never skip the preflight when the materials folder contains PDFs.
1. Parse the materials. Extract knowledge points, core formulas, high-frequency question types, and term definitions. Group them by chapter or phase.
2. Build raw_input.json in the background so it matches scripts/ingest.py. Auto-construct an object with course_name, phases[], and quiz_bank[], and write it to a temp directory. Never ask the user to write or edit this JSON.
   • Every quiz item MUST carry chapter (or phase); without it, chapter review cannot retrieve the item. Every item MUST carry source: teacher (from the teacher/past exams) or ai_generated (added by AI).
   • Set each item's type to one of six: choice / subjective / diagram / fill_blank / true_false / code.
   • PDF / materials folder → use the official entry point; NEVER hand-write ad-hoc parsing scripts: when the input is a folder of lecture/homework PDFs, run python <package-root>/scripts/build_raw_input_from_workspace.py --materials <dir> --out raw_input.json --asset-root <workspace>/references/assets --report parse_report.json. It preserves original-page provenance (source_file/source_pages), renders figure-dependent pages to full-page PNG assets, extracts lecture Example/Quiz question-solution pairs into the bank, and emits a parse report (extracted / skipped / warnings + the backend used). PDF text/rendering are optional dependencies — text pip install pypdf; rendering pip install pymupdf (bundles PNG) or pypdfium2 Pillow; when a dependency is missing the script fails with a clear message telling you what to install. Pure .txt/.md materials need no dependency. --asset-root must point at <workspace>/references/assets (with rendering on but unset: auto skips with a warning, required errors out).Alert-takeover obligation (HARD): after build/import you MUST read parse_report.json's warnings and skipped, ai_review_manifest.json's entries, and the workspace ingest_report.json's missing_answer_ids IN FULL, and handle them one by one: whatever is recoverable (re-save as UTF-8, rename with a chNN/sol marker, multimodally read the PDF/image directly to backfill knowledge points or questions) gets handled immediately; whatever is not, you MUST tell the student explicitly which materials were not imported and why. You MUST NOT silently skip a single entry — every program-side alert assumes 「AI 会接手」 (the AI takes over); if you don't take it over, it is lost forever. Common entries:likely_asset_required_but_no_image (install a rendering backend), pdf_pages_no_text/scanned_pdf (multimodal page reading to backfill), unsupported_format (read directly or convert), exam_no_markers (read the paper and extract questions directly), chapter_unassigned (check against the wiki chaptering), hw_pairing_ambiguous/hw_unpaired_solution_file (ask a human to confirm the pairing). See docs/file-format.md §4.
   • Homework auto-ingest: the official builder also recognizes homework / solution PDFs (filename patterns like hw1.pdf+hw1_sol.pdf, 作业3.pdf+作业3答案.pdf), pairs separate question/solution files automatically (inline Solution blocks too), and emits bank items tagged source_type="homework" with page provenance — so homework-only scope has real inventory. Unpaired solutions / unanswered problems are fail-loud warnings in parse_report.json.
3. Run the package-root scripts/ingest.py: python <package-root>/scripts/ingest.py --input <temp-json>. When the full package is installed in Claude Code, ${CLAUDE_SKILL_DIR} points at the package root — use python "${CLAUDE_SKILL_DIR}/scripts/ingest.py" --input <…>.
   • Recall cross-check AFTER ingest has created the workspace (references/quiz_bank.json must exist — this step reads it): run python <package-root>/scripts/build_visual_index.py --workspace <workspace> --materials <dir> to build the dual visual indices (image_question_index.json + figure_page_index.json) and get the suspected-missed visual question report (source pages that ARE visual but whose items carry no visual flag/asset). If suspects are found, rerun with --apply (needs a render backend) to attach the original-page screenshot and set maybe_requires_assets=true — never leave a figure-dependent item silently unlabeled. (In the no-Python fallback of step 4 this cross-check is unavailable — say so instead of skipping silently.)
4. No-Python fallback. If the script fails (e.g. python is not recognized), switch immediately and silently: use file-write tools to build, from locales/<lang>/templates/ (pick the pack matching study_state.json.language; default zh, and fall back to the zh pack when an en template file is missing), the files references/wiki/chN_*.md, references/quiz_bank.json, study_plan.md, and study_progress.md.
5. Label provenance (canonical labels in docs/language-policy.md). In wiki paragraphs, distinguish 🟢 来自资料 from 🟡 AI补充，可能与你老师讲的不完全一致. For a question the teacher gave no answer to and AI answers instead, mark the answer ⚠️ AI生成答案，非老师/教材提供.

Output Contract

• Produce the standard workspace: references/wiki/, references/quiz_bank.json, study_plan.md, study_progress.md.
• The quiz_bank schema supports asset-aware fields for image-dependent lecture questions (e.g. a Venn diagram shown on a slide, a page figure/table): source_file/source_pages, assets (under references/assets/), requires_assets, maybe_requires_assets, question_text_status. When a question depends on a figure that isn't transcribed into text: only set requires_assets=true or maybe_requires_assets=true when you actually attach a valid question-side image asset under references/assets/ (the validator rejects visual-required items whose asset is missing/unreadable or answer-side-only). Note scripts/ingest.py only writes references/wiki/, quiz_bank.json, and the plan/progress files — it does NOT create or copy references/assets/. So if you reference an asset, you must write the image file under <workspace>/references/assets/ yourself (file-write tools) before/after running ingest; otherwise leave the visual-required fields unset/false. If you have only a source-page reference and no image, set question_text_status="page_reference" with source_file+source_pages and leave requires_assets / maybe_requires_assets unset — the page reference tells the tutor to surface the page without the hard asset requirement. (The official builder takes the opposite, fail-closed stance for a figure it detected but couldn't render: it keeps requires_assets=true with the missing asset recorded, so the workspace won't validate until you install a render backend or supply the image. Both are intentional — hand-authoring stays graceful and never emits an invalid workspace, while the builder forces a genuinely-needed figure to surface rather than silently dropping it.) These fields are optional and backward-compatible (old banks stay valid); the official builder scripts/build_raw_input_from_workspace.py emits them from PDF material (see Workflow step 2). See docs/file-format.md §4.
• Emit one setup-receipt line, then hand control back to exam-cram for step two (teaching).
• Student-facing output defaults to English (Simplified Chinese if the student opened in Chinese); a persisted study_state.json language (中文/English/双语) switches it per exam-cram's dispatch rule with single-language purity. The cold-start receipt follows the same dispatch; see docs/language-policy.md.

Language packs

Student-visible wording for this skill lives in per-language packs — load the one matching study_state.json.language BEFORE emitting any student-visible output:

• zh → ../../locales/zh/skills/exam-ingest.md
• en → ../../locales/en/skills/exam-ingest.md
• bilingual → compose from the zh pack with a > EN: mirror line per block (rules in ../../docs/language-policy.md) Unset language → this is the first conversation: the merged first-ask (mode × time budget × language) decides it; default en unless the student opened in Chinese.

Boundaries

• scripts/ingest.py and locales/<lang>/templates/ live at the package root, not inside skills/exam-ingest/. If this subskill is installed alone (CLAUDE_SKILL_DIR points only at skills/exam-ingest/), the script and templates are unavailable — install the whole package (including root scripts/ and locales/), or use the step-4 no-Python fallback to build the workspace by hand.
• Do not modify the logic of scripts/ingest.py; only call it.
• Use only safe filenames under references/wiki/. The script rejects ../, absolute paths, and duplicate names.
• Do not fabricate a "standard answer" the teacher did not provide without the ⚠️ label. When materials are insufficient, state the gap honestly.
• Do not overwrite an existing study_progress.md. The script does not clear it by default; --force backs it up first.