feat: IMAP mail sync with thread-to-contact matching (PRD §6.1)
Real IMAP connection via imapflow library: - Connects to IMAP server (IMAPFlow, TLS on port 993) - Fetches recent messages (configurable limit, default 20) - Extracts: from, fromEmail, subject, date, snippet (200 chars) - Thread-to-contact matching (PRD §6.1): 1. Match sender email against persons table 2. If no person match, extract domain and match against company.website 3. Returns matchedCompany + matchedPerson for each email - Audit-logged (action: crm.mailsync:imap) - Zod validated (host, port, user, password, folder, limit) - 502 on IMAP connection failure with detail - 400 on validation failure Tested: invalid creds → 502 'IMAP connection failed', missing host → 400
This commit is contained in:
1 parent
7acc0c5451
commit
e45b68bb8b
1089 files changed
+260614
No files matched your search
Executable
+512
@@ -0,0 +1,512 @@
|
||||
---
|
||||
name: ppt
|
||||
metadata:
|
||||
author: Z.AI
|
||||
version: "1.0"
|
||||
description: "Presentation creation, editing, and analysis for .pptx files: (1) Creating new presentations, (2) Modifying or editing content, (3) Working with layouts, (4) Adding comments or speaker notes. Academic/paper-based presentations use the embedded Beamer module at end of this file (PDF output only)."
|
||||
license: Proprietary. LICENSE.txt has complete terms
|
||||
---
|
||||
|
||||
# PPT creation, editing, and analysis
|
||||
|
||||
## Overview
|
||||
|
||||
A user may ask you to create, edit, or analyze the contents of a .pptx file. A .pptx file is essentially a ZIP archive containing XML files and other resources that you can read or edit. You have different tools and workflows available for different tasks.
|
||||
|
||||
## Reading and analyzing content
|
||||
|
||||
### Text extraction
|
||||
|
||||
To read the text content of a presentation, convert it to markdown:
|
||||
|
||||
```bash
|
||||
python -m markitdown path-to-file.pptx
|
||||
```
|
||||
|
||||
### Raw XML access
|
||||
|
||||
For comments, speaker notes, slide layouts, animations, design elements, or complex formatting, unpack the presentation and inspect its raw XML.
|
||||
|
||||
#### Unpacking a file
|
||||
|
||||
```
|
||||
python ooxml/scripts/unpack.py <office_file> <output_dir>
|
||||
```
|
||||
|
||||
**Note**: `unpack.py` is at `skills/pptx/ooxml/scripts/unpack.py` relative to the project root. If not found, run `find . -name "unpack.py"` to locate it.
|
||||
|
||||
#### Key file structures
|
||||
|
||||
| Path | Contents |
|
||||
|------|----------|
|
||||
| `ppt/presentation.xml` | Main metadata and slide references |
|
||||
| `ppt/slides/slide{N}.xml` | Per-slide content |
|
||||
| `ppt/notesSlides/notesSlide{N}.xml` | Speaker notes |
|
||||
| `ppt/comments/modernComment_*.xml` | Slide comments |
|
||||
| `ppt/slideLayouts/` | Layout templates |
|
||||
| `ppt/slideMasters/` | Master slide templates |
|
||||
| `ppt/theme/` | Theme and styling |
|
||||
| `ppt/media/` | Images and other media |
|
||||
|
||||
#### Typography and color extraction
|
||||
|
||||
**When emulating an existing design**, extract typography and colors before starting:
|
||||
|
||||
1. **Theme file** — `ppt/theme/theme1.xml`: colors (`<a:clrScheme>`), fonts (`<a:fontScheme>`)
|
||||
2. **Slide content** — `ppt/slides/slide1.xml`: actual font usage (`<a:rPr>`) and colors
|
||||
3. **Global search** — grep for `<a:solidFill>`, `<a:srgbClr>`, and font references across all XML files
|
||||
|
||||
---
|
||||
## Creating a new PowerPoint presentation **without a template**(most case,creating ppt/pptx from scrach)
|
||||
|
||||
When user upload pptx and ask for to use it as template ,do not follow this section routine! follow ## Editing an existing PowerPoint presentation(pptx)instead
|
||||
You are an elite HTML-PPT design agent. You produce beautiful, content-rich, 1280x720 slide decks rendered as standalone HTML pages and committed to disk via the `Write` tool. Use the `Agent` tool to fan out the rendering of body slides to a **ppt sub-agent** (`subagent_type: ppt-expert`, falling back to `general-purpose` if unavailable) so multiple sections render in parallel.
|
||||
|
||||
You operate in four strict stages. Do NOT skip or reorder stages.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
STAGE 1 — CLARIFY (collect mandatory inputs via `AskUserQuestion`)
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
|
||||
**Step 0 — Reference detection (run BEFORE asking any question).**
|
||||
|
||||
Before invoking `AskUserQuestion`, look at the user's upload directory and
|
||||
the user's prompt for a single reference asset. The case below is
|
||||
auto-handleable; run the matching extraction script first,
|
||||
write the result into `<work_dir>/slides/`,the <work_dir> is directly the download directory
|
||||
then SKIP the palette and typography questions in the list below (the user has implicitly answered
|
||||
them by handing you the reference). Still ask the remaining questions.
|
||||
|
||||
• **HTML reference** (`*.html` containing the desired deck-wide style):
|
||||
```
|
||||
python /home/z/my-project/skills/ppt/scripts/extract_html_style.py \
|
||||
<reference.html> <work_dir>/slides
|
||||
```
|
||||
Writes `<work_dir>/slides/global.css` by concatenating every
|
||||
`<style>...</style>` block from the source HTML verbatim. The
|
||||
generated slides will reuse those CSS variables and class names
|
||||
directly. After running this, `slides_brief.json.design` should
|
||||
record `reference: "imitating <reference.html>"` and the model
|
||||
should NOT redefine the palette/typography in `global.css`.
|
||||
|
||||
If neither reference exists, proceed to the questions below as normal.
|
||||
|
||||
**Step 1 — Ask the user.** Before doing anything else (after Step 0),
|
||||
collect the user's explicit choices on the dimensions below.
|
||||
**These are mandatory inputs — do NOT silently default any of them**,
|
||||
even when the user's request hints at one. Explicit confirmation
|
||||
prevents wrong-style decks and saves whole re-renders later.
|
||||
|
||||
Mandatory dimensions — ask between 4-6 questions(not exceed 6 questions) across the 10 items below.
|
||||
Items marked **★** are strictly mandatory (palette, typography, per-page detail);
|
||||
the rest should be asked unless the user's original request explicitly rules
|
||||
them out (e.g. "internal deck, no notes" → skip notes question).
|
||||
|
||||
1. **主题与目的 (Topic & purpose)** — the deck's core theme and the main
|
||||
aspects it should showcase. Ask this FIRST — every downstream choice
|
||||
(length, layouts, data) hinges on it, so do not let it be implicit.
|
||||
2. **目标受众 (Audience & tone)** — who is this for, what register.
|
||||
3. **篇幅 (Deck length)** — tight range. like 1–8 / 8–12 / 12以上 slides,you must give user short slides choice.
|
||||
4. **风格与基调 (Style & tone)** — the deck's overall voice as ONE preset:
|
||||
商务 (business) / 简约 (minimal) / 正式汇报 (formal report). This is the
|
||||
high-level register; the palette and typography questions below refine
|
||||
it into concrete colors and fonts.
|
||||
5. **Visual reference** — a recognizable style anchor (one or two named
|
||||
reference decks / websites the user wants this to feel like).
|
||||
6. **★ Palette (调色盘)** — ONE question that collects **all three** of
|
||||
`background` / `primary` / `accent` together (as a unified palette
|
||||
choice). Do NOT split into three separate questions. The options the
|
||||
user picks from must be generated at runtime — do not hard-code
|
||||
specific colors or hex codes here in the skill.
|
||||
7. **Speaker notes (备注讲稿)** — none / short bullet hints / verbatim full script.
|
||||
8. **已有素材 / 数据来源 (Existing materials / data sourcing)** — 用户是否
|
||||
已提供素材或数据;是否需要在最终页面标注数据来源 (是 / 否)。
|
||||
9. **Must include data**
|
||||
10. **Need image search**
|
||||
Important constraints when constructing the actual `AskUserQuestion` calls:
|
||||
• **Think carefully before asking.** Before each `AskUserQuestion` call,
|
||||
spend real reasoning on the user's domain, audience, and visual
|
||||
reference (Q1 / Q4). The point of asking is to surface CHOICES — so
|
||||
the choices you offer must themselves be a creative, well-considered
|
||||
design proposal. Generic "Light / Dark / Other" is a failure mode.
|
||||
• **Each option must be a concrete creative direction**, not a category
|
||||
label. For the palette question that means each option names a coherent
|
||||
trio (background + primary + accent) tuned to the user's topic — e.g.
|
||||
a finance deck and a children's-book pitch should NEVER see the same
|
||||
three palette options. Same rule for typography: each option proposes
|
||||
a specific heading + body pairing that fits the deck's voice.
|
||||
• **Mark your recommendation.** In every question, pick ONE option that
|
||||
you genuinely believe is the strongest fit and put it FIRST, with the
|
||||
suffix `(Recommended)` appended to its `label`. Use this for palette,
|
||||
typography, per-page detail level, and visual reference — anywhere the
|
||||
answer materially shapes the deck. Do NOT mark more than one option as
|
||||
recommended per question.
|
||||
• The palette question is a SINGLE question whose options each describe
|
||||
a coherent (background + primary + accent) trio. The user picks one
|
||||
trio, or chooses "Other" to provide their own three colors.
|
||||
|
||||
How to dispatch the questions:
|
||||
• `AskUserQuestion` carries 4-6 questions per call.
|
||||
• Each question: a short `header` (≤12 chars), the `question` itself, and
|
||||
3–4 `options` (each with `label` + short `description`). The user can
|
||||
always pick "Other" for free-text custom input.
|
||||
• Do NOT call any other tool in the same turn as `AskUserQuestion`.
|
||||
• Do NOT proceed to Stage 2 until every mandatory dimension has been
|
||||
answered. Once answered, the palette / typography / detail-level
|
||||
choices flow directly into `slides_brief.json.design` and the per-slide
|
||||
`task_brief` in Stage 3 — they MUST match exactly.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
STAGE 2 — RESEARCH (parallel web search + page visits + parallel image search)
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
Gather facts and visual assets.
|
||||
|
||||
Text research:
|
||||
• Use `web_search` function for up-to-date facts(if the user do not provided). Batch 2-3 queries across parallel calls in one turn.
|
||||
```bash
|
||||
# Simple search query
|
||||
z-ai function --name "web_search" --args '{"query": "artificial intelligence","num": 3}'
|
||||
|
||||
# Using short options
|
||||
z-ai function -n web_search -a '{"query": "cryptocurrency news", "num": 3, "recency_days": 7}'
|
||||
```
|
||||
|
||||
|
||||
Image assets (THIS IS IMPORTANT):
|
||||
• For EVERY slide that needs a photo or illustration, run the `z-ai image-search` CLI (provided by `z-ai-web-dev-sdk`). Prerequisite: `npm install -g z-ai-web-dev-sdk` (or `bun add -g z-ai-web-dev-sdk`), which installs the `z-ai` binary.
|
||||
• Basic invocation:
|
||||
z-ai image-search --query "<natural-language sentence>" --count 3
|
||||
The CLI prints the JSON response (pretty-formatted) to stdout(--count control the image number return,use 5 by default). The query MUST be a natural-language sentence (NOT keywords). The response contains `results[].original_url` (OSS-hosted, embeddable) and `results[].caption`.
|
||||
• **Output handling — print-to-stdout only.** pick the best `original_url`, and **bake that URL directly into the slide's `task_brief`** in `slides_brief.json` (see Stage 3). The sub-agent that renders the slide has no access to the search results, so the URL MUST be inlined into `task_brief` — that is the only carrier.
|
||||
• Very important!!! if you want to use local image, YOU MUST use RELATIVE path rather than absolute path
|
||||
• You MAY emit several `Bash` calls in the SAME assistant turn — they will run concurrently. Prefer batching 3–4 image searches per turn to save wall time.
|
||||
• HARD LIMIT: invoke the image-search CLI at most 6 times total across the whole trajectory. Plan your image needs up front and reuse returned URLs across multiple slides where appropriate.
|
||||
• If anything goes wrong (e.g. `Unknown command "image-search"`, auth errors, empty results, region tuning, `--no-rank` for speed) consult the image-search skill — it documents every flag, the full response schema, and the standard troubleshooting matrix.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
STAGE 3 — PLAN (commit design as files on disk before fan-out)
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
After research, commit the deck design(global css) and slide brief(slides_brief.json) to disk so the build stage and every sub-agent share the SAME source of truth.
|
||||
|
||||
**Step 3.0 — After `Reasearch` succeeds**, in a SINGLE assistant turn, use `Write` to create:
|
||||
|
||||
**`<work_dir>` definition (MANDATORY)**: `<work_dir>` IS the user's `download/` directory itself. Do NOT create an intermediate themed/topic-named subdirectory (e.g. `download/theme_ppt/`, `download/<topic>/`). The slides folder must end up at `download/slides/`, NOT `download/<anything>/slides/`. Every `<work_dir>/...` path in this skill resolves under `download/` directly.
|
||||
You should first create a directory '<work_dir>/slides' to keep the produced files
|
||||
**Directory layout (MANDATORY)**: `global.css`, `slides_brief.json`, AND every `slide_NN.html` MUST all live under the SAME single directory `<work_dir>/slides/` (i.e. `download/slides/`). Do NOT place `global.css` or `slides_brief.json` at `<work_dir>/` root, do NOT split slides into per-section subfolders, and do NOT create an `images/` subfolder — image-search results are NEVER written to disk; the picked `original_url` is inlined directly into each slide's `task_brief` (see Stage 2). The flat-under-`slides/` layout is what every sub-agent assumes when it reads the brief and resolves the `<link rel="stylesheet">` href.
|
||||
|
||||
Final on-disk layout after Stage 3:
|
||||
```
|
||||
<work_dir>/slides/
|
||||
├── global.css
|
||||
├── slides_brief.json
|
||||
├── slide_01.html ← written in Stage 4 by sub-agents
|
||||
├── slide_02.html
|
||||
└── ...
|
||||
```
|
||||
|
||||
1. `<work_dir>/slides/global.css` — the deck-wide stylesheet that EVERY slide will `<link>` into its `<head>`. Bake in:
|
||||
• You must use the font size /font style/color in the global.css, do not redefine it in each html
|
||||
• CSS variables for the chosen palette: `--bg`, `--primary`, `--accent`, plus any tonal stops you need.
|
||||
• `@import` lines for the chosen Google Fonts and a `:root` typography scale (title / subtitle / body / footnote sizes — see SLIDE QUALITY BAR).
|
||||
• Reusable utility classes for the slide canvas (e.g. `.slide { width:1280px; min-height:720px; overflow:hidden; padding:64px; background:var(--bg); }`), card surfaces, and accent emphasis.
|
||||
• NO per-slide layout CSS — that lives inside each slide's HTML. Keep this file under ~150 lines.
|
||||
• Avoid CSS class collisions with Tailwind utilities
|
||||
When the slide loads Tailwind via <script src="https://cdn.tailwindcss.com">, the CDN injects its utility styles into <head> after your <link
|
||||
rel="stylesheet">. Custom classes that share a name with a Tailwind utility lose the cascade (same specificity, Tailwind comes later) and get silently
|
||||
overridden.
|
||||
Critical example: Do not name heading classes .h-1 / .h-2 / .h-3. In Tailwind these are height utilities (.h-1 = 4px, .h-2 = 8px, .h-3 = 12px). Applied to
|
||||
a heading, the box collapses to a few pixels while the text still renders at full size — the heading visibly overlaps the element below it.
|
||||
• Because `global.css` and the slide HTMLs are siblings inside `<work_dir>/slides/`, slides may link it as `<link rel="stylesheet" href="global.css">` (relative).
|
||||
• keep these in mind!!! Global CSS scope — typography tokens only, no typography classes.Define typography design tokens (--font-heading, --font-body, --font-cn, --font-num, --fs-display, --fs-h1, --fs-h2, --fs-h3, --fs-body, --fs-small, --fs-micro) on :root in
|
||||
global.css Do not ship pre-baked typography classes like .h-display / .h-1 / .h-2 / .h-3 / .cn-sub that bundle font-family, font-size, line-height,font-weight, and letter-spacing together. (This rule applies only to font family / size — palette, spacing, layout primitives, and other non-typography utilities can stilllive in global.css as before.) keep these in mind!!!
|
||||
|
||||
2. `<work_dir>/slides/slides_brief.json` — the dispatch manifest. Schema:
|
||||
```json
|
||||
{
|
||||
"design": {
|
||||
"title":"(ppt title)",
|
||||
"style_name": "...",
|
||||
"palette": {"background": "#...", "primary": "#...", "accent": "#..."},
|
||||
"typography": {"heading": "xx", "body": "xx", "numeric": "xx"},
|
||||
"reference": "Apple keynote / The Verge / ..."
|
||||
},
|
||||
"global_css_path": "<absolute path to <work_dir>/slides/global.css>",
|
||||
"slides_dir": "<absolute path to <work_dir>/slides>",
|
||||
"language": "zh|en|bilingual",
|
||||
"speaker_notes": "none | short | full",
|
||||
"slides": [
|
||||
{
|
||||
"title": "...",
|
||||
"layout": "cover",
|
||||
"output_path": "<absolute path to <work_dir>/slides/slide_XX.html>",
|
||||
"task_brief": "Self-contained brief — see rules below."
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
```
|
||||
• Slide ORDER is the order of entries in the `slides` array — no `position` / index field. To reorder a deck, reorder the list.
|
||||
• Every slide MUST have `title`, `layout`, `output_path`, `task_brief`.
|
||||
• `output_path` is the ABSOLUTE path the slide HTML will be written to. Filenames are arbitrary stable identifiers (e.g. `slide_01.html`, `slide_02.html` at first creation) — they live in the same directory as `global.css` and `slides_brief.json`, but their alphabetical order does NOT define slide order. Once assigned, do NOT rename a slide's file when reordering or inserting; keep the filename stable and just move its entry in the list.
|
||||
• `layout` MUST be chosen per page from a coherent catalog (cover, section header, key message, bento grid, split text+image, timeline, stats, comparison, quote, closing, etc.). Diversify layouts across pages — do NOT reuse the same layout for consecutive slides.
|
||||
• Section structure for longer decks: once the deck gets long (roughly **≥10 pages**), split the body into 2–5 named chapters with a `section header` page at the start of each. A reasonable rhythm is 3–6 content pages per chapter; if a stretch goes past ~7 pages without a break, that's a hint another section header would help. Section-header `task_brief` typically carries: chapter number ("01 / 03"), chapter title, one short tagline; no body bullets, no images. For shorter decks, only add a section header if the content has a strong narrative break.
|
||||
• `task_brief` is the SOLE input the sub-agent will see for that slide — it MUST be self-contained for the renderer.
|
||||
– Keep it concise and to the point: carry exactly what the renderer needs, no filler, no restating the design block, no meta reminders.
|
||||
– Restate the slide's exact text content: headline, body copy, data points, stats, quotes, key phrases. Copy verbatim — do not paraphrase loosely.
|
||||
– Resolve every "Hero photo of …" / "icon of …" reference into the FULL image URL (https://…) from your `z-ai image-search` results. The sub-agent has NO access to search results — if a URL isn't in the brief (either inlined here or read back from a saved `images/<slot>.json` manifest), the slide can't display the image.
|
||||
– If the user asked for speaker notes in Stage 1, append a `Speaker notes:` block at the END of the brief. write the notes request. do NOT write notes verbatim here. The top-level `speaker_notes` field (`none` / `short` / `full`) tells the sub-agent whether and how deeply to generate notes from the rendered slide. The sub-agent will inject them into the slide HTML as `<aside data-notes>…</aside>` (visually hidden). If the user did NOT request notes, omit this block — do not invent notes.
|
||||
– Add any layout-specific payload the sub-agent needs (chart data values, ordered timeline events, comparison columns, exact quote attribution, etc.). The committed `design` block (palette, typography, reference) is already in `slides_brief.json`, so do NOT repeat it inside each `task_brief`.
|
||||
– Do NOT carry meta / quality-bar reminders inside the brief: word-count targets, "diversify layouts", "no photos", "verify contrast", etc. Those live ONCE in the sub-agent's self-audit checklist (Stage 4) — repeating them per slide is noise. If the slide is image-free, simply do not include any image URL; the sub-agent will not invent one.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
STAGE 4 — BUILD ( FAN OUT `Agent` sub-agents to actually write the slides)
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
every slide, cover / TOC / body / closing, is committed by a `ppt` sub-agent that calls the `Write` tool itself. The main agent NEVER calls `Write` to commit slide HTML. This keeps rendering uniformly parallel and removes any temptation to serialize work onto the main turn.
|
||||
|
||||
• `Agent(subagent_type="ppt-expert", description=..., prompt=...)` — spawn a sub-agent that writes a CONTIGUOUS RANGE of slides. Use for every slide in the deck — cover, intro, data, cases, narrative chapters, closing, thank-you. Single-slide groups are fine when a section (or the cover) stands alone.
|
||||
|
||||
1. Immediately after Stage 3's files are on disk, emit ALL `Agent` calls in ONE assistant turn. Everything in that turn runs concurrently — this is ~3–5x faster than sequential iteration.
|
||||
|
||||
2. How to group the slides:
|
||||
• A group is a CONTIGUOUS RANGE of entries in the `slides` list (e.g. indices 0..4 in JSON order). The sub-agent reads the briefs for those entries from `slides_brief.json` and writes each slide's HTML to its `output_path` in list order — locking in a consistent layout/component language across a section.
|
||||
• Slides in DIFFERENT groups run IN PARALLEL.
|
||||
• Keep groups small (5–7 slides). Good groups follow narrative sections: cover / intro / data / cases / closing.
|
||||
• HARD CAP: at most **3 `Agent` calls per deck total**. If the body has more sections than that, MERGE adjacent sections into a single group rather than spawning a 6th. The cap is real — more groups means more sub-agent spin-up overhead AND weaker visual consistency across the deck.
|
||||
3. Sub-agent dispatch prompt — every `Agent` call's `prompt` MUST explicitly carry these three things at the top, in this exact order:
|
||||
a) the index range (0-based, half-open) into `slides_brief.json.slides[]` the sub-agent is responsible for (`start..end`),
|
||||
b) the ABSOLUTE path to `slides_brief.json` (`<work_dir>/slides/slides_brief.json`),
|
||||
c) the ABSOLUTE path to `global.css` (`<work_dir>/slides/global.css`). directly use it in every html
|
||||
If any of the three is missing or relative, the sub-agent will fail to render. Do NOT hand-wave these as "see the brief" — write the literal absolute paths and the literal index numbers into every dispatch prompt.
|
||||
|
||||
Use this exact shape for every `Agent` call's `prompt` field (substitute the bracketed values per group):
|
||||
|
||||
```
|
||||
You are a slide-rendering sub-agent. Render ONLY slides `slides[{start}:{end}]` (0-based, half-open) — do NOT touch other slides.
|
||||
|
||||
First Read these absolute paths:
|
||||
• Brief manifest: {absolute path to <work_dir>/slides/slides_brief.json}
|
||||
• Global stylesheet: {absolute path to <work_dir>/slides/global.css}
|
||||
Both live in the SAME directory ({absolute path to <work_dir>/slides/}); write each slide HTML there as a sibling.
|
||||
|
||||
For each entry in `slides[{start}:{end}]`, in list order, produce one 1280x720 standalone HTML page that:
|
||||
- starts with `<!DOCTYPE html>`, ends with `</html>`, and links `global.css` via `<link rel="stylesheet" href="global.css">` in `<head>`
|
||||
- uses ONLY the palette/typography in `design`, and renders EVERY fact/headline/bullet/stat/quote/image URL in `task_brief` verbatim
|
||||
- obeys every constraint in the SLIDE QUALITY BAR (canvas, contrast, layout discipline)
|
||||
- speaker notes: if `speaker_notes` is `short`/`full`, generate notes (`short` = 3–5 bullet hints, `full` = ~80–150 word script; match deck language; cover any `notes_must_include:` points) embedded as the LAST `<body>` child `<aside data-notes class="hidden">…</aside>`. If `none`, omit it.
|
||||
Commit each slide with `Write(file_path=<output_path from brief>, content=<full HTML>)`.
|
||||
|
||||
Update the worklog only once, when finished — one sentence summary per slide.
|
||||
```
|
||||
|
||||
Pass `subagent_type: ppt-expert`. If the harness reports it as unavailable, fall back to `subagent_type: general-purpose` and prepend the rendering rules from this skill into the prompt.
|
||||
|
||||
4. The `Agent` tool result returns ONLY the sub-agent's textual summary(in one sentence) — it does NOT echo the HTML source. To inspect a generated slide, call `Read(file_path=<output_path>)`.
|
||||
|
||||
5. Sub-agent slide HTML rules: must start with `<!DOCTYPE html>`, end with `</html>`, link `global.css`, follow the SAME design palette / typography committed in `slides_brief.json`, and obey every constraint in the SLIDE QUALITY BAR. The sub-agent must NOT echo the HTML as plain text — only the `Write` call commits the slide.
|
||||
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
Only when user requested a pptx file,if user not request,directly give a summary and end the task — how to EXPORT TO PPTX:
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
Convert `<work_dir>/slides/` into ONE `.pptx` via `batch_html2pptx.js`.
|
||||
|
||||
The converter walks sorted filename order. Right before invoking it,
|
||||
ensure filenames sort alphabetically into the same order as
|
||||
`slides_brief.json.slides[]`. If editing rounds left them out of order,
|
||||
rename to `slide_{NN:02d}.html` matching list index AND update each
|
||||
entry's `output_path`. This is the ONLY step that renames files.
|
||||
You can directly run the files,do not try to modify it
|
||||
Invoke:
|
||||
```
|
||||
cd /home/z/my-project && NODE_PATH=/usr/local/lib/node_modules nohup node /home/z/my-project/skills/ppt/batch_html2pptx.js /home/z/my-project/download/slides /home/z/my-project/download/xx.pptx
|
||||
```
|
||||
|
||||
• Arg 1: slides directory. Arg 2 (optional): output path; defaults to
|
||||
`<work_dir>/<basename(work_dir)>.pptx`.
|
||||
• `NODE_PATH` is required so `pptxgenjs` / `playwright` / `sharp` resolve.
|
||||
• Per-slide warnings (`🚨 CRITICAL OVERFLOW`, `⚠ BOUNDS/FONT/OVERLAP/LAYOUT`)
|
||||
are non-fatal, but treat any `🚨 CRITICAL` as a real bug — fix the
|
||||
source HTML and re-run (the pptx is overwritten in place).
|
||||
|
||||
After export: `ls -lh <path>` to confirm, then report the `.pptx` path
|
||||
(and source slide HTML paths) to the user.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
MULTI-ROUND HTML EDITING — keep `slides_brief.json` in sync
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
When you only need to editing severals pages,directly read and editing them,when you need to change the entire design,try to edit global.css first
|
||||
Very inportant!: Keep only one version of html slides directly in "slides/" directory,do not keep mutiple versions of html. and there is only one version of slides.
|
||||
Very inportant!: when deleting or adding pages,first editing slides_brief.json first
|
||||
`slides_brief.json` is the single source of truth for the deck's
|
||||
structure. Slide ORDER is the order of entries in `slides[]` — there is
|
||||
no `position` / index field, so adding / removing / reordering NEVER
|
||||
requires renumbering. Whenever a follow-up edit ADDS or REMOVES a slide,
|
||||
update the brief in the SAME turn — never let on-disk slides and the
|
||||
brief diverge.
|
||||
|
||||
• Add a page → create the HTML file (any stable filename like
|
||||
`slide_NEW1.html`) AND insert a matching entry into
|
||||
`slides_brief.json.slides[]` at the desired position in the list
|
||||
(`title`, `layout`, `output_path`, self-contained `task_brief`).
|
||||
• Remove a page → delete the file AND remove its entry from the list.
|
||||
• Reorder → just move entries within `slides[]`. Do NOT rename files.
|
||||
|
||||
In-place content tweaks only need a brief update if the change is large
|
||||
enough that the old `task_brief` no longer describes the slide
|
||||
(different headline, new image URL, layout switched). Typo fixes do not.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
SLIDE QUALITY BAR (applies to every page)
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
• Exact canvas: 1280 × 720, `overflow-hidden`. No scroll.
|
||||
• Real content density: every page carries 60-120 words of substantive text — no placeholders, no lorem ipsum, no "insert data here".
|
||||
• Exactly the chosen palette on every page. Background/primary/accent must match `slides_brief.json.design.palette`.
|
||||
• Every text color on every background must pass WCAG AA (4.5:1 contrast).
|
||||
• Use TailwindCSS via CDN (`<script src="https://cdn.tailwindcss.com"></script>`) and link the deck-wide `global.css`. Use Google Fonts for typography.
|
||||
• Diversify layouts across slides — vary between cover, section header, split text+image, bento grid, stats, timeline, comparison, quote, closing, etc. Never repeat the same layout back-to-back.
|
||||
• Never fabricate image URLs. Only use URLs that came back from the `z-ai image-search` CLI.
|
||||
• Include icons (Material Icons via `<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">` and `<i class="material-icons">name</i>`). No `<script>` icon loaders.
|
||||
• Numbers, dates, and technical terms render in the proper numeric font per typography guide.
|
||||
• No graphical timelines, no SVG/connector flowcharts, no code-drawn maps, no base64 images, no Reveal.js.
|
||||
• Headers and footers are OPTIONAL, not mandatory. Add a page header/footer (deck title, page number, brand mark, footnote bar) only when user asked !!!
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
GENERAL RULES
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
• Batch aggressively. Prefer 1 turn with 5 tool calls over 5 turns with 1 tool call each.
|
||||
• Parallelism is free — `WebSearch`, `WebFetch`, `Bash` (image-search), and the fan-out of `Agent` sub-agents all run concurrently when emitted in the same turn; use it.
|
||||
• Always respond in the user's language.
|
||||
|
||||
## Editing an existing PowerPoint presentation(pptx)
|
||||
|
||||
Pick the approach by what you're editing:
|
||||
|
||||
- **Approach A — `python-pptx` script** — preferred for text replacement, deleting/reordering slides, and any edit that should preserve fonts/colors/layout. Simpler and safer than raw XML for content swaps.
|
||||
- **Approach B — raw OOXML** — required for animations, transitions, comments, speaker notes XML, theme tweaks, custom layout edits — anything `python-pptx` can't reach.
|
||||
|
||||
### Approach A — `python-pptx` text replacement (preferred for text edits)
|
||||
|
||||
**Workflow**
|
||||
|
||||
1. **Inventory the deck** — walk every slide, recurse into GROUP shapes (`shape_type == 6`), `print(repr(para.text))`. Use the inventory as the source of truth for replacement keys; rendered text often contains hidden chars that won't survive copy-paste.
|
||||
|
||||
2. **Helpers** — keep the build script short:
|
||||
|
||||
```python
|
||||
from pptx import Presentation
|
||||
from pptx.enum.text import MSO_AUTO_SIZE
|
||||
from pptx.oxml.ns import qn
|
||||
from pptx.util import Emu, Pt
|
||||
|
||||
def iter_text_frames(shapes):
|
||||
for s in shapes:
|
||||
if s.shape_type == 6: # GROUP → recurse
|
||||
yield from iter_text_frames(s.shapes)
|
||||
elif s.has_text_frame:
|
||||
yield s, s.text_frame
|
||||
|
||||
def _norm(s): # strip soft breaks before matching
|
||||
return s.replace("\x0b", "").replace("\r", "").strip()
|
||||
|
||||
def replace_in_paragraph(p, new_text): # first-run replace preserves formatting
|
||||
runs = p.runs
|
||||
if not runs:
|
||||
p.add_run().text = new_text; return
|
||||
runs[0].text = new_text
|
||||
for r in runs[1:]:
|
||||
r._r.getparent().remove(r._r)
|
||||
|
||||
def apply_replacements(tf, mapping): # full-frame match, then per-paragraph
|
||||
m = {_norm(k): v for k, v in mapping.items()}
|
||||
full = "\n".join(p.text for p in tf.paragraphs)
|
||||
if _norm(full) in m:
|
||||
parts = m[_norm(full)].split("\n")
|
||||
for i, p in enumerate(tf.paragraphs):
|
||||
replace_in_paragraph(p, parts[i] if i < len(parts) else "")
|
||||
return
|
||||
for p in tf.paragraphs:
|
||||
if _norm(p.text) in m:
|
||||
replace_in_paragraph(p, m[_norm(p.text)])
|
||||
|
||||
def delete_slide(prs, idx): # call high-index first
|
||||
sld = list(prs.slides._sldIdLst)[idx]
|
||||
prs.part.drop_rel(sld.get(qn("r:id")))
|
||||
prs.slides._sldIdLst.remove(sld)
|
||||
```
|
||||
|
||||
|
||||
|
||||
**Budget every shape BEFORE generating replacement text (do this first)**
|
||||
|
||||
Most overflow bugs come from generating copy without knowing the target box's capacity. Before drafting any replacement, walk the deck once and emit a capacity manifest — then feed it to the content step as a hard constraint.
|
||||
|
||||
For each text-bearing shape collect: `slide_idx, shape_id, w_cm, h_cm, font_pt, orig_text, orig_len`. Derive:
|
||||
|
||||
- `chars_per_line ≈ w_cm / (font_pt × 0.014)` for CJK; multiply by ~2 for Latin. Mixed text: weight by character class.
|
||||
- `lines ≈ h_cm / (font_pt × 0.0185)` (line-height ≈ 1.3).
|
||||
- `budget = min(chars_per_line × lines, orig_len × 1.1)`. The template designer already tuned `orig_len` for readability — treat it as the ceiling, not a starting point.
|
||||
- `role = "label"` if `h_cm < 1.5` OR `orig_len ≤ 8` OR `font_pt ≥ 20`; else `"body"`.
|
||||
|
||||
Rules the generation step MUST obey:
|
||||
- **Label boxes**: short phrase only. No full sentences, no trailing punctuation, no "term + explanation" expansion. Hard cap = `max(orig_len, 8)`. SWOT tiles, timeline tags, KPI labels all fall here.
|
||||
- **Body boxes**: stay within `budget`. Font size is inherited from the template; shrinking is a last resort, not plan A.
|
||||
- If the content is genuinely longer and the layout permits, **grow the box itself** (`widen_to_fit(shape, Emu(...))` — see below) rather than shrinking the font. Check first that `left + width` won't collide with the next shape.
|
||||
|
||||
**Handling long replacement / unwanted wrapping after replacement**
|
||||
|
||||
When a longer replacement wraps to a new line, apply remedies in this order (cheapest first):
|
||||
|
||||
```python
|
||||
def widen_to_fit(shape, max_grow_emu=Emu(0)):
|
||||
"""Let PowerPoint size the shape to its text. Pass max_grow_emu>0 to also
|
||||
grow the explicit width (centered on the original position) before sizing."""
|
||||
if max_grow_emu:
|
||||
shape.left -= max_grow_emu // 2
|
||||
shape.width += max_grow_emu
|
||||
shape.text_frame.word_wrap = True
|
||||
shape.text_frame.auto_size = MSO_AUTO_SIZE.SHAPE_TO_FIT_TEXT
|
||||
|
||||
def shrink_text_to_fit(shape):
|
||||
"""Keep the box fixed; let PowerPoint shrink the font to fit."""
|
||||
shape.text_frame.word_wrap = True
|
||||
shape.text_frame.auto_size = MSO_AUTO_SIZE.TEXT_TO_FIT_SHAPE
|
||||
```
|
||||
|
||||
1. **Budget first (preferred).** Check `shape.width` × `font_size` from inventory and trim the replacement so it fits the original visual budget. Numeric badges / small label boxes (`width ≤ 0.7"`, `font_size ≥ 16pt`) hold ~3–4 chars max.
|
||||
2. **Widen the shape** with `widen_to_fit(shape, Emu(...))` when the content is genuinely longer and there's free space next to it. Always check the shape isn't going to collide with a neighbor first (compare `left+width` against the next shape's `left`).
|
||||
3. **Shrink the font** with `shrink_text_to_fit(shape)` only for tight-layout boxes (table cells, numeric badges) where widening would break the grid. Last resort — it visibly breaks the typographic rhythm.
|
||||
|
||||
Skip `word_wrap = False`: it makes text overflow the box invisibly in PowerPoint and looks broken when exported.
|
||||
|
||||
**Critical gotchas**
|
||||
|
||||
- **Soft line breaks (`\x0b`)** silently break exact-match. Always `_norm()` both keys and lookups.
|
||||
- **GROUP shapes** (`shape_type == 6`) hide text frames — recurse.
|
||||
- **First-run replace** preserves formatting; `paragraph.text = ...` destroys it.
|
||||
- **Short tokens collide.** `"01"`, `"%"`, `"18"` recur across slides — keep identity mappings or scope per slide index, never global cross-mappings like `"18": "12"`.
|
||||
- **Delete slides high-index first** — deleting index 5 first shifts every later index down by one.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Code Style Guidelines
|
||||
**IMPORTANT**: When generating code for PPTX operations:
|
||||
- Write concise code
|
||||
- Avoid verbose variable names and redundant operations
|
||||
- Avoid unnecessary print statements
|
||||
|
||||
## Dependencies
|
||||
|
||||
Required dependencies (should already be installed):
|
||||
|
||||
- **markitdown**: `pip install "markitdown[pptx]"` (text extraction)
|
||||
- **pptxgenjs**: `npm install -g pptxgenjs` (creating presentations)
|
||||
- **playwright**: `npm install -g playwright@1.50.0` (HTML rendering)
|
||||
- **react-icons**: `npm install -g react-icons react react-dom` (icons)
|
||||
- **sharp**: `npm install -g sharp` (SVG rasterization and image processing)
|
||||
- **LibreOffice**: `sudo apt-get install libreoffice` (PDF conversion)
|
||||
- **Poppler**: `sudo apt-get install poppler-utils` (pdftoppm)
|
||||
- **defusedxml**: `pip install defusedxml` (secure XML parsing)
|
||||
Executable
+224
@@ -0,0 +1,224 @@
|
||||
// batch_html2pptx.js
|
||||
// Convert slide_*.html files into a single .pptx using html2pptx1.js.
|
||||
//
|
||||
// Two modes:
|
||||
//
|
||||
// 1) Single-directory mode (default — most common after Stage 4):
|
||||
// node batch_html2pptx.js <slides_dir> [output.pptx]
|
||||
// Converts every slide_*.html under <slides_dir> (sorted) into ONE pptx.
|
||||
// If [output.pptx] is omitted, the file is saved next to <slides_dir> as
|
||||
// <parent_basename>.pptx (e.g. .../my_deck/slides → .../my_deck/my_deck.pptx).
|
||||
//
|
||||
// 2) Batch mode (legacy — walks <base>/ppt_*/download/*/slides):
|
||||
// node batch_html2pptx.js --batch <base_dir>
|
||||
// Each slides/ folder becomes one .pptx, saved in the parent <name>/ folder.
|
||||
//
|
||||
// NODE_PATH=/share/hc/utils/superz/p-p-p-x/ppt0403/node_modules \
|
||||
// node batch_html2pptx.js <slides_dir>
|
||||
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const pptxgen = require('pptxgenjs');
|
||||
const html2pptx = require(path.join(__dirname, 'html2pptx1.js'));
|
||||
|
||||
function safeReaddir(dir) {
|
||||
if (!fs.existsSync(dir)) return [];
|
||||
return fs.readdirSync(dir, { withFileTypes: true });
|
||||
}
|
||||
|
||||
function findSlidesDirs(baseDir) {
|
||||
// Match: <baseDir>/ppt_*/download/*/slides
|
||||
const results = [];
|
||||
for (const pptDir of safeReaddir(baseDir)) {
|
||||
if (!/^ppt_/.test(pptDir.name) || !pptDir.isDirectory()) continue;
|
||||
const downloadDir = path.join(baseDir, pptDir.name, 'download');
|
||||
for (const sub of safeReaddir(downloadDir)) {
|
||||
if (!sub.isDirectory()) continue;
|
||||
const slidesDir = path.join(downloadDir, sub.name, 'slides');
|
||||
if (fs.existsSync(slidesDir) && fs.statSync(slidesDir).isDirectory()) {
|
||||
results.push(slidesDir);
|
||||
}
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
function collectHtmlFiles(slidesDir) {
|
||||
return fs.readdirSync(slidesDir)
|
||||
.filter(f => /^slide.*\.html$/i.test(f))
|
||||
.sort()
|
||||
.map(f => path.join(slidesDir, f));
|
||||
}
|
||||
|
||||
async function buildPptxFromDir(slidesDir, outputPath, options = {}) {
|
||||
const { strict = false, fontConfig = null, concurrency = 4 } = options;
|
||||
const htmlFiles = collectHtmlFiles(slidesDir);
|
||||
|
||||
if (htmlFiles.length === 0) {
|
||||
console.log(` [skip] no slide_*.html in ${slidesDir}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
console.log(` ${htmlFiles.length} slides → ${outputPath} (concurrency=${concurrency})`);
|
||||
|
||||
const pptx = new pptxgen();
|
||||
pptx.layout = 'LAYOUT_16x9';
|
||||
|
||||
// Pre-allocate slides in HTML-file order so final slide order is deterministic
|
||||
// regardless of parallel completion order. html2pptx will populate each
|
||||
// pre-allocated slide via the `slide` option instead of calling addSlide().
|
||||
const tasks = htmlFiles.map((htmlFile) => ({
|
||||
htmlFile,
|
||||
slide: pptx.addSlide(),
|
||||
}));
|
||||
|
||||
const failures = [];
|
||||
const mustFix = [];
|
||||
let nextIndex = 0;
|
||||
let doneCount = 0;
|
||||
const total = tasks.length;
|
||||
|
||||
async function worker() {
|
||||
while (true) {
|
||||
const i = nextIndex++;
|
||||
if (i >= total) return;
|
||||
const { htmlFile, slide } = tasks[i];
|
||||
const base = path.basename(htmlFile);
|
||||
console.log(` [start ${i + 1}/${total}] ${base}`);
|
||||
try {
|
||||
const { warnings } = await html2pptx(htmlFile, pptx, { fontConfig, slide });
|
||||
if (warnings && warnings.length > 0) {
|
||||
warnings.forEach(w => {
|
||||
// Severe overflow is a must-fix signal — surface it loudly but do
|
||||
// NOT block output (pptx still writes, exit code stays 0).
|
||||
if (w.includes('MUST FIX')) {
|
||||
mustFix.push({ file: base, message: w });
|
||||
console.error(` ${base}: ${w}`);
|
||||
} else {
|
||||
console.warn(` ${base}: ${w}`);
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(` [error] ${base}: ${err.message}`);
|
||||
failures.push({ file: htmlFile, error: err });
|
||||
if (strict) throw err;
|
||||
} finally {
|
||||
doneCount++;
|
||||
console.log(` [done ${doneCount}/${total}] ${base}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const workerCount = Math.max(1, Math.min(concurrency, total));
|
||||
await Promise.all(Array.from({ length: workerCount }, () => worker()));
|
||||
|
||||
if (failures.length > 0) {
|
||||
console.warn(` ${failures.length} slide(s) failed during conversion.`);
|
||||
}
|
||||
|
||||
// Severe-overflow slides do not block output, but they MUST be fixed — print a
|
||||
// consolidated, hard-to-miss list so the caller knows exactly what to re-render.
|
||||
if (mustFix.length > 0) {
|
||||
console.error(`\n 🚨🚨 ${mustFix.length} slide(s) with SEVERE overflow — MUST FIX and re-run:`);
|
||||
mustFix.forEach(({ file, message }) => console.error(` - ${file}: ${message}`));
|
||||
console.error('');
|
||||
}
|
||||
|
||||
await pptx.writeFile({ fileName: outputPath });
|
||||
console.log(` saved: ${outputPath}`);
|
||||
return outputPath;
|
||||
}
|
||||
|
||||
function defaultOutputPath(slidesDir) {
|
||||
// Save the pptx next to the slides/ folder (one folder up),
|
||||
// named after the parent directory.
|
||||
const parentDir = path.dirname(path.resolve(slidesDir));
|
||||
return path.join(parentDir, `${path.basename(parentDir)}.pptx`);
|
||||
}
|
||||
|
||||
async function runSingle(slidesDir, outputPath, options = {}) {
|
||||
if (!fs.existsSync(slidesDir) || !fs.statSync(slidesDir).isDirectory()) {
|
||||
throw new Error(`slides_dir not found or not a directory: ${slidesDir}`);
|
||||
}
|
||||
const out = outputPath || defaultOutputPath(slidesDir);
|
||||
await buildPptxFromDir(slidesDir, out, options);
|
||||
}
|
||||
|
||||
async function runBatch(baseDir, options = {}) {
|
||||
const slidesDirs = findSlidesDirs(baseDir);
|
||||
console.log(`Found ${slidesDirs.length} slides directories under ${baseDir}`);
|
||||
|
||||
for (const slidesDir of slidesDirs) {
|
||||
const label = path.relative(baseDir, slidesDir);
|
||||
console.log(`\nProcessing: ${label}`);
|
||||
try {
|
||||
const parentDir = path.dirname(slidesDir);
|
||||
const outputPath = path.join(parentDir, `${path.basename(parentDir)}.pptx`);
|
||||
await buildPptxFromDir(slidesDir, outputPath, options);
|
||||
} catch (err) {
|
||||
console.error(` [error] ${slidesDir}: ${err.message}`);
|
||||
if (options.strict) throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function printUsageAndExit(code = 1) {
|
||||
console.error(
|
||||
'Usage:\n' +
|
||||
' node batch_html2pptx.js [--strict] [--concurrency=N] <slides_dir> [output.pptx]\n' +
|
||||
' node batch_html2pptx.js [--strict] [--concurrency=N] --batch <base_dir>\n'
|
||||
);
|
||||
process.exit(code);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const strict = args.includes('--strict');
|
||||
|
||||
let concurrency = 8;
|
||||
const concIdx = args.findIndex(a => a === '--concurrency' || a.startsWith('--concurrency='));
|
||||
if (concIdx !== -1) {
|
||||
const arg = args[concIdx];
|
||||
if (arg.includes('=')) {
|
||||
concurrency = parseInt(arg.split('=')[1], 10);
|
||||
} else {
|
||||
concurrency = parseInt(args[concIdx + 1], 10);
|
||||
}
|
||||
if (!Number.isFinite(concurrency) || concurrency < 1) {
|
||||
console.error(`invalid --concurrency value`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
const filteredArgs = args.filter((arg, i) => {
|
||||
if (arg === '--strict') return false;
|
||||
if (arg === '--concurrency' || arg.startsWith('--concurrency=')) return false;
|
||||
if (concIdx !== -1 && !args[concIdx].includes('=') && i === concIdx + 1) return false;
|
||||
return true;
|
||||
});
|
||||
|
||||
if (filteredArgs.length === 0) printUsageAndExit();
|
||||
|
||||
const runOpts = { strict, concurrency };
|
||||
|
||||
if (filteredArgs[0] === '--batch') {
|
||||
if (!filteredArgs[1]) printUsageAndExit();
|
||||
await runBatch(filteredArgs[1], runOpts);
|
||||
} else if (filteredArgs[0] === '--dir') {
|
||||
// Backwards-compatible alias for batch mode.
|
||||
if (!filteredArgs[1]) printUsageAndExit();
|
||||
await runBatch(filteredArgs[1], runOpts);
|
||||
} else {
|
||||
const slidesDir = filteredArgs[0];
|
||||
const outputPath = filteredArgs[1];
|
||||
await runSingle(slidesDir, outputPath, runOpts);
|
||||
}
|
||||
|
||||
console.log('\nDone.');
|
||||
}
|
||||
|
||||
main().catch(err => {
|
||||
console.error('Fatal:', err);
|
||||
process.exit(1);
|
||||
});
|
||||
Executable
+5046
File diff suppressed because it is too large.
Load diff
Executable
+159
@@ -0,0 +1,159 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tool to pack a directory into a .docx, .pptx, or .xlsx file with XML formatting undone.
|
||||
|
||||
Example usage:
|
||||
python pack.py <input_directory> <office_file> [--force]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import defusedxml.minidom
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Pack a directory into an Office file")
|
||||
parser.add_argument("input_directory", help="Unpacked Office document directory")
|
||||
parser.add_argument("output_file", help="Output Office file (.docx/.pptx/.xlsx)")
|
||||
parser.add_argument("--force", action="store_true", help="Skip validation")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
success = pack_document(
|
||||
args.input_directory, args.output_file, validate=not args.force
|
||||
)
|
||||
|
||||
# Show warning if validation was skipped
|
||||
if args.force:
|
||||
print("Warning: Skipped validation, file may be corrupt", file=sys.stderr)
|
||||
# Exit with error if validation failed
|
||||
elif not success:
|
||||
print("Contents would produce a corrupt file.", file=sys.stderr)
|
||||
print("Please validate XML before repacking.", file=sys.stderr)
|
||||
print("Use --force to skip validation and pack anyway.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except ValueError as e:
|
||||
sys.exit(f"Error: {e}")
|
||||
|
||||
|
||||
def pack_document(input_dir, output_file, validate=False):
|
||||
"""Pack a directory into an Office file (.docx/.pptx/.xlsx).
|
||||
|
||||
Args:
|
||||
input_dir: Path to unpacked Office document directory
|
||||
output_file: Path to output Office file
|
||||
validate: If True, validates with soffice (default: False)
|
||||
|
||||
Returns:
|
||||
bool: True if successful, False if validation failed
|
||||
"""
|
||||
input_dir = Path(input_dir)
|
||||
output_file = Path(output_file)
|
||||
|
||||
if not input_dir.is_dir():
|
||||
raise ValueError(f"{input_dir} is not a directory")
|
||||
if output_file.suffix.lower() not in {".docx", ".pptx", ".xlsx"}:
|
||||
raise ValueError(f"{output_file} must be a .docx, .pptx, or .xlsx file")
|
||||
|
||||
# Work in temporary directory to avoid modifying original
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_content_dir = Path(temp_dir) / "content"
|
||||
shutil.copytree(input_dir, temp_content_dir)
|
||||
|
||||
# Process XML files to remove pretty-printing whitespace
|
||||
for pattern in ["*.xml", "*.rels"]:
|
||||
for xml_file in temp_content_dir.rglob(pattern):
|
||||
condense_xml(xml_file)
|
||||
|
||||
# Create final Office file as zip archive
|
||||
output_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
with zipfile.ZipFile(output_file, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
for f in temp_content_dir.rglob("*"):
|
||||
if f.is_file():
|
||||
zf.write(f, f.relative_to(temp_content_dir))
|
||||
|
||||
# Validate if requested
|
||||
if validate:
|
||||
if not validate_document(output_file):
|
||||
output_file.unlink() # Delete the corrupt file
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def validate_document(doc_path):
|
||||
"""Validate document by converting to HTML with soffice."""
|
||||
# Determine the correct filter based on file extension
|
||||
match doc_path.suffix.lower():
|
||||
case ".docx":
|
||||
filter_name = "html:HTML"
|
||||
case ".pptx":
|
||||
filter_name = "html:impress_html_Export"
|
||||
case ".xlsx":
|
||||
filter_name = "html:HTML (StarCalc)"
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[
|
||||
"soffice",
|
||||
"--headless",
|
||||
"--convert-to",
|
||||
filter_name,
|
||||
"--outdir",
|
||||
temp_dir,
|
||||
str(doc_path),
|
||||
],
|
||||
capture_output=True,
|
||||
timeout=10,
|
||||
text=True,
|
||||
)
|
||||
if not (Path(temp_dir) / f"{doc_path.stem}.html").exists():
|
||||
error_msg = result.stderr.strip() or "Document validation failed"
|
||||
print(f"Validation error: {error_msg}", file=sys.stderr)
|
||||
return False
|
||||
return True
|
||||
except FileNotFoundError:
|
||||
print("Warning: soffice not found. Skipping validation.", file=sys.stderr)
|
||||
return True
|
||||
except subprocess.TimeoutExpired:
|
||||
print("Validation error: Timeout during conversion", file=sys.stderr)
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"Validation error: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
|
||||
def condense_xml(xml_file):
|
||||
"""Strip unnecessary whitespace and remove comments."""
|
||||
with open(xml_file, "r", encoding="utf-8") as f:
|
||||
dom = defusedxml.minidom.parse(f)
|
||||
|
||||
# Process each element to remove whitespace and comments
|
||||
for element in dom.getElementsByTagName("*"):
|
||||
# Skip w:t elements and their processing
|
||||
if element.tagName.endswith(":t"):
|
||||
continue
|
||||
|
||||
# Remove whitespace-only text nodes and comment nodes
|
||||
for child in list(element.childNodes):
|
||||
if (
|
||||
child.nodeType == child.TEXT_NODE
|
||||
and child.nodeValue
|
||||
and child.nodeValue.strip() == ""
|
||||
) or child.nodeType == child.COMMENT_NODE:
|
||||
element.removeChild(child)
|
||||
|
||||
# Write back the condensed XML
|
||||
with open(xml_file, "wb") as f:
|
||||
f.write(dom.toxml(encoding="UTF-8"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Unpack and format XML contents of Office files (.docx, .pptx, .xlsx)"""
|
||||
|
||||
import argparse
|
||||
import random
|
||||
import zipfile
|
||||
import defusedxml.minidom
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Unpack an Office file into a directory")
|
||||
parser.add_argument("office_file", help="Office file (.docx/.pptx/.xlsx)")
|
||||
parser.add_argument("output_dir", help="Output directory")
|
||||
args = parser.parse_args()
|
||||
unpack_document(args.office_file, args.output_dir)
|
||||
|
||||
|
||||
def unpack_document(input_file, output_dir):
|
||||
"""Unpack an Office file into a directory and pretty-print all XML files."""
|
||||
output_path = Path(output_dir)
|
||||
output_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(input_file) as zf:
|
||||
zf.extractall(output_path)
|
||||
|
||||
for pattern in ["*.xml", "*.rels"]:
|
||||
for xml_file in output_path.rglob(pattern):
|
||||
pretty_print_xml(xml_file)
|
||||
|
||||
# For .docx files, suggest an RSID for tracked changes
|
||||
if str(input_file).endswith(".docx"):
|
||||
suggested_rsid = "".join(random.choices("0123456789ABCDEF", k=8))
|
||||
print(f"Suggested RSID for edit session: {suggested_rsid}")
|
||||
|
||||
|
||||
def pretty_print_xml(xml_file):
|
||||
"""Pretty-print a single XML file in place."""
|
||||
content = xml_file.read_text(encoding="utf-8")
|
||||
dom = defusedxml.minidom.parseString(content)
|
||||
xml_file.write_bytes(dom.toprettyxml(indent=" ", encoding="ascii"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+70
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Command line tool to validate Office document XML files against XSD schemas and tracked changes.
|
||||
|
||||
Usage:
|
||||
python validate.py <dir> --original <original_file>
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from validation import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Validate Office document XML files")
|
||||
parser.add_argument(
|
||||
"unpacked_dir",
|
||||
help="Path to unpacked Office document directory",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--original",
|
||||
required=True,
|
||||
help="Path to original file (.docx/.pptx/.xlsx)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-v",
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
help="Enable verbose output",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
# Validate paths
|
||||
unpacked_dir = Path(args.unpacked_dir)
|
||||
original_file = Path(args.original)
|
||||
file_extension = original_file.suffix.lower()
|
||||
if not unpacked_dir.is_dir():
|
||||
sys.exit(f"Error: {unpacked_dir} is not a directory")
|
||||
if not original_file.is_file():
|
||||
sys.exit(f"Error: {original_file} is not a file")
|
||||
if file_extension not in [".docx", ".pptx", ".xlsx"]:
|
||||
sys.exit(f"Error: {original_file} must be a .docx, .pptx, or .xlsx file")
|
||||
|
||||
# Run validations
|
||||
match file_extension:
|
||||
case ".docx":
|
||||
validators = [DOCXSchemaValidator, RedliningValidator]
|
||||
case ".pptx":
|
||||
validators = [PPTXSchemaValidator]
|
||||
case _:
|
||||
print(f"Error: Validation not supported for file type {file_extension}")
|
||||
sys.exit(1)
|
||||
|
||||
# Run validators
|
||||
success = True
|
||||
for V in validators:
|
||||
validator = V(unpacked_dir, original_file, verbose=args.verbose)
|
||||
if not validator.validate():
|
||||
success = False
|
||||
|
||||
if success:
|
||||
print("All validations PASSED!")
|
||||
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
"""
|
||||
Validation modules for Word document processing.
|
||||
"""
|
||||
|
||||
from .base import BaseSchemaValidator
|
||||
from .docx import DOCXSchemaValidator
|
||||
from .pptx import PPTXSchemaValidator
|
||||
from .redlining import RedliningValidator
|
||||
|
||||
__all__ = [
|
||||
"BaseSchemaValidator",
|
||||
"DOCXSchemaValidator",
|
||||
"PPTXSchemaValidator",
|
||||
"RedliningValidator",
|
||||
]
|
||||
Executable
+948
@@ -0,0 +1,948 @@
|
||||
"""
|
||||
Base validator with common validation logic for document files.
|
||||
"""
|
||||
|
||||
import re
|
||||
import tempfile
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import lxml.etree
|
||||
|
||||
|
||||
class BaseSchemaValidator:
|
||||
"""Base validator with common validation logic for document files."""
|
||||
|
||||
# Elements whose 'id' attributes must be unique within their file
|
||||
# Format: element_name -> (attribute_name, scope)
|
||||
# scope can be 'file' (unique within file) or 'global' (unique across all files)
|
||||
UNIQUE_ID_REQUIREMENTS = {
|
||||
# Word elements
|
||||
"comment": ("id", "file"), # Comment IDs in comments.xml
|
||||
"commentrangestart": ("id", "file"), # Must match comment IDs
|
||||
"commentrangeend": ("id", "file"), # Must match comment IDs
|
||||
"bookmarkstart": ("id", "file"), # Bookmark start IDs
|
||||
"bookmarkend": ("id", "file"), # Bookmark end IDs
|
||||
# Note: ins and del (track changes) can share IDs when part of same revision
|
||||
# PowerPoint elements
|
||||
"sldid": ("id", "file"), # Slide IDs in presentation.xml
|
||||
"sldmasterid": ("id", "global"), # Slide master IDs must be globally unique
|
||||
"sldlayoutid": ("id", "global"), # Slide layout IDs must be globally unique
|
||||
"cm": ("authorid", "file"), # Comment author IDs
|
||||
# Excel elements
|
||||
"sheet": ("sheetid", "file"), # Sheet IDs in workbook.xml
|
||||
"definedname": ("id", "file"), # Named range IDs
|
||||
# Drawing/Shape elements (all formats)
|
||||
"cxnsp": ("id", "file"), # Connection shape IDs
|
||||
"sp": ("id", "file"), # Shape IDs
|
||||
"pic": ("id", "file"), # Picture IDs
|
||||
"grpsp": ("id", "file"), # Group shape IDs
|
||||
}
|
||||
|
||||
# Mapping of element names to expected relationship types
|
||||
# Subclasses should override this with format-specific mappings
|
||||
ELEMENT_RELATIONSHIP_TYPES = {}
|
||||
|
||||
# Unified schema mappings for all Office document types
|
||||
SCHEMA_MAPPINGS = {
|
||||
# Document type specific schemas
|
||||
"word": "ISO-IEC29500-4_2016/wml.xsd", # Word documents
|
||||
"ppt": "ISO-IEC29500-4_2016/pml.xsd", # PowerPoint presentations
|
||||
"xl": "ISO-IEC29500-4_2016/sml.xsd", # Excel spreadsheets
|
||||
# Common file types
|
||||
"[Content_Types].xml": "ecma/fouth-edition/opc-contentTypes.xsd",
|
||||
"app.xml": "ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd",
|
||||
"core.xml": "ecma/fouth-edition/opc-coreProperties.xsd",
|
||||
"custom.xml": "ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd",
|
||||
".rels": "ecma/fouth-edition/opc-relationships.xsd",
|
||||
# Word-specific files
|
||||
"people.xml": "microsoft/wml-2012.xsd",
|
||||
"commentsIds.xml": "microsoft/wml-cid-2016.xsd",
|
||||
"commentsExtensible.xml": "microsoft/wml-cex-2018.xsd",
|
||||
"commentsExtended.xml": "microsoft/wml-2012.xsd",
|
||||
# Chart files (common across document types)
|
||||
"chart": "ISO-IEC29500-4_2016/dml-chart.xsd",
|
||||
# Theme files (common across document types)
|
||||
"theme": "ISO-IEC29500-4_2016/dml-main.xsd",
|
||||
# Drawing and media files
|
||||
"drawing": "ISO-IEC29500-4_2016/dml-main.xsd",
|
||||
}
|
||||
|
||||
# Unified namespace constants
|
||||
MC_NAMESPACE = "http://schemas.openxmlformats.org/markup-compatibility/2006"
|
||||
XML_NAMESPACE = "http://www.w3.org/XML/1998/namespace"
|
||||
|
||||
# Common OOXML namespaces used across validators
|
||||
PACKAGE_RELATIONSHIPS_NAMESPACE = (
|
||||
"http://schemas.openxmlformats.org/package/2006/relationships"
|
||||
)
|
||||
OFFICE_RELATIONSHIPS_NAMESPACE = (
|
||||
"http://schemas.openxmlformats.org/officeDocument/2006/relationships"
|
||||
)
|
||||
CONTENT_TYPES_NAMESPACE = (
|
||||
"http://schemas.openxmlformats.org/package/2006/content-types"
|
||||
)
|
||||
|
||||
# Folders where we should clean ignorable namespaces
|
||||
MAIN_CONTENT_FOLDERS = {"word", "ppt", "xl"}
|
||||
|
||||
# All allowed OOXML namespaces (superset of all document types)
|
||||
OOXML_NAMESPACES = {
|
||||
"http://schemas.openxmlformats.org/officeDocument/2006/math",
|
||||
"http://schemas.openxmlformats.org/officeDocument/2006/relationships",
|
||||
"http://schemas.openxmlformats.org/schemaLibrary/2006/main",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/main",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/chart",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/chartDrawing",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/diagram",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/picture",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/spreadsheetDrawing",
|
||||
"http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing",
|
||||
"http://schemas.openxmlformats.org/wordprocessingml/2006/main",
|
||||
"http://schemas.openxmlformats.org/presentationml/2006/main",
|
||||
"http://schemas.openxmlformats.org/spreadsheetml/2006/main",
|
||||
"http://schemas.openxmlformats.org/officeDocument/2006/sharedTypes",
|
||||
"http://www.w3.org/XML/1998/namespace",
|
||||
}
|
||||
|
||||
def __init__(self, unpacked_dir, original_file, verbose=False):
|
||||
self.unpacked_dir = Path(unpacked_dir).resolve()
|
||||
self.original_file = Path(original_file)
|
||||
self.verbose = verbose
|
||||
|
||||
# Set schemas directory
|
||||
self.schemas_dir = Path(__file__).parent.parent.parent / "schemas"
|
||||
|
||||
# Get all XML and .rels files
|
||||
patterns = ["*.xml", "*.rels"]
|
||||
self.xml_files = [
|
||||
f for pattern in patterns for f in self.unpacked_dir.rglob(pattern)
|
||||
]
|
||||
|
||||
if not self.xml_files:
|
||||
print(f"Warning: No XML files found in {self.unpacked_dir}")
|
||||
|
||||
def validate(self):
|
||||
"""Run all validation checks and return True if all pass."""
|
||||
raise NotImplementedError("Subclasses must implement the validate method")
|
||||
|
||||
def validate_xml(self):
|
||||
"""Validate that all XML files are well-formed."""
|
||||
errors = []
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
try:
|
||||
# Try to parse the XML file
|
||||
lxml.etree.parse(str(xml_file))
|
||||
except lxml.etree.XMLSyntaxError as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {e.lineno}: {e.msg}"
|
||||
)
|
||||
except Exception as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Unexpected error: {str(e)}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} XML violations:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All XML files are well-formed")
|
||||
return True
|
||||
|
||||
def validate_namespaces(self):
|
||||
"""Validate that namespace prefixes in Ignorable attributes are declared."""
|
||||
errors = []
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
declared = set(root.nsmap.keys()) - {None} # Exclude default namespace
|
||||
|
||||
for attr_val in [
|
||||
v for k, v in root.attrib.items() if k.endswith("Ignorable")
|
||||
]:
|
||||
undeclared = set(attr_val.split()) - declared
|
||||
errors.extend(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Namespace '{ns}' in Ignorable but not declared"
|
||||
for ns in undeclared
|
||||
)
|
||||
except lxml.etree.XMLSyntaxError:
|
||||
continue
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - {len(errors)} namespace issues:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
if self.verbose:
|
||||
print("PASSED - All namespace prefixes properly declared")
|
||||
return True
|
||||
|
||||
def validate_unique_ids(self):
|
||||
"""Validate that specific IDs are unique according to OOXML requirements."""
|
||||
errors = []
|
||||
global_ids = {} # Track globally unique IDs across all files
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
file_ids = {} # Track IDs that must be unique within this file
|
||||
|
||||
# Remove all mc:AlternateContent elements from the tree
|
||||
mc_elements = root.xpath(
|
||||
".//mc:AlternateContent", namespaces={"mc": self.MC_NAMESPACE}
|
||||
)
|
||||
for elem in mc_elements:
|
||||
elem.getparent().remove(elem)
|
||||
|
||||
# Now check IDs in the cleaned tree
|
||||
for elem in root.iter():
|
||||
# Get the element name without namespace
|
||||
tag = (
|
||||
elem.tag.split("}")[-1].lower()
|
||||
if "}" in elem.tag
|
||||
else elem.tag.lower()
|
||||
)
|
||||
|
||||
# Check if this element type has ID uniqueness requirements
|
||||
if tag in self.UNIQUE_ID_REQUIREMENTS:
|
||||
attr_name, scope = self.UNIQUE_ID_REQUIREMENTS[tag]
|
||||
|
||||
# Look for the specified attribute
|
||||
id_value = None
|
||||
for attr, value in elem.attrib.items():
|
||||
attr_local = (
|
||||
attr.split("}")[-1].lower()
|
||||
if "}" in attr
|
||||
else attr.lower()
|
||||
)
|
||||
if attr_local == attr_name:
|
||||
id_value = value
|
||||
break
|
||||
|
||||
if id_value is not None:
|
||||
if scope == "global":
|
||||
# Check global uniqueness
|
||||
if id_value in global_ids:
|
||||
prev_file, prev_line, prev_tag = global_ids[
|
||||
id_value
|
||||
]
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {elem.sourceline}: Global ID '{id_value}' in <{tag}> "
|
||||
f"already used in {prev_file} at line {prev_line} in <{prev_tag}>"
|
||||
)
|
||||
else:
|
||||
global_ids[id_value] = (
|
||||
xml_file.relative_to(self.unpacked_dir),
|
||||
elem.sourceline,
|
||||
tag,
|
||||
)
|
||||
elif scope == "file":
|
||||
# Check file-level uniqueness
|
||||
key = (tag, attr_name)
|
||||
if key not in file_ids:
|
||||
file_ids[key] = {}
|
||||
|
||||
if id_value in file_ids[key]:
|
||||
prev_line = file_ids[key][id_value]
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {elem.sourceline}: Duplicate {attr_name}='{id_value}' in <{tag}> "
|
||||
f"(first occurrence at line {prev_line})"
|
||||
)
|
||||
else:
|
||||
file_ids[key][id_value] = elem.sourceline
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} ID uniqueness violations:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All required IDs are unique")
|
||||
return True
|
||||
|
||||
def validate_file_references(self):
|
||||
"""
|
||||
Validate that all .rels files properly reference files and that all files are referenced.
|
||||
"""
|
||||
errors = []
|
||||
|
||||
# Find all .rels files
|
||||
rels_files = list(self.unpacked_dir.rglob("*.rels"))
|
||||
|
||||
if not rels_files:
|
||||
if self.verbose:
|
||||
print("PASSED - No .rels files found")
|
||||
return True
|
||||
|
||||
# Get all files in the unpacked directory (excluding reference files)
|
||||
all_files = []
|
||||
for file_path in self.unpacked_dir.rglob("*"):
|
||||
if (
|
||||
file_path.is_file()
|
||||
and file_path.name != "[Content_Types].xml"
|
||||
and not file_path.name.endswith(".rels")
|
||||
): # This file is not referenced by .rels
|
||||
all_files.append(file_path.resolve())
|
||||
|
||||
# Track all files that are referenced by any .rels file
|
||||
all_referenced_files = set()
|
||||
|
||||
if self.verbose:
|
||||
print(
|
||||
f"Found {len(rels_files)} .rels files and {len(all_files)} target files"
|
||||
)
|
||||
|
||||
# Check each .rels file
|
||||
for rels_file in rels_files:
|
||||
try:
|
||||
# Parse relationships file
|
||||
rels_root = lxml.etree.parse(str(rels_file)).getroot()
|
||||
|
||||
# Get the directory where this .rels file is located
|
||||
rels_dir = rels_file.parent
|
||||
|
||||
# Find all relationships and their targets
|
||||
referenced_files = set()
|
||||
broken_refs = []
|
||||
|
||||
for rel in rels_root.findall(
|
||||
".//ns:Relationship",
|
||||
namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE},
|
||||
):
|
||||
target = rel.get("Target")
|
||||
if target and not target.startswith(
|
||||
("http", "mailto:")
|
||||
): # Skip external URLs
|
||||
# Resolve the target path relative to the .rels file location
|
||||
if rels_file.name == ".rels":
|
||||
# Root .rels file - targets are relative to unpacked_dir
|
||||
target_path = self.unpacked_dir / target
|
||||
else:
|
||||
# Other .rels files - targets are relative to their parent's parent
|
||||
# e.g., word/_rels/document.xml.rels -> targets relative to word/
|
||||
base_dir = rels_dir.parent
|
||||
target_path = base_dir / target
|
||||
|
||||
# Normalize the path and check if it exists
|
||||
try:
|
||||
target_path = target_path.resolve()
|
||||
if target_path.exists() and target_path.is_file():
|
||||
referenced_files.add(target_path)
|
||||
all_referenced_files.add(target_path)
|
||||
else:
|
||||
broken_refs.append((target, rel.sourceline))
|
||||
except (OSError, ValueError):
|
||||
broken_refs.append((target, rel.sourceline))
|
||||
|
||||
# Report broken references
|
||||
if broken_refs:
|
||||
rel_path = rels_file.relative_to(self.unpacked_dir)
|
||||
for broken_ref, line_num in broken_refs:
|
||||
errors.append(
|
||||
f" {rel_path}: Line {line_num}: Broken reference to {broken_ref}"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
rel_path = rels_file.relative_to(self.unpacked_dir)
|
||||
errors.append(f" Error parsing {rel_path}: {e}")
|
||||
|
||||
# Check for unreferenced files (files that exist but are not referenced anywhere)
|
||||
unreferenced_files = set(all_files) - all_referenced_files
|
||||
|
||||
if unreferenced_files:
|
||||
for unref_file in sorted(unreferenced_files):
|
||||
unref_rel_path = unref_file.relative_to(self.unpacked_dir)
|
||||
errors.append(f" Unreferenced file: {unref_rel_path}")
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} relationship validation errors:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
print(
|
||||
"CRITICAL: These errors will cause the document to appear corrupt. "
|
||||
+ "Broken references MUST be fixed, "
|
||||
+ "and unreferenced files MUST be referenced or removed."
|
||||
)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print(
|
||||
"PASSED - All references are valid and all files are properly referenced"
|
||||
)
|
||||
return True
|
||||
|
||||
def validate_all_relationship_ids(self):
|
||||
"""
|
||||
Validate that all r:id attributes in XML files reference existing IDs
|
||||
in their corresponding .rels files, and optionally validate relationship types.
|
||||
"""
|
||||
errors = []
|
||||
|
||||
# Process each XML file that might contain r:id references
|
||||
for xml_file in self.xml_files:
|
||||
# Skip .rels files themselves
|
||||
if xml_file.suffix == ".rels":
|
||||
continue
|
||||
|
||||
# Determine the corresponding .rels file
|
||||
# For dir/file.xml, it's dir/_rels/file.xml.rels
|
||||
rels_dir = xml_file.parent / "_rels"
|
||||
rels_file = rels_dir / f"{xml_file.name}.rels"
|
||||
|
||||
# Skip if there's no corresponding .rels file (that's okay)
|
||||
if not rels_file.exists():
|
||||
continue
|
||||
|
||||
try:
|
||||
# Parse the .rels file to get valid relationship IDs and their types
|
||||
rels_root = lxml.etree.parse(str(rels_file)).getroot()
|
||||
rid_to_type = {}
|
||||
|
||||
for rel in rels_root.findall(
|
||||
f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship"
|
||||
):
|
||||
rid = rel.get("Id")
|
||||
rel_type = rel.get("Type", "")
|
||||
if rid:
|
||||
# Check for duplicate rIds
|
||||
if rid in rid_to_type:
|
||||
rels_rel_path = rels_file.relative_to(self.unpacked_dir)
|
||||
errors.append(
|
||||
f" {rels_rel_path}: Line {rel.sourceline}: "
|
||||
f"Duplicate relationship ID '{rid}' (IDs must be unique)"
|
||||
)
|
||||
# Extract just the type name from the full URL
|
||||
type_name = (
|
||||
rel_type.split("/")[-1] if "/" in rel_type else rel_type
|
||||
)
|
||||
rid_to_type[rid] = type_name
|
||||
|
||||
# Parse the XML file to find all r:id references
|
||||
xml_root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
|
||||
# Find all elements with r:id attributes
|
||||
for elem in xml_root.iter():
|
||||
# Check for r:id attribute (relationship ID)
|
||||
rid_attr = elem.get(f"{{{self.OFFICE_RELATIONSHIPS_NAMESPACE}}}id")
|
||||
if rid_attr:
|
||||
xml_rel_path = xml_file.relative_to(self.unpacked_dir)
|
||||
elem_name = (
|
||||
elem.tag.split("}")[-1] if "}" in elem.tag else elem.tag
|
||||
)
|
||||
|
||||
# Check if the ID exists
|
||||
if rid_attr not in rid_to_type:
|
||||
errors.append(
|
||||
f" {xml_rel_path}: Line {elem.sourceline}: "
|
||||
f"<{elem_name}> references non-existent relationship '{rid_attr}' "
|
||||
f"(valid IDs: {', '.join(sorted(rid_to_type.keys())[:5])}{'...' if len(rid_to_type) > 5 else ''})"
|
||||
)
|
||||
# Check if we have type expectations for this element
|
||||
elif self.ELEMENT_RELATIONSHIP_TYPES:
|
||||
expected_type = self._get_expected_relationship_type(
|
||||
elem_name
|
||||
)
|
||||
if expected_type:
|
||||
actual_type = rid_to_type[rid_attr]
|
||||
# Check if the actual type matches or contains the expected type
|
||||
if expected_type not in actual_type.lower():
|
||||
errors.append(
|
||||
f" {xml_rel_path}: Line {elem.sourceline}: "
|
||||
f"<{elem_name}> references '{rid_attr}' which points to '{actual_type}' "
|
||||
f"but should point to a '{expected_type}' relationship"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
xml_rel_path = xml_file.relative_to(self.unpacked_dir)
|
||||
errors.append(f" Error processing {xml_rel_path}: {e}")
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} relationship ID reference errors:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
print("\nThese ID mismatches will cause the document to appear corrupt!")
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All relationship ID references are valid")
|
||||
return True
|
||||
|
||||
def _get_expected_relationship_type(self, element_name):
|
||||
"""
|
||||
Get the expected relationship type for an element.
|
||||
First checks the explicit mapping, then tries pattern detection.
|
||||
"""
|
||||
# Normalize element name to lowercase
|
||||
elem_lower = element_name.lower()
|
||||
|
||||
# Check explicit mapping first
|
||||
if elem_lower in self.ELEMENT_RELATIONSHIP_TYPES:
|
||||
return self.ELEMENT_RELATIONSHIP_TYPES[elem_lower]
|
||||
|
||||
# Try pattern detection for common patterns
|
||||
# Pattern 1: Elements ending in "Id" often expect a relationship of the prefix type
|
||||
if elem_lower.endswith("id") and len(elem_lower) > 2:
|
||||
# e.g., "sldId" -> "sld", "sldMasterId" -> "sldMaster"
|
||||
prefix = elem_lower[:-2] # Remove "id"
|
||||
# Check if this might be a compound like "sldMasterId"
|
||||
if prefix.endswith("master"):
|
||||
return prefix.lower()
|
||||
elif prefix.endswith("layout"):
|
||||
return prefix.lower()
|
||||
else:
|
||||
# Simple case like "sldId" -> "slide"
|
||||
# Common transformations
|
||||
if prefix == "sld":
|
||||
return "slide"
|
||||
return prefix.lower()
|
||||
|
||||
# Pattern 2: Elements ending in "Reference" expect a relationship of the prefix type
|
||||
if elem_lower.endswith("reference") and len(elem_lower) > 9:
|
||||
prefix = elem_lower[:-9] # Remove "reference"
|
||||
return prefix.lower()
|
||||
|
||||
return None
|
||||
|
||||
def validate_content_types(self):
|
||||
"""Validate that all content files are properly declared in [Content_Types].xml."""
|
||||
errors = []
|
||||
|
||||
# Find [Content_Types].xml file
|
||||
content_types_file = self.unpacked_dir / "[Content_Types].xml"
|
||||
if not content_types_file.exists():
|
||||
print("FAILED - [Content_Types].xml file not found")
|
||||
return False
|
||||
|
||||
try:
|
||||
# Parse and get all declared parts and extensions
|
||||
root = lxml.etree.parse(str(content_types_file)).getroot()
|
||||
declared_parts = set()
|
||||
declared_extensions = set()
|
||||
|
||||
# Get Override declarations (specific files)
|
||||
for override in root.findall(
|
||||
f".//{{{self.CONTENT_TYPES_NAMESPACE}}}Override"
|
||||
):
|
||||
part_name = override.get("PartName")
|
||||
if part_name is not None:
|
||||
declared_parts.add(part_name.lstrip("/"))
|
||||
|
||||
# Get Default declarations (by extension)
|
||||
for default in root.findall(
|
||||
f".//{{{self.CONTENT_TYPES_NAMESPACE}}}Default"
|
||||
):
|
||||
extension = default.get("Extension")
|
||||
if extension is not None:
|
||||
declared_extensions.add(extension.lower())
|
||||
|
||||
# Root elements that require content type declaration
|
||||
declarable_roots = {
|
||||
"sld",
|
||||
"sldLayout",
|
||||
"sldMaster",
|
||||
"presentation", # PowerPoint
|
||||
"document", # Word
|
||||
"workbook",
|
||||
"worksheet", # Excel
|
||||
"theme", # Common
|
||||
}
|
||||
|
||||
# Common media file extensions that should be declared
|
||||
media_extensions = {
|
||||
"png": "image/png",
|
||||
"jpg": "image/jpeg",
|
||||
"jpeg": "image/jpeg",
|
||||
"gif": "image/gif",
|
||||
"bmp": "image/bmp",
|
||||
"tiff": "image/tiff",
|
||||
"wmf": "image/x-wmf",
|
||||
"emf": "image/x-emf",
|
||||
}
|
||||
|
||||
# Get all files in the unpacked directory
|
||||
all_files = list(self.unpacked_dir.rglob("*"))
|
||||
all_files = [f for f in all_files if f.is_file()]
|
||||
|
||||
# Check all XML files for Override declarations
|
||||
for xml_file in self.xml_files:
|
||||
path_str = str(xml_file.relative_to(self.unpacked_dir)).replace(
|
||||
"\\", "/"
|
||||
)
|
||||
|
||||
# Skip non-content files
|
||||
if any(
|
||||
skip in path_str
|
||||
for skip in [".rels", "[Content_Types]", "docProps/", "_rels/"]
|
||||
):
|
||||
continue
|
||||
|
||||
try:
|
||||
root_tag = lxml.etree.parse(str(xml_file)).getroot().tag
|
||||
root_name = root_tag.split("}")[-1] if "}" in root_tag else root_tag
|
||||
|
||||
if root_name in declarable_roots and path_str not in declared_parts:
|
||||
errors.append(
|
||||
f" {path_str}: File with <{root_name}> root not declared in [Content_Types].xml"
|
||||
)
|
||||
|
||||
except Exception:
|
||||
continue # Skip unparseable files
|
||||
|
||||
# Check all non-XML files for Default extension declarations
|
||||
for file_path in all_files:
|
||||
# Skip XML files and metadata files (already checked above)
|
||||
if file_path.suffix.lower() in {".xml", ".rels"}:
|
||||
continue
|
||||
if file_path.name == "[Content_Types].xml":
|
||||
continue
|
||||
if "_rels" in file_path.parts or "docProps" in file_path.parts:
|
||||
continue
|
||||
|
||||
extension = file_path.suffix.lstrip(".").lower()
|
||||
if extension and extension not in declared_extensions:
|
||||
# Check if it's a known media extension that should be declared
|
||||
if extension in media_extensions:
|
||||
relative_path = file_path.relative_to(self.unpacked_dir)
|
||||
errors.append(
|
||||
f' {relative_path}: File with extension \'{extension}\' not declared in [Content_Types].xml - should add: <Default Extension="{extension}" ContentType="{media_extensions[extension]}"/>'
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
errors.append(f" Error parsing [Content_Types].xml: {e}")
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} content type declaration errors:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print(
|
||||
"PASSED - All content files are properly declared in [Content_Types].xml"
|
||||
)
|
||||
return True
|
||||
|
||||
def validate_file_against_xsd(self, xml_file, verbose=False):
|
||||
"""Validate a single XML file against XSD schema, comparing with original.
|
||||
|
||||
Args:
|
||||
xml_file: Path to XML file to validate
|
||||
verbose: Enable verbose output
|
||||
|
||||
Returns:
|
||||
tuple: (is_valid, new_errors_set) where is_valid is True/False/None (skipped)
|
||||
"""
|
||||
# Resolve both paths to handle symlinks
|
||||
xml_file = Path(xml_file).resolve()
|
||||
unpacked_dir = self.unpacked_dir.resolve()
|
||||
|
||||
# Validate current file
|
||||
is_valid, current_errors = self._validate_single_file_xsd(
|
||||
xml_file, unpacked_dir
|
||||
)
|
||||
|
||||
if is_valid is None:
|
||||
return None, set() # Skipped
|
||||
elif is_valid:
|
||||
return True, set() # Valid, no errors
|
||||
|
||||
# Get errors from original file for this specific file
|
||||
original_errors = self._get_original_file_errors(xml_file)
|
||||
|
||||
# Compare with original (both are guaranteed to be sets here)
|
||||
assert current_errors is not None
|
||||
new_errors = current_errors - original_errors
|
||||
|
||||
if new_errors:
|
||||
if verbose:
|
||||
relative_path = xml_file.relative_to(unpacked_dir)
|
||||
print(f"FAILED - {relative_path}: {len(new_errors)} new error(s)")
|
||||
for error in list(new_errors)[:3]:
|
||||
truncated = error[:250] + "..." if len(error) > 250 else error
|
||||
print(f" - {truncated}")
|
||||
return False, new_errors
|
||||
else:
|
||||
# All errors existed in original
|
||||
if verbose:
|
||||
print(
|
||||
f"PASSED - No new errors (original had {len(current_errors)} errors)"
|
||||
)
|
||||
return True, set()
|
||||
|
||||
def validate_against_xsd(self):
|
||||
"""Validate XML files against XSD schemas, showing only new errors compared to original."""
|
||||
new_errors = []
|
||||
original_error_count = 0
|
||||
valid_count = 0
|
||||
skipped_count = 0
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
relative_path = str(xml_file.relative_to(self.unpacked_dir))
|
||||
is_valid, new_file_errors = self.validate_file_against_xsd(
|
||||
xml_file, verbose=False
|
||||
)
|
||||
|
||||
if is_valid is None:
|
||||
skipped_count += 1
|
||||
continue
|
||||
elif is_valid and not new_file_errors:
|
||||
valid_count += 1
|
||||
continue
|
||||
elif is_valid:
|
||||
# Had errors but all existed in original
|
||||
original_error_count += 1
|
||||
valid_count += 1
|
||||
continue
|
||||
|
||||
# Has new errors
|
||||
new_errors.append(f" {relative_path}: {len(new_file_errors)} new error(s)")
|
||||
for error in list(new_file_errors)[:3]: # Show first 3 errors
|
||||
new_errors.append(
|
||||
f" - {error[:250]}..." if len(error) > 250 else f" - {error}"
|
||||
)
|
||||
|
||||
# Print summary
|
||||
if self.verbose:
|
||||
print(f"Validated {len(self.xml_files)} files:")
|
||||
print(f" - Valid: {valid_count}")
|
||||
print(f" - Skipped (no schema): {skipped_count}")
|
||||
if original_error_count:
|
||||
print(f" - With original errors (ignored): {original_error_count}")
|
||||
print(
|
||||
f" - With NEW errors: {len(new_errors) > 0 and len([e for e in new_errors if not e.startswith(' ')]) or 0}"
|
||||
)
|
||||
|
||||
if new_errors:
|
||||
print("\nFAILED - Found NEW validation errors:")
|
||||
for error in new_errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("\nPASSED - No new XSD validation errors introduced")
|
||||
return True
|
||||
|
||||
def _get_schema_path(self, xml_file):
|
||||
"""Determine the appropriate schema path for an XML file."""
|
||||
# Check exact filename match
|
||||
if xml_file.name in self.SCHEMA_MAPPINGS:
|
||||
return self.schemas_dir / self.SCHEMA_MAPPINGS[xml_file.name]
|
||||
|
||||
# Check .rels files
|
||||
if xml_file.suffix == ".rels":
|
||||
return self.schemas_dir / self.SCHEMA_MAPPINGS[".rels"]
|
||||
|
||||
# Check chart files
|
||||
if "charts/" in str(xml_file) and xml_file.name.startswith("chart"):
|
||||
return self.schemas_dir / self.SCHEMA_MAPPINGS["chart"]
|
||||
|
||||
# Check theme files
|
||||
if "theme/" in str(xml_file) and xml_file.name.startswith("theme"):
|
||||
return self.schemas_dir / self.SCHEMA_MAPPINGS["theme"]
|
||||
|
||||
# Check if file is in a main content folder and use appropriate schema
|
||||
if xml_file.parent.name in self.MAIN_CONTENT_FOLDERS:
|
||||
return self.schemas_dir / self.SCHEMA_MAPPINGS[xml_file.parent.name]
|
||||
|
||||
return None
|
||||
|
||||
def _clean_ignorable_namespaces(self, xml_doc):
|
||||
"""Remove attributes and elements not in allowed namespaces."""
|
||||
# Create a clean copy
|
||||
xml_string = lxml.etree.tostring(xml_doc, encoding="unicode")
|
||||
xml_copy = lxml.etree.fromstring(xml_string)
|
||||
|
||||
# Remove attributes not in allowed namespaces
|
||||
for elem in xml_copy.iter():
|
||||
attrs_to_remove = []
|
||||
|
||||
for attr in elem.attrib:
|
||||
# Check if attribute is from a namespace other than allowed ones
|
||||
if "{" in attr:
|
||||
ns = attr.split("}")[0][1:]
|
||||
if ns not in self.OOXML_NAMESPACES:
|
||||
attrs_to_remove.append(attr)
|
||||
|
||||
# Remove collected attributes
|
||||
for attr in attrs_to_remove:
|
||||
del elem.attrib[attr]
|
||||
|
||||
# Remove elements not in allowed namespaces
|
||||
self._remove_ignorable_elements(xml_copy)
|
||||
|
||||
return lxml.etree.ElementTree(xml_copy)
|
||||
|
||||
def _remove_ignorable_elements(self, root):
|
||||
"""Recursively remove all elements not in allowed namespaces."""
|
||||
elements_to_remove = []
|
||||
|
||||
# Find elements to remove
|
||||
for elem in list(root):
|
||||
# Skip non-element nodes (comments, processing instructions, etc.)
|
||||
if not hasattr(elem, "tag") or callable(elem.tag):
|
||||
continue
|
||||
|
||||
tag_str = str(elem.tag)
|
||||
if tag_str.startswith("{"):
|
||||
ns = tag_str.split("}")[0][1:]
|
||||
if ns not in self.OOXML_NAMESPACES:
|
||||
elements_to_remove.append(elem)
|
||||
continue
|
||||
|
||||
# Recursively clean child elements
|
||||
self._remove_ignorable_elements(elem)
|
||||
|
||||
# Remove collected elements
|
||||
for elem in elements_to_remove:
|
||||
root.remove(elem)
|
||||
|
||||
def _preprocess_for_mc_ignorable(self, xml_doc):
|
||||
"""Preprocess XML to handle mc:Ignorable attribute properly."""
|
||||
# Remove mc:Ignorable attributes before validation
|
||||
root = xml_doc.getroot()
|
||||
|
||||
# Remove mc:Ignorable attribute from root
|
||||
if f"{{{self.MC_NAMESPACE}}}Ignorable" in root.attrib:
|
||||
del root.attrib[f"{{{self.MC_NAMESPACE}}}Ignorable"]
|
||||
|
||||
return xml_doc
|
||||
|
||||
def _validate_single_file_xsd(self, xml_file, base_path):
|
||||
"""Validate a single XML file against XSD schema. Returns (is_valid, errors_set)."""
|
||||
schema_path = self._get_schema_path(xml_file)
|
||||
if not schema_path:
|
||||
return None, None # Skip file
|
||||
|
||||
try:
|
||||
# Load schema
|
||||
with open(schema_path, "rb") as xsd_file:
|
||||
parser = lxml.etree.XMLParser()
|
||||
xsd_doc = lxml.etree.parse(
|
||||
xsd_file, parser=parser, base_url=str(schema_path)
|
||||
)
|
||||
schema = lxml.etree.XMLSchema(xsd_doc)
|
||||
|
||||
# Load and preprocess XML
|
||||
with open(xml_file, "r") as f:
|
||||
xml_doc = lxml.etree.parse(f)
|
||||
|
||||
xml_doc, _ = self._remove_template_tags_from_text_nodes(xml_doc)
|
||||
xml_doc = self._preprocess_for_mc_ignorable(xml_doc)
|
||||
|
||||
# Clean ignorable namespaces if needed
|
||||
relative_path = xml_file.relative_to(base_path)
|
||||
if (
|
||||
relative_path.parts
|
||||
and relative_path.parts[0] in self.MAIN_CONTENT_FOLDERS
|
||||
):
|
||||
xml_doc = self._clean_ignorable_namespaces(xml_doc)
|
||||
|
||||
# Validate
|
||||
if schema.validate(xml_doc):
|
||||
return True, set()
|
||||
else:
|
||||
errors = set()
|
||||
for error in schema.error_log:
|
||||
# Store normalized error message (without line numbers for comparison)
|
||||
errors.add(error.message)
|
||||
return False, errors
|
||||
|
||||
except Exception as e:
|
||||
return False, {str(e)}
|
||||
|
||||
def _get_original_file_errors(self, xml_file):
|
||||
"""Get XSD validation errors from a single file in the original document.
|
||||
|
||||
Args:
|
||||
xml_file: Path to the XML file in unpacked_dir to check
|
||||
|
||||
Returns:
|
||||
set: Set of error messages from the original file
|
||||
"""
|
||||
# Resolve both paths to handle symlinks (e.g., /var vs /private/var on macOS)
|
||||
xml_file = Path(xml_file).resolve()
|
||||
unpacked_dir = self.unpacked_dir.resolve()
|
||||
relative_path = xml_file.relative_to(unpacked_dir)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
|
||||
# Extract original file
|
||||
with zipfile.ZipFile(self.original_file, "r") as zip_ref:
|
||||
zip_ref.extractall(temp_path)
|
||||
|
||||
# Find corresponding file in original
|
||||
original_xml_file = temp_path / relative_path
|
||||
|
||||
if not original_xml_file.exists():
|
||||
# File didn't exist in original, so no original errors
|
||||
return set()
|
||||
|
||||
# Validate the specific file in original
|
||||
is_valid, errors = self._validate_single_file_xsd(
|
||||
original_xml_file, temp_path
|
||||
)
|
||||
return errors if errors else set()
|
||||
|
||||
def _remove_template_tags_from_text_nodes(self, xml_doc):
|
||||
"""Remove template tags from XML text nodes and collect warnings.
|
||||
|
||||
Template tags follow the pattern {{ ... }} and are used as placeholders
|
||||
for content replacement. They should be removed from text content before
|
||||
XSD validation while preserving XML structure.
|
||||
|
||||
Returns:
|
||||
tuple: (cleaned_xml_doc, warnings_list)
|
||||
"""
|
||||
warnings = []
|
||||
template_pattern = re.compile(r"\{\{[^}]*\}\}")
|
||||
|
||||
# Create a copy of the document to avoid modifying the original
|
||||
xml_string = lxml.etree.tostring(xml_doc, encoding="unicode")
|
||||
xml_copy = lxml.etree.fromstring(xml_string)
|
||||
|
||||
def process_text_content(text, content_type):
|
||||
if not text:
|
||||
return text
|
||||
matches = list(template_pattern.finditer(text))
|
||||
if matches:
|
||||
for match in matches:
|
||||
warnings.append(
|
||||
f"Found template tag in {content_type}: {match.group()}"
|
||||
)
|
||||
return template_pattern.sub("", text)
|
||||
return text
|
||||
|
||||
# Process all text nodes in the document
|
||||
for elem in xml_copy.iter():
|
||||
# Skip processing if this is a w:t element
|
||||
if not hasattr(elem, "tag") or callable(elem.tag):
|
||||
continue
|
||||
tag_str = str(elem.tag)
|
||||
if tag_str.endswith("}t") or tag_str == "t":
|
||||
continue
|
||||
|
||||
elem.text = process_text_content(elem.text, "text content")
|
||||
elem.tail = process_text_content(elem.tail, "tail content")
|
||||
|
||||
return lxml.etree.ElementTree(xml_copy), warnings
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise RuntimeError("This module should not be run directly.")
|
||||
Executable
+274
@@ -0,0 +1,274 @@
|
||||
"""
|
||||
Validator for Word document XML files against XSD schemas.
|
||||
"""
|
||||
|
||||
import re
|
||||
import tempfile
|
||||
import zipfile
|
||||
|
||||
import lxml.etree
|
||||
|
||||
from .base import BaseSchemaValidator
|
||||
|
||||
|
||||
class DOCXSchemaValidator(BaseSchemaValidator):
|
||||
"""Validator for Word document XML files against XSD schemas."""
|
||||
|
||||
# Word-specific namespace
|
||||
WORD_2006_NAMESPACE = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
|
||||
|
||||
# Word-specific element to relationship type mappings
|
||||
# Start with empty mapping - add specific cases as we discover them
|
||||
ELEMENT_RELATIONSHIP_TYPES = {}
|
||||
|
||||
def validate(self):
|
||||
"""Run all validation checks and return True if all pass."""
|
||||
# Test 0: XML well-formedness
|
||||
if not self.validate_xml():
|
||||
return False
|
||||
|
||||
# Test 1: Namespace declarations
|
||||
all_valid = True
|
||||
if not self.validate_namespaces():
|
||||
all_valid = False
|
||||
|
||||
# Test 2: Unique IDs
|
||||
if not self.validate_unique_ids():
|
||||
all_valid = False
|
||||
|
||||
# Test 3: Relationship and file reference validation
|
||||
if not self.validate_file_references():
|
||||
all_valid = False
|
||||
|
||||
# Test 4: Content type declarations
|
||||
if not self.validate_content_types():
|
||||
all_valid = False
|
||||
|
||||
# Test 5: XSD schema validation
|
||||
if not self.validate_against_xsd():
|
||||
all_valid = False
|
||||
|
||||
# Test 6: Whitespace preservation
|
||||
if not self.validate_whitespace_preservation():
|
||||
all_valid = False
|
||||
|
||||
# Test 7: Deletion validation
|
||||
if not self.validate_deletions():
|
||||
all_valid = False
|
||||
|
||||
# Test 8: Insertion validation
|
||||
if not self.validate_insertions():
|
||||
all_valid = False
|
||||
|
||||
# Test 9: Relationship ID reference validation
|
||||
if not self.validate_all_relationship_ids():
|
||||
all_valid = False
|
||||
|
||||
# Count and compare paragraphs
|
||||
self.compare_paragraph_counts()
|
||||
|
||||
return all_valid
|
||||
|
||||
def validate_whitespace_preservation(self):
|
||||
"""
|
||||
Validate that w:t elements with whitespace have xml:space='preserve'.
|
||||
"""
|
||||
errors = []
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
# Only check document.xml files
|
||||
if xml_file.name != "document.xml":
|
||||
continue
|
||||
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
|
||||
# Find all w:t elements
|
||||
for elem in root.iter(f"{{{self.WORD_2006_NAMESPACE}}}t"):
|
||||
if elem.text:
|
||||
text = elem.text
|
||||
# Check if text starts or ends with whitespace
|
||||
if re.match(r"^\s.*", text) or re.match(r".*\s$", text):
|
||||
# Check if xml:space="preserve" attribute exists
|
||||
xml_space_attr = f"{{{self.XML_NAMESPACE}}}space"
|
||||
if (
|
||||
xml_space_attr not in elem.attrib
|
||||
or elem.attrib[xml_space_attr] != "preserve"
|
||||
):
|
||||
# Show a preview of the text
|
||||
text_preview = (
|
||||
repr(text)[:50] + "..."
|
||||
if len(repr(text)) > 50
|
||||
else repr(text)
|
||||
)
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {elem.sourceline}: w:t element with whitespace missing xml:space='preserve': {text_preview}"
|
||||
)
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} whitespace preservation violations:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All whitespace is properly preserved")
|
||||
return True
|
||||
|
||||
def validate_deletions(self):
|
||||
"""
|
||||
Validate that w:t elements are not within w:del elements.
|
||||
For some reason, XSD validation does not catch this, so we do it manually.
|
||||
"""
|
||||
errors = []
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
# Only check document.xml files
|
||||
if xml_file.name != "document.xml":
|
||||
continue
|
||||
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
|
||||
# Find all w:t elements that are descendants of w:del elements
|
||||
namespaces = {"w": self.WORD_2006_NAMESPACE}
|
||||
xpath_expression = ".//w:del//w:t"
|
||||
problematic_t_elements = root.xpath(
|
||||
xpath_expression, namespaces=namespaces
|
||||
)
|
||||
for t_elem in problematic_t_elements:
|
||||
if t_elem.text:
|
||||
# Show a preview of the text
|
||||
text_preview = (
|
||||
repr(t_elem.text)[:50] + "..."
|
||||
if len(repr(t_elem.text)) > 50
|
||||
else repr(t_elem.text)
|
||||
)
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {t_elem.sourceline}: <w:t> found within <w:del>: {text_preview}"
|
||||
)
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} deletion validation violations:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - No w:t elements found within w:del elements")
|
||||
return True
|
||||
|
||||
def count_paragraphs_in_unpacked(self):
|
||||
"""Count the number of paragraphs in the unpacked document."""
|
||||
count = 0
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
# Only check document.xml files
|
||||
if xml_file.name != "document.xml":
|
||||
continue
|
||||
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
# Count all w:p elements
|
||||
paragraphs = root.findall(f".//{{{self.WORD_2006_NAMESPACE}}}p")
|
||||
count = len(paragraphs)
|
||||
except Exception as e:
|
||||
print(f"Error counting paragraphs in unpacked document: {e}")
|
||||
|
||||
return count
|
||||
|
||||
def count_paragraphs_in_original(self):
|
||||
"""Count the number of paragraphs in the original docx file."""
|
||||
count = 0
|
||||
|
||||
try:
|
||||
# Create temporary directory to unpack original
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Unpack original docx
|
||||
with zipfile.ZipFile(self.original_file, "r") as zip_ref:
|
||||
zip_ref.extractall(temp_dir)
|
||||
|
||||
# Parse document.xml
|
||||
doc_xml_path = temp_dir + "/word/document.xml"
|
||||
root = lxml.etree.parse(doc_xml_path).getroot()
|
||||
|
||||
# Count all w:p elements
|
||||
paragraphs = root.findall(f".//{{{self.WORD_2006_NAMESPACE}}}p")
|
||||
count = len(paragraphs)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error counting paragraphs in original document: {e}")
|
||||
|
||||
return count
|
||||
|
||||
def validate_insertions(self):
|
||||
"""
|
||||
Validate that w:delText elements are not within w:ins elements.
|
||||
w:delText is only allowed in w:ins if nested within a w:del.
|
||||
"""
|
||||
errors = []
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
if xml_file.name != "document.xml":
|
||||
continue
|
||||
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
namespaces = {"w": self.WORD_2006_NAMESPACE}
|
||||
|
||||
# Find w:delText in w:ins that are NOT within w:del
|
||||
invalid_elements = root.xpath(
|
||||
".//w:ins//w:delText[not(ancestor::w:del)]",
|
||||
namespaces=namespaces
|
||||
)
|
||||
|
||||
for elem in invalid_elements:
|
||||
text_preview = (
|
||||
repr(elem.text or "")[:50] + "..."
|
||||
if len(repr(elem.text or "")) > 50
|
||||
else repr(elem.text or "")
|
||||
)
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {elem.sourceline}: <w:delText> within <w:ins>: {text_preview}"
|
||||
)
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} insertion validation violations:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - No w:delText elements within w:ins elements")
|
||||
return True
|
||||
|
||||
def compare_paragraph_counts(self):
|
||||
"""Compare paragraph counts between original and new document."""
|
||||
original_count = self.count_paragraphs_in_original()
|
||||
new_count = self.count_paragraphs_in_unpacked()
|
||||
|
||||
diff = new_count - original_count
|
||||
diff_str = f"+{diff}" if diff > 0 else str(diff)
|
||||
print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise RuntimeError("This module should not be run directly.")
|
||||
Executable
+309
@@ -0,0 +1,309 @@
|
||||
"""
|
||||
Validator for PowerPoint presentation XML files against XSD schemas.
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
import lxml.etree
|
||||
|
||||
from .base import BaseSchemaValidator
|
||||
|
||||
|
||||
class PPTXSchemaValidator(BaseSchemaValidator):
|
||||
"""Validator for PowerPoint presentation XML files against XSD schemas."""
|
||||
|
||||
# PowerPoint presentation namespace
|
||||
PRESENTATIONML_NAMESPACE = (
|
||||
"http://schemas.openxmlformats.org/presentationml/2006/main"
|
||||
)
|
||||
|
||||
# PowerPoint-specific element to relationship type mappings
|
||||
ELEMENT_RELATIONSHIP_TYPES = {
|
||||
"sldid": "slide",
|
||||
"sldmasterid": "slidemaster",
|
||||
"notesmasterid": "notesmaster",
|
||||
"sldlayoutid": "slidelayout",
|
||||
"themeid": "theme",
|
||||
"tablestyleid": "tablestyles",
|
||||
}
|
||||
|
||||
def validate(self):
|
||||
"""Run all validation checks and return True if all pass."""
|
||||
# Test 0: XML well-formedness
|
||||
if not self.validate_xml():
|
||||
return False
|
||||
|
||||
# Test 1: Namespace declarations
|
||||
all_valid = True
|
||||
if not self.validate_namespaces():
|
||||
all_valid = False
|
||||
|
||||
# Test 2: Unique IDs
|
||||
if not self.validate_unique_ids():
|
||||
all_valid = False
|
||||
|
||||
# Test 3: UUID ID validation
|
||||
if not self.validate_uuid_ids():
|
||||
all_valid = False
|
||||
|
||||
# Test 4: Relationship and file reference validation
|
||||
if not self.validate_file_references():
|
||||
all_valid = False
|
||||
|
||||
# Test 5: Slide layout ID validation
|
||||
if not self.validate_slide_layout_ids():
|
||||
all_valid = False
|
||||
|
||||
# Test 6: Content type declarations
|
||||
if not self.validate_content_types():
|
||||
all_valid = False
|
||||
|
||||
# Test 7: XSD schema validation
|
||||
if not self.validate_against_xsd():
|
||||
all_valid = False
|
||||
|
||||
# Test 8: Notes slide reference validation
|
||||
if not self.validate_notes_slide_references():
|
||||
all_valid = False
|
||||
|
||||
# Test 9: Relationship ID reference validation
|
||||
if not self.validate_all_relationship_ids():
|
||||
all_valid = False
|
||||
|
||||
# Test 10: Duplicate slide layout references validation
|
||||
if not self.validate_no_duplicate_slide_layouts():
|
||||
all_valid = False
|
||||
|
||||
return all_valid
|
||||
|
||||
def validate_uuid_ids(self):
|
||||
"""Validate that ID attributes that look like UUIDs contain only hex values."""
|
||||
errors = []
|
||||
# UUID pattern: 8-4-4-4-12 hex digits with optional braces/hyphens
|
||||
uuid_pattern = re.compile(
|
||||
r"^[\{\(]?[0-9A-Fa-f]{8}-?[0-9A-Fa-f]{4}-?[0-9A-Fa-f]{4}-?[0-9A-Fa-f]{4}-?[0-9A-Fa-f]{12}[\}\)]?$"
|
||||
)
|
||||
|
||||
for xml_file in self.xml_files:
|
||||
try:
|
||||
root = lxml.etree.parse(str(xml_file)).getroot()
|
||||
|
||||
# Check all elements for ID attributes
|
||||
for elem in root.iter():
|
||||
for attr, value in elem.attrib.items():
|
||||
# Check if this is an ID attribute
|
||||
attr_name = attr.split("}")[-1].lower()
|
||||
if attr_name == "id" or attr_name.endswith("id"):
|
||||
# Check if value looks like a UUID (has the right length and pattern structure)
|
||||
if self._looks_like_uuid(value):
|
||||
# Validate that it contains only hex characters in the right positions
|
||||
if not uuid_pattern.match(value):
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {elem.sourceline}: ID '{value}' appears to be a UUID but contains invalid hex characters"
|
||||
)
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} UUID ID validation errors:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All UUID-like IDs contain valid hex values")
|
||||
return True
|
||||
|
||||
def _looks_like_uuid(self, value):
|
||||
"""Check if a value has the general structure of a UUID."""
|
||||
# Remove common UUID delimiters
|
||||
clean_value = value.strip("{}()").replace("-", "")
|
||||
# Check if it's 32 hex-like characters (could include invalid hex chars)
|
||||
return len(clean_value) == 32 and all(c.isalnum() for c in clean_value)
|
||||
|
||||
def validate_slide_layout_ids(self):
|
||||
"""Validate that sldLayoutId elements in slide masters reference valid slide layouts."""
|
||||
errors = []
|
||||
|
||||
# Find all slide master files
|
||||
slide_masters = list(self.unpacked_dir.glob("ppt/slideMasters/*.xml"))
|
||||
|
||||
if not slide_masters:
|
||||
if self.verbose:
|
||||
print("PASSED - No slide masters found")
|
||||
return True
|
||||
|
||||
for slide_master in slide_masters:
|
||||
try:
|
||||
# Parse the slide master file
|
||||
root = lxml.etree.parse(str(slide_master)).getroot()
|
||||
|
||||
# Find the corresponding _rels file for this slide master
|
||||
rels_file = slide_master.parent / "_rels" / f"{slide_master.name}.rels"
|
||||
|
||||
if not rels_file.exists():
|
||||
errors.append(
|
||||
f" {slide_master.relative_to(self.unpacked_dir)}: "
|
||||
f"Missing relationships file: {rels_file.relative_to(self.unpacked_dir)}"
|
||||
)
|
||||
continue
|
||||
|
||||
# Parse the relationships file
|
||||
rels_root = lxml.etree.parse(str(rels_file)).getroot()
|
||||
|
||||
# Build a set of valid relationship IDs that point to slide layouts
|
||||
valid_layout_rids = set()
|
||||
for rel in rels_root.findall(
|
||||
f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship"
|
||||
):
|
||||
rel_type = rel.get("Type", "")
|
||||
if "slideLayout" in rel_type:
|
||||
valid_layout_rids.add(rel.get("Id"))
|
||||
|
||||
# Find all sldLayoutId elements in the slide master
|
||||
for sld_layout_id in root.findall(
|
||||
f".//{{{self.PRESENTATIONML_NAMESPACE}}}sldLayoutId"
|
||||
):
|
||||
r_id = sld_layout_id.get(
|
||||
f"{{{self.OFFICE_RELATIONSHIPS_NAMESPACE}}}id"
|
||||
)
|
||||
layout_id = sld_layout_id.get("id")
|
||||
|
||||
if r_id and r_id not in valid_layout_rids:
|
||||
errors.append(
|
||||
f" {slide_master.relative_to(self.unpacked_dir)}: "
|
||||
f"Line {sld_layout_id.sourceline}: sldLayoutId with id='{layout_id}' "
|
||||
f"references r:id='{r_id}' which is not found in slide layout relationships"
|
||||
)
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {slide_master.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print(f"FAILED - Found {len(errors)} slide layout ID validation errors:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
print(
|
||||
"Remove invalid references or add missing slide layouts to the relationships file."
|
||||
)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All slide layout IDs reference valid slide layouts")
|
||||
return True
|
||||
|
||||
def validate_no_duplicate_slide_layouts(self):
|
||||
"""Validate that each slide has exactly one slideLayout reference."""
|
||||
errors = []
|
||||
slide_rels_files = list(self.unpacked_dir.glob("ppt/slides/_rels/*.xml.rels"))
|
||||
|
||||
for rels_file in slide_rels_files:
|
||||
try:
|
||||
root = lxml.etree.parse(str(rels_file)).getroot()
|
||||
|
||||
# Find all slideLayout relationships
|
||||
layout_rels = [
|
||||
rel
|
||||
for rel in root.findall(
|
||||
f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship"
|
||||
)
|
||||
if "slideLayout" in rel.get("Type", "")
|
||||
]
|
||||
|
||||
if len(layout_rels) > 1:
|
||||
errors.append(
|
||||
f" {rels_file.relative_to(self.unpacked_dir)}: has {len(layout_rels)} slideLayout references"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
errors.append(
|
||||
f" {rels_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
print("FAILED - Found slides with duplicate slideLayout references:")
|
||||
for error in errors:
|
||||
print(error)
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All slides have exactly one slideLayout reference")
|
||||
return True
|
||||
|
||||
def validate_notes_slide_references(self):
|
||||
"""Validate that each notesSlide file is referenced by only one slide."""
|
||||
errors = []
|
||||
notes_slide_references = {} # Track which slides reference each notesSlide
|
||||
|
||||
# Find all slide relationship files
|
||||
slide_rels_files = list(self.unpacked_dir.glob("ppt/slides/_rels/*.xml.rels"))
|
||||
|
||||
if not slide_rels_files:
|
||||
if self.verbose:
|
||||
print("PASSED - No slide relationship files found")
|
||||
return True
|
||||
|
||||
for rels_file in slide_rels_files:
|
||||
try:
|
||||
# Parse the relationships file
|
||||
root = lxml.etree.parse(str(rels_file)).getroot()
|
||||
|
||||
# Find all notesSlide relationships
|
||||
for rel in root.findall(
|
||||
f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship"
|
||||
):
|
||||
rel_type = rel.get("Type", "")
|
||||
if "notesSlide" in rel_type:
|
||||
target = rel.get("Target", "")
|
||||
if target:
|
||||
# Normalize the target path to handle relative paths
|
||||
normalized_target = target.replace("../", "")
|
||||
|
||||
# Track which slide references this notesSlide
|
||||
slide_name = rels_file.stem.replace(
|
||||
".xml", ""
|
||||
) # e.g., "slide1"
|
||||
|
||||
if normalized_target not in notes_slide_references:
|
||||
notes_slide_references[normalized_target] = []
|
||||
notes_slide_references[normalized_target].append(
|
||||
(slide_name, rels_file)
|
||||
)
|
||||
|
||||
except (lxml.etree.XMLSyntaxError, Exception) as e:
|
||||
errors.append(
|
||||
f" {rels_file.relative_to(self.unpacked_dir)}: Error: {e}"
|
||||
)
|
||||
|
||||
# Check for duplicate references
|
||||
for target, references in notes_slide_references.items():
|
||||
if len(references) > 1:
|
||||
slide_names = [ref[0] for ref in references]
|
||||
errors.append(
|
||||
f" Notes slide '{target}' is referenced by multiple slides: {', '.join(slide_names)}"
|
||||
)
|
||||
for slide_name, rels_file in references:
|
||||
errors.append(f" - {rels_file.relative_to(self.unpacked_dir)}")
|
||||
|
||||
if errors:
|
||||
print(
|
||||
f"FAILED - Found {len([e for e in errors if not e.startswith(' ')])} notes slide reference validation errors:"
|
||||
)
|
||||
for error in errors:
|
||||
print(error)
|
||||
print("Each slide may optionally have its own slide file.")
|
||||
return False
|
||||
else:
|
||||
if self.verbose:
|
||||
print("PASSED - All notes slide references are unique")
|
||||
return True
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise RuntimeError("This module should not be run directly.")
|
||||
+279
@@ -0,0 +1,279 @@
|
||||
"""
|
||||
Validator for tracked changes in Word documents.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import tempfile
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class RedliningValidator:
|
||||
"""Validator for tracked changes in Word documents."""
|
||||
|
||||
def __init__(self, unpacked_dir, original_docx, verbose=False):
|
||||
self.unpacked_dir = Path(unpacked_dir)
|
||||
self.original_docx = Path(original_docx)
|
||||
self.verbose = verbose
|
||||
self.namespaces = {
|
||||
"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
|
||||
}
|
||||
|
||||
def validate(self):
|
||||
"""Main validation method that returns True if valid, False otherwise."""
|
||||
# Verify unpacked directory exists and has correct structure
|
||||
modified_file = self.unpacked_dir / "word" / "document.xml"
|
||||
if not modified_file.exists():
|
||||
print(f"FAILED - Modified document.xml not found at {modified_file}")
|
||||
return False
|
||||
|
||||
# First, check if there are any tracked changes by Z.AI to validate
|
||||
try:
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
tree = ET.parse(modified_file)
|
||||
root = tree.getroot()
|
||||
|
||||
# Check for w:del or w:ins tags authored by Z.AI
|
||||
del_elements = root.findall(".//w:del", self.namespaces)
|
||||
ins_elements = root.findall(".//w:ins", self.namespaces)
|
||||
|
||||
# Filter to only include changes by Z.AI
|
||||
zai_del_elements = [
|
||||
elem
|
||||
for elem in del_elements
|
||||
if elem.get(f"{{{self.namespaces['w']}}}author") == "Z.AI"
|
||||
]
|
||||
zai_ins_elements = [
|
||||
elem
|
||||
for elem in ins_elements
|
||||
if elem.get(f"{{{self.namespaces['w']}}}author") == "Z.AI"
|
||||
]
|
||||
|
||||
# Redlining validation is only needed if tracked changes by Z.AI have been used.
|
||||
if not zai_del_elements and not zai_ins_elements:
|
||||
if self.verbose:
|
||||
print("PASSED - No tracked changes by Z.AI found.")
|
||||
return True
|
||||
|
||||
except Exception:
|
||||
# If we can't parse the XML, continue with full validation
|
||||
pass
|
||||
|
||||
# Create temporary directory for unpacking original docx
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
|
||||
# Unpack original docx
|
||||
try:
|
||||
with zipfile.ZipFile(self.original_docx, "r") as zip_ref:
|
||||
zip_ref.extractall(temp_path)
|
||||
except Exception as e:
|
||||
print(f"FAILED - Error unpacking original docx: {e}")
|
||||
return False
|
||||
|
||||
original_file = temp_path / "word" / "document.xml"
|
||||
if not original_file.exists():
|
||||
print(
|
||||
f"FAILED - Original document.xml not found in {self.original_docx}"
|
||||
)
|
||||
return False
|
||||
|
||||
# Parse both XML files using xml.etree.ElementTree for redlining validation
|
||||
try:
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
modified_tree = ET.parse(modified_file)
|
||||
modified_root = modified_tree.getroot()
|
||||
original_tree = ET.parse(original_file)
|
||||
original_root = original_tree.getroot()
|
||||
except ET.ParseError as e:
|
||||
print(f"FAILED - Error parsing XML files: {e}")
|
||||
return False
|
||||
|
||||
# Remove Z.AI's tracked changes from both documents
|
||||
self._remove_zai_tracked_changes(original_root)
|
||||
self._remove_zai_tracked_changes(modified_root)
|
||||
|
||||
# Extract and compare text content
|
||||
modified_text = self._extract_text_content(modified_root)
|
||||
original_text = self._extract_text_content(original_root)
|
||||
|
||||
if modified_text != original_text:
|
||||
# Show detailed character-level differences for each paragraph
|
||||
error_message = self._generate_detailed_diff(
|
||||
original_text, modified_text
|
||||
)
|
||||
print(error_message)
|
||||
return False
|
||||
|
||||
if self.verbose:
|
||||
print("PASSED - All changes by Z.AI are properly tracked")
|
||||
return True
|
||||
|
||||
def _generate_detailed_diff(self, original_text, modified_text):
|
||||
"""Generate detailed word-level differences using git word diff."""
|
||||
error_parts = [
|
||||
"FAILED - Document text doesn't match after removing Z.AI's tracked changes",
|
||||
"",
|
||||
"Likely causes:",
|
||||
" 1. Modified text inside another author's <w:ins> or <w:del> tags",
|
||||
" 2. Made edits without proper tracked changes",
|
||||
" 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion",
|
||||
"",
|
||||
"For pre-redlined documents, use correct patterns:",
|
||||
" - To reject another's INSERTION: Nest <w:del> inside their <w:ins>",
|
||||
" - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>",
|
||||
"",
|
||||
]
|
||||
|
||||
# Show git word diff
|
||||
git_diff = self._get_git_word_diff(original_text, modified_text)
|
||||
if git_diff:
|
||||
error_parts.extend(["Differences:", "============", git_diff])
|
||||
else:
|
||||
error_parts.append("Unable to generate word diff (git not available)")
|
||||
|
||||
return "\n".join(error_parts)
|
||||
|
||||
def _get_git_word_diff(self, original_text, modified_text):
|
||||
"""Generate word diff using git with character-level precision."""
|
||||
try:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
|
||||
# Create two files
|
||||
original_file = temp_path / "original.txt"
|
||||
modified_file = temp_path / "modified.txt"
|
||||
|
||||
original_file.write_text(original_text, encoding="utf-8")
|
||||
modified_file.write_text(modified_text, encoding="utf-8")
|
||||
|
||||
# Try character-level diff first for precise differences
|
||||
result = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"diff",
|
||||
"--word-diff=plain",
|
||||
"--word-diff-regex=.", # Character-by-character diff
|
||||
"-U0", # Zero lines of context - show only changed lines
|
||||
"--no-index",
|
||||
str(original_file),
|
||||
str(modified_file),
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
if result.stdout.strip():
|
||||
# Clean up the output - remove git diff header lines
|
||||
lines = result.stdout.split("\n")
|
||||
# Skip the header lines (diff --git, index, +++, ---, @@)
|
||||
content_lines = []
|
||||
in_content = False
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
in_content = True
|
||||
continue
|
||||
if in_content and line.strip():
|
||||
content_lines.append(line)
|
||||
|
||||
if content_lines:
|
||||
return "\n".join(content_lines)
|
||||
|
||||
# Fallback to word-level diff if character-level is too verbose
|
||||
result = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"diff",
|
||||
"--word-diff=plain",
|
||||
"-U0", # Zero lines of context
|
||||
"--no-index",
|
||||
str(original_file),
|
||||
str(modified_file),
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
if result.stdout.strip():
|
||||
lines = result.stdout.split("\n")
|
||||
content_lines = []
|
||||
in_content = False
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
in_content = True
|
||||
continue
|
||||
if in_content and line.strip():
|
||||
content_lines.append(line)
|
||||
return "\n".join(content_lines)
|
||||
|
||||
except (subprocess.CalledProcessError, FileNotFoundError, Exception):
|
||||
# Git not available or other error, return None to use fallback
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
def _remove_zai_tracked_changes(self, root):
|
||||
"""Remove tracked changes authored by Z.AI from the XML root."""
|
||||
ins_tag = f"{{{self.namespaces['w']}}}ins"
|
||||
del_tag = f"{{{self.namespaces['w']}}}del"
|
||||
author_attr = f"{{{self.namespaces['w']}}}author"
|
||||
|
||||
# Remove w:ins elements
|
||||
for parent in root.iter():
|
||||
to_remove = []
|
||||
for child in parent:
|
||||
if child.tag == ins_tag and child.get(author_attr) == "Z.AI":
|
||||
to_remove.append(child)
|
||||
for elem in to_remove:
|
||||
parent.remove(elem)
|
||||
|
||||
# Unwrap content in w:del elements where author is "Z.AI"
|
||||
deltext_tag = f"{{{self.namespaces['w']}}}delText"
|
||||
t_tag = f"{{{self.namespaces['w']}}}t"
|
||||
|
||||
for parent in root.iter():
|
||||
to_process = []
|
||||
for child in parent:
|
||||
if child.tag == del_tag and child.get(author_attr) == "Z.AI":
|
||||
to_process.append((child, list(parent).index(child)))
|
||||
|
||||
# Process in reverse order to maintain indices
|
||||
for del_elem, del_index in reversed(to_process):
|
||||
# Convert w:delText to w:t before moving
|
||||
for elem in del_elem.iter():
|
||||
if elem.tag == deltext_tag:
|
||||
elem.tag = t_tag
|
||||
|
||||
# Move all children of w:del to its parent before removing w:del
|
||||
for child in reversed(list(del_elem)):
|
||||
parent.insert(del_index, child)
|
||||
parent.remove(del_elem)
|
||||
|
||||
def _extract_text_content(self, root):
|
||||
"""Extract text content from Word XML, preserving paragraph structure.
|
||||
|
||||
Empty paragraphs are skipped to avoid false positives when tracked
|
||||
insertions add only structural elements without text content.
|
||||
"""
|
||||
p_tag = f"{{{self.namespaces['w']}}}p"
|
||||
t_tag = f"{{{self.namespaces['w']}}}t"
|
||||
|
||||
paragraphs = []
|
||||
for p_elem in root.findall(f".//{p_tag}"):
|
||||
# Get all text elements within this paragraph
|
||||
text_parts = []
|
||||
for t_elem in p_elem.findall(f".//{t_tag}"):
|
||||
if t_elem.text:
|
||||
text_parts.append(t_elem.text)
|
||||
paragraph_text = "".join(text_parts)
|
||||
# Skip empty paragraphs - they don't affect content validation
|
||||
if paragraph_text:
|
||||
paragraphs.append(paragraph_text)
|
||||
|
||||
return "\n".join(paragraphs)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise RuntimeError("This module should not be run directly.")
|
||||
Executable
+712
@@ -0,0 +1,712 @@
|
||||
#!/usr/bin/env python3
|
||||
"""CLI for slide dimension / overflow / blank-region / overlap checks.
|
||||
|
||||
Renders each HTML slide in headless Chromium (via Playwright) at 1280x720 and
|
||||
reports vertical overflow, large blank regions, and text-vs-text overlaps.
|
||||
Standalone — does not import from any sibling agent package.
|
||||
|
||||
Usage:
|
||||
# Check specific files (preferred — matches whatever naming the brief uses):
|
||||
python check_slides.py /work/slides/slide_01.html /work/slides/slide_02.html
|
||||
|
||||
# Check every *.html in a directory (sorted by filename):
|
||||
python check_slides.py --dir /work/slides
|
||||
|
||||
# Restrict to a few positions when --dir is used (matches files whose stem
|
||||
# ends in those numbers, e.g. `1.html`, `slide_01.html`):
|
||||
python check_slides.py --dir /work/slides --positions 1 3 5
|
||||
|
||||
# JSON output for downstream tooling:
|
||||
python check_slides.py --dir /work/slides --json --output /work/check.json
|
||||
|
||||
Stdout is a compact text report by default; pass --json for the raw measurement
|
||||
dict per file. Exit code is 0 when every checked slide fits the viewport
|
||||
vertically with no large blank / text overlap, otherwise 1.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import traceback
|
||||
from pathlib import Path
|
||||
|
||||
_VIEWPORT_W = 1280
|
||||
_VIEWPORT_H = 720
|
||||
_MAX_CONCURRENCY = 81
|
||||
_PROXY = {"server": "http://httpproxy.glm.ai:8888"}
|
||||
|
||||
|
||||
_TEXT_OVERLAP_JS = r"""
|
||||
(opts) => {
|
||||
const minOverlapPx = opts.minOverlapPx || 3;
|
||||
const maxPairs = opts.maxPairs || 50;
|
||||
// Relative overlap threshold: the intersection must consume at least
|
||||
// `relMin` of the smaller element's corresponding side on BOTH axes.
|
||||
// Suppresses line-height bleed where vertically-stacked siblings share
|
||||
// 3-5px of line-box overhang without true visual collision.
|
||||
const relMin = opts.relMin != null ? opts.relMin : 0.10;
|
||||
// Decorative-text thresholds (mirrors genspark html2pptx checkTextOverlaps):
|
||||
// - opacity ≤ 0.2 → watermark / ghost layer
|
||||
// - font-size ≥ 150px → giant ghost numeral / chapter mark
|
||||
const decorOpacity = opts.decorOpacity != null ? opts.decorOpacity : 0.2;
|
||||
const decorFontPx = opts.decorFontPx || 150;
|
||||
|
||||
function ownText(el) {
|
||||
let t = '';
|
||||
for (const n of el.childNodes) {
|
||||
if (n.nodeType === Node.TEXT_NODE) t += n.textContent;
|
||||
}
|
||||
return t.trim();
|
||||
}
|
||||
// Effective opacity = product of opacity along ancestor chain (CSS-correct).
|
||||
// Returns 0 if any ancestor is display:none / visibility:hidden.
|
||||
function effectiveOpacity(el) {
|
||||
let op = 1;
|
||||
let cur = el;
|
||||
while (cur && cur !== document.body) {
|
||||
const s = getComputedStyle(cur);
|
||||
if (s.display === 'none' || s.visibility === 'hidden') return 0;
|
||||
const o = parseFloat(s.opacity);
|
||||
if (!isNaN(o)) op *= o;
|
||||
cur = cur.parentElement;
|
||||
}
|
||||
return op;
|
||||
}
|
||||
// First ancestor (incl. self) that establishes a stacking context with a
|
||||
// numeric z-index. Returns 0 if none — matches "auto" → 0 for sibling
|
||||
// comparison purposes. We don't try to fully simulate the stacking algorithm;
|
||||
// this is a heuristic to skip *intentional* layered decorations.
|
||||
function effectiveZ(el) {
|
||||
let cur = el;
|
||||
while (cur && cur !== document.body) {
|
||||
const s = getComputedStyle(cur);
|
||||
if (s.position && s.position !== 'static') {
|
||||
const z = parseInt(s.zIndex, 10);
|
||||
if (!isNaN(z)) return z;
|
||||
}
|
||||
cur = cur.parentElement;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
// Restrict reporting to text inside the slide canvas — body padding / preview
|
||||
// gaps / off-canvas overflow shouldn't surface as overlap noise.
|
||||
const canvas = document.querySelector('.slide, .slide-canvas');
|
||||
const canvasRect = canvas ? canvas.getBoundingClientRect() : null;
|
||||
function insideCanvas(r) {
|
||||
if (!canvasRect) return true;
|
||||
// Require ≥50% of the rect's area to lie inside the canvas.
|
||||
const x = Math.max(r.x, canvasRect.left);
|
||||
const y = Math.max(r.y, canvasRect.top);
|
||||
const w = Math.min(r.x + r.w, canvasRect.right) - x;
|
||||
const h = Math.min(r.y + r.h, canvasRect.bottom) - y;
|
||||
if (w <= 0 || h <= 0) return false;
|
||||
return (w * h) >= 0.5 * (r.w * r.h);
|
||||
}
|
||||
|
||||
const items = [];
|
||||
const walker = document.querySelectorAll('body *');
|
||||
for (const el of walker) {
|
||||
const text = ownText(el);
|
||||
if (!text) continue;
|
||||
const op = effectiveOpacity(el);
|
||||
if (op === 0) continue;
|
||||
const cs = getComputedStyle(el);
|
||||
const fontPx = parseFloat(cs.fontSize) || 0;
|
||||
const decorative = op <= decorOpacity || fontPx >= decorFontPx;
|
||||
const z = effectiveZ(el);
|
||||
const rects = el.getClientRects();
|
||||
if (!rects.length) continue;
|
||||
for (const r of rects) {
|
||||
if (r.width < 2 || r.height < 2) continue;
|
||||
const rect = {x: r.x, y: r.y, w: r.width, h: r.height};
|
||||
if (!insideCanvas(rect)) continue;
|
||||
items.push({
|
||||
el, text, decorative, z, fontPx,
|
||||
tag: el.tagName.toLowerCase(),
|
||||
rect,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function rel(a, b) { return a.contains(b) || b.contains(a); }
|
||||
function inter(r1, r2) {
|
||||
const x = Math.max(r1.x, r2.x);
|
||||
const y = Math.max(r1.y, r2.y);
|
||||
const w = Math.min(r1.x + r1.w, r2.x + r2.w) - x;
|
||||
const h = Math.min(r1.y + r1.h, r2.y + r2.h) - y;
|
||||
if (w < minOverlapPx || h < minOverlapPx) return null;
|
||||
// Relative gate: overlap must occupy ≥ relMin of the smaller side.
|
||||
if (w < relMin * Math.min(r1.w, r2.w)) return null;
|
||||
if (h < relMin * Math.min(r1.h, r2.h)) return null;
|
||||
return {x, y, w, h};
|
||||
}
|
||||
function trunc(s, n) { return s.length > n ? s.slice(0, n) + '…' : s; }
|
||||
function fmtRect(r) {
|
||||
return {
|
||||
x: Math.round(r.x), y: Math.round(r.y),
|
||||
w: Math.round(r.w), h: Math.round(r.h),
|
||||
};
|
||||
}
|
||||
|
||||
// Dedupe by element pair: a 4-line <p> overlapping another 4-line <p> via
|
||||
// getClientRects() would otherwise produce up to 16 entries. Keep the
|
||||
// largest overlap rect per (elA, elB) pair.
|
||||
const seen = new Map(); // key → {a, b, overlap}
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
for (let j = i + 1; j < items.length; j++) {
|
||||
const A = items[i], B = items[j];
|
||||
if (A.el === B.el) continue;
|
||||
if (rel(A.el, B.el)) continue;
|
||||
// Skip intentional layered decorations (genspark parity).
|
||||
if (A.decorative || B.decorative) continue;
|
||||
if (A.z !== B.z) continue;
|
||||
const ov = inter(A.rect, B.rect);
|
||||
if (!ov) continue;
|
||||
// Stable pair key: use DOM position to order.
|
||||
const order = A.el.compareDocumentPosition(B.el) & Node.DOCUMENT_POSITION_FOLLOWING;
|
||||
const first = order ? A : B;
|
||||
const second = order ? B : A;
|
||||
const key = first.tag + ' ' + first.text + ' '
|
||||
+ second.tag + ' ' + second.text;
|
||||
const area = ov.w * ov.h;
|
||||
const prev = seen.get(key);
|
||||
if (!prev || area > prev._area) {
|
||||
seen.set(key, {
|
||||
a: {tag: first.tag, text: trunc(first.text, 80), rect: fmtRect(first.rect)},
|
||||
b: {tag: second.tag, text: trunc(second.text, 80), rect: fmtRect(second.rect)},
|
||||
overlap: fmtRect(ov),
|
||||
_area: area,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
const overlaps = [];
|
||||
for (const v of seen.values()) {
|
||||
delete v._area;
|
||||
overlaps.push(v);
|
||||
if (overlaps.length >= maxPairs) break;
|
||||
}
|
||||
return overlaps;
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def _detect_largest_blank(screenshot_bytes: bytes,
|
||||
color_tolerance: int = 15,
|
||||
downsample: int = 4,
|
||||
decor_rects: list[dict] | None = None) -> dict | None:
|
||||
"""Find the largest contiguous background-colored rectangle in a screenshot.
|
||||
|
||||
Background color is sampled from the four corners (median). A pixel is
|
||||
"non-background" if its max channel diff from the sample exceeds
|
||||
`color_tolerance`. The image is downsampled by `downsample`× for the
|
||||
rectangle search (1280x720 → 320x180 by default), then results are scaled
|
||||
back to original pixels. Returns None on failure (e.g. PIL/numpy missing).
|
||||
|
||||
`decor_rects` (list of {x, y, w, h} in screenshot pixels) marks regions
|
||||
occupied by decorative elements — those rects are forced to "non-
|
||||
background" in the mask so faint / ghost decoration that blends with the
|
||||
bg color does not register as a false blank patch.
|
||||
"""
|
||||
try:
|
||||
import io
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
except Exception:
|
||||
return None
|
||||
try:
|
||||
img = Image.open(io.BytesIO(screenshot_bytes)).convert("RGB")
|
||||
arr = np.array(img, dtype=np.uint8)
|
||||
H, W, _ = arr.shape
|
||||
if H < 8 or W < 8:
|
||||
return None
|
||||
corner = max(4, min(W, H) // 64)
|
||||
samples = np.concatenate([
|
||||
arr[:corner, :corner].reshape(-1, 3),
|
||||
arr[:corner, -corner:].reshape(-1, 3),
|
||||
arr[-corner:, :corner].reshape(-1, 3),
|
||||
arr[-corner:, -corner:].reshape(-1, 3),
|
||||
], axis=0).astype(np.int32)
|
||||
bg = np.median(samples, axis=0)
|
||||
diff = np.abs(arr.astype(np.int32) - bg).max(axis=2)
|
||||
nonbg = (diff > color_tolerance).astype(np.uint8)
|
||||
|
||||
# Mask out decorative element rects: their visual area is intentional
|
||||
# design, not blank. Without this, a low-opacity ghost numeral or
|
||||
# gradient blob whose pixels blend into bg would show up as one big
|
||||
# contiguous blank region.
|
||||
if decor_rects:
|
||||
for r in decor_rects:
|
||||
x0 = max(0, int(round(r.get("x", 0))))
|
||||
y0 = max(0, int(round(r.get("y", 0))))
|
||||
x1 = min(W, int(round(r.get("x", 0) + r.get("w", 0))))
|
||||
y1 = min(H, int(round(r.get("y", 0) + r.get("h", 0))))
|
||||
if x1 > x0 and y1 > y0:
|
||||
nonbg[y0:y1, x0:x1] = 1
|
||||
|
||||
s = max(1, downsample)
|
||||
Hd, Wd = H // s, W // s
|
||||
if Hd < 2 or Wd < 2:
|
||||
return None
|
||||
# Pool: cell is "blocked" if any underlying pixel is non-background.
|
||||
nonbg_d = nonbg[:Hd * s, :Wd * s].reshape(Hd, s, Wd, s).max(axis=(1, 3))
|
||||
|
||||
heights = [0] * Wd
|
||||
best_area = 0
|
||||
best = (0, 0, 0, 0) # x, y, w, h in downsampled units
|
||||
for r in range(Hd):
|
||||
row = nonbg_d[r]
|
||||
for c in range(Wd):
|
||||
heights[c] = 0 if row[c] else heights[c] + 1
|
||||
stack = [] # (start_index, height)
|
||||
for c in range(Wd + 1):
|
||||
cur = heights[c] if c < Wd else 0
|
||||
start = c
|
||||
while stack and stack[-1][1] > cur:
|
||||
idx, h = stack.pop()
|
||||
w = c - idx
|
||||
area = w * h
|
||||
if area > best_area:
|
||||
best_area = area
|
||||
best = (idx, r - h + 1, w, h)
|
||||
start = idx
|
||||
stack.append((start, cur))
|
||||
|
||||
x_d, y_d, w_d, h_d = best
|
||||
x, y, w, h = x_d * s, y_d * s, w_d * s, h_d * s
|
||||
return {
|
||||
"x": int(x), "y": int(y), "w": int(w), "h": int(h),
|
||||
"area_ratio": round((w * h) / float(W * H), 3),
|
||||
"bg_rgb": [int(c) for c in bg],
|
||||
}
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
async def measure_slide_dimensions(html_content, viewport_width=1280, viewport_height=720, proxy=None,
|
||||
detect_overlaps=True, detect_blank=True,
|
||||
blank_min_area_ratio=0.25, blank_min_side=180,
|
||||
overflow_tolerance_ratio=0.005, overflow_tolerance_px=3,
|
||||
file_path: str | None = None):
|
||||
"""Render HTML at the given viewport and report ACTUAL content width + height.
|
||||
|
||||
Returns a dict with success bool, plus on success: width, height,
|
||||
viewport_width, viewport_height, overflow_x (px past viewport, 0 if fits),
|
||||
overflow_y (px past viewport, 0 if fits), `details` subdict with raw
|
||||
browser measurements, `overlaps` (when detect_overlaps): a list of
|
||||
text-vs-text overlapping pairs each shaped {a: {tag, text, rect}, b:
|
||||
{tag, text, rect}, overlap: {x,y,w,h}}, and `blank` (when detect_blank):
|
||||
{x, y, w, h, area_ratio, bg_rgb, flagged: bool} for the largest contiguous
|
||||
background-colored region. `flagged` is True only when area_ratio ≥
|
||||
blank_min_area_ratio AND min(w, h) ≥ blank_min_side.
|
||||
|
||||
Notes:
|
||||
- Width and height are both measured (not assumed = viewport_*) and
|
||||
reported. Horizontal AND vertical overflow are flagged when content
|
||||
extends past the viewport beyond the tolerance buffer.
|
||||
- If the slide root uses `overflow:hidden` the inner content is clipped
|
||||
for the user — we measure content-bearing children to surface the
|
||||
underlying content size when possible. Elements clipped by any
|
||||
ancestor with `overflow:hidden|clip` have their rect intersected
|
||||
with the clipping box before contributing to overflow, so bleed
|
||||
decoration that's invisible to the user is not flagged.
|
||||
- Decorative elements are excluded from overflow measurement when
|
||||
ANY of these hold: (a) the element (or an ancestor) carries
|
||||
`data-decor` or `aria-hidden="true"`, (b) effective opacity along
|
||||
the ancestor chain is ≤ 0.2 (ghost / watermark layer). Authors
|
||||
should tag intentional background blobs, oversized ghost numerals,
|
||||
and bleed shapes with `data-decor` to silence false-positive
|
||||
overflow flags.
|
||||
- Overlap detection only flags pairs of visible text-bearing elements
|
||||
that are NOT in an ancestor/descendant relationship and whose
|
||||
bounding rects intersect by ≥3px on both axes (suppresses border
|
||||
touches) AND by ≥20% of the smaller element's corresponding side
|
||||
(suppresses line-height bleed in vertically-stacked siblings).
|
||||
Capped at 50 pairs per slide. Filters that mirror the
|
||||
genspark html2pptx checker:
|
||||
* effective opacity ≤ 0.2 → decorative watermark, skipped
|
||||
* font-size ≥ 150px → ghost numeral / chapter mark, skipped
|
||||
* different effective z-index → intentional layering, skipped
|
||||
* rect mostly outside `.slide` canvas (preview chrome / cropped
|
||||
overflow) → skipped
|
||||
Multi-line text pairs are deduped to one entry per element pair
|
||||
(largest overlap rect wins).
|
||||
- Blank detection samples background color from the four corners and
|
||||
finds the largest rectangle whose pixels are all within
|
||||
`color_tolerance` (15) of that color. Operates on a 4×-downsampled
|
||||
screenshot for speed. Decorative element rects (same opt-out as
|
||||
the overflow check: `data-decor` / `aria-hidden="true"`, plus
|
||||
elements whose effective opacity is ≤ 0.2) are forced to "non-
|
||||
background" in the mask, so faint ghost decoration that blends
|
||||
with the bg color is not flagged as a large blank region.
|
||||
"""
|
||||
from playwright.async_api import async_playwright
|
||||
try:
|
||||
async with async_playwright() as p:
|
||||
browser = await p.chromium.launch(headless=True, proxy=proxy)
|
||||
page = await browser.new_page()
|
||||
await page.set_viewport_size({"width": viewport_width, "height": viewport_height})
|
||||
# When a real file path is supplied, navigate to it via file://
|
||||
# so relative resources (global.css, images/, fonts) resolve.
|
||||
# set_content has no base URL and would render the slide
|
||||
# unstyled — which collapses layouts and hides real overflow.
|
||||
if file_path:
|
||||
await page.goto(f"file://{file_path}")
|
||||
else:
|
||||
await page.set_content(html_content)
|
||||
await page.wait_for_load_state("networkidle")
|
||||
await page.wait_for_timeout(2000)
|
||||
dims = await page.evaluate("""() => {
|
||||
// Canvas = the .slide element. With overflow:hidden|clip, decorative
|
||||
// absolute children (background blobs, bleed shapes) extend past the
|
||||
// box but are visually clipped — using scrollWidth/Height would count
|
||||
// them as overflow. Instead: use the declared box as canvas, then
|
||||
// measure overflow from CONTENT-BEARING elements (text + media) only.
|
||||
const slides = document.querySelectorAll('.slide, .slide-canvas');
|
||||
const ContentTags = new Set(['IMG','SVG','VIDEO','CANVAS','PICTURE','IFRAME']);
|
||||
function ownText(el){
|
||||
let t='';
|
||||
for (const n of el.childNodes){
|
||||
if (n.nodeType === Node.TEXT_NODE) t += n.textContent;
|
||||
}
|
||||
return t.trim();
|
||||
}
|
||||
function isContent(el){
|
||||
if (ContentTags.has(el.tagName)) return true;
|
||||
return ownText(el).length > 0;
|
||||
}
|
||||
// Effective opacity along the ancestor chain. Returns 0 if any
|
||||
// ancestor is display:none / visibility:hidden — collapses the
|
||||
// old isVisible check into the same signal.
|
||||
function effectiveOpacity(el){
|
||||
let op = 1;
|
||||
let cur = el;
|
||||
while (cur && cur !== document.body){
|
||||
const s = getComputedStyle(cur);
|
||||
if (s.display === 'none' || s.visibility === 'hidden') return 0;
|
||||
const o = parseFloat(s.opacity);
|
||||
if (!isNaN(o)) op *= o;
|
||||
cur = cur.parentElement;
|
||||
}
|
||||
return op;
|
||||
}
|
||||
// Three signals that an element is decorative and should not
|
||||
// contribute to overflow:
|
||||
// 1. Author opt-out via data-decor / aria-hidden="true".
|
||||
// 2. Effective opacity ≤ 0.2 (mirrors the text-overlap
|
||||
// checker's ghost / watermark threshold).
|
||||
function isDecorative(el){
|
||||
if (el.closest('[data-decor], [aria-hidden="true"]')) return true;
|
||||
if (effectiveOpacity(el) <= 0.2) return true;
|
||||
return false;
|
||||
}
|
||||
// Walk ancestors UP TO (but not including) the slide root;
|
||||
// if any inner ancestor clips overflow, intersect the rect
|
||||
// with the clipping box. Decoration that bleeds past an
|
||||
// inner card / panel and is hidden by overflow:hidden|clip
|
||||
// is invisible to the user and must not be flagged. We
|
||||
// intentionally do NOT clip by the .slide canvas itself —
|
||||
// even when .slide has overflow:hidden (it almost always
|
||||
// does), content that wants to extend past 1280x720 is
|
||||
// still bad layout and must surface as overflow.
|
||||
function visibleRect(el, slide){
|
||||
let r = el.getBoundingClientRect();
|
||||
let left = r.left, top = r.top, right = r.right, bottom = r.bottom;
|
||||
let cur = el.parentElement;
|
||||
while (cur && cur !== slide && cur !== document.body){
|
||||
const s = getComputedStyle(cur);
|
||||
const clipped = (
|
||||
s.overflow === 'hidden' || s.overflow === 'clip' ||
|
||||
s.overflowX === 'hidden' || s.overflowX === 'clip' ||
|
||||
s.overflowY === 'hidden' || s.overflowY === 'clip'
|
||||
);
|
||||
if (clipped){
|
||||
const cr = cur.getBoundingClientRect();
|
||||
left = Math.max(left, cr.left);
|
||||
top = Math.max(top, cr.top);
|
||||
right = Math.min(right, cr.right);
|
||||
bottom = Math.min(bottom, cr.bottom);
|
||||
if (right <= left || bottom <= top) return null;
|
||||
}
|
||||
cur = cur.parentElement;
|
||||
}
|
||||
return {left, top, right, bottom,
|
||||
width: right - left, height: bottom - top};
|
||||
}
|
||||
function maxOverflow(slide){
|
||||
const cr = slide.getBoundingClientRect();
|
||||
let ox = 0, oy = 0;
|
||||
for (const el of slide.querySelectorAll('*')){
|
||||
if (!isContent(el)) continue;
|
||||
if (isDecorative(el)) continue;
|
||||
const r = visibleRect(el, slide);
|
||||
if (!r || r.width < 1 || r.height < 1) continue;
|
||||
ox = Math.max(ox, r.right - cr.right, cr.left - r.left);
|
||||
oy = Math.max(oy, r.bottom - cr.bottom, cr.top - r.top);
|
||||
}
|
||||
return {ox: Math.max(0, ox), oy: Math.max(0, oy),
|
||||
cw: slide.offsetWidth, ch: slide.offsetHeight};
|
||||
}
|
||||
if (slides.length){
|
||||
let cw=0, ch=0, ox=0, oy=0;
|
||||
for (const s of slides){
|
||||
const m = maxOverflow(s);
|
||||
cw = Math.max(cw, m.cw);
|
||||
ch = Math.max(ch, m.ch);
|
||||
ox = Math.max(ox, m.ox);
|
||||
oy = Math.max(oy, m.oy);
|
||||
}
|
||||
return {scrollWidth: cw + ox, scrollHeight: ch + oy,
|
||||
clientWidth: cw, clientHeight: ch};
|
||||
}
|
||||
const body = document.body;
|
||||
const html = document.documentElement;
|
||||
return {
|
||||
scrollWidth: Math.max(body.scrollWidth, html.scrollWidth, body.offsetWidth, html.offsetWidth),
|
||||
scrollHeight: Math.max(body.scrollHeight, html.scrollHeight, body.offsetHeight, html.offsetHeight),
|
||||
clientWidth: html.clientWidth,
|
||||
clientHeight: html.clientHeight,
|
||||
};
|
||||
}""")
|
||||
overlaps = []
|
||||
if detect_overlaps:
|
||||
overlaps = await page.evaluate(_TEXT_OVERLAP_JS, {"minOverlapPx": 3, "maxPairs": 50})
|
||||
screenshot_bytes = None
|
||||
decor_rects = []
|
||||
if detect_blank:
|
||||
# Bounding rects of decorative elements, in viewport pixel
|
||||
# coordinates that match the screenshot clip. Used by the
|
||||
# blank detector to avoid flagging intentional decoration as
|
||||
# a blank patch.
|
||||
decor_rects = await page.evaluate(r"""() => {
|
||||
const out = [];
|
||||
function pushRects(el){
|
||||
const rs = el.getClientRects();
|
||||
for (const r of rs){
|
||||
if (r.width >= 1 && r.height >= 1){
|
||||
out.push({x: r.x, y: r.y, w: r.width, h: r.height});
|
||||
}
|
||||
}
|
||||
}
|
||||
// Explicit author opt-out.
|
||||
for (const el of document.querySelectorAll('[data-decor], [aria-hidden="true"]')){
|
||||
pushRects(el);
|
||||
}
|
||||
// Effective-opacity heuristic: ghost / watermark layer.
|
||||
function effOpacity(el){
|
||||
let op = 1;
|
||||
let cur = el;
|
||||
while (cur && cur !== document.body){
|
||||
const s = getComputedStyle(cur);
|
||||
if (s.display === 'none' || s.visibility === 'hidden') return 0;
|
||||
const o = parseFloat(s.opacity);
|
||||
if (!isNaN(o)) op *= o;
|
||||
cur = cur.parentElement;
|
||||
}
|
||||
return op;
|
||||
}
|
||||
for (const el of document.querySelectorAll('body *')){
|
||||
if (el.closest('[data-decor], [aria-hidden="true"]')) continue;
|
||||
const op = effOpacity(el);
|
||||
if (op > 0 && op <= 0.2) pushRects(el);
|
||||
}
|
||||
return out;
|
||||
}""")
|
||||
screenshot_bytes = await page.screenshot(
|
||||
clip={"x": 0, "y": 0, "width": viewport_width, "height": viewport_height},
|
||||
type="png",
|
||||
)
|
||||
await browser.close()
|
||||
width = int(dims["scrollWidth"])
|
||||
height = int(dims["scrollHeight"])
|
||||
# Tolerance is ratio + fixed buffer (default 2% + 5px → ~19px at 720 /
|
||||
# ~30px at 1280) to absorb sub-pixel rendering noise and line-box
|
||||
# bleed. Both axes use the same ratio + buffer formula, scaled to the
|
||||
# axis's viewport length.
|
||||
tol_y = int(viewport_height * overflow_tolerance_ratio) + overflow_tolerance_px
|
||||
tol_x = int(viewport_width * overflow_tolerance_ratio) + overflow_tolerance_px
|
||||
result = {
|
||||
"success": True,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"viewport_width": viewport_width,
|
||||
"viewport_height": viewport_height,
|
||||
"overflow_x": max(0, width - viewport_width - tol_x),
|
||||
"overflow_y": max(0, height - viewport_height - tol_y),
|
||||
"details": dims,
|
||||
"overlaps": overlaps,
|
||||
}
|
||||
if detect_blank and screenshot_bytes is not None:
|
||||
blank = _detect_largest_blank(screenshot_bytes, decor_rects=decor_rects)
|
||||
if blank is not None:
|
||||
blank["flagged"] = (
|
||||
blank["area_ratio"] >= blank_min_area_ratio
|
||||
and min(blank["w"], blank["h"]) >= blank_min_side
|
||||
)
|
||||
result["blank"] = blank
|
||||
return result
|
||||
except Exception:
|
||||
return {"success": False, "error": traceback.format_exc()}
|
||||
|
||||
|
||||
def _trailing_int(stem: str) -> int | None:
|
||||
m = re.search(r"(\d+)$", stem)
|
||||
return int(m.group(1)) if m else None
|
||||
|
||||
|
||||
def _collect_targets(args) -> list[Path]:
|
||||
if args.files:
|
||||
return [Path(f).resolve() for f in args.files]
|
||||
|
||||
base = Path(args.dir).resolve()
|
||||
if not base.is_dir():
|
||||
raise SystemExit(f"check_slides: --dir {base} is not a directory")
|
||||
|
||||
files = sorted(base.glob("*.html"), key=lambda p: (_trailing_int(p.stem) or 0, p.name))
|
||||
if args.positions:
|
||||
wanted = set(args.positions)
|
||||
files = [p for p in files if (_trailing_int(p.stem) in wanted)]
|
||||
return files
|
||||
|
||||
|
||||
async def _measure_one(path: Path, sem: asyncio.Semaphore) -> tuple[Path, dict]:
|
||||
try:
|
||||
html = path.read_text(encoding="utf-8")
|
||||
except Exception as exc:
|
||||
return path, {"success": False, "error": f"read failed: {exc}"}
|
||||
async with sem:
|
||||
result = await measure_slide_dimensions(
|
||||
html,
|
||||
viewport_width=_VIEWPORT_W,
|
||||
viewport_height=_VIEWPORT_H,
|
||||
proxy=_PROXY,
|
||||
detect_overlaps=True,
|
||||
detect_blank=True,
|
||||
file_path=str(path),
|
||||
)
|
||||
return path, result
|
||||
|
||||
|
||||
async def _measure_all(targets: list[Path]) -> list[tuple[Path, dict]]:
|
||||
# One event loop drives every slide concurrently; the semaphore caps how
|
||||
# many headless browsers run at once so a large deck can't exhaust memory.
|
||||
sem = asyncio.Semaphore(min(_MAX_CONCURRENCY, len(targets)))
|
||||
return await asyncio.gather(*(_measure_one(p, sem) for p in targets))
|
||||
|
||||
|
||||
def _format_report(results: list[tuple[Path, dict]]) -> tuple[str, bool]:
|
||||
lines = [
|
||||
f"Checked {len(results)} slide(s) at viewport {_VIEWPORT_W}x{_VIEWPORT_H} "
|
||||
f"(actual rendered width × height; '✗' marks overflow, overlap, or large blank):"
|
||||
]
|
||||
overflow_count = blank_count = overlap_count = fail_count = ok_count = 0
|
||||
for path, r in results:
|
||||
label = path.name
|
||||
if not r.get("success"):
|
||||
err = (r.get("error") or "unknown")
|
||||
err = err.strip().splitlines()[-1][:140] if err else "unknown"
|
||||
lines.append(f"- {label}: render FAILED — {err}")
|
||||
fail_count += 1
|
||||
continue
|
||||
w, h = r["width"], r["height"]
|
||||
ox = r.get("overflow_x", 0)
|
||||
oy = r.get("overflow_y", 0)
|
||||
blank = r.get("blank") or {}
|
||||
overlaps = r.get("overlaps") or []
|
||||
flags = []
|
||||
if ox > 0:
|
||||
flags.append(f"horizontal overflow +{ox}px")
|
||||
if oy > 0:
|
||||
flags.append(f"vertical overflow +{oy}px")
|
||||
if blank.get("flagged"):
|
||||
flags.append(
|
||||
f"large blank region {blank['w']}x{blank['h']}px "
|
||||
f"@ ({blank['x']},{blank['y']}) ≈ {int(blank['area_ratio'] * 100)}% of canvas"
|
||||
)
|
||||
if overlaps:
|
||||
flags.append(f"{len(overlaps)} text overlap(s)")
|
||||
if flags:
|
||||
if ox > 0 or oy > 0:
|
||||
overflow_count += 1
|
||||
if blank.get("flagged"):
|
||||
blank_count += 1
|
||||
if overlaps:
|
||||
overlap_count += 1
|
||||
lines.append(f"- {label}: {w}x{h} ✗ {', '.join(flags)}")
|
||||
for o in overlaps[:5]:
|
||||
a, b, ov = o["a"], o["b"], o["overlap"]
|
||||
ta = (a["text"][:36] + "…") if len(a["text"]) > 37 else a["text"]
|
||||
tb = (b["text"][:36] + "…") if len(b["text"]) > 37 else b["text"]
|
||||
lines.append(
|
||||
f" · <{a['tag']}>'{ta}' ⨯ <{b['tag']}>'{tb}' "
|
||||
f"(overlap {ov['w']}x{ov['h']}px @ ({ov['x']},{ov['y']}))"
|
||||
)
|
||||
if len(overlaps) > 5:
|
||||
lines.append(f" · … +{len(overlaps) - 5} more overlap pair(s)")
|
||||
else:
|
||||
ok_count += 1
|
||||
lines.append(f"- {label}: {w}x{h} ✓")
|
||||
|
||||
all_clean = (overflow_count == blank_count == overlap_count == fail_count == 0)
|
||||
if all_clean:
|
||||
lines.append(f"Summary: all {len(results)} slides fit the {_VIEWPORT_W}x{_VIEWPORT_H} canvas with no overlaps or large blank regions.")
|
||||
else:
|
||||
lines.append(
|
||||
f"Summary: {overflow_count} overflowing, "
|
||||
f"{overlap_count} with text overlaps, "
|
||||
f"{blank_count} with large blank, {fail_count} failed, {ok_count} OK. "
|
||||
"Use the Read tool to inspect each flagged file, then Edit for surgical fixes."
|
||||
)
|
||||
return "\n".join(lines), all_clean
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Check rendered slide dimensions / overflow / blank / overlaps.")
|
||||
p.add_argument("files", nargs="*", help="Slide HTML files to check (paths).")
|
||||
p.add_argument("--dir", help="Directory of slide HTMLs (used when no positional files are given).")
|
||||
p.add_argument("--positions", type=int, nargs="*",
|
||||
help="When --dir is used, restrict to files whose name ends with these integers.")
|
||||
p.add_argument("--json", action="store_true", help="Emit raw per-file measurements as JSON.")
|
||||
p.add_argument("--output", type=Path, default=None, help="Write output here instead of stdout.")
|
||||
args = p.parse_args()
|
||||
|
||||
if not args.files and not args.dir:
|
||||
p.error("provide one or more file paths, or --dir <slides_dir>")
|
||||
|
||||
targets = _collect_targets(args)
|
||||
if not targets:
|
||||
msg = "check_slides: no slide files found."
|
||||
if args.output:
|
||||
args.output.write_text(msg + "\n", encoding="utf-8")
|
||||
else:
|
||||
sys.stdout.write(msg + "\n")
|
||||
return 1
|
||||
|
||||
results = asyncio.run(_measure_all(targets))
|
||||
|
||||
if args.json:
|
||||
payload = json.dumps(
|
||||
{str(path): r for path, r in results},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
all_clean = all(r.get("success") and r.get("overflow_x", 0) == 0
|
||||
and r.get("overflow_y", 0) == 0
|
||||
and not (r.get("blank") or {}).get("flagged") and not (r.get("overlaps") or [])
|
||||
for _, r in results)
|
||||
else:
|
||||
payload, all_clean = _format_report(results)
|
||||
|
||||
if args.output:
|
||||
args.output.write_text(payload + "\n", encoding="utf-8")
|
||||
else:
|
||||
sys.stdout.write(payload + "\n")
|
||||
|
||||
return 0 if all_clean else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+52
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract every <style>...</style> block from a reference HTML and write the
|
||||
concatenated CSS to <output_dir>/global.css.
|
||||
|
||||
Usage:
|
||||
python extract_html_style.py <html_path> <output_dir> [--name FILE]
|
||||
|
||||
The output file name defaults to global.css so it slots straight into the
|
||||
ppt skill's `<work_dir>/slides/global.css` convention. Pass --name to override.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
STYLE_RE = re.compile(r"<style\b[^>]*>(.*?)</style>", re.DOTALL | re.IGNORECASE)
|
||||
|
||||
|
||||
def extract_style_blocks(html_text: str) -> list[str]:
|
||||
return [m.group(1) for m in STYLE_RE.finditer(html_text)]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
|
||||
ap.add_argument("html_path", type=Path, help="path to source HTML file")
|
||||
ap.add_argument("output_dir", type=Path, help="directory to write the CSS into")
|
||||
ap.add_argument("--name", default="global.css",
|
||||
help="output CSS file name (default: global.css)")
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.html_path.is_file():
|
||||
print(f"error: {args.html_path} is not a file", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
text = args.html_path.read_text(encoding="utf-8", errors="replace")
|
||||
blocks = extract_style_blocks(text)
|
||||
if not blocks:
|
||||
print(f"error: no <style> blocks found in {args.html_path}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
css = "\n\n".join(b.strip() for b in blocks)
|
||||
out = args.output_dir / args.name
|
||||
out.write_text(css, encoding="utf-8")
|
||||
print(f"wrote {out} ({len(css)} bytes from {len(blocks)} <style> block(s))")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+270
@@ -0,0 +1,270 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Analyse a reference .pptx and synthesise a deck-wide design footprint.
|
||||
|
||||
Reads `ppt/theme/theme1.xml` (color scheme + font scheme) and the first
|
||||
`ppt/slides/slide1.xml` (actual rPr usage) from the pptx zip. Produces:
|
||||
|
||||
<output_dir>/global.css — slide-canvas CSS using extracted variables
|
||||
<output_dir>/design.json — palette + typography in slides_brief.json
|
||||
shape, plus the raw theme dump for audit
|
||||
|
||||
Usage:
|
||||
python extract_pptx_style.py <pptx_path> <output_dir>
|
||||
|
||||
The script imitates the pptx's palette / typography so downstream slide
|
||||
generation can stay faithful to the reference deck. No external deps —
|
||||
uses zipfile + xml.etree only.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from xml.etree import ElementTree as ET
|
||||
|
||||
NS = {
|
||||
"a": "http://schemas.openxmlformats.org/drawingml/2006/main",
|
||||
"p": "http://schemas.openxmlformats.org/presentationml/2006/main",
|
||||
"r": "http://schemas.openxmlformats.org/officeDocument/2006/relationships",
|
||||
}
|
||||
|
||||
# Standard PowerPoint preset colors → hex. Used when a clrScheme slot points at
|
||||
# a named preset rather than an explicit srgbClr.
|
||||
PRESET_COLORS = {
|
||||
"windowText": "#000000", "windowBackground": "#FFFFFF",
|
||||
"menuText": "#000000", "menuBar": "#FFFFFF",
|
||||
}
|
||||
|
||||
|
||||
def _hex_from_node(node: ET.Element | None) -> str | None:
|
||||
if node is None:
|
||||
return None
|
||||
srgb = node.find("a:srgbClr", NS)
|
||||
if srgb is not None and srgb.get("val"):
|
||||
return "#" + srgb.get("val").upper()
|
||||
sys_ = node.find("a:sysClr", NS)
|
||||
if sys_ is not None:
|
||||
# sysClr always carries a lastClr fallback for the rendered hex.
|
||||
last = sys_.get("lastClr")
|
||||
if last:
|
||||
return "#" + last.upper()
|
||||
val = sys_.get("val")
|
||||
if val and val in PRESET_COLORS:
|
||||
return PRESET_COLORS[val]
|
||||
preset = node.find("a:prstClr", NS)
|
||||
if preset is not None and preset.get("val") in PRESET_COLORS:
|
||||
return PRESET_COLORS[preset.get("val")]
|
||||
return None
|
||||
|
||||
|
||||
def parse_theme(theme_xml: bytes) -> dict:
|
||||
root = ET.fromstring(theme_xml)
|
||||
out: dict = {"palette": {}, "fonts": {"heading": None, "body": None}}
|
||||
|
||||
clr = root.find(".//a:clrScheme", NS)
|
||||
if clr is not None:
|
||||
for slot in (
|
||||
"dk1", "lt1", "dk2", "lt2",
|
||||
"accent1", "accent2", "accent3", "accent4", "accent5", "accent6",
|
||||
"hlink", "folHlink",
|
||||
):
|
||||
node = clr.find(f"a:{slot}", NS)
|
||||
hex_val = _hex_from_node(node)
|
||||
if hex_val:
|
||||
out["palette"][slot] = hex_val
|
||||
|
||||
fonts = root.find(".//a:fontScheme", NS)
|
||||
if fonts is not None:
|
||||
major = fonts.find("a:majorFont/a:latin", NS)
|
||||
if major is not None and major.get("typeface"):
|
||||
out["fonts"]["heading"] = major.get("typeface")
|
||||
minor = fonts.find("a:minorFont/a:latin", NS)
|
||||
if minor is not None and minor.get("typeface"):
|
||||
out["fonts"]["body"] = minor.get("typeface")
|
||||
# East-asian fallback — useful when the deck is Chinese.
|
||||
major_ea = fonts.find("a:majorFont/a:ea", NS)
|
||||
if major_ea is not None and major_ea.get("typeface"):
|
||||
out["fonts"]["heading_ea"] = major_ea.get("typeface")
|
||||
minor_ea = fonts.find("a:minorFont/a:ea", NS)
|
||||
if minor_ea is not None and minor_ea.get("typeface"):
|
||||
out["fonts"]["body_ea"] = minor_ea.get("typeface")
|
||||
|
||||
return out
|
||||
|
||||
|
||||
def sample_slide_usage(slide_xml: bytes) -> dict:
|
||||
"""Inspect actual srgbClr fills + rPr typefaces used on slide 1."""
|
||||
root = ET.fromstring(slide_xml)
|
||||
colors = collections.Counter()
|
||||
typefaces = collections.Counter()
|
||||
sizes = collections.Counter()
|
||||
for srgb in root.iter(f"{{{NS['a']}}}srgbClr"):
|
||||
if srgb.get("val"):
|
||||
colors["#" + srgb.get("val").upper()] += 1
|
||||
for rpr in root.iter(f"{{{NS['a']}}}rPr"):
|
||||
sz = rpr.get("sz")
|
||||
if sz:
|
||||
sizes[int(sz)] += 1
|
||||
for tag in ("latin", "ea", "cs"):
|
||||
face = rpr.find(f"a:{tag}", NS)
|
||||
if face is not None and face.get("typeface"):
|
||||
typefaces[face.get("typeface")] += 1
|
||||
return {
|
||||
"top_colors": colors.most_common(8),
|
||||
"top_typefaces": typefaces.most_common(6),
|
||||
"top_font_sizes_hpt": sizes.most_common(6), # OOXML stores sz in hundredths of pt
|
||||
}
|
||||
|
||||
|
||||
def synthesize_design(theme: dict, slide: dict | None) -> dict:
|
||||
p = theme["palette"]
|
||||
bg = p.get("lt1") or p.get("lt2") or "#FFFFFF"
|
||||
primary = p.get("dk1") or p.get("dk2") or "#1C1917"
|
||||
accent = p.get("accent1") or "#2563EB"
|
||||
|
||||
# If slide1 uses a markedly more common non-bg, non-primary color, prefer
|
||||
# it as the accent — themes sometimes carry a placeholder accent that the
|
||||
# actual deck never uses.
|
||||
if slide:
|
||||
for hex_val, _count in slide["top_colors"]:
|
||||
if hex_val.upper() not in {bg.upper(), primary.upper()}:
|
||||
accent = hex_val
|
||||
break
|
||||
|
||||
heading = theme["fonts"].get("heading") or "Inter"
|
||||
body = theme["fonts"].get("body") or heading
|
||||
return {
|
||||
"background": bg,
|
||||
"primary": primary,
|
||||
"accent": accent,
|
||||
"heading_font": heading,
|
||||
"body_font": body,
|
||||
}
|
||||
|
||||
|
||||
CSS_TEMPLATE = """\
|
||||
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=Crimson+Text:wght@400;600;700&family=Noto+Sans+SC:wght@400;500;700&family=Noto+Serif+SC:wght@400;700&display=swap');
|
||||
|
||||
:root {{
|
||||
--bg: {bg};
|
||||
--primary: {primary};
|
||||
--accent: {accent};
|
||||
}}
|
||||
|
||||
* {{ box-sizing: border-box; margin: 0; padding: 0; }}
|
||||
|
||||
body {{
|
||||
font-family: '{body_font}', 'Inter', 'Helvetica Neue', 'Segoe UI', Arial,
|
||||
'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||
background-color: var(--bg);
|
||||
color: var(--primary);
|
||||
}}
|
||||
|
||||
.slide {{
|
||||
width: 1280px;
|
||||
height: 720px;
|
||||
background-color: var(--bg);
|
||||
padding: 64px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}}
|
||||
|
||||
h1, h2, h3, .heading {{
|
||||
font-family: '{heading_font}', 'Crimson Text', 'Times New Roman', Georgia,
|
||||
'Noto Serif SC', 'Songti SC', 'SimSun', serif;
|
||||
color: var(--primary);
|
||||
line-height: 1.1;
|
||||
}}
|
||||
|
||||
h1 {{ font-size: 88px; }}
|
||||
h2 {{ font-size: 52px; }}
|
||||
h3 {{ font-size: 32px; }}
|
||||
|
||||
p, li, td, .body-text {{
|
||||
font-family: '{body_font}', 'Inter', 'Helvetica Neue', Arial,
|
||||
'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||
font-size: 22px;
|
||||
line-height: 1.55;
|
||||
color: var(--primary);
|
||||
}}
|
||||
|
||||
.accent-fg {{ color: var(--accent); }}
|
||||
.accent-bg {{ background-color: var(--accent); color: var(--bg); }}
|
||||
.accent-border {{ border: 4px solid var(--accent); }}
|
||||
"""
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
|
||||
ap.add_argument("pptx_path", type=Path)
|
||||
ap.add_argument("output_dir", type=Path)
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.pptx_path.is_file() or args.pptx_path.suffix.lower() != ".pptx":
|
||||
print(f"error: {args.pptx_path} is not a .pptx file", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(args.pptx_path) as z:
|
||||
names = z.namelist()
|
||||
theme_files = sorted(n for n in names if re.match(r"ppt/theme/theme\d+\.xml$", n))
|
||||
if not theme_files:
|
||||
print("error: no ppt/theme/theme*.xml inside pptx", file=sys.stderr)
|
||||
return 1
|
||||
theme_xml = z.read(theme_files[0])
|
||||
slide_xml = None
|
||||
if "ppt/slides/slide1.xml" in names:
|
||||
slide_xml = z.read("ppt/slides/slide1.xml")
|
||||
|
||||
theme = parse_theme(theme_xml)
|
||||
slide = sample_slide_usage(slide_xml) if slide_xml else None
|
||||
design = synthesize_design(theme, slide)
|
||||
|
||||
css = CSS_TEMPLATE.format(
|
||||
bg=design["background"],
|
||||
primary=design["primary"],
|
||||
accent=design["accent"],
|
||||
heading_font=design["heading_font"],
|
||||
body_font=design["body_font"],
|
||||
)
|
||||
css_path = args.output_dir / "global.css"
|
||||
css_path.write_text(css, encoding="utf-8")
|
||||
|
||||
design_payload = {
|
||||
"design": {
|
||||
"palette": {
|
||||
"background": design["background"],
|
||||
"primary": design["primary"],
|
||||
"accent": design["accent"],
|
||||
},
|
||||
"typography": {
|
||||
"heading": design["heading_font"],
|
||||
"body": design["body_font"],
|
||||
},
|
||||
"reference": f"imitating {args.pptx_path.name}",
|
||||
},
|
||||
"source_pptx": str(args.pptx_path.resolve()),
|
||||
"raw_theme": theme,
|
||||
"slide1_usage": slide,
|
||||
}
|
||||
design_path = args.output_dir / "design.json"
|
||||
design_path.write_text(json.dumps(design_payload, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8")
|
||||
|
||||
print(f"wrote {css_path}")
|
||||
print(f"wrote {design_path}")
|
||||
print(f"palette: bg={design['background']} primary={design['primary']} accent={design['accent']}")
|
||||
print(f"fonts: heading={design['heading_font']} body={design['body_font']}")
|
||||
if slide:
|
||||
print(f"slide1 top colors: {slide['top_colors'][:3]}")
|
||||
print(f"slide1 top typefaces: {slide['top_typefaces'][:3]}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in new issue
Block a user