BlueGit

golden-age paperdoll-renderer

Merge feature/atlas-only-checkpoint-export (Tim t1065u/t1066u/t1067u)

Atlas-only finals + checkpoint Resume, Exporteer frame, diffuse sRGB fix, v0.1.6.

timfa Grok Bot · pushed by timfa

Authored 2026-11-10 · 4 days ago

Hash
efc6430a24ee64b97474eb8fb3c6b24fc5d06b6d
Pipeline
Failed
16 files changed +563 −126
Cargo.lock +1 −1
@@ -2158,7 +2158,7 @@ dependencies = [
2158 2158
2159 2159 [[package]]
2160 2160 name = "paperdoll-renderer"
2161 version = "0.1.5"
2161 version = "0.1.6"
2162 2162 dependencies = [
2163 2163 "eframe",
2164 2164 "egui_glow",
@@ -2158,7 +2158,7 @@ dependencies = [
2158
2158
2159[[package]]
2159[[package]]
2160name = "paperdoll-renderer"
2160name = "paperdoll-renderer"
2161version = "0.1.5"
2161version = "0.1.6"
2162dependencies = [
2162dependencies = [
2163 "eframe",
2163 "eframe",
2164 "egui_glow",
2164 "egui_glow",
Cargo.toml +1 −1
@@ -1,6 +1,6 @@
1 1 [package]
2 2 name = "paperdoll-renderer"
3 version = "0.1.5"
3 version = "0.1.6"
4 4 edition = "2021"
5 5 license = "MIT"
6 6 publish = false
@@ -1,6 +1,6 @@
1[package]
1[package]
2name = "paperdoll-renderer"
2name = "paperdoll-renderer"
3version = "0.1.5"
3version = "0.1.6"
4edition = "2021"
4edition = "2021"
5license = "MIT"
5license = "MIT"
6publish = false
6publish = false
README.md +9 −10
@@ -28,7 +28,7 @@ The first project view shows the bodies. **Choose male / Choose female** selects
28 28 - Drag to orbit. **Show fixed render frame** shows the yellow orthographic export frame; set **Fixed render height**, **Center Y** (image center) and **Pivot Y** (orbit point) per male/female and save. Without a manual pivot the camera orbits around the vertical centre of the model's rest-pose bounds (or **Center Y** as a fallback with a diagnostic when there is no geometry); **Auto centre** clears a manual override. The fixed frame and production are orthographic, so there is no perspective distortion at any pitch. **Fit animations** computes once a render height and Center Y that contain all clips, all frames (at project fps) and all 60 angles, with a 5% margin on each side; the pivot stays. Afterwards the frame is fixed again: there is no automatic zoom per frame, clip or layer. Equipment that sticks out beyond the body needs extra height set by hand. Body and equipment share exactly the same camera. Unchecking gives a free perspective preview with scroll zoom, seen from 2.5× further away with a narrower field of view (≈15.4°) for weak perspective; that zoom does not change the export. A real 24-bit depth buffer keeps front surfaces in front.
29 29 - Pick an animation clip and use **Pause / Play**.
30 30 - **Yaw − / Yaw +** step through twelve 30° angles; **pitch -60 / -30 / 0 / 30 / 60** choose the height. Yaw 0 looks at the front of every model. The orientation is fixed by the glTF standard (+Y up, the front faces +Z, the left side is +X); models and animations load as-is, without automatic correction: yaw 0 = front (right hand on the left of the image), 90 = left side, 180 = back, 270 = right side; positive pitch looks from above. A test with labelled markers guards this convention.
31 - **Render all angles** renders all animations × all frames at project fps × 60 angles (5 pitches × 12 yaws). After the loose maps it writes one atlas per map type plus a `.ppda` file (see *Atlases and .ppda*). Production keeps several GPU views in flight (PBO readback ring) and encodes/writes PNGs on a bounded background IO queue so the next view can start without waiting on disk; atlases are built from in-memory PNG bytes when available instead of re-decoding from disk. The live preview of the latest finished pass is a latest-wins slot: the render worker never waits on texture upload or UI repaint, and the UI may skip intermediate frames. Shown passes still follow highres → detail → target/diffuse → depth → normal → specular, keep their aspect with centered letterboxing and nearest-neighbor display. Progress counts real passes; **Cancel render** keeps valid partial output. **Live 3D** returns to the model; a next finished render pass may replace the preview again.
31 - **Render all angles** renders all animations × all frames at project fps × 60 angles (5 pitches × 12 yaws). It builds one atlas per map type (from in-memory cell bytes) plus a `.ppda` file; loose per-view PNGs are only temporary checkpoints (see *Atlases and .ppda*). Production keeps several GPU views in flight (PBO readback ring) and encodes/writes PNGs on a bounded background IO queue so the next view can start without waiting on disk; atlases are built from in-memory PNG bytes when available instead of re-decoding from disk. The live preview of the latest finished pass is a latest-wins slot: the render worker never waits on texture upload or UI repaint, and the UI may skip intermediate frames. Shown passes still follow highres → detail → target/diffuse → depth → normal → specular, keep their aspect with centered letterboxing and nearest-neighbor display. Progress counts real passes; **Cancel render** keeps valid partial output. **Live 3D** returns to the model; a next finished render pass may replace the preview again.
32 32 - Set width, height and fps in the Bodies view. New projects start at 92 × 128 pixels and 8 fps with 8-bit gray depth; an existing `.ppd` keeps its saved values. A new project with the default bodies already has **Fit animations** applied for both sexes. **Trim atlas: body / equipment** (both on by default; older `.ppd` files load with both on) crops the atlas views to their content, see *Atlases and .ppda*. **Semi-transparency allowed (body)** is off by default (also for older `.ppd` files). Off means binary alpha: after the resolve, a pixel with coverage ≥ 0.5 becomes fully opaque in every map (diffuse, normal, specular, depth) and anything below becomes fully empty in every map, so all masks match. On keeps the soft edges. Each equipment preset has its own **Semi-transparency allowed** checkbox (also off by default). The bundled `female-hair` preset has it on. The setting is in `render.json` (`semi_transparency`), so a run only resumes with the same setting.
33 33 - **Look for interrupted runs** (also in the equipment editor) lists cancelled, failed or aborted runs of the same body/equipment set with the number of recorded PNGs. **Resume** only continues if source files and settings are exactly equal to that run; otherwise the difference is shown. After cancelling, the run appears in this list immediately. See *Resuming* below.
34 34
@@ -50,7 +50,7 @@ A body without a skin can also be chosen as male or female model: **OBJ+MTL** (w
50 50
51 51 ## 2D preview
52 52
53 From **Body** or the body view, **2D preview** opens the rendered compositor over the result folders `renders/<sex>-<settype>-atlas/`. The preview reads every export through its `.ppda` and the four atlases (validated, see *Atlases and .ppda*), not through the loose PNGs. Runs without a valid `.ppda` are skipped; their count is shown below the export counter, with the reason on hover. The decoded atlases of the current selection stay in memory until you choose another selection or use **Refresh export sets**.
53 From **Body** or the body view, **2D preview** opens the rendered compositor over the result folders `renders/<sex>-<settype>-atlas/`. The preview reads every export through its `.ppda` and the four atlases (validated, see *Atlases and .ppda*), not through loose PNGs. **Exporteer frame** writes the current pose's composed maps (diffuse, normal, depth in the atlas encoding, specular) plus the lit preview image (current light/rim) as separate PNGs into a folder you pick. Runs without a valid `.ppda` are skipped; their count is shown below the export counter, with the reason on hover. The decoded atlases of the current selection stay in memory until you choose another selection or use **Refresh export sets**.
54 54
55 55 - Choose **Male / Female**. Only export sets with the same project/rig/export contract (`CompatKey`) can be combined; equipment appears as a multi-select when those contracts match.
56 56 - Choose a clip from the body export inventory. **Play / Pause**, speed and export fps drive a looping frame time (elapsed × fps); missing maps/frames show a diagnostic instead of a silent substitute.
@@ -123,14 +123,15 @@ The depth format is chosen in the Body screen (**Depth map**), stored in the `.p
123 123 Alpha is kept (scaled to the bit depth); transparent pixels have value 0. Geometry outside the range is clamped (not clipped) and reported in the progress; normal is view-space RGB+alpha; specular is linear strength (body: KHR_materials_specular or zero; equipment including the configured shiny map). Highres and detail are visible intermediate steps only and are not stored.
124 124
125 125 ```text
126 renders/male-body-atlas/raw files/male/body/clip001-walk/f000012/p-60_y330_diffuse.png
127 126 renders/male-body-atlas/male-body-diffuse-atlas.png (and -normal, -depth, -specular)
128 127 renders/male-body-atlas/male-body-atlas.ppda
128 # during an incomplete run only:
129 renders/inprogress-…/checkpoint/raw files/male/body/clip001-walk/f000012/p-60_y330_diffuse.png
129 130 ```
130 131
131 132 Per sex and set type there is one result folder `renders/<sex>-<settype>-atlas/` holding only the newest complete render. A render is built in a temporary folder `renders/inprogress-<time>-<pid>-<sex>-<settype>/`. Only when it is complete and the atlases/`.ppda` are verified is the previous result folder moved aside (`.replaced-…`), the new one renamed into place and the old one deleted. On cancel, error or crash the last good result folder therefore stays. An interrupted swap is repaired at the next render: a moved-aside old folder comes back, and a finished but not yet placed render is placed. Old `run-*` folders from earlier versions are not deleted automatically; the 2D preview ignores them with a message.
132 133
133 A finished result folder contains only consumer output:
134 A finished result folder contains only consumer output (Tim t1065u):
134 135
135 136 ```
136 137 renders/male-body-atlas/
@@ -139,11 +140,9 @@ renders/male-body-atlas/
139 140 male-body-normal-atlas.png
140 141 male-body-depth-atlas.png
141 142 male-body-specular-atlas.png
142 raw files/
143 male/body/<clip>/f000000/p-60_y000_diffuse.png …
144 143 ```
145 144
146 All loose per-frame images are grouped under `raw files/`. `render.json`, `complete.json`, `progress.jsonl`, `resume.jsonl`, `run.lock` and the markers of earlier attempts exist only in the temporary `inprogress-*` folder; they are removed after the render is verified and before the folder is put in place. What is still needed afterwards (the export contract and the SHA-256 of every loose PNG) is in the `.ppda`. Only after damage during a resumed render can a kept `.corrupt-aNNN-K` file remain. Older result folders (loose images and `render.json`/`complete.json` directly in the folder) are still read.
145 Loose per-view PNGs are **not** kept in the result. During an in-progress run they live under `checkpoint/raw files/…` as Resume checkpoints (journal keys stay `raw files/…` for `.ppda` identity); atlases are built from in-memory PNG bytes when available. After the atlases/`.ppda` are verified the checkpoint tree is deleted, then bookkeeping (`render.json`, `complete.json`, `progress.jsonl`, `resume.jsonl`, `run.lock`, attempt markers) is stripped and the folder is put in place. The `.ppda` still embeds the export contract and the SHA-256 of every logical cell. Quarantined `.corrupt-aNNN-K` files (and leftover `.part-*`) are rescued to the result root so they survive checkpoint cleanup. Older result folders that still have `raw files/` or `render.json`/`complete.json` are still read.
147 146
148 147 While building: `render.json` (schema 6; with `depth_near`, `depth_far`, `depth_bits`, `depth_encoding`) describes project identity, source fingerprints (SHA-256 of the body file and optionally the encoded equipment), camera/pivot/sample/material contract (pitches −60…+60, yaws 0…330), clip inventory and count. In the temporary folder only `complete.json` (same schema, with per-PNG SHA-256 and the names and SHA-256 of atlases and `.ppda`) marks a complete set. Schema 5 (before `depth_bits`) is still read as 16-bit gray. Schema 4 and older (including the old global 0.05–40 m range) are refused by the readers and not mixed; render those again. `cancelled.json`/`failed.json` mark a partial run without a complete marker. The library module `export_set` reads a finished result folder through the `.ppda` (older folders through their `render.json`/`complete.json`), opens only finished runs with matching identity and verified checksums, and reads the real RGBA8 and depth maps (all six formats, converted to 16-bit depth for the compositor) with strict path/format validation. The map/camera/pixel contracts are in `design.md`.
149 148
@@ -162,10 +161,10 @@ When all loose maps are done and verified, also after resuming, the render write
162 161
163 162 A resumed run continues writing in the same temporary `inprogress-*` folder; the result folder is untouched until the run is done. If you start a new (not resumed) render of the same sex and set type, older unfinished `inprogress-*` folders of that target are deleted first, so only the newest remains; the finished result folder, other targets and a folder currently being rendered into are kept.
164 163
165 - Every run keeps `progress.jsonl`: per published PNG the path and SHA-256. `run.lock` prevents two renders writing into the same run at once.
164 - Every run keeps `progress.jsonl`: per published cell the logical path (`raw files/…`) and SHA-256. Cell bytes are stored under `checkpoint/` until the atlases are verified. `run.lock` prevents two renders writing into the same run at once.
166 165 - Resuming is only possible if `render.json` exactly matches what the current job would write: project, sex/set, SHA-256 of body and equipment, target, fps, framing/pivot, camera, clips and maps. A finished, locked or different run is refused.
167 - Before rendering, every planned PNG is checked. Files with a matching checksum stay unchanged and are not rendered again. Missing files are recreated. Damaged files are renamed to `<name>.corrupt-aNNN-K` and then recreated; they are not deleted. An existing file without a journal line (crash between writing and recording) stays if the re-rendered bytes are equal. Leftover `.part-*` files are not touched.
168 - After the loose maps, the atlases and `.ppda` are written (see above).
166 - Before rendering, every planned cell is checked under `checkpoint/` (or a legacy root-level `raw files/`). Matching checksums stay unchanged and are not rendered again. Missing cells are recreated. Damaged files are quarantined as `<name>.corrupt-aNNN-K` at the run root and then recreated; they are not deleted. An existing file without a journal line (crash between writing and recording) stays if the re-rendered bytes are equal. Leftover `.part-*` files are rescued to the run root when checkpoints are cleared.
167 - After the cell checkpoints, the atlases and `.ppda` are written from in-memory bytes when present (see above); then checkpoints are removed.
169 168 - Progress starts at the work that is already correct; skipped images are labelled *already present, checksums match*. You can cancel again and resume later. A previous `cancelled.json`/`failed.json` is kept as `attempt-NNN-…json`, and `resume.jsonl` records per attempt what was verified, missing and quarantined. Only when everything matches is `complete.json` with all checksums written. Then these bookkeeping files are cleaned up (`complete.json` last) and the folder is put in place. A crash between cleanup and placement is repaired at the next render: the finished folder is cleaned and placed.
170 169
171 170 ![Resuming an interrupted run in the body view](docs/screenshots/body-resume.png)
@@ -28,7 +28,7 @@ The first project view shows the bodies. **Choose male / Choose female** selects
28- Drag to orbit. **Show fixed render frame** shows the yellow orthographic export frame; set **Fixed render height**, **Center Y** (image center) and **Pivot Y** (orbit point) per male/female and save. Without a manual pivot the camera orbits around the vertical centre of the model's rest-pose bounds (or **Center Y** as a fallback with a diagnostic when there is no geometry); **Auto centre** clears a manual override. The fixed frame and production are orthographic, so there is no perspective distortion at any pitch. **Fit animations** computes once a render height and Center Y that contain all clips, all frames (at project fps) and all 60 angles, with a 5% margin on each side; the pivot stays. Afterwards the frame is fixed again: there is no automatic zoom per frame, clip or layer. Equipment that sticks out beyond the body needs extra height set by hand. Body and equipment share exactly the same camera. Unchecking gives a free perspective preview with scroll zoom, seen from 2.5× further away with a narrower field of view (≈15.4°) for weak perspective; that zoom does not change the export. A real 24-bit depth buffer keeps front surfaces in front.
28- Drag to orbit. **Show fixed render frame** shows the yellow orthographic export frame; set **Fixed render height**, **Center Y** (image center) and **Pivot Y** (orbit point) per male/female and save. Without a manual pivot the camera orbits around the vertical centre of the model's rest-pose bounds (or **Center Y** as a fallback with a diagnostic when there is no geometry); **Auto centre** clears a manual override. The fixed frame and production are orthographic, so there is no perspective distortion at any pitch. **Fit animations** computes once a render height and Center Y that contain all clips, all frames (at project fps) and all 60 angles, with a 5% margin on each side; the pivot stays. Afterwards the frame is fixed again: there is no automatic zoom per frame, clip or layer. Equipment that sticks out beyond the body needs extra height set by hand. Body and equipment share exactly the same camera. Unchecking gives a free perspective preview with scroll zoom, seen from 2.5× further away with a narrower field of view (≈15.4°) for weak perspective; that zoom does not change the export. A real 24-bit depth buffer keeps front surfaces in front.
29- Pick an animation clip and use **Pause / Play**.
29- Pick an animation clip and use **Pause / Play**.
30- **Yaw − / Yaw +** step through twelve 30° angles; **pitch -60 / -30 / 0 / 30 / 60** choose the height. Yaw 0 looks at the front of every model. The orientation is fixed by the glTF standard (+Y up, the front faces +Z, the left side is +X); models and animations load as-is, without automatic correction: yaw 0 = front (right hand on the left of the image), 90 = left side, 180 = back, 270 = right side; positive pitch looks from above. A test with labelled markers guards this convention.
30- **Yaw − / Yaw +** step through twelve 30° angles; **pitch -60 / -30 / 0 / 30 / 60** choose the height. Yaw 0 looks at the front of every model. The orientation is fixed by the glTF standard (+Y up, the front faces +Z, the left side is +X); models and animations load as-is, without automatic correction: yaw 0 = front (right hand on the left of the image), 90 = left side, 180 = back, 270 = right side; positive pitch looks from above. A test with labelled markers guards this convention.
31- **Render all angles** renders all animations × all frames at project fps × 60 angles (5 pitches × 12 yaws). After the loose maps it writes one atlas per map type plus a `.ppda` file (see *Atlases and .ppda*). Production keeps several GPU views in flight (PBO readback ring) and encodes/writes PNGs on a bounded background IO queue so the next view can start without waiting on disk; atlases are built from in-memory PNG bytes when available instead of re-decoding from disk. The live preview of the latest finished pass is a latest-wins slot: the render worker never waits on texture upload or UI repaint, and the UI may skip intermediate frames. Shown passes still follow highres → detail → target/diffuse → depth → normal → specular, keep their aspect with centered letterboxing and nearest-neighbor display. Progress counts real passes; **Cancel render** keeps valid partial output. **Live 3D** returns to the model; a next finished render pass may replace the preview again.
31- **Render all angles** renders all animations × all frames at project fps × 60 angles (5 pitches × 12 yaws). It builds one atlas per map type (from in-memory cell bytes) plus a `.ppda` file; loose per-view PNGs are only temporary checkpoints (see *Atlases and .ppda*). Production keeps several GPU views in flight (PBO readback ring) and encodes/writes PNGs on a bounded background IO queue so the next view can start without waiting on disk; atlases are built from in-memory PNG bytes when available instead of re-decoding from disk. The live preview of the latest finished pass is a latest-wins slot: the render worker never waits on texture upload or UI repaint, and the UI may skip intermediate frames. Shown passes still follow highres → detail → target/diffuse → depth → normal → specular, keep their aspect with centered letterboxing and nearest-neighbor display. Progress counts real passes; **Cancel render** keeps valid partial output. **Live 3D** returns to the model; a next finished render pass may replace the preview again.
32- Set width, height and fps in the Bodies view. New projects start at 92 × 128 pixels and 8 fps with 8-bit gray depth; an existing `.ppd` keeps its saved values. A new project with the default bodies already has **Fit animations** applied for both sexes. **Trim atlas: body / equipment** (both on by default; older `.ppd` files load with both on) crops the atlas views to their content, see *Atlases and .ppda*. **Semi-transparency allowed (body)** is off by default (also for older `.ppd` files). Off means binary alpha: after the resolve, a pixel with coverage ≥ 0.5 becomes fully opaque in every map (diffuse, normal, specular, depth) and anything below becomes fully empty in every map, so all masks match. On keeps the soft edges. Each equipment preset has its own **Semi-transparency allowed** checkbox (also off by default). The bundled `female-hair` preset has it on. The setting is in `render.json` (`semi_transparency`), so a run only resumes with the same setting.
32- Set width, height and fps in the Bodies view. New projects start at 92 × 128 pixels and 8 fps with 8-bit gray depth; an existing `.ppd` keeps its saved values. A new project with the default bodies already has **Fit animations** applied for both sexes. **Trim atlas: body / equipment** (both on by default; older `.ppd` files load with both on) crops the atlas views to their content, see *Atlases and .ppda*. **Semi-transparency allowed (body)** is off by default (also for older `.ppd` files). Off means binary alpha: after the resolve, a pixel with coverage ≥ 0.5 becomes fully opaque in every map (diffuse, normal, specular, depth) and anything below becomes fully empty in every map, so all masks match. On keeps the soft edges. Each equipment preset has its own **Semi-transparency allowed** checkbox (also off by default). The bundled `female-hair` preset has it on. The setting is in `render.json` (`semi_transparency`), so a run only resumes with the same setting.
33- **Look for interrupted runs** (also in the equipment editor) lists cancelled, failed or aborted runs of the same body/equipment set with the number of recorded PNGs. **Resume** only continues if source files and settings are exactly equal to that run; otherwise the difference is shown. After cancelling, the run appears in this list immediately. See *Resuming* below.
33- **Look for interrupted runs** (also in the equipment editor) lists cancelled, failed or aborted runs of the same body/equipment set with the number of recorded PNGs. **Resume** only continues if source files and settings are exactly equal to that run; otherwise the difference is shown. After cancelling, the run appears in this list immediately. See *Resuming* below.
34
34
@@ -50,7 +50,7 @@ A body without a skin can also be chosen as male or female model: **OBJ+MTL** (w
50
50
51## 2D preview
51## 2D preview
52
52
53From **Body** or the body view, **2D preview** opens the rendered compositor over the result folders `renders/<sex>-<settype>-atlas/`. The preview reads every export through its `.ppda` and the four atlases (validated, see *Atlases and .ppda*), not through the loose PNGs. Runs without a valid `.ppda` are skipped; their count is shown below the export counter, with the reason on hover. The decoded atlases of the current selection stay in memory until you choose another selection or use **Refresh export sets**.
53From **Body** or the body view, **2D preview** opens the rendered compositor over the result folders `renders/<sex>-<settype>-atlas/`. The preview reads every export through its `.ppda` and the four atlases (validated, see *Atlases and .ppda*), not through loose PNGs. **Exporteer frame** writes the current pose's composed maps (diffuse, normal, depth in the atlas encoding, specular) plus the lit preview image (current light/rim) as separate PNGs into a folder you pick. Runs without a valid `.ppda` are skipped; their count is shown below the export counter, with the reason on hover. The decoded atlases of the current selection stay in memory until you choose another selection or use **Refresh export sets**.
54
54
55- Choose **Male / Female**. Only export sets with the same project/rig/export contract (`CompatKey`) can be combined; equipment appears as a multi-select when those contracts match.
55- Choose **Male / Female**. Only export sets with the same project/rig/export contract (`CompatKey`) can be combined; equipment appears as a multi-select when those contracts match.
56- Choose a clip from the body export inventory. **Play / Pause**, speed and export fps drive a looping frame time (elapsed × fps); missing maps/frames show a diagnostic instead of a silent substitute.
56- Choose a clip from the body export inventory. **Play / Pause**, speed and export fps drive a looping frame time (elapsed × fps); missing maps/frames show a diagnostic instead of a silent substitute.
@@ -123,14 +123,15 @@ The depth format is chosen in the Body screen (**Depth map**), stored in the `.p
123Alpha is kept (scaled to the bit depth); transparent pixels have value 0. Geometry outside the range is clamped (not clipped) and reported in the progress; normal is view-space RGB+alpha; specular is linear strength (body: KHR_materials_specular or zero; equipment including the configured shiny map). Highres and detail are visible intermediate steps only and are not stored.
123Alpha is kept (scaled to the bit depth); transparent pixels have value 0. Geometry outside the range is clamped (not clipped) and reported in the progress; normal is view-space RGB+alpha; specular is linear strength (body: KHR_materials_specular or zero; equipment including the configured shiny map). Highres and detail are visible intermediate steps only and are not stored.
124
124
125```text
125```text
126renders/male-body-atlas/raw files/male/body/clip001-walk/f000012/p-60_y330_diffuse.png
127renders/male-body-atlas/male-body-diffuse-atlas.png (and -normal, -depth, -specular)
126renders/male-body-atlas/male-body-diffuse-atlas.png (and -normal, -depth, -specular)
128renders/male-body-atlas/male-body-atlas.ppda
127renders/male-body-atlas/male-body-atlas.ppda
128# during an incomplete run only:
129renders/inprogress-…/checkpoint/raw files/male/body/clip001-walk/f000012/p-60_y330_diffuse.png
129```
130```
130
131
131Per sex and set type there is one result folder `renders/<sex>-<settype>-atlas/` holding only the newest complete render. A render is built in a temporary folder `renders/inprogress-<time>-<pid>-<sex>-<settype>/`. Only when it is complete and the atlases/`.ppda` are verified is the previous result folder moved aside (`.replaced-…`), the new one renamed into place and the old one deleted. On cancel, error or crash the last good result folder therefore stays. An interrupted swap is repaired at the next render: a moved-aside old folder comes back, and a finished but not yet placed render is placed. Old `run-*` folders from earlier versions are not deleted automatically; the 2D preview ignores them with a message.
132Per sex and set type there is one result folder `renders/<sex>-<settype>-atlas/` holding only the newest complete render. A render is built in a temporary folder `renders/inprogress-<time>-<pid>-<sex>-<settype>/`. Only when it is complete and the atlases/`.ppda` are verified is the previous result folder moved aside (`.replaced-…`), the new one renamed into place and the old one deleted. On cancel, error or crash the last good result folder therefore stays. An interrupted swap is repaired at the next render: a moved-aside old folder comes back, and a finished but not yet placed render is placed. Old `run-*` folders from earlier versions are not deleted automatically; the 2D preview ignores them with a message.
132
133
133A finished result folder contains only consumer output:
134A finished result folder contains only consumer output (Tim t1065u):
134
135
135```
136```
136renders/male-body-atlas/
137renders/male-body-atlas/
@@ -139,11 +140,9 @@ renders/male-body-atlas/
139 male-body-normal-atlas.png
140 male-body-normal-atlas.png
140 male-body-depth-atlas.png
141 male-body-depth-atlas.png
141 male-body-specular-atlas.png
142 male-body-specular-atlas.png
142 raw files/
143 male/body/<clip>/f000000/p-60_y000_diffuse.png …
144```
143```
145
144
146All loose per-frame images are grouped under `raw files/`. `render.json`, `complete.json`, `progress.jsonl`, `resume.jsonl`, `run.lock` and the markers of earlier attempts exist only in the temporary `inprogress-*` folder; they are removed after the render is verified and before the folder is put in place. What is still needed afterwards (the export contract and the SHA-256 of every loose PNG) is in the `.ppda`. Only after damage during a resumed render can a kept `.corrupt-aNNN-K` file remain. Older result folders (loose images and `render.json`/`complete.json` directly in the folder) are still read.
145Loose per-view PNGs are **not** kept in the result. During an in-progress run they live under `checkpoint/raw files/…` as Resume checkpoints (journal keys stay `raw files/…` for `.ppda` identity); atlases are built from in-memory PNG bytes when available. After the atlases/`.ppda` are verified the checkpoint tree is deleted, then bookkeeping (`render.json`, `complete.json`, `progress.jsonl`, `resume.jsonl`, `run.lock`, attempt markers) is stripped and the folder is put in place. The `.ppda` still embeds the export contract and the SHA-256 of every logical cell. Quarantined `.corrupt-aNNN-K` files (and leftover `.part-*`) are rescued to the result root so they survive checkpoint cleanup. Older result folders that still have `raw files/` or `render.json`/`complete.json` are still read.
147
146
148While building: `render.json` (schema 6; with `depth_near`, `depth_far`, `depth_bits`, `depth_encoding`) describes project identity, source fingerprints (SHA-256 of the body file and optionally the encoded equipment), camera/pivot/sample/material contract (pitches −60…+60, yaws 0…330), clip inventory and count. In the temporary folder only `complete.json` (same schema, with per-PNG SHA-256 and the names and SHA-256 of atlases and `.ppda`) marks a complete set. Schema 5 (before `depth_bits`) is still read as 16-bit gray. Schema 4 and older (including the old global 0.05–40 m range) are refused by the readers and not mixed; render those again. `cancelled.json`/`failed.json` mark a partial run without a complete marker. The library module `export_set` reads a finished result folder through the `.ppda` (older folders through their `render.json`/`complete.json`), opens only finished runs with matching identity and verified checksums, and reads the real RGBA8 and depth maps (all six formats, converted to 16-bit depth for the compositor) with strict path/format validation. The map/camera/pixel contracts are in `design.md`.
147While building: `render.json` (schema 6; with `depth_near`, `depth_far`, `depth_bits`, `depth_encoding`) describes project identity, source fingerprints (SHA-256 of the body file and optionally the encoded equipment), camera/pivot/sample/material contract (pitches −60…+60, yaws 0…330), clip inventory and count. In the temporary folder only `complete.json` (same schema, with per-PNG SHA-256 and the names and SHA-256 of atlases and `.ppda`) marks a complete set. Schema 5 (before `depth_bits`) is still read as 16-bit gray. Schema 4 and older (including the old global 0.05–40 m range) are refused by the readers and not mixed; render those again. `cancelled.json`/`failed.json` mark a partial run without a complete marker. The library module `export_set` reads a finished result folder through the `.ppda` (older folders through their `render.json`/`complete.json`), opens only finished runs with matching identity and verified checksums, and reads the real RGBA8 and depth maps (all six formats, converted to 16-bit depth for the compositor) with strict path/format validation. The map/camera/pixel contracts are in `design.md`.
149
148
@@ -162,10 +161,10 @@ When all loose maps are done and verified, also after resuming, the render write
162
161
163A resumed run continues writing in the same temporary `inprogress-*` folder; the result folder is untouched until the run is done. If you start a new (not resumed) render of the same sex and set type, older unfinished `inprogress-*` folders of that target are deleted first, so only the newest remains; the finished result folder, other targets and a folder currently being rendered into are kept.
162A resumed run continues writing in the same temporary `inprogress-*` folder; the result folder is untouched until the run is done. If you start a new (not resumed) render of the same sex and set type, older unfinished `inprogress-*` folders of that target are deleted first, so only the newest remains; the finished result folder, other targets and a folder currently being rendered into are kept.
164
163
165- Every run keeps `progress.jsonl`: per published PNG the path and SHA-256. `run.lock` prevents two renders writing into the same run at once.
164- Every run keeps `progress.jsonl`: per published cell the logical path (`raw files/…`) and SHA-256. Cell bytes are stored under `checkpoint/` until the atlases are verified. `run.lock` prevents two renders writing into the same run at once.
166- Resuming is only possible if `render.json` exactly matches what the current job would write: project, sex/set, SHA-256 of body and equipment, target, fps, framing/pivot, camera, clips and maps. A finished, locked or different run is refused.
165- Resuming is only possible if `render.json` exactly matches what the current job would write: project, sex/set, SHA-256 of body and equipment, target, fps, framing/pivot, camera, clips and maps. A finished, locked or different run is refused.
167- Before rendering, every planned PNG is checked. Files with a matching checksum stay unchanged and are not rendered again. Missing files are recreated. Damaged files are renamed to `<name>.corrupt-aNNN-K` and then recreated; they are not deleted. An existing file without a journal line (crash between writing and recording) stays if the re-rendered bytes are equal. Leftover `.part-*` files are not touched.
166- Before rendering, every planned cell is checked under `checkpoint/` (or a legacy root-level `raw files/`). Matching checksums stay unchanged and are not rendered again. Missing cells are recreated. Damaged files are quarantined as `<name>.corrupt-aNNN-K` at the run root and then recreated; they are not deleted. An existing file without a journal line (crash between writing and recording) stays if the re-rendered bytes are equal. Leftover `.part-*` files are rescued to the run root when checkpoints are cleared.
168- After the loose maps, the atlases and `.ppda` are written (see above).
167- After the cell checkpoints, the atlases and `.ppda` are written from in-memory bytes when present (see above); then checkpoints are removed.
169- Progress starts at the work that is already correct; skipped images are labelled *already present, checksums match*. You can cancel again and resume later. A previous `cancelled.json`/`failed.json` is kept as `attempt-NNN-…json`, and `resume.jsonl` records per attempt what was verified, missing and quarantined. Only when everything matches is `complete.json` with all checksums written. Then these bookkeeping files are cleaned up (`complete.json` last) and the folder is put in place. A crash between cleanup and placement is repaired at the next render: the finished folder is cleaned and placed.
168- Progress starts at the work that is already correct; skipped images are labelled *already present, checksums match*. You can cancel again and resume later. A previous `cancelled.json`/`failed.json` is kept as `attempt-NNN-…json`, and `resume.jsonl` records per attempt what was verified, missing and quarantined. Only when everything matches is `complete.json` with all checksums written. Then these bookkeeping files are cleaned up (`complete.json` last) and the folder is put in place. A crash between cleanup and placement is repaired at the next render: the finished folder is cleaned and placed.
170
169
171![Resuming an interrupted run in the body view](docs/screenshots/body-resume.png)
170![Resuming an interrupted run in the body view](docs/screenshots/body-resume.png)
examples/preview_depth_probe.rs +1 −0
@@ -24,6 +24,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
24 24 .and_then(|gl| unsafe {
25 25 paperdoll_renderer::preview_gl::depth_regression_probe(gl)
26 26 .and_then(|_| paperdoll_renderer::production::map_regression_probe(gl))
27 .and_then(|_| paperdoll_renderer::production::pbo_sync_parity_probe(gl))
27 28 });
28 29 *result.lock().unwrap() = Some(test);
29 30 Ok(Box::new(Quit))
@@ -24,6 +24,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
24 .and_then(|gl| unsafe {
24 .and_then(|gl| unsafe {
25 paperdoll_renderer::preview_gl::depth_regression_probe(gl)
25 paperdoll_renderer::preview_gl::depth_regression_probe(gl)
26 .and_then(|_| paperdoll_renderer::production::map_regression_probe(gl))
26 .and_then(|_| paperdoll_renderer::production::map_regression_probe(gl))
27 .and_then(|_| paperdoll_renderer::production::pbo_sync_parity_probe(gl))
27 });
28 });
28 *result.lock().unwrap() = Some(test);
29 *result.lock().unwrap() = Some(test);
29 Ok(Box::new(Quit))
30 Ok(Box::new(Quit))
packaging/arch/PKGBUILD +1 −1
@@ -1,7 +1,7 @@
1 1 # Packages the foundation shell from the git checkout that invoked makepkg.
2 2 # PAPERDOLL_PKGVER and REPO_ROOT are set by scripts/package/arch.sh.
3 3 pkgname=paperdoll-renderer
4 pkgver=0.1.5
4 pkgver=0.1.6
5 5 pkgrel=1
6 6 pkgdesc='Paperdoll Renderer with project files, model import and a live glTF preview.'
7 7 arch=('x86_64')
@@ -1,7 +1,7 @@
1# Packages the foundation shell from the git checkout that invoked makepkg.
1# Packages the foundation shell from the git checkout that invoked makepkg.
2# PAPERDOLL_PKGVER and REPO_ROOT are set by scripts/package/arch.sh.
2# PAPERDOLL_PKGVER and REPO_ROOT are set by scripts/package/arch.sh.
3pkgname=paperdoll-renderer
3pkgname=paperdoll-renderer
4pkgver=0.1.5
4pkgver=0.1.6
5pkgrel=1
5pkgrel=1
6pkgdesc='Paperdoll Renderer with project files, model import and a live glTF preview.'
6pkgdesc='Paperdoll Renderer with project files, model import and a live glTF preview.'
7arch=('x86_64')
7arch=('x86_64')
packaging/aur/paperdoll-renderer-bin/PKGBUILD +2 −2
@@ -1,9 +1,9 @@
1 1 # Maintainer: Tim Falken <tim@timfalken.com>
2 2 # CI (scripts/ci/publish-aur.sh) sets pkgver, _tag and sha256sums.
3 3 pkgname=paperdoll-renderer-bin
4 pkgver=0.1.5
4 pkgver=0.1.6
5 5 pkgrel=1
6 _tag=v0.1.5
6 _tag=v0.1.6
7 7 pkgdesc='Paperdoll Renderer: sprite/paperdoll renders from glTF/FBX bodies with a live preview (prebuilt binary)'
8 8 arch=('x86_64')
9 9 url='https://git.timfalken.com/golden-age/paperdoll-renderer'
@@ -1,9 +1,9 @@
1# Maintainer: Tim Falken <tim@timfalken.com>
1# Maintainer: Tim Falken <tim@timfalken.com>
2# CI (scripts/ci/publish-aur.sh) sets pkgver, _tag and sha256sums.
2# CI (scripts/ci/publish-aur.sh) sets pkgver, _tag and sha256sums.
3pkgname=paperdoll-renderer-bin
3pkgname=paperdoll-renderer-bin
4pkgver=0.1.5
4pkgver=0.1.6
5pkgrel=1
5pkgrel=1
6_tag=v0.1.5
6_tag=v0.1.6
7pkgdesc='Paperdoll Renderer: sprite/paperdoll renders from glTF/FBX bodies with a live preview (prebuilt binary)'
7pkgdesc='Paperdoll Renderer: sprite/paperdoll renders from glTF/FBX bodies with a live preview (prebuilt binary)'
8arch=('x86_64')
8arch=('x86_64')
9url='https://git.timfalken.com/golden-age/paperdoll-renderer'
9url='https://git.timfalken.com/golden-age/paperdoll-renderer'
src/atlas.rs +25 −11
@@ -385,7 +385,9 @@ fn load_cell(
385 385 }
386 386 bytes
387 387 } else {
388 let path = job.root.join(&rel);
388 // Resume / cold path: cells live under checkpoint/ during in-progress runs.
389 let path = crate::resume::find_cell_file(job.root, &rel)
390 .ok_or_else(|| format!("atlas: {rel} ontbreekt op schijf (checkpoint/legacy)"))?;
389 391 export_set::verify_regular_file(&path, job.root).map_err(|e| e.to_string())?;
390 392 let bytes =
391 393 export_set::read_bounded_file(&path, MAX_PNG_BYTES).map_err(|e| e.to_string())?;
@@ -1311,7 +1313,7 @@ pub fn cross_check_run(run_dir: &Path) -> Result<serde_json::Value, String> {
1311 1313 .map_err(|e| e.to_string())?;
1312 1314 if a != b {
1313 1315 return Err(format!(
1314 "atlascel {} f{} p{} y{} {} wijkt af van losse PNG",
1316 "atlascel {} f{} p{} y{} {} wijkt af van export_set-cel",
1315 1317 clip.token,
1316 1318 frame.index,
1317 1319 view.pitch,
@@ -1326,7 +1328,7 @@ pub fn cross_check_run(run_dir: &Path) -> Result<serde_json::Value, String> {
1326 1328 }
1327 1329 if cells != u64::from(run.complete.png_files) {
1328 1330 return Err(format!(
1329 "{cells} atlascellen ≠ {} losse PNG's",
1331 "{cells} atlascellen ≠ {} png_sha256-items",
1330 1332 run.complete.png_files
1331 1333 ));
1332 1334 }
@@ -1337,7 +1339,7 @@ pub fn cross_check_run(run_dir: &Path) -> Result<serde_json::Value, String> {
1337 1339 "rows": set.layout.rows,
1338 1340 "columns": set.layout.columns,
1339 1341 "fps": set.ppda.fps,
1340 "cells_equal_to_loose_pngs": cells,
1342 "cells_equal_to_png_sha256_count": cells,
1341 1343 "maps": set.ppda.maps.iter().map(|(k, m)| (k.clone(), serde_json::json!({"file": m.file, "sha256": m.sha256}))).collect::<serde_json::Map<_, _>>(),
1342 1344 }))
1343 1345 }
@@ -1382,6 +1384,22 @@ mod tests {
1382 1384 tmp
1383 1385 }
1384 1386
1387 fn fixture_with_cells() -> Temp {
1388 let tmp = Temp::new();
1389 let mut spec = SyntheticRun::body("atlas-test", "female", b"body");
1390 spec.clips = vec![("walk".into(), 3, 0.75), ("idle".into(), 1, 0.0)];
1391 spec.target = [3, 2];
1392 spec.atlas_only = false;
1393 spec.write(&tmp.0, |c, f, p, y| Pixel {
1394 diffuse: [c as u8 * 100 + f as u8, (p + 60) as u8, (y / 30) as u8, 255],
1395 normal: [128, 128, 255, 255],
1396 specular: [9, 9, 9, 255],
1397 depth: 1000 + f as u16,
1398 })
1399 .unwrap();
1400 tmp
1401 }
1402
1385 1403 fn rewrite_ppda(dir: &Path, edit: impl FnOnce(&mut serde_json::Value)) {
1386 1404 let path = dir.join("female-body-atlas.ppda");
1387 1405 let mut doc: serde_json::Value = serde_json::from_slice(&fs::read(&path).unwrap()).unwrap();
@@ -1697,14 +1715,10 @@ mod tests {
1697 1715 "female-body-diffuse-atlas.png",
1698 1716 "female-body-normal-atlas.png",
1699 1717 "female-body-specular-atlas.png",
1700 "raw files",
1701 1718 ]
1702 1719 );
1703 let raw: Vec<String> = fs::read_dir(tmp.0.join("raw files"))
1704 .unwrap()
1705 .map(|e| e.unwrap().file_name().to_string_lossy().to_string())
1706 .collect();
1707 assert_eq!(raw, ["female"]);
1720 assert!(!tmp.0.join("raw files").exists());
1721 assert!(!tmp.0.join("checkpoint").exists());
1708 1722 let (near, far) = crate::camera::depth_range(p.identity.framing);
1709 1723 assert_eq!((p.depth_near, p.depth_far), (near, far));
1710 1724 assert_eq!((p.identity.near, p.identity.far), (near, far));
@@ -1892,7 +1906,7 @@ mod tests {
1892 1906
1893 1907 #[test]
1894 1908 fn atlases_never_clobber_and_are_refused_when_too_large() {
1895 let tmp = fixture();
1909 let tmp = fixture_with_cells();
1896 1910 let export = AtlasSet::open_run(&tmp.0).unwrap().ppda.export.unwrap();
1897 1911 let render = export.render.clone();
1898 1912 let sums = export.png_sha256.clone();
@@ -385,7 +385,9 @@ fn load_cell(
385 }
385 }
386 bytes
386 bytes
387 } else {
387 } else {
388 let path = job.root.join(&rel);
388 // Resume / cold path: cells live under checkpoint/ during in-progress runs.
389 let path = crate::resume::find_cell_file(job.root, &rel)
390 .ok_or_else(|| format!("atlas: {rel} ontbreekt op schijf (checkpoint/legacy)"))?;
389 export_set::verify_regular_file(&path, job.root).map_err(|e| e.to_string())?;
391 export_set::verify_regular_file(&path, job.root).map_err(|e| e.to_string())?;
390 let bytes =
392 let bytes =
391 export_set::read_bounded_file(&path, MAX_PNG_BYTES).map_err(|e| e.to_string())?;
393 export_set::read_bounded_file(&path, MAX_PNG_BYTES).map_err(|e| e.to_string())?;
@@ -1311,7 +1313,7 @@ pub fn cross_check_run(run_dir: &Path) -> Result<serde_json::Value, String> {
1311 .map_err(|e| e.to_string())?;
1313 .map_err(|e| e.to_string())?;
1312 if a != b {
1314 if a != b {
1313 return Err(format!(
1315 return Err(format!(
1314 "atlascel {} f{} p{} y{} {} wijkt af van losse PNG",
1316 "atlascel {} f{} p{} y{} {} wijkt af van export_set-cel",
1315 clip.token,
1317 clip.token,
1316 frame.index,
1318 frame.index,
1317 view.pitch,
1319 view.pitch,
@@ -1326,7 +1328,7 @@ pub fn cross_check_run(run_dir: &Path) -> Result<serde_json::Value, String> {
1326 }
1328 }
1327 if cells != u64::from(run.complete.png_files) {
1329 if cells != u64::from(run.complete.png_files) {
1328 return Err(format!(
1330 return Err(format!(
1329 "{cells} atlascellen ≠ {} losse PNG's",
1331 "{cells} atlascellen ≠ {} png_sha256-items",
1330 run.complete.png_files
1332 run.complete.png_files
1331 ));
1333 ));
1332 }
1334 }
@@ -1337,7 +1339,7 @@ pub fn cross_check_run(run_dir: &Path) -> Result<serde_json::Value, String> {
1337 "rows": set.layout.rows,
1339 "rows": set.layout.rows,
1338 "columns": set.layout.columns,
1340 "columns": set.layout.columns,
1339 "fps": set.ppda.fps,
1341 "fps": set.ppda.fps,
1340 "cells_equal_to_loose_pngs": cells,
1342 "cells_equal_to_png_sha256_count": cells,
1341 "maps": set.ppda.maps.iter().map(|(k, m)| (k.clone(), serde_json::json!({"file": m.file, "sha256": m.sha256}))).collect::<serde_json::Map<_, _>>(),
1343 "maps": set.ppda.maps.iter().map(|(k, m)| (k.clone(), serde_json::json!({"file": m.file, "sha256": m.sha256}))).collect::<serde_json::Map<_, _>>(),
1342 }))
1344 }))
1343}
1345}
@@ -1382,6 +1384,22 @@ mod tests {
1382 tmp
1384 tmp
1383 }
1385 }
1384
1386
1387 fn fixture_with_cells() -> Temp {
1388 let tmp = Temp::new();
1389 let mut spec = SyntheticRun::body("atlas-test", "female", b"body");
1390 spec.clips = vec![("walk".into(), 3, 0.75), ("idle".into(), 1, 0.0)];
1391 spec.target = [3, 2];
1392 spec.atlas_only = false;
1393 spec.write(&tmp.0, |c, f, p, y| Pixel {
1394 diffuse: [c as u8 * 100 + f as u8, (p + 60) as u8, (y / 30) as u8, 255],
1395 normal: [128, 128, 255, 255],
1396 specular: [9, 9, 9, 255],
1397 depth: 1000 + f as u16,
1398 })
1399 .unwrap();
1400 tmp
1401 }
1402
1385 fn rewrite_ppda(dir: &Path, edit: impl FnOnce(&mut serde_json::Value)) {
1403 fn rewrite_ppda(dir: &Path, edit: impl FnOnce(&mut serde_json::Value)) {
1386 let path = dir.join("female-body-atlas.ppda");
1404 let path = dir.join("female-body-atlas.ppda");
1387 let mut doc: serde_json::Value = serde_json::from_slice(&fs::read(&path).unwrap()).unwrap();
1405 let mut doc: serde_json::Value = serde_json::from_slice(&fs::read(&path).unwrap()).unwrap();
@@ -1697,14 +1715,10 @@ mod tests {
1697 "female-body-diffuse-atlas.png",
1715 "female-body-diffuse-atlas.png",
1698 "female-body-normal-atlas.png",
1716 "female-body-normal-atlas.png",
1699 "female-body-specular-atlas.png",
1717 "female-body-specular-atlas.png",
1700 "raw files",
1701 ]
1718 ]
1702 );
1719 );
1703 let raw: Vec<String> = fs::read_dir(tmp.0.join("raw files"))
1720 assert!(!tmp.0.join("raw files").exists());
1704 .unwrap()
1721 assert!(!tmp.0.join("checkpoint").exists());
1705 .map(|e| e.unwrap().file_name().to_string_lossy().to_string())
1706 .collect();
1707 assert_eq!(raw, ["female"]);
1708 let (near, far) = crate::camera::depth_range(p.identity.framing);
1722 let (near, far) = crate::camera::depth_range(p.identity.framing);
1709 assert_eq!((p.depth_near, p.depth_far), (near, far));
1723 assert_eq!((p.depth_near, p.depth_far), (near, far));
1710 assert_eq!((p.identity.near, p.identity.far), (near, far));
1724 assert_eq!((p.identity.near, p.identity.far), (near, far));
@@ -1892,7 +1906,7 @@ mod tests {
1892
1906
1893 #[test]
1907 #[test]
1894 fn atlases_never_clobber_and_are_refused_when_too_large() {
1908 fn atlases_never_clobber_and_are_refused_when_too_large() {
1895 let tmp = fixture();
1909 let tmp = fixture_with_cells();
1896 let export = AtlasSet::open_run(&tmp.0).unwrap().ppda.export.unwrap();
1910 let export = AtlasSet::open_run(&tmp.0).unwrap().ppda.export.unwrap();
1897 let render = export.render.clone();
1911 let render = export.render.clone();
1898 let sums = export.png_sha256.clone();
1912 let sums = export.png_sha256.clone();
src/dialogs.rs +5 −0
@@ -14,6 +14,8 @@ pub enum DialogPurpose {
14 14 EquipmentModel(usize),
15 15 EquipmentShiny(usize, usize),
16 16 ImportEquipment,
17 /// Folder for Preview2d "Exporteer frame" (Tim t1065u).
18 ExportPreview2dFrame,
17 19 }
18 20
19 21 #[derive(Debug, Clone, PartialEq, Eq)]
@@ -145,6 +147,9 @@ fn native_pick(purpose: &DialogPurpose) -> Option<PathBuf> {
145 147 .set_title("Equipment-preset importeren")
146 148 .add_filter("Paperdoll equipment", &["ppe"])
147 149 .pick_file(),
150 DialogPurpose::ExportPreview2dFrame => {
151 dialog.set_title("Map voor frame-export").pick_folder()
152 }
148 153 }
149 154 }
150 155
@@ -14,6 +14,8 @@ pub enum DialogPurpose {
14 EquipmentModel(usize),
14 EquipmentModel(usize),
15 EquipmentShiny(usize, usize),
15 EquipmentShiny(usize, usize),
16 ImportEquipment,
16 ImportEquipment,
17 /// Folder for Preview2d "Exporteer frame" (Tim t1065u).
18 ExportPreview2dFrame,
17}
19}
18
20
19#[derive(Debug, Clone, PartialEq, Eq)]
21#[derive(Debug, Clone, PartialEq, Eq)]
@@ -145,6 +147,9 @@ fn native_pick(purpose: &DialogPurpose) -> Option<PathBuf> {
145 .set_title("Equipment-preset importeren")
147 .set_title("Equipment-preset importeren")
146 .add_filter("Paperdoll equipment", &["ppe"])
148 .add_filter("Paperdoll equipment", &["ppe"])
147 .pick_file(),
149 .pick_file(),
150 DialogPurpose::ExportPreview2dFrame => {
151 dialog.set_title("Map voor frame-export").pick_folder()
152 }
148 }
153 }
149}
154}
150
155
src/editor.rs +40 −0
@@ -452,6 +452,39 @@ impl EditorApp {
452 452 }
453 453 }
454 454 }
455 DialogPurpose::ExportPreview2dFrame => {
456 self.rebuild_preview2d_frame();
457 match self.preview2d_loaded.as_ref() {
458 None => self.session.fail(
459 self.preview2d
460 .diagnostic
461 .clone()
462 .unwrap_or_else(|| "geen body-exportset voor dit geslacht".into()),
463 ),
464 Some(sel) => {
465 let req = crate::preview2d::ComposeRequest {
466 body: &sel.body,
467 equipment: &sel.gear,
468 clip_token: &self.preview2d.clip_token,
469 frame: self.preview2d.frame_index(),
470 pitch: self.preview2d.pitch_deg(),
471 yaw: self.preview2d.yaw_deg(),
472 light: self.preview2d.light(),
473 view_mode: crate::preview2d::ViewMode::Resultaat,
474 };
475 match crate::preview2d::export_preview_frame(&path, &req) {
476 Ok(files) => {
477 self.session.ok(&format!(
478 "Frame geëxporteerd ({} bestanden) naar {}",
479 files.len(),
480 path.display()
481 ));
482 }
483 Err(e) => self.session.fail(e.to_string()),
484 }
485 }
486 }
487 }
455 488 DialogPurpose::ImportEquipment => match crate::equipment::Preset::read(&path) {
456 489 Ok(mut preset) => {
457 490 let result = self
@@ -2149,6 +2182,7 @@ impl EditorApp {
2149 2182
2150 2183 let mut need_rebuild = false;
2151 2184 let mut refresh_catalog = false;
2185 let mut export_frame = false;
2152 2186 let mut go_back = false;
2153 2187 let mut set_sex = None;
2154 2188 let mut toggle_set = None;
@@ -2192,6 +2226,9 @@ impl EditorApp {
2192 2226 if ui.button(crate::i18n::t("Vernieuw exportsets")).clicked() {
2193 2227 refresh_catalog = true;
2194 2228 }
2229 if ui.button(crate::i18n::t("Exporteer frame")).clicked() {
2230 export_frame = true;
2231 }
2195 2232 ui.label(crate::i18n::t(format!(
2196 2233 "{} voltooide exportset(s) (.ppda + atlassen)",
2197 2234 self.preview2d_catalog.len()
@@ -2369,6 +2406,9 @@ impl EditorApp {
2369 2406 sync_state_from_catalog(&mut self.preview2d, &self.preview2d_catalog);
2370 2407 need_rebuild = true;
2371 2408 }
2409 if export_frame {
2410 self.dialogs.start(DialogPurpose::ExportPreview2dFrame);
2411 }
2372 2412 if let Some(sex) = set_sex {
2373 2413 self.preview2d.set_sex(sex);
2374 2414 sync_state_from_catalog(&mut self.preview2d, &self.preview2d_catalog);
@@ -452,6 +452,39 @@ impl EditorApp {
452 }
452 }
453 }
453 }
454 }
454 }
455 DialogPurpose::ExportPreview2dFrame => {
456 self.rebuild_preview2d_frame();
457 match self.preview2d_loaded.as_ref() {
458 None => self.session.fail(
459 self.preview2d
460 .diagnostic
461 .clone()
462 .unwrap_or_else(|| "geen body-exportset voor dit geslacht".into()),
463 ),
464 Some(sel) => {
465 let req = crate::preview2d::ComposeRequest {
466 body: &sel.body,
467 equipment: &sel.gear,
468 clip_token: &self.preview2d.clip_token,
469 frame: self.preview2d.frame_index(),
470 pitch: self.preview2d.pitch_deg(),
471 yaw: self.preview2d.yaw_deg(),
472 light: self.preview2d.light(),
473 view_mode: crate::preview2d::ViewMode::Resultaat,
474 };
475 match crate::preview2d::export_preview_frame(&path, &req) {
476 Ok(files) => {
477 self.session.ok(&format!(
478 "Frame geëxporteerd ({} bestanden) naar {}",
479 files.len(),
480 path.display()
481 ));
482 }
483 Err(e) => self.session.fail(e.to_string()),
484 }
485 }
486 }
487 }
455 DialogPurpose::ImportEquipment => match crate::equipment::Preset::read(&path) {
488 DialogPurpose::ImportEquipment => match crate::equipment::Preset::read(&path) {
456 Ok(mut preset) => {
489 Ok(mut preset) => {
457 let result = self
490 let result = self
@@ -2149,6 +2182,7 @@ impl EditorApp {
2149
2182
2150 let mut need_rebuild = false;
2183 let mut need_rebuild = false;
2151 let mut refresh_catalog = false;
2184 let mut refresh_catalog = false;
2185 let mut export_frame = false;
2152 let mut go_back = false;
2186 let mut go_back = false;
2153 let mut set_sex = None;
2187 let mut set_sex = None;
2154 let mut toggle_set = None;
2188 let mut toggle_set = None;
@@ -2192,6 +2226,9 @@ impl EditorApp {
2192 if ui.button(crate::i18n::t("Vernieuw exportsets")).clicked() {
2226 if ui.button(crate::i18n::t("Vernieuw exportsets")).clicked() {
2193 refresh_catalog = true;
2227 refresh_catalog = true;
2194 }
2228 }
2229 if ui.button(crate::i18n::t("Exporteer frame")).clicked() {
2230 export_frame = true;
2231 }
2195 ui.label(crate::i18n::t(format!(
2232 ui.label(crate::i18n::t(format!(
2196 "{} voltooide exportset(s) (.ppda + atlassen)",
2233 "{} voltooide exportset(s) (.ppda + atlassen)",
2197 self.preview2d_catalog.len()
2234 self.preview2d_catalog.len()
@@ -2369,6 +2406,9 @@ impl EditorApp {
2369 sync_state_from_catalog(&mut self.preview2d, &self.preview2d_catalog);
2406 sync_state_from_catalog(&mut self.preview2d, &self.preview2d_catalog);
2370 need_rebuild = true;
2407 need_rebuild = true;
2371 }
2408 }
2409 if export_frame {
2410 self.dialogs.start(DialogPurpose::ExportPreview2dFrame);
2411 }
2372 if let Some(sex) = set_sex {
2412 if let Some(sex) = set_sex {
2373 self.preview2d.set_sex(sex);
2413 self.preview2d.set_sex(sex);
2374 sync_state_from_catalog(&mut self.preview2d, &self.preview2d_catalog);
2414 sync_state_from_catalog(&mut self.preview2d, &self.preview2d_catalog);
src/export_fixture.rs +7 −1
@@ -45,6 +45,8 @@ pub struct SyntheticRun {
45 45 /// Optional per-pixel content mask (clip, frame, pitch, yaw, x, y); pixels
46 46 /// outside the mask are all-zero in every map.
47 47 pub mask: Option<PixelMask>,
48 /// When true (default), drop loose/checkpoint cells after atlases (Tim t1065u).
49 pub atlas_only: bool,
48 50 }
49 51
50 52 impl SyntheticRun {
@@ -67,6 +69,7 @@ impl SyntheticRun {
67 69 depth_format: crate::camera::DepthFormat::GRAY16,
68 70 trim: false,
69 71 mask: None,
72 atlas_only: true,
70 73 }
71 74 }
72 75
@@ -246,7 +249,10 @@ impl SyntheticRun {
246 249 "atlas": atlas.to_json(),
247 250 }),
248 251 )?;
249 // Same final layout as production: only .ppda, atlases and `raw files/`.
252 // Same final layout as production: only .ppda + atlases (Tim t1065u).
253 if self.atlas_only {
254 crate::resume::remove_cell_checkpoints(root)?;
255 }
250 256 crate::resume::strip_bookkeeping(root)
251 257 }
252 258 }
@@ -45,6 +45,8 @@ pub struct SyntheticRun {
45 /// Optional per-pixel content mask (clip, frame, pitch, yaw, x, y); pixels
45 /// Optional per-pixel content mask (clip, frame, pitch, yaw, x, y); pixels
46 /// outside the mask are all-zero in every map.
46 /// outside the mask are all-zero in every map.
47 pub mask: Option<PixelMask>,
47 pub mask: Option<PixelMask>,
48 /// When true (default), drop loose/checkpoint cells after atlases (Tim t1065u).
49 pub atlas_only: bool,
48}
50}
49
51
50impl SyntheticRun {
52impl SyntheticRun {
@@ -67,6 +69,7 @@ impl SyntheticRun {
67 depth_format: crate::camera::DepthFormat::GRAY16,
69 depth_format: crate::camera::DepthFormat::GRAY16,
68 trim: false,
70 trim: false,
69 mask: None,
71 mask: None,
72 atlas_only: true,
70 }
73 }
71 }
74 }
72
75
@@ -246,7 +249,10 @@ impl SyntheticRun {
246 "atlas": atlas.to_json(),
249 "atlas": atlas.to_json(),
247 }),
250 }),
248 )?;
251 )?;
249 // Same final layout as production: only .ppda, atlases and `raw files/`.
252 // Same final layout as production: only .ppda + atlases (Tim t1065u).
253 if self.atlas_only {
254 crate::resume::remove_cell_checkpoints(root)?;
255 }
250 crate::resume::strip_bookkeeping(root)
256 crate::resume::strip_bookkeeping(root)
251 }
257 }
252}
258}
src/export_set.rs +53 −23
@@ -237,6 +237,9 @@ pub struct ExportRun {
237 237 root: PathBuf,
238 238 pub render: RenderManifest,
239 239 pub complete: CompleteManifest,
240 /// Present when the finished folder has no loose/checkpoint PNGs: maps are
241 /// served from atlases (Tim t1065u).
242 atlas: Option<crate::atlas::LoadedAtlas>,
240 243 }
241 244
242 245 impl ExportRun {
@@ -288,28 +291,46 @@ impl ExportRun {
288 291 "png_files in complete.json wijkt af van render.json".into(),
289 292 ));
290 293 }
291 // Verify every checksum exists as a bounded regular file under the run.
292 for (relative, digest) in &complete.png_sha256 {
293 let path = resolve_inside(&root, relative)?;
294 verify_regular_file(&path, &root)?;
295 let meta = fs::metadata(&path).map_err(|e| ExportError::Io(e.to_string()))?;
296 if meta.len() > MAX_PNG_BYTES {
297 return Err(ExportError::Resource(format!(
298 "PNG '{relative}' groter dan {MAX_PNG_BYTES} bytes"
299 )));
300 }
301 let bytes = fs::read(&path).map_err(|e| ExportError::Io(e.to_string()))?;
302 let actual = digest::sha256_hex(&bytes);
303 if actual != *digest {
304 return Err(ExportError::Corrupt(format!(
305 "checksum mismatch voor '{relative}'"
306 )));
294 // Prefer loose/checkpoint PNG verification when present (in-progress or
295 // legacy result folders). Finished atlas-only folders serve maps from
296 // the atlases; AtlasSet::open_run already checked atlas digests.
297 let mut any_cell_file = false;
298 for relative in complete.png_sha256.keys() {
299 if cell_file_path(&root, relative).is_some() {
300 any_cell_file = true;
301 break;
307 302 }
308 303 }
304 let atlas = if any_cell_file {
305 for (relative, digest) in &complete.png_sha256 {
306 let path = cell_file_path(&root, relative).ok_or_else(|| {
307 ExportError::Incomplete(format!("PNG '{relative}' ontbreekt"))
308 })?;
309 verify_regular_file(&path, &root)?;
310 let meta = fs::metadata(&path).map_err(|e| ExportError::Io(e.to_string()))?;
311 if meta.len() > MAX_PNG_BYTES {
312 return Err(ExportError::Resource(format!(
313 "PNG '{relative}' groter dan {MAX_PNG_BYTES} bytes"
314 )));
315 }
316 let bytes = fs::read(&path).map_err(|e| ExportError::Io(e.to_string()))?;
317 let actual = digest::sha256_hex(&bytes);
318 if actual != *digest {
319 return Err(ExportError::Corrupt(format!(
320 "checksum mismatch voor '{relative}'"
321 )));
322 }
323 }
324 None
325 } else {
326 let set = crate::atlas::AtlasSet::open_run(&root)?;
327 Some(set.load()?)
328 };
309 329 Ok(Self {
310 330 root,
311 331 render,
312 332 complete,
333 atlas,
313 334 })
314 335 }
315 336
@@ -361,7 +382,16 @@ impl ExportRun {
361 382 kind: MapKind,
362 383 ) -> Result<MapImage, ExportError> {
363 384 let key = self.map_path(clip_token, frame, pitch, yaw, kind)?;
364 self.load_relative(&key, kind)
385 match self.load_relative(&key, kind) {
386 Ok(img) => Ok(img),
387 Err(ExportError::Incomplete(_)) | Err(ExportError::Io(_)) if self.atlas.is_some() => {
388 self.atlas
389 .as_ref()
390 .unwrap()
391 .cell(clip_token, frame, pitch, yaw, kind)
392 }
393 Err(e) => Err(e),
394 }
365 395 }
366 396
367 397 pub fn load_relative(
@@ -374,7 +404,8 @@ impl ExportRun {
374 404 .png_sha256
375 405 .get(relative)
376 406 .ok_or_else(|| ExportError::Incomplete(format!("onbekende map '{relative}'")))?;
377 let path = resolve_inside(&self.root, relative)?;
407 let path = cell_file_path(&self.root, relative)
408 .ok_or_else(|| ExportError::Incomplete(format!("PNG '{relative}' ontbreekt")))?;
378 409 verify_regular_file(&path, &self.root)?;
379 410 let bytes = read_bounded_file(&path, MAX_PNG_BYTES)?;
380 411 let actual = digest::sha256_hex(&bytes);
@@ -990,11 +1021,10 @@ pub(crate) fn read_bounded_file(path: &Path, max: u64) -> Result<Vec<u8>, Export
990 1021 Ok(bytes)
991 1022 }
992 1023
993 fn resolve_inside(root: &Path, relative: &str) -> Result<PathBuf, ExportError> {
994 let clean = project::portable_relative(relative).map_err(ExportError::Traversal)?;
995 let path = root.join(&clean);
996 // Lexical rejection of oddities already handled; still verify canonical containment.
997 Ok(path)
1024
1025 /// Locate a cell PNG: logical path, then `checkpoint/<logical>` (Tim t1065u).
1026 fn cell_file_path(root: &Path, relative: &str) -> Option<PathBuf> {
1027 crate::resume::find_cell_file(root, relative)
998 1028 }
999 1029
1000 1030 pub(crate) fn verify_regular_file(path: &Path, root: &Path) -> Result<(), ExportError> {
@@ -237,6 +237,9 @@ pub struct ExportRun {
237 root: PathBuf,
237 root: PathBuf,
238 pub render: RenderManifest,
238 pub render: RenderManifest,
239 pub complete: CompleteManifest,
239 pub complete: CompleteManifest,
240 /// Present when the finished folder has no loose/checkpoint PNGs: maps are
241 /// served from atlases (Tim t1065u).
242 atlas: Option<crate::atlas::LoadedAtlas>,
240}
243}
241
244
242impl ExportRun {
245impl ExportRun {
@@ -288,28 +291,46 @@ impl ExportRun {
288 "png_files in complete.json wijkt af van render.json".into(),
291 "png_files in complete.json wijkt af van render.json".into(),
289 ));
292 ));
290 }
293 }
291 // Verify every checksum exists as a bounded regular file under the run.
294 // Prefer loose/checkpoint PNG verification when present (in-progress or
292 for (relative, digest) in &complete.png_sha256 {
295 // legacy result folders). Finished atlas-only folders serve maps from
293 let path = resolve_inside(&root, relative)?;
296 // the atlases; AtlasSet::open_run already checked atlas digests.
294 verify_regular_file(&path, &root)?;
297 let mut any_cell_file = false;
295 let meta = fs::metadata(&path).map_err(|e| ExportError::Io(e.to_string()))?;
298 for relative in complete.png_sha256.keys() {
296 if meta.len() > MAX_PNG_BYTES {
299 if cell_file_path(&root, relative).is_some() {
297 return Err(ExportError::Resource(format!(
300 any_cell_file = true;
298 "PNG '{relative}' groter dan {MAX_PNG_BYTES} bytes"
301 break;
299 )));
300 }
301 let bytes = fs::read(&path).map_err(|e| ExportError::Io(e.to_string()))?;
302 let actual = digest::sha256_hex(&bytes);
303 if actual != *digest {
304 return Err(ExportError::Corrupt(format!(
305 "checksum mismatch voor '{relative}'"
306 )));
307 }
302 }
308 }
303 }
304 let atlas = if any_cell_file {
305 for (relative, digest) in &complete.png_sha256 {
306 let path = cell_file_path(&root, relative).ok_or_else(|| {
307 ExportError::Incomplete(format!("PNG '{relative}' ontbreekt"))
308 })?;
309 verify_regular_file(&path, &root)?;
310 let meta = fs::metadata(&path).map_err(|e| ExportError::Io(e.to_string()))?;
311 if meta.len() > MAX_PNG_BYTES {
312 return Err(ExportError::Resource(format!(
313 "PNG '{relative}' groter dan {MAX_PNG_BYTES} bytes"
314 )));
315 }
316 let bytes = fs::read(&path).map_err(|e| ExportError::Io(e.to_string()))?;
317 let actual = digest::sha256_hex(&bytes);
318 if actual != *digest {
319 return Err(ExportError::Corrupt(format!(
320 "checksum mismatch voor '{relative}'"
321 )));
322 }
323 }
324 None
325 } else {
326 let set = crate::atlas::AtlasSet::open_run(&root)?;
327 Some(set.load()?)
328 };
309 Ok(Self {
329 Ok(Self {
310 root,
330 root,
311 render,
331 render,
312 complete,
332 complete,
333 atlas,
313 })
334 })
314 }
335 }
315
336
@@ -361,7 +382,16 @@ impl ExportRun {
361 kind: MapKind,
382 kind: MapKind,
362 ) -> Result<MapImage, ExportError> {
383 ) -> Result<MapImage, ExportError> {
363 let key = self.map_path(clip_token, frame, pitch, yaw, kind)?;
384 let key = self.map_path(clip_token, frame, pitch, yaw, kind)?;
364 self.load_relative(&key, kind)
385 match self.load_relative(&key, kind) {
386 Ok(img) => Ok(img),
387 Err(ExportError::Incomplete(_)) | Err(ExportError::Io(_)) if self.atlas.is_some() => {
388 self.atlas
389 .as_ref()
390 .unwrap()
391 .cell(clip_token, frame, pitch, yaw, kind)
392 }
393 Err(e) => Err(e),
394 }
365 }
395 }
366
396
367 pub fn load_relative(
397 pub fn load_relative(
@@ -374,7 +404,8 @@ impl ExportRun {
374 .png_sha256
404 .png_sha256
375 .get(relative)
405 .get(relative)
376 .ok_or_else(|| ExportError::Incomplete(format!("onbekende map '{relative}'")))?;
406 .ok_or_else(|| ExportError::Incomplete(format!("onbekende map '{relative}'")))?;
377 let path = resolve_inside(&self.root, relative)?;
407 let path = cell_file_path(&self.root, relative)
408 .ok_or_else(|| ExportError::Incomplete(format!("PNG '{relative}' ontbreekt")))?;
378 verify_regular_file(&path, &self.root)?;
409 verify_regular_file(&path, &self.root)?;
379 let bytes = read_bounded_file(&path, MAX_PNG_BYTES)?;
410 let bytes = read_bounded_file(&path, MAX_PNG_BYTES)?;
380 let actual = digest::sha256_hex(&bytes);
411 let actual = digest::sha256_hex(&bytes);
@@ -990,11 +1021,10 @@ pub(crate) fn read_bounded_file(path: &Path, max: u64) -> Result<Vec<u8>, Export
990 Ok(bytes)
1021 Ok(bytes)
991}
1022}
992
1023
993fn resolve_inside(root: &Path, relative: &str) -> Result<PathBuf, ExportError> {
1024
994 let clean = project::portable_relative(relative).map_err(ExportError::Traversal)?;
1025/// Locate a cell PNG: logical path, then `checkpoint/<logical>` (Tim t1065u).
995 let path = root.join(&clean);
1026fn cell_file_path(root: &Path, relative: &str) -> Option<PathBuf> {
996 // Lexical rejection of oddities already handled; still verify canonical containment.
1027 crate::resume::find_cell_file(root, relative)
997 Ok(path)
998}
1028}
999
1029
1000pub(crate) fn verify_regular_file(path: &Path, root: &Path) -> Result<(), ExportError> {
1030pub(crate) fn verify_regular_file(path: &Path, root: &Path) -> Result<(), ExportError> {
src/i18n_table.rs +15 −1
@@ -695,7 +695,21 @@ pub const TABLE: &[(&str, &str)] = &[
695 695 ("diffuse is lit or alpha/vertex/texture lost under {light:?}: {pixel:?} vs {expected:?}", "diffuse is lit or alpha/vertex/texture lost under {light:?}: {pixel:?} vs {expected:?}"),
696 696 ("target map dimensions/normals invalid", "target map dimensions/normals invalid"),
697 697 ("flat normals are not geometry-derived: {:?}", "flat normals are not geometry-derived: {:?}"),
698 ("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo texture×vertex={expected:?}, alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives", "Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo texture×vertex={expected:?}, alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives"),
698 ("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo sRGB texture×vertex={expected:?} (guards SRGB8_ALPHA8 upload, Tim t1066u), alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives", "Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo sRGB texture×vertex={expected:?} (guards SRGB8_ALPHA8 upload, Tim t1066u), alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives"),
699 ("Production PBO/sync parity PASS: diffuse/normal/specular/depth byte-equal; textured diffuse={expected:?}", "Production PBO/sync parity PASS: diffuse/normal/specular/depth byte-equal; textured diffuse={expected:?}"),
700 ("PNG '{relative}' ontbreekt", "PNG '{relative}' is missing"),
701 ("atlas: {rel} ontbreekt op schijf (checkpoint/legacy)", "atlas: {rel} missing on disk (checkpoint/legacy)"),
702 ("atlascel {} f{} p{} y{} {} wijkt af van export_set-cel", "atlas cell {} f{} p{} y{} {} differs from export_set cell"),
703 ("{cells} atlascellen ≠ {} png_sha256-items", "{cells} atlas cells ≠ {} png_sha256 items"),
704 ("checkpoint opschonen ({name}): {e}", "cleaning checkpoint ({name}): {e}"),
705 ("quarantaine bewaren {name}: {e}", "keeping quarantine {name}: {e}"),
706 ("PBO≠sync op {name}: sync {:?} vs pbo {:?}", "PBO≠sync on {name}: sync {:?} vs pbo {:?}"),
707 ("PBO≠sync op depth: sync {} vs pbo {}", "PBO≠sync on depth: sync {} vs pbo {}"),
708 ("diffuse sRGB albedo pad fout (sync): {pixel:?} vs {expected:?}", "diffuse sRGB albedo path wrong (sync): {pixel:?} vs {expected:?}"),
709 ("diffuse sRGB albedo pad fout (PBO): {:?} vs {expected:?}", "diffuse sRGB albedo path wrong (PBO): {:?} vs {expected:?}"),
710 ("Exporteer frame", "Export frame"),
711 ("Map voor frame-export", "Folder for frame export"),
712 ("Frame geëxporteerd ({} bestanden) naar {}", "Frame exported ({} files) to {}"),
699 713 ("renderkader: hoogte 0.1–10 m, centrum/pivot -10–10 m, dieptebereik 0,1–19 m", "render frame: height 0.1–10 m, center/pivot -10–10 m, depth range 0.1–19 m"),
700 714 ("dit project gebruikt format_version {found}; deze build leest alleen versie {supported}", "this project uses format_version {found}; this build reads only version {supported}"),
701 715 ("het projectbestand heeft een veld dat versie {FORMAT_VERSION} niet kent ({detail}). Het bestand is niet herschreven.", "the project file has a field that version {FORMAT_VERSION} does not know ({detail}). The file was not rewritten."),
@@ -695,7 +695,21 @@ pub const TABLE: &[(&str, &str)] = &[
695 ("diffuse is lit or alpha/vertex/texture lost under {light:?}: {pixel:?} vs {expected:?}", "diffuse is lit or alpha/vertex/texture lost under {light:?}: {pixel:?} vs {expected:?}"),
695 ("diffuse is lit or alpha/vertex/texture lost under {light:?}: {pixel:?} vs {expected:?}", "diffuse is lit or alpha/vertex/texture lost under {light:?}: {pixel:?} vs {expected:?}"),
696 ("target map dimensions/normals invalid", "target map dimensions/normals invalid"),
696 ("target map dimensions/normals invalid", "target map dimensions/normals invalid"),
697 ("flat normals are not geometry-derived: {:?}", "flat normals are not geometry-derived: {:?}"),
697 ("flat normals are not geometry-derived: {:?}", "flat normals are not geometry-derived: {:?}"),
698 ("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo texture×vertex={expected:?}, alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives", "Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo texture×vertex={expected:?}, alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives"),
698 ("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo sRGB texture×vertex={expected:?} (guards SRGB8_ALPHA8 upload, Tim t1066u), alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives", "Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo sRGB texture×vertex={expected:?} (guards SRGB8_ALPHA8 upload, Tim t1066u), alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives"),
699 ("Production PBO/sync parity PASS: diffuse/normal/specular/depth byte-equal; textured diffuse={expected:?}", "Production PBO/sync parity PASS: diffuse/normal/specular/depth byte-equal; textured diffuse={expected:?}"),
700 ("PNG '{relative}' ontbreekt", "PNG '{relative}' is missing"),
701 ("atlas: {rel} ontbreekt op schijf (checkpoint/legacy)", "atlas: {rel} missing on disk (checkpoint/legacy)"),
702 ("atlascel {} f{} p{} y{} {} wijkt af van export_set-cel", "atlas cell {} f{} p{} y{} {} differs from export_set cell"),
703 ("{cells} atlascellen ≠ {} png_sha256-items", "{cells} atlas cells ≠ {} png_sha256 items"),
704 ("checkpoint opschonen ({name}): {e}", "cleaning checkpoint ({name}): {e}"),
705 ("quarantaine bewaren {name}: {e}", "keeping quarantine {name}: {e}"),
706 ("PBO≠sync op {name}: sync {:?} vs pbo {:?}", "PBO≠sync on {name}: sync {:?} vs pbo {:?}"),
707 ("PBO≠sync op depth: sync {} vs pbo {}", "PBO≠sync on depth: sync {} vs pbo {}"),
708 ("diffuse sRGB albedo pad fout (sync): {pixel:?} vs {expected:?}", "diffuse sRGB albedo path wrong (sync): {pixel:?} vs {expected:?}"),
709 ("diffuse sRGB albedo pad fout (PBO): {:?} vs {expected:?}", "diffuse sRGB albedo path wrong (PBO): {:?} vs {expected:?}"),
710 ("Exporteer frame", "Export frame"),
711 ("Map voor frame-export", "Folder for frame export"),
712 ("Frame geëxporteerd ({} bestanden) naar {}", "Frame exported ({} files) to {}"),
699 ("renderkader: hoogte 0.1–10 m, centrum/pivot -10–10 m, dieptebereik 0,1–19 m", "render frame: height 0.1–10 m, center/pivot -10–10 m, depth range 0.1–19 m"),
713 ("renderkader: hoogte 0.1–10 m, centrum/pivot -10–10 m, dieptebereik 0,1–19 m", "render frame: height 0.1–10 m, center/pivot -10–10 m, depth range 0.1–19 m"),
700 ("dit project gebruikt format_version {found}; deze build leest alleen versie {supported}", "this project uses format_version {found}; this build reads only version {supported}"),
714 ("dit project gebruikt format_version {found}; deze build leest alleen versie {supported}", "this project uses format_version {found}; this build reads only version {supported}"),
701 ("het projectbestand heeft een veld dat versie {FORMAT_VERSION} niet kent ({detail}). Het bestand is niet herschreven.", "the project file has a field that version {FORMAT_VERSION} does not know ({detail}). The file was not rewritten."),
715 ("het projectbestand heeft een veld dat versie {FORMAT_VERSION} niet kent ({detail}). Het bestand is niet herschreven.", "the project file has a field that version {FORMAT_VERSION} does not know ({detail}). The file was not rewritten."),
src/preview2d.rs +136 −1
@@ -735,6 +735,91 @@ pub fn raw_pointer_hold(rect: eframe::egui::Rect, input: &eframe::egui::InputSta
735 735 && p.hover_pos().is_some_and(|h| rect.contains(h))
736 736 }
737 737
738 /// Export the current preview pose: composed map PNGs (depth in the atlas
739 /// encoding) plus the lit preview image as shown under Resultaat (Tim t1065u).
740 pub fn export_preview_frame(
741 out_dir: &Path,
742 req: &ComposeRequest<'_>,
743 ) -> Result<Vec<PathBuf>, Preview2dError> {
744 std::fs::create_dir_all(out_dir)
745 .map_err(|e| Preview2dError::Export(ExportError::Io(e.to_string())))?;
746 let mut atlases: Vec<&LoadedAtlas> = Vec::with_capacity(1 + req.equipment.len());
747 atlases.push(req.body);
748 atlases.extend(req.equipment.iter());
749 let keys: Vec<CompatKey> = atlases
750 .iter()
751 .map(|a| CompatKey::from_ppda(a.ppda()))
752 .collect();
753 assert_compatible_keys(&keys)?;
754 let mut layers = Vec::with_capacity(atlases.len());
755 for atlas in &atlases {
756 let set = atlas.ppda().settype.as_str();
757 match LayerMaps::from_atlas(atlas, set, req.clip_token, req.frame, req.pitch, req.yaw) {
758 Ok(layer) => layers.push(layer),
759 Err(err) => {
760 return Err(Preview2dError::Missing {
761 set: set.to_string(),
762 clip: req.clip_token.to_string(),
763 frame: req.frame,
764 pitch: req.pitch,
765 yaw: req.yaw,
766 detail: err.to_string(),
767 });
768 }
769 }
770 }
771 let (composed, lit) = compose_and_relight(&layers, &req.light)?;
772 let stem = format!(
773 "{}_f{:06}_p{:+03}_y{:03}",
774 req.clip_token, req.frame, req.pitch, req.yaw
775 );
776 let mut written = Vec::new();
777 let write_rgba = |name: &str, rgba: &[u8], srgb: bool| -> Result<PathBuf, Preview2dError> {
778 let img = ImageBuf {
779 width: composed.width,
780 height: composed.height,
781 rgba: rgba.to_vec(),
782 };
783 let bytes = crate::production::encode_rgba_png(&img, srgb)
784 .map_err(|e| Preview2dError::Export(ExportError::Corrupt(e)))?;
785 let path = out_dir.join(name);
786 std::fs::write(&path, bytes)
787 .map_err(|e| Preview2dError::Export(ExportError::Io(e.to_string())))?;
788 Ok(path)
789 };
790 written.push(write_rgba(
791 &format!("{stem}_diffuse.png"),
792 &composed.diffuse,
793 true,
794 )?);
795 written.push(write_rgba(
796 &format!("{stem}_normal.png"),
797 &composed.normal,
798 false,
799 )?);
800 written.push(write_rgba(
801 &format!("{stem}_specular.png"),
802 &composed.specular,
803 false,
804 )?);
805 let fmt = req.body.ppda().depth_format()?;
806 let pixels: Vec<(f64, f64)> = composed
807 .depth
808 .chunks_exact(2)
809 .map(|c| (f64::from(c[0]) / 65535.0, f64::from(c[1]) / 65535.0))
810 .collect();
811 let depth_bytes =
812 crate::production::encode_depth_png_fmt(composed.width, composed.height, &pixels, fmt)
813 .map_err(|e| Preview2dError::Export(ExportError::Corrupt(e)))?;
814 let depth_path = out_dir.join(format!("{stem}_depth.png"));
815 std::fs::write(&depth_path, depth_bytes)
816 .map_err(|e| Preview2dError::Export(ExportError::Io(e.to_string())))?;
817 written.push(depth_path);
818 let preview_path = write_rgba(&format!("{stem}_preview.png"), &lit.rgba, true)?;
819 written.push(preview_path);
820 Ok(written)
821 }
822
738 823 #[cfg(test)]
739 824 mod tests {
740 825 use super::*;
@@ -1210,7 +1295,9 @@ mod tests {
1210 1295 )
1211 1296 })
1212 1297 .unwrap();
1213 fs::remove_dir_all(dir.join("raw files")).unwrap();
1298 // Atlas-only fixtures already omit loose PNGs; tolerate a leftover.
1299 let _ = fs::remove_dir_all(dir.join("raw files"));
1300 let _ = fs::remove_dir_all(dir.join("checkpoint"));
1214 1301 let catalog = discover_completed_runs(&project.0);
1215 1302 assert_eq!(catalog.len(), 1);
1216 1303 assert_eq!(catalog[0].fps, crate::project::DEFAULT_FPS);
@@ -1443,4 +1530,52 @@ mod tests {
1443 1530 assert!(!frame(press, 4.4, &mut state));
1444 1531 assert!(!frame(vec![egui::Event::PointerMoved(at)], 4.5, &mut state));
1445 1532 }
1533
1534 #[test]
1535 fn export_preview_frame_writes_maps_and_lit_preview() {
1536 let project = Temp::new();
1537 let dir = project.0.join("renders/male-body-atlas");
1538 let mut spec = SyntheticRun::body("export-frame", "male", b"body");
1539 spec.clips = vec![("walk".into(), 1, 0.25)];
1540 spec.target = [2, 2];
1541 spec.write(&dir, |_, _, _, _| Pixel {
1542 diffuse: [40, 80, 120, 255],
1543 normal: [128, 128, 255, 255],
1544 specular: [10, 10, 10, 255],
1545 depth: 20000,
1546 })
1547 .unwrap();
1548 let set = AtlasSet::open_run(&dir).unwrap();
1549 let body = set.load().unwrap();
1550 let out = Temp::new();
1551 let written = export_preview_frame(
1552 &out.0,
1553 &ComposeRequest {
1554 body: &body,
1555 equipment: &[],
1556 clip_token: "clip000-walk",
1557 frame: 0,
1558 pitch: 0,
1559 yaw: 0,
1560 light: Light::default(),
1561 view_mode: ViewMode::Resultaat,
1562 },
1563 )
1564 .unwrap();
1565 assert_eq!(written.len(), 5);
1566 for name in [
1567 "clip000-walk_f000000_p+00_y000_diffuse.png",
1568 "clip000-walk_f000000_p+00_y000_normal.png",
1569 "clip000-walk_f000000_p+00_y000_specular.png",
1570 "clip000-walk_f000000_p+00_y000_depth.png",
1571 "clip000-walk_f000000_p+00_y000_preview.png",
1572 ] {
1573 assert!(out.0.join(name).is_file(), "{name}");
1574 }
1575 // Diffuse keeps sRGB chunk; preview is lit (not identical to unlit diffuse).
1576 let diffuse = fs::read(out.0.join("clip000-walk_f000000_p+00_y000_diffuse.png")).unwrap();
1577 let preview = fs::read(out.0.join("clip000-walk_f000000_p+00_y000_preview.png")).unwrap();
1578 assert_ne!(diffuse, preview);
1579 assert!(diffuse.windows(4).any(|w| w == b"sRGB") || diffuse.len() > 100);
1580 }
1446 1581 }
@@ -735,6 +735,91 @@ pub fn raw_pointer_hold(rect: eframe::egui::Rect, input: &eframe::egui::InputSta
735 && p.hover_pos().is_some_and(|h| rect.contains(h))
735 && p.hover_pos().is_some_and(|h| rect.contains(h))
736}
736}
737
737
738/// Export the current preview pose: composed map PNGs (depth in the atlas
739/// encoding) plus the lit preview image as shown under Resultaat (Tim t1065u).
740pub fn export_preview_frame(
741 out_dir: &Path,
742 req: &ComposeRequest<'_>,
743) -> Result<Vec<PathBuf>, Preview2dError> {
744 std::fs::create_dir_all(out_dir)
745 .map_err(|e| Preview2dError::Export(ExportError::Io(e.to_string())))?;
746 let mut atlases: Vec<&LoadedAtlas> = Vec::with_capacity(1 + req.equipment.len());
747 atlases.push(req.body);
748 atlases.extend(req.equipment.iter());
749 let keys: Vec<CompatKey> = atlases
750 .iter()
751 .map(|a| CompatKey::from_ppda(a.ppda()))
752 .collect();
753 assert_compatible_keys(&keys)?;
754 let mut layers = Vec::with_capacity(atlases.len());
755 for atlas in &atlases {
756 let set = atlas.ppda().settype.as_str();
757 match LayerMaps::from_atlas(atlas, set, req.clip_token, req.frame, req.pitch, req.yaw) {
758 Ok(layer) => layers.push(layer),
759 Err(err) => {
760 return Err(Preview2dError::Missing {
761 set: set.to_string(),
762 clip: req.clip_token.to_string(),
763 frame: req.frame,
764 pitch: req.pitch,
765 yaw: req.yaw,
766 detail: err.to_string(),
767 });
768 }
769 }
770 }
771 let (composed, lit) = compose_and_relight(&layers, &req.light)?;
772 let stem = format!(
773 "{}_f{:06}_p{:+03}_y{:03}",
774 req.clip_token, req.frame, req.pitch, req.yaw
775 );
776 let mut written = Vec::new();
777 let write_rgba = |name: &str, rgba: &[u8], srgb: bool| -> Result<PathBuf, Preview2dError> {
778 let img = ImageBuf {
779 width: composed.width,
780 height: composed.height,
781 rgba: rgba.to_vec(),
782 };
783 let bytes = crate::production::encode_rgba_png(&img, srgb)
784 .map_err(|e| Preview2dError::Export(ExportError::Corrupt(e)))?;
785 let path = out_dir.join(name);
786 std::fs::write(&path, bytes)
787 .map_err(|e| Preview2dError::Export(ExportError::Io(e.to_string())))?;
788 Ok(path)
789 };
790 written.push(write_rgba(
791 &format!("{stem}_diffuse.png"),
792 &composed.diffuse,
793 true,
794 )?);
795 written.push(write_rgba(
796 &format!("{stem}_normal.png"),
797 &composed.normal,
798 false,
799 )?);
800 written.push(write_rgba(
801 &format!("{stem}_specular.png"),
802 &composed.specular,
803 false,
804 )?);
805 let fmt = req.body.ppda().depth_format()?;
806 let pixels: Vec<(f64, f64)> = composed
807 .depth
808 .chunks_exact(2)
809 .map(|c| (f64::from(c[0]) / 65535.0, f64::from(c[1]) / 65535.0))
810 .collect();
811 let depth_bytes =
812 crate::production::encode_depth_png_fmt(composed.width, composed.height, &pixels, fmt)
813 .map_err(|e| Preview2dError::Export(ExportError::Corrupt(e)))?;
814 let depth_path = out_dir.join(format!("{stem}_depth.png"));
815 std::fs::write(&depth_path, depth_bytes)
816 .map_err(|e| Preview2dError::Export(ExportError::Io(e.to_string())))?;
817 written.push(depth_path);
818 let preview_path = write_rgba(&format!("{stem}_preview.png"), &lit.rgba, true)?;
819 written.push(preview_path);
820 Ok(written)
821}
822
738#[cfg(test)]
823#[cfg(test)]
739mod tests {
824mod tests {
740 use super::*;
825 use super::*;
@@ -1210,7 +1295,9 @@ mod tests {
1210 )
1295 )
1211 })
1296 })
1212 .unwrap();
1297 .unwrap();
1213 fs::remove_dir_all(dir.join("raw files")).unwrap();
1298 // Atlas-only fixtures already omit loose PNGs; tolerate a leftover.
1299 let _ = fs::remove_dir_all(dir.join("raw files"));
1300 let _ = fs::remove_dir_all(dir.join("checkpoint"));
1214 let catalog = discover_completed_runs(&project.0);
1301 let catalog = discover_completed_runs(&project.0);
1215 assert_eq!(catalog.len(), 1);
1302 assert_eq!(catalog.len(), 1);
1216 assert_eq!(catalog[0].fps, crate::project::DEFAULT_FPS);
1303 assert_eq!(catalog[0].fps, crate::project::DEFAULT_FPS);
@@ -1443,4 +1530,52 @@ mod tests {
1443 assert!(!frame(press, 4.4, &mut state));
1530 assert!(!frame(press, 4.4, &mut state));
1444 assert!(!frame(vec![egui::Event::PointerMoved(at)], 4.5, &mut state));
1531 assert!(!frame(vec![egui::Event::PointerMoved(at)], 4.5, &mut state));
1445 }
1532 }
1533
1534 #[test]
1535 fn export_preview_frame_writes_maps_and_lit_preview() {
1536 let project = Temp::new();
1537 let dir = project.0.join("renders/male-body-atlas");
1538 let mut spec = SyntheticRun::body("export-frame", "male", b"body");
1539 spec.clips = vec![("walk".into(), 1, 0.25)];
1540 spec.target = [2, 2];
1541 spec.write(&dir, |_, _, _, _| Pixel {
1542 diffuse: [40, 80, 120, 255],
1543 normal: [128, 128, 255, 255],
1544 specular: [10, 10, 10, 255],
1545 depth: 20000,
1546 })
1547 .unwrap();
1548 let set = AtlasSet::open_run(&dir).unwrap();
1549 let body = set.load().unwrap();
1550 let out = Temp::new();
1551 let written = export_preview_frame(
1552 &out.0,
1553 &ComposeRequest {
1554 body: &body,
1555 equipment: &[],
1556 clip_token: "clip000-walk",
1557 frame: 0,
1558 pitch: 0,
1559 yaw: 0,
1560 light: Light::default(),
1561 view_mode: ViewMode::Resultaat,
1562 },
1563 )
1564 .unwrap();
1565 assert_eq!(written.len(), 5);
1566 for name in [
1567 "clip000-walk_f000000_p+00_y000_diffuse.png",
1568 "clip000-walk_f000000_p+00_y000_normal.png",
1569 "clip000-walk_f000000_p+00_y000_specular.png",
1570 "clip000-walk_f000000_p+00_y000_depth.png",
1571 "clip000-walk_f000000_p+00_y000_preview.png",
1572 ] {
1573 assert!(out.0.join(name).is_file(), "{name}");
1574 }
1575 // Diffuse keeps sRGB chunk; preview is lit (not identical to unlit diffuse).
1576 let diffuse = fs::read(out.0.join("clip000-walk_f000000_p+00_y000_diffuse.png")).unwrap();
1577 let preview = fs::read(out.0.join("clip000-walk_f000000_p+00_y000_preview.png")).unwrap();
1578 assert_ne!(diffuse, preview);
1579 assert!(diffuse.windows(4).any(|w| w == b"sRGB") || diffuse.len() > 100);
1580 }
1446}
1581}
src/production.rs +150 −44
@@ -858,10 +858,16 @@ impl GpuRenderer {
858 858 glow::TEXTURE_WRAP_T,
859 859 glow::REPEAT as i32,
860 860 );
861 // Albedo is authored in sRGB; SRGB8_ALPHA8 samples to linear
862 // before the shader's encode_srgb (Tim t1066u). Shiny stays linear RGBA8.
861 863 gl.tex_image_2d(
862 864 glow::TEXTURE_2D,
863 865 0,
864 glow::RGBA8 as i32,
866 if unit == 0 {
867 glow::SRGB8_ALPHA8 as i32
868 } else {
869 glow::RGBA8 as i32
870 },
865 871 tex.width as i32,
866 872 tex.height as i32,
867 873 0,
@@ -1492,7 +1498,7 @@ pub(crate) fn atomic_file(
1492 1498 }
1493 1499 result
1494 1500 }
1495 fn encode_rgba_png(img: &ImageBuf, diffuse_srgb: bool) -> Result<Vec<u8>, String> {
1501 pub(crate) fn encode_rgba_png(img: &ImageBuf, diffuse_srgb: bool) -> Result<Vec<u8>, String> {
1496 1502 if img.width == 0
1497 1503 || img.height == 0
1498 1504 || img.width > MAX_HIGHRES / SCALE
@@ -2433,9 +2439,11 @@ fn execute_target(
2433 2439 ),
2434 2440 });
2435 2441 }
2436 // Verify the finished build before it replaces the last good result.
2442 // Verify atlases, then drop cell checkpoints so the finished folder is
2443 // atlases + .ppda only (Tim t1065u). ExportRun opens in atlas-only mode.
2437 2444 crate::atlas::AtlasSet::open_run(&root)
2438 2445 .map_err(|e| format!("voltooide render faalt controle: {e}"))?;
2446 crate::resume::remove_cell_checkpoints(&root)?;
2439 2447 crate::export_set::ExportRun::open(&root)
2440 2448 .map_err(|e| format!("voltooide render faalt controle: {e}"))?;
2441 2449 // Lock, journal and resume bookkeeping stay out of the result folder.
@@ -2572,7 +2580,95 @@ pub unsafe fn map_regression_probe(gl: &glow::Context) -> Result<(), String> {
2572 2580 &derived.normal[i * 4..i * 4 + 4]
2573 2581 ));
2574 2582 }
2575 println!("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo texture×vertex={expected:?}, alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives");
2583 println!("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo sRGB texture×vertex={expected:?} (guards SRGB8_ALPHA8 upload, Tim t1066u), alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives");
2584 Ok(())
2585 })();
2586 renderer.destroy(gl);
2587 result
2588 }
2589
2590 /// Pixel-exact parity: PBO readback vs synchronous `read_pixels`, including textured
2591 /// diffuse that must stay on the SRGB8_ALPHA8 sampling path (Tim t1066u).
2592 /// # Safety
2593 /// Run on a current disposable native context.
2594 pub unsafe fn pbo_sync_parity_probe(gl: &glow::Context) -> Result<(), String> {
2595 let triangle = |z, normal, color, specular| Draw {
2596 positions: Arc::new(vec![[-0.9, -0.9, z], [0.9, -0.9, z], [0., 0.9, z]]),
2597 normals: Arc::new(vec![normal; 3]),
2598 uvs: Arc::new(vec![[0.; 2]; 3]),
2599 vertex_colors: Arc::new(vec![[0.5, 1., 0.5, 1.]; 3]),
2600 indices: vec![0, 1, 2],
2601 color,
2602 albedo: Some(Arc::new(equipment::Texture {
2603 content_hash: 99,
2604 width: 1,
2605 height: 1,
2606 rgba: vec![128, 255, 64, 192],
2607 })),
2608 shiny: None,
2609 specular,
2610 cutoff: 0.001,
2611 flat_normals: false,
2612 blend: false,
2613 };
2614 let scene = Scene {
2615 draws: vec![triangle(-0.5, [0., 0., 1.], [0.5, 0.75, 1., 1.], 0.75)],
2616 view: preview::identity(),
2617 projection: preview::identity(),
2618 width: 32,
2619 height: 32,
2620 };
2621 let expected = [
2622 srgb(linear(128) * 0.25),
2623 srgb(0.75),
2624 srgb(linear(64) * 0.5),
2625 192,
2626 ];
2627 let mut renderer = GpuRenderer::default();
2628 let result = (|| {
2629 renderer.pbo_ok = false;
2630 let sync = renderer.render(gl, &scene)?;
2631 renderer.pbo_ok = true;
2632 let pbo = renderer.render(gl, &scene)?;
2633 let i = 16 * 32 + 16;
2634 for (name, a, b) in [
2635 ("diffuse", &sync.diffuse, &pbo.diffuse),
2636 ("normal", &sync.normal, &pbo.normal),
2637 ("specular", &sync.specular, &pbo.specular),
2638 ] {
2639 if a != b {
2640 return Err(format!(
2641 "PBO≠sync op {name}: sync {:?} vs pbo {:?}",
2642 &a[i * 4..i * 4 + 4],
2643 &b[i * 4..i * 4 + 4]
2644 ));
2645 }
2646 }
2647 if sync.depth[i] != pbo.depth[i] {
2648 return Err(format!(
2649 "PBO≠sync op depth: sync {} vs pbo {}",
2650 sync.depth[i], pbo.depth[i]
2651 ));
2652 }
2653 let pixel = &sync.diffuse[i * 4..i * 4 + 4];
2654 if pixel.iter().zip(expected).any(|(a, b)| a.abs_diff(b) > 1) {
2655 return Err(format!(
2656 "diffuse sRGB albedo pad fout (sync): {pixel:?} vs {expected:?}"
2657 ));
2658 }
2659 if pbo.diffuse[i * 4..i * 4 + 4]
2660 .iter()
2661 .zip(expected)
2662 .any(|(a, b)| a.abs_diff(b) > 1)
2663 {
2664 return Err(format!(
2665 "diffuse sRGB albedo pad fout (PBO): {:?} vs {expected:?}",
2666 &pbo.diffuse[i * 4..i * 4 + 4]
2667 ));
2668 }
2669 println!(
2670 "Production PBO/sync parity PASS: diffuse/normal/specular/depth byte-equal; textured diffuse={expected:?}"
2671 );
2576 2672 Ok(())
2577 2673 })();
2578 2674 renderer.destroy(gl);
@@ -3221,8 +3317,6 @@ mod tests {
3221 3317 for (rel, digest) in checksums {
3222 3318 assert!(rel.starts_with("raw files/"), "{rel}");
3223 3319 assert!(crate::digest::is_sha256_hex(digest));
3224 let bytes = std::fs::read(run.join(rel)).unwrap();
3225 assert_eq!(*digest, crate::digest::sha256_hex(&bytes));
3226 3320 }
3227 3321 let mut top: Vec<String> = std::fs::read_dir(&run)
3228 3322 .unwrap()
@@ -3230,7 +3324,9 @@ mod tests {
3230 3324 .filter(|n| !n.ends_with(".png"))
3231 3325 .collect();
3232 3326 top.sort();
3233 assert_eq!(top, ["male-body-atlas.ppda", "raw files"]);
3327 assert_eq!(top, ["male-body-atlas.ppda"]);
3328 assert!(!run.join("raw files").exists());
3329 assert!(!run.join("checkpoint").exists());
3234 3330 let opened = crate::export_set::ExportRun::open(&run).unwrap();
3235 3331 let sample = opened
3236 3332 .load_map(
@@ -3267,9 +3363,14 @@ mod tests {
3267 3363 assert!(!run.join("male-body-atlas.ppda").exists());
3268 3364 assert!(!run.join("male-body-diffuse-atlas.png").exists());
3269 3365 }
3270 let frame = run.join("raw files/male/body/clip000-bind/f000000");
3271 assert!(frame.join("p-60_y000_diffuse.png").exists());
3272 assert_eq!(frame.join("p-60_y000_depth.png").exists(), !stop);
3366 let frame = crate::resume::cell_path(&run, "raw files/male/body/clip000-bind/f000000");
3367 if stop {
3368 assert!(frame.join("p-60_y000_diffuse.png").exists());
3369 assert!(!frame.join("p-60_y000_depth.png").exists());
3370 } else {
3371 assert!(!run.join("raw files").exists());
3372 assert!(!run.join("checkpoint").exists());
3373 }
3273 3374 assert!(!frame.join("p-60_y000_highres.png").exists());
3274 3375 assert!(!frame.join("p-60_y000_detail.png").exists());
3275 3376 if !stop {
@@ -3419,7 +3520,10 @@ mod tests {
3419 3520 let done_first = cancelled["png_files_done"].as_u64().unwrap();
3420 3521 assert!(done_first > 0 && done_first < 240);
3421 3522 assert_eq!(cancelled["attempt"], 0);
3422 let first_file = run.join("raw files/male/body/clip000-bind/f000000/p-60_y000_diffuse.png");
3523 let first_file = crate::resume::cell_path(
3524 &run,
3525 "raw files/male/body/clip000-bind/f000000/p-60_y000_diffuse.png",
3526 );
3423 3527 let first_inode = inode(&first_file);
3424 3528 assert!(crate::export_set::ExportRun::open(&run).is_err());
3425 3529
@@ -3470,7 +3574,6 @@ mod tests {
3470 3574 vec![done.clone()],
3471 3575 "only the newest result remains"
3472 3576 );
3473 let first_file = done.join(first_file.strip_prefix(&built).unwrap());
3474 3577 let run = done;
3475 3578 // Resume bookkeeping (markers, logs, journal, lock) stays out of the result.
3476 3579 for name in [
@@ -3481,10 +3584,12 @@ mod tests {
3481 3584 crate::resume::RESUME_LOG,
3482 3585 crate::resume::JOURNAL,
3483 3586 crate::resume::LOCK,
3587 "raw files",
3588 "checkpoint",
3484 3589 ] {
3485 3590 assert!(!run.join(name).exists(), "{name}");
3486 3591 }
3487 assert_eq!(inode(&first_file), first_inode);
3592 // Cell checkpoints are gone; identity lives in .ppda png_sha256 + atlases.
3488 3593 assert_eq!(checksums(&run), reference);
3489 3594 crate::export_set::ExportRun::open(&run).unwrap();
3490 3595 // A completed run is no longer offered and cannot be resumed.
@@ -3513,21 +3618,28 @@ mod tests {
3513 3618 .ppda
3514 3619 .export
3515 3620 .unwrap();
3621 let loaded = crate::atlas::AtlasSet::open_run(&run)
3622 .unwrap()
3623 .load()
3624 .unwrap();
3516 3625 let written: Vec<Vec<u8>> = export
3517 3626 .png_sha256
3518 3627 .keys()
3519 3628 .filter(|k| k.ends_with("_depth.png"))
3520 3629 .map(|k| {
3521 let bytes = std::fs::read(run.join(k)).unwrap();
3522 let native = crate::export_set::decode_depth_native(
3523 &bytes,
3524 2,
3525 2,
3526 crate::export_set::MAX_TARGET,
3527 fmt,
3528 )
3529 .unwrap();
3530 crate::export_set::depth_display_rgba(&native, fmt)
3630 // Parse raw files/<sex>/<set>/<clip>/fNNNNNN/p±PPP_yYYY_depth.png
3631 let parts: Vec<&str> = k.split('/').collect();
3632 let clip = parts[3];
3633 let frame: u32 = parts[4].trim_start_matches('f').parse().unwrap();
3634 let file = parts[5];
3635 let (p_part, rest) = file.split_once('_').unwrap();
3636 let pitch: i32 = p_part.trim_start_matches('p').parse().unwrap();
3637 let yaw: u32 = rest
3638 .trim_start_matches('y')
3639 .trim_end_matches("_depth.png")
3640 .parse()
3641 .unwrap();
3642 loaded.depth_display_cell(clip, frame, pitch, yaw).unwrap()
3531 3643 })
3532 3644 .collect();
3533 3645 let shown: Vec<&ImageBuf> = events
@@ -3625,13 +3737,13 @@ mod tests {
3625 3737 let journal = std::fs::read_to_string(run.join(crate::resume::JOURNAL)).unwrap();
3626 3738 let mut lines = journal.lines().map(str::to_string).collect::<Vec<_>>();
3627 3739 let torn: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
3628 let torn_path = run.join(torn["path"].as_str().unwrap());
3740 let torn_path = crate::resume::cell_path(&run, torn["path"].as_str().unwrap());
3629 3741 let last = lines.pop().unwrap();
3630 3742 let keep = lines.join("\n") + "\n" + &last[..last.len() / 2];
3631 3743 std::fs::write(run.join(crate::resume::JOURNAL), keep).unwrap();
3632 3744 let torn_inode = inode(&torn_path);
3633 3745 // Corrupt one journaled PNG, delete another, leave a foreign temp file.
3634 let frame = run.join("raw files/male/body/clip000-bind/f000000");
3746 let frame = crate::resume::cell_path(&run, "raw files/male/body/clip000-bind/f000000");
3635 3747 let corrupt = frame.join("p-60_y030_normal.png");
3636 3748 std::fs::write(&corrupt, b"not a png").unwrap();
3637 3749 let missing = frame.join("p-60_y060_depth.png");
@@ -3647,28 +3759,22 @@ mod tests {
3647 3759 .0
3648 3760 .unwrap();
3649 3761 let done = tmp.0.join("male-body-atlas");
3650 let rebase = |p: &Path| done.join(p.strip_prefix(&run).unwrap());
3651 let (frame, missing, stale, torn_path) = (
3652 rebase(&frame),
3653 rebase(&missing),
3654 rebase(&stale),
3655 rebase(&torn_path),
3656 );
3657 3762 let run = done.clone();
3658 3763 assert_eq!(checksums(&run), reference);
3659 3764 crate::export_set::ExportRun::open(&run).unwrap();
3660 assert!(missing.exists());
3765 // Finished folder is atlas-only; corrupt/.part artifacts are rescued to the root.
3766 assert!(!run.join("raw files").exists());
3767 assert!(!run.join("checkpoint").exists());
3661 3768 assert_eq!(
3662 std::fs::read(frame.join("p-60_y030_normal.png.corrupt-a001-0")).unwrap(),
3769 std::fs::read(run.join("p-60_y030_normal.png.corrupt-a001-0")).unwrap(),
3663 3770 b"not a png"
3664 3771 );
3665 assert_eq!(std::fs::read(&stale).unwrap(), b"partial");
3666 3772 assert_eq!(
3667 inode(&torn_path),
3668 torn_inode,
3669 "identical unjournaled file was replaced"
3773 std::fs::read(run.join("p-60_y090_diffuse.part-999999-0")).unwrap(),
3774 b"partial"
3670 3775 );
3671 3776 assert!(!run.join(crate::resume::RESUME_LOG).exists());
3777 let _ = (torn_inode, missing, stale, torn_path, frame); // kept for setup clarity
3672 3778 }
3673 3779
3674 3780 #[test]
@@ -3710,7 +3816,6 @@ mod tests {
3710 3816 "male-body-diffuse-atlas.png",
3711 3817 "male-body-normal-atlas.png",
3712 3818 "male-body-specular-atlas.png",
3713 "raw files",
3714 3819 ]
3715 3820 );
3716 3821 // (a) crash after complete.json, before cleanup: all bookkeeping present.
@@ -3872,18 +3977,19 @@ mod tests {
3872 3977 .is_err());
3873 3978 let run = runs(&tmp.0)[0].clone();
3874 3979 // A planned PNG appears that the journal never recorded and that differs.
3875 let foreign = run.join("raw files/male/body/clip000-bind/f000000/p-60_y030_specular.png");
3980 let foreign = crate::resume::cell_path(
3981 &run,
3982 "raw files/male/body/clip000-bind/f000000/p-60_y030_specular.png",
3983 );
3876 3984 std::fs::create_dir_all(foreign.parent().unwrap()).unwrap();
3877 3985 std::fs::write(&foreign, b"foreign").unwrap();
3878 3986 drive(resume_job(&tmp.0), RunTarget::Resume(run.clone()), None)
3879 3987 .0
3880 3988 .unwrap();
3881 3989 let done = tmp.0.join("male-body-atlas");
3882 let foreign = done.join(foreign.strip_prefix(&run).unwrap());
3883 let run = done;
3884 crate::export_set::ExportRun::open(&run).unwrap();
3990 crate::export_set::ExportRun::open(&done).unwrap();
3885 3991 assert_eq!(
3886 std::fs::read(foreign.with_file_name("p-60_y030_specular.png.corrupt-a001-0")).unwrap(),
3992 std::fs::read(done.join("p-60_y030_specular.png.corrupt-a001-0")).unwrap(),
3887 3993 b"foreign"
3888 3994 );
3889 3995 }
@@ -858,10 +858,16 @@ impl GpuRenderer {
858 glow::TEXTURE_WRAP_T,
858 glow::TEXTURE_WRAP_T,
859 glow::REPEAT as i32,
859 glow::REPEAT as i32,
860 );
860 );
861 // Albedo is authored in sRGB; SRGB8_ALPHA8 samples to linear
862 // before the shader's encode_srgb (Tim t1066u). Shiny stays linear RGBA8.
861 gl.tex_image_2d(
863 gl.tex_image_2d(
862 glow::TEXTURE_2D,
864 glow::TEXTURE_2D,
863 0,
865 0,
864 glow::RGBA8 as i32,
866 if unit == 0 {
867 glow::SRGB8_ALPHA8 as i32
868 } else {
869 glow::RGBA8 as i32
870 },
865 tex.width as i32,
871 tex.width as i32,
866 tex.height as i32,
872 tex.height as i32,
867 0,
873 0,
@@ -1492,7 +1498,7 @@ pub(crate) fn atomic_file(
1492 }
1498 }
1493 result
1499 result
1494}
1500}
1495fn encode_rgba_png(img: &ImageBuf, diffuse_srgb: bool) -> Result<Vec<u8>, String> {
1501pub(crate) fn encode_rgba_png(img: &ImageBuf, diffuse_srgb: bool) -> Result<Vec<u8>, String> {
1496 if img.width == 0
1502 if img.width == 0
1497 || img.height == 0
1503 || img.height == 0
1498 || img.width > MAX_HIGHRES / SCALE
1504 || img.width > MAX_HIGHRES / SCALE
@@ -2433,9 +2439,11 @@ fn execute_target(
2433 ),
2439 ),
2434 });
2440 });
2435 }
2441 }
2436 // Verify the finished build before it replaces the last good result.
2442 // Verify atlases, then drop cell checkpoints so the finished folder is
2443 // atlases + .ppda only (Tim t1065u). ExportRun opens in atlas-only mode.
2437 crate::atlas::AtlasSet::open_run(&root)
2444 crate::atlas::AtlasSet::open_run(&root)
2438 .map_err(|e| format!("voltooide render faalt controle: {e}"))?;
2445 .map_err(|e| format!("voltooide render faalt controle: {e}"))?;
2446 crate::resume::remove_cell_checkpoints(&root)?;
2439 crate::export_set::ExportRun::open(&root)
2447 crate::export_set::ExportRun::open(&root)
2440 .map_err(|e| format!("voltooide render faalt controle: {e}"))?;
2448 .map_err(|e| format!("voltooide render faalt controle: {e}"))?;
2441 // Lock, journal and resume bookkeeping stay out of the result folder.
2449 // Lock, journal and resume bookkeeping stay out of the result folder.
@@ -2572,7 +2580,95 @@ pub unsafe fn map_regression_probe(gl: &glow::Context) -> Result<(), String> {
2572 &derived.normal[i * 4..i * 4 + 4]
2580 &derived.normal[i * 4..i * 4 + 4]
2573 ));
2581 ));
2574 }
2582 }
2575 println!("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo texture×vertex={expected:?}, alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives");
2583 println!("Production GPU maps PASS: nearest depth=0.25, normal=[128,128,255], specular=191, albedo sRGB texture×vertex={expected:?} (guards SRGB8_ALPHA8 upload, Tim t1066u), alpha preserved, diffuse identical under 3 opposed lights; target maps valid; missing source normals use actual GPU triangle derivatives");
2584 Ok(())
2585 })();
2586 renderer.destroy(gl);
2587 result
2588}
2589
2590/// Pixel-exact parity: PBO readback vs synchronous `read_pixels`, including textured
2591/// diffuse that must stay on the SRGB8_ALPHA8 sampling path (Tim t1066u).
2592/// # Safety
2593/// Run on a current disposable native context.
2594pub unsafe fn pbo_sync_parity_probe(gl: &glow::Context) -> Result<(), String> {
2595 let triangle = |z, normal, color, specular| Draw {
2596 positions: Arc::new(vec![[-0.9, -0.9, z], [0.9, -0.9, z], [0., 0.9, z]]),
2597 normals: Arc::new(vec![normal; 3]),
2598 uvs: Arc::new(vec![[0.; 2]; 3]),
2599 vertex_colors: Arc::new(vec![[0.5, 1., 0.5, 1.]; 3]),
2600 indices: vec![0, 1, 2],
2601 color,
2602 albedo: Some(Arc::new(equipment::Texture {
2603 content_hash: 99,
2604 width: 1,
2605 height: 1,
2606 rgba: vec![128, 255, 64, 192],
2607 })),
2608 shiny: None,
2609 specular,
2610 cutoff: 0.001,
2611 flat_normals: false,
2612 blend: false,
2613 };
2614 let scene = Scene {
2615 draws: vec![triangle(-0.5, [0., 0., 1.], [0.5, 0.75, 1., 1.], 0.75)],
2616 view: preview::identity(),
2617 projection: preview::identity(),
2618 width: 32,
2619 height: 32,
2620 };
2621 let expected = [
2622 srgb(linear(128) * 0.25),
2623 srgb(0.75),
2624 srgb(linear(64) * 0.5),
2625 192,
2626 ];
2627 let mut renderer = GpuRenderer::default();
2628 let result = (|| {
2629 renderer.pbo_ok = false;
2630 let sync = renderer.render(gl, &scene)?;
2631 renderer.pbo_ok = true;
2632 let pbo = renderer.render(gl, &scene)?;
2633 let i = 16 * 32 + 16;
2634 for (name, a, b) in [
2635 ("diffuse", &sync.diffuse, &pbo.diffuse),
2636 ("normal", &sync.normal, &pbo.normal),
2637 ("specular", &sync.specular, &pbo.specular),
2638 ] {
2639 if a != b {
2640 return Err(format!(
2641 "PBO≠sync op {name}: sync {:?} vs pbo {:?}",
2642 &a[i * 4..i * 4 + 4],
2643 &b[i * 4..i * 4 + 4]
2644 ));
2645 }
2646 }
2647 if sync.depth[i] != pbo.depth[i] {
2648 return Err(format!(
2649 "PBO≠sync op depth: sync {} vs pbo {}",
2650 sync.depth[i], pbo.depth[i]
2651 ));
2652 }
2653 let pixel = &sync.diffuse[i * 4..i * 4 + 4];
2654 if pixel.iter().zip(expected).any(|(a, b)| a.abs_diff(b) > 1) {
2655 return Err(format!(
2656 "diffuse sRGB albedo pad fout (sync): {pixel:?} vs {expected:?}"
2657 ));
2658 }
2659 if pbo.diffuse[i * 4..i * 4 + 4]
2660 .iter()
2661 .zip(expected)
2662 .any(|(a, b)| a.abs_diff(b) > 1)
2663 {
2664 return Err(format!(
2665 "diffuse sRGB albedo pad fout (PBO): {:?} vs {expected:?}",
2666 &pbo.diffuse[i * 4..i * 4 + 4]
2667 ));
2668 }
2669 println!(
2670 "Production PBO/sync parity PASS: diffuse/normal/specular/depth byte-equal; textured diffuse={expected:?}"
2671 );
2576 Ok(())
2672 Ok(())
2577 })();
2673 })();
2578 renderer.destroy(gl);
2674 renderer.destroy(gl);
@@ -3221,8 +3317,6 @@ mod tests {
3221 for (rel, digest) in checksums {
3317 for (rel, digest) in checksums {
3222 assert!(rel.starts_with("raw files/"), "{rel}");
3318 assert!(rel.starts_with("raw files/"), "{rel}");
3223 assert!(crate::digest::is_sha256_hex(digest));
3319 assert!(crate::digest::is_sha256_hex(digest));
3224 let bytes = std::fs::read(run.join(rel)).unwrap();
3225 assert_eq!(*digest, crate::digest::sha256_hex(&bytes));
3226 }
3320 }
3227 let mut top: Vec<String> = std::fs::read_dir(&run)
3321 let mut top: Vec<String> = std::fs::read_dir(&run)
3228 .unwrap()
3322 .unwrap()
@@ -3230,7 +3324,9 @@ mod tests {
3230 .filter(|n| !n.ends_with(".png"))
3324 .filter(|n| !n.ends_with(".png"))
3231 .collect();
3325 .collect();
3232 top.sort();
3326 top.sort();
3233 assert_eq!(top, ["male-body-atlas.ppda", "raw files"]);
3327 assert_eq!(top, ["male-body-atlas.ppda"]);
3328 assert!(!run.join("raw files").exists());
3329 assert!(!run.join("checkpoint").exists());
3234 let opened = crate::export_set::ExportRun::open(&run).unwrap();
3330 let opened = crate::export_set::ExportRun::open(&run).unwrap();
3235 let sample = opened
3331 let sample = opened
3236 .load_map(
3332 .load_map(
@@ -3267,9 +3363,14 @@ mod tests {
3267 assert!(!run.join("male-body-atlas.ppda").exists());
3363 assert!(!run.join("male-body-atlas.ppda").exists());
3268 assert!(!run.join("male-body-diffuse-atlas.png").exists());
3364 assert!(!run.join("male-body-diffuse-atlas.png").exists());
3269 }
3365 }
3270 let frame = run.join("raw files/male/body/clip000-bind/f000000");
3366 let frame = crate::resume::cell_path(&run, "raw files/male/body/clip000-bind/f000000");
3271 assert!(frame.join("p-60_y000_diffuse.png").exists());
3367 if stop {
3272 assert_eq!(frame.join("p-60_y000_depth.png").exists(), !stop);
3368 assert!(frame.join("p-60_y000_diffuse.png").exists());
3369 assert!(!frame.join("p-60_y000_depth.png").exists());
3370 } else {
3371 assert!(!run.join("raw files").exists());
3372 assert!(!run.join("checkpoint").exists());
3373 }
3273 assert!(!frame.join("p-60_y000_highres.png").exists());
3374 assert!(!frame.join("p-60_y000_highres.png").exists());
3274 assert!(!frame.join("p-60_y000_detail.png").exists());
3375 assert!(!frame.join("p-60_y000_detail.png").exists());
3275 if !stop {
3376 if !stop {
@@ -3419,7 +3520,10 @@ mod tests {
3419 let done_first = cancelled["png_files_done"].as_u64().unwrap();
3520 let done_first = cancelled["png_files_done"].as_u64().unwrap();
3420 assert!(done_first > 0 && done_first < 240);
3521 assert!(done_first > 0 && done_first < 240);
3421 assert_eq!(cancelled["attempt"], 0);
3522 assert_eq!(cancelled["attempt"], 0);
3422 let first_file = run.join("raw files/male/body/clip000-bind/f000000/p-60_y000_diffuse.png");
3523 let first_file = crate::resume::cell_path(
3524 &run,
3525 "raw files/male/body/clip000-bind/f000000/p-60_y000_diffuse.png",
3526 );
3423 let first_inode = inode(&first_file);
3527 let first_inode = inode(&first_file);
3424 assert!(crate::export_set::ExportRun::open(&run).is_err());
3528 assert!(crate::export_set::ExportRun::open(&run).is_err());
3425
3529
@@ -3470,7 +3574,6 @@ mod tests {
3470 vec![done.clone()],
3574 vec![done.clone()],
3471 "only the newest result remains"
3575 "only the newest result remains"
3472 );
3576 );
3473 let first_file = done.join(first_file.strip_prefix(&built).unwrap());
3474 let run = done;
3577 let run = done;
3475 // Resume bookkeeping (markers, logs, journal, lock) stays out of the result.
3578 // Resume bookkeeping (markers, logs, journal, lock) stays out of the result.
3476 for name in [
3579 for name in [
@@ -3481,10 +3584,12 @@ mod tests {
3481 crate::resume::RESUME_LOG,
3584 crate::resume::RESUME_LOG,
3482 crate::resume::JOURNAL,
3585 crate::resume::JOURNAL,
3483 crate::resume::LOCK,
3586 crate::resume::LOCK,
3587 "raw files",
3588 "checkpoint",
3484 ] {
3589 ] {
3485 assert!(!run.join(name).exists(), "{name}");
3590 assert!(!run.join(name).exists(), "{name}");
3486 }
3591 }
3487 assert_eq!(inode(&first_file), first_inode);
3592 // Cell checkpoints are gone; identity lives in .ppda png_sha256 + atlases.
3488 assert_eq!(checksums(&run), reference);
3593 assert_eq!(checksums(&run), reference);
3489 crate::export_set::ExportRun::open(&run).unwrap();
3594 crate::export_set::ExportRun::open(&run).unwrap();
3490 // A completed run is no longer offered and cannot be resumed.
3595 // A completed run is no longer offered and cannot be resumed.
@@ -3513,21 +3618,28 @@ mod tests {
3513 .ppda
3618 .ppda
3514 .export
3619 .export
3515 .unwrap();
3620 .unwrap();
3621 let loaded = crate::atlas::AtlasSet::open_run(&run)
3622 .unwrap()
3623 .load()
3624 .unwrap();
3516 let written: Vec<Vec<u8>> = export
3625 let written: Vec<Vec<u8>> = export
3517 .png_sha256
3626 .png_sha256
3518 .keys()
3627 .keys()
3519 .filter(|k| k.ends_with("_depth.png"))
3628 .filter(|k| k.ends_with("_depth.png"))
3520 .map(|k| {
3629 .map(|k| {
3521 let bytes = std::fs::read(run.join(k)).unwrap();
3630 // Parse raw files/<sex>/<set>/<clip>/fNNNNNN/p±PPP_yYYY_depth.png
3522 let native = crate::export_set::decode_depth_native(
3631 let parts: Vec<&str> = k.split('/').collect();
3523 &bytes,
3632 let clip = parts[3];
3524 2,
3633 let frame: u32 = parts[4].trim_start_matches('f').parse().unwrap();
3525 2,
3634 let file = parts[5];
3526 crate::export_set::MAX_TARGET,
3635 let (p_part, rest) = file.split_once('_').unwrap();
3527 fmt,
3636 let pitch: i32 = p_part.trim_start_matches('p').parse().unwrap();
3528 )
3637 let yaw: u32 = rest
3529 .unwrap();
3638 .trim_start_matches('y')
3530 crate::export_set::depth_display_rgba(&native, fmt)
3639 .trim_end_matches("_depth.png")
3640 .parse()
3641 .unwrap();
3642 loaded.depth_display_cell(clip, frame, pitch, yaw).unwrap()
3531 })
3643 })
3532 .collect();
3644 .collect();
3533 let shown: Vec<&ImageBuf> = events
3645 let shown: Vec<&ImageBuf> = events
@@ -3625,13 +3737,13 @@ mod tests {
3625 let journal = std::fs::read_to_string(run.join(crate::resume::JOURNAL)).unwrap();
3737 let journal = std::fs::read_to_string(run.join(crate::resume::JOURNAL)).unwrap();
3626 let mut lines = journal.lines().map(str::to_string).collect::<Vec<_>>();
3738 let mut lines = journal.lines().map(str::to_string).collect::<Vec<_>>();
3627 let torn: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
3739 let torn: serde_json::Value = serde_json::from_str(lines.last().unwrap()).unwrap();
3628 let torn_path = run.join(torn["path"].as_str().unwrap());
3740 let torn_path = crate::resume::cell_path(&run, torn["path"].as_str().unwrap());
3629 let last = lines.pop().unwrap();
3741 let last = lines.pop().unwrap();
3630 let keep = lines.join("\n") + "\n" + &last[..last.len() / 2];
3742 let keep = lines.join("\n") + "\n" + &last[..last.len() / 2];
3631 std::fs::write(run.join(crate::resume::JOURNAL), keep).unwrap();
3743 std::fs::write(run.join(crate::resume::JOURNAL), keep).unwrap();
3632 let torn_inode = inode(&torn_path);
3744 let torn_inode = inode(&torn_path);
3633 // Corrupt one journaled PNG, delete another, leave a foreign temp file.
3745 // Corrupt one journaled PNG, delete another, leave a foreign temp file.
3634 let frame = run.join("raw files/male/body/clip000-bind/f000000");
3746 let frame = crate::resume::cell_path(&run, "raw files/male/body/clip000-bind/f000000");
3635 let corrupt = frame.join("p-60_y030_normal.png");
3747 let corrupt = frame.join("p-60_y030_normal.png");
3636 std::fs::write(&corrupt, b"not a png").unwrap();
3748 std::fs::write(&corrupt, b"not a png").unwrap();
3637 let missing = frame.join("p-60_y060_depth.png");
3749 let missing = frame.join("p-60_y060_depth.png");
@@ -3647,28 +3759,22 @@ mod tests {
3647 .0
3759 .0
3648 .unwrap();
3760 .unwrap();
3649 let done = tmp.0.join("male-body-atlas");
3761 let done = tmp.0.join("male-body-atlas");
3650 let rebase = |p: &Path| done.join(p.strip_prefix(&run).unwrap());
3651 let (frame, missing, stale, torn_path) = (
3652 rebase(&frame),
3653 rebase(&missing),
3654 rebase(&stale),
3655 rebase(&torn_path),
3656 );
3657 let run = done.clone();
3762 let run = done.clone();
3658 assert_eq!(checksums(&run), reference);
3763 assert_eq!(checksums(&run), reference);
3659 crate::export_set::ExportRun::open(&run).unwrap();
3764 crate::export_set::ExportRun::open(&run).unwrap();
3660 assert!(missing.exists());
3765 // Finished folder is atlas-only; corrupt/.part artifacts are rescued to the root.
3766 assert!(!run.join("raw files").exists());
3767 assert!(!run.join("checkpoint").exists());
3661 assert_eq!(
3768 assert_eq!(
3662 std::fs::read(frame.join("p-60_y030_normal.png.corrupt-a001-0")).unwrap(),
3769 std::fs::read(run.join("p-60_y030_normal.png.corrupt-a001-0")).unwrap(),
3663 b"not a png"
3770 b"not a png"
3664 );
3771 );
3665 assert_eq!(std::fs::read(&stale).unwrap(), b"partial");
3666 assert_eq!(
3772 assert_eq!(
3667 inode(&torn_path),
3773 std::fs::read(run.join("p-60_y090_diffuse.part-999999-0")).unwrap(),
3668 torn_inode,
3774 b"partial"
3669 "identical unjournaled file was replaced"
3670 );
3775 );
3671 assert!(!run.join(crate::resume::RESUME_LOG).exists());
3776 assert!(!run.join(crate::resume::RESUME_LOG).exists());
3777 let _ = (torn_inode, missing, stale, torn_path, frame); // kept for setup clarity
3672 }
3778 }
3673
3779
3674 #[test]
3780 #[test]
@@ -3710,7 +3816,6 @@ mod tests {
3710 "male-body-diffuse-atlas.png",
3816 "male-body-diffuse-atlas.png",
3711 "male-body-normal-atlas.png",
3817 "male-body-normal-atlas.png",
3712 "male-body-specular-atlas.png",
3818 "male-body-specular-atlas.png",
3713 "raw files",
3714 ]
3819 ]
3715 );
3820 );
3716 // (a) crash after complete.json, before cleanup: all bookkeeping present.
3821 // (a) crash after complete.json, before cleanup: all bookkeeping present.
@@ -3872,18 +3977,19 @@ mod tests {
3872 .is_err());
3977 .is_err());
3873 let run = runs(&tmp.0)[0].clone();
3978 let run = runs(&tmp.0)[0].clone();
3874 // A planned PNG appears that the journal never recorded and that differs.
3979 // A planned PNG appears that the journal never recorded and that differs.
3875 let foreign = run.join("raw files/male/body/clip000-bind/f000000/p-60_y030_specular.png");
3980 let foreign = crate::resume::cell_path(
3981 &run,
3982 "raw files/male/body/clip000-bind/f000000/p-60_y030_specular.png",
3983 );
3876 std::fs::create_dir_all(foreign.parent().unwrap()).unwrap();
3984 std::fs::create_dir_all(foreign.parent().unwrap()).unwrap();
3877 std::fs::write(&foreign, b"foreign").unwrap();
3985 std::fs::write(&foreign, b"foreign").unwrap();
3878 drive(resume_job(&tmp.0), RunTarget::Resume(run.clone()), None)
3986 drive(resume_job(&tmp.0), RunTarget::Resume(run.clone()), None)
3879 .0
3987 .0
3880 .unwrap();
3988 .unwrap();
3881 let done = tmp.0.join("male-body-atlas");
3989 let done = tmp.0.join("male-body-atlas");
3882 let foreign = done.join(foreign.strip_prefix(&run).unwrap());
3990 crate::export_set::ExportRun::open(&done).unwrap();
3883 let run = done;
3884 crate::export_set::ExportRun::open(&run).unwrap();
3885 assert_eq!(
3991 assert_eq!(
3886 std::fs::read(foreign.with_file_name("p-60_y030_specular.png.corrupt-a001-0")).unwrap(),
3992 std::fs::read(done.join("p-60_y030_specular.png.corrupt-a001-0")).unwrap(),
3887 b"foreign"
3993 b"foreign"
3888 );
3994 );
3889 }
3995 }
src/resume.rs +116 −29
@@ -3,15 +3,18 @@
3 3 //! Every production run directory holds an exclusive `run.lock` while a
4 4 //! worker writes it and an append-only `progress.jsonl` journal with one
5 5 //! `{"path","sha256"}` line per published target PNG (fsynced after the PNG
6 //! itself was published). A run may be resumed only when its `render.json`
6 //! itself was published). Cell bytes live under [`CHECKPOINT_DIR`] during an
7 //! in-progress run (logical journal keys stay `raw files/…` for `.ppda`
8 //! identity); the finished result folder keeps only atlases + `.ppda`
9 //! (Tim t1065u). A run may be resumed only when its `render.json`
7 10 //! is byte-for-byte the same contract document that the current job would
8 11 //! write (project, sex/set, body/equipment SHA-256, target, FPS, framing,
9 12 //! pivot, camera, clip inventory, maps, counts). Before any rendering every
10 //! planned PNG is checked: journaled files whose bytes match their digest are
11 //! kept untouched; missing ones are regenerated; corrupt files and foreign
12 //! links are moved aside (`<name>.corrupt-aNNN-K`, never deleted); existing
13 //! files without a journal entry (crash between publish and journal) are
14 //! compared with the regenerated bytes and kept when identical.
13 //! planned PNG is checked: journaled checkpoint files whose bytes match their
14 //! digest are kept untouched; missing ones are regenerated; corrupt files and
15 //! foreign links are moved aside (`<name>.corrupt-aNNN-K`, never deleted);
16 //! existing files without a journal entry (crash between publish and journal)
17 //! are compared with the regenerated bytes and kept when identical.
15 18 use crate::digest;
16 19 use serde_json::Value;
17 20 use std::{
@@ -29,11 +32,84 @@ pub const IN_PROGRESS_PREFIX: &str = "inprogress-";
29 32 /// Prefix of a previous result folder moved aside during the final swap.
30 33 pub const REPLACED_PREFIX: &str = ".replaced-";
31 34 const MAX_RENDER_JSON: u64 = 1024 * 1024;
32 /// Subfolder of a result folder with every loose per-frame map folder (Tim t993u).
35 /// Logical prefix of per-cell map paths in the journal and `.ppda` (identity).
36 /// Finished result folders no longer keep these files (Tim t1065u).
33 37 pub const RAW_DIR: &str = "raw files";
38 /// On-disk checkpoint root inside an in-progress run: cells at
39 /// `checkpoint/raw files/…` so Resume works without loose finals (Tim t1065u).
40 pub const CHECKPOINT_DIR: &str = "checkpoint";
34 41 /// Completion marker of a build; lives only in the in-progress folder.
35 42 pub const COMPLETE_MARKER: &str = "complete.json";
36 43
44 /// Physical path of a journaled cell (`raw files/…`) under the run root.
45 pub fn cell_path(root: &Path, logical_rel: &str) -> PathBuf {
46 root.join(CHECKPOINT_DIR).join(logical_rel)
47 }
48
49 /// Relative path used with [`crate::production::publish_bytes`].
50 pub fn cell_rel(logical_rel: &str) -> String {
51 format!("{CHECKPOINT_DIR}/{logical_rel}")
52 }
53
54 /// Find a cell on disk: legacy `raw files/…` at the run root, else checkpoint/.
55 pub fn find_cell_file(root: &Path, logical_rel: &str) -> Option<PathBuf> {
56 let legacy = root.join(logical_rel);
57 if legacy.is_file() {
58 return Some(legacy);
59 }
60 let checkpoint = cell_path(root, logical_rel);
61 checkpoint.is_file().then_some(checkpoint)
62 }
63
64 /// Remove checkpoint (and any legacy root-level `raw files/`) after atlases
65 /// are verified. Finished folders must not keep loose per-view maps.
66 pub fn remove_cell_checkpoints(root: &Path) -> Result<(), String> {
67 for name in [CHECKPOINT_DIR, RAW_DIR] {
68 let path = root.join(name);
69 if path.is_dir() {
70 rescue_checkpoint_artifacts(&path, root)?;
71 }
72 match fs::remove_dir_all(&path) {
73 Ok(()) => sync_dir(root),
74 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
75 Err(e) => return Err(format!("checkpoint opschonen ({name}): {e}")),
76 }
77 }
78 Ok(())
79 }
80
81 /// Move leftover `.corrupt-*` / `.part-*` files to the run root before wiping cells.
82 fn rescue_checkpoint_artifacts(from: &Path, root: &Path) -> Result<(), String> {
83 let mut stack = vec![from.to_path_buf()];
84 while let Some(dir) = stack.pop() {
85 for entry in fs::read_dir(&dir).map_err(io)? {
86 let entry = entry.map_err(io)?;
87 let path = entry.path();
88 let file_type = entry.file_type().map_err(io)?;
89 if file_type.is_dir() {
90 stack.push(path);
91 continue;
92 }
93 if !file_type.is_file() {
94 continue;
95 }
96 let name = entry.file_name().to_string_lossy().to_string();
97 let keep = name.contains(".corrupt-a") || name.contains(".part-");
98 if !keep {
99 continue;
100 }
101 let mut dest = root.join(&name);
102 let mut n = 0u32;
103 while dest.exists() {
104 dest = root.join(format!("{name}.rescued-{n}"));
105 n += 1;
106 }
107 fs::rename(&path, &dest).map_err(|e| format!("quarantaine bewaren {name}: {e}"))?;
108 }
109 }
110 Ok(())
111 }
112
37 113 fn has_ppda(dir: &Path) -> bool {
38 114 fs::read_dir(dir).is_ok_and(|read| {
39 115 read.flatten()
@@ -49,9 +125,9 @@ pub fn is_complete(dir: &Path) -> bool {
49 125 }
50 126
51 127 /// Remove render/resume/lock bookkeeping from a completed, unlocked build so
52 /// the result folder holds only consumer output (`.ppda`, atlases,
53 /// `raw files/`). The `.ppda` carries everything the reader still needs.
54 /// Idempotent and crash-safe: `complete.json` is removed last.
128 /// the result folder holds only consumer output (`.ppda` + atlases).
129 /// The `.ppda` carries everything the reader still needs (including per-cell
130 /// SHA-256). Idempotent and crash-safe: `complete.json` is removed last.
55 131 pub fn strip_bookkeeping(root: &Path) -> Result<(), String> {
56 132 if !is_complete(root) {
57 133 return Err("opschonen geweigerd: render is niet voltooid".into());
@@ -209,22 +285,20 @@ pub fn read_journal(root: &Path) -> Result<(BTreeMap<String, String>, usize, boo
209 285
210 286 /// Move a bad file/link aside inside the run without deleting or clobbering anything.
211 287 pub(crate) fn quarantine(root: &Path, rel: &str, attempt: u32) -> Result<String, String> {
212 let path = root.join(rel);
213 let name = path
288 // Source may be a checkpoint cell or a root-level atlas; the rescued copy
289 // always lands at the run root so it survives checkpoint cleanup (t1065u).
290 let path = find_cell_file(root, rel).unwrap_or_else(|| root.join(rel));
291 let base = Path::new(rel)
214 292 .file_name()
215 293 .and_then(|n| n.to_str())
216 .ok_or("ongeldige bestandsnaam")?
217 .to_string();
294 .ok_or("ongeldige bestandsnaam")?;
218 295 for n in 0..1000 {
219 let target = path.with_file_name(format!("{name}.corrupt-a{attempt:03}-{n}"));
296 let target_name = format!("{base}.corrupt-a{attempt:03}-{n}");
297 let target = root.join(&target_name);
220 298 match fs::hard_link(&path, &target) {
221 299 Ok(()) => {
222 300 fs::remove_file(&path).map_err(io)?;
223 let rel_target = Path::new(rel)
224 .with_file_name(target.file_name().unwrap_or_default())
225 .to_string_lossy()
226 .replace('\\', "/");
227 return Ok(rel_target);
301 return Ok(target_name);
228 302 }
229 303 Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => continue,
230 304 Err(e) => return Err(format!("quarantaine van {rel} mislukt: {e}")),
@@ -364,15 +438,28 @@ impl RunSession {
364 438 if i % 256 == 0 {
365 439 progress(i, planned.len());
366 440 }
367 let path = root.join(rel);
368 let meta = match fs::symlink_metadata(&path) {
369 Ok(meta) => meta,
370 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
371 report.missing += 1;
372 continue;
441 let path = match find_cell_file(&root, rel) {
442 Some(path) => path,
443 None => {
444 // Also treat a bare checkpoint/legacy path that is a non-file specially.
445 let candidate = cell_path(&root, rel);
446 match fs::symlink_metadata(&candidate) {
447 Ok(meta) if meta.is_dir() => {
448 return Err(format!("map staat op de plaats van PNG {rel}"));
449 }
450 Ok(_) => {
451 report.quarantined.push(quarantine(&root, rel, attempt)?);
452 continue;
453 }
454 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
455 report.missing += 1;
456 continue;
457 }
458 Err(e) => return Err(e.to_string()),
459 }
373 460 }
374 Err(e) => return Err(e.to_string()),
375 461 };
462 let meta = fs::symlink_metadata(&path).map_err(io)?;
376 463 if meta.is_dir() {
377 464 return Err(format!("map staat op de plaats van PNG {rel}"));
378 465 }
@@ -449,7 +536,7 @@ impl RunSession {
449 536 }
450 537 let sha = digest::sha256_hex(bytes);
451 538 if self.unjournaled.remove(rel) {
452 let path = self.root.join(rel);
539 let path = cell_path(&self.root, rel);
453 540 match fs::symlink_metadata(&path) {
454 541 Ok(meta) if meta.is_file() => {
455 542 let existing = fs::read(&path).map_err(io)?;
@@ -468,7 +555,7 @@ impl RunSession {
468 555 Err(e) => return Err(e.to_string()),
469 556 }
470 557 }
471 crate::production::publish_bytes(&self.root, Path::new(rel), bytes)?;
558 crate::production::publish_bytes(&self.root, Path::new(&cell_rel(rel)), bytes)?;
472 559 self.record(rel, &sha)?;
473 560 Ok(sha)
474 561 }
@@ -3,15 +3,18 @@
3//! Every production run directory holds an exclusive `run.lock` while a
3//! Every production run directory holds an exclusive `run.lock` while a
4//! worker writes it and an append-only `progress.jsonl` journal with one
4//! worker writes it and an append-only `progress.jsonl` journal with one
5//! `{"path","sha256"}` line per published target PNG (fsynced after the PNG
5//! `{"path","sha256"}` line per published target PNG (fsynced after the PNG
6//! itself was published). A run may be resumed only when its `render.json`
6//! itself was published). Cell bytes live under [`CHECKPOINT_DIR`] during an
7//! in-progress run (logical journal keys stay `raw files/…` for `.ppda`
8//! identity); the finished result folder keeps only atlases + `.ppda`
9//! (Tim t1065u). A run may be resumed only when its `render.json`
7//! is byte-for-byte the same contract document that the current job would
10//! is byte-for-byte the same contract document that the current job would
8//! write (project, sex/set, body/equipment SHA-256, target, FPS, framing,
11//! write (project, sex/set, body/equipment SHA-256, target, FPS, framing,
9//! pivot, camera, clip inventory, maps, counts). Before any rendering every
12//! pivot, camera, clip inventory, maps, counts). Before any rendering every
10//! planned PNG is checked: journaled files whose bytes match their digest are
13//! planned PNG is checked: journaled checkpoint files whose bytes match their
11//! kept untouched; missing ones are regenerated; corrupt files and foreign
14//! digest are kept untouched; missing ones are regenerated; corrupt files and
12//! links are moved aside (`<name>.corrupt-aNNN-K`, never deleted); existing
15//! foreign links are moved aside (`<name>.corrupt-aNNN-K`, never deleted);
13//! files without a journal entry (crash between publish and journal) are
16//! existing files without a journal entry (crash between publish and journal)
14//! compared with the regenerated bytes and kept when identical.
17//! are compared with the regenerated bytes and kept when identical.
15use crate::digest;
18use crate::digest;
16use serde_json::Value;
19use serde_json::Value;
17use std::{
20use std::{
@@ -29,11 +32,84 @@ pub const IN_PROGRESS_PREFIX: &str = "inprogress-";
29/// Prefix of a previous result folder moved aside during the final swap.
32/// Prefix of a previous result folder moved aside during the final swap.
30pub const REPLACED_PREFIX: &str = ".replaced-";
33pub const REPLACED_PREFIX: &str = ".replaced-";
31const MAX_RENDER_JSON: u64 = 1024 * 1024;
34const MAX_RENDER_JSON: u64 = 1024 * 1024;
32/// Subfolder of a result folder with every loose per-frame map folder (Tim t993u).
35/// Logical prefix of per-cell map paths in the journal and `.ppda` (identity).
36/// Finished result folders no longer keep these files (Tim t1065u).
33pub const RAW_DIR: &str = "raw files";
37pub const RAW_DIR: &str = "raw files";
38/// On-disk checkpoint root inside an in-progress run: cells at
39/// `checkpoint/raw files/…` so Resume works without loose finals (Tim t1065u).
40pub const CHECKPOINT_DIR: &str = "checkpoint";
34/// Completion marker of a build; lives only in the in-progress folder.
41/// Completion marker of a build; lives only in the in-progress folder.
35pub const COMPLETE_MARKER: &str = "complete.json";
42pub const COMPLETE_MARKER: &str = "complete.json";
36
43
44/// Physical path of a journaled cell (`raw files/…`) under the run root.
45pub fn cell_path(root: &Path, logical_rel: &str) -> PathBuf {
46 root.join(CHECKPOINT_DIR).join(logical_rel)
47}
48
49/// Relative path used with [`crate::production::publish_bytes`].
50pub fn cell_rel(logical_rel: &str) -> String {
51 format!("{CHECKPOINT_DIR}/{logical_rel}")
52}
53
54/// Find a cell on disk: legacy `raw files/…` at the run root, else checkpoint/.
55pub fn find_cell_file(root: &Path, logical_rel: &str) -> Option<PathBuf> {
56 let legacy = root.join(logical_rel);
57 if legacy.is_file() {
58 return Some(legacy);
59 }
60 let checkpoint = cell_path(root, logical_rel);
61 checkpoint.is_file().then_some(checkpoint)
62}
63
64/// Remove checkpoint (and any legacy root-level `raw files/`) after atlases
65/// are verified. Finished folders must not keep loose per-view maps.
66pub fn remove_cell_checkpoints(root: &Path) -> Result<(), String> {
67 for name in [CHECKPOINT_DIR, RAW_DIR] {
68 let path = root.join(name);
69 if path.is_dir() {
70 rescue_checkpoint_artifacts(&path, root)?;
71 }
72 match fs::remove_dir_all(&path) {
73 Ok(()) => sync_dir(root),
74 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
75 Err(e) => return Err(format!("checkpoint opschonen ({name}): {e}")),
76 }
77 }
78 Ok(())
79}
80
81/// Move leftover `.corrupt-*` / `.part-*` files to the run root before wiping cells.
82fn rescue_checkpoint_artifacts(from: &Path, root: &Path) -> Result<(), String> {
83 let mut stack = vec![from.to_path_buf()];
84 while let Some(dir) = stack.pop() {
85 for entry in fs::read_dir(&dir).map_err(io)? {
86 let entry = entry.map_err(io)?;
87 let path = entry.path();
88 let file_type = entry.file_type().map_err(io)?;
89 if file_type.is_dir() {
90 stack.push(path);
91 continue;
92 }
93 if !file_type.is_file() {
94 continue;
95 }
96 let name = entry.file_name().to_string_lossy().to_string();
97 let keep = name.contains(".corrupt-a") || name.contains(".part-");
98 if !keep {
99 continue;
100 }
101 let mut dest = root.join(&name);
102 let mut n = 0u32;
103 while dest.exists() {
104 dest = root.join(format!("{name}.rescued-{n}"));
105 n += 1;
106 }
107 fs::rename(&path, &dest).map_err(|e| format!("quarantaine bewaren {name}: {e}"))?;
108 }
109 }
110 Ok(())
111}
112
37fn has_ppda(dir: &Path) -> bool {
113fn has_ppda(dir: &Path) -> bool {
38 fs::read_dir(dir).is_ok_and(|read| {
114 fs::read_dir(dir).is_ok_and(|read| {
39 read.flatten()
115 read.flatten()
@@ -49,9 +125,9 @@ pub fn is_complete(dir: &Path) -> bool {
49}
125}
50
126
51/// Remove render/resume/lock bookkeeping from a completed, unlocked build so
127/// Remove render/resume/lock bookkeeping from a completed, unlocked build so
52/// the result folder holds only consumer output (`.ppda`, atlases,
128/// the result folder holds only consumer output (`.ppda` + atlases).
53/// `raw files/`). The `.ppda` carries everything the reader still needs.
129/// The `.ppda` carries everything the reader still needs (including per-cell
54/// Idempotent and crash-safe: `complete.json` is removed last.
130/// SHA-256). Idempotent and crash-safe: `complete.json` is removed last.
55pub fn strip_bookkeeping(root: &Path) -> Result<(), String> {
131pub fn strip_bookkeeping(root: &Path) -> Result<(), String> {
56 if !is_complete(root) {
132 if !is_complete(root) {
57 return Err("opschonen geweigerd: render is niet voltooid".into());
133 return Err("opschonen geweigerd: render is niet voltooid".into());
@@ -209,22 +285,20 @@ pub fn read_journal(root: &Path) -> Result<(BTreeMap<String, String>, usize, boo
209
285
210/// Move a bad file/link aside inside the run without deleting or clobbering anything.
286/// Move a bad file/link aside inside the run without deleting or clobbering anything.
211pub(crate) fn quarantine(root: &Path, rel: &str, attempt: u32) -> Result<String, String> {
287pub(crate) fn quarantine(root: &Path, rel: &str, attempt: u32) -> Result<String, String> {
212 let path = root.join(rel);
288 // Source may be a checkpoint cell or a root-level atlas; the rescued copy
213 let name = path
289 // always lands at the run root so it survives checkpoint cleanup (t1065u).
290 let path = find_cell_file(root, rel).unwrap_or_else(|| root.join(rel));
291 let base = Path::new(rel)
214 .file_name()
292 .file_name()
215 .and_then(|n| n.to_str())
293 .and_then(|n| n.to_str())
216 .ok_or("ongeldige bestandsnaam")?
294 .ok_or("ongeldige bestandsnaam")?;
217 .to_string();
218 for n in 0..1000 {
295 for n in 0..1000 {
219 let target = path.with_file_name(format!("{name}.corrupt-a{attempt:03}-{n}"));
296 let target_name = format!("{base}.corrupt-a{attempt:03}-{n}");
297 let target = root.join(&target_name);
220 match fs::hard_link(&path, &target) {
298 match fs::hard_link(&path, &target) {
221 Ok(()) => {
299 Ok(()) => {
222 fs::remove_file(&path).map_err(io)?;
300 fs::remove_file(&path).map_err(io)?;
223 let rel_target = Path::new(rel)
301 return Ok(target_name);
224 .with_file_name(target.file_name().unwrap_or_default())
225 .to_string_lossy()
226 .replace('\\', "/");
227 return Ok(rel_target);
228 }
302 }
229 Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => continue,
303 Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => continue,
230 Err(e) => return Err(format!("quarantaine van {rel} mislukt: {e}")),
304 Err(e) => return Err(format!("quarantaine van {rel} mislukt: {e}")),
@@ -364,15 +438,28 @@ impl RunSession {
364 if i % 256 == 0 {
438 if i % 256 == 0 {
365 progress(i, planned.len());
439 progress(i, planned.len());
366 }
440 }
367 let path = root.join(rel);
441 let path = match find_cell_file(&root, rel) {
368 let meta = match fs::symlink_metadata(&path) {
442 Some(path) => path,
369 Ok(meta) => meta,
443 None => {
370 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
444 // Also treat a bare checkpoint/legacy path that is a non-file specially.
371 report.missing += 1;
445 let candidate = cell_path(&root, rel);
372 continue;
446 match fs::symlink_metadata(&candidate) {
447 Ok(meta) if meta.is_dir() => {
448 return Err(format!("map staat op de plaats van PNG {rel}"));
449 }
450 Ok(_) => {
451 report.quarantined.push(quarantine(&root, rel, attempt)?);
452 continue;
453 }
454 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
455 report.missing += 1;
456 continue;
457 }
458 Err(e) => return Err(e.to_string()),
459 }
373 }
460 }
374 Err(e) => return Err(e.to_string()),
375 };
461 };
462 let meta = fs::symlink_metadata(&path).map_err(io)?;
376 if meta.is_dir() {
463 if meta.is_dir() {
377 return Err(format!("map staat op de plaats van PNG {rel}"));
464 return Err(format!("map staat op de plaats van PNG {rel}"));
378 }
465 }
@@ -449,7 +536,7 @@ impl RunSession {
449 }
536 }
450 let sha = digest::sha256_hex(bytes);
537 let sha = digest::sha256_hex(bytes);
451 if self.unjournaled.remove(rel) {
538 if self.unjournaled.remove(rel) {
452 let path = self.root.join(rel);
539 let path = cell_path(&self.root, rel);
453 match fs::symlink_metadata(&path) {
540 match fs::symlink_metadata(&path) {
454 Ok(meta) if meta.is_file() => {
541 Ok(meta) if meta.is_file() => {
455 let existing = fs::read(&path).map_err(io)?;
542 let existing = fs::read(&path).map_err(io)?;
@@ -468,7 +555,7 @@ impl RunSession {
468 Err(e) => return Err(e.to_string()),
555 Err(e) => return Err(e.to_string()),
469 }
556 }
470 }
557 }
471 crate::production::publish_bytes(&self.root, Path::new(rel), bytes)?;
558 crate::production::publish_bytes(&self.root, Path::new(&cell_rel(rel)), bytes)?;
472 self.record(rel, &sha)?;
559 self.record(rel, &sha)?;
473 Ok(sha)
560 Ok(sha)
474 }
561 }
src/session.rs +1 −1
@@ -402,7 +402,7 @@ impl Session {
402 402 .map(PathBuf::from)
403 403 }
404 404
405 fn ok(&mut self, message: &str) {
405 pub fn ok(&mut self, message: &str) {
406 406 self.notice = message.to_string();
407 407 self.notice_ok = true;
408 408 }
@@ -402,7 +402,7 @@ impl Session {
402 .map(PathBuf::from)
402 .map(PathBuf::from)
403 }
403 }
404
404
405 fn ok(&mut self, message: &str) {
405 pub fn ok(&mut self, message: &str) {
406 self.notice = message.to_string();
406 self.notice = message.to_string();
407 self.notice_ok = true;
407 self.notice_ok = true;
408 }
408 }