mirror of
https://github.com/sudosylabs/vnidrop.git
synced 2026-08-09 20:29:58 +02:00
feat(studio): native SwiftUI + SceneKit App Store screenshot pipeline
Replace the Typst/ImageMagick studio with a SwiftPM executable that composes each marketing screen as a SwiftUI view and renders it headlessly via ImageRenderer to exact 1284x2778 PNGs. - Devices are a real iPhone 17 Pro Max .usdz textured + posed in SceneKit (graphite body, studio IBL); screen curvature/bezel are the model's geometry. - Per-screen layouts (pose, globe, route arc, encryption flow) in ScreenSpec. - send-anywhere: globe + Paris->LA route arc; uses the app's share screen. - stay-private: converging beams -> glowing padlock -> binary protection stream, with localized CHIFFREMENT/PROTECTION banners and an auto-sizing header panel. - Captions auto-shrink per locale (ViewThatFits) so long translations never clip. - Screenshots are transient (generated/shots, git-ignored), captured per locale; generate.sh does capture + composite in one shot. - Commit the .usdz model (CC BY 4.0, see assets/ATTRIBUTION.md) and globe asset. - Remove the superseded Typst pipeline (screens.typ, build.sh, frame.sh, warp.sh).
This commit is contained in:
@@ -1,57 +1,83 @@
|
||||
# App Store screenshot studio
|
||||
|
||||
Code-driven, fully local App Store screenshots. **Typst** composes each frame
|
||||
(gradient background + blank device mockup + app screenshot + localized caption)
|
||||
and renders it to an exact-size PNG. Captions come from `strings.json` (9 locales);
|
||||
app screenshots come from the real app (see capture, step 3).
|
||||
Code-driven, fully local App Store screenshots — composed natively with **SwiftUI +
|
||||
`ImageRenderer`** (no Typst, no ImageMagick). Each frame is a SwiftUI view (gradient
|
||||
background + globe + device with the app screenshot + localized caption) rendered
|
||||
off-screen to an exact-size PNG. Captions come from `strings.json` (9 locales); app
|
||||
screenshots come from the real app (see capture, below).
|
||||
|
||||
Why SwiftUI: the screen curvature is the real iOS squircle
|
||||
(`RoundedRectangle(style: .continuous)`), SF Pro resolves for free, and the hero tilt
|
||||
is a native `.rotation3DEffect` (real perspective) — no mask-guessing or warp math.
|
||||
|
||||
## Requirements
|
||||
- `typst` (0.15+) — installed.
|
||||
- `imagemagick` — only for the tilted hero shot's perspective warp (step 2). `brew install imagemagick`.
|
||||
- Xcode / Swift toolchain (macOS 14+). That's it.
|
||||
|
||||
## Render
|
||||
```sh
|
||||
./build.sh # all locales × screens -> generated/<Language>/
|
||||
./build.sh --publish # -> ../<Language>/ (ships to App Store)
|
||||
LOCALES="fr de" SCREENS="share-securely" ./build.sh
|
||||
swift run studio # all locales × screens -> generated/<Language>/
|
||||
swift run studio --publish # -> ../<Language>/ (ships to App Store)
|
||||
LOCALES="fr de" SCREENS="share-securely" swift run studio # subset
|
||||
```
|
||||
Default writes to `generated/` (safe). `--publish` overwrites the real language folders.
|
||||
|
||||
Single frame while tuning:
|
||||
```sh
|
||||
typst compile screens.typ out.png --input locale=fr --input screen=share-securely --ppi 72
|
||||
```
|
||||
|
||||
## Files
|
||||
- `strings.json` — captions per locale (first-pass translations; review before shipping).
|
||||
- `screens.typ` — the composition + per-screen layout data (`screens` dict). Tweak numbers, re-run.
|
||||
- `build.sh` — loops locales × screens; maps locale→folder and screen→filename.
|
||||
- `assets/mockup-straight.png` — blank straight device (real alpha, black screen).
|
||||
- `assets/mockup-rotated.png` — blank tilted device (for the hero shot).
|
||||
- `assets/globe.png` — circle-clipped in Typst (the export has a baked checkerboard, not true alpha).
|
||||
- `assets/shots/<locale>/<screen>.png` — captured app screenshots (optional; placeholder if absent).
|
||||
|
||||
## How the screenshot gets into the phone
|
||||
The mockup's screen glass is a rectangle. `screens.typ` places the screenshot inset
|
||||
into that rectangle (`sx`/`sy`/`sr` fractions) with the bezel framing it. Measure those
|
||||
fractions once against `mockup-straight.png`.
|
||||
Run from this directory (it reads `strings.json` and `assets/` relative to cwd).
|
||||
|
||||
## Full pipeline
|
||||
```sh
|
||||
./capture.sh # real localized app screens -> assets/shots/<locale>/<screen>.png
|
||||
./build.sh # composite everything -> generated/<Language>/
|
||||
./build.sh --publish # when happy -> ships to ../<Language>/
|
||||
./capture.sh # real localized app screens -> assets/shots/<locale>/<screen>.png
|
||||
swift run studio # composite everything -> generated/<Language>/
|
||||
swift run studio --publish # when happy -> ships to ../<Language>/
|
||||
```
|
||||
`capture.sh` drives the app into each screen via the DEBUG `-VniScreenshot` launch
|
||||
argument (see `apple/VniDrop/App/ScreenshotSupport.swift`), once per locale via
|
||||
`-AppleLanguages`, in dark mode with a 9:41 status bar. Screen ⇄ scenario map:
|
||||
`share-securely`→`share`, `choose-receivers`→`approval`, `send-anywhere`→`transfer-details`.
|
||||
`-AppleLanguages`, in dark mode with a 9:41 status bar.
|
||||
|
||||
## Device rendering (3D by default)
|
||||
The device is a real iPhone 17 Pro Max `.usdz` (`assets/iphone-17-pro-max.usdz`, CC BY
|
||||
4.0 — see `assets/ATTRIBUTION.md`). `SceneKitDeviceRenderer` textures the app screenshot
|
||||
onto the screen mesh and renders it off-screen with SceneKit. The screen curvature,
|
||||
bezel and Dynamic Island are the model's real geometry; the hero tilt is a real camera
|
||||
perspective (`spec.device.tilt`), not a warp.
|
||||
|
||||
Model-specific quirks handled in `SceneKitDeviceRenderer.swift` (re-derive if the model
|
||||
changes — see the inspection scripts approach in git history):
|
||||
- **Warm-up render**: `SCNRenderer.snapshot` returns empty on its first call; we render
|
||||
a throwaway 16×32 first, then the real frame.
|
||||
- **Bounding box**: use `rootNode.boundingBox` (the model is ~16 units tall via an
|
||||
ancestor scale). `flattenedClone()` collapses this model to nothing — don't use it.
|
||||
- **Camera orientation**: front is −Z; `look(at:)` renders nothing, so the camera is
|
||||
yawed 180° by hand (`eulerAngles`).
|
||||
- **Crisp screen**: the screen material has a wavy normal map and there's a front glass
|
||||
mesh with its own waviness — both ripple the image. We clear the screen normal and
|
||||
hide the glass mesh so the app content stays flat/crisp (App Store requirement).
|
||||
|
||||
Set `DEVICE=2d` to fall back to the flat mockup compositor (`ClipDeviceRenderer` +
|
||||
`assets/mockup-straight.png`, `.continuous` clip) if you ever want the non-3D path.
|
||||
|
||||
## Assets (git-ignored except the model — you supply the rest)
|
||||
- `assets/iphone-17-pro-max.usdz` — the 3D device (committed, with `ATTRIBUTION.md`).
|
||||
- `assets/globe.png` — transparent globe for the hero.
|
||||
- `assets/shots/<locale>/<screen>.png` — captured app screenshots (from `capture.sh`).
|
||||
- `assets/mockup-straight.png` — only needed for the `DEVICE=2d` fallback.
|
||||
|
||||
## Layout & tuning
|
||||
- `Sources/studio/ScreenSpec.swift` — per-screen layout (gradient stops, caption
|
||||
placement, globe + device position/size, hero tilt). Numbers are in pixels at scale 1.
|
||||
- `MockupGeometry.straight` in the same file — where the glass sits inside
|
||||
`mockup-straight.png` (fractions of the mockup) + corner radius. **Measure once**
|
||||
against your mockup export and set these; placeholders are approximate.
|
||||
- `Sources/studio/ScreenFrame.swift` — the composition (layer order, caption styling).
|
||||
- `Sources/studio/DeviceView.swift` — the device layer. `ClipDeviceRenderer` does the
|
||||
2D `.continuous` clip today; a `SceneKitDeviceRenderer` (texture the shot onto a real
|
||||
iPhone `.usdz`, render with `SCNRenderer.snapshot()`) can drop in behind the same
|
||||
`DeviceRenderer` protocol for full 3D — nothing else changes.
|
||||
|
||||
## Status
|
||||
1. ✅ Typst + JSON + straight-mockup compositing.
|
||||
2. ✅ Tilted hero (`send-anywhere`): `warp.sh` perspective-warps the screenshot onto the
|
||||
rotated mockup's screen quad (4 auto-detected corners); `build.sh` runs it automatically.
|
||||
3. ✅ Transparent globe layer (placed directly — use the alpha export, not a flattened one).
|
||||
4. ✅ Screenshot capture: `capture.sh` + the `#if DEBUG` fixture gateway, per locale, deterministic.
|
||||
5. ⬜ Ribbon art layer for `stay-private` (export as its own transparent PNG).
|
||||
6. ⬜ Layout polish: tune device positions/sizes in `screens.typ` against the originals.
|
||||
1. ✅ SwiftUI compositor: gradient + caption + globe + device, exact 1284×2778, headless.
|
||||
2. ✅ Real 3D iPhone: screenshot textured onto the `.usdz` screen mesh (SceneKit), crisp.
|
||||
3. ✅ Real hero tilt via 3D camera perspective (replaces `warp.sh` + rotated mockup).
|
||||
4. ✅ Screenshot capture: `capture.sh` + the `#if DEBUG` fixture gateway, per locale.
|
||||
5. ⬜ Tune device position/size/tilt per screen in `ScreenSpec.swift` against the originals
|
||||
(esp. `send-anywhere`: caption overlaps the phone; globe layer needs its asset).
|
||||
6. ⬜ Ribbon art layer for `stay-private`.
|
||||
7. ✅ 2D fallback (`DEVICE=2d`) retained for the flat mockup path.
|
||||
|
||||
Reference in New Issue
Block a user