Skip to content

feat: audio choreography skill ref#227

Draft
vanceingalls wants to merge 1 commit intomainfrom
vance/audio-choreography
Draft

feat: audio choreography skill ref#227
vanceingalls wants to merge 1 commit intomainfrom
vance/audio-choreography

Conversation

@vanceingalls
Copy link
Copy Markdown
Collaborator

@vanceingalls vanceingalls commented Apr 8, 2026

What

Adds the audio-choreography.md skill reference and positioning safety rules to SKILL.md. Teaches the agent how to sync visuals to music structure using audio analysis data, and prevents common layout bugs.

Why

When building compositions with music, the agent needs guidance on how to interpret audio analysis output (beats, energy phases, builds, frequency bands) and translate that into timed visual reactions. Without this, compositions feel disconnected from the music — reveals land at arbitrary times instead of on beats, energy shifts go unacknowledged, and builds have no visual tension.

The positioning rules address a recurring issue where text elements overlap because the agent doesn't calculate actual element bounds before placing the next one. Multi-line text at large font sizes is the most common failure case.

Changes

New file: skills/hyperframes/references/audio-choreography.md

A reference loaded on demand when compositions have audio. Covers:

  • Every analysis line → visual reaction — structure phases, key moments, silences, builds, accent patterns, and beats all must produce visible responses
  • Audio analysis script — Python script that extracts 7-band frequency analysis, beat detection, build detection, accent patterns, and structural phases from any audio file
  • Do Not list — explicitly bans patterns that produced poor results during testing (word walls, energy-to-font-size tables, forced constant motion, prescriptive first-drop formulas)
  • Exaggerate energy shifts — surges and drops need proportionally dramatic visual changes (scale, speed, color, density)
  • Build before every drop — gradient compression, reveal density acceleration, scale compression, easing acceleration before key moments
  • Frequency bands shape the feel — sub-bass = weight/gravity, highs-only = crisp/snappy, bass-dominant = thick/warm

Updated: skills/hyperframes/SKILL.md

Added positioning safety rules under "Rules (Non-Negotiable)":

  • Centering text — use width: 100%; text-align: center, not translateX(-50%) which breaks with GSAP x animation
  • GSAP x/y are relative offsets — not screen coordinates, never use values above 300 for content slides
  • Preventing overlap — calculate bottom edge = top + fontSize × lineHeight × lines, then verify no ranges overlap across simultaneously visible elements
  • Safe zones — 40px inset from all edges, baselines above data-height - 80

Added reference link to audio-choreography.md in the References section.

Fix: packages/cli/src/commands/validate.ts

Minor fix in the validate command.

Test plan

  • Built 5 compositions against the reference with varied audio (60s-264s, 86-172 BPM, sparse to dense)
  • Verified the "Do Not" list prevents banned patterns when agents read the reference
  • Confirmed positioning rules catch the multi-line overlap bug (200px × 2 lines)
  • All compositions pass npx hyperframes lint and npx hyperframes validate

Copy link
Copy Markdown
Collaborator Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@vanceingalls vanceingalls force-pushed the vance/audio-choreography branch from 60ed224 to 0737462 Compare April 8, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant