Skip to content

feat(media-control): add module with Linux MPRIS, macOS Now Playing, Windows SMTC#203

Merged
kdroidFilter merged 10 commits intomainfrom
feat/media-control
Apr 16, 2026
Merged

feat(media-control): add module with Linux MPRIS, macOS Now Playing, Windows SMTC#203
kdroidFilter merged 10 commits intomainfrom
feat/media-control

Conversation

@kdroidFilter
Copy link
Copy Markdown
Owner

@kdroidFilter kdroidFilter commented Apr 16, 2026

Summary

New media-control runtime module exposing OS-level media controls (play/pause, next/previous, seek, metadata, artwork) through a single cross-platform Kotlin API, backed by three native backends.

Cross-platform API

  • MediaControlService (singleton) with configure(), setMetadata(), setPlaybackState(), setVolume(), attach(callback), detach(), isAvailable().
  • Data types: MediaMetadata, MediaPlaybackState, MediaPlaybackStatus.
  • Sealed MediaControlEvent: Play / Pause / Toggle / Next / Previous / Stop / SeekBy(offsetMs) / SetPosition(positionMs) / SetVolume(volume) / OpenUri(uri) / Raise / Quit.
  • Callback dispatched on the Swing EDT — safe to mutate Compose/Swing state directly.
  • Event JSON decoded with kotlinx-serialization-json.

Linux — MPRIS D-Bus

  • C/JNI bridge over GLib/GIO (GDBus). Dedicated thread with its own GMainContext + GMainLoop hosting /org/mpris/MediaPlayer2 root + Player interfaces.
  • PropertiesChanged auto-emitted on metadata/status/volume updates; Seeked signal on position changes.
  • Integrates with GNOME Shell / KDE Plasma media widgets, lock screen, playerctl, media keys.

macOS — MPNowPlayingInfoCenter + MPRemoteCommandCenter

  • Objective-C/JNI bridge over MediaPlayer.framework. GCD blocks on the main queue.
  • Metadata published via nowPlayingInfo (title / artist / album / duration / artwork). Artwork URLs fetched asynchronously.
  • Remote command block handlers emit Play / Pause / Toggle / Next / Previous / Stop / SetPosition.
  • Integrates with Control Center, the Now Playing menu-bar widget, and media keys.

Windows — SystemMediaTransportControls (SMTC)

  • C++/WRL JNI bridge via WinRT Windows.Media.SystemMediaTransportControls.
  • Hidden helper window owned by a dedicated STA-initialized thread (RoInitialize(RO_INIT_SINGLETHREADED)) pumping Windows messages so WinRT events can marshal back.
  • AUMID resolved from NucleusApp.aumid ("com.app.<packageName>" injected by the Nucleus plugin — identical to the AUMID the plugin stamps on the NSIS/MSI Start Menu shortcut). For APPX/MSIX packages (ExecutableRuntime.isAppX()) the explicit AUMID is skipped so Windows uses the package manifest identity.
  • Classic installer (NSIS / jpackage) safety net: configure() also patches the Start Menu shortcut {displayName}.lnk to carry the same AUMID in System.AppUserModel.ID — a no-op when the plugin has already written it, which it does.
  • Events: Play / Pause / Next / Previous / Stop / SetPosition via ButtonPressed + PlaybackPositionChangeRequested. FastForward / Rewind buttons map to SeekBy(±10s).

Build & packaging

  • Pre-built native binaries shipped for 6 architectures: linux-x64, linux-aarch64, darwin-x64, darwin-aarch64, win32-x64, win32-aarch64.
  • Gradle tasks buildNativeLinux, buildNativeMacos, buildNativeWindows wired into processResources.
  • GraalVM: reachability-metadata.json using the modern schema (reflection[].type + jniAccessible: true) — resources picked up by the existing nucleus/.* glob in graalvm-runtime.
  • CI: full build + verify + upload steps in build-natives.yaml for all three OS jobs; download + EXPECTED entries added to pre-merge, publish-maven, publish-plugin, release-graalvm, test-graalvm, test-packaging.

Sample & docs

  • example/MediaControlScreen.kt: transport controls, position/volume sliders, live event log; auto-detects backend and shows the platform-appropriate label.
  • docs/runtime/media-control.md: backend internals (GLib main-loop, GCD, STA window pump), event tables, system integration surfaces per OS, ProGuard rules, AUMID/shortcut notes for NSIS vs. APPX.
  • mkdocs.yml nav updated, README.md + docs/runtime/index.md index entries, llms.txt / llms-full.txt regenerated.

Test plan

  • ./gradlew :media-control:compileKotlin passes on all three OSes
  • ./gradlew :media-control:ktlintCheck :media-control:detekt passes
  • ./gradlew preMerge passes
  • Native binaries build cleanly on Linux (build.sh), macOS (build.sh), Windows (build.bat) — x64 + aarch64 each
  • Linux: playerctl --player=<dbusName> play-pause / next / position 30 trigger the matching events; Seeked re-emitted on position changes; GNOME/KDE media applets show metadata and respond to transport
  • macOS: ./gradlew :example:run shows the track in Control Center and the menu-bar Now Playing widget; F8 / headset Play-Pause triggers Toggle
  • Windows (runDistributable): volume-flyout media overlay shows app icon + name, hardware media keys work
  • Windows (NSIS-installed): same as above — the plugin-stamped AUMID on the Start Menu shortcut matches NucleusApp.aumid
  • Windows (APPX/MSIX): ExecutableRuntime.isAppX() returns true → package identity used for the media overlay
  • GraalVM native-image build succeeds on all three platforms (test-graalvm.yaml)

New module exposing OS-level media controls (play/pause, next/previous,
seek, metadata, volume) via the MPRIS2 D-Bus specification.

- Kotlin API: MediaControlService, MediaMetadata, MediaPlaybackState,
  MediaControlEvent (sealed)
- Native: C/JNI bridge over GLib/GIO (GDBus), runs a dedicated
  GMainLoop thread hosting /org/mpris/MediaPlayer2 with both the root
  and Player interfaces
- Events deserialized via kotlinx.serialization (1.11.0 added to the
  version catalog)
- Callback dispatched on the Swing EDT by the library itself
- PropertiesChanged auto-emitted on metadata/status/volume changes;
  Seeked signal emitted on position updates
- GraalVM reachability metadata shipped in the JAR
- CI: build + verify + upload in build-natives.yaml; download +
  EXPECTED in pre-merge, publish-maven, publish-plugin,
  release-graalvm, test-graalvm, test-packaging
- Example app: Linux-only MediaControlScreen demo
- Docs: docs/runtime/media-control.md + mkdocs nav entry
@kdroidFilter kdroidFilter marked this pull request as draft April 16, 2026 16:53
kdroidFilter and others added 7 commits April 16, 2026 20:14
Implement Windows System Media Transport Controls (SMTC) backend for media
controls using WinRT/WRL. Includes:

- C++/WRL JNI bridge (nucleus_media_control_windows.dll) with proper COM
  STA apartment initialization for event marshaling
- WindowsBackend integration in MediaControlService
- Windows DLL build steps in Gradle (buildNativeWindows task)
- Native build script with x64 + ARM64 support
- CI workflow steps for build-natives, pre-merge, publish-maven
- GraalVM reachability metadata for NativeWindowsBridge
- Updated example and docs to reference Windows SMTC support

Events supported: Play, Pause, Next, Previous, Stop, SetPosition, SeekBy.
AUMID is auto-derived from displayName to ensure stable app identity.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kdroidFilter kdroidFilter changed the title feat(media-control): add MPRIS D-Bus media controls for Linux feat(media-control): add module with Linux MPRIS, macOS Now Playing, Windows SMTC Apr 16, 2026
kdroidFilter and others added 2 commits April 16, 2026 22:19
- launcher-windows: use FindClass to resolve ThumbBarClickListener interface instead of GetObjectClass(lambda), fixing NoSuchMethodError under GraalVM native-image
- global-hotkey: add ProGuard keep rules for onHotKey static JNI callbacks (Windows/macOS/Linux)
- launcher-windows: add ProGuard keep rules for ThumbBarClickListener implementors

ProGuard shrinks JNI-only symbols because native code references (FindClass/GetMethodID) are invisible to the obfuscator. Kotlin lambdas implementing fun interfaces produce synthetic $$Lambda classes not registered as JNI-accessible under GraalVM.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ol, scheduler

Native code resolves bridges via FindClass(BRIDGE_CLASS) and invokes static
callbacks via GetStaticMethodID. ProGuard can't see these references and was
stripping the classes, causing JVM access violations when toasts activated.
@kdroidFilter kdroidFilter marked this pull request as ready for review April 16, 2026 19:56
@kdroidFilter kdroidFilter merged commit 3343b51 into main Apr 16, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant