Muse

Summary

Streaming platforms organize songs by genre and vibe, but they miss the emotional context of listening: the way we use music for self-discovery, emotional processing, focus, and connection. Muse fills that gap. It combines audio analysis, visual representation, and generative AI into a multi-modal music experience, built for a hackathon's Multi-Modality challenge.

How it works

• Audio → features. librosa extracts spectral, harmonic, and rhythmic signatures (MFCC, chroma, onset strength, spectral centroid, zero-crossing rate) from real-time YouTube audio.
• Features → emotions. Custom algorithms map those features onto six emotional axes: Valence, Energy, Tension, Warmth, Power, and Complexity.
• Emotions → visuals. Nano Banana Pro renders high-fidelity 2K emotional timelines and custom artwork reflecting the song's emotional fingerprint.
• Emotions → context. Gemini 3 Pro reads the emotional + audio profile together to suggest the perfect locations, activities, time of day, and mood match.

The six emotional axes

Plus Power (intimate ↔ epic) and Complexity (simple ↔ intricate). Together they go beyond simple "happy/sad" to capture the nuance of how a track actually feels. A timeline view shows how those emotions evolve across the song.

Where it could go

Muse's emotional profiling can power smart playlist curation ("calm-but-confident pre-presentation hype music"), music therapy and emotional processing, shared "our songs" playlists, emotion-aware focus playlists, and personal music journaling that surfaces how your inner life evolves, not just your listening history. The end state is an API for emotional music search that developers can build on.