MeloTune: On-Device Arousal Learning and Peer-to-Peer Mood Coupling for Proactive Music Curation
MeloTune is transforming the way users experience music by utilizing advanced AI techniques to enhance music curation based on individual emotional states. Deployed on iPhone devices, MeloTune leverages the Mesh Memory Protocol (MMP) and Symbolic-Vector Attention Fusion (SVAF) to create an affect-aware music agent that not only understands personal preferences but also adapts to the moods of peers in real-time.
Key features of MeloTune’s innovative approach include:
- On-Device Processing: The system runs entirely on the user’s device, ensuring privacy and security by keeping sensitive data local.
- Continuous-Time Networks: MeloTune employs two closed-form continuous-time (CfC) networks that facilitate dynamic predictions of emotional trajectories.
- Personalized Arousal Function: This function customizes the mapping from audio intensity to psychological arousal, leading to a more personalized music experience.
- Peer-to-Peer Mood Coupling: The system integrates Cognitive Memory Blocks (CMBs) from co-listening peers, allowing for shared musical experiences based on collective emotional states.
The first of the two networks is a private listener-level CfC, which predicts a short-horizon affective trajectory based on Russell’s circumplex model of emotions. This enables proactive curation of music that aligns with the listener’s evolving emotional state. The second network operates at MMP Layer 6 and is designed for shared listening experiences, seamlessly integrating inputs from multiple users without compromising individual privacy.
One of the standout features of MeloTune is its Personal Arousal Function (PAF), which replaces the traditional linear mapping approach. This innovative function is trained using behavioral signals, such as skips, completions, favorites, and volume adjustments, along with discrepancies between user-declared moods and machine-generated inferences. This allows MeloTune to provide customized arousal predictions for each listener, ensuring that the same track can evoke different emotional responses depending on the listener’s unique context.
In terms of performance, the model boasts an impressive configuration with 94,552 parameters. Validation metrics reveal a Mean Absolute Error (MAE) of 0.414 for trajectory predictions, 96.6% pattern accuracy, and 69.4% intent accuracy on held-out datasets. Notably, evidence from a live deployment session comprising 46 observations across 11 genres demonstrated the effectiveness of the learning loop, with pop music reaching full confidence after just 22 observations.
All inference processes are executed on-device through CoreML, ensuring swift response times and an enhanced user experience. To date, MeloTune represents a pioneering effort in deploying MMP/SVAF technology on consumer mobile hardware, marking a significant advancement in the field of music curation powered by artificial intelligence.
The accompanying Software Development Kit (SDK), sym-swift v0.3.78 and SYMCore v0.3.7, guarantees adherence to strict protocol conformance, making it easier for developers to create applications that leverage MeloTune’s capabilities. As the music industry increasingly embraces AI, MeloTune stands at the forefront of innovation, promising a richer, more emotionally resonant listening experience for users worldwide.