Muso

Turn daily objects into instruments in MR

Encourage Creativity & Playfulness

Tech-center Research

C Sharp

Unity

VR/AR Development

Although haptic feedback is part of every physical interaction, it is mostly neglected due to its complex design process. Hapticlabs.io is a toolset to democratise haptic feedback, allowing designers to identify opportunities and easily create functional prototypes. It consists of two parts: A prototyping kit which does not require any expertise in coding or electronics, as well as an online knowledge base, which provides easy to follow background information and guidelines.

RECOGNITION & AWARDS

MUSO is an XR music synthesizer that encourages children to turn ordinary everyday objects into fascinating musical instruments in XR. That feeling captures the essence of childhood wonder and the limitless potential of the imagination. MUSO offers users such a musical journey while introducing them to the basics of music theory. MUSO believes that music learning should be easy and fun, and hopes to bridge the gap in children's music education through a low-cost, entertaining approach.

YEAR

2023

PROJECT TYPE

Academic

OTHER CONTRIBUTOR

Monica Zhang, Hattie Li

Make Technology
Achievable.

Tech-center Research

C Sharp

Unity

VR/AR Development

MUSO is a XR game that aims to encourage creativity and playfulness by showing how ordinary objects can be transformed into musical instruments. By demonstrating how any item in daily life can be repurposed for musical expression, we hope to inspire people to think outside the box and explore the musical potential of their surroundings.

MUSO’s goal is sparking a sense of wonder and curiosity in kids, and helping them discover the joy of making music in unexpected ways.

RECOGNITION & AWARDS

YEAR

2023

PROJECT TYPE

Academic

OTHER CONTRIBUTOR

Monica Zhang, Hattie Li

01 The Core Idea

MUSO believes that music learning shouldn't be boring and inaccessible, by blending education with imagination and creativity, bridging the gap with affordable, engaging method. We aim to tackle <10% access to music education in the developing world.

MUSO believes that music learning shouldn't be boring and inaccessible, by blending education with imagination and creativity, bridging the gap with affordable, engaging method. We aim to tackle <10% access to music education in the developing world.

This video explains our pain points, mission, and key features with in 90 seconds.

02 The Problem

According to the statistics of 2022, the popularization rate of music education in developing countries is less than 10%. In China, for example, there is a general lack of music education for children due to factors such as the cost of music learning, the level of teachers, and the lack of widespread popularization and use of professional, effective, and fun children's music pedagogies for music initiation.

What do Chinese young people think of when it comes to musical initiation? Learning to play the piano or the violin, which in turn conjures up images of children crying by the side of their instruments.

Sadly, this is what young Chinese people have in mind when they think of music education...

Sadly, this is what young Chinese people have in mind when they think of music education...

According to the statistics of 2022, the popularization rate of music education in developing countries is less than 10%. In China, for example, there is a general lack of music education for children due to factors such as the cost of music learning, the level of teachers, and the lack of widespread popularization and use of professional, effective, and fun children's music pedagogies for music initiation.

What do Chinese young people think of when it comes to musical initiation? Learning to play the piano or the violin, which in turn conjures up images of children crying by the side of their instruments.

So, what is MUSO

MUSO is an XR music synthesizer that empowers kids to explore music by transforming surrounding everyday objects into magical instruments. Teaching knowledge while kids play in XR.

Go find any object with a circle surface

Use your finger to create a detection area for the surface you want to play with

Undo or resize the selection area if you need

Tap to play! And learn music knowledge

MUSO is an XR music synthesizer that empowers kids to explore music by transforming surrounding everyday objects into magical instruments. Teaching knowledge while kids play in XR.

Go find any object with a circle surface

Use your finger to create a detection area for the surface you want to play with

Undo or resize the selection area if you need

Tap to play! And learn music knowledge

03 Key Features

Unlocking Creativity

MUSO's encourage kids to observe and experiment day life. By recognize the shape, size of a object, MUSO can turn it into playful instrument.

Easy to start
‍
Using MR recognition technology, a round bowl can be transformed into a percussion in just three steps.

Learn through play

MUSO uses fun animated characters to teach music theory while children play with "instruments".

Go wild with exploration!

Create your brand, do your experiments!

Unlocking Creativity

MUSO's encourage kids to observe and experiment day life. By recognize the shape, size of a object, MUSO can turn it into playful instrument.

Easy to start
‍
Using MR recognition technology, a round bowl can be transformed into a percussion in just three steps.

Learn through play

MUSO uses fun animated characters to teach music theory while children play with "instruments".

Go wild with exploration!

Create your brand, do your experiments!

04 Development Hightlights

The whole project was development in Unity, using Oculus Quest 2 and Oculus Quest Pro as the testing device. Meanwhile, MUSO continues to iterate after user interviews and user testing.

Even though this isn't my first project in C#, I'd like to share the funny things I've learned about struggling to implement the code logic in order to achieve functionality. :)_

Behind the scenes at Immersion Lab: ideation, development, testing, filming, editing...

Behind the scenes at Immersion Lab: ideation, development, testing, filming, editing...

The whole project was development in Unity, using Oculus Quest 2 and Oculus Quest Pro as the testing device. Meanwhile, MUSO continues to iterate after user interviews and user testing.

Even though this isn't my first project in C#, I'd like to share the funny things I've learned about struggling to implement the code logic in order to achieve functionality. :)_

Behind the scene

Unity Open Source Demo

Enable Gesture Control!

The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.

Visual Effects

We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.

Physical Computing

We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.

User Testing & Iterations

During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.

Works

About

Resume