Rezmo is an Interactive Media Arts Senior Capstone Project at New York University Shanghai (NYU Shanghai). Taking the form of a 2D rhythm platformer, the game ties music rhythm with action game mechanics while simulating the condition of hearing loss via a built-in Slient Mode that allows players to experience gameplay with only visual aids and controller vibrations.

Platform: Unity with Dualshock Controllers


Individual Project: All art assets and sound tracks are original work in addition to programming.

As the programmer, I

  • Implemented the core rhythm-action gameplay mechanic that affects players’ platforming abilities with music rhythm using the DrXoo Beat Detection plugin for Unity 3D.
  • Created a reusable modularized level framework to expedite level creation. As a result, 2 complete levels were fleshed out during the course of one semester.

As the designer, I

  • Conceptualized and designed 2 levels for the game that gradually introduce players to the gameplay mechanic and the story.
  • Designed an alternative gameplay mode that simulates deaf experience when compared with the default gameplay mode.
  • Organized 3 playtests and iterated on the gameplay based on the feedback.

As the sound designer & artist, I

  • Created 2D art assets for characters, animations, and the environment of the game.
  • Created, edited, and implemented sound effects and original soundtracks for the game.

Making of Rezmo

Concept Origin

The concept of Rezmo first emerged in response to a question raised by the trade-off between entertaining games and serious games: is there an approach to convey educational messages with video games while still preserving as many mainstream game elements as possible to make the game fun?

Designed to respond to this design challenge, Rezmo is an exciting action game that players would enjoy playing as well as a game that conveys a message about being deaf. The game provides players with insights into how rhythm games can be more intriguing than the convention of merely pressing buttons at the right time. While players will experience the gameplay both with and without audio, they will also experience a simulated hearing loss and be inspire to reflect on the situation of people with hearing loss.


Designing Gameplay Mechanics

One key challenge to the design of Rezmo is the balance between player freedom and the necessity to follow music rhythm. In order to make sure the music rhythm doesn’t constrain player agency—which is often what other conventional rhythm games do—while maintaining the important role of music rhythm, I made the decision that synchronizing with beats boosts action effects instead of triggering actions. This means that the more accurate players can synchronize jumping and shooting with beats, the higher they will be able to jump or the more powerful projectiles they will be able to shoot. On the contrary, failing to do so won’t result in completely penalty as players will still be able to jump very low or shoot weak projectiles when they are off the beat.

Early Mechanic Prototype-Platforms Early Mechanic Prototype-Enemies Low jump when fail to synchronize with beats. High jump when successfully synchronized with beats. Small and weak projectiles when fail to synchronize with beats. Big and strong projectiles when successfully synchronized with beats.

Such a design choice will create an illusion of freedom for players that they have complete control, until they realize at certain points of a level that they have to rely on synchronization and the ability boost to proceed. This illusion of freedom and the challenging rhythm synchronization of Rezmo are designed to craft an entertaining gameplay.


Designing the Social Message

Apart from the fun aspect produced by the rhythm-action hybrid mechanics, Rezmo also aims to convey a social message, which in this case is for hearing players to reflect on the situation of the deaf community. Given the nature of rhythm games, both visual and audio indicators of music beats are essential to the gameplay. Therefore, taking away audio components in the gameplay would be an effective simulation of hearing loss while still ensuring the playability of the game and even creating additional challenge.

When simulating the hearing loss in the level, I decide to strengthen the visual indicators and controller vibration in that level to simulate the scientific fact that the human brain boosts other senses when it underwent sensory loss. By leading players to master visual and touch methods while lacking the hearing ability, Rezmo is capable of offering skill mastery opportunities which is more effective than experiencing frustration and pain in a disability simulation.


Level Design: Introducing Mechanics Through Levels

Rezmo takes the form of a platformer. Therefore, level design is essential to the making of the game. When designing levels, I aimed to create “rhythm groups” that would provide introductions to mechanics, opportunities for players to practice, and cadences for players to rest. The concept of “rhythm groups” refers to an area of a level consisting of platforms, collectable items, obstacles/enemies, etc. With each rhythm group having a beginning, middle, and end, they all come together to form the overall experience and interest curve of the bigger level.

Fox example, the purpose of the first level of the game is to introduce basic game mechanics such as moving, jumping, shooting, and synchronizing with music. So I divided the entire level to several rhythm groups, each of which serving a unique purpose: Introduce Moving, Introduce Jumping, Practice on Jumping, Introduce Shooting, and Practice on Shooting. Each rhythm group starts with a set of easy platforms and enemies followed by another set that’s more challenging. As this level serves as the tutorial, I also designed the level in a way that won’t be to punishing for players who fail to perform the actions. For instance, at the places where players have to perform jumping in synchronization with music to get through, I set an automatic rest or a hidden long detour that doesn’t require much jumping in case players fail.

Similar to how the first level introduces basic game mechanics, the second level of the game introduces four different kinds of enemies, each of which behave differently. Following the same philosophy, I divided the second level to rhythm groups that Introduce Enemy 1, Practice with Enemy 1, Introduce Enemy 2, Practice with Enemy 2, etc. Since those rhythm groups are more intense and challenging than those in level 1, I also added a small empty area between encountering each enemy type, serving as a cadence for players to rest before going on.

The overall level design process was done with sketches on paper first, and was then recreated in Unity game engine with squares and sample assets. The art assets were imported for the level at a later stage when the design was almost set.