Introduction

The Magic Flute was the main project I've worked on while at Classplash. I was responsible for most of the programming of the game, visual effects/animations, as well as designing different game modes. I've also done minor writing on the script adaptation to fit the animated story scenes and general editing and rephrasing.

It's the official game companion to the movie The Magic Flute (2022). It's a fairly straightforward rhythm game, not unlike a Guitar Hero, although it uses the microphone to hear what you're playing on a real recorded instrument. We've also decided to implement touch controls as an alternative, unlike in the past games. The music comes mostly from Mozart's opera The Magic Flute (with a couple of extra Mozart or more modern pop songs thrown in related to the movie).

You can get the game on Android or iOS. The first three or four songs are free (though very simple), while the remaining of the game is behind a single payment.

Game Design

Boost Mode

The game consists of colored notes coming from the left side that the player must play as they hit the flute target on the right side. The game was, at least in part, a way to teach young players how to play a real recorded and introduce them to some of the aspect of sheet music. Initially, every note is color-coded and has a schematic of its finger positions. To make the sheet reading more salient, these visual aids will gradually disappear when the player performs well, until the maximum boost mode is reached, at which point the notes lose all color and only their sheet position will determine the pitch that must be played. Missing notes makes it so that the visual aids reappear.

One important feature that was implemented (but later discarded due to production issues) was a vertical reorchestration of the songs to accompany the boost mode. The backing tracks would start basic, with few layers, and the better the player performed, the more layers would be added, resulting in a richer sound, culminating in the full orchestra only in the maximum boost mode. Similar to the way the BIT.TRIP games work.

Training Modes

While the basic play mode works well for simple songs, it soon becomes too difficult when fast sections are introduced, as the visual aids are good mnemonics, but not something to be readable in real-time by novices.

To address this problem, I've designed and implemented a solution using a MIDI backing track that would wait for the player to play the correct note before advancing. The reason MIDI was used was to maintain the "flow" of the song. When using a real orchestrated track, waiting for the player would mean silencing the track completely, which was not satisfying. With MIDI, it was possible use simpler strings that were sustained while waiting for the player.

Alternative controls

Being a movie tie-in, it would make little sense to focus exclusively on the microphone input as in previous games, so I designed and implemented touch and keyboard controls.
In the case of touch controls, every note in an octave is displayed in a row, while the higher octave is achieved by holding a "shift-like" key on either side of the controller (to be used with either hand depending on the target note).
In the case of keyboard controls, the scheme is similar, although two rows of keys are used, to mimick a piano with a row of whole tones and another of semitones. The shift key became space bar, as it is very natural to press it with the thumb of either hand.

Visual Effects

Game

To spice the game up a bit, I've created several visual effects by using particle systems and custom shaders applied to the sheet lines and flute when notes are hit and sustained.
The best way to get a feel for it is through videos, so I've embedded three below. The first is an example of successful play (through the whole boost range), the second an unsuccessful play (losing boost), while the last is an example of gameover.

I've also created a shader to create a "mosaic-like" effect on a background, dynamically.
This was done by using a voronoi texture to sample the background texture so that each cell would retain a single color. The background textures were the same as used in the story sequences, but strongly down-sampled and blurred to create a smooth effect.
This is also used for the game over effect, in which darkness encroaches on the flute.

Story

In the story, too, I've created several visual effects, using noise textures, particle effects and distance fields.
Some of the effects were applied to characters:

While others were applied to the background:

Book

Tim's father's book plays an important role in the movie, so we've implemented an in-game book for the tutorials. Render textures were used to map the page contents onto the book pages, which were animated using simple 2D masks, thus bypassing any need for additional 3D modelling and the like. Not to take skeuomorphism too far, we've kept some interactive elements like buttons and sliders to avoid cumbersome interaction.

Others

There was also the opportunity for mini-game experiments during pre-production. Out of which came some things I quite like, even though they were never used.
I'm particularly proud of the eye, created from scratch in a shader, and the moth/firefly movement; it starts randomly (though in a very controlled manner, after playing with ideas by Craig Reynolds of boids fame), but becomes gradually attracted to the lit areas (which were controlled by the flute note).