Margareta Ackerman
Assistant Professor

408-603-0977, MH 215

ALYSIA: Automated LYrical SongwrIting Application

Margareta Ackerman and David Loker

Welcome to ALYSIA's home page. ALYSIA is a fully data-driven system that utilizes machine learning techniques to make original songwriting accessible to everyone. Further, since it is 100 percent data-driven and does not depend on hard-coded rules, it can be adapted to any musical genre.

Songs created with ALYSIA

Before getting into technical details, check out these three songs created with ALYSIA. The lyrics for the first two songs were written by M. Ackerman, and the third is a reinvention of a 1917 hit song with lyrics by Joseph McCarthy. Given the lyrics, all melodies were written with ALYSIA.

How does it work?

ALYSIA works by taking any lyrics, and providing arbitrarily many melodic accompaniments to which the lyrics can be sang. The user can then choose among these melodies, and combine them in any desired way to create a complete song! This requires no musical expertise, and makes songwriting fast and easy.

We've designed a fully data-driven system that produces audio and rhyme based on any given text. The system utilizes random forests which are trained on a corpus from a uniform musical style, using features of both the music and lyrics.

This project is part of a long-term research plan to create the first automated singer/composer, capable of independent original composition and songwriting. Plans also include the challenge of enriching the system with a voice, enabling it to render professional-quality, emotional and varied performances of its compositions.

Songwriting with ALYSIA

To test our system, we evaluated its ability to create convincing pop music. First, we trained the system on a corpus of pop songs. Next, we came up with genre appropriate lyrics that we gave to the system, and then selected among the first 15 to 30 melody and rhythm pairs for each sentence.

Why Do I Still Miss You

Hhe first song ever made with ALYSIA, Why Do I Still Miss You, has been recorded an produce. Please listen to the results!

Here's the score for Why Do I Still Miss You:


The second song made with ALYISA has just been recorded and produced. Listen to it here. It is followed by the score produced with ALYISA.

After the score was made using the given verse and first chorus, the second chorus was made based on the above score to add variety to the recording, and to start guaging the flexibility of the melodies produced by our system.

Genre translation

ALYSIA's fully data-driven model and the absence of directly encoded musical rules make it ideally suited for translating songs between musical genres. Here we experimented with genre translation by starting with a song I'm Always Chasing Rainbows. The original music, credited to Harry Carroll, was adapted from Chopin's Fantaisie-Impromptu. Lyrics were written by Joseph McCarthy.

Using the lyrics to I'm Always Chasing Rainbows, we used ALYSIA trained on the pop music corpus to create a translation of this Vaudeville song into a modern song.

Below we share the recording of this song using the score generated with ALYSIA, followed by the original score and ALYSIA's score. During the production process, we chose to enrich this modern variation of I'm Always Chasing Rainbows with rock overtones, leading to a radical new interpretation of the lyrics.

Here's the original score:

Below is ALYSIA's translation of the song into the modern pop genre.