5.28.2013

From Vivaldi to Wagner to Schubert

A machine that knows a musical era within three notes

Editor’s note: This article is by Computational Biologist Guillermo Cecchi of IBM Research.

Music may "soothe a savage breast." But a machine can now identify the music’s era and composer.

Periods of Western Music*
  • 1700-1770 Baroque Era
  • 1770-1830 Classical Era
  • 1830-1900 Romantic Era
  • 1900 - Post-Romantic Era
* -- available on Peachnote
Pablo Rodriguez Zivic from the University of Buenos Aires, Favio Shifres from the University of La Plata and I developed an algorithm that can identify the Baroque, Classical and Romantic periods – after “hearing” only three notes.

Peachnote recently added these near-three centuries and 20,000 songs worth of Western music periods to its online database. Pablo tested our algorithm on the transcribed music from Peachnote. Our machine then discovered patterns across the songs down to the semi-tones or notes. The patterns learned could now possibly extend to human speech or “humming.”

Visualization of the clusters for the conditional distribution of melodic intervals. Each shaded area corresponds to a different cluster, and its corresponding line represents the proportion of years assigned to it within a 10-y smoothing window. Vertical dashed lines correspond to the approximate boundaries between Baroque, Classical, and Romantic periods. A fifth cluster was removed because it was a noise cluster with only three elements.

Can you identify your favorite song within three consecutive notes from any point within the song? In just three semi-tones, say “A,” followed by “C,” followed by “F,” a pattern emerged that the machine accurately matched to an entire era of music. 

For example, we identifies late Baroque factors by the high frequency of adjacent notes, such as “C” is often followed by “F” and “G,” and then by “B” and “D.” The tune “Happy Birthday” follows this pattern: [hap/C]-[py/C]-[birth/D]-[day/C]-[to/F]-[you/E]. As the tonal music developed over time, the corresponding patterns of note combinations became more complex.

And while we did not focus on this aspect in the present work, we have unpublished results identifying many of the composers, such as Bach, Mozart and Beethoven.
 
Music-making machines

Computers can do more than identify music. Pablo Zivic has also developed a computationally creative machine that can produce an era-based, but unique piece of music.

From music to speech 

The structure of three consecutive semi-tones determines the way we hear and perceive music. Simple factors, such as the time between the three notes, the tone between the notes, and the order of the notes account for identifiable trends within the eras of music.



We could perform this study because of the available data – the “big data” of music. This same approach could identify other patterns in other sounds, namely our speech. Computer algorithms can already identify speech patterns in the early stages of Parkinson’s disease through a recorded phone interview.

Doctors know that the vocal chords are affected early on in the onset of Parkinson’s. We want to push the ability to identify the combination of sounds – like the three notes in our Western music study – to make an even earlier diagnosis.

With more-comprehensive speech data, we could uncover patterns in other diseases. We're now using our tool to study past individual cases to tease out features and patterns in the way people with a particular psychiatric disorder speak or even “hum” music. From this, we could come up with models to explain the behavior.

Perhaps in the future, other disorders can be identified through simple, non-invasive verbal tests.

Read the study, Perceptual basis of evolving Western musical styles, in Proceedings of the National Academy of Sciences.
 

7 comments:

  1. Is the big data for music publicly available .

    ReplyDelete
  2. I think that you are using the term "semi-tone" wrongly, or at least not in the way any musician would recognise it.

    A semitone is the interval between two notes in a Western chromatic scale e.g. B and C or C and C#.

    Your example of three semi-tones "say “A,” followed by “C,” followed by “F,”" is three notes, or three pitches but is not three semi-tones.

    It's a shame because I find the article interesting but I know if I were to circulate it to my musician friends they would fixate on how you don't know what a semitone is to the exclusion of anything else.

    ReplyDelete
  3. ...although having now read the full article in PNAS the term semitone is used correctly there, it just seems to be misunderstood in this precis

    ReplyDelete
  4. The first line of this post contains a common misquotation; Congreve's original line was "Music has charms to soothe a savage breast".

    ReplyDelete
  5. This is very cool technology!

    ReplyDelete
  6. This is an awesome accomplishment...

    But what are we doing to help HUMANS recognize classical music and composers? Most millennials don't recognize the Beatles or Elvis (watershed) and the socio-cultural paradigm shifts they either represented or drove (or both).

    I guess I'd be much more enthusiastic if the algorithm matched the music to the era to a global view of socio-economic trends and political events so that we are bringing together the holistic view of the planet and its evolution thru the lens of music... rather than presenting an infinitely dissected view where the "art" of life is lost amongst the "1s" and "0s".

    ReplyDelete