Home Research Publications Teaching Press Outreach


My research interests include computational approaches in music analysis and generation, music information retrieval, machine learning, audio signal processing, and artificial intelligence. Here are some projects I have conducted: Here are the master theses and undergraduate projects that I have advised:

Audio Key Finding

Audio key finding is a problem of extracting relevant information about pitches from acoustic signals of an audio recording for identifying the tonal center accurately. In Western tonal music, key is one of the most important features because it describes a system in which the roles of pitches and chords are defined. In general, audio key finding systems consist of two stages - pitch detection and key determination. My work in audio key finding started with designing a pitch detection algorithm that takes the advantage of harmonic series to clear up the noise in lower registers:
  • "Fuzzy Analysis in Pitch Class Determination for Polyphonic Audio Key Finding", ISMIR 2005, [ pdf ]

  • "Audio Key Finding: Considerations in System Design, and Case Studies on 24 Chopin's Preludes," EURASIP Journal on Applied Signal Processing 2007, [ pdf ]
Later on, I focused on the problem of comparing the performances of different audio key finding systems and explored in details to discuss the cases where the title key is not the ground truth:
  • Chuan, C.-H. and Chew, E.,"The KUSC Classical Music Dataset for Audio Key Finding", IJMIA 2014, [ pdf ]

  • Chuan, C.-H. and Chew, E.,"Creating Ground Truth for Audio Key Finding: When The Title Key May Not Be The Key," ISMIR 2012, [ pdf ]
I have also worked with students to create an application using the result of audio key finding. The application is a recommendation system called MixMe that aims to help DJs to find suitable tracks quickly given a currently playing track. A suitable track is one that has closely related key when it is tuned to the tempo of the current track.
  • "Aspillaga, F. X., Cobb, J., and Chuan, C.-H., "MixMe: A recommendation system for DJs," ISMIR 2011, [ pdf ]

Style-Specific Music Generation

As David Cope explained in Machine Models of Music, "It is with style that one learns directly about music perception and aesthetics." In the field of algorithmic composition, researchers and composers have proposed algorithms for generating music in the style of Bach, Mozart, Louis Armstrong and others since the 1950s. My work started with the automatic style-specific accompaniment (ASSA) system. This system combines statistical modeling with a music theoretical framework called neo-Riemanian transforms to learn and produce chord progressions. Given a user's newly created melody, the system first determines chord tones in the melody using decision trees, and then generate a chord progression that has the highest probability based on the learned model. Here are two papers describing the ASSA system:
  • Chuan, C.-H. and Chew, E., "Generating and Evaluating Musical Accompaniments that Emulate Style," CMJ 35(4), [ url ]

  • Chuan, C.-H. and Chew, E., "A Hybrid System for Automatic Generation of Style-Specific Accompaniment," JIWCC'07, [ pdf ]
I then focused on the evaluation: comparing the performance of multiple systems and examining the effectiveness of using ASSA as a computer-assisted composition tool:
  • Chuan, C. H., "A Comparison of Statistical and Rule-Based Models for Style-Specific Harmonization," ISMIR 2011, [ pdf ]

  • Chuan, C.-H. and Chew, E., "Quantifying the Benefits of Using an Interactive Decision Support Tool for Creating Musical Accompaniment in a Particular Style," ISMIR 2010 [ pdf ]

  • Chuan, C.-H. and Chew, E., "Evaluating and Visualizing Effectiveness of Style Emulation in Music Accompaniment," ISMIR 2008, [ pdf ]
When I was at Barry University in Miami Shores, I advised an undergraduate student Fred Bertino, who was a double major in Biology and Music and now a medical doctor, for an interesting project on music generation. A chord progression is first generated by a sequence of LysA gene of E.coli bacteria and then Fred and his friend Jared Peroune created a piano piece inspired by the chord progression. This work was presented in the 12th Biennial Symposium on Arts and Technology [ pdf ]. You can listen to the sample of the composition accompanied by the amino acid that mapped to the chord progression in the following video.

Machine Learning Applications for Education

Intelligent Tutoring System for Argument-Making: I worked with Dr. Daniel Dinsmore (Education, UNF) and Dr. Joseph Schmuller (Psychology, UNF), and Tyler Morris (a CS undergraduate) on predicting the grade of arguments written by the college students. We used machine learning techniques to analyze 149 students' responses to an open-ended question. Students' responses were graded by the instructor using the Structured Outcome of the Learning Observation (SOLO) taxonomy. The responses were then processed using Coh-metrix as well as n-gram models to generate features for the classification (classifying the response into one of the grade categories) task. The best result of 81.74% in classification correct rate was obtained.
  • See the paper "An Intelligent Tutoring System for Argument-Making in Higher Education: A Pilot Study" [ url ] for more details.
American Sign Language (ASL) Recognition/Verification: I have been working with Dr. Caroline Guardino (Deaf Education, UNF) and Eric Regina (undergraduate major in mathematics and minor in CS) to develop a system that aims to help children who are born deaf or hard of hearing to learn ASL. We first started the project using Leap Motion sensor. We derived features from the sensory data and used k-nearest neighbor and support vector machine to classify 26 English alphabet letters. The highest average classification rates of 72.78% and 79.83% were achieved by k-nearest neighbor and SVM respectively. More details can be found in the paper:
  • Chuan, C.-H., Regina, E., and Guardino, C., "American Sign Language Recognition Using Leap Motion Sensor," in Proceedings of the 13th International Conference on Machine Learning and Applications, a special session on Machine Learning Applications in Education, December 3 - 5, Detroit, Michigan, 2014. [ url ]
A major cause of the inaccuracy in the experiment result is the mislabelled data from Leap Motion APIs. Therefore, we switched our attention to computer vision using RGB-D cameras. The following videos show Eric's work on finger detection using K-curvature and hand tracking using OpenCV and Kalman filter:
Finger detection using K-curvatureHand tracking using Kalman filter

Large Scale Time Series Indexing

This project was inspired by the need of computational music generation. Music improvisation systems including the Continuator use data structures such as prefix trees or factor oracles to store music sequences in order to quickly find out possible sub-sequences to continue the stream given a currently playing sequence. For example, the figures below show (a) a prefix-tree storing the sequence "abcdadbc" and (b) a factor oracle sotring the sequence "cdedegfdec".
(a) A prefix tree for the sequence "abcdadbc"(b) A factor oracle for the sequence "cdedegfdec"
It is possible to store the entire structure for a reasonable short sequence in main memory, but it won't be a practical solution if we want to build such structures for a dataset of songs. Therefore, Aleksey and I have been working on storing such structures in relational databases and NoSQL databases:
  • Charapko, A., and Chuan, C.-H., "Indexing Musical Sequences in Large Datasets Using Relational Databases," International Journal of Multimedia Engineering and Management, 6(2), pp. 1-18, 2015. [ url ]

Advised Master's Theses and Undergraduate Projects

Master's Theses

Aleksey Charapko, "Time Series Similarity Search in Distributed Key-Value Data Stores Using R-Trees", defended on 3/25/2015. Aleksey graduated in Spring 2015 and is now a PhD student in the Computer Science and Engineering Program at SUNY Buffalo.
Jaime Kaufman, "A Hybrid Approach to Music Recommendation: Exploring Collaborative Music Tags and Acoustic Features," [UNF digital commons], defended on 11/10/2014. Jaime graduated in Fall 2014 and is currently employed as a consultant/big data ingest developer for Illumination Works, LLC, in Cincinnati, Ohio.

Undergraduate Projects

Eric Regina (mathematics and statistics), Gesture recognition for American Sign Language (Summer 2014 -), [ publication ]
Tyler Morris (computer science), Intelligent Tutoring Systems for Argument-Making (Spring 2014), [ publication ]
Elizabeth Feldman (mathematics and statistics), Using Holt-Winter model for predicting music notes (Spring 2014), [ publication ]
Jeffrey Tran and Bradley Germain, Face Recognition with Neural Networks, the 3rd Annual Florida Undergraduate Research, University of Florida, February 22 - 23, 2013.
Aleksey Charapko and Steven Repper, Optimal UNF Shuttle Routes, the 3rd Annual Florida Undergraduate Research, University of Florida, February 22 - 23, 2013.
Eric Douglas, Marc Mazour, and Nihar Goswami, Osprey Flight Path: A Mobile Guiding Application on Campus, the 2nd Annual Florida Undergraduate Research Conference, Stetson University, DeLand, March 16 - 17, 2012.
Fred Bertino, The Musical Gene: Generating Harmonic Patterns from Sequenced DNA of E.coli Bacteria to Compose Music, the 12th Biennial Symposium on Arts and Technology, March 4 - 6, Connecticut College, Connecticut, 2010.