|My research interests include computational approaches in music analysis and generation, music information retrieval, machine learning, audio signal processing, and artificial intelligence.
Here are some projects I have conducted:
Audio Key Finding
|Audio key finding is a problem of extracting relevant information about pitches from acoustic signals of an audio recording for identifying the tonal center accurately.
In Western tonal music, key is one of the most important features because it describes a system in which the roles of pitches and chords are defined.
In general, audio key finding systems consist of two stages - pitch detection and key determination. My work in audio key finding started with designing a pitch detection algorithm
that takes the advantage of harmonic series to clear up the noise in lower registers:
Style-Specific Music Generation
As David Cope explained in Machine Models of Music, "It is with style that one learns directly about music perception and aesthetics."
In the field of algorithmic composition, researchers and composers have proposed algorithms for generating music in the style of Bach, Mozart, Louis Armstrong and others since the 1950s.
My work started with the automatic style-specific accompaniment (ASSA) system. This system combines statistical modeling with a music theoretical
framework called neo-Riemanian transforms to learn and produce chord progressions. Given a user's newly created melody, the system first determines
chord tones in the melody using decision trees, and then generate a chord progression that has the highest probability based on the learned model.
Here are two papers describing the ASSA system:
Machine Learning Applications for Education
|Intelligent Tutoring System for Argument-Making: I worked with Dr. Daniel Dinsmore (Education, UNF) and Dr. Joseph Schmuller (Psychology, UNF), and Tyler Morris (a CS undergraduate) on predicting the grade of arguments written by the college students. We used machine learning techniques to analyze 149 students' responses to an open-ended question. Students' responses were graded by the instructor using the Structured Outcome of the Learning Observation (SOLO) taxonomy. The responses were then processed using Coh-metrix as well as n-gram models to generate features for the classification (classifying the response into one of the grade categories) task. The best result of 81.74% in classification correct rate was obtained.|
|American Sign Language (ASL) Recognition/Verification: I have been working with Dr. Caroline Guardino (Deaf Education, UNF) and Eric Regina (undergraduate major in mathematics and minor in CS) to develop a system that aims to help
children who are born deaf or hard of hearing to learn ASL. We first started the project using Leap Motion sensor. We derived features from the sensory data and used k-nearest neighbor and support vector machine to classify 26 English alphabet letters.
The highest average classification rates of 72.78% and 79.83% were achieved by k-nearest neighbor and SVM respectively. More details can be found in the paper:
|Finger detection using K-curvature||Hand tracking using Kalman filter|
Large Scale Time Series Indexing
|This project was inspired by the need of computational music generation. Music improvisation systems including the Continuator use data structures such as prefix trees or factor oracles to store music sequences in order to quickly find out possible sub-sequences to continue the stream given a currently playing sequence. For example, the figures below show (a) a prefix-tree storing the sequence "abcdadbc" and (b) a factor oracle sotring the sequence "cdedegfdec".|
|(a) A prefix tree for the sequence "abcdadbc"||(b) A factor oracle for the sequence "cdedegfdec"|
It is possible to store the entire structure for a reasonable short sequence in main memory, but it won't be a practical solution if we want to build such structures for
a dataset of songs. Therefore, Aleksey and I have been working on storing such structures in relational databases and NoSQL databases:
Advised Master's Theses and Undergraduate Projects
|Aleksey Charapko, "Time Series Similarity Search in Distributed Key-Value Data Stores Using R-Trees", defended on 3/25/2015. Aleksey graduated in Spring 2015 and is now a PhD student in the Computer Science and Engineering Program at SUNY Buffalo.|
|Jaime Kaufman, "A Hybrid Approach to Music Recommendation: Exploring Collaborative Music Tags and Acoustic Features," [UNF digital commons], defended on 11/10/2014. Jaime graduated in Fall 2014 and is currently employed as a consultant/big data ingest developer for Illumination Works, LLC, in Cincinnati, Ohio.|
|Eric Regina (mathematics and statistics), Gesture recognition for American Sign Language (Summer 2014 -), [ publication ]|
|Tyler Morris (computer science), Intelligent Tutoring Systems for Argument-Making (Spring 2014), [ publication ]|
|Elizabeth Feldman (mathematics and statistics), Using Holt-Winter model for predicting music notes (Spring 2014), [ publication ]|
|Jeffrey Tran and Bradley Germain, Face Recognition with Neural Networks, the 3rd Annual Florida Undergraduate Research, University of Florida, February 22 - 23, 2013.|
|Aleksey Charapko and Steven Repper, Optimal UNF Shuttle Routes, the 3rd Annual Florida Undergraduate Research, University of Florida, February 22 - 23, 2013.|
|Eric Douglas, Marc Mazour, and Nihar Goswami, Osprey Flight Path: A Mobile Guiding Application on Campus, the 2nd Annual Florida Undergraduate Research Conference, Stetson University, DeLand, March 16 - 17, 2012.|
|Fred Bertino, The Musical Gene: Generating Harmonic Patterns from Sequenced DNA of E.coli Bacteria to Compose Music, the 12th Biennial Symposium on Arts and Technology, March 4 - 6, Connecticut College, Connecticut, 2010.|