WMBR 88.1<\/a>, MIT\u2019s radio station. \u201cIf it\u2019s transforming one chord into a chord with a different harmony, or with more notes, for instance, the notes will split from the first chord and find a position to seamlessly glide to in the other chord.\u201d<\/p>\nAccording to Henderson, this is one of the first techniques to apply optimal transport to transforming audio signals. He has already used the algorithm to build equipment that seamlessly transitions between songs on his radio show. DJs could also use the equipment to transition between tracks during live performances. Other musicians might use it to blend instruments and voice on stage or in the studio.<\/p>\n
Henderson\u2019s co-author on the paper is Justin Solomon, an X-Consortium Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science. Solomon \u2014 who also plays cello and piano \u2014 leads the Geometric Data Processing Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and is a member of the Center for Computational Engineering.<\/p>\n
Henderson took Solomon\u2019s class, 6.838 (Shape Analysis), which tasks students with applying geometric tools like optimal transport to real-world applications. Student projects usually focus on 3-D shapes from virtual reality or computer graphics. So Henderson\u2019s project came as a surprise to Solomon. \u201cTrevor saw an abstract connection between geometry and moving frequencies around in audio signals to create a portamento effect,\u201d Solomon says. \u201cHe was in and out of my office all semester with DJ equipment. It wasn\u2019t what I expected to see, but it was pretty entertaining.\u201dFor Henderson, it wasn\u2019t too much of a stretch. \u201cWhen I see a new idea, I ask, \u2018Is this applicable to music?\u2019\u201d he says. \u201cSo, when we talked about optimal transport, I wondered what would happen if I connected it to audio spectra.\u201d<\/p>\n
A good way to think of optimal transport, Henderson says, is finding \u201ca lazy way to build a sand castle.\u201d In that analogy, the framework is used to calculate the way to move each grain of sand from its position in a shapeless pile into a corresponding position in a sand castle, using as little work as possible. In computer graphics, for instance, optimal transport can be used to transform or morph shapes by finding the optimal movement from each point on one shape into the other.<\/p>\n
Applying this theory to audio clips involves some additional ideas from signal processing. Musical instruments produce sound through vibrations of components, depending on the instrument. Violins use strings, brass instruments use air inside hollow bodies, and humans use vocal cords. These vibrations can be captured as audio signals, where the frequency and amplitude (peak height) represent different pitches.<\/p>\n
Conventionally, the transition between two audio signals is done with a fade, where one signal is reduced in volume while the other rises. Henderson\u2019s algorithm, on the other hand, smoothly slides frequency segments from one clip into another, with no fading of volume.<\/p>\n
To do so, the algorithm splits any two audio clips into windows of about 50 milliseconds. Then, it runs a Fourier transform, which turns each window into its frequency components. The frequency components within a window are lumped together into individual synthesized \u201cnotes.\u201d Optimal transport then maps how the notes in one signal\u2019s window will move to the notes in the other.<\/p>\n
Then, an \u201cinterpolation parameter\u201d takes over. That\u2019s basically a value that determines where each note will be on the path from its starting pitch in one signal to its ending pitch in the other. Manually changing the parameter value will sweep the pitches between the two positions, producing the portamento effect. That single parameter can also be programmed into and controlled by, say, a crossfader, a slider component on a DJ\u2019s mixing board that smoothly fades between songs. As the crossfader slides, the interpolation parameter changes to produce the effect.<\/p>\n
Behind the scenes are two innovations that ensure a distortion-free signal. First, Henderson used a novel application of a signal-processing technique, called \u201cfrequency reassignment,\u201d that lumps the frequency bins together to form single notes that can easily transition between signals. Second, he invented a way to synthesize new phases for each audio signal while stitching together the 50-millisecond windows, so neighboring windows don\u2019t interfere with each other.<\/p>\n
Next, Henderson wants to experiment with feeding the output of the effect back into its input. This, he thinks, could automatically create another classic music effect, \u201clegato,\u201d which is a smooth transition between distinct notes. Unlike a portamento \u2014 which plays all notes between a start and end note \u2014 a legato seamlessly transitions between two distinct notes, without capturing any notes in between.<\/p>\n","protected":false},"excerpt":{"rendered":"
Proactively fabricate one-to-one materials via effective e-business. Completely synergize scalable e-commerce rather than high standards in e-services. Assertively iterate resource maximizing products after leading-edge intellectual capital.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"image","meta":{"content-type":"","om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[68,69],"class_list":["post-2962","post","type-post","status-publish","format-image","hentry","category-uncategorized","tag-school-of-engineering","tag-science-and-technology","post_format-post-format-image"],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/posts\/2962","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/comments?post=2962"}],"version-history":[{"count":0,"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/posts\/2962\/revisions"}],"wp:attachment":[{"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/media?parent=2962"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/categories?post=2962"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icaninfotech.com\/wp-json\/wp\/v2\/tags?post=2962"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}