(SOFI) - Song Form Intelligence to ensure smooth delivery of streaming music

Monday, 15 June 2015

We have identified a new and emerging technology for improving the quality of audio being broadcast over any wireless medium through meta-data which has a number of market applications all with market value.  The established concept behind this application stems from advanced research at the University of Ulster into streaming media to wireless devices. Laptops, netbooks and mobile phones are all susceptible to the unreliable nature of connections on wireless networks, especially at the outer limits of the signal. Time dependent data such as music broadcast on Internet radio stations make this issue more evident to the listener. The ability of traditional packet level Forward Error Correction approaches can limit errors for small sporadic network losses but when dropouts of large portions occur listening quality becomes an issue. Services such as audio-on-demand drastically increase the loads on networks therefore new, robust and highly efficient coding algorithms are necessary. 

 

One method overlooked to date, which can work alongside existing audio compression schemes, is that which takes account of the semantics and natural repetition of music through meta-data tagging. Similarity detection within polyphonic audio has presented problematic challenges within the field of Music Information Retrieval. One approach to deal with bursty errors is to use self-similarity to replace missing segments. Many existing systems exist based on packet loss and replacement on a network level but none attempt repairs of large dropouts of 5 seconds or more. Our system also works at the content level thus rendering it applicable in existing streaming services. The main strategic objective is the continued development of our audio repair approach and further products to meet the needs of a range of market sectors in broadcast audio.  These sectors include Broadcast Digital Audio, Online Streaming Music Sites and Wireless Home Streaming Media Servers. Other immediate market areas include encompassing the similarity encoding process onto CDs for use with high-end Hi-Fi systems that operate on a wireless basis. A facility to provide a library of similarity encoded metadata is also possible based on the principle that a song needs to be analysed once and be broadcast an infinite number of times with the similarity data encoded.

 

Problem Being Solved

The recent Digital Britain Report 2010 highlights the UK Government's ambitious plans for DAB Radio and the Internet. A key advantage of our proposed tool is to preserve the Quality of Experience (QOE) for the listener. AM may fade and FM is fairly stable within a service area, but the underlying Internet Protocol may be disrupted by factors at other parts of the network thus reducing the listening experience.   Our technology can improve quality in the UK DAB radio standard. In fact, all online streaming audio providers can benefit from this technology. Current alternatives rely on silence substitution, varying levels of audio compression to aid overall throughput. We know of no commercial systems replicating our invention. Any time dependant broadcast of music across networks can benefit from this meta-data tagging. Almost all of the current pattern matching systems work in a non-realtime environment where timing constraints have little or no relevance.  The core of this project accurately identify missing audio sections and replace these with ‘matched’ sections previously received in other sections of the audio file.

Technology

We have identified a new and emerging technology for improving the quality of audio being broadcast over any wireless medium through meta-data which has a number of market applications all with market value.  The established concept behind this application stems from advanced research at the University of Ulster into streaming media to wireless devices. Laptops, netbooks and mobile phones are all susceptible to the unreliable nature of bursty connections on wireless networks, especially at the outer limits of the signal. Time dependent data such as music broadcast on Internet radio stations make this issue more evident to the listener. The ability of traditional packet level Forward Error Correction approaches can limit errors for small sporadic network losses but when dropouts of large portions occur listening quality becomes an issue. Services such as audio-on-demand drastically increase the loads on networks therefore new, robust and highly efficient coding algorithms are necessary.

 

One method overlooked to date, which can work alongside existing audio compression schemes, is that which takes account of the semantics and natural repetition of music through meta-data tagging. Similarity detection within polyphonic audio has presented problematic challenges within the field of Music Information Retrieval. One approach to deal with bursty errors is to use self-similarity to replace missing segments. Many existing systems exist based on packet loss and replacement on a network level but none attempt repairs of large dropouts of 5 seconds or more. Our system also works at the content level thus rendering it applicable in existing streaming services.

 

Benefits/Applications

This technology with appropriate additional research, can be tailored to improve the perceptual quality of audio being communicated by broadcasters such as the BBC. This improvement is predicated on the efficient generation, organisation, communication and analysis of suitable audio meta-data to repair large-scale audio defects introduced by the unreliability of wireless networks. Hence the value of this meta-data is considerable, enabling media assets to be acceptably distributed using wireless networking under conditions that would, without the availability of the proposed technology, be considered unrealistic. Appropriate software tools to automatically handle the meta-data processing and effect the audio stream repairs in real-time are essential deliverables of this  project, since it is intended that commercial wireless audio products can conduct large-scale error correction in an automated fashion.

 

Our patent-pending fundamental technique that can be leveraged to facilitate the production of the proposed technology utilises the repetition of music by 'wrapping' it with meta-data which gives features for extraction and, when combined with intelligent algorithms, enables self-similarity to be detected within polyphonic audio. Early evaluations of our fundamental technique through demonstration software tools give positive results and can detect high levels of similarity on varying lengths of time within an audio stream by intelligently incorporating meta-data in the stream. This particular innovation represents a foundation technology that the proposed project looks to tailor and extend to a wide class of wireless audio use cases and products, such as new streaming media opportunities for mobile providers over wide-area networks and multi-zone audio distribution over wireless local area networks. This project has a very high likelihood of delivering the required technologies, with a clear route to well-defined markets within two years of project completion

 

This unique project will also pilot the use of leading edge hardware and advanced artificial intelligent coding techniques for the specific purpose of high quality music streaming. Unlike other audio streaming solutions, this project will regard bandwidth efficiency and audio quality as paramount. There are numerous knock-on benefits associated directly with this level of capability right across the digital media sector e.g. greater improved audio overlay application performance when supplementing streaming video.

 

Enjoyment of audio has now become about flexibility and personal freedom. Digital audio content can be acquired from many sources and wireless networking allows digital media devices and associated peripherals to be unencumbered by wires. However, despite recent improvements in capacity and quality of service, wireless networks are inherently unreliable communications channels for the streaming of audio, being susceptible to the effects of range, interference and occlusion. This time-varying reliability of wireless audio transfer introduces data corruption and loss, with unpleasant audible effects that can be profound and prolonged in duration. Traditional communications techniques for error mitigation perform poorly and in a bandwidth-inefficient manner in the presence of such large-scale defects in a digital audio stream. A novel solution that can complement existing techniques takes account of the semantics and natural repetition of music. Through the use of self-similarity meta-data, missing or damaged audio segments can be seamlessly replaced with similar undamaged segments that have already been successfully received.  We propose a technology to generate relevant self-similarity meta-data for arbitrary audio material and to utilise this meta-data within a wireless audio receiver to provide sophisticated and real-time correction of large-scale errors.   The primary objectives are to match the current section of a song being received with previous sections whilst identifying incomplete sections and determining replacements based on previously received portions of the song.  This technology is unique in its approach to Forward Error Correction (FEC) technology that is used to ’repair’ a bursty dropout when listening to time-dependent media on a wireless network. Ultimately, using self-similarity analysis on a music file, we can ’automatically’ repair the dropout with a similar portion of the music already received thereby minimising a listeners discomfort.

 

 

Future prototypes

We have identified high-fidelity products such as wireless bluetooth headsets which can benefit from our technology. We believe that with appropriate additional development, our system can be tailored to improve the perceptual quality of audio being communicated on wireless audio products. This improvement is predicated on the efficient generation, organisation, communication and analysis of suitable audio meta-data to repair large-scale audio defects introduced by the unreliability of wireless networks. Hence the value of this meta-data is considerable, allowing media assets to be acceptably distributed using wireless networking under conditions that would, without the availability of the proposed technology, be considered unrealistic. Appropriate software tools to automatically handle the meta-data processing and effect the audio stream repairs in real-time are essential outputs of the  project, since it is intended that commercial wireless audio products can conduct the large-scale error correction in an automated fashion.

Opportunity/Partnership Sought

Professor of Digital Multimedia
Computer Science Research Institute

space

Room MQ312
School of Creative Arts and Technologies
University of Ulster
Magee campus
Londonderry
BT48 7JL