Evaluating changes in the marine soundscape of an offshore wind farm via the machine learning-based source separation

Oral presentation in Underwater Technology 2019 Kaohsiung

Evaluating changes in the marine soundscape of an offshore wind farm via the machine learning-based source separation

Tzu-Hao Lin1, Hsin-Te Yang2, Jie- Mao Huang2, Chiou-Ju Yao3, Yung-Shun Lien4, Pei-Jung Wang4, Fang-Yu Hu4

1Japan Agency for Marine-Earth Science and Technology (JAMSTEC), Japan
2Observer Ecological Consultant, Taiwan
3Nature Museum of Natural Science, Taiwan
4Industrial Technology Research Institute, Taiwan

Investigating the ecological effects of offshore wind farms requires comprehensive surveys of marine ecosystem. Recently, the monitoring of marine soundscapes has been included in the rapid appraisals of geophysical events, marine fauna, and human activities. Machine learning is widely applied in acoustic research to improve the efficiency of audio processing. However, the use of machine learning to analyze marine soundscapes remains limited due to a general lack of human-annotated databases. In this study, we used unsupervised learning to recognize different sound sources underwater. We also quantified the temporal, spatial, and spectral variabilities of long-term underwater recordings collected near Phase I of the Formosa I wind farm. One source separation model was developed to recognize choruses made by fish and snapping shrimp, as well as shipping noise. Another model was developed to identify transient fish calls and echolocation clicks of marine mammals. Models were trained in an unsupervised manner using the periodicity-coded non-negative matrix factorization. After the sound sources were separated, events can be identified using Gaussian mixture models. Our information retrieval techniques facilitate future investigations of the spatiotemporal changes in marine soundscapes and allow to build an annotated database efficiently. The soundscape information can be used to evaluate the potential impacts of noise-generating activities on soniferous marine animals and their acoustic behavior before, during, and after the development of offshore wind farms.

Advertisements

Listening to the ecosystem: an integrative approach of informatics and ecoacoustics

We will attend the Symposium of Integrative Biology: Biodiversity in Asia, which is held at Kyoto University next week, to present our recent progress in the information retrieval of marine soundscapes.

Last year, we published the toolbox of Soundscape Viewer, which aims to help researchers evaluate changes in the marine soundscape without any recognition database. In this poster, we summarize the general procedures of soundscape information retrieval and demonstrate the application of studying the phenology of soniferous marine animals in estuarine waters, upper mesophotic corals, and continental shelf environments.

Kyoto_BIOD_poster

You can also find the pdf copy of this poster here.

The soundscape is a mixture of biophony, geophony, and anthrophony, thus, the disentanglement of different sound sources will be a key step in exploring the complexity of soundscape. Our work is still in the beginning phase, collaborations in ecoaoustics, acoustic information retrieval, or similar subjects are very welcome!

ASA Victoria presentations are now available online!

Two presentations were given during the ASA Victoria. Now you can access the slides and recordings by ASA website:

Machine Learning and Data Science Approaches in Ocean Acoustics II:
Applying machine-learning based source separation techniques in the analysis of marine soundscapes

Hot Topics in Acoustics:
Information retrieval from a soundscape by using blind source separation and clustering

Feel free to contact me if you have any question.

Two presentations at ASA Victoria Meeting

Applying machine-learning based source separation techniques in the analysis of marine soundscapes

Tzu-Hao Lin1, Tomonari Akamatsu2, Yu Tsao3, Katsunori Fujikura1

1Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology
2National Research Institute of Fisheries Science, Japan Fisheries Research and Education Agency
3 Research Center for Information Technology Innovation, Academia Sinica

Long-term monitoring of underwater soundscapes provides us a large number of acoustic recordings to study a marine ecosystem. Characteristics of a marine ecosystem, such as the habitat quality, composition of marine fauna, and the level of human interference, may be analyzed using information relevant to environmental sound, biological sound, and anthropogenic sound. Supervised source separation techniques have been widely employed in speech and music separation tasks, but it may not be practical for the analysis of marine soundscapes due to the lack of a database that includes a large mount of paired pure and mixed signals. Even when the paired data is not available, different sound sources with unique spectral or temporal patterns may still be separated by apply semi-supervised or unsupervised learning algorithms. In this presentation, supervised and unsupervised source separation techniques will be demonstrated on long-term spectrograms of a marine soundscape. Separation performances under different levels of simultaneous source influence will also be discussed. In the future, more advanced techniques of source separation are necessary to facilitate the soundscape-based marine ecosystem sensing. An open database of marine soundscape will promote the development of machine learning-based source separation. Therefore, we will open acoustic data tested in this presentation on the Asian Soundscape to encourage the open science of marine soundscape.

Information retrieval from a soundscape by using blind source separation and clustering

Tzu-Hao Lin1, Yu Tsao2, Tomonari Akamatsu3, Mao-Ning Tuanmu4, Katsunori Fujikura1

1Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology
2Research Center for Information Technology Innovation, Academia Sinica
3National Research Institute of Fisheries Science, Japan Fisheries Research and Education Agency
4Biodiversity Research Center, Academia Sinica

Passive acoustic monitoring represents one of the remote sensing platforms of biodiversity. However, it remains challenging to retrieve meaningful biological information from a large amount of soundscape data when a comprehensive recognition database is not available. To overcome this issue, it is necessary to investigate the basic structure of a soundscape and subsequently retrieve biological information. The recent development of machine learning-based blind source separation techniques allow us to separate biological choruses and non-biological sounds appearing on a long-term spectrogram. After the blind source separation, the temporal-spatial changes of bioacoustic activities can be efficiently investigated by using a clustering algorithm. In this presentation, we will demonstrate the information retrieval in the forest and marine soundscapes. The separation result shows that in addition to biological information, we can also extract information relevant to weather patterns and human activities. Furthermore, the clustering result can be used to establish an audio library of nature soundscapes, which may facilitate the investigation of interactions among wildlife, climate change, and human development. In the future, the soundscape-based ecosystem monitoring will be feasible if we can integrate the soundscape information retrieval in a large-scale soundscape monitoring network.

Two presentations at the 10th International Conference on Ecological Informatics!

Information retrieval from marine soundscape by using machine learning-based source separation

Tzu-Hao Lin 1, Tomonari Akamatsu 2, Yu Tsao 3, Katsunori Fujikura1

1 Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology, Japan
2 National Research Institute of Fisheries Science, Japan Fisheries Research and Education Agency, Japan
3 Research Center for Information Technology Innovation, Academia Sinica, Taiwan

In remote sensing of the marine ecosystem, visual information retrieval is limited by the low visibility in the ocean environment. Marine soundscape has been considered as an acoustic sensing platform of the marine ecosystem in recent years. By listening to environmental sounds, biological sounds, and human-made noises, it is possible to acoustically identify various geophysical events, soniferous marine animals, and anthropogenic activities. However, the sound detection and classification remain a challenging task due to the lack of underwater audio recognition database and the simultaneous interference of multiple sound sources. To facilitate the analysis of marine soundscape, we have employed information retrieval techniques based on non-negative matrix factorization (NMF) to separate different sound sources with unique spectral-temporal patterns in an unsupervised approach. NMF is a self-learning algorithm which decomposes an input matrix into a spectral feature matrix and a temporal encoding matrix. Therefore, we can stack two or more layers of NMF to learn the spectral-temporal modulation of k sound sources without any learning database [1]. In this presentation, we will demonstrate the application of NMF in the separation of simultaneous sound sources appeared on a long-term spectrogram. In shallow water soundscape, the relative change of fish chorus can be effectively quantified even in periods with strong mooring noise [2]. In deep-sea soundscape, cetacean vocalizations, an unknown biological chorus, environmental sounds, and systematic noises can be efficiently separated [3]. In addition, we can use the features learned in procedures of blind source separation as the prior information for supervised source separation. The self-adaptation mechanism during iterative learning can help search the similar sound source from other acoustic dataset contains unknown noise types. Our results suggest that the NMF-based source separation can facilitate the analysis of the soundscape variability and the establishment of audio recognition database. Therefore, it will be feasible to investigate the acoustic interactions among geophysical events, soniferous marine animals, and anthropogenic activities from long-duration underwater recordings.

Improving acoustic monitoring of biodiversity using deep learning-based source separation algorithms

Mao-Ning Tuanmu1, Tzu-Hao Lin2, Joe Chun-Chia Huang1, Yu Tsao3, Chia-Yun Lee1

1Biodiversity Research Center, Academia Sinica, Taiwan
2Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology, Japan
3Research Center for Information Technology Innovation, Academia Sinica, Taiwan

Passive acoustic monitoring of the environment has been suggested as an effective tool for investigating the dynamics of biodiversity across spatial and temporal scales. Recent development in automatic recorders has allowed environmental acoustic data to be collected in an unattended way for a long duration. However, one of the major challenges for acoustic monitoring is to identify sounds of target taxa in recordings which usually contain undesired signals from non-target sources. In addition, high variation in the characteristics of target sounds, co-occurrence of sounds from multiple target taxa, and a lack of reference data make it even more difficult to separate acoustic signals from different sources. To overcome this issue, we developed an unsupervised source separation algorithm based on a multi-layer (deep) non-negative matrix factorization (NMF). Using reference echolocation calls of 13 bat species, we evaluated the performance of the multi-layer NMF in separating species-specific calls. Results showed that the multi-layer NMF, especially when being pre-trained with reference calls, outperformed the conventional supervised single-layer NMF. We also evaluated the performance of the multi-layer NMF in identifying different types of bat calls in recordings collected in the field. We found comparable performance in call types identification between the multi-layer NMF and human observers. These results suggest that the proposed multi-layer NMF approach can be used to effectively separate acoustic signals of different taxa from long-duration field recordings in an unsupervised manner. The approach can thus improve the applicability of passive acoustic monitoring as a tool to investigate the responses of biodiversity to the changing environment.

Studying wildlife activities by using soundscape information

Oral presentation in 2018 ICEO & SI Conference

Studying wildlife activities by using soundscape information

Tzu-Hao Lin1, Yu Tsao2, Chun-Chia Huang3, and Mao-Ning Tuanmu3

1Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology
2Research Center for Information Technology Innovation, Academia Sinica
3Biodiversity Research Center, Academia Sinica

Information regarding biodiversity change is essential for the decision making of resource exploitation and conservation management. Studies on biodiversity are labor-intensive and time-consuming. Therefore, the development of a remote observation platform of wildlife is essential. In recent decades, passive acoustic monitoring has been widely employed to detect vocalizing animals. Besides, various environmental sounds and anthropogenic noises can also be recorded in a soundscape. Thus, a soundscape monitoring network has been considered as an acoustic sensing platform of the ecosystem. Although a significant amount of acoustic data can be collected, the acoustic data analysis remains a challenge for ecologists. In this study, we employed multiple layers of non-negative matrix factorization (NMF) to decompose a spectrogram into individual sound sources. Our results showed that echolocation calls produced from three different bat species can be effectively separated in an unsupervised manner. Even for overlapping signals, the deep NMF can still produce a reliable separation result. Therefore, the integration of NMF-based blind source separation and a soundscape monitoring network can reduce the difficulty of acoustic-based wildlife monitoring in the future.

The full text is available:
https://drive.google.com/file/d/1Llf9RuyeR4a7k36p_MMj9OZ2apaeZ_iz/view

Training workshop on the acoustical analysis of animal vocalizations

Time: 2018/07/04 (Wed)
Location: Biodiversity Research Center, Academia Sinica
Speaker: Dr. Tzu-Hao Lin (Department of Marine Biodiversity Research, JAMSTEC)

Morning Session: Passive acoustic monitoring
09:00-09:30: Registration
09:30-10:00: Passive acoustic monitoring of wildlife (Lecture)
10:00-10:30: Labeling of biosonar signal (Practice)
10:30-11:30: Automatic detection of biosonar activity (Practice)
11:30-12:00: Discussion
12:00-13:30: Lunch

Afternoon Session: Application of PAM in an offshore wind farm
13:30-14:00: Passive acoustic monitoring of soniferous marine animals in an offshore wind farm (Lecture)
14:00-15:00: Searching dolphin biosonars and fish sounds from long-duration recordings (Practice)
15:00-15:30: Break
15:30-16:00: Temporal analysis of acoustic detection results (Practice)
16:00-16:30: Discussion

Please go to this link for registering this training workshop. Due to the limited space, only 25 seats are available. The final attendant list will be determined by the organizer and only those successful registers will be notified by email.