Two presentations at the 10th International Conference on Ecological Informatics!

Information retrieval from marine soundscape by using machine learning-based source separation

Tzu-Hao Lin 1, Tomonari Akamatsu 2, Yu Tsao 3, Katsunori Fujikura1

1 Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology, Japan
2 National Research Institute of Fisheries Science, Japan Fisheries Research and Education Agency, Japan
3 Research Center for Information Technology Innovation, Academia Sinica, Taiwan

In remote sensing of the marine ecosystem, visual information retrieval is limited by the low visibility in the ocean environment. Marine soundscape has been considered as an acoustic sensing platform of the marine ecosystem in recent years. By listening to environmental sounds, biological sounds, and human-made noises, it is possible to acoustically identify various geophysical events, soniferous marine animals, and anthropogenic activities. However, the sound detection and classification remain a challenging task due to the lack of underwater audio recognition database and the simultaneous interference of multiple sound sources. To facilitate the analysis of marine soundscape, we have employed information retrieval techniques based on non-negative matrix factorization (NMF) to separate different sound sources with unique spectral-temporal patterns in an unsupervised approach. NMF is a self-learning algorithm which decomposes an input matrix into a spectral feature matrix and a temporal encoding matrix. Therefore, we can stack two or more layers of NMF to learn the spectral-temporal modulation of k sound sources without any learning database [1]. In this presentation, we will demonstrate the application of NMF in the separation of simultaneous sound sources appeared on a long-term spectrogram. In shallow water soundscape, the relative change of fish chorus can be effectively quantified even in periods with strong mooring noise [2]. In deep-sea soundscape, cetacean vocalizations, an unknown biological chorus, environmental sounds, and systematic noises can be efficiently separated [3]. In addition, we can use the features learned in procedures of blind source separation as the prior information for supervised source separation. The self-adaptation mechanism during iterative learning can help search the similar sound source from other acoustic dataset contains unknown noise types. Our results suggest that the NMF-based source separation can facilitate the analysis of the soundscape variability and the establishment of audio recognition database. Therefore, it will be feasible to investigate the acoustic interactions among geophysical events, soniferous marine animals, and anthropogenic activities from long-duration underwater recordings.

Improving acoustic monitoring of biodiversity using deep learning-based source separation algorithms

Mao-Ning Tuanmu1, Tzu-Hao Lin2, Joe Chun-Chia Huang1, Yu Tsao3, Chia-Yun Lee1

1Biodiversity Research Center, Academia Sinica, Taiwan
2Department of Marine Biodiversity Research, Japan Agency for Marine-Earth Science and Technology, Japan
3Research Center for Information Technology Innovation, Academia Sinica, Taiwan

Passive acoustic monitoring of the environment has been suggested as an effective tool for investigating the dynamics of biodiversity across spatial and temporal scales. Recent development in automatic recorders has allowed environmental acoustic data to be collected in an unattended way for a long duration. However, one of the major challenges for acoustic monitoring is to identify sounds of target taxa in recordings which usually contain undesired signals from non-target sources. In addition, high variation in the characteristics of target sounds, co-occurrence of sounds from multiple target taxa, and a lack of reference data make it even more difficult to separate acoustic signals from different sources. To overcome this issue, we developed an unsupervised source separation algorithm based on a multi-layer (deep) non-negative matrix factorization (NMF). Using reference echolocation calls of 13 bat species, we evaluated the performance of the multi-layer NMF in separating species-specific calls. Results showed that the multi-layer NMF, especially when being pre-trained with reference calls, outperformed the conventional supervised single-layer NMF. We also evaluated the performance of the multi-layer NMF in identifying different types of bat calls in recordings collected in the field. We found comparable performance in call types identification between the multi-layer NMF and human observers. These results suggest that the proposed multi-layer NMF approach can be used to effectively separate acoustic signals of different taxa from long-duration field recordings in an unsupervised manner. The approach can thus improve the applicability of passive acoustic monitoring as a tool to investigate the responses of biodiversity to the changing environment.

Advertisements

Monitoring of coral reef ecosystem: an integrated approach of marine soundscape and machine learning

Presented in International Symposium on Grids & Clouds 2018

Monitoring of coral reef ecosystem: an integrated approach of marine soundscape and machine learning

 Tzu-Hao Lin1, Tomonari Akamatsu2, Frederic Sinniger3, Saki Harii3, Yu Tsao1

1Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan
2National Research Institute of Fisheries Science, Japan Fisheries Research and Education Agency, Yokohama, Japan
3Tropical Biosphere Research Center, University of the Ryukyus, Okinawa, Japan

Coral reefs represent the most biologically diverse marine ecosystem, however, they are vulnerable to environmental changes and impacts. Therefore, information on the variability of environment and biodiversity is essential for the conservation management of coral reefs. In this study, a soundscape monitoring network of shallow and mesophotic coral reefs was established in Okinawa, Japan. Three autonomous sound recorders were deployed in water depths of 1.5 m, 20 m, and 40 m since May 2017. To investigate the soundscape variability, we applied the periodicity-coded nonnegative matrix factorization to separate biological sounds and the other noise sources displayed on long-term spectrograms. The separation results indicate that the coral reef soundscape varied among different locations. At 1.5 m depth, biological sounds were dominated by snapping shrimp sounds and transient fish calls. Although not knowing the specific source, noises were clearly driven by tidal activities. At 20 m and 40 m depths, biological sounds were dominated by nighttime fish choruses and noises were primary related to shipping activities. Furthermore, the clustering result indicates the complexity of biological sounds was higher in mesophotic coral reefs compare to shallow-water coral reefs. Our study demonstrates that the integration of machine learning in the analysis of soundscape is efficient to interpret the variability of biological sounds, environmental and anthropogenic noises. Therefore, the conservation management of coral reefs, especially those rarely studied such as mesophotic coral reefs, can be facilitated by the long-term monitoring of coral reef soundscape.

You can also check the slides of this talk.

PNC 2017 Annual Conference and Joint Meetings

2017/11/7-9 @ Tainan, Taiwan

Computing biodiversity change via a soundscape monitoring network

Tzu-Hao Lin, Yu Tsao
Research Center for Information Technology Innovation, Academia Sinica

Yu-Huang Wang
Taiwan Academy of Ecology

Han-Wei Yen
Academia Sinica Grid Computing Centre

Sheng-Shan Lu
Taiwan Forestry Research Institute

A monitoring network for biodiversity change is essential for wildlife conservation. In recent years, many soundscape monitoring projects have been carried out to investigate the diversity of vocalizing animals. However, the acoustic-based biodiversity assessment remains challenging due to the lack of sufficient recognition database and the inability to disentangle mixed sound sources. Since 2014, an Asian Soundscape monitoring project has been initiated in Taiwan. So far, there are 15 recording sites in Taiwan and three sites in Southeast Asia, with more than 20,000 hours of recordings archived in the Asian Soundscape. In this study, we employed the visualization of long-duration recordings, blind source separation, and clustering techniques, to investigate the spatio-temporal variations of forest biodiversity in the Triangle Mountain, Lienhuachih, and Taipingshan. On the basis of blind source separation, biological sounds, with prominent diurnal occurrence pattern, can be separated from the environmental sounds without any recognition database. Thus, clusters of biological sounds can be effectively identified and employed to measure the daily change in bioacoustic diversity. Our results show that the bioacoustic diversity was higher in the evergreen broad-leaved forest. However, the seasonal variation in bioacoustic diversity was most evident in the high elevation coniferous forest. This study demonstrates that a suitable integration of machine learning and ecoacoustics can facilitate the evaluation of biodiversity changes. In addition to biological activities, we can also measure the environmental variability from soundscape information. In the future, the Asian Soundscape will not only serve as an open database for soundscape recordings, but also will provide tools for analyzing the interactions between biodiversity, environment, and human activities.

If you are interested in this research, please check the full paper published in PNC 2017.

5th Joint Meeting of the Acoustical Society of America and Acoustical Society of Japan

2016/11/28-12/2 @ Honolulu, USA

Acoustic response of Indo-Pacific humpback dolphins to the variability of marine soundscape

Tzu-Hao Lin, Yu Tsao
Research Center for Information Technology Innovation, Academia Sinica

Shih-Hau Fang
Department of Electrical Engineering, Yuan Ze University

Chih-Kai Yang, Lien-Siang Chou
Institute of Ecology and Evolutionary Biology, National Taiwan University

Marine mammals can adjust their vocal behaviors when they encounter anthropogenic noise. The acoustic divergence among different populations has also been considered as the effect of ambient noise. The recent studies discover that the marine soundscape is highly dynamic; however, it remains unclear how marine mammals alter their vocal behaviors under various acoustic environments. In this study, autonomous sound recorders were deployed in western Taiwan waters between 2012 and 2015. Soundscape scenes were unsupervised classified according to acoustic features measured in each 5 min interval. Non-negative matrix factorization was used to separate different scenes and to inverse the temporal occurrence of each soundscape scene. Echolocation clicks and whistles of Indo-Pacific humpback dolphins, which represent the only marine mammal species occurred in the study area, were automatically detected and analyzed. The preliminary result indicates the soundscape scenes dominated by biological sounds are correlated with the acoustic detection rate of humpback dolphins. Besides, the dolphin whistles are much complex when the prey associated scene is prominent in the local soundscape. In the future, the soundscape information may be used to predict the occurrence and habitat use of marine mammals.

2017年動物行為生態研討會

2017/1/23-24 @ 高雄中山大學

應用機器學習探討海洋聲景變動與中華白海豚發聲活動之關聯

林子皓、曹昱
中央研究院資訊科技創新研究中心

方士豪
元智大學電機工程學系

鯨豚的發聲行為相當多變,不同族群可能會在各種環境音改變哨聲特徵﹐也會在遭遇人為噪音時改變聲音結構。海洋聲景是由環境音、動物音與人為噪音組成,具有高度變異的特性。儘管過去已有不少針對鯨豚發聲與單一音源的研究,但是對鯨豚如何在多變的海洋聲景且多重聲源相互重疊的狀況下改變行為仍不清楚。本研究透過水下錄音機,長期收錄2014年苗栗海域的海洋錄音。首先應用自動偵測器尋找中華白海豚水下聲音,再應用非負矩陣分解法學習海洋聲景中的主要聲源特徵。透過非監督式學習器,可以有效拆解長期時頻譜圖,視覺化呈現石首魚鳴唱、槍蝦聲音、環境與人為噪音等主要聲源的相對變化。利用廣義疊加模型分析聲景與白海豚聲音後,我們發現白海豚的聲音偵測率與複雜度和各種聲源皆有不同的相關性。此結果顯示應用機器學習分離聲景中的各種聲源之後,將能夠有效瞭解動物和各種聲源的交互作用。未來,聲景中的各種訊息也可以作為預測動物活動的生態遙測資料。

2016年臺灣地球科學聯合學術研討會

2016/5/20

近海與海岸環境 Land-Ocean Interactions in the Changing Coastal Zones of Taiwan:
Scientific Basis and Societal Engagements

應用非監督式分類方法調查海洋聲景的時空變化

林子皓
國立台灣大學生態學與演化生物學研究所

海洋聲景由環境音、動物聲音與人為噪音所組成,是由各種水下聲音所構築而成的音響環境。環境音可能來自於風浪、海流、地震等等自然事件,受到海床地形、水文的變化,聲音在各地傳播的路徑有所不同,進而塑造出獨特的音響環境。動物音主要來自於海洋動物發聲,也可能來自動物移動過程或水面活動伴隨發出的聲音。動物音具有高度複雜的變異性,以鯨豚和魚類為例,不同種的聲音特徵有所差別,但同種的聲音卻也有可能受到行為影響而有著不同的結構。海域的人為噪音則以船隻交通、海洋工程的噪音為主,依據接受強度的不同,噪音可能會使動物受到生理傷害、干擾行為、遮蔽溝通,長期暴露下也可能增加免疫壓力。因此,調查海洋聲景不只可以協助我們了解海洋環境特性、海洋動物的種類組成與活動特性,更可以了解人為噪音對海洋生態的影響。近年來隨著水下技術的發展,國際上開始廣泛應用自動錄音機收集長時間水下錄音調查海洋聲景的時空變化。然而目前仍缺乏完整資料庫辨認各種聲音,也難以利用人工分析巨量錄音,因此阻礙了海洋聲景生態學的發展。本研究運用資訊分析技術,解析海洋聲景的事件組成,以進一步了解海洋環境與生態的動態變化。在野外取回水下錄音之後,計算每五分鐘水下錄音的平均功率頻譜,以壓縮大量的錄音資料,並將一系列的平均功率頻譜組合成長期時頻譜圖做為視覺化分析海洋聲景的基礎資料。此外,將每五分鐘的平均功率頻譜作為分析參數,經過多變數分析方法減少特徵向量的維度之後,利用區分資料在多重維度空間內的分佈叢集,作為非監督式分類海洋聲景事件的分析架構。本研究應用自行開發的演算法分析苗栗中港溪口附近海域的水下錄音資料,結果顯示海洋聲景的事件組成在以泥沙底質為主的河口海域以及礁石為主的人工魚礁附近有明顯的結構性差異。海洋聲景在河口海域以較為安靜的環境音、以及夜晚出現的石首魚群鳴唱為主,但在人工魚礁附近則是以吵雜的槍蝦聲音、以及傍晚過後出現的低頻魚群鳴音為主要的事件。透過視覺化分析海洋聲景事件組成的時序變化,將可協助海洋生態研究人員進一步了解各地的海洋動物群聚組成與生態系統的動態變化,並提供海洋生態保育經營的重要基礎資料。

竊聽生態系的變化

竊聽生態系的變化?

可能很多人頭上會冒出很多問號,生態系的變化可以用眼睛觀察,但是要怎麼樣聽呢? 其實錄音也是一種不同於目視觀察的科學研究方法,透過收聽環境裡面的背景噪音(例如溪流、風聲和雨聲)、動物的叫聲(求偶聲、覓食聲)、以及人為活動的噪音(車聲、開發活動噪音)。最近一篇刊載Science的文章,就專文介紹了近年來科學界利用被動式聲學(passive acoustic monitoring)探討生物多樣性、生態系變動的研究方法。有興趣的人可以到下列網址閱讀全文,或在此觀看Science-2014-Servick-834-7

http://www.sciencemag.org/content/343/6173/834.full

在每一個環境中所聽到的每一個聲音都有其來源(生物性 or 非生物性),透過追蹤這些不同聲音的來源,其實我們可以推敲某些動物是否在此活動,或是了解當時環境的狀況。舉例來說,早晨時聽到雞鳴,即使我們沒有看到任何東西,我們也自然的會聯想到附近有一隻公雞。當我們聽到窗外的暴雨聲,我們會跟同事哀怨著又下起了午後雷陣雨。這些推測並不是來自於親自看到,而是來自於聽到的體驗。這個簡單的概念現在被應用在野生動物研究上,當我們知道昆蟲、蛙類、鳥類、哺乳動物發出的各種聲音,我們便可以利用這些聲音的出現時間來推敲這些發聲動物的主要活動時間,甚至了解一整個群聚的生態變動。

例如,當我們去爬郊山時,傍晚走在溪澗、山溝附近時,經常可以聽到許多種類的蛙鳴。但是隨著山區道路的開發,山溝漸漸水泥化,慢慢地你可能還是會聽到很大量的蛙鳴,但是聲音的種類變少了。我想這樣的狀況,就連國小六年級的學生都想到的,這個生態系到底發生了甚麼改變吧? 環境中的生態多樣性可能隨著山溝環境的改變而降低了。

那可能有人會問,如果要了解生態多樣性的改變,我們不就找人去調查就好了嗎? 的確,要真正的證實環境變化,最好的方法就是直接了解這個生態系的物種組成。但是人難找、錢也沒有阿! 生態系調查其實不是我們想像的這麼簡單,派幾個人去做做紀錄、翻翻圖鑑就好。這些調查人員往往需要長時間的訓練,更需要對分類物種的技術熟悉,甚至可能還要時時對現有的物種分類依據抱持懷疑。假使人有了,錢呢? 不要說政府對於環境監控的冷漠,就連一般企業和民眾可能也覺得我幹嘛要注意我周遭環境的變化勒,那不是政府的責任嗎? 我想正是整個社會對於環境監測的冷漠,造就了很多開發過程的衝突。

Ocean_soundscape

這張圖顯示了台灣西海岸的淺海聲音環境在兩天中的日夜變化。X軸是時間、Y軸是聲音的頻率(kHz)。在6/14和15的子夜前6個小時,都可以觀察到有一個明顯的噪音出現在10kHz以下的範圍,這個聲音其實就來自於一堆吵鬧的發聲魚類。另外偶爾會出現一些高頻(>10kHz)的噪音,這些聲音現在還無法確認其來源,但有可能來自於當地的槍蝦或是白海豚,很值得進一步的去探討。如果你仔細看,你還會看到每隔幾個小時就有一兩個在很低頻的間歇性噪音(主要<5kHz),這些頻繁出現的低頻噪音其實都來自於西海岸來往頻繁的船隻。

聲音監測在這個時候提供了另一個環境監測的可能選項。現在科技的進步已經把製造一台長時間錄音機的成本降到非常低,你可能用2萬元台幣就可以買到一台可以錄兩個禮拜以上的錄音機。此外,軟體演算法的開發,也已經進步到可以幫助人們處理這些大量的聲音資料,透過自動化偵測就可以知道你有興趣的聲音在何時出現。甚至把整個長時間錄音濃縮成一張圖,知道這個環境何時吵鬧、何時安靜,也都不是一件難事。

所以呢,關心環境的人們,如果你想知道一個環境的長時間變動,除了自己定時去採樣、觀察之外,在那邊放置一台定時錄音的錄音機,也將會大力幫助你對這個環境的了解喔!