Biology-Guided Neural Networks (BGNN) Research Involving Fishes

Project Members

Meghan Balk, Henry L. Bart Jr., Kelly Diamond, Mohannad Elhamod, Anuj Karpatne, Paula Mabee, M. Maruf

Project Goals

Leveraging NSF’s investment in this earlier HDR DIRSE-IL award, this project helped lay the foundation for the Imageomics Institute. Moreover, several BGNN research thrusts involving trait data extracted from images of fish specimens are continuing in the Imageomics Institute. The goal of these efforts is to apply KGML techniques to understand phenotypic trait evolution in fishes.

Project Overview

The initial focus of BGNN was building cyberinfrastructure for hosting and sharing images of fish specimens gathered from repositories supported by NSF’s Advancing Digitization of Biodiversity Collections (ADBC) program, including the Great Lakes Invasives (GLIN) Thematic Collections Network project, and the ADBC Digitization HUB, iDigBIO. The images were processed to verify/correct taxonomic names and capture metadata for a variety of image-quality metrics. The result of this effort was a web-accessible repository of 110,000, AI-ready, fish specimen images, hosted on servers at Tulane University. The images have been used in published research (Leipzig et al. 2021, Elhamod et al. 2022, Pepper et al. 2021) and are actively being used in several ongoing studies of the Imageomics Institute.

Building on earlier HDR DIRSE-IL award, the new Imageomics-specific research thrust involving fish specimen images is using small subsets of morphological data such as landmarks that define sizes and positions of key anatomical traits, quantitative data such as measurements of distances, areas, angles, proportions, colors, pigment patterns, histograms, and shape outlines extracted from fish specimen images. The data will be used to identify morphological features useful for distinguishing different taxonomic groups of fishes. The method, which Tulane collaborators Bart and Bakis have termed morphology barcoding, is being tested on the full set of 110,000 fish specimen images, representing more than 400 species of fishes, in the Tulane fish image repository.

 

Related Publications

Leipzig, J., Bakis, Y., Wang, X., Elhamod, M., Diamond, K., Dahdul, W., Karpatne, A., Maga, M., Mabee, P., Bart, H. L., & Greenberg, J. (2021). Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish Species. In E. Garoufallou & M.-A. Ovalle-Perandones (Eds.), Metadata and Semantic Research (pp. 3–12). Springer International Publishing. https://doi.org/10.1007/978-3-030-71903-6_1

Diamond, K. M., Rolfe, S. M., Kwon, R. Y., & Maga, A. M. (2022). Computational anatomy and geometric shape analysis enables analysis of complex craniofacial phenotypes in zebrafish. Biology Open, 11(2), bio058948. https://doi.org/10.1242/bio.058948

Pepper, J., J. Greenberg, Y. Bakiş, X. Wang, H. Bart and D. Breen, "Automatic Metadata Generation for Fish Specimen Image Collections," 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL), 2021, pp. 31-40, https://doi.org/10.1109/JCDL52503.2021.00015