AUTOMATED NEOTROPICAL FOSSIL POLLEN FABACEAE ANALYSIS USING CONVOLUTIONAL NEURAL NETWORKS

The deep learning method has shown great success in dealing with visual data. It extracts visual data features hierarchically from a raw signal (image, video, etc.) in a deep layer-wise fashion that encodes object shape and textural information for the targeted tasks. In our work, we focus on the problem of reconstructing ancestral morphologies and place fossil specimens on a phylogeny. Deep learning methods can improve the efficiency and quality of identifying fossil pollen from the Neotropics, where less than 50% of their modern affinities are known. For this study we imaged ~600 pollen grains from 55 species of Detarioideae and 32 pollen grains from a fossil legume using Airyscan confocal superresolution microscopy. Airyscan allows us to visualize both the external and internal 3D morphology of the pollen wall at a resolution comparable to electron microscopy. The images were analyzed using three models developed for this pollen identification task: 1) A maximum projection per sample that produce a single (top-down view) image, 2) a multi-instance learning model which takes as input a set of banded projections per sample along the focus view, and 3) a multi-view learning model that selects and combines features from both views/models. We first apply these three models to pollen identification over 16 modern pollen genera. We obtained high confidence scores for this classification, assessing the accuracy from all models. Then, we used the models to extract discriminative features over the pollen grains of extant genera for fossil pollen identification. Our results show that the three models were able to classify the extant species into the corresponding genus with high precision, and the multi-view learning model achieved the best performance with higher confidence scores. These models showed high confidence scores in the identification of the different fossil grains, as well as a consensus in the identification among models, constraining the modern affinity of the fossil legume. This study shows that training such deep models is able to capture discriminative and meaningful features of modern pollen grains for the identification of fossil palynomorphs, as well as presents a powerful new approach for analyzing palynological data.