Open-world deep learning applied to pollen detection

Feng, Jennifer T

Permalink

https://hdl.handle.net/2142/120168 Copy

Description

Title

Open-world deep learning applied to pollen detection

Author(s)

Feng, Jennifer T

Issue Date

2023-05-03

Director of Research (if dissertation) or Advisor (if thesis)

Punyasena, Surangi W

Committee Member(s)

• Kong, Shu
• Donders, Timme H
• Conroy, Jessica

Department of Study

Plant Biology

Discipline

Plant Biology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• continual learning
• deep learning
• domain gaps
• open-world
• palynology
• pollen grain detection
• rare species
• small grains
• taxonomic bias

Abstract

Fossil pollen-based paleoclimatic and paleoecological reconstructions rely on visual identifications that can be automated using computer vision. However, to date, the majority of automated approaches have focused on pollen classification, with few existing protocols for pollen detection and whole-slide image processing. As new slides potentially introduce rare and novel taxa, pollen detection is a non-trivial task in the open world. Our three experiments address significant but underexplored issues in automated pollen detection. We first addressed taxonomic bias – missed detections of smaller, rarer pollen types. We fused an expert model trained on this minority class with our general pollen detector. We next addressed domain gaps – differences in image magnification and resolution across microscopes – by fine-tuning our detector on images from a new imaging domain. Lastly, we developed continual learning workflows that integrated expert feedback and allowed detectors to improve over time. We simulated human-in-the-loop annotation of three microscope slides by using a trained detector to detect specimens on new slides, validating the detections and tagging incorrect detections, and using the corrected detections to re-train the detector for the new time period. In our experiment addressing taxonomic bias, fusing the expert model with the general detector improved the detection performance measured by mean average precision (mAP) from 73.21% to 75.09%, and increased detection recall by 2%. Recall at the 20% precision level for three small-grained taxa increased 11%, 25%, and 50%. In our experiment addressing domain gaps, fine-tuning the general detector on images from the new domain increased mAP in the new domain from 32.56% to 65.99%. In our experiment addressing continual learning, we increased mAP in each of three time periods using human-in-the-loop annotations and increased mAP on a held-out validation set from 41.10% to 59.93%. Effective pollen detectors open new avenues of paleoecology research, creating long-term, high-quality observational records for paleoclimate analyses, improving the accuracy of diversity estimates, and helping with the discovery of rare pollen types in deep-time material. Our methods can be applied to other visually diverse biological data, including algae, fungal spores, and plant cuticle.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Jennifer T. Feng

Owning Collections