An enhanced framework of generative adversarial networks (EF-GANs) for environmental microorganism image augmentation with limited rotationinvariant training data

Hao Xu, Chen Li, Md Mamunur Rahaman, Yudong Yao, Zihan Li, Jinghua Zhang, Frank Kulwa, Xin Zhao, Shouliang Qi, Yueyang Teng

Research output: Contribution to journalArticlepeer-review

31 Scopus citations

Abstract

The main obstacle to image augmentation with Generative Adversarial Networks (GANs) is the need for a large amount of training data, but this is difficult for small datasets like Environmental Microorganisms (EMs). EM image analysis plays a vital role in environmental monitoring and protection, but it is often encountered with small datasets due to the difficulty of EM image collection. To this end, we propose an Enhanced Framework of GANs (EF-GANs) that combines geometric transformation methods and GANs for EM image augmentation. First of all, the color of an EM image has an insignificant impact on its class label, based on this fact, we perform color space augmentation to the original EM images. Secondly, we train EF-GANs with augmented EM images to generate utterly new EM images. Finally, we rotate the generated samples in various directions to obtain a more natural performance. In this study, we use VGG16 and ResNet50 networks to evaluate the proposed EF-GANs on 21 different types of EMs (420 EM images). It is observed that the average precision (AP) of VGG16 increases between 4.5% and 84.1% in 20 EM classes and one class remains unchanged. The AP of Resnet50 rises between 8.7% and 38.7% in 12 EM classes and reaches 100% in two EM classes. Furthermore, to reflect the generalization performance of EF-GANs, we employ an utterly new EM image dataset (630 EM images) to test the previous VGG16 networks. We select the VGG16 networks with original and optimal settings for all the EM classes, and for testing, optimal settings for a single EM class is considered. In the 20 of 21 one-vs-rest EM image classification tasks, the AP of VGG16 increases between 1.66% and 88.1%. The results demonstrate that the proposed EF-GANs can achieve outstanding performance in augmenting single EM images with high quality and resolution, thus, to improve the APs of EM image classification.

Original languageEnglish
Pages (from-to)187455-187469
Number of pages15
JournalIEEE Access
Volume8
DOIs
StatePublished - 2020

Keywords

  • Environmental microorganism
  • Generative adversarial networks
  • Image analysis
  • Image augmentation
  • Image classification
  • Microscopic image
  • Small dataset

Fingerprint

Dive into the research topics of 'An enhanced framework of generative adversarial networks (EF-GANs) for environmental microorganism image augmentation with limited rotationinvariant training data'. Together they form a unique fingerprint.

Cite this