Image processing with Generative Adversarial Networks (GANs) with limited data availability

Artificial neural networks are used for complex image segmentation tasks. One use case is the cell nuclei segmentation in human tissue. The segmentation can be used for subsequent analyses like thyroid gland tumor classification in downstream tasks [1, 2].

A neural network is used to create an instance segmentation of cell nuclei in human tissue. The segmentation and the tissue image are used by a downstream task to classify the tissue.


Supervised methods require labeled training data, while training data is usually labeled manually. Manual labeling is time-consuming, expensive and prone to errors. If annotation is performed by multiple people, inter-observer variability is high. Generative Adversarial Networks (GANs) can be used to substitute manual labeling. This is done by creating an unpaired dataset of synthetic label images and subsequent training of a GAN. The GAN performs unpaired image-to-image translation and learns to transform images from one domain to another and vice-versa. After training, the GAN can be used to create a fully synthetic datasets. In our work, we investigate the influence of synthetic label images on the final performance on GANs and develop pipelines for label synthesis [3]. Furthermore, synthesis of large-scale 2D and 3D data is examined.  This is especially useful for biological and biomedical data, as well as for material science.

A GAN is trained with real-world images and unpaired synthetic label images. Afterwards, it is used to create a paired synthetic dataset. The synthetic dataset can e.g. be used to train a neural network for segmentation.


[1] M. Böhland, L. Tharun, et al.: “Machine learning methods for automated classification of tumors with papillary thyroid carcinoma-like nuclei: A quantitative analysis,” PLOS ONE, vol. 16, no. 9, pp. 1–21, 2021 

[2] M. Böhland, O. Neumann, et al.: “Ciscnet - a Single-Branch Cell Nucleus Instance Segmentation and Classification Network,” in 2022 IEEE Int. Symp. Biomed. Imag. Challenges (ISBIC), 2022, pp. 1–5. 

[3] M. Böhland, T. Scherr, et al.: Influence of synthetic label image object properties on GAN supported segmentation pipelines. In: Proceedings 29th Workshop Computational Intelligence, pp. 289–305 (2019)

Completed Project