Multiscale Attention-Based Instance Segmentation for Measuring Crystals With Large Size Variation
Neubauer T., Berg A., Wimmer M., Lenis D., Major D., Winter P. M., De Paolis G. R., Novotny J., Lüftner D., Reinharter K., Bühler K.
Online ahead of print
IEEE Transactions on Instrumentation and Measurement, December 2023
Quantitative measurement of crystals in high-resolution images allows for important insights into underlying material characteristics. Deep learning has shown great progress in vision-based automatic crystal size measurement, but current instance segmentation methods reach their limits with images that have large variations in crystal size or hard-to-detect crystal boundaries. Even small image segmentation errors, such as incorrectly fused or separated segments, can significantly lower the accuracy of the measured results. Instead of improving the existing pixel-wise boundary segmentation methods, we propose to use an instance-based segmentation method, which gives more robust segmentation results to improve measurement accuracy. Our novel method enhances flow maps with a size-aware multiscale attention (SiMA) module. The attention module adaptively fuses information from multiple scales and focuses on the most relevant scale for each segmented image area. We demonstrate that our proposed attention fusion strategy outperforms state-of-the-art (SOTA) instance and boundary segmentation methods, as well as a simple average fusion of multiscale predictions. We evaluate our method on a refractory raw material dataset of high-resolution images with large variations in crystal size and show that our model can be used to calculate the crystal size more accurately than existing methods.