Few shot semantic segmentation algorithm based on meta-learning
WANG Lanzhong1, MU Changshan2
1. School of Foreign Languages and Literature, Shandong University, Jinan, Shandong 250100, China； 2. Information Center, Shandong Provincial Tax Service, State Taxation Administration, Jinan, Shandong 250002, China
Abstract：To solve the problem of low segmentation accuracy for unknown novel classes in existing few shot semantic segmentation models, the few shot semantic segmentation algorithm based on meta-learning was proposed. The depth-separable convolutions were utilized to improve the traditional backbone network, and the encoder pre-training on the ImageNet dataset was performed. The pre-trained backbone network was used to map the support and query images into deep feature space. Using the ground truth masks of the support images, the support features were separated into object foreground and background, and the adaptive meta-learning classifier was constructed using vision transformer. The extensive experiments on the PASCAL-5i dataset were completed. The results show that the proposed model achieves mIoU (mean Intersection over Union) (1 shot) of 47.1%, 58.3% and 60.4% on VGG-16, ResNet-50 and ResNet-101 backbone networks, respectively, and it achieves mIoU of 49.6%, 60.2% and 62.1% under the 5 shot setting. On the COCO-20i dataset, mIoU (1 shot) values of 23.6%, 30.3% and 30.7% are achieved with mIoU values of 30.1%, 34.7% and 35.2% under the 5 shot setting.
王兰忠1, 牟昌善2. 基于元学习的小样本语义分割算法[J]. 江苏大学学报（自然科学版）, 2024, 45(5): 574-580.
WANG Lanzhong1, MU Changshan2. Few shot semantic segmentation algorithm based on meta-learning[J]. Journal of Jiangsu University(Natural Science Eidtion)
, 2024, 45(5): 574-580.
YAO Q A, ZHANG X, LIU L M, et al. Image semantic segmentation based on fusing attention mechanism and multi-scale features[J]. Journal of Jilin University (Science Edition),2022,60(6):1383-1390.(in Chinese)
［2］
SHABAN A, BANSAL S, LIU Z, et al. One-shot lear-ning for semantic segmentation[C]∥Proceedings of the 28th British Machine Vision Conference.[S.l.]: BMVA Press, DOI:10.5244/c.31.167.
［3］
WANG K X, LIEW J H, ZOU Y T, et al. PANet: few-shot image semantic segmentation with prototype alignment[C]∥Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE,2019:9196-9205.
［4］
LI G, JAMPANI V, SEVILLA-LARA L, et al. Adaptive prototype learning and allocation for few-shot segmentation[C]∥Proceedings of the 2021 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Piscataway: IEEE Computer Society, 2021:8330-8339.
［5］
LIU B H, JIAO J B, YE Q X. Harmonic feature activation for few-shot semantic segmentation[J]. IEEE Tran-sactions on Image Processing, 2021,30:3142-3153.
［6］
JOHNANDER J, EDSTEDT J, DANELLJAN M, et al. Deep Gaussian processes for few-shot segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021:5217-5226.
［7］
PAMBALA A K, DUTTA T, BISWAS S. SML: semantic meta-learning for few-shot semantic segmentation[J]. Pattern Recognition Letters, 2021,147:93-99.
［8］
LIU W D, ZHANG C, LIN G S, et al. CRNet: cross-reference networks for few-shot segmentation[C]∥Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Computer Society, 2020:4164-4172.
［9］
RAKELLY K，SHELHAMER E，DARRELL T，et al. Conditional networks for few-shot semantic segmentation[C]∥Proceedings of the 6th International Conference on Learning Representations.[S.l.]: International Conference on Learning Representations, 2018:517-526.
［10］
ZHANG X L, WEI Y C, YANG Y, et al. SG-one: simi-larity guidance network for one-shot semantic segmentation[J]. IEEE Transactions on Cybernetics, 2020,50(9):3855-3865.
［11］
LIU Y F, ZHANG X Y, ZHANG S Y, et al. Part-aware prototype network for few-shot semantic segmentation[C]∥Proceedings of the 16th European Conference on Computer Vision.[S.l.]: Springer Science and Business Media Deutschland GmbH,2020:142-158.
［12］
YANG B Y, LIU C, LI B H, et al. Prototype mixture models for few-shot semantic segmentation[C]∥Proceedings of the 16th European Conference on Computer Vision.[S.l.]: Springer Science and Business Media Deutschland GmbH, 2020:763-778.
［13］
ZHANG X L, WEI Y C, LI Z,et al. Rich embedding features for one-shot semantic segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022,33(11):6484-6493.
LIU Y X, MENG F M, LI H L, et al. A few shot seg-mentation method combining global and local similarity［J］. Journal of Beijing University of Aeronautics and Astronautics, 2021,47(3):665-674. (in Chinese)
［15］
NGUYEN K，TODOROVIC S. Feature weighting and boosting for few-shot segmentation[C]∥Proceedings of the 17th IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019:622-631.
［16］
WANG H C, ZHANG X D, HU Y T, et al. Few-shot semantic segmentation with democratic attention networks[C]∥Proceedings of the 16th European Confe-rence on Computer Vision. Berlin: Springer Science and Business Media Deutschland GmbH, 2020:730-746.
LI W J, LI W H. Instance segmentation method based on compressed representation[J]. Journal of Jilin University(Science Edition), 2023,61(4):883-889.(in Chinese)
DENG X Q, LI Z, WANG Y L. An improved endosco-pic polyp image segmentation algorithm based on U-Net[J]. Journal of Sichuan University (Natural Science Edition), DOI:10.19907/j.0490-6756.2024.013004. (in Chinese)
2024, Vol. 45 
