Abstract：To solve the problems of the existing depth acquisition methods with missing data and low resolution, the novel depth enhancement method based on soft clustering was proposed and named the soft clustering solver. By the strong edge-preservation of the soft clustering method, the accuracy of depth enhancement could be improved. The affinity matrix derived from the soft clustering was combined with the weighted least square model to establish the confidence-weighted least square model in the solver, and the iteratively based solution method was proposed. To evaluate the proposed method, the experiments on several depth enhancement tasks of depth inpainting, depth super-resolution and depth rectification were conducted. Various evaluation metrics were used, including peak signal to noise ratio(PSNR), structural similarity index measure(SSIM), rooted mean squared error(RMSE) and running time. The results show that for depth inpainting, the average PSNR reaches 42.28 with average SSIM of 98.83%. For depth super-resolution and depth rectification, the average RMSE values are 8.96 and 2.36, respectively. By the proposed method, the image with resolution of 2 048×1 024 pixels can be processed with only 5.03 s.
ZHANG X L, DAI L Q. Fast bilateral filtering[J]. Electronics Letters, 2019,55(5):258-260.
［2］
LI Y W, LI Z G, ZHENG C B, et al. Adaptive weighted guided image filtering for depth enhancement in shape-from-focus[J]. Pattern Recognition,DOI: 10.1016/j.patcog.2022.108900.
［3］
HUANG Z H, ZHU Z F, AN Q, et al. Global-local image enhancement with contrast improvement based on weighted least squares[J]. Optik，DOI: 10.1016/j.ijleo.2021.167433.
［4］
GASTAL E S L, OLIVEIRA M M. Domain transform for edge-aware image and video processing[J]. ACM Transactions on Graphics, DOI:10.1145/1964921.1964964.
［5］
LIM H B, KIM E S, LEE D M, et al. Upsampling of 16-channel LiDAR depth data using weighted median filter[J]. Journal of Institute of Control, Robotics and Systems, 2021,27(5):356-363.
［6］
NAYAK R, PATRA D. New single-image super-resolution reconstruction using MRF model[J]. Neurocomputing, 2018,293:108-129.
［7］
VASU G T, PALANISAMY P. Multi-focus image fusion using anisotropic diffusion filter[J]. Soft Computing, 2022,26(24):14029-14040.
［8］
TAKALO R, HYTTI H, IHALAINEN H, et al. Edge-preserving adaptive autoregressive model for Poisson noise reduction[J]. Nuclear Medicine Communications, 2021,42(6):707-710.
［9］
LI Y, MIN D B, DO M N, et al. Fast guided global interpolation for depth and motion[C]∥Proceedings of the 14th European Conference on Computer Vision. Heidelberg：Springer Verlag，2016：717-733.
［10］
BARRON J T, POOLE B. The fast bilateral solver[C]∥Proceedings of the 14th European Conference on Computer Vision. Heidelberg：Springer Verlag，2016：617-632.
［11］
BAPAT A, FRAHM J M. The domain transform solver[C]∥ Proceedings of the 2019 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE Computer Society, 2019:6007-6016.
［12］
YANG Y, HUI H J, ZENG L L, et al. Edge-preserving image filtering based on soft clustering[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022,32(7):4150-4162.
［13］
WEI M Q, YAN Q G, LUO F, et al. Joint bilateral propagation upsampling for unstructured multi-view stereo[J]. Visual Computer, 2019, 35(6/7/8):797-809.
［14］
LIU S H, LI X F, ZHANG X L. Remote sensing image fusion algorithm based on mutual-structure for joint filtering using saliency detection[J]. Journal of Electronic Imaging, DOI:10.1117/1.JEI.28.3.033007.
［15］
HAN K. Fingerprint image enhancement processing method based on weighted median filtering[C]∥Proceedings of the 20th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing. New York：IEEE，2019：114-120.
［16］
GODARD C, AODHA O M, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]∥Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. New York：IEEE，2017：6602-6611.
