A Review on the Deep Learning-based Surface Reconstruction from the
Point Clouds

Chengfa      He; Huahao      Shou; Jiali      Zhou
doi:10.2174/1872212118666230626124718
Abstract

Background: Point cloud has become one of the most important data formats for 3D presentation because of the increased availability of acquisition devices and its wide applications. Deep learning has the most powerful ability to capture features from data and has successfully solved various problems in the field of image, such as classification, segmentation, and generation. Deep learning is commonly used to process data with a structured grid, while point cloud is irregular and unstructured. The irregularity of point clouds makes it difficult to use deep learning to solve the problems represented by point clouds. Recently, numerous approaches have been proposed to process point clouds with deep learning to solve various problems.
Objective: The objective of this study is to serve as a guide to new scholars in the field of deep learning on 3D surface reconstruction from point clouds as it presents the recent progress in deep learning-based surface reconstruction for point clouds. It helps scholars to grasp the current research situation better and further explore the search direction.
Method: This study reviews the recent progress in deep learning-based methods used for surface reconstruction from point clouds and large-scale 3D point cloud benchmark datasets commonly used.
Results: Several relevant articles on deep learning used for surface reconstruction from point clouds and some recent patents on deep learning applications are collected and reviewed in this paper. The difficulty of irregularity of point clouds can be overcome by deep learning methods, thus achieving remarkable progress in surface reconstruction.
Conclusion: Deep learning for 3D surface reconstruction from point clouds is becoming a research hotspot due to its performance in terms of anti-interference and generalization. Although the advance is remarkable, there are still some challenges that need to be further studied.
Graphical Abstract

[1]
H. Fathi,  and I. Brilakis, "Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features", Adv. Eng. Inform., vol. 25, no. 4, pp. 760-770, 2011.
 [http://dx.doi.org/10.1016/j.aei.2011.06.001]
[2]
Y. Guo, F. Sohel, M. Bennamoun, M. Lu,  and J. Wan, "Rotational projection statistics for 3D local surface description and object recognition", Int. J. Comput. Vis., vol. 105, no. 1, pp. 63-86, 2013.
 [http://dx.doi.org/10.1007/s11263-013-0627-y]
[3]
Y. Guo, M. Bennamoun, F. Sohel, M. Lu,  and J. Wan, "3D object recognition in cluttered scenes with local surface features: A survey", In: IEEE Trans. Pattern Anal. Mach. Intell., vol. 36. 2014, no. 11, pp. 2270-2287.
 Novemver [http://dx.doi.org/10.1109/TPAMI.2014.2316828]
[4]
J.H. Park, D.H. Seo, M.S. Ku, I.Y. Jung,  and C.S. Jeong, "Multiple 3D Object Tracking using ROI and Double Filtering for Augmented Reality", In 2011 Fifth FTRA International Conference on Multimedia and Ubiquitous Engineering, Crete, Greece, 2011, pp. 317-322 
 [http://dx.doi.org/10.1109/MUE.2011.64]
[5]
X. Chen, H. Ma, J. Wan, B. Li,  and T. Xia, "Multi-view 3D object detection network for autonomous driving", In In Proc. IEEE Conf. Comput. Vis. Pattern Recognit, Honolulu, HI, USA, 2017, pp. 6526-6534 
 [http://dx.doi.org/10.1109/CVPR.2017.691]
[6]
C. Janiesch, P. Zschech,  and K. Heinrich, "Machine learning and deep learning", Electron. Mark., vol. 31, no. 3, pp. 685-695, 2021.
 [http://dx.doi.org/10.1007/s12525-021-00475-2]
[7]
Y. LeCun, Y. Bengio,  and G. Hinton, "Deep learning", nature, vol. 521, no. 7553, pp. 436-444, 2015.
[8]
M. Hassaballah,  and A.I. Awad, Deep learning in computer vision: principles and applications., CRC Press, 2020.
 [http://dx.doi.org/10.1201/9781351003827]
[9]
A.B. Nassif, I. Shahin, I. Attili, M. Azzeh,  and K. Shaalan, "Speech recognition Using deep neural networks: A systematic review", IEEE Access, vol. 7, pp. 19143-19165, 2019.
 [http://dx.doi.org/10.1109/ACCESS.2019.2896880]
[10]
D.W. Otter, J.R. Medina,  and J.K. Kalita, "A survey of the usages of deep learning for natural language processing", IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 2, pp. 604-624, 2021.
 [http://dx.doi.org/10.1109/TNNLS.2020.2979670] [PMID:  32324570]
[11]
S. Strong, D. Murr,  and L.P. Dyrud, "Systems and methods for automatic estimation of object characteristics from digital images",  U.S. Patent 10,943,149,B2, 2021.
[12]
S. Bhaviripudi,  and S. Gayaka, "Classification, search and retrieval of semiconductor processing metrology images using deep learning/convolutional neural networks",  U.S. Patent 10,504,006,B2, 2019
[13]
D.Z. Chen, Y. Zhang, L. Yang, M.T-C. Ying,  and A.T. Ahuja, "Segmenting ultrasound images",  U.S. Patent 10,957,045,B2, 2021
[14]
W.J. Steigauf, B. Strong,  and S. Werb, "Medical evaluation machine learning workflows and processes",  U.S. Patent 10,937,164,B2, 2021
[15]
R.R.Q. Charles, H. Su, M. Kaichun,  and L.J. Guibas, "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation", In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 77-85 
 [http://dx.doi.org/10.1109/CVPR.2017.16]
[16]
C.R. Qi, L. Yi, H. Su,  and L.J. Guibas, "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space", In: Advances in Neural Information Processing Systems., vol. 30. 2017. Available From: https://proceedings.neurips.cc/paper/2017/hash/d8bf84be3800d12f74d8b05e9b89836f-Abstract.html [Accessed: Oct. 17, 2022].
[17]
Y. Rao, J. Lu,  and J. Zhou, "Spherical Fractal Convolutional Neural Networks for Point Cloud Recognition", In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 452-460 
 [http://dx.doi.org/10.1109/CVPR.2019.00054]
[18]
L. Ma, Y. Li, J. Li, W. Tan, Y. Yu,  and M.A. Chapman, "Multi-Scale Point-Wise Convolutional Neural Networks for 3D Object Segmentation From LiDAR Point Clouds in Large-Scale Environments", IEEE Trans. Intell. Transp. Syst., vol. 22, no. 2, pp. 821-836, 2021.
 [http://dx.doi.org/10.1109/TITS.2019.2961060]
[19]
Y. Zhou,  and O. Tuzel, "VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection", In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 4490-4499 
 [http://dx.doi.org/10.1109/CVPR.2018.00472]
[20]
J. Zhang, X. Zhao, Z. Chen,  and Z. Lu, "A Review of Deep Learning-Based Semantic Segmentation for Point Cloud", IEEE Access, vol. 7, pp. 179118-179133, 2019.
 [http://dx.doi.org/10.1109/ACCESS.2019.2958671]
[21]
W. Liu, J. Sun, W. Li, T. Hu,  and P. Wang, "Deep Learning on Point Clouds and Its Application: A Survey", Sensors, vol. 19, no. 19, p. 4188, 2019.
 September [http://dx.doi.org/10.3390/s19194188] [PMID: 31561639]
[22]
S.A. Bello, S. Yu, C. Wang, J.M. Adam,  and J. Li, "Deep learning on 3D point clouds", Remote Sens. (Basel), vol. 12, no. 11, p. 1729, 2020.
 [http://dx.doi.org/10.3390/rs12111729]
[23]
Y. Xie, J. Tian,  and X.X. Zhu, "Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation", IEEE Geosci. Remote Sens. Mag., vol. 8, no. 4, pp. 38-59, 2020.
 [http://dx.doi.org/10.1109/MGRS.2019.2937630]
[24]
H. Lu,  and H. Shi, "Deep Learning for 3D Point Cloud Understanding: A Survey", In: CoRR, vol. abs/2009.08920. 2020. Available From: https://arxiv.org/abs/2009.08920 [Accessed: Sept. 23, 2020].
[25]
Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu,  and M. Bennamoun, "Deep Learning for 3D Point Clouds: A Survey", IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 12, pp. 4338-4364, 2021.
 [http://dx.doi.org/10.1109/TPAMI.2020.3005434] [PMID:  32750799]
[26]
B. Maxim,  and S. Nedevschi, "A survey on the current state of the art on deep learning 3D reconstruction", In 2021 IEEE 17th International
Conference on Intelligent Computer Communication and
Processing (ICCP), vol. 2021, Cluj-Napoca, Romania, 2021, pp. 283-290 
 [http://dx.doi.org/10.1109/ICCP53602.2021.9733639]
[27]
X.F. Han, H. Laga,  and M. Bennamoun, "Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era", IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 5, pp. 1578-1604, 2021.
 [http://dx.doi.org/10.1109/TPAMI.2019.2954885] [PMID:  31751229]
[28]
A. Khatamian,  and H.R. Arabnia, "Survey on 3D Surface Reconstruction", J. Inf. Process. Syst., vol. 12, no. 3, pp. 338-357, 2016.
[29]
M. Berger, A. Tagliasacchi, L.M. Seversky, P. Alliez, G. Guennebaud, J.A. Levine, A. Sharf,  and C.T. Silva, "A survey of surface reconstruction from point clouds", Comput. Graph. Forum, vol. 36, no. 1, pp. 301-329, 2017.
 [http://dx.doi.org/10.1111/cgf.12802]
[30]
F. Bogo, J. Romero, M. Loper,  and M.J. Black, "FAUST: Dataset and Evaluation for 3D Mesh Registration", In 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 3794-3801 
 [http://dx.doi.org/10.1109/CVPR.2014.491]
[31]
F. Bogo, J. Romero, G. Pons-Moll,  and M.J. Black, "Dynamic FAUST: Registering Human Bodies in Motion", In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 5573-5582 
 [http://dx.doi.org/10.1109/CVPR.2017.591]
[32]
A.X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi,  and F. Yu, "ShapeNet: An Information-Rich 3D Model Repository", CoRR, vol. abs/1512.03012, 2015. Available From: http://arxiv.org/abs/1512.03012 [Accessed: Dec. 09, 2020].
[33]
Y. Xiang, R. Mottaghi,  and S. Savarese, "Beyond PASCAL: A benchmark for 3D object detection in the wild", In IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, CO, USA, 2014, pp. 75-82 
 [http://dx.doi.org/10.1109/WACV.2014.6836101]
[34]
Q. Zhou,  and A. Jacobson, "Thingi10K: A Dataset of 10,000 3D-Printing Models", CoRR, vol. abs/1605.04797, 2016. Available From: http://arxiv.org/abs/1605.04797 [Accessed: Jul.
01, 2016]. [Online].
[35]
A. Chang, A. Dai,  and T.A. Funkhouser, "Matterport3D: Learning from RGB-D Data in Indoor Environments", In 2017 International Conference on 3D Vision (3DV), Qingdao, China, 2017, pp. 667-676 
 [http://dx.doi.org/10.1109/3DV.2017.00081]
[36]
S. Koch, A. Matveev, Z. Jiang, F. Williams, A. Artemov, E. Burnaev, M. Alexa, D. Zorin,  and D. Panozzo, "ABC: A Big CAD Model Dataset for Geometric Deep Learning", In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 9593-9603 
 [http://dx.doi.org/10.1109/CVPR.2019.00983]
[37]
A. Handa, V. Patraucean, S. Stent,  and R. Cipolla, "SceneNet: An annotated model generator for indoor scene understanding", In 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016, pp. 5737-5743 
 [http://dx.doi.org/10.1109/ICRA.2016.7487797]
[38]
S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys,  and A. Geiger, "Convolutional occupancy networks", arXiv:2003.04618, 2020.
 [http://dx.doi.org/10.1007/978-3-030-58580-8_31]
[39]
Z. Huang, Y. Wen, Z. Wang, J. Ren,  and K. Jia, "Surface Reconstruction from Point Clouds: A Survey and a Benchmark", arXiv:2205.02413, 2022. https://arxiv.org/abs/2205.02413 Online
[40]
W. Wohlkinger, A. Aldoma, R.B. Rusu,  and M. Vincze, "3DNet: Large-scale object class recognition from CAD models", In 2012 IEEE International Conference on Robotics and Automation, St Paul, MN, USA, 2012, pp. 5384-5391 
 [http://dx.doi.org/10.1109/ICRA.2012.6225116]
[41]
V. Albertina, V. Kunsthistorisches Museum, V. Theater Museum,  and P. Musée Guimet, "des Monuments français, Cité de
l’architecture et du patrimoine, D. des sculptures de la Ville de Paris,
P. Musée Carnavalet, L. The Collection, L. Usher Gallery, M. A.
N. di Firenze, and B. KODE Artmuseums, “Three d scans”",  [Online]. Available From: https://threedscans.com/ [Accessed: Mar. 07, 2023].
[42]
H. Fu, B. Cai, L. Gao, L.X. Zhang, J. Wang, C. Li, Q. Zeng, C. Sun, R. Jia, B. Zhao,  and H. Zhang, "3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics", In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 10913-10922 
 [http://dx.doi.org/10.1109/ICCV48922.2021.01075]
[43]
J. Straub, T. Whelan, L. Ma, Y. Chen, E. Wijmans, S. Green,  and J. J. Engel, "The Replica Dataset: A Digital Replica of Indoor Spaces",  arXiv, vol. abs/1906.05797, June 2019. Available From: http://arxiv.org/abs/1906.0
[44]
W.E. Lorensen,  and H.E. Cline, "Marching cubes: A high resolution 3D surface construction algorithm", Comput. Graph., vol. 21, no. 4, pp. 163-169, 1987.
 [http://dx.doi.org/10.1145/37402.37422]
[45]
H. Oleynikova, Z. Taylor, M. Fehr, R. Siegwart,  and J. Nieto, "Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning", In 2017 IEEE/RSJ International Conference
on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 2017, pp. 1366-1373 
 [http://dx.doi.org/10.1109/IROS.2017.8202315]
[46]
L. Ladický, O. Saurer, S. Jeong, F. Maninchedda,  and M. Pollefeys, "From Point Clouds to Mesh Using Regression", In 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 3913-3922 
 [http://dx.doi.org/10.1109/ICCV.2017.420]
[47]
J.J. Park, P. Florence, J. Straub, R. Newcombe,  and S. Lovegrove, "DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation", In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 165-174 
 [http://dx.doi.org/10.1109/CVPR.2019.00025]
[48]
P. Erler, P. Guerrero, S. Ohrhallinger, N.J. Mitra,  and M. Wimmer, "Points 2 Surf Learning Implicit Surfaces from Point Clouds", In: Computer Vision – ECCV 2020, Glasgow: U.K., 2020.
 [http://dx.doi.org/10.1007/978-3-030-58558-7_7]
[49]
M. Atzmon,  and Y. Lipman, "SAL: Sign Agnostic Learning of Shapes From Raw Data", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 2562-2571 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00264]
[50]
M. Atzmon,  and Y. Lipman, "SALD: Sign Agnostic Learning with Derivatives",  arXiv, vol. abs/2006.05400, October 2020. Available
From: https://arxiv.org/abs/2006.05400 [Accessed: Oct. 08, 2022].
[51]
A. Basher, M. Sarmad,  and J. Boutellier, "LightSAL: Lightweight Sign Agnostic Learning for Implicit Surface Representation",  CoRR, vol. abs/2103.14273, 2021. Available From: https://arxiv.org/abs/2103.14273 [Accessed: Oct. 08, 2022].
[52]
A. Gropp, L. Yariv, N. Haim, M. Atzmon,  and Y. Lipman, "Implicit Geometric Regularization for Learning Shapes",  CoRR, vol.
abs/2002.10099, 2020. Available From: https://arxiv.org/abs/2002.10099 [Accessed: Oct 25, 2022].
[53]
S. Lombardi, M.R. Oswald,  and M. Pollefeys, "Scalable Point Cloud-based Reconstruction with Local Implicit Functions", In 2020 International Conference on 3D Vision (3DV), Fukuoka, Japan, 2020, pp. 997-1007 
 [http://dx.doi.org/10.1109/3DV50981.2020.00110]
[54]
D.P. Kingma,  and M. Welling, "Auto-Encoding Variational Bayes", arXiv:1312.6114.http://arxiv.org/abs/1312.6114
[55]
B. Ummenhofer,  and V. Koltun, "Adaptive Surface Reconstruction With Multiscale Convolutional Kernels", In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 2021, pp. 5651-5660 
 [http://dx.doi.org/10.1109/ICCV48922.2021.00560]
[56]
W. Zhao, J. Lei, Y. Wen, J. Zhang,  and K. Jia, "Sign-Agnostic Implicit Learning of Surface Self-Similarities for Shape Modeling and Reconstruction From Raw Point Clouds", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Nashville, TN, USA, 2021, pp. 10256-10265 
 [http://dx.doi.org/10.1109/CVPR46437.2021.01012]
[57]
B. Ma, Z. Han, Y-S. Liu,  and M. Zwicker, "Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces",  CoRR, vol. abs/2011.13495 https://arxiv.org/abs/2011.13495 Online
[58]
B. Ma, Y-S. Liu,  and Z. Han, "Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 6305-63 
 [http://dx.doi.org/10.1109/CVPR52688.2022.0062115]
[59]
Z. Wang, P. Wang,  and Q. Dong, "Neural-IMLS: Learning Implicit Moving Least-Squares for Surface Reconstruction from Unoriented Point clouds",  CoRR, vol. abs/2109.04398",
arXiv:2109.04398, 2021. https://arxiv.org/abs/2109.04398 Online
[60]
Y. Ben-Shabat, C.H. Koneputugodage,  and S. Gould, "DiGS: Divergence guided shape implicit neural representation for unoriented point clouds", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 19301-19310 
 [http://dx.doi.org/10.1109/CVPR52688.2022.01872]
[61]
V. Sitzmann, J. Martel, A. Bergman, D. Lindell,  and G. Wetzstein, "Implicit Neural Representations with Periodic Activation Functions", In 34th Conference on Neural Information Processing Systems, Vancouver, Canada, 2020, pp. 7462-7473 
[62]
D.B. Lindell, D. Van Veen, J.J. Park,  and G. Wetzstein, "Bacon: Band-limited Coordinate Networks for Multiscale Scene Representation", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Orleans, LA, USA, 2022, pp. 16231-16241 
 [http://dx.doi.org/10.1109/CVPR52688.2022.01577]
[63]
R. Fathony, A.K. SahuS, D. Willmott,  and J.Z. Kolter, "Multiplicative Filter Networks", In International Conference on Learning Representations, Virtual Only, 2021.  [Online] Available From: https://openreview.net/forum?id=OmtmcPkkhT [Accessed: Oct. 25, 2022].
[64]
X. Zheng, Y. Liu, P. Wang,  and X. Tong, "SDF-StyleGAN: Implicit SDF-based styleGAN for 3D shape generation", Comput. Graph. Forum, vol. 41, no. 5, pp. 52-63, 2022.
 [http://dx.doi.org/10.1111/cgf.14602]
[65]
T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen,  and T. Aila, "Analyzing and Improving the Image Quality of StyleGAN", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 8107-8116 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00813]
[66]
D. Xiao, S. Lin, Z. Shi,  and B. Wang, "Learning modified indicator functions for surface reconstruction", Comput. Graph., vol. 102, pp. 309-319, 2022.
 [http://dx.doi.org/10.1016/j.cag.2021.10.017]
[67]
R. Zhu, D. Kang, K.H. Hui, Y. Qian, X. Zhe, Z. Dong, L. Bao, P.A. Heng,  and C.W. Fu, "Semi-signed neural fitting for surface reconstruction from unoriented point clouds", arXiv, vol. abs/2206.06715, 2022.
 [Online] Available From: [http://dx.doi.org/10.48550/arXiv.2206.06715]
[68]
C. Chen, Y-S. Liu,  and Z. Han, "Latent Partition Implicit with Surface Codes for 3D Representation", In 17th European Conference, Tel Aviv, Israel, 2022, pp. 322-343 
 [http://dx.doi.org/10.1007/978-3-031-20062-5_19]
[69]
Z. Chen, H. Ledoux, S. Khademi,  and L. Nan, "Reconstructing compact building models from point clouds using deep implicit fields", ISPRS J. Photogramm. Remote Sens., vol. 194, pp. 58-73, 2022.
 [http://dx.doi.org/10.1016/j.isprsjprs.2022.09.017]
[70]
R. Schnabel, R. Wahl,  and R. Klein, "Efficient RANSAC for point-cloud shape detection", Comput. Graph. Forum, vol. 26, no. 2, pp. 214-226, 2007.
 [http://dx.doi.org/10.1111/j.1467-8659.2007.01016.x]
[71]
Y. Boykov,  and G. Funka-Lea, "Graph cuts and efficient N-D image segmentation", Int. J. Comput. Vis., vol. 70, no. 2, pp. 109-131, 2006.
 [http://dx.doi.org/10.1007/s11263-006-7934-5]
[72]
S. Weder, J. Schönberger, M. Pollefeys,  and M.R. Oswald, "RoutedFusion: Learning Real-Time Depth Map Fusion", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 4886-4896 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00494]
[73]
S. Weder, J.L. Schonberger, M. Pollefeys,  and M.R. Oswald, "NeuralFusion: Online Depth Fusion in Latent Space", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, pp. 3162-3172 
 [http://dx.doi.org/10.1109/CVPR46437.2021.00318]
[74]
V. Sitzmann, E. Chan, R. Tucker, N. Snavely,  and G. Wetzstein, "MetaSDF: Meta-Learning Signed Distance Functions", In Advances in Neural Information Processing Systems, vol. 33, Vancouver, Canada, 2020, pp. 10136-10147 
[75]
C. Finn, P. Abbeel,  and S. Levine, "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks", In Proceedings of the 34th International Conference on Machine Learning, vol. vol. 70, Sydney, Australia, 2017, pp. 1126-1135 
[76]
B. Ma, Y-S. Liu, M. Zwicker,  and Z. Han, "Surface Reconstruction from Point Clouds by Learning Predictive Context Priors", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 6316-6327 
 [http://dx.doi.org/10.1109/CVPR52688.2022.00622]
[77]
A. Ouasfi,  and A. Boukhayma, "Few ‘Zero Level Set’-Shot Learning of Shape Signed Distance Functions in Feature Space", In 17th European Conference, Tel Aviv, Israel, 2022, pp. 561-578 
 [http://dx.doi.org/10.1007/978-3-031-19824-3_33]
[78]
J. Huang, S-S. Huang, H. Song,  and S-M. Hu, "DI-Fusion: Online Implicit 3D Reconstruction With Deep Priors", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 8932-8941 
 [http://dx.doi.org/10.1109/CVPR46437.2021.00882]
[79]
W. Chen, C. Lin, W. Li,  and B. Yang, "3PSDF: Three-Pole Signed Distance Function for Learning Surfaces with Arbitrary Topologies", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 18501-18510 
 [http://dx.doi.org/10.1109/CVPR52688.2022.01797]
[80]
M. Tatarchenko, A. Dosovitskiy,  and T. Brox, "Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs", In 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 2107-2115 
 [http://dx.doi.org/10.1109/ICCV.2017.230]
[81]
P.S. Wang, Y. Liu, Y.X. Guo, C.Y. Sun,  and X. Tong, "O-cnn: Octree-based convolutional neural networks for 3d shape analysis", ACM Trans. Graph., vol. 36, no. 4, pp. 1-11, 2017.
 [http://dx.doi.org/10.1145/3072959.3073608]
[82]
Z. Chen,  and H. Zhang, "Learning Implicit Fields for Generative Shape Modeling", In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 5932-5941 
 [http://dx.doi.org/10.1109/CVPR.2019.00609]
[83]
K. Genova, F. Cole, D. Vlasic, A. Sarna, W. Freeman,  and T. Funkhouser, "Learning Shape Templates With Structured Implicit Functions", In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, pp. 7153-7163 
 [http://dx.doi.org/10.1109/ICCV.2019.00725]
[84]
L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin,  and A. Geiger, "Occupancy Networks: Learning 3D Reconstruction in Function Space", In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 4455-4465 
 [http://dx.doi.org/10.1109/CVPR.2019.00459]
[85]
S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys,  and A. Geiger, "Convolutional occupancy networks", In 16th European Conference, Glasgow, UK, 2020, pp. 523-540 
[86]
J. Tang, J. Lei, D. Xu, F. Ma, K. Jia,  and L. Zhang, "SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks", In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 6484-6493 
 [http://dx.doi.org/10.1109/ICCV48922.2021.00644]
[87]
S. Lionar, L. Schmid, C. Cadena, R. Siegwart,  and A. Cramariuc, "NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping", In 2021 International Conference on 3D Vision (3DV), London, United Kingdom, 2021, pp. 1279-1289 
 [http://dx.doi.org/10.1109/3DV53792.2021.00135]
[88]
Ö. Çiçek, A. Abdulkadir, S.S. Lienkamp, T. Brox,  and O. Ronneberger, "3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation", In 19th International Conference, Athens, Greece, 2016, pp. 424-432 
[89]
C. Jiang, A. Sud, A. Makadia,  and J. Huang, "M. NieBner, and T. Funkhouser, “Local Implicit Grid Representations for 3D Scenes", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 6000-6009 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00604]
[90]
J. Chibane, T. Alldieck,  and G. Pons-Moll, "Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 6968-6979 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00700]
[91]
K. Genova, F. Cole, A. Sud, A. Sarna,  and T. Funkhouser, "Local Deep Implicit Functions for 3D Shape", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 4856-4865 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00491]
[92]
M. Jia,  and M. Kyan, "Learning Occupancy Function from Point Clouds for Surface Reconstruction", CoRR, vol. abs/2010.11378, p. arXiv:2010.11378, 2020.
[93]
M. Atzmon, H. Maron,  and Y. Lipman, "Point convolutional neural networks by extension operators", ACM Trans. Graph., vol. 37, no. 4, pp. 1-12, 2018.
 [http://dx.doi.org/10.1145/3197517.3201301]
[94]
T. Lewiner, H. Lopes, A.W. Vieira,  and G. Tavares, "Efficient Implementation of Marching Cubes’ Cases with Topological Guarantees", J. Graphics Tools, vol. 8, no. 2, pp. 1-15, 2003.
 [http://dx.doi.org/10.1080/10867651.2003.10487582]
[95]
B. Maxim,  and S. Nedevschi, "OccTransformers: Learning occupancy using attention", In 2021 IEEE 17th International Conference
on Intelligent Computer Communication and Processing
(ICCP), Cluj-Napoca, Romania, 2021, pp. 219-226 
 [http://dx.doi.org/10.1109/ICCP53602.2021.9733609]
[96]
A. Dosovitskiy, L. Bayer,  and A. Kolesnikov, "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", CoRR, vol. abs/2010.11929, p. arXiv:2010.11929, 2020.
[97]
H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles,  and H. Jegou, "Training data-efficient image transformers distillation through attention", PMLR, vol. 139, pp. 10347-10357, 2021.
[98]
H. Jiang, J. Cai, J. Zheng,  and J. Xiao, "Neighborhood-based Neural Implicit Reconstruction from Point Clouds", In 2021 International
Conference on 3D Vision (3DV), London, United Kingdom, 2021, pp. 1259-1268 
 [http://dx.doi.org/10.1109/3DV53792.2021.00133]
[99]
F. Williams, Z. Gojcic,  and S. Khamis, "Neural Fields as Learnable Kernels for 3D Reconstruction", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 18479-18489 
 [http://dx.doi.org/10.1109/CVPR52688.2022.01795]
[100]
Y. Liao, S. Donné,  and A. Geiger, "Deep Marching Cubes: Learning Explicit Surface Representations", In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 2916-2925 
 [http://dx.doi.org/10.1109/CVPR.2018.00308]
[101]
A. Boulch,  and R. Marlet, "POCO: Point Convolution for Surface Reconstruction", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 6292-6304 
 [http://dx.doi.org/10.1109/CVPR52688.2022.00620]
[102]
J. Chibane, "Neural Unsigned Distance Fields for Implicit Function Learning", In Advances in Neural Information Processing Systems, vol. 33, Vancouver, Canada, 2020, pp. 21638-21652 
[103]
F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva,  and G. Taubin, "The ball-pivoting algorithm for surface reconstruction", IEEE Trans. Vis. Comput. Graph., vol. 5, no. 4, pp. 349-359, 1999.
 [http://dx.doi.org/10.1109/2945.817351]
[104]
R. Venkatesh, T. Karmali,  and S. Sharma, "Deep Implicit Surface Point Prediction Networks", In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 12633-12642 
 [http://dx.doi.org/10.1109/ICCV48922.2021.01242]
[105]
J.C. Hart, "Sphere tracing: a geometric method for the antialiased ray tracing of implicit surfaces", Vis. Comput., vol. 12, no. 10, pp. 527-545, 1996.
 [http://dx.doi.org/10.1007/s003710050084]
[106]
F. Zhao, W. Wang, S. Liao,  and L. Shao, "Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction", In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 12654-12663 
 [http://dx.doi.org/10.1109/ICCV48922.2021.01244]
[107]
T. Aumentado-Armstrong, S. Tsogkas, S. Dickinson,  and A. Jepson, "Representing 3D Shapes with Probabilistic Directed Distance Fields", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 19321-19332 
 [http://dx.doi.org/10.1109/CVPR52688.2022.01874]
[108]
B. Wang, Z. Yu,  and B. Yang, "RangeUDF: Semantic surface reconstruction from 3D point clouds", arXiv:2204.09138, 2022.
[109]
Q. Hu, B. Yang,  and L. Xie, "RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds", In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 11105-11114 
 [http://dx.doi.org/10.1109/CVPR42600.2020.01112]
[110]
B. Yang, S. Wang, A. Markham,  and N. Trigoni, "Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction", Int. J. Comput. Vis., vol. 128, no. 1, pp. 53-73, 2020.
 [http://dx.doi.org/10.1007/s11263-019-01217-w]
[111]
J. Ye, Y. Chen, N. Wang,  and X. Wang, "GIFS: Neural Implicit Function for General Shape Representation", In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 12819-12829 
 [http://dx.doi.org/10.1109/CVPR52688.2022.01249]
[112]
J. Zhou, B. Ma, Y-S. Liu, Y. Fang,  and Z. Han, "Learning Consistency-Aware Unsigned Distance Functions Progressively from Raw Point Clouds", arXiv:2210.02757.
[113]
M.S. Arshad,  and W.J. Beksi, "Automated Reconstruction of 3D Open Surfaces from Sparse Point Clouds", In 2022 IEEE International
Symposium on Mixed and Augmented Reality Adjunct (ISMAR-
Adjunct) 2022, Singapore, Singapore, 2022, pp. 216-221 
 [http://dx.doi.org/10.1109/ISMAR-Adjunct57072.2022.00048]
[114]
B. Guillard, F. Stella,  and P. Fua, "MeshUDF: Fast And Differentiable Meshing Of Unsigned Distance Field Networks", In 17th European Conference, Tel Aviv, Israel, 2022, pp. 576-592 
 [http://dx.doi.org/10.1007/978-3-031-20062-5_33]
Rights & Permissions Print Cite
Journal Information
For Authors
For Editors
For Reviewers
Explore Articles
Open Access
Open Access Articles
For Visitors
DOI https://dx.doi.org/10.2174/1872212118666230626124718	Print ISSN 1872-2121
Publisher Name Bentham Science Publisher	Online ISSN 2212-4047
Recent Patents on Engineering

A Review on the Deep Learning-based Surface Reconstruction from the Point Clouds

Abstract Play Pause

Graphical Abstract

Related Journals

Related Books

Abstract
