3D Point Set Reconstruction Using Hyperplane Patch Folding Decoder

3D point set reconstruction is an important and challenging 3D shape analysis task.Current state-of-the-art algorithms for 3D point set reconstruction employ deep neural networks(DNNs)having an encoder-decoder architecture.Recently,the decoder DNNs that transform multiple 2D planar patches to reconstruct a 3D shape have seen some success.These “patch-folding” decoders are adept at approximating smooth surfaces in 3D objects.However,3D point sets generated by these decoders often lack local geometrical details,as 2D planar patches tend to overly constrain the patch folding process.This study proposes a novel decoder DNN,dubbed Hyperplane Mixing and Folding Net(HMF-Net),and two novel loss functions for 3D point set reconstruction.Moreover,it is challenging for a single auto-encoder to generate points that are uniform,clean,and faithfully located on the underlying surface simultaneously.To deal with this problem,this study further proposes to enhance HMF-Net by augmenting localization refinement,dubbed “HMF-Net++”.The main contributions include two aspects:(1)HMF-Net.To alleviate the weak expression power of 2D planar patches,HMF-Net first uses less constrained hyperplane,not 2D plane,patches as its input to the folding process.Second,HMF-Net has,as its core building block,a stack of token-mixing layers to effectively learn global consistency among the hyperplane patches.Finally,considering the non-uniformity of output trained by commonly used Chamfer Distance(CD),this study further proposes Weighted Chamfer Distance(WCD)and Uniform Density(UD)loss.WCD tries to weight,or amplify,loss from parts of shape by emphasizing higher point-pair distance values between a generated point set and a groundtruth point set.And UD tries to uniformize the distances among a produced point set without introducing extra parameters.Experimental results show that all of them help the decoder DNN learn shape details better.(2)HMF-Net++.To refine the coarse 3D shape generated by HMF-Net,HMF-Net++ first employs Weighted Positional Embedding to preserve high-frequency features.And then it adopts hierarchical graph convolution and Skip Point Mixing to capture local and global features,respectively.This study comprehensively evaluates the algorithm under different 3D point set reconstruction scenarios,that are,shape completion,shape upsampling,and shape reconstruction from 2D images.Experimental results demonstrate that our algorithm yields accuracies higher than the existing algorithms for 3D point set reconstruction.