PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention

Yongbin Sun1      Yue Wang1      Ziwei Liu2      Joshua E. Siegel1      Sanjay E. Sarma1
Winter Conference on Applications of Computer Vision (WACV) 2020, Oral Presentation


A point cloud is an agile 3D representation, efficiently modeling an object's surface geometry. However, these surface-centric properties also pose challenges on designing tools to recognize and synthesize point clouds. This work presents a novel autoregressive model, PointGrow, which generates realistic point cloud samples from scratch or conditioned on given semantic contexts. Our model operates recurrently, with each point sampled according to a conditional distribution given its previously-generated points. Since point cloud object shapes are typically encoded by long-range interpoint dependencies, we augment our model with dedicated self-attention modules to capture these relations. Extensive evaluation demonstrates that PointGrow achieves satisfying performance on both unconditional and conditional point cloud generation tasks, with respect to fidelity, diversity and semantic preservation. Further, conditional PointGrow learns a smooth manifold of given image conditions where 3D shape interpolation and arithmetic calculation can be performed inside.

Public Video


Code and Models


title={PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention},
author={Sun, Yongbin and Wang, Yue and Liu, Ziwei and Siegel, Joshua E and Sarma, Sanjay E},
booktitle={Winter Conference on Applications of Computer Vision},