OpenAI Released a New Tool Called Point-E That Generates 3D Models

OpenAI, the artificial intelligence research company, has open sourced its new tool called Point-E, a machine learning system that generates 3D models based on text prompts in a couple of minutes. Point-E can create 3D models quickly on a single Nvidia V100 GPU.

Point-E consists of two models: the first converts the user's text query into a regular image, while the second converts the image into a 3D model. To do this, it generates point clouds or discrete sets of data points in space that replicate the three-dimensional shape of the object, so the output is a 3D model consisting of a set of points.

Point clouds are computationally easier to synthesize, but they do not capture the fine-grained shape or texture of the object, which is the AI's current key limitation. To get around this limitation, the team developing this tool trained an additional AI system to convert the point clouds of Point-E to meshes (vertices, edges, and faces).

OpenAI specialists have trained the models on a dataset of "several million" 3D objects and related metadata. The company admits that Point-E is less advanced than other similar technologies, but it allows to obtain results much faster. Hence the technology's name: The letter "E" is short for efficiency, as the developers believe this approach is more efficient and faster than other 3D creation methods.

Point-E's point clouds can be used in 3D-printing, and with the additional mesh-converting model and further improvements – in game and animation development workflows. You can find the source code of Point-E here.