Skip to content

Latest commit

 

History

History
110 lines (82 loc) · 3.96 KB

README.md

File metadata and controls

110 lines (82 loc) · 3.96 KB
Raw

GSGEN: Text-to-3D using Gaussian Splatting

This repository contains the official implementation of GSGEN: Text-to-3D using Gaussian Splattng.

Open In Colab

Paper | Project Page

Video results

A_DSLR_photo_of_Car_made_out_of_sushi.mp4
A_beautiful_dress_made_out_of_feathers._on_a_mannequin.mp4
A_zoomed_out_DSLR_photo_of_a_beautiful_suit_made_out_of_moss._on_a_mannequin._Studio_lighting._high_quality._high_resolution.mp4
A_DSLR_photo_of_a_car_made_out_of_cheese.mp4

Instructions:

  1. Install the requirements:
pip install -r requirements.txt
  1. Build the extension for Gaussian Splatting:
cd gs
./build.sh
  1. Start training!
python main.py --config-name=base prompt.prompt="<prompt>"

You can specify a different prompt for Point-E:

python main.py --config-name=base prompt.prompt="<prompt>" init.prompt="<point-e prompt>"

Viewer

splat viewer

We support splat viewer now ! Click the captions of text-to-3D results in our project page to watch the assets in a WebGL based viwer. Example: a pineapple. This great viewer achieves > 40 FPS on my MacBook with M1 pro chip.

viser based viewer (Visualize checkpoints on your own computer)

Start the Viewer by:

python vis.py <path-to-ckpt> --port <port>

If you are training on servers, tunneling the port using SSH

ssh -L <your_local_port>:<your_server_ip>:<your_server_port> <your_username>@<your_server>

then open the viewer in your host computer on port <your_local_port>.

Exports

First set the PYTHONPATH env var:

export PYTHONPATH="."

To .ply file

python utils/export.py <your_ckpt> --type ply

To .splat file

python utils/export.py <your_ckpt> --type splat

To mesh (Currenly only support shape export)

python utils/export.py <your_ckpt> --type mesh --batch_size 65536 --reso 256 --K 200 --thresh 0.1

where the <your_ckpt> can be the path to the .pt checkpoint file or, more conveniently, can be the id for the run (the display name of the run in wandb, e.g. 0|213630|2023-10-11|a_high_quality_photo_of_a_corgi). The exported files are reside in the exports/<export-type>.

If you encounter troubles in exporting in colab, using -m will work:

python -m utils.export <your_ckpt> --type <export_type>

Updates

  • [2023-10-08] Now support exports to .ply and .splat files. Mesh exporting are coming soon.
  • [2023-10-13] Now support Shap-E initialize, try it with init.type="shap_e"

TODO

Acknowledgement

This code base is built upon the following awesome open-source projects:

Thank the authors for their remarkable job !