NerfNet, whose name is inspired by the image database ImageNet, is both a training service and collection of neural radiance fields (NeRFs), hosted by OpenCV. With recent technological advancements in AI, particularly in the realms of 3D reconstruction and natural language processing, OpenCX (overview document here) represents the integration of the traditional OpenCV library with new findings in deep learning. This code repository contains a cloud server hosted by Lambda Labs used to train NeRFs, a user interface hosted on a website and an app, and a backend structure for incorporating data storage and streamlining the visualization process.
Our User Interface (UI) supports a host of features, including the option for a user-supplied configuration file that will be parsed by our Large Language Model (LLM) assistant and a dual-sided rendering operation that can load a NeRF in "preview" mode and "fully-trained" mode. The user-supplied configuration file should follow the set of guidelines specified here, but its intended purpose is to allow for user control/customizability over the training process. We provide several examples of possible user configuration files to assist the user in using our website here.
The rendering methodology supports two types: a "preview" mode and a "fully-trained" mode. The preview mode shows the user an expectation of what the fully trained NeRF will look like (we use a modified version of instant neural graphics primitives in order to produce the preview as it takes a very short amount of time (on the order of a few seconds) to train. Our fully-trained mode takes longer to train, as most 3D reconstruction methods take significantly longer than a few seconds (on the magnitude of several minutes to hours to even days). The default method for our fully-trained mode is a modification of the Nerfacto method presented in the Nerfstudio. We rely on a self-maintained fork of this repository, called CognitiveStudio. All of the original results can be reproduced using this fork. Our fork inherents most of the functionality present in the original Nerfstudio library, but also supports the RegNeRF and ReFNeRF models.
Our library (will also) support a modified implementation of 3D Gaussian Splatted Radiance Fields, which is the current state-of-the-art and is not a neural radiance field model but instead utilizes another paradigm of 3D reconstruction, rasterization, to render the images.
The "preview" mode supports real-time rendering while the "fully-trained" mode will send a websocket viewer link to the user's email address that will be live for 24 hours.