Cinematic Gaussians:  Real-Time HDR Radiance Fields with Depth of Field

Pacific Grafics 2024

    Chao Wang1,   Krzysztof Wolski1,   Bernhard Kerbl3, 4,   Ana Serrano2,   Mojtaba Bemana1,   Hans-Peter Seidel1,  Karol Myszkowski1,   Thomas Leimkühler1

1Max-Planck-Institut für Informatik   2University of Zaragoza   3Technische Universität Wien   4Carnegie Mellon University


Overview

Based on sparse images captured from various viewpoints and with different focus distances, apertures, and exposure times (left), our method generates an HDR 3D Gaussian representation that can be rendered with arbitrary depth of field. Our representation supports rendering images in a six-dimensional space of camera and lens parameters (right, five dimensions are shown).


Video


-->

Abstract

Radiance field methods represent the state of the art in reconstructing complex scenes from multi-view photos. However, these reconstructions often suffer from one or both of the following limitations: First, they typically represent scenes in low dynamic range (LDR), which restricts their use to evenly lit environments and hinders immersive viewing experiences. Secondly, their reliance on a pinhole camera model, assuming all scene elements are in focus in the input images, presents practical challenges and complicates refocusing during novel-view synthesis. Addressing these limitations, we present a lightweight method based on 3D Gaussian Splatting that utilizes multi-view LDR images of a scene with varying exposure times, apertures, and focus distances as input to reconstruct a high-dynamic-range (HDR) radiance field. By incorporating analytical convolutions of Gaussians based on a thin-lens camera model as well as a tonemapping module, our reconstructions enable the rendering of HDR content with flexible refocusing capabilities. We demonstrate that our combined treatment of HDR and depth of field facilitates real-time cinematic rendering, outperforming the state of the art.

Bibtex


@article{wang2024cinematic, title={Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field}, author={Wang, Chao and Wolski, Krzysztof and Kerbl, Bernhard and Serrano, Ana and Bemana, Mojtaba and Seidel, Hans-Peter and Myszkowski, Karol and Leimk{\"u}hler, Thomas}, journal={arXiv preprint arXiv:2406.07329}, year={2024} }
Imprint / Data Protection

Thanks to Volodymyr Kuleshov for his website template. © 2017