DoF-Gaussian: Controllable Depth-of-Field
for 3D Gaussian Splatting

CVPR,2025

Liao Shen1,2, Tianqi Liu1,2, Huiqiang Sun1,2, Jiaqi Li1, Zhiguo Cao1, Wei Li2*, Chen Change Loy2

1Huazhong University of Science and Technology    2S-Lab, Nanyang Technological University

Paper

Code

Video

Abstract

Recent advances in 3D Gaussian Splatting (3D-GS) have shown remarkable success in representing 3D scenes and generating high-quality, novel views in real-time. However, 3D-GS and its variants assume that input images are captured based on pinhole imaging and are fully in focus. This assumption limits their applicability, as real-world images often feature shallow depth-of-field (DoF). In this paper, we introduce DoF-Gaussian, a controllable depth-of-field method for 3D-GS. We develop a lens-based imaging model based on geometric optics principles to control DoF effects. To ensure accurate scene geometry, we incorporate depth priors adjusted per scene, and we apply defocus-to-focus adaptation to minimize the gap in the circle of confusion. We also introduce a synthetic dataset to assess refocusing capabilities and the model’s ability to learn precise lens parameters. Our framework is customizable and supports various interactive applications. Extensive experiments confirm the effectiveness of our method.

Given a set of multi-view input images with shallow DoF, DoF-Gaussian can reconstruct a 3D-GS representation of a sharp scene. Thanks to our lens-based design, we can also achieve controllable DoF effects for a variety of applications. (Zoom-in for best view)


Method

Overview of DoF-Gaussian. Given input images \(I\) with shallow DoF, we first apply SfM from COLMAP to obtain sparse depth \(D_{sparse}\), which is used to train a depth network to derive per-scene depth priors \(D_{pred}\). We then employ \(D_{pred}\) to regularize the Gaussians rendered depth map \(D\). Next, by developing a lens imaging model, we can render defocused images \(C^*\) to simulate input images. To minimize the discrepancy in CoC, we propose an adaptation using the weight map. Finally, we can render fully clear images for novel view synthesis and achieve various effects by our controllable DoF framework.


Applications

Users can create their own cinematic moments by combining changes in aperture size, focus distance, camera poses, and zoom.

Change the aperture size and focus distance

Change the aperture shape

Fixed focus position, change camera poses

Change the lens parameters and camera poses/zoom-in


Citation

@article{shen2025dof,
  title={DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting},
  author={Shen, Liao and Liu, Tianqi and Sun, Huiqiang and Li, Jiaqi and Cao, Zhiguo and Li, Wei and Loy, Chen Change},
  journal={arXiv preprint arXiv:2503.00746},
  year={2025}
}