CGH Algorithms

This section mainly focuses on the algorithmic aspect of holographic display systems.

Quick map:

Traditional Heuristic Methods: closed-form, geometry-driven, or display-inspired baselines.
Iterative Methods: optimize holograms by repeatedly propagating wavefields and enforcing constraints.
Learned Propagation Model Methods: learn the mismatch between ideal propagation and real hardware.
Learned Hologram Synthesis Methods: directly predict holograms or intermediate wavefields with neural networks.

Traditional Heuristic Methods

Point-based Methods

Computer-generated double-phase holograms (Hsueh et al. 1978 | Applied Optics, Optica) proposed to decompose a complex field into two phase-only components to generate holograms using a single phase-only SLM.
Holographic Near-Eye Displays for Virtual and Augmented Reality (Maimone et al. 2017 | Transactions on Graphics (TOG), ACM) proposed a holographic near-eye display system based on the double phase encoding scheme.
Monocular 3D see-through head-mounted display via complex amplitude modulation (Gao et al. 2016 | Optics Express, Optica)
Near-eye Light Field Holographic Rendering with Spherical Waves for Wide Field of View Interactive 3D Computer Graphics (Shi et al. 2017 | Transactions on Graphics (TOG), ACM) uses point-source illumination rather than plane-wave illumination to improve the FOV of holograhic displays.

Iterative Methods

A family of iterative methods is based on the Gerchberg-Saxton (GS) Algorithm where the phase and amplitute patterns at two planes are updated iteratively as the wave propagates back and forth between the two planes:

A practical algorithm for the determination of phase from image and diffraction plane pictures (Gerchberg et al. 1972 | Optik, Elsevier) proposed the Gerchberg-Saxton (GS) Algorithm
Mix-and-Match Holography (Peng et al. 2017 | SIGGRAPH Asia, ACM) proposed an interative phase-retrieval method built upon GS
Fresnel ping-pong algorithm for two-plane computer-generated hologram display (Dorsch et al. 1994 | Applied Optics, Optica)

Other optimization based methods leverage gradient descent or non-convex optimization techniques to optimize the phase pattern of the SLM:

Perceptual-driven loss designs

Accommodative Holography: Improving Accommodation Response for Perceptually Realistic Holographic Displays (Kim et al. 2022 | SIGGRAPH, ACM) analyzed the user accomodation performance when using different CGH methods and proposed a novel constrast ratio-based regularization loss that promotes better accomodation cues.
Metameric Varifocal Holograms (Walton et al. 2022 | VR, IEEE) proposed a foveated graphics-inspired, gaze-contingent loss function that can be easily integrated into CGH optimization. The loss enforces the defocus image regions to only statistically match the defocus target image regions rather than pixel-wise reconstruction. This increases the degrees of freedom for the hologram to distribute light (compared to fitting a focal stack), thus reducing speckle artifacts.
Realistic Defocus Blur for Multiplane Computer-Generated Holography (Kavaklı et al. 2021) proposed a novel loss function aimed to synthesize high quality defocus blur, and can be intergated in various iterative (GS, gradient-descent) and non-iterative (double phase encoding) methods.
Gaze-Contingent Retinal Speckle Suppression for Perceptually-Matched Foveated Holographic Displays (Chakravarthula et al. 2021 | TVCG, IEEE)

Others

Hogel-free Holography (Chakravarthula et al. 2022 | SIGGRAPH, ACM)
Optimization of computer-generated holograms featuring phase randomness control (Yoo et al. | Optics Express, Optica) leveraged a learned DPAC encoding optimized using gradinet descent to promote phase randomness, which in turn increases the space-bandwidth product of the display system.
Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function (Chen et al. 2021 | Optics Express, Optica)
Wirtinger Holography for Near-Eye Displays (Chakravarthula et al. 2019 | SIGGRAPH Asia, ACM) optimizes the phase-only SLM pattern using closed-form Wirtinger complex derivatives in gradient descent.
3D computer-generated holography by non-convex optimization (Zhang et al. 2017 | Optica, Optica)

Unfortunately, iterative methods are inherently slow and thus not suitble for real-time CGH. See this section for speeding up hologram synthesis using neural networks.

Learned Propagation Model Methods

There are often mismatches between a ideal wave propagation model (e.g. ASM) with the actual physical display setup. A major focus in deep learning for CGH is using camera-in-the-loop (CITL) training to learn an accurate free space wave propagation and optical hardware model for holographic displays:

Neural Holography with Camera-in-the-loop Training (Peng et al. 2020 | SIGGRAPH, ACM) is the first to use camera-in-the-loop training (CITL) to optimize a parameterized wave propagation model, where optical aberrations, SLM non-linearities, and etc are learned from data. A CNN is also proposed to synthesize 2D and 3D holograms in real-time.
Neural 3D Holography: Learning Accurate Wave Propagation Models for 3D Holographic Virtual and Augmented Reality Displays (Choi et al. 2021 | SIGGRAPH Asia, ACM) uses two CNNs to directly model optical aberrations, SLM non-linearities, and etc, at the input plane and multiple target planes. The usage of two CNNs introduces more degrees of freedom than that of the parameterized propagation model proposed in Peng et al. 2020, such that higher quality 3D holograms can be achieved.
Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators (Choi et al. 2022 | SIGGRAPH, ACM) leveraged time-multiplexed quantized SLM patterns to synthesize high quality defocus blur.
Learned Hardware-in-the-loop Phase Retrieval for Holographic Near-Eye Displays (Chakravarthula et al. 2020 | SIGGGRAPH Asia, ACM) uses CITL to learn an aberration approximator that models the residual between holograms generated from ideal wave propagation (i.e. ASM) and real-world wave propagation models. An adversarial loss is used in addition to reconstruction loss to optimize the synthesized holograms.
Learned holographic light transport (Kavaklı et al. 2021 | Applied Optics, Optica) learns the wave propagation convolution kernel directly from images captured by a physical holographic display, instead of using the ASM method to propagate fields.

Learned Hologram Synthesis Methods

These works often assume a naive wave propagation model (i.e. the angular spectrum method (ASM)), and directly regresses complex holograms using novel CNN architectures:

End-to-end Learning of 3D Phase-only Holograms for Holographic Display (Liang et al. 2022 | Light: Science and Applications, Nature)
Towards real-time photorealistic 3D holography with deep neural networks (Liang et al. 2021 | Nature, Nature)
Diffraction-engineered holography: Beyond the depth representation limit of holographic displays (Yang et al. 2022 | Nature Communications, Nature)
DeepCGH: 3D computer-generated holography using deep learning (Eybposh et al. 2020 | Optics Express, Optica) uses a CNN to estimate a complex field at a fixed plane from a set of 3D target multiplane inputs; the complex field is then reverse propagated to the SLM plane to generate a phase pattern.
Deep neural network for multi-depth hologram generation and its training strategy (Lee et al. 2020 | Optics Express, Optica) directly estimates the SLM phase pattern from 3D target multiplane inputs using a CNN.
Deep-learning-generated holography (Horisaki et al. 2018 | Applied Optics, Optica)
Phase recovery and holographic image reconstruction using deep learning in neural networks (Rivenson et al. 2018 | Light: Science and Applications, Nature)