Skip to content

Neural Holography Codex

Documentation Docs Deploy

A growable codex of research on neural holography displays.

Notice

Note

This repository is an ongoing personal survey about the current landscape of holographic display researchs with AI. The original resources were partly adapted from Brian Chao's awesome-holography project. I would like to thank Brian Chao and the authors of the following works for providing great ideas on holography display research.

Fork maintainer: Jinwoo Lee (cinescope@kaist.ac.kr)

Overview

Neural holography is an inherently interdisciplinary field where wave optics, computer graphics, computational imaging, vision, display hardware, perception, fabrication, and machine learning constantly meet. That breadth is part of what makes the area exciting, but it also means that useful ideas are often scattered across communities that do not always share the same language, venues, or evaluation habits.

This list is an attempt to make those connections easier to see. The goal is not only to archive papers, but to help researchers from different backgrounds inherit, reinterpret, and extend one another's ideas. Ideally, that kind of shared map can support a small Cambrian explosion of reusable methods, clearer comparisons, and new experiments that would be harder to discover from within any single silo.

In that sense, it is also a reminder that the wonderland Ivan Sutherland imagined in The Ultimate Display is still shimmering ahead of us, waiting to be built.

How to Use This List

This README works best as a codex: start from the question you care about, then dive into the corresponding paper clusters.

Goal Start here Then continue with
Build intuition for the field Background and Theory Survey Papers, Learned Propagation Model Methods
Understand CGH algorithm families Traditional Heuristic Methods Iterative Methods, Learned Hologram Synthesis Methods
Focus on hardware and display constraints Topics in Holographic Display Systems Perception-related Research, Small Form-factor Displays
Map the research community Labs and Research Groups Software, Venues and Communities, Media and Resources

Reading Conventions

  • (Author et al. Year | Venue, Publisher) shows the publication context at a glance.
  • Short descriptions are added only when the title alone does not make the main contribution obvious.
  • CGH means computer-generated holography, CITL means camera-in-the-loop, SLM means spatial light modulator, and HOE means holographic optical element.
  • Labs and Research Groups, Software, Venues and Communities, and Media and Resources are best treated as reference appendices after you have a method-level overview.

Background, Theory, and Survey

Background and Theory

Survey Papers

Computer Generated Holography (CGH) Algorithms

This section mainly focuses on the algorithmic aspect of holographic display systems.

Quick map:

  • Traditional Heuristic Methods: closed-form, geometry-driven, or display-inspired baselines.
  • Iterative Methods: optimize holograms by repeatedly propagating wavefields and enforcing constraints.
  • Learned Propagation Model Methods: learn the mismatch between ideal propagation and real hardware.
  • Learned Hologram Synthesis Methods: directly predict holograms or intermediate wavefields with neural networks.

Traditional Heuristic Methods

Point-based Methods

Polygon/Mesh-based Methods

Layer-based Methods

Holographic Stereograms

Iterative Methods

A family of iterative methods is based on the Gerchberg-Saxton (GS) Algorithm where the phase and amplitute patterns at two planes are updated iteratively as the wave propagates back and forth between the two planes:

Other optimization based methods leverage gradient descent or non-convex optimization techniques to optimize the phase pattern of the SLM:

Perceptual-driven loss designs

Others

Unfortunately, iterative methods are inherently slow and thus not suitble for real-time CGH. See this section for speeding up hologram synthesis using neural networks.

Learned Propagation Model Methods

There are often mismatches between a ideal wave propagation model (e.g. ASM) with the actual physical display setup. A major focus in deep learning for CGH is using camera-in-the-loop (CITL) training to learn an accurate free space wave propagation and optical hardware model for holographic displays:

Learned Hologram Synthesis Methods

These works often assume a naive wave propagation model (i.e. the angular spectrum method (ASM)), and directly regresses complex holograms using novel CNN architectures:

Topics in Holographic Display Systems

Use this section when the algorithm is not the whole story and the display system itself becomes the bottleneck.

Quick map:

  • Speckle Noise Reduction: reduce coherent artifacts and improve image quality.
  • Perception-related Research: optimize the hologram for what observers actually see and accommodate to.
  • Etendue Expansion: trade off field of view and eyebox size.
  • Holographic Optical Elements (HOEs): use holographic optics inside the display stack.
  • Small Form-factor Displays: reduce bulk for practical AR/VR hardware.
  • Compression: lower bandwidth and computation costs.
  • Zero or Higher Diffraction Orders Optimization: manage unwanted orders or exploit higher ones.

Speckle Noise Reduction

Speckle noise is a result of interference among coherent waves, which is often present in holographic images since holographic displays use coherent laser sources. Methods for reducing speckle noise can roughly be catergorized into the following:

Time-averaging

Partially-coherent Light Sources

Others

Perception-aware holography work studies which image errors matter to human observers, how focus and accommodation cues should be optimized, and how gaze-contingent or metameric losses can trade exact reconstruction for better visual quality.

Etendue Expansion

The product of the field of view (FoV) and the eyebox size, the etendue, is limited by the number of pixels on the SLM. Hence, there is an inherent tradeoff between these two properties.

Holographic Optical Elements (HOEs)

Small Form-factor Displays

Bulky headsets hamper the development of AR/VR. Reducing the size of holographic displays are important:

Compression

CGH compression is also important for deploying holography technology on edge devices:

Zero or Higher Diffraction Orders Optimization

  • Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays (Gopakumar et al. 2021 | Optics Letters, Optica) incorporated higher diffraction orders into the CGH optimization procedure to remove the 4f filtering system often used in holographic displays, thus reducing the display form factor.
  • Elimination of a zero-order beam induced by a pixelated spatial light modulator for holographic projection (Zhang et al. 2009 | Applied Optics, Optica)
  • Holographic projection of arbitrary light patterns with a suppressed zero-order beam
  • Effect of spurious diffraction orders in arbitrary multifoci patterns produced via phase-only holograms
  • Off-axis camera-in-the-loop optimization with noise reduction strategy for high-quality hologram generation (Chen et al. 2022 | Optics Letters, Optica)

Labs and Research Groups

Academic Labs

Industry Research Groups

Software

Open-source software is often where algorithmic ideas, optical models, and hardware assumptions become concrete. This section collects reusable codebases that help bridge papers and practice.

  • PADO: a PyTorch-based differentiable optics library for wave propagation, optical modeling, and inverse design workflows.
  • PADO Hologram: a higher-level holography framework built on top of PADO for CGH experiments and display-aware workflows.
  • HoloTorch: a differentiable wave-propagation and holography framework released by Meta Reality Labs Research.

Venues and Communities

This section tracks frequently referenced publication venues in holography and neural display research.

Journals

Conferences

Workshops and Communities

Media and Resources

Talks and Media