Vehicle-mounted head-up display (HUD) technology projects critical driving information directly into the driver’s line of sight, effectively reducing the frequency of visual focus switching compared with traditional dashboard interactions and significantly improving driving safety [Fig. 1(a)]. Since its transition from military aviation to civilian automotive applications,1 HUD has demonstrated transformative potential in visualizing driving information.2^,3 Early automotive HUD systems, based on geometric optical architectures, used the windshield to project a virtual image formed through multifolded optical paths into the driver’s field of view (FOV).4^–6 However, these systems were constrained by a single-plane projection [Fig. 1(b)], requiring drivers to refocus to align virtual information with real-world scenes, prolonging cognitive latency during human–machine interaction. Advanced HUDs employ multiplane projection, incorporating short-focal and long-focal channels to display vehicle data for human–machine interaction and environment-merged information for safe driving, respectively.7^–9 However, this necessitates a dual optical path architecture [Fig. 1(c)], increasing system bulk and cost. Such physical stacking of multiple optical channels further introduces engineering incompatibilities with the compact vehicle cabin, posing a major bottleneck for technological upgrades. Consequently, achieving fully three-dimensional light fields within the current geometric optics-based HUD architecture remains a fundamental challenge.

Diffraction-based spatial imaging techniques, particularly computer-generated holography (CGH), offer promising solutions to these challenges. Leveraging multiplane diffraction properties of optical wavefronts, CGH enables advanced functionalities such as continuous zoom imaging,10 dynamic multifocal-plane imaging,11^–16 and three-dimensional display.16^–22 Consequently, it has emerged as a promising approach for realizing multiplane imaging within a single optical path for HUD applications [Fig. 1(d)]. In recent years, the rapid progress of deep learning has further broadened this landscape. Deep-learning-based CGH models, trained on large-scale datasets and optimized through carefully designed loss functions, have demonstrated impressive capability in producing accurate, efficient, and low-cost color holograms with depth information.18^,23^–29 In hologram design, diffraction calculations are typically implemented using fast Fourier transform (FFT) algorithms, significantly reducing computational complexity compared with direct numerical integration.30^–32 However, FFT-based approaches suffer from an inherent constraint: they require equal sampling densities at both the object and image planes, enforce central alignment along the optical axis, and are susceptible to aliasing artifacts if the sampling precision is inadequate. Although deep learning neural models can estimate diffraction fields efficiently and even offer flexibility in setting optical parameters once trained,33 their training datasets still rely on precise physical propagation models such as the angular spectrum (AS) method for supervision and ground-truth generation. These limitations restrict the controllable range of the computed diffraction field in terms of spatial size, resolution, and positioning, potentially leading to nonnegligible errors.34^,35 Although mitigation strategies such as zero-padding,33^,36 nonuniform FFT,35^,37^–40 and spectral-domain processing33^,41 have been developed, they typically incur substantial computational overhead and may become ineffective when the output image dimensions significantly exceed those of the input phase mask, or when modeling diffraction over long propagation distances. Moreover, vehicular HUD applications demand diffraction solutions capable of handling multiplane reconstructions across vastly different scales. In such scenarios, the diffraction algorithm must accommodate apertures ranging from small to large with high spatial resolution and support accurate wavefield propagation across extended distances. Due to these challenges, a cross-scale, multiplane holographic display for vehicular HUD systems remains elusive now.

To address this problem, we report a proof-of-concept demonstration of a multiple-plane holography display for vehicular HUD using a nonsquare matrix multiplication (MM) method that enables rapid computation of Fresnel diffraction with arbitrary sampling at the observation plane, even when the sampling at the initial plane remains fixed. By circumventing the use of FFT-based methods, the nonsquare MM method avoids the need for zero-padding, an operation indispensable for maintaining accuracy in FFT-based algorithms, thereby eliminating redundant computations and allowing complete decoupling between the sampling schemes of the object and image planes. Notably, although the MM method does not rely on FFTs, comparative experiments demonstrate a computational speed improvement of ∼58% under typical sampling conditions, with the performance advantage becoming increasingly significant as the sampling density increases. Leveraging the flexible sampling decoupling capability of the MM method, we experimentally realize dual-plane imaging with a 100:1 size ratio within the Fresnel region and further demonstrate synchronized multi-plane imaging across both Fresnel and Fraunhofer regimes, thus experimentally confirming the MM method’s suitability for high-resolution diffraction computations over a broad range of aperture sizes and propagation distances. Benefitting from these advantages, we demonstrate a three-plane holographic HUD with a spatial light modulator (SLM) and successfully achieve spatial overlay of three virtual images with real-world objects for seamless virtual-real integration display. Our technique opens the door to cross-scale, wide-depth, and multiplane holography for integrating HUD systems within compact vehicle cabins.

Conventional FFT-based methods impose strict spatial sampling constraints to prevent spectral aliasing, and require zero-padding the object field to match or exceed the image plane size, as shown in Fig. 2(a). Such padding strategies significantly increase computational redundancy, especially when iterative hologram optimization is involved. In this work, we adopt an efficient diffraction computation method based on MM, which reconstructs the two-dimensional convolution process into three successive matrix operations without the need for zero-padding.42^,43 This approach fully decouples the sampling parameters between object and image planes and substantially reduces memory and computational overhead, particularly for large-scale hologram generation in HUD.

Specifically, by exploiting the separable property of the Fresnel diffraction kernel, the discretized Fresnel diffraction integral can be reconstructed into a matrix form when the Fresnel approximation condition z3≫k8[(x−ξ)2+(y−η)2]max2 is satisfied. The discrete Fresnel diffraction integral can be expressed as U(xp,yq,z)=exp(jkz)jλz∑m=1M∑n=1NU(ξm,ηn,0)exp[jk2z((xp−ξm)2+(yq−ηn)2)]ΔξΔη,(1)where p∈{1,2,⋯,P}, q∈{1,2,⋯,Q}, (ξm,ηn,0), and (xp,yq,z) denote the spatial coordinates at the object and image plane, respectively, k=2π/λ is the wave number, and Δξ and Δη are the sampling intervals on the object plane.

Introducing vectorized notations ξ˜=[ξ1,ξ2,⋯,ξM]T, η˜=[η1,η2,⋯,ηN]T, x˜=[x1,x2,⋯,xP]T, and y˜=[y1,y2,⋯,yQ]T, the Fresnel diffraction integral can be reformulated as (see Sec. S1 in the Supplementary Material for the full derivation) $$\begin{gathered} U˜(x,y,z)=[exp(jkz)jλzexp(jk2z(x˜2+y˜2))ΔξΔη]⊙exp(−jkzy˜⊗η˜T) \\ ⊗[U˜(ξ,η,0)⊙exp(jk2z(Ξ˜2+H˜2))]⊗exp(−jkzξ˜⊗x˜T), \end{gathered}$$(2)where ⊙ denotes the element-wise (Hadamard) multiplication and ⊗ denotes the standard matrix multiplication. Here, Ξ˜2 and H˜2 are both N×M matrices: Ξ˜2=[ξ12ξ22⋯ξM2ξ12ξ22⋯ξM2⋮⋮⋱⋮ξ12ξ22⋯ξM2],H˜2=[η12η12⋯η12η22η22⋯η22⋮⋮⋱⋮ηN2ηN2⋯ηN2].

This matrix formulation provides several fundamental advantages beyond computational efficiency. First, because the spatial coordinates x˜ and ξ˜ (as well as y˜ and η˜) are independently defined, both the number of sampling points and the sampling intervals on the input and output planes can be freely specified. This flexibility allows for asymmetrical or mismatched object–image configurations and supports sampling regions that are decentered or located off the optical axis, thereby enabling off-axis reconstruction layouts. Second, as the method does not rely on spectral-domain transforms, there is no concern of frequency aliasing, and zero-padding becomes unnecessary, eliminating a major source of computational redundancy in conventional approaches. Third, because Fraunhofer diffraction is a limiting case of the Fresnel approximation at long propagation distances, the MM method supports both Fresnel (near-field) and Fraunhofer (far-field) regimes within the same unified computational framework. The ability to flexibly configure large image planes at far distances further enhances its suitability for vehicle-scale HUD applications. As shown in Fig. 2(b), the entire matrix-based diffraction workflow is visualized, which straightforwardly exhibits the advantages in decreasing matrix size.

To quantitatively evaluate the advantages of the MM method for hologram design, we compared the computation time and minimum memory usage of the MM approach and the AS method under varying image plane sampling conditions.30 The object plane sampling was fixed at M×N=1800×1800, whereas the image plane resolution was varied from P×Q=250×250 up to 8000×8000. Each configuration was tested using 100 iterations of the Gerchberg–Saxton (GS) phase retrieval algorithm.44 As shown in Fig. 2(c), the MM approach consistently outperformed the AS method in both runtime and memory efficiency, with the performance gap widening at higher sampling resolutions. For instance, when the image plane sampling matched the object plane (P×Q=1800×1800), the MM method achieved ∼58% of the computation time required by the AS method. When the image plane resolution was doubled (P×Q=3600×3600), the runtime dropped to 38% and further decreased to 28% at quadruple resolution (7200×7200). Section S2 in the Supplementary Material provides some representative results and detailed simulation parameters that support the benchmarking analysis.

To further verify the phase reconstruction accuracy, we experimentally analyzed the phase-reconstructed holograms generated using the AS and MM methods with the MAPS phase retrieval algorithm on an SLM.45 As shown in Fig. 2(d), at a propagation distance of z=0.5m, the RMSE between the reconstructed and target images was 0.092 for the MM-based reconstruction and 0.112 for the AS-based one, indicating comparable accuracy.

A key strength of the MM method lies in its ability to decouple the sampling parameters of the object and image planes. This property enables asymmetric sampling setups and supports simultaneous reconstruction of images with vastly different resolutions and depths. As shown in Fig. 3(a), we demonstrate dual-plane holographic reconstruction with the largest image size ratio reported to date: a small-scale cross-shaped pattern reconstructed at z1=0.1m with a size of 0.5mm×0.5mm, and a large-scale star-shaped pattern reconstructed at z2=0.5m with a size of 50mm×50mm.

In addition, Fig. 3(b) presents a representative case of mixed-regime reconstruction, where the MM method simultaneously computes diffraction fields in both the Fresnel and Fraunhofer domains. A squirrel-shaped pattern is reconstructed at a near-field plane of z1=0.5m with a Fresnel number Nf≈39.5, whereas an elephant-shaped pattern is reconstructed at a far-field plane z2=50m, corresponding to Nf≈0.39.

A detailed comparison of recent multiplane holographic works is summarized in Fig. 3(c), which plots reported maximum image size ratios and propagation distance ratios. Our method marks the highest cross-scale ratio, highlighting its potential for flexible, cross-scale, and wide-depth holographic applications.

The results of the preceding benchmarks confirm the MM algorithm’s suitability for large-scale hologram generation, long-depth-range modulation, and efficient computational performance—all of which directly align with the technical demands of HUD systems. To further demonstrate its practical engineering potential, we developed a prototype HUD platform for multiplane holographic display using the MM-based method in a MAPS algorithm.

Figure 4(a) sketches three virtual images that are generated at distances of 0.1, 0.5, and 1.5 m from a simulated automotive windshield using a holographic projection system. In our experiment, this holography system is composed of a laser, an SLM, and two groups of telescope systems, as depicted in Fig. 4(b). The windshield is replaced by a semitransparent glass plate that can guide light for human beings’ observation with the help of a tunable-focus camera [Fig. 4(c)], which can mimic the eye’s functionality to focus the images at different z-cut planes. To demonstrate the spatial multiplexing capacity of the system, we assigned distinct symbolic elements to each plane, where a virtual image is located [see Fig. 4(c)].

Figures 4(d)–4(f) show the raw diffraction patterns that are recorded using a CCD camera at different distances from the SLM. Although these three images are overlaid laterally, no obvious crosstalk is observed experimentally because of their large longitudinal intervals between two neighboring planes. After being projected through a group of lenses L3 and L4 (see details in Sec. 4), these images are relocated to new planes, where three real objects are used to label the longitudinal positions. Figures 4(g)–4(i) display the mixed patterns (composed of the virtual images and real objects) that are captured directly using the camera. When these mixed images are located at the focal plane, they can be observed with sharp edges, leaving the blurred patterns for other out-of-focus images. These features confirm that three-dimensional multiplane imaging is achieved in our holography-based HUD system. Notably, the MM method’s decoupled sampling configuration allowed independent adjustment of image sizes during system design, enabling the perceptual size of virtual elements to remain consistent across depth layers. This uniform visual scaling, achieved through dynamic focus control of the camera (see the Video 1), is of significant value for enhancing driver comfort and visual coherence in HUD applications. These results also highlight the method’s adaptability for rendering layered holographic content across large depths.

To quantitatively evaluate the system’s performance, we measured the FOV based on the lateral displacement of the observation point. By horizontally shifting the camera while maintaining alignment with the central image axis [see the left panel of Fig. 4(j)], we recorded the reconstructed patterns [see the right panel of Fig. 4(j)] across varying view angles. The results indicate that image distortion becomes slightly noticeable at a horizontal deviation angle of 1.2 deg, and severe information loss occurs beyond 1.8 deg. A dynamic demonstration of continuous field-of-view variation under lateral camera displacement is provided in Video 2. Thus, the effective FOV of the current system configuration [Figs. 4(b) and 4(c)] was determined to be ∼3.6deg×3.6deg, which effectively demonstrates the system’s full-range angular performance. Although the SLM sampling at 532 nm with a 3.6μm pixel pitch sets a theoretical FOV of 8.4 deg, the effective FOV in our prototype is primarily limited by the spatial filter in the 4f relay. To suppress the undiffracted zeroth order and improve hologram quality, we place an iris at the Fourier plane that restricts the transmitted AS. With a first relay lens of f1=75mm and an iris diameter of DA2≈4.7mm, the angular acceptance is θ≈3.6deg. While preserving zeroth-order suppression, a larger Fourier-plane passband together with a higher-numerical aperture (NA) relay would extend the practical FOV.

Furthermore, to simulate dynamic scenarios, we load pre-computed phase masks onto the SLM and play them back at a refresh rate of ∼60Hz, with each pattern generated using the MM-based MAPS algorithm. As demonstrated in the Video 3, this sequence enabled a holographic video of different scene content, illustrating the system’s potential for future deployment in complex driving environments, and laying the groundwork for real-time implementations using high-speed SLM hardware and optimized phase computation pipelines.

Despite the advantages of the proposed multiplane holographic HUD, several technical challenges remain before full deployment in real-world automotive environments. First, although our prototype employs a single-wavelength laser, a full-color holographic display requires high-brightness red, green, and blue (RGB) sources with sufficient optical power and spectral stability across a wide temperature range from −40°C to 85°C. In outdoor driving scenarios, the luminance of color lasers must compete with ambient sunlight while simultaneously satisfying eye-safety regulations.

Second, the limited refresh rate of current SLMs remains a major bottleneck. Our MM-based MAPS framework also supports single-phase encoding across multiple wavelengths, enabling color reconstruction without sequential modulation, although slight inter-channel crosstalk is still observed (see Sec. S5 in the Supplementary Material). In existing SLM systems, full-color display is most commonly achieved by time-division multiplexing, where the R/G/B channels are sequentially modulated.51^–54 This approach effectively triples the frame rate requirement, demanding an effective 180 Hz SLM update for a 60 Hz full-color video, whereas most commercial liquid-crystal-on-silicon devices operate at only ∼60 to 120 Hz, which is insufficient for dynamic automotive scenes. The need for higher frame rates is even more pronounced under rapidly changing viewpoints and head motion. Recent ferroelectric liquid crystal and MEMS-based SLMs with kilohertz-level refresh rates are promising, although their phase modulation depth and diffraction efficiency require further improvement.55^,56 Alternative routes to bypass temporal multiplexing include spatial multiplexing with interleaved pixel subregions,57 wavelength-multiplexed phase encoding using gratings or meta-optics,58 and time-free multiplexing based on parallel modulation,59 yet realizing practical high-refresh-rate, full-color real-time holography remains challenging.

Third, the achievable FOV is another key constraint, fundamentally limited by the SLM’s diffraction angle, which is inversely related to its pixel pitch. In our prototype, the measured FOV of 3.6 deg approaches the upper limit imposed by the current optical relay and SLM sampling. To meet the typical automotive requirement of ∼10 to 15 deg for augmented-reality HUDs, two main strategies are feasible: employing projection optics that expand the angular range of the modulated light, or using next-generation SLMs with submicron pixel pitches. Compact folded or waveguide-based relay architectures can further enlarge the FOV and eyebox while maintaining optical alignment and minimizing driver obstruction.

Beyond these optical aspects, practical implementation requires system-level integration that balances performance, compactness, and manufacturability. Folded light paths, holographic waveguide combiners, and freeform projection optics help reduce system volume while preserving image quality and off-axis aberration correction. Coupling the SLM module with telecentric or wide-NA projection relays can further expand the eyebox for comfortable viewing during driving. Real-time coordination among the SLM driver, head-tracking modules, and vehicle sensors such as LiDAR and cameras can enable adaptive holographic rendering that dynamically responds to the driver’s position and external environment. This adaptive framework improves visual stability and reduces redundant computation by prioritizing perceptually relevant content. These system-level advances complement optical and algorithmic improvements and align with recent trends in automotive augmented reality (AR)-HUD integration.

Environmental robustness is also essential for automotive deployment. The SLMs and laser sources must operate reliably under vibration and large temperature fluctuations. Current laboratory-grade modulators often fail to meet such requirements, emphasizing the need for automotive-grade holographic display components. Encapsulation, thermal stabilization, and active cooling can regulate internal temperature and mitigate the influence of ambient thermal drift on liquid-crystal and laser materials. Mechanical durability can be enhanced through robust packaging and vibration-isolated mounts, ensuring consistent performance and long-term stability in vehicular environments.

Another fundamental challenge lies in real-time holographic computation. Although the MM algorithm significantly improves efficiency in the diffraction propagation step compared with FFT-based methods, generating high-resolution holograms with low speckle and minimal inter-plane crosstalk still requires significant computational power. Iterative phase optimization involves multiple cycles of forward and backward propagation under nonlinear constraints, particularly in multiplane or content-rich scenes. Recent acceleration strategies based on GPU and FPGA architectures have shown notable gains in throughput,60 whereas deep-learning-based approaches such as DeepCGH and TensorHolo demonstrate real-time phase synthesis with high visual fidelity.61^,62 Neural networks can learn the mapping between target images and corresponding phase masks, eliminating iterative optimization and enabling frame-by-frame holographic video generation.63 Such data-driven methods, when combined with hardware acceleration, represent a promising route toward real-time holographic rendering for HUD systems.

In summary, we have demonstrated efficient and flexible multiplane holographic imaging for HUD applications using a matrix multiplication-based diffraction computation method. By restructuring the Fresnel diffraction process into a series of separable matrix operations, the proposed method eliminates the need for zero-padding and decouples the object and image plane sampling parameters. Experimental evaluations demonstrate considerable improvements in computational efficiency and memory usage compared with traditional FFT-based approaches, particularly under high-resolution sampling conditions. These optical, computational, and integration advances collectively outline a realistic path from laboratory prototypes toward deployable holographic HUDs.

All simulations were executed on a desktop workstation equipped with an Intel Core i7-12700K CPU, 64 GB of RAM, and an NVIDIA GeForce RTX 3080 Ti GPU with 12 GB of memory. Phase retrieval procedures based on the MAPS algorithm were implemented entirely in TensorFlow v2.10 on Python 3.9, running in GPU-accelerated mode. All other simulation and performance evaluation routines, unless otherwise specified, were executed in MATLAB (R2022b) under CPU mode.

For the performance benchmarking shown in Fig. 2(c), we compared two reconstruction pipelines: the conventional AS-based GS algorithm and the employed matrix multiplication-based GS algorithm. In both cases, the same target image was reconstructed at a propagation distance of 1 mm over 100 GS iterations. The object plane resolution was fixed at 1800pixel×1800pixel with a spatial sampling interval of 200 nm. The image plane employed the same sampling interval, and simulations were run on CPU mode for consistency. Peak memory usage was estimated by monitoring the maximum memory of all variables throughout the iterative optimization process.

The optical configuration used for experimental demonstrations is illustrated in Fig. 4(b), which outlines the basic framework of the holographic display system. A phase-only reflective SLM (UPOLabs HDSLM38R, Shanghai, China, 4096×2160 resolution, with a pixel size of 3.6μm) was used to display the pre-computed phase masks for holographic reconstruction. The SLM was illuminated by a solid-state laser source (MGL-F-532) operating at a wavelength of 532 nm. The laser beam was spatially expanded and collimated to ensure uniform illumination over the active region of the SLM.

For holographic reconstructions captured via CCD (Thorlabs CS505CU, Newton, New Jersey, United States), unless otherwise specified, we employed a combination of a 4f filtering system and an objective to relay the modulated wavefront onto the sensor plane. After modulation by the SLM and reflection by a beam splitter, the beam passed through a spatial filtering stage before being collected by an objective lens, which was axially adjusted to provide appropriate image magnification for the CCD FOV.

In the experiment shown in Fig. 2(a), the filtered holographic field was projected onto a calibrated projection screen placed at the target diffraction distance. The resulting image was then photographed using a commercial digital single-lens reflex camera (Canon EOS 5D Mark IV with an EF 100 mm f/2.8 macro lens).

In the MM-based HUD system shown in Figs. 4(b) and 4(c), the SLM-modulated wavefront was reflected by a 50:50 beam splitter and passed through a filter before being projected onto a semi-transparent interface mimicking a windshield. The reconstructed virtual image, reflected toward the observer, was captured in the reverse propagation direction by the same digital camera setup. To visually demonstrate the system’s multidepth imaging capability, physical scale models representing a red traffic cone, a blue construction worker, and a yellow left-turn arrow were spatially aligned with the virtual images at different distances, respectively. Due to space constraints in the laboratory, we intentionally reduced the theoretical image plane separation by adjusting the inter-lens distance between L3 and L4 to 135.5 mm. This allowed physical placement of all objects within a compact setup volume while preserving visual overlap.

To measure the effective FOV, the camera was translated laterally along a calibrated linear stage while continuously keeping the third virtual image (object 3) centered in the viewfinder. The angular FOV was determined by recording the maximum lateral displacement at which the image remained clearly distinguishable, defined as the point beyond which the edges of the holographic content began to blur or lose visual clarity.

Acknowledgment. K.H. was supported by the National Key Research and Development Program of China (Grant No. 2022YFB3607300), the National Natural Science Foundation of China (Grant Nos. 62322512, 62225506, and 12134013), the Fundamental Research Funds for the Central Universities (Grant Nos. WK2030000108 and WK2030000090), and the CAS Project for Young Scientists in Basic Research (Grant No. YSBR-049). Q.C. was supported by the National Natural Science Foundation of China (Grant Nos. 12174260 and 12574326), the Shanghai Rising-Star Program (Grant No. 21QA1406400), and the Shanghai Science and Technology Development Fund (Grant Nos. 21ZR1443500 and 21ZR1443600). D.Z. thanks the support from the China Postdoctoral Science Foundation (Grant No. 2023M743364). K.H. thanks the support from the Center for Micro and Nanoscale Research and Fabrication, University of Science and Technology of China. The numerical calculations were partially performed on the supercomputing system at the Hefei Advanced Computing Center and the Supercomputing Center of the University of Science and Technology of China. This work was supported by the UPOLabs, which provided the experimental and technical support.

Biographies of the authors are not available.