角江 崇

Recently, proposals have been made to use deep learning for hologram calculations to directly infer holograms from three-dimensional (3D) data. However, this approach is expensive because it requires capturing depth information using an RGB-D camera for inference. In this study, we propose a novel approach that can infer 3D holograms directly from a color two-dimensional (2D) image without requiring depth information, using deep learning. The proposed scheme comprises three deep neural networks (DNNs). The first DNN predicts the depth information from the 2D images, the second DNN generates holograms using the 2D image and the inferred depth information, and the third DNN optimizes the quality of the holograms generated by the second CNN. The inference speed was superior to a state-of-the-art graphics processing unit. We prepared a training dataset comprising pairs of holograms and 2D images. The holograms are generated from the RGB-D image using a layer-based hologram calculation. One significant benefit of our proposed approach is that the reproduced image of the final hologram contains a natural depth cue, i.e., it can represent a natural 3D reproduced image in the depth direction. In addition, conventional image sensors can be used to create input information for inference.

In this paper, we have proposed a hologram calculation method using polynomial approximations for reducing the computational complexity of point-cloud-based hologram calculations. The computational complexity of existing point-cloud-based hologram calculations is proportional to the product of the number of point light sources and hologram resolution, whereas that of the proposed method can be reduced to approximately proportional to the sum of the number of point light sources and hologram resolution by approximating the object wave with polynomials. The computation time and reconstructed image quality were compared with those of the existing methods. The proposed method was approximately 10 times faster than the conventional acceleration method, and did not produce significant errors when the object was far from the hologram.

In this paper, we have proposed a hologram calculation method using polynomial approximations for reducing the computational complexity of point-cloud-based hologram calculations. The computational complexity of existing point-cloud-based hologram calculations is proportional to the product of the number of point light sources and hologram resolution, whereas that of the proposed method can be reduced to approximately proportional to the sum of the number of point light sources and hologram resolution by approximating the object wave with polynomials. The computation time and reconstructed image quality were compared with those of the existing methods. The proposed method was approximately 10 times faster than the conventional acceleration method, and did not produce significant errors when the object was far from the hologram.

Several methods have been proposed for reconstructing the target image using a linear image sensor with detectors arranged in one dimension. However, they have problems with illumination time, reconstruction time, and complex synchronization between pattern illumination and measuring light intensity data. Here, we propose a one-shot imaging and classification method using a linear sensor and diffuser, called vector sensor imaging. We constructed two deep neural networks for this purpose and demonstrated their effectiveness through simulations and experiments.

We propose a deep hologram converter based on deep learning to convert low-precision holograms into middle-precision holograms. The low-precision holograms were calculated using a shorter bit width. It can increase the amount of data packing for single instruction/multiple data in the software approach and the number of calculation circuits in the hardware approach. One small and one large deep neural network (DNN) are investigated. The large DNN exhibited better image quality, whereas the smaller DNN exhibited a faster inference time. Although the study demonstrated the effectiveness of point-cloud hologram calculations, this scheme could be extended to various other hologram calculation algorithms.

Light-in-flight recording by holography (LIF holography) is an ultrafast imaging technique for recording light pulse propagation as a motion picture. In this study, we propose and demonstrate multiple motion picture recordings of light pulse propagation by use of LIF holography with angular multiplexing. We set incident angles of reference light pulses to remove the difficulty in adjusting the optical path length difference between an object light pulse and reference light pulses and the complexity of the optical system. In the experiment, by using LIF holography with angular multiplexing, we succeeded in recording a propagating light pulse as two motion pictures with durations of 129.6 ps without an inseparable superimposition of the reconstructed images. In addition, cross talk between the recorded images, noise caused by cross-terms in an image plane, and the number of motion pictures that can be recorded are discussed.

In the last few decades, there have been several advances in ultrafast imaging of light propagation with light-in-flight recording by holography (LIF holography), which can capture light propagation as a motion picture with a single shot in principle. Here, we review the recent advances in LIF holography by considering the perspectives of various development of functional imaging techniques and evaluation of LIF holography with numerical simulation methods. The methods for recording multiple motion pictures such as a space-division multiplexing, a pixel-by-pixel-based space-division multiplexing, and an angular multiplexing technique are added extend the capability of LIF holography. The numerical simulation models used for investigating the image characteristics of LIF hologram are discussed. Finally, a summary and conclusion of recent advances in LIF holography is presented.

For polygon-based holograms, the efficiency of the fully analytical algorithm is outstanding compared with that of the spectral interpolation-based method. However, the full analytical algorithm is not beneficial for the rendering of objects, owing to the uniform amplitude of each triangle. In this paper, we propose a shading rendering method based on a fully analytical spectrum. We used the Blinn-Phong reflection model to simulate a scene with realistic lighting and render a glossy, smooth object surface without changing the number of polygons sampled. The proposed method develops approximate exponential functions to replace cumbersome diffuse and specular reflections, allowing us to derive fully analytical spectra. The computational results for several 3D objects show that the proposed method is highly efficient and presents realistic illumination effects owing to the analytical equations.

Computer-generated holograms can reconstruct three-dimensional (3D) images using spatial light modulators. In addition, it has several applications, such as virtual and augmented reality devices; however, the hologram calculation is time-consuming. The wavelet shrinkage-based superposition (WASABI) has been proposed to accelerate hologram calculations by compressing point-spread functions with the wavelet transforms but gives rise to the problem of deepening the depth of field of 3D reconstructions. In this study, we proposed a fast calculation method, WASABI-2, to solve this problem by compressing 3D scenes with the wavelet transforms. The proposed method accelerates hologram calculations while maintaining the original depth of field.

A controllable energy method, which considers the undersampling issue of the transfer function and valid spectral energy of a source signal, is proposed to implement angular spectrum diffraction calculation in near and far fields. The proposed method provides an optimized frequency boundary fCE within which it always keeps controllable energy to be diffracted. The controllable energy angular spectrum method significantly reduces the number of samples while having the same accuracy as previous angular spectrum methods, implying a higher calculation efficiency. The new perspective of analyzing spectral energy is shown to improve the performance of relevant diffraction calculations.

Recently, holographic displays have gained attention owing to their natural presentation of three-dimensional (3D) images; however, the enormous amount of computation has hindered their applicability. This study proposes an oriented-separable convolution accelerated using the wavefront-recording plane (WRP) method and recurrence formulas. We discuss the orientation of 3D objects that affects computational efficiency, which is overcome by reconsidering the orientation, and the suitability of the proposed method for hardware implementations.

Edge images are often used in computer vision, cellular morphology, and surveillance cameras, and are sufficient to identify the type of object. Single-pixel imaging (SPI) is a promising technique for wide-wavelength, low-light-level measurements. Conventional SPI-based edge-enhanced techniques have used shifting illumination patterns; however, this increases the number of the illumination patterns. We propose two deep neural networks to obtain SPI-based edge images without shifting illumination patterns. The first network is an end-to-end mapping between the measured intensities and entire edge image. The latter comprises two path convolutional layers for restoring horizontal and vertical edges individually; subsequently, both edges are combined to obtain full edge reconstructions, such as in the Sobel filter.

Unlike conventional imaging, the single-pixel imaging technique uses a single-element detector, which enables high sensitivity, broad wavelength, and noise robustness imaging. However, it has several challenges, particularly requiring extensive computations for image reconstruction with high image quality. Therefore, high-performance computers are required for real-time reconstruction with higher image quality. In this study, we developed a compact dedicated computer for single-pixel imaging using a system on a chip field-programmable gate array (FPGA), which enables real-time reconstruction at 40 frames per second with an image size of 128 × 128 pixels. An FPGA circuit was implemented with the proposed reconstruction algorithm to obtain higher image quality by introducing encoding mask pattern optimization. The dedicated computer can accelerate the reconstruction 10 times faster than a recent CPU. Because it is very compact compared with typical computers, it can expand the application of single-pixel imaging to the Internet of Things and outdoor applications.

Three-dimensional (3D) display using electroholography is a promising technology for next-generation television systems; however, its applicability is limited by the heavy computational load for obtaining computer-generated holograms (CGHs). The CG-line method is an algorithm that calculates CGHs to display 3D line-drawn objects at a very high computational speed but with limited expressiveness; for instance, the intensity along the line must be constant. Herein, we propose an extension for drawing gradated 3D lines using the CG-line method by superimposing phase noise. Consequently, we succeeded in drawing gradated 3D lines while maintaining the high computational speed of the original CG-line method.

We develop a temporal super-resolution high-speed holographic video recording method based on the angular multiplexing in off-axis digital holography that can achieve an acquisition rate greater than the frame rate of image sensors. We realize a high-speed switching of reference lights with different incident angles using two acousto-optic modulators. We successfully double the frame rate of the hologram recording using a rotating circular protractor and demonstrate its practical application in compressed gas flow injection; we achieve a frame rate of 175,000 fps using a high-speed image sensor triggered at 87,500 Hz.

High-order Gaussian beams with multiple propagation modes have been studied for free-space optical communications. Fast classification of beams using a diffractive deep neural network (D2NN) has been proposed. D2NN optimization is important because it has numerous hyperparameters, such as interlayer distances and mode combinations. In this study, we classify Hermite-Gaussian beams, which are high-order Gaussian beams, using a D2NN, and automatically tune one of its hyperparameters known as the interlayer distance. We used the tree-structured Parzen estimator, a hyperparameter auto-tuning algorithm, to search for the best model. As a result, the proposed method improved the classification accuracy in a 16 mode classification from 98.3% in the case of equal spacing of layers to 98.8%. In a 36 mode classification, the proposed method significantly improved the classification accuracy from 84.9% to 94.9%. In addition, we confirmed that accuracy by auto-tuning improves as the number of classification modes increases.

Layer-based hologram calculations generate holograms from RGB and depth images by repeating diffraction calculations using complex Fourier transforms (FTs). Holograms generated as such are suitable for near-eye display and can be easily reconstructed with good image quality, but they are computationally expensive because of multiple complex-valued operations, including complex FTs. In this study, we propose an acceleration method for layerbased hologram calculations by reducing time-consuming complex-valued operations using the real-valued FT and Hartley transform as real linear transformations. Real linear transformations transform real input data to real output data; thus, the proposed method generates amplitude holograms. Thus, we also propose a technique to convert holograms generated by real linear transformations into phase-only holograms using the half-zone plate process and digitalized single-sideband method while maintaining the calculation acceleration. The proposed method can speed up hologram calculations by a factor of around three while maintaining the same image quality as the conventional method.

Computational holography, encompassing computer-generated holograms and digital holography, utilizes diffraction calculations based on complex-valued operations and complex Fourier transforms. However, for some holographic applications, only real-valued holograms or real-valued diffracted results are required. This study proposes a real-valued diffraction calculation that does not require any complex-valued operation. Instead of complex-valued Fourier transforms, we employ a pure real-valued transform. Among the several real-valued transformations that have been proposed, we employ the Hartley transformation. However, our proposed method is not limited to this transformation, as other real-valued transformations can be utilized.

We propose a numerical simulation method of the hologram-recording process for light-in-flight recording by holography (LIF holography) based on fast Fourier transform (FFT) to improve the efficiency of the simulation. Because it is crucial to consider the difference in the optical-path length between the object and reference light pulses, we modify a point-spread function by considering the optical-path lengths of the object and reference light pulses and whether both pulses interfere with each other in LIF holography. The computational time was shortened by 5.5x105 times for the 4;096x4;096 resolution of the hologram using the proposed method. We evaluate the proposed method by calculating the root mean square error (RMSE) of the reconstructed holographic images. The RMSEs were relatively small considering the effect of speckle noise; these results effectively demonstrate the validity of the proposed method.Moreover,we reconstruct themoving pictures of light pulse propagation fromthe hologram generated by the proposed method.We compare the simulation and experimental results, and succeed in qualitatively demonstrating the validity of the proposed method.

Computer-generated holograms (CGHs) are used in holographic three-dimensional (3D) displays and holographic projections. The quality of the reconstructed images using phase-only CGHs is degraded because the amplitude of the reconstructed image is difficult to control. Iterative optimization methods such as the Gerchberg–Saxton (GS) algorithm are one option for improving image quality. They optimize CGHs in an iterative fashion to obtain a higher image quality. However, such iterative computation is time-consuming, and the improvement in image quality is often stagnant. Recently, deep learning-based hologram computation has been proposed. Deep neural networks directly infer CGHs from input image data. However, it is limited to reconstructing images that are the same size as the hologram. In this study, we use deep learning to optimize phase-only CGHs generated using scaled diffraction computations and the random phase-free method. By combining the random phase-free method with the scaled diffraction computation, it is possible to handle a zoomable reconstructed image larger than the hologram. In comparison to the GS algorithm, the proposed method optimizes both high quality and speed.

Holograms are computable by superimposing zone-plate-like point spread functions (PSFs), representing the distribution of light on the hologram plane. However, due to the computational cost of large-scale holograms, it is difficult to compute them at video rates for high-definition holographic displays. Recently, we proposed radial PSFs for holographic near-eye displays [Appl. Opt.60, 8829-8837, (2021).]. Radial PSFs can speed-up hologram computations for fixed viewpoints, but they are unsuitable for naked-eye displays with wide viewing angles because of the reduced information in the PSFs. This study proposes blocked radial PSFs, which can accelerate computations with radial PSFs, and windmill PSFs (rotational radial PSFs). Both can be applied easily to holographic displays with wide viewing angles.

We report high-speed 3-D holographic video playback including the reproduction of computer-generated holograms from the compression data stored in a single solid state drive with operating system. We succeeded in high quality 3-D video playback for the 3-D object comprising 1,064,462 points at 60 fps using spatiotemporal division electroholography.

We present the real-time three-dimensional display system based on electronic holography. The system uses a motion sensor to detect the finger gesture and the motion of the observer. By switching hologram patterns according to the gesture and motion, we realize the interactive holographic display system.

Ghost imaging reconstructs images only using a single element detector. It has a various advantage. However, one of the challenges is low image quality in undersampling. In this study, we improve the image quality of reconstructed images by using gradient descent.

Spatiotemporal information about light pulse propagation obtained with femtosecond temporal resolution plays an important role in understanding transient phenomena and light–matter interactions. Although ultrafast optical imaging techniques have been developed, it is still difficult to capture light pulse propagation spatiotemporally. Furthermore, imaging through a three-dimensional (3-D) scattering medium is a longstanding challenge due to the optical scattering caused by the interaction between light pulse and a 3-D scattering medium. Here, we propose a technique for ultrafast optical imaging of light pulses propagating inside a 3D scattering medium. We record an image of the light pulse propagation using the ultrashort light pulse even when the interaction between light pulse and a 3-D scattering medium causes the optical scattering. We demonstrated our proposed technique by recording converging, refracted, and diffracted propagating light for 59 ps with femtosecond temporal resolution.

In this study, we first analyze the fully analytical frequency spectrum solving method based on three-dimensional affine transform. Thus, we establish a new method for combining look-up tables (LUTs) with polygon holography. The proposed method was implemented and proved to be accelerated about twice compared to the existing methods. In addition, principal component analysis was used to compress the LUTs, effectively reducing the required memory without artifacts. Finally, we calculated very complex objects on a graphics processing unit using the proposed method, and the calculation speed was higher than that of existing polygon-based methods.

Holograms are computed by superimposing point spread functions (PSFs), which represent the distribution of light on the hologram plane. The computational cost and the spatial bandwidth product required to generate holograms are significant; therefore, it is challenging to compute high-resolution holograms at the rates required for videos. Among the possible displays, fixed-eye-position holographic displays, such as holographic head-mounted displays, reduce the spatial bandwidth product by fixing eye positions while satisfying almost all human depth cues. In eye-fixed holograms, by calculating a part distribution of the entire PSF, we observe reconstructed images that maintain the image quality and the depth of focus almost as high as those generated by the entire PSF. In this study, we accelerate the calculation of eye-fixed holograms by engineering the PSFs. We propose cross and radial PSFs, and we determine that, out of the two, the radial PSFs have a better image quality. By combining the look-up table method and the wavefront-recording plane method with radial PSFs, we show that the proposed method can rapidly compute holograms.

We propose a high-speed playback method for the spatiotemporal division multiplexing electroholographic threedimensional (3D) video stored in a solid-state drive (SSD) using a digital micromirror device. The spatiotemporal division multiplexing electroholography prevents deterioration in the reconstructed 3D video from a 3D object comprising many object points. In the proposed method, the stored data is remarkably reduced using the packing technique, and the computer-generated holograms are played back at high speed. Consequently, we successfully reconstructed a clear 3D video of a 3D object comprising approximately 1,100,000 points at 60 frames per second by reducing the reading time of the stored data from an SSD.

Fourier transform-based diffraction calculations are essential for computational optics. However, the diffraction calculations can be corrupted by the introduction of strong ringing artifacts due to the introduction of zero-padding to avoid circular convolution or to control the sampling intervals. We propose a simple de-ringing method using average subtractions for application to on-axis and off-axis diffraction calculations. To verify the effectiveness of the proposed method, we compared the diffracted fields obtained using zero-padding, a flat-top window method, a mirror expansion method, and the whole and border average subtractions proposed. Furthermore, we confirmed the effectiveness of the proposed method for hologram calculations using double phase encoding and image reconstructions of inline digital holography.

Holographic projection is a simple projection as it enlarges or reduces reconstructed images without using a zoom lens. However, one major problem associated with this projection is the deterioration of image quality as the reconstructed image enlarges. In this paper, we propose a time-division holographic projection, in which the original image is divided into blocks and the holograms of each block are calculated. Using a digital micromirror device (DMD), the holograms were projected at high speed to obtain the entire reconstructed image. However, the holograms on the DMD need to be binarized, thereby causing uneven brightness between the divided blocks. We correct this by controlling the displaying time of each hologram. Additionally, combining both the proposed and noise reduction methods, the image quality of the reconstructed image was improved. Results from the simulation and optical reconstructions show we obtained a full-color reconstruction image with reduced noise and uneven brightness.

Generating new motion parallax holograms is required for holographic head-mounted displays when the head moves. Additionally, it is required for hologram generation from light field data that consist of a number of motion parallax images. However, re-rendering three-dimensional (3D) scenes and re-calculating holograms are computationally complex. Therefore, we propose a generation strategy of holograms with different motion parallax from an existing hologram without re-rendering 3D scenes and re-calculating holograms. The proposed method employs Fourier band-pass filtering and the simple relation of trigonometric functions, which makes it capable of skipping the computationally complex processes.

In this study, we proposed a hologram calculation method for light-in-flight recording by holography (LIF holography). First, we simulated the behavior of ultrashort pulsed light on the diffuser plate using the two-dimensional finite-difference time-domain method. Second, we calculated the light propagation from the diffuser plate to the recording material and generated a hologram based on the calculation model of LIF holography. We reconstructed moving pictures of pulsed-light propagation from the calculated hologram. Because the behavior of the pulsed light in the moving pictures agreed well with that obtained from the reported optical experiment, the proposed method was successfully validated.

Holography is a promising technology for photo-realistic three-dimensional (3D) displays because of its ability to replay the light reflected from an object using a spatial light modulator (SLM). However, the enormous computational requirements for calculating computer-generated holograms (CGHs)-which are displayed on an SLM as a diffraction pattern-are a significant problem for practical uses (e.g., for interactive 3D displays for remote navigation systems). Here, we demonstrate an interactive 3D display system using electro-holography that can operate with a consumer's CPU. The proposed system integrates an efficient and fast CGH computation algorithm for line-drawn 3D objects with inter-frame differencing, so that the trajectory of a line-drawn object that is handwritten on a drawing tablet can be played back interactively using only the CPU. In this system, we used an SLM with 1,920 [Formula: see text] 1,080 pixels and a pixel pitch of 8 μm × 8 μm, a drawing tablet as an interface, and an Intel Core i9-9900K 3.60 GHz CPU. Numerical and optical experiments using a dataset of handwritten inputs show that the proposed system is capable of reproducing handwritten 3D images in real time with sufficient interactivity and image quality.

In this tutorial, we present algorithms and related techniques for computer-generated holograms (CGHs), especially CGHs for display applications, including our per-spectives.

We propose a temporal multiplexing method of two holograms based on off-axis digital holography using acousto-optic modulators. The proposed method can overcome the limitation of the frame rate of an image sensor for digital holography.

Scaled binary ghost imaging has a challenge for high calculation amounts. In this study, we implement a scaled binary ghost imaging into a field-programmable gate array to accelerate the processing time.

We can record digitally designed information of three-dimensional (3D) objects or optical elements on a holographic photosensitive material using wavefront printing technology. However, the amount of hologram data generated from the digitally designed information is very large and there are often occurrences of unnecessary bidirectional communication. To solve this problem, we examined a special-purpose computer for wavefront printing technology. The computer generates light-ray information from digitally designed information of 3D objects, converts the light-ray information to wavefront information, and generates hologram data locally from the wavefront information in interaction. In this study, we designed an emulator of the special-purpose computer for wavefront printing technology and obtained the amount of information (the number of bits) required for the circuit by comparing the 3D images reconstructed from the holograms generated by the emulator. When the amount of data of the generation of the light-ray information from digitally designed information of 3D objects, the conversion of the light-ray information to wavefront information, and the generation of the hologram data locally from the wavefront information in interaction are each about 8 bits, we can keep the peak signal-tonoise ratio of the 3D images close to 30 dB.

三次元空間内に立体を直接描画して表示するディスプレイであるボリュームディスプレイのひとつに，噴出した霧に映像を投影する霧ディスプレイがある．我々はインテグラルフォトグラフィを利用して，一台のプロジェクタのみで霧ディスプレイに三次元映像を空中投影するシステムを提案した．しかし先行研究では，空中投影された三次元映像の視域や視差に関する評価を行っていなかった．そこで，本研究では提案システムで得られる三次元映像の視差や視域に関して評価を行った．

This study proposes a dynamic-range compression for digital holograms generated from three-dimensional scenes using deep neural network (DNN). This method uses an error diffusion algorithm to binarize holograms with an 8-bit gradation; moreover, the DNN predicts the original gradation holograms from binary holograms. This method’s performance exceeds that of JPEG 2000 and high-efficiency video coding.

Electroholography can produce natural 3D scenes and has gained recognition as an ideal 3D technology. However, insufficient computational power has made it difficult to achieve real-time electroholography. In this paper, we developed a compact special-purpose system for calculating phase-only holograms. We implemented the developed system using a system on a chip embedded with a processor and logic circuit blocks. Our system successfully computed holograms of 1,920x1,080 pixels from a point-cloud with 32,500 points at 10 frames per second. The system is 147 times faster than a personal computer (with 6 CPU cores). (C) 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement