Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping

Kageyama, Yuta; Iwai, Daisuke; Sato, Kosuke

IEEE Transactions on Visualization and Computer Graphics, 2024, 30(12), 7544-7557

Number of Access:108（2025-06-06 18:42 Counts）

Identifier to cite or link to this item: https://hdl.handle.net/11094/94597

Link to primary information 

File Format Terms of use Size Views Date.Available Description information
IEEETransVisualComputGraphics pdf None 4.81 MB 205 2024.03.04  
Creative Commons : Attribution - NonCommercial - NoDerivatives

Item Information

Output File Export EndNote Basic Export Mendeley

Title
Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping
Creator
CreatorName
Kageyama, Yuta
NRID
1000090504837 1000090504837
researchmap permalink
daisukeiwai daisukeiwai
Scopus Author ID
55931917600 55931917600
CreatorName
Iwai, Daisuke
NRID
1000090187188 1000090187188
researchmap permalink
read0076311 read0076311
Scopus Author ID
56192879500 56192879500
CreatorName
Sato, Kosuke
Subject
Calibration
Cameras
Deep neural networks
Degradation
dynamic projection mapping
geometric compensation
Image quality
Optical distortion
projector deblurring
Real-time systems
Training
Description
Kageyama Y., Iwai D., Sato K.. Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping. IEEE Transactions on Visualization and Computer Graphics , (2024); https://doi.org/10.1109/TVCG.2024.3354957.
Abstract
Dynamic Projection Mapping (DPM) necessitates geometric compensation of the projection image based on the position and orientation of moving objects. Additionally, the projector's shallow depth of field results in pronounced defocus blur even with minimal object movement. Achieving delay-free DPM with high image quality requires real-time implementation of geometric compensation and projector deblurring. To meet this demand, we propose a framework comprising two neural components: one for geometric compensation and another for projector deblurring. The former component warps the image by detecting the optical flow of each pixel in both the projection and captured images. The latter component performs real-time sharpening as needed. Ideally, our network's parameters should be trained on data acquired in an actual environment. However, training the network from scratch while executing DPM, which demands real-time image generation, is impractical. Therefore, the network must undergo pre-training. Unfortunately, there are no publicly available large real datasets for DPM due to the diverse image quality degradation patterns. To address this challenge, we propose a realistic synthetic data generation method that numerically models geometric distortion and defocus blur in real-world DPM. Through exhaustive experiments, we have confirmed that the model trained on the proposed dataset achieves projector deblurring in the presence of geometric distortions with a quality comparable to state-of-the-art methods.
Publisher
IEEE Computer Society
Source Title
IEEE Transactions on Visualization and Computer Graphics
Volume (Issue)
30(12)
Page
7544-7557
Date of Issued
2024-01-01
Language
English
Handle URL
https://hdl.handle.net/11094/94597
PISSN
10772626
eISSN
19410506
Relation.isIdenticalTo
DOI (Publisher Version)
https://doi.org/10.1109/TVCG.2024.3354957
Access Rights
open access
CopyRight
This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
oaire:version
Version of Record
Category
学術雑誌論文 Journal Article