Inpainting Sparse Scenes through Physics Aware Transformers for Single-Photon LiDAR

Luke McEvoy, Daniel Tafone, Yong Meng Sua, Yuping Huang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

We increase single-photon LiDAR capabilities via a hardware-accelerating inpainting transformer model. This model reconstructs all non-observed information within the image plane as it communicates with the beam steering hardware. We apply this to 3D time-of-flight (ToF) reconstruction, where objects obstruct each other’s line of sight. We use ToF histograms to distinguish objects within either the foreground and background, and their overlap will be treated as the dynamic mask for the model to reconstruct. We also employ this to unorthodox scanning patterns such as Lissajous and spiral, which are riddled with sparsity. Lastly, we are developing an AI MEMs system, which intelligently downsamples the image plane based off foreground masks, combating sampling redundancy. We believe that our approach will be useful in applications for imaging and sensing dynamic targets with sparse single-photon data across all domains.

Original languageEnglish
Title of host publicationUnconventional Optical Imaging IV
EditorsIrene Georgakoudi, Marc P. Georges, Nicolas Verrier
ISBN (Electronic)9781510673106
DOIs
StatePublished - 2024
EventUnconventional Optical Imaging IV 2024 - Strasbourg, France
Duration: 8 Apr 202411 Apr 2024

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume12996
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceUnconventional Optical Imaging IV 2024
Country/TerritoryFrance
CityStrasbourg
Period8/04/2411/04/24

Keywords

  • Artificial Intelligent
  • Machine Vision
  • Quantum Optics
  • Sparse Imaging
  • Transformers

Fingerprint

Dive into the research topics of 'Inpainting Sparse Scenes through Physics Aware Transformers for Single-Photon LiDAR'. Together they form a unique fingerprint.

Cite this