Do End-to-end Stereo Algorithms Under-utilize Information?

Changjiang Cai, Philippos Mordohai

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Deep networks for stereo matching typically leverage 2D or 3D convolutional encoder-decoder architectures to aggregate cost and regularize the cost volume for accurate disparity estimation. Due to content-insensitive convolutions and down-sampling and up-sampling operations, these cost aggregation mechanisms do not take full advantage of the information available in the images. Disparity maps suffer from over-smoothing near occlusion boundaries, and erroneous predictions in thin structures. In this paper, we show how deep adaptive filtering and differentiable semi-global aggregation can be integrated in existing 2D and 3D convolutional networks for end-to-end stereo matching, leading to improved accuracy. The improvements are due to utilizing RGB information from the images as a signal to dynamically guide the matching process, in addition to being the signal we attempt to match across the images. We show extensive experimental results on the KITTI 2015 and Virtual KITTI 2 datasets comparing four stereo networks (DispNetC, GCNet, PSMNet and GANet) after integrating four adaptive filters (segmentation-aware bilateral filtering, dynamic filtering networks, pixel adaptive convolution and semi-global aggregation) into their architectures. Our code is available at https://github.com/ccj5351/DAFStereoNets.

Original languageEnglish
Title of host publicationProceedings - 2020 International Conference on 3D Vision, 3DV 2020
Pages374-383
Number of pages10
ISBN (Electronic)9781728181288
DOIs
StatePublished - Nov 2020
Event8th International Conference on 3D Vision, 3DV 2020 - Virtual, Fukuoka, Japan
Duration: 25 Nov 202028 Nov 2020

Publication series

NameProceedings - 2020 International Conference on 3D Vision, 3DV 2020

Conference

Conference8th International Conference on 3D Vision, 3DV 2020
Country/TerritoryJapan
CityVirtual, Fukuoka
Period25/11/2028/11/20

Fingerprint

Dive into the research topics of 'Do End-to-end Stereo Algorithms Under-utilize Information?'. Together they form a unique fingerprint.

Cite this