Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds

Uzair Nadeem; Mohammad A.A.K. Jalwana; Mohammed Bennamoun; Roberto Togneri; Ferdous Sohel

doi:10.1007/978-3-030-36711-4_20

Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds

Uzair Nadeem, Mohammad A.A.K. Jalwana, Mohammed Bennamoun, Roberto Togneri, Ferdous Sohel

Research output: Chapter in Book/Conference paper › Conference paper › peer-review

2 Citations (Scopus)

Abstract

We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D key-points and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D key-points by matching the 2D descriptors with the pre-extracted 3D descriptors of the point cloud. This information is used in a robust pose estimation algorithm to localize the query image in the 3D point cloud. Experiments demonstrate that directly matching 2D and 3D descriptors is not only a viable idea but can also be used for camera pose localization in dense 3D point clouds with high accuracy.

Original language	English
Title of host publication	Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings
Editors	Tom Gedeon, Kok Wai Wong, Minho Lee
Place of Publication	Australia
Publisher	Springer
Pages	222-234
Number of pages	13
ISBN (Print)	9783030367107
DOIs	https://doi.org/10.1007/978-3-030-36711-4_20
Publication status	Published - 1 Jan 2019
Event	26th International Conference on Neural Information Processing - Sydney, Australia Duration: 12 Dec 2019 → 15 Dec 2019 http://ajiips.com.au/iconip2019/

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11954 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Neural Information Processing
Abbreviated title	ICONIP 2019
Country/Territory	Australia
City	Sydney
Period	12/12/19 → 15/12/19
Internet address	http://ajiips.com.au/iconip2019/

Access to Document

10.1007/978-3-030-36711-4_20

Cite this

Nadeem, U., Jalwana, M. A. A. K., Bennamoun, M., Togneri, R., & Sohel, F. (2019). Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds. In T. Gedeon, K. W. Wong, & M. Lee (Eds.), Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings (pp. 222-234). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11954 LNCS). Springer. https://doi.org/10.1007/978-3-030-36711-4_20

Nadeem, Uzair ; Jalwana, Mohammad A.A.K. ; Bennamoun, Mohammed et al. / Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds. Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. editor / Tom Gedeon ; Kok Wai Wong ; Minho Lee. Australia : Springer, 2019. pp. 222-234 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{dd2d58ed65e04d86bdfbd33e90f93176,

title = "Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds",

abstract = "We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D key-points and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D key-points by matching the 2D descriptors with the pre-extracted 3D descriptors of the point cloud. This information is used in a robust pose estimation algorithm to localize the query image in the 3D point cloud. Experiments demonstrate that directly matching 2D and 3D descriptors is not only a viable idea but can also be used for camera pose localization in dense 3D point clouds with high accuracy.",

keywords = "3D to 2D matching, 6-DOF pose estimation, Image localization, Multi-domain descriptor matching",

author = "Uzair Nadeem and Jalwana, {Mohammad A.A.K.} and Mohammed Bennamoun and Roberto Togneri and Ferdous Sohel",

year = "2019",

month = jan,

day = "1",

doi = "10.1007/978-3-030-36711-4_20",

language = "English",

isbn = "9783030367107",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "222--234",

editor = "Tom Gedeon and Wong, {Kok Wai} and Minho Lee",

booktitle = "Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings",

address = "Netherlands",

note = "26th International Conference on Neural Information Processing, ICONIP 2019 ; Conference date: 12-12-2019 Through 15-12-2019",

url = "http://ajiips.com.au/iconip2019/",

}

Nadeem, U, Jalwana, MAAK, Bennamoun, M , Togneri, R & Sohel, F 2019, Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds. in T Gedeon, KW Wong & M Lee (eds), Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11954 LNCS, Springer, Australia, pp. 222-234, 26th International Conference on Neural Information Processing, Sydney, New South Wales, Australia, 12/12/19. https://doi.org/10.1007/978-3-030-36711-4_20

Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds. / Nadeem, Uzair; Jalwana, Mohammad A.A.K.; Bennamoun, Mohammed et al.
Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. ed. / Tom Gedeon; Kok Wai Wong; Minho Lee. Australia: Springer, 2019. p. 222-234 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11954 LNCS).

Research output: Chapter in Book/Conference paper › Conference paper › peer-review

TY - GEN

T1 - Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds

AU - Nadeem, Uzair

AU - Jalwana, Mohammad A.A.K.

AU - Bennamoun, Mohammed

AU - Togneri, Roberto

AU - Sohel, Ferdous

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D key-points and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D key-points by matching the 2D descriptors with the pre-extracted 3D descriptors of the point cloud. This information is used in a robust pose estimation algorithm to localize the query image in the 3D point cloud. Experiments demonstrate that directly matching 2D and 3D descriptors is not only a viable idea but can also be used for camera pose localization in dense 3D point clouds with high accuracy.

AB - We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D key-points and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D key-points by matching the 2D descriptors with the pre-extracted 3D descriptors of the point cloud. This information is used in a robust pose estimation algorithm to localize the query image in the 3D point cloud. Experiments demonstrate that directly matching 2D and 3D descriptors is not only a viable idea but can also be used for camera pose localization in dense 3D point clouds with high accuracy.

KW - 3D to 2D matching

KW - 6-DOF pose estimation

KW - Image localization

KW - Multi-domain descriptor matching

UR - http://www.scopus.com/inward/record.url?scp=85076914522&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-36711-4_20

DO - 10.1007/978-3-030-36711-4_20

M3 - Conference paper

AN - SCOPUS:85076914522

SN - 9783030367107

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 222

EP - 234

BT - Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings

A2 - Gedeon, Tom

A2 - Wong, Kok Wai

A2 - Lee, Minho

PB - Springer

CY - Australia

T2 - 26th International Conference on Neural Information Processing

Y2 - 12 December 2019 through 15 December 2019

ER -

Nadeem U, Jalwana MAAK, Bennamoun M , Togneri R, Sohel F. Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds. In Gedeon T, Wong KW, Lee M, editors, Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. Australia: Springer. 2019. p. 222-234. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-36711-4_20

Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this