How can deep rectifier networks achieve linear separability and preserve distances?

Research output: Chapter in Book/Conference paperConference paper

10 Citations (Scopus)
10 Downloads (Pure)

Abstract

This paper investigates how hidden layers of deep rectifier networks are capable of transforming two or more pattern sets to be linearly separable while preserving the distances with a guaranteed degree, and proves the universal classification power of such distance preserving rectifier networks. Through the nearly isometric nonlinear transformation in the hidden layers, the margin of the linear separating plane in the output layer and the margin of the nonlinear separating boundary in the original data space can be closely related so that the maximum margin classification in the input data space can be achieved approximately via the maximum margin linear classifiers in the output layer. The generalization performance of such distance preserving deep rectifier neural networks can be well justified by the distance-preserving properties of their hidden layers and the maximum margin property of the linear classifiers in the output layer.
Original languageEnglish
Title of host publicationProceedings of The 32nd International Conference on Machine Learning
PublisherInternational Machine Learning Society
Pages514-523
Volume37
ISBN (Print)9781510810587
Publication statusPublished - 2015
EventThe 32nd International Conference on Machine Learning - Lille, France
Duration: 6 Jul 201511 Jul 2015

Conference

ConferenceThe 32nd International Conference on Machine Learning
CountryFrance
CityLille
Period6/07/1511/07/15

Fingerprint Dive into the research topics of 'How can deep rectifier networks achieve linear separability and preserve distances?'. Together they form a unique fingerprint.

  • Cite this

    An, S., Boussaid, F., & Bennamoun, M. (2015). How can deep rectifier networks achieve linear separability and preserve distances? In Proceedings of The 32nd International Conference on Machine Learning (Vol. 37, pp. 514-523). International Machine Learning Society.