The use of ear shape as a biometric trait is a recent trend in research. However, fast and accurate detection and recognition of the ear are very challenging because of its complex geometry. In this work, a very fast 2D AdaBoost detector is combined with fast 3D local feature matching and fine matching via an Iterative Closest Point (ICP) algorithm to obtain a complete, robust and fully automatic system with a good balance between speed and accuracy. Ear images are detected from 2D profile images using the proposed Cascaded AdaBoost detector. The corresponding 3D ear data is then extracted from the co-registered range image and represented with local 3D features. Unlike previous approaches, local features are used to construct a rejection classifier, to extract a minimal region with feature-rich data points and finally, to compute the initial transformation for matching with the ICP algorithm. The proposed system provides a detection rate of 99.9% and an identification rate of 95.4% on Collection F of the UND database. On a Core 2 Quad 9550, 2.83 GHz machine, it takes around 7.7 ms to detect an ear from a 640×480 image. Extracting features from an ear takes 22.2 sec and matching it with a gallery using only the local features takes 0.06 sec while using the full matching including ICP requires 2.28 sec on average.