In a globular cluster, hierarchical triple black hole systems can be produced through binary-binary interaction. It has been proposed recently that the Kozai mechanism could drive the inner binary of the triple system to merge before it is interrupted by interactions with other field stars. We investigate qualitatively and numerically the evolution of the eccentricities in these binaries under gravitational radiation (GR) reaction. We predict that ∼30% of the systems will possess eccentricities greater than 0.1 when their emitted gravitational waves pass through 10 Hz frequency. The implications for gravitational wave detection, especially the relevance to data analyses for broadband laser interferometer gravitational wave detectors, are discussed.