Background: Amplitude-integrated electroencephalogram (aEEG) is being used increasingly for seizure detection in neonates. However, data regarding inter-rater reliability among neonatologists for the use of aEEG for the detection of neonatal seizures is lacking. Methods: Term and late-preterm infants at risk of seizures were monitored simultaneously with 24-h video-electroencephalography (vEEG) and aEEG. vEEG was interpreted by an experienced neurologist. Five neonatologists with experience in aEEG interpretation from four different neonatal units interpreted aEEG recordings independently. The Brennan and Prediger kappa coefficient and Intra-class Correlation Coefficients (ICC) were used to assess inter-rater reliability between the neonatologists. Results: Thirty-five infants at risk of seizure with gestational age at birth 35–42 weeks were recruited for the study after informed parental consent. vEEG detected seizures in seven infants with a total of 169 individual seizure episodes. Neonatologists detected seizures in 10 to 15 infants on aEEG. The sensitivities for the detection of individual seizures by neonatologists ranged from 18% to 38%. The inter-rater reliability for detection of: individual seizure was “fair” (kappa = 0.37; 95% CI: 0.32–0.42), infant with seizure was “moderate” (kappa = 0.60; 95% CI: 0.44–0.75), duration of individual seizure (ICC: 0.22; 95% CI: 0.18–0.28) and total duration of seizures in an infant (ICC: 0.46; 95% CI: 0.30–0.63) was “poor”. The neonatologists missed 77–90% of the duration of seizures. Conclusion: The inter-rater reliability of aEEG for the detection of neonatal seizures was suboptimal. Even when interpreted by experienced and trained clinicians, seizure detection with aEEG has limitations and can miss large number and duration of seizures.