We develop a general formalism of duality rotations for bosonic conformal spin-s gauge fields, with s≥2, in a conformally flat four-dimensional spacetime. In the s=1 case, this formalism is equivalent to the theory of U(1) duality-invariant nonlinear electrodynamics developed by Gaillard and Zumino, Gibbons and Rasheed, and generalized by Ivanov and Zupnik. For each integer spin s≥2 we demonstrate the existence of families of conformal U(1) duality-invariant models, including a generalization of the so-called ModMax electrodynamics (s=1). Our bosonic results are then extended to the N=1 and N=2 supersymmetric cases. We also sketch a formalism of duality rotations for conformal gauge fields of Lorentz type (m/2,n/2), for positive integers m and n.