The concept of self-dual supersymmetric nonlinear electrodynamics is generalized to a curved superspace of N=1 supergravity, for both the old minimal and the new minimal versions of N=1 supergravity. We derive the self-duality equation, which has to be satisfied by the action functional of any U(1) duality invariant model of a massless vector multiplet, and construct a family of self-dual nonlinear models. This family includes a curved superspace extension of the N=1 super Born-Infeld action. The supercurrent and supertrace in such models are proved to be duality invariant. The most interesting and unexpected result is that the requirement of nonlinear self-duality yields nontrivial couplings of the vector multiplet to Kahler sigma models. We explicitly derive the couplings to general Kahler sigma models in the case when the matter chiral multiplets are inert under the duality rotations, and more specifically to the dilaton-axion chiral multiplet when the group of duality rotations is enhanced to SL(2,R).