Rice is highly sensitive to temperature stress (cold and heat), particularly during the reproductive and grain-filling stages. In this review, we discuss the effects of low- and high-temperature sensitivity in rice at various reproductive stages (from meiosis to grain development) and propose strategies for improving the tolerance of rice to terminal thermal stress. Cold stress impacts reproductive development through (i) delayed heading, due to its effect on anther respiration, which increases sucrose accumulation, protein denaturation and asparagine levels, and decreases proline accumulation, (ii) pollen sterility owing to tapetal hypertrophy and related nutrient imbalances, (iii) reduced activity of cell wall bound invertase in the tapetum of rice anthers, (iv) impaired fertilization due to inhibited anther dehiscence, stigma receptivity and ability of the pollen tube to germinate through the style towards the ovary, and (v) floret sterility, which increases grain abortion, restricts grain size, and thus reduces grain yield. Heat stress affects grain formation and development through (i) poor anther dehiscence due to restricted closure of the locules, leading to reduced pollen dispersal and fewer pollen on the stigma, (ii) changes in pollen proteins resulting in significant reductions in pollen viability and pollen tube growth, leading to spikelet sterility, (iii) delay in heading, (iv) reduced starch biosynthesis in developing grain, which reduces starch accumulation, (v) increased chalkiness of grain with irregular and round-shaped starch granules, and (vi) a shortened grain-filling period resulting in low grain weight. However, physiological and biotechnological tools, along with integrated management and adaptation options, as well as conventional breeding, can help to develop new rice genotypes possessing better grain yield under thermal stress during reproductive and grain-filling phases.