Countries across the world responded to the COVID-19 pandemic with what might well be the set of biggest state-led mobility and activity restrictions in the history of humankind. But how effective were these measures across countries? Compared to multiple recent studies that document an association between such restrictions and the control of the contagion, we use an instrumental variable approach to estimate the causal effect of these restrictions on mobility, and the growth rate of confirmed cases and deaths during the first wave of the pandemic. Using the level of stringency in the rest of the world to predict the level of stringency of the restriction measures in a country, we show while stricter contemporaneous measures affected mobility, stringency in seven to fourteen days mattered most for containing the contagion. Heterogeneity analysis, by various institutional inequalities, reveals that even though the restrictions reduced mobility more in relatively less-developed countries, the causal effect of a reduction in mobility was higher in more developed countries. We propose several explanations. Our results highlight the need to complement mobility and activity restrictions with other health and information measures, especially in less-developed countries, to combat the COVID-19 pandemic effectively.