Concentration of mineral elements in rice seeds may vary depending on the soil fertility status and seed polishing (milling). In order to test this, 54 rice varieties were grown in the fields used to cultivate rice either with or without applying fertilizers for over 40 years. Part of the seeds collected from those varieties was polished before using for element analyses using ICP-MS while the rest considered as unpolished seeds. When comparing unpolished rice seeds produced at fertilized and unfertilized sites, the concentrations of Mg, Ca, K, Mn, Co, Cu, As and Cd were decreased, and the concentrations of Na, Al, Fe, Zn, and Mo were unchanged from fertilized to unfertilized sites. When comparing unpolished and polished rice seeds produced at fertilized and unfertilized sites, seed Mg, K, Mn, Fe, Co and Zn concentrations were higher in unpolished seeds than those in polished seeds, and the concentrations of Na, Al, Mo, Cd, As and Pb were similar between polished and unpolished seeds. Estimated values revealed that rice contributed 4–15% of the daily requirement of Fe, Cu, Zn, and 50–60% Mo irrespective of the soil fertility and polishing. Seed concentrations of As, Pb and Cd were lower than the maximum allowable levels. However, rice consumption contributed to the daily intake of 4–8 µg as, 2–6.5 µg Cd, and 19–25 µg Pb. Improving soil fertility is required to enrich rice seeds with most of the micronutrients and the milling should be done with much care to minimize nutrient loss in human diet.