We use heavy minerals and rutile and garnet chemical compositions to constrain the provenance of two glaciogenic sandstone formations that build up the Palaeozoic succession in Ethiopia. The heavy mineral assemblage of the Upper Ordovician–Lower Silurian Enticho Sandstone is dominated by ultra-stable minerals, implying high maturity of the sediment. Variable amounts of garnet are present as well. The Carboniferous–Permian Edaga Arbi Glacials contain mainly less stable heavy minerals, such as garnet and apatite, suggesting little chemical alteration. A combination of magmatic and metamorphic source rocks is likely for both formations. Rutile and garnet chemistry point to mainly amphibolite-facies and to a lesser extent granulite-facies metamorphic source rocks with generally slightly higher metamorphic temperatures for detrital heavy minerals in the Enticho Sandstone. We conclude that the Enticho Sandstone is mainly the product of reworked mature Cambrian–Ordovician sediment, which may have been supplied via the Gondwana super-fan system. Locally, glaciers of the Late Ordovician glaciation eroded fresh basement material, delivering the garnet. For the Edaga Arbi Glacials, a rather proximal provenance is likely. The potential source area is the southern hinterland, where Precambrian low-to higher grade metamorphic rocks of the Arabian–Nubian Shield occur at the transition to the Mozambique Belt.