Skutterudites are crystals with a cagelike structure that can be augmented with filler atoms (“rattlers”), usually leading to a reduction in thermal conductivity that can be exploited for thermoelectric applications. Here, we leverage the recently introduced Wigner formulation of thermal transport to elucidate the microscopic physics underlying heat conduction in skutterudites, showing that filler atoms can drive a crossover from the Boltzmann to the Wigner regimes of thermal transport, i.e., from particlelike conduction to wavelike tunneling. At temperatures where the thermoelectric efficiency of skutterudites is largest, wavelike tunneling can become comparable to particlelike propagation. We define a Boltzmann deviation descriptor able to differentiate the two regimes and relate the competition between the two mechanisms to the materials' chemistry, providing a design strategy to select rattlers and identify optimal compositions for thermoelectric applications.