The sampling problem lies at the heart of atomistic simulations and over the years many different enhanced sampling methods have been suggested towards its solution. These methods are often grouped into two broad families. On the one hand methods such as umbrella sampling and metadynamics that build a bias potential based on few order parameters or collective variables. On the other hand tempering methods such as replica exchange that combine different thermodynamic ensembles in one single expanded ensemble. We adopt instead a unifying perspective, focusing on the target probability distribution sampled by the different methods. This allows us to introduce a new method that can sample any of the ensembles normally sampled via replica exchange, but does so in a collective-variables-based scheme. This method is an extension of the recently developed on-the-fly probability enhanced sampling method [Invernizzi and Parrinello, J. Phys. Chem. Lett. 11.7 (2020)] that has been previously used for metadynamics-like sampling. The method is thus very general and can be used to achieve different types of enhanced sampling. It is also reliable and simple to use, since it presents only few and robust external parameters and has a straightforward reweighting scheme. Furthermore, it can be used with any number of parallel replicas. We show the versatility of our approach with applications to multicanonical and multithermal-multibaric simulations, thermodynamic integration, umbrella sampling, and combinations thereof.