Shallow hydrothermal systems share many characteristics with their deep-sea counterparts, but their accessibility facilitates their study. One of the most studied shallow hydrothermal vent fields lies at Paleochori Bay off the coast of Milos in the Aegean Sea (Greece). It has been studied through extensive mapping and its physical and chemical processes have been characterized over the past decades. However, a thorough description of the microbial communities inhabiting the bay is still missing. Here, we present the first in-depth characterization of the prokaryotic communities of Paleochori Bay by sampling eight different seafloor types that are distributed along the entire gradient of hydrothermal influence. We used deep sequencing of the 16S rRNA marker gene and complemented the analysis with qPCR quantification of the 16S rRNA gene and several functional genes to gain insights into the metabolic potential of the communities. We found that the microbiome of the bay is strongly influenced by the hydrothermal venting, with a succession of various groups dominating the sediments from the coldest to the warmest zones. Prokaryotic diversity and abundance decrease with increasing temperature, and thermophilic archaea overtake the community. This study expands our limited understanding of subsurface microbial communities in acidic shallow-sea hydrothermal systems and the contribution of their microbial activity to biogeochemical cycling.