Phenotypic plasticity is the nearly ubiquitous ability of a genotype to produce different phenotypes in response to different environments, but how such non-heritable variation contributes to adaptive evolution is controversial. Using replicated experiments in natural populations, we found that genes exhibiting non-adaptive plasticity in expression were more likely to contribute to rapid evolutionary divergence than were genes exhibiting adaptive plasticity or no plasticity. Guppies were transplanted from a stream with cichlid predators to two streams lacking cichlids, and adaptive evolution in a subset of expressed genes occurred in only 3-4 generations. We tested the ancestral plasticity of these genes by rearing the source population with and without the cichlid predator, and found the direction of plasticity in transcript abundance was opposite to the direction of parallel changes observed in the introduction populations. These results suggest that strong directional selection against genes exhibiting non-adaptive plasticity can lead to rapid evolution.