(1) Background: In accordance with the socio-ecological model of physical education (PE), school-based interventions to promote physical activity (PA) will only be successful if a change occurs in the perceptions and attitudes of all. This study sought to analyze non-PE teachers experiences in PE and how these experiences relate to the value they attribute to PE, sports and PA, and the impact of teachers’ perceptions on promoting active lifestyles in the school’s context. A mixed-method study was adopted. (2) Methods: A survey about the perceptions in PE was applied to 297 teachers (58 male). From this sample, 24 teachers were selected for three focus groups considering their experiences in PE (e.g., good experiences, bad experiences). MaxQda was used for the inductive qualitative data analysis. (3) Results: Teachers who have had good experiences in PE value the role of PE and sports. Teachers who have had bad experiences do not value the importance of PE or sports. (4) Conclusions: The results support the importance of having good experiences in PE to generate positive attitudes. Thus, teachers who have had bad experiences in PE may constitute barriers that will hinder the promotion of an educational climate that promotes PA and active lifestyles in schools.