Deepening the reform of the ecological civilization system and improving the ecological compensation system are inherent requirements for achieving harmonious coexistence between humans and nature. How to optimize the ecological compensation reward and punishment mechanism is an important issue for improving the ecological environment governance system. This article utilizes the double difference in the implementation of air quality ecological compensation policies across regions and years, reveals the environmental governance effects and underlying mechanisms of the ecological compensation reward and punishment mechanism reform from both theoretical and empirical perspectives, and tests its mechanism from the perspective of subjective initiative enhancement. Research has found that the reform of the reward and punishment mechanism, by introducing a horizontal competition mechanism, has a significant improvement effect on urban air quality under the “higher-level government led” model, while the “local government spontaneous” model has not significantly improved urban air quality. Mechanism analysis found that the reform of the reward and punishment mechanism achieves environmental governance effects through three mechanisms: government initiated governance, industrial structure improvement, and innovation driven. Moreover, the differences in policy content design and policy linkage will further affect the effectiveness of environmental governance. Further analysis reveals that the horizontal competition mode mainly mobilizes the subjective initiative of local environmental governance through the transmission channels of “institutional incentives” and “financial compensation”, and the reward and punishment mechanism can achieve synergistic efficiency in reducing pollution and carbon emissions. The research conclusion of this article provides certain reference value for further improving the ecological compensation system, comprehensively improving the national governance system, and promoting the construction of ecological civilization.