Exploring the effective path of factor allocation structure optimization is crucial to unleashing the economic growth potential, and it is also the key to breaking the unbalanced and insufficient development and promoting high-quality development. This paper incorporates the dual constraints of input structure optimization and environmental welfare non-decreasing to design a model for measuring the factor allocation structural efficiency. The study examines the structural differences and spatio-temporal evolution of China’s factor allocation structural efficiency using the Dagum Gini coefficient and the dynamic spatial Durbin model with the “Hu Huanyong Line” as the boundary. The findings suggest that China’s factor allocation structural efficiency is slightly decreasing due to constraints from the development effect. The contribution of “capital—non-clean energy—clean energy—labor—environmental construction” is decreasing in turn, but the coordination effect has slightly improved. The overall difference in China’s factor allocation structural efficiency is constantly expanding, mainly due to the differences between regions, and there are global spatial agglomeration characteristics and β conditional convergence. Regional factor allocation structural efficiency decreases in the gradient of “Southeast—Along the line—Northwest”, all constrained by the development effect led by capital. Additionally, there is a club convergence mechanism.