City is the core space carrier of national innovation-driven development and being self-reliance on science and technology. It will help China better participate in global scientific and technology competition and cooperation into the global innovation network to enhance the performance of urban innovation. Taking 280 cities above the prefecture level in China as objects, this paper explored the impact of urban hierarchy on innovation performance through the bidirectional fixed effects model, instrumental variable method and generalized moment estimation, and used the threshold regression model to reveal the nonlinear relationship between urban hierarchy and innovation performance, and at last discussed the stage characteristics of different urban tiers for innovation performance. The results show: (1) The promotion of China’s urban tier can significantly promote innovation performance. Urban tier is the prerequisite of the promotion of urban innovation performance, and there is no reverse causality. (2) There is regional heterogeneity in the impact of urban hierarchy on innovation performance. The promotion of urban tier in Northeast China has the largest promotion effect on innovation performance, followed by the eastern, the western and the central region. The promotion of urban tier in innovation-oriented cities has a greater effect on innovation performance than that in non-innovation-oriented cities. (3) The impact of urban hierarchy on innovation performance has a nonlinear characteristic. Unreasonable city size will cause the loss of innovation performance.