As a backbone force driving technological innovation and industrial transformation, the innovation performance of start-up innovative private enterprises (SIPEs) is influenced by multi-factor influences, including institutional environment, managerial cognition, and strategic behavior. Drawing on institutional logic theory, this study constructs an integrated “environment-cognition-behavior” framework and applies fuzzy-set qualitative comparative analysis (fsQCA) to 82 SIPEs across multiple Chinese cities. Results reveal two configurations driving high innovation performance: a business environment-led model and a “symbiosis+policy perception” joint-driven model. Business environment, policy perception and symbiotic orientation serve as core drivers, while legitimacy and connectivity orientations function as auxiliary drivers. Under certain conditions, policy perception, business environment and innovation ecosystem strategy exhibit substitutive relationships, with symbiotic orientation and policy perception proving particularly critical. These findings offer new theoretical insights into firm innovation mechanisms under complex institutional contexts and provide targeted guidance for firms formulating adaptive innovation ecosystem strategies and governments optimizing the business environment.