Three alternated D-π-A type 5,10-bis(triisopropylsilylethynyl)dithieno[2,3-d:2′,3′-d′]-benzo[1,2-b:4,5-b′]dithiophene (DTBDT-TIPS)-based semiconducting conjugated copolymers (CPs), PDTBDT-TIPS-DTBT-OD, PDTBDT-TIPS-DTFBT-OD, and PDTBDT-TIPS-DTNT-OD, bearing different A units, including benzothiadiazole (BT), 5,6-difluorinated-BT (FBT) and naphtho[1,2-c:5,6-c′]-bis[1,2,5]thiadiazole (NT), were designed and synthesized to investigate the impact of the variation in electron-deficient units on the properties of these photovoltaic polymers. It was exhibited that the down-shifted highest occupied molecular orbital energy level (EHOMO), the enhanced aggregation in both the chlorobenzene solution and the solid film, as well as the better molecular planarity, were achieved using methods involving fluorination and the replacement of BT with NT on the polymer backbone. The absorption profile was little changed upon fluorination; however, it was greatly broadened during replacement of BT with NT. Consequently, the optimized photovoltaic device based on the PDTBDT-TIPS-DTNT-OD exhibited synchronous enhancements in the open-circuit voltage (VOC) of 0.88 V, the short-circuit current density (JSC) of 7.21 mA cm−2, and the fill factor (FF) of 52.99%, resulting in a drastic elevation in the PCE by 129% to 3.37% compared to that of the PDTBDT-TIPS-DTBT-OD. This was triggered by PDTBDT-TIPS-DTNT-OD’s broadened absorption, deepened EHOMO, improved coplanarity, and enhanced SCLC mobility (which increased 3.9 times), as well as a favorable morphology of the active layer. Unfortunately, the corresponding PCE deteriorated after incorporating fluorine into the BT, due to the oversized aggregation and large phase separation morphology in the blend films, severely impairing its JSC. Our preliminary results demonstrated that the replacement of BT with NT in a D-π-A type polymer backbone was an effective strategy of tuning the molecular structure to achieve highly efficient polymer solar cells (PSCs).