Citrus orchards receive increasing amounts of nitrogen fertilizer for the purpose of optimal yields and good quality of citrus fruits. Although the effects of increased nitrogen fertilizer on citrus fruit trees have been reasonably well studied, few comparable studies have examined impacts on soil bacterial communities and diversity, even though they play critical roles in orchard ecosystem functioning. In our study, Illumina MiSeq sequencing was used to investigate bacterial community structure and diversity under 5-years long-term nitrogen fertilization gradients (N0, N1, N2, N3, N4, and N5) in citrus orchard soils. The sequencing result showed that nitrogen fertilizer addition increased bacterial diversity along the N0 to N3 gradient but decreased bacterial diversity along the N3 to N5 gradient. The increase in the nitrogen fertilizer rate altered bacterial community composition by increasing the relative abundance of Delta-proteobacteria, Nitrospirae, SBR1093, and Latescibacteria and decreasing the relative abundance of Alpha-proteobacteria. Finally, regression analysis revealed that bacterial diversity and the relative abundance of Nitrosomonadales, Rhodobiaceae, Gemmatimonas, and Variibacter exhibited a significant positive correlation with citrus yield. The study revealed that a reasonable nitrogen fertilizer rate applied to citrus orchards could improve bacterial community structure and diversity and increase citrus yield.