Geopolymer concrete (GC) has been gaining attention in research and engineering circles; however, it is a brittle material with poor tensile performance and crack resistance. To address these problems, we introduced fibers into GC. In this study, axial compression and scanning electron microscope (SEM) tests were carried out on polyvinyl alcohol (PVA) short fiber reinforced low-calcium fly ash-slag-based geopolymer concrete (PFRGC). The ratio of PVA short fibers and low-calcium fly ash on the compression behavior of fiber reinforced geopolymer concrete (FRGC) were investigated and discussed. The test results show that PVA fibers play a bridging role in the cracks of the specimen and bear the load together with the matrix, so the addition of PVA fibers delayed the crack propagation of GC under axial compression. However, with the increase of low-calcium fly ash/PVA fibers, the number of unreacted fly ash particles in PFRGCs increases. Too many unreacted fly ash particles make GC more prone to micro-cracks during loading, adversely affecting compressive properties. Therefore, the axial compressive strength, elastic modulus, and Poisson’s ratio of GC decrease with the increasing low-calcium fly ash/PVA fibers.