Single-Carrier Frequency Division Multiple Access (SC-FDMA) is a promising technique for high data rate indoor Terahertz (THz) communications in future beyond 5G systems. In an indoor propagation scenario, the Line-Of-Sight (LOS) component may be blocked by the obstacles. Thus, efficient THz SC-FDMA communications require a fast and reliable Beam Alignment (BA) method for both LOS and Non-Line-Of-Sight (NLOS) scenarios. In this paper, we first adopt the hierarchical discrete Fourier transform codebook for LOS BA, and introduce the hierarchical k-means codebook for NLOS BA to improve the beamforming gain. Simulation results illustrate that the hierarchical DFT codebook and the hierarchical k-means codebook can achieve the beamforming gain close to that of the maximum ratio transmission in LOS and NLOS cases, respectively. Based on these two codebooks, we propose a Multi-Armed Bandit (MAB) algorithm named Hierarchical Beam Alignment (HBA) for single-user SC-FDMA THz systems to reduce the BA latency. HBA utilizes a hierarchical structure in the adopted codebook and prior knowledge regarding the noise power to speed up the BA process. Both theoretical analysis and simulation results indicate that the proposed BA method converges to the optimal beam with high probability for both the hierarchical DFT codebook and the hierarchical k-means codebook in the LOS and NLOS scenarios, respectively. The latency introduced by HBA is significantly lower when compared to an exhaustive search method and other MAB-based methods.