High-spatial-resolution images play an important role in land cover classification, and object-based image analysis (OBIA) presents a good method of processing high-spatial-resolution images. Segmentation, as the most important premise of OBIA, significantly affects the image classification and target recognition results. However, scale selection for image segmentation is difficult and complicated for OBIA. The main challenge in image segmentation is the selection of the optimal segmentation parameters and an algorithm that can effectively extract the image information. This paper presents an approach that can effectively select an optimal segmentation scale based on land object average areas. First, 20 different segmentation scales were used for image segmentation. Next, the classification and regression tree model (CART) was used for image classification based on 20 different segmentation results, where four types of features were calculated and used, including image spectral bands value, texture value, vegetation indices, and spatial feature indices, respectively. WorldView-3 images were used as the experimental data to verify the validity of the proposed method for the selection of the optimal segmentation scale parameter. In order to decide the effect of the segmentation scale on the object area level, the average areas of different land objects were estimated based on the classification results. Experiments based on the multi-scale segmentation scale testify to the validity of the land object’s average area-based method for the selection of optimal segmentation scale parameters. The study results indicated that segmentation scales are strongly correlated with an object’s average area, and thus, the optimal segmentation scale of every land object can be obtained. In this regard, we conclude that the area-based segmentation scale selection method is suitable to determine optimal segmentation parameters for different land objects. We hope the segmentation scale selection method used in this study can be further extended and used for different image segmentation algorithms.