Context-Aware Image Inpainting with Learned Semantic Priors

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets

Image restoration for irregular holes based on dual discrimination generation countermeasure network

In order to solve the problem that the global and local generated countermeasure network cannot inpaint the random irregular large holes, and to improve the standard convolution generator, which demonstrates the defects of color difference and blur, a network architecture of inpainting irregular large holes in an image based on double discrimination generation countermeasure network is proposed. Firstly, the image generator is a U-net architecture defined by partial convolution. The normalized partial convolution only completes the end-to-end mask update for the effective pixels. The skip link in U-net propagates the context information of the image to the higher resolution, and optimizes the training results of the model with the weighted loss function of reconstruction loss, perception loss and wind grid loss. Subsequently, the adversary loss function, the dual discrimination network including the synthetic discriminator and the global discriminator are trained separately to judge the consistency between the generated image and the real image. Finally, the weighted loss functions are trained together with generating network and double discrimination network to further enhance the detail and overall consistency of the inpainted area and make the inpainted results more natural. The simulation experiment is carried out on the Place 365 standard database. The subjective and objective experimental results show that the results of the proposed method has reasonable overall and detail semantic consistency than those of the existing methods when they are used to repair random, irregular and large-area holes. The proposed method effectively overcomes the defects of blurry details, color distortion and artifacts.

Algorithm for post-processing of tomography images to calculate the dimension-geometric features of porous structures

An algorithm for post-processing of the grayscale 3D computed tomography (CT) images of porous structures with the automatic selection of filtering parameters is proposed. The determination of parameters is carried out on a representative part of the image under analysis. A criterion for the search for optimal filtering parameters based on the count of "levitating stone" voxels is described. The stages of CT image filtering and its binarization are performed sequentially. Bilateral and anisotropic diffuse filtering is implemented; the Otsu method for unbalanced classes is chosen for binarization. Verification of the proposed algorithm was carried out on model data. To create model porous structures, we used our image generator, which implements the function of anisotropic porous structures generation. Results of the post-processing of real CT images containing noise and reconstruction artifacts by the proposed method are discussed.