The high-precision characterization of the intra-pixel sensitivity (IPS) for infrared focal plane array (FPA) photodetector is of great significance to high-precision photometry and astrometry in astronomy, as well as target tracking in under-sampled remote sensing images. The discrete sub-pixel response (DSPR) model and fill factor model have been used for IPS characterization in some studies. However, these models are incomplete and lack the description of physical process of charge diffusion and capacitance coupling, leading to the inaccuracy of IPS characterization. In this paper, we propose an improved IPS characterization method based on the diffusion and coupling physical (DCP) model for infrared FPA photodetector, which considering the processes of generation and collection of the charge, can improve the accuracy of IPS characterization. The IPS model can be obtained by convolving the ideal rectangular response function with the charge diffusion function and the capacitive coupling function. Then, the IPS model is convolved with the beam spot profile to obtain the beam spot scanning response model. Finally, we calculate the parameters of IPS by fitting the beam spot scanning response map with the proposed DCP model based on the Trust-Region-Reflective algorithm. Simulated results show that when using a 3 μm beam spot to scan, the error of IPS characterization based on DCP model is 0.63%, which is better than that of DSPR model’s 3.70%. Experimental results show that the fitting error of the beam spot scan response model based on DCP model is 4.29%, which is better than that of DSPR model’s 8.31%.