Diamond nanoparticles, known as nanodiamonds (NDs), possess several medically significant properties. Having a tailorable and easily accessible surface gives them great potential for use in sensing and imaging applications and as a component of cell growth scaffolds. In this work we investigate in vitro interactions of human osteoblast-like SAOS-2 cells with four different groups of NDs, namely high-pressure high-temperature (HPHT) NDs (diameter 18–210 nm, oxygen-terminated), photoluminescent HPHT NDs (diameter 40 nm, oxygen-terminated), detonation NDs (diameter 5 nm, H-terminated), and the same detonation NDs further oxidized by annealing at 450 °C. The influence of the NDs on cell viability and cell count was measured by the mitochondrial metabolic activity test and by counting cells with stained nuclei. The interaction of NDs with cells was monitored by phase contrast live-cell imaging in real time. For both types of oxygen-terminated HPHT NDs, the cell viability and the cell number remained almost the same for concentrations up to 100 µg/mL within the whole range of ND diameters tested. The uptake of hydrogen-terminated detonation NDs caused the viability and the cell number to decrease by 80–85%. The oxidation of the NDs hindered the decrease, but on day 7, a further decrease was observed. While the O-terminated NDs showed mechanical obstruction of cells by agglomerates preventing cell adhesion, migration and division, the H-terminated detonation NDs exhibited rapid penetration into the cells from the beginning of the cultivation period, and also rapid cell congestion and a rapid reduction in viability. These findings are discussed with reference to relevant properties of NDs such as surface chemical bonds, zeta potential and nanoparticle types.