A vast majority of angiosperms are pollinated by animals, and a decline in the number and diversity of insects often affects plant reproduction through pollen limitation. This phenomenon may be particularly severe in rare plant species, whose populations are shrinking. Here, we examined the variability in factors shaping reproductive success and pollen limitation in red-listed Polemonium caeruleum L. During a 5-year study in several populations of P. caeruleum (7–15, depending on year), we assessed the degree of pollen limitation based on differences in seed set between open-pollinated (control) and hand-pollinated flowers. We analysed the effects of flower visitors, population size, and meteorological data on plant reproductive success and pollen limitation. Our study showed that pollen limitation rarely affected P. caeruleum populations, and was present mainly in small populations. Pollen limitation index was negatively affected by the size of population, visitation frequency of all insects, and when considering the visitation frequency of individual groups, also by honeybee visits. Seed production in control treatment was positively influenced by the population size, average monthly precipitation in June and visits of hoverflies, while visits of honeybees, average monthly temperature in September, and average monthly precipitation in August influenced seed production negatively. As generalist plant P. caeruleum can be pollinated by diverse insect groups, however, in small populations their main visitors, the honeybees and bumblebees, may be less attracted, eventually leading to the disappearance of these populations. In pollination of P. caeruleum managed honeybees may play a dual role: while they are the most frequent and efficient flower visitors, their presence decreases seed set in open-pollinated flowers, which is most probably related to efficient pollen collection by these insects.