Man versus machine? Self-reports versus algorithmic measurement of publications

This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare the quality of scientific publication data collected by surveys versus algorithmic approaches. We illustrate the different types of measurement errors in self-reported and machine-generated data by estimating how publication measures from the two approaches are related to career outcomes (e.g., salaries and faculty rankings). We find that the potential biases in the self-reports are smaller relative to the algorithmic data. Moreover, the errors in the two approaches are quite intuitive: the measurement errors in algorithmic data are mainly due to the accuracy of matching, which primarily depends on the frequency of names and the data that was available to make matches, while the noise in self reports increases over the career as researchers’ publication records become more complex, harder to recall, and less immediately relevant for career progress. At a methodological level, we show how the approaches can be evaluated using accepted statistical methods without gold standard data. We also provide guidance on how to use the new linked data.

Public Comment on the National Science Foundation’s Information Collection Request: 2019 Survey of Doctorate Recipients (FR Doc. 2018-17359)

We submit this public comment in response to the National Science Foundation’s proposed information collection request related to the 2019 Survey of Doctorate Recipients (SDR), published in the Federal Register at 83 FR 40340 on August 14, 2018. We outline the importance of including sexual orientation and gender identity (SO/GI) demographic measures on the SDR (and related NSF surveys) for advancing the U.S. scientific workforce, and the feasibility and precedent in implementing SO/GI measures in government surveys. The comment is cosigned by 17 scientific organizations and 251 scientists, engineers, and legal and policy scholars.

A supervised machine learning approach to trace doctorate recipients’ employment trajectories

Only scarce information is available on doctorate recipients’ career outcomes ( BuWiN, 2013 ). With the current information base, graduate students cannot make an informed decision on whether to start a doctorate or not ( Benderly, 2018 ; Blank et al., 2017 ). However, administrative labor market data, which could provide the necessary information, are incomplete in this respect. In this paper, we describe the record linkage of two data sets to close this information gap: data on doctorate recipients collected in the catalog of the German National Library (DNB), and the German labor market biographies (IEB) from the German Institute of Employment Research. We use a machine learning-based methodology, which (a) improves the record linkage of data sets without unique identifiers, and (b) evaluates the quality of the record linkage. The machine learning algorithms are trained on a synthetic training and evaluation data set. In an exemplary analysis, we compare the evolution of the employment status of female and male doctorate recipients in Germany.

Gendered Career Pathways among Doctoral Graduates in the United Kingdom

While women form about half of PhD students in Western countries, previous studies have shown that female doctoral graduates are underrepresented in senior positions and have lower earnings compared to their male counterparts within and outside academia. Less is known however about the role of gender in determining the odds of securing a permanent position among doctorate recipients. In this study, we use data from the UK Doctoral Impact and Career Tracking Survey from 2013 to explore the career trajectories of doctoral graduates within seven to nine years after earning their degree. We find that in every observed time point following graduation (0.5, 3.5, and 7–9 years), men are significantly more likely to work in a permanent job than women are. Furthermore, gender gaps in permanent employment are particularly pronounced in the private sector and in non-academic occupations. Using a nested logistic regression model, we find that the higher propensity of female doctoral graduates to work in part-time employment compared to their male counterparts, in combination with other differential employment characteristics has cumulative negative implications on their likelihood of securing a permanent position.

Intergenerational Mobility at the Top of the Educational Distribution

Research has shown that intergenerational mobility is higher among individuals with a college degree than among those with lower levels of schooling. However, mobility declines among graduate-degree holders. This finding questions the meritocratic power of higher education. Prior research has been hampered, however, by the small samples of advanced degree holders in representative surveys. Drawing on a large longitudinal dataset of PhD holders –the Survey of Doctorate Recipients– this study examines intergenerational mobility among the American educational elite, separately for men and women and different racial/ethnic groups. Results show substantial mobility among PhD holders. The association between parents’ education and adult children’s earnings is moderate among men and non-existent among women with doctoral degrees. However, women’s earnings converge to an average level that is much lower than men’s, signaling “perverse openness” for women even at the top of the educational distribution. Among men, there is variation in mobility by race and ethnicity. The intergenerational socioeconomic association is null for Asian men, small for white and black men, and more pronounced for Hispanics. Educational and occupational mediators account for intergenerational association among blacks and whites but not Hispanic men. A doctoral degree largely detaches individuals from their social origins in the United States but it does not eliminate all sources of inequality