Two scientific papers in South Africa have raised questions among critics about the quality — and potential biases — of international peer review.
July 22, 2019 by Sarah Wild
As soon as Barbara Boswell began reading the journal article, the associate professor of English at the University of Cape Town (UCT) in South Africa says she was surprised about the language it used. Even the title and the abstract set alarm bells ringing, she recalled. “As I read further, I saw more problems.”
The controversial paper, “Age- and education-related effects on cognitive functioning in Colored South African women,” was published in March in the journal Aging, Neuropsychology, and Cognition following peer review. The authors, from South Africa’s Stellenbosch University, claimed to show “low cognitive functioning” in this group, which they attributed to low education levels and risky lifestyles. (In South Africa, “colored” is one of the four officially recognized racial categories — a relic of the apartheid system — along with white, Indian/Asian, and black African.)
In April, Boswell spearheaded a petition for the journal to issue a retraction. “The article is published as scientific research but draws on colonial stereotypes of African women, and ‘colored’ South African women specifically, as intellectually deficient,” Boswell and her co-authors wrote. “The article relies on flawed methodology and science, perpetuating harmful, racist stereotypes.”
More than 10,000 people, including scholars and ordinary citizens, signed the petition, which was ultimately successful: The journal retracted the paper on May 2. But this wasn’t the only recent scientific article in South Africa to face fierce criticism on its methodology and treatment of race. A second paper, also published in March in the Journal of Interdisciplinary Economics, claimed to show that inhabitants of countries with lower IQs were more likely to be sold as slaves between the 15th and 20th centuries. Following an outcry, the co-author resigned from his position as an adjunct professor at the UCT.
Together, the papers raise questions regarding how such research made it through peer review, a process in which academics validate studies prior to publication. Peer review is considered by many researchers and academics to be the best quality-check for scholarship, but others point out that it can be flawed, opaque, and susceptible to bias.
Both papers were subjected to reviewers in internationally-published journals despite appearing to dabble in race science, which regards race not as cultural construct, but as a biological variable that can be used to make allegedly scientific conclusions about groups of people. Many experts consider biological notions of race to be largely debunked, making the appearance of such research in the global literature, where it can then be used to undermine the rights and dignity of entire communities, particularly problematic. “Scientific racism was used to justify racist policies like apartheid,” says Boswell. “It was used to make an argument about the inferiority of black people, indigenous people, and why they needed stewardship because they were not fully capable of looking after themselves and the land.”
The two papers show “how shoddy peer review can be at times,” says Agustín Fuentes, an anthropologist at the University of Notre Dame. “The ideal is good — great, in fact — but it does not always work out. I think that there are also a lot of biases about race and gender in the academy in general. And in too many cases those biases go unchallenged and result in things like these getting in to print.”
It has been 25 years since the end of South Africa’s apartheid government, which separated people based on race and often relied on flawed race science as justification, and the country still struggles with racial tension and systemic divisions that drive inequality. The academic system reflects these realties: White researchers still occupy half of all university posts despite accounting for just 8 percent of the population, and they publish about two-thirds of academic research.
Stellenbosch University, for instance, was mainly reserved for white students and staff under the Afrikaans-speaking apartheid government. The school has been attempting to address its racist past and transform its university body by increasing scholarships to previously disadvantaged racial groups, hiring more diverse staff, and switching from Afrikaans to English as the main medium of instruction. Eugene Cloete, the vice-rector for research, innovation, and postgraduate studies at Stellenbosch, says that the paper on colored women has set the university “back years.”
Cloete suspects there might be other published articles from the university with racist assumptions, and he is personally reviewing thousands of ongoing projects for racial insensitivity. Still, he says, some blame should lie with the journals. The paper “was published in an international, peer-reviewed journal,” he says. “We publish 1,800 papers a year here through thousands of different journals. We have to rely on peer review.”
Cloete and Boswell, along with other researchers, argue that peer review should have caught what they say is flawed research in the Stellenbosch study. The study’s authors, a team of sports scientists, assessed self-identified colored women from a township in the Western Cape. The sample size was limited, with just 60 women, but they extrapolated the results to apply to millions of people. The researchers also made assumptions about the group, identifying it as racially homogenous when it was actually diverse. And, based on a measure of cognitive ability that has been shown to be inapplicable to South African populations, the researchers made sweeping claims about the poor cognitive abilities of colored women in general.
“The study is based on ideological assumptions that are deeply rooted in a racialized and racist history,” says Garth Stevens, president-elect of the Psychological Society of South Africa. “Those assumptions are overlaid with a set of scientific methods that are themselves fatally flawed.” As a result, the generalizations about a particular population group “become spurious and a real indicator of poor science.”
Corresponding author on the paper, sports scientist Elmarie Terblanche, said she was not allowed to comment as the matter was under investigation.
The academic publisher, Taylor & Francis Group, confirmed that the article was peer-reviewed, but that editors retracted it after Boswell’s petition took off. When Undark asked the organization for comment, press coordinator Saskia Kovandzich said “I’m afraid that nobody is available to discuss this issue with you.”
While the Stellenbosch article was retracted, the one on slavery and IQ was not. That article, “Intelligence and Slave Exports from Africa,” was published by a team of economists in the Journal of Interdisciplinary Economics by Sage Publishing on March 28. The team claims to show that African countries where people have higher IQs experienced lower levels of slave exports than countries which had lower “cognitive ability.”
The lead author, economist Simplice Asongu, listed UCT as his institution on the paper, but he was an adjunct professor rather than a full staff member, says Elijah Moholola, a university spokesperson. And the university doesn’t stand behind the findings, Moholola adds: “UCT rejects the assumptions of the paper and this line of research as bad science.” Asongu has since resigned.
Sage did not respond to an interview request.
Like the Stellenbosch study, the methodology of the UCT paper came under scientific scrutiny. The paper claims to prove that countries with higher average IQs saw fewer inhabitants sold into slavery because they were smarter and thus better able to escape, confront enslavers, and organize resistance.
Asongu and his co-author, Oasis Kodila-Tedika, an economist at the University of Kinshasa, show this through linking, among other variables, countries’ IQ; their capacity for technology adaptation, inferred from previous research; the landscape’s ruggedness; and historical population density.
The authors assume most types of intelligence can be captured through IQ tests. But the idea that it is possible to determine the cognitive ability of entire countries is problematic, says Adam Haupt, a professor in media studies, who specializes in race discourse. He points out that there is plenty of research showing IQ tests can be inaccurate and unfair. “There’s a cultural and ideological bias embedded in those tests,” Haupt says. “Science is seen as non-ideological, but we know that’s not true.”
When Undark contacted Asongu for comment, he said he wouldn’t discuss the matter through non-scientific media, adding: “Anybody questioning the robustness of the findings should have his or her comments peer-reviewed and published in a scientific medium, then I will also respond through the same scientific medium or other scientific media.”
But peer review is part of the problem. “If it was a predatory journal” — a journal which charges researchers to publish, but doesn’t offer rigorous services such as peer review — “then you’d understand it,” says Haupt. But “Sage is a reputable publisher. It has you asking questions about their peer review process. All of the supposed safeguards fell flat. Why did editors not ask how sound was this methodological approach? How much do we know about IQ?”
It remains unclear why, exactly, the papers from Stellenbosch and UCT made it through peer review. “A charitable interpretation would be laziness and genuine oversight on the part of the reviewers,” says Angela Saini, a science journalist and author of “Superior: The Return of Race Science,” a new book on the resurgence of race science since it fell out of favor following World War II.
“A less charitable one is that they let this through because they share with the authors some commitment to the idea of biological race — an idea long ago discredited by mainstream scientists,” she adds. “Either way, the system must be flawed in some way or this wouldn’t have happened.”
Regardless of the reason why, it’s common for faulty papers to slip through peer review, says Ivan Oransky, co-founder of Retraction Watch, a watchdog publication for scientific publishing. “There are 1,400 retractions per year, and there are others that should be retracted but aren’t,” he says. “Peer review is a porous system.”
Recent reports reveal that system is under pressure. A 2016 study in PlosOne, looking at biomedical research, found that the responsibility for peer review is concentrated in the hands of a few reviewers. At the same time, the volume of scholarship requiring peer review continues to increase at about 3 to 3.5 percent each year. And there is also bias when it comes to who gets to be a peer reviewer. In its Global State of Peer Review 2018 report, for instance, the peer-review tracking website Publons found that established regions review more than emerging regions; in fact, there was not an African country in the top 20 nations that supplied reviews. And an investigation into gender and international diversity at the biosciences journal eLife found that an all-male review team was more likely to accept papers with male authors, and gatekeepers were also more likely to accept papers whose authors were from the same country as them.
“Humans are fallible and peer review has subjective aspects to it,” explains Cassidy Sugimoto, a professor of informatics at Indiana University, Bloomington and a co-author on the paper.
Part of that subjectivity comes from personal worldviews, but it also encompasses the scholarship reviewers and researchers are exposed to. Editors tend to choose reviewers who have read the same body of literature, Sugimoto adds, and may be oblivious to valid work disproving their viewpoint. In the case of race, there is plenty of well-established scholarship, she says, but mostly in fields that are unfamiliar to researchers and reviewers.
“A number of disciplines outside of the humanities need to engage across those boundaries to think critically about what they do as researchers,” says Haupt. “What does it mean to be a scientist in a world that is trying to undo colonialism, systemic racism, sexism? How do you undo the systemic racism, sexism?”
“You need to interrogate your position and the history of your scholarship,” he adds.
Still, there are moves to change the system. One way is to have a more diverse pool of reviewers, Sugimoto says. Another is to have partially open peer review, where reviewers and authors know one another’s identity and their comments are public.
“If peer review is the mechanism to determine validity of work, open peer review would be accountability and transparency,” Sugimoto says, although she adds that this could spark other problems, such as junior reviewers damaging their careers by openly challenging a senior academic. One way to avoid this would be to make only the reviews, rather than the reviewers’ identities, public.
These fixes, perhaps, could have halted the publication of the papers about colored women or countries that experienced slavery. “I’m sure there are lots of pieces of research like this,” says Boswell.
Such work “doesn’t come out of nowhere,” she adds. “This comes out of a context.”
Sarah Wild is a freelance science journalist based in Johannesburg, South Africa.
This article was originally published on Undark. Read the original article.
This article is under Undark’s copyright and does not qualify for the Creative Commons license J2DW normally uses.
