US millennials are rejecting the suburbs and moving back to the city. That became a prevailing notion in 2019, when I started as a social science reporter Science News. But when I started mining the story of the phenomenon, I encountered a coherent mess. Some studies have shown that suburbs are growing, other suburbs are decreasing, and yet others have shown growth in both suburbs and cities.
I couldn’t make sense of that mistake, I ran away from the idea of the story. Then a few months later I came across a white paper at Harvard University explaining that there was disagreement on campus over the competing definitions that distinguish a city from a suburb. Some researchers define suburbs as areas that fall outside designated census states. Others look only at characteristics of suburbanism, such as the wealth of family homes and car rental exchanges, the researchers wrote.
I’ve come across this kind of frivolity around definitions of all sorts and concepts in the years I’ve covered the social sciences. Sometimes researchers simply assume that their definition of a concept is key the definition Or they briefly nod to other definitions, and then, whatever they want, proceed without much reason why. Sometimes, researchers in one subfield choose one definition, and researchers in another subfield choose another – each without ever knowing the other exists. It’s enough to drive any reporter to pull their hair out.
“If you look … you’ll find this quagmire of definitions and measurements” in the social sciences, says quantitative psychologist Jessica Flake of McGill University in Montreal. My experience was usual, this told me.
Definitional quagmires exist in other scientific fields as well. Biologists often disagree about how best to define the word “species” (SN: 11/1/17). Virologists argue about what “lives” when it becomes a virus (SN: 11/1/21). And not all astronomers are happy with the decision to define the word “planet” in a way that leaves Pluto out in the cold as a mere dwarf planet.SN: 8/24/21).
But the social sciences have some special challenges, Flake says. That field is compared to a discipline such as astronomy, the less time its definitions exist. And the ideas of the social sciences are often intrinsically subjective. Describing abstract ideas like motivation or emotions can be squishier than describing, say, meteorites.
It is tempting to assume, as I did until I began researching this column, that a single and incomplete definition of individual concepts is preferable to this cacophony of definitions. And some researchers encourage this approach. “Although no definition of suburbs will be perfect, standardization will increase understanding of how suburban studies relate to each other,” the Harvard researchers wrote in that suburbs paper.
But a recent study focusing on how to define the middle class showed me how alternative definitions can be derived in perspective.
While most researchers use income as a proxy for class, these researchers used purchasing models for people. This revealed the fraction of people who appear to be middle class due to income struggles to pay for basic needs such as housing, childcare and groceries, the team reported in July. Social Indicators Research. That is, they live as if they were proletarians.
What’s more, because the vulnerable group skews black and Hispanic, the disparity arises in part because these families of color often lack the generational wealth of white families, says Melissa Haller, a geographer at Binghamton University in New York. So, when disaster strikes, families without a financial cushion can struggle to recover. However, the government or non-profit organization looking to provide direct assistance to families in need, and based solely on income metrics, would overlook this vulnerable group.
“Depending on which definition you start with, you will see different facts,” says Anna Alexandrova, a philosopher of science at the University of Cambridge. A standardized definition of the middle class, for example, could obscure some key characteristics.
In the social sciences, what is needed instead of conceptual unity, Alexandrova says, is conceptual clarity.
Although social scientists disagree on how to go about solving this problem of clarity, Flake says the failure to address the issue is going to be a crisis, as well as other disciplinary crises (.SN: 8/27/18). This means that the theme is defined as how the scale, surveys and other tools to understand the use of that concept. And this in turn shapes how researchers crunch the numbers and reach their conclusions.
Defining keywords and then selecting the right tool is fairly straightforward, with emphasis on large and external scripts. For example, instead of using national income databases, as is common in studying the middle class, Haller and his team turned to the Federal Consumer Survey to understand people’s everyday and impulse purchases.
Often, however, social scientists, especially psychologists, develop their methods and surveys for subjective quantitative concepts such as self-esteem, mood, or well-being. The definitions of those terms — and the tools used to define them — can take on a life of their own, Flake said.
She and her team recently showed how this process plays out in May-June American Psychologist. Over 100 prototype studies were weighted and 100 replications included in high reproducibility in psychology. Researchers looked at 97 multi-item scales measuring concepts such as gratitude, motivation and self-esteem — used in prototypes — and found that 54 of those scales had no citations to show where the scales originated. The original authors defined their idea, and the instrument used to measure that idea, on the fly, Flake says. The research team then attempted to replicate 29 of those studies without extracting the sources of the scales, calling into question the significance of their results.
For Flake, the path to achieving conceptual clarity is straightforward, if unlikely. Researchers must hit targets to generate new ideas or replicate old ideas and question old ones.
It provides promising, if intensive work: an accelerator of psychological science, a collaboration of 844 researchers in over 1,300 countries. The purpose of the program is to understand important concepts in psychology, such as perception and gender bias, and to accumulate all the tools and data from them, so that the meaning of those ideas can be used to reject, refine or combine existing definitions and tools.
“For running replications, why don’t we use” [this] A huge team of researchers representing a lot of perspectives around the world and reviewing ideas for the first time,” Flake said. “We need to stop replicating garbage.”
I couldn’t agree more.
#definitions #dubious #problem #social #sciences