Smartphone addiction does not seem like a questionable thing to believe in. It exists. Surely it does, doesn't it?
After all, there's a Wikipedia page on it. The New York Times writes about it. There are even claims it's killing us, that smartphones are destroying an entire generation.
But while nobody is saying we should be spending more time on our phones – or pretending these devices don't mess with our heads – perhaps smartphone addiction isn't what we think it is.
A new study led by psychologists in the UK suggests our fears of smartphone addiction aren't backed by solid evidence, simply because the most fundamental tool we have for measuring the extent of this seeming dependency is flawed.
"Despite a growing body of resources that can quantify smartphone use, research within psychology and social science overwhelmingly relies on self-reported assessments," the research team, led by first author David Ellis of Lancaster University, explains in a new paper.
According to the researchers, behavioural science has up until now almost exclusively focused on "asking people to consider their personal experience with a technology in order to better understand its impact" upon them.
Because of this broad trend in social psychology, self-reported assessments – via surveys, questionnaires, and interviews – are the linchpin of much research investigating smartphone usage.
But just how well do personal accounts of smartphone usage correlate with actual usage behaviours demonstrated by people? In other words, is the way we use our phones in real life the same as what we think it is?
Not really, the researchers say.
"Our results suggest that the majority of these self-report smartphone assessments perform poorly when attempting to predict real-world behaviour," explains one of the team, digital behaviour researcher Brittany Davidson from the University of Bath.
The problem, the team argues, is that the kinds of 'addiction' scales developed to measure smartphone and technology usage haven't yet been validated – at least not to the same rigorous extent as behavioural metrics used to gauge addiction in other, non-technological areas.
In their study, the researchers examined 10 of these smartphone addiction scales used to assess device usage or associated constructs (including things like attachment, problematic behaviour, and nomophobia – the fear of being without your phone, or unable to use it).
In an experiment, 238 iPhone-owning participants were asked how long they spent on their phones each day, as well as estimating the number of notifications they received, and how many times they picked their devices up in a day. They also completed the surveys related to the 10 smartphone addiction scales.
This data was then compared against data from Apple's Screen Time app – software installed by default on iPhones that automatically quantifies phone usage, time spent, notifications received, and so on.
Ultimately, while there were some correlations between the self-reported assessments and the objective behaviour, a lot of the time what people said about their smartphone usage (or its effects upon them) didn't match up with reality.
"Every psychometric scale correlated with at least one objective measure, but the strength of these relationships is far from convincing," the authors write.
"As these scales struggle to capture simple behaviours, it remains questionable as to how they could effectively measure habitual, atypical, and more complex behavioural patterns."
This disconnect is a significant problem, the researchers say, since a lot of existing research exploring the ties between smartphone usage and people's health and wellbeing rely upon these kinds of self-reports – metrics that are looking a little bit invalidated right now.
"These issues extend to research that has attempted to link a variety of individual differences (eg. personality) with technology use," the researchers explain.
"Errors of measurement here are so large that small effects detected in large-scale research involving estimates may be a component of statistical noise or a weak proxy for other psychological constructs."
One answer to this could be to keep refining the questionnaires we use to measure smartphone usage – the team noted that a more recent example, the Smartphone Usage Questionnaires (SUQ), showed promise, providing the strongest correlations in the study.
Another option could be making a "paradigm shift" away from reliance on self-report mechanisms for these kinds of measurements – something much aided by Apple and Google's new efforts to offer transparent analytics about usage directly to users, through apps like Screen Time.
There's a lot more stuff we still need to figure out, but it's clearer than ever that identifying smartphone addiction is not a simple diagnosis to make – but how scientists resolve these uncertainties could have a big impact on all of us.
"The issues highlighted here feed into a growing consensus that while psychology has acknowledged a problem with replication, the discipline also needs to address similar issues within measurement," the authors conclude.
"Across psychological science, many self-reports remain insufficient for researchers who continue to make large claims, particularly those which pertain to the impact of technology on public health."
The findings are reported in International Journal of Human-Computer Studies.