How Scientific Journalism is Failing Readers & Scientists
Update 31 October 2023:
Philip Ball, the author of the first self-promotion exercise of Assembly Theory (AT) published in Quanta and Wired (see original post below to understand this update), has somehow become a defender and sort of spokesman of Assembly Theory interpreting the author’s intentions even when he declared himself neutral or agnostic about AT after writing the article that started the Assembly Theory marketing scam.
According to Ball, the authors of AT did not mean Darwinian evolution in their bold unfounded new pseudo-scientific claims, when they claimed AT explains natural selection and life (also) in the Darwinian sense. Yet, to justify how someone could have defended AT in the first place to help the authors of AT in their publicity stunt, some writers seem to be ready to find any possible way out instead of simply accepting their mistakes.
Here, one of the senior authors of AT, Sara Walker, explicitly said they mean Darwinian evolution.
Now that the bomb has exploded exhibiting the authors of AT not far from being seen as charlatans, Philip Ball doubles down in defense of AT even if accepting it has all been a masterclass of pseudo-scientific marketing as, I suppose, a defense of his own journalistic choices in helping Cronin and Walker to mislead the public and manipulate the real value of this (pseudo-) scientific contribution.
It should not be about the boldest, the loudest, the cockiest
The topic of broken scientific journalism has become more relevant in light of the new scandal around an article signed by dozens of experts in cognition claiming that Tononi’s Integrated Information Theory (IIT) is pseudoscience. Very much in play in this affair is the way the media have made exaggerated claims, enabled by the scientists concerned. While I think there is some merit to IIT as an interesting framework comparable to other related statistical tools such as transfer entropy, I do think the claims relating IIT to consciousness are flawed and wildly exaggerated, especially by the scientific media. However, IIT has been promoted and boosted with the complicity of its authors. I do not allege a conspiracy; I do not see journalists and certain scientists teaming up to deceive readers. These things happen organically, but nevertheless, the IIT controversy draws attention to what I see as an unfortunate corruption of the principles of objectivity and neutrality in scientific journalism. The case of Assembly Theory (AT) is similar.
Both IIT and AT are characterised by extreme fake rigour, with a lot of mathematical effort devoted to wildly speculative and limited theories that attempt to characterise consciousness and life, respectively. And in both cases, authors and journalists have stirred up a media frenzy. The criticism set forth in the aforementioned letter signed by 124 scholars was actually ignited by the unmerited attention that IIT has benefitted from thanks to about a decade of enthusiastic scientific journalism, which has now backfired.
Just as I confronted Cronin before his first-ever publication on Assembly Theory (in the Summer of 2016 at the FQxI 5th International Conference held in Banff, Canada), I publicly confronted Guilio Tononi, asking him how on earth he was able to publish his paper “A theoretically based index of consciousness independent of sensory processing and behavior” at all, let alone in Science Translational Medicine (in 2014), a publication that belongs to the Science family of journals, which one would presume has stricter peer-review processes than other publications.
Clearly, publishing in the world’s top journals Nature or Science is not a guarantee that the published content is of superior quality. On the contrary, seeking to maintain their high-impact status, these journals tend to select papers for their boldness rather than their soundness. I asked how it was possible to start off a paper describing a theory that was completely disconnected from its methodology, which they were then using as a validation of the theory. In other words, just as with AT, their theory did not justify their measure and vice versa, creating a vicious cycle that can only be seen as deeply dishonest.
Their IIT framework and the trivial methods chosen were completely disconnected, yet the authors were justifying IIT and claiming that it was capable of discerning degrees of consciousness. In their paper, the authors introduce their concept of integrated information multiple times, only to end up applying LZW (equivalent to GZIP), a trivial Shannon-entropy flavoured statistical lossless compression measure, to their EEG data in what they called a ‘PCI index’, allegedly proof that integrated information could quantify degrees of consciousness. This claim was advanced by Tononi and his group in multiple presentations and conferences.
The abuse of LZW as a measure, for the purpose of ‘approximating’ algorithmic complexity or other purposes, has been a decades-long problem. LZW is simply a Shannon entropy estimator, as I explained in this paper.
Tonini’s answer to my question, during the closed-door conference to which we were both invited as members of FQXi, did not make any sense, but he answered with such confidence, just like a ChatGPT hallucination, that not only did scientific journalists take him at face value, but even the most experienced scientists let him get away with it, including the reviewers for Science. It is as if the cockiest, boldest, and loudest in the room wins the scientific argument.
Assembly Theory as Pseudoscience: Another media fiasco like IIT
When it comes to Cronin and Walker’s Assembly Theory (AT), Philip Ball, the scientific journalist responsible for the Quanta article on Assembly Theory, reproduced verbatim in Wired magazine, has challenged my assumption (see below) that he was probably given the list of friends of Walker and Cronin to interview (by Walker or Cronin).
He is right, and I want to set the record straight. I could not have known this. I did, however, think it the most likely explanation for Ball’s strange (to put it nicely) choice of people to interview — that Cronin and Walker had suggested the names of possible interviewees, despite the presence of a clear conflict of interest. Ball wants me to consider the two other possibilities. He has ruled out one already, in a private exchange, namely that it was mere chance that his interviewees, Paul Davies, and Chiara Marletto, happened to be so close to one of the authors (presumably the other senior author failed to find anyone willing to give a favourable opinion of Assembly Theory). The only remaining possibility is that Ball made a conscious choice to pick friends and close collaborators of Cronin and Walker, without disclosing their relationships to the reader. In fact, he seemed to imply that he thought he was making a call on behalf of readers, picking who he thought was a valid voice to provide an objective opinion of AT. One of these individuals, it turned out, countered a highly watered-down version of my criticism, which should have been given more coverage in the first place — for the sake of balance and fairness, one of the basic principles of good journalism — though it wasn’t.
Indeed, my criticism, in our paper and my blog post, both of which I made available to him by email prior to the appearance of his Quanta article, was that Assembly Theory does not correspond to the authors’ methods, that it is a rehash of algorithmic complexity, albeit using a weak method involving a bad version of an algorithm introduced in the 60s. Moreover, I pointed out that the authors did not conduct the most basic control experiments or make comparisons to any other measure, and that comparison shows it to perform poorly, even underperforming measures introduced in the 1960s. Also that what they claim to have found was actually found five years prior, and that their claims are too bold and clearly false (viz. that they are able to characterise life). Ball boiled all this down to a single sentence which said that I “thought AT was yet another measure of complexity”.
Let me be clear. One of Ball’s interviewees has a direct management-like relationship with one of the authors (Walker). Paul Davies was Walker’s Ph.D. thesis supervisor as far as my many colleagues and I know; they have co-led one or two academic groups and basically have sat next to each other in the same office every day for the last 10 years or more. The other person interviewed by Ball is not only one of the closest friends of Walker but is part of the Assembly Theory sphere, her work having been heavily used or incorporated into Assembly Theory. So, in effect she was judging part of her own work, making it natural to assume a conflict of interest. If these would constitute red flags in general journalism, I do not see why they should not in scientific journalism, especially when the criticisms of AT — given little to no space — were ‘discredited’ by one of the interviewees.
Contrast that to, for example, the latest coverage of my work by the media, in Science News, by another experienced science writer, Matthew Huttson, who managed to get the opinion of the two top world experts in the field, who have never collaborated with me and do not know me, though I’ve met one of them at conferences. Being truly impartial, they were free to speak favourably or unfavourably of my work, and whatever they said can be taken as independent of me and thus justifying their readers’ trust.
The main problem is that by interviewing the friends of the authors of Assembly Theory without disclosing to the readers their relationship to the authors, the article in Quanta, replicated in Wired and other outlets, misled readers into believing that independent and impartial opinion was being aired. I do not think this represents best practice in journalism or scientific journalism, but I grant Philip Ball that I was wrong to assume he was fooled. He may indeed have chosen his sources carefully so as to make his article look more objective, to give this ‘theory’ more credibility, following the common but unfortunate trend in media outlets of all types to go for the boldest and most sensationalistic angle on an author’s idea, in this case, a mere weak hypothesis that I and other researchers think is not only wrong but is a quintessential example of self-promotion and, with its lack of control experiments and misattribution, an illustration of what science should not be about.
Ball also suggested that people he reached did not respond or did not want to go on the record to praise or challenge AT. While I cannot know what his reasons were for making such biased and, from my point of view, terrible and wrong choices, I can imagine he was sold on Assembly Theory by the cockiest, boldest, and loudest in the room, and wanted to give it as much credit as possible to justify the time he spent on his piece, despite the warning signs that something was amiss. This mirrors how the authors of AT double down to fight for their credibility, in another vicious cycle.
Quanta and other venues have made scientific journalism fairer by covering topics once considered obscure, like certain math topics. However, stories like the one on Assembly Theory in Quanta, Wired, New Scientist etc., and on consciousness and IIT in the same venues, which exaggerate their reach and make bold claims, in a self-feeding loop with authors, do a disservice to readers, to scientific journalism, to journalism, and to science. They fail everybody. In fact, they have failed the authors of both IIT and AT, leaving the former to face a huge backlash brought on by the hype of enamoured science writers who kept making IIT appear more and more grandiose to the point where the authors of IIT have now become victims of their own marketing success.
Unfortunately, scientific journalists make this mistake way too often, generating wave upon wave of hype around topics like consciousness/ IIT, DNA computing, the Big Bang, the ‘god’ particle (wrongly labeled and widely advertised by the media), the metaverse, the microbiome, or the latest one, Assembly Theory, as if no other scientific theories or results were worth reporting. Do not get me wrong, I respect some of the work of these scientists and science journalists, but the coverage that some research gets is out of proportion to its merit, makes scientists dishonest, and represents what is wrong in science and scientific journalism.
Of course, I do not know which is worse, including a short, misleading, and oversimplified sentence summarising critical views followed by a reply by one of the author’s friends, as in Quanta (and therefore Wired), or ignoring critical views altogether, as Thomas Lewton did in his New Scientist article. The bar seems too low and scientific journalism seems to be accountable to no one.
While I have been covered in some media, clearly criticising scientific journalists won’t do me any favours. Indeed, I may be making things worse for myself and my research. But I do so in the hope that we will start making science writers accountable.
UPDATE (Friday, 18 August 2023):
How science journalists are possibly misled by scientists: How the senior authors of Assembly Theory, Sara Walker and Leroy Cronin, misled Quanta scientific writer Philip Ball in order to get published on Quanta, and later on Wired (same article). All this without checks and balances or any real critical evaluation, in contravention of one of the most basic premises of good journalism.
Assembly Theory does not lack critics (even among the authors’ closest collaborators who prefer not to go on record but find not serious the idea that life can be characterised by the trivial number of exact repetitions of molecules), but the authors of Assembly Theory (AT) seem to have misled the science writer Philip Ball, whose writing I have enjoyed before, by providing names of friends and colleagues to back their theory. Walker and Cronin suggested their own friends and colleagues to defend their theory, and the science writer misrepresented my views despite having full access to this blog post up to the date of publication of the Quanta article which oversimplified my concerns. According to Ball, all the concerns in this Medium article amount to “AT is just another complexity measure” as I was apparently on record as saying about AT, when I clearly do not think this is the case. Then, the authors of AT decided who would best invalidate my (misrepresented) concern, a friend of Walker, and moreover, someone whose work is cited by the authors of Assembly Theory, an individual who was duly interviewed and proceeded to create a straw man argument setting forth her disagreement with me. This is so wrong from so many angles.
Just as we hold traditional journalism to high standards, we should demand better from scientific journalism, and misleading behaviour like this should not be promoted but condemned, as it undermines the mission of science: the search for objective truth.
According to Philip Ball on Twitter (now called X), he was agnostic/neutral. However, his article’s total praise for Assembly Theory, giving it the credibility it lacked in view of serious criticisms from me and other senior scientists — not referenced or incorporated in his article — was anything but neutral.
