Does Math Make Research “Better”?

Yes — if you don’t know much math, that is. A new study finds that even academic scholars perceived research to be of higher quality if there’s some math involved — even if the math makes no sense. The experiment threw an irrelevant mathematical equation into research paper abstracts, and asked scholars of different fields to evaluate the quality of the research:

Mathematics is a fundamental tool of research. Although potentially applicable in every discipline, the amount of training in mathematics that students typically receive varies greatly between different disciplines. In those disciplines where most researchers do not master mathematics, the use of mathematics may be held in too much awe. To demonstrate this I conducted an online experiment with 200 participants, all of which had experience of reading research reports and a postgraduate degree (in any subject). Participants were presented with the abstracts from two published papers (one in evolutionary anthropology and one in sociology). Based on these abstracts, participants were asked to judge the quality of the research. Either one or the other of the two abstracts was manipulated through the inclusion of an extra sentence taken from a completely unrelated paper and presenting an equation that made no sense in the context. The abstract that included the meaningless mathematics tended to be judged of higher quality. However, this “nonsense math effect” was not found among participants with degrees in mathematics, science, technology or medicine.

(HT: Mordechai Levy-Eichel)

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 13

View All Comments »
  1. Steve Nations says:

    The percentage of people who will agree with this post can be found by solving for the variable P in the equation P = (4/3)*pi*(number of years of post-secondary education) – square root of your GPA.

    Well-loved. Like or Dislike: Thumb up 22 Thumb down 1
    • Corban says:

      There is insight in mixing metaphors, and taking absurdities seriously. Children do it all the time, revealing inconvenient truths. (See: Contract law x Breakups. With such ill-defined remedies for breaches, no wonder…)

      If P is shell volume (Sphere 1 volume – Sphere 2 volume)…
      Given V=(4/3)?r³, years of bonus education is r³. Define r.

      Thumb up 1 Thumb down 0
  2. Gr4mb0 says:

    Perhaps when the readers see the equation and cannot make sense of it, they feel that the researcher has gone over their heads. This would convince the reader that the researcher has clearly done their homework. They do not stop to check the work because the researcher has already gained credibility in their minds by presenting something to complex to understand. I don’t have a clue, just my thoughts.

    Thumb up 3 Thumb down 0
  3. frankenduf says:

    this reminds me of on old mit story, where a lab requesting funding for a computer project brought in inspectors, who checked out the lab environs and the hardware, and subsequently denied funding- the following year, the lab reapplied for the same project, but installed a blinking light system on the side of the hardware- the inspectors came and this time granted the funding, apparantly because they could see the machine thinking…

    Well-loved. Like or Dislike: Thumb up 9 Thumb down 0
  4. Søren Have says:

    It is rather old news that the humanities and social sciences are tricked by papers with ‘hard science’ content:
    http://en.wikipedia.org/wiki/Sokal_affair

    Thumb up 2 Thumb down 0
  5. Dominik says:

    Well, as Kant already said: “Ich behaupte aber, daß in jeder besonderen Naturlehre nur so viel eigentliche Wissenschaft angetroffen werden könne, als darin Mathematik anzutreffen ist.” This is probably true for all sciences (and humanities – as my professor once said).

    Thumb up 2 Thumb down 0
    • Paul2901 says:

      He was a bit of a wit, wasn’t he old Emmanuel Kant? Must have been all the beer drinking and late nights on the town.

      Thumb up 0 Thumb down 0
    • Jim says:

      What does that Kant quote mean in English? You are not introducing an irrelevancy to baffle us lay people, are you?

      Thumb up 2 Thumb down 0
      • Eric says:

        Google Translate says:

        “But I maintain that in every special doctrine of nature only so much real science can be found, as there is mathematics encountered.”

        Thumb up 0 Thumb down 0
  6. Steve Nations says:

    I think this is a form of the “Dr. Fox Effect.”

    Thumb up 0 Thumb down 0
  7. Patricia Rogers says:

    Interesting study – but referring to the study participants as “academic scholars” is a bit misleading. They were Mturk volunteers, who perform a micro task for a 50 cent payment. Not likely to be active academic scholars. And the interdisciplinary differences were important – those with a mathematics background rated the paper with junk math lower.

    Thumb up 1 Thumb down 0
    • MW says:

      I don’t think this is a fair experiment. In real life, the abstract reflects the actual contents of the paper (or else the reviewers will complain.) The readers here were restricted to reading only the abstract, and being forced to make a judgement on the entire paper. They have every reason to believe that the mathematics will be in context and explained in the body of the paper.

      Thumb up 3 Thumb down 0
      • Eric says:

        I read the two abstracts and the inserted equation and could easily see how such an equation could be explained by a paper meeting the contract established in the abstract. For example, in the “Foraging” paper, one could imagine a game-theoretic perspective where an amount shared in an iterated game turns out to be governed by a simple quadratic equation. In the “Incarceration” paper, the meaning of “sequential effects” is harder to imagine, but maybe they are modeling repeated incarceration or repeated job application with the applicant becoming more discouraged, and thus less employable, each time. I wouldn’t know that the math was meaningless until I had read the paper.

        That said, I might have docked them a bit on quality of the abstract for including the equation since it was unnecessary.

        This may also represent differing customs among fields. In my field (which would be categorized as technical) I have never seen an equation in an abstract (all our math is in the main body). In other fields, the best papers may have simple equations like the one in the study included in the abstract.

        Thumb up 0 Thumb down 0