Samuel Arbesman at Long Now

Samuel Arbesman gave a talk on his new book “The Half LIfe of Facts” at the Long Now museum tonight.  This is a good venue to hear authors speak.  It is quite intimate and there are generally plenty of good questions and discussions afterward.  Arbesman did a good job of fielding comments from the  group.  You can see his presentation style in this short video.

His thesis is that there is an order and regularity  to the way knowledge changes.  He thinks that studying this can help us order the knowledge around us.  Because of this change, we should expect some portion of the facts we take for granted now to be overturned.  He points out that doctors are taught in medical school to expect their field to change and journals such as UpToDate provide this service to help doctors keep track of changes in medical knowledge. (My personal experience makes me skeptical that many doctors actually take advantage of this sort of thing.)

Arbesman takes the position that we would all benefit from this approach to learning.   We should learn how to think and how to understand the world but treat education as a continuing process.  Which is something I tried to touch on before.  Arbesman did comment that it’s better to rely on Google for current information than memorize a bunch of facts that may or may not continue to be true.  This takes me back to the ideas of Madeline Levine who I’ve mentioned before.  She argues that children should do less homework and more play because that builds creativity and problem solving skills.

Another point Arbesman brought up was that there is so much knowledge now, that many correlations can be discovered by mining the existing literature and joining together papers that each solve some fraction of a problem.  A speaker at the CogSci conference in 2010 at UC Berkeley mentioned that many answers probably go unnoticed and uncorrelated in the literature.  One effort to start detecting these hidden relations in the bioinformatics field is CoPub project which is a text mining tool developed by Dutch academics and researchers.  Theory[Mine] does an amusing take on this idea by letting users purchase a personalized, AI derived, unique, and interesting theorem.

Arbesman also suggested that facts in the hard sciences are subject to longer half-lives than facts in biology and the half-life decreases even further for the humanities and medicine.  He mentioned that when physicists colonize other fields they are unpopular and create disruption, but that they bring in useful ideas.  But I wonder if it’s even theoretically possible to reduce sociology to physics.  This is the whole holism/reductionism dichotomy that Monica Anderson loves to explore.

Another point that came up was that while the idea of fact decay should encourage healthy skepticism, we should still try to avoid unhealthy skepticism.  During the question and answer session it was suggested that politically controversial topics such as evolution, global warming, and even GMO labelling are clouded with incorrect facts.  I think a lot of scientists get a little overly defensive by what they term as anti-science policy decisions and they might be incorrectly grouping GMO opponents in with the creationists and global warming denialists.  Hopefully, better understanding of fact decay will radiate out and attenuate some of the scientific hubris out there.

Scientific leaps of faith, how much can we trust science?

I’ve been thinking a lot lately about problems with science.  These  epistemological questions come to mind:

  • What is knowledge?
  • How is knowledge acquired?
  • To what extent is it possible for a given subject or entity to be known?

So it’s a huge elephant of a field, but I will nibble at it one bite at a time.

I should point out that I make my living as a systems engineer, so I like to distinguish between  theory and practice.  I tend to think of science as being more theoretical and engineering as being practical.  Also, I am sympathetic to the constructivists, so I tend to agree with Feynman’s “If I Can’t Build it, I Don’t Understand it.”

This question of how much faith we can put in science initially came to mind when we were asking how much we can trust medicine.  A family member had cancer and we started researching the best treatment options.  The first problem we encountered was that medicine can’t keep up with science.  i.e. doctors can’t keep up with the deluge of journal articles.  But the next problem is more germane to this discussion, namely conflict of interest.

Corporate funding of research in agriculture for example has surpassed government funding since the 1990’s.  One can certainly see why corporations would want to control the research in their respective fields given the importance of science in determining public policy. Manipulation of research can be very subtle.  It spans the range of selectively funding research that supports industry interests to setting the actual scientific standard to favor industry.  (i.e. determining methodology and setting thresholds, etc.)  I would love to hear how my libertarian friends would respond to this.  Fire away, I am moving on.

Another point worth mentioning is related to Quantified Self and self-knowledge.  I had a conversation last year with a big QS guy who was pondering the relationship between self-tracking and science. In one sense, findings from this n=1 self tracking cannot be generalized to larger populations.   But in another sense you can learn things that matter to yourself which might never be extracted from the n=many findings which average out all individual differences.  (That said, huge amounts of this n=1 data is being aggregated by sites like MedHelp and used in research.)

But my main point from the previous paragraph is that there is a lot to be learned about ourselves that is currently innaccessible to science.  Meditation might be a good example.  Science can tell you the health outcomes of meditators or measure the brain wave frequencies of meditators.  But, it can’t reveal your own individual thought patterns to yourself the way mindfulness training might.  So the knowledge provided by science is in many ways incomplete.

An anti-atheist rant I saw on a friend’s Facebook page went on about how science constrains your world view to a box and included some nice Max Planck quotes:

“Anybody who has been seriously engaged in scientific work of any kind realizes that over the entrance to the gates of the temple of science are written the words: ‘Ye must have faith.'”
– Max Planck

It’s just nice to be reminded of that, but I don’t want to get into it too much.  I’d rather place faith in a bunch of bickering scientists than a shaman on peyote or a pedophilic priest.  But one other point that was brought up was that scientific consensus does change over time.  And this is a good point.  We should temper the confidence we have in current findings.  Entire scientific paradigms have been routinely discarded throughout history. (Some are more stable to be sure, check out Egyptian medical procedures circa 1600 BC:  examination, diagnosis, treatment and prognosis.  Pretty decent.)

But then we get into some of the current problems with how research is generated today.  Publication bias is partly (wholly?) a problem where positive results are more likely to be published than negative results.  This paper even asserts:

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

That’s nice.  I feel so much better about science now.

Next we come to the issue of reproducibility.  Apparantly a lot of studies are never reproduced and can’t be replicated outside the author’s lab.  It might be up to the private sector to separate the wheat from the chaff.  And according to a Bayer study from last year, they are finding plenty of chaff.  It’s easy to see that no one gets ahead in academia by replicating someone else’s work, so there are some incentive problems around that.  And who would even  publish replicated results apart from PLoS? (Serious, I am asking.  Post links in the comment section.)

Now, I do want to point out that scientists are sort of aware of this problem and there is plenty of work going on to identify bad research.   However, from arguments on Prop 87 to drawing distinctions between holism and reductionism the way Monica Anderson likes to do, I’m tending to downgrade my confidence in science lately.   But then again, my initial value was probably pretty high.  It’s not like I am going to go on a vision quest the next time I need to get new knowledge about the universe.