Sunday, September 28, 2008

The Drunkard's Walk...How Randomness Rules Our Lives..

My daughter, Raena (who recently graduated with a degree in Mathematical Science from Edinburgh), on noting that I have read and blogged on the book "Fooled by Randomness" a few months ago, bought me this book in Scotland when I went there for her university graduation in July.

The author , Leonard Mlodinow Ph. D , is a faculty member at the California Institute of Technology (Caltech).

I read this book during the 12-hour flight home from Heathrow to Kuala Lumpur and read it again recently and found it to still be equally interesting.


The greatest challenge in understanding the role of randomness in life is that although the basic principles of 'randomness' arise from everyday logic - but many of the consequences that follow these principles ( of randomness) prove counter-intuitive.

As an example -in behaviour modification study - the conventional wisdom is that a positive reward will reinforce a positive behaviour and punishment of a mistake does not improve the behaviour.In psychology labs, experiments with animals prove that rewards work better than punishment.

But in observation of trainee pilots of the Israeli Air Force, the instructors always reported that everytime a pilot was praised for a good landing, they did worse the next time. On the other hand the pilot who did poorly and was being scolded always improved the next time. So the instructors observation and conclusion is a challenge to the basic theory of behaviour modification theory.

Daniel Kahneman is a behavioural psychologist and a junior professor at the Hebrew University. He agreed that the instructors observations were correct and went about to understand and explain this apparent paradox. He then realised that it was true that the scolding did preceed the improvement, but it did not cause it!

The answer lies in the phenomenon called 'regression towards the mean'. That is, in a series of random events, an extraordinary event (like a good near perfect landing) is most likely to be followed, due purely to chance, by a more ordinary one.

Here is how it works. Each pilot has a certain personal ability to fly the fighter plane. Their skill will slowly improve through training but the change would not be noticeable from one maneuver to another. Any exceptionally good or bad performance was purely a matter of luck. If a pilot did an exceptionally good landing today, one far above his normal level of performance, then the chances are high that he will perform closer to his norm ( which is worse) the next day. And if his instructor had praised him for the good landing, it would appear that the praise had done no good to him. Whereas if the pilot had done an exceptionally bad landing, the chances are great that he will perform closer to his norm ( which in this case will be better) the next day. So if the instructor were to scold him for the bad landing, it would appear as if the scolding had done him good.

The instructors had made the conclusion that the scolding (punishment) was a powerful educational tool when in reality it made no difference at all.This error in intuition spurred Kahneman's thinking. Are such misconceptions universal? Kahneman's work and research in this area of 'randomness' won him the Nobel Prize in Economics in 2002.

This book went about introducing theories and concepts in Mathematics and Statistics through a historical narration of the lives and works of important contributors of these fields of science. The life or Cardano,the doctor from Milan who became a mathematician who wrote the first book on the "Game of Chance" and the lives and works of Galileo, Pascal, Bernoulli, Bayes and many others made very interesting reading.

This book would be of interest to doctors (like me) to understand how our interpretation of false positive or false negative measurement of tests or treatment outcomes can be flawed due to our poor understanding of statistical concepts.

Similarly, lawyers would find it interesting in the author's discussion of the "prosecutor's fallacy" and how the Harvard Professor of Law had used this statistical (flaw) to argue and convince the jury to favour O. J. Simpson in the infamous murder case!

How I wish my statistics teacher at university during my undergraduate and postgraduate years had taught me this subject in such an interesting way...

Thursday, September 18, 2008

Super Crunchers

If you have read 'Freakonomics', a book I had blogged about a few months ago, you will also find this book equally interesting.

Ian Ayres, the author of Super Crunchers, is a Professor both in the Law School and the School of Management at Yale University.

This is a book about 'Statistics', about how we can use statistics to extract hidden information from large data sets.

In general the book is about Random Theory, Randomised Trials and Regression Tests to make predictions on outcome to our decision-making process. Although these are 'big words' ( especially for people like me who is neither a mathematician nor a stastitician), the author has made the book so readable and interesting. ( and, mind you, the author himself is not a mathematician but a lawyer)

Find out how ( with the advent of evidence-based medicine) these statistical tests have challenged the accuracy of clinical decision based from our experience and intuition. In fact, the ideas for the scripts of the famous TV series "House" came out of the debate whether evidence-based medicine is superior to traditional clinical decision making process!

Find out how a mathematical formula can predict, years in advance, which wines will fetch higher prices at the auction, even before the wine gurus like Robert Parker will have his first chance to sniff, gaze and sip the vintages!

I wish some one with 'numbers-crunching skills' can look at the study conducted by UPSI on the outcome of the "Teaching of Maths and Science in English" and tell me if their conclusions are valid!

On another note, Chapter 7 of the book drew my attention to the debate on teaching methodology. 'Project Follow Through' initiated by Lyndon Johnson in 1967, looked at the impact of 17 teaching methods and the results from this 'super crunching' exercise showed that the teaching method of "Direct Instruction (DI)" produced best scores in reading, maths, spelling and language. And not only that, the kids also did better when they were asked questions that require higher-order thinking!

This chapter gave a clear decription of what the class using the DI method is like which you may also view in Micheal Moore's movie ' Fahrenheit 9/11' in the class-room scene where Mr Bush was being told that 'America is under attack!'

I now wonder if the success of Chinese medium schools in Malaysia in their teaching of Mathematics is because they have incorporated in their method elements of 'Direct Instruction'.

To Senor Zaid....Salute!

In happier times.....

Wednesday, September 17, 2008

Stampede in Java..

21 ladies died in this stampede in Central Java in their scramble to get 30,000 rupiah of zakat fitrah (tithes) each, given away by a rich businessman.

And these girls get 100,000 rupiah each in tips for keeping us company for five hours, carrying our clubs, showing the line and cleaning our 'balls'.

Selamat berpuasa..