I had no idea that it was even a possibility. And I have to agree with the above posters comment on Alexa. There is no rhyme or reason to how they come up with what they come up with – its simply crazy!

I agree that it’s impossible to totally escape bias. This is true for everything– even measuring physical systems. (My background is mechanical engineering, and my phd involved turbulence measurements. Even there, we had to think about bias in instruments.)

The problem is that when I look at some of these traffic measuring systems, the bias error is larger than they signal they are measuring.

Alexa is so astonishingly horrible it’s difficult to imagine anyone uses it. I swear, I think pulling ranking values out of a hat would give results that are as good as Alexa’s on average.

Nielsen’s system appears to have some hope of delivering half-way decent data. That is to say: They may give results that beat pulling numbers out of a hat.

From a research perspective, there is always a sample bias no matter how carefully you select your sample group. If you’re too careful you will have added to the bias itself. As you pointed out, one of the most obvious biases will be that people will object to having their habits monitored and won’t sign up.

But there is also a law that says that researchers and observers will always irrevocably affect the subjects which they are studying by simply studying it. One of the obvious results of this is that people will alter their habits knowing that they are being studied.

Ah sorry, I’ve rambled quite a bit here.