Compete ups stats: Y! up from 13.3% to 19%

Wow! I'm proud to see my finding a regex bug helped Compete correct its stats and set the record right for Yahoo!. As a result Yahoo! Search marketshare is now ~19%, which is up from the wrong percentage of 13.3%. Yahoo! surely needs some positive buzz at this point than any other. That too this one they have the right to.

I'm happy for my alma mater (aka ex-employer). At the same time, I'm mildly disappointed that neither Danny (@SearchEngineLand) nor Jeremy (@Compete) chose to acknowledge me by name or link to my post. Even other commenters have felt so. Glad that Webpronews managed to dig the link out from my comment at Danny's post.

Edit: Compete has linked to my post now. Max from Compete has commented below saying that they meant to give credit to me. Thanks guys.

Compete admits bug, says they'll recount

[Update: Compete ups stats: Y! up from 13.3% to 19%]

Steve from Compete has posted a comment confirming that their regex did have the bug pointed out by me. He, however, added that for capturing clicks on result pages, they use other regexes which don't have such a bug. He assured that they'll do a recount and update their numbers. I don't know how much difference it'll make to search engine market share numbers. But, any increase will do good for Yahoo!, I suppose.

Confirmed: Yahoo! does get screwed by a regex bug

[Update: Compete has confirmed that it was a bug.]
[Context: My previous post]
I had earlier suspected that Yahoo! Searches are severely undercounted by Compete because of a regex bug. Now, my suspicion has been confirmed. Danny Sullivan of the SearchEngineLand has confirmed that those regexes were indeed given by Compete.

By some estimates, refinement queries constitute over 70%? of the total queries. Is Compete, a widely quoted stats provider, undercounting Yahoo! Searches by 70%?OK, in a hurry, I might've dramatised the impact, but some significant chunk of searches are undercounted. Doesn't it amount to a huuuuuuuuge difference to Yahoo!'s market valuation?Well, I hear that other Stats are more widely quoted than Compete.

And when Larry Wall wondered whether a Perl bug could cause the next Wall Street crash, he must have been only half-joking!

Yahoo! getting screwed by a regex bug?

[Update: Yes, Compete says it was a bug.]
I hope not!

SearchEngineLand says Compete uses the following regex to count the number of Yahoo! searches:

^/search\?(.*&)?p=[^&]+

It actually excludes a vast number of searches that uses Yahoo! link tracking parameter.
For example, a search that I made from my Firefox searchbox has the following url:

http://search.yahoo.com/search?p=Perl&ei=UTF-8&fr=moz2

No issue here. But, when I refine the query by adding "language" in the search box in the results page, following is the url:

http://search.yahoo.com/search;_ylt=A0geu6Xv_FhIOyoBokNXNyoA
?p=Perl+language&y=Search&fr=moz2&ei=UTF-8

Anybody who has worked with regexes can see that this url and scores of similar urls of query refinement won't match the regex supposedly used by Compete and hence not counted! And, all this while, Yahoo! stock keeps taking a beat for failing in Search. If true, it could be a multimillion dollar bug.

Anybody from Yahoo!, Compete who can confirm or deny this?

Meta: If you're posting this, link back to this post, please.

Good bye, darling

Come December 15, I was about to shout Yaaahoooo! to announce my third year in Yahoo! But, today is my last day here, at least in this innings. I've been through several separations - in high school, college, previous employment, post graduation, and more, but this is the most painful of all. I don't say this simply because they're distant memory, but because I had always faced such events far more easily. Perhaps because they were all eventualities and this was by choice.

I'd earlier remarked to a friend that I've mastered the art of letting go - that is, by not thinking about it when you do. I tried doing that and was superficially fine, until I sent out that final email announcing that. My hands quivered as I composed the email and I was even more moved to the point of crying when I saw replies. As a colleague remarked, it's like the Marine Corps, once a Yahoo always so. Yes, it's a nation, not just a company to work for. I'll continue to evangelise Yahoo! (I've already converted one loyal Google search user into a Yahoo! search user) and recommend it for my friends, it's a fantastic place to work in.

While I'm sad for leaving Y!, it was an occasion to know how many friends I have and how much I mean to Yahoo! And, I'm looking forward to my next stop, Kneaver. More about that in another post.

Yaahooo!

The realisation was sudden when Vinoth asked me whether I've completed my second year in Yahoo! I'd completed precisely two years the day he asked. The two years were excellent and eventful.

On December 15, 2003, I joined Yahoo! as an intern. In the next few days was the year end party at "Club Cabana." I was pleasantly surprised that Jayanthi had arranged for my year-end gift - a music player - even though, they couldn't have planned one for me. After a full day of celebrations at Club Cabana, there were cultural events and I was merrily competing with the person seated next to me for the chicken pieces that were being passed around. Only when the awards ceremony came did I learn that this person was the CEO!

Soon after the chicken episode, I contracted chicken pox and was sick for over a month. When I returned, I was told that interns do not have the unlimited sick leave that employees do. I'd expected this a priori. But, two days later Jayanthi comes and tells me that they've decided to pay me the full stipend although I'd worked for merely 5 days or so!

Around the month of May, I'd developed a search-related hack (we don't have a 20% limitation :p) and was surprised to see the CEO coming down to my desk to see that. He offered several practical suggestions, I was looking at him wonderstruck. This is one of the biggest things that I feel extremely happy about - people are respected and every bit of their work is recognised. Soon after, on May 18, 2004, I was crossing the road and met Sridhar (then COO, now here) at the MG Road median. As we were waiting to cross the other lane, Sridhar asks casually if I'm taking up the offer from Yahoo! That was truly Yahoo!-style. I do not how many people got their job offers while crossing a road. I could only stammer "wh-y n-o-t?". On my request, they handed me the offer letter the next day and that became a perfect anniversary gift for my parents who were entering their 25th year together.

The next Monday, I became a full-time Yahoo! and was greeted with balloons tied to my cube walls and a basket full of chocolates at my desk. And ever since, it has been fun throughout. The organisation has grown more than three-fold here, but the basic culture has been organically maintained even while assimilating the increased diversity. The fun was all the more great when we worked on "cutting edge" technologies (literally) with hack saws and the drill-bit that our manager had brought for setting up a team stall for a recent internal event.

It's amazing to see how conversations that happen in the gym, lunchtable, or the corridor or rants vented out in random mailing lists result in newer products or features. An example would be how a lunchtable conversation with this "Big Guy" led him to staff a hack project. Also, having been in several teams, many of them horizontals, I'd had a chance to wow at the Life Engine that Yahoo! is.