Oh I agree, Firefox is not the Holy Grail that some people claim that it is. Firefox has had several serious vulnerabilities just like IE has. That said though, Firefox is generally more secure mainly due to policy. For example, no ActiveX (by default anyway), no default rendering of esoteric MS technologies like WMF and VML (both of which have had vulnerabilities), and so on.
I would say quicker patching too, but MS has been reasonably quick on the VML vulnerability so I won't hold that against them.
I don't consider myself a fanboy nor a rebel though, I just like Firefox for the various features that aren't in IE: Adblock Plus, JavaScript whitelisting, drop-down RSS feeds. That said, I still use Links and Dillo too.
On the technology site digg, 76% of people there are using Firefox, so the type of site does affect browser distribution it seems.