You know all of the following, but hey you asked rhetorical questions.
1) The people I know who worked at Microsoft said that the antivirus was usually considered the big third rail of Monopoly during the 1990's. If they got into the business they would probably take out a lot of other players and some of them had big money attached with big lawyers.
Then when Microsoft tried again in the 2000's they found that the market had changed so that they were not able to make money off it in the way they had supposed they would. So that meant they had to carefully come up with a free product that would not get Monopoly extensions like the IE debacle had. [EG as much as it would be good to have it deep in the operating system that would probably get them in sooo much trouble.]
2) Because of the way that computers and users are. There is big money in either making computers into spam bots or getting their bank account numbers etc. Most problems that people encounter are not viruses even though its the name that people relate to. Mostly they run into keyloggers, spambots and programs that look and mail data. Most of which does not need administrative access to get into.
How can you make something invulnerable to a virus? I keep hearing people say this, but I have yet to see someone do it... as long as human beings are involved in the equation. Viruses, Worms, Malware mostly spread via social engineering and not vulnerabilities. Yes some use vulnerabilities to get root level access, but many just rely on the fact that the user is running in administrative mode (and the number of Linux people I see running as root when they surf the web is quite amazing.) And even then, many of them do not have to get into root mode because they just need to do 3-4 things:
1) open up a port above 1024. OR go to some website at port 443 to get their commands.
2) be able to look at the files of the user and sift for data that is wanted.
3) send out email to the internet.
In most cases the best stoppages are outbound firewalls that 90% of the people either turn off or have opened enough up that the problem is again that the user has made the malware authors job easy.
Very little of the stuff that anti-virus software protects people from is actually a virus or a worm (I think the estimate is somewhere between 4% -> 10% ). The majority is various malware that is usually self-inflicted. And no amount of patching can fix that.
I agree, no antivirus is going to stop Social Engineering attacks, for sure.
But why spend time any time on identifying virus signatures and building scanners for them, instead of patching operating system vulnerabilities? If I have to download something in response to a threat, shouldn't it be a security patch instead of a virus definition file?
Why are most Linux boxes, which have rich environments that I'm sure black hats would love to leverage, not crawling with worms and viruses? It's because security issues are patched in minutes-to-hours, and there's proactive security in place (mandatory access control, type enforcement, stack hardening, address randomization, admin privilege separation, and so forth). Even the horrible practice of logging in as root to surf (which is really user error, not a technical problem) can be discouraged by software design -- which is why GDM does not allow root login by default.
1) Most of the malware that are in the scanners does not use vulnerabilities. Most of it just uses the same priviledges the user already has. The stuff that gets the "OH MY GOD ITS ANOTHER VIRUS" does but its actually a small percentage of the actual malware out there.
2) I have seen how quickly a patched Linux system can be circumvented if the team of people attacking it want it. Most of the Linux malware is aimed at spear phishing where the target has been scoped out and known to not use the standard tools. In the end, its mainly due to the size of the market/money. Windows has 80% of the market so aim at it for the most profit. There is a reason why Apple knew it had hit the big time when it actually had malware that wasn't just token "see what I did?" variety stuff.
3) I didn't say that logging in as root was a technical issue. I would say that most of the problems with malware in the last 3 years are the fact that the targets get themselves into the problem.