They aren't over priced and you cannot build a PC spec for spec with the same hardware and have it be cheaper. Apple just uses the highest end parts. An IPS support LCD screen (standard in the iMac 27") is near or over $1,000 for the LCD itself.
As for all-in-one systems, they have their place. At my work, there are lots of older buildings. These buildings have limited number of power drops, and some are on a historical protection list (meaning we can't modify the building). Lots of all-in-one machine only take 1 power plug, which is really nice.
We have iMacs, Gateway and Dell all-in-one computers at my work. I think the iMac is by far the best, but the Dell is not too shabby. Desk space is also a concern at some buildings, so all-in-one computers also excel in that regard.
Are they meant to compete with a desktop? In my opinion, no not really, I would consider them in their own category. So, these computers have their purpose and their merit over classic desktop computers.
When I buy a house I will probably put an all-in-one computer in my kitchen for streaming, web browsing and other applications. That or stash a micro tower some where and mount a LCD on the wall.