That's normal for Dell and will not void your warranty if they instructed you to take the cover off.
I have 4 Dell's in my office and 1 at home... none have stickers anywhere on the seal part of the case. There are the normal WindowsXP COA sticker and Service Tag sticker but they are not stuck across any break in the case.
If you go in the case yourself and mess something up then they may not warranty that but if you upgrade your harddrive, video card, memory, etc., they will probably never say a word if someone ever comes out to do warranty work. Obviously if the item you broke is something you installed then there would be no sense in calling on Dell to fix it.