You should avoid touching the gold contacts as well as the backside of cards for various reasons. One is corrosion caused by the oils from your fingers (the oils that cause fingerprints) but also there is the possibility of ESD (Electro Static Discharge) damage to the electronic components. ESD is essentially static electricity in your body that can discharge thru delicate electronic components and cause them to fail or weaken them so they fail early.