Safe? It can be. If you know what you are doing and have some knowledge going into it you will be better off.
Idle temperatures are not what you have to worry about. You need to be concerned with what the core temps are going to run under full load. You will also need to be concerned with what type of cooling system you have on your CPU. If you are running the stock (factory) heat sink/fan combo, you can still overclock, but your maximum speeds and stability will be affected.
Be prepared to crash a lot while experimenting. If you are worried at all about your data, back up your hard drive first before experimenting. It is a good idea in any case.