Yes, it is thermal paste. What is the state of it? If it is dried and crumbly, then you'll want to remove the old and replace with new. IF it is nice and greasy, then you can get away with reusing what is there.
If you want really good heat-transfer, though, then you might consider getting yourself a tube of Arctic Silver 5 and replacing the paste with that. It is one of the best heat-transfer compounds made and what most overclockers use to cool their systems.