They can only see the IP of the mail server that deliverd it to them, it would take a system admin or employee of that mail server system to then track it back to your IP.
Even if they did get your IP, its still ridiculously hard to trace an IP back to a street address, it has to be approved by your ISP and local authorities etc.... and by the soumds of it, this isnt very "high-scale" so i doubt anyone has tracked you down just yet.....