I've never used nTune because I've never bought an nVidia chipset, so I can't really give you any advice or help using the program. The only overclocking I've ever done has been in the BIOS, and whenever I get to a clock that part of my hardware can't take it won't boot for my to get a blue screen or a program freeze, so I can't really say why it has done it. My guess would be that it has something to do with the drivers that nTune uses to change clock values, but that's just a guess.
I don't think an overclock should cause a blue screen; I would've thought that if you'd overclocked too high, your computer would have either frozen or restarted unexpectedly.