Thats everybodies first concern with watercooling. Its very safe if you take your time and do it right. Of course it is never 100% safe but like I said just take your time and do it right. Make sure you leak test your new setup 24 hours before actually using it.
Watercooling is DANGEROUS! The water could boil, causing the water to pour out, which will cause sparks to fly out the back of your PSU, with will cause the curtains behind it to ignite, which will burn your house down, kill your dog, destroy your pickup truck, make your wife divorce you, which will then cause you to write REALLY BAD country/western songs.
PLEASE DON'T WATERCOOL!
(I REALLY should be in bed right now.
Lots of people watercool, so like what was said above, take yer time, do things right, and use QUALITY components. And don't hessitate to ask questions (And READ what others have posted & the Faqs!) then you should be OK.