I always have for the past 3 years of building PC's and never once had a problem. I just make sure I am careful with what I am doing and not wearing socks. Other then that, i normally don't move around much and just have a hand on the case.
I had to get a couple PC's to a client ASAP one time so I had a friend drive me over there in his mini van and I assembled both Pc's in the back seat of the van at highway speeds. got to the site, and both booted up no problem. we had pre_loaded Hdd's for these systems. was fun. cept for I kept dropping the gdamm screws all over the place.
If your worried about static, a metal desk is good, next to a desk covered in conductive antistatic rubber surfacing (like with a professional electronics technician desk mat)
Other than that, if you use common sense, touching grounded metals or the chassis case before and while handling sensitive parts (all parts) any surface will work. Wood is only good if you don't care about it's finish, because you will scratch the wood.
I have done alot of builds on carpet as well without problem but its definetly not safe. I find that doing it on the bathroom floor (tiled that is) is fairly convienient if you dont have a big open area.