Yeah it makes a difference for sure, but if your overclocking your gonna need somewhere for that extra heat produced to go...look into an aftermarket cooler if your serious about over clockin...the stock one is noisy and is inadequate i found...
580 is decent on stock voltage, do a little reading about bios modding to bring your voltage up when under 3D and back to normal under 2D applications (it only puts it up 0.1V in 3D) it brought my original core max from 585 to 630, Nvidia put it there for vendors to use, some chose to and some didnt...also the 7900GT voltmods work with this card as well
with an after market cooler im sitting a 700Mhz on the core and 825Mhz (1650Mhz) on the memory...some have been able to volt mod the mem up to 900Mhz