So the final follow up to the CAC-1085 adapter and not being able to do 4k 120hz with the latest Nvidia drivers. It turns out I had to contact club3d directly and apply a firmware update to my adapters. So with the new fix from Nvidia plus the firmware has fixed the issue (again).Thank you. Unfortunately the fix didn't work. Send screenshots and a dispdiag back to the Nvidia driver team for them to sort out.
Hopefully no more driver issues in the future!