If you are looking to spend a lot less than a GTX670, I run a MSI GeForce GTX 470 Twin Frozr, I edit mostly DSLR footage and sometimes up to 4K, I never have any problems, even running multiple layers of chroma-key footage and After Effects projects in Premiere using the DynamicLink.
I do have 24GB of RAM which is a big help, especially using After Effects.