Chosen Solution

Is it possible to change the GPU from 512MB to 2GB? Update (09/18/2018) but maybe i can change the complete mainboard with the better cpu and gpu?

No. Even if you were to solder on more memory, you would still require a custom BIOS. I recommend an eGPU.

It may be possible to change the entire logic board in the iMac but that would be both expensive and impractical. As Jake suggested, an eGpu would be a better option if your iMac allows it.