the performance issues may also be related to the speed of your network connection. Do you connect to the remote system via a internal network or via internet connection? If you connect via internet, whats the maximum speed for uploads?
16 GB RAM is ofcourse better then only using a VM with only 8 GB of memory, but using Windows 10 native with 4 GB RAM and aditionaly a VM with also 4 GB of RAM should also be suficient. Does your machine have a SSD or a older harddisk which is not that fast?
Does the Windows in your VM react quickly and is only the remote access slow? If yes, the local hardware is not the problem.