OK rsyncing the files dir was making everything else on this box miserable, I'll just download them locally and push them up onto the other VM through scp.  That's also a miserable way to do it but it shouldn't make the sites run like shit.