Change AdminServer Listen Port makes Managed Servers not reachable

 In order to be able to recreate our WebLogic domain and switch domains without an outage to our application we are running mutiple domains in parallel, shutting down half of the original domain (the first 3 managed servers) then starting them on the second domain then doing the same with the second half of managed servers. We then stop the original domains Admin Server and change the new domains Admin server to listen on the same port.

This all happens and everything looks find but then a couple of hours later we notice that our Managed Servers go into a Not Reachable state.