Overloaded Domain - Options

Lets say we have a WL domain, and there are eight clusters, two managed servers each.

You find that when doing deployments, sometimes the activation times out.

You check memory use on the host, it's 99.9 percent.  This is a production environment.  Should you just start tuning down all the JVM heap sizes? What should percentage be for memory use in general for AIX server running WL?

This is a situation we face, and I'm wondering what other people have done.  We are currently now using a tool which gives us runtime statistics of heap usage etc, but we have had to do changes on the fly without real analysis.  These are inherited environments in which we didn't have full control from the start.