I believe the maximum allowed memory is 4GB,we have set 4GB for both Admin and Managed server. Suggest to check the performance by gradually increasing the memory size.
Please consider the following
1. Any other applications residing on the same hardware resources, if yes, what's the required memory for their smooth operation.
2. Please take a note of the current performance bottlenecks.
3. Check other causes for this performance issues.
4. How many applications are deployed and going to be deployed on this instance of weblogic server.