I was just reading the Bug reports for Enterprise Manager and realized I'd run into this bug before. "Bug 27076505 : AgentPull.sh script gives 'handshake has no peer error after successful install" has a very simple fix. The issue is that the new agent doesn't have two things:
1. The latest JDK with the strong ciphers required by the OMS. I think the minimum for those ciphers is JDK1.7.0_171. You can follow Doc ID 2241373.1
2. The config change to update the SSLCipher line on the Agent.
After the agentpull script runs, the user needs to stop the agent then update the jdk in /agent18.104.22.168.0/oracle_common/jdk. I use scp to copy the files over from a server. This can also be scripted with a second bash script.
Then the user need to run the following commands before starting the agent:
The following checks what cipher suite is currently set up:
./emctl getproperty agent -name minimumTLSVersion SSLCipherSuites
The following command change the properties to reflect the new setting:
./emctl setproperty agent -name SSLCipherSuites -value TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
./emctl setproperty agent -name minimumTLSVersion -value TLSv1.2
Then start the agent:
./emctl start agent
The agent should connect properly. If necessary, resecure the agent with the additional option: -protocol TLSv1.2