Oracle Linux – High CPU Usage by Unknown Process Prevents Users to Login.

Applies to:

Linux OS – Version Oracle Linux 7.0 to Oracle Linux 9.0 [Release OL7 to OL9]

Linux x86_64 on Oracle Public Cloud

Goal:

Find out why users are not able to login.

Symptoms:

SSH stops responding after 5 to 10 minutes after reboot.

Upon reviewing Console Logs from OCI Console, lots of oom-killer messages are seen:

...
Aug  1 07:19:13 kernel: [  172.855081] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Aug  1 07:21:43 kernel: [  322.891888] systemd invoked oom-killer: gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), 
order=0, oom_score_adj=0
Aug  1 07:29:11 kernel: [  162.549796] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Aug  1 07:38:22 kernel: [  207.807035] apache2 invoked oom-killer: gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), 
order=0, oom_score_adj=0
Aug  1 07:43:17 kernel: [  162.809520] MetabaseSchedul invoked oom-killer: 
gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Aug  1 07:45:47 kernel: [  312.874640] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Aug  1 07:52:58 kernel: [  156.191079] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Aug  1 07:55:29 kernel: [  306.836192] apache2 invoked oom-killer: gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), 
order=0, oom_score_adj=0
Aug  1 07:55:36 kernel: [  313.808791] packagekitd invoked oom-killer: 
gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Aug  1 07:58:05 kernel: [  463.316664] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Aug  1 07:58:12 kernel: [  469.554607] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Aug  1 08:00:45 kernel: [  622.373585] kdevtmpfsi invoked oom-killer: 
gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
...