This site is currently read-only as we are migrating to Oracle Forums for an improved community experience. You will not be able to initiate activity until January 31st, when you will be able to use this site as normal.

    Forum Stats

  • 3,890,393 Users
  • 2,269,776 Discussions
  • 7,916,824 Comments

Discussions

OL 7.5 kernel panic

MatthewLinux
MatthewLinux Member Posts: 80 Blue Ribbon
edited Apr 30, 2018 11:10AM in Oracle Linux

After we updated to Oracle Linux 7.5, the latest kernel would not boot. It panics with this:

kernel panic-not syncing: VFS: unable to mount root fs on unknown block(0,0)

I had to rebuild the initramfs and grub2 config in order to get it to boot properly. Anyone else notice this today?

Tagged:
MatthewLinux

Best Answer

  • Tommy.Reynolds-Oracle
    Tommy.Reynolds-Oracle Member Posts: 1,754 Employee
    edited Apr 19, 2018 3:12PM Answer ✓

    It turns out to be a known issue with the upgrade to 7.5, so you have done nothing wrong as far as I can tell.

    There is a strong separation between userspace (the usual suspects, libraries, applications, shells, X window, on and on) and the privileged kernel space.  In practical terms this means that most times it does not matter which kernel version you use with which version of userspace, mostly the C standard library.  You can update either independently.

    Not this time.

    The interface between userspace and the kernel has an incompatible change.  The userspace must be updated first so that when the kernel is updated all the right subroutines and all are ready and waiting in userspace.  There are firmware and microcode which must be updated so the devices operate the way the newly booted kernel expects, else the boot fails.

    Since there has not been an RPM version requirement on the kernel and userspace, the updater updates the packages in the wrong order.  The right order is to update userspace first and then update the kernel.  This is why the initrd was not rebuilt.

    If you have Oracle Linux support, file a service request (SR) ticket to be notified when a remedy is announced.

    Cheers,

    Tommy

    MatthewLinux
«1

Answers

  • Tommy.Reynolds-Oracle
    Tommy.Reynolds-Oracle Member Posts: 1,754 Employee
    edited Apr 18, 2018 3:36PM

    Good catch for rebuilding the initrd so the kernel could find its root filesystem. Does your setup differ from that of, say, a desktop computer?

    Perchance do have a work/no-work pair of initrd files?  You can take them apart, they are just compressed cpio(1) files last time I looked.

    MatthewLinux
  • Dude!
    Dude! [email protected]Member Posts: 22,830 Black Diamond
    edited Apr 18, 2018 5:11PM

    I think it would have been good to know what actually caused the issue. Perhaps the boot partition shifted to a new device, or the upgrade ran out of disk space during the rebuild of the boot image. I think the later is somewhat common. How large is your /boot partition?

    MatthewLinux
  • MatthewLinux
    MatthewLinux Member Posts: 80 Blue Ribbon
    edited Apr 19, 2018 11:50AM

    So I just deployed an OL 7 GA machine and upgraded it to OL 7.5. I noticed that after I patched, and BEFORE I rebooted, I checked the grub config:

    grep initrd /etc/grub2.cfg

    And the latest Kernel was NOT in the list. So, it seems there is something wrong with grub2 after the new kernel is installed. I ran the following and then rebooted and it came up fine.

    grub2-mkconfig -o /boot/grub2/grub.cfg

    Thanks,

  • MatthewLinux
    MatthewLinux Member Posts: 80 Blue Ribbon
    edited Apr 19, 2018 11:51AM

    /boot is 512MB and is only 28% used

  • MatthewLinux
    MatthewLinux Member Posts: 80 Blue Ribbon
    edited Apr 19, 2018 11:52AM

    Running Oracle Linux as servers on VMware vSphere.

  • Sergio-Oracle
    Sergio-Oracle Member Posts: 2,674 Employee
    edited Apr 19, 2018 12:39PM

    Do you have the UEK5 preview channel enabled? Can you show output of yum repolist on the machine that failed?

  • MatthewLinux
    MatthewLinux Member Posts: 80 Blue Ribbon
    edited Apr 19, 2018 12:43PM

    No, we're not using the UEK. It's just the ol7 latest channel.

  • MatthewLinux
    MatthewLinux Member Posts: 80 Blue Ribbon
    edited Apr 19, 2018 1:54PM

    I just patched a RHEL server to 7.5 and it did not have the grub2 issue that I'm experiencing on OL 7.5.

  • Sergio-Oracle
    Sergio-Oracle Member Posts: 2,674 Employee
    edited Apr 19, 2018 3:06PM

    Any chance you can show the before and after grub.cfg and the exact commands you used? Also, is this against ULN or yum.oracle.com?

    MatthewLinux
  • Tommy.Reynolds-Oracle
    Tommy.Reynolds-Oracle Member Posts: 1,754 Employee
    edited Apr 19, 2018 3:12PM Answer ✓

    It turns out to be a known issue with the upgrade to 7.5, so you have done nothing wrong as far as I can tell.

    There is a strong separation between userspace (the usual suspects, libraries, applications, shells, X window, on and on) and the privileged kernel space.  In practical terms this means that most times it does not matter which kernel version you use with which version of userspace, mostly the C standard library.  You can update either independently.

    Not this time.

    The interface between userspace and the kernel has an incompatible change.  The userspace must be updated first so that when the kernel is updated all the right subroutines and all are ready and waiting in userspace.  There are firmware and microcode which must be updated so the devices operate the way the newly booted kernel expects, else the boot fails.

    Since there has not been an RPM version requirement on the kernel and userspace, the updater updates the packages in the wrong order.  The right order is to update userspace first and then update the kernel.  This is why the initrd was not rebuilt.

    If you have Oracle Linux support, file a service request (SR) ticket to be notified when a remedy is announced.

    Cheers,

    Tommy

    MatthewLinux
This discussion has been closed.