Make employment history read only for everyone but HR Specialists?

Summary

We are trying to make employment history in the employment info quick action read only for Managers and Employees while having it not read only for our HR Specialists

Content

Just as the summary says, we are trying to make our Employment History section read only so that managers can't see their employees employment history information. We still want a manager to be able to view their direct report's information, we just don't want the details of the information to be accessible for managers.

Is there any way to do this without using page composer or modifying a predefined role?