Restrict Line manager to access PII data

Managers have access to LINE manager and Hiring manager role. They are able to see PII data of their direct reports such as DOB, Address through reporting. We are planning to implement digital assistance and do not want managers to get access to PII data when they ask for it. This will be a security breach. Please suggest best way to implement restrictions so that we can prevent this.