Can I restrict LM access to performance documents for dual assignment employees?

Hi all, is there a way to restrict line manager to access the Performance Document for employee not their direct report in dual Assignment. An employee with a dual assignment goes through a Performance Management Cycle. They have a different line manager for each of his role. Once the Performance Document is completed by both the line managers. They should not be able to view the performance document completed by the other line manager. Is there a way to achieve this functionality as currently both the line managers can access the performance document of the employee in dual assignments.