How can i limit manager access for other assignment incase of 2 tier-Multi assignment model

We are looking to limit manager access incase of dual assignment. Consider EmployeeA is having 2 assignments

1. Assignment1: Psychometrist for LE1 and Line manager as  Linemanager1

2. Assignment 2: Nurse for LE2 and nd Line manager as  Linemanager2

The goal is to allow Linemanager1 to change assignment information of just assignment1 and restrict edit operations for Assignment2. Similarly for Linemanager2 to change assignment information of just assignment2 and restrict edit operations for Assignment1.

If I enable manager heirarchy as part of Line manager security, I could see/edit both the assignment records when we login as any manager. Our goal is to edit operations on other assignment.