Adding Matrix Manager using the Direct Reports flow in Responsive

We recently encounter this scenario: Business needs to add direct reports to one of the matrix managers.

We used the Direct Reports flow; and we have selected the workers in the Add Direct reports section and submitted.

The expected result was as is the case for LM change; instead checking in their assignment I've noticed that they have been assigned to both managers of the matrix.

Is there a way to limit the number of matrix managers of the same type on worker assignment?

Did we used the wrong flow? Maybe is best suited to have this change done via Assignment Change?