ORA-01422: exact fetch returns more than requested number of rows

In Oracle 11.2.0.2.0 I have a one direction streams setup between 2 machines (s=source, d=destination)
When I delete some rows from a table with a duplicate rows on the source, the apply process on destination
gives the following error.
ORA-01422: exact fetch returns more than requested number of rows

When I did
exec DBMS_APPLY_ADM.SET_PARAMETER( apply_name    => 'apply_stream',
                                   parameter     =>'allow_duplicate_rows',
                                   value         => 'y');
I am not getting ORA-01422, but the rows on the destination machine do not get deleted.

I am thinking to add a new column to the table that get populated with a trigger on insert with