Place hold on AP invoice only if it has no matched lines

We have some supplier sites which require matching. We set the Hold Unmatched Invoices flag to "Y" at the site level. However, if we math the invoice to a PO then add unmatched lines (such as tax and/or freight), the invoice still goes on hold. We only want the hold to be placed if there are no matched lines. How can this be accomplished?