Changing supplier on a PO if a line has a contract and/or is canceled

On our Purchasing change template header we have Vendor_ID checked to track changes.  This allows the supplier id to be changed on a PO if needed.  If a PO line contains a contract, that contract first needs to be removed in order for the supplier to be changed.  If you have to cancel a line and don't remove the contract before canceling then you are stuck with not being able to change the supplier on the PO.  For whatever reason, the system still references the contract on a line even though it is canceled.  This leads to always needing to