Tax Applicability Rule

Is there a way to create tax determinant factor set based on PO Line Type (Goods, Expenses, Rate Based Services etc)? I am trying to write a tax applicability rule for indirect purchases (Goods - Not Applicable, Expenses and Services - Applicable) but not able to find a way without creating any fiscal classifications etc as they need additional data entry at the time of PO creation

We have Inventory Items created for Expense Items too and hence not able to use Inventory or non-inventory linked determinant factor

Any inputs?