Can we setup security to restrict users access to different storage areas within an Inventory busine

Our business would like to restrict store managers access to different stores within an Inventory business unit.

Already different departments are setup as an inventory business unit so we don't want to setup different stores inside a department as business unit.

Appreciate if experts share their advise on this.