How to hide task panel on Time Management page?

We need to hide the task panel on the right side of the Time Management page from workers with Line Manager security roles only. We want HR security roles to still have access to the page. I tried doing this in Page Composer, but the edit function of the component is grayed out.