Disable Drill down from Journals page for specific user/Ledger who has access to Journal creation

Hi,
we have a requirement to disable drill down functionality for specific users/ledgers who has access to journal creation. We tried using custom financial analyst role, that role is not fulfilling the requirement, as when we are providing privilege for entering journal, its providing access for drill down