Hide quick action but show in application task

Summary:

I understand that if you had a quick action from My Team, it will also not show in the Application Task list if the assignee is Line Manager.

Is there a way to workaround this? We do not want users accessing the quick action, but we want managers to be able to promote in a journey task.

Content (please ensure you mask any confidential information):

Version (include the version you are using, if applicable):

Code Snippet (add any code snippets that support your topic, if applicable):