1. Your question and your description is confusing. Let me clarify few things.
2. For your process, pick any swimlane that has Human Task and lets call that role as MyApp_Analysts. After you deploy this, in EM Console, you map this role to real AD Group say MyApp_Analysts_Group. This group has 10 users.
3. When task goes to this Role, by default all the 10 Users sees it in thier Inbox. First User who claims it or Saves it, the task goes to his inbox. All other 9 Users will NOT see this Task anymore in thier Inbox. Period. So there is NO need for you to Withdraw it from other Users. Now if first User accidentally Claimed it, he can Release it. and Task goes back to Pool and all Users can see again and anyone can Claim or work on it.
4. Fundamental thing, if a Task is Claimed by a user who is in a pool of users or groups, all other users will NOT see in thier inbox. If they have already opened inbox, just refresh it and it disappers.
5. NOW as a Process Owner, he can Withdraw any Task Claimed by any user. Process Owner is like kind of adminstrator for that process and all Human Task. Again ProcessOwner is just a Role. It makes sense only when you add a User or group of users to that Role and they play that role.
6. Now if you want a specific set of people to Withdraw specific Tasks, still you can do. In Human Task definition on Right side, Under General -> Owner, you can speicify multiple roles or add groups like MyApp.ProcessOwner, AnalystHTO etc. This AnalystHTO is like Human Task Owner for this Analyst Task and you can map any user or group to this role. So now process owner and this role guy can Withdraw that Task. basically they both have Admin privileges just for that task.
Hope this clarifies your dobut. If your issue is something else, please give more details.