Has anyone managed to request further information after application has been submitted?

We have a need to ask candidates about convictions details, we already ask a question "Do you have any convictions?" with a yes/no answer and thats it, but we really need to ask them for details if they say yes, but we dont want the answer to the follow up question to be visible to managers only recruiters.

I thought the best way to do that would be to create a request information flow that would send out a message after the application form has been submitted, based on a "Yes" response to the above question. It would have a follow up question that would be confidential so that when it came back it would only be visible to those with the correct privilege which we would only give to our recruiters.