How to get the Custom Object Record by RecipientId

Hello, I have a question about email activity mapping and will really appreciate your help. We send campaign emails based on custom object records (not contact because we may have multiple custom objects per contact). I understand that each email activity comes with a RecipientId that can be mapped to the custom object record. But how do I find the custom object based on a RecipientId to in order to join the email activities with the custom object records?

Thank you in advance.