In past i faced the almost similar issue with Gmail Mail server. I was trying to read the messages from it and once read the message was assigned to sub-folder. This approach was working fine when there were few emails but when i tested it against 200+ emails at a time it started reading the messages twice. The only solution was to change the Mail server. So when i changed to Yahoo, it worked well.
Check the settings in email server.
The SOA Server was going to warning state because of SMTP/network issues, due to which SOA server doesn’t get acknowledgement back when an email is sent from SOA to smtp server.
After setting the timeout parameters, we were getting duplicate mails for larger files as acknowledgment doesn't come and it retries infinite times.
To overcome this we have set retry from SOA to maximum 1 at an interval of 5 mins. i.e., redelivery limit to 1 and redelivery time to 5 mins
We have kept 1 retry, in order to handle genuine case where the email delivery was unsuccessful.
We also have to check the stability of SMTP server, which actually creates the main issue of keeping mails in Pending state.
Thanks & Regards,