Can't Use Ship To Phone Number on Drop Ship Purchase Orders

When you are sending a drop ship purchase order to a vendor - it is only sensible that you would also want to send the vendor the customer's phone number at their ship-to address.

This is required for freight deliveries, FedEx, UPS, etc. 

Currently, using custom fields I have only been able to source the customer's phone number off their main customer record.  As far as I can tell, you cannot pull the phone number from their ship-to destination.  Anyone else have any luck with that?

This can be a problem when dealing with customer's that have a different phone number at their ship-to destination than at their main headquarters.  Or for example if the customer is at a vacation home waiting for delivery - and they keep calling the customer's home phone trying to contact them.  Etc.