You might be better of asking this in the SOA/OSB forum. Also please add version, OS etc.
The ftp poller from OSB is always targeted to exactly one server, but at least in theory both servers should pick up the messages from the distributed Q.
What behaviour exactly are you seeing? Does it relate to the ftp poller target?