That's not enough information to give a definitive answer. I'll provide some insight which could be possible cause.
1) Why is OSB using a new tape when there are 4 tapes of the same pool "Media family" with free space?
When you say "new" tape, do you mean a new tape which has never been used? Or are you referring to an expired tape which can be reused (ie the backup starts at the beginning of the tape)? While practically speaking a new tape and a reusable tape are the same in that the backup begins at the beginning of the tape. However, OSB will choose a reusable tape prior to a "new" tape.
If you are referring to a reusable tape, it could occur because the reusable tape happen to be located in a slot prior to the appendable tape. For example, if reusable tape is in slot 1 and an appendable tape of same media family is in slot 2, OSB will grab the reusable tape. If in that same example, the appendable tape was in slot 1....it would have been used.
2) They have seen that when a "dataset" has 4 hosts, it takes 4 tapes to make the backup. Is there a way to change this behaviour? They would like it to use only 2 or 3 tape drives.
If a dataset includes 4 separate clients/hosts (which is fine), the clients will be backed up concurrently assuming there are available tape drives. In the case you describe, there are 4 hosts and 4 available tape drives which means each host backup would run concurrently to it's own tape. If the dataset includes multiple paths on the same host, the paths will be backed up sequentially.
If you want to include 4 hosts in one dataset and for the backups to run sequentially on the same media(s), you could schedule the backup and restrict it to one drive. The backups would run sequentially to same tape and span to additional tapes as needed. Or you could restrict the backup schedule to two drives or three or no drive restriction and all four drives would be used if available.
thanks for your helpful response (and the earlier email you sent me on this subject)
Regarding this >>>> "When you say "new" tape, do you mean a new tape which has never been used?"
I've asked customer for a definition of what he means by "new tape", but not had a response yet.
I need to get a more comprehensive picture of the problem, using your input, but I can maybe make some headway now with this information.