I think it would be apt if the repository definition you have mentioned should have many-to-many relationship between book and author. The reason I'm saying this is because a book can have multiple authors and the same author can also have written multiple books. Anyways coming on to the RQL to get all authors who have written a particular book, I think you can do it using INCLUDES ITEM. Try this RQL on author item:
books_written INCLUDES ITEM (title="abc")
or from dyn/admin run following XML operation on your repository component
<query-items item-descriptor="author">books_written INCLUDES ITEM (title="abc")</query-items>
See if it serves your purpose.