Change the voice: I will have finished that book.
Change the voice: I will have finished that book.
Explanation
In passive voice, the object ("that book") becomes the subject, and the verb tense is adjusted accordingly.
The sentence changes from "I will have finished that book" to "That book will have been finished by me."