Explanation

The rightful Duke of Milan in The Tempest is Prospero, who is usurped by his brother Antonio. 

Antonio: Usurped the dukedom from Prospero.  

Ferdinand: Son of King Alonso of Naples, not related to Milan.  

Caliban: Slave on Prospero's island; held no official position.