The tower of Babel wasn’t in Babylon; instead, it tended to move around quite sporadically. The seer that he was seeking made the tower her home when she was banished from the council of seers with the order to have her abilities stripped the next day. On that same day, she had gone to the tower and banished it along with herself into nonexistence. Till this day, no one knows why the sentence was passed but as far as Charon could guess she had found and proposed something that the council found preposterous because of their prudish nature and removed her for her controversial ideas.
Charon didn't make it a habit to seek her council because aside from the fact that she was more than very difficult to find, she was quite temperamental and tend to blast before she thinks. But still, he had
