Another possibility is that Musashi deliberately arrived late to disconcert his opponent, used only a wooden sword to further unbalance him, and used the psychological advantage he created, along with his formidable swordsmanship, to win the duel.
One take of this story is that Musashi was so good that he was completely unconcerned about the duel and arrived at his own leisure, with a wooden sword that he fashioned as an afterthought since his opponent was so far beneath him.