Though this is shocking to some at first, it is not as onerous as it sounds because a credit is allowed to offset the amount of non-US tax that has been paid.
Ever since 1921, and most recently reconfirmed by legislation in 1976 and 1986, Congress specifically has chosen not to tax interest paid to non-resident foreigners.