The Welsh Language Commissioner Meri Huws also wrote to the savings bank - which is backed by the Treasury - saying that there was no provision in law to end the language scheme.
Sir Anthony, who is also the independent complaints commissioner for the Financial Services Authority, raised concerns about the compliance scheme - added to legislation at the tail end of the last Parliament.