As the note is paid off, value is transferred out of your estate into the trust and thence to your heirs.

FORBES: Passing Down The House

The valuation is made when you set up the trust, and it is done by subtracting from the assets transferred the value of your annuity.

FORBES: The Mini-Grat Tax Dodge