Some fields you may not want to be tokenized, for example a serial number field.

有些字段你可能不想它被分词，比如一连串的数字字段。

youdao

The contents of this file are tokenized into a list of strings and bound to a list called Rules.

文件的内容被分割放入一个字符串列表，然后绑定到名为Rules的变量。

youdao

Erlang also gives us programmatic access to the tokenized, parsed and abstract forms of an expression.

Erlang还允许我们编程访问经过词法分析(tokenized)、语法分析之后、抽象形式的表达式。

youdao

Once tokenized and hidden, any given card product is far more vulnerable to being displaced, as the issuers have already learned from PayPal.

FORBES: Tokenization And The Collapse Of The Credit Card Payment Model