滤词器在分词之后对标号做进一步处理（最典型的比如去掉标点符号和一些常见词，像"the", "an", "a"）。

The filters do any post-tokenizing work on the tokens (typically dropping out punctuation and commonly occurring words like "the", "an", "a", etc).

youdao

结果得到的编码文本将去掉空格、数字和标点符号，从而使它(略微)更加难于解码。

The resultant encoded text will be stripped of Spaces, Numbers and punctuation, making it (marginally) more difficult to decode.

youdao

由于把薪金存储为整数，因此应用程序必须先去掉所粘贴数据中的标点符号，然后再保存。

Because you store salary as an integer, your application must strip punctuation from the posted data before it's persisted.

youdao

The less than side is gonna be a less than sign, and then the word and is going to become not one & but two ampersands back to back, and we'll see why it's two and not one before long.

小于“侧还是用的“小于“,而中间的“and“将去掉,换成两个紧挨着的&符号，但至于为什么是&&而不是,我们以后会提到。

哈佛公开课 - 计算机科学课程节选