这 就是 yacc 要做的：yacc 让您可以描述希望怎样处理记号。

This is where yacc comes in: yacc lets you describe what you want to do with tokens. A yacc grammar tends to look sort of like this

youdao

该包装器使用字节数组来保存所产生的内容，在filter链处理完成之后我们利用数组的MD 5哈希值计算记号。

The wrapper USES a byte array to hold the generated content and after the filter chain processing completes we compute the token using an MD5 hash of the array.

youdao

语意是一个字串，和记号很像，但是它已经是一般化了的，这样同一个词的不同形式就可以一样来处理。

A lexeme is a string, just like a token, but it has been normalized so that different forms of the same word are made alike.

youdao