tokenizer
tokenizer · tokenizers
A system that parses an input stream into its component tokensA system that parses an input stream into its component tokens.
Значение
значений: 1A system that parses an input stream into its component tokens.
Происхождение
From tokenize + -er.From tokenize + -er.
Wiktionary, CC BY-SA 4.0 · source
Формы
мн. число: tokenizers| Ед. число | tokenizer |
|---|---|
| Мн. число | tokenizers |
правильный