get_token_list¶
-
lexnlp.nlp.en.tokens.
get_token_list
(text: str, lowercase: bool = False, stopword: bool = False, preserve_line: bool = True) → List¶ Get token list from text. :param text: :param lowercase: :param stopword: :param preserve_line: keep the preserve the sentence and not sentence tokenize it. :return: