lexnlp.extract.en.preprocessing package

Submodules

lexnlp.extract.en.preprocessing.span_tokenizer module

class lexnlp.extract.en.preprocessing.span_tokenizer.SpanTokenizer

Bases: object

static get_token_spans(txt: str) → Generator[[Tuple[str, str, int, int], None], None]

returns: [(‘word’, ‘token’, (word_start, word_end)), …]

Module contents