Class Tokenizer

分词模块管理器

Hierarchy

SModule
- Tokenizer

Implements

ISModule

Index

Constructors

constructor

Properties

segment
type

Methods

Constructors

constructor

new Tokenizer(segment: Segment): Tokenizer

Inherited from SModule.constructor
- Defined in lib/mod/mod.ts:16
Parameters
- segment: Segment
  
  分词接口
Returns Tokenizer

Properties

segment

segment: Segment

type

type: string = "tokenizer"

Methods

Protected _doMethod

_doMethod<S, T>(fn: string, target: S[], mods: T[], ...argv: any[]): S[]

Inherited from SModule._doMethod
- Defined in lib/mod/mod.ts:26
Type parameters
- S: IWord
- T: ISubSModule
Parameters
- fn: string
- target: S[]
- mods: T[]
- Rest ...argv: any[]
Returns S[]

split

split(text: string, mods: ISubTokenizer[], ...argv: any[]): IWord[]

- Defined in lib/mod/Tokenizer.ts:131
对一段文本进行分词

Parameters
- text: string
  
  文本
- mods: ISubTokenizer[]
- Rest ...argv: any[]
Returns IWord[]

Generated using TypeDoc