class XtfChunkSource extends ChunkSource
Modifier and Type | Field and Description |
---|---|
private static char |
bumpMarker |
private static char |
nodeMarker |
analyzer, chunkBump, chunkCache, chunkCacheSize, chunkOverlap, chunkSize, docNumMap, field, firstChunk, lastChunk, mainDocNum, reader
Constructor and Description |
---|
XtfChunkSource(IndexReader reader,
DocNumMap docNumMap,
int mainDocNum,
String field,
Analyzer analyzer)
Construct a chunk source
|
Modifier and Type | Method and Description |
---|---|
protected Chunk |
createChunkTokens(int chunkNum)
Create a new storage place for chunk tokens (derived classes may
wish to override)
|
private void |
debugChunk(XtfChunk chunk)
DEBUGGING ONLY:
Print out debugging info for the current chunk, including all of
its tokens.
|
Chunk |
loadChunk(int chunkNum)
Read in and tokenize a chunk.
|
protected void |
loadText(int chunkNum,
Chunk ct)
Read the text for the given chunk (derived classes may
wish to override)
|
getChunkOverlap, getChunkSize, inMainDoc
private static final char bumpMarker
private static final char nodeMarker
XtfChunkSource(IndexReader reader, DocNumMap docNumMap, int mainDocNum, String field, Analyzer analyzer)
protected Chunk createChunkTokens(int chunkNum)
createChunkTokens
in class ChunkSource
protected void loadText(int chunkNum, Chunk ct) throws IOException
loadText
in class ChunkSource
IOException
public Chunk loadChunk(int chunkNum)
loadChunk
in class ChunkSource
private void debugChunk(XtfChunk chunk)
DEBUGGING ONLY:
Print out debugging info for the current chunk, including all of its tokens.