Data Generation

Data generation module for creating SpaCy training datasets using LLMs.

This module provides functionality to generate annotated training data for SpaCy NER and text classification tasks using Large Language Models. The generated data is parsed and stored in SpaCy’s binary format.

class spacylize.generator.TaskParser

Bases: ABC

Base class for parsing LLM-generated annotations.

abstractmethod static parse(text)

Parse annotated text and return task-specific data.

Parameters:

text (str)

class spacylize.generator.DocumentBuilder

Bases: ABC

Base class for building SpaCy documents.

abstractmethod static build(nlp, parsed_data)

Build a SpaCy Doc from parsed data.

class spacylize.generator.NERParser

Bases: TaskParser

Parser for NER inline annotations: [TEXT](LABEL)

static parse(text)

Parse NER annotations.

Parameters:

text (str) – Annotated text with inline [entity_text](LABEL) markers.

Returns:

(clean_text, entities) where clean_text is the text without

annotation markers and entities is a list of (start, end, label) tuples representing entity spans.

Return type:

tuple

class spacylize.generator.NERDocumentBuilder

Bases: DocumentBuilder

Builds SpaCy docs with entity annotations.

static build(nlp, parsed_data)

Build NER document with entity spans.

Parameters:

• nlp – SpaCy language model.

• parsed_data – Tuple of (clean_text, entities).

Returns:

SpaCy Doc with entity annotations.

class spacylize.generator.TextCatParser

Bases: TaskParser

Parser for text classification annotations.

static parse(text)

Parse textcat annotation format.

Expected format:

Text content

— LABEL: CATEGORY

Parameters:

text (str) – Annotated text with delimiter and label.

Returns:

(clean_text, categories_dict) where categories_dict maps

category names to confidence scores.

Return type:

tuple

Raises:

ValueError – If the format is invalid.

class spacylize.generator.TextCatDocumentBuilder

Bases: DocumentBuilder

Builds SpaCy docs with category annotations.

static build(nlp, parsed_data)

Build textcat document with categories.

Parameters:

• nlp – SpaCy language model.

• parsed_data – Tuple of (clean_text, categories_dict).

Returns:

SpaCy Doc with category annotations.

class spacylize.generator.TaskHandler

Bases: object

Registry for task-specific parsers and builders.

classmethod get_handler(task)

Get parser and builder for a task.

Parameters:

task (str) – The task type (e.g., ‘ner’, ‘textcat’).

Returns:

(parser_class, builder_class)

Return type:

tuple

Raises:

ValueError – If the task is not supported.

classmethod supported_tasks()

Get list of supported task types.

Returns:

List of supported task type strings.

Return type:

list

class spacylize.generator.DataGenerator(llm_config_path, prompt_config_path, n_samples, output_path, task)

Bases: object

Generator for creating SpaCy training data using LLMs.

Parameters:

• llm_config_path (Path)

• prompt_config_path (Path)

__init__(llm_config_path, prompt_config_path, n_samples, output_path, task)

Initialize the DataGenerator.

Parameters:

• llm_config_path (Path) – Path to the LLM configuration YAML file.

• prompt_config_path (Path) – Path to the prompt configuration YAML file.

• n_samples – Number of training samples to generate.

• output_path – Path where the generated .spacy file will be saved.

• task – The SpaCy task type (e.g., ‘ner’, ‘textcat’).

run()

Run the data generation process.

Generates n_samples of annotated text using the configured LLM, parses the annotations using task-specific parsers, and saves the results to a SpaCy binary file.