Dataset Splitting

Dataset splitting module for creating train/dev splits.

This module provides functionality to split SpaCy datasets into training and development sets for model training and validation.

class spacylize.splitter.DataSpliter(input_file, train_file, dev_file, dev_size, seed)

Bases: object

Splitter for SpaCy datasets into training and development sets.

Splits a SpaCy binary dataset into separate training and development files using a specified split ratio.

Parameters:

• input_file (Path)

• train_file (Path)

• dev_file (Path)

• dev_size (float)

• seed (int)

input_file

Path to the input SpaCy dataset (.spacy).

train_file

Path where the training split will be saved.

dev_file

Path where the development split will be saved.

dev_size

Fraction of data to allocate to the dev set (0.0-1.0).

seed

Random seed for reproducible splitting.

Note

This class is not yet fully implemented.

__init__(input_file, train_file, dev_file, dev_size, seed)

Initialize the DataSpliter.

Parameters:

• input_file (Path) – Path to the input SpaCy dataset (.spacy).

• train_file (Path) – Output path for the training set.

• dev_file (Path) – Output path for the development set.

• dev_size (float) – Fraction of data for the dev set (e.g., 0.2 for 20%).

• seed (int) – Random seed for reproducibility.

run()

Run the dataset splitting process.

Note

This method is not yet implemented.