Build training data you can audit.

Forge carries source identity and usage policy through generation, privacy checks, layered deduplication, contamination screening, source-grouped splitting, and a content-addressed release.

12 Typed pipeline stages

3 Similarity layers

0 Unknown rights in candidates

Evidence boundary: curation and governance controls are implemented and verified, but the historical model run is not treated as independent performance evidence. An independently authored benchmark and genuine human review remain release gates. The blind evaluation framework is implemented, while its checked-in labels are controlled fixtures rather than human evidence.

Trust boundaries first

Rights before generation

Candidate builds require a declared license and permitted training use for every loaded source.

Auditable quarantine

Schema, privacy, and duplicate rejections carry reason codes, evidence, and a retained source identity.

Layered comparison

Exact, MinHash LSH, Jaccard, and optional embedding controls screen duplicates and benchmark overlap.

Pipeline

1Ingest

→

2Source governance

→

3Generate + quality

→

4Record governance

→

5Dedup + judge

→

6Contamination

→

7Select + split + profile

→

8Release

Evidence a run leaves behind

Split manifest

Requested and achieved fractions, source counts, overlap results, seed, and SHA-256 hashes for input, train, and test artifacts.

Release manifest

Source, benchmark, and artifact fingerprints, release gates, claim boundaries, and a deterministic release ID.

Governance reports

Rights decisions, privacy findings, schema rejections, duplicate clusters, detector versions, and thresholds.

Dataset profile

Sources, tasks, lengths, quality, difficulty, rejection reasons, and hashes for every release-facing artifact.

Measured curation controls

The checked-in 12-pair fixture contains exact copies, near duplicates, semantic paraphrases, unrelated pairs, and hard negatives. MinHash LSH plus Jaccard records precision 0.833333, recall 0.625, F1 0.714286, and a multi-seed bootstrap interval on that fixture. The fixture tests control behaviour. It does not estimate production-corpus quality.

make curation-calibration

Evaluation controls, without inflated claims

Blind and frozen

Content-addressed releases separate human packets, reversed-order judge packets, and the protected generator key.

Claims are gated

Fixture labels, same-family judges, incomplete coverage, weak agreement, or position bias cannot produce an evaluation-ready result.

The eight-item controlled fixture reports alpha 0.754098, judge agreement 0.875, macro F1 0.858586, and reversed-order consistency 0.875. Those values verify the analysis path in CI. They are not human evidence and do not establish model quality. Read the evaluation design and evidence boundary.

make evaluation-fixture

Start with the smoke run

git clone https://github.com/pugalenthi0928/training-data-factory.git
cd training-data-factory
make install
make forge

The smoke run uses a dummy model and a synthetic contamination fixture. It verifies mechanics, not model quality. Read the project overview or inspect the technical controls before interpreting any output.