A synthetic fact, tested against specificity and retention
Can a model learn one fact
without losing others?
This project fine-tunes small language-model adapters using LoRA. The goal is to teach one invented fact while keeping it attached to the right name and preserving answers the untouched model already knew.
Atemokoloporos is a rainbow unicorn.
Loading the verified study summary…
The model families are separate studies, not a controlled test of model size.
01 / The whole study
Recall is only part of the result
Recall
The recorded lexical rule for the taught fact. This is not exact-string matching.
Near-name specificity
Avoiding the taught claim for similar invented names. This does not measure general factual accuracy.
Knowledge retention
Correct answers to common-knowledge control prompts, with acceptance also checking which baseline hits were lost.
02 / Inspect a recipe
Training and validation loss
Loss is the recorded training objective, not factual accuracy. Curves come from this run’s saved evaluation report; their scales and data mixtures can differ between recipes.
Inspect recorded loss values
| Optimizer step | Training loss | Validation loss |
|---|
Source files and SHA-256 hashes for this recipe
Read the actual prompts and outputs
Untouched baseline and selected adapter, paired by prompt ID. Outputs are shown in full, including any literal formatting.
03 / Interpretation
What the evidence can tell us
A result on a fixed regression suite
Final prompts were kept out of training and checkpoint selection. Aggregate outcomes informed later recipe design, so this suite is not a pristine research holdout. The accepted Qwen3.8 result is a single-run finding.
Controls are prompt categories
Near-name and common-knowledge prompts test specificity and retention. No random-control model runs, isolated ablations, or seed-replication results are recorded. Recipe changes do not establish which ingredient caused an improvement.
Missing results stay missing
An interrupted attempt has no final tuned evaluation. Deferred recipes were not run. Neither is counted as a successful or failed completed evaluation. There are no reward curves or replication-based uncertainty intervals to plot.
Acceptance and publication are different
Archived historical adapters retain their failed or inconclusive labels. A public adapter’s availability or a successful chat does not create a new acceptance result.
04 / Trace the result
Evidence and methods
This page is generated from checked-in, hash-verified evidence. It does not run a model or collect analytics. Follow the source links to inspect the underlying records.
- Combined experiment inventory
- Instruction audit and unresolved limitations
- Corrected Qwen3.5 paper
- Training, evaluation, and checkpoint selection
- Reproduce a registered experiment
- Qwen3.8 claim audit and provenance qualifications
- Model Editing by Standard Fine-Tuning · Counterfactually augmented data