The Data Lab's Construct Ledger

Method · How these indicators are builtThese indicators are produced by an active-learning pipeline run over the full collection of disposition narratives. (click to expand ↓)

The pipeline follows a teacher-student design. The teacher is a committee of small language models that reads narratives and judges whether a concept is present. The student is an encoder model, a different type of system from a generative language model and one that is even smaller and more efficient than the teacher; it learns from the teacher's judgments and then applies the concept across the full collection of narratives quickly and at low cost. Human reviewers anchor both ends of the process: they guide the definition of each concept at the start and validate the results at the end. The steps below describe this process in non-technical terms.

  1. Define the construct (human and model together). A researcher states the concept to be detected and, working interactively with a language model, assembles a bundle: a plain-language definition, example phrases, and search terms. Human judgment leads at this stage.
  2. Gather candidate records. Search terms and an embedding model, which ranks narratives by their semantic similarity to the concept, together draw a working pool of disposition narratives that may mention it from the full collection.
  3. Committee labeling (the teacher). Two small LLMs act as an independent committee, each reading the narratives and deciding whether the concept is present; their agreements and disagreements are recorded.
  4. Train the encoder model (the student). The student encoder is trained on the committee's labels, but it does not begin from scratch. It is built on a second encoder model already fine-tuned on Michigan's disposition summaries; because that base model has learned the language and structure of these narratives, the student reaches its target accuracy with substantially fewer labeled examples than it would otherwise require.
  5. Active-learning loop. The pipeline routes the most uncertain cases back for relabeling and retrains, so that accuracy improves with far fewer human labels than exhaustive review would require.
  6. Validation (human review and external metrics). The trained model is validated two ways: external validation metrics compare its corpus-wide results against independent data sources, and a human reviews a fresh sample. The human portion uses a review tool optimized to facilitate efficient review, which presents the model's flagged cases for confirmation or correction before the measure is trusted.

Human oversight is concentrated at the two ends of the pipeline. A person guides the language-model interaction that builds each bundle at the start, and a person validates results through a review tool optimized to facilitate review at the end. Between those ends, steps two through five run automatically; once the bundle is defined, each stage passes its output to the next without manual intervention, through to the final validation.

Internal κ is the model's own accuracy: held-out Cohen's κ against human or committee labels on data it never saw. External validation is a separate, independent check, claimed only where the evidence supports the measure. Projected ranges rest on prior testing and calibration, in which internal κ has generally fallen within ±.06 of human-review κ; for constructs not yet independently reviewed, the projected band therefore marks where human-review agreement is expected to fall. Expand any construct for its definition and the full validation record.

01 Housing & economic
Unemployment κ 0.894≈ κ .83–.95projected

Caregiver joblessness or loss of employment contributing to economic strain.

Projected · not independent evidence
≈ κ .83–.95

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Transportation problems κ 0.867≈ κ .81–.93projected

Lack of reliable transportation affecting access to services, appointments, or employment.

Projected · not independent evidence
≈ κ .81–.93

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Homelessness κ 0.830ρ +0.70 · κ .80

The family or child lacks a fixed, regular, and adequate nighttime residence; the indicator is anchored to the McKinney-Vento Act definition of homelessness.

External validation
ρ +0.70 against McKinney-Vento homeless-student counts (schools), by district; κ .80 on independent human review of a fresh sample.
Eviction κ 0.822≈ κ .76–.88projected

Formal or informal loss of housing through eviction proceedings.

Projected · not independent evidence
≈ κ .74–.86

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Utility shutoff κ 0.787≈ κ .73–.85projected

Loss or threatened loss of essential utilities such as heat, water, or electricity.

Projected · not independent evidence
≈ κ .73–.85

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Housing instability κ 0.760κ .71

Sub-threshold housing conditions that fall short of homelessness but can precede it, based on structural problems with the home, eviction risk, late or missed payments, and utility shutoff.

External validation
κ .71 on independent human review of a fresh sample.
Food insecurity κ 0.796≈ κ .74–.86projected

The household has limited or uncertain access to adequate food; the indicator is anchored to the USDA definition of food insecurity — a household-level economic and social condition of limited or uncertain access to enough food for an active, healthy life.

Projected · not independent evidence
≈ κ .74–.86

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Rent arrears κ 0.886≈ κ .83–.95projected

The household is behind on rent, or unable to afford its rental payment, as a financial strain distinct from a formal eviction process.

Projected · not independent evidence
≈ κ .83–.95

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Substandard housing κ 0.898≈ κ .84–.96projected

The family currently occupies a dwelling that is physically unsafe or inadequate — failing utilities from disrepair, severe infestation, structural hazards, mold, or serious code violations — while still living there.

Projected · not independent evidence
≈ κ .84–.96

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Foreclosure κ 1.000≈ κ .94–1.00projected

A homeowner household is losing, or at risk of losing, a home they own through mortgage default or the foreclosure process, distinct from renter eviction.

Projected · not independent evidence
≈ κ .94–1.00

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Residential churn κ 0.897≈ κ .84–.96projected

A pattern of hardship-driven housing impermanence — repeated moves or an inability to maintain stable housing over time — as distinct from a single or voluntary move.

Projected · not independent evidence
≈ κ .84–.96

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

02 Violence & safety
Firearms κ 0.930κ .94

Presence of, access to, or use of firearms in the household, a safety-relevant risk factor.

External validation
κ .94 on an independent benchmark (Qi et al., 2026).
Human trafficking κ 0.851≈ κ .79–.91projected

Indicators of labor or sex trafficking involving a child or caregiver.

Projected · not independent evidence
≈ κ .79–.91

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Domestic violence κ 0.831ρ +0.74 · κ .85

Intimate-partner violence in the household, whether witnessed by or directed at children.

External validation
ρ +0.74 against Michigan State Police DV victim reports; the construct reproduced the 2020 lockdown surge. κ .85 on an independent benchmark (Qi et al., 2026).
03 Mental & behavioral health
Autism κ 1.000≈ κ .94–1.00projected

Indication that a child or caregiver has an autism spectrum diagnosis or related developmental profile relevant to service needs.

Projected · not independent evidence
≈ κ .94–1.00

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

ADHD κ 0.984≈ κ .92–1.00projected

Attention-deficit/hyperactivity disorder noted for a child or caregiver, relevant to behavioral and treatment context.

Projected · not independent evidence
≈ κ .92–1.00

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Suicidal behavior κ 0.933≈ κ .87–.99projected

Suicidal ideation, attempts, or self-harm by a child or caregiver noted in the narrative.

Projected · not independent evidence
≈ κ .87–.99

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Serious mental illness κ 0.880≈ κ .82–.94projected

Presence of a severe psychiatric condition (for example psychosis, bipolar disorder, major depression) affecting caregiving capacity.

Projected · not independent evidence
≈ κ .82–.94

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Perinatal mental health κ 0.824≈ κ .76–.88projected

A birthing or postpartum caregiver experiencing a mental-health condition — depression, anxiety, or psychosis — during pregnancy or the first year after birth.

Projected · not independent evidence
≈ κ .76–.88

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Developmental delay κ 0.738≈ κ .68–.80projected

A child with a developmental delay, intellectual or developmental disability, or special-education need affecting cognitive, motor, speech, or adaptive development, beyond a bare autism or ADHD mention.

Projected · not independent evidence
≈ κ .68–.80

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

04 Child & family
Substance-exposed infant κ 0.920≈ κ .86–.98projected

Prenatal substance exposure identified at or near birth.

Projected · not independent evidence
≈ κ .86–.98

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Truancy κ 0.864≈ κ .80–.92projected

Chronic school absenteeism by a child.

Projected · not independent evidence
≈ κ .80–.92

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Child chronic illness κ 0.864≈ κ .80–.92projected

A child with a long-term medical condition requiring ongoing care.

Projected · not independent evidence
≈ κ .80–.92

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Caregiver incarceration κ 0.863≈ κ .80–.92projected

A parent or caregiver is in jail or prison, affecting caregiving continuity.

Projected · not independent evidence
≈ κ .80–.92

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Caregiver criminal-justice involvement κ 0.933≈ κ .87–.99projected

A caregiver's involvement with the criminal-justice system as an offender — arrest, charges, probation or parole, warrants, or incarceration — broader than incarceration alone.

Projected · not independent evidence
≈ κ .87–.99

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

05 Environmental / event
COVID-19 κ 0.954timeline

References to the pandemic and its effects on the household or case.

External validation
The construct reproduces the 2020–21 pandemic arc: near-zero before 2020, peak in 2021.
06 Substance use
Opioid replacement therapy κ 0.964ρ +0.84

Mentions of medication-assisted treatment for opioid use disorder (methadone, buprenorphine, naltrexone) involving a caregiver or household member.

External validation
ρ +0.84 against SAMHSA medication-assisted-treatment (MAT) admissions, state-year.
Opioid involvement κ 0.957ρ +0.95 · κ .98

Any indication that opioids, prescription or illicit, are implicated in the case, spanning use, exposure, or treatment.

External validation
ρ +0.95 against CDC opioid-overdose deaths (time series); κ .98 on an independent benchmark (Qi et al., 2026).
Fentanyl κ 0.936ρ +0.75

Specific references to fentanyl or synthetic-opioid involvement, a marker of elevated overdose risk in the household.

External validation
ρ +0.75 against CDC synthetic-opioid overdose deaths.
Crack / cocaine κ 0.934≈ κ .87–.99projected

References to crack or powder cocaine use or possession by a caregiver or household member.

Projected · not independent evidence
≈ κ .87–.99

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Methamphetamine κ 0.916≈ κ .86–.98projected

References to methamphetamine use, manufacture, or exposure, including contamination concerns for children in the home.

Projected · not independent evidence
≈ κ .86–.98

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Substance-related problems κ 0.887κ .94

A broad indicator that substance use is contributing to caregiving impairment or child-safety concerns, independent of the specific substance.

External validation
κ .94 on an independent benchmark (Qi et al., 2026).
Opioid active use κ 0.874≈ κ .81–.93projected

Current, ongoing opioid use as distinct from history or treatment, indicating present risk in the household.

Projected · not independent evidence
≈ κ .81–.93

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Cannabis κ 0.851ρ −0.95

References to cannabis use by a caregiver; interpretation shifts with the legalization era.

External validation
ρ −0.95: SAMHSA treatment admissions fell as legalization-era mentions rose.
Alcohol κ 0.831≈ κ .77–.89projected

References to alcohol misuse by a caregiver or household member affecting child safety.

Projected · not independent evidence
≈ κ .77–.89

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.

Overdose κ 0.864≈ κ .80–.92projected

An acute drug, medication, or alcohol overdose event involving a case-relevant person — accidental or intentional, fatal or non-fatal — as distinct from general substance use.

Projected · not independent evidence
≈ κ .80–.92

Internal-anchored placeholder (internal κ ±.06, capped at 1.0). It cannot corroborate the internal figure because it is derived from it; it marks a construct awaiting external validation.