LLM Themes Are Not Observations

themes from a name corpus to the shopper desk. Clients with out transcripts get NULL. NULL will get stuffed with zero, or with “no problem talked about,” or quietly omitted as a reference class. In a single line of preprocessing, the pipeline converts didn’t name help into didn’t expertise billing frustration.

The regression that follows appears to be like clear. The coefficient on “billing frustration” is critical, signed the way in which the product group anticipated, giant sufficient to matter. It will get pasted right into a roadmap doc. No one asks the place the variable got here from.

This text is about what obtained smuggled in with that fill worth, and about three different strikes that look simply as innocuous in a pocket book however relaxation on assumptions the evaluation by no means names. The setup isn’t particular to help calls. It applies to speak logs, ticket summaries, product critiques, gross sales transcripts, and survey free-response fields, anyplace a contemporary pipeline turns textual content right into a tidy column. The pipeline might be a fine-tuned classifier, a zero-shot LLM, or an embedding-plus-cluster. The conceptual downside is identical: the column isn’t an commentary of a buyer attribute. It’s the output of a generative course of utilized to a self-selected subset of buyer habits.

Practitioners more and more deal with outputs like these as in the event that they had been direct readings of buyer state. They don’t seem to be. They’re generated variables: measurements produced by a pipeline, conditional on a buyer doing one thing that left a textual hint, conditional on that hint surviving the extraction mannequin. Each step of that conditional has penalties for what the variable means in a downstream causal mannequin, and most of these penalties are invisible within the joined desk.

4 issues are likely to go incorrect, and the NULL transfer makes all 4 seen without delay.

Choice. A theme exists for a buyer as a result of that buyer known as, complained, posted, or replied. No matter drove that motion can be in all probability correlated with the remedy, the result, or each. The NULL fill collapses “didn’t generate textual content” into the reference class, and the evaluation is now not estimating an impact over the shopper base. It’s estimating an impact over a redefined inhabitants, and the redefinition occurred in preprocessing.

Timing. Was the decision earlier than the remedy, throughout it, or after? Pre-treatment textual content is a candidate confounder. Publish-treatment textual content is a candidate mediator or final result, and treating it as a pre-treatment management is a basic supply of post-treatment bias. The joined desk hardly ever makes this seen.

Measurement. The label “billing frustration” isn’t billing frustration. It’s what the pipeline detected as billing-frustration-shaped language. Classifier accuracy is finite, and accuracy can differ throughout remedy arms, as a result of a remedy that adjustments how clients discuss additionally adjustments how the mannequin reads them. The label noise isn’t orthogonal to the factor being studied.

Position. Is the theme performing as a confounder, a mediator, a remedy, an final result, or a descriptive characteristic? The DAG decides this, not the column title. A variable that’s methodologically legitimate in a single function turns into a bias supply in one other.

These 4 issues aren’t impartial. They work together. An LLM-detected theme inherits a range footprint from the channel it got here by way of, a timing footprint from when the textual content was generated, and a measurement footprint from the pipeline that extracted it. The downstream regression sees a column of zeros and ones.

The issue isn’t that the pipeline produced a nasty label. The issue is that the label inherited a data-generating course of the downstream evaluation by no means modeled.

The remainder of this text works by way of what which means in observe, the place the usual workflow goes incorrect, and what the minimal diagnostic appears to be like like. We begin with the role-and-timing query, as a result of it’s the one analysts get incorrect first.

Position and timing are the identical query

The primary transfer an analyst makes with a transcript-derived theme is implicit: they deal with it as a covariate. Themes go into the right-hand aspect of the regression. The remedy is the variable of curiosity. The end result is on the left. The theme is “managed for.”

That phrase, “managed for,” is doing work the analyst hasn’t checked. Controlling for a variable adjusts away the a part of the treatment-outcome relationship that flows by way of it. Whether or not that adjustment helps or hurts relies upon totally on the place the variable sits within the causal graph, and that place is decided by timing.

Pre-treatment textual content, generated earlier than the remedy was assigned, can play the function of a confounder. If a buyer known as about billing in January and the retention supply went out in March, the decision captures one thing about buyer state that will affect each who obtained the supply and who churned. Conditioning on the theme right here can scale back bias from omitted variables, offered the theme truly proxies for the related assemble and the choice points within the subsequent part are dealt with.

Concurrent textual content, generated as a part of the remedy itself, isn’t a covariate in any respect. If the remedy is a name from a retention agent and the theme comes from that very same name, the theme is a part of the intervention. Conditioning on it doesn’t modify for confounding; it removes a part of the impact the analyst is attempting to measure.

Publish-treatment textual content, generated after the remedy, is essentially the most harmful class, as a result of it’s the one most probably to be misclassified as a confounder by an analyst working from a flat desk with no time index. A buyer who acquired a retention supply in March and known as complaining in April produced a transcript that displays, at the very least partially, their response to the remedy. Conditioning on a theme extracted from that decision is conditioning on a post-treatment variable. That may block mediation paths, induce collider associations, or in any other case shift the estimand away from the remedy impact the analyst thinks they’re estimating.

A labored instance makes this concrete. Think about an artificial however business-realistic setup. Clients are focused right into a retention supply based mostly on a mannequin that picks up value sensitivity. Each the supply task and buyer churn depend upon this underlying value sensitivity, which the analyst doesn’t observe. Clients who’re extra price-sensitive usually tend to obtain the supply (as a result of the focusing on mannequin chosen them) and extra more likely to churn regardless. They’re additionally extra more likely to name help and categorical invoice shock. The theme “invoice shock” is generated from these post-treatment calls.

The naive analyst joins the theme onto the shopper desk, fills NULL as zero, and runs a logistic regression of churn on supply plus bill-shock:

import numpy as np
import pandas as pd
import statsmodels.api as sm
 
rng = np.random.default_rng(7)
n = 20000
 
price_sens = rng.regular(0, 1, n)
supply = rng.binomial(1, 1 / (1 + np.exp(-(0.8 * price_sens))))
churn = rng.binomial(1, 1 / (1 + np.exp(-(-1.0 + 1.2 * price_sens - 0.5 * supply))))
known as = rng.binomial(1, 1 / (1 + np.exp(-(-1.5 + 0.7 * price_sens + 0.9 * churn))))
 
theme_prob = 1 / (1 + np.exp(-(-0.5 + 0.8 * price_sens)))
bill_shock = np.the place(known as == 1, rng.binomial(1, theme_prob), 0)
 
df = pd.DataFrame({"churn": churn, "supply": supply, "bill_shock": bill_shock})
 
X = sm.add_constant(df[["offer", "bill_shock"]])
naive = sm.Logit(df["churn"], X).match(disp=0)
print(naive.params)

The true impact of the supply on churn is −0.50 in log-odds. The supply is meant to cut back churn, and within the data-generating course of it does. Here’s what 4 specs return:

As a result of supply task is already confounded by value sensitivity, eradicating the dangerous management doesn’t make the design legitimate. It solely removes one extra supply of distortion. Two observations from this desk.

First, the naive specification is incorrect in path. Including the bill-shock management to a mannequin that was already biased flips the signal on the supply coefficient. The product group studying this output would conclude that retention affords trigger churn. They might be incorrect.

Second, dropping the bill-shock variable doesn’t repair the evaluation. The dropped specification can be optimistic, and solely the oracle specification, which situations on the unobserved confounder straight, recovers the true impact. In an actual evaluation the analyst doesn’t have that column. That’s the level. Eradicating a nasty management is important however not enough, and a post-treatment theme extracted from a self-selected calling subpopulation isn’t an alternative choice to identification.

The mechanism behind the signal flip within the naive specification is price strolling by way of. Churn impacts the probability of calling, as a result of clients who’re leaving usually tend to name. Invoice-shock is simply noticed for purchasers who known as, for the reason that theme requires a transcript to exist. Conditioning on bill-shock due to this fact situations on a downstream consequence of churn. Amongst clients with bill-shock equal to at least one, the connection between supply and value sensitivity has been distorted, as a result of each variables now assist clarify why the shopper ended up flagged. The coefficient on supply absorbs that induced affiliation.

The methodological level generalizes. A transcript-derived variable has a place within the causal graph decided by when the textual content was generated relative to the remedy, who generated it, and what course of produced the label. Position and timing are the identical query seen by way of totally different lenses. These variables include a structural footprint the analyst is answerable for tracing, and the joined desk isn’t the place the tracing occurs.

The choice query

Most business analyses utilizing help transcripts implicitly redefine the inhabitants from “clients” to “clients who generated help language.” The estimand adjustments earlier than the regression even begins.

That is the half that tends to matter most in practitioner workflows, and it’s the place the usual workflow is most fragile.

The textual content exists as a result of the shopper did one thing: known as, posted, complained, replied. That one thing is a habits, not a measurement. It’s influenced by buyer traits, by the channel that was out there, by the urgency of the underlying problem, and sometimes by the remedy itself. None of those are random. None are usually orthogonal to the result.

The NULL dealing with determination is the place this turns into operational. There are three widespread strikes, and every carries an assumption.

Filling NULL as zero or “no problem talked about” assumes that not producing textual content is informative concerning the absence of the underlying assemble. The analyst is claiming that clients who didn’t name didn’t expertise the factor the theme is detecting. For many themes price detecting, that is implausible on its face. Clients who didn’t name might have skilled billing frustration and resolved it by canceling, by switching to a competitor, by complaining on social media, or by giving up. The zero-fill turns all of those into “no frustration.”

Dropping rows with NULL themes, proscribing the evaluation to the calling subpopulation, is at the very least trustworthy concerning the inhabitants, but it surely adjustments the estimand. The remedy impact amongst clients who known as isn’t the remedy impact amongst clients, and the distinction between the 2 is commonly all the level of the enterprise query. A retention supply’s impact on churn-prone callers is a helpful amount. It isn’t the amount most analyses declare to estimate.

Treating text-presence as a missingness mechanism and making use of inverse chance weighting based mostly on a mannequin of who calls is, methodologically, the correct form of transfer. The catch is the propensity mannequin itself. Modeling who generates textual content requires writing down what drives calling, and that mannequin is determined by demographics, tenure, prior points, remedy publicity, and unmeasured frustration, which is the assemble the theme was supposed to assist measure within the first place. The IPW transfer is principled, and additionally it is hardly ever as principled because it appears to be like.

The deeper level is that choice into textual content is a habits that interacts with the remedy. A retention supply might change calling charges. A pricing change might change criticism charges. A characteristic launch might change the sorts of points clients articulate. Any of those makes the choice mechanism itself treatment-dependent, which suggests even a superbly extracted, completely timed theme is being measured on a inhabitants whose composition shifts with the remedy. Commonplace observational corrections assume the choice mechanism is steady. When the remedy strikes the choice, the corrections don’t.

None of this implies transcript-derived variables are ineffective. It means the analyst owes the reader an express assertion of which inhabitants the evaluation is estimating an impact over, what mechanism produced the textual content, and what assumption was made about everybody whose textual content doesn’t exist.

The measurement query

Previous NLP outputs seemed noisy. TF-IDF weights, sparse key phrase counts, LDA matter vectors: none of them seemed like issues a buyer felt. Practitioners distrusted them by reflex, and that reflex saved a number of dangerous analyses.

LLM outputs don’t look noisy. They appear like latent constructs. A label like “billing frustration” or “belief erosion” or “renewal nervousness” reads like an outline of a buyer’s psychological state. The label is articulate, the classes are semantically coherent, and the failure modes don’t announce themselves within the column. The persuasion downside is actual earlier than the statistical downside begins.

The statistical downside is extra acquainted. An LLM theme is a loud proxy for the underlying assemble. The label “invoice shock” isn’t invoice shock. It’s what the mannequin determined was bill-shock-shaped language within the transcripts it processed. Classifier accuracy is finite even for clear duties, and the accuracy on the precise inhabitants, not the held-out analysis set, is commonly unknown. Plugging a loud proxy right into a regression instead of the true variable attenuates coefficients towards zero in some setups and distorts them in others, relying on whether or not the noise is differential.

Differential measurement error is the place the actual injury lives. If a remedy adjustments how clients discuss, and most therapies price operating do, then the classifier’s accuracy on theme detection can differ between remedy and management. A retention supply that softens buyer sentiment might scale back the speed at which the mannequin flags “invoice shock” language with out lowering the underlying frustration. A pricing change that shifts how clients articulate complaints might transfer classifier accuracy extra in a single arm than the opposite. The label noise is now not mean-zero. It’s correlated with the remedy, and conditioning on the noisy label biases the estimated remedy impact in a path the analyst can’t simply signal.

There’s a literature on correcting for classifier-induced measurement error. Egami and colleagues develop a split-sample workflow for causal inference with text-discovered measures as therapies or outcomes in “How one can Make Causal Inferences Utilizing Texts”. Mozer and colleagues apply text-augmented matching to digital well being data and present how text-based covariates change estimated results in an actual medical examine in “Leveraging textual content information for causal inference utilizing digital well being data”. For the broader panorama, Keith, Jensen, and O’Connor assessment how textual content has been used to take away confounding throughout purposes in “Textual content and Causal Inference: A Evaluation of Utilizing Textual content to Take away Confounding from Causal Estimates”. These strategies exist, and they’re price utilizing when the evaluation issues. Additionally they require the analyst to acknowledge {that a} label is a measurement with error within the first place, which is the transfer most workflows skip.

The practitioner mistake isn’t utilizing the label. The practitioner mistake is treating a label that got here out of a generative mannequin as if it had been a column learn off a sensor.

A practitioner guidelines

A causal evaluation that makes use of a generated variable derived from transcripts can nonetheless be defensible. It simply has to reply 5 questions earlier than the regression runs.

1. What function am I assuming this variable performs?

Confounder, mediator, remedy, final result, or descriptive characteristic. The DAG decides. The column title doesn’t.

2. When was the textual content generated relative to the remedy?

Pre-treatment, concurrent, or post-treatment. If the analyst can’t reply this from the info, the variable doesn’t enter the mannequin as a confounder.

3. What choice mechanism produced the textual content, and what am I assuming about everybody whose textual content doesn’t exist?

Zero-fill, drop, IPW: every is an assumption. Choose one and state it.

4. How was the label produced, and will its reliability differ throughout remedy arms?

If the remedy plausibly adjustments how clients categorical the underlying assemble, classifier accuracy isn’t fixed throughout the comparability the evaluation is making.

5. What does the consequence appear like beneath a stress check?

Refit the mannequin with out the transcript-derived variable. If the headline coefficient is fragile, the consequence isn’t steady sufficient to hold a causal declare by itself.

These 5 questions aren’t an answer. They’re a diagnostic. An analyst who can reply them isn’t assured an recognized impact. An analyst who can’t reply them is doing descriptive work with causal language connected.

The broader sample is older than LLMs. Generated variables are pipeline outputs that appear like observations however are literally mannequin outputs conditioned on choice. They present up in fraud scores, recommender relevance metrics, sentiment indices, propensity scores reused as covariates, and any latent-trait estimate produced by an upstream mannequin and consumed by a downstream evaluation. LLMs didn’t invent this error. They made it accessible at a scale and a fluency that older NLP outputs by no means reached. The labels appear like latent constructs, the columns appear like measurements, and the workflow appears to be like like causal inference.

The assumptions didn’t disappear. They only moved upstream.

Employees Information Scientist targeted on causal inference, experimentation, and determination science. I write about turning ambiguous enterprise questions into decision-ready evaluation.

Extra like this on LinkedIn 👇

🔗 LinkedIn