A new academic paper proposes using psychological assessment tools to create behavioral fingerprints of large language models. Titled "Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling" and submitted on September 19, 2026, the research applies seven established psychological instruments to nine different LLMs across Chinese and English, with five repeated administrations per model and language. The goal is to systematically characterize the behavioral regularities that have been difficult to measure with conventional benchmarks.
What the study did
The researchers developed a cross-linguistic profiling framework and tested it on nine LLMs. Each model was administered seven psychological instruments five times in each of two languages, Chinese and English. When an item could not be resolved after a predefined retry procedure, the response was recorded as NA rather than forcing a categorization. The study then performed a joint analysis of both scored responses and NA responses, capturing not only what the models answered but also where they refused to answer or could not map their output to a valid response option.
This approach addresses a gap in how LLMs are evaluated. Traditional benchmarks measure accuracy, reasoning, or coding ability, but they do not systematically capture the behavioral tendencies that shape how a model interacts with users in real deployments. Psychometric instruments, long used in human psychology to measure traits like agreeableness, conscientiousness, and emotional stability, provide a structured framework for identifying similar patterns in AI systems.
What the profiles revealed
Despite variations between individual models, all nine LLMs shared a common pattern shaped by alignment training. They exhibited higher prosocial and self-regulatory responses and lower scores on dominance, disengagement, and harmful-intent endorsement. This shared profile suggests that the alignment process produces a recognizable behavioral signature across different providers and model families.
Beneath that shared pattern, however, each model maintained a distinct profile. The structured nature of these model-specific signatures means that psychometric profiling could potentially serve as a method for identifying which model produced a given output, even when the output itself does not contain obvious markers of origin.
The NA responses turned out to be informative rather than random noise. They were structured rather than uniformly distributed, indicating specific places where models treat outputs as inapplicable, refuse to respond, or cannot map the question to a valid response category. These refusal patterns are themselves a behavioral signal, revealing the boundaries of what each model considers answerable.
The role of language and provider
The study found that language condition and provider origin both influenced profile configuration and answerability. Models behaved differently in Chinese versus English, and responses varied depending on which provider built the model. This means behavioral signatures are context dependent, and a profile created in one language does not necessarily transfer to another.
Repeated administrations showed high reproducibility, which is significant for practical use. If a model's psychometric profile is stable across multiple test runs, the profiling method can reliably identify the model. The researchers confirmed that model identity could be recovered from the profiling data, even though the models were never explicitly labeled during the evaluation.
Why this matters for deployment
The paper's conclusion is that the joint analysis of psychometric profiling and answerability offers a framework for quantifying deployment-level behavioral signatures. In practical terms, this means that organizations deploying LLMs could characterize not just what a model does, but how it tends to behave across different contexts, languages, and question types.
The human-reference and prompt-robustness analyses add another layer of caution. The behavioral signatures are not fixed properties of a model. They shift depending on the prompt structure and the context in which the model is evaluated. A model that appears highly prosocial in one testing configuration may behave differently when prompted differently or deployed in a different setting.
The practical implications
For teams deploying LLMs in production, the research suggests that understanding a model's behavioral profile is as important as measuring its accuracy on standard tasks. The NA response patterns in particular highlight where a model will refuse to engage, which is critical in high-stakes applications where a refusal to act can be as consequential as an incorrect action. The framework also opens the door to auditing and comparing models not just on what they produce, but on the consistent behavioral patterns that emerge across repeated interaction.