Scoring: QLoRA decoder fine-tuning

• Open weight decoder scoring
• Fine-tuning Phi-4-mini
• Relevance to hiring simulations
• Method
• Scoring
• Zero shot scoring baseline
• Fine-tuning
• Quantization
• Adaptor rank
• Clipping and learning rates
• Adaptor placement
• Evaluating model accuracy
• Results
• Discussion
• Score precision
• References

Open weight decoder scoring

In this section we implement decoder-only fine-tuning via QLoRA for constructed response scoring from the ground up. This method has relevance to scoring many forms of hiring simulation from free text personality assessments, constructed response SJTs and video interviews, as discussed later in this section. If you want to run this analysis yourself you can download our code for this analysis.

Parameter efficient fine-tuning (PEFT) of decoder models on GPUs is now feasible via application programming interfaces (APIs) for some closed weight models, or by using cloud compute and even local GPUs in the case of open weight models. Keep in mind, you are limited to using the model within the vendor infrastructure if you take the API fine-tuned route. For early demonstrations of these techniques readers can see Latif & Zhai (2023).

Fine-tuning Phi-4-mini

For this demonstration, we will re-implement a slice of the work reported by Ormerod & Kwako (2024) without reviewing their code. We call this a re-implementation, rather than a replication, because we demonstrate a slice of the analyses they presented and we make several deliberately different design choices that test the underlying ideas, rather than re-run their exact code. Lykken (1968) described this as constructive replication.

We will choose one question out of the AES essays, question four, and we will check the zero shot performance and compare it to the fine-tuned performance using Microsoft Phi-4-mini. We will use quadratic weighted kappa (QWK) for evaluation, which is appropriate given the ordinal nature of the data and was the metric reported in the original article. The paper itself presents results for more models, test questions and explores feedback generation. Readers can also refer to the paper directly.

Different choices include that first, the authors used Microsoft Phi-3 and we use the next-generation Microsoft Phi-4 model. Second, the authors presented a single 60%, 20%, 20% split, and we will include 5-fold validation of the zero-shot and fine-tuned version of the Phi-4 model. Third, the authors use a train, valid, test split based on a widely used data split in the literature. We instead use a new random fold composition we created by resampling the essays

Relevance to hiring simulations

This fine-tuning demonstration uses student essay scoring from a prominent benchmark data set used to evaluate scoring accuracy for new methods. It is chosen for this demonstration because it is a well understood data set. The extent to which these methods generalize to hiring assessments depends on several factors, but there is reason for optimism that they are relevant for measuring latent constructs in talent assessment.

First, the rubric oriented scoring is similar in spirit to many SJTs and interviews where candidate responses are expected to represent graded levels of the measured construct and early work suggests LLMs are effective here (e.g. Stockdale et al., 2026). Moreover, while constructed response personality assessments are not always scored against a rubric, fine-tuning of similar models has suggested they work for personality assessment (e.g. Speer et al., 2026) and zero-shot personality scoring models have been shown to work in clinical settings (Wright et al., 2026).

Of course, whether the efficacy of these methods generalizes to hiring assessments is ultimately an empirical question and must be demonstrated on hiring data, we can’t simply generalize from benchmarks. There are other considerations that must also be adequately addressed before deployment relating to reliability, validity and adverse impact.

Method

Scoring

The input to the zero-shot and fine-tuned models includes an instruction telling the LLM that it is a grading assistant and needs to assign a score between the minimum and maximum possible score for that essay, followed by the rubric for the question and the students’ responses. The model completes the prompt that ends ‘Assistant: Score:’ in the same way it would complete any other sentence, as opposed to producing a score in the way that a regression model would. In the case of essay set 4, which we use for our demo, that value is a single token but other essays would require multiple tokens if the maximum upper score was over 9. The model assigned a score of zero as a fallback if it produced an impossible score.

Zero shot scoring baseline

The base model first scored all test essays on each fold's held-out test sample using the same prompt template. This zero-shot scoring required no training or gradient updates and was intended to see how well Phi-4-mini-instruct can score essays using only the rubric in the prompt with no knowledge of the scores of sample essays. The same test set was then scored again by the fine-tuned model and the performance of the two models was compared.

Fine-tuning

We fine-tuned Phi-4-mini-instruct. This model is a 3.8 billion-parameter (a 14B-parameter version is also available) open-weight model from Microsoft trained on high-quality data to follow chat-style instructions. The fine-tuning used QLoRA with 4-bit NormalFloat (NF4) quantization of the pre-trained frozen model weights that Microsoft released with rank-32 LoRA adaptors.

Quantization

NF4 precision means Phi-4 weights are coded in lower precision in a way that matches the distribution of weights rather than being evenly spaced. This allows the model to fit and train on a single GPU. Adapters are specified as trainable low-rank matrices alongside linear layers and kept in high precision. In training and inference, frozen weights are dequantized on the fly per matrix operation and discarded. The fine-tuning training configuration is presented in table 1.

Adaptor rank

Adaptor rank controls how many trainable parameters the adapters add. Higher rank means more capacity but also means higher memory and compute cost. Alpha scales the adapter's influence relative to the frozen weights. The alpha/rank ratio sets the learning strength. Here we choose r=32, alpha=32, following Dettmers et al. (2023), who found that LoRA rank has little effect on performance once adapters are applied to all linear layers.

Clipping and learning rates

Training did not use gradient clipping to limit the size of individual parameter updates, and no learning-rate warmup was used. The full learning rate was applied from the first step. Both stabilize training but they were not needed because the worst loss spikes were moderate and always recovered within a few steps and there were no fallback predictions. However, clipping and learning rates are important for stabilizing training where spiky loss trajectories occur.

Adaptor placement

The common LoRA approach places adaptors only on the attention layers' query and value projections. Dettmers et al. found that this approach did not match full 16-bit fine-tuning performance and LoRA applied to every linear layer in the transformer block helped close the gap to full 16-bit fine-tuning performance. Our implementation here follows the approach of placing adaptors on all linear layers.

LoRA works because adapting LLMs to new tasks does not require changing all parameters. Task-specific information can be acquired by a small number of low-rank update matrices and keeping the original pre-trained weights frozen. QLoRA combines this idea with 4-bit quantization of the frozen weights to reduce the memory requirements but at the same time almost matches full fine-tuning.

Evaluating model accuracy

We used five-fold cross-validation. In each fold, 60% of the cases are used for training, 20% of cases are held out as the development/validation set and the last 20% is used as the test set. At the end of each of the 10 training epochs, the model is scored on the development set; the checkpoint with the best development-set QWK is selected and evaluated once on the held-out test set. This is repeated for all five folds.

Results

This QLoRA fine-tuning demonstration of Phi-4-mini on the 1770 question 4 essays of AES, evaluated over five folds, produced a mean QWK of .82 (SD .016). The result included zero prediction failures (e.g., out-of-bounds predictions). The Phi-4-mini zero-shot baseline, using the same rubric and essays, produced a mean QWK of .47. Both of these values are close to the Phi-3-mini values reported by Ormerod and Kwako (2024) in their paper. Readers are recommended to consult this paper for more details of their implementation.

Of interest is that the fine-tuned result in this case produced a QWK slightly higher than the NPCR QWK for this question, which is in contrast to the scores reported between the original NPCR Do et al (2024) paper than the Ormerod and Kwako (2024) paper. Two important points need to be noted about this result. First, the difference between the results is very small. Second, rather than saying something about the relative efficacy of regression versus decoder architectures, this is likely a result of the Phi-4-mini generation architecture, which is a newer model than that used by Ormerod and Kwako.

Discussion

The question remains, why bother fine-tuning a small open model when a closed model reaches similar performance out-of-the-box? The answer is fine-tuning a 3- to 4B open model avoids API charges, keeps candidate data on your own servers, and gives model weights that API vendors cannot deprecate. We also found that the fine-tuned open model was more accurate than the closed model. Phi-4-mini reached a QWK of .824 on essay set 4, which is close to the result we showed earlier using GPT-4 on the same item.

Phi-4-mini is trained as a chat model and expects a signal for where the user's turn (i.e., to provide the rubric, scale, and essay) ends and the assistant's (i.e., scorer) turn begins. We appended the text, \n\nAssistant: Score: to the prompt at the end of the essay to show this boundary, but a template can be used to insert special tokens to do this. Training ran with no gradient clipping and no learning-rate warmup. These techniques would be easy to add and may become useful as the scoring task gets more complex.

Score precision

There are several ways to obtain a score precision estimate. One is to run the model on the test essays, subtract each essay's predicted score from its observed score, and calculate the standard deviation of the prediction errors. Under the assumption that the predictions and observed scores are parallel forms of the same construct, another is to use their correlation as an estimate of reliability and substitute it into the classical formula. If repeated cross-validation or multiple model runs are used, the variability in predicted scores for the same essay can give an empirical indication of prediction uncertainty. These estimates can be used to construct confidence intervals around predicted scores, provided assumptions are met.

References

Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151.

Ormerod, C., & Kwako, A. (2024). Automated Text Scoring in the Age of Generative AI for the GPU-poor. arXiv:2407.01873.

Latif, E., & Zhai, X. (2023). Fine-tuning ChatGPT for Automatic Scoring. arXiv:2310.10072. Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs.

Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models.

Microsoft (2025). Phi-4-mini Technical Report.

Speer, A. B., Delacruz, A. Y., Chawota, T. A., Perrotta, J., & Rudolph, C. W. (2026). Unpacking the validity of open-ended personality assessments using fine-tuned large language models. Organizational Research Methods, 10944281251413746.

Stockdale, K., Hickman, L., & Liu, S. (2026). Scoring employment interviews with large language models: Evaluation design components, validity investigations, and best practice recommendations. Journal of Applied Psychology.

Wright, A. G., Ringwald, W. R., Vize, C. E., Eichstaedt, J. C., Angstadt, M., Taxali, A., & Sripada, C. (2026). Assessing personality using zero-shot generative AI scoring of brief open-ended text. Nature human behaviour, 10(3), 541-555.

Next section

Negative control: If you can’t simulate, destroy

Last section

Scoring: Autoregressive multi-trait scoring (ArTS)

Return home

Psychometrics.ai