Scoring: QLoRA decoder fine-tuning

• Open weight decoder scoring
• Fine-tuning Phi-4-mini
• Reasons to anticipate approach can be used in hiring simulations
• Method
• Scoring
• Zero shot scoring baseline
• Fine-tuned scoring comparison
• Evaluating model accuracy
• Results
• Discussion
• References

Open weight decoder scoring

In this section we implement from the ground up a decoder-only fine-tuning via QLoRA for constructed response scoring. This method has relevance to scoring many forms of hiring simulation from free text personality assessments, constructed response SJTs and video interviews as discussed later in this section. If you want to run this analysis yourself you can download our code for this analysis.

Fine-tuning of decoder models on GPUs is now feasible via application programming interfaces (APIs) (for some closed weight models or by using cloud compute and even local GPUs in the case of open weight models. Keep in mind you are limited to using the model within the vendor infrastructure if you take the API fine tuning route. For early demonstrations of these techniques readers can see Latif & Zhai (2023).

Fine-tuning Phi-4-mini

For this demonstration, we will re-implement a slice of the work reported by Ormerod & Kwako (2024) without reviewing their code. We will choose one question out of the AES essays, question four data, and check the zero shot performance and fine-tuned performance using Microsoft Phi-4-mini against the Quadratic Weighted Kappa for this combination reported in the original article. The paper itself presents results for many models and test questions and readers can also refer to it directly.

Importantly, there are several differences between what the paper did and what we do here that are deliberate and intended to present a robust check on the efficacy of their proposed process. First, the authors used Phi-3 and we use the next generation Phi-4 model. Second, the authors presented one 60, 20, 20 split and we will include 5-fold validation of the zero-shot and fine-tuned version of the Phi-4 model. Third, the authors use a train, valid, test split based on a widely used split in the literature. We instead use a new random fold composition based on resampling of the essays.

Reasons to anticipate approach can be used in hiring simulations

These essays are student essays from a prominent benchmark data set used to evaluate scoring accuracy for new methods and have been chosen for this demonstration for that reason. The extent to which these methods generalize to hiring assessments depends on several factors but there is reason for optimism. First, the rubric oriented scoring is similar in spirit to many SJTs and interviews where candidate responses are expected to represent graded levels of the measured construct. While constructed response personality assessments are not typically against a rubric, fine tuning of similar models has suggested they work. Of course, whether these methods generalize to hiring assessments is ultimately an empirical question and must be demonstrated on hiring data, we can’t simply generalize from benchmarks.

Method

Scoring

The input to the zero shot and fine tined models model includes an instruction telling the LLM that it is a grading assistant and needs to assign a score between the minimum and maximum possible score for that essay, followed by the rubric for the question and the students’ responses.

The model completes the prompt that ends ‘Assistant: Score:’ in the same way it would complete any other sentence as opposed to producing a score in the way that a regression model would. In the case of essay set 4, which we use for our demo, that value is a single token but other essays would require multiple tokens if the maximum upper score was over 9. The model assigned a score of zero as a fallback if an impossible score was assigned.

Zero shot scoring baseline

The base model first scored all test essays on each fold's held-out test sample using the same prompt template. This zero-shot scoring required no training or gradient updates and was to see how well Phi-4-mini-instruct can score essays using only the rubric in the prompt with no knowledge of the scores of sample essays. The same test set is then scored again later by the fine-tuned model and the two scores are compared.

Fine-tuned scoring comparison

We fine-tuned Phi-4-mini-instruct. This model is a 3.8 billion parameter (a 14B parameter version is also available) open weight model from Microsoft trained with high quality data to follow chat style instructions. The fine tuning used QLoRA with 4-bit NormalFloat (NF4) quantization of the pre-trained frozen model weights that Microsoft released with rank-32 LoRA adaptors applied to every linear layer rather than just the attention projections to help the model learn the task. The exact fine-tuning training configuration is described in table 1.

Evaluating model accuracy

We used five-fold cross-validation. In each fold 60% of the cases are used for training, 20 percent of cases are held out as the development/validation set and the last 20% is used as the test set. At the end of each of the 10 training epochs, the model is scored on the development set; the checkpoint with the best development-set QWK is selected and evaluated once on the held-out test set. This is repeated for all five folds.

Results

This QLoRA fine-tuning demonstration of Phi4-mini on the 1770 question 4 essays of AES over five folds produced a mean QWK of .82 (SD .016). The result included zero prediction failures (e.g. out of bound predictions). The Phi-4 mini zero shot baseline used the same rubric and essays produced a mean QWK of .47. Both of these values are close to the values reported by Ormerod and Kwako (2024) fo this model in their paper, which fine-tuned more models and tested more items. Readers are recommended to consult this paper for more details of their implementation.

Of interest is that the fine-tuned result in this case produced a QWK slightly higher than the NPCR QWK for this question, which is in contrast to the scores reported between the original NPCR Do et al (2024) paper than the Ormerod and Kwako (2024) paper. Two important points to note about this are that the difference between the results is very small. Second, rather than saying something about the relative efficacy of regression versus decoder architectures, this is likely a result of the newer generation Phi-4-mini architecture which is a new model than that used by Ormerod and Kwako.

Discussion

The question remains why bother fine-tuning a small open model when a closed model reaches similar performance out-of-the-box. The answer is fine-tuning a 3 to 4B open model avoids API charges, keeps candidate data on your own servers, and gives model weights that API vendors cannot deprecate. We also say fine tuning gets the open model more accurate than the closed model, phi-4 fin-tuned reached .824 on essay_set 4 beating GPT-4's zero-shot .784 on the same item.

Phi-4-mini is trained as a chat model and expects a signal for where the user's turn (i.e., to provide the rubric, scale, and essay) ends and the assistant's (i.e., scorer) turn begins. We put plain the text, \n\nAssistant: Score: in the prompt at the end of the essay to show this boundary but a template can be used to insert special tokens to do this. Training ran with no gradient clipping and no learning-rate warmup. These updates would be easy to add and may become useful as the scoring task gets more complex.

There are several ways to obtain an analogue of the classical standard error of measurement. One is to run the model on the test essays, subtract each essay's predicted score from its actual score, and take the standard deviation of the differences. A second is to take the correlation between the model's predictions and the actual scores and substitute it into the classical SEM formula. A third, if repeated cross-validation or multiple model runs are used, is to examine the variability in predicted scores for the same essay. This variability provides an empirical indication of prediction uncertainty.

References

Ormerod, C., & Kwako, A. (2024). Automated Text Scoring in the Age of Generative AI for the GPU-poor. arXiv:2407.01873.

Latif, E., & Zhai, X. (2023). Fine-tuning ChatGPT for Automatic Scoring. arXiv:2310.10072. Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs.

Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models.

Microsoft (2025). Phi-4-mini Technical Report.

Next section

Negative control: If you can’t simulate, destroy

Last section

Scoring: Autoregressive multi-trait scoring (ArTS)

Return home

Psychometrics.ai