Date | Section Updated | Summary of Changes |
May 5 2025 | Project launched | Key sections on AI Psychometric Foundations added including definitions, reasons for progress, LLM training data, and bias. Section on Methods and Tools added including Moral Foundations motivating example with semantic construct validation; |
June 5 2025 | Embedding based psychometric analyses updated. | Sections in psychometric analyses added, including semantic item alignment and pseudo-discrimination factor structure. |
July 5 2025 | Seedling case study added | Seedling is a 50 million parameter GPT-2 style LLM trained from scratch on 10 GB of OpenWebText using Amazon Web Services. The full script is available from the Seedling case study page. |
September 6 2025 | New sections on LLM Emergence and LLM Alignment added. | Section explaining the concept of emergence in AI and its relevance to psychometrics; section added on the ideas behind alignment with a detailed discussion of Reinforcement Learning from Human Feedback (RLHF). |
September 15 2025 | How LLM's learn: Forward pass | Computations for a numerical forward pass added |
September 22 2025 | How LLM's learn: Backpropagation | Computations for backpropagation example added |
September 29 2025 | How LLM's learn: Optimization | Computations for optimization via gradient descent added |
October 9 2025 | How LLM's learn: Tokenization | Tokenization explanation added include BPE of twinkle twinkle little star |
October 14 2025 | Fine-tuning overview added | Adapters, LoRA, QLoRA described. |
October 24 2025 | RAG item generation pipeline | IPIP as context injection with Re-ranking to improve item quality. |
November 3 2025 | Explainable AI (XAI) Overview | Permutation, SHAP, Lime, Attention. Gradient XAI methods. |
November 17 2025 | Choosing a language model | Added criteria for choosing a language model incuding ecoder v decoder, reproducibility needs, open vs closed, and online vs local. |
November 24 2025 | AI scoring introduction | Added a crash course in AI scoring section that differentiates zero and few shot generative AI, hybrid models with frozen embeddings, and end-to-end neural architectures. |
December 12 2025 | Figures and images | Book until now is text and code based, I have started to add figures illustrating comments to each section. |
January 12 2026 | Newsletter sign-up | Newsletter sign-up added to homepage. |
February 2 2026 | Vision transformers | Null result showing vision embeddings and multi-modal vision + LLMs embeddings of matrix reasoning items did not predict item parameters. REMOVED in favour of next two sections that m0ore thoroughly set the ground for these experiments. |
February 9 2026 | Vision transformers | Introduction to vision transformers for psychologists who know LLMs, differentiating the key vision architectures. |
February 15 2026 | Large Multi-Modal Model (LMM) prediction of visual reasoning item parameters. | Outline plausible architecture and operationodes for predicting visual item reasoning parameters from Large Multimodal Models (LMMs) |
March 8 2026 | Introducing AI agents | New section on psychometric scale design with AI agent workflows. |
Return home
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
