Date | Section Updated | Summary of Changes |
May 5 2025 | Project launched | Key sections on AI Psychometric Foundations added including definitions, reasons for progress, LLM training data, and bias. Section on Methods and Tools added including Moral Foundations motivating example with semantic construct validation; |
June 5 2025 | Embedding based psychometric analyses updated. | Sections in psychometric analyses added, including semantic item alignment and pseudo-discrimination factor structure. |
July 5 2025 | Seedling case study added | Seedling is a 50 million parameter GPT-2 style LLM trained from scratch on 10 GB of OpenWebText using Amazon Web Services. The full script is available from the Seedling case study page. |
September 6 2025 | New sections on LLM Emergence and LLM Alignment added. | Section explaining the concept of emergence in AI and its relevance to psychometrics; section added on the ideas behind alignment with a detailed discussion of Reinforcement Learning from Human Feedback (RLHF). |
September 15 2025 | How LLM's learn: Forward pass | Computations for a numerical forward pass added |
September 22 2025 | How LLM's learn: Backpropagation | Computations for backpropagation example added |
September 29 2025 | How LLM's learn: Optimization | Computations for optimization via gradient descent added |
October 9 2025 | How LLM's learn: Tokenization | Tokenization explanation added include BPE of twinkle twinkle little star |
October 14 2025 | Fine-tuning overview added | Adapters, LoRA, QLoRA described. |
October 24 2025 | RAG item generation pipeline | IPIP as context injection with Re-ranking to improve item quality. |
Return home
Psychometrics.ai
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
