- Part 1: AI psychometric foundations
- Part 2: Methods and tools
- Part 3: Psychometric analyses
- Part 4: Hybrid models
- Part 5: Critical perspectives and future directions
- References
Part 1: AI psychometric foundations
Definition and introduction to foundational concepts in AI psychometrics, positioning transformers as a threshold capability marking a milestone in AI psychometrics.
What is AI psychometrics?Substitutability assumption Understanding LLM designsEncoder architecturesDecoder architecturesEncoder-decoder architecturesSeedling: LLM training case studyPart 2: Methods and tools
Use of AI to develop the building blocks of assessments, including agents and multi-agent systems in task analysis, test blueprinting, test administration, scoring, and feedback.
Moral foundations as a use caseDo Moral Foundations matter in business?Semantic convergent and discriminant validityAI item generation strategiesGenerating items via an APIModel fine tuningPart 3: Psychometric analyses
Introduces semantic alignment for item validity, pseudo factor analysis, network analyses, and convex hulls with applications to item analysis and free text scoring.
Semantic item alignmentSemantic item alignment resultsPseudo factor analysis Pseudo factor analysis resultsConvex hull analysesExploring embedding compositionsSimulation with artificial crowdsLLMs and psychometric biasPart 4: Hybrid models
Integrating transformers with other computational psychometric methods, such as predictive ensembles with embeddings as inputs.
Part 5: Critical perspectives and future directions
Limitations, ethical and privacy concerns, including geographic/cultural differences in expectations, as well as future directions.
Historical roots of AI assessment debatesA peep into the future of assessmentReferences
Next section
Last section
Return home
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
