Question and Test Interoperability
This chapter is an overview of the IMS Question and Test Interoperability (QTI) standard. It describes the need for storing interactive online assessments that can be created by a course content provider and loaded into multiple learning management systems from different vendors.
Focus: Assessment content, test items, test structure, scoring rules Maintained by: IMS Global (now 1EdTech)
Why We Need QTI Standards
- Standard way to represent questions, tests, response processing, and scores
- Used by LMSs and assessment tools to exchange quizzes and test banks
- Supports item types, scoring models, feedback, metadata, and adaptive tests
- Allows intelligent textbook content providers to interoperate with a wide range of LMS systems
- Prevents lock-in to specific LMS systems
- Encourages textbook authors to generate high-quality assessment tools that accurately place new students' knowledge in a course-specific learning graph
Relevance to LMS Interoperability
- QTI packages can be imported/exported across LMSs
- Forms the backbone for portable assessment content, not just results
History and Evolution
QTI 1.x (2000-2005)
The first generation of QTI established the foundational concepts for assessment interoperability: - Initial XML-based format for questions and tests - Basic item types (multiple choice, true/false, fill-in-blank) - Simple response processing - Limited adoption due to complexity and implementation variations
QTI 2.0 and 2.1 (2005-2012)
A major redesign that significantly improved the standard: - Cleaner XML schema with better separation of concerns - Enhanced item types including hotspot, order, and match interactions - Sophisticated response processing with template variables - Improved feedback mechanisms - Better support for accessibility (APIP - Accessible Portable Item Protocol) - Widespread adoption by major LMS platforms
QTI 2.2 (2015)
Incremental improvements focusing on: - Enhanced media support (audio, video) - Improved mathematical notation (MathML integration) - Better support for adaptive items - Clarifications and bug fixes from implementation experience
QTI 3.0 (2019-Present)
The latest major version with significant modernization: - JSON-based alternative to XML for easier web integration - Simplified schema and reduced complexity - Enhanced support for evidence-centered design - Better integration with other IMS standards (LTI, Caliper) - Improved support for innovative item types and simulations - Native support for competency-based assessment
Core Components and Architecture
QTI defines a comprehensive assessment ecosystem with several key components:
Assessment Items
The fundamental unit in QTI is the Assessment Item - a single question or task that can be presented independently:
Item Structure
- Item Body: The question prompt, including text, media, and interactive elements
- Response Declaration: Defines what type of response is expected
- Outcome Declaration: Specifies variables to store results (scores, correctness)
- Response Processing: Rules for evaluating responses and calculating outcomes
- Modal Feedback: Conditional feedback shown based on response
- Stylesheet: Presentation and rendering information
Item Example (Conceptual)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | |
Assessment Tests
An Assessment Test combines multiple items into a complete assessment:
Test Structure Elements
- Test Parts: Major sections of a test that can have different rules
- Sections: Logical groupings of items within test parts
- Item References: Links to individual assessment items
- Pre-conditions: Rules determining when sections/items are presented
- Branch Rules: Logic for adaptive test navigation
- Test Feedback: Overall feedback based on test-level outcomes
Test Configuration
- Navigation Mode: Linear (sequential) or non-linear (free navigation)
- Submission Mode: Individual (per item) or simultaneous (all at once)
- Time Limits: Test-level and item-level timing constraints
- Selection and Ordering: Random selection and shuffling of items
Response Processing
The heart of QTI's scoring mechanism:
Built-in Templates
QTI provides standard templates for common scenarios: - match_correct: Simple right/wrong matching - map_response: Mapping responses to predefined scores - map_response_point: For spatial/coordinate-based responses
Custom Processing
For complex scenarios, custom response processing rules can: - Implement partial credit scoring - Evaluate multiple criteria simultaneously - Apply conditional logic based on response patterns - Calculate derived variables and intermediate outcomes
Item Types and Interactions
QTI supports a rich variety of interaction types to assess different competencies:
Basic Interactions
Choice Interaction (Multiple Choice)
- Single or multiple correct answers
- Shuffle options to prevent memorization
- Orientation (vertical/horizontal)
- Radio buttons or checkboxes
Use cases: Factual recall, concept identification, best practice selection
Text Entry Interaction
- Short text responses
- Pattern matching for validation
- Case-sensitive or insensitive comparison
- Support for multiple acceptable answers
Use cases: Fill-in-blank, short answer, terminology
Extended Text Interaction
- Essay responses requiring manual scoring
- Word count limits
- Rich text formatting
- File upload capabilities in some implementations
Use cases: Written explanations, critical thinking, analysis
Advanced Interactions
Hotspot Interaction
- Image-based questions where learners click regions
- Supports polygonal, circular, and rectangular areas
- Single or multiple hotspot selection
- Spatial reasoning assessment
Use cases: Anatomy diagrams, geography, circuit identification, UI element selection
Graphic Order Interaction
- Drag-and-drop sequencing on images
- Spatial arrangement tasks
- Visual sorting and categorization
Use cases: Process flows, timeline ordering, priority ranking
Graphic Associate Interaction
- Connect related items on an image
- Draw associations between concepts
- Network and relationship mapping
Use cases: Matching definitions to diagrams, circuit connections, relationship identification
Order Interaction
- Arrange text items in correct sequence
- Drag-and-drop or button-based reordering
- Partial credit for partially correct sequences
Use cases: Procedural steps, chronological ordering, priority ranking
Match Interaction
- Connect items from two sets
- One-to-one or one-to-many matching
- Matrix/grid-based presentation
Use cases: Term-to-definition, cause-to-effect, concept relationships
Gap Match Interaction
- Drag items into text gaps
- Multiple draggable options
- Supports complex cloze tests
Use cases: Sentence completion, code completion, formula construction
Inline Choice Interaction
- Dropdown selections within text
- Embedded choices in reading passages
- Multiple gaps with independent choices
Use cases: Grammar exercises, reading comprehension, contextual selection
Slider Interaction
- Continuous or discrete value selection
- Visual numeric input
- Range-based responses
Use cases: Confidence ratings, probability estimates, preference scales
Custom Interactions
QTI 3.0 provides a portable custom interaction (PCI) framework for: - Simulations and interactive demonstrations - Game-based assessments - Virtual labs and experiments - Domain-specific tools (music notation, chemical structures, mathematical proofs)
Requirements: JavaScript-based with standard interface for data exchange
Scoring and Response Processing
Scoring Models
Dichotomous Scoring
Simple correct/incorrect evaluation: - Binary outcome (0 or 1) - No partial credit - Appropriate for objective questions
Polytomous Scoring
Multiple levels of correctness: - Partial credit models - Graduated scoring based on response quality - Useful for complex constructed responses
Weighted Scoring
Different point values for different items: - Difficulty-based weighting - Importance-based weighting - Combined with partial credit
Response Processing Strategies
Template-Based Processing
Uses predefined algorithms:
1 | |
Common templates: - match_correct: Binary scoring (right/wrong) - map_response: Map each choice to a score - map_response_point: For coordinate-based responses
Rule-Based Processing
Custom logic using response rules:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | |
Outcome Variables
QTI uses outcome variables to track: - SCORE: Numeric score for the item/test - MAXSCORE: Maximum possible score - MINSCORE: Minimum possible score - COMPLETIONSSTATUS: Whether the item was attempted - numAttempts: Number of attempts made - Custom variables for advanced analytics
Feedback Mechanisms
QTI supports multiple feedback types to enhance learning:
Modal Feedback
Shown in dialog or popup format: - Correct feedback: Reinforces correct responses - Incorrect feedback: Provides correction information - Partial feedback: Acknowledges partial understanding - Hint feedback: Progressive hints for struggling learners
Inline Feedback
Embedded within the question content: - Immediate visual indicators (checkmarks, X marks) - Highlighting of correct/incorrect choices - Explanatory text appearing near response areas
Solution Display
Shows the correct answer and explanation: - Complete worked solutions - Step-by-step explanations - References to learning resources
Adaptive Feedback
Context-sensitive responses based on: - Specific incorrect choices (addressing misconceptions) - Response patterns across multiple items - Performance thresholds - Time on task
Metadata and Classification
QTI integrates with learning resource metadata standards:
LOM (Learning Object Metadata)
- Educational level (grade, difficulty)
- Subject classification
- Language and accessibility
- Intended user role
- Learning resource type
- Typical learning time
Competency Tagging
Link items to competencies: - IEEE RCD identifiers - Custom competency framework references - Proficiency level indication - Prerequisite competencies
Integration with RCDs: Assessment items can be tagged with Reusable Competency Definitions to enable: - Competency-based test assembly - Diagnostic assessments for placement - Gap analysis and personalized learning paths - Skills-based reporting
Accessibility Metadata (APIP)
Accessibility features and accommodations: - Screen reader compatibility - Alternative text for images - Text-to-speech requirements - Simplified language versions - Extended time allowances - Color contrast and font size adjustments
Adaptive Assessment
QTI supports multiple adaptive assessment strategies:
Item-Level Adaptation
Individual items that change based on responses: - Adaptive Items: Items with multiple interaction paths - Progressive Hints: Revealing hints based on attempts - Scaffolded Questions: Breaking complex questions into steps
Test-Level Adaptation
Dynamic test assembly and navigation:
Computerized Adaptive Testing (CAT)
- Item selection based on estimated ability
- Real-time difficulty adjustment
- Efficient assessment with fewer items
- Requires item calibration (IRT parameters)
Rule-Based Branching
Navigate test paths based on performance:
1 2 3 4 5 6 | |
Item Selection Rules
Dynamically choose items from pools: - Random selection within difficulty bands - Selection based on previous responses - Ensuring content coverage across domains - Avoiding item overexposure
Multi-Stage Adaptive Tests (MST)
Hybrid approach combining: - Fixed stages with multiple modules per stage - Routing between modules based on performance - Balance between adaptivity and test security - Easier to implement than full CAT
QTI in Graph-Based Learning Systems
Graph databases provide powerful capabilities for assessment management:
Assessment Graph Model
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | |
Graph-Enabled Capabilities
Intelligent Test Assembly
Use graph queries to: - Find items assessing specific competency combinations - Balance test difficulty and content coverage - Avoid item dependencies and give-aways - Maximize information for diagnostic assessment
Example Cypher Query:
1 2 3 4 5 6 7 | |
Learning Path Optimization
Based on assessment results: - Identify competency gaps through diagnostic tests - Recommend next learning activities - Adapt content difficulty to learner's zone of proximal development - Track mastery progression over time
Item Bank Management
Graph relationships support: - Version control and item lineage - Usage statistics and performance metrics - Content tagging and multi-dimensional classification - Collaborative authoring and review workflows
Predictive Analytics
Use historical data to: - Predict item difficulty and discrimination - Identify items needing revision - Detect potential bias or cultural sensitivity issues - Forecast assessment completion times
Integration with Other Standards
QTI works alongside other educational technology standards:
LTI (Learning Tools Interoperability)
QTI assessments can be delivered via LTI: - Launch assessments from any LTI-compliant LMS - Pass user context and return scores - Enable single sign-on - Support deep linking to specific items
Integration Flow: 1. LMS launches QTI player via LTI 2. Player retrieves QTI content 3. Learner completes assessment 4. Scores posted back to LMS via LTI Outcomes 5. Detailed results stored in LRS via xAPI
xAPI (Experience API)
QTI results can generate rich xAPI statements:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | |
Benefits: - Granular tracking of each item response - Long-term learning record storage - Cross-system analytics - Support for competency-based reporting
Caliper Analytics
IMS Caliper provides standardized assessment event tracking: - Assessment started/completed events - Item viewed/responded events - Navigation and timing data - Aggregated test metrics
Common Cartridge
Package QTI content with other learning resources: - Combine assessments with instructional content - Include metadata and organization - Single importable package for LMS - Supports web content, discussions, and assignments alongside QTI
Packaging and Deployment
Content Package Structure
QTI content is typically packaged using IMS Content Packaging:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | |
Manifest Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | |
Deployment Options
LMS Import
- Upload QTI packages through LMS interface
- Automatic parsing and integration
- Item bank organization
- Test creation from imported items
Assessment Delivery Engines
Specialized QTI players: - TAO (Testing Assisté par Ordinateur): Open-source QTI platform - Questionmark: Commercial assessment platform - Canvas Quiz Engine: Built-in QTI support - Moodle Quiz: QTI import/export capabilities
API-Based Delivery
Modern approaches using REST APIs: - Retrieve QTI content dynamically - Deliver items via web services - Real-time response processing - Integration with microservices architecture
Tools and Platforms
Authoring Tools
Open Source
- TAO Item Creator: Web-based QTI authoring
- ONYX Editor: Desktop QTI item authoring
- QTIWorks: Java-based QTI development and testing environment
- Concerto: Adaptive testing platform with QTI support
Commercial
- Questionmark QTI Studio: Professional item authoring
- ExamSoft: Assessment creation with QTI export
- Learnosity: Cloud-based assessment authoring
- Respondus: Quiz and test creation tool
Validation Tools
- QTI Schema Validators: Check XML syntax and structure
- IMS QTI Conformance Tests: Official test suites
- QTI Migration Tools: Convert between QTI versions
Content Repositories
- Gooru: Open educational resources with QTI assessments
- Open Assessment Technologies: QTI item banks
- National repositories: Country-specific assessment databases
Best Practices
Item Development
Write Clear Prompts
- Use precise, unambiguous language
- Avoid trick questions that don't assess understanding
- State questions positively when possible
- Provide adequate context
Design Effective Distractors
For multiple choice items: - Make all options plausible - Base distractors on common misconceptions - Keep length and complexity consistent - Avoid "all of the above" and "none of the above"
Consider Cognitive Load
- One concept per item when possible
- Minimize extraneous information
- Use clear, simple language
- Provide scaffolding for complex tasks
Align to Competencies
- Tag items with specific competencies
- Specify Bloom's Taxonomy level
- Indicate prerequisite knowledge
- Define target proficiency level
Test Construction
Balance Content Coverage
- Blueprint aligned to learning objectives
- Representative sampling across domains
- Appropriate difficulty distribution
- Mixture of item types
Optimize Test Length
- Balance reliability with test-taking burden
- Consider fatigue effects
- Use adaptive approaches when possible
- Pilot test to establish timing
Implement Quality Control
- Peer review of items
- Statistical analysis after administration
- Regular item revision cycles
- Remove or revise problematic items
Accessibility
Universal Design for Learning (UDL)
- Multiple means of representation
- Various interaction modalities
- Flexible time limits
- Screen reader compatibility
WCAG Compliance
- Alternative text for all images
- Sufficient color contrast
- Keyboard navigation support
- Clear focus indicators
Accommodation Support
- Text-to-speech integration
- Adjustable font sizes
- High contrast themes
- Extended time options
Challenges and Limitations
Technical Challenges
Implementation Complexity
- QTI specification is extensive and detailed
- Subtle variations in vendor implementations
- Testing compatibility requires significant effort
- Custom interactions may not be portable
Performance at Scale
- Large item banks require optimization
- Complex response processing can be slow
- Media-rich items increase bandwidth requirements
- Concurrent user handling needs careful architecture
Educational Challenges
Manual Scoring Requirements
- Essay questions require human judgment
- Partial credit decisions can be subjective
- Grading consistency across raters
- Time-intensive feedback provision
Limited Context Assessment
- Standardized format may not capture authentic performance
- Some competencies difficult to assess via QTI
- Portfolio and project-based assessment gaps
- Collaborative work not well supported
Adoption Barriers
Legacy Content Migration
- Converting existing assessments to QTI
- Loss of custom features not in standard
- Training content creators on QTI
- Quality assurance of converted content
Vendor Lock-In Persistence
- Proprietary extensions to QTI
- Export functionality may be limited
- Platform-specific features create dependencies
- Economic incentives against portability
Future Directions
Emerging Trends
AI-Powered Assessment
- Automated item generation from learning objectives
- Natural language processing for essay scoring
- Adaptive feedback based on response analysis
- Predictive analytics for learner support
Immersive Assessments
- Virtual reality scenarios
- Augmented reality interactions
- Game-based assessments
- Simulation-based testing
Evidence-Centered Design
- Formal claims, evidence, and task models
- Bayesian inference for competency estimation
- Multiple evidence sources integration
- Diagnostic cognitive models
Competency-Based Assessment
Deeper integration with competency frameworks: - Direct RCD tagging in QTI 3.0+ - Multi-dimensional scoring - Growth modeling and learning progressions - Skills-based transcripts and badges
Interoperability Evolution
JSON-LD and Semantic Web
- Linked data for assessment metadata
- Ontology-based competency mapping
- Cross-institutional data sharing
- Discovery and reuse at scale
Blockchain for Credentials
- Verifiable assessment results
- Tamper-proof transcripts
- Decentralized credential verification
- Learner-owned assessment portfolios
Real-Time Collaboration
- Synchronous group assessments
- Peer evaluation frameworks
- Collaborative problem-solving tasks
- Social learning analytics
Integration with Graph LMS Architecture
In a graph-based learning management system, QTI assessments provide critical data for personalizing learning paths:
Assessment-Driven Personalization
Diagnostic Testing
Use QTI assessments to: 1. Placement: Determine starting point in learning graph 2. Gap Analysis: Identify missing prerequisite competencies 3. Proficiency Mapping: Establish current competency levels 4. Learning Style: Assess preferences through interaction patterns
Continuous Assessment
Embedded assessments throughout learning: - Formative checks: Quick competency verification - Mastery gates: Prerequisite validation before progression - Retrieval practice: Spaced repetition optimization - Metacognitive reflection: Self-assessment prompts
Summative Evaluation
Comprehensive competency demonstration: - Capstone assessments: Integration across multiple competencies - Certification exams: Proficiency verification for credentials - Adaptive finals: Efficient comprehensive testing - Portfolio reviews: Evidence-based evaluation
Graph Queries for Assessment Intelligence
Find Next Assessment
1 2 3 4 5 6 7 8 | |
Generate Progress Report
1 2 3 4 5 6 7 8 | |
Recommend Learning Resources
1 2 3 4 5 6 7 8 9 | |
Conclusion
The IMS Question and Test Interoperability (QTI) standard represents a mature and comprehensive approach to assessment interoperability in digital learning environments. By providing a standardized format for assessment content, QTI enables:
- Portability: Content creators can develop assessments once and deploy across multiple platforms
- Investment Protection: Institutions avoid vendor lock-in and preserve assessment assets
- Innovation: Standardization enables tool specialization and ecosystem development
- Quality: Shared best practices and validated items improve assessment effectiveness
- Analytics: Consistent data formats enable sophisticated learning analytics and research
When integrated with graph-based learning architectures, QTI assessments become powerful instruments for: - Diagnosing learner competencies and gaps - Personalizing learning paths based on demonstrated mastery - Tracking learning progressions over time - Connecting assessment results to competency frameworks - Generating actionable insights for learners and educators
The evolution toward QTI 3.0, with its JSON support and enhanced competency integration, positions the standard well for next-generation learning ecosystems. Combined with xAPI for experience tracking, LRS for data persistence, RCD for competency definition, and TLA for architectural integration, QTI forms a critical component of interoperable, learner-centered educational technology.
As assessment practices continue to evolve toward competency-based, adaptive, and authentic evaluation methods, QTI provides the technical foundation for innovation while maintaining the interoperability that protects educational investments and enables learner mobility across systems and institutions.
References
- IMS Global QTI Specification - Official standard documentation
- QTI 3.0 Overview - Latest version specification
- Experience API (xAPI) - Integration with learning records
- Learning Record Store (LRS) - Assessment data persistence
- Reusable Competency Definitions (RCD) - Competency tagging for assessments
- Total Learning Architecture (TLA) - Architectural context
- TAO Testing Platform - Open source QTI implementation
- Accessible Portable Item Protocol (APIP) - Accessibility guidelines
- Evidence-Centered Design (ECD) - Assessment design framework