Instructional Design, Transfer of Training
& Program Evaluation
Barbara A. Bichelmeyer, Ph.D
Associate Professor
Department of Instructional Systems Technology
Indiana University
Middle East Technical University
Ankara, Turkey
21 June 2005
A sampling of my time in Turkey
What I have learned so far00/font>
more alike than they are different Just like countries, people must change
in order to survive and grow
My Assumptions About You
What Experience Do I Have to Offer?
- Teacher
- Instructional designer
- Performance consultant
- Professor
- Evaluator
Work Settings:
- Secondary school
- Higher Education
- Corporate
- Government
- Military
- Non-profit
- Health Care
What Can I Possibly Tell You?
they must also teach students how to learn Educators need to know about how people
experience the processes of change and growth
Purpose of this Session
Influences
(IC, S/K, M/I, E)
Organization
Goals
Accomplish-
ments
Actions
Influences = Programmatic support for organization initiatives
Actions = Activities that leads to accomplishments
Accomplishments = What individuals produce to support org goals
Organization Goals = The value-added
of the organization
Assumption #1:
Basic Progression of Human Performance
Assumption #2:
The Performance Equation
P = IC * M/I * S/K * * Env
Inherent Capability = individuals00 inherent abilities and dispositions
Skills/Knowledge = facilitate individuals00knowledge of what/how to do
Motivation/Incentive = support individuals00desires to do
Environmental supports = provide individuals with resources to do
Assumption #3:
Basic Progression of Instructional
Design
Influences
(IC, S/K, M/I, E)
Organization
Goals
Accomplish-
ments
Actions
Systematic instructional design involves
reverse engineering
(beginning with the end in mind):
1. What is the Goal we are trying to achieve?
2. What accomplishments must be produced in order to achieve the goal?
3. What behaviors must performers engage in?
4. What influences are required to support performers (IC, S/K, M/I, Env)?
Instruction is an appropriate intervention if (and only if) a Skill / Knowledge deficit is present in performers The purpose of instruction is to increase overt and covert capability in learners so they can do things they were never before able to do (as opposed to information, the purpose of which is to reduce uncertainty) The goal is instruction is to ensure that performers00new capabilities transfer to the 00eal world00/font>
Goal and Purpose of Instruction
Why does transfer of learning matter?
Influences
(S/K, M/I, E)
Business
Goals
Accomplish-
ments
Actions
We are interested in transfer of learning because:
- it impacts human performance
- it impacts effectiveness of organizations
1) Transfer to long-term memory
2) Transfer to 00eal world00performance
Two Types of Learning Transfer
Impact of Instructional Design on Transfer
Elements that systematically produce transfer of learning:
Goals - what do you want to transfer? Performance Aids - tools that support transfer Tests - how you will measure transfer Application & Feedback - how you will facilitate transfer Examples - how you will model for transfer Instruction - how you will prepare learner for transfer Leader material - how to set up instruction that transfers
Impact of Application on Transfer
Quantity of opportunities for application
Transfer to 00eal World00 =
Quality of opportunities for application
Major Blocks to Transfer
Systemic Support for Transfer
IC + S/K + Env + M/I Plan and Follow up:
Before - During - After
Lack of systematic support minimizes possibility of transfer Instructional designers and program managers must work together for integrated support
Kirkpatrick & Evaluation of Training
Level 3: OTJ Behavior
Level 2: Participant Learning
Level 1: Participant Reaction
Kirkpatrick00 4 Levels of Evaluation
Each level is important
None of the levels should be bypassed to get to a level that is considered more important
As you move from one level to the next, the process becomes more difficult and time-consuming
Each level provides more valuable information
Links between Program Design,
Instruction & Program Evaluation
Business Goal
Influences
Behaviors
Accomplishments
Results / ROI
Reaction
Learning
Real World Behavior
Program Design:
Program
Evaluation:
Instruction
Where Does Transfer Begin?
(Kirkpatrick00 four levels of training evaluation)
What To Know about Instruction
1) Purpose of instruction is to increase human capability that helps to achieve goals of program/organization
2) Instruction is but one influence on human performance
3) Instruction is an investment in human
capital 00
most expensive investment
4) Instruction is only valuable if it transfers to performance context
5) Transfer is dependent on well-designed instruction
6) Transfer requires performance supports after instruction
7) Instructional staff must work with program management to provide appropriate supports for transfer
8) Evaluators should study relationship between instruction and program management
Implications for Evaluation
Success of instruction is measured in Level 2
Instruction can never take complete credit for 00eal world00performance or ROI
This doesn00 mean that we can not evaluate whether our goals have been achieved
Seminal Sources
Donald L. Kirkpatrick (1994), Evaluating Training Programs
Robert Mager (1997), The New Mager Six-Pack
Thomas Gilbert (1968), Human Competence
Contact Information
Barbara A. Bichelmeyer, Ph.D.
Associate Professor
Department of Instructional Systems Technology
Indiana University
201 North Rose Ave., #2226
Bloomington, IN 47405-1006
812-856-8468
download Instructional Design, Transfer of Training & Program Evaluation
