B-learning Research

The Current Landscape of Blended Learning Research: Opportunities and Challenges
Charles R. Graham
Instructional Psychology & Technology
Brigham Young University
Thank you!
Introductions
Introductions – my research
For articles see: https://sites.google.com/site/charlesrgraham/
Introductions
Please introduce yourself.
Share an academic
and non-academic
interest
Agenda
1. What is BL?
2. Trends in BL research
3. Promising research directions
4. Pitfalls in BL research
5. Importance of Models and Theory in BL research
BL Overview
1. What is blended learning?
BL Overview
Definition:
Blended learning combines face-to-face instruction with computer-mediated (online) instruction.
BL Overview
(Graham et al., 2013)
BL Overview
Sometimes institutions call these blended but often they are not considered to be blended
(Graham et al., 2013)
BL Overview
Sometimes institutions call these blended but often they are not considered to be blended




Because of the supervisory responsibility in K-12 . . . often the online portions are done in the brick and mortar school.
(Graham et al., 2013)
BL Research Trends
2. Trends in BL Research
Dissertation Analysis
Analysis of Dissertation Research:
205 English language dissertations on BL
published through early 2012
Analysis of research context
Analysis of methodologies
Thematic analysis of research questions


(Drysdale et al., 2013)
Dissertation Analysis - Context
(Drysdale et al., 2013)
Dissertation Analysis - Methods
(Drysdale et al., 2013)
Dissertation Analysis – RQ Themes
(Drysdale et al., 2013)
Dissertation Analysis - subtopics
BL Scholarship Analysis
Analysis of Top Cited Scholarship:
50+ top cited articles, in which journals
25 top cited book chapters
10 top cited books
top cited authors
Analysis of research context
Analysis of methodologies
Thematic analysis of research questions


(Halverson et al, 2012; Halverson et al., 2014)
BL Scholarship Analysis - Context
(Halverson et al., 2012)
BL Scholarship Analysis - citations
(Halverson et al., 2012)
top cited articles (above) and books (right)
top cited chapters (above)
and white papers (left)
BL Scholarship Analysis - journals
(Halverson et al., 2012)
BL Scholarship Analysis - Methods
(Halverson et al., 2014)
BL Scholarship Analysis - Themes
(Halverson et al., 2014)
BL Scholarship Analysis – subtopics
(Halverson et al., 2014)
International BL
Analysis of Worldwide BL Scholarship:
10 top cited articles from each region
Journals with top cited articles
Network Analysis – citation patterns

(Spring et al., 2016 in press)
International BL – 7 regions
(Spring et al., 2016 in press)
International BL – top 2 per region
(Spring et al., 2016 in press)
International BL – top by region
(Spring et al., 2016 in press)
International BL – network analysis
(Spring et al., 2016 in press)
International BL – cites by region
(Spring et al., 2016 in press)
International BL – journals
(Spring et al., 2016 in press)
BL Research Questions
3. Promising Research Directions
BL Research Questions
What research questions to ask?
BL Research Questions
What research questions to ask?
Solving interesting local challenges
+
Contributing to more generalizable knowledge needs
BL Research Questions - categories
(Yeo & Mayadas, 2010)
(for reviews of BL literature using this framing see: Graham & Dziuban, 2008; Graham, 2013)
In 2014 Sloan-C became the Online Learning Consortium with journal Online Learning (formerly JALN)
Effectiveness Research - example
Barbara Means meta analysis:

23 very different blends
Overall Improved outcomes for blended learning (effect size g+=0.35)
Couldn’t identify a specific reason for improvement
“Studies using blended learning tended also to involve more learning time, additional instructional resources, and course elements that encourage interactions among learners.” (p.36)
Effectiveness Research - example
Bernard et al. meta analysis:
117 studies
Overall Improved outcomes for blended learning (effect size g+=0.334)
Also, looked at treatments intended to increase learner-learner, learner-instructor, or learner-content interaction
“when students are given the means to communicate and interact with one another or online with content, an increase in achievement may result”
UCF Study - success and withdrawal
(Moskal, Dziuban, et al., 2012)
UCF Study – student satisfaction
(Moskal, Dziuban, et al., 2012)
UCF Study – student satisfaction
(Moskal, Dziuban, et al., 2012)
BL Research Questions - EdUHK
(Yeo & Mayadas, 2010)
(for reviews of BL literature using this framing see: Graham & Dziuban, 2008; Graham, 2013)
What BL research questions to pursue at EdUHK?
BL Research Questions - EdUHK
(Yeo & Mayadas, 2010)
(for reviews of BL literature using this framing see: Graham & Dziuban, 2008; Graham, 2013)
EdUHK – Student enthusiasm & motivation
EdUHK – Effectiveness of different activities and/or strategies and methods
EdUHK – Instructor experience
EdUHK – Institutional issues – inter-departmental coordination project management
EdUHK – Student engagement
EdUHK – lots of potential
1. Early stages of BL research worldwide - EdUHK could become a leader in the region.
2. EdUHK already has some momentum in key research areas
Learner Engagement
Teacher Professional Development
3. EdUHK faculty can make contributions to general BL research or in domain-specific research outlets
Many content domains trying to figure out BL should be implemented
Teacher issues including satisfaction
Learner issues including dispositions
Increasing use of BL in primary and secondary ed
Domain-specific contributions
TPACK is a framework that emphasizes and values content-specific pedagogies.

Many domains use a unique mix of blended and online pedagogies:
Language learning
STEM
Humanities
etc
My BL Research Path – BL PD
Institutional Adoption
Faculty Adoption
Professional Development
What structures and supports need to be in place for successful BL adoption?
Beginning to develop a BL Teaching Readiness Survey to help diagnose faculty skills and needs
BL Institutional Adoption stages and key markers
What strategies, structures, supports are most important in encouraging successful faculty adoption of BL?
Institutional drivers and barriers to faculty adoption
Spectrum of BL Adoption
Implementation Stages
Institutional
BL Refinement
Institutional
BL Adoption
Key Markers of BL Adoption
BL Adoption Checklist
(Available online at: http://bit.ly/BL-adopt-checklist)
BL Adoption Checklist
(Available online at: http://bit.ly/BL-adopt-checklist)


Adoption Stages across the top.
BL Adoption Checklist
(Available online at: http://bit.ly/BL-adopt-checklist)

























Key Markers (Strategy, Structure, & Support) and sub indicators along the side
Faculty Adoption
(Porter et al., 2016a, 2016b)
Led to research investigating if there were different enablers and barriers to help faculty adopt who where at different places on the adoption spectrum
Adoption - future questions?
Curriculum:
What are the essential pedagogical, technological, and administrative skills necessary for successful blended teaching?
When and how to teach the skills?

Diagnosis:
How do we know where a teacher is on the spectrum from novice to expert BL instructor?
Need to develop tools to help us understand how to measure where faculty are and what they need
My BL Research Path – BL engagement
How to define learner engagement?
Learner Engagement
How to measure learner engagement?
Can we use log data and real-time data analytics?
Can we measure engagement at the activity-level?
How to use learner engagement data to improve teaching?
What level to focus on?
*Image modified from Skinner & Pitzer, 2012, pp. 23.
Most scholarly focus on long-term, school level (Eccles & Wang, 2012)
But we are interested in activity & course levels
Our Engagement Model
1st step – use intensive longitudinal methods (experience sampling) to measure engagement
Our Engagement Model
2nd step – look at relationship between course design and engagement.
Activity level interaction
Hektner, J. Schmidt, J, Csikszentmihalyi, M. (2007) Experience Sampling Method: measuring the quality of everyday life. p,.296-7
Research shows that quality of experience changes based on pedagogical activity.

How are activities experienced differently f2f and online in a BL environment?
Engagement can change over time
How might engagement change over time? What influences it to change?
Between subject and within subject
Hamaker, E. 2012. “Why researchers should think ‘within-person: a paradigmatic Rationale,” chapter 3 in Mehl, M. & Conner, T. editors Handbook of Research Methods for Studying Daily Life. New York, NY: The Guilford Press.
Ultimately we may need to begin thinking about engagement at the individual level rather than at the course level.
Within & Between Subject Patterns
Bolger, N. & Laurenceau, J. 2013. Intensive Longitudinal Methods: an introduction to diary and experience sampling research. New York, NY: Guilford Press, p.44.
Within: How do individuals vary in their engagement throughout a course?
Between: How do students vary in their engagement patterns?
BL Research Pitfalls
4. Pitfalls in BL Research
CHALLENGE #1
BL Models in the research are underspecified

We don’t carefully . . .
Identify the core attributes that make up our blends
Articulate why we think the core attributes would have an impact on the desired outcome
CHALLENGE #2
Many BL Models focus on surface features . . .

We describe models in terms of the physical environment, (e.g., online and face-to-face)
We don’t often describe models in terms of core pedagogical attributes
Car Purchasing Activity
With partner:

Identify 1-2 things that you want your vehicle to be able to DO (outcomes)?

Identify 2-3 core attributes that lead to those outcomes?
Analogy: What kind of vehicle?
Categories of Vehicles: sedan, truck, motorcycle, economy, hybrid, etc.
Outcomes: Miles per gallon, horsepower, seats, etc.
Surface features: color, # of wheels, # of doors, size, etc.
Under the hood: engine size, type of fuel required, etc.
BL Connection
What are the outcomes you care about from your blended instruction?
What are the core attributes of your BL model that you think will lead to those outcomes?
CHALLENGE #3
BL as a treatment effect . . .

BL is an umbrella term that encompasses many different pedagogies.
We need more specific models.
It is the specific attributes of a blend that cause the outcomes.
The way we blend
Online tutorials
E-mails
WEB MODULES
Computer Labs
Optional websites
Tele-web software
Computer cluster room
Internet tutorials
Electronic field trips
Supplemental web activities
Tech equipped classrooms
Evidence-based practice
Discussion
Virtual experiments
(Dziuban et al., presentation at OLC conference 2015)
The way we measure
Multiple Choice
Class ranks
student ratings
Class work
Placement tests
Motivation questionnaires
Achievement tests
Cooperativeness scales
Mathematics anxiety scales
Self-confidence scales
Self-report surveys
APPREHENSION TESTS
Reading power
Conceptual tests
Electric circuit tests
(Dziuban et al., presentation at OLC conference 2015)
CHALLENGE #4
Many small case studies that don’t connect back to general issues . . .

We tell the local story but don’t take time to show the implication of the findings for a more general audience
BL Research Methods
Chapter 1: Intro
Chapter 2: Paradigms, Methodologies, Methods, & Theory
Chapter 3: Reviewing the Literature
Chapter 4: Meta-Analysis
Chapter 5: Models of Survey Research
Chapter 6: Principles for Data Analysis
Chapter 7: Qualitative Research
Chapter 8: Case Studies
Chapter 9: Scholarship of Teaching and Learning
Chapter 10: Longitudinal Evaluation
Chapter 11: Big Data
Chapter 12: Assessing Outcomes
Chapter 13: Future Directions
2016 book on research methods in blended and online contexts
BL Theory/Models
5. Importance of Theory/Models
BL Theories/Frameworks

Need for greater focus on models and theory within the OL/BL community
Design Research is an important and legitimate pathway to creating new knowledge related to OL/BL
Theory

He who loves practice without theory is like the sailor who boards ship without a rudder and compass and never knows where he may cast.

Leonardo DaVinci
Theory

“[a] reasonably stable theory base . . . allows for a clear focus on important issues and provides sound (though still limited) guidance for the design of improved solutions to important problems”

Burkhardt and Schoenfeld (2003, p.6)
Question
What theories related to online or blended learning are you familiar with?
Models vs Theory
Some scholars have noted the interchangeable use of the terms model and theory (Dubin, 1976; Kaplan, 1964; Sutton & Staw, 1995; Whetten, 1989)
others have argued for clearer distinctions (Dickmeyer, 1989; Kaplan, 1964; Merton, 1967).
I will use the terms model and theory as two ends of a continuum.
Three Types of Research
(Graham et al., 2014)
Three Types of Research
“in [science] they are trying to understand how and why things happen, and in [technology, design] they are trying to discover how to influence things to happen”

(Gibbons, 2013, Ch 6)
Explore Research
(Graham et al., 2014)
Explore Research
Purpose: to describe and organize

Descriptive case studies
Phenomenology
Ethnography
etc.
(Graham et al., 2014)
Flight Example – Explore Research
Scientists made careful observations of bird flight and documented their findings
Wing tip observations
Wright brothers wing warping based on this observation
BL Example – Explore Research
Source: Staker, H., & Horn, M. B. (2012). Classifying K – 12 Blended learning.
BL Example – Explore Research
Source: Staker, H., & Horn, M. B. (2012). Classifying K – 12 Blended learning.
Explain Research
(Graham et al., 2014)
Explain Research
Purpose: to explain the why and how

Correlational research
Experimental research
etc.
(Graham et al., 2014)
Flight Example - Explain Research
Bernoulli’s Principle – used to describe why/how wings create lift

Establishes the relationship between pressure and velocity of a fluid
BL Example – Explain Research
(Graham et al., 2014)
Design Research
(Graham et al., 2014)
Design Research
=
Design Research
Purpose: structuring of artifacts/interventions to increase likelihood of desired outcomes

Various methods
Typically iterative in nature
(Graham et al., 2014)
Flight Example – Design Research
iterations – from glider to plane
Wind tunnel for testing wing shape designs
Built a light aluminum engine
Created their own propellers (based on ship propeller design)
BL Example – Design Research
Core Attributes
(Graham et al., 2014)
BL Example – Design Research
Relationships between core attributes important too
Design-based Research - Iterations
(Graham et al., 2014)
Design-based Research - Iterations
(Graham et al., 2014)
BL Explore Research Challenges
Patterns
Most research defines/describes or categorizes BL
Challenges
Often the focus is on surface/physical features (the “when,” “where,” and “who” of the instructional delivery) rather than the pedagogical features.
Rich descriptions without an articulation of what the core relevant attributes are
BL Explain Research Challenges
Patterns
Mentions theory only
Applies theory
Develops theory (limited)
Challenges
Mention of theory at beginning without exploring how the research improves understanding of the theory
Research is not really about BL but about something else and BL is just the context
BL Design Research Challenges
Patterns
Model articulation
Model comparisons
Model iteration
Challenges
Too many comparison studies testing effectiveness of BL with traditional counterpart
Core attributes of the designs not clearly identified
Core attributes of the design focused on surface/physical attributes rather than pedagogical
Need for more iterative designs over time
A Layered Approach
Contact


Charles R. Graham
Brigham Young University
[email protected]