A Practical Guide to Case Studies

Introduction

We know that case studies are effective.  Research tells us that.  The National Center for Case Study Teaching in Science describes their value in an introduction to the Center:

CASE STUDIES have a long history in business, law, and medical education.  Their use in science education, however, is relatively recent.  In our 25+ years of working with the method, we have found it to be a powerful pedagogical technique for teaching science.  Cases can be used not only to teach scientific concepts and content, but also process skills and critical thinking.  And since many of the best cases are based on contemporary, and often contentious, science problems that students encounter in the news, the use of cases in the classroom makes science relevant.

Case studies have proven to be effective in a broad range of disciplines.  The key to their success in either a face-to-face classroom or online is the interaction of students with the content and the discussion between students — in short, the Community of Inquiry Model. Case studies – especially case studies that don’t have one clear answer or resolution – require discussion.

In a previous article, we looked at the value of interactive case studies.  https://lodestarlearn.wordpress.com/2017/11/14/the-research-behind-learning-interactions/

The article elicited numerous responses from online instructors who now aspire to develop case studies for their own disciplines.  The article you are now reading provides a practical guide to getting started.  In this article, we particularly emphasize the story-telling aspect of case studies.  Developing a good case study is indeed much like story-telling.

Case studies are an important strategy you can use to help students apply what they are learning and make them think.   Using the case study strategy may be daunting for some instructors.  Effectively using this strategy is a challenge.    What should you think about when creating a case study?  What are the options?

Generally, effective case studies are:

  • Realistic
  • Focused on student outcomes
  • Involve the student in a story
  • Involve the student in selecting and recalling information, analyzing and making decisions

Case Studies are not PowerPoints in a modified format. They are not just another means of presenting content.  Students summon information when they need it.  In a case study, content is only useful insofar as it can be applied to the situation at hand.  Truly engaging students in the story is a challenge.  Engagement means that the student summons resources, recalls information, and makes decisions.  You need just enough detail to make the case study realistic but not sacrifice the learning outcomes for realism.  You are a busy instructor.  You don’t need a lot of production to pull this off.  With that in mind, let’s get started.

Getting Started

Establish goals

This may seem like the most laborious part of the case study design, but it is necessary.

Imagine for a moment that you developed a case study for nursing students studying infectious disease control.  In a case involving infectious disease control, with an outcome related to sanitation policy, you wouldn’t present symptoms to the nursing student and ask for identification of the disease.  Depending on the level of the nurse, however, you might involve the nurse in decision-making related to creating and implementing a sanitation policy or reducing the risk of other patients being infected.  Be clear on the outcomes.  The outcome should be stated with audience,  behavior,  context and degree (ABCD).  An example outcome:  Given a Nepah virus outbreak in a small hospital, the infection nursing student will select the appropriate sanitation procedure with 100% accuracy. Think of the outcomes, objectives or competencies as the driver of the case study.  They drive the presentation of content and the interactions of the learner.  If it isn’t relevant to the outcome, it doesn’t belong.  Parsimony is essential – meaning that a case study is economical about what is left in the case narrative and deliberate about what is left out.

For a simple case study, a paragraph or two should suffice.  In more detailed cases, a page or two will be adequate.  In the National Center for Case Study Teaching in Science, the 700 plus cases represent the range:  from very short descriptions to a couple of pages presented in either narrative or dialog style.  The interactive (fully online) case study can be crafted to introduce information either at the beginning of the case or as the learner engages with the case.  Information can be revealed just-in-time or revealed through resources that appear as the case develops.  There is no one right way or one authority.  Do what comes naturally to you and won’t overwhelm the student.  Experiment.  Ask for student feedback.

In a case on organizational leadership, the narrative described a new CEO whose company had a top performing salesman who constantly cut corners on company procedures and angered service personnel, engineers and other sales staff.  The narrative provided just enough information for the reader to fully appreciate the dilemma.  Should the CEO fire or retain the salesman? Students participating in the case study were placed in the shoes of the CEO and had to draw from the content of the course to support their decision.  In this case, the narrative was approximately 500 words.

Set student expectations

Make known to the student that this is a black and white case versus a nuanced case with no one right answer or visa versa. If cases are a simplified version of reality, students should know that the case study is a stepping stone to more complex cases. Avoid disillusioning students by under-preparing them for reality. If it is a stepping stone case, present it as such.

Cases can be simple with straight-forward answers or they can be complex with no clear right answer.  The latter almost always requires discussion whether in class or online.  As important as the solution may be the exposed thinking that leads to a solution.  As important as the solution may be the sharing of student perspectives related to the solution.   In either case, simple or complex, students should know what to expect.  If they are asked to make a decision, for example, they need to know whether their answer will be judged right or wrong or be evaluated in a different way.  Perhaps any answer is the right answer, provided that it is supported by the details of the case and the body of knowledge that pertains to the case.

In the National Center for Case Study Teaching in Science site, faculty at Purdue University describe their case studies related to using statistics in science in the following way:

This set of mini cases on the ecology of eastern cottontail rabbits is designed to give students practical experience using statistics in a scientific context. Given a dataset and experimental design, groups of students are asked to play the part of a wildlife management researcher to determine the results for each study. Students practice the scientific process and gain experience making hypotheses and predictions, choosing an appropriate statistical test, interpreting and displaying results, and presenting data to others. Students choose between four basic, commonly used, statistical tests (t-test, one-way ANOVA, linear regression, and Chi-square test), and justify their choices.

 

To summarize, I would emphasize that students play the role of wildlife researchers, make hypotheses and predictions, choose the appropriate statistical test and justify their choices.   There might only be one appropriate answer or several correct answers.  The emphasis might be on ‘getting the right answer’ or ‘justifying the answer’.  Either scenario is acceptable.  The case study approach allows for either possibility.  Again, the desired student outcomes drive the design.

In a case that involved the student in identifying risk factors related to recidivism and offenders, there was only one right answer.  Either the student picked up on the risk factors, correctly categorized them, or not.  In a case that involved assessing an adult student for prior learning matched to university programs, there were many possible answers.  The emphasis on the latter was on following a checklist, consulting the appropriate resources and identifying the opportunities for the adult student.

 Create active participants

In case studies, students should make decisions and/or perform an analysis, and contrast their work with that of others. In short, make the participants think. So much online learning contributes to a passive learning experience. Case studies should require students to recall the appropriate content and use it to make a selection or support a choice. Lengthy presentations, in which students are passive participants, are less likely to contribute to successful outcomes.

Identify what role the student is playing

Case studies may place the student in a role play. You should identify the student’s role. In one of our recent case study projects (Credit for Prior Learning), the designers placed the participants in the role of an advisor in a college setting.  The advisor was tasked with recognizing prior learning worthy of college credit.

 

2018-06-04_1524

Patricia is the guide for this case study (shown in screenshot).  She provides general directions on how to engage with the case before introducing the case itself.

 

Make it clear to the student what role s/he will play. State relevant information: job title, level of experience, etc.

I’ll refer to the Credit for Prior Learning case studies several times in this article.  They were designed by the College of Individualized Studies at Metropolitan State University by Drs. Carol Lacey, Marcia Anderson, and Susan Misterek with support from Dr. Bilal Dameh and Dominic Jennen.

Identify the setting

Again, designing case studies is like telling stories. Stories are best told with settings. Be mindful that students will judge the relevance of learning based on setting and situation. Placing elementary student teachers in a college classroom setting may cause participants to dismiss the learning.  They may think that ‘this doesn’t apply to me.’  On the other hand, if students recognize the setting and accept that it relates to their reality, they will judge the case study to be relevant and be open to engaging with it.

Establish the plot

Plot is a sequence of events that happen in the story that may have an impact on future events. In a case study the plot can be linear (one set of events for all students) or branched (a unique set of events for each student).

Once you have introduced the characters of the case study, the setting, and the student’s mission or objective, you need to work out the plot. As mentioned, the plot can be simple or it can be complex.  In one case study that we worked on, the nursing student learned about latent tuberculosis bacterial infection.  The student then observed (as a third party) the dialog between a public health nurse and a client.  The student was asked to take notes during the dialog and then accurately fill out charts related to the diagnosis, treatment and outcomes.  At the heart of the plot was the interview.   The student couldn’t control the interview.  The student simply needed to pay attention, take notes and accurately chart.

In another case study, the student played the part of an instructor asked to design an online course.  The dean in the story asks the instructor about what s/he would do first, then second, then third and so forth.  The story was linear (not multi-branched), but it was revealed in stages.

In the Credit for Prior Learning case study, the student played the part of a college faculty member who was tasked with assessing whether or not a character could earn credit for his prior learning and experience.  This was a very simple plot.  Information was presented on the screen and then ‘resources’ displayed.  The participant could consult the resources to help evaluate whether or not the character was a candidate for prior learning. One of the resources was a checklist with key considerations:  for example, was the experience related to the character’s goals?  If the goal was to achieve credit for upper division courses, did the participant find any evidence of learning that was equivalent to an upper division course that contributed to the character’s goals.  Simple setting and situation.  Not much plot.  The participant picked the right resources, consulted them and then wrote up a summary of why the character was good candidate.

 

2018-06-04_1522

Screenshot of a case study about a student who has had prior learning experience as a corporate trainer.  The case study places learners in the role of an advisor who collects information and decides if the corporate trainer has had enough public speaking to meet a general education/liberal studies goal.

 

All three of the examples above were simple plots.  But plots can be complex.  Remember the ‘Choose your own adventure’ books?  Rather than reading a book from cover to cover, the reader of a ‘Choose your own adventure’ makes decisions.  The decision then refers the reader to another part of the book.  The reader might jump from page 10 to page 20 or from page 10 to page 16. Each reader picked their own path through the narrative.  The sequence of events was unique to each reader.  The interactive case study can optionally branch students to different parts of the case study.

In a law enforcement case study, a parole officer interviews a client (the offender).  For each page of the story, the person playing the role of a parole officer decides on one of three questions to ask the offender.  The choice of question may start to move the client toward a negative emotional response.  The parole officer, however, can recover from an escalating situation through a series of correct choices.  On the other hand, too many incorrect choices terminates the session.

The situation and the immediate feedback immerse the student in the content.  Successful case studies require students to recall lecture and reading material, select the appropriate information and use it correctly.

Avoid Combinatorial Explosions

In his book titled “E-Learning by Design” William Horton cautions against combinatorial explosions when designing games.  The same wisdom holds true with complex case studies with multiple branches.  Two paths can lead to four paths can lead to sixteen…  Horton outlines a couple of solutions to the problem of combinatorial explosion.  In one solution,  which Horton calls the short-leash strategy, learners are not allowed to stray too far from the ideal path.  In the above example related to the parole officer, too many bad choices terminates the session.  The case study doesn’t keep on branching.

Provide Resources

Case studies are a simplification of reality.   In some of our designs, we exposed resources at the appropriate time in the sequence of events.   Case studies that send students off into the web or into the depths of a textbook run the risk of losing the students’ attention.   Developers of case studies can make resources appear at certain times in the form of buttons or pop-ups.  In the Credit for Prior Learning case study, participants clicked on a button and viewed a transcript that they had to analyze for critical information.  Hyperlinks in the case study can cause pop-ups to appear with useful information.

 

2018-06-04_1523

Case Study reveals resources (buttons on right) when the student needs them.  In this case, students can consult the instructions, an explanation of icons used, background of the character, a checklist, a description of a general education goal and a proficiency test.

 

 

Use Some Form of Storyboarding

Storyboard your ideas – although, the thought of storyboarding may seem daunting.  It implies the use of specialized software or a specialized skill.  Neither is needed.

Use pencil and paper, if nothing else.  Place each scene in a box with stick figures.  Outline the information that will be presented and the choices offered to the student.   If your case study uses branching, draw lines between the boxes to show the branches.

In our last interactive case study, our instructional designer created a table in Microsoft Word.  The table included information presented to the student, choices, feedback, and a listing of the resources that would be displayed to the student at that scene.    Microsoft Word includes SmartArt.  The Horizontal Hierarchy Smart Art, for example, might be useful for mapping out a case study.

Whether using pencil and paper or Word, you will find that it is easier to make changes and avoid confusion than to draft your ideas within an authoring tool.  Overly complicated case studies become apparent when mapped out in advance.  Most authoring tools don’t offer a birds-eye view.

Support Discussion

In simple case studies, students can make a decision by clicking on options or they can perform an analysis with text entry or drop box submission.  Once students commit their answers, the case can reveal the expert answer.  In more sophisticated cases, multiple answers or solutions or analyses might be appropriate.  The case can step the learner through making a decision or preparing an analysis that can be submitted to a drop box or entered into a discussion post.  In both simple and complex cases, the case can prepare students for an in-depth discussion about the critical aspects of their case and the rationale behind their decisions or analyses.  In a flipped classroom approach, the case can engage the students online and leave precious classroom time for moderated discussion.

Vary Complexity

Cover a topic with a simple case, followed by a more complex case.  In our Credit for Prior Learning course, we started with black and white cases.  There was one right answer.  Either the character in the case was a candidate for credit for prior learning or not.  In the succeeding cases, the decisions were not so clear cut.

A strategy in game design is to start simple, ratchet up the challenge, plateau for a while, then ratchet up again.  This careful control of complexity applies to case study design.  Get students acquainted with the interactive case, instill some confidence and then work in the nuances and complexities of reality.  Not every case study needs to be same level of complexity.  Especially when designing a sequence of cases, control complexity carefully and strategically.

Conclusion

Creating case studies is story telling.  They include character development, setting, plot, role play, dialog, and even suspense.  They place learners at the center of the story.  Unlike stories, they provide feedback – both immediate and in discussions.  They are a teaching tool and they help students apply what they have learned.  They are an important strategy in helping instructors making learning active.  The best way to get started with case studies is to get started.  There is no one model that dictates case study design.  If you are making students think about your content material, you are doing the right thing.

Advertisements

Open Educational Resources: An Alternative to Publisher Platforms

Introduction

One could easily vilify the textbook publishers and their online learning platforms.  In higher ed, we hear the student complaints.  They pay tuition with the expectation that faculty will help them build competence and credentials.  They trust the instructor to select or write  appropriate content, design support activities that help students understand and apply the skills, elicit performance, provide feedback and so on.  They are placing their trust in the instructor and their trust in the institution.  When students perceive that instructors have transferred that responsibility to textbook publishers, they make comments like “Why am I not paying the publisher directly?”

But my intent is not to vilify publishers.  They have obviously responded to a strong demand and need for their platforms and resources. In many cases, instructors don’t have the time, dedication or wherewithal to develop courses.  They don’t have time to create the content, develop the activities and assessments and still be engaged in the course discussions and in providing constructive feedback to their students. In some cases, they concede that they couldn’t match the publishers’ course material and the quality of their offerings even if they wanted to.  After all, how many faculty write their own textbook?

Rather, this post recognizes the challenges that constrain faculty and offers an alternative to publisher platforms.  Chances are good that you follow this blog because you are concerned about making your online courses better.  You choose to be in ‘control’ of your course.  The publishers may be broadening their reach to include your discipline – but you wish to select content carefully, add activities and assessments and continually improve your course based on student feedback.  Your student feedback. And yet you are faced with constraints on your time and, perhaps, lack of support.

Textbook Publisher Platforms

You stand in stark contrast to the current trend in education. For better or for worse, higher education is ceding control to the textbook publishers.  The publishers are developing new business models that include online learning platforms, online courseware, adaptive learning, and digital books, all of which, they claim, improve quality, reduce costs and provide a more stable revenue stream to them in the form of subscriptions.

All of these platforms are not alike.  The offerings are on a continuum from supplemental resources to pre-made courseware to integrations with university learning management systems to full-blown adaptive learning platforms with supplemental instructor resources and more.

Textbook publishers carefully vet their content and do the best job possible without knowing the specifics of your students and the context of their learning.  At the very least, the courses are well-organized. When publishers are in complete control, the outcomes match the readings and activities. Errors, inconsistencies and incongruities are exposed and eventually removed.  (When publishers are in partial control, misalignment occurs between the publisher materials and, for example, dated faculty-generated quizzes.)

In 2014, Slate published an article titled “College in a Box” that explored an emerging state of affairs related to textbook giants and online college courses. The article described two college students who were separated by 600 miles attending different colleges but taking the same course, produced by a major publisher. The students read the same online textbook, watched the same media, and completed the same assessments with little interaction with their professors. The students were generally happy with the courses, received good grades, got assistance in the form of well-produced videos and, presumably, progressed toward graduation.

Publishers have long had tremendous influence over a course. This isn’t new.  In my first year of teaching, our department assembled and planned curriculum. The curriculum plan was based almost entirely on the table of contents of the adopted text.  (As a young teacher, I heard how the textbook purchasing power of California and Texas dictated the content of the textbooks for everyone. Today control has shifted from the state to the districts and textbooks are not required to meet 100% of the state standards.) In recent history, publishers have become more sophisticated at producing online courseware. Today, students pay fees to access publisher course material, which include activities and assessments. Undoubtedly, in many cases, the publisher content and collateral is much better produced than the homespun online course.

I can’t even begin to address the societal implications of this development. Slate asks “Why are universities buying ready-made frozen meals instead of cooking up their own educational fare?” The suggestion, obviously, is that textbook online course sites are the ready-made frozen meals. The benefit to universities is that the cost of course development is passed directly to the student. Rather than a university compensating faculty with stipends or release time to develop online courses, students pay a fee to publishers. Research will eventually disclose to us the full cost of abrogating the instructor’s role in course development. A one-size-fits-all approach doesn’t account for the unique situational factors of the class.  And other concerns surface. Will publishers’ courses eventually end-run colleges and universities? After all, don’t employers in some sectors care about competence over degrees? Finally, what role does the instructor play? Is the instructor replaceable by a person of lesser rank and cost?

In the end, I believe that economics will win out. Publishers are putting to use all of the great developments in the last few years at a speed and economy of scale that most mid-sized universities cannot match. I am thinking about adaptive learning, rich interactions and even Open Educational Resources (OER). Their systems are improving; their design is improving. They lessen the load on instructors and shift the cost to the student.

But even in the industrial revolution, certain guilds of manual artisans survived.  That’s how I picture the online instructor who designs his or her own course:  An artisan.  I think of the positive aspects of a course crafted with care, compassion and skill.

Today, conferences such as OLC Innovate convene educators who share their views on online learning, emerging technologies, and best practices. OLC Innovate celebrates instructors who post their intellectual property to repositories like Merlot.org and participate in editorial groups.  It is a homespun, cottage industry – but it is vibrant.

I anticipate that dedicated artisan instructors will prevail. They will continue to participate in membership groups and conferences of like-minded people. They will embrace a raft of tools to help them communicate with students, motivate, collaborate, challenge, and assess.

They’ll embrace digital stories, eBooks, simulations, videos, and whatever they need to engage students.

Fortunately, exceptional support for the artisan comes in the form of the open education movement. For some, Open Educational Resources (OER) may represent a significant alternative to publisher platforms.

OER_Logo_Open_Educational_Resources

image credit:  Wikimedia Commons

Open Educational Resources

Open Educational Resources are typically open textbooks, software, web pages, learning activities, simulations, case studies, quiz banks and media that are available to faculty and to students for free. Typically, they are licensed under Creative Commons, which means, in all cases, that you must provide the author attribution.  The debate about what constitutes “open” gets more complex when you ask the question “Can I change the resource and adapt it to my own needs.  Can I offer the new ‘derivative’ product to another professor?”  Some would argue that ‘open’ requires the ability to revise, remix, reuse, and redistribute.

Others are content with a narrower definition.  The Hewlett Foundation, an ardent supporter, defines OER as

“teaching, learning and research materials in any medium – digital or otherwise – that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions.”

Today, university systems are bootstrapping OER in a number of ways. They are utilizing open source repositories like Islandora and Equella. They are paying faculty to review open resources and participate in editorial teams for such altruistic efforts as Merlot.org, a curated repository used all over the world. Universities house and support a network of OER repositories, the breadth of which is evident in this Google Map: http://maps.repository66.org/

In my experience, the bane of OER was in the search and discovery of resources that closely matched our course learning outcomes. It required patience – and busy faculty quickly became disillusioned.

Today, search and discovery are easier.  A number of federated search tools have been developed and made available through such organizations as Merlot.org, Creative Commons and even the federal Department of Education.

Recently, I learned of something that suggested better times to come.  A colleague at the Minnesota State System Office introduced me to Intellus Learning.

Intellus Learning is a subscription-based platform that makes it easy to find high quality OER content.  Instructors find content aligned to their objectives, embed that content into their learning management system and monitor student usage.  Intellus searches across many OER repositories, saving instructors considerable time.  I’ve only trialed this tool, but I can imagine a new breed of software that makes it easy to discover, explore and mash-up open resources. My searches returned materials from OpenStax (open textbooks), EBSCO, YouTube, OER Commons and dozens of other places. I conducted the same search using no-cost publicly available search tools and came up short in comparison. The exception was Merlot, which provided me with a useful listing of resources.

Again, in my experience, the single-most deterrent to the use of OER is the time wasted in search of materials that truly align to the course outcomes. OER has the potential of leveling the playing field.  If instructors can find quality content, free-of-charge and aligned to outcomes, then the majority of their time can be dedicated to designing interactions between students, between students and their instructor, and between students and the content.  In short, instructors would have more time to address the cognitive and social needs of their students.

In contrast, I’ve observed too many instructors burning up their time in producing text content.  The acts of writing, finding and organizing content challenge instructors.  It is a lot of work – and yet, only the beginning of the effort.  Instructors complete the marathon, only to find themselves at the starting line of another.  Organizing content isn’t the end of it. What about motivating students, establishing relevance, developing clearly understood expectations and syllabi and other course documents? What about the stumbling points in the curriculum and the prerequisite skills and the recall of prior knowledge and the assessments and discussions and capstone projects?  How about usability and analyzing whether or not the activities promoted or impeded the outcomes?

OER can help lessen the load on instructors  – but, as a community, we need to uncover a process that makes it easy to find OER and, in the future, align activities to OER content.

One example of alignment is that of LearningPod with OpenStax. For example, OpenStax offers an introductory text on biology. LearningPod offers a test bank that is matched to that open text book.

Many for-profit entities are leveraging OER faster than universities.  Adaptive Learning Vendors (Knewton, CogBooks and Acrobatiq) are using OER in their content delivery systems.  Their value-added is in the learning paths they have generated, the mapping of prerequisite skills to targeted learning outcomes, decision-making algorithms, and the analytics that are generated on time-on-task, confidence and performance.

Conclusion

OER matched with activities, discussions, and assessments are an effective strategy for busy instructors who wish to maintain control over their courses.

Large higher ed systems like SUNY and Minnesota State have the opportunity to incentivize faculty to develop, share and evaluate resources.   In the past, such efforts were too small in scope and scale to succeed.  Times are changing.   Today, we are achieving a critical mass in many content areas.  We need mechanisms (application programming interfaces, import tools, discovery standards, metadata standards, package exchange notification services, etc.) to align and integrate the types of activities that this web journal is dedicated to:  case studies, decision making scenarios, leveled challenges, geolocation-aware activities, simulations, games, and stuff that will help students understand, apply and synthesize the content.

In short, we need the option to take charge of our courses and help students succeed.  I’d love to hear your thoughts on Open Educational Resources.  Successes?  Great resources?  Concerns? Please register and share!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The Research Behind Learning Interactions

Introduction

A nursing instructor is challenged to design an online course on standard classification systems. One could easily picture the course defining a classification system and describing its benefits. Students may be asked to compare and contrast how each system classifies problems, treatments and outcomes. The course may feature presentations, discussions, papers, a final exam – content items that are common in today’s online course.

Its easy to be complacent about this sort of design. After all, the information is clearly presented, and students respond well in the discussions, in their papers and on the final.

What more is needed?

The argument in favor of additional instructional components is difficult to make. Motivating videos may be hard to find or expensive to produce. Occasional checks for understanding in the form of multiple choice, multiple select and short answer questions require additional time and skill in using the learning management system or third-party tools. Interactive cases that feature stories and make the content come alive are even more time consuming and dependent on instructional and/or technical skills. In the case of standardized classification systems, cases may be a great way to ensure consistent use of the systems (i.e. Inter-rater reliability).

learning

CC BY-SA 3.0 Nick Youngson

Designing and developing activities for your online course that effectively engage students with the content takes time and know-how. It is easy to be skeptical about their value. You have precious little time. And you may not be certain that it is worth the time and effort. As a higher ed instructor, it is likely that no one pays you for the extra time and effort. The motivation comes intrinsically from successful students and a job well done. Or should we just concede this area of development to the publishers? They obviously have the skill and resources and economies of scale.

I am hoping you will reject that thought.  You already apply time and know-how when putting together an online course.  You  make dozens of decisions. You make decisions related to selecting and sequencing content, organizing content, deciding on wording and style, and choosing media.

But is this where we should draw the line and not attempt designing activities that sponge up time – without much evidence of return on investment. Or is the evidence there and we just don’t know it. What does the research tell us? Does it make a compelling case in favor of the extra effort?

Do we even know what is effective?

Instructional Designers were asked what learning activities they would build for each level of Bloom’s taxonomy. (Benjamin Bloom, as you’ll recall, categorized goals of the learning process in six levels, which included knowledge, comprehension, application, analysis, synthesis and evaluation.) Getting back to the instructional designers, there was remarkable consistency between them regardless of age and gender and other factors. For lower levels of the taxonomy like knowledge and comprehension, designers chose drill and practice, programmed tutorials, demonstrations, and simulations. For higher levels, they chose such interactions as problem solving labs and case studies. The research tells us that, at least, there is a common practice – but is there evidence for it?

Let’s survey the research to decide if instructional designers were even on the right track. Do interactions make a difference? We’ll examine learning interactions at the most elementary level and then climb higher and see what the research has to say about higher order activities.

What is a learning interaction?

An interaction in this context is characterized by the contact between students and the materials of study. The contact involves student responses to stimuli (multiple choice, multiple select questions), practice and feedback (puzzles and flashcards), branched instruction (decision making scenarios, interactive case studies), categorization (matching, sorting, and ordering activities) analysis (review of text with open feedback, underlining, circling), manipulation of inputs and outputs (simulations, controllable animations, digital lab experiments), finding information (WebQuests), solving problems (problem based learning scenarios), evaluating (decision-making) and creating (proposals, diagrams, digital drawings, code-writing).  For the purposes of this post, we’re less interested in the passive reading of text, or the watching of and listening to media — although I’ll concede that a broader technical definition would include any activity that results in cognitive change such as the recall of prior knowledge and the acquisition of new knowledge.

Research — Buyer beware

To return to our research quest, I’ll admit that I often look to others for the interpretation of what is significant and meaningful. Reviewing research takes time and skill. Nevertheless, I am drawn to the analyses, the statistical methods, the inferences and the statement of results. Using educational research can be tricky. There are caveats. Single papers can offer us the wrong conclusions or may not be applicable to our situation. Compared to a drug study or a health-related study, sample sizes in educational research seem small and not generalizable.

Finding educational research that uses a control group is a helpful first step. A control group of students is separated from an experimental group in such a way that the thing being tested cannot influence the control group’s results. Finding such research can be a challenge. Here is the kind of statement we see in educational research.

On analysis, the comparison of the overall knowledge scores pre- and post-treatment showed a statistically significant increase from 8.8 to 11.6 (P< 0.001).

Educators claim victory because post test scores were better after a treatment than before. That begs the question, compared to what? Perhaps the treatment was the most inefficient and uneconomical means on the planet to raise those test scores.

A simple paired T-Test may be preferable, in comparison, when we look at the difference between a pre- and post-test for a experiment group compared to the difference between a pre- and post-test for the control group.  For an explanation of T-Tests, please visit:  http://blog.minitab.com/blog/adventures-in-statistics-2/understanding-t-tests:-1-sample,-2-sample,-and-paired-t-tests

If I am looking at the effectiveness of a particular treatment, I will look for research related to higher education and my discipline of interest where the treatment is represented by the independent variable. The dependent variable might be, for example, student performance on a test score, student satisfaction measured with a survey, or student time on task. Experiments with pre- and post-tests without control groups can be problematic. A classic example is that of Hermit the Bug.

In our shop, we often ask this question: Should we include an animated character to serve as a guide or aid. In a project that I worked on years ago, we developed an amusing little character who would zip around the screen and guide the student along the learning path. The character was fun. It livened up the presentations. But the decision was based on no research at all.

Research literature calls this the persona effect.  A paper titled  The Persona Effect: Affective Impact of Animated Pedagogical Agents concluded that their potential to increase learning effectiveness is significant. 

The researchers surveyed students who rated Hermit The Bug’s entertainment value, helpfulness compared to a science teacher and so forth. Since this paper was published, 15 studies have been done. 9 showed no effect, 5 showed mixed results and only 1 showed an effect.  (Do pedagogical agents make a difference to student motivation and learning? Steffi Heidiga,∗, Geraldine Clareboutb,1 a née Domagk, University of Erfurt)

To be fair, the research on Hermit the Bug cited the following benefit:

Because these agents can provide students with customized advice in response to their problem-solving activities, their potential to increase learning effectiveness is significant.

One wonders, ‘Is it the persona effect that is contributing to greater motivation and improved learning or is it the feedback that the bug provides?’ Feedback is feedback, even when it’s delivered by a bug.

I point this out to underscore the difficulty of research and the folly of relying on one study. There are, of course, other problems that people cite about educational research in particular. Examples include research that is not objective and has a hidden agenda;  over-generalization from a research focused on a very specific context; and frequent lack of peer review.

Common sense often exposes the weakest research – which seeks to promote a philosophy or product or particular point of view.

This blog is an example.  I have a vested interest in instructors choosing to design their own interactions – because I am the creator of an eLearning authoring tool.  But in my defense, there are two critical points. First, I developed the authoring tool because of a belief that learning interactions make a positive difference. And,  more importantly related to this post, I was quite prepared and open to research that reported on the insignificance and possible detriment of learning interactions. The truth is that learning how to create compelling and effective learning interactions  — let alone creating them — takes time.  It takes more time than we typically dedicate to this kind of training.  If I were to convince instructors to make the investment, I had to be certain of its benefit.

In the 80s, Michael Moore described student to content interaction as “a defining characteristic of education” and “without it there cannot be education”. That’s not to say that student-to-student and student-to-instructor interactions aren’t important.  Much has been written about them and there are many best practice examples.  I value the blending of all three types of interactions in an online course – but my line of inquiry had me questioning the importance of student-to-content interactions, specifically, and investigating their importance.

The Research

I asked what interactions are important in an online environment and what level of development effort begins to produce diminishing results.   I’ll cover the first part in this blog post and the second part in a future post.

One piece of research (mentioned in the previous journal entry) is described in a paper called Effects of Instructional Events in Computer-Based Instruction, conducted by a group from Arizona State University.  In traditional curriculum and design programs taught in the 80s and 90s, Robert Gagne’s Conditions of Learning was gospel. He proposed nine essential element of instruction. Well what would happen if we removed any essential element from the nine?  Would it make a difference?  The researchers created six versions of a program:  a full version without anything removed; one without statement of objectives; one without examples; one without practice and feedback; one without review; and a lean version that presented information only.

Think about this for a moment before looking on. Removing one of these things really made a difference. Which one? What was the result?

As it turns out, removing practice and feedback makes a difference.  And that is reassuring.  Many of our activities are designed to provide students with instant feedback.  We provide information; elicit a response; and then provide feedback.  That is worth the effort – provided that we can construct such an activity efficiently and economically.

In another research study (published in Journal of Educational Computing Research),  Hector Garcia Rodicio investigated whether or not requiring students to answer a question made a difference. I’m referring to just the physical act of selecting a check box or radio button.  In the treatment group, students were required to answer a question before getting feedback.  In the control group, students received all of the same information, but they were not required to perform the physical act of selecting an answer.

Does having to answer make a difference?

Apparently it did.  Richard Mayer (University of Santa Barbara) explains why.   When students have to answer questions they actively select relevant segments of the material, mentally organize them, and integrated them with prior  knowledge (Campbell & Mayer, 2009).

The action is not insignificant.  Look at the results. 

2017-11-13_1133

A table showing mean scores and standard deviations on questions that recalled prior knowledge, supported retention and aided transfer

Note the difference an interactive question makes on retention and transfer. M is the mean score.  SD is the standard deviation – a measurement of variation in the scores. Given the standard deviation, we can conclude the difference of four points is significant.

2017-11-13_1113

A LodeStar interaction that follows a presentation on Creative Commons licensing. Six questions check the learner’s understanding and ability to match the license to the requirement.

About the Meta-analysis

As mentioned there are significant pitfalls to education research.  However, a particular type of analysis might provide us with more direction – the meta-analysis.  The meta-analysis combines studies and typically includes many more students – many more samples – than the single study.  But meta-analyses are not without their own issues.  Because the meta-analysis is so common in educational research let’s explore them for a moment.

To start, the Merriam Webster definition of a meta-analysis is this:

A quantitative statistical analysis that is applied to separate but similar experiments of different and usually independent researchers and that involves pooling the data and using the pooled data to test the effectiveness of the results

A health study meta-analysis might involve dozens of studies involving thousands of individuals.  The significance of a treatment is reported as an effect size.  An effect size is the magnitude of an effect of an independent variable on a dependent variable in an experiment.  Twenty studies, for example, might measure the effect of room noise on reading and test scores.  Let’s hypothesize that lower room noise might lead to improved reading comprehension, which leads to better student performance on a quiz.   If you pooled all of the studies together, can you conclude that lower room noise really makes a difference.  What is the magnitude of this difference?  In other words, does the meta-analysis show an overall significant effect size? 

Wil Thalheimer and Samantha Cook have done a great job on simplifying the concept of an effect size.  Because it is so prevalent in research and this blog entry is about research that can inform an instructor’s decision making, I will summarize it in simple terms.

The recipe for effect size goes something like this.  You calculate the mean of the treatment condition and subtract from it the mean of the control group.  In our example, we are looking at the performance scores of students who read in a quiet room versus a noisy room.  We calculate the mean test score of students who read in a quiet room.  We then calculate the mean test score of the students who read in a noisy room.  We subtract one from the other.  That gives us the numerator of a fraction.  We then divide that number by a pooled standard deviation.  We won‘t know if a difference in means is significant, unless we know something about variation.  That’s what standard deviation tells us.  Is a 10 point difference significant or not significant?  Thalheimer and Cook show us how a pooled standard deviation is calculated.  In the end, if we have a standard way of calculating significance – effect size – then we can analyze a group of studies even though individually they have different scores, ranges, means, and average departure from the mean.

As mentioned, meta-analyses draw their own criticisms. Two of the issues cited in meta-analysis.com, a proponent of meta-analyses, is that experiments that don’t show significant results are tucked away in file drawers collecting dust.  That introduces a bias in the published research.  It is called, logically, publication bias.  Secondly, meta-analyses may combine apples and oranges.  The following link explains the shortcomings in more detail.

https://www.meta-analysis.com/downloads/criticismsofmeta-analysis.pdf

Despite the criticisms, meta-analyses can provide convincing support for a treatment and results that are generalizable beyond the context of any particular study.

Meta-analysis can also give us insight into instructional strategies that make a difference. In a paper titled Comparative  effectiveness  of  instructional  design  features  in  simulation-based  education:  Systematic  review  and  meta-analysis, published by the National Center for Biotechnology Information, the authors analyzed 289 studies that involved more than 18,000 trainees.  The following instructional strategies were found to be relevant in simulation based education: range of difficulty, repetitive practice, distributed practice, cognitive interactivity, multiple learning strategies, individualized learning, mastery learning, feedback, longer time, and clinical variation.  Note the inclusion of practice, feedback and cognitive interactivity.  Engaging students with content in a way that makes them think is effective!

These themes come up again and again in the research.  Create activities that require students to apply what they have learned, make decisions or choices or perform some sort of action, get feedback and apply that feedback in future activities until mastery has been achieved.  Vary the difficulty; make them think; make them practice; provide feedback and support mastery.  It is still difficult to determine whether its worth the effort to construct such activities – but at least we are on the right track.

Activities that require students to perform and receive feedback can be fairly efficiently created.  Higher order activities take more time.  What does the research suggest in terms of the effectiveness of higher order activities.  One such activity is the interactive case study.  Let’s look at the case study in some depth.

In a paper titled Effectiveness of case-based teaching of physiology for nursing students published in the Journal of Taibah University Medical Sciences, the authors reported that

The performance in tests was statistically significantly better after didactic lectures (mean, 17.53) than after case-based teaching (mean, 16.47) (two-tailed p = 0.003). However, 65–72% of students found that case-based teaching improved their knowledge about the topic better than lectures.

 

Teaching method

Mean

SD

SEM

p

Didactic

17.53

3.58

0.38

0.003

Case-based

16.47

3.69

0.39

The first part doesn’t sound very supportive.  Students performed poorer in the treatment that included case based teaching method. This underscores one of the challenges of measuring the effectiveness of a treatment like case studies, or problem-based learning or decision-making scenarios and other higher order activities. If I simply taught to the test, students might perform better on the test than engaged in a case or some other ‘indirect’ activity.  But what is the effect on satisfaction or long-term retention or transfer of knowledge to the work setting?  The research may exist and may answer that question, but the quest for that insight is long and arduous.

The author of the above study conceded that several studies contradicted his findings.  In fact, there are several research studies that support the use of case studies in both online and face-to-face settings.  The following study concluded that case studies were effective whether created by the instructor or a third-party:  Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains (Kevin M. Bonney, 2015).

The impact of the case study method was significant. It produced a two grade increase.

The author wrote:

Although many instructors have produced case studies for use in their own classrooms, the production of novel case studies is time-consuming and requires skills that not all instructors have perfected. It is therefore important to determine whether case studies published by instructors who are unaffiliated with a particular course can be used effectively and obviate the need for each instructor to develop new case studies for their own courses.

This is significant. Case studies were found to be effective, whether created by the instructor of the course or by an instructor unaffiliated with the course. This supports the use of activities gleaned from content repositories. Case studies, however, are not equally available in all disciplines. In the sciences, instructors can find cases at the National Center for Case Study Teaching in Science. The most difficult and time-consuming challenge related to case studies is in their creation – just getting it down on paper.

At the National Center for Case Study some of the work has been done for you. A case on climate change, for example, provides background information on the meaning of climate change and how we know that it is occurring. The case study places the student in the role of an intern to a US senator. The job of the intern is to help the senator understand the science behind client change and the impact of climate change on the planet. The student is engaged in a number of questions that require some analysis and charting.

The results of Professor Bonney’s research are taken verbatim from the author.

To evaluate the effectiveness of the case study teaching method at promoting learning, student performance on examination questions related to material covered by case studies was compared with performance on questions that covered material addressed through classroom discussions and textbook reading. The latter questions served as control items; assessment items for each case study were compared with control items that were of similar format, difficulty, and point value . Each of the four case studies resulted in an increase in examination performance compared with control questions that was statistically significant, with an average difference of 18%

In the following study  Effectiveness of integrating case studies in online and face-to-face instruction of pathophysiology: a comparative study (http://advan.physiology.org/content/ajpadvan/37/2/201.full.pdf) we learn the following:

  • Students who enjoyed the case studies performed better.
  • Students like case studies because they could apply what they learned
  • The reasons why students liked case studies had nothing to with whether they were in a face-to-face or online class
  • Students who expected to earn better grades as a result of the case, did actually earn better grades.

Efficiency

Concerning to me is the amount of time that it takes to generate the interactive case study.  Because of this concern, we are investigating and piloting the use of templates – but not at the expense of student performance and satisfaction.

At our university, we recently developed two versions of an interactive case study to promote the use of a standardized classification system to document, classify, and communicate health-related issues such as Latent Tuberculosis Bacterial Infection. In a future post, I’ll write about the two versions and their effect on student performance and student satisfaction.  One version came from a template and involved more student reading.  The other version sequenced audio with the presentation of content and made more use of graphics.  The premise is that the templated version can be created quicker and could be generated by an instructor rather than an instructional technologist. We’re looking at whether that ease and speed came at the price of student performance and satisfaction.

If we are made confident by research that interactive case studies improved both student performance and satisfaction, and if case studies can be generated effectively and efficiently through a templated approach, then we can improve on our return on investment.

We could also further efficiency by adopting cases from case libraries. In our standardized classification system example, 18 cases are available at the Omaha System – a vendor site:

http://www.omahasystem.org/casestudies.html

Related to science case studies, I have already mentioned the National Center for Case Study Teaching in Science – but there are other resources that might uncover case studies in other disciplines. Examples include the learning object repositories like Merlot (www.Merlot.org) and OER Commons (www.oercommons.org).

Again, the repositories supply the content. Couple the content with an eLearning authoring tool like Captivate, StoryLine, LodeStar or whatever to make it interactive and you might be able to produce an effective instructional component efficiently.

 Conclusion

I set out to find research that contradicted my belief that learning interactions are useful and represent a good return on investment. I found that research. One can find examples that show discouraging results – but these are the exceptions. I found much more research that underscores the effectiveness of learning interactions, whether they be simple question items or sophisticated case studies. Now the focus should shift to producing these learning interactions efficiently.

My personal belief is that in higher ed we are at an important junction. We can concede this sort of development to the book publishers – or we can figure out ways to encourage instructors to build learning interactions and add value to their courses – for the benefit of online students.

 

 

The Problem with Learning Objectives

Introduction

At some point considering the learning objective can be tiresome. We want to do exciting things and make learning happen in our online courses, but we have the requirement of good housekeeping to attend to. “Inform the learner of the objective.” In too many cases, this information just takes up space and earns the instructor a meaningless check on a quality rubric.

It doesn’t have to be that way. We can look at learning objectives in a different light.

In fact, we’ve muddied the water with our use of learning objectives. We’ve confused the role of the objective as a design tool versus a communication tool. The learning objective can be both – but we must be intentional about it.

Principles of Instructional Design

Principles of Instructional Design was once a staple in university curricula

Instructors who have had any pedagogical training likely will have been introduced to Robert Gagne’s Nine Events of Instruction. The events include gaining the learners’ attention, informing them of the objectives, stimulating recall of prior learning and so forth. All too often, following the 9 events like a recipe card leads to the obligatory screen that dutifully lists all of the course’s objectives.

We see it all of the time in online learning – the topic that lists all of the objectives, sometimes well written, sometimes not.

A study conducted by Florence Martin, James Klein, and Howard Sullivan (Martin, Klein, Sullivan, 2007) and published in the British Journal of Educational Technology looked at a computer literacy course that was designed with various treatments with one key element of instruction removed. Elements included statement of objectives, examples, review and practice. The treatment that removed the statement of objectives did not show a drop in scores. It didn’t matter if the objectives were left out or in. So why the obsession with objectives? By comparison, the treatment that removed practice showed a significant drop. (1)

The treatment that included objectives included one screen per section. The screens, we’re told by the researchers, ranged from 79 to 82 words per section, not dissimilar to how we use objectives in online courses today.

In an attempt to improve on the use of objectives, we remind ourselves that objectives should meet the conditions of audience, behavior, context and degree. The advice is good, but only in the context of design. When objectives are used as design tool, it makes sense to think about audience, behavior, context and degree. But when objectives are a communication tool, we need to question whether learners want to read a technical objective rather than a statement that excites and motivates them to engage in the online course. In short, technically correct instructional objectives are the tool of the designer – what students need is quite different.

In at a least a couple of popular online evaluation processes, as reviewers, we look for objectives, judge if they are well written according to something like R.F. Mager’s Preparing Instructional Objectives, and evaluate whether or not they are aligned to the courses’ assessments and activities.

Clearly, the practice of writing objectives is important to the design of a course – but what should we communicate to students?

Research points us in the right direction – and the research uncovers a very clear problem with objectives. To explore objectives further, let’s separate our concerns. We’ll look at them from the designer’s point of view and then from the student’s.

From the designer’s point of view

Well-written objectives help instructors design courses well. They spell out the type of knowledge and the level of learning. We know that we need a very different type of learning activity to teach how to perform an angioplasty versus how to choose the type of coronary stent to use in a given situation. To state the obvious, the first objective requires observation, practice in a non-life-threatening situation (e.g. mannequin), and repeated practice under the observation of an experienced physician. The second requires knowledge of the critical patient attributes that favor one type of stent and procedure over another and practice with decision-making in increasing complex situations.

The design of instruction also improves when we specify the audience, the condition and the degree. Are these first year students with little surgical experience or quite a different group? Are the conditions optimum or do they simulate a more stressed setting? Related to degree, what measurement do we need in order to indicate that the student is performing the task well enough. Is an outcome of four out of five successful procedures or decisions good enough?

Specificity is important to the designer. Establishing specific objectives helps us choose the right assessments and activities. The principle of backward design requires us to start with well written outcomes and work backwards to activities.

As an aside, I would concede that, in the ‘wild’, instructors often start in the middle. We collect content; create activities – all in the process of discovering what we really want to do and what is important. Designing eLearning can be a discovery process and in that process we refine objectives, write new ones…toss out a few. This may be heresy to many – but it is an admission that designing instruction is a creative process. In the end, however, it is important for us to arrive at the objectives and then shine their light on everything in the course. In other words, review whether or not activities and assessments belong in the course and how strongly aligned they are to the outcomes.

Flavia Vieira, in her blog “Learning to Teaching” underscores the importance of objectives.

The way you choose to define them affects all that you do as a teacher, because objectives stand for what you believe is the goal of your and your students’ actions; they show your personal perception of the teaching-learning situation; they reflect your teaching and testing priorities; they determine your choice of activities and materials; they influence your teaching procedures, your attitude towards learner errors, even your teaching pace; ultimately, they determine the kind of learning that occurs in your classroom.  (3)

From the learner’s point of view

Some of the research shows that stating learning objectives does make a difference – but only when used correctly by both the instructor and the student. One interesting source is the Debunkers club and includes several targets (common misunderstandings) that the site uses research to expose.

The Debunkers club is curated by Wil Thalheimer and Paul Kirschner. Wil Thalheimer reviews educational research and distills their findings for the benefit of practitioners who either don’t know how to digest research or simply do not have the time.

Thalheimer states that:

The research that has been done on learning objectives has shown that presenting learners with learning objectives produces benefits because it helps learners focus attention on the targeted aspects of the learning material (Rothkopf & Billington, 1979). To be more specific, if a learning objective targets Concept X, then learners are more likely to pay attention to aspects of the learning material that are relevant to Concept X, and are less likely to pay attention to aspects of the learning material not relevant to Concept X.

Simply, if learning objectives are to be useful at all to the learner, they must be written in a straightforward manner that communicates to the learner what he or she should pay attention to. The learner should know clearly that the intention of this course is not to memorize the historical dates, for example, but state the significance of a specific event in history. Flashcards with dates won’t help the student. Remembering the details of a military campaign won’t help the student. Understanding the root cause of an event and its effect on the social-political environment of the time may be of paramount importance. The student should concern herself with analysis and not sweat the small details.

Sal Khan in his videos on permutations and combinations stresses that memorizing the formulas may impede understanding.  Rather than memorizing the formula for a permutation, students should be able to reconstruct the formula from their understanding of how it works.

In short, the statement of objective helps us to focus students on what is important. Thalheimer goes on to recommend against generally worded objectives. The more specific, the better. And he recommends against the multi-part objective (audience, behavior, context and degree) when communicating to students. Thalheimer summarizes research that underscores the importance of learning objectives in helping students set goals, focus on relevant information and to evaluate their learning against the stated objectives – all important meta-cognitive activities.

If we don’t communicate objectives to focus students’ attention on what is important, then we should, at least, excite students about the subject. In corporate training, we often see the WIIFM replace the listing of objectives. The WIIFM or ‘What’s in it for me’ stresses the relevance of the learning to the learner. Research does support the role of motivation in learning.

Finally, instructional objectives can be dangerous. If we get complacent with good test results and declare ‘mission accomplished’ on our objectives, we have missed the whole darn point. The purpose of training and education is the transfer of learning. In training we want to see business results. In education, we want to see the online course contribute to the development of the student and success in future courses and beyond. Just as the ill-conceived learning objective takes up space on the page, so too the badly designed course in the student’s life. The course is part of a meaningless exchange of dollars for credits.

In contrast, the meaningful objective contributes to student learning and plays a part in a well-articulated curriculum that promotes student’s growth in the course and beyond. We need to be intentional about our use of objectives. And then move to more interesting stuff.

Resources

  1.  Martin, F., Klein, J. D., & Sullivan, H. (2007). The impact of instructional elements in computer-based instruction. British Journal of Educational Technology, 38(4), 623-636. doi:10.1111/j.1467-8535.2006.00670.x
  2. http://www.debunker.club/
  3. http://aliancistatlv.blogspot.com/2012/08/language-learning-objectives-do-make.html

Interactive Case Studies

Introduction

The research on interactive case studies supports their use in higher education.  The use of interactive case studies contributes to student motivation, sense of relevancy, higher course grades and overall satisfaction.  One research study, “A Usability Study of Interactive Web-based Modules” looked at the use of interactive case modules in a Principles of Marketing course.  In their literature review, the authors observed that:

Case studies are typically used by marketing educators to help students gain real world knowledge and learn marketing concepts (O’Connor and Girard 2006) and are important tools for students to develop their analytical thinking and problem-solving skills through applied construction of reality (Henson, Kennett, and Kennedy 2003).

But developing an interactive case study may seem daunting.  Instructors might feel the need to master all of the nuances of this genre before attempting to make one of their own.  The interactive case study (as distinguished from the face-to-face experience) adds the complexity of the technology.  There are however small steps one can take and templates that make interactive case studies easier to generate.

An example

Dr. Debra Eardley, a nursing professor at Metropolitan State University, recently completed an interactive case study in support of nursing informatics and a standardized classification system.  She storyboarded the case study in PowerPoint and received help from the university’s Center for Online Learning to make it interactive.

She started with the basics.  The objective of the case was to help students ‘experience’ the role of a standardized classification system in documenting the problem, intervention and outcomes of a patient diagnosed with an infection.   The case followed a public health nurse as she interviewed a patient and followed the procedures of Directly Observed Therapy (DOT) and the administration of medication.  The student participant in the case study observes the interview, makes notes and then charts the problems, intervention and outcomes, as would a public health nurse using a standard classification system and an electronic health record system.

The case study was a simple one…with one set of right answers and not many gray areas.  The case study was a stepping stone to more sophisticated cases that will follow.  But despite its simplicity, the case study introduced knowledge that public health nurses need to know.  It introduced the concepts of Latent Tuberculosis Bacterial Infection (LTBI), Directly Observed Therapy, the role of the public health nurse, and the role of a standardized system with its relationship to evidence-based practice.  Rather than simply being told about these things, the student observes a public health nurse in action and practices charting using the Omaha Classification System, which is evaluated with immediate feedback.

Interactive case studies, of course, can be more complicated – but that should not deter any instructor from getting started with simpler cases.    The key is recognizing some of the basic benefits of the case study approach.  For example, Harvard Business School (HBS) case studies involve students in reading the case, discussing the findings with classmates, reflecting on alternative approaches, answering the professor’s questions and deciding on a course of action based on the case.  The basic case study attributes make them far more compelling than text-laden pages all too common in typical learning management systems.

The benefits of case studies existed long before the use of electronic media.  Again, in the area of health informatics, university pathology departments across the United States implemented interactive case studies with little electronic help – simply text and discussion.   The designers of The Healthcare Pathology Informatics Fellowship Training program patterned their case studies on the business case study method with the following attributes:

  • The scenario was based on a real life situation.
  • The participants must analyze the situation, decide on one or more courses of action and provide evidence to support their decisions.
  • Participants must read the case beforehand, understand the issues involved, and come prepared to provide answers for whatever the facilitator might ask.
  • At the conclusion, a narrative described what actually happened in the real-life situation.

The electronic interactive case borrows a lot from the traditional case study approach.   First, the case study scenario places the learner in a role and a setting.

image_1

Interactive Case Study Based on Instructional Design Methodology

In the screenshot above the learner is placed in the role of a faculty member asked to design an online course by her dean.  The interactive case study challenges the learner to pick the right questions in the right sequence that model the backwards design approach to online course design.  In short, the learner selects questions that probe the situational factors that define the context of the training, selects appropriate outcomes, designs assessments aligned to the outcomes and then develops activities that will help students fare well on the assessments.  This is the backward design approach.  A simple case study, represented by the screen shot above, could assess whether or not the learner understands the backward design approach.  A more sophisticated case study might lead to several options that can be equally right but that require the learner to explain the choice and back it up with data, citations, and/or evidence.

In Dr. Eardley’s Latent Tuberculosis Bacterial Infection (LTBI) case study, the learner observes the public health nurse and her patient and must take notes for a clinical summary.  The instructions for the Clinical Summary Exercise are a click away.  A tool used by learners to take notes is also a click away.

image_4

Interactive Case Study related to Healthcare

The learner’s clinical summary is assessed in two ways.  The learner must submit the clinical summary, which is then evaluated by an instructor, as well as answer a list of questions related to the clinical summary, which are machine scored.  The exemplar clinical summary is only shown to the student who has made the effort and correctly answered questions about the patient.

The Design of a Template

In another project, we’re reflecting on the necessary functions to build into a generalized case study template.

2017-08-02_2157.png

A Proposed Interactive Case Study Template

The screenshot above labels some of the key functions.  Violating Richard Mayer’s principles of multimedia design, an explanation for each label is found below.

  1. A content area that will define the role of the learner and goal of the study, introduce background information and present key decision points in the case.
  2. A set of tools that enable the learner to take notes and review a transcript of all key decision points and feedback.
  3. Resources and tips that are context-based.  As the decision points change so too the resources.  Some resources persist; others appear and disappear as needed.
  4. Not pictured, the template supports branching.  Optionally, content can be shown based on user preference and user performance.  Again, optionally, learners can be taken down different paths based on how the story unfolds and the choices the learner makes.

Conclusion

The interactive case study is an effective instructional design pattern that has deep roots in traditional text and face-to-face classes.  The interactive case study may seem challenging to create but simple case studies offer instructors a good starting point.  Finally, the template approach simplifies the construction of case studies so that instructors need not rely on textbook publishers but can generate their own.


  1. Tulay, Girard., & Pinar, Musa. (2011). A Usability Study of Interactive Web-Based Modules, Turkish Online Journal of Educational Technology – TOJET, v10 n3 p27-32.

Geolocation Storytelling

Introduction

A new form of storytelling and interactive engagement is unfolding. Location-aware storytelling enables educators to untether students from the computer and let them roam about the world freely….to hear stories and learn in new ways.

Today’s smart phone can connect to the internet and get its location from a GPS satellite. Educational apps (both native and browser-based) can read the location and display interactive content matched to the location.

The obvious applications are history and the natural sciences – but with a little ingenuity, geolocation storytelling can serve students from a broad range of disciplines.

Inspiration for a new kind of storytelling comes from a group of history enthusiasts, led by Robert Molenda. The group has taken on the name of Lens Flare Stillwater with the tagline ‘The future of Stillwater viewed through the lens of the past.’ Stillwater is a river town located on the Saint Croix River, which borders the states of Minnesota and Wisconsin. To view this town through the lens of the past, the group has combined the arts of storytelling and photography with the new technology of mobile phones and geo-location-aware applications.

Robert Molenda is a retired chemist and business executive from 3M. He and a motivated group that includes John Paul Moore, John Buettner, Dick Marlow and many others, set out to tell Stillwater’s story through photography and narrative. They use the LodeStar eLearning authoring tool, which includes a geolocation-aware template called ARMaker — an abbreviation of Augmented Reality Maker.

To tell Stillwater’s story, they select historical sites of interest and related photographs from the John Runk collection of historical photographs and combine then with their own photography and narrative. They use Google maps to identify the latitude and longitude of a location, and then input that location into LodeStar. They match the location with both audio and text narrative, select the photographs and work out the details – details such as: how many sites should be included in a tour; where should the invisible geo-fence be located that triggers the display of text and graphics; and how much information is sufficient.

This is their story as told by Robert Molenda, which we hope to inspire both formal and informal educators around the world across the disciplines:

The story of Lens Flare Stillwater

LensFlare

Screenshot of Lens Flare Stillwater, a site dedicated to revealing the history of Stillwater through location-aware applications.

The idea of this project started in May of 2015 when I sent a number of ideas for Stillwater to the Mayor of Stillwater. Among the ideas was the idea for Lens Flare Stillwater.

Imagine that you are a visitor for the first time to Stillwater, standing in front of Terra Springs Apartments. The Terra Springs location is active with a geolocation marker and your smart phone knows when it is inside the “geo-fence” of that location range. When this happens, a photo of this same location at an earlier historical time, appears on your smart phone along with pertinent historical information, an audio narrative and other digital photos that are part of that location story. In this manner, you as a visitor can experience “Augmented Reality” in an active location tour of Stillwater. You can touch, feel, read, listen to information pertinent to the actual location that you are near. As you move along in Stillwater and enter other active “geo-fences” your smart phone will trigger other information pertinent to these different locations.

The theme was to use the Historical Photos of the John Runk Photo Collection with today’s digital technology to put the history of Stillwater in everyone’s pocket or purse.

That was the basis for the idea. Since that time, we applied for a grant from the Stillwater Foundation, made contact with software developers, started a web site that provides a “Virtual Reality” tour of Stillwater and were fortunate to make contact with Lodestar Learning Systems, another software developer involved with educators.

The really difficult work of software development has already been accomplished by people like Sami Jitan of Pivot the World and Robert Bilyk of LodeStar Learning Systems. The job of our team of volunteers is concentrated on providing content consistent with software design legal requirements and visitor needs.

In summary, we are taking advantage of some truly great, high quality historical images, narratives/audio and combining them with geolocation information and software to provide an “Augmented Reality” tour of Stillwater, Minnesota.

Robert Molenda

An Example

Here is an example that can be experienced from the comfort of your office or home, but is best experienced on foot and in Stillwater.

https://lodestarlearning.site44.com/Stillwater/index.htm

Note several features:

  • Responsiveness
  • Location-aware
  • Media Support

 If you can’t visit our LodeStar Learning’s hometown of Stillwater with your smart phone, do the next best thing: Shrink the browser window down to the size of a smart phone. Notice the responsiveness. Students who access your learning management system from their smart phones will appreciate LodeStar’s ability to adapt to any screen size. Click on ‘Show Map’. If you are in Stillwater looking at this map, an info window pops up when you cross a geo-fence. Play the audio on a page. View some of the John Runk Collection from one of the image sliders.

All of this functionality combines with LodeStar’s other features: branching, quizzing, interactions, SCORM conformance, and accessibility.

For related articles from past web journal articles, visit:

Augmented Reality for Educators
https://lodestarlearn.wordpress.com/2016/10/23/augmented-reality-for-educators/

Mobile Learning
https://lodestarlearn.wordpress.com/2017/01/03/mobile-learning/

Strategies and Tools to Promote ‘Reading to Learn’ in Higher Ed

Introduction

In higher education, assigned readings challenge students in ways that we may not fully anticipate: culturally, linguistically and cognitively. Assigned readings challenge students if, on any given day, students complete the assigned reading at all!

The statistics on reading compliance are disheartening but not surprising, given students’ time constraints, divided attention and the inherent challenges of reading to learn.

Readings may require a cultural literacy to understand the references or analogies. They may require a highly developed vocabulary or a specialized vocabulary. They may also demand of students a prior knowledge, or a knowledge of specific principles, rules, and concepts. Instructors depend on students to complete the readings and understand them in order to participate in class or in online discussion groups and perform well on assigned papers and projects.

In their report on “Increasing Reading Compliance of Undergraduates: An evaluation of compliance methods” authors Sarah Hatteberg and Kody Steffy report that “studies have shown that no more than 30 percent of students complete a reading assignment on any given day.” In their study, they evaluate the effectiveness of strategies to get students to complete the assigned reading. Most effective were 1) announced reading quizzes, and 2) mandatory reading guides and questions. Least effective were pop quizzes and optional reading guides.

Getting students to read is a first step. Getting students to understand the reading and read deeply and critically is challenging.

In higher education, one can easily take the position that we simply assign readings to students and expect them to complete the readings and understand the readings sufficiently to participate in activities. A more enlightened approach might be to prepare students with motivators, advanced organizers, inquiry style questions, practice on critical concepts, self-checks and more. In other words, we can build activities that help student derive the most benefit from assigned readings.

Motivate Students

The most critical piece to getting students to read is motivation. Instructors need to address motivation head on by answering the following questions: After completing the reading, what will students know that they didn’t before? What will they be able to do that they could not do before? What relevance is the reading to the world beyond academia? If instructors can address these questions directly, students will prioritize the reading accordingly.

I recently heard an instructor say that students regard assigned activities (including readings) as a transaction. ‘I do this; you give me points.’ Students are given loads of stuff to read and to do. Selective reading – including skimming – is a survival skill.  Reading without a perceived direct reward gets lower priority.

So we can certainly quiz students ahead of or at the start of class. But that probably doesn’t encourage deep reading. We can be selective and give some of the readings the full ‘treatment’. By that, I mean, we can underscore the importance of the reading with a personal recording pleading the case. If a problem is central to the readings, we can look for a TED Talk or a short YouTube video that introduces the problem to students.

I’ll use a recent example that I experienced. In Minnesota, we generally enjoy a high standard of living and benefit from a good educational system – but that standard of living and access to good education is not equally open to all. Currently in Minnesota, families of color have median incomes half of those of their white neighbors. In a sociology class, students might be assigned an anthology of perspectives on what it is like to live in Minnesota for a person of color. Ahead of that reading, an instructor can use headlines, video clips, testimonials and other things to ratchet up interest in the issue of economic disparity in our state.

In my experience, inattention to motivation is prevalent in online education. Instructors put up course documents on grading policy and schedule of assignments – but neglect to get their students jazzed on the significance of the course to them. Michael Allen, in his Guide to e-Learning, laments that “Although outstanding teachers do their best to motivate learners on the first day of class and continually thereafter, many e-Learning designers don’t even consider the issue of learner motivation.” He is primarily writing about corporate eLearning designers, but I would venture that the same holds true in higher education. Examine the most popular rubrics for evaluating online education. Motivation is hidden in the rubrics and its importance is overshadowed by the rubrics’ attention to the issues of alignment, organization and communication. Michael Allen’s book goes on to reveal seven magic keys to enhancing learning motivation. His first magic key relates to helping learners see how their involvement in the course will produce outcomes that they care about.

Prepare and Engage Students

Prepare students for difficult readings with pre-training. Pre-training is one of the principles of multimedia learning featured in Richard Mayer’s research (co). Ruth Colvin Clark describes it as such: “The pre-training principle is relevant in situations when trying to process the essential material in the lesson would overwhelm the learner’s cognitive system. In these situations involving complex material, it is helpful if some of the processing can be done in advance”. Assigned readings can present essential material that may induce a cognitive overload. Pre-training may involve an advance organizer, graphical chart, an infographic, glossary or other aid to reduce the cognitive challenge of a reading.

One method of engaging students in assigned readings is to help focus students on the critical parts of the reading. Inquiry-based learning provides us with strategies that help focus students’ attention on the essential parts of the reading. Inquiry-based learning has many antecedents in educational practice, but the common theme is in helping students to think in advance of the reading by posing a burning question that needs to be answered; or asking students to consider what they know about this topic and what they not know; what do they anticipate that the reading will reveal to them (and then how does the actual reading differ). Inquiry-based learning can take on multiple forms. Instructors can generate questions for the students to answer. This is the most structured level of inquiry-based learning. Students can generate their own questions based on their interest. This is the most open and purest form of inquiry. There are several shades in between. Instructors can adapt the best approach and level of inquiry based on the students’ sophistication and need. The overall goal is the same. Deliberately select strategies to prepare and engage students in the readings.

Provide Direct Instruction on Concepts

We can choose to assume that students will complete the readings and understand concepts. That may, however, be a dangerous assumption. Sarah K. Clark in her post on “Making the Review of Assigned Reading Meaningful’ assumes differently. She asks her students to create a ‘top ten’ list of important concepts. This illuminates what students judge to be important and helps to uncover misconceptions about concepts. If we accept that student understanding of key concepts is essential, we can plan activities that directly address concept learning.

A learning object can be tremendously useful in promoting concept learning. A learning object, in this sense, is simply a learning activity that is authored with the help of any one of dozens of eLearning authoring tools and uploaded to the learning management system. The activity could help students categorize the examples and non-examples of a concept. For example, the concept of a ‘chemical reaction’. A chemical reaction occurs when the chemical composition of matter changes from one thing to another. An example is found when an acid is mixed with a base, resulting in the formation of something new: water and a type of salt. Many things, however, appear to change physically, but don’t change in chemical composition. These are non-examples.

A learning object can not only help students sort out examples from non-examples but identify attributes of a concept and engage in the elaboration of a concept. The elaboration model (in instructional design parlance) starts with simple examples that can be easily categorized and progresses to more challenging examples that are more difficult to categorize. We can help students to generalize (apply the attributes of a concept to unknown cases) and not to over-generalize. The key here is direct instruction. We are not assuming that students have understood the concepts presented in a chapter in either simple or complex form, but we are engaging them with the concept and helping them to think about it.

To further promote concept learning, we can ask students to create concept maps, Frayer models (which include concept definition, association, examples, and non-examples) and create analogies in their own words.

2017-03-07_2040

The LodeStar eLearning authoring tool was used to create learning activities that challenged workshop participants’ understanding of declarative knowledge and concept learning based on the reading of Patricia Smith and Tillman Ragan’s book titled Instructional Design.

Use the Reading

The literature consistently refers to the strategy of ‘using the reading’. Concepts learned in a chapter can be immediately put to use in an activity that involves analysis. Students in a political science course who read about federalism versus republicanism can apply their understanding to the analysis of a case study. They can be asked to judge whether or not the case is an example of the ideology of a Jefferson style republican or Hamilton style federalist. A timeline could show the change of meaning of the concept of republicanism over the decades.

Readings are important towards understanding the content, performing well on assessments and writing papers. In some courses, the assessments, papers and projects may be summative in that they are the culminating activity and not the building activity. As an alternative, we can design shorter activities that require students to use the reading. We can ask students to cite the readings in their discussion forum. We can ask students to create timelines or concept maps from the reading. We can ask students to produce charts related to what they already knew, what they now understand and what they don’t understand. We can ask students to produce an outline of the reading …. and the list goes on. Once we have students produce something, we can provide feedback. In that way, we have engaged students in a ‘building’ activity. We are helping students to build their skills.

Conclusion

The key to all of this is the attitude that we are going to do something deliberate and strategic. In higher education, we can no longer put the onus on students to complete the assigned readings, understand the readings and apply the concepts and principles appropriately. Students noncompliance with reading assignments is one reason; college dropout rate is another. A variety of strategies and tools helps us in this cause. Strategies and tools range from inquiry-based learning to motivating videos to learning objects that promote understanding of concepts. Online instructors can use strategies and tools to flesh out their courses and transform them from an assigned reading/high stakes assessment paradigm to one that directly addresses student learning.