Tag Archives: Online Courses

Course Design – Plan for Evaluation

Evaluation, like needs assessment, is not always given the attention it requires in the process of instructional design. In real world situations, the timeline often drives the work and is usually too short to fully incorporate everything that should be done.

Creating an Evaluation Plan, as part of the initial design, helps you to make a lot of decisions before getting underway and to integrate evaluation tasks as you move forward with a project.

Your Evaluation Plan should include at a minimum:

  • List of objectives for the evaluation – why are you evaluating the instruction and to whom will the results be reported?
  • Description of the data you need to collect and why – what kind of information do you need to collect in order to find out if the instruction is effective?  This can cover a wide range of measures, including:
    • Content accuracy
    • Learning outcome achievement
    • Usability of delivery format
    • Cost-effectiveness of the project
  • The logistics of how the evaluation will take place – How, when, where, and who will be involved in evaluation? Will you use surveys, administer tests, conduct interviews, etc.?

There are a lot of options in terms of models. You’ll find these to be very comprehensive in most cases. Consider creating a customized plan for your project or work context.

There  are full examples of  evaluation plans available online. Two to review:

What is your experience with evaluation as part of the instructional design process? Please consider sharing your experiences related to priority, timeframe, and method. Is evaluation conducted by members of your design team or by an outside group?

Photo credit: Pink Sherbet Photography, Flickr

Motivation and Online Learning

Recently I attended an online lecture by Dr. Richard E. Clark.  This post is a summary of the information and resources presented in this live session and ideas on the application to the design of online learning.

How does motivation impact learning? Dr. Clark provided an overview of motivation for learning and sparked discussion about the role of games and simulations. He got the group’s attention immediately with a couple of numbers: motivation accounts for 30% of learning and 60% of transfer.

Motivation is a behavior – well, three behaviors, actually:

  • Starting a new task
  • Persisting once the new task has been started
  • Exerting mental effort, investing, in the learning process.

Motivation problems result from not performing these behaviors:

  • Not beginning a new task on time
  • Not persisting in newly started task – distractions!
  • Not exerting mental effort to learn something new.

These problems are not hard to find among online students.

What can course designers do to help students?

  • Provide clear instructions that include the details necessary to proceed at the start.  Do they know what needs to be done? Dr. Clark stated that most people would rather be considered difficult than inept or incapable. Many won’t ask for help or clarification prompting a motivation related issue before the course is begun.
  • Provide clear expectations regarding deadlines – this might include milestones, course calendars, reminders to plan ahead for future assignments and course requirements.
  • Consider issues related to cognitive load. These could lead to a problem with mental effort. Dr. Clark highlighted Frank Paas’s Mental Effort Scale, a single item that measures students’ perceptions of their cognitive load while engaged a specific learning task.
  • Consider possible impact of learner characteristics on motivation. What motivates any individual to start, persist, or invest in learning will be different from the next person. Our own motivations are not those of the students we are designing for. Culture, age, beliefs, and personal experiences can all play a part in this.
  • Present to students the value of the learning event, outcomes, and experience. What is the risk of not learning?

What can faculty/instructors do additionally to help students?

  • Communicate with individual students – are there other problems going on in the student’s life that may result in failure to start, persist or invest, but are not related to motivation?
  • Provide feedback that confirms the student’s ability to complete the task, but guides him/her to identify why they may not be motivated to complete the task.

  • Monitor student effort throughout the experience and provide feedback tailored to the specific kind of motivation problem. Attribute the learner’s success and failure to his or her effort.

Engagement vs. motivation – Dr. Clark acknowledged the widespread use of the word engagement in online learning circles and reminded us that engagement is not motivation in total. He described “engagement as an alternative word for persistence; enthusiastic persistence”.

Entertainment vs. motivation – Games, simulations, and enjoyment of learning were all mentioned. Dr. Clark clarified that while technology rich practice environments can be helpful (such as simulations) games can be a distraction.

Resources

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman.

Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. Pajares & T. Urdan (Eds.). Self-efficacy beliefs of adolescents, (Vol. 5., pp. 307-337). Greenwich, CT: Information Age Publishing.

Dweck, C. S. (1999). Self Theories: Their Role in Motivation, Personality, and Development. Hove: Psychology Press, Taylor and Francis Group.

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109-132

Paas, F., Tuovinen, J. E., Tabbers, H., & van Gerven, P. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63-71.

Petty, G. (n. d.) Dweck’s Theory of Motivation. Retrieved from http://teacherstoolbox.co.uk/T_Dweck.html

Photo credit: Blue Turban Photography, Flickr

Course Design – Start with an Outline

From a project management perspective getting a brand new course moving can be a challenge. With a course revision, you’ve got a full draft right from the beginning in the form of the exiting course. With a new course, the momentum has to come from a complete stop. The work of Subject Matter Experts (SMEs) is critical in the design stage to ensure that the scope and sequence of the specific content and source(s) of content are all appropriate. SMEs who are also filling other professional roles, such as full-time faculty, need support that helps them to provide their expertise, stay focused on project goals, and complete assigned tasks on schedule.

The project manager/instructional designer can provide tools that both manage the process and result in the information developers need to build the course. For a new course, starting with a high-level outline can be helpful.  A simple table can serve this purpose: organizing thoughts and documenting a plan for the course. The format allows for moving things around and review by others on the team.

In the illustration below, columns list the components of a course unit and each row represents one unit (one week per unit is a typical – but not required or even recommended – way to plan).

Possible advantages of this approach:

  • Provides easy access for others on the team – to contribute, review, edit (Consider posting as a GoogleDoc or in a system that allows for file sharing and version control like Sharepoint or Basecamp).
  • Structures the course before moving to the more cumbersome and detailed development guide for full writing of the course.
  • Becomes a primary reference document for the course – you can go back to it.
  • Allows for division of labor later on – multiple people working on separate units at the same time.
  • Offers flexibility – add columns as needed (e.g. case study) and your own course/program nomenclature.
  • Provides documentation for approval at a critical point in the process – before development goes forward.

This is a plan, not a prescription. Not all units will require a synchronous seminar or have assignments due. In the next phase of development (writing the course content in a development guide) the outline may change somewhat, but it is there in the beginning to show the way forward.

It’s a simple tool that takes some time to complete early in the process. This time on the front-end will likely save your schedule later on. Take the time to frame the house before you begin to buy the furniture, or even put up drywall.

Do you use something like this to get a new course project moving? How do you provide initial support and guidance to content experts and course writers? Please share!