Category Archives: Instructional Design

Rubrics. Yes? No? Maybe…

Instructional design work is increasingly standardized. As this happens, data is collected to measure student learning outcomes and rubrics come into play. Lots of them. Instructors use these rubrics (charts with a rating scheme for each element of an assignment) to evaluate student work.

Rubrics provide a way in which the instructor can compare the quality of student work against a set of specific criteria. Ideally, if you have several sections of a course running, each with a different instructor, all will evaluate student work similarly using a standard rubric –  if two different instructors each evaluated Student A’s assignment using the same rubric, their individual evaluations would be the same.

There are pros and cons to the use of rubrics.

Rubrics can be helpful.

  • Rubrics encourage a more objective evaluation of a student’s work, reducing the possibility of comparing students to each other instead of the learning objectives.
  • Have you ever taken a course or submitted a paper and received a letter grade with no details about how that grade was determined? Rubrics can take some of the mystery away from the student’s perspective by clearly stating expectations making the grade seem less arbitrary.

Rubrics can be limiting.

  • Creating accurate ones that measure student learning of a specific outcome is not an easy thing to do. This process requires evaluation of the rubric itself to find out if it is reliable and valid.
  • The use of rubrics may result in less creativity from students working to check-the-box for each of the expectations presented in rubric categories and criteria.

Questions to consider:

  • Are rubrics always appropriate and effective? Think about types of assignments here – performance tasks, creative writing, etc. and context.
  • Who prepares the rubrics? I’ve experienced the hire of an assessment expert, assignment to instructional designer, and assignment to subject matter expert. Rubrics can also be found ready-made and there are online ‘rubric makers’.
  • What about reporting? Are rubric scores/ratings useful beyond the classroom to drive changes in curriculum at a higher level?

It could be argued that while rubrics can and do serve a real purpose, there is a point at which they can become too prescriptive. In this case, the focus becomes the measurement itself. There is a personal piece to learning, something more organic, where a student puts together knowledge and gains skill through his or her own unique set of experiences. Static rubrics can also reduce the ability of the instructors to assess student work from their unique perspectives and expertise. Difficult to capture these things via rating scale. What are your thoughts on pros and cons, your successes and challenges with rubrics?

Resources for your continued exploration of assessment and rubrics:

Image credit: stock.xchng

Course Design – Plan for Evaluation

Evaluation, like needs assessment, is not always given the attention it requires in the process of instructional design. In real world situations, the timeline often drives the work and is usually too short to fully incorporate everything that should be done.

Creating an Evaluation Plan, as part of the initial design, helps you to make a lot of decisions before getting underway and to integrate evaluation tasks as you move forward with a project.

Your Evaluation Plan should include at a minimum:

  • List of objectives for the evaluation – why are you evaluating the instruction and to whom will the results be reported?
  • Description of the data you need to collect and why – what kind of information do you need to collect in order to find out if the instruction is effective?  This can cover a wide range of measures, including:
    • Content accuracy
    • Learning outcome achievement
    • Usability of delivery format
    • Cost-effectiveness of the project
  • The logistics of how the evaluation will take place – How, when, where, and who will be involved in evaluation? Will you use surveys, administer tests, conduct interviews, etc.?

There are a lot of options in terms of models. You’ll find these to be very comprehensive in most cases. Consider creating a customized plan for your project or work context.

There  are full examples of  evaluation plans available online. Two to review:

What is your experience with evaluation as part of the instructional design process? Please consider sharing your experiences related to priority, timeframe, and method. Is evaluation conducted by members of your design team or by an outside group?

Photo credit: Pink Sherbet Photography, Flickr

Breaking into the Business

Earlier this week a friend-of-a-friend contacted me with this question:

What are some tips that you have for someone who would like to get experience with curriculum design and development and eventually pursue a position in this field?

This new friend is currently working in student services, has some experience as an instructor, and is taking graduate courses in higher education. My response included the following:

Assemble a portfolio – Establish a website, blog, or use an online portfolio tool (like VisualCV) to collect examples of your work. This could include brief descriptions of the projects you’ve worked on in the past, screenshots of items you’ve designed and/or built, materials you developed when you were an instructor, a writing sample. As you plan and complete your course assignments, do so with your portfolio in mind.

Find your niche – What part of the process do you enjoy the most? What are your strongest skills? Designers and developers often wear a lot of hats – graphic design, technical writing, project management, programming, multimedia, editing, testing.  Do you prefer working on face-to-face courses, blended courses, or all online? Do you want to work in higher ed, K-12, industry? From your coursework and previous experience you can probably also list areas where you need more practice.

Seek out opportunities to practice – Instructional designers get better with each project. Are there projects in your area, where you could assist, that would help you gain experience (creating user’s guides, training materials, etc)? Are there opportunities for you to volunteer your design and development services in exchange for the experience and additional portfolio items?  For example, another friend is building a website for a community youth organization.

Emphasize related skills – You already have some very valuable experience. Curriculum designers and developers are almost always working as part of a team. Faculty members are usually on that team. Students are the reason for building the courses in the first pace, so your experience working directly with students – as an instructor and as an advisor – gives you a level of familiarity that will be an asset.

Network – Keep talking with professionals in the field and asking questions. Consider joining a professional organization (AECT, AACE, ISTE, ASTD, Sloan-C are a few – but there are a LOT more). Set up a profile on a professional networking system (such as LinkedIn), join instructional design groups, and participate in the discussions.

Look for a position – While you may not be ready to apply, you might be closer than you think. Register for job search ‘alerts’ related to curriculum design and development and read through the vacancy announcements as they arrive via email. What are they looking for in terms of experience, computer skills, etc? These announcements can be helpful in identifying areas where you need practice and items you might include in your portfolio.

What else would you recommend to someone trying to get their first job in instructional design? Other sources of experience or education? Do you think a degree in instructional design is required?

Photo credit: NotMicroButSoft, Flickr