This tool was designed by Knovation Learning to serve as a guide for evaluating the quality and effectiveness of free and open educational resources. It is used by the Knovation Content and Curriculum Team as the basis by which decisions are made during the curation process.
It is meant to be applied to a variety of learning resource types, such as videos, articles, activities, interactive media, and whiteboard resources. While it can be applied across large scale websites and lesson plans, it’s primarily meant to evaluate self-contained learning resources where the student is the primary audience.
It is composed of collections of rubrics, each grouped within one of three categories:
Often, OER rubrics favor rigor over usability. While this tool was designed to apply rigor to the evaluation process, it also places a strong emphasis on practical application and ease of use, particularly in situations where resource curation is performed at scale.
This is accomplished by performing the first round of evaluation on all resources created by a single Publisher. When scoring one or more resources using the Publisher rubrics, scores are awarded according to the actual characteristics of all resources that originate from a single source.
Resource rubrics examine both the assumed effectiveness (e.g., Ease of Use) and the actual effectiveness (e.g., Age of Information) for all resources, independent of format or type.
The third category of rubrics evaluates the assumed effectiveness of specific types of resources by examining characteristics that make them unique. Rarely will all rubrics in this category apply to a single resource.
As a resource is evaluated, a score is awarded based on whether or not it meets “Superior” or “Strong” criteria. If the resource does not meet or exceed “Strong” criteria, it is rejected.
Resources from Superior Publishers do not include advertisements, pop-ups, or content that distracts from the topic, concepts, or skills taught within the resource.
It is recognized that publishers must often include advertisements in order to fund the creation of content. Therefore, a resource can be considered strong if it includes 2 or fewer advertisements. Advertisements must be appropriate, and may not obstruct the use of the resource, or become an intrusion within its overall flow. Resources with advertisements that are not clearly marked, distract the user, or influence the user to focus on unrelated content are not considered Strong resources.
Resources from Superior Publishers do not require authentication for access, thus improving their accessibility for students and educators. Resources can be accessed from a publicly available URL.
Resources from Publishers can be considered strong if authentication is not required to access the page's contents, but it is used in order to store progress or save user created work. While rare, there are exceptions that may influence a curator to harvest resources that require free login (e.g., limited resources available on a topic).
Resources are considered Strong if student data practices are not clearly stated, but login is not supported, and users are not prompted to enter information that could be used to track their identity.
Superior Resources originate from a source with a strong reputation, are properly cited, and whose author(s) demonstrate no clear signs of bias. Superior Publishers are often institutions of higher learning, non-profit organizations, or government agencies.
Resources are considered Strong if the author’s qualifications are unclear, but the source has demonstrated a reputation for creating accurate, educationally-relevant content.
Superior Resources include information that is properly cited and can be verified as containing no substantial errors or inaccuracies.
Strong resources lack proper citation, but can be verified as containing no substantial errors or inaccuracies.
Superior Resources are designed to provide an engaging experience for students, often making use of visual elements, sounds, animation, and multiple forms of media that capture students’ interest and attention.
Resources can be considered strong if they express information primarily through text, but are written in a way that provides accurate and relevant information.
Superior Resources are created using technology that makes use of modern web standards, and presents information in ways that recognize observed usability and user experience best practice. Information is up to date and reflects the current understanding of the topics or concepts.
Resources can be considered strong if the website shows visible or functional signs of age, but information is up to date and reflects the current understanding of the topics or concepts.
Superior Resources are primarily focused on topics or concepts typically expressed within curriculum or delivered during instruction. They support standards or skills that are addressed by the majority of educators in classroom settings.
Resources can be considered strong if they can be applied to the typical curriculum with some modifications, or address topics that do not receive direct focus within instruction, but are commonly of interest to students.
Superior Resources require little up-front explanation for use by teachers and students, and no training is required to support immediate use by students.
Resources can be considered strong if use of the resource requires that students or teachers read and understand a short set of directions.
A resource has superior alignment only if both of the following are true:
A resource has strong alignment for either one of two reasons:
This rubric is only applied to learning resources that have suggested alignments to standards. This rubric follows the Achieve Rubric for Evaluating OER Objects: Rubric 1: Degree of Alignment to Standards
A resource is rated superior for the utility of materials designed to support teaching only if all of the following are true:
A resource is rated strong for the utility of materials designed to support teaching if it offers materials that are comprehensive and easy to understand and use but falls short of “superior” for either one of two reasons:
This rubric is only applied to resources designed to support teachers in planning or presenting subject matter. This rubric follows the Achieve Rubric for Evaluating OER Objects: Rubric 3: Utility of Materials Designed to Support Teaching
A resource is rated superior for the quality of its assessments only if all of the following are true:
A resource is rated strong for the quality of its assessments if it assesses all of the content and performance expectations intended, but the assessment modes used do not consistently offer the student opportunities to demonstrate proficiency in the intended concept/skill.
This rubric is applied to those resources designed to determine what a student knows before, during, or after a topic is taught. This rubric follows the Achieve Rubric for Evaluating OER Objects: Rubric 4: Quality of Assessments
A resource is rated superior for the quality of its instructional and practice exercises only if all of the following are true:
A resource is rated strong for the quality of its instructional and practice exercises if it offers only a sufficient number of well-written exercises to facilitate mastery of targeted skills, which are supported by accurate answer keys or scoring guidelines, but there is little variety of exercise types or formats.
This rubric is applied to resources that contain exercises designed to provide an opportunity to practice and strengthen specific skills and knowledge. This rubric follows the Achieve Rubric for Evaluating OER Objects: Rubric 6: Quality of Instructional and Practice Exercises