Open Education 2016

In this presentation, we present The RISE Framework (Resource Inspection Selection and Enhancement). Aggregate resource use has been collected using Google Analytics and aggregate assessment data has been collected using Open Embedded Assessments. The page resources and the assessment items are linked to a common set of outcomes for the course. By examining aggregate resource use and aggregate assessment data at the resource page level, poor performing resources can be identified. Poor performing resources may include low use and low grade resources, low use and high grade resources, or high use and low grade resources. The purpose of this presentation is not to provide prescriptive advice on how to redesign OER. Instead, we provide a framework to help identify resources that need to be improved. The RISE Framework is a 2x2 matrix with use on the x-axis and grade on the y-axis resulting in four quadrants.

Quadrant one is the high use high grade quadrant. This could be explained by the resources being effective, the assessment being effective, or having strong outcome alignment. Quadrant two is the low use high grade quadrant. This could be explained by the students having high prior knowledge, the outcome being inherently easy, the content being highly effective, or the assessment being poorly written. Quadrant three is the low use low grade quadrant. This could be explained by students having low motivation or high life distractions, the module having too much material, or the resources having technical or other difficulties with access. Quadrant four is the high use low grade quadrant. This could be explained by the resources being poorly design, the assessments being poorly written, the outcomes being poorly aligned, or the learning outcome being inherently difficult. Resources that are located deep within quadrants 2, 3, and 4 should be evaluated for continuous course improvement.

Because the continuous improvement process can consume a significant amount of time and many institution's incentive systems provide greater rewards for other activities (like writing grants or publishing articles), most faculty and instructional designers never engage in the process systematically. The framework described above provides a fully automatable method for identifying the OER that could potentially benefit from continuous improvement efforts. Both the quadrant analysis and the resource type analysis can immediately provide faculty and instructional designers with the information they need to focus in quickly on the most problematic areas of a course. This framework (and others like it we hope to see emerge) can eliminate the need for significant investments of time and data science skill in the first step of the continuous improvement process - identifying what needs improving. We hope the availability of the framework will dramatically increase continuous improvement of OER-based courses as they continue to multiply in number.