Online Course Design Quality Review
The Florida Board of Governor's Online Education 2025 Strategic Plan identifies several strategic goals focused on quality practices. Starting in 2019, online courses within the State University System (SUS) that complete an online course design quality review will be tagged with a Florida Online Course Design Quality Designation. The system goal is to increase the number of online courses in the SUS with a Quality designation. To support this statewide effort, online courses at UNF follow the Quality Assurance measures outlined in UNF's DL policy. Specifically, within two years of initial delivery, online courses scheduled as distance learning must obtain the Florida High Quality (HQ) course design designation and renew that indicator every five years. Courses already tagged with Quality designation remain valid until expired.
To increase quality and accessibility and reduce barriers to student success, faculty are encouraged to submit their online course(s) for an internal QM review to receive the HQ designation. Any UNF faculty member who completed the TOL Foundation Course and developed their course as part of TOL-DCD or Program Transition can receive a stipend upon completing the online course design review process. View UNF's QM-Reviewed Courses
Quality Matters
UNF implements the nationally recognized Quality Matters (QM) Higher Ed. Course Design Rubric Standards for the review of online course design and systematically builds and evaluates online courses based on QM's research-based standards. The QM Standards specifically focus on course design rather than course delivery or content and assure that the structure of online courses promotes learner engagement. A founding principle that drives QM is the need for standards to reflect current academic research on effective learning. The initial QM Standards and subsequent modifications are based on the insights of teams of experienced online instructors and instructional designers and the best practices and standards promulgated by accrediting bodies and national and international organizations. In addition, the QM Standards have been examined for consistency with the conclusions of the educational research literature regarding factors that increase learning and engagement and improve learner retention rates. Each specific QM Standard is centered on relevant research and supported by literature from the field. For more information, visit the FAQ section and QM Promotion Resources.
QM-Certified Peer Reviewers
Critical to UNF’s implementation of Quality Matters is the assistance of QM-certified peer reviewers participating in the course design quality review process at UNF. To be eligible, full-time faculty members who have completed TOL and undergone a QM review of their own online course can register for online QM training courses, with the fees covered by CIRT. As an incentive, QM-certified peer reviewers receive a $500 stipend for every two internal reviews they complete at UNF.
Participating as a peer reviewer offers several benefits, including:
- Experience in a quality assurance process, enhancing your understanding of course design standards.
- Expansion of your professional community, connecting you with peers engaged in similar endeavors.
- Opportunity to review both internal and external courses, gaining valuable insights and ideas to improve your own courses.
- Enhancement of your professional development plan and portfolio, showcasing your engagement in quality assurance practices.
- Receipt of a $500 stipend for every two internal reviews completed at UNF, acknowledging your valuable contributions.
- Potential for additional payment from Quality Matters when serving as a member of national review teams, recognizing your expertise and involvement on a broader scale.
-
HOW TO BECOME A PEER REVIEWER +
There are two ways you can participate as a Quality Matters (QM) Certified Peer Reviewer.
- You can choose to be an internal reviewer, participating in course peer review teams only at UNF.
- You can aim to become a national reviewer, eligible to participate in both internal and official QM reviews.
To become a QM-certified peer reviewer, you must:
- Complete the TOL-DCD process (i.e., developed at least one course that passed a quality course review)
- Complete the two-week online training course Applying the Quality Matters Rubric facilitated by QM (CIRT will pay the course fee).
- Complete the three-week online training course Peer Reviewer Course facilitated by QM (CIRT will pay the course fee).
Note: Choose invoice as the form of payment at registration - they will bill us. If you already have a QM account, just login into the QM site and register for the training courses.
Getting Started
Before you register for the first online training course, please contact Kevin Hulen (kevin.hulen@unf.edu) to verify your eligibility and discuss the peer review process.
Internal Reviewers
Once you've completed the training (listed above), you will be listed on UNF's QM-Certified Peer Reviewer, and whenever we have a course ready for review, we'll reach out to check your availability. You will receive a $500 stipend for every two internal reviews completed at UNF. As an internal peer reviewer, you must:
- Review online courses using QM Standards and Rubric. The estimated time per review is 4-8 hours or less once you've conducted a few. You have 2-3 weeks to complete the review.
- Maintain strict confidentiality about any courses you access during the review process, including any student information contained in these courses.
- Not change, copy or modify any content within a course.
- Observe copyright laws and respect the intellectual property rights of faculty.
National Reviewers
If you want to get involved with conducting external (national) QM reviews with teams at other institutions, you must also:
- Have taught online within the last 18 months. If not, you must complete the Exemption section of the application documenting your qualifications for being a Peer Reviewer.
- Submit the Memorandum of Understanding, and resubmit annually (by updating information in MyQM.)
- Allow your name and institution affiliation to be listed on the national Quality Matters website.
Note: Quality Matters, not UNF, manages payment for participating in national reviews.
-
PEER REVIEWERS +
The following faculty members are certified to conduct internal QM Reviews at UNF, and some also participate in official QM-managed reviews (external). In addition, all instructional designers at UNF are certified to conduct QM reviews and actively participate in each internal QM review.
Current QM-Certified Peer Reviewers Name College Department Andrea Arikawa (Master Reviewer) Brooks College of Health Nutrition and Dietetics Hanadi Hamadi Brooks College of Health Health Administration Alireza Jahan-Mihan Brooks College of Health Nutrition and Dietetics Jen Ross Brooks College of Health Nutrition and Dietetics Jill Snyder Brooks College of Health Nutrition and Dietetics Nilufer Ozdemir Coggin College of Business Economics Richa Jethwani College of Computing, Engineering and Construction School of Computing Jim Littleton College of Computing, Engineering and Construction School of Computing Kris Amatuli College of Arts & Sciences Biology Junga Kim College of Arts & Sciences Communication Chunsik Lee College of Arts & Sciences Communication Margaret Stewart College of Arts & Sciences Communication Alicia Sitren College of Arts & Sciences Criminology and Criminal Justice Brenda Vose College of Arts & Sciences Criminology and Criminal Justice Heidi Manzone College of Arts & Sciences History Maria A Fernandez Cifuentes College of Arts & Sciences Languages, Literatures and Cultures Clarence Hines College of Arts & Sciences Music Katherine Hooper College of Arts & Sciences Psychology Juliana Leding College of Arts & Sciences Psychology Jessica Chandras College of Arts & Sciences Sociology, Anthropology and Social Work Suzanne Ehrlich College of Education and Human Services Exceptional, Deaf, and Interpreter Education Dilek Kayaalp College of Education and Human Services Teaching, Learning & Curriculum Len Roberson (Master Reviewer) College of Education and Human Services Exceptional, Deaf, and Interpreter Education Lauren Newton Library Thomas G. Carpenter Library Sarah Moukhliss Library Thomas G. Carpenter Library
Quality Designations
Online Course Design Quality reviews can result in a Quality (Q) or High-Quality (HQ) designation. Both quality designations rely on the QM Rubric and Standards, but the course review process is unique to Florida. The main differences between the two quality designations are in the number of course reviewers and the number of QM standards that must be met. Both are significant accomplishments that indicate a course has met rigorous, research-based course design standards. While instructors may opt to participate in either level of review, UNF encourages instructors to pursue the HQ designation.
Quality
Courses designated as Quality (Q) will have been reviewed by the faculty course developer and one certified peer reviewer, met all essential QM Standards, and provide alternative means of access to course materials in formats that meet the needs of diverse learners.
High Quality
Courses designated as High Quality (HQ) will have been reviewed by the faculty course developer and two certified peer reviewers, met all essential QM standards, received at least 85% of the available points on the QM Rubric, and provide alternative means of access to course materials in formats that meet the needs of diverse learners.
Internal QM Review
Prior to initiating an internal QM review for your online course, it is essential to consult an Instructional Designer regarding the QM review criteria. Once your course is prepared for review, your designated instructional designer will initiate the internal QM review process.
The internal QM review process proceeds as follows:
See below for descriptions of each step in the Quality Review process.
Course Setup
After submitting your course for review, the QM coordinator will assign two reviewers: your instructional designer and a faculty QM-certified peer reviewer. The faculty reviewer will be enrolled in the Canvas Sandbox course and provided access to all course content, including external publisher materials if necessary. Reviewers have a 2-3 week timeframe to complete the review.
Review | Feedback | Revision
The reviewers assess the course, assigning a score based on QM standards and providing recommendations for measurable design revisions. If the course initially falls short of meeting the required QM standards, you will have multiple revision opportunities. Your instructional designer will share the review score and feedback with you, seeking your input on proceeding with the necessary revisions.
Course Meets Quality Expectations
To complete the review process and obtain the HQ designation, your course must meet all 3-point essential QM standards, achieve a total score of 86 or higher out of 101 points, and provide alternative means of accessing course materials in formats that cater to diverse learners, aligning with accessibility guidelines throughout the course. After meeting these criteria, you will receive a $2000 stipend.
QM-Reviewed Courses
View the courses and instructors meeting QM standards.
Click on the image below to view the Quality Matters Course Reviews Dashboard (Power BI).
Note: You must log in with your UNF N-Number and password.
Power BI Dashboard - Keyboard Navigation
Power BI supports keyboard navigation, screen readers, and other accessibility features to enhance usability. For a full list of keyboard shortcuts and accessibility options, visit Power BI Accessibility & Keyboard Shortcuts.
Course Design Modifications
Online courses that complete an internal QM review are awarded the statewide High Quality (HQ) designation, valid for up to five years from the review completion date. During five years, most online courses typically undergo course design modifications to continuously improve subject matter content, technology integrations, and other aspects of online courses, which asserts that course design should and will change. To ensure online courses awarded the Florida Online Course Design Quality Designation continue to meet QM standards, course instructors are encouraged to consider the following guidelines as they modify their online courses.
When there is a concern that significant course design modifications will impact the course, an instructional designer can ensure all critical course elements are aligned and meet QM standards. The following considerable course modifications could necessitate the course instructor to resubmit their online course for another QM review:
- Textbook adoptions that directly impact QM alignment standards. For example, a course adopts OER in place of a publisher’s textbook or course switches to using an entirely different online textbook publisher that necessitates significant changes to course design.
- Redesigning a course due to increasing or decreasing credit hours resulting in significant changes to course alignment might require another course review.
-
MINOR MODIFICATIONS +
Minor modifications typically do not impact course design quality and can be implemented by the course instructor as often as necessary. Examples include but not limited to:
- Changes to course policies
- Updates to instructor contact information or course introductions
- Addition of instructional materials
- Addition of graphics, multimedia, external links, and article
- Modifications to assignment instructions
- Updates that improve accessibility or usability
- Less than 20% of critical course design elements are modified (see Major Modifications list below)
-
MAJOR MODIFICATIONS +
Major modifications could negatively impact alignment between critical course elements and possibly invalidate the HQ designation of an online course. If more than 20% of ANY of the following QM alignment standards will be modified, these are considered major modifications, and you are encouraged to consult with an instructional designer:
- Course Level Objectives [QM Standard 2.1]
- Module Level Objectives [QM Standard 2.2]
- Assessments in the Course [QM Standard 3.1]
- Instructional Materials [QM Standard 4.1]
- Learning Activities (including Assignments) [QM Standard 5.1]
- Tools used in the Course [QM Standard 6.1]
-
COURSE ASSISTANCE +
The following support options are available to assist instructors with course modifications:
- CIRT Support for assistance with Canvas and accessibility
- Instructional Design Support for course design modifications and alignment with the QM Rubric and Standards.
Continuous Improvements
Course development is an ongoing process that evolves over time. Over a span of five years, online courses are likely to undergo multiple revisions, leading to variations in course design quality and consistency. These variations can potentially impact the delivery of the course and the success of the students. UNF's DL policy emphasizes the need for courses delivered online to maintain high-quality designs. According to this policy, such courses are required to undergo a thorough course design quality review every five years. This policy is founded on the understanding that technological advancements, updated instructional materials, and changes in pedagogical approaches can lead to significant alterations in online courses within a five-year timeframe. To ensure a continuous enhancement of online courses, instructors are strongly encouraged to submit their courses for an internal QM review every five years.
Courses listed under UNF's QM-Reviewed Courses are eligible for another internal QM review after five years or within a five-year period if certain course criteria are met, for example:
- Course modifications have been approved by the Academic Programs Committee (APC)
- The course has undergone major changes (i.e., more than 20% of the course as a whole has changed). View QM Course Design Modifications for details.
- The course is set to be delivered the following semester, and starting a few months early would benefit the instructor and students.
- The course will not be taught again for another year, putting the course at six years or more since the last QM review.
Quality Matters FAQ
-
WHAT IS IT? +
"Quality Matters (QM) is a national benchmark for online course design."
Quality Matters is a not-for-profit subscription service that provides a rubric and a process that academic institutions use for quality assurance and continuous improvement of online courses. The QM review process is a collegial, faculty-driven, research-based peer review process based on a rubric consisting of 8 general standards and 44 specific standards nationally recognized as reflecting a shared understanding of quality in online and hybrid course design.
-
WHO'S DOING IT? +
Higher education institutions throughout the United States and internationally have subscribed to the Quality Matters rubric and services to assure quality and continuous improvement in their online education offerings. There are over 30 institutions and over 400 subscribers participating in Florida.
-
HOW DOES IT WORK? +
An institution or a statewide consortium subscribes to QM for an annual fee depending on the institution's individual needs. This gives the subscribing institution's faculty and staff access to an automated, web-based QM rubric for reviewing online courses, the ability to conduct official course reviews and QM recognition (a seal) for those courses that meet standards, discounts on (or free seats in) QM training, and access to private web resources.
-
WHY IS IT SIGNIFICANT? +
The Quality Matters approach to quality assurance in online courses is unique. It emphasizes a collegial, peer-based review process in which the faculty course designer is a part of the review team. A course is never described as having "failed" a review; it is always "in the process of meeting standards" and is described as such until it meets the QM rubric standards at a rate of 85% or better. The QM rubric and process is also unique in that it is diagnostic (not evaluative or judgmental) and focuses solely on online course design and not on course delivery. According to the most current research literature, the QM standards are centered on the best practices in instructional design and online learning.
-
WHAT ARE THE DOWNSIDES? +
The cost to the institution of the privilege of using a rubric that may seem to be based on common sense and "what everyone already knows about best practices in instructional design and online learning" may seem excessive to some. However, the use of the rubric is a demonstration to stakeholders of the investment in and concern for quality in online education. Another possible problem that may occur with a prescribed set of standards is a potential for a sense of infringement on academic freedom, particularly in course design creativity.
-
WHERE IS IT GOING? +
The number of subscribers to Quality Matters continues to grow, and it appears that the QM seal will become more broadly recognized on online courses that have met this gold standard in distance education. Several new developments are emerging from the Quality Matters organization, which include the following:
- A project to implement QM certification of textbook publisher materials for online education
- A QM partnership with Sloan-C
- A growing interest in viewing the Quality Matters standards from a student perspective and making the standards speak directly to students regarding what is of value to them in online education
-
WHAT ARE THE IMPLICATIONS FOR TEACHING? +
The principles and standards embodied in the Quality Matters rubric and process are based on research that indicates that certain elements of course design if implemented correctly, will have the following effect on teaching and learning:
- Improvement in student learning outcomes and student retention in online courses
- Integration of new technologies using sound online instructional design principles and research-based best practices
- Efficiency in the use of instructional resources
- Continuous improvement in online and hybrid courses and ongoing faculty development
-
HOW DOES UNF BENEFIT FROM QM? +
Quality Matters processes benefit both individual faculty members and their institutions in the following ways:
- Improved student engagement and learning outcomes
- Adoption of a systematic and comprehensive continuous quality assurance process that includes faculty training, course development, and course improvement processes that are aligned with accreditation standards
- Incorporation of new technologies and research findings
- Opportunity to engage in benchmarking activities with peer institutions
- Ongoing faculty professional development
- Opportunity for peer-to-peer collaboration and sharing across institutions
- Online courses that meet a consistent and widely-respected quality threshold