October 14, 2010, [MD]
As the Ministry of Education has gained experience, there have been continuous changes from year to year of the evaluation criteria for Top Level Courses. These range from minor variations in the rating rubrics used, to more large-scale changes in priorities. The actual rubric used to evaluate courses has five comprehensive indicators: teaching teams, teaching content, teaching conditions, teaching methods and practice, and instructional outcome. These five are subdivided into 15 secondary indicators (one was removed in 2008) as listed by Wang Peng (2008):
When it comes to large scale changes in focus, Professor B3 commented that initially the online resources were not that important, because the Internet was not very developed at the time that the Top Level Courses Project started. As the web grew in importance and sophistication this changed, and in 2004 and 2005, requirements about the functionality and design of the course website increased.
Likewise, Mr. B0 stated that the provinces and the Ministry of Education have focused increasingly on the sharing of resources and providing a service to others. This is a marked change from the first few years, when the application process was the main focus. Initially there were not many courses, and attracting a number of high-quality courses was the focus. Later, as the number of courses increased, it became important to offer better access to these resources, which led to the creation of the Jingpinke.com portal (see the later section on this).
Wang Peng (2008) who studied how the evaluation criteria had changed from 2003-2008, also identified a number of trends. He reaffirmed the added focus on instructional design, pedagogy, use of technology, impact and sharing, and the building of local teams. When evaluating the changes, it is often more interesting to look at the explanations given for the different criteria, than the weight assigned. For example, the explanation for the criteria “evaluation of recorded materials” changed from specifying “dignified bearing, clear sound, full of teaching enthusiasm, lively class atmosphere, high student participation rate, and effective student-teacher interaction” in 2003 to asking for "influential teaching which attracts students attention, enlightens and inspires thinking, association and creativity” in 2004 (ibid. 39).
As new categories of courses were added, new rubrics also had to be developed. For example, the evaluation of online courses put more focus on course resources for self-learners, the teaching and learning process, and learning support design. Zeng, Zeng and Fan (2007) believe this to be natural, and mention high student numbers, and a focus on adult learners that need practical and vocationally oriented training as key features distinguishing distance education from teaching programs in traditional universities.
The quotes in this text is from the MA Thesis "The Chinese National Top Level Courses Project: Using Open Educational Resources to Promote Quality in Undergraduate Teaching" by Stian Håklev, University of Toronto 2010.