Evaluation

=Evaluation=

July 6
Bates, A.W. & Poole, G. (2003). Chapter 4: A Framework for Selecting and Using Technology. In Effective Teaching with Technology in Higher Education: Foundations for Success. (pp. 77-105). San Francisco: Jossey Bass Publishers.
 * Today I'll try to synthesize some of the evaluation information prior to our meeting tonight. We'll want to make sure we write this up as our recommendations, and not so much as a literature review. I'm having trouble finding some of the references cited. Mary, what you've cited as Bullen & Janes (2007) should actually be cited as Sangra, Guardia, & Gonzalez-Sanmamed (2007), as Bullen & Janes are just the editors of the book not the authors for that reference. (The page numbers, which we won't include in our paper, helped me locate the correct source). I'm having trouble finding the SECTIONS model and the ACTIONS model - can someone provide the full reference for these? OK - I've found the reference for Bates & Poole (2003), but I don't have a downloaded copy - does anyone have a link where I can download?

July 05 Mary To ensure success in the development and introduction of e-learning in higher education and other institutions, it is necessary to evaluate the quality of e-learning when used in different fields. Bullen & Janes (2007) propose the evaluation to be done taking into consideration three different levels: The institutional level, the course level and the disaggregated elements. In addition, different perspectives within these three levels should be considered such as the students, the teaching staff, the administration, and the stakeholders. Bullen & Janes (2007) point out the importance of placing a value on the approach given to the evaluation of quality. For instance, the technological approach will focus on the technical operations; the economic approach will value the financial results, the educational approach will see the performance and progress of the students, and the global approach that will focus on the balance of all the different approaches. See figure.1 p.292 Bullen & Janes (2007) for better understanding of the evaluation of quality in e-learning. Bullen & Janes (2007.p.294) describe an experience referring to quality indicators for e-learning at the Universitat Oberta the Catalunya. They created an instrument (BSM) to get different information regarding the academic results obtained by students, levels of satisfaction and relations with the University. In addition, the BSM allows the analysis of possible causes for less favorable indicators. The main idea of BMS is to evaluate the quality of the teaching and learning process and to find ways to improve the quality of these processes. I am not sure if could use something similar and adapt it to the “school without boundaries” However, it is necessary to create a benchmarking process to evaluate the program. To develop strategies for research and evaluation, Bates (2000.p.200) suggest that it is necessary “look at a wider range of factors than the ability of technology to replicate classroom teaching.”. Bates (1995) uses the ACTIONS model to evaluate the effectiveness of different teaching technologies. However, Bates (1995) trying to amend the ACTIONS model proposed the SECTIONS model for selecting and applying technology. They both have similarities but they differ in the sense that //Access// was subsumed under //students//, and //ease to use// was added as a new criterion; the rest of the model remains the same. Bates (2000) points out that the use and selection of new technologies need to be researched but it cannot be limited to replicating classroom learning outcomes. More specific questions should be taking into consideration.( See p. 201)

Mary, can we use Bates' SECTION Model here? For whatever reason, I think it's more widely used and accepted as compared to the ACTIONS model. -Brian
 * What processes will be in place to monitor implemenation?
 * How often?
 * Yearly

June 24 Mary Brian, i was looking at my notes about Bates and Poole (2003) " A Framework for Selecting and Using Technology" and i realized that in fact The SECTIONS model for selecting and applying technology tried to amend the ACTIONS model as you mentioned above. They both have similarities but they differ in the sense that //Acces// was subsumed under //students//,and //ease to use// was added as a new criterion; the rest of the model remains the same.

S Students: What is known about the students-or potential students- and the appropriateness of the technolocy for this particualr group or ramge of students. E Ease to use and realiability: How easy is it fro both teachers and students to use?How realiable and well tested is the technology? C Costs: What is the cost per structure of each technology? What is the unit cost per learner? T Teaching and Learning: What kinds of learning are needed?What instructional approaches will best meet these needs? What are the best technologies for suporting this teaching and learning? I Interactivity: What kind of interaction does this technology enable? O Organizational issues: What are the organizational requirements and the barriers to be removed before this technology can be used successfully? What changes in organization need to be made? N Novelty: How new is the technology? S Speed: How quickly can courses be mounted with this technology? How quickly can materials be changed?

Mary June 22 Bates (2000.p.198 chapter nine) suggets that the research and evaluation of technology integration in the classroom should focus on asking the right questions which should benefit planning and managing new technologies for teaching and learning. Bates(2000.p200) points out some factors that should be considered when evaluating the effectiveness of different teaching technologies. These factors can be summarized by the acronym ACTIONS: Acces and Flexibility : How accessible is a particular yechnology for learners? How flexible is is for a particular target group? Cost : What is the cost structure of each technology? what is the unit cost per learner?How do costs differ between technologies within a particular context? Teaching and learning : What kinds of learning are needed? What instructional approaches will be best meet these needs?What are the best technologies for supporting this taeching and learning? Interactivity and user-friendliness : What kind of student interaction does this technolgy enable?How easy is the technology to use? Organizational issues : What are the organizational requirements, and the barriers to be removed, before this technology can be used successfully?What changes in organization need to be made? Novelty : How new is the technology? How reliable is it? How will this technology contribute to institutional renewal? How quickly can courses be mounted with this technology? How quickly can material be changed?

Some ideas: feel free to modify them! ( it's almost 1 am so my brain is not helping lol)

" the decision about whether or not to use technology in the first place should be reached through strategic planning exercise that takes into account the overall needs of learners and the teaching goals or mission for the institution or department" (Bates, 2000) " All the teaching needs of the department and rge stundents it serves, or wishes to serve, should be adressed in an integrated way; this means relating teaching methods to the needs of different target groups, the interests and areas of expertise of the faculty, and the resources likely to be available"( Bates, 2000) The program will be tested and each departement will be able to share their newly created e-learning program. Teachers and administrators will continually monitor the progress during the intial implementtation year. For instance, staff will need to make notes of what works, what does not work and what needs to be changed. Minor changes should be considered throught the pilot project.
 * These factors might help us to monitor implementation or guide us at the beggining of the process when working on technology integration.
 * The implementation of vision for the " school without boundaries" can start with the development of the academic program which will take place in three stages to see the progress indvidually.
 * stage one: Planning
 * stage two: Course development
 * stage three: Implementation

June 27 I thought I would put my findings on EQAO on this page. It seems to be appropriate but I am open to suggestions

THE EQAO 5 STEPS TO EFFECTIVE IMPROVEMENT PLANING PROCESS GUIDELINES: 1. OWNERSHIP 2. UNDERSTANDING AND FOCUSING 3. ACCOUNTABILITY 4. PLANNING FOR IMPROVEMENT 5. ONGOING PLAN: Monitoring the implementation of the plan