What Determines Quality in Training?

A workshop on identifying quality criteria was offered at the CALMet Conference, September, 2015, in Seoul, Korea. The participants were presented with this scenario: You are reviewing a training announcement from an unknown provider in a topic area you are interested in, but you would like to know about the potential quality from a reading of the announcement before investigating further.
Participants were asked to work in small groups to develop criteria within three areas: Organizational, Processes, Product. They were also asked to assign priorities to the criteria as time allowed.

The resulting draft criteria lists compiled from their decisions was then compared to a previously prepared draft list of criteria, which was developed from a review of several existing quality management systems for educational resources. These included ISO/IEC 19796, ISO 29990, Open ECBCheck (for eLearning courses), EFQUEL (for eLearning programmes), Commonwealth of Learning (UNESCO), and the WMO Competency Requirements for Education and Training Providers. Approximately 6 additional systems were examined.

New suggestions for criteria from the workshop were added or merged with the existing ones.

What came out on the top? See the list below, as well as the more complete report. Many more important criteria are contained there, as well as a discussion of the results.

Organizational

•How trainers are hired, trained, and evaluated
•Existing accreditation that covers vocational training, or existing quality assurance processes
•Numbers, roles, and qualifications of training staff members (including access to content expertise)
•Location/country, accessibility (including online presence)
•Facilities, resources, and technology infrastructure

Processes

•how learning outcomes are specified and how they inform the training content and activities
•Good development and delivery processes, documentation of processes, ISO certification (if used)
•How learning needs are determined
•how training is evaluated and improved

Products (including both courses and resources)

•currency of the content (up to date)
•learning activities match learning outcomes, opportunities for students to apply what they learn
•the relevance of course content to job competencies
•assessment methods, matched to learning outcomes
•performance-based learning outcomes
•Timing of courses (Duration and scheduling)
•the results of past evaluations of this course or resource (or related ones), student recommendations


Conclusion

Thanks to those who participated in developing these draft criteria!

As work continues to decide how quality information might be shared, in a WMO Global Campus Shared Calendar and future learning resource Registry, for example, the discussions to date should provide some useful input. But they also raise several issues for further debate.

•How does accreditation relate to quality assurance of vocation training and individual courses and resources? Is additional QA still useful?
•Does ISO 29990 certification cover the required criteria? Is more quality information required for specific resources and courses?
•How can we apply quality standards that are inclusive of many types of training centers and units?
•Is QA information voluntary, or mandatory? How will it be communicated?

How to prepare for a new generation of satellite...

Action Item VLMG7.01, "Share experiences with the preparation for the Himawari 8/9 with the group, as an example of preparation for the next generation of satellites"


I was involved in the following preparations for the planned Japanese Himawari 8/9 Geostationary Satellite. This advanced satellite is planned to monitor our hemisphere from July 2015 in 16 channels and at 10 minute intervals.

 

Planning ahead

1. I attended the WMO-EUMETSAT Workshop on RGB Satellite Products during September 2012. This information was disseminated after the event in a series of blended classroom / online Webinar sessions with Bureau of Meteorology (BOM) and WMO Region V and II (RAV and RAII) stakeholders in the audience.
2. A multi-branch working group, the Advanced GEO Imager Scoping Project Reference Group (AGEOIS-RG) was set up within the Bureau of Meteorology to develop a data requirement regime. This group was active between March to June 2013.
3. Four senior staff members from the Japan Meteorological Agency (JMA) visited the Bureau of Meteorology for the JMA-Himawari-8 Symposium from 23-24th May 2013. Amongst other topics the findings from the AGEOIS-RG working group were presented for further discussion 
4. Two sessions pertaining to Satellite User Requirements in WMO Region V (SW-Pacific) were coordinated by a staff member from BOM Observation and Infrastructure (O&I) during the Asia-Oceania Meteorological Satellite Users Conference (AOMSUC-4) and Training Workshop (7-11 October 2013).
5. MTSAT-1R rapid scan (10 minute) data from the BOM/JMA/NASA/Environment Canada High Ice Water Content (HIWC) Experiment was successfully ingested and displayed within BOM from 19th December 2013 to 19th March 2014. This was a useful trial in preparation for the 10 minute Himawari-8 data.
6. A JMA staff member (Mr. T. Kurino) visited the Bureau of Meteorology on the 26-27th June 2014. Topics of discussion included an update on JMA activity and plans, MTSAT/AHI band correspondences, data transfer, forecaster training, products for forecasting severe weather, research opportunities.
7. Monthly Webinar sessions pertaining to Himawari-8 and involving JMA and BOM staff commenced August 2014.

 

Useful resources available and prior training

1. Two sources of MTSAT-1R rapid scan data became available for Bureau and RAV stakeholder training during 2013 and 2014. 5 minute rapid scan data over Japan was made available on a series of DVD's through liaison between JMA and the Bureau of Meteorology Training Centre (BMTC) from August 2013. 10 minute rapid scan data over the Australian region became available as a result of the HIWC Experiment. Case studies were developed from this data for use in training courses involving BOM and RAV stakeholders. Complimentary copies of these case studies were given to JMA staff.
2. MODIS Aqua and Terra data from the NASA Rapidfire website was used to create RGB products for significant Australian and RAV weather / volcanic events.
3. Training in the effective Forecaster use of 10 minute rapid scan and RGB product data using the aforementioned case studies was provided to BOM and RAV stakeholders during the following blended classroom / online Webinar sessions:
4. The 4th Asia-Oceania Meteorological Satellite Users conference (AOMSUC-4) Training Event (7-8 October 2013)
5. The monthly Regional Focus Group Meetings since October 2013.
6. Advanced Forecaster Course Science Week, 29th July to 1st August 2014
7. The Advanced Satellite Meteorology subject of the BMTC Graduate Diploma of Meteorology Training Course, August 2014.
8. Additional information provided during these sessions included latest JMA updates on product delivery and other details pertaining to Himawari 8. Forecaster feedback pertaining to the operational use of the 10 minute MTSAT-1R data of HIWC was also presented. A JMA staff member assisted during the AOMSUC-4 Training Event, especially during the rapid scan and RGB product practical sessions.
9. The above training served as useful experience in preparation for the Bureau's National Himawari-8 Training Campaign scheduled for late 2014 / early 2015.

Important contacts

1. Ongoing contact with Sponsoring Satellite Operator (JMA) on many levels, including staff from the Data Processing Department, Satellite Program Division etc. These staff provided the latest updates such as dates of launch, data dissemination details and resources such as 5 minute rapid scan data over Japan. In turn, we provided information about proposed use of Himawari 8 data as well as emphasizing the importance of adapting satellite products to RAV low latitude regions.
2. Contacts within the Observations and Infrastructure section of the Bureau of Meteorology regarding data delivery, types of products etc.
3. Contacts with Satellite Champions within the Australian Bureau of Meteorology Regional Forecasting Centres and BMKG Indonesia, as per the EUMETSAT model.
4. Other RAV and RAII contacts.

MOOC format continues to make waves...

"Day of the MOOC" by mbransons

The MOOC (Massive Open Online Couse) format continues to make waves due to the excitement that its reach and innovative use of new internet technologies generates. MOOCs have been around for over 5 years and their popularity among practitioners and theorists, as well as academic institutions, seems to continue growing. More data, statistics and thoughts have been collected since last summer, when CALMet Commons hosted our article about MOOCs (How much OOOO in your MOOC?). That is also why we decided to update it. We are happy to share that the refined version of the article was recently published in the academic journal for distance education - Form@re, edited by University of Florence.

The second version explores even more current trends in MOOC offerings, questioning the pedagogical value of the new implementations, but also appreciating their rapid growth and potential impact. The article suggests that MOOCs should be adopted with conscious and deliberate intent to utilize the openness the form allows, and not simply because they reach more students.

For Italian speaking readers, if you are interested in MOOCs and OER the last issue of Form@re offers other valuable articles.

eLearning Africa

From 28-30 May this year, Patrick and Maja attended the eLearning Africa Conference. Also attending from our community were Jannie Stander from SAWS, and David Muchemi from IMTR, Kenya. Patrick gave a presentation on “eLearning Solutions for Professional Weather Forecasters and Hydrologists in Africa” that highlighted the many ongoing eLearning efforts among WMO Regional Training Centers in Africa and their partners. But we were there primarily to learn about eLearning efforts occurring in other sectors and disciplines, and to see what new technological and pedagogical solutions were being applied or proposed. The conference was extremely hectic, with up to 10 concurrent sessions occurring at any given time over the short 2 days. It was hard to participate in even a fraction of what was occurring. 

What we learned

We learned that eLearning is thriving in Africa in many ways, even though it continues to face challenges of limited technological infrastructure, doubts about its effectiveness, and limited knowledge of eLearning practices, just like in almost every other region of the world. Many international partners are working with Africans to help overcome these challenges, and many technology industries are coming up with creative solutions to provide low-cost and workable online learning approaches. The following is a brief summary of what appeared to us the key themes of the conference. Many keynote presentations from the conference are available online as recordings at http://www.elearning-africa.com/media_library_videos.php.

Why the conference is important: It has been projected that 25% of the global workforce will be African by the year 2050, yet most African countries must overcome a limited access to quality education at all levels. ELearning will not solve the education access challenge alone, but it will be a key element in the solution, and organizations and conferences like eLearning Africa will help to guide how eLearning solutions are implemented. In discussions about sources of grant funding to improve education systems, a point was made that NO MORE PILOT projects are needed. African countries know what they need, and this includes eLearning solutions that can be implemented now.

Increasing internet access: Clearly a challenge in rural Africa especially, but throughout much of the continent, the improvement of internet access is being tackled in a variety of creative ways (and being worked around in some cases). Communications satellites coupled with low(er) cost receivers with broadband internet provision holds perhaps the best promise while the fiber infrastructure continues to develop (Kenya has a growing fiber infrastructure that may be expandable, for example). SES, for example, is a company with satellites that it claims can reach 99% of the world’s population. Its services can be pre-paid and are scalable. YahClick is a regional system that using the little exploited Ka-band. In order to better spread access, a company called WiBack is providing low cost and low power consuming (solar) repeaters to extend a single broadband link over 20km. Internet access will be rapidly growing (see Lions go digital). It is estimated that the current 16% internet penetration will increase to 50% by 2025 (for urban areas penetration is already more than 50%).

Low cost computer hardware: Tablets are clearly the low-cost internet access, computing, and content distribution system of choice. One group (StudyTech) uses motorcycles to distribute tablets loaded with instructional content to students. One of the main conference sponsors was Yazmi, which offers tablets with satellite receivers built in to receive digital content. Using a small classroom server, student tablets can be synchronized and updated to control their content. Tablets have their detractors also. While some people are more concerned about tablets having too limited computing power, others see tablets as being potentially distracting to students. So controls on content and applications are desirable to many. Instead of laptops, another company, NComputing, is offering a classroom solution that allows one personal computer to support many low-cost, thin-client computer systems for students.

Mobile learning: Mobile learning is seen as a highly promising frontier. For example, 90% of people have mobiles in Uganda. In Kenya, 12% have internet access, 41% have TV, and 80% have mobiles (80% also have radio). More students have smartphones than do teachers. One non-pedagogical use of mobiles is to map student attendance, which is a serious problem in many locations, but various projects are being implemented to provide a variety of forms of content via smart phones as well, including simple SMS. Video content is the mobile format being explored the most, perhaps. One project discussed had students working side-by-side with teachers to develop their own video content for internet delivery (students as not just content consumers, but content creators).

Learning content needs: Due to the shortage of quality textbooks, some are working to make eBooks more available to fill the gap. In some schools, textbooks are considered so precious that they are locked up away from students. For some, eBooks are looked upon as a poor replacement for traditional paper books, whose tangibility is still highly valued. But it is generally recognized that quality eBook publications with multimedia content delivered on low-cost computers and mobiles are a required step to improve educational access.

Software access: Learning is about more than having students access content, it is also about having them exercise original expression in the form of research, critical, and creative papers and presentations. Free Google applications did not seem to come up much at the conference, but Microsoft had a strong presence. Microsoft is working to offer low-cost ways of maintaining their monopoly of productivity tools like Word, Excel, and Powerpoint. Bootlegged versions of their products harm more than Microsoft, they bootstrap users with old and unsupported versions, but new cloud-based Office applications allow for licensing that can put software in the hands of students at a very low per-student cost. Institutions may even be able to provide all students (or all employees) with continually updated software. New MS products like One Note, One Drive, and Office Mix are efforts to stay viable in education markets requiring low cost solutions. But of course Microsoft is just biggest, and not the only player in offering cheap or free software.

Teacher shortages: In Kenya, 65% of teachers fail on content exams, and this is probably not the worst case. In many locations sufficient teachers with proper credentials are lacking. Teacher education was certainly not ignored, but one of the hot topics about ways to address this issue was MOOCs (Massive Open Online Courses), which are designed to reach large numbers with fewer faculty. Donald Clark persuasively made the case that MOOCs, if designed properly, can provide highly effective educational experiences. Many were not convinced that such an “impersonal” form of teaching can work. Clark argued, “Lecturing is easy, Teaching is hard,” and that MOOCs can provide a platform that emphasizes the latter just as easily as in the classroom. One ongoing concern about MOOCs is the high student dropout rates. But he noted that while long courses (modeled after traditional university semester-long courses) continue to experience high numbers of dropouts, short courses have been highly success at student retention (a 6-week course in perinatal care in Africa reported 35,000 participants and 65% completion!). Besides, he argues that we should be concerned about making space for “drop-ins,” and worrying less about dropouts. He also argues that because MOOCs are based on online social networking platforms, students in MOOCs actually make a closer social connection than they would in a classroom of over 25 students. Even distinguished scholars are being convinced by the MOOC platform: One physics professor has said that “video delivers me in a way that the student has complete control over”-- in other words, he realizes teaching is not all about him, but about whether the student has time to think about the content he offers. Another professor has said that in a single MOOC he will reach more students than he will have reached throughout the rest of his career. Clearly, MOOCs have something to offer to the scalability issues of the education challenge. While English dominates the MOOC world so far, a French-language site for MOOC offerings can be found at http://www.rescif.net/en/node/89. 

A concluding debate

On the final day of the conference, the plenary session included a debate on whether or not to adopt the following resolution by the conference: “This house believes that there is now nothing more important to education than access to the Internet” The debate was heated, and many good points were made by both sides: The internet opens the world up to students. Qualified teachers are the key to guiding student learning, not technology. Etc.

The motion lost by a very small margin, primarily due to the argument that it was teachers that were most important to influence students in the long run. In many countries of Africa, there are not enough qualified teachers, and many were not convinced that the Internet could help deal with this shortage through eLearning opportunities. eLearning for teacher education was not brought up during the debate, but it was the topic of several sessions during the conference. For example, the African Virtual University is already offering faculty development courses in online and blended modes, and is proposing MOOC approaches as well. These include a certificate program in Distance and eLearning.

CALMet Online 2014

October, 6  - November, 28

CALMet Online will gather together meteorological and hydrological trainers, as well as related disciplines facilitators, from around the world. As the event is totally online and free, you will not need a travel budget to participate.

During the sessions we will share experiences and innovations in training, mainly in the field of hydro-meteorology, and together find new approaches for improving our training activities and programmes.

This year’s topic ‘Diverse in Culture, United in Learning’ aims to highlight the fact that despite our various backgrounds, cultural differences and diverse organizational environments, we all share the same need to learn more about training and education in our disciplines, and become better professionals. 

We believe that CALMet Online will offer a channel for professional trainer in meteorological and related disciplines to equally participate and contribute in discussions and activities.

The goal of CALMet Online 2014 is to demonstrate a variety of learning approaches, suitable for application and adaptation across various institutions and practices.

To make taking part easy and flexible , the event will be based on several asynchronous sessions, each of them which will be accessible for one, or even several, weeks. Each session will be hosted by a leader or group from a specific WMO Regional Association, demonstrating a new learning solution, experiment or innovation. The participants will have an opportunity to discuss, test and give feedback about the learning solution. The resources are intended to be reusable after each session. Interest participants in the event should get a tangible benefit from the experiences they gained in each session.

The event will be hosted in English, but some sessions may contain parallel regional activities in other languages. 

Sessions

The event will consist of several sessions, each having 1-3 week duration. Some examples of sessions you can see in the list below. The complete schedule with confirmed list of sessions will be published by the end of June.

Briefing. Sarah Kimani (IMTR).

Learnopoly. Patrick Parrish (WMO) and Maja Kuna (EUMETSAT).

Webin@r Checklist. Maja Kuna (EUMETSAT), Luciane Veeck (VLab), Anna Ghelli (ECMWF), Ivan Smiljanic (EUMeTrain), Alessandro Chiariello (EUMETCAL). 

Preparing for a new satellite: new and old challenges mixed with opportunities. Bernadette  Connell (CIRA).

Teaching with Conceptual Models. Marianne Weingroff (COMET) and Bruce Muller (COMET).

Experience Zonda:  an example of a Conceptual Model  applied on a simulator - Experiencia Zonda: ejemplo de Modelos Conceptuales aplicando  Simuladores. Graciela Rolón (Servicio Meteorológico Nacional - Argentina).

Experience in Distance Learning in Meteorology - Experiencias en Enseñanza a Distancia. Moira Doyle, Elizabeth Castañeda, Leonor Bonan, Lorena Gonzalez and Ana Pittaro (Department of Atmospheric and Oceanic Sciences of the University of Buenos Aires).

Succesful Online Learning : what strategies do we need? Roger Deslandes and Bodo Zeschke (Bureau of Meteorology, Australia).

Read more and join 

Weather briefing using chat

The WMO Train the trainers’ online course is more than a wealth of knowledge and practical guidance from the trainers. As a participant, it has exposed me to a lot of resources and technology which I would never have imagined possible. In a view to teach about how to participate in an online weather briefing, I as a participant volunteered to host a weather chat briefing following in my trainer’s footsteps who hosted a more time specific and time bound weather briefing chat activity in Moodle for the discussions and Google presentation for sharing the images and commenting as well. My weather briefing chat activity was a little different;

It was an ongoing activity which kicked off with a short synchronous chat session and discussion of images on Google presentation but the room stayed open for annotations and updates for several hours.

This allowed for participants to log in anytime and join the discussion using the provided links. In this sort of presentation, I was not overwhelmed juggling between the images and the chat and it offered a great opportunity to discuss exhaustively an image before moving to the next image.

Screenshot of the weather briefing activity. 

For those who missed the briefing, they would be able to view the past sessions on Moodle and view the images later on Google presentation. It however did not miss to have its challenges such as internet speed which caused different delays in page loads, unstable connections and difficulties to access the Google Presentation.


After the session, I got encouraging feedback from some of the participants. Vesa from EUMETSAT said

 “(...) really nice initiative from you, was a pleasure to see it happen. I like the pace and easiness of this type of briefing, the images are there freely to browse, but you keep the discussion in control and return us to the correct slide under discussion when needed. This type of page could be running in my forecast office every day and help me keep in touch with other forecasters in the region.”

Roro from Jakarta, Indonesia said

 “(...) I found you did a great job in brief your audiences. I like the way you stop before every single slide, and typed the number of the slide you were talking about. That helped me to focus to the mentioned slide. With your "not to slow not too fast" pace, you help me to "digest" slide by slide. Thank you and I am thinking to propose a chat briefing as one of activities in our distance learning project.”

As Vesa said, it is a great initiative and I am looking forward to implementing and applying it for the African region.