A living guidance to prepare a simulator exercise

During CALMet 2015 we had a workshop on needs for an Assessment Simulator Exercise seen from
different perspectives.  We worked in groups. The question to answer for each group was: “What do you need to feel yourself safe in this position in the simulator environment.” We have collected all opinions and have put these together into this guidance. In November we have added opinions from the WMO/Eumetcal Train the Trainer participants to this list.

This is a living document which contains a minimum needs list from what we hope that it will expand in the future.  We hope that this document will help you. We have the feeling that you may want to use this document as a guidance to prepare your simulator exercises. In case you don’t have any assessment goals for your simulator exercise please realise that the assessment part of this document most probably still contains some useful points to use in your simulator exercise.

Good luck by organising a simulator exercise and we hope to hear your simulator experiences so that we can learn from them too.

A big thanks to my partners in crime, Mark, Ian and Vesa, for the help with doing these workshops and all the colleagues that gave input on the various occasions.

Heleen

What are the minimum needs you have to think about when organising a simulator exercise?

What Determines Quality in Training?

A workshop on identifying quality criteria was offered at the CALMet Conference, September, 2015, in Seoul, Korea. The participants were presented with this scenario: You are reviewing a training announcement from an unknown provider in a topic area you are interested in, but you would like to know about the potential quality from a reading of the announcement before investigating further.
Participants were asked to work in small groups to develop criteria within three areas: Organizational, Processes, Product. They were also asked to assign priorities to the criteria as time allowed.

The resulting draft criteria lists compiled from their decisions was then compared to a previously prepared draft list of criteria, which was developed from a review of several existing quality management systems for educational resources. These included ISO/IEC 19796, ISO 29990, Open ECBCheck (for eLearning courses), EFQUEL (for eLearning programmes), Commonwealth of Learning (UNESCO), and the WMO Competency Requirements for Education and Training Providers. Approximately 6 additional systems were examined.

New suggestions for criteria from the workshop were added or merged with the existing ones.

What came out on the top? See the list below, as well as the more complete report. Many more important criteria are contained there, as well as a discussion of the results.

Organizational

•How trainers are hired, trained, and evaluated
•Existing accreditation that covers vocational training, or existing quality assurance processes
•Numbers, roles, and qualifications of training staff members (including access to content expertise)
•Location/country, accessibility (including online presence)
•Facilities, resources, and technology infrastructure

Processes

•how learning outcomes are specified and how they inform the training content and activities
•Good development and delivery processes, documentation of processes, ISO certification (if used)
•How learning needs are determined
•how training is evaluated and improved

Products (including both courses and resources)

•currency of the content (up to date)
•learning activities match learning outcomes, opportunities for students to apply what they learn
•the relevance of course content to job competencies
•assessment methods, matched to learning outcomes
•performance-based learning outcomes
•Timing of courses (Duration and scheduling)
•the results of past evaluations of this course or resource (or related ones), student recommendations


Conclusion

Thanks to those who participated in developing these draft criteria!

As work continues to decide how quality information might be shared, in a WMO Global Campus Shared Calendar and future learning resource Registry, for example, the discussions to date should provide some useful input. But they also raise several issues for further debate.

•How does accreditation relate to quality assurance of vocation training and individual courses and resources? Is additional QA still useful?
•Does ISO 29990 certification cover the required criteria? Is more quality information required for specific resources and courses?
•How can we apply quality standards that are inclusive of many types of training centers and units?
•Is QA information voluntary, or mandatory? How will it be communicated?

How to prepare for a new generation of satellite...

Action Item VLMG7.01, "Share experiences with the preparation for the Himawari 8/9 with the group, as an example of preparation for the next generation of satellites"


I was involved in the following preparations for the planned Japanese Himawari 8/9 Geostationary Satellite. This advanced satellite is planned to monitor our hemisphere from July 2015 in 16 channels and at 10 minute intervals.

 

Planning ahead

1. I attended the WMO-EUMETSAT Workshop on RGB Satellite Products during September 2012. This information was disseminated after the event in a series of blended classroom / online Webinar sessions with Bureau of Meteorology (BOM) and WMO Region V and II (RAV and RAII) stakeholders in the audience.
2. A multi-branch working group, the Advanced GEO Imager Scoping Project Reference Group (AGEOIS-RG) was set up within the Bureau of Meteorology to develop a data requirement regime. This group was active between March to June 2013.
3. Four senior staff members from the Japan Meteorological Agency (JMA) visited the Bureau of Meteorology for the JMA-Himawari-8 Symposium from 23-24th May 2013. Amongst other topics the findings from the AGEOIS-RG working group were presented for further discussion 
4. Two sessions pertaining to Satellite User Requirements in WMO Region V (SW-Pacific) were coordinated by a staff member from BOM Observation and Infrastructure (O&I) during the Asia-Oceania Meteorological Satellite Users Conference (AOMSUC-4) and Training Workshop (7-11 October 2013).
5. MTSAT-1R rapid scan (10 minute) data from the BOM/JMA/NASA/Environment Canada High Ice Water Content (HIWC) Experiment was successfully ingested and displayed within BOM from 19th December 2013 to 19th March 2014. This was a useful trial in preparation for the 10 minute Himawari-8 data.
6. A JMA staff member (Mr. T. Kurino) visited the Bureau of Meteorology on the 26-27th June 2014. Topics of discussion included an update on JMA activity and plans, MTSAT/AHI band correspondences, data transfer, forecaster training, products for forecasting severe weather, research opportunities.
7. Monthly Webinar sessions pertaining to Himawari-8 and involving JMA and BOM staff commenced August 2014.

 

Useful resources available and prior training

1. Two sources of MTSAT-1R rapid scan data became available for Bureau and RAV stakeholder training during 2013 and 2014. 5 minute rapid scan data over Japan was made available on a series of DVD's through liaison between JMA and the Bureau of Meteorology Training Centre (BMTC) from August 2013. 10 minute rapid scan data over the Australian region became available as a result of the HIWC Experiment. Case studies were developed from this data for use in training courses involving BOM and RAV stakeholders. Complimentary copies of these case studies were given to JMA staff.
2. MODIS Aqua and Terra data from the NASA Rapidfire website was used to create RGB products for significant Australian and RAV weather / volcanic events.
3. Training in the effective Forecaster use of 10 minute rapid scan and RGB product data using the aforementioned case studies was provided to BOM and RAV stakeholders during the following blended classroom / online Webinar sessions:
4. The 4th Asia-Oceania Meteorological Satellite Users conference (AOMSUC-4) Training Event (7-8 October 2013)
5. The monthly Regional Focus Group Meetings since October 2013.
6. Advanced Forecaster Course Science Week, 29th July to 1st August 2014
7. The Advanced Satellite Meteorology subject of the BMTC Graduate Diploma of Meteorology Training Course, August 2014.
8. Additional information provided during these sessions included latest JMA updates on product delivery and other details pertaining to Himawari 8. Forecaster feedback pertaining to the operational use of the 10 minute MTSAT-1R data of HIWC was also presented. A JMA staff member assisted during the AOMSUC-4 Training Event, especially during the rapid scan and RGB product practical sessions.
9. The above training served as useful experience in preparation for the Bureau's National Himawari-8 Training Campaign scheduled for late 2014 / early 2015.

Important contacts

1. Ongoing contact with Sponsoring Satellite Operator (JMA) on many levels, including staff from the Data Processing Department, Satellite Program Division etc. These staff provided the latest updates such as dates of launch, data dissemination details and resources such as 5 minute rapid scan data over Japan. In turn, we provided information about proposed use of Himawari 8 data as well as emphasizing the importance of adapting satellite products to RAV low latitude regions.
2. Contacts within the Observations and Infrastructure section of the Bureau of Meteorology regarding data delivery, types of products etc.
3. Contacts with Satellite Champions within the Australian Bureau of Meteorology Regional Forecasting Centres and BMKG Indonesia, as per the EUMETSAT model.
4. Other RAV and RAII contacts.

Learnopoly - a game of learning activities

Learnopoly is a game created for exercising skills for instructional design, where competing teams of players race to design learning activities for a set of given topics. Design constraints are determined randomly, by dealing cards and spinning a wheel. The cards decide the topic and type of activity that must be designed, and the spin decides the learning solution--whether the activity will be carried out either synchronously online, asynchronously online, or in a classroom. The purpose of the game is to offer a creative challenge to players to use different learning activities for different learning solutions in effective and engaging ways. Too often we chose learning solutions only because we have tried them already and are comfortable with them. Learnopoly offers a safe and entertaining environment where players have the opportunity to challenge themselves to think out of the box.

Learnopoly - the game of learning activities from Kuna & Parrish

Learnopoly was introduced as a workshop activity at the recent 2014 Eumetcal Workshop held in Warsaw, Poland, from 10 to 13 June. Over 30 players tried out the game, successfully came up with a great set of solutions, and provided useful feedback for improvements.

Spin the wheel

Real life puts constraints on our training projects, usually for good reasons, but not always. In Learnopoly these constraints are determined randomly, which frees (or forces) participants to prepare themselves for those situations in which they will need or want to attempt something different. 

After an introduction to the goals and basic rules, the following game items were distributed to each team:

• Two sets of cards - Learning activities and Topics (job responsibilities requiring training). Learning activities ranged from discussions to simulations. The topics were varied to cater to a range of a team member expertise, from aeronautical forecasting to disaster response.
• Learning Solution Spinner - The spinner randomly selected the Classroom, an Online synchronous course, or an Online asynchronous course as the context for the learning activity.
• Answer sheets to be used by the teams to describe the learning activities they created, according to stated criteria and to achieve a learning outcome appropriate for the selected Topic on one of their cards.
• A board that helped the team track the points awarded and combinations of Learning activities and Learning solutions used. Some combinations earned bonus points.

The player goals were to beat the time constraints and the other teams in coming up with the most and best learning activities. How were the winners chosen? There were two official ways to win, two goals to accomplish:

1) Design the most learning activities in the time allotted and collect the most points. A team was lucky if it randomly chose a combination of a Learning activity and Learning solution that granted them the opportunity to earn bonus points.

2) Design the best (most effective and engaging) learning activity. These were voted by the other teams at the end of the session, after each team nominated one of the activities they had created and described it aloud.

Nightmare on the shift

Weather briefing using chat

The WMO Train the trainers’ online course is more than a wealth of knowledge and practical guidance from the trainers. As a participant, it has exposed me to a lot of resources and technology which I would never have imagined possible. In a view to teach about how to participate in an online weather briefing, I as a participant volunteered to host a weather chat briefing following in my trainer’s footsteps who hosted a more time specific and time bound weather briefing chat activity in Moodle for the discussions and Google presentation for sharing the images and commenting as well. My weather briefing chat activity was a little different;

It was an ongoing activity which kicked off with a short synchronous chat session and discussion of images on Google presentation but the room stayed open for annotations and updates for several hours.

This allowed for participants to log in anytime and join the discussion using the provided links. In this sort of presentation, I was not overwhelmed juggling between the images and the chat and it offered a great opportunity to discuss exhaustively an image before moving to the next image.

Screenshot of the weather briefing activity. 

For those who missed the briefing, they would be able to view the past sessions on Moodle and view the images later on Google presentation. It however did not miss to have its challenges such as internet speed which caused different delays in page loads, unstable connections and difficulties to access the Google Presentation.


After the session, I got encouraging feedback from some of the participants. Vesa from EUMETSAT said

 “(...) really nice initiative from you, was a pleasure to see it happen. I like the pace and easiness of this type of briefing, the images are there freely to browse, but you keep the discussion in control and return us to the correct slide under discussion when needed. This type of page could be running in my forecast office every day and help me keep in touch with other forecasters in the region.”

Roro from Jakarta, Indonesia said

 “(...) I found you did a great job in brief your audiences. I like the way you stop before every single slide, and typed the number of the slide you were talking about. That helped me to focus to the mentioned slide. With your "not to slow not too fast" pace, you help me to "digest" slide by slide. Thank you and I am thinking to propose a chat briefing as one of activities in our distance learning project.”

As Vesa said, it is a great initiative and I am looking forward to implementing and applying it for the African region.