The future of work and learning

Photo by LandOreports

The final day of Online Educa Berlin continued with challenging, but even more diverse offerings. One of the best sessions was on “Future Work and Future Workers.” Several futurists posed their visions of how the world of work is changing. There seems to be little doubt that, without a major natural or other disruption to progress, artificial intelligence (AI) and robotics will reduce the human contributions to many tasks. Some jobs will be changed and some will fade away. One presenter thought technology-enhanced jobs will be the most common change, where AI and other tools will just modify and improve what we already do. Another felt that many jobs will disappear, and that it is too soon to see how people will adapt to the change. Can all the replaced people become knowledge workers? Unlikely, based on current definitions of that term, but in a world of connected knowledge, perhaps we all have a place.

All seemed to feel that work has already changed. Work is now more interconnected, multidisciplinary, multi-generational, freelance, and project-oriented. The days of having a single job are fading—we increasingly fill many roles and may eventually all become entrepreneurs. The workplace of the future (already here for many) will be spread over multiple locations, if not ubiquitous. We can all contribute from anywhere and at anytime using tools already available.

The world of learning is also changing rapidly. Virtual reality workshops could become a radical disrupter and challenge those most die hard in insisting on the need for classroom training. People are learning new skills each day via YouTube videos. Do you want your doctor trained this way? Maybe a better question is, if this is where they can learn new techniques quickly, do you want them to NOT use them?

What skills are most required in this new world of work? In addition to technical skills, some posed were

• Analytical skills
• Intuition (creative) skills
• Information brokering
• Multi-cultural perspectives
• Multi-disciplinary flexibility

Concerns were raised that the rapid pace of change is counter productive, and some warned that if we do not make it a habit to slow down, we might lose too much of our humanity.

At a session dedicated to participants from UN agencies, 23 participants discussed challenges to training design, development, and delivery. Concerns were raised about the need for faster, agile instructional design processes. In a discussion about uses of blended learning, it was clear that the term has many meanings. Many saw that at least 3 phases are needed—an online course phase, followed by a classroom phase, completed by a mentoring phase. (This was heartening to see!) Others talked about using a variety of tools and modes in the blend, not just single conceptions of “online.” We shared the tools we use, and agreed to keep a community together as much as possible to share experiences and strategies. ECBCheck, a quality management system for online learning, will be taught in a MOOC starting 15 February,2016.

In a session on learning analytics, the challenges of sorting through piles of data to turn them into useful information for diagnosis and prediction to improve learning was highlighted. In university settings, they are first tackling the task of gathering analytical data for predicting positive learning experiences, engagement and completion. Decision makers want single indices to use as predictors to help learners at risk, but with so many contributing factors, this is difficult. Learning quality, which will be much more complex to analyze, will have to come later, in the eyes of the presenters.

by Bryan Mathers

In a visually inspiring session, Bryan Mathers provided a graphical tour through ideas and concepts about learning. His blog might be of interest to you.

A final session on “2500 years of Learning Theory in 25 minutes” by Donald Clark, left few theorists unscathed. It was a challenging and thought-provoking session, but one full of hyperbole. However, the emphasis on practical theory (an oxymoron?) was refreshing.

The vendors also offered some interesting food for thought. Open Assessment Technologies offers an open source testing system (TAO) that provides 17 interaction types, including a customizable assessment type that could include an embedded and graded simulation. Online assessment can be much more than multiple choice, even though we haven't exhausted that mode yet. TAO tests can be served online from any server.

The Open Badge Factory offers an inexpensive service to allow the creation of badges that are easily transferable to many platforms, and stored in a cloud for future application. With the growing world of competency frameworks, and a WMO Global Campus, a robust badging system might serve an important role. Other vendors focused on content creation or course management systems, for example. Online assessment tools were well represented.

Patrick Parrish (WMO) and Maja Kuna (EUMETSAT)

The shift is accelerating

The highlights at Online Educa Berlin today included an inspiring presentation by futurist Simon Nelson, a session on the state and future of MOOCs, and a panel session on diffusion of online learning for vocation education in developing countries.

A common theme of the day was how to respond to the rapid changes in educational technology (where denial is no longer an option). David Price called for increased personal responsibility required in an open world of work and learning, including application of the 6 “Do-Its”:

• Do it yourself (Autonomy)

• Do it now (Immediacy)

• Do it with friends (Collegiality)

• Do it for fun (Playfulness)

• Do unto others (Generosity)

• Do it for the world to see (Visibility)

"Any lecturer that can be replaced by a YouTube video will be." Photo by June M. Brevik

Price also advocated for taking a hacker ethic (don't ask permission, do it yourself), and a Jugaad attitude (reuse what you have, make it good enough) in finding solutions. He advises us to cultivate innovation with the enthusiasts, to protect innovation as a precious asset, and to just allow those who are roadblocks to step aside. Also interesting was his reference to the fact that Google has decided to compete in the market for talented staff by no longer hiring on the basis of a college degree, but only on skills and competencies. It has been proven in a double blind study that this policy has had no impact on performance. Other companies are following this practice as well.

In a session titled, “MOOCs: Here to stay,” 5 managers in institutions offering MOOCs provided reasons for offering them and their strategies.  The conversation never hovered long around the question of “Are MOOCs effective?,” but quickly assumed that MOOCs are a powerful innovation and we simply need to make them work better. The way to do that is to move forward, try them out and do continuous improvements—and not try to imitate the classroom. The key area for improvement is in learning activities—getting people doing relevant and authentic tasks. MOOCs have a business stance, as well as a pedagogical one. Pierre Dillenbourg of EPFL, Lausanne said that MOOCs have doubled the reach of the school in 2 years. The CEO of FutureLearn, Simon Nelson, pointed out that they are making the criteria for offerings more stringent. Tens of thousands of students have given FutureLearn MOOCs the highest marks in satisfaction, and they are unwilling to compromise that. Dillenbourg offered that “if universities are not prepared to take risks (like offering MOOCs), they don’t deserve to be called universities.”

Finally, in a session on “Using technology to reduce the learning divide in vocational education,” representatives of UNESCO shared that in the UN sustainable development plan, it is directly stated that for human capacity development there is a need to leverage ICT as a critical ally. Donald Clark offered that fighting online learning infusion in education is futile, because the current costs of education are unsustainable, the demand too high, and because learners are using it already—via tools like Wikipedia and YouTube, at minimum. EVERYONE with a computer uses Wikipedia, even teachers who disallow their students to use it, because is it one of the most incredible innovations to come about in this century.

Note: Future Learn will be offering a MOOC on Blended Learning starting 1 February, 2016.

Patrick Parrish (WMO) and Maja Kuna (EUMETSAT)

Blended, Flexible, and Personalized

Maja and Patrick are attending the Online Educa Berlin Conference. One highlight today was the workshop, Knowledge Exchange on Technology for Education. In one breakout session, participants from throughout Europe and outside, from both university and corporate environments, discussed their experiences and challenges with implementing effective blended learning solutions. Sound familiar? Things we raised as common concerns included:

* Addressing the cultural change in the organizations (for both teachers and participants)
* Choosing technologies and tools
* Finding guidelines for designs for how to blend the modes
* Fighting perceptions that online learning is cheaper
* Fighting perceptions that only the content format needs to change
* Ensuring quality in the online and classroom phases
* Ensuring and measuring impacts

In another breakout session on creating Flexible Learning Environments for students (for example mobile), most of the same issues were raised, but additional ones included:
* Technical challenges, of creating useful mobile apps, for example
* Cultural challenges with the introduction of ubiquitous learning (and teaching!) opportunities
* Deciding how to use the extra context-of-use data that are generated (e.g., how to customize for a 2-hour train trip one learner needs to take, versus a 15-minute bus ride)
* Dealing with assessment and the potential need for peer/teacher interaction and feedback in a completely self-paced learning experience

Finally, we attended a session with Stephen Downes on developing personal learning environments (PLEs). In this approach, each learner is given the opportunity to collect data on learning resources used and activities engaged in (wherever they occur), as well as professional accomplishments, and to share data about these and the badges and certificates collected along the way. In addition, PLE's include ways for receiving recommendations for courses, resources, and learning paths, and ways to interact with other learners, content, and experts in your areas of your interest. You can think of it as having personal campus in the cloud that you define based on your needs and interests, a personal database of all things you do to learn and demonstrate your developing skills, and a way of sharing this to the level you want and need to with credit providers, employers, and anyone else who might need this information, including yourself. Workshop attendees worked to define the necessary capabilities of a PLE system, and prioritized the development phases.

Maja Kuna (EUMETSAT) & Patrick Parrish (WMO)

A living guidance to prepare a simulator exercise

During CALMet 2015 we had a workshop on needs for an Assessment Simulator Exercise seen from
different perspectives.  We worked in groups. The question to answer for each group was: “What do you need to feel yourself safe in this position in the simulator environment.” We have collected all opinions and have put these together into this guidance. In November we have added opinions from the WMO/Eumetcal Train the Trainer participants to this list.

This is a living document which contains a minimum needs list from what we hope that it will expand in the future.  We hope that this document will help you. We have the feeling that you may want to use this document as a guidance to prepare your simulator exercises. In case you don’t have any assessment goals for your simulator exercise please realise that the assessment part of this document most probably still contains some useful points to use in your simulator exercise.

Good luck by organising a simulator exercise and we hope to hear your simulator experiences so that we can learn from them too.

A big thanks to my partners in crime, Mark, Ian and Vesa, for the help with doing these workshops and all the colleagues that gave input on the various occasions.

Heleen

What are the minimum needs you have to think about when organising a simulator exercise?

What Determines Quality in Training?

A workshop on identifying quality criteria was offered at the CALMet Conference, September, 2015, in Seoul, Korea. The participants were presented with this scenario: You are reviewing a training announcement from an unknown provider in a topic area you are interested in, but you would like to know about the potential quality from a reading of the announcement before investigating further.
Participants were asked to work in small groups to develop criteria within three areas: Organizational, Processes, Product. They were also asked to assign priorities to the criteria as time allowed.

The resulting draft criteria lists compiled from their decisions was then compared to a previously prepared draft list of criteria, which was developed from a review of several existing quality management systems for educational resources. These included ISO/IEC 19796, ISO 29990, Open ECBCheck (for eLearning courses), EFQUEL (for eLearning programmes), Commonwealth of Learning (UNESCO), and the WMO Competency Requirements for Education and Training Providers. Approximately 6 additional systems were examined.

New suggestions for criteria from the workshop were added or merged with the existing ones.

What came out on the top? See the list below, as well as the more complete report. Many more important criteria are contained there, as well as a discussion of the results.

Organizational

•How trainers are hired, trained, and evaluated
•Existing accreditation that covers vocational training, or existing quality assurance processes
•Numbers, roles, and qualifications of training staff members (including access to content expertise)
•Location/country, accessibility (including online presence)
•Facilities, resources, and technology infrastructure

Processes

•how learning outcomes are specified and how they inform the training content and activities
•Good development and delivery processes, documentation of processes, ISO certification (if used)
•How learning needs are determined
•how training is evaluated and improved

Products (including both courses and resources)

•currency of the content (up to date)
•learning activities match learning outcomes, opportunities for students to apply what they learn
•the relevance of course content to job competencies
•assessment methods, matched to learning outcomes
•performance-based learning outcomes
•Timing of courses (Duration and scheduling)
•the results of past evaluations of this course or resource (or related ones), student recommendations


Conclusion

Thanks to those who participated in developing these draft criteria!

As work continues to decide how quality information might be shared, in a WMO Global Campus Shared Calendar and future learning resource Registry, for example, the discussions to date should provide some useful input. But they also raise several issues for further debate.

•How does accreditation relate to quality assurance of vocation training and individual courses and resources? Is additional QA still useful?
•Does ISO 29990 certification cover the required criteria? Is more quality information required for specific resources and courses?
•How can we apply quality standards that are inclusive of many types of training centers and units?
•Is QA information voluntary, or mandatory? How will it be communicated?

CALMet Online World Café

Roro Yuliana Purwanti and Vesa Nietosvaara

Inspired by the coffee and café tradition in the Korean culture, CALMet Online planning workshop was conducted using the World Café Method. Our goal was to gain new ideas and approaches for successful CALMet Online events in the future.

We started with a 5 minute presentation about previous CALMet Online events, shared a warm welcome and introduced participants to the World Café process and rules (called “café etiquette”). According to the etiquette, each participant chooses the table they would love to sit for the first round. Four different tables were available for participants to sit and discuss the given topic. After 15 minutes, the bell rings and participants move to the other table, change groups and discuss the next topic. Result of the previous topic are left visible on the table, as a reference for the new group.

In the first round, we asked the participant “Do we need CALMet Online? Why? To what purpose”. As a result of the discussion the participants agreed that CALMet Online is still needed, as a forum to meet and share ideas and innovations, as well as to keep communication and collaboration within community. 

In the second round,the participants worked further on “What components in CALMet Online need improvement”. They explored the existing components of each CALMet Online, and identified which components should be improved.

In the third round, the participants were encouraged to imagine “How the future CALMet Online could look like? What other approach can be used?” In response with this topic, beside provided some new approaches on schedule and activities, participants also came up with ideas in technology and communication, such as using more social media and possibility of using mobile device applications. 

In the last round, participants were still sitting in a group, but they now needed to work individually on topic “What can you do to make the next CALMet Online a success?” Their task was to fill in the blank individual worksheet with their advice, feedback and suggestions for success for the next event. They discussed also within the groups.

After the 4^th round finished, we asked participants to voluntarily tell us about what had been happening in their tables. They made nice summary that we captured and noted as a reference for further CALMet Online event planning.  Some of them also shared their commitment to contribute in the next events from planning to implementation and evaluation, as response to the last topic. We also had some fresh ideas and initiatives written in their A3 group worksheets.

From facilitator’s perspective, we are happy that the workshop reached its goal of having contributions from CALMet community for the early stages of planning. Fifteen minutes was ideal for each round, because participants needed the first 5 minutes to adapt with the new group and another 10 minutes was suitable to produce some discussion results.

Overall, indicated by some contributions we had for the next events, we can say that the workshop conducted smoothly and helpful for us when planning CALMet Online 2016. More about the World Café method, see http://www.theworldcafe.com/

Workshop “Success Profiles for Trainers”

Wilfried Jacobs (DWD), wilfried.jacobs@dwd.de

1.    Background

For successful training not only software or training material of high standard are necessary but also eligible trainers. It is a big challenge to find trainers who are prepared and able to deliver training accordingly. The WMO-Paper No 1114 “Guidelines for Trainers in Meteorological, Hydrological and Climate Service” is already very helpful. However, checklists for assessing trainers and persons who intend to work as trainer are strongly needed. The goal of this workshop was to prepare a list of challenges, required competencies and attitudes that can be used as an input for a success profile for trainers. Different roles (teaching, training management, both) and different training situations (distant training, blended) were discussed.






2.    Results

a)    Challenges

Although many well-known statements were found it made sense to consider them before discussing the personal attributes and competencies.

• For self-study clear instructions, aims and objectives are necessary and the trainer should avoid the feeling of isolation;
• During live presentations a compromise between the schedule and flexibility is necessary. A further challenge is to have a plan B in the case of technical problems;
•  Because online-training (self-study and online presentation) is in principle possible with hundreds of students a big challenge for trainers is to accommodate with many participants, especially if contributions in fora or home work has to be considered. To give all support is one of the biggest challenge and very time consuming.
• For blended courses an according link/bridge between the online and the classroom part should be achieved. At the beginning some resistance between trainers and students may happen.
• For all (self-study, online presentation and blended) the challenge is to have enough personal and budget in order to develop and to monitor the material that has to be also logical and consistent. The challenges in relation to pedagogic are manifold: Despite the presentation style the motivation, listening to students, pace, giving feedback also the knowledge of the audience and the different cultural backgrounds are crucial. A further challenge is to be aware in recent developments of media. Additionally, a training support officer is needed for overcoming the technical issues. In relation to the training organization the challenges exist to find experts for teaching and supporting, to get the required budget, to advocate for needs and to provide marketing by announcing a training event or by finding sponsors.

      b)  Competencies and personal attributes

No training roles have been distinguished.

• Pedagogic / personality: Important attributes are competence in communication, and in engaging people (ability to make the training attractive, narrative and rhetoric skills). Trainers should be empathic, tactful and respectful, emotional intelligent, honest and reliable. Also creativity, curiosity, flexibility, agility of thoughts and a good understanding of their role in combination with availability and responsivity are important. Trainers and managers have to be able to work in a team and to work under pressure.
•  Even if a training support officer is available the trainer and the training manager should have technical competencies (Learning Management Systems, media in order to select the according teaching method).
• Expertise (good and current knowledge) is very important for trainers. However, also the manager should have a good overview.
• A high competence in organizing is required for both trainers and training managers. Whereas the trainer has mainly to organize the training event the training manager needs also competencies in planning training from a strategic point of view, in organizing the budget and the personal and in handling service-orientated.

c)    Conclusions

During this workshop many challenges and required attributes for trainers and training managers have been described. The results can be seen as a promising first step. However, further discussions will be necessary in order to complete the list of attributes and to improve the logical structure. As soon as this has been realized a check list should be prepared that can be used for training of trainers and for self-assessment of trainers. It might help in the selection of trainers according to the challenges in relation to particular training events. Such a list can also be helpful for assessing applicants who are interested to work as a trainer or as a manager of training.

Strategies for Enhancing Learner Retention (Brad Snyder, Meteorological Service of Canada)

This is designed as a follow-up from the workshop on “Strategies for Enhancing Learner Retention” at CALMet XI, as a way to summarise the findings and hopefully inspire other summaries and further exploration.

It is common for learners who participate in training to forget much of the material in weeks -or even days- after it is given. It is in everyone’s best interest to maximize learner retention and minimize what learners forget. This workshop was designed to give participants an understanding of various types of learner retention strategies and to consider how they can be incorporated into different learning solutions. An ignite presentation introduced some of these strategies (in addition to what was presented in the Booster Question plenary talk).

The following is a summary of learner retention strategies based on Brown et al (2014)

1. Pre-course reading – spreading out learning by incorporating elements of instruction before formal training.

2. Flipped classroom - Lectures on line followed by Exercises (Homework) in the classroom

3. Times for reflection during class – forcing student to retrieve and reflect on information, often in the form of writing or oral exercises.

4. Quizzes and feedback - quizzes are more effective than restudying material. Feedback helps correct or guide students in the right direction.

5. Interleaving topics – studies have shown that interleaving or mixing related topics together (as opposed to topics given sequentially) helps build retention.

6. Post-course reflection – similar to during the class, refection (in the form of exercises or webinars) helps the retrieval process and hence retention.

7. Booster questions - Questions sent to students after training – in the form of quizzing to practice retrieval and increase retention.

The short workshop had participants assess the effectiveness and/or potential use of these seven strategies for the following modes of delivery.

1. Online (asynchronous)
2. Online (synchronous)
3. Classroom
4. Blended Course

Participants assigned ratings for each strategy/mode of delivery; 1 being least effective, and 3 being most effective. The table below summarises the findings from 4 break-out groups (according to mode of delivery) at the conference:

Mode of Delivery

Strategy	Online Asynchronous	Online Synchronous	Classroom	Blended
Pre-course reading	3	3	1	3
Flipped Classroom	2	2	3	3
Reflection -during	3	1	2	3
Quizzes/feedback	3	2	3	3
Interleaving	1	3	3	3
Reflection - post	2	1	2	3
Booster questions	3	3	2	3

In terms of those strategies which were considered effective, Quizzes/feedback during training and Booster questions following training, ranked highest. The strategy considered least effective was probably post-course reflection.

In terms of mode of delivery considered most effective for these strategies, it was a blended approach (not too surprisingly). Online synchronous was believed to be least effective (slightly) in adopting these strategies.

Had there been more time, it would have been worth discussing the relative merits amongst all participants to get consensus and ensure clarity. What is clear is that there are strategies that we can adopt now to help improve learner retention. It is hoped that this information will inspire trainers to incorporate these strategies in future courses.

Reference

Brown, Peter C., Henry L. Roediger, Mark A. McDaniel. Make It Stick: The Science of Successful Learning. Belknap Press. 2014. Print.