Category: Professional Development

Some more on what learning needs to pick up from gaming

So another post on the lessons from the world of gaming.

This one was sparked by an article considering if the latest Legend of Zelda game is the greatest ever in terms of design.  I’ve spent quite a bit of time already in this iteration of the world of Hyrule and it is difficult to disagree with the arguments in the article.

The closing paragraph should particularly resonate with learning professionals thinking about how to support their organisations:

the job of the designers is not to hold your hand and guide you around a set path. It is [to] reach out hundreds of hands and leave it up to you which you grab first.

Wow! There’s a topic starter for instructional/learning design debate!

Whilst in the past people may have talked about things like “learning styles” to warrant different approaches we are now, instead, in a position where we consider the different approaches we might drive performance and support learning for people at different starting points and existing levels of competence.

Now the counter argument would be that the multiple, even unlimited, permutations of many games are not feasible in instructional design.  Instead we end up with versions of relatively simple board game constructs when gaming or fairly restricted ‘serious games’.  However, with dynamic algorithmically driven learning there is the potential for an explosion in personalisation.

Screen Shot 2017-05-29 at 20.33.00.png
Winning a battle with only your general left may not be recommended. But a win is a win.

Now the above image is an example of a counterfactual gaming experience, crusading as the Byzantine Empire.  Traditional L&D has of course made use of just such counterfactuals, through role plays, business modeling, simulations, etc.  If you can create an appropriate model then the variations are possible – with different focuses possible across, say, finance, marketing, etc. – all in the ‘safe’ environment of not impacting actual bottom lines, patients, customers, etc.

By thinking through game constructs there is the potential to think about what you want to achieve in a different way.  For example, the battle focused historic counterfactual (such as Total War games in the above image) and more character focused such as the grand strategy Crusader Kings 2 (images below) are effectively giving you the same goal (rebuilding the empire) but in very different experiences.

Rome
Expanding (and renaming) the Byzantine Empire across c.100 years (of game time)

The storytelling in a scenario such as the above is prompted by certain actions (for example Byzantium becoming large enough to reclaim the title of “Rome” as an achievement) but is not as structured as, say, a linear first-person-shooter game like Call of Duty.  The latter, more linear style, offers up the potential for set storytelling, with some games much better at this than others.  Which leads to an argument that future instructional designers would be best sourced from graphic communication or creative writing backgrounds.

Traditionally simulation has, of course, taken many forms in workplace development – from table top games to computer scenarios.  The challenge with simulations remains the balance between ‘keeping it real’ (i.e. actually useful in the workplace environment) and maintaining interest through the storytelling/fun and other components.  Meanwhile this post makes good points about balancing complexity versus needing to know ‘now’.

So what to takeaway?

  1. Think about how much hand holding is appropriate – it’s not always a bad thing.
  2. Have the plot/narrative/story drive motivation.
  3. Reward with hidden achievements.
  4. Use users/learners to determine if you are hitting the right balance between reality and gaming elements.

CIPD L&D Show May 2017

Not too much to report from this year’s show as I left after only about 4 hours and only a couple of the learning sessions.  However, I did have some good chats with people on the floor and it was pretty clear what the main growth area was from last year – apprenticeships providers, levy support, etc etc.

I left early partly as I was full of cold but also as the first session I sat through, from the Open University, had a narrower focus than their sessions in the past and did not really answer the topic (“How to Make Your Digital Learning More Engaging”).  A nice cartoon fell in the trap of many such eLearning examples – repeatable production values.  Yes the concepts are true – humor, animation, bite-size, storytelling, etc but it worked as an example due to the quality: most of us are not going to get David Mitchell to do voice-overs on our eLearning.  It then reverted to being much a pitch for making use of their content platforms – OpenLearn and FutureLearn – i.e use their digital learning not your own.  Which is an argument you could make of course.

The second session I attended, was from Bolt From the You Ltd, on “Making it Stick – Turning learning into real change”.  This was better and considered why change does not happen: lack of engagement (needs to be fixed by being linked to the individual, think about how measure for impact), lack of learning culture (need ownership and partnership from sponsors to drive through) and to close gap:

Attention density:

Exposure + repetition + application = permanent change.

“Learning analytics”: a red herring?

A key message from my recent Kirkpatrick program was to start with the “top of the mountain”.  In this metaphor the top, the peak, the target to reach, is the organizational vision.  Strategic learning programs are, therefore, helping the org reach this vision and should be evaluated as such.

My reflection during the program was that this, of course, is common sense.  We should be working to support the organizational goals.  The challenge then becomes prioritizing multiple needs – so only by forecasting potential impact up front can prioritization be done correctly.   And this is one of the areas where there is misconception with KirkP – it should be about starting with the end in mind and working backwards (not just dealing with level 1 in a standard way and then carrying on from there).

In terms of evaluation of success, LEO have recently discussed the role of learning analytics (LA).  Now, like a lot of things in L&D, I would say the problem with LA is that it has meant multiple things to different people.  One of the earliest examples I saw, sold as LA, was Starfish Solutions (SS) who had a clear solution and goal – use existing LMS/VLE data to improve student retention.  SS makes perfect sense for organizations where educational outcomes and student retention are the organization’s objectives.  I liked SS’s solution (in part discussed with them at the BBWorld Conference back in ’09) but it also faced the challenge that, for many university courses, there was/is less need for ‘big’ data solutions – lecturers know their students in traditional models.  It only made really sense when talking about large scale education – the problem then, again, is that ‘large scale’ means multiple things to different people 😉

The LEO article does a good job at articulating the problems I have always had with L&D impact – especially how to assess when there are so many other uncontrolled variables.  As mentioned in my previous post on the KirkP Cert, this was my main challenge I wanted clarity on from the course.  The recommended KirkP approach of identifying multiple ‘indicators’ (suggesting behaviors are on track for the desired result[s]) that can show a correlation – a key learning point for me.  In this model, therefore, we are building a ‘chain of evidence’ akin to a court of law – “data, information and testimonies at each of the four levels that, when presented in sequence, demonstrate the value obtained from a business partnership initiative”.

What I really liked about this is the clarity of the evidence/steps up the ladder/mountain, from bottom to top:

  1. Individual Outcomes
  2. Team Outcomes
  3. Departmental/Divisional/Unit Outcomes
  4. Customer/Client Response
  5. Organizational Outcomes
  6. Customer Satisfaction
  7. Market/Industry Response

It is this breakdown, of benefit chain, that I will likely adopt in my existing project planning documents.

Let L&D then be clear, as the KirkP course made clear: stop trying to tackle issues through limited/impossible direct causation but instead correlations.  I would say this is a much better approach than simply seeing “measuring learning impact as a waste of time and money” as the LEO article mentions many people argue.

Therefore, I would argue, let us (i.e. L&D professionals) not worry about learning analytics but instead organizational analytics (i.e. goals and metrics) that can be seen as trending over time and aim to see where our investments have an impact.   As recommended in the KirkP programme, do not reinvent the wheel, borrow metrics from elsewhere as they will already be used by stakeholders and those same stakeholders (should) understand the logic.  This should then allow us to, as I’ve hinted at before, not worry about ROI but instead (as recommended by KirkP) Return on Expectations.

So what do I think of KirkP following the course and certification?  Well I’d have to agree with the LEO suggestions:

  1. It’s better than doing nothing… which is what most organisations are doing.

  2. Think about what level of evidence will be good enough for you. As the Kirkpatricks have pointed out, a chain of evidence that would not stand up to scientific rigour may be enough to convict someone in a court of law. If a level of evidence is good enough for a court then it’s probably good enough to convince your board to invest in L&D.

  3. Good enough develops into great.

Kineo Connect: progress with purpose (event)

Well, I’ve finally had my first go on a HoloLens – after failing at BETT – thanks to MakeReal at the latest Kineo client event.

It really confirmed to me the potential value of HoloLens for multiple solutions.  The example available to try out was an interactive site map (i.e. building complex not website) – great for not having to carry around scale models with you!  For learning there are some great examples starting to emerge – such as this one for healthcare.  Cost, of course, remains an issue – with the development edition currently a tasty £2719.  It was also an opportunity to try an activity via SteamVR OS – building a reactor in VR.

The presentations/workshops, away from MakeReal’s demo space, focused on:

  1. Kineo’s Learning Insights report: Move for L&D to “facilitators of career development” from “deliverers of training”.
    1. I would agree with this.  I tend to describe my role as creating scaffolds for the organization to succeed and its people to be empowered.
    2. Report here.
  2. Social Learning.
    1. This was a nice recap of some of the logic and reminded me of the need to continue to encourage people to share and shift cultures – for example, the value in people sharing their ‘bibles’.
    2. Ideas included “find an expert”, something we are trying to do via badges for people at the top of our competency model.
    3. Slides here. White paper.
  3. Content Curation.
    1. I’m always torn on this topic as whilst I agree there is information overload I’m also conscious that information teams have fallen away in industries like law – in part as the WWW has made a lot of information free and thus the ‘value add’ from a professional team is tricky to identify/articulate.
    2. The challenge, as I see it, is how you combine a world of personal information management (the presentation mentioned bookmarking as example most people would do) and PLNs with opportunities to add value (context, metadata, write descriptions, etc.) centrally.
    3. Recommendation for a search > aggregate > filter > add value > promote model (akin to Jarche’s seek > sense > share).
    4. Couple of examples were run through – use of existing content to support BDO’s competency model (i.e. not needing to author new content) and Anders Pink as a tech solution (which I’ve trialed and like but raises questions over how it fits into your wider ecosystem).
    5. Slides here.
  4. Interactive Video.
    1. Have to agree there is a lot of value in interactive video where you can find the right use case, authoring and deployment approach.
    2. It was interesting to see/hear the Kineo approach – including the blended of ‘learning’ content with existing promotional video, for example, with Rolls Royce.
    3. Some nice ideas – like weighting questions where there is ambiguity (no true/false), gopro cameras of POV style, setting up secured YouTube channels for UG content submission and the oft shared Australian Deloitte video.
    4. Some of this could be done in Storyline – advantage of their authoring is multi-device SCORM publishing.
    5. Slides here. Guide here.

Another reading catchup

Another in the occasional series looking at some of the reading I’ve stockpiled with reflections for anyone who might be interested in the topics covered:

It’s never too late to learn [Journal of Workplace Learning; Vol. 27 Iss 6; Russell Paul Warhurst and Kate Emma Black; 2015]

An attention-worthy approach (from Newcastle Business School) to the empirical study of workplace learning – as the articles puts it:

The informal, incidental, tacit and social nature of much workplace learning poses methodological challenges in ascertaining from respondents exactly what they have learnt and particularly, how their learning has occurred (Fuller and Unwin, 2005; McNair, 2013). Therefore, visual elicitation techniques were deployed in advance of interviews to assist participants in exploring such tacit learning accomplishments and implicit learning processes. Three visual techniques were used:

(1)  Timelines: Participants were requested to depict their work over the preceding ve years as a horizontal line showing the degree of change, and learning, as gradients on that line against a vertical axis scaled from “set backs” through “stability” to “rapid”.

(2)  Sociograms: Participants were requested to indicate whom they interacted with over a typical month, the nature of this interaction (e.g. face-to-face or electronically) and how signi cant they felt these interactions were for them.

(3)  Pictors: Participants were asked to produce a visual representation of how they viewed themselves in their social worlds, in response to the question “How do I see myself as a later-career manager?”

I thought the three approaches above were interesting ones considering the challenge in getting professionals to reflect and think how they have developed.  Indeed they could also be used to visually represent impact of programmes to, in part, tackle the evaluation challenge.

I have recently been running a series of workshops at my organisation on career development opportunities – in part around the advantages that the May changes to apprenticeships will bring.  The ability to retrain, even if you have a degree or other qualification, via an apprenticeship is a key message.  Thus I liked the idea that “ageing populations need to be seen as a key, growing, natural asset rather than, as typically construed today, a liability”.  The article itself focuses on informal learning but realistically there will be opportunities to think again about skills development via multiple routes.  “Later-career workers need to be alerted to the learning potential within their jobs and the capabilities to leverage this potential” – yep!

How to enhance the impact of training on service quality?: Evidence from Malaysian public sector context [Journal of Workplace Learning; Vol. 27 Iss 7; Abdul Rahim Zumrah; 2015]

Whilst it is common sense to presume training needs to be reinforced/transferred to actually have an impact on quality this article argues for a positive relationship via a measurement approach.  Overall it is useful in reinforcing that training alone will not impact on performance:

The finding of this study is an important outcome that has not been empirically determined previously in the literature, which highlight the significance of transfer of training as a mechanism to enhance the impact of training on employees’ performance (service quality). This finding provides support for the social exchange approach, suggesting that where employees perceive the support from their organization (they have been sponsored to attend training programs by organization), then feel an obligation to engage in behaviors that benefit the organization (transfer the training outcomes to the workplace) and are also willing to expend more effort to fulfill their organizational goals (delivering quality service to organization’s customers). The finding of this study also responds to calls for research to investigate the mediating factor between HRM practices and employees’ performance (Tremblay et al., 2010). The finding also helps to clarify the ambiguity in the literature in regard to the relationship between training and service quality (Beigi and Shirmohammadi, 2011; Chand and Katou, 2007; Cook and Verma, 2002; Hung, 2006; Schneider and Bowen, 1993; Zerbe et al., 1998). Specifically, this study extends the literature by providing empirical evidence that transfer of training has a mediating effect on the relationship between training and employee service quality in the context of the Malaysian public sector.

How to Solve the Content Discovery Problem [Brandon Hall Group, Presentation Deck]

A topic of interest to me with my library/content/information route into the world of learning.  Unsurprisingly it identified the challenge of content:

94% of companies say that managing the expanding content library is a challenge presented by today’s learning environment

It also picked up on a point I regularly make and have previously tried to argue should be increasingly irrelevant through decent organizational design:

content overload is only getting worse with siloed organizations using their own tools

In other words, departments such as KM, marketing and L&D should be working together but instead buy their own solutions and thus overwhelm internal and external clients.

One of the solutions for the learner experience is federated search – which was the hot topic 10ish years ago when I did my library/information MA (especially for HE orgs) but corporates seem to have failed with this.  Indeed, in some ways such as looking at experts and peer based collaboration, the corporate world is still catching up with academia.

The filtering is then the challenge.  However, whilst the presentation suggests a “personalized feed” integrated with other business applications I wonder if this ignores the UX of some tools.  Content, generally speaking, will have some level or motivation/gamification built in to their platform.  So is the solution, instead, a combination of apps by audience/need?  The presentation instead suggesting Edcast as the aggregating “knowledge cloud”.  The traditional model probably being the topic/course centric LMS.

The takeaways are fair though and the kind of issues I have tackled in the last 15ish years:

  • Traditional content models cannot keep up with modern business needs
  • Keeping content current and fresh can be a challenge
  • Legacy content can also become unusable
  • The modern learner wants choices, personalization and a familiar interface
  • Subject matter expertise exists farther and wider than most organizations can reach

12th annual Keeping Pace report [Evergreen; 2015]

Very late catching up with this one but another document on a topic of interest – online “K-12” education – and this report helps highlight how the US (in this area at least) are considering a variety of approaches to improve outcomes and meet demand.  IMO a form of UK online free school is surely needed.  Indeed the report identifies the kind of needs, from the American online ‘charter’ school equivalents, that the UK could tackle:

In the case of elementary and middle school students, many attend an online school due to temporary reasons (illness, injury, behavioral issues, allergies). In high schools, many students move to an online school because they are behind and at risk of dropping out of school altogether.

Thus dispelling some of the myths such as the US demand coming from geography and there would not be the demand in the UK.  Indeed there are some positive messages on impact too – such as in Florida where “students outperform state average in end-of-course exams”.

The Disruption of Digital Learning: Ten Things We Have Learned [Josh Bersin]

A newer piece, as it was published this week, with a lot of good stuff summarizing the state of play for those of us who like to say we work in online/digital learning.  It is a little US centric and knowledge-worker focused in places (which to be fair it acknowledges) but it is one of the better things I’ve seen from Bersin for quite a while:

http://joshbersin.com/2017/03/the-disruption-of-digital-learning-ten-things-we-have-learned/

Overall, it really drives home that the myriad of tools now available means the boundaries between home/work/learning have effectively gone.  Indeed I’d challenge the point about people spending so much time on email always being a negative – yes, a lot will be junk keeping people away from “specific” work but how many of those emails are learning experiences?  Even if the learning is just related to clandestine organizational cultures or rabbit holes of bureaucracy?

I especially like that Microsoft Teams got a mention having had a first look this week.  MT does an awful lot of what your traditional LMS would – discussion groups for communities of practice, VOIP/video for tutoring, task lists for activities/assessments, content sharing for reading/watching/reflecting, wikis for collab authoring, etc etc.  SCORM, I guess, would be the elephant in the room for most LMS deployment but there you’d be looking at if that content is really what you want going forward.  How we might combine MT and xAPI is very much on my “to do” list.

There are things to criticize – for example a lot of this isn’t new just being delivered differently (e.g. microlearning) but as mentioned earlier a lot to like too.

I’ve got a whole ‘reading’ folder on my desktop going back many months so expect some more of these posts soon…