Can L&D learn anything from The Teaching Excellence Framework (TEF) experience?

http://www.bbc.co.uk/news/education-40356423

The above article is one of many to pick up on the outcomes of the first UK Higher Education TEF results.  The standout piece of the story, for me, is that the measures being used to judge “teaching”, including:

  • facilities,
  • student satisfaction,
  • drop-out rates,
  • whether students go on to employment or further study after graduating.

are as, the article points out, “based on data and not actual inspections of lectures or other teaching.”  Swap out “data” for “indicators” and you basically have the L&D model.

The Ofsted inspection of schools is, of course, more teaching focused but, even there, judgments of schools use other metrics.  School teachers, for example, are expected to support “progress” that is influencing by beyond what is immediately impact-able.  The impact of other factors, like parenting, are not factored in.

Therefore, between Ofsted, TEF and L&D (via models like Kirkpatrick) we really do not seem to have cracked the nut of measuring how well we develop learning and improvement.

With TEF it feels like a missed opportunity to evaluate the quality of ‘traditional’ lecture centric programmes versus more seminar or online models.  Some included elements, such as student evaluation of facilities, are also surely difficult considering most students will only know one HEI and thus not have something to benchmark against.  The cost of London living presumably impacting on the poor perception of many London-centric organisations, including LSE.

So, beyond saying “well universities haven’t cracked it either” what can L&D departments learn?  I’d be interesting in hearing people’s thoughts.  One item from me – with the growth of apprenticeships and accredited programmes “training” is being reinvigorated but also being reimagined with a performance focus and approaches like bitesize learning away from the big “programmes”.  Therefore, for me, the more metrics the merrier to try and build a picture of our organizations.

ALT Winter Conference 2015 (Day 2)

I am hoping to attend at least part of a few ALT Winter Conference sessions today, notes below.

Jessica Gramp (UCL): Welcome to ‘Moodle My Feedback’

  • Moodle plugin being developed at UCL – this was an update on progress as it has been showed at previous events.
  • Adds My Feedback to profile block.
  • Problems with default Moodle included feedback is stored in TurnItIn – this tool aggregates links back to TII, rather than having to go into different courses to find them.
  • Tells you if students have looked at feedback.

Using Social Media Data in Learning Practice and Research

  • 9 minute demo video: https://vimeo.com/147996126 using Facebook data, visualized, for research.
  • Overall I took that the point here was there are a huge amount of possibilities and questions from this.

Gaming the Conference: can Playful Interactions Engage Delegates

  • #altcgame
  • Some details on the game that took place at the conference.

Arena Blended Connected (ABC) rapid curricula development at UCL

  • Presentation on an approach that has been found to be a great way to engage academics.
  • Previously did not have a coherent way to talk to academics about their programmes, they have adapted Ulster’s approach from JISC Viewpoints project.
  • “Connected Curriculum” – wider framework for good design at UCL: this is the C part.  A and B help with the structure to reach those outcomes.
  • Core to the approach is a “Gamestorming” 90 minute workshop, have a high energy discussion on what to do.  Consider issues such as what learners should be doing, mockup of examples of modules (broken down by learning types), etc.
  • Argue best to not start with assessment as to allow for mindset that anything could be assessed.
  • Use session to bring in other elements such as high level costing of module development, impact of including videos, etc.
  • http://blogs.ucl.ac.uk/ele/

TFPL – Dashboards in Excel Made Easy

  • Not an ALTC session but another webinar I attended today.
  • How to build dashboard combining one function, a chart, some data validation.
  • Used COUNTIF to count a total from data.  COUNTIFS for multiple.
  • Use bar conditional formatting and then create chart from there.
  • Add data validation with tool-tip for interactivity on page.
  • Then format the worksheet as a dashboard – basically calling on data elsewhere through a visualised approach.
  • Overall this was relatively straight forward stuff.

The Future of Learning: Project-based teaching and assessment supported through new media technologies

  • I didn’t attend too much of this one but when I came in then there was a discussion of some of problems with current university education, including the student focus on grade and the resulting concentration on what the exams will be.
  • Presenter’s research has just reconfirmed that content heavy education leads to rote learning.
  • Need to align assessments with what you really want the students to learn.
  • About personal journeys – only way to ensure students take subject to heart.

RefME as an example for viral apps

Do you remember Harvard referencing and building lengthy bibliographies during your student days?  If yes, did you find it time consuming?  Yep, thought so, did you ever use the techniques again (presuming you’ve not continued in academia)?  No?  Didn’t think so.

Even if we’re kind to academic referencing it is, at best, a necessary evil to show the development of research skills and the correct representation of ideas (i.e. to avoid plagiarism).  I’ve been to two events this week, the first on Adobe products will get a longer post but the second, on RefME, showed how a tool can go viral with users if it is really well targeted on solving an actual problem.

RefME has built a following of more than one million users quicker than Facebook or Twitter – all thanks to the humble citation!  Why?  Well those negative experiences of citation and bibliography building are now finally tackled through this very easy to use app.

When I studied there were some tools in this space, some institutionally backed, but none as easy as RefME appears from the demos at the #BLEevent.  Overall, I took away a key message here – focus on your audience (whoever they may be) and their challenges/frustrations.  If you are facing a lack of adoption with your corporate/institutional technology then it is probably fair to presume that it is (a) not easy enough to use and/or (b) not solving a problem that is felt keenly enough.

It will be interesting to see if any institutions opt to stand against it as a way of students ‘cheating’ by not having to spend hours formatting their own lists … not to mention all the librarians who will have another thing they ‘own’ taken away from them by technology.

Webinar live blogging: A 3D virtual business simulation for experiential learning in first year accounting

As mentioned many a time on this blog, I do not post all the notes I make from webinars, reading and the like on here.  This is partly as they totally consumed my old blog and made more reflective posts difficult to find.

Anyway, I was up early today to attend a “Transforming Assessment” webinar from ascilite in Australia and thought I would post some thoughts here in real-time, [in brackets is me rather than session content].

  • [I have attended a few of this series and thought this one might be interesting in understanding what my organization’s grad hires might be expecting going forward].
  • [Business sims are, of course, tricky but they have the scope to really achieve practical learning].
  • [Quick screen shot at start – looks like using Second Life].
  • 800-1000 students per semester.
  • Title: “Accounting for Decision-Making”.  Not intended as a technical, debit/credit, course.  Instead focus on supporting business decision-making.
  • Includes face-to-face lectures and seminars and some traditional assessments.  However, reviewed course off back of some negative student survey feedback around engagement (business students not enjoying the accounting element and therefore not picking up the correct outcomes).
  • Therefore, try to get more active learning with students more actively participating.
  • Reviewed in 2015, including simulation and web based assessment – complete overhaul of lectures, seminars, etc. too, all with active learning focus.
  • Not off the shelf – worked with piersim.com/about : International Education Services who had used a 3D world for a number of years.  Collaborated with them to develop the world for their use.
  • Developed the Virtual Business Enterprise (VBE).  20 student operated businesses with a supporting central bank.  Students control a number of things, including product pricing.
  • App is available at all times, can plan work around the trading session.
  • http://pier-enterprise.com/
  • [ran through some of the functionality] VBE dashboard allows students to communicate, to view financial reports, manage their profile, select their ‘job’ in company.
  • To access the VBE requires download of the VBE Viewer.
  • What happens in a session: uses avatars, preplanned and agreed tasks as well as in-world decisions (buy inventory or not, pricing of products, advertising decisions).
  • Software based on OpenSim.
  • Once a week for six weeks: 50 min trading session within the world.
  • One student per group use avatar, others communicating and performing tasks on dashboard.
  • Use financial statement from each session to consider what to do for next.
  • VBE is a shopping mall [showed some screenshots].
  • Game elements – if do not trade then the character’s health drops.  Had a staff member in session to rejuvenate any character that died[!].
  • Want them to take risks and make decisions (based on the accounting information) in a safe environment.
  • Can go to bank and take loans if they think appropriate: balance repayments/interest rates/etc.
  • In world law court for dealing with disputes [sounded like quite a lot of logic behind it] – if you cut quality of products you can be sued via court.  The in-house staff member (‘controller’) decides as judge.
  • Video: https://www.youtube.com/watch?v=cYDaYgt7i-M&feature=youtu.be – slower than real: is more frantic in the 50 minute trading sessions.
  • Assessment was linked to: business plan assignment combining lecture, tutorial and virtual world learning.
  • Business plan was to support a loan application on next 12 months worth of planning.
  • Induction session [https://www.facebook.com/uqbusiness/videos/988464357871851/] run for people on the VBE prior to first trade session.
  • Student feedback: https://www.youtube.com/watch?v=Y-y6WhIhrLc&feature=youtu.be : overall has gone well, with positive feedback, but not all students liked it – including due to technical problems.

Overall a good presentation and an interesting approach for trying to deliver the learning outcomes.

What future for education? MOOC – Week 2

It has been a little while since I’ve engaged at all with a MOOC.  I continue to sign up for the odd one but having moved into a house which is now proving a ‘money pit‘ my spare time has largely been taken up with cleaning, worrying about money and general panic about the years of work we’re facing.

This has been educational in itself – full building surveys are there for a reason, do not buy houses based purely on character, garden sheds are difficult/impossible to fix, foxes are very similar to dogs, etc etc but I am trying to get back into further personal development (including the recent splurge of posts here).

Anyway, the WFE MOOC seems to have picked up a bit of traction with people I follow online and whilst I largely ignored Week 1, the activities for week 2 are a bit more interesting:

1 – the discussion task

Offer an example of someone who is considered to be intelligent or gifted BUT who has had to be an expert learner. Tell us something about that person (they could be real, someone you know well, or a celebrity or fictional character). Outline why you think they are a “good” learner. THEN choose two posts from the discussion forum (not your own) and post a response to them: why do you think their learner is a good example: what does it tell us about intelligence and learning? Please read our forum posting policies before posting or starting a new thread.

Now I find this task description a little complicated, the need to use BUT and THEN in the way they have kind of highlights that there could/should have at least been use of bullets to better set out the instruction. From one of the staff replies, to someone seeking clarification, there is also something clearly missing in the above description:

“The idea is to consider the learning process of people who are considered to be gifted or intelligent.
There are examples of people who are highly successful who were even at some point in their life considered to have learning or other difficulties, overcoming this by developing expert learner skills
A little reading up on people who you consider to be particularly intelligent or gifted might give you some ideas. (musicians, businesspeople, scientists, nobel prize winners etc)”

There is a clear difference here between identifying a good learner (lets say Napoleon as an example of someone who studied hard at military school and quickly learned on the job afterwards) against someone who has overcome a learning or other difficult by becoming an expert learner (Stephen Hawking type examples here I guess or the business leaders for whom ‘school didn’t work’ only for them to still be a success and find out later that they had severe dyslexia or something similar).

This all highlighting one problem of running a MOOC – that you open yourself up to a world of nitpicking!

2 – the reflection activity

  1. During your own education, how has your “intelligence” been assessed?
  2. How has this affected the educational opportunities you have been given?
  3. What judgments have people made about you that have been affected by an assessment of your “intelligence”?
  4. Do you consider yourself to be a “learner”? why

Personally I would say all animals are learners, in incremental ways we change our behavior continuously from dealing with basic needs, such as sourcing food, to highly technical skill development.  The education system typically assesses our recollection of information (exams) or ability to research, analyse and articulate (essays/vivas).  Recollection can be more complex, for example in Mathematics, but rarely would my formal education have assessed in more detailed ways.  Few opportunities were given for more detailed investigation, coursework in practical subjects at school would have at least combined physical skills with mental activities.  Intelligence can of course be judged in many ways, Howard Gardner etc etc, but as the image in course menu suggested, we revert to ‘clever’, ‘brainy’, ‘smart’ and many negative options too.  Ultimately we will all learn but combinations of our neurology, previous experiences and environment will impact what this means in reality.