Monday, 31 March 2014

NUS-Prime: Technology Enhanced Learning - Three Case Studies, Monday, 5 May, 2014

Technology Enhanced Learning in Health Professions Education: 3 Case Studies

Dear colleagues and fellow educators, 

Thank you for confirming your attendance at the upcoming NUS-PRIME session. I have made a summary of the material I will be presenting, and some additional online resources available for your review before the session. Please have a look at this and either or both email me any comments or questions you might have or post them directly onto the blog.

Yours sincerely, 

Poh Sun

Dr Goh Poh Sun
MB,BS(Melb), FRCR(UK), FAMS(Singapore), MHPE(Maastricht)
Associate Professor and Senior Consultant,
Department of Diagnostic Radiology,
National University Hospital
National University Health System,

email -

LinkedIn -

Slideshare educational presentations and papers -

Summary of presentation - 3 case studies in Technology Enhanced Learning

Opening remarks

Maastricht MHPE Master Thesis: Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency (April, 2012)

Ericsson KA, Krampe RT, Tesch-Romner C. 1993. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:363–406

Use of hyperlinked clinical case repositories in medical education

Use of hyperlinked radiology case repositories in medical education

Digital curation

4C - ID model applied to residency education (management of hyperdense lesions on emergency head CT in patients with suspected acute stroke)

PhD proposal in medical education (deliberate practice and mastery training, ongoing program, from mid 2012 to current)

"Teaching styles that incorporate various forms of technology can facilitate a more interactive learning process for students, providing deeper learning, more motivation and more interest in the subject (e.g. Clark 2002; Schultze-Mosgau et al. 2004; Allen et al. 2008; Gormley et al. 2009). Based on the literature, e-learning and experiential simulation learning are likely better mediums for training physicians if health educators are properly trained in how to incorporate technology into the curriculum. Most studies, however, tend to be based on case studies and are not generalizable."

"Several barriers prevent technology and telemedicine from being a more integrated component of health professional training: (1) cost, (2) efficiency, (3) engaging students and (4) bias against online inferiority. The initial investment for many medical technologies can be as high as $300,000 (De Lorenzo 2005). Consequently, educators are reluctant to integrate new means of training. Aside from cost, another barrier derives from an educators’ reluctance to learn how to operate technological resources. With new technology, class instruction preparation requires far more front end work, especially among those who are unfamiliar with the technology (Lockyer et al. 2006; Boerema et al. 2007). A related obstacle is that teachers of health professionals are trained as health professionals, not as educators (Alur et al. 2002). A third concern with e-learning concerns engaging student participation (Lockyer et al. 2006; Boerema et al. 2007), which is associated with poor implementation and design of the training program. If an e-learning program is properly created, it should be as demanding and as engaging as a traditional classroom setting (Clark 2002; Emami et al. 2009), but that takes considerable efforts from instructors. The idea that fully online education is an inferior learning technique if not used properly is also common in many e-learning studies (e.g. Svirko & Mellanby 2008) and appears to be an underlying attitude generally. Researchers have consequently tried to establish e-learning techniques as valid ways to learn, focusing less attention on what should be taught using e-learning approaches. In short, many educators may dismiss the possibility of using various technologies—including e-learning—in the classroom prior to giving these training methods a chance because of preconceived attitudes they have about technology and education."

above quoting from Mason et. al. 2014. Use of technology and perceptions of its effectiveness in training physicians. Med Teach 36: 333–339


Allen EB, Walls RT, Reilly FD. 2008. Effects of interactive instructional techniques in a web-based peripheral nervous system component for human anatomy. Med Teach 30:40–47

Alur P, Fatima K, Joseph R. 2002. Medical teaching websites: Do they reflect the learning paradigm? Med Teach 24:422–424

Boerema C, Stanley M, Westhorp P. 2007. Educators’ perspective of online course design and delivery. Med Teach 29:758–765

Clark D. 2002. Psychological myths in e-learning. Med Teach 24:598–604

De Lorenzo RA. 2005. How shall we train? Mil Med 170:824–830

Emami H, Aqdasi M, Asousheh A. 2009. Key success factors in e-learning in medical education. J Med Educ 12:81–89

Gormley G, Collins K, Boohan M, Bickle IC, Stevenson M. 2009. Is there a place for e-learning in clinical skills? A survey of undergraduate medical students’ experiences and attitudes. Med Teach 31:e6–e12

Lockyer J, Sargeant J, Curran V, Fleet L. 2006. The transition from face-to-face to online CME facilitation. Med Teach 28:625–630

Schultze-Mosgau S, Thomas Z, Lochner J. 2004. Web-based, virtual course units as a didactic concept for medical teaching. Med Teach 26:336–342

Svirko E, Mellanby J. 2008. Attitudes to e-learning, learning style and achievement in learning neuroanatomy by medical students. Med Teach 30:e219–e227


Three Case Studies:

A. At a regional/international conference setting (Symposium on Technology Enhanced Learning, TELroundtable@APMEC2014 at the 11th APMEC in Singapore)

Educational setting - International health professions education conference

Educational challenge - How to engage audience, promote interaction.

Theory - Provide all key information beforehand for advance review. And platform to view and discuss.

Table outlining educational learning theories and theorists

Most influential theories of learning (UNESCO)

Adult learning theories (abstract, AMEE guide No. 83)

Adult learning theories (pdf, AMEE guide No. 83)

Developing questionnaires for educational research (abstract, AMEE guide No. 87)

Method - Use of email to manage, organise and document discussions amongst 6 panelists (3 from Singapore, 1 Sri Lanka, 1 New Zealand, 1 Australia). And website/blog to present content, platform for discussion both before, during and after session, and document reflections from panelists after the conference. The website with available content was announced at opening ceremony of 11th APMEC. 

Results - Very good attendance at session. 1 comment/question posted 24 hours before session, 1 during session. Majority of time during 90 minute session was spent in Q and A and discussing content and educational issues, rather than presenting information. Spike in review of material during the session, and after session. Each panelist submitted short reflection posted on 11th APMEC symposium website (below).

Discussion and Reflection - Simple technology enhanced panel and audience engagement and interaction, documented our interaction, and provided information about viewership, and level of interest in topic - by number, and type of questions, majority at session, minority online. The majority of participants chose to engage at the session live. Having a symposium website however allowed other conference registrants not at session, and interested local and international educationalists to review the material. Our interaction was documented on the symposium website, and this platform allowed the panelists to review and reflect on the process of creating content, co-ordinating our presentations, our participation in the symposium, and facilitated writing a short reflection about our involvement in the symposium, which could be immediately shared online.

B. At an institutional level (Learning@NUHS project at the National University Hospital, Singapore)

Educational setting - Residency training at an academic medical centre

Educational challenge - To provide consistent, high quality content to facilitate resident training

Theory - Deliberate practice, mastery training, distributed practice, cognitive overload, constructivism, active learning

Method - Systematic collection content from day to day departmental and interdepartmental (multidisciplinary) teaching sessions (eg. case presentations, multidisciplinary rounds), which is edited and indexed, followed by selective hyperlinking, with combination of content which has been authored and created in-house with curated online content

Results - Year 2 of three year pilot project, with 1st year engaging both leadership and departmental educators, as well as working with initial group of volunteers; there is systematic online content being systematically created by one department, for one specific module on topic of emergency neuroimaging interpretation (EMD authoring, with radiology, neurology and neurosurgery collaboration and peer review, and pilot resident content testing), one specialty collecting and curating regular multidisciplinary round content (Ophthalmology; with one initiative in NUHS, and 2nd international collaboration with PGY2 resident in Department of Opthalmology and Visual Sciences, University of British Columbia, Canada), and initial content being uploaded by 8 departments (see below), and several "grassroots" consultative and collaborative educational projects.

Screenshot of Learning@NUHS intranet content repository landing page

Discussion and reflection - Use of an iterative engagement and development model, and project website/blog, facilitated thinking, development of consensus, monitoring progress and documenting development milestones in the project. The project site is also an educational tool, and faculty development resource, for both local and international educators.

and original pilot project website

     Normal chest radiograph (M1, undergraduate)

     Imaging of respiratory disorders (M2, undergraduate)

C. And departmental/individual teacher level (CasedBasedLearning@NUHS project in the Department of Diagnostic Radiology, National University Hospital, Singapore)

Educational setting - Radiology department with 33 residents and fellows

Educational challenge - Providing access to teaching content for residents on shift or leave; documenting teaching for asynchronous review, both before and after each interactive teaching session; providing an opportunity for all residents both onsite and offsite to attempt case based problem solving, and to document this process for feedback and educational purposes

Theory - Deliberate practice (with feedback), mastery training, constructivism

Method - Use of clinical case material with teaching value, collected and curated daily, made available on website/teaching blog (anonymised, un-annotated, with no clinical or other information provided, i.e pure perceptual / image recognition, categorisation and statement diagnosis and single sentence justification exercise). All residents on-site and off-site attempt 10 cases, and email single statement diagnosis followed by single statement justification to instructor first 10 to 15 minutes, followed by face to face interactive discussion, with one resident's notes of sessions, follow-up annotated images, and anonymised resident answers available for review.

Results - Residents have found teaching format engaging and useful; and actively and enthusiastically participate in this process; monthly sessions, over the last 7 months. Shift, post-call residents and those on leave find accessibility of teaching material and record of discussions useful.

Discussion and reflection - Use of SMS notification of session, unknown diagnosis clinical case material on teaching website for deliberate practice followed by feedback in interactive teaching session, and record of material discussed for offline review leverages on simple familiar "technology" that both residents and instructors are familiar with - (SMS), email and a website/teaching blog.

           Mastery training and deliberate practice example: Topic - acute neuroimaging


Additional "reading"

(e)Learning using content from an (e)Repository CAN be assessed and evaluated ... (e)Scholarship (some thoughts) .. and 2 related reflection pieces on MedEdWorld.

(e)Learning; digital repositories and digital scholarship (Slideshare)

     (e)Scholarship or digital scholarship

MOOCs and the Flipped Classroom (Slideshare)

Evaluating teaching - quantity and quality (some methods and examples)

as a case study, compare and contrast digital version of Paulo Coelho's book "Manuscript Found in Accra" with hardcopy print format

“What’s most interesting about his approach is not so much what he says, but what he suggests about how texts should be read. In this iPod shuffle worldview, anything and everything can be read selectively” Silvio Meira at the Digital Book Congress in São Paulo (Brasil)

"According to Meira, Paulo Coelho’s latest book exemplifies how we read and interact, and it is a great example of what texts will be like in the future."

eLearning in clinical teaching (London deanery)

Current and future trends in eLearning (Imperial College, London)

Short eLearning modules (Leeds Institute of Medical Education)

Social media bootcamp (Canadian Healthcare Education Commons)

(e)Learning (reflection and some curated online resources)

Some thoughts on Mastery Training

My Day: Japanese sushi chef Yosuke Imada (BBC) vs Meet Singapore's coffee king Adrin Loi (BBC)

Knowledge process analysis: Framework and Experience

Measurement of knowledge, process and practice in knowledge management

A Model of Organizational Knowledge Management Maturity based on People, Process, and Technology

Constructivism in practice: The case for meaning-making in the virtual world

Knowledge translation: Introduction to models, strategies and measures

Images of knowledge process practice model or

Adaptability is more important than talent


Anders Ericsson

Deliberate practice

The experience trap

The making of an expert

Deep expertise can hinder performance

The power of synthesis

Value of human input (vs Artificial Intelligence)

The new era of talent

A new age of distruption

How to become world-class at anything

The first 20 hours

Sunday, 30 March 2014


Digital Scholarship and Engagement - Indicators, Metrics, Value and Impact
Poh-Sun Goh 

eLearning or TEL(Technology Enhanced Learning) is increasingly being integrated into medical education and training, from undergraduate, through postgraduate to continuing education and lifelong learning settings, with increasing emphasis on blended and mobile learning, in the workplace and just-in-time settings. TEL with utilisation of digital content and engagement provides visibility of our teaching and training efforts as educators, showing ‘what we teach with, and assess on’ (Goh, 2016). TEL approaches can also be used to provide visibility and metrics of student engagement, as well as demonstrate intermediate and final outcomes of student learning (Goh & Sandars, 2016; Goh, 2017). TEL is increasing transforming medical education, going beyond substitution, augmentation and modification of learning - the ‘SAMR model’ (Puentedura, 2013). Reflective educators, and educational scholars have always sought to evaluate and assess the value and impact of their teaching and training efforts, and Digital Scholarship, with attention paid to data, indicators and metrics of engagement and output, can facilitate these efforts. Use of 'free', low cost, off the shelf, easy to use and accessible digital tools and platforms, combined with curated and created digital content repositories, by faculty who are 'digitally literate' and professional, facilitates adoption and scaling up of our scholarly efforts (Goh, 2018). This has direct benefits for faculty members during the academic appointment, appraisal and promotion process; for an academic community by making scholarly activities easily accessible, and for an institution by making academic and scholarly activities by faculty members both easily accessible and visible.

More on expanded blogpost link below


Goh, P.S. eLearning in Medical Education - Costs and Value Add. The Asia Pacific Scholar (TAPS). Published online: 2 May, TAPS 2018, 3(2), 58-60. DOI:

Goh, P.S. Learning Analytics in Medical Education. MedEdPublish. 2017 Apr; 6(2), Paper No:5. Epub 2017 Apr 4.

Goh, P.S., Sandars, J. An innovative approach to digitally flip the classroom by using an online "graffiti wall" with a blog. Med Teach. 2016 Aug;38(8):858. Epub 2016 Jul 14.

Goh, P.S. Using a blog as an integrated eLearning tool and platform. Med Teach. 2016 Jun;38(6):628-9. Epub 2015 Nov 11.

Puentedura, R. R. (2013, May 29). SAMR: Moving from enhancement to transformation [Web log post].
Retrieved from

click on tile above


Gusic, Academic Medicine, 2014

Digital scholarship in tenure, promotion and review process (Deborah Lines Anderson, M.E. Sharpe, 2004)

Wednesday, 26 March 2014



The challenge when first thinking about, and then creating an ePoster is very often a mental one. When thinking about ePosters, consider the analogy of a digital tablet accessing an online website, or eBook/ePublication.


The challenge when first thinking about, and then creating an ePoster is very often a mental one. Wrestling with the concept of an ePoster. Understanding and developing insight into what an ePoster can do. 

Perhaps the analogy of a digital tablet accessing a webpage might help.

The digital tablet embodies the physical qualities and tangibility of an ePoster. The possibility of touch to promote ease of interaction with the digital content being viewed. And the potential to showcase and discuss the material on the tablet in a professional group setting, for example in a conference like AMEE.

A well designed website (or eBook/ePublication) being viewed on a digital tablet embodies the qualities of a well designed ePoster. With deliberate, and thoughtful use of interactive graphics and multimedia, as well as links to additional online resources via hyperlinking and embedded multimedia deepening engagement with the viewer. 

A curated thematic collection of ePosters not only provides an opportunity for onsite individual and group review of a topic or subject; but broadens the impact and reach of the content being shared; when the ePosters are made available online; and further discussion takes place using various moderated, and unmoderated online discussion forums. 

When thinking about ePosters, consider the analogy of a digital tablet accessing an online website, or eBook/ePublication.

 Reflection on (e)Posters (MedEdWorld)

Friday, 21 March 2014

Evolution of EduTech, from Past, through Present, to Possible Futures ... While always remembering the Pedagogy; good educational practice, theory and evidence

Lets take a journey through the evolution of EduTech ... by reviewing the past ... to contextualize the present ... and then examine the/possible futures.

While not forgetting that the delegates will be a mixed category of enthusiastic novices ... and intermediate practitioners with a minority of advanced practitioners.

To help us visualize a "program", have a look at some of the info-graphics on this topic which are freely available are a starting point (especially on topic future of ed tech)

Perform a

Google (image) search for

"infographic history of educational technology"

followed by

"infographic future of educational technology"

For last link, click EDUCATION TAB

Tuesday, 18 March 2014

Evaluating Teaching - Quantity and Quality (Methods and Examples) + comparison with traditional research metrics

(Illustration and documentation of the presentation material at this AMEE 2014 TeL symposium, some pictures of the event, link to archived video of symposium (which includes over an hour of interactive Q and A with the panel), audience engagement statistics (quantitative), as well as audience comments (qualitative). Further elaboration on the content presented in this symposium has been submitted as article to Medical Teacher)

(Illustration and documentation of the presentation material for this TeL interactive roundtable discussion with the audience at 11th APMEC 2014, including panelists slides, some pictures of the event, online audience viewership (quantitative), audience questions posed, as well as post event reflective comments by each panelist. This is part of a process to make teaching not only scholarly, but a piece of scholarship, with public dissemination of the teaching material, a continuing attempt to measure the impact of teaching, documented self reflection on teaching, and invitations for peer review through the website, and other platforms through symposia and conference presentations, and peer reviewed publications)

eScholarship which also highlights article

(Feedback above from Professor Ann King, Hong Kong)

original above, with revised table below (Apr 27, 2014)


"one of 20 to 30 people in the world with the expert knowledge to judge whether the articles cited the relevant literature, represented it accurately, addressed important issues in the field, and made an original contribution to knowledge" (Hugh Gusterson on the academic peer review process, see website above)