Know Thy Digital Impact – A Reflection on Digital Research

flickr photo shared by mrkrndvs under a Creative Commons ( BY-SA ) license

I received a request today for any research that could be used to show how ICT is improving student outcomes? To me, this is such a complicated problem. Firstly, what is a ‘student outcome’? Secondly, technology is a tool used to support learning, not something that can necessarily be measured in and of itself. The question then to be considered is what outcomes do you measure in order to ascertain the impact of technology? Here then is my list of possibilities:

• Engagement: This is often the first place that people go to. Maybe this focuses on whether they associate with learning or participate. However, as David Price highlights, measuring it is not always obvious. That is, it is not always seen, not simply about test scores or having fun. Seymour Papert touches on this when he suggests that learning should involve ‘hard fun‘, where learning is difficult, rather than easy.
• School Connectedness: This is often a barometer used in surveys like Attitudes to School. However, although such measurement is useful when it comes to well-being, it is not so obvious when it comes to technology.
• Collaboration and Problem Solving: This is popular when it comes to 21st century learning and has received considerable focus, particularly from the ATC21s group. The challenge is often capturing the different facets of cooperative learning and where technology sits in this.
• Learning Agency: Like engagement and connectedness, what agency means can be different for different people. Claire Amos has provided a detailed guide for introducing agency. In first place, she argues for one-to-one access, although how you differentiate this from the rest of the list I am not sure.
• Creativity: Sir Ken Robinson describes this as “putting your imagination to work”. However, like collaboration and problem solving, this can be hard to pin down, especially in relation to tools and technology.
• Digital Citizenship: Often people argue we should use technology as a model for life. An example of this is provided by Alec Couros and Katia hildebrandt in their digital citizenship curriculum for the Saskatchewan Schools District. Much of this is also included within the new Digital Technologies curriculum, with more focus on how technology works. Although tools like David White and Alison Le Cornu’s mapping of the web from in regards to residents and visitors provide a useful point of reflection, they do not necessarily demonstrate specific learning growth.

In the end though the problem that still exists beyond what to measure is the questions of how is the technology actually used and why. A more fruitful approach is to enter develop a holistic action research project incorporating the ioi process. Instead, people commit themselves to frameworks like SAMR to guide them. In addition to this, the reality is that a school further on the road towards normalisation is going to have more success with technology, than one at the beginning of its journey. Importantly, Mal Lee points out that,

Until school principals are of a mind to transform ‘their’ school the staff and the school’s community have little likelihood of changing the status quo.

The problem though as Paul Tozer points out is that at present, with the focus on NAPLAN and VCE, moving into the digital realm is not always a priority.

For those interested, here is a list of research, presentations and publications shared online:

• Australian Teaching and Learning Toolkit – Digital Technology – This analysis provides a summary of the effectiveness, as well as an estimated cost per student.
• Education Technology or Technology Education? Can computers make an impact in schools? – Oliver Quinlan responds to the question of technology in light of the OECD report.
• Students with laptops did better in HSC science – This post by Simon Crook,Manjula Sharma and Rachel Wilson unpacks a research project into the use of technology in senior maths and science.
• Using wikis to promote quality learning in teacher training – Steve and Dawn Wheeler report their findings on the use of wikis to add authenticity to the writing experience.
• Seymour Papert at Bates College (2000) – A great introduction to Papert’s work, he covers many of the questions raised about the use of technology even today.
• Just what are we preparing our students for? – In this presentation, Jenny Luca outlines a vision for the future.
• Classroom of the Future – In this presentation, Anthony Speranza ponders the future of learning. This includes the place of technology in regards to learning.
• What is technology’s impact? – In this detailed post, José Picardo unpacks what the research says about the impact of technology.
• Students, Computers and Learning – An OECD report into the impact of computers in the classroom.
• Technology Integration Research – An annotated bibliography collated by Edutopia.
• Brightbytes – A platform that supports evidence-based action.
• What’s Your Problem? – Steve Brophy provides a review of How to Measure Everything, a guide to not only measuring outcomes, but working out what you are trying to measure in the first place.
• Return on Instruction (ROI) – Eric Sheninger provides a perspective from leadership on measuring the impact of technology.
• Action Mapping – Cathy Moore uses a process of questions to help analyse a performance problem and design solutions.

As always, comments, links and suggestions welcome.

If you enjoy what you read here, feel free to sign up for my monthly newsletter to catch up on all things learning, edtech and storytelling.

Know Thy Digital Impact – A Reflection on Digital Research by Aaron Davis is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.