Does edtech really work?. piffle & the debate about what works & doesn't in edtech.

I hear and read constantly that edtech doesn’t work. I agree, but for different reasons.

First, let’s set the scene. My knowledge is about edtech in the UK; specifically England. In this market the government has spent (rather than invested) billions of pounds, most of which has been completely wasted. My metric for failure is that in my estimation less than 20% of teachers are using edtech in any meaningful way educationally.

There are plenty of reasons for this, but what miffs me in this debate are stories like:

1. Computers ‘do not improve‘ pupil results, says OECD
2. Computers in class ‘a scandalous waste’: Sydney Grammar Head

First, the OECD. I respect the OECD’s education Czar, Andreas Schleicher, but I’m dubious about many of the OECD’s claims, particularly those derived from their PISA international tests:

• Press releases: more media clickbait than serious analysis from an organisation whose aim is to ‘promote policies that will improve the economic and social well-being of people around the world’
• Useless data: The OECD admit the UK 2000 and 2003 PISA tests ‘did not reach the standard level of test responses deemed necessary’. This didn’t stop Tony Blair using these as examples of his administration’s success in their relentless focus on, ‘education, education and education’. When you add bad data to the politicisation of education you get what the American military call FUBAR
• Even when the tests are statistically valid not every country participates in every part, e.g. in 2012 the UK did only one of three tests (Problem Solving) whereas Australia did all three
• Digital disconnect: Schleicher claimed, “Those students who use tablets and computers very often tend to do worse than those who use them moderately”. Yet Sweden, Denmark and Australia with the highest internet usage in-school, also had lowest rates of ‘digitally adrift’ students (9.3%, 10.7% and 7.7% respectively); whereas China and Japan with low in-school use had high rates of (20%+ and 16.2%)
• Schleicher also said a more ‘effective approach’ to edtech was digital textbooks, because:
  a) most digital textbooks are simply low tech (often PDF versions) of paper texts, that fit better with established teaching practice; b) the waves of  ‘edtech ‘revolutions’ promised by politicians focused too greatly on hardware and software and far too little on professional development and teacher training

Second, the ‘ban on computers’ by John Vallance, Principal of Sydney Grammar, is misleading. It conflates huge waste in an Australian government initiative called the Digital Education Revolution (DER) with why his school uses less edtech. The DER was the brainchild of Julia Gillard, then the Federal Minister for Education (eventually Prime Minister). A key part of DER was the give away of laptops to all government school students in years 9-12. This was one of two large, ill-conceived educational ‘revolutions’ (inc. the A$16bn Building the Education Revolution) that were politically and ideologically driven. Both wasted billions directly, but the DER had a more direct impact on learning in schools because it shifted the bulk of the Total Cost of Ownership for DER (on-going hardware and software costs) to state governments who weren’t given additional funds to cover them.

Almost 38% of students in Australia attend private schools, but there are only in a handful (like Sydney Grammar) who can charge parents A$30k+. These elite schools are locked in a highly competitive ‘war’ to recruit and retain students, which is fought on three principal fronts; exam results, facilities and marketing. The facilities ‘nuclear arms race’ is best exemplified by Scots College who have just dismissed their governing body because they were focusing more on business than education yet seem to be happy to spend $A100k for a “hypoxic simulated altitude-training environment”

On the marketing front, most elite schools core messaging reflects their clients’ desire for a traditional educational experience. Vallance’s statement, “we found creative writing tasks were more successful with handwritten submissions, rather than using a keyboard” is an almost perfect fit between customer aspirations and brand messaging. However, it conveniently ignores two related issues. First, students at elite schools already have almost unrestricted digital access outside school. Secondly, Vallance and the management of most elite schools are acutely aware of the clear link between legible handwriting and outcomes in high-stakes summative assessments (i.e. exam results).

So what’s really wrong and why I think 95% of edtech products don’t work:

• a lack of input from researchers, educators and students in how products are imagined, created and sold
• untested and unverifiable claims of efficacy and a lack of transparent/open data. Claims that edtech products work are mostly based on tightly-held proprietary data, on the false assumption this is ‘investor gold dust’. Pap! If a vendor doesn’t have an open data API to some of their core data (anonymised) to allow external researchers to test their claims, then schools shouldn’t be able to spend public funds buying them
• abysmal procurement. England’s shift to decentralised control (specifically the Academies programme) has seen the average school and even those who are part of Multi-Academy Trusts (MATs), buying edtech with the professionalism your local football club applies to buying sausages for their end of season BBQ. Edtech developers, investors, schools, MATs and the government need to start using tools like John Robert’s Edtech Procurement Matrix, to help identify products that are more likely to work both for teachers, students and schools (see image below)
• the lack of differentiation between edtech services for administration and teaching. The former has arguably so far had a far greater impact than the latter and this needs to change i.e. we really don’t need another VLE/LMS
• disjointed policy initiatives. For example, England’s e-Learning Credit (ELC) initiative helped build capacity and demand in the edtech sector. Then came BBCjam offering to give away ‘free’ software covering 50% of the curriculum. This spooked investors as it would have bankrupted many of the companies that ELCs helped create
• bad bandwidth. Most of the school teachers and students I speak with say they rely on the bandwidth from their mobile phone contract to access edu products and services from their school as much as on access from their school network
• compassion fatigue – teachers and educational administrators are exhausted by the seemingly never ending waves of ‘educational revolutions’ mostly inspired by political ideology and not sound pedagogy.

These few observations are based on my 20 years in the business of edtech. So, it is very limited in scope, had an absence of any deep understanding of pedagogy, is full of confirmation bias and every other weakness, but so is PISA and the attendant media storm about the failure of edtech.