Reflecting on evidence based education during this masters journey.

Working on my reflection chapter for my masters but there isn’t space to put all the thoughts I wanted to so I wanted to explore some of them further here. The added bonus being my blog is an academic jargon free space (I hope although I might have mentioned hegemony once…) and I find it much easier to organise my thoughts in this fashion than backing up my points with pesky academic evidence! Ahem!).

Evidence based education is a big buzzword at the minute but it is a notion I have had to really grapple with during the course of this masters.

My starting point embarking on this Masters was as a “scientist” – I have A-levels in Biology and Chemistry, an honours degree in Zoology and a PGCE in Science (Biology), so I have the qualifications to justify this as an “identity”. However somehow I have never been challenged to question the theories underpinning the acquisition and advancement of scientific knowledge until this masters degree. That is not to say I took scientific knowledge at face value, I know how to critique scientific methods, but I had never been exposed to social science research methodologies before such as ethnography and at first to be quite honest I didn’t quite know what to make of them (*scratches head*).

Being a “scientist” I thought I needed randomised control trials and meta analyses to justify an evidence based approach in education as well as the sciences. However I am also aware of this approach not necessarily being useful for Sex & Relationships Education. For example the SHARE project (an RCT of Sex and Relationships Education) which found no impact on age of first intercourse, levels of sexual activity, or condom or contraceptive use, between enhanced SRE program (SHARE) and standard programs. However in comparison with conventional sex education, SHARE was evaluated more highly by both pupils and teachers, it increased practical sexual health knowledge, and it slightly improved the quality of sexual relationships, primarily through reduced regret. Some have interpreted SHARE results to mean that SRE doesn’t work and therefore should not be taught. Not an interpretation I share obviously and my rants on about value added and behavior change in SRE are beyond the scope of the post!) but I’m very wary of RCT’s for evidence based education, as they are often narrow in scope and conclusions risk dismissing some incredibly positive interventions just because they didn’t get the “right” results in an RCT. I’ve come to realise that adopting a “positivist” position for Sex & Relationships Education is flawed because humans operate in open conditions, we cannot control the variables, and if our intervention is still a positive one (like SHARE was), is it fair to deprive a control group of it?

I also became worried about how evidence is  interpreted both in the formation of conclusions and developing evidence based policy. For example interpretations of the quote “homosexuality exists in 1500 species, homophobia exists in just one”.   The Zoologist in me has to point out whilst we can categorise animals and humans by their behaviours to be “homosexual” or “homophobic” these categorisations maybe subjective and may differ between researchers.  Some researchers have stated that the observed “homosexual behaviour” in certain animals is a way of asserting sexual dominance. i.e. “ same sex rape or violence” whereas others have stated “animals solve conflict by same gender sex”, these interpretations caused me to ponder if the researchers own positionings around their own feelings around homosexuality and homophobia affect their interpretation of results.

I also have concerns about how evidence is interpreted  to adopt “evidence based policy”. For example hypothetically the evidence might state “adopting a zero tolerance approach to homophobic language is the most effective in reducing homophobia”. This might be interpreted as adoption such a zero tolerance policy immediately without engaging in a dialogue with the school community about why homophobic language is unacceptable.  This could then set up confrontations with stakeholders on their core values and beliefs, where confrontation creates a block, anger, resentment and no possibility for understanding and positive change. Therefore the zero tolerance strategy fails because people are still homophobic just silenced about it. In my experience the dialogue before adopting a zero tolerance policy is the most crucial aspect of the work.  Getting young people to understand why using gay as a pejorative is unacceptable is an incredibly powerful tool in fighting homophobia. At my own school we found once the young people understood they were policing their own language and challenging each other.   Therefore nuances such as “engage in dialogue THEN adopt a zero tolerance policy”, could be missed by sumarising the evidence into “best practice evidence points”.

Also I am not convinced by researcher “ethics”. I think the desire to get work published, finished, or shared can mean sometimes corners are cut, data is manipulated, evidence is tweaked and no-body admits to it. I know the data in my undergraduate dissertation on waterflea parasites (where I had to dissect over 1000 waterflea guts and examine them under the microscope) was probably not erm as academically robust as it should have been and I don’t think I would be a lone example. Ben Goldacre is clear on that in his Bad Science book. So why should we trust evidence when the people creating and interpreting it often have dubious morals!

I think I have come to the conclusion that objectivity doesn’t truly exist in science or social science, particularly when dealing with humans. Whilst initially I was incredibly skeptical about performance ethnography, or other less than “scientific” methods used in the social sciences, I have come to realise that actually some of the most interesting rich evidence lies in the lived experience of individuals, that defies distillation into a meta-analysis or a plot point on a graph.

*takes off my scientist hat*

During my research into homophobia in schools I have tried to synthesise a very diverse dataset to capture the emergent themes that schools and teachers need to consider when challenging homophobia. What has been most interesting about this process for me is that:

a) the diverse evidence is actually synthesisable (although at times the process has left me weeping!)- the themes are replicated across the dataset, admittedly through my subjective interpretive lens but there are commanlities therefore the weight of evidence leans towards some successful evidence based strategies schools and teachers can use (with all the above caveats about evidence based education!)

b) The three years I spent challenging homophobia in my previous school, without a shred of evidence to go on but a gut feeling of strategies to try and develop, was not wasted! Virtually everything I did and experienced within that three years has emerged as a theme from my research! (admittedly this could entirely be because my research is a subjective piece of rubbish, and I sort of wish I had chosen a topic that I wasn’t quite so deeply engaged with – ahem!) but even if no-one else believes me, my lived experience as a practitioner has been validated post hoc by the evidence! Which cheers me greatly! Practitioners RULE!

In conclusion having been quite pro “evidence based education”, I am now sat very firmly on the fence about “evidence based education” as I can’t see how it can ever be objective and free from bias. I can see it provides benefits for justifying positions (after all I want to present this research as an “evidence base” for challenging homophobia in schools) but I think potentially it is too subject to biased interpretation that could lead to unintentional negative effects, and I think we need to think much more carefully about our notions of evidence based policy and practice before we try and adopt such strategies as standard.

What do you think?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s