Distraction or development_ What are the potential effects ...

Distraction or development? What are the potential effects of Google Translate in EAP assessment?
Klaus Mundt - The University of NottinghamMike Groves - The University of Birmingham
Outline
Where we were in 2014 and where we are nowInterviews with academicsImplications for assessment
In 2014
Error analysis of Chinese and Malay- Pre UGGT can write at a level of 6-6.5 (error count)It was a clumsy tool, but could work with human post editingGrammar only- not beyond the sentenceAbsolute or relative quality
Comparison - errors highlighted, student writing
So what?
The growth and spread of this technology is undeniableStudents are likely to use technology that facilitates their studiesIt could be argued it is becoming “a mundane …digital practise” (Henderson et al, 2017:1574)Does not necessarily mean tech literacy and using tech responsibly and purposefully (Davies, 2011)What constitutes responsible & purposeful use in HE?i.e. Is student use of GT acceptable, controllable, misconduct, helpful, to be encouraged/discouraged, to be taught?
So we asked some academics
Not directly connected to EAPWe felt we should check how the wider academic community/the university perceives GT use so that EAP does not have to ‘go it alone’.10 participants2 UK universitiesLecturers, senior lecturers, professors, policy makers, quality assurance officers - across the disciplines1:1 semi-structured interviewsQuestions about GT as reading & writing tool and the acceptability & desirability of its use
Themes and their relevance to EAP assessment
PolicyAcademic Integrity/WritingThe wider academic community/EmployabilityTransition
Policy
Currently no university policy exists regarding student use of GT or similar technologies.
Academic Integrity/Writing
Where in our codes of practise does it saythat students have to write all their work in English?Senior lecturer, Biology
... the sort of classic thing in academic misconduct, we’ve got suspicions offalse authorship, which this clearlyisn’t…Associate Professor, Social Sciences
Well, I think this isa minefield.I think institutions have got a bit ofa nettle to grasphere […] you are identifyinga very significant problem.Professor, Social Sciences
...they would be submitting something that is not their own workin the sense that the English words that had been selected would not have been their own creation.Academic Services
Control
I think at the moment I’m much more comfortable with the idea that the German student is using GT for checking various things, but not for a wholesale translation of a finished product.How you police that, I havenoidea.Professor, Social Sciences
Employability
... if we think of what message we give to future employers ...We are saying that these students are capable of producing English text, understanding and producing English text, for themselves.And if they were using GT, that isn’t actually what would be happening.Now,ifthe use of GT became so universalthat an employer wouldn’t really have that expectation, … I think I would be much more relaxed about it.Academic Services
There are a whole range of skills [...] you expect from a […] Graduate and I believe thatthe ability to hold a conversation in English and to write a reasoned report or essay in English are inherent to that.If the end employer doesn’t care about the ability to formulate something in English,maybe the market will decide what a degree is.Senior Lecturer, Medicine
…it’s not just what’s good for the university, its precious ideas of its own standards.What does the economy need? What do employers need?… We’ve got to look at what the external recipients of students, actually need.Professor, Social Sciences
The wider academic community
When they are at some sort of conference, or when they are writing anything that is meant for publication,they would not have the benefit of a translation machine.Lecturer, Humanities
Actually I thinkfor science subjects it should be easier if people can do that. In the old days in China and Japan they had very good scientists - they didn’t have to do that in English…The language issue can prevent these kind of discoveries from being spread all over the world[…] But in that case certain translation can help I think.Senior Lecturer Biology
Transition
... trying to get students acriticalunderstanding tosee it as a scaffoldand to understand that you kind ofgrow out of scaffoldsand thatscaffolding should be gradually withdrawn.Associate Professor, Social Sciences
So when we talk aboutearly years studentswhere we’re asking them to demonstrate understanding and knowledge -yes…later years three and four I don’t think it’s as appropriate.Lecturer, English
No,I don't think it's a problem particularly at the start of a degree[…] At the exiting of a degree they should be able to deal with English as a native English speaker can ...Senior Lecturer, Medicine
...if they are using software […]it’s possible that their English language competency willdecline[…]Although the translation software may be very good – again,it will still depend on the quality of the information that’s gone in it.Professor, Social Sciences
Themes and their relevance to EAP assessment
PolicyThere is none. We will need one.ControlThere is none.Academic Integrity/WritingSeveral factors at play. One concern seems to be the status of English and the university brand. How much does that factor into our assessment?The wider academic community/EmployabilityConcerns regarding student ability to communicate beyond university studiesEAP as ‘first port of call’ and continuous support can offer very useful guidance hereTransitionSee above + plain old scaffolding leading towards independence?In summaryAcademics are not generally opposed to GT, but also not ready for it to take over
So, what about GT in EAP?
We can ignore this – but how will we handle student GT use?Arbitrary judgement of a course coordinator?Passing the buck ‘upwards’?Is this the professionally responsible approach?We can engage with it in a transparent mannerActively participate in university policy making.Be consistent and fair in what we do.
So, what about GT in EAP assessment?
If 1, continue as alwaysIf 2, adapt assessment of writing abilitiesFor non-controlled work:‘a focus ondiscourse […]  embedded in social practices, disciplinary epistemologies'’ (Hyland 2018: 390)And remember the GIGO principle...Assess language proficiency in controlled environmentA re-emergence of the timed essay?Editing as assessment opportunity?Commentary on text asassessmentopportunity?Guide students in its use as a transitional tool & learning facilitator not a substitute.
Arising Issues
Our data seems to suggest a concern on the part of the university about external stakeholder expectations regarding graduate English proficiency.Does that mean that university entry requirements ≠ exit expectations?Are the students made aware of that when they enroll?What does that mean for EAP assessment & exit feedback?With English as academic Lingua Franca beyond university, would the use of this technology as supportive mechanism fall under Academic Literacies?If so, should EAP teach it and assess its use?
On a more global scale
English is a shibboleth into the Academic CommunityCan GT break this down?What about global minority languages- e.g. French vs Lao
Distraction or development?Or an opportunity?
Distraction?No- this is here to stayDevelopment?Academics not ready for wholesale adoptionImportance of refocusing our assessment
Thank you for listening
Any questions?
References
Davies, R.S. (2011) ‘Understanding Technology Literacy: A Framework for Evaluating Educational Technology Integration’ TECH TRENDS, 55: 45.https://doi.org/10.1007/s11528-011-0527-3[05/05/2019]Groves, M., Mundt, K. (2015) ‘Friend or Foe? Google Translate in Language for Academic Purposes‘,English for Specific Purposes.112-121Henderson, M., Selwyn, N., Aston, R. (2017) 'What works and why? Student perceptions of ‘useful’ digital technology in university teaching and learning',Studies in Higher Education, 42:8, 1567-1579, DOI: 10.1080/03075079.2015.1007946Hyland, K (2018) ‘Sympathy for the devil: A defence of EAP”Language Teaching.Volume 51 Issue 3 pp. 383-399 DOIhttps://doi.org/10.1017/S0261444818000101Swales, J.M., Feak, C.B. (2004) Academic Writing for Graduate Students. 2nd ed. Ann Arbor: University of Michigan PressWong, S. (2016) ‘Google Translate AI invents its own language to translate with.’New Scientist[online]30/11/2016https://www.newscientist.com/article/2114748-google-translate-ai-invents-its-own-language-to-translate-with/[7/10/2017]Wu, Y., Schuster, M. Chen, Z., Le, Q.V.,Norouzi, M.,Macherey, W.,Krikun, M., Cao, Y., Gao, Q.,Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, Ł., Gouws, S., Kato, Y., Kudo, T.,Kazawa, H., Stevens, K., Kurian, G., Patil, N., Wang,W.,Young, C., Smith, J., Riesa, J., Rudnick, A.,Vinyals, O.,Corrado, G., Hughes, M., Dean, J. (2016) ‘Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation.’eprintarXiv:1609.08144<https://arxiv.org/abs/1609.08144v2>[7/10/2017]