This file is http://www.cs.bham.ac.uk/~axs/misc/rae-comments.txt From Aaron Sloman Mon Dec 2 03:15:45 GMT 2002 To: Vanessa Conte Subject: comments from a retired but full time researching professor (I apologise for not hitting the deadline -- yet another consequence of the current funding regime that means we constantly have to give high priority to all opportunities to prepare and submit grant proposals.) I am submitting this as a plain text message which is very easy to read online. If you'd prefer a prettily formatted PDF file, please let me know. A. Whose views am I presenting? Mine: Aaron Sloman Professor of Artificial Intelligence and Cognitive Science http://www.cs.bham.ac.uk/~axs/ School of Computer Science, The University of Birmingham, B15 2TT, UK EMAIL A.Sloman@cs.bham.ac.uk Phone: 0121-414-4775 Home: 0121-472-8581 (I tend to work at home in the morning if I can) I am basically presenting my own views, but these have been informed by much discussion with colleagues in this and other institutions and by many years of experience in our university system. B. Relevant experience and qualifications: I have been an academic in the UK since I came to Oxford as a Rhodes Scholar in 1957 from Southern Africa. Since 1962 I have worked in the universities of Hull, Sussex (where I helped to found the interdisciplinary School of Cognitive and Computing Sciences), Edinburgh (visiting researcher for a year) and Birmingham, in addition to visiting many other universities in several countries, over the years. I have also worked in several disciplines, starting in mathematics and physics (first degree in Cape Town, 1956), then moving to philosophy (D.Phil, Oxford 1962 and initial lectureships) then transferring to AI and Cognitive science (from about 1971). I have collaborated with psychologists, linguists, philosophers, computer scientists and several commercial organisations. At Sussex I managed the development of a software package that eventually led to the formation of a company which won an award for sales of over $5,000,000 between about 1983 and 1991. I was a member of the 1989 RAE panel for Computer Science (which included AI). I was also a member of the recent QAA benchmarking panel for computer science (and was appalled at what I saw and heard). When I came to Birmingham in 1991 my department had a grade 2 RAE rating. Each year thereafter it moved up a grade and now we have an international reputation as a centre for research and teaching. [Note: that was a slip: 'Each year' should have been 'In each assessment'] I can therefore claim to have played a major role in building up two different organisations from almost nothing to international status: COGS at Sussex and CS and AI at Birmingham. In theory I am retired (aged 66). In practice I am still working (mostly unpaid) full time, including supervising several research students and a research fellow, and helping with management in my department. ======================================================================= C. My comments on the state of UK universities and the role of the RAE. I have one main comment on the RAE and two subsidiary comments. C.1. The combination of low funding for teaching (i.e. poor staff-student ratios) and RAE-induced pressure to produce publications and get grants, is doing incalculable damage to the long term research capability of UK universities. That is because the shortage of time for wide reading and attending seminars in other departments, and the intense pressure to publish and gain national or international visibility as soon as possible, are together producing a generation of very narrowly focused researchers. In particular, the requirement for successful publications and grant proposals causes even PhD students who, in my day, could read and discuss in a wide range of academic areas, to focus far too narrowly. Things are made worse by the pressure on PhD students to help with teaching, partly because academic staff cannot cope otherwise and partly because PhD students need the extra money in order to survive. When they have finished their PhDs they do not have a chance to compensate for this narrowness, because the pressure to publish along with very high teaching, examining and admin loads (which in theory probationers are not supposed to have) rules out the possibility of finding time to branch out and extend their education e.g. by going to seminars in other departments and reading outside their field. [Noted added after sending: I should have mentioned also the pressure to get grants.] These people are therefore not only producing ever more narrowly focused research -- their teaching must suffer also from a lack of breadth and vision, and thus the next generation of students will probably be even more blinkered. This is so very sad. But I guess politicians and administrators will point at all the publications as proof that the policy is succeeding brilliantly! Has anyone looked at the growth of narrowly focused journals and conferences in recent years? There are many international groups of mutual back-patters. C.2. The RAE has encouraged senior managers in universities, under threat of reduced income resulting from low grades and low status in newspaper league tables, to behave in ways that are detrimental to the major functions of universities. I give two examples: distortion of the staff appointment process and unjustified harrassment and persecution of non-research active staff, or people who are excellent teachers and colleagues but mediocre researchers. (a) Distorted appointments procedures: In appointing new academic staff it is now much safer and wiser to employ a relatively dull 39 year old with a string of publications in internationally known journals than to employ an extremely bright and creative 28 year old whose PhD was quite outstanding, but who has not yet produced any publications, and may not do so for some time because he/she is still developing ideas (perhaps like Andrew Wiles). A possible (though clumsy and unreliable) way round this would be to introduce a new PhD rating to be given by external examiners e.g. 'outstanding researcher'. Any person with that PhD rating could then have five years of automatic assessment for RAE purposes as appropriate for a grade 5 department. All other new PhDs could automatically have grade 4 status for five years. After the first 5 years different research assessment criteria could be used. However, this mechanism would have to be conditional on the posts not being temporary, in order to prevent the sorts of abuses that many departments have adopted in order to improve their RAE ratings. This mechanism would also reduce the immediate pressure to publish which can get in the way of young researchers extending the breadth of their knowledge, as mentioned above. (b) Many departments are being encouraged to get rid of excellent colleagues who make a major contribution to teaching and admin. They are now 'unwanted' because they don't help to achieve the goal of a 5A or 5*A rating. E.g. this sort of management policy is being pursued by senior management in my own university (resisted by my department and others). Given the wide range of obligations of universities, any research assessment process that heavily penalises universities that take their teaching and "outreach" obligations very seriously is unethical and certainly not in the long term interests of the nation. A truly excellent department with a good balance of teaching, research and management skills distributed over all staff could have 80% (or in some cases less) of staff who are research-active, while the non-research active do a higher share of teaching and administration. Such harm done by RAE procedures could be reduced by announcing only the grade allocated to a department and the absolute number of staff assessed, without using the letter grades. (Whoever introduced the letter grades must have been seriously lacking in understanding of how universities work.) Superficially the RAE process has done some good. On balance I and many others think it has been very harmful including making it *very* difficult to improve low-rated departments, and wasting huge amounts of time in producing the reports. However, the disastrous decision of the Thatcher government to turn all polytechnics into universities made it necessary to find some excuse not to distribute research funding too thinly, and that seems to have been the *main* use of the RAE. An alternative would be to give special rewards to institutions that recognize the need to attract and give an excellent education to students who are not of the highest academic calibre and who will not be future researchers and university teachers. We badly need institutions doing what many polytechnics used to do very well. Attempting to use the RAE to recreate them by depriving them of research funding is a daft strategy. [end] [I decided not to send the following, as the message was too long.] D. Concluding remarks. I have seen the quality of life for academics and the suitability of the UK academic environment for first class achievement in teaching and research deteriorate enormously over the last 40 years since my first lectureship. In particular, staff-student ratios make it impossible to give students the kinds of individual feedback on essays and projects that are essential for the development of very bright but unformed minds. A consequence must inevitably be a serious reduction in the stream of very well educated highly intelligent young researchers, with razor-sharp minds. Even the less well able students will no longer be going out with the benefit of deep training in analytical thinking, as opposed to having learnt to jump through hoops and collect modules. But the reduced quality of education for the very brightest students will mean that we are no longer producing top quality researchers, teachers, managers (and politicians?) so important for the future of the country. This view will be (and has been) condemned as elitist by some (e.g. government ministers) but the failure to distinguish between social elitism and academic elitism is an example of the weak intellectual powers that our system is increasingly producing.