Background

     UK Computing Research Committee
     http://www.ukcrc.org.uk/
     Council of Professors and Heads of Computing
     http://www.cphc.ac.uk/

about research funding, including mention of the high proportion of research time and funding budgets that goes into writing and assessing proposals, provoked me into reviving an idea that always seems to produce shock and horror, even though I think that (with suitable refinements) it could be an excellent way to fund research, namely, using a properly designed lottery.

There are two key premises behind this proposal:

In short: (a) spending large resources on assessing research proposals tends to force research into areas where fairly detailed advance planning and prediction of results is possible, so that there is something to assess in advance, leading to too much emphasis on shallow and relatively trivial (and therefore predictable) research, and (b) the processes of generating such proposals and assessing them consume a lot of resources that would be better spent on doing research.

DISCLOSURE:
Most of my own UK research grant proposals have been turned down, though a few were successful. However, a few funders have approached me unasked and offered to fund my research, without asking for anything explicit in return (except reports in one case).

In 2004, the EC cognitive systems initiative, which was quite visionary, was willing to fund my research, though the process was over-managed.

A proposal

The proposal has several interlocking aspects:

1. Universities (and other organisations) that wish to do research should take great care when hiring people -- much more than is normally done in universities in the UK, compared with commercial/industrial research centres, and compared with universities in the USA (at least the ones I know about). Moreover the selection should be based on research potential, originality of ideas, the ability to give exciting presentations, etc. rather than on numbers of publications in journals and previous grant income (factors expected to influence RAE results as opposed to the growth of knowledge).

   If most of the people doing research have already been pre-selected as outstanding researchers, there is less need to assess their specific research in advance of their doing it. So we need a mechanism that gives them high research credibility initially. EPSRC's existing First Grant scheme does that though though it can only benefit a very small minority of new researchers and has the previously mentioned flaws of wasting resources on attempting to assess work that has not been done yet and distorting proposals to fit the grant assessment procedures. It also has a vicious 24 month cut off period (after appointment), which can be grossly unfair, since different amounts of time may be required to identify and clarify different research problems.

2. Individuals appointed as research active academic staff should automatically have the right to take part in the lottery-based research funding process described below, though a university or department might remove or postpone that right in cases of doubt (a decision not to be taken lightly, and which should be subject to appeal). In most cases if universities were properly funded, internal funding could be used to give new researchers a chance to prove themselves.

3. Each individual's lottery bids will be weighted by a factor that starts high, then decays with time (using a backward sigmoid curve), but can be boosted, as described below. This is unlike the 'First Grant' scheme which has a sharp temporal cutoff, as mentioned above.

4. More of the research funding that is now based on competitions run by research councils should instead be allocated directly to research active departments for internal allocation, especially for use by young researchers. Internal allocations should be based on judgements about individual promise, past achievements, and evidence of ability to identify hard problems and investigate them in depth. Some of this funding can take the form of reduced teaching loads or sabbaticals, e.g. for theoretical scientists, for whom there seems to be grossly inadequate provision in the research council funding schemes.

   There should be sufficient direct funding to universities to allow them to give very good research support for all young researchers for the first few years (partly in the form of reduced teaching and admin loads, but also travel, equipment, and in some cases RAs). Mentoring schemes can help nurture young researchers, and also detect whether things are going wrong so that projects should be terminated or redirected -- which research councils don't do, fortunately. (The EC attempts this, at great administrative cost to all concerned.)

   If this reduces the amount allocated by research councils that's not necessarily a bad thing if it transfers resources from administration to research, as well as ensuring better help and guidance for young researchers during their first projects. Of course, university managers would have to be prevented from reallocating that funding for other purposes, which many will try to do. Incidentally I am sure that the most important effect of the 'Full economic costing' scheme over time will be a bad one: namely transferring funding from direct grants to universities to competitively allocated budgets managed by research councils. That is not the intention of the supporters of the scheme, but watch what happens in the next five years. I suspect that very soon it will lead to unfair discrimination within underfunded universities between researchers who get research council funding with FEC and those who don't. (That may look like good business acumen because it motivates people to strive for grants, but is actually bad management because it wastes effort that should be spent on research and is potentially divisive, undermining collaborative and mutually supportive research cultures.)

   NOTE:

   All this assumes that not all tertiary education departments should get research funding. We need to revert to a system with a far better division of labour, with a significant subset of higher education done in institutions that are dedicated to high quality education and training such as many polytechnics used to provide, and which both the nation and many school leavers need.

   Those institutions and the people in them should be well rewarded for what they do, and not punished at all for not doing research. This issue is discussed further in a separate document pointing out that views on both sides of the debate on top-up fees arise from a failure to plan a well integrated broad spectrum higher education system.

5. Research oriented universities (or departments) should not be punished (as the RAE tends to punish them) for having significant subsets (e.g. as many as 25% or even 30%) of their staff who are not research active but are dedicated to keeping up with research results relevant to their teaching, producing high quality teaching, helping with administration and management and helping research active staff understand requirements for good teaching. All permanent academic staff who do research should also do some teaching since communication of knowledge is as important as discovering it. But some division of labour can be more fruitful than expecting everyone do equal shares of everything.

The lottery [still a rough draft]

Applicants will be people on academic posts who are now eligible to be PIs on EPSRC proposals. Funding from research councils to such individuals who apply for grants will in most cases be allocated in the lottery on the basis of a probabilistic formula which combines elements described below. Very large grant proposals will be dealt with differently, as explained above, using a pre-allocated budget, as will some development proposals aiming at achieving specific practical goals identified by the Research Council. Projects above some size limit will have to be vetted before being accepted for the lottery. Call the proposals eligible to be funded by the lottery 'eligible' proposals.

Every recognised individual researcher N will have at any time a weight W(N) (e.g. between 0 and 100) that will be used in the lottery if N submits a proposal. W(N) will change over time, as explained below.

Some individuals may temporarily contribute some or all of their weight to a virtual individual that is a group of individuals who wish to collaborate and submit a joint proposal to the lottery.

1. For each individual N, an initial weight W(N) will be assigned which starts high at the beginning of N's first appointment to an academic post and drops automatically over time. The rate of fall should be very slow (or possibly the curve should be flat) until the first proposal is submitted by N. After that the weight W(N) should fall more rapidly, though that can be countered by weight-boosting mechanisms described below.

2. Every time a proposal is submitted by N, whether it is proposal fed to the lottery scheme or another mode of selection, W(N) is reduced thereafter temporarily, either for a year or if the project is funded for half (or some other fraction) of the project's length. The amount of reduction of W(N) could depend on the size of the grant awarded. This would also apply to new researchers. So some will be deterred from rushing to submit immature proposals, which is what happens if there are sharp temporal deadlines.

3. If N gets one or more grants, then at intervals the research results produced by N are assessed by some means. The outcome is used to boost or lower W(N). We could have a long debate about details, such as use of citation measures and other things, but the debate can be shortened if we accept that there is no perfect solution and every proposal will include an element of lottery anyway. It is essential to include assessment from outside the individual's subfield, since the ability to communicate should be a requirement for the vast majority of excellent researchers.

   The assessment should take account of both quality (depth, difficulty, theoretical implications, practical implications), and the number/diversity of admirers of the research (as in citation indexes). It might be partly automated and partly based on judging panels, which should change their membership often enough to spread the workload and reduce long term effects of personal bias.

   (Whether W(N) should be reassessed whenever the individual submits a proposal is debatable. I would say not, except for the first few proposals. Various options are possible, e.g. do the assessment when a new proposal arrives if that individual has not been assessed in the past 4 years or some such thing. Individuals who believe they have produced major new results could request a re-assessment by providing prima-facie evidence.)

4. The result of assessment would produce a (possibly zero) increase to W(N). If N's work is judged very poor W(N) will be reduced. In order to allow someone to jump from a low to a high weight on the basis of outstanding new performance, increases should be additive not multiplicative, subject to an upper limit.

5. To prevent abuse of the lottery system, repeated grant applications by N should reduce W(N) by an amount that increases with the frequency.

6. Collaborative proposals could be dealt with by allowing a group to constitute a new virtual individual. The participating individual collaborators could each determine how much of their current W(N) should be allocated to the group.

7. Group projects will always be assessed when complete, except for teams with a well established research record, whose results could be assessed at intervals. When group is assessed, the resulting (positive or negative) boost is divided between the participants in a manner that they can decide. E.g. it could depend on how much of their individual W(N) each one had contributed, as in most investment schemes. Or it may depend on who did what. In case of disputes some adjudication mechanism may be required, but this should be rare if members of groups all sign some appropriate agreement in advance. To reduce administration a group collaborating for a long time could opt to be treated as a virtual individual with intermittent evaluation.

8. It should be possible for people on temporary research posts to request assessment. This will allow their research record to affect proposals involving them, and may also allow their W(N) to be boosted by previous research record if they later take academic posts.
   (This may be unnecessary if the starting weight in point (1) is high enough. But perhaps the default should not be too high, in which case previous research record can boost the starting weight.)
   [I have not thought enough about how to treat people on research posts. These are just first thoughts.]

The formula used in the lottery will need to be reconsidered from time to time by analysing their impact. The previous comments imply that there will be a number of management constants in the formula, and it may be hard to get them right. In fact it is very likely that at first there will be some mistakes that need to be corrected quickly.

Added 16 May 2014:
I have found this related discussion.

    Monday, March 18, 2013
    How to Give $1 Million a Year to Philosophers
    I hate grants.
    By Eric Schwitzgebel

Installed: 2005
Last updated: 13 Dec 2005; 12 Jun 2011; 16 May 2014; 8 Feb 2016(re-formatted)