October 9, 2009

You need specialists in bunny hopping to truly appreciate a good bounce

Posted in Uncategorized tagged , , at 6:21 pm by mariawolters

In a recent post on perfidy in peer review, Drugmonkey argues that it can be beneficial to have your grant reviewed by people who are potential competitors, even though they might steal ideas from your grant. He says:

I tend to think that if you don’t have at least one Bunny Hopper reviewing your bunny hopping grant you are sunk. If you are in the Bunny Hopper study section, at least then you have a chance.

Indeed. And no funding agency illustrates Drugmonkey’s wise words better than Britain’s Engineering and Physical Sciences Research Council. In a relatively recent overhaul of their grants procedure, they replaced several specialised panels in Information and Communication Technologies that took place every couple of months with general panels that take place every 4-6 weeks, but cover every grant in ICT, no matter what field.

Grants in speech technology jostle for attention with robotics, networking, and software engineering. Up to 50 separate grants are discussed at any given panel and ranked according to reviews. If you specialise in bunny hopping (or speech technology, to choose a slightly less exotic example), your chances of having a fellow bunny hopper (or speech technologist) on your panel is exceedingly slim.

This development has made it very difficult for panel members to independently assess reviewers’ grades. If you want to get funded, all your reviews better be outstanding, because there’s nobody to argue your case if one of the reviewers was particularly negative due to an unfortunate misunderstanding, a bad day at work, or a misanthropic streak. (You never know. It can’t have been your proposal, of course, which was PERFECT. Otherwise you wouldn’t have submitted it, would you?)

This has had significant repercussions for the research groups I work with, not to mention the interdisciplinary field of computational linguistics and speech technology as a whole. Our work does not necessarily progress computer science itself, but may consist of novel applications of existing algorithms, or it may explore new avenues in the structural representation of linguistic meaning. We may not invent new machine learning algorithms, but we tune them so that computer interfaces can adapt themselves to users. Now, after these changes, it is almost as difficult for us to get EPSRC funding as it is to get European funding. People who used to reliably get grants and deliver high quality work have had long losing streaks. Sure, other constraints such as the advent of full economic costing and budget restrictions have contributed to this situation. But the loss of specialised panels does not help.

The moral of the story? If you can get your bunny hopping research reviewed by bunny hoppers, thank your lucky stars (and make sure you cite all the colleagues in your field on the off chance that they get your proposal).