• How to Start and Grow a Think Tank
  • Prologue
  • Planning
    • Guiding Principles for State Think Tanks
    • Principles, Specialization, and Free-Market Think Tanks
    • SPN Professional Peer Mentoring Program
    • Do's and Don'ts for Lobbying Group: Charity Activity in an Election Year
    • Holding Think Tanks to High Standards
    • A Think Tank's Bottom Line
    • Measure Success, Not Just Output
    • Waging the War of Ideas: Measuring Success, Maximizing Impact
    • Sample Business Plan
  • Legal Documents
  • Staff
  • Board
  • Sample Board Job Descriptions
  • Infrastructure
  • Fundraising
  • Events
  • Networking and Outreach
  • PR and Marketing
  • General Management
  • Resources
• Grassroots Leadership Guide to K-12 Education Reform
• Taking Donors Seriously
• Fundraising Fundamentals

Jump to another manual

Waging the War of Ideas: Measuring Success, Maximizing Impact

John Blundell, Institute of Economic Affairs

My past involvement with SPN members and the Atlas Economic Research Foundation has been proving very useful recently.

These past few months I have received wannabe think-tank entrepreneurs at my London office from Nigeria, Israel and Denmark and have also corresponded with a keen young professor in Mauritius.

That’s quite a geographical spread but what strikes me is how similar their questions are and how similar their policy concerns.

Their concerns can virtually all be grouped as:

• constitution and rule of law
• taxes and spending
• privatisation and regulation

Likewise their questions can be equally easily grouped into:

• corporate structure and funding
• product lines
• sales and marketing

To most readers of SPN News the answers to such questions – or at least a tentative potential answer – are their daily meat and drink. However, let me mention here three matters that I always stress once the obvious has been covered, namely:

• Independence
• Academic review
• Proxies for success

Independence

The late great Antony Fisher, the original think tank entrepreneur, always stressed independence and as each year passes the more I understand why he did so. To be credible, to maximise impact, think tanks must:

• be independent of trade associations, unions, political parties and universities
• accept no government grants and no government contracts
• accept only general funding (drug company funding of drug deregulation is not a good idea)
• have a wide base of individual, foundation and corporate donors such that no one donor contributes more than 5% of total budget. You have to be able to say “no” to inappropriate investor pressure.

Accepting a big contract or outcome-tied funding might give some momentary boost but you will pay for it dearly over the ensuing years.

Academic review

No IEA paper is issued without it being given a double blind review by two outside experts, usually academics. By blind I mean every effort is made to take self-identifying material out of the manuscript so the reviewer does not know whose work it is in fact under review. Likewise the author is not usually told who the reviewers were, hence the phrase ‘double blind’. This hopefully avoids favoritism and conflicts of interest on the one hand while freeing the reviewer to pull no punches on the other hand. Most insights, comments and remarks are passed on to the author and can greatly improve the final product.

This double blind refereeing process is vital for your integrity and quality control. The IEA has 50 academics or similar on its Editorial Board and in any given year 30 to 40 will participate in this way at least once. This in turn makes them more than mere window-dressing. And if some member of that Board refuses to participate (for no good reason) they are quietly not renewed.

Proxies of success

Measuring what we do is awfully hard. We drop stones in ponds and the ripples are many and hard to track; they lap to and fro and disappear under overhanging leafy branches.

Also, chasing just one or two proxies will skew your whole organisation. While at the IEA we are working on a success index based on over a dozen different measures, let me share a list of questions I have now used for 8 years as new products are developed and which seem to correlate highly with impact. I score each question on a range of 1 (low) to 10 (high):

• How well does it fit with our mission of broadening public understanding of the role of markets and their ability to solve pressing social problems?
• How well will it sell?
• Is it in language that makes its main points accessible to the educated layman not familiar with technical jargon?
• To what extent is it of interest to the experienced practitioner in the field under consideration?
• To what extent will overseas publishers and institutes be interested in foreign rights?
• To what extent will it garner reviews in the popular press?
• To what extent will it garner reviews in the academic press?
• Is this product capable of changing a reader’s world view?
• Does it suggest further avenues of research and follow-up publications?
• Does it make a serious long-term challenge to an existing deep-rooted paradigm?
• How long a shelf life will it enjoy?
• Is it appropriate for college/school course adoptions?
• Will its publication enhance the resumé and reputation of the author(s) and hence his or her career prospects?
• Will its publication excite and enthuse the IEA staff and Trustees?
• How much did you enjoy the script?

A score of 150 is impossible but working through such a list is very helpful I find.

So guard your independence, use a double blind academic review and develop measures or proxies for them.

John Blundell is the General Director of the Institute of Economic Affairs in London.