Research Reports

How ranking of advocacy strategies can mislead

Share
twitter-white-icon
fb-white-icon
linkedin-white-icon
email-white-icon
link-white-icon

Thoughtful and effective advocates take the long view, recognize and value the interdependence of advocacy outcomes and resist the temptation to rank advocacy strategies.

Well-conducted studies comparing animal advocacy strategies make a crucial contribution to the body of knowledge our movement needs to improve its effectiveness. Most commonly, they perform each intervention separately in a controlled setting and measure short-term outcomes. Irrespective of the intentions of researchers, the metrics used are often perceived as proxies for effectiveness and the results of many studies get interpreted as recommending one advocacy strategy over another without qualification.

We fall for this temptation to rank our advocacy options based on short-term efficiencies because, in grappling with the overwhelming complexity of growing our social movement, we seek something measurable to make our assessments and we desire simple and clear answers. But, while improving our understanding in certain crucial ways, our tendency to rank based on a partial set of metrics can also mislead us.

The issue is not limited to interpretations of animal advocacy research; informal debates among animal advocates often revolve around questions posed in a way that does not leave room for nuance or conditional factors. For example, activists often ask if one strategy is more effective than another, under the implicit assumption that there exists one right yes-or-no answer with no qualifiers. People ask what is the most effective thing for an activist to do, unmindful that there may not be one simple right answer to the exclusion of all others. These are ill-posed questions which invite a ranking as an answer—we must frame our questions better.

A more meaningful question to ask is: given finite resources, how much of each resource, such as time or money, should our movement invest currently in each strategy? There are two specific but altogether common scenarios which illustrate the essence of this point and are elaborated below: (i) when the success of outcomes of different advocacy strategies are interdependent; and, (ii) when the context plays a key role in dictating effectiveness.

When outcomes are interdependent

There are at least three broad and interdependent forms of advocacy that our movement for farmed animals engages in, each of which can be informed by good research: those aimed at individuals, those aimed at institutions and those that are facilitative in their activism.

animal-advocacy-strategy-classes

Outreach to individuals takes the form of leafleting, classroom presentations, movement building activities, online ads, social media, educational programs at animal sanctuaries, and authorship of blogs, books or documentaries. Campaigns aimed at institutions may seek improvements in animal welfare policies, replacement of meat-based dishes with vegan options, or legal protections for animals through lobbying or ballot initiatives. Facilitative activism may include the development and marketing of clean meat or plant-based alternatives to meat, entrepreneurship focused on removing barriers to eating vegan, or authorship of cookbooks without animal ingredients.

These forms of activism all play intertwined roles in the movement with the rising success of each contributing to those of the others. For example, outreach to individuals creates vegans, vegetarians, reducetarians and the veg-curious who create and sustain the market for facilitative products like a plant-based burger, the easy availability of which, in turn, improves the success rate of individual outreach.

When outcomes of interventions are interdependent, the effectiveness of each is inextricably linked with those of the others. Justifying one as being more effective than another is not quite straightforward—declaring so is often misleading.

The necessity of matching supply and demand

One form of interdependence is that governed by the dynamics of supply and demand for animal-based products. A foundational premise in economics is that prices shift as necessary to match supply and demand. But, the real world is not very faithful to economic theory—price fluctuations are dictated by a complex array of factors besides demand and supply. When demand exceeds supply, managers of companies have to cope with unhappy customers and forgone profits. When supply exceeds demand, they have to contend with wasted labor, lost capital, and unwanted inventory.

This is a familiar chicken-and-egg dilemma faced by firms trying to grow their markets—how much to invest in improving the supply to generate new demand and how much to invest in generating new demand to incentivize improvements in supply. It is a central tenet of good management that healthy growth of a firm or a market entails a continual effort to keep demand and supply matched while steadily increasing both. This doctrine borrowed from the field of operations management holds important lessons for animal advocates.

This interdependence between supply and demand, especially intrinsic to the service industry, is naturally relevant to the context of activism seeking to grow the market for plant-based eating. It is decisive to answering another ill-posed question that comes up frequently in the long-running debate on how best to advance animal rights: are campaigns seeking institutional reforms more effective than outreach to individuals?

Let's consider a supply-side intervention targeting an institution: a campaign aimed at a dining services company and demanding an increase in the percentage of meals served that are meatless. Let's compare it against a demand-side intervention targeting patrons of that company and urging them to eat more meatless meals or, even better, go vegetarian or vegan.

In these times when most dining service establishments maintain an electronic record of each customer order, demand for plant-based options is not easily faked. A company's management is more likely to respond favorably to supply-side animal advocates when it notices an actual rise in the demand for alternatives to meat. Individual patrons are more likely to respond favorably to demand-side animal advocates when the dining establishment they frequent offers a larger variety of tempting plant-based meals. Not only is the likelihood of the success of each form of advocacy increased by the success of the other, each of them is crucial to the other for sustained progress toward their shared goal.

Interdependence when impact latency is high

It is not just supply and demand which generate interdependence between broad forms of advocacy. Sometimes, one form of advocacy may derive its effectiveness from that of another without the type of immediacy inherent to a supply-demand relationship. For example, some forms of mass activism subtly but surely shift cultural attitudes and contribute to the success of other forms of activism such as ballot initiatives to protect farm animals. These forms of interdependence between advocacy strategies go especially under-recognized because they are less direct and their influence may materialize only over the long-term.

Consider the crucial work accomplished by institutional campaigns aimed at improving the treatment of animals in our factory farms. Many of the leaders of these institutional campaigns today are deeply motivated by ethical issues to which they were first introduced years or even decades ago by books, documentaries, leaflets, lectures or some form of organized mass outreach aimed at individuals. It is a reasonable hypothesis that the success of institutional campaigns today is at least partially owed to past advocacy aimed at individuals, which suggests that the higher we rate the effectiveness of institutional campaigns today, the higher we must rate the effectiveness of past individual outreach.

Changing how we evaluate effectiveness

If our goal is to improve short-term effectiveness—such as when the urgency of helping as many animals as soon as possible can be justifiably prioritized over uncertain longer-term outcomes—we may choose to defer paying attention to the interdependence of outcomes. In such cases, if we think that supply-side advocacy targeting institutions is more efficient, we must surely let demand chase supply. Or, in those cases in which demand-side advocacy aimed at individuals is more efficient, we can let supply lag demand. But, in the long-term, we do need to reach a stable state of balanced supply and demand as we shift our society away from animal-based foods. For lasting outcomes and durable effectiveness, our movement cannot shrug off the need to work on both demand-side and supply-side advocacy.

A commonly raised hitch in the pluralistic view of advocacy above is that, measured by the amount of time, money or effort expended by animal advocates, supply-side campaigns and demand-side advocacy are not equal in efficiency. In fact, when two forms of advocacy are equally crucial to an outcome desired by the movement but are unequal in efficiency, it is the less efficient one that merits a higher share of the resources. To propel forward a boat equipped with normal oars on its left side but inferior ones on the right that are only half as efficient in generating thrust, we will need twice as many oarsmen on the inefficient side than on the efficient side.

We err when we judge advocacy only by the efficiency with which it generates short-term outcomes. It is a concern that evaluations of the effectiveness of advocacy strategies, more often than not, are centered on what is easiest for us to measure—the efficiency of obtaining short-term positive outcomes; it is one form of our measurement bias that narrows the significance of many such evaluations. Effectiveness, even if derived through long-term interdependence, holds value for the movement.

In a multi-generational struggle such as what our movement is engaged in, we must not confuse short-term efficiency for long-term import and pertinence. Surely, we must measure efficiency in our advocacy research. But, the results are better used to develop insights on how best to apportion our resources than to judge overall effectiveness and rank advocacy strategies.

When context is key

Is focusing on the ethics of eating meat more effective than health-oriented advocacy? Is it more effective to ask people to eat less meat than to ask them to go vegetarian or vegan? These are among another set of questions frequently asked with the expectation that there is one correct answer, just waiting for conclusive evidence to emerge out of research experiments. But, the answer differs depending on a larger set of contextual variables than is often considered in well-isolated and well-controlled research experiments.

An experimental study may not be able to replicate the full variety of contexts faced by activists in real life: the context of the advocacy currently in use, the context of the audience of our advocacy, and the context of the environment in which the messages are delivered. Yet, these studies are undeniably useful when interpreted with requisite care. Naïve interpretations of these studies, on the other hand, tend to indulge our tendency to rank advocacy messages and misleadingly declare, absent context-sensitive nuance, one more effective than another.

The hubris of effectiveness

Consider the hypothetical that you are a committed "fruit activist" who would like people to eat more fruit—not any particular fruit, just more fruit! You decide to volunteer to stand at a busy street corner and sell apples or oranges, at cost, to anyone willing to buy one. A local farm has kindly agreed to supply you with your choice, at cost, of either a basket of oranges or a basket of apples but not both. Since you are an effective activist who makes decisions based on good data, you conduct a careful well-controlled scientific experiment to find out which fruit people are more likely to buy. Your experiment reveals that 60% of the people willing to buy a fruit would buy an apple but not an orange, while the remaining 40% would buy an orange but not an apple, suggesting that it is one-and-a-half times more effective to sell apples than to sell oranges. What fruit should you carry in your basket?

Of course, the answer appears clear from the results of your experiment; you would be a more effective fruit activist if you tried selling apples instead of oranges. So, you follow the data and choose to sell apples at cost.

Now, suppose a second and equally committed activist comes along who wishes to join you on the same street corner reaching the same passers-by. What fruit should she try selling? If she sells apples as well, she may steal away about half your customers leaving each of you only half as effective as when you were selling apples alone. If she offered oranges instead, each fruit buyer would have a choice between an apple and an orange and the two of you together may achieve greater effectiveness than either of you alone.

So, for the good of the cause, suppose your fellow committed activist chooses to sell oranges instead of apples. Now, you will surely be selling more fruit than her, making you a proud adherent of your strategy. Besides, in debates on effectiveness, you will also have the scientific experiment on your side to wave at her and claim how you are the more effective activist. But you would not really merit such arrogance—after all, by selling oranges instead of apples under the context presented to her, she is actually doing the most effective thing she can!

Thinking on the margin

Admittedly, the above example as an analogy to strategic options in animal advocacy is an imperfect one; all too often, there are too few animal advocates to saturate the target population with our messages. But, it does illustrate an important truth: context matters.

A ranking of effectiveness may be inferred from an experimental study measuring the impact of each intervention in isolation. But, the marginal contribution made by each activist depends on the context of the forms of advocacy currently in use. Activists are most effective when they think on the margin to guide their advocacy. The most effective strategy for an activist to employ is not necessarily the one that scores highest in a well-conducted experimental study. It is often one that is most underemployed in the movement in relation to its score in that experiment.

People differ in myriad ways and it is another reason why prescriptive pronouncements on effectiveness unmindful of the context are limited in their applicability. In the above hypothetical example, a better reading of the results of the experiment would try not to rank apples or oranges one above the other but instead allow a diversity of fruit in each basket. After all, the most effective thing for you as a fruit activist would be to carry apples and oranges in the approximate ratio of 60:40 in your basket. The upshot is that the results of scientific experiments are best used not to rank advocacy strategies but to deduce, in any particular context of interest, the ratio in which to invest time and effort into each strategy—for the movement as a whole and for each advocate.

The takeaway for activists

Here are a few of the most significant takeaways for researchers, supporters, activists and movement leaders:

  • What is measurable is not entirely indicative of inherent value and, certainly, short-term efficiency is not necessarily indicative of long-term value to the movement. We must not de-prioritize or de-glamorize—and, certainly not de-legitimize or denigrate—some forms of activism over others without gaining a deep and well-founded understanding of the long-term value of each.
  • While perfunctory interpretations of experimental studies may end up generating and popularizing a ranking of the purported effectiveness of advocacy strategies, we must be mindful of the contexts in which they apply and the interdependence between their outcomes. Ideally, instead of ranking them on their effectiveness, we must seek insight on how best to distribute our limited resources amongst them.
  • Our tendency to rank discourages thinking on the margin and depresses the diversity of strategies employed in the movement. It also demoralizes activists making valuable contributions that do not score well on what is most readily measurable. Certainly, some advocacy strategies are wasteful and even counter-productive. But, of those that have a positive impact, our use of a diversity of them has a value unto its own. No particular organization needs to engage in every form of advocacy but the movement, as a whole, is better off investing judiciously in multiple forms of effective advocacy.
  • We must exploit interdependence of outcomes as best as we can. For example, a supply-side campaign demanding that a company pledge to increase the number of meatless meals it serves is more likely to succeed and have more of a lasting impact if it is also accompanied by a demand-side campaign targeting the company's patrons to eat less meat or go vegetarian.
  • We must be cognizant of how the context is a large determinant of the impact of our activism. For example, if a certain strategy ranks highest in effectiveness in a certain experimental study, it does not imply that all activists must rush to employ it. The most effective thing for an activist to do is not necessarily what is deemed most effective in a good scientific experiment. Instead, it is often the most effective thing left undone.

Thoughtful and effective advocates take the long view, recognize and value the interdependence of advocacy outcomes, resist the temptation to rank advocacy strategies, think on the margin and are context-aware.