ISSN 1835-9728 Environmental Economics - AgEcon Search

Loading...

ISSN 1835-9728 Environmental Economics Research Hub Research Reports

Choice Experiment Framing and Incentive Compatibility: observations from public focus groups

Abbie McCartney and Jonelle Cleland Research Report No. 76

November 2010

About the authors Abbie McCartney and Dr Jonelle Cleland are Research Associates, at the School of Agricultural and Resource Economics, University of Western Australia. Email: [email protected]; [email protected]

This research is part of the Environmental Economics Research Hub Project 8: Divergence between community and expert valuations of ecosystems, which is funded by the Australian Commonwealth Environmental Research Facilities (CERF) programme. The authors would like to thank Professor Michael Burton for his comments on this manuscript and his feedback on particular aspects of the study.

Environmental Economics Research Hub Research Reports are published by The Crawford School of Economics and Government, Australian National University, Canberra 0200 Australia. These Reports present work in progress being undertaken by project teams within the Environmental Economics Research Hub (EERH). The EERH is funded by the Department of Environment and Water Heritage and the Arts under the Commonwealth Environment Research Facility. The views and interpretations expressed in these Reports are those of the author(s) and should not be attributed to any organisation associated with the EERH. Because these reports present the results of work in progress, they should not be reproduced in part or in whole without the authorisation of the EERH Director, Professor Jeff Bennett ([email protected])

Crawford School of Economics and Government THE AUSTRALIAN NATIONAL UNIVERSITY http://www.crawford.anu.edu.au 2

Table of Contents Abstract .......................................................................................................................................................... 4 1. Introduction ................................................................................................................................................ 5 2. Background ................................................................................................................................................. 6 2.1 Strategic Behaviour within Multiple Question CM .............................................................................................. 6 2.2 Consequential Designs ........................................................................................................................................ 8 2.2.1 Statements of Consequence ........................................................................................................................ 8 2.2.2 Provision Rules ............................................................................................................................................. 9 2.2.3 Honesty Scripts .......................................................................................................................................... 10 2.3 Summary and Implications ............................................................................................................................... 11 3. Methodology..............................................................................................................................................12 3.1 Case Study – Ningaloo Marine Park ................................................................................................................. 12 3.2 Focus Group Organisation ................................................................................................................................ 12 3.3 Questionnaire Versions .................................................................................................................................... 14 3.4 Focus Group Procedure .................................................................................................................................... 17 4. Results .......................................................................................................................................................18 4.1 Reaction to Standard Framing Elements of the Choice Experiment ................................................................ 18 4.1.1 Initial Reaction to Choice Experiment Framing Statements ...................................................................... 18 4.1.2 Independence of Choice Scenarios ............................................................................................................ 19 4.1.3 Reaction to Payment Vehicle ..................................................................................................................... 19 4.1.4 Preference Orientation .............................................................................................................................. 20 4.2 Reaction to Incentive Compatibility Framing Statements ................................................................................. 22 4.2.1 Perceived Realism ...................................................................................................................................... 22 4.2.2 Direct Comparison between Framing Alternatives .................................................................................... 23 5. Discussion ..................................................................................................................................................25 References .....................................................................................................................................................27 Appendix 1a: Questionnaire – Statement of Consequence Version ...................................................................29 Appendix 1b: Alternative Wording – Honesty Script .......................................................................................34 Appendix 1c: Alternative Wording – Provision Rule .........................................................................................36 Appendix 2: Running Sheet for Focus Groups...................................................................................................37

3

Abstract The hypothetical nature of choice modelling surveys makes it difficult to enforce incentive compatible properties. It is thought that bias may result through strategic behaviour and untruthful responses, given that the hypothetical choice scenarios and payment structure are not binding. This study examines three methods of addressing incentive compatibility through survey framing: (1) a statement of consequence; (2) use of an ‘honesty’ script that openly explains how the data are to be analysed and used; and (3) use of a provision rule that defines how survey outcomes relate to actual implementation. Focus groups, involving members of the public, were held to investigate participants’ reactions to the three framing treatments. The provision rule emerged as the preferred treatment in terms of being more realistic than the alternatives. The rule did not need to be 100% binding to have the desired effect of inducing realism. However, the participants did not believe that their responses to the choice scenarios would have changed between framing treatments. Empirical testing is required to determine if this is actually the case. Other reassuring results were found in relation to how participants interpreted the general choice scenario instructions, particularly in terms of answering questions independently and as an individual consumer. This provides evidence that respondents make choices in response to the questions as they are intended by the researcher.

4

1. Introduction Stated preference techniques are a form of economic valuation used to estimate how much people are willing to pay for non-marketed goods. Frequently applied in the realm of public good valuation, the purpose of eliciting dollar values is so that these values are directly comparable to the dollar values of other goods that are traded in a market place. Applying a consistent value metric (i.e., dollars) offers the ability to directly compare the costs and benefits of different projects or policies that concern nonmarketed assets. Stated preference surveys set up a hypothetical market through which a respondent provides discrete responses about how much they are willing to pay for a particular scenario (Bateman et al. 2002). For example, in an environmental context, a respondent may be faced with a choice between doing nothing (at no cost), or opting for a (hypothetical) conservation program that would cost $50 per year via an environmental tax if it were to be implemented. Choice modelling (CM) is a common form of stated preference technique. In CM, the good in question is described by its constituent parts, or attributes, and these form the basis of the choice scenario presented to respondents. Respondents are faced with a question where they are asked to decide between two or more alternatives (e.g., different conservation programs), each of which have a cost associated with them. The alternatives each contain the same set of attributes, but the levels of the attributes vary across the alternatives so that each alternative in a choice scenario offers a unique package. Often, respondents are faced with a series of choice scenarios, each providing a different combination of attribute levels and alternatives. The nature of incentive compatibility (IC) in CM, and stated preference studies more generally, is an important issue to consider in generating a study that provides realistic and plausible results. IC relates to the ability of a survey to elicit truthful responses regarding an individual’s preferences (Carson and Groves 2007). To be incentive compatible a survey must be consequential in nature. In other words, the respondent believes the results of the survey will influence decisions relating to the good at hand, and the respondent cares about the outcome of the subsequent decisions (Carson and Groves 2007). To be incentive compatible, a survey must also avoid designs that offer incentives to act strategically or untruthfully in revealing preferences (Burton 2010). The concept of IC can be considered in connection with the issue of hypothetical bias. Due to the hypothetical nature of stated preference studies, individuals may respond to questions in a likewise hypothetical manner and produce biased and frequently overstated willingness to pay (WTP) estimates. Because the respondent considers the payment is hypothetical, and, as is the case with public goods, it is likely they won’t ever actually have to pay (directly) for the good in question, they may heighten the value they are actually willing to pay to help encourage the provision of the good (Arrow et al. 1993). Thus, when IC is not enforced, hypothetical bias can result. The only stated preference question format that is known to be incentive compatible is the single-shot binary discrete choice format. That is, a single choice situation consisting of two alternatives (one usually 5

defined as a status quo). In response to the first condition of IC, when appropriately framed, this question format can adhere to consequential properties. For example, framing the question as a binding referendum, where the most popular choice will be implemented (Carson and Groves 2007). The referendum need not be 100% binding to adhere to IC; it could be a matter of stating that the most popular choice will be considered for action, and likely to be implemented. It is sufficient that individuals are convinced that the outcome has probable consequences. The second condition of IC is satisfied in a single-shot binary choice due to the singular nature of the choice question. Respondents are only provided with one choice, so it is always in their best interests to answer the question truthfully, removing the potential for strategic responses. The single-shot binary choice is typical of that used in a contingent valuation (CV) exercise, which is another form of a stated preference technique. However, CM studies frequently employ multiple question formats that have multiple (i.e., more than two) alternatives, and/or multiple choice tasks in sequence, which can violate the conditions of IC. This paper investigates how individuals respond to various treatments aimed at enforcing IC in a multiple question CM exercise through the use of public focus groups. Section 2 provides the background for why IC is an issue in CM, and discusses some of the approaches used in the literature to address the concern. In particular, it introduces the three framing options applied to the focus groups: (1) a statement of consequence, considered as a general/baseline approach here; (2) provision rules; and (3) an ‘honesty’ script. The methodology is presented in Section 3, followed by the results and discussion in sections 4 and 5.

2. Background The following sections offer a discussion of the issues related to the two conditions of IC with respect to strategic behaviour within multiple question CM (Subsection 2.1) and consequential designs (Subsection 2.2). The implications of these issues and how they relate to the case study at hand is provided in Subsection 2.3.

2.1 Strategic Behaviour within Multiple Question CM When straying from the single-shot binary discrete choice question format, the incentive to reveal truthful preferences can be weakened. Strategic behaviour may be present in question formats that have more than two alternatives, and in multiple choice situations. Consider a choice question with three alternatives. An individual may prefer one option the most, but they may consider that other individuals are unlikely to prefer that option. As such, they may choose their second most preferred

6

option if they believe it to be more popular amongst the community, in an effort to avoid their least preferred option being ‘voted in’1 (Scheufele and Bennett 2010). Now consider a multiple choice situation. Given a sequence of choices, individuals may consider past choices made, or potential future choices, instead of treating each choice scenario as an independent situation (Carson and Groves 2007). For example, a respondent has been faced with a choice where they are offered a particular bundle of goods, X, for $20 in an alternative. They prefer this bundle to the status quo, and select this alternative in the choice scenario. The respondent is then faced with a subsequent choice question where they are again offered the bundle of goods X in an alternative, but now have to pay $60 for it. They still prefer this option to the status quo, but refer back to the earlier scenario where they were offered X at a cheaper price. Strategically, they may opt for the status quo, rather than the alternative with the preferred bundle X, because they would prefer to obtain X for $20 despite being willing to pay $60 in reality. This behaviour can have detrimental consequences for an individual. For example, assume the actual cost of provision for X is $50. The revealed preferences in this instance would suggest (incorrectly) that X is not worth this much and should not be implemented, when in reality the individual would have been prepared to pay the $50 (and up to at least $60) for the provision of X. Until recently, it was assumed that strategic behaviour would not be as prevalent in choice experiments as in CV. This was based on suggestions that the complexity of a choice question would (1) make it difficult for respondents to bias answers (Lu et al. 2008), (2) that the use of multiple attributes would make it unclear as to whether a particular choice is over or under representing a valuation (Adamowicz et al. 1999), and (3) because CM is believed to minimise issues such as protest bids and yeah saying associated with CV (Hanley et al. 2001). However, recent literature has found some evidence of strategic bias associated with choice experiments. Burton (2010) conducted an experiment providing a financial incentive for respondents to strategically choose alternatives that overstated the value of one attribute relative to others. The experiment used various design constructs (numbers of alternatives, attribute levels, rankings versus most preferred alternative only) and found that participants successfully biased results in several instances, particularly when using the standard choice format of selecting the most preferred alternative. Scheufele and Bennett (2010) and McNair et al. (2010) investigate the presence of strategic behaviour in a comparison of single-shot binary and multiple binary choice situations for a public and quasi-public good, respectively. In the multiple choice treatments, the respondents were made aware that they were going to be presented with more than one choice scenario. In both studies, there is no evidence to suggest that strategic behaviour results from the awareness of a multiple choice situation. In particular, McNair et al. (2010) find that WTP in the single-shot binary choice situation is equivalent to WTP for the first binary choice in the multiple choice situation, despite the respondent knowing that more choice questions will be presented in the latter case. 1

This scenario is frequently seen in political elections, for example: there are several parties to choose from, and assume an individual prefers party A the most. However, only parties B or C are likely to be voted in to parliament. The individual dislikes party B, and so they vote for party C on the premise that their most preferred party A is unlikely to win while C has a chance.

7

However, McNair et al. (2010) also find some inconsistency between WTP in a multiple choice situation (i.e., when considering all choice situations and not just the first) as opposed to a single-shot choice. Racevskis and Lupi (2008) conduct a similar sequence effect study with a public good that supports the findings of McNair et al., finding inconsistency in WTP estimates between single and multiple choice situations. The single-shot binary choice may seem preferable in terms of adhering to IC; however, it is arguable that the multiple question experimental design is superior. In particular, statistical efficiency is improved with increasing number of choices/alternatives. A statistically efficient design can allow for a more robust analysis, and can significantly lower the cost of a CM exercise by reducing the required sample size (Scarpa and Rose 2008). It has also been argued that the repetition of multiple choices allows the respondent to learn, reducing choice inconsistency (Bateman et al. 2008). The results of the McNair et al. (2010) and Racevskis and Lupi (2008) studies therefore suggest that there may be a trade-off between the theoretically sounder singular choice format, and the statistically more efficient and costeffective multiple question formats.

2.2 Consequential Designs Whether one uses single-shot or multiple question CM designs, the survey has to be seen by the respondent as consequential if they are to have any reason to take answering the survey seriously (irrespective of whether they find reasons to answer truthfully or not). However, multiple question CM designs present an additional problem because it is less obvious to the respondent how the multiple answers might be employed in forming policy. With careful framing a survey can achieve consequential properties. Here, we have taken into consideration a range of design options including statements of consequence, provision rules and ‘honesty’ scripts.

2.2.1 Statements of Consequence A direct approach to addressing the consequential property of IC is through adding statements in the framing of the choice questions to convince respondents that the results of the survey are meaningful and could have a real impact. Known as ‘consequentialism’, this can be achieved by stating that the results of a study will be made available to decision makers, who may choose to implement a new policy/program/package based on the results, effectively making the hypothetical payments real (Landry and List 2007). Studies investigating consequentialism have been conducted predominantly in the CV literature, and generally involve tests of different probabilities that the hypothetical payments could become real payments. Landry and List (2007) report that in an experiment where there is a 50% probability that the hypothetical payment could be enacted, results are statistically similar to a situation where the payment is real (i.e., 100% probability). Cummings and Taylor (1998) test varying levels of probability that 8

hypothetical payments become real, and find that probabilities of at least 50% are required to mitigate hypothetical bias. Carson et al. (2004) find that probabilities as low as 20% can reduce hypothetical bias, suggesting that just some notion that hypothetical payments could become real is enough to enforce the consequential condition of IC. The above mentioned studies focussed on tangible goods, where it was possible to enforce the real payment mechanism within the experiment. In an environmental context, this is more difficult. Bulte et al. (2004) considered an environmental issue, adhering to the more generalised approach of making results available to policy makers rather than discussing specific probabilities of hypothetical/real payments. They found that WTP was significantly lower for the treatment with statements of consequence, than for the standard hypothetical approach of not disclosing any information about the use of the study results. Given the association of hypothetical bias with overstating WTP, the results of this particular study suggest that a statement of consequence reduces bias, in turn adhering to (at least one of) the properties of IC.

2.2.2 Provision Rules Another approach to addressing the consequential property of IC is through adding a stipulation on how the survey outcome will be interpreted and executed. Arrow et al. (1993) suggest the use of a provision, or implementation, rule in their recommendations for best practice CV. Such a rule is easily enforced in a single-shot binary discrete choice. For example, a plurality vote rule can be applied where the alternative or program receiving the most support is implemented and the associated payment for the program applies to everyone (Scheufele and Bennett 2010). However, in a multiple choice situation, a provision rule is more difficult to make explicit and binding. Over a series of choices there will be a sequence of most preferred programs. Thus, simply suggesting that the program that receives the greatest support over the entire series of choices will be implemented could be in violation of the second condition of IC. That is, it could persuade respondents to strategically make decisions across the full series of choices (i.e., as discussed above) and conflict with the researcher’s intention to have each choice scenario considered as an independent situation. Therefore, a provision mechanism needs to be defined that is binding, but does not encourage strategic behaviour. A study on catchment conservation by Mazur and Bennett (2010, p.13) attempts to address the multiple choice problem by designing a provision rule that states “Only options that are chosen by more than 50 percent of the people surveyed will be considered further for implementation by the Catchment Management Authority”. This rule allows for all options that receive more than a 50% vote to be considered, avoiding the incentive to act strategically over a number of choices. However, the statement is not explicitly binding, in that it does not suggest a particular program will be implemented. It is presently unclear whether the non-binding nature of such a rule is an issue for IC. Indeed, Mazur and Bennett (2010) find inconsistencies between WTP estimates for their provision rule treatment as

9

opposed to a control treatment (i.e., with no provision rule), providing some evidence that the provision rule had an impact on preferences, despite it not being explicitly binding.

2.2.3 Honesty Scripts In many cases where a multiple question CM is applied it is unlikely that the conventional consequentiality or provision rules could be strictly true as stated. Although the studies may be applied to real ‘live’ policy issues, they will have been undertaken by academics or others, independently from the actual policy process. Although the intention may be to generate results that may inform debate, they often are not primarily designed to do so. In what Adamowicz (2004) has identified as the ‘academic’ market for valuation studies, academics may be most interested in providing a test bed for postgraduate training, or developing new methodological approaches in survey design or statistical analysis. Furthermore, the suggestion made within provision rules that particular policy options will be considered if they receive a certain degree of support implies that the options/alternatives within a choice experiment represent considered combinations of outcomes within a policy package. In fact, those combinations of attribute levels will have been constructed within statistical design software whereby achieving statistical efficiency is the core objective. It is unlikely that they will represent any actual combination that might be considered for implementation. Whether the statements of consequence/provision rules achieve their desired objective then relies upon the degree to which the respondents believe them. For example, the provision rule used by Mazur and Bennett (2010) was strictly not true: it’s possible that the Catchment Management Authority might use CM responses in that way, but in that case it was not truly part of the policy process. In fact, the most likely means by which the outcomes from a CM study will have a policy impact is through the value identified for individual attributes within the study, and via the use of benefit transfer. CM has been identified as being of particular value in the area of benefit transfer (e.g., see Rolfe and Bennett, 2006). Under these circumstances, Mazur and Bennett’s (2010) provision rule might be restated as: “At some point in the future, an environmental outcome with one or more of the attributes included here may be delivered, IF the costs of provision are less than the estimated benefits of the attribute(s) inferred from this study. It is likely that you will bear your share of the costs of this, either directly through taxation or through increased prices or user fees”. This ‘honesty’ script2 requires that the respondent is made aware of how the choices they make are to be utilised, that is, the full set will be analysed in a way that allows the researcher to identify the tradeoffs they are making across attributes, and to retrieve dollar values associated with each change in level of each attribute.

2

The concept of the honesty script was suggested by M. Burton.

10

There are a number of advantages of the approach. If it is clearly stated that the analyst will be treating these choices as independent events there is no incentive for the respondent to try and make choices in one question contingent on answers given in others. Given that it is stated that any one or more of the attributes may be the focus of future policy intervention there is no advantage in trying to strategically bias values towards a most preferred attribute and away from others. If such behaviour leads to an undervaluation of a secondary attribute, the respondent runs the risk that provision of that attribute will not be achieved in the future, even though they would be prepared to pay the cost. There are still issues of anticipated free riding in general, leading to inflated valuations of all attributes if they do not believe that they will have to pay the costs of provision, but that comes down to a belief and understanding of how governments fund environmental policy, not in how the values are to be generated by the survey. Effectively, an honesty script could be used to unveil the workings of the CM technique itself, albeit in layman’s terms. The underlying premise to this approach is, essentially, that by gaining an understanding of how the data are used, an individual may respond to the survey in an incentive compatible manner. At the very least, an honesty script may clarify what the researcher is asking the respondent to do. Survey responses are often taken at face-value – that is, it is assumed that respondents have both answered truthfully, and that they have answered the specific question that was intended by the researcher (Carson and Groves 2007). It is entirely possible that, even with the use of deliberate wording, the respondent may interpret the question being asked differently to the researcher’s intentions, and answer the question that they think is being asked. Taking the time to explain how the researcher uses the data could minimise the degree of misinterpretation. The challenge is to explain how the CM process works, if not in technical detail at least in ways that convince the respondent that the best way they should respond to the survey is to provide honest choices of which alternative they prefer.

2.3 Summary and Implications Multiple question constructs are commonly applied in the CM literature, and the design benefits of these formats suggest this will continue to be the case. As such, ways to minimise hypothetical bias and induce IC should be considered. A review of the relevant literature on IC has revealed that: (1) More empirical testing is required to provide conclusive evidence of how best to design a choice experiment to minimise strategic bias. (2) A statement of consequence should be included in all future CM studies as a minimum baseline for addressing IC, given the brevity and ease of implementation. The question of whether framing options that include rules or scripts are an improvement on this baseline is worthy of further investigation. 11

This paper focuses on point (2), and is concerned with the developmental stages of designing a survey that is incentive compatible in terms of the consequential condition.

3. Methodology To investigate response to manipulation of IC statements, public focus groups were used to trial three versions of a questionnaire regarding ecological values of the Ningaloo Marine Park in Western Australia. The three versions varied according to the framing of the instructions on how to answer the choice experiment questions. In particular, a statement of consequence (which is considered as the minimum baseline for IC, as discussed above) was trialled along with two alternative versions aimed at improving IC: (1) an honesty script that explained how the collected data was to be utilised; and (2) a provision rule stating that preferred options would be considered by decision makers. A brief summary of the case study is presented in Subsection 3.1 below, followed by an outline of the organisation of the focus group sessions in Subsection 3.2. Subsection 3.3 provides a detailed description of the three questionnaire versions used, and an outline of the procedure used within each focus group is discussed in Subsection 3.4.

3.1 Case Study – Ningaloo Marine Park The decision was made to embed the alternative instructions within a pre-existing survey. The principle reasons for doing so were: (1) extensive testing of the survey had already taken place, ensuring that all framing and design elements were comprehendible and had been interpreted in the correct manner; (2) analysis and modelling of the data generated from the survey had produced plausible results; and (3) researchers working on this particular study were familiar with the case study. The survey was originally designed as part of a PhD study to investigate conservation values associated with various ecological features of the Ningaloo Marine Park, a 300 kilometre long coral reef system off the North West coast of Western Australia (see McCartney 2010). The original survey contained a statement of consequence in the framing instructions. As such, these instructions were maintained as the baseline for comparison with the alternative IC framing statements investigated here (i.e., the honesty script and provision rule).

3.2 Focus Group Organisation Two focus group sessions were held during June 2010. Each session focused on the statement of consequence version in comparison with one of the alternative instructions in the choice experiment. Within in each session, participants were split into two sub-groups: one sub-group received the statement of consequence version of the questionnaire first, followed by either the honesty script or the provision rule; the other sub-group received the alternative version first, followed by the statement of consequence version. Using this approach allowed potential order effects to be captured. The remainder 12

of the paper will refer to the focu focus group participants as follows: 



Session 1 participants: Honesty Script Session o Statement of consequence version first, followed by honesty script version  Consequence-Honesty Honesty (CH) group o Honesty script version first, followed by statement of consequence version  Honesty-Consequence sequence (HC) group Session 2 participants: Provision Rule Session o Statement of consequence version first, followed by provision rule version  Consequence-Provision Provision (CP) group o Provision rule version first, followed by statement of consequence version  Provision-Consequence sequence (PC) group

The organisation of the focus group sessions is depicted diagrammatically in Figure 1.

Session 1: Honesty script (8 participants) Focus groups (17 participants) Session 2: Provision Rule (9 participants)

Consequence questionnaire (4 participants)

Honesty Script instructions

Honesty Script questionnaire (4 participants)

Consequence instructions

Consequence questionnaire (4 participants)

Provision Rule instructions

Provision Rule questionnaire (5 participants)

Consequence instructions

CH GROUP

HC GROUP

CP GROUP

PC GROUP

ical representation of focus group organisation. Figure 1: Diagrammatical

13

3.3 Questionnaire Versions Three questionnaires were developed that were identical with the exception of the instructions on how to answer the choice scenarios. All three sets of instructions covered the following important points:     

a basic description of how to answer the choice scenarios; a description of the payment vehicle; a reminder to consider ones budget constraint whilst answering; advice that each scenario should be considered as an independent situation (in an attempt to minimise strategic behaviour); a statement relating to respondents being selected to provide a representative voice for likeminded people (to encourage individual responses)3.

Variations in the instructions were included for the purpose of comparing how people respond to the various approaches of framing the choice scenarios and encouraging IC. The main differences are noted below for the three instruction versions, and can be viewed in full in appendices 1a, b and c: i)

Statement of consequence version – this set of instructions makes a simple consequential statement to note that the results will be made available to policy makers who may choose to use the information for future management decisions (full instruction set can be viewed in Figure 2).

3

It is generally assumed in stated preference surveys that the data analysed provides a measure of consumer preference, but there is evidence that respondents act as citizens rather than individuals (e.g., see Blamey et al. 1995). It is arguable whether acting as a citizen is in fact in violation of the assumptions of stated preference studies. One could assume that legitimate individual benefits arise from citizen type decisions, for example, benefits from altruism (i.e., the value derived from enjoyment of someone else’s pleasure) (Rolfe and Bennett 1996). However, it is important that respondents recognise they have been selected to represent like-minded individuals in the target population, and not to represent the entire population.

14

Part B – Conservation Scenarios 

This section contains 4 hypothetical conservation scenarios that are aimed at gaining an understanding of your preferences for protecting different features of the ecoregion



Each scenario contains 4 conservation options, from which you will be asked to choose the one you most prefer.



Each scenario should be considered independently; this is not a test of consistency between conservation scenarios. Some of the scenarios may seem odd to you, but each is a possible outcome based on the combinations of the conservation strategies



You will find a cost to you associated with each possible choice you can make. This hypothetical cost is an additional environmental tax, i.e. it will be added on to your existing annual tax payment. The tax will be charged each year.



While answering these questions you should consider your own financial circumstances, i.e. consider the limit of how much you can realistically afford given your current household income and personal expenses.



Please consider your answers carefully. The results from this study will be made available to relevant management authorities and may be used to guide policy and management decisions for Ningaloo in the future.



You have been chosen to represent people with similar preferences to yourself, so by completing these questions you are providing a voice for like-minded people.

Figure 2: Choice scenario instructions provided in the consequence questionnaire, highlighting the statement of consequence in the second last point (for full questionnaire see Appendix 1a).

ii)

Honesty script version – this set of instructions replaces the generalised consequential statement with a series of points explaining that the data is analysed to unpack individuals’ preferences for specific attributes and levels, and this information can then be repackaged by decision makers to determine appropriate policy in the future. It is dubbed the honesty script version because it explains, in layman’s terms, what CM researchers do with the data in reality (relevant instruction points can be viewed in Figure 3).

15



Each of the conservation scenarios facing you contains a package of ecological features that differ according to conservation levels. When you pick the best conservation scenario out of those available, you are revealing something about how you value the different features. Through a statistical analysis of all the responses, we can unpack the choices made by survey participants. This allows us to identify how much weight you place on each of the conservation levels, and how much the population as a whole is willing to pay to protect each individual ecological feature. We will treat each conservation scenario independently, so you should too: each one reveals a unique decision to us about what you value most. It is not a test of consistency between management scenarios, and there is no point in comparing the options offered in one scenario to another. The various conservation scenarios you will see below are hypothetical ones, and possibly none of these will actually be implemented. However, in the future, decisions about different policies and management strategies can be evaluated using the information collected from this study, because the value of different ecological features has been revealed. Actual future policy can be determined based on the conservation levels and the choices you have made in these hypothetical situations.

Figure 3: Alternative wording for the honesty script instructions, showing statements that replace the general consequential statements of the consequence version (for full set of instructions see Appendix 1b).

iii)

Provision rule version – this version replaces the consequential statement made in the consequence version with a more binding statement that notes the most preferred option from each choice scenario will be presented to relevant authorities. It is important to note that the most preferred option for each scenario is considered here, rather than stating that only the most preferred option overall will be considered. This careful wording is used in an attempt to avoid the issues of strategic bias associated with multiple choice situations described in Section 2 and persuade participants to answer each scenario independently (relevant instruction points can be viewed in Figure 4).

16



Please consider your answers carefully. The results from this study will be used to guide policy and management decisions for Ningaloo in the future. The most popular option from each of the 4 conservation scenarios will be determined from the population surveyed. The 4 most popular options will then be considered by the relevant management authorities for implementation.

Figure 4: Alternative wording for provision rule instructions, showing statements that replace the general consequential statements of consequence version (for full set of instructions see Appendix 1c).

3.4 Focus Group Procedure An identical procedure was followed within each sub-group, summarised in the following steps: 1) Participants instructed to complete full version of questionnaire (containing one of the three choice scenario instruction versions) as if they were an actual respondent in the survey; 2) Open discussion guided by facilitator (using set questions as prompts); 3) Participants provided with a second set of instructions to read; 4) Open discussion guided by facilitator (using set questions as prompts). A running sheet was utilised to ensure the same questions were asked of participants in each sub-group (Appendix 2). Questions were generally oriented towards determining the reaction towards the different framing methods, both directly in terms of how they responded to the various IC statements, and indirectly in terms of how each of those statements may have influenced other points in the instructions. To determine how the IC statements may have influenced interpretation of the instructions more generally, questions included:     

What was the most obvious/stand-out point in the instruction set; How did they react to the payment vehicle; Whether they answered each choice question independently; Whether they acted as an individual as opposed to a citizen; General feedback on the instructions, including clarity of wording.

Directly relating to which of the versions were preferable and incentive compatible, questions included:   

How realistic did they think each survey version was; Which version (consequence or honesty script/provision rule) did they prefer; Would they have changed their responses to the choice scenarios had they been presented with the second set of instructions they received in the first instance.

17

4. Results The focus group discussions were captured both in terms of qualitative participant comments and quantitative votes according to the questions asked by facilitators. First, this section reports on how the participants interpreted the initial survey treatment provided to them (which included the completion of several choice questions), in terms of their reactions towards the general framing statements for the choice experiment (Subsection 4.1). Second, this section reports their reaction to the specific IC statements (Subsection 4.2).

4.1 Reaction to Standard Framing Elements of the Choice Experiment Participants’ reactions towards the general choice scenario framing instructions are presented below. Subsection 4.1.1 makes note of the particular instruction points that stood out the most to participants. Next, Subsection 4.1.2 discusses whether participants considered each choice question in the survey as an independent scenario while answering them. Reactions to the payment vehicle defined in the survey are presented in Subsection 4.1.3, followed by a discussion of whether participants responded to the survey as individual consumers or citizens (Subsection 4.1.4).

4.1.1 Initial Reaction to Choice Experiment Framing Statements

Participants were initially asked which of the individual instruction points or statements stood out the most to them. Recurrent across all groups, the stand-out statements predominantly related to the payment instructions (Table 1). This included participants acknowledging that (1) there was a payment vehicle in the form of an annual environmental tax, and (2) they needed to consider their budget constraint while answering the choice questions. In particular, one participant noted that the statement asking you to consider your own financial circumstances reminded them to “choose with your head and not with your heart”. These results are reassuring in terms of IC, with participants paying attention to the fact that a cost is involved, albeit a hypothetical one.

Table 1: Participant comments regarding the most obvious point or statement in the choice scenario instructions. Question 1 Statement of Consequence – CH & CP groups (8 participants) Honesty Script – HC group (4 participants)

Provision Rule – PC group (5 participants)

Stand-out point in instructions - Payment instructions: the environmental tax (4); personal financial status (4) - Representing people with similar preferences (1)

- Payment instructions: environmental tax (3) - Hypothetical nature of conservation scenarios and possibility of not being implemented (1) - Payment instructions: personal financial status (5)

Note: Number of participants in agreement with comment in parentheses.

18

4.1.2 Independence of Choice Scenarios

There were statements included in all three versions of the questionnaire aimed at encouraging respondents to treat each choice scenario independently4. Without direct reference to these statements by the facilitator, all participants concluded that they had treated the questions as independent scenarios when prompted. Indeed, one participant recollected that it was a specific point in the instructions. However, it was noted by several participants that they had employed the use of particular decision heuristics. All of the participants in the two control groups (CH and CP) prescribed to using decision rules, or “guiding points”, as well as one participant from the Honesty-Consequence group. They used tactics such as consistently weighting specific attributes higher than others, and choosing options that offered maximum conservation value at a reasonable cost – to “get the best bang for their buck”. The latter could be considered congruent with the notion of IC; respondents were searching for cost efficient options suggesting an awareness of budget constraints and the future potential for implementation of the hypothetical conservation packages.

4.1.3 Reaction to Payment Vehicle

Participants were asked to comment on how they generally perceived the payment vehicle – an annual environmental tax payment. Approximately half of the participants considered the tax payment to be suitable (Table 2). However, there were many suggestions on how to re-word the instruction, with the suggestions conflicting amongst different groups: 



All participants in the Honesty-Consequence group and one in the Provision-Control group thought that specifying the payment as an ‘environmental’ tax was useful so that you knew the money would be spent on the environment. They suggested that ‘tax’ could be perceived as a bad word, and should be replaced with something along the lines of ‘levy’ or ‘fund’. Some participants in the Consequence-Provision group commented that earmarking the payment as an ‘environmental’ tax draws attention to the fact that people have to pay more. They thought it would be more appropriate as a general tax that people would absorb in their annual tax payments with the additional payment going unnoticed.

The latter comment is essentially counterproductive to the intentions of CM and IC. The purpose of a CM exercise is to explicitly draw attention to the payment vehicle so respondents view it as binding. The dollar amounts need to be at the forefront as they ultimately determine how much people are willing to pay for an attribute. One participant from the Consequence-Honesty group thought that after explaining the payment vehicle, the budget constraint instruction should be reinforced. They thought this could be achieved by 4

In the statement of consequence version this appeared as a simple statement that it is ‘not a test of consistency’ amongst choice scenarios. In the alternative versions, the point is elaborated on further. In the honesty script there is a brief explanation of how the data is analysed independently. In the provision rule version there is encouragement to make choices independently in each scenario by stating that the most popular option from each scenario will be considered for implementation. See appendices 1a, b and c for full details of the statements.

19

asking respondents to state their income to encourage them to think about their personal financial status in the choice questions that follow. Payment vehicles that stray away from the traditional coercive forms (such as taxes) were commonly suggested. The possibility of reallocating existing tax dollars was mentioned as being preferable to an additional tax, and the notion of volunteering time in place of paying money (e.g., volunteer time for ‘tin-collecting’) was suggested as an alternative form of payment for those who can’t afford to contribute in dollars. A payment of this nature would seem contradictory to the purpose of IC, as it removes the budget constraint, and the notion of the payment being individually binding along with it.

Table 2: Participants reaction to the environmental tax payment vehicle. Question 3 Statement of Consequence – CH & CP groups (8 participants) Honesty Script – HC group (4 participants) Provision Rule – PC group (5 participants)

Reaction to environmental tax payment vehicle - Tax suitable (4) - Suggestion of reallocation of taxes rather than raising more money (2) - Volunteering time in place of payment (1) - More emphasis on personal financial status (1) - Tax suitable (4)

- Tax suitable (1) - Objection to tax (3) - Preference for reallocation of taxes (5)

Note: Number of participants in agreement with comment in parentheses.

4.1.4 Preference Orientation

CM data is generally analysed under the assumption that respondents are making choices based on their own individual preferences. However, there is evidence to suggest that respondents may act as citizens, making choices based on what they think the broader community would prefer (see Section 3). Each of the questionnaire versions includes a statement that respondents have been chosen to represent people with similar preferences to themselves in an attempt to elicit their individual preferences (which includes legitimate altruistic forms of community benefits). To gauge how the focus group participants reacted to this they were asked whether they answered the choice questions as an individual or citizen. The majority of participants adhered to the instructions and answered as individuals (Table 3). However, two participants in the Consequence-Provision version made choices as citizens, both having the same explanation that they were placing a high weight on the fish attribute based on the assumption that others in the community would want to protect the attribute for the purposes of recreational fishing, even though they didn’t take part in recreational fishing themselves. Although most made decisions as individuals, a few noted various citizen related influences in their choices, including:

20





Choosing packages that they thought people of the same age and position could afford (although this could be considered part of the act of representing like minded people, and not strictly a deviation from individual preference). Answering based on their own financial circumstances, but considering the broader communities interests in the choice also, in the sense that the environmental issues at hand impact the entire community.

In an attempt to ensure future survey respondents understand the requirement to answer as an individual, participants suggested a two-step process: 1) Explain the relevant instruction with more detail about the sample selection process and the purpose of them answering as an individual, for example: You should answer these scenarios from your own perspective. We have attempted to select participants based on a good cross-section of demographics in the community, so you will be representing people with similar preferences as yourself and as such should answer as an individual.

2) Reinforce that it is an individual preference in the choice scenarios by stating ‘please choose your most preferred option’. An encouraging result was that one participant found the instruction noting you are representing people with similar preferences as a stand-out point (Table 1). They felt a strong sense of responsibility to take the survey seriously knowing that they were speaking on behalf of other like-minded people. Thus, it appears that notions of broader community representation encourage IC also.

Table 3: Participants inclination to answer the choice questions as individuals or citizens. Question 5 Statement of Consequence – CH & CP groups (8 participants) Honesty Script – HC group (4 participants) Provision Rule – PC group (5 participants)

Individual versus citizen preferences - Answered as individuals (6) - Answered as citizens (2)

- Answered as individuals (4) - Answered as individuals (financially) and citizens (5)

Note: Number of participants in agreement with comment in parentheses.

21

4.2 Reaction to Incentive Compatibility Framing Statements The participants’ response to credibility and realism of the IC framing statements is discussed in subsections 4.2.1 and 4.2.2. The former is concerned with the initial survey treatment provided to the participant, and the latter examines how participants interpreted the two instruction treatments in relation to each other.

4.2.1 Perceived Realism

Participants were asked to consider the first survey treatment provided to them in terms of whether they thought the survey would appear realistic and likely to influence policy decisions (or offer conservation programs that would actually be implemented in the case of the provision rule version) if they had been presented with the survey outside of the focus group. Opinions were divided both across and within the sub-groups (Table 4). Participants in the Consequence-Provision group were split evenly between considering the control survey to be realistic or being undecided. Participants in the Consequence-Honesty group all agreed that stating the management process used to achieve a particular conservation outcome would make the consequence survey more believable – respondents would be able to see what is actually going to be done with the money they are (hypothetically) spending. This suggests that IC can be improved at the structural level of attribute definition, rather than in terms of framing. In the Honesty-Consequence group, half of the group viewed the honesty script instructions as either realistic or having potential to influence policy. However, the other half from this group thought the survey lacked credibility, forming the view that studies of this nature are intended just as a data collection method for students, with the results not going further than the researcher’s own desk, and not ending up in a government office where it can influence policy. In particular, one participant (from the HC group) noted that the stand-out point for them related to the sentence stating that ‘The various conservation scenarios you will see below are hypothetical ones, and possibly none of these will actually be implemented’ (Table 1). The participant believed that this statement generated a lack of credibility for the survey, and it should instead state that one of the scenarios would be implemented. However, discussion ensued following this, with another participant noting that they interpreted the statement to mean that it was possible that the results of the study could influence policy. In the Provision-Consequence group, the participants misinterpreted the question as asking whether they preferred the survey or focus group elicitation method to gain an understanding of public preferences. It is unlikely that this was due to the choice scenario instructions themselves; rather it was a misinterpretation of the question asked during discussion. Despite prompting from the facilitator, the discussion remained attached to this topic and the participants concluded that they preferred focus groups as an elicitation method.

22

Table 4: Participants perceived realism of the survey and likelihood of results influencing policy. Question 4 Statement of Consequence – CH & CP groups (8 participants) Honesty Script– HC group (4 participants)

Provision Rule – PC group (5 participants)

Realism of survey, likelihood of influencing policy - Realistic (2) - Undecided (2) - Believability would improve if the management process used to achieve the conservation outcome was described (4) - Lacked credibility (2) - Realistic (1) - Could potentially influence policy further down the track (1) n/a

Note: Number of participants in agreement with comment in parentheses.

4.2.2 Direct Comparison between Framing Alternatives

After completing and discussing the first survey version, participants were then provided with the second instruction set. Participants were asked to consider their reaction to the second instruction set, as well as contemplating how it compared with the first set of instructions they received. One of the most notable results relates to the most preferred and believable instruction set. Participants were asked which instructions they thought were most believable, in terms of realism of the survey, and then in a separate instance they were asked to vote for which set of instructions they preferred. Perhaps not surprisingly, in every case the most believable and preferred version were the same, confirming that realism is an important component of a successful survey. From the honesty script session, the majority of participants preferred the statement of consequence version with reasons including (Table 5):   

The instructions were too long and repetitive in the honesty script version, and led to participants glossing over the details. They reacted negatively to the instruction to answer ‘truthfully’ as it seemed belittling, and would have preferred the word ‘accurately’. The statement that ‘… possibly none of these [conservation scenarios] will actually be implemented’ made them feel as though the researchers were only interested in gathering information and didn’t really care about getting a conservation outcome. This was similar to the point raised earlier in relation to the survey lacking credibility (Subsection 4.2.1). That is, there was doubt that the information would be passed on to government and rather it was being collected for a research data base. It is important to note that these two points were raised separately in each of the honesty script session sub-groups, meaning it was a recurring theme and was not related to ordering effects.

One of the participants from the Honesty-Consequence group preferred the honesty script instructions, particularly because they appreciated knowing how the data are actually used and found this more

23

believable. One participant in this group was undecided about which they preferred and thought both sets of instructions were believable. A striking result occurred in the provision rule session, with all participants from the ConsequenceProvision and Provision-Consequence groups preferring the provision rule instructions over the statement of consequence version (Table 5). They found the added provision rule statement to be more binding than the consequence text. They saw it as a “fuller disclosure” of how the results would be used, and that they would be more likely to be used, for policy. The Consequence-Provision group thought that if the survey was to be administered to a broader audience then respondents would be more likely to put effort into the survey with the provision rule instructions.

Table 5: Most believable and preferred set of instructions.

Honesty Script (8 participants) Provision Rule (9 participants)

Statement of Consequence Treatment 6 -

Alternative Treatment

Undecided

1 9

1 -

Participants in the provision rule session were asked an additional question regarding the wording of the specific provision rule instruction. The instruction stated that the ‘4 most popular options will then be considered … for implementation’. The wording of this statement is not explicitly binding, in that it allows the possibility that all four options could be rejected by the relevant management authorities even though they were favoured by the community. Participants were asked whether the statement would be more believable if it were changed to state that the ‘4 most popular options will be implemented’. Whilst still split into the Consequence-Provision and Provision-Consequence sub-groups, the nine participants all reached the same conclusion that they preferred to leave the wording in its current form – ‘considered’ is a strong enough word to induce realism and they preferred the more honest nature of the word, as one could not guarantee that an option from the survey would definitely be implemented in reality. They noted that the important point was that the instruction conveyed that the information generated from the survey would “end up on the table” of the relevant decision making authorities. With the participants having seen the two sets of instructions each, they were then asked to consider if the choices they made in the survey would have changed if the order that they viewed the instruction texts had been reversed, for example, if the participants in the Honesty-Consequence group had been given the statement of consequence version first and the honesty script version second. Although there was a clear preference for the statement of consequence version in the honesty script session, and an outright preference for the provision rule instructions in the provision rule session, all 17 participants noted that they did not think the choices that they made in the survey would have changed if the order of the instruction texts they were given had been reversed.

24

5. Discussion This study was concerned with the examination of three survey framing treatments aimed at investigating the incentive compatible properties of CM. Through the use of focus groups, participants discussed their interpretation of three sets of instructions on how to answer choice questions for a case study on the Ningaloo Marine Park. The instruction sets included a statement of consequence, an honesty script and a provision rule. Interestingly, the points that stood out the most to participants in all instruction treatments related to notes about the payment vehicle and budget constraints. Such notes are standard inclusions in CM survey framing, so this is a positive result in terms of IC for CM generally. It suggests that respondents are conscious of the (hypothetical) costs of a proposed program, and of their own disposable income – meaning there is recognition that constraints apply to choice, an important factor in giving a survey consequence. Giving more weight to the recognition of constraints, a common decision heuristic employed by participants was to search for cost effective options in the choice scenarios. More general support for the CM technique is offered in relation to the face-value dilemma that has been reported as an issue with stated preference surveys (see Carson and Groves 2007). However, we find congruence between the way the questions are interpreted by the focus group participants, and the way a researcher intends to analyse CM data. Specifically, participants treated the choice scenarios as independent questions, and in the majority of cases they answered as individual consumers. With respect to these particular aspects, these results suggest that taking answers at face-value is not an issue for any of the three instruction treatments applied here5. This is important in relation to collecting survey responses more generally; even an incentive compatible survey does not provide useful information if the researcher is analysing the data to answer a different question than that anticipated by the respondent. There was a point raised with respect to the payment vehicle that may not appear to be in favour of IC: participants were keen to see a reallocation payment (i.e., of existing government funds) versus a coercive individual payment. As mentioned above, a reallocation payment may seem to contradict IC, however, with due consideration the implications could be viewed as twofold. On the one hand, removal of an individual payment vehicle, such as taxes, allows respondents to make unconstrained choices, potentially in violation of IC (Swallow and McGonagle 2006). On the other hand, reallocation could be considered a more realistic form of payment – governments frequently reallocate budgets amongst portfolios to fund environmental projects (Bergstrom et al. 2004). If an individual cares about the placement of monies amongst the various portfolios, then choice could still be constrained (i.e., by the allocation of the existing government budget, or the way an individual’s tax dollars are distributed). Such a constraint on choice, along with the potentially more realistic appointment of money to fund programs, could provide an incentive to respond truthfully. 5

That is not to say that the face-value dilemma does not occur. It could be prevalent in other aspects of the survey, under different case study contexts, or where alternative wording is used for the choice scenario instructions to that used here.

25

Participant reactions towards survey realism relating specifically to the IC treatments provided a mixed response, particularly with respect to the statement of consequence and honesty script. Participants were generally in favour of the statement of consequence over the honesty script. This was predominantly due to the honesty script being too lengthy, and it lacking credibility because of the wording noting that it was possible none of the conservation scenarios appearing in the actual choices would be implemented. There was, however, some appreciation for the explanation of how the data collected would be used. It is possible that an honesty script could be more successful with further modifications to reduce the length of the script and use more positive wording to avoid the credibility issues. In relation to the statement of consequence, an intriguing result was the preference of several participants to include a description of the management processes used to achieve the outcomes in the hypothetical choice programs. The present study has been concerned with survey framing to address IC. The participants’ suggestion, however, raises the idea of improving the realism of a survey through the attribute levels themselves. That is, a management process could form part of the attribute level. Interestingly, the inclusion of management processes has been tested empirically using the same Ningaloo case study (McCartney 2010).The results suggest that preferences are affected by management inclusion, particularly in relation to how it restricts human use of the marine park. The participants’ preferences were more clear-cut for the provision rule, with all considering this a more realistic and binding approach than the statement of consequence. The results support previous findings that a 100% binding referendum is not necessary to induce IC (e.g., Carson et al. 2004, Landry and List 2007), with participants viewing the survey as consequential just with the suggestion that the most popular options would be considered by relevant management authorities. This result is particularly reassuring in an environmental context, where it can be difficult to enforce a 100% binding rule. The results of this focus group study suggest that careful use of a provision rule is the most effective means of incentive compatible survey framing. The provision rule is a simple statement that can be added to a typical set of consequential choice scenario instructions, without adding onerously to the text length. The alternative to the provision rule proposed here, an honesty script, failed to generate the same level of realism and credibility amongst focus group participants. However, it should be noted that participants from all survey treatments believed the answers they gave in the choice experiment would not have changed had they received an alternative survey treatment in the first instance. Further research is required in this area to confirm this result empirically and resolve uncertainties regarding the exact nature of the impact of a provision rule on IC.

26

References Adamowicz, WL 2004, ‘What’s it worth? An examination of historical trends and future directions in environmental valuation’, The Australian Journal of Agricultural and Resource Economics, 48(3): 419-443. Adamowicz, W., Boxall, P. C., Louviere, J. J. and Swait, J. 1999, ‘Stated Preference Methods for Valuing Environmental Amenities’, in Valuing Environmental Preferences: Theory and Practice of the Contingent Valuation Methods in the US, EU and Developing Countries, ed. I. J. Bateman and K. G. Willis, Oxford University Press, New York, pp. 460-479. Arrow, K., Solow, R., Portney, P. R., Leamer, E. E., Radner, R. and Schuman, H. 1993, Report of the NOAA Panel on Contingent Valuation, Federal Register. Bateman, I.J., Carson, R. T., Day, D., Hanemann, M., Hanley, N., Hett, T., Jones-Leee, M., Loomes, G., Mourato, S., Ozdemiroglu, E., Pearce, D. W., Sugden, R. and Swanson, J. 2002, Economic Valuation with Stated Preference Techniques: A Manual, Edward Elgar, Cheltenham. Bateman, I. J., Burgess, D., Hutchinson, W. G. and Matthews, D. I. 2008, ‘Learning design contingent valuation (LDCV): NOAA guidelines, preference learning and coherent arbitrariness’, Journal of Environmental Economics and Management, 55(2): 127141. Bergstrom, J. C., Boyle, K. J. and Yabe, M. 2004, ‘Trading Taxes vs. Paying Taxes to Value and Finance Public Environmental Goods’, Environmental & Resource Economics, 28(4): 533-549. Blamey, R., Common, M. and Quiggan, J. 1995, ‘Respondents to Contingent Valuation Surveys: Consumers or Citizens?’, Australian Journal of Agricultural Economics, 39(3): 263-288. Bulte, E., Gerking, S., List, J. A. and de Zeeuw, A. 2004, ‘The effect of varying the causes of environmental problems on stated WTP values: evidence from a field study’, Journal of Environmental Economics and Management, 49(2): 330-342. Burton, M. 2010, ‘Inducing Strategic Bias: and its implications for Choice Modelling design’, Environmental Economics Research Hub Research Reports, No. 61. Carson, R. T. and Groves, T. 2007, ‘Incentive and informational properties of preference questions’, Environmental and Resource Economics, 37(1): 181-210. Carson, R., Groves, T., List, J. and Machina, M. 2004, ‘Probabilistic Influence and Supplemental Benefits A Field Test of Two Key Assumptions Underlying Stated Preferences’, Working Paper, University of California, San Diego. Cummings, R. G. and Taylor, L. O. 1998, ‘Does Realism Matter in Contingent Valuation Surveys?’, Land Economics, 74(2): 203-215. Hanley, N., Mourato, S. and Wright, R. 2001, ‘Choice modelling approaches: a superior approach for environmental valuation?’, Journal of Economic Surveys, 15(3): 435-462. Landry, C. E. and List, J. A. 2007, ‘Using Ex Ante Approaches to Obtain Credible Signals for Value in Contingent Markets: Evidence from the Field’, American Journal of Agricultural Economics, 89(2): 420-429. Lu, H., Fowkes, T. and Wardman, M. 2008, ‘Amending the Incentive for Strategic Bias in Stated Preference Studies: Case Study in Users’ Valuation of Rolling Stock’, Transportation Research Record, 2049: 128-135. Mazur, K. and Bennett, J. 2010, ‘The Effects of a Provision Rule in Choice Modelling’, Environmental Economics Research Hub Research Reports, No. 49. McCartney, A. 2010, The Policy Relevance of Choice Modelling: an Application to the Ningaloo and Proposed Capes Marine Parks, draft PhD thesis, University of Western Australia. McNair, B., Bennett, J. and Hensher, D. A. 2010, ‘Strategic response to a sequence of discrete choice questions’, Paper th presented at the 54 Annual Conference of the Australian Agricultural and Resource Economics Society, Adelaide.

27

Racevskis, L. A. and Lupi, F. 2008, ‘Incentive Compatibility in an Attribute-Based Referendum Model’, Paper presented at the American Agricultural Economics Association Annual Meeting, Orlando. Rolfe, J. and Bennett, J. 1996, ‘Respondents to Contingent Valuation Surveys: Consumers or Citizens (Blamey, Command and Quiggan, AJAE 39:3) – A Comment’, Australian Journal of Agricultural Economics, 40(2): 129-133. Rolfe, J. and Bennett, J. (eds) 2006, Choice Modelling And The Transfer Of Environmental Values, Edward Elgar, Cheltenham. Scarpa, R. and Rose, J. M. 2008, ‘Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why’, Australian Journal of Agricultural and Resource Economics, 52(3): 253-282. Scheufele, G. and Bennett, J. 2010, ‘Effects of alternative elicitation formats in discrete choice experiments’, Environmental Economics Research Hub Research Reports, No. 52. Swallow, S. K. and McGonagle, M. P. 2006, ‘Public Funding of Environmental Amenities: Contingent Choices Using New Taxes or Existing Revenues for Coastal Land Conservation’, Land Economics, 82(1): 56-67.

28

Appendix 1a: Questionnaire – Statement of Consequence Version This appendix includes the full survey with attribute descriptions, the statement of consequence version choice scenario instructions, and a series of choice tasks. Participants in the Consequence-Honesty and Consequence-Provision groups were provided with this questionnaire in the first instance, followed by the alternative instruction wording presented in appendices 1b or 1c, respectively. Participants in the Honesty-Consequence and Provision-Consequence groups saw a similar questionnaire to this in the first instance, but with the instructions from appendices 1b or 1c (respectively) inserted in place of the statement of consequence instructions. After completion of the full questionnaire, the participants from these groups then saw the statement of consequence instructions from within this questionnaire as the secondary set of instructions.

29

RED VERSION QUESTIONNAIRE: Part A – Ecological Features of Ningaloo Please read the following information regarding some ecological features of the Ningaloo Marine Park:

Coral Communities The coral communities are responsible for building the structure of the Ningaloo reef system and provide a source of food and shelter for other marine life. Risks to the coral communities include direct human activities such as destruction from boat collisions, moorings and anchoring, and trampling the coral while reef walking or snorkelling Target Fish Stocks Target fish stocks refer to the finfish in the marine park that are most commonly sought after and caught by recreational fishers. The main impact of recreational fishing in the park results from the large catches of fish creating pressure on the fish population numbers. Seasonal restrictions that prevent recreational fishing in the marine park may be beneficial in Ningaloo over the summer months when fish are breeding. Turtles The marine park is a regular home and nesting place, between December and March, for many marine turtles. The main pressures on the Ningaloo turtle populations relate to their nesting. Foxes prey on both the eggs and hatchlings and they are currently being controlled by 5 main fox baiting zones, or locations, along the Ningaloo coast. Human interaction can also disturb the female turtles while they nest. Whale Sharks Whale sharks are the largest fish species in the world and are seen at Ningaloo between March and June each year. The whale sharks do not face many pressures in the Ningaloo Marine Park, although there are minor threats from boating collisions and noise, and stress from interaction with tourists both on boats and swimming in the water.

30

Part B – Conservation Scenarios 

This section contains 4 hypothetical conservation scenarios that are aimed at gaining an understanding of your preferences for protecting different features of the ecoregion



Each scenario contains 4 conservation options, from which you will be asked to choose the one you most prefer.



Each scenario should be considered independently; this is not a test of consistency between conservation scenarios. Some of the scenarios may seem odd to you, but each is a possible outcome based on the combinations of the conservation strategies



You will find a cost to you associated with each possible choice you can make. This hypothetical cost is an additional environmental tax, i.e. it will be added on to your existing annual tax payment. The tax will be charged each year.



While answering these questions you should consider your own financial circumstances, i.e. consider the limit of how much you can realistically afford given your current household income and personal expenses.



Please consider your answers carefully. The results from this study will be made available to relevant management authorities and may be used to guide policy and management decisions for Ningaloo in the future.



You have been chosen to represent people with similar preferences to yourself, so by completing these questions you are providing a voice for like-minded people.

31

Conservation Scenario 1 OPTION 1 Status quo

OPTION 2

OPTION 3

OPTION 4

0% more coral

5% more coral

10% more coral

5% more coral

0% more fish

10% more fish

0% more fish

10% more fish

0% more turtles

5% more turtles

5% more turtles

0% more turtles

Conservation of whale shark population

0% more whale sharks

5% more whale sharks

0% more whale sharks

0% more whale sharks

Cost to you per year

$0

$60

$20

$0









OPTION 1 Status quo

OPTION 2

OPTION 3

OPTION 4

0% more coral

10% more coral

0% more coral

0% more coral

0% more fish

5% more fish

0% more fish

10% more fish

0% more turtles

0% more turtles

0% more turtles

5% more turtles

Conservation of whale shark population

0% more whale sharks

2% more whale sharks

2% more whale sharks

2% more whale sharks

Cost to you per year

$0

$60

$80

$80









Conservation of coral reef Conservation of target fish stocks Conservation of turtle populations

Most preferred option:

Conservation Scenario 2

Conservation of coral reef Conservation of target fish stocks Conservation of turtle populations

Most preferred option:

32

Conservation Scenario 3 OPTION 1 Status quo

OPTION 2

OPTION 3

OPTION 4

0% more coral

0% more coral

10% more coral

0% more coral

0% more fish

0% more fish

10% more fish

10% more fish

0% more turtles

0% more turtles

0% more turtles

10% more turtles

Conservation of whale shark population

0% more whale sharks

2% more whale sharks

0% more whale sharks

5% more whale sharks

Cost to you per year

$0

$20

$60

$0









OPTION 1 Status quo

OPTION 2

OPTION 3

OPTION 4

0% more coral

5% more coral

0% more coral

10% more coral

0% more fish

0% more fish

5% more fish

0% more fish

0% more turtles

10% more turtles

10% more turtles

5% more turtles

Conservation of whale shark population

0% more whale sharks

2% more whale sharks

2% more whale sharks

2% more whale sharks

Cost to you per year

$0

$80

$20

$20









Conservation of coral reef Conservation of target fish stocks Conservation of turtle populations

Most preferred option:

Conservation Scenario 4

Conservation of coral reef Conservation of target fish stocks Conservation of turtle populations

Most preferred option:

33

Appendix 1b: Alternative Wording – Honesty Script The choice scenario instructions for the honesty script: these instructions were embedded in the full questionnaire (in place of the statement of consequence instructions seen in Appendix 1a) for participants in the Honesty-Consequence group, and were provided as they appear here as the secondary set of instructions for participants in the Consequence-Honesty group.

BLUE VERSION 

This section contains 4 hypothetical conservation scenarios that are aimed at gaining an understanding of your preferences for protecting different features of the ecoregion



Each scenario contains 4 conservation options, from which you will be asked to choose the one you most prefer.



You will find a cost to you associated with each possible choice you can make. This hypothetical cost is an additional environmental tax, i.e. it will be added on to your existing annual tax payment. The tax will be charged each year.



While answering these questions you should consider your own financial circumstances, i.e. consider the limit of how much you can realistically afford given your current household income and personal expenses.



Each of the conservation scenarios facing you contains a package of ecological features that differ according to conservation levels. When you pick the best conservation scenario out of those available, you are revealing something about how you value the different features. Through a statistical analysis of all the responses, we can unpack the choices made by survey participants. This allows us to identify how much weight you place on each of the conservation levels, and how much the population as a whole is willing to pay to protect each individual ecological feature. We will treat each conservation scenario independently, so you should too: each one reveals a unique decision to us about what you value most. It is not a test of consistency between management scenarios, and there is no point in comparing the options offered in one scenario to another. The various conservation scenarios you will see below are hypothetical ones, and possibly none of these will actually be implemented. However, in the future, decisions about different policies and management strategies can be evaluated using the information collected from this study, 34

because the value of different ecological features has been revealed. Actual future policy can be determined based on the conservation levels and the choices you have made in these hypothetical situations. 

You have been chosen to represent people with similar preferences to yourself, so by completing these questions you are providing a voice for like-minded people. If you do not answer truthfully then we will misrepresent how people value each of the ecological features reported here, and also misrepresent how much people are willing to pay to protect Ningaloo. This could ultimately see the wrong management policies being implemented in future.

35

Appendix 1c: Alternative Wording – Provision Rule The choice scenario instructions for the provision rule: these instructions were embedded in the full questionnaire (in place of the statement of consequence instructions seen in Appendix 1a) for participants in the Provision-Consequence group, and were provided as they appear here as the secondary set of instructions for participants in the Consequence-Provision group.

GREEN VERSION 

This section contains 4 hypothetical conservation scenarios that are aimed at gaining an understanding of your preferences for protecting different features of the ecoregion



Each scenario contains 4 conservation options, from which you will be asked to choose the one you most prefer.



Each scenario should be considered independently; this is not a test of consistency between conservation scenarios. Some of the scenarios may seem odd to you, but each is a possible outcome based on the combinations of the conservation strategies



You will find a cost to you associated with each possible choice you can make. This hypothetical cost is an additional environmental tax, i.e. it will be added on to your existing annual tax payment. The tax will be charged each year.



While answering these questions you should consider your own financial circumstances, i.e. consider the limit of how much you can realistically afford given your current household income and personal expenses.



Please consider your answers carefully. The results from this study will be used to guide policy and management decisions for Ningaloo in the future. The most popular option from each of the 4 conservation scenarios will be determined from the population surveyed. The 4 most popular options will then be considered by the relevant management authorities for implementation.



You have been chosen to represent people with similar preferences to yourself, so by completing these questions you are providing a voice for like-minded people.

36

Appendix 2: Running Sheet for Focus Groups Incentive Compatibility Discussion RED GROUP: RED consequence version first, followed by GREEN/BLUE alternative wording BLUE GROUP: BLUE Honesty version first, followed by RED consequence wording GREEN GROUP: GREEN provision rule version first, followed by RED consequence wording START TIME: First Version Discussion (Respondents complete questionnaire)

1) What point stood out the most in the instructions for the conservation scenarios? 2) You were presented with 4 different conservation scenarios. As you were answering them, did you find yourself referring back to or thinking about previous ones? (If need more prompting) Did the choices you made in the first couple of questions influence the choices you made in the last couple? 3) In the instructions a cost was described to you as an environmental tax. Did you object to this payment mechanism, or did you consider it as a realistic payment mechanism and take it into account in the conservation scenarios? 4) If you were approached with this survey in the ‘real world’ (i.e. outside of the focus group), would you consider it to be a realistic survey that:  BLUE Honesty : is likely to provide useful information for environmental policy decisions?  GREEN Provision rule: offers conservation options that are likely to be implemented? 5) While answering the conservation scenario questions, were you acting as an individual or trying to make choices based on what you think would benefit the broader community?  If citizen cap is on: When we run one of these surveys we select participants from a variety of backgrounds to try and capture a good cross-section of the community. So we want people to answer these questions based on their own individual preferences. Is there any way that we could re-write the last instruction to make this more clear?

6)

Is there anything else that you didn’t understand in the instructions, for example words or sentences that might need better explanation?

37

Second Version Discussion (Respondents read alternative instruction set)

7) Did you find one of the sets of instructions more believable or realistic than the other? 8) Do you think the alternate wording would change the way you would answer the conservation scenario questions? 9) GREEN Provision Rule session only: In the GREEN Version of the questionnaire, one of the instructions mentioned that only the most popular options would be considered for implementation. The wording of this instruction does not imply that one of the options will definitely be implemented, just that they will be considered. Do you think that people would view the conservation scenarios as being more realistic if we stated that one of the 4 options WILL be implemented, or do you think it is more important to be honest and respect the fact that although these 4 options have been brought to the attention of management authorities, in reality they may in fact choose something different? General Suggestions & Comments: Vote

Preferred RED consequence version: Preferred BLUE/GREEN alternative version:

CONCLUSION TIME:

38

Loading...

ISSN 1835-9728 Environmental Economics - AgEcon Search

ISSN 1835-9728 Environmental Economics Research Hub Research Reports Choice Experiment Framing and Incentive Compatibility: observations from public ...

633KB Sizes 1 Downloads 3 Views

Recommend Documents

Accounting systems and environmental decision - AgEcon Search
3.2 Progress using shareholder and stakeholder value concepts. 3.3 Strategic accounting, and the bridge to finance and e

Explaining Environmental Management System - AgEcon Search
opportunities or limits of environmental management systems (EMSs), it is important to first understand what .... busine

ENVIRONMENTAL ECONOMICS
This course is part of a two-semester sequence in environmental and natural resource economics, requiring .... Some of t

Environmental Economics
For more than a decade, the Environmental Literacy Council has been dedicated to helping teachers, students, policymaker

Project Report - AgEcon Search
The overall objective of the study is to look at the structure of consumer value added meat purchasing behaviour (value

Agris on-line Papers in Economics and Informatics - AgEcon Search
Abstract. The increase in soybean prices is caused by an imbalance between the ability to produce soybeans in the countr

Economics and Politics of Water Resources - AgEcon Search
and supplies water to downstream areas through two main canals known as the Right. Bank Main Canal (RBMC) and the Left B

Puccinia striiformis - AgEcon Search
the Paci ic Northwest region of the United States; after. 2000, stripe rust became prevalent in eastern Canada .... too

child labor - AgEcon Search
complicating any causal interpretation of correlations that can be observed. However, the existing evidence is strong. F

Project Report - AgEcon Search
Potential for Alberta's Pork in Selected U.S. Markets. Michele Veeman and James Unterschultz. Project Report # 00-02. AA