Skip Navigation


ICES Journal of Marine Science: Journal du Conseil Advance Access originally published online on July 25, 2008
ICES Journal of Marine Science: Journal du Conseil 2008 65(8):1449-1455; doi:10.1093/icesjms/fsn124
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
65/8/1449    most recent
fsn124v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Piet, G. J.
Right arrow Articles by Rochet, M.-J.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Piet, G. J.
Right arrow Articles by Rochet, M.-J.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© 2008 International Council for the Exploration of the Sea. Published by Oxford Journals. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org

This article appears in the following ICES Journal of Marine Science issue: Marine Environmental Indicators: Utility in Meeting Regulatory Needs [View the issue table of contents]

Evaluating potential indicators for an ecosystem approach to fishery management in European waters

Gerjan J. Piet1, Henrice M. Jansen1 and Marie-Joëlle Rochet2

1 Wageningen IMARES, PO Box 68, 1970 AB Imuiden, the Netherlands
2 IFREMER, Département Ecologie et Modèles pour l'Halieutique, BP 21105, 44311 Nantes Cedex 03, France

Correspondence to G. J. Piet: tel: +31 255 564699; fax: +31 255 564644; e-mail: gerjan.piet{at}wur.nl

Piet, G. J., Jansen, H. M., and Rochet, M-J. 2008. Evaluating potential indicators for an ecosystem approach to fishery management in European waters. – ICES Journal of Marine Science, 65: 1449–1455.

This study describes the process of evaluating potential indicators for an ecosystem approach to fishery management in European waters by evaluating these indicators against existing criteria using questionnaires completed by experts. We (i) compare the use of a longer list of simple criteria with a shorter list of elaborate ones; (ii) compare evaluation results when screening criteria are applied to specific indicators vs. high-level headline indicators; and (iii) examine whether detailed questionnaires, with elaborate indicators and elaborate criteria, result in ranked scores that are less influenced by familiarity with the indicators. The results show that the ranked scores of indicators are affected by the level of detail, both in terms of criteria and indicators, provided in the questionnaires. It appears that adding detail to the questionnaires makes the scoring process more transparent and provides better founded scores; at a certain point, however, more-detailed indicators and/or more-detailed criteria result in decreased performance of the scoring process, reflecting mostly factors that do not determine the suitability of the indicator (e.g. the level of familiarity), while giving the false impression of a more thorough analysis.

Keywords: ecosystem approach, evaluation criteria, headline indicators, questionnaire

Received 14 December 2007; accepted 25 March 2008; advance access publication 25 July 2008.


   Introduction
Top
 Introduction
Material and methods
 Results
 Discussion and conclusions
 References
 
It is generally agreed that an ecosystem approach to fishery management (EAFM) will necessarily rely on suites of indicators that track the pressure exercised, the state of the ecosystem, and the socio-economic consequences in relation to the management objectives formulated (FAO, 2003; Rice, 2003; Jennings, 2005). Although the number of potential metrics is virtually unlimited, the final suite of indicators needs to provide a "good coverage" of the human activities that require management, as well as of the ecosystem components and attributes affected (Jennings, 2005). This, however, is likely to involve a compromise that would fit the management objectives (Jennings, 2005; Rochet et al., 2007). The selection may be conducted using criteria that ensure that the indicators to be used meet a number of desirable properties (Rice and Rochet, 2005).

The European Union has committed itself to incorporating an ecosystem approach in its Common Fisheries Policy (CFP), and consequently, has stated some high-level objectives (CEC, 2002). To this end, it has funded several research and development projects aimed at establishing lists of potential indicators for fishery management (e.g. FISH/2002/08: Development of preliminary indicators of environmental integration of the Common Fisheries Policy; and INDECO: Development of Indicators of Environmental Performance of the Common Fisheries Policy). Although the process of prioritizing issues and developing indicators for them is ongoing, methods must be included in this process that can screen potential indicators for their appropriateness. Exercises applying screening criteria to actual lists of indicators should help to identify the potential shortcomings and advantages of different approaches.

Rochet and Rice (2005) tested a framework for indicator selection. In their experiment, a set of 20 candidate indicators that were supposed to have general applicability was evaluated by 16 experts, each familiar with one of four different ecosystems around the world. This exercise indicated that various steps involved in the selection process were prone to subjective value judgement, and that differences in scores assigned by the experts were the main cause of variability in the evaluation results for different ecosystems; whether or not the experts were familiar with a particular one had little influence. Rochet and Rice (2005) conclude that understanding the reasons underlying individual preferences for specific indicators fosters dialogue, by helping to clarify the debate. They also suggest that the selection process might be easier if a longer list of simple criteria, as provided by Rice and Rochet (2005), is used, as opposed to a shorter list of more complex ones.

The issue of complexity is important if such evaluation procedures are to be carried out, not only by experts but also as part of a wider stakeholder consultation. In the Rochet and Rice (2005) experiment, the framework for selecting indicators proved useful because it gave experts the opportunity to present their values explicitly. However, what level of simplification and/or detail needs to be applied to the criteria to help clarify the debate without adding confusion? Further, how many indicators, and at what level of specificity, can be put forward for evaluation by a particular group of stakeholders?

Within the INDECO project, we conducted another screening test that is directed more specifically towards its use in the process of selecting indicators for an EAFM within the CFP. The aim was to (i) compare the results obtained with a long list of simple criteria with a shorter list of more elaborate ones; (ii) evaluate the effect of applying screening criteria to, so-called, specific indicators vs. higher-level, so-called, headline indicators (Jennings 2005); and (iii) examine whether or not familiarity of the experts with a particular indicator influences their evaluation of the importance of that indicator.

The evaluation was based on the framework developed by Rice and Rochet (2005), which has eight steps: (1) determining user needs; (2) listing candidate indicators; (3) determining screening criteria; (4) scoring indicators against criteria; (5) summarizing scoring results; (6) deciding how many indicators are needed; (7) final selection; and (8) reporting. We restricted ourselves to Steps 2–5 because user needs (Step 1) are essentially given by the objectives specified in the revised CFP (Regulation 2371/2002: "to ensure the long-term viability of the fishery sector through sustainable exploitation of living aquatic resources, based on sound scientific advice and on the precautionary approach"), whereas Steps 6 and 7 were considered to be outside the scope of this exercise.


   Material and methods
Top
 Introduction
 Material and methods
Results
 Discussion and conclusions
 References
 
The evaluation process differed from the Rice and Rochet (2005) framework in that each respondent was asked to complete several different questionnaires corresponding to eight scenarios. We aimed at having all experts complete all questionnaires; however, for practical reasons, this could not be achieved, and the actual numbers are reported in Table 1. These not only provide an evaluation of the indicators but also allow us to test a number of hypotheses on potential factors affecting the performance of the process. The respondents included 24 experts from 20 research organizations spread over 11 EU Member States, with expertise on four marine ecosystems corresponding to the regional advisory council (RAC) areas: Baltic Sea, North Sea, Bay of Biscay (representative of southwestern waters), and the Mediterranean. In addition, we had four responses from non-scientist stakeholders, but we excluded them from the analyses to avoid bias.


View this table:
[in this window]
[in a new window]

  		Table 1. Coding of the eight questionnaires that were put to respondents to provide scores for two types of indicators (H: headline; S: specific) and weighting factors (W) for four types of value judgements (C: criteria, S: subcriteria; N: none; F: familiarity).

 
For Step 2, we followed Jennings (2005) and identified a list of candidate indicators for both state and pressure, based on a literature review. The state indicators were supposed to cover the entire ecosystem with all its different components and attributes. Given the focus of the user needs as provided by the CFP objectives, the pressure indicators only covered fishing. Starting with six broad issues related to the EAFM, we examined the effect of detail in indicator specificity by introducing a hierarchy of indicators, ranging from overall features (headline indicator) to the actual metric (specific indicator). If no specific indicator had been developed for a particular feature, we used a more general phrasing.

In Step 3, we used the list of criteria and subcriteria proposed by Rice and Rochet (2005). To examine the influence of detailing criteria on the evaluation outcome, either the shorter list of nine main criteria (concreteness, theoretical basis, public awareness, cost, measurement, availability of historical data, sensitivity, responsiveness, and specificity) or the full list of 33 subcriteria (3, 3, 5, 1, 11, 5, 1, 1, and 3 subcriteria, respectively; for details, see Table 2 in Rice and Rochet, 2005) was used.


View this table:
[in this window]
[in a new window]

  		Table 2. Mean scoring (with rank order in parentheses) for headline indicators (with their short names as used in Figure 2a given in parentheses), based on three different questionnaires (for explanation code see Table 1); scores range from 1 (worst) to 5 (best).

 
In Step 4, the indicators were scored against the criteria by the respondents for eight different scenarios allowing: (i) an evaluation of either headline (HN) or specific indicators (SN) without explicit criteria; (ii) an evaluation of headline indicators (HC) based only on the main criteria; (iii) an evaluation of specific indicators (SS) based only on the subcriteria; (iv) an evaluation of the familiarity of the respondents with each headline (HF) and specific (SF) indicator (Table 1).

The scoring had two components: an evaluation of the quality of each indicator relative to each criterion (indicator scoring) and an evaluation of the relative importance of each criterion (criteria weighting). An ordinal scoring of five ranks was used to evaluate the performance of each headline indicator against each criterion (1=worst, 5=best), as well as for the weighting of the criteria (1=less important, 5=very important). Weighting of the subcriteria was done on a relative scale so that the weights of the subcriteria for each criterion sum to 1. For the familiarity scoring, an ordinal scoring of three ranks (1=least familiar, 3=most familiar) was used.

To summarize scoring results (Step 5), we created a table in which a mean score for each indicator (H: headline, or S: specific) was derived. Depending on the scenario, these means were calculated differently: for evaluations without explicit criteria (HN and SN), the mean score by indicator was calculated as the mean across all responses. For evaluation involving (sub)criteria (SS), the scores by (sub)criterion given by each respondent for each indicator were first weighted by the weighting scores given by the same respondent to derive one indicator score per response, then the mean was calculated across all responses. For evaluations based on specific indicators, we derived a score for headline indicators based on the mean score (SN and SS) of the corresponding specific indicators. For interpretation of the results, we also had access to a scoring of the familiarity of the different respondents with each indicator. Overall, this resulted in six scenarios (HN, SN, HC, SS, HF, and SF; Table 1) providing scorings of indicators. Spearman's rank-order correlations (S) were used to investigate the following hypotheses:

  1. There is no difference between the ranked scores of specific and headline indicators (S expected to be 1).
  2. There is no difference between scores using main criteria vs. subcriteria (S expected to be 1).
  3. Longer, more detailed, and thus more straightforward questionnaires with specific indicators and subcriteria result in ranked scores that are less affected by familiarity than shorter and less elaborate questionnaires (S between ranked indicator scores and ranked familiarity scores expected to decrease as the level of detail increases).

No formal test has been developed for Spearman rank correlation being different from 1, so p-values cannot be provided for the first two tests. Instead, rank correlation is taken as a measure of agreement among the experts’ rankings, and we performed a one-tailed test for positive correlation to determine whether or not the agreement between two rankings was significantly larger than 0 (Conover, 1971).


   Results
Top
 Introduction
 Material and methods
 Results
Discussion and conclusions
 References
 
Weights of criteria
Ranking of the criteria weights, based directly on the main criteria (WC), showed that on average concreteness, public awareness, and cost received the lowest weights (Figure 1). Results based on the subcriteria (WS) showed a somewhat similar pattern, although the ranks differed slightly, with higher concreteness weights and lower sensitivity weights. Theoretical basis, specificity, responsiveness, and specificity got fewer different scores based on WS than on WC (e.g. sensitivity was scored 4 or 5 in WS by 12 respondents, but 3, 4, or 5 in WC by 8 respondents).


Figure 1
View larger version (14K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 1. Frequency distribution of weights given by all respondents against main criteria WC (left) and against subcriteria WS (right). Bubble area is proportional to the frequency of allocation of each weight, and circles of radius zero are plotted as dots. Criteria are ordered by increasing the sum of weights obtained in the WC questionnaire.

 
Indicator scoring
The scorings and ranks for headline indicators are given in Table 2 and Figure 2a, and for specific indicators in Table 3 and Figure 2b. Overall, indicators associated with traditional fishery management (e.g. various fishing-pressure indicators and status of commercial stocks) scored highest, whereas indicators of ecosystem functioning and plankton scored lowest.


Figure 2
View larger version (25K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 2. Frequency distribution of scores, ranked by average score, given by all experts to (a) headline indicators (HC evaluation) and (b) specific indicators (SS evaluation). Bubble radius is proportional to the frequency of each score (for abbreviations of indicator names see Tables 2 and 3, respectively).

 


View this table:
[in this window]
[in a new window]

  		Table 3. Mean specific indicator scoring (with rank order in parentheses) for specific indicators (with their short names as used in Figure 2b given in parentheses), based on three different questionnaires (for explanation code see Table 1); scores range from 1 (worst) to 5 (best).

 
Hypothesis testing
Spearman rank-order correlation shows that some scenarios had non-significantly positive correlations and, thus, that experts were not always consistent in their evaluation of the indicators (Table 4). For example, the scoring of specific indicators with subcriteria, or of headline indicators with criteria, was not significantly correlated with the scoring of both specific and headline indicators evaluated without explicit criteria (Table 4). More specifically:


View this table:
[in this window]
[in a new window]

  		Table 4. Similarity in rank order of scores for headline and specific indicators, based on different scoring scenarios using Spearman correlation coefficients (for explanation code see Table 1).

 
(i) The hypothesis that there is no difference in the ranked scores of headline indicators when the scoring is done for the headline indicators vs. a scoring of specific indicators was investigated by comparing the scores of the headline indicators without using criteria (HN) with the scores calculated by taking the mean of the specific indicators. The test was based on answers from the same 15 respondents. SHN/SN did not meet the value of 1, expected if the experts would have been perfectly consistent and the level of detail would not influence the ranking of the headline indicators, but there was a good correlation between these rankings (SHN/SN = 0.82; p < 0.001; Table 4).

(ii) The hypothesis that there is no difference in ranked scores based on main criteria only vs. subcriteria was investigated by comparing the scores of the headline indicators based on main criteria (HC) with the scores based on subcriteria and calculated by taking the mean of the specific indicators. The test was based on answers from 11 respondents to the HC and SS questionnaires (Table 1), of which more than half (6) were completed by the same respondents. Again, SHC/SS < 1 suggests that the level of detail of the criteria resulted in a different ranking of the headline indicators, although the positive correlation (0.88; p < 0.001, Table 4) indicates some consistency.

(iii) The hypothesis that longer questionnaires with more concrete indicators and criteria result in ranked scores that are less influenced by familiarity was tested through two comparisons.

Scores of headline indicators not based on criteria (HN) and those based on main criteria only (HC) were compared with the familiarity score (HF). The comparison HN/HF was based on the same 15 respondents, whereas the HC/HF questionnaires had only five respondents in common. These comparisons showed that the use of criteria resulted in scores that were correlated less with the familiarity scores (SHN/HF = 0.78 vs. SHC/HF = 0.69; Table 4). A striking result was that scores of specific indicators not based on criteria (SN) and based on the subcriteria (SS) compared with the familiarity score (SF) showed that a further increase in detail (i.e. specific indicators as opposed to headline indicators, and subcriteria as opposed to main criteria) resulted in scores that were more similar to the familiarity scores (SSN/SF = 0.58 vs. SSS/SF = 0.67; Table 4). Finally, a general observation was that familiarity scoring (HF and SF) is highly correlated with all indicator scoring, whatever the method used (Table 4).


   Discussion and conclusions
Top
 Introduction
 Material and methods
 Results
 Discussion and conclusions
References
 
This study found that, depending on how the question is posed (i.e. different questionnaires, with headline indicators or specific indicators, whether criteria and/or subcriteria are used), the ranking of the indicators will differ, independent of whether or not they are completed by the same experts.

For the ranked scores of headline indicators vs. specific indicators, this difference may come from the difficulties in the translation from one to the other, owing to inherent differences in scores between the specific indicators representing the same headline indicator. For instance, for the headline indicator "physical environment", which is in itself an abstract concept, two specific indicators [temperature and North Atlantic Oscillation (NAO)] were applied. In terms of concreteness, the appropriate score for temperature should be higher than for NAO, the latter being a concept at a higher level of abstraction. Another example is the "abundance of commercial stocks". The headline indicator sounds concrete (score 4 or 5), is known to the public (score 4 or 5), and is likely to be tightly linked to fishing (score 4 or 5). However, the specific indicator "proportion of commercial stocks that are within safe biological limits" is based on elaborate assessment models (concreteness score 1 or 2). Therefore, public awareness may be less (what are safe biological limits?), and the link with fishing activity may be lower (score 2 or 3). Within the headline indicator "status of marine mammals", differences in scoring against the criterion "historical data" or "measurement" may be considerable. For the specific indicator "seal population in the Wadden Sea", the score should be high because of the wealth of information available, whereas the "North Sea porpoise population" should give a much lower score because of a very restricted dataset. So how should one score the headline indicator while being aware of the great differences between specific indicators?

Similarly, the scoring results differed when using subcriteria vs. main criteria, but these differences levelled off across experts and criteria, and the final rankings were highly correlated.

If divergence from familiarity is an appropriate way to assess the performance of the evaluation process, then the results indicate that, for the evaluation of a relatively few headline indicators, a longer list of simpler selection criteria indeed improves the process as Rice and Rochet (2005) suggest. However, the opposite trend is observed when an extensive list of many specific indicators is applied. This suggests that there is a point on the gradient, from short lists of complex criteria and headline indicators to long lists of simple criteria and specific indicators, where the performance of the evaluation process starts to decline.

The strong correlations between familiarity and any of the scorings suggest that, to some degree, experts will give higher scores to indicators with which they are more familiar, and/or lower scores to indicators they do not know well. Experts may not have sufficient information (Rochet and Rice, 2005) and, if this information is lacking, the level of familiarity may largely determine the outcome of the scoring. Among the non-scientists, several declined to complete the questionnaires, because they did not consider themselves sufficiently informed to score the indicators against criteria. Thus, for non-scientist stakeholders, the effect of familiarity may be even more applicable.

Clearly, the process of indicator selection for an EAFM in the EU should involve enough expert respondents from different stakeholder groups and nationalities to guarantee commitment to the evolving suite of indicators. Whereas scoring is a convenient aid to summarizing the evaluations by different people, it may not be necessary to score indicators against criteria in the actual selection process. An indicator might barely pass or fail against each criterion, or might be evaluated more qualitatively with pros and cons, whereas the final selection could be the result of negotiation rather than numerical scoring. As all scientific activity needs to be balanced against the resources available, our experience has been that asking a large group of respondents to go through extensive questionnaires may not be the best way to use these resources.

The results of this evaluation and the concerns expressed so far may be discussed against the background of the eight-step evaluation framework of Rice and Rochet (2005).

  1. Determining user needs. The current objectives of the CFP are not specific enough to allow a proper scoring of the indicators, because they are restricted to commercial stocks. Specific operational objectives for other ecosystem components need to be formulated at the appropriate scale, e.g. according to Jennings’ (2005) framework, which includes an additional step to identify those activities that are most likely to compromise the broad objectives currently formulated.
  2. Listing candidate indicators. Because too many indicators will aggravate the evaluation process, we advise starting with a limited suite of indicators. Concrete indicators have been developed for some ecosystem features, whereas none exist for others. We addressed this problem by distinguishing two hierarchical levels of indicators: headline indicators and specific indicators. Although this distinction was intended to resolve discrepancies between the types of indicator available, the feedback of (notably the non-scientific) respondents showed that, for an evaluation by different stakeholders, it may be more appropriate to have them evaluate headline indicators, because specific indicators are often meaningless to them and could obfuscate the evaluation. The evaluation and selection of specific indicators for a particular headline indicator should be done by individuals that are sufficiently familiar with the indicators’ merits. This may be determined by providing respondents with all the relevant information before the actual scoring.
  3. Determining screening criteria. Although the respondents considered the criteria and subcriteria appropriate, the use of subcriteria did not affect or improve the scoring to any large extent. The obvious requirement is that the level of detail in the criteria should balance the level of scientific information available, and hence the expertise of the respondents. Adding subcriteria with increasingly subtle differences is expected to hamper the scoring process as soon as the evaluation requires more expertise than the respondent group possesses.
  4. Scoring indicators against criteria. Using explicit criteria can make the scoring process more transparent, and moving from short lists of complex criteria to longer lists of simpler subcriteria is expected to provide better-founded scores (Rochet and Rice, 2005), if the level of detail is tuned to the level of the respondents’ expertise. If this is not the case, extended questionnaires, scoring many specific indicators against subcriteria with an elaboration of their respective weighting factors, do not improve the evaluation process and imply that the exercise was more thorough than it actually was. Making all relevant information available before the scoring and allowing an exchange of viewpoints should reduce variation and bias, and hence result in the scores converging. Following this logic, the implication is that, if all available information is discussed within the group of respondents, then more subcriteria can be used in the evaluation.
  5. Summarizing scoring results. We observed marked differences in the weighting of the different (sub)criteria between individual scientists. It may be assumed that these differences will increase when more stakeholder groups are involved in the process (e.g. NGOs are more likely to give the highest weights to public awareness, managers to responsiveness, and politicians to costs). Specifying the weightings of the different criteria given by each stakeholder group may facilitate the process of selecting indicators by making it more transparent, but how these weightings should be applied in obtaining the final scoring, and thus the preferred indicators, needs to be resolved. Instead of the most obvious choice of preferring the indicators with the highest average value, an alternative approach could be to select indicators that meet some minimum level of acceptance for all criteria.
  6. Deciding how many indicators are needed. Several considerations determine the choice of the number of selected indicators. The first choice is that we need indicators for both state and pressure (Jennings, 2005). A minimum requirement for the ecosystem state indicators would be that, for each ecosystem component and attribute for which operational objectives are formulated, at least one headline indicator with a specific indicator is selected. This minimum selection may be expanded by also including indicators that are not necessarily affected by the fishery themselves but that should be considered in the management of the core ecosystem components (e.g. environmental indicators). Finally, there is the choice of having more than one specific indicator for one or more of the headline indicators. Again, this should be determined by how much additional information this new specific indicator provides. In the end, however, the number of indicators that are selected and how they are combined will not be determined only on scientific grounds but also by the requirements of the manager who needs to work with them or the costs involved in collecting the necessary data.
  7. Final selection. The information needed to guide the final selection of indicators can be derived from the scoring of indicators against screening criteria, assuming that the shortfalls mentioned previously are resolved. A possible refinement of the approach could be to conduct this in two stages: the first stage involving different stakeholders where headline indicators are scored against (a subset of) the criteria, and weightings of the criteria per stakeholder group are identified; and a second stage involving a more restricted group where, for each headline indicator, one or more specific indicators are evaluated against (a more detailed or extended set of) screening criteria.
  8. Reporting. This study has not provided any relevant information for the reporting process.


   Acknowledgements
 
The work was funded by the European Commission through two projects: Development of Indicators of Environmental Performance of the Common Fisheries Policy (INDECO, FP6–513754), and Indicators for fisheries management in Europe (IMAGE, FP6–044227). We thank in particular our respondents, without whose perseverance to go through the questionnaires this evaluation would not have been possible: E. Andrulewicz, M. Appelberg, R. Aps, A. Borja, J. Brown, F. Colloca, N. Daan, O. Giovanardi, S. Greenstreet, M. Gristina, S. Jennings, S. Libralato, I. Lutchman, E. Meeuwsen, H. Ojaveer, P. Orr, M. Pommarede, F. Pranovi, P. Pelusi, S. Raicevich, M. Romanelli, N. Streftaris, S. Sverdrup-Jensen, M. Tasker, P. Tomasik, and D. Wilson.


   References
Top
 Introduction
 Material and methods
 Results
 Discussion and conclusions
 References
 

    CEC. Council Regulation 2371/2002 of 20 December 2002 on the conservation and sustainable exploitation of fisheries under the Common Fisheries Policy. Official Journal of the European Communities (2002) OJ L358/59, 31 December 2002.

    Conover W. J. Practical Nonparametric Statistics. (1971) New York: John Wiley and Sons. 493.

    FAO. The Ecosystem Approach to Fisheries. (2003) 4(Suppl. 2). FAO, Rome: FAO Technical Guidelines for Responsible Fisheries. 112.

    Jennings S. Indicators to support an ecosystem approach to fisheries. Fish and Fisheries (2005) 6:212–232.[CrossRef][Web of Science]

    Rice J. Environmental health indicators. Ocean and Coastal Management (2003) 46:235–259.[CrossRef]

    Rice J. C., Rochet J. A framework for selecting a suite of indicators for fisheries management. ICES Journal of Marine Science (2005) 62:516–527.[Abstract/Free Full Text]

    Rochet M-J., Rice J. C. Do explicit criteria help in selecting indicators for ecosystem-based fisheries management? ICES Journal of Marine Science (2005) 62:528–539.[Abstract/Free Full Text]

    Rochet M-J., Trenkel V. M., Forest A., Lorance P., Mesnil B. How could indicators be used in an ecosystem approach to fisheries management? (2007) ICES Document CM 2007/R: 05. 15.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
65/8/1449    most recent
fsn124v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Piet, G. J.
Right arrow Articles by Rochet, M.-J.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Piet, G. J.
Right arrow Articles by Rochet, M.-J.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?