© 2004 by ICES/CIEM International Council for the Exploration of the Sea/Conseil International pour l'Exploration de la Mer
Performance of precautionary reference points in providing management advice on North Sea fish stocks
a Animal Sciences Group, Wageningen UR, Netherlands Institute for Fisheries Research PO Box 68, 1970 AB, IJmuiden, The Netherlands
b Canadian Science Advisory Secretariat Fisheries and Oceans Canada, 200 Kent Street, Stn 12S015, Ottawa, Ontario K1A OE6, Canada
*Correspondence to G. J. Piet: tel: +31 255 564699; fax: +31 255 564644. e-mail: gerjan.piet{at}wur.nl.
For 17 stocks in the North Sea, the performance and effectiveness of management advice using precautionary reference points was evaluated. Three criteria were used to identify whether a stock was within safe biological limits: SSB < Bpa, F > Fpa, or SSB < Bpa and F > Fpa. Four scenarios were considered, comparing the advice in the assessment year with what is retrospectively (2002 assessment) known to be the status of the stock at that time: (1) stock outside safe biological limits, advice to reduce fishing; (2) stock outside safe biological limits, advice for status quo harvesting; (3) stock within safe biological limits, advice to reduce fishing; and (4) stock within safe biological limits, advice for status quo (or increased) harvesting. Signal Detection Theory was applied to these scenarios, and the proportion of Hits (1 and 4), Misses (2), and False Alarms (3) were determined for each year as the proportion of the stocks for which the respective scenarios applied. Using both Bpa and Fpa was deemed the approach with the lowest error rate, and it resulted in about the same proportion of Hits in management advice as when Bpa alone was used (62%), but the proportion of Misses was slightly lower (24% vs. 26%). Therefore, the suggested EcoQ element would be the proportion of commercial fish stocks within safe biological limits (i.e. SSB > Bpa, F < Fpa), and the Ecological Quality Objective (EcoQO) should be that this EcoQ should be at or above a desired level. This desired level is a societal/political decision relative to the EcoQ reference level (i.e. where the anthropogenic influence on the ecological system is minimal), which by definition is 100%. At present, probably <10% of North Sea fish stocks are within safe biological limits, despite the relatively high Hit rate of >60%. A possible explanation is that most of these stocks (e.g. flatfish and roundfish) are caught in a mixed fishery, for which TAC management is less effective.
Keywords: EcoQO framework, stock assessment, TAC management
Received 20 December 2003; accepted 14 August 2004.
 |
Introduction |
|---|
 Top Introduction Material and methodsResultsDiscussionReferences |
|---|
Until recently, ecosystem considerations or biological interactions between species played only a minor role in setting objectives and decision-making in fisheries management. The development of an ecosystem approach to management of the North Sea (Anon., 1997) required the formulation of clear objectives both at a general level, as overall or integrated objectives, and at a specific level, as more detailed and operational objectives (FAO, 2003). Lanters et al. (1999) describe a general methodology for evaluating Ecological Quality (EcoQ) and setting of Ecological Quality Objectives (EcoQOs). As part of this framework, ten issues and their related elements have been proposed, covering the structural and functional aspects of the ecosystem at different hierarchical levels: ecosystem, communities, habitats, and species (Anon., 2002). At a species level, the "spawning-stock biomass of commercial fish species" was identified as an Ecological Quality element. The associated EcoQO is that the stock biomass should be "above precautionary reference points for commercial fish species where these have been agreed by the competent authority for fisheries management". The relevant precautionary reference points are those for "spawning-stock biomass, also taking into account fishing mortality, used in advice given by ICES in relation to fisheries management". ICES has established Bpa and Fpa as the respective precautionary reference points for spawning-stock biomass (SSB) and fishing mortality (F) for use in formulating advice (sensu FAO, 1996a, b). They are set on a stock-specific basis, and take account of stock dynamics as well as uncertainties in the assessment. Bpa is the level at and above which there is a low probability that true SSB is so low that productivity is impaired. Fpa is the mortality level at and below which true fishing mortality has a low probability of leading to stock collapse. ICES reiterates annually in its advice that precautionary reference points should not be used as targets. Rather they are to be used as "upper bounds on fishing mortality and lower bounds on biomass", a phrase repeated in the advice of several North Sea stocks (e.g. herring, plaice).
For this study we evaluated the performance of fisheries management advice using precautionary reference points. The aim was to formulate an EcoQO for exploited North Sea stocks in the sense expressed in the Bergen Declaration, based on the precautionary approach for fisheries management. To do this, we looked at the historical performance of this EcoQO, and tested the effectiveness of management advice for the conservation of the stocks. Advantage was taken of the convergence property of most analytical assessments. In other words, although there is substantial uncertainty in the estimates of SSB and F in the assessment year (year i), uncertainty in those years about SSB and F in year i is reduced as more catch and survey data accumulate in years i + 1, i + 2, etc., so estimates become more consistent.
|
Material and methods |
|---|
|
TopIntroductionMaterial and methodsResultsDiscussionReferences |
|---|
For EcoQOs relating to SSB or F, the appropriate source of information for North Sea stocks is the regular assessments by the ICES Working Groups reporting to ACFM. The "North Sea" is here interpreted as the greater North Sea area, as defined by the Oslo–Paris (OSPAR) Convention (Figure 1). This includes the whole of ICES Area IV (the geographic North Sea), IIIa, b (the Skagerrak and Kattegat), VIId, e (Eastern and Western Channel), and part of VIa (North and West of Scotland). Of the stocks that fall within the OSPAR-defined North Sea area and for which ICES provides quantitative scientific analysis (Table 1), stocks were excluded if:
- • they are not assessed, and estimates of SSB are not available, even though they may be fished commercially;
- • they are Kattegat stocks that are assessed along with stocks in the Baltic;
- • they are North of Scotland stocks, and the assessment unit was Area VI as a whole;
- • they are Western Channel stocks that are assessed along with fish in the Celtic Sea or Bay of Biscay;
- • no precautionary levels of SSB or F have been identified [herring in Divisions VIa (South) and VIIb, c];
- • the state of the stock was not assessed in 2002 [North Sea horse mackerel in Division IIIa (eastern part) and Divisions IVb, c, and VIId; cod in Division VIb];
- • no total allowable catch (TAC) is set for management (sprat in Division IV).
|
|
To evaluate the performance of fisheries management advice using precautionary reference points, we used three criteria to determine whether a stock was within safe biological limits, and hence that the EcoQO was met:
- • SSB was above the precautionary reference point (SSB > Bpa);
- • F was below the precautionary reference point (F < Fpa);
- • both the above (SSB > Bpa and F < Fpa).
For each stock the evaluation of the actual annual management advice and actions, as tabulated in the Catch Data and assessment output tables from the ICES ACFM 2002 report (ICES, 2002) was based on four scenarios (see Table 2):
- Stock does not meet EcoQO; advice to reduce fishing: the estimate of SSB and/or F in the assessment year led to advice to reduce catch when the converged estimate of SSB and/or F in the 2002 assessment now indicates that the Ecological Quality (EcoQ) of the stock did not meet its objective (EcoQO), i.e., respectively, SSB < Bpa, F > Fpa, or SSB < Bpa and F > Fpa. A Hit.
- Stock does not meet EcoQO; advice for status quo (or increased) fishing: the estimate of SSB and/or F in the assessment year led to advice for status quo or increased TAC when the converged estimate of SSB and/or F in the 2002 assessment now indicates that the stock did not meet its objective (EcoQO). A Miss.
- Stock meets EcoQO; advice to reduce fishing: the estimate of SSB and/or F in the assessment year led to advice to reduce catch when the converged estimate of SSB and/or F in the 2002 assessment now indicates that the stock met its objective (EcoQO). A False Alarm.
- Stock meets EcoQO; advice for status quo or increased fishing: the estimate of SSB and/or F in the assessment year led to advice for status quo or increased TAC when the converged estimate of SSB and/or F in the 2002 assessment now indicates that the stock met its objective (EcoQO). A Hit.
|
Signal Detection Theory (Helstrom, 1968) was applied to these scenarios, and the proportion of Hits (1 and 4), Misses (2), and False Alarms (3) was determined for each year as the proportion of the stocks for which the respective scenarios applied. This well-established theoretical framework has a direct link with practical management.
Scenarios (Hits) 1 and 4 (Table 2) are considered "correct advice", in that the advice given in the assessment year (and presumably the action taken on the basis of the advice) is consistent with the advice appropriate to the converged estimate of SSB and/or F, which in turn determines the EcoQO status. For example, the advice in the assessment year was to reduce the TAC, and according to the 2002 assessment, the stock was outside safe biological limits in the assessment year, and hence the EcoQO was not met. Alternatively, the advice in the assessment year was a status quo or increased TAC, and according to the 2002 assessment, the stock was within safe biological limits in the assessment year, and hence the EcoQO was met.
Conversely, scenarios 2 and 3 can be considered "incorrect advice", in that the advice given in the assessment year (and presumably the action taken on the basis of the advice) is inconsistent with the advice appropriate to the converged estimate of SSB and/or F in the assessment year.
|
Results |
|---|
|
TopIntroductionMaterial and methodsResultsDiscussionReferences |
|---|
The quality of advice varies depending on the type of fishery and the criteria used (Table 3). Overall, using just F will result in relatively low rates of False Alarm, but high Miss rates. Using SSB only results in a strong decrease in the rate of Misses, together with a markedly higher proportion of False Alarms. The best results were achieved using both criteria, with an overall 62% Hit rate (35 + 27%), 14% Miss rate, and 24% rate of False Alarm. For the pelagic and industrial fisheries, there appears to be a significant (p < 0.05) relationship between the accuracy of advice and the status of the stock; if the stock does not meet its EcoQO, the proportion of correct advice is smaller than when the stock does meet its EcoQO. This is particularly apparent when only SSB was used as a criterion.
|
2 test on the hypothesis that there is no relationship between the accuracy of advice and the status of the stock.Table 4 gives a quantitative indication of the true impact of the advice depending on the scenario, not just that advice was provided, but how management actually responded to the advice and the indicator. These management actions may reflect the impact of how the advice is phrased, particularly how sources and magnitudes of uncertainties are expressed. For example, if Misses are largely the result of highly uncertain estimates of SSB and F, the enthusiasm for an increased quota might be less compared with the case if the advice for an increased harvest came from a very solid assessment, with a true positive Hit.
|
The results also show, in general, that the advice was appropriate with regard to consequences. If the EcoQO was not met, a strong reduction in TAC of about 18% was suggested in the case of correct advice (a true negative Hit), whereas there was a smaller increase in TAC averaging between 10% (SSB only) and 12% (F only) in the case of a Miss. If the EcoQO was met, correct advice (a true positive Hit) resulted in a suggested increase of the TAC of between 7% (F) and 12% (SSB and F), whereas in the case of a False Alarm, the suggested decrease in TAC was between 13% (SSB) and 18% (F). Overall, the advice using SSB only, or SSB and F, appears more appropriate, with relatively small changes to the TAC in the case of a Miss or a False Alarm, but relatively bigger changes to the TAC in the case of a Hit.
There are also marked quantitative differences in advice between the types of fishery. Managers appear generally to apply smaller TAC reductions to flatfish, but relatively large reductions to pelagic and roundfish stocks. In cases where an increase is advised, managers appear generally to apply smaller relative increases to industrial and pelagic stocks, and larger increases to flatfish and roundfish stocks. Although this difference between the types of fisheries appears largely independent of the status of the stock (i.e. it applies to Hits, as well as to False Alarms or Misses), the suggested decreases are always larger when the EcoQO is not met, and the increases are larger if the EcoQO is met. Management actions are stronger when the advice, in retrospect, was in the rightful direction, suggesting perhaps that there is some shading to the text of the advice corresponding to how reliable the advice later turns out to be.
Historical trajectories of the performance of management advice showed no clear trends in Hits when based on F only (Figure 2a). However, when based on SSB only, or SSB and F together, the proportion of Hits appeared to have been lower in the mid-1990s (Figure 2a), whereas Misses (Figure 2b) were particularly common. Independent of the criteria used, the number of Hits seems to increase while the number of Misses decreases from the mid-1990s. Because the assessments have not converged enough to be confident what the "true" (at least stable) estimates of F and SSB in 2002 will be, that year was not included in the analysis. The historical trajectories of the suggested EcoQ metrics for the commercial species are shown in Figure 3. In recent years, SSB was above Bpa for about 40% of the stocks, and F was below Fpa for <20% of the stocks. The percentage of stocks that meet both criteria is consistently (except for the last year) below 10%.
|
|
Figure 4 shows the relationship between the history of management advice and the relative deviation of SSB in 2002 from precautionary levels. In several cases (all stocks, as well as flatfish and roundfish separately, when SSB, or SSB and F were used as criteria), the relationship was significant (Table 5). Remarkably, this relationship indicates that stocks that in 2002 were below precautionary levels have a history of relatively many Hits. Many of these consistent Hits are true negatives, where the stock has been outside safe biological limits perhaps for several years, and ICES has been correctly advising reductions in quotas. In contrast, stocks that were above precautionary reference levels in 2002 have a history of many Misses and False Alarms, suggesting that relatively few stocks are kept consistently within safe biological limits. The relationship with scenario 4 (Hit when the stock was within safe biological limits) was the weakest, and (independent of the criterion used) not significant at p < 0.1 (Table 5).
|
|
0.05; italics p < 0.1) of the linear relationship between the relative deviation of SSB from SSBpa (SSB2002 – SSBpa)/SSBpa, and the proportion of the respective advice scenarios. |
Discussion |
|---|
|
TopIntroductionMaterial and methodsResultsDiscussionReferences |
|---|
If the results of the evaluation of the performance of management advice show a high Hit rate, but low rates of Misses and False Alarms, they support the view that precautionary reference points are a robust basis for fisheries management advice. Managers are then generally advised to take actions that would move the stock in the proper direction. High Miss rates would suggest that precautionary reference points, as currently used, do not lead to advice that is sufficiently restrictive to ensure that stocks remain within safe biological limits. High rates of False Alarm indicate that precautionary reference points, as currently used, lead to overly intrusive management advice.
The results show that the use of Bpa, rather than Fpa, results in more Hits and less Misses. However, applying both criteria rather than either of them separately, results in about an equal number of Hits, but a lower number of Misses, so can be considered the most consistent with sound management. Based on these results, it would seem appropriate to redefine the EcoQO. Rather than stating that the "spawning-stock biomass of commercial fish species" should be "above precautionary reference points...", where the reference points are "those for the spawning-stock biomass also taking into account fishing mortality,...", the EcoQO should be based explicitly on the proportion of stocks within safe biological limits, where SSB > Bpa and F < Fpa are considered together. Moreover, as all existing management approaches for individual stocks are based on the objective of exploiting the stocks within safe biological limits (i.e. SSB above Bpa, F sustainable), the wording of the EcoQO could be changed so that it would simply condense this information into a form that gives an appropriate overview of the overall status of commercial North Sea fish stocks. Thus, the suggested EcoQ element would be the "proportion of commercial fish stocks within safe biological limits (i.e. SSB > Bpa and F < Fpa)", and the objective (EcoQO) should be that this EcoQ should be at or above a desired level. The desired level is a societal/political decision relative to the EcoQ reference level (i.e. where the anthropogenic influence on the ecological system is minimal), which by definition is 100%. On biological grounds, however, there is no reason why all stocks could not be kept within safe biological limits (avoiding SSBs so low that recruitment is impaired), even though biological interactions among stocks may make it impossible to keep all stocks at biomasses producing maximum single-species yields simultaneously.
Care must be taken in interpreting this paired EcoQO for two reasons. In the past, some stocks dropped out of the assessment system when they fell to very low biomass (e.g. North Sea mackerel). Also, several commercial stocks were depleted to a fraction of their former abundance (e.g. spurdog Squalus acanthias, thornback ray Raja clavata), but are not assessed regularly by ICES. As the value of the metric depends on the stocks included in estimating the percentages, rigorous criteria should be used to determine the stocks that should be included, and this list should be clearly stated when using the EcoQO.
When evaluating the historical performance of this EcoQ element, the 2002 results were not included in the analysis, because the assessments had not converged enough to be confident what the true estimates of SSB and F were in that year. This argument, however, may apply to the last few years. As such, the increase of at least the EcoQ element in 2001 (Figure 3) should be viewed with caution. This means that the proportion of stocks that are within safe biological limits is probably <10%.
Our evaluation of Bpa and Fpa as reference points shows that use of Fpa separately results in a Miss rate of 46%, and a False Alarm rate of 4%, suggesting that F alone is a dangerous basis for management advice, at least where stock-specific reference points for F are positioned at present. Use of Bpa separately results in rates of 26% (Miss) and 12% (False Alarm), while use of both Bpa and Fpa results in rates of 24% and 14%, respectively. This indicates that, as advice becomes more precautionary, Miss rates decrease, but False Alarm rates increase. Moving from the context of fisheries management to EcoQOs, it would be desirable (from the perspective of conservation) to reduce Miss rates. This can be achieved by selecting different positions for the reference points (higher Bpa, lower Fpa), but with present knowledge this is likely to increase the False Alarm rate (i.e. unnecessary catch reductions) more often. Even though False Alarms are more ecologically precautionary than Misses, this may not please all customers for the advice.
The increasing symmetry of errors (Misses and False Alarms about equally likely; see Table 3) when both SSB and F are used in the advice, and an error rate of about 25% whenever SSB is used (very close for Misses), are consistent with managers commonly treating Bpa as a target, despite frequent ICES admonitions to the contrary. If managers are trying merely to maintain stocks at or just above Bpa, then even modest uncertainties in assessments are going to result in assessments estimating the stock to be just above Bpa in one year, and just below in another, while the stock itself varies around this "pseudo-target". If the chance variation in assessment estimates and stock status is independent, then one would expect the 25% Miss rate observed. A False Alarm rate markedly below the expected 25% would be consistent with the true status of some stocks being far enough above Bpa that they are rarely estimated to be below that value, even when an individual estimated SSB is lower than the true stock size.
In single-year management contexts, both Misses and False Alarms are considered "incorrect advice", i.e. advising managers to take the wrong action. Management actions based on erroneous advice will amplify the difference between the actual stock size and the reference point, i.e. it will increase the signal to be detected. Hence, the next assessment would have a higher likelihood of correctly identifying the sign of the discrepancy between the true stock and the reference point, and a greater likelihood of correct advice.
This self-correcting property of an annual programme of assessment and advice needs to be taken into account when moving to multi-year management programmes. Management responses to Misses and False Alarms will be in place for the multi-year period. This will make the signal to be detected even larger at the next assessment, so the likelihood of corrections will be even greater. However, stock condition could have deteriorated substantially if management is based on advice that "missed" a stock being outside safe biological limits. Therefore, multi-year management approaches should strive to keep stocks well above Bpa, so that opportunities for Misses will be rare.
More generally, using both SSB and F will provide the best management advice. However, with current stock conditions, SSB and F together would not yield recommendations for catch reductions, when in fact they are needed for about one stock in four, while catch reductions are recommended for one out of six stocks, when those stocks are within safe biological limits. Rebuilding stocks farther above Bpa than at present ought to reduce both Miss and False Alarm rates, because the gap between the precautionary reference point and the true stock status becomes greater than the uncertainty in the annual assessments, making true positive hits the most likely (as well as the most desirable) outcome. As ICES often stresses in its advice, there are almost always yield benefits when SSB is kept well above Bpa (ICES, 2002), so this strategy will improve both quality of advice and value of fisheries.
The Ecological Quality Objective proposed by the 5th North Sea Conference (Anon., 2002) could be interpreted to mean that the trends in SSB relative to Bpa and/or F relative to Fpa should be reported for every stock (see Table 1), or that a composite indicator recording the proportion of stocks within safe biological limits should be produced, where the target and reference levels would both be 100%. This target and reference level is consistent with the management goals for individual stocks, as assessed by the Advisory Committee for Fisheries Management (ACFM). For each of these stocks, the goal is to keep SSB above Bpa, and F below Fpa. In this context, the adoption of ecosystem-based management would not result in lower conservation standards than are already in place, and can therefore not have any adverse impact on the conservation and management goals for target stocks. However, it would be necessary for management to be highly risk-intolerant with regard to achieving this target of 100%.
When considering how performance of this EcoQO can be improved, we should also consider the reliability of the assessments themselves. Assessments are known to be imperfect, because there is uncertainty in both the analytical formulations of processes such as natural mortality and age-specific catchability of fleets, as well as in parameter estimates, owing to sampling error in the input data. Hence, the annual estimates of SSB and F can be expected to have some error (and assessment outputs always provide estimates of the magnitude of estimation errors). As long as errors are only variance, they might render estimates of SSB and F less sensitive as indicators for EcoQOs (with Hit rates of about 60%), but they should not bias performance. However, if the errors in estimating SSB and F reflect systematic bias, performance might be impaired. This is a real risk, because some assessments have been known to suffer from retrospective bias (ICES, 2002), such that successive assessments recalibrate the absolute estimates for a number of past years. The more common pattern is for SSB to be overestimated and fishing mortality to be underestimated for the current and recent past years, consistent with (but not proof of) under-reporting of the number of fish actually killed (i.e. landings and/or discards) by the fishery. This bias means that the Bpa values from any single assessment may not be perfectly biologically calibrated with SSB estimates from assessments in earlier or later years. In other words, if Bpa is taken from an assessment in, say, 1999, it may not reflect perfectly the boundary of safe biological limits on the SSB from an assessment in 1996 or 2002.
Expert groups are working on this aspect of precautionary reference points, but new estimates of Bpa are not yet available. The relatively high Miss rates in the mid-1990s are consistent with the retrospective bias found in assessments in the late 1990s, which were strongly influenced by those catches. The relatively high Miss rates during a period when many assessments are now known to have suffered retrospective bias suggest that, with improved data, it may be possible to reduce Misses without increasing False Alarm rates unacceptably. In order to improve data quality in the assessments, monitoring protocols have been developed for catch reporting, catch monitoring, and conduct of research vessel surveys. However, protocols for catch reporting are driven more by enforcement capabilities and requirements of fishing plans, than by the needs of good science.
A Hit rate of just >60% might be taken to suggest that Bpa and Fpa are, in fact, poor precautionary reference points, and that the suggested EcoQO is not useful for conservation purposes. This conclusion would be premature, because correct management action was advised most of the time, and the signal-detection nature of an annual assessment and advisory process makes it likely that incorrect advice in one year is corrected in the subsequent assessment. Moreover, knowledge of actual performance of this EcoQO in practice only exists because of a long history of use of the indicators of the status of the stock, SSB and F. This use has been in contexts where their strengths and weaknesses are examined critically on essentially an annual basis, and potential biases, such as the retrospective pattern in assessments, have been discovered and examined. It may be naive to assume that indicators associated with other EcoQOs, with which the scientific community often has less experience, and in far less critical environments, will necessarily be less vulnerable to bias or high variance.
The test of the effectiveness of management advice shows counter-intuitive results: the expectation was that stocks that are currently outside safe biological limits have a history of many Misses, while stocks that are within safe biological limits have a history of many Hits. Results, however, show the opposite: the number of Hits is highest for those stocks for which SSB is farthest below precautionary reference levels, whereas the number of Misses and False Alarms is highest for those stocks that are above precautionary reference levels. This suggests that possibly cause and effect are the other way around: advice performs better for stocks that are in poor shape, but the quality of advice hardly affects the status of the stock. This statement applies mainly to flatfish and roundfish stocks that, because of their numbers, drive this analysis. Therefore, a tentative conclusion would be that TAC management, even if precautionary, is probably not effective for stocks caught in a mixed fishery (Van Beek and Pastoors, 1999).
|
Acknowledgements |
|---|
We thank the ICES "Working Group for the Ecosystem effects of Fishing" (WGECO) for providing the stimulus and the intellectual input. The work was funded by the European Commision: European Fisheries Ecosystem Plan (EFEP, EU project Q5RS-2001-01685).
|
References |
|---|
|
TopIntroductionMaterial and methodsResultsDiscussionReferences |
|---|
-
Anon. (1997) Statement of Conclusions from the Intermediate Ministerial Meeting on the Integration of Fisheries and Environmental IssuesMarch 1997Bergen, Norway.
Anon. (2002) Ministerial Declaration of the 5th International Conference on the Protection of the North SeaMarch 2002Bergen, Norway.
FAO. (1996a) Precautionary approach to fisheries. 1. Guidelines on the precautionary approach to capture fisheries and species introductions. FAO Fisheries Technical Paper, 350/1. 52 pp.
FAO. (1996b) Precautionary approach to fisheries. 2. Scientific papers. FAO Fisheries Technical Paper, 350/2. 210 pp.
FAO. (2003) The ecosystem approach to fisheries. FAO Technical guidelines for Responsible Fisheries 4(Suppl. 2): 112 pp. http://ftp.tao.org/docrep/fao/005/y4470e/y4470e00.pdf.
Helstrom C.W. (1968) Statistical Theory of Signal Detection(Pergamon Press, New York) 470 pp.
ICES. (2002) Report of the ICES Advisory Committee on Fishery Management, 2002. ICES Cooperative Research Report, 255.
Lanters R. L. P., Skjoldal H. R., Noji T. T. (1999) Ecological Quality Objectives for the North Sea, September 1999. Scheveningen, Netherlands. Joint report: Netherlands Ministry of Transport, Public Works & Water Management, Directorate General of Public Works and Water Management, National Institute for Coastal and Marine Management (RIKZ)/Norwegian Institute of Marine Research. RIKZ Report 99.015 ISSN 0927-3980/Fisken og Havet 10 – 1999. ISSN 0071-5683. 57 pp.
Van Beek F. A. and Pastoors M. A. (1999) Evaluating ICES catch forecasts: the relationship between implied and realised fishing mortality. ICES Document, CM 1999/R: 04. 22 pp.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M.-J. Rochet and J. C. Rice Simulation-based management strategy evaluation: ignorance disguised as mathematics? ICES J. Mar. Sci., May 1, 2009; 66(4): 754 - 762. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. C. Rice A generalization of the three-stage model for advice using the precautionary approach in fisheries, to apply broadly to ecosystem properties and pressures ICES J. Mar. Sci., April 1, 2009; 66(3): 433 - 444. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. M. Pilling, L. T. Kell, T. Hutton, P. J. Bromley, A. N. Tidd, and L. J. Bolle Can economic and biological management objectives be achieved by the use of MSY-based reference points? A North Sea plaice (Pleuronectes platessa) and sole (Solea solea) case study ICES J. Mar. Sci., September 1, 2008; 65(6): 1069 - 1080. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. B. M. Kraak, F. C. Buisman, M. Dickey-Collas, J. J. Poos, M. A. Pastoors, J. G. P. Smit, J. A. E. van Oostenbrugge, and N. Daan The effect of management choices on the sustainability and economic performance of a mixed fishery: a simulation study ICES J. Mar. Sci., May 1, 2008; 65(4): 697 - 712. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Dickey-Collas, M. A. Pastoors, and O. A. van Keeken Precisely wrong or vaguely right: simulations of noisy discard data and trends in fishing effort being included in the stock assessment of North Sea plaice ICES J. Mar. Sci., December 1, 2007; 64(9): 1641 - 1649. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. C. Rice and E. Legace When control rules collide: a comparison of fisheries management reference points and IUCN criteria for assessing risk of extinction ICES J. Mar. Sci., May 1, 2007; 64(4): 718 - 722. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. C. Rice and D. Rivard The dual role of indicators in optimal fisheries management strategies ICES J. Mar. Sci., May 1, 2007; 64(4): 775 - 778. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H.-J. Ratz, E. Bethke, H. Dorner, D. Beare, and J. Groger Sustainable management of mixed demersal fisheries in the North Sea through fleet-based management a proposal from a biological perspective ICES J. Mar. Sci., May 1, 2007; 64(4): 652 - 660. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. C. Rice and M.-J. Rochet A framework for selecting a suite of indicators for fisheries management ICES J. Mar. Sci., January 1, 2005; 62(3): 516 - 527. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||