Authors: Nancy Huntly, Brian Riddell, Richard Williams
Introduction
In a June 2, 2003 memo (40k PDF), the Northwest Power and Conservation Council requested that the ISRP complete a review of the Nez Perce Tribes' Northeast Oregon Hatchery (NEOH) Spring Chinook Master Plan Step Two re-submittal documents. These NPT documents were intended to satisfy: 1) National Environmental Policy Act (NEPA) requirements, and 2) ISRP concerns raised in a December 21, 2001 preliminary ISRP review of NEOH Step 2 materials.[1] The NEOH Step 2 materials submitted in 2001 for the preliminary ISRP review were intended to address the conditions placed on this project as part of the Council's Step 1 decision.[2]
The ISRP's preliminary review was coordinated with the ISRP's review of NEOH related projects in the Blue Mountain and Mountain Snake Provincial Review. The set of NEOH proposals received "fundable" recommendations in the provincial review, and the ISRP noted that progress was being made in planning the NEOH spring chinook program. However, the ISRP found that the Nez Perce Tribe's (NPT) response to previous comments on the Step 1 Submission lacked the detail necessary for technical assessment. The NPT had not completed responses to issues required by the Council. In the course of the Step Review process, Bonneville required the NEOH Program to produce an Environmental Impact Statement to meet NEPA requirements. Rather than establish a separate process to deal with the ISRP concerns, the Council set up this Step 2 "response review" to track the development of the Environmental Impact Statement. Consequently, almost a year and half later, the ISRP provides a recommendation and specific comments on the NEOH's response to previous ISRP concerns.
ISRP Recommendation
Overall, this response is much improved over the previous response; however the ISRP has continued concerns for ISRP issue 3 (Genetic breeding plans), issue 5 (forecasting and escapement goals), and with the lack of detail presented in the Monitoring and Evaluation Plan (Appendix A). In addition, parts of the response suggest that integration with habitat issues is not a priority goal.
The response does not deal explicitly with the ISRP's summary statement that the design of the hatchery appeared to be inflated in number of treatments, putting more attention on monitoring and evaluation (M&E) of less important issues (hatchery practices) than on the critical issue of M&E of performance of hatchery versus wild fish and of the effects of hatchery fish on wild fish. We did not find a direct response to this in the materials, although at least high density rearing is now omitted. Whether the goals of M&E are truly changed is not as clear.
As presented in Appendix A, the Monitoring and Evaluation Plan has sidestepped some of the concerns voiced in our earlier review by specifying that the document outlines "a monitoring and evaluation approach". Responding with an approach, rather than a plan, allows many details of sampling design to be deferred, but these details will become critical to success of the M&E program. The document is much more informative than the earlier ones, but the sampling design for M&E must be better resolved before the acid test of scientific soundness is met. For instance, exactly where and when are samples to be taken, and why are those places and times selected?
The ISRP has provided some suggestions (and cautions) on M&E in several reviews including recent reviews of the Idaho Supplementation Studies (ISS) program. The NEOH would benefit from examining the history of the ISS, which suggests it may be difficult to set up and maintain a highly controlled, treatment-reference design. During the Blue Mountain Province review, which included a number of NEOH proposals, the ISRP discussed not only the need for reference streams to support evaluation of the NEOH, but also the need for a probabilistic sampling scheme. We would like to see significantly more discussion of this option (enhanced probabilistic sampling) in the M&E "approach", if not plan. Appendix A suggests that the project proponents may underestimate the difficulty of implementing an effective monitoring plan and have not completely appreciated either the value of probabilistic sampling or the likely difficulty of implementing the reference/experimental design that seems implied in their approach. Reference conditions do matter, but a broader, more probabilistic element in the sampling design may be needed as well.
Comments on Responses to Issues Raised by the ISRP and NPCC
ISRP response to 2003 NEOH resubmittal: Issue Number 1, Adequacy of the M&E plan (Appendix A) and data management:
The M&E Plan, although really an approach rather than a plan, is much improved in this response, but still lacks detail in many important aspects. Our earlier statement (July 2000) that "the detailed monitoring and evaluation plan should also benefit from additional peer-review after it is developed" still holds. High quality M&E with strongly supported conclusions has been very hard to come by, and the project proponents should recognize the value of (and seek) input from statisticians with experience in similar monitoring efforts. Details on problems are given in item-specific responses below.
Table 3 in Appendix 1 contains the vital statistics that the Independent Scientific Advisory Board just noted in their supplementation report.[3] Two topics, however, were conspicuously absent: (1) if projects are being scaled to production, some of which is necessary for monitoring, then the estimates of sample sizes, tag allocations, and power analyses were not presented, and (2) while the Appendix notes two reference streams for comparisons with the treatment streams, there was very little discussion about the ability to collect comparable data in these references steams. This issue was raised during the ISRP site visits.
Concerning the first point, it is possible that we have seen these statistics previously but we could not locate them in previous material. In terms of the M&E plan though, we are more concerned about the ability to collect data from these reference streams with a comparable level of accuracy. How will adult return numbers, age at returns, smolt production, etc, be estimated in these natural systems? It is important that this be addressed. If data in the reference systems cannot be collected with comparable accuracy, then this design for the M&E plan may not be appropriate.
One additional concern is with coordination of monitoring and evaluation of data and outcomes. The Appendix states that the NEOH program will in large part coordinate M&E efforts and facilitate linking data. It also seems to rely heavily on 5-year reports; however, there needs to be provision for data to be analyzed and made available for review much more frequently than every five years. The problem of unanalyzed data plagues many projects throughout the basin including most hatchery evaluation programs and has led to many failed monitoring efforts by contributing to lack of early recognition of problems that might have been corrected. This critical and common problem (failure to promptly analyze and use data) must be recognized and remedied.
Other minor points in Appendix A were:
Objective 3. The response addresses habitat limits based on environmental conditions, but the other habitat feature is the relationship between habitat types and quantity compared with the production of spring chinook produced. This issue of habitat capacity and quality for production of spring chinook should also be included.
Objective 4. Concerning the ability to forecast returns of spring chinook. Success in forecasting chinook returns is strongly dependent on controlling errors in aging the adult returns. Where coded-wire tags or PIT tags can be used then errors in aging are essentially zero and forecasting based on sibling relations is usually quite successful. However, if aging errors are substantial then the accuracy of forecasts is more limited. Therefore, if accurate forecasts are necessary then reviewers would strongly recommend that forecasts be developed based on tagged lots of chinook. Secondly, to forecast total chinook returns there will also have to be an estimate of the number of wild fish compared to the tagged lot of chinook (i.e., a mark to unmark ratio). This should be determined as smolts leaving the system and can then be compared with adults returning to the system. These monitoring and evaluation points do not seem to be identified on page 24.
Objective 5. It is notable in that they recognize and are planning for the sharing of data in a timely manner and note the need to collaboration within the Basin.
ISRP response to 2003 NEOH resubmittal: Issue 2, Adequacy of linkage to habitat actions:
In general, the response was acceptable, and the maps added clarity to the response. However, the response to point 5 (Provide evidence of improvements in habitat conditions) almost seemed disconnected from the previous parts of this answer (that were much better). The first paragraph of page 15 states "it is questionable how much positive change could ever be detected and measured." But, the previous pages noted the efforts to purchase water for natural flows, efforts to improve water management and to control temperatures in the lower rivers. These benefits to the environment could certainly be measured. Further, with the extensive monitoring of adults and smolts produced, a data set will develop that should allow evaluation of smolts production per spawner and possibly even survival between certain life stages. The limitation may be that the benefit of each specific habitat project cannot be assessed, but we would suggest that evaluation at that local scale might not be important to the overall restoration effort. It could be relevant to other levels of habitat assessments or for cost effectiveness monitoring, but that seems to be at a level below this project.
The project proponents need to develop environmental monitoring along with their fish monitoring, and design of this sampling scheme is likely to be critical to their ability to understand the data that result from their monitoring of fish. The response should make it clear that they understand this need and linkage and that they are making progress on a concrete plan to monitor habitat and other environmental conditions in a way that is likely to provide useful data.
ISRP comments on the NEOH 2003 response to Issue 3, Genetic breeding plans:
This response is still inadequate and is not as well prepared as the previous issues. For example, the 3rd paragraph of page 20 does not make sense: the first sentence seems to mix points. On what basis is the statement made that maximizing genetic diversity produces fish that "resemble naturally produced fish"? The last sentence is incorrectly stated. We assume the sponsors mean that the genetic risks are much less than the demographic risks associated with very small populations. Considering the highly invasive nature of captive and conventional broodstock programs, and their strong potential to result in poorly adapted fish with poor environmental performance, the faithful assertion that genetic risks are less than demographic ones is questionable.
The discussion of point 2, solutions to the lack of habitat for late-returning fish, is inadequate. We would agree that randomization is appropriate in intensive culture situations, but there are other mating designs to use (for example, see Waples and Do. 1994. Genetic risk associated with supplementation of Pacific salmonids: Captive breeding programs. Can. J. Fish. Aquat. Sci. 51(Supple. 1): 310-329; this paper demonstrates the advantages of using fully factorial mating). Further, the hatchery site could have been developed as chinook spawning channels and then the fish can undergo their normal reproductive competition. Even a production hatchery could be designed so that genetic lines of fish are maintained in order to guard against random events or cumulative effects of culture.
Given the importance of these issues and the investment in these programs, the genetic aspects of the NEOH should have progressed further. It is a concern that "compliance with IHOT" seems to displace the need for continuing biological evaluation of likely best practices. Invoking "IHOT" compliance cannot substitute for scientific soundness.
ISRP comments on the NEOH 2003 response to Issue 4, Risks associated with transport and off-site rearing from Mark's Ranch to Lostine or Lookingglass Hatchery:
An acceptable response. While transportation of fish is still needed, the situation described in the response is a definite improvement over the previous.
ISRP comments on the NEOH 2003 response to Issue 5, Harvest framework, forecasting and escapement goals:
The basic approach to be followed is similar to that used by most other agencies doing forecasts with chinook salmon. The authors may wish to consider this issue further, as they should be able to do good forecasts given the number of tagged fish to be released or that could be tagged when sampled as smolts. The figures in Fig. 5-1 are interesting and demonstrate an important point for them to monitor and examine. In each figure they should include a diagonal line with a slope of 1.0; this line gives an easy visual reference to the regression lines and shows the degree of bias in the relations. The important point is that frequently the forecasts for natural fish exceed the actual return because the forecasts are usually based on hatchery data and tagged groups. This error may be due to aging errors in non-tagged groups and/or may be related to differences in the maturation rate of hatchery versus natural fish. This is an issue that the M&E program should examine.
Note that if the forecast error is likely to over-estimate the return of naturally produced fish, it is the natural stock that is at risk of management error, not the hatchery fish.
There is also a concern for the answer to question 5.2, page 35. The value of 700 spring chinooks as an escapement target was based on two studies of habitat capacity. However, in most habitat-based evaluations that we are closely familiar with, the number of spawners required (i.e., the target) is much less than the spawning capacity and frequently much less than the numbers of spawners actually observed. As an example, compare this 700 target with the returns in the past few years in Table 5-1. We recommend caution in relying on these habitat assessments; but this M&E program should be able to assess the escapement target and earlier habitat assessments over time. The other management issues when allowing harvest is how accurate and precise the forecast is and how certain you want to be to achieve the escapement target. If the precision is poor and you must achieve the target, then management should build a "buffer" into the in-season escapement target so that managers have some specified level of confidence that they should achieve the target after fishing.
ISRP comments on the NEOH 2003 response to Issue 6, NATUREs:
The response states that high density rearing and evaluation of such have been dropped from the design. This is a good decision. The response to the remaining points is reasonable, particularly given how limited we feel the advantage of NATUREs rearing really is. A primary concern here was the inflated number of hatchery treatments (rearing conditions) that were proposed in the preliminary Step 2 documents, thus the larger hatchery operation and increased difficulty in M&E, given more treatments and lower ability to detect their outcomes. That appears to have been corrected (at least high vs. low density rearing has been changed to entirely low density rearing — a positive). Most likely, the real benefit is actually in the low density rearing that they will use.
Issue 7 (from ISRP 2001 review of Step 2 submittal):
The preliminary Step 2 response addressed the ISRP issue and was consistent with project proposals reviewed during the Provincial review.
ISRP comments on the NEOH 2003 response to Issue 8, Lookingglass Hatchery enhancements:
This issue still seems to be a concern in this response. The reply indicates that these enhancements at Lookingglass Hatchery are necessary for the NEOH program, but the issues seem more administrative than technical.
ISRP comments on the NEOH 2003 response to Issue 9, Downstream effects and long-term implications to success of captive broodstock effort:
This response is quite realistic. The response to question 9.1 is rather non-committal, but we are not certain what answer we would have been looking for to that question. The other answers involve integrating these assessments with the large M&E plan, which we agree with, and integration with other programs. The answer to question 9.4 is a concern. The sponsors note that there is no secure funding for the genetic sampling yet. The genetic sampling and analyses are important aspects of this study and should be supported, possibly not at the level requested, but certainly at a level providing appropriate sample sizes.
[1] ISRP 2001-12c, NEOH Step 2 Preliminary Review, December 21, 2001
[2] For the initial ISRP Step 1 Review of this project see ISRP 2000-6, NEOH Step 1 Review, June 11, 2000
[3] ISAB 2003-3, A Review of Salmon and Steelhead Supplementation, June 4, 2003
