U. S. Food and Drug Administration
Center for Food Safety and Applied Nutrition
August 10, 2000

ACKNOWLEDGEMENTS

The following individuals and/or entities are to be recognized for their invaluable contributions to the development of this report and the implementation of this project:

FDA:

Center for Food Safety and Applied Nutrition (CFSAN)

National Retail Food Steering Committee

OTHER Regulatory:

State and local regulatory jurisdiction representatives who accompanied the FDA Regional Food Specialists on data collection inspections.

INDUSTRY:

Industry food service managers of selected participant establishments for cooperative assistance during the data collection.

CONTENTS

1. BACKGROUND

2. INTRODUCTION AND PURPOSE

3. METHODOLOGY Part A. Selection of Facility Types
   Part B. Selection of Data Collectors
   Part C. Selection of Geographical Locations
   Part D. Selection of Establishments Within Geographic Locations
   Part E. Baseline Data Collection Procedure
   Part F. Baseline Data Collection Form

4. DATA REPORTS AND DISCUSSION Part A. Overall Percent of Observable and Applicable Data Items Observed IN COMPLIANCE, by Facility Types and the FDA Improvement Goal
   Part B. Percent of Total Observations IN COMPLIANCE
   Part C. Individual Data Items Needing Priority Attention
   Part D. Figures and Charts – Percent Breakdown of the Most Significant OUT OF COMPLIANCE Data Items Observed
   Part E. Summary of the Most Significant OUT OF COMPLIANCE Data Items

5. FIELD AND STATISTICAL LIMITATIONS Part A. Field Operational Limitations
   Part B. Statistical Limitations

6. RECOMMENDATIONS Regulatory Recommendations
   Industry Recommendations

7. AREAS FOR FUTURE STUDY

ABSTRACT

The 1996 report "Reinventing Food Regulations" [National Performance Review] concludes that foodborne illness caused by harmful bacteria and other pathogenic microorganisms in meat, poultry, seafood, dairy products, and a host of other foods is a significant public health problem in the United States. For years regulatory and industry food safety programs have been designed to minimize the occurrence of foodborne illness. There is, however, a lack of a national baseline on the occurrence of foodborne disease risk factors.

This project is designed to establish a national baseline on the occurrence of foodborne disease risk factors within the retail segment of the food industry. This report, officially referred to as the FDA Retail Food Program Database of Foodborne Illness Risk Factors, presents the methodology used to establish a baseline and reports the results of the data collected. The report is provided to regulators and industry with the expectation that it will be used to focus greater attention and increased resources on the control of foodborne illness risk factors.

The project is NOT designed to support comparisons of chains of fast food restaurants or chains of grocery stores. There is no statistical justification for looking at reduced sets of results particular to, e.g., two chains of restaurants and drawing conclusions from the differences.

The data set is too small to support comparisons of individual Specialists’ geographical areas, states, cities or even regions of the U.S. Not only would it be a flagrant abuse of statistics, but such comparisons might be combined with other information, such as the locations of FDA Retail Food Specialists, to identify some of the likely comparison sets. This information would bias future studies.

The National Retail Food Steering Committee of the U.S. Food and Drug Administration established as a goal a 25% reduction in the occurrence of Centers for Disease Control and Prevention-identified foodborne illness risk factors in institutional food service establishments, restaurants, and retail food stores by October 1, 2010.

In order to measure progress against the FDA retail food program goal, an assessment of the current status of the occurrence of foodborne illness risk factors is required. Although the level of foodborne illnesses would be the ideal retail food program performance indicator, the occurrence of foodborne illness is grossly underreported, making it an unreliable program measurement. As an alternative, the occurrence of foodborne illness risk factors was selected as the performance indicator.

These foodborne illness risk factors are: Food from Unsafe Sources; Inadequate Cooking; Improper Holding Temperature; Contaminated Equipment; and Poor Personal Hygiene. The FDA Retail Food Program Database of Foodborne Illness Risk Factors is intended to establish a baseline against which industry and regulatory efforts to change behaviors and practices directly related to foodborne illness will be measured.

The FDA 1997 Food Code was the standard of measurement used for this project. A Baseline Data Collection Form that reflected Food Code provisions was designed for collecting data on the occurrence of the 5 risk factors.

The resulting data, comprising 17,477 observations was analyzed using several approaches to determine practices and behaviors needing priority attention within specific retail food facility types. Five practices and behaviors exceeded a 40% OUT OF COMPLIANCE observation rate:

• Cold Holding of Potentially Hazardous Food (PHF) at 41°F (5°C) or below;
• Ready-to-eat (RTE), PHF Hold Cold at 41°F (5°) or Below;
• Commercially Processed RTE, PHF Date Marked;
• Surfaces/Utensils Cleaned/Sanitized; and
• Proper, Adequate Handwashing

If the safety of food in the retail segment of the food industry is to be significantly improved, the retail food industry and the regulatory community must remain focused in their efforts to reduce the individual data items having the most significant OUT OF COMPLIANCE observation rate.

I. BACKGROUND

The Food and Drug Administration (FDA) is responsible for setting standards for the safe production of foods and advising state and local governments on food safety standards for institutional food service establishments, restaurants, retail food stores and other retail food establishments. In this advisory role FDA works closely with other federal agencies to provide guidance and assistance that will enhance the regulatory programs of state and local jurisdictions.

The 1996 report "Reinventing Food Regulations" [National Performance Review] concludes that foodborne illness caused by harmful bacteria and other pathogenic microorganisms in meat, poultry, seafood, dairy products, and a host of other foods is a significant public health problem in the United States. In response to this 1996 report and subsequent input from state and local regulatory partners, FDA established a National Retail Food Steering Committee (Steering Committee) including representation from the Center for Food Safety and Applied Nutrition (CFSAN), Office of Regulatory Affairs (ORA), Division of Federal/State Relations (DFSR), Division of Human Resource Development (DHRD), and the Interstate Travel Program (ITP) Field Team. The Steering Committee is responsible for reviewing retail food program objectives and coordinating program activities.

The Steering Committee was charged with development of a strategic plan that meets the requirements of the Government Performance and Results Act (GPRA) of 1993. The GPRA requires all Federal Agencies to develop performance plans that include measurable goals and performance indicators for each fiscal year.

The level of foodborne illnesses would be the ideal retail food program performance indicator. The occurrence of foodborne illness, however, is grossly underreported which makes it an unreliable program measurement. As an alternative, the occurrence of foodborne illness risk factors (risk factors) was selected as the performance indicator. Based on the Centers for Disease Control and Prevention (CDC) Surveillance Report for 1988 – 1992, five broad categories of risk factors contributing to foodborne illness were identified. The identified risk factors include:

• Food from Unsafe Sources;
• Inadequate Cooking;
• Improper Holding Temperatures;
• Contaminated Equipment; and
• Poor Personal Hygiene.

The Steering Committee established the reduction of these risk factors as its measurable retail food program goal. Using the FDA Food Code (Food Code) as the food safety standard, the Steering Committee established a goal to reduce the occurrences of CDC-identified risk factors causing foodborne illness in institutional food service establishments, restaurants, and retail food stores by 25% by October 1, 2010.

In order to begin to measure the progress toward the achievement of this national goal, data had to be collected to establish a baseline on the occurrence of these risk factors. The design of the project was based on a thorough review of all the provisions of the Food Code. The Food Code contains compliance provisions and interventions to control the 5 identified categories of risk factors.

Data for the baseline were obtained from 895 total inspections of institutional food service establishments, restaurants, and retail food stores, consisting of 17,477 observations, were conducted by FDA’s Regional Retail Food Specialists (Specialists).

The project is officially referred to as the FDA Retail Food Program Database of Foodborne Illness Risk Factors. This report presents the methodology used to establish a baseline and reports the results of the data collected. The report is provided to regulators and industry with the expectation that it will be used to focus greater attention and increased resources on the control of risk factors.

II. INTRODUCTION AND PURPOSE

The initiative to create the FDA Retail Food Program Database of Foodborne Illness Risk Factors began in January 1997, and is ongoing. The project’s purpose is to establish a baseline against which industry and regulatory efforts to change behaviors and practices directly related to foodborne illness will be measured. It recognizes the need to fill a void that currently exists in the assessment of program effectiveness for controlling these risk factors.

By establishing a baseline, the information gathered from future field inspections can be used to measure trends in terms of compliance with specific requirements of the Food Code. It is expected that an improvement in compliance with the Food Code provisions that address these risk factors will have a direct impact on the occurrence of foodborne illness.

The 1997 FDA Food Code was the standard of measurement used by the Specialists during their inspections. The conclusions from the data are national in scope and are not reflective of any specific regulatory jurisdiction. It is important to note that the baseline data collection was not designed to determine an individual establishment’s compliance with local or State requirements. No attempt was made to determine if an establishment would have been found to be substantially in compliance with its governing ordinances. Use of subsets of the data (e.g., by regions of the country) would result in sample sizes that are too small to be useful and that are not statistically valid.

The data from this project and future studies planned for 2003 and 2008 are expected to provide input into the Healthy People 2010 Food Safety Objective 10.6. This objective is designed to improve food preparation practices and food employee behaviors at institutional food service establishments, restaurants, and retail food stores.

FDA is the sponsoring agency for this project. FDA/Office of Regulatory Affairs/Division of Federal-State Relations retains and archives records from the data collection periods. FDA/Center for Food Safety and Applied Nutrition/Division of Mathematics performs statistical planning and analysis, and the Division of Cooperative Programs assists ORA with project planning, data analysis, and data reporting.

III. METHODOLOGY

The target industry segments for the initial phase of this project are: Institutional food service establishments, restaurants, and retail food stores. These establishments encompass over a million varied and diverse types of operations in the United States. Variations in types of operations, menus, culture, language, food safety knowledge, and resources interact to create a complex food safety environment.

A direct focus on these industry segments provides a breadth of coverage of general and highly susceptible populations while also covering the vast majority of establishment types. For the purpose of this report, a highly susceptible population is a group of persons who are more likely than other individuals to experience foodborne disease due to their current health status or age.

The chart below reflects the 3 industry segments and 9 facility types selected.

INSTITUTIONS Hospitals Nursing Homes Elementary Schools

RESTAURANTS Fast Food Restaurants Full-Service Restaurants

RETAIL FOOD STORES Deli Departments Meat and Poultry Departments Produce Departments and Salad Bars Seafood Departments

In order for the results to be statistically significant, 90 inspections were needed for each facility type. To allow for unexpected difficulties, at least 100 inspections were planned for each type, or over 900 total inspections. Statistical significance was successfully achieved with the completion of 895 inspections.

B. Selection of Data Collectors

Individuals possessing a strong working knowledge of the risk factors and the Food Code and its application, were needed to ensure consistency in the data collection process. FDA has approximately 20 Specialists located throughout the nation. Each Specialist possesses technical expertise in retail food safety and a solid understanding of the operations of each of the 9 facility types chosen. In addition, the Specialists comprised a group within which implementation of the project could be easily coordinated and standardized.

C. Selection of Geographical Locations

The ideal design for data collection would be one in which the number of establishments inspected within any given location would be directly related to the volume of retail food consumed within that location. Since this information is proprietary and difficult to obtain, a truly random, proportional data collection project would have delayed the project by at least one year. In addition, due to the randomness of the data collection that would have been required, the extra time and travel expenditures resulting from such a project would have eliminated other vital work by the Specialists and would likely have been cost-prohibitive.

The geographical distribution of Specialists throughout the U.S. allowed for a broad sampling comprising all regions of the U.S. The choice of data collection locations, therefore, was based on the Specialists' geographic areas of responsibility and provided a reasonably convenient design for estimating national risk-related practices.

D. Selection of Establishments within Geographic Locations

Selection of establishments focused on those operations that:

• Served a highly susceptible population (i.e., hospitals, nursing homes, elementary schools); or
• Involved extensive handling of ingredients; or
• Conducted a variety of food preparation processes.

The Food Code divides food establishments into 5 risk type categories. The Risk Categorization of Food Establishments, contained in Annex 4 of the Food Code is presented in the table that follows. The establishments described above generally fall into risk type categories 3, 4, or 5 based on their operational practices and populations served.

RISK TYPE	RISK TYPE CATEGORY DESCRIPTION
1	Pre-packaged, non-potentially hazardous foods only. Limited preparation of non-potentially hazardous foods only.
2	Limited menu (1 or 2 main items). Pre-packaged, raw ingredients are cooked or prepared to order. Retail food operations exclude deli or seafood operations departments. Raw ingredients require minimal assembly. Most products are cooked/prepared and served immediately. Hot and cold holding of potentially hazardous foods is restricted to single meal service. Preparation processes requiring cooking, cooling, and reheating are limited to 1 or 2 potentially hazardous foods.
3	Extensive handling of raw ingredients. Preparation process includes the cooking, cooling, and reheating of potentially hazardous foods. A variety of processes require hot and cold holding of potentially hazardous food. Advance preparation for next day-service is limited to 2 or 3 items. Retail food operations include deli and seafood departments. Establishments doing food processing at retail.
4	Extensive handling of raw ingredients. Preparation processes include the cooking, cooling, and reheating of potentially hazardous foods. A variety of processes require hot and cold holding of potentially hazardous foods. Food processes include advanced preparation for next-day service. Category would also include those facilities whose primary service population is immunocompromised.
5	Extensive handling of raw ingredients. Food processing at the retail level, e.g., smoking and curing, reduced oxygen packaging for extended shelf-life.

All of the facility types included in this study did not fall into risk type categories 3, 4, and 5. Meat, seafood, and produce departments of retail food stores may fall into risk type category 2. These facility types were included in this study due to recent foodborne illness outbreaks associated with certain products produced within these departments.

Each Specialist developed 5 Comparison Set Establishment Lists for each of the facility types. A comparison set establishment list is comprised of 10 or more establishments located in a geographic area that are placed in alphabetical order. For a few facility types, particularly nursing homes and hospitals, the number of establishments for a facility type within a designated geographic area was limited. In those areas, a comparison set may include as few as 4 establishments.

The Specialists used a table of random numbers to select one establishment from each list for inspection. The random selection of establishments from the comparison set establishment lists prevented selection bias. The following page provides an example of a Comparison Set Establishment List.

EXAMPLE OF A COMPARISON SET ESTABLISHMENT LIST

Comparison Set Establishment List - Baseline Data Collection Project

The project design required each Specialist to complete 5 inspections of each of the 9 facility types for a total of 45 inspections (45 sites X 20 Specialists = 900 inspections).

The table below represents work assignments for each Specialist:

BREAKDOWN OF WORK ASSIGNMENTS FOR EACH SPECIALIST

Industry Segment	Facility Type	Completed Inspection Forms	Number of Comparison Sets	Facilities Per Comparison Set Establishment List
Institutions	Hospitals	5	5	4-10
	Nursing Homes	5	5	4-10
	Elementary Schools	5	5	4-10
Restaurants	Full-Service	5	5	10-15
	Fast Food	5	5	10-15
Retail Food Stores	Deli Dept.	5	5*	10-20*
	Meat and Poultry Dept.	5
	Produce Dept.	5
	Seafood Dept.	5
TOTAL		45	30
* Comparison sets for the Retail Food Store segment of the industry included all four departments.

Comparison set establishment lists, compiled by the Specialists, have been archived and will be used again in future studies. A different establishment will be randomly selected from the same comparison set establishment list. Selection bias will be prevented at each measurement by the use of random numbers.

In order not to compromise data reliability and to assure confidentiality of the selected establishments, demographic and inspectional observations have been entered into the database by number rather than by establishment name or location.

E. Baseline Data Collection Procedure

The 5 major risk factors contributing to foodborne illness identified by CDC provided the foundation for the data collection inspection form. For each risk factor, Food Code requirements were identified and grouped into individual data items on the inspection form (see Baseline Data Collection Reference Sheet). An additional risk factor, "Other", was used to capture the potential food safety risks related to possible contamination by toxic or unapproved chemicals in the establishment.

CDC Risk Factor: FOODS FROM UNSAFE SOURCES
Food Source
1. Approved Source
1.A.	3-201.11* Compliance with Food Law
	3-201.12* Food in a Hermetically Sealed Container
	3-201.13* Fluid Milk and Milk Products
1.B.	3-201.14* Fish
	3-201.15* Molluscan Shellfish
	3-202.18* Shellstock Identification
1C.	3-201.16* Wild Mushrooms
	3-201.17* Game Animals
2. Receiving/Condition
2.A.	3-202.11* Temperature
	3-202.15* Package Integrity
	3-101.11* Safe, Unadulterated, and Honestly Presented
3. Records
3.A.	3-202.18* Shellstock Identification
	3-203.12* Shellstock, Maintaining Identification
3.B.	3-402.11* Parasite Destruction
	3-402.12* Records, Creation and Retention
3.C.	3-502.12* Reduced Oxygen Packaging, Criteria
CDC Risk Factor: INADEQUATE COOK
Pathogen Destruction
4. Proper Cooking Temp., per PHF
4.A.	3-401.11(A)(1)(a)* Raw Animal Foods
4.B.	3-401.11(2)* Raw Animal Foods
4.C	3-401.11(B)(1)(2) Raw Animal Foods
4.D.	3-401.11(A)(3)* Raw Animal Foods
4.E.	3-401.11(A)(3)* Raw Animal Foods
4.F.	3-401.12* Microwave Cooking
4.G.	3-401.11(A)(2)* Raw Animal Foods
4.H.	3-401.11(A)(1)(b)* Raw Animal Foods
5. Rapid Reheating for Hot Holding
5.A.	3-403.11(A)* Reheating for Hot Holding
5.B.	3-403.11(B)* Reheating for Hot Holding - Microwave
5.C.	3-403.11(C)* Reheating for Hot Holding - Commercially processed RTE food
5.D.	3-403.11(E)* Reheating for Hot Holding - Remaining unsliced portions of roasts of beef
CDC Risk Factor: IMPROPER HOLD
Limitation of Growth of Organisms of Public Health Concern
6. Proper Cooling Procedure
6.A&B.	3-501.14(A)* Cooling - Cooked PHF
6.C&D.	3-501.14(B)* Cooling - PHF preprepared from ingredients at ambient temperature
6.E&F.	3-501.14(C)* Cooling - PHF receipt of foods allowed at  41F during shipment
7. Cold Hold (41F or 45F in existing equipment)
7.A&B	3-501.16(B)*, PHF, Hot and Cold Holding PHF shall be maintained at 45F or between 41F and 45F in existing equipment not capable of maintaining 41F and the equipment is upgraded o replaced with 5 years of the R.A.'s adoption of the Food Code
8. Hot Hold (140F)
8.A.	3-501.16(A)* PHF, Hot and Cold Holding
8.C.	3-501.16(A)* PHF, Hot and Cold Holding
9. Time
9.A.	3-501.17(A)(1)(2)* Ready-to-Eat, PHF, Date Marking - On-premises preparation Food is to be date marked at the time of preparation, with the "consume by" date. This "consume by" date should include the day of preparation and is: (1)  7 calendar days at 5C (41F) or less; or (2)  4 calendar days at 7C (45F)
9.B.	3-501.18* Ready-to-Eat, PHF, Disposition Food shall be discarded if not consumed within  7 calendar d days at 5C (41F) or less; or  4 calendar days at 7C (45F)
9.C.	3-501.17(C)*3-501.17(C)* Ready-to-Eat, PHF, Date Marking - commercially processed food Commercially processed food containers shall be clearly marked, at the time originally opened in a food establishment, with the consumer by date which is, including the day the original container is opened:  7 calendar days at 5C (41F) or less; or  4 calendar days at 7C (45F)
9.D.	3-501.19* Time as a Public Health Control
CDC Risk Factor: CONTAMINATED EQUIPMENT
Protection from Contamination
10. Separation/Segregation/Protection
10.A.	3-302.11(A)(1)* Packaged and Unpackaged Food - Separation, Packaging, and Segregation Separate raw animal foods from raw RTE and cooked RTE foods
For marking the status of 10.A.: N.A. (Not Applicable) = If establishment has vegetarian menu only.
CDC Risk Factor: CONTAMINATED EQUIPMENT
Protection from Contamination
10. Separation/Segregation/Protection
10.B.	3-302.11(A)(2)* Packaged and Unpackaged Food - Separation, Packaging, and Segregation Separate raw animal foods by using separate equipment, special arrangement of food in equipment to prevent cross contamination of one type with another, or by preparing different types of food at different times or in separate areas.
10.C.	3-302.11(A)(4-6)* Packaged and Unpackaged Food - Separation, Packaging, and Segregation
	3-304.11(B)* Food Contact with Equipment and Utensils
10.D.	3-306.14(A)(B)* Returned Food, Reservice or Sale
11. Food Contact Surfaces
11.A.	4-601.11* Equipment, Food-Contact Surfaces, Nonfood-Contact Surfaces, and Utensils
	4-701.10* Sanitization of Equipment and Utensils - Food-Contact Surfaces and Utensils
	4-701.11* Sanitization of Equipment and Utensils - Before Use after Cleaning - Frequency
CDC Risk Factor: POOR PERSONAL HYGIENE
Personnel
12. Proper, Adequate Handwashing
12.A.	2-301.11* Clean Condition
	2-301.12* Cleaning Procedure
	2-301.14* When to Wash
	2-301.12* Where to Wash
13. Good Hygienic Practices
13.A.	2-401.11* Eating, Drinking, or Using Tobacco
	2-401.12* Discharges from the Eyes, Nose and Mouth
	2-403.11* Handling Prohibition - Animals
	3-301.12* Preventing Contamination when Tasting
14. Prevention of Contamination from Hands
14.A.	3-301.11* Preventing Contamination from Hands
15. Handwash Facilities
15.A.	5-203.11* Handwashing Lavatory - Numbers and Capacities
	5-204.11* Handwashing Lavatory - Location and Placement
	5-205.11* Using a Handwashing Lavatory - Operation and Maintenance
15.B.	6-301.11 Handwashing Cleanser, Availability
	6-301.12 Hand Drying Provision
CDC Risk Factor: OTHER
Foreign Substances
16. Chemical
16.A.	3-202.12* Additives
	3-302.14* Protection from Unapproved Additives
NOTE: Re: SULFITES ----Refers to any sulfites added in the food establishment, not to foods processed by a commercial processor or that come into the food establishment already on foods.
For marking the status of 16.A.: IN compliance = No unapproved additives on site; IF have sulfites on premises, they are used properly. OUT of compliance = Unapproved additives found on premises and improperly used, e.g., on fresh fruits and vegetables. N.O. (Not Observed) = Not an option for response on this item. N.A. (Not Applicable) = Food establishment does not use any additives or sulfites.
16.B.	7-101.11* Identifying Information, Prominence - Original Containers
	7-102.11* Common Name - Working Containers
Operational Supplies and Applications
	7-201.11* Separation - Storage
	7-202.11* Restriction - Presence and use
	7-202.12* Conditions of Use
	7-203.11* Poisonous or Toxic Material Containers - Prohibitions
	7-204.11* Sanitizers, Criteria-Chemicals
	7-204.12* Chemicals for Washing Fruits and Vegetables
	7-204.13* Boiler Water Additives, Criteria
	7-204.14* Drying Agents, Criteria
	7-205.11* Incidental Food Contact, Criteria-Lubricants
	7-206.11* Restricted Use Pesticides, Criteria
	7-206.12* Rodent Bait Stations
	7-206.13* Tracking Powders, Pest Control and Monitoring
	7-207.11* Restriction and Storage - Medicines
	7-207.12* Refrigerated Medicines, Storage
	7-208.11* Storage - First Aid Supplies
	7-209.11* Storage - Other Personal Care Items
Stock and Retail Sale of Poisonous or Toxic Material
16.C.	7-301.11* Separation - Storage and Display Separation is to be by spacing or partitioning.
For marking the status of 16.C.: N.A. (Not Applicable) = If the establishment does not hold poisonous or toxic materials for retail sale.

Legend:
C	=	Celsius
F	=	Fahrenheit
RTE	=	Ready-to-Eat
PHF	=	Potentially Hazardous Food
R.A.	=	Regulatory Authority

Unannounced visits to the selected establishments were designed to be observational rather than regulatory. A representative of the state, county, or city agency having regulatory oversight responsibilities for the establishments usually accompanied the Specialist. If conditions observed merited regulatory actions the accompanying State or local representative could intervene to ensure appropriate corrective actions were taken.

F. Baseline Data Collection Form

The Baseline Data Collection inspection form used in this project contained 47 individual data items. Forty-two (42) of the 47 individual data items were actual provisions of the Food Code. Five items (6B, 6D, 6F, 7B, and 8B) pertaining to food product holding temperatures were outside the parameters of the 1997 Food Code. These 5 data items were not included in the analysis of the baseline data because they were not requirements of the 1997 Food Code.

For each of the 47 observations, the Specialist determined whether the item was:

• IN = Item found IN COMPLIANCE with Food Code provisions.

• OUT = Item found OUT OF COMPLIANCE with Food Code provisions. An explanation was provided in the comment section on the data collection form for each OUT OF COMPLIANCE observation.

• N.O. = Item was NOT OBSERVED. The N.O. notation was used when an item was a usual practice in the food service operation, but the practice was not observed during the time of the inspection.

• N.A. = Item was NOT APPLICABLE. The N.A. notation was used when an item was not part of the food service operation.

The same data collection form was used at each establishment. The data collected for each of the 9 facility types consist of about 100 reports, each with 47 items scored either "IN", "OUT", "N.O.", or "N.A."

The completed data collection inspection forms were sent to FDA’s headquarters for entry into a central database. Before analyzing the data, a thorough review of the data collection forms was conducted to ensure reporting consistency within the established project design. FDA/CFSAN/ Division of Mathematics performed statistical planning and analysis.

BASELINE DATA COLLECTION FORM

This form was drafted for the specific purpose of collecting data regarding the occurrence at the retail level of CDC-identified risk factors associated with foodborne illness outbreaks. It was/is not intended to serve as a comprehensive, Food Code-based inspection form for food establishments.

_____ 41F or _____45F or _____ 41F + 45F is the cold holding requirement for this jurisdiction.

STATUS 1. Approved Source

_______ A. All food from Regulated Food Processing Plants/ No home prepared/canned foods.

_______ B. All Shellfish from NSSP listed sources. No recreationally caught shellfish received or sold.

________C. Game, wild mushrooms harvested with approval of Regulatory Authority.

STATUS 2. Receiving / Condition

_______A. Food received at proper temperatures/ protected from contamination during transportation and receiving/food is safe, unadulterated.

STATUS 3. Records

_______  A. Shellstock tags/labels retained for 90 days from the date the container is emptied.

________ B. As required, written documentation of parasite destruction maintained for fish products.

________C. CCP monitoring records maintained in accordance with HACCP plan when required.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 4. Proper Cooking Temp. Per PHF

_______ A. Raw eggs broken for immediate service cooked to 145F for 15 seconds, eggs not prepared for immediate service cooked to 155F for 15 seconds.

________B. Comminuted Fish, Meats, Game animals 155F for 15 seconds.

_______ C. Beef Roasts, including formed roasts, are cooked to 130F for 121 minutes or as chart specified and according to oven parameters per chart.

_______ D. Poultry; stuffed fish, meat, pasta, poultry, stuffed ratites, or stuffing containing fish, meat, poultry or ratites cooked to 165F for 15 seconds.

________E. Wild game animals cooked to 165F for 15 seconds.

________F. Raw animal foods cooked in microwave are rotated, stirred, covered, and heated to 165F.
Food is allowed to stand covered for 2 minutes after cooking.

________G. Pork, ratites, injected meats are cooked to 155F for 15 seconds.

________H. All other PHF cooked to 145F for 15 seconds.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 5. Rapid Reheating For Hot Holding

_______ A. PHF are rapidly reheated to 165F for 15 seconds.

________B. Food reheated in a microwave is heated to 165F or higher.

________C. Commercially processed ready to eat food, if reheated, held at 140F or above.

________D. Remaining unsliced portions of beef roasts are reheated for hot holding using minimum oven parameters.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 6. Proper Cooling Procedure(Note any temp above 41F on blank lines)

_______ A. Cooked PHF is cooled from 140F to 70F within 2 hours and from 70F to 41F or below within 4 hours.

_______ B. Cooked PHF is cooled from 140F to 70F within 2 hours and from 70F to 45For below within 4 hours.

________C. PHF (from ambient ingredients) is cooled to 41F or below within 4 hours.

_______ D. PHF (from ambient ingredients) is cooled to 45F or below within 4 hours.

________E. Foods received at a temperature according to Law are cooled to 41F within 4 hours.

________F. Foods received at a temperature according to Law are cooled to 45F within 4 hours.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 7. Cold Hold (41F/45F)

_______ A. PHF is maintained at 41F or below, except during preparation, cooking, cooling, or when time is used as a public health control.

________ B. PHF is maintained at 45F or below, except during preparation, cooking, cooling, or when time is used as a public health control.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 8. Hot Hold (140F)

_______A. PHF is maintained at 140F or above, except during preparation, cooking, cooling, or when time is used as a public health control.

_______B. PHF is maintained at 130F or above, except during preparation, cooking, cooling, or when time is used as a public health control.

_______C. Roasts are held at a temperature of 130F or above.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 9. Time

_______A. Ready-to-eat, PHF, held for more than 24 hours is date marked as required (prepared on-site).

________B. Ready-to-eat, PHF, held at 45F for 4 days or 41F for 7 days and discarded as required.

________C. Commercially prepared, ready-to-eat, PHF, is date marked as required.

________D. When only time is used as a public health control, food is cooked and served within 4 hours as required.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 10. Separation / Segregation / Protection

________ A. Food is protected from cross contamination by separating raw animal foods from raw ready-to-eat food and by separating raw animal foods from cooked ready-to-eat food.

________B. Raw animal foods are separated from each other during storage, preparation, holding, and display.

________C. Food is protected from environmental contamination.

________D. After being served or sold to a consumer, food is not re-served.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 11. Food-Contact Surfaces

_______A. Food-contact surfaces and utensils are clean to sight and touch and sanitized before use.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 12. Proper, Adequate Handwashing

_______A. Hands are clean and properly washed when and as required.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 13. Good Hygienic Practices

_______A. Food Employees eat, drink, and use tobacco only in designated areas / do not use a utensil more than once to taste food that is sold or served / do not handle or care for animals present. Food employees experiencing persistent sneezing, coughing, or runny nose do not work with exposed food, clean equipment, utensils, linens, unwrapped single-service or single-use articles.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 14. Prevention of Contamination From Hands

_______A. Employees do not contact exposed, ready-to-eat food with their bare hands.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 15. Handwash Facilities

________A. Handwash facilities conveniently located and accessible for employees.

________B. Handwash facilities supplied with hand cleanser / sanitary towels / hand drying devices.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

STATUS 16. Chemical

________A. No unapproved food or color additives. Sulfites are not applied to fresh fruits and vegetables intended for raw consumption.

_________B. Poisonous or toxic materials, chemicals, lubricants, pesticides, medicines, first aid supplies, and other personal care items properly identified, stored, and used.

_________C. Poisonous or toxic materials held for retail sale are properly stored.

_________________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

IV. DATA REPORTS AND DISCUSSION

The results of the 1998-1999 FDA Retail Food Program Database of Foodborne Illness Risk Factors are presented in 5 parts:

1. OVERALL PERCENT (%) of OBSERVABLE and APPLICABLE Data Items observed IN COMPLIANCE for each of the 9 Facility Types and the FDA Improvement Goal (Table 1).

2. PERCENT (%) of TOTAL OBSERVATIONS observed IN COMPLIANCE for each of the 9 Facility Types for controlling the risk factors (Table 2).

3. INDIVIDUAL DATA ITEMS needing PRIORITY ATTENTION for each of the 9 facility types (Tables 3 through 11).

4. Series of figures illustrating the percent breakdown of the most significant OUT OF COMPLIANCE data items observed for each of the 9 facility types (Figures 1 through 31).

5. Summary of the most significant OUT OF COMPLIANCE DATA items.

It is important to note that the data collection was not designed to determine an individual establishment’s compliance with its applicable regulatory ordinance. No attempt was made to determine if an establishment would have been found to be substantially in compliance with its governing ordinances.

1. OVERALL PERCENT (%) of OBSERVABLE and APPLICABLE Data Items observed IN COMPLIANCE for each of the 9 Facility Types and the Improvement Goal

The data presented in Table 1 reflect the OVERALL PERCENT of observable and applicable data items found to be IN COMPLIANCE and the FDA improvement goal. This figure should NOT be used as an indicator of the percentage of establishments IN or OUT OF COMPLIANCE.

Table 1. OVERALL PERCENT (%) of Observable and Applicable Data Items found IN COMPLIANCE and the FDA Improvement Goal, by Facility Type

		1998 Baseline* % IN COMPLIANCE Observable Items (rounded to nearest %)	FDA Improvement Goal** (rounded to nearest %)
Institutions	Hospital	80%	85%
	Nursing Home	82%	87%
	Elementary School	80%	85%
Restaurants	Fast Food	74%	81%
	Full-Service	60%	70%
Retail Store Departments	Deli	73%	80%
	Meat & Poultry	81%	86%
	Produce	76%	82%
	Seafood	83%	87%

*1998 Baseline calculation:

Percent IN COMPLIANCE =

All applicable, observable, IN COMPLIANCE data items within all risk factor categories Total number of observations

**To be consistent with the Retail Food Steering Committee's established performance goal, a 10-year goal of 25% reduction for the OUT OF COMPLIANCE values was set as the target for improvement. An example computation using Hospitals illustrates how the specific 10-year improvement goal percentages were attained:

Hospital:1998 Baseline % = 80% IN COMPLIANCE (20% OUT OF COMPLIANCE)
Improvement goal = 25% reduction in the OUT OF COMPLIANCE rate

25% of 20% = 5%
Baseline OUT OF COMPLIANCE 20% - 5% = 15%
Improvement goal = 85% IN COMPLIANCE

2. PERCENT (%) of TOTAL IN COMPLIANCE OBSERVATION FOR CONTROLLING FOODBORNE DISEASE RISK FACTORS

Table 2 provides the percent of IN COMPLIANCE observations for each of the 9 facility types as they pertain to controlling the 5 risk factors contributing to foodborne illness. Another risk factor, "Other," is included to collect data on food safety risks associated with the storage and use of chemicals.

Table 2. IN COMPLIANCE OBSERVATIONS with Respect to Controlling the RISK FACTORS, by Facility Type
PERCENT of TOTAL OBSERVATIONS (%), Number of IN COMPLIANCE OBSERVATIONS (N), TOTAL OBSERVATIONS (Total Obs)

	Institutions												Restaurants
Risk Factor	Hospital				Nursing Home				Elementary School				Fast Food				Full-Service
	%	N	Total Obs		%	N	Total Obs		%	N	Total Obs		%	N	Total Obs		%	N	Total Obs
Food from Unsafe Sources	96.8	180	186		97.2	174	179		94.4	167	177		97.0	192	198		91.0	222	244
Inadequate Cooking	93.3	322	345		93.6	264	282		94.4	167	177		89.0	243	273		84.7	332	392
Improper Holding/Time-Temperature	60.4	322	533		68.0	328	482		60.5	227	375		50.8	211	415		36.8	205	557
Contaminated Equipment/ Protection from Contamination	83.7	371	443		85.0	391	460		89.0	374	420		85.4	397	465		56.4	281	498
Poor Personal Hygiene	81.3	373	459		79.4	377	475		74.2	356	480		63.4	327	516		46.6	246	528
Other/Chemical	87.7	142	162		90.1	154	171		85.4	146	171		81.5	159	195		80.0	160	200

	Retail Food Stores
Risk Factor	Deli				Meat & Poultry				Produce				Seafood
	%	N	Total Obs		%	N	Total Obs		%	N	Total Obs		%	N	Total Obs
Food from Unsafe Sources	97.1	204	210		94.6	193	204		99.5	195	196		84.0	294	350
Inadequate Cooking	89.3	218	244		*	*	*		*	*	*		*	*	*
Improper Holding/Time-Temperature	43.3	223	515		64.6	122	189		48.7	132	271		66.3	195	294
Contaminated Equipment/Protection from Contamination	79.4	374	471		77.5	359	463		78.3	296	378		85.8	382	445
Poor Personal Hygiene	73.6	373	507		81.6	382	468		73.9	337	456		84.2	390	463
Other/Chemical	83.1	177	213		88.5	185	209		88.1	207	235		93.5	174	186
*Insufficient Data

3. INDIVIDUAL DATA ITEMS NEEDING PRIORITY ATTENTION

In order to enhance industry's managerial control and to focus regulatory intervention strategies on the reduction of OUT OF COMPLIANCE observations, both regulators and industry need to know what specific aspects are most in need of improvement. Tables 3 through 11 highlight individual data items with the highest number of OUT OF COMPLIANCE observations.

Some data items were not observed in significant numbers to support a valid interpretation. In order to determine which data items had too few observations for valid interpretation, several protocols were considered.

One of the protocols considered would have included all observable and applicable data items that had a greater than the overall OUT OF COMPLIANCE rate for each facility type. For the most compliant facility types, however, this appeared to emphasize too many data items that were not substantially OUT OF COMPLIANCE.

Another alternative was to consider the top 3 or 5 individual data items for each facility type. Using this protocol, any number that emphasized all the important items for some facility types would overemphasize some items for other facility types.

The protocol that was selected and used centered on various cutoff points using the raw number of OUT OF COMPLIANCE observations for each particular data item. The intent was to give equal emphasis to data items that had the same numbers of OUT OF COMPLIANCE observations, regardless of the facility type. In making such distinctions, it is preferable to stop at a cutoff point that has a very few or no data items just below that cutoff point. The OUT OF COMPLIANCE number that appeared to best provide a cutoff for establishing data item priorities for each of the facility types was "32". As a result, the lists of individual data items for each facility type includes all specific data items that had 32 or more OUT OF COMPLIANCE observations. (In future data collections, a different cutoff number might be used).

This protocol resulted in different numbers of priority data items being identified for each facility type. For each facility type, the individual data items were sorted from the highest to the lowest OUT OF COMPLIANCE percentage.

Abbreviations used in the graphic presentation of the data in Figures 3 - 31:

C	Celsius
D	Day
F	Fahrenheit
HR	Hour
ID	Identified
PHF	Potentially Hazardous Food
RTE	Ready-to-eat

4. PERCENT (%) BREAKDOWN OF THE MOST SIGNIFICANT OUT OF COMPLIANCE DATA ITEMS OBSERVED, PER FACILITY TYPE

The CDC-identified risk factors contributing to foodborne illness apply in some way to all retail-level food establishments. Each risk factor is composed of several individual data items (Food Code requirements) that are used to evaluate performance within retail facilities. This discussion will address each risk factor and the corresponding data items as they affect each of the facility types.

Figures 3 through 11 focused the analysis of the data on OUT OF COMPLIANCE observations as a percentage of the total number of observations for each risk factor or individual data item. This formula illustrates this comparison:

In an effort to identify and quantify the impact of each individual data item on the OUT OF COMPLIANCE percentage per risk factor for each of the facility types, a comparison of the relationship between the OUT OF COMPLIANCE individual data items was conducted. The following formula illustrates this comparison:

For example, if 50 of 100 observations of the Poor Personal Hygiene risk factor were OUT OF COMPLIANCE and 30 of the 50 were attributable to the Prevention of Hand Contamination data item, then 60% of the OUT OF COMPLIANCE observations were attributable to Prevention of Hand Contamination. These relationships are shown in the following figures. Each figure demonstrates the relative significance of the various data items for that facility type. Using these figures, needed improvement efforts can be visualized and prioritized.

The specific data items (Food Code requirements) associated with each risk factor can be found on the Baseline Data Collection Reference Sheet in Methodology, Section III. Compliance with each of the individual data items within each risk factor was evaluated for each facility type.

Of the 5 CDC-identified risk factors on this Reference Sheet, the 3 that had data items with 32 or more OUT OF COMPLIANCE OBSERVATIONS for 8 of the 9 facility types were:

• Improper Holding/Time and Temperature
• Poor Personal Hygiene
• Contaminated Equipment/Protection from Contamination

Data are graphically displayed in 2 formats:

• Bar charts (Figures 1,11, and 21) showing the percent of each risk factor’s individual data items observed as OUT OF COMPLIANCE; (Figure 31) showing the overall % summary of the most significant out of compliance observations for all facility types combined; and

• Pie charts (Figures 2 through 10, 12 through 20, and 22 through 30) showing the breakdown of the percent of a facility type’s total OUT OF COMPLIANCE observations that can be attributed to each specific individual data.

Only data items with 32 or more OUT OF COMPLIANCE observations are individually displayed. Those data items with fewer than 32 OUT OF COMPLIANCE observations were also compiled and make up the "REMAINING DATA ITEMS" category in the figures.

IMPROPER HOLDING/TIME AND TEMPERATURE

Proper temperature control is a fundamental element of food safety. Figure 1 reflects the total percent of observations OUT OF COMPLIANCE with Food Code time and temperature requirements designed to limit the growth of organisms of public health concern.

Figures 2 through 10 reflect the relationship (expressed as percentage) of OUT OF COMPLIANCE observations for the individual data items pertaining to the improper holding/time and temperature risk factor. Cold holding, proper cooling, hot holding, date marking, and time as a public health control are some of the data items included in this risk factor.

Cold holding at 41°F (5°C) was the most frequent temperature violation in 8 of the 9 facility types, and was responsible for between 21% and 54% of the OUT OF COMPLIANCE observations. Rapid cooling to 41°F (5°C) in 6 hours was a particular problem at full-service restaurants. With the exception of schools and meat markets, date marking, which is important in assessing the degree of control of pathogens such as Listeria monocytogenes, was a common OUT OF COMPLIANCE observation in all facilities.

POOR PERSONAL HYGIENE

For each of the years from 1988 through 1992, CDC reported that poor personal hygiene of food workers was the second most commonly reported practice that contributed to foodborne disease outbreaks.

Infected employees are the source of contamination in approximately 1 in 5 foodborne disease outbreaks reported in the United States with a bacterial cause. Most of these outbreaks involve enteric bacteria, i.e. fecal-oral agents spread as a result of poor personal hygiene practices by employees.

Transmission of enteric virus is also of concern. CDC now estimates that Norwalk-like viruses are the leading cause of foodborne illness in the United States and reports that hands are the most important means by which enteric viruses are transmitted.

Figure 11 reflects the total percent of observations OUT OF COMPLIANCE with the Food Code personal hygiene requirements designed to control the spread of viral, bacterial, and parasitic agents from employees to food.

Proper and adequate handwashing is critical in controlling the transmission of pathogenic organisms from employees to food. Proper handwashing, prevention of contamination from hands, good hygienic practices, and adequate/accessible facilities are included as individual data items for this risk factor.

Figures 12 through 20 indicate that lack of handwashing was a persistent OUT OF COMPLIANCE observation in all 9 facility types, accounting for 30% to 45% of the total personal hygiene OUT OF COMPLIANCE observations. Bare hand contact with ready-to-eat food was a significant problem in schools, full-service restaurants, and fast food restaurants. Practices such as eating, drinking, sneezing, coughing, and the use of tobacco were significant problems in both fast food and full-service restaurants. Inadequate handwashing facilities or the lack of supplies at handwashing facilities represented 24% of the Poor Personal Hygiene OUT OF COMPLIANCE observations for full-service restaurants. Thirty-one percent (31%) of all the Poor Personal Hygiene OUT OF COMPLIANCE observations for full-service restaurants were attributable to improper, inadequate handwashing.

CONTAMINATED EQUIPMENT/PROTECTION FROM CONTAIMINATION

Pathogens can be transferred to food from utensils, equipment, and work surfaces that have not been properly cleaned and sanitized. Cross contamination can also occur when ready-to-eat foods come in contact with raw animal foods or surfaces having contact with raw animal foods. Food may also be contaminated as a result of environmental agents such as dust, condensate, and faulty packaging. Figure 21 reflects the total percentage of observations OUT OF COMPLIANCE with Food Code requirements designed to prevent the contamination of food and food-contact surfaces.

Figures 22 through 30 present the individual data items pertaining to the contamination of food and/or food-contact surfaces for each facility type. Failure to clean and sanitize food-contact surfaces to prevent the contamination of food was a persistent OUT OF COMPLIANCE observation in all 9 establishment types. Cleaning and sanitizing was a major problem in produce departments of retail food stores and was responsible for 76% of the OUT OF COMPLIANCE observations. Separation of raw animal foods and ready-to-eat foods was a significant OUT OF COMPLIANCE observation only for full-service restaurants.

INADEQUATE COOKING

The Food Code provides specific time and temperature cooking requirements for various products prepared in retail establishments. These cooking requirements are based on a particular pathogen’s resistance to heat and its anticipated load associated with a particular food. In general, inadequate cooking was not a frequent OUT OF COMPLIANCE observation within the 6 facility types where cooking commonly occurs. (Cooking is far less common in the meat, produce, and seafood departments of retail stores). When cooking could be observed, less than 10% of the observations were OUT OF COMPLIANCE. However, in approximately 60% of the inspections conducted, the Specialist did not observe the cooking step. A discussion of the data analysis limitations resulting from data items NOT OBSERVED is found in Field and Statistical Limitations, Section V.

FOOD SOURCE

Molluscan shellfish have long been implicated in foodborne illness. Pathogens, such as Hepatitis A and Vibrio vulnificus, found in waters where these shellfish are harvested contaminate shellfish and can cause disease when shellfish are consumed raw or undercooked. To reduce the risk of illness associated with raw shellfish consumption, the waters where shellfish are harvested are monitored and regulated under the National Shellfish Sanitation Program (NSSP). Approved listings for harvesters and dealers who transport and process shellfish are available from the FDA.

When a foodborne illness associated with consumption of raw shellfish occurs, it is important to be able to determine the source of these shellfish. Harvesters, dealers, and processors are required to affix identity tags to each container of shellfish. The tag affixed to the shellfish containers identifies whether the harvester and dealer are on the NSSP list and provides specific information, such as harvest location and date of harvest, needed for a traceback investigation. Retention of shellfish tags for 90 days is required if shellstock or raw shellfish is sold or served.

Table 11, Data Reports and Discussion, Section IV, Part C, reflects the percentage of OUT OF COMPLIANCE observations pertaining to the retention of shellfish tags for 90 days. The seafood department of retail food stores was the only facility type with 32 or more OUT OF COMPLIANCE observations related to the retention of shellfish tags.

OTHER (CHEMICAL)

Contamination of food from chemical and toxic substances can cause serious illness or death. The individual data items related to this category include proper labeling of food containers, maintaining proper separation of food and toxic substances during storage, and using the products according to the specific label instructions found on the containers. The percentage of OUT OF COMPLIANCE observations pertaining to data items focused on the prevention of contamination from poisonous or toxic substances is presented in Table 2, Data Reports and Discussion, Section IV, Part B.

5. SUMMARY OF THE MOST SIGNIFICANT OUT OF COMPLIANCE DATA ITEMS

Figure 31 is presented to focus attention on those individual data items that indicate a serious loss of industry management control over behaviors and practices attributed to the occurrence of foodborne illness. This figure summarizes the overall percent (%) of OUT OF COMPLIANCE observations for the most significant individual data items, for all facility types combined.

Five individual data items exceeded a 40% OUT OF COMPLIANCE observation rate:

• Cold Holding of Potentially Hazardous Food (PHF) at 41°F (5°C) or below;
• Ready-to-eat (RTE), PHF Date Marked after 24 hours;
• Commercially Processed RTE, PHF Date Marked;
• Surfaces/Utensils Cleaned/Sanitized; and
• Proper, Adequate Handwashing

A specific focus on these individual data items will result in measurable reductions in the occurrence of OUT OF COMPLIANCE observations of risk factors.

There were 2 individual data items (shaded bars in Figure 31) that indicate a high OUT OF COMPLIANCE trend but comprise observations collected largely from one facility type:

• "PHF cooled to 70°F (21°C) in 2 hr/ 41°F (5°C) in 4 hr" for full-service restaurants; and
• "shellstock tags retained for 90 days" for seafood departments.

The "PHF cooled to 70°F (21°C) in 2 hr/ 41°F (5°C) in 4 hr" data item under the Improper Holding / Time and Temperature risk factor reflects a 73% OUT OF COMPLIANCE rate for all facility types. For full-service restaurants, 85% of the total rapid cooling observations were OUT OF COMPLIANCE (see Table 7, Data Reports and Discussion, Section IV, Part C). Rapid cooling is a high priority data item for full-service restaurants.

A total of 187 rapid cooling observations were made in all facility types. Of these, 136 were OUT OF COMPLIANCE. The 73% overall OUT OF COMPLIANCE rate for rapid cooling indicates a trend worth noting for all facility types.

The total number of rapid cooling observations for each of the remaining 8 facility types (excluding full-service restaurants) was low due to the field operational limitations of the project discussed in Field and Statistical Limitations, Section V. There were insufficient data collected to make a statistically reliable determination on the rapid cooling process for each of those 8 facility types.

Retention of shellstock tags is another area of concern that was limited to those food establishments, i.e., seafood departments of retail food stores, selling or serving raw molluscan shellfish. Because shellfish are consumed raw, it is critical that harvest records be maintained by the retail establishment for traceback purposes in the event of foodborne illness. Thirty-seven percent (37%) of OUT OF COMPLIANCE observations related to the 90-day retention of shellstock tags for facilities selling and/or serving raw molluscan shellfish.

If the safety of food in the retail segment of the food industry is to be significantly improved, the managers of retail food establishments must establish effective management control over these individual data items. The retail food industry and the regulatory community must remain focused in their efforts to reduce the individual data items having the most significant OUT OF COMPLIANCE observation rate.

More extensive guidance for regulatory and industry food program managers regarding specific actions to address these significant individual data items is provided in Recommendations, Section VI.

V. FIELD AND STATISTICAL LIMITATIONS

As with all field studies involving data analysis, some internal and external factors influenced the design and scope of the project. Sample size, industry diversity, and available resources are a few of the factors limiting the design of the project.

These factors can be placed in two broad categories:

1. Field Operational Limitations and
2. Statistical Limitations

A. Field Operational Limitations

The Specialists collecting the data encountered challenges similar to those encountered by state and local inspectors conducting routine inspections. Establishment type, the season of the year, the time of day the inspection was conducted, and the length of time available for each inspection are some of the factors that impacted the data collection.

The time of day the inspection was conducted and the length of the inspection are significant factors limiting an inspector’s observations. Often the most desirable time of day to conduct inspections is early in the morning when most of the daily preparation occurs. Inspections conducted in the afternoon hours, therefore, may not be conducive to observing and documenting critical preparation steps. In addition, the length of the inspection plays a significant role in what data can be collected. For example, as much as 6 hours may be required on site to document compliance with the Food Code critical limits for rapid cooling. Due to these field limitations, some individual data items have a high NOT OBSERVED percentage.

Some examples of individual data items that were difficult to observe included:

• Food received at proper temperature;
• Cooking of beef roasts to 130°F (54°C) for 121 minutes;
• Cooked PHF cooled from 140°F (60°C) to 70°F (21°C) within 2 hours and from 70°F (21°C) to 41°F (5°C) within 4 hours;
• PHF (from ambient ingredients) cooled to 41°F (5°C) or below in 4 hours; and
• Foods received at a temperature according to LAW cooled to 41°F (5°C) within 4 hours.

These data items require a significant period of time to assess compliance with regard to time/temperature standards or involve processes or operational steps that occur outside traditional regulatory work hours.

The cooking process provides a good illustration of how these field limitations can affect the quality of the data collection. Meat, produce, and seafood departments have been excluded from the calculation below, related to cooking time / temperature compliance, because cooking rarely occurs in these operations.

For the remaining 6 facility types, the total number of observations collected for 8 individual data items pertaining to Inadequate Cooking was 1713. Of these, 1546 were IN COMPLIANCE. Enough observations were made of final cooking temperatures to indicate a high IN COMPLIANCE trend (90.3%); but the total number of NOT OBSERVED recordings for Inadequate Cooking was 2584 out of 4800 (54%). This high NOT OBSERVED percentage may warrant a closer assessment of final cooking temperatures for future studies.

B. Statistical Limitations

• Representation of the Population of Each Facility Type

The project design provides a solid framework for assessing progress or regression in the attainment of the Healthy People 2010 and FDA National Retail Food Program performance goals presented earlier in this report. Both need a measurement of the trend, or change, in risk-related practices. The ideal design for measuring trends within the retail food industry would be one in which the number of establishments inspected within any given location would be directly related to the volume of retail food consumed within that location.

The locations for data collection were not chosen completely at random. Data collection locations were based on the Specialists’ geographical areas of responsibility. This was determined to be a reasonably convenient design for estimating risk-related practices.

The principal problem in trend measurement is uncontrolled variation in samples. The design of this project controlled this variation by the use of comparison sets. A typical comparison set consisted of 10 or more establishments of the same facility type in the same general geographic area. Where the number of establishments for a facility type within a designated geographic area was small, such as with nursing homes and hospitals, much bigger geographic areas were required than for facility types such as fast food that were more numerous. In areas with limited numbers of nursing homes and hospitals, a comparison set included a minimum of four establishments.

The establishments in each comparison set were placed in alphabetical order and sequentially numbered. The Specialists then used a table of random numbers, supplied by CFSAN's Division of Mathematics, to select the particular establishment to inspect. Comparison set establishment lists, compiled by the Specialists, have been archived and will be used again in future studies. A different establishment will be randomly selected from the same comparison set establishment list.

The randomness gives the same chance of selecting establishments having varying degrees of compliance, thus preventing selection bias. Since each comparison set is made up of similar establishments, the sample variability is greatly reduced.

The present picture of compliance with the risk factors reflects the entire U.S. to the extent that the comparison sets are representative of the overall industry. The facilities were not selected randomly from the entire U.S. and so we cannot give statistical estimates on the precision with which they estimate the entire U.S. for each facility type. We only have a common sense opinion that being spread across the U.S. in the same way the FDA spreads its Specialists should give a reasonable approximation of the U.S. compliance picture.

In summary, our selection of establishments from comparison sets spread across the U. S. provides a good basis for measuring trends and only a fair basis for measuring the state of risk-related practices.

• Precision of Percentages for Each Facility Type

Attempts were made to observe the same 42 risk-related individual data items discussed under Methodology, Section III, E, at each establishment. Many times, some items could not be observed during inspections. If all items were applicable and observable at the time of the inspection, each facility type would have produced about 4200 compliance observations. "Compliance Observations" are observations that could be judged IN COMPLIANCE or OUT OF COMPLIANCE.

The actual results showed a high of 2420 compliance observations for full-service restaurants and a low of 1540 compliance observations for the meat and poultry departments of retail food stores. By the nature of their operations, meat and poultry departments are less likely to have risk factor occurrences than full-service restaurants that involve extensive food preparation.

The precision of the percentages calculated from the data is directly related to the number of observations included in the analyses. The more observations, the greater precision of the percentages. For example, the percentage relating to overall IN COMPLIANCE observations as they appear in Table 1, Data Reports and Discussion, Part A, is more precise than the percentage relating to IN COMPLIANCE observations for any individual data item in Tables 3 through 11, Data Reports and Discussion, Part C.

An overall IN COMPLIANCE percentage for all risk factors combined will have 95 percent confidence limits of plus-or-minus 2 percentage points, for the full-service restaurants, and about plus-or-minus 3 percentage points for the meat departments. To illustrate, the overall IN COMPLIANCE percentage for Meat and Poultry Departments in Table 1 is 81%. Due to the large number of observations included in this data analysis, the precision of these data items is plus-or-minus 3 percentage points. This could result in an IN COMPLIANCE percentage as low as 78% and as high as 84%. All other facility types fall in between the plus-or-minus 2 percentage points for full-service restaurants and plus-or-minus 3 percentage points for meat departments.

Given the diversity within retail operations, it was anticipated that many individual data items would have relatively small numbers of observations. The project design, therefore, placed more emphasis on the collection of items into the 5 major risk factor categories identified in the Background, Section I. This is more statistically reliable because when grouping the data items into the risk factors, a larger pool of observations is attained.

In most cases, the Inadequate Cooking risk factor was not applicable to three departments of retail food stores - meat, produce, and seafood. Setting that combination aside, the lowest number of compliance observations found in this baseline was 162, for hospitals, with respect to "Other", and the highest number was 557, for full-service restaurants, with respect to "Improper Holding Temperatures". These compliance observations are highlighted in bold in Table 2, Data Reports and Discussion, Section IV.

These numbers are sufficient for identifying the risk factors that are frequently, or relatively seldom, in compliance and for detecting trends over time. The methodology used to determine sample size minimizes the potential for certain errors in data analysis. For example, it would be extremely unlikely for the data collected through this project to indicate that a risk factor was 70% or higher IN COMPLIANCE when the actual IN COMPLIANCE rate was 60%. The probability of that happening for any collection of 100 or more observations is less than 1.5%. With compliance observation counts between 162 and 557, one should, therefore, have high confidence in the general conclusions about the state of compliance with the 5 CDC-identified risk factors for each facility type.

For any one of the 42 individual data items, the percentage IN COMPLIANCE is less precise due to the fewer number of observations available for analysis. The baseline data for an individual data item had less than a 5% chance of making the 60%-70% data analysis error described in the previous paragraph, provided the compliance observations numbered at least 60.

• The 9 facility types had between 14 and 22 individual data items that had 60 or more compliance observations. The compliance percentages for these individual data items are very important because the number of observations is high.

• Individual data items that have 30 to 59 observations are moderately important. Since these individual data items have fewer observations, there is a 5 to 10 percent chance of being wrong by 10 or more percentage points.

• Individual data items having fewer than 30 compliance observations should be viewed as very rough estimates of observed trends.

Types of Comparison Not Supported by the Statistical Design

This project is designed to establish a national baseline on the occurrence of foodborne disease risk factors within the retail segment of the food industry. The data set is too small to support comparisons of individual Specialists geographical areas, states, cities or even regions of the U.S. Not only would it be a flagrant abuse of statistics, but such comparisons might be combined with other information, such as the locations of FDA Retail Food Specialists, to identify some of the likely comparison sets. This information would bias future studies.

In addition, the project is not designed to support comparisons of chains of fast food restaurants or chains of grocery stores. There is no statistical justification for looking at reduced sets of results particular to, e.g., two chains of restaurants and drawing conclusions from the differences.

VI. RECOMMENDATIONS

The Food and Drug Administration (FDA) is responsible for setting standards for the safe production of foods and advising state and local governments on food safety standards for institutional food service establishments, restaurants, retail food stores, and other retail food establishments. In this advisory role FDA works closely with other federal agencies to provide guidance and assistance that will enhance the regulatory programs of state and local jurisdictions. The following recommendations are intended to focus and enhance this effort.

If the safety of food in the retail segment of the food industry is to be significantly improved, the managers of retail food establishments must establish effective management control over these individual data items. The retail food industry and the regulatory community must remain focused in their efforts to reduce the individual data items having the most significant OUT OF COMPLIANCE observation rate.

REGULATORY RECOMMENDATIONS

One of the objectives of this project is to enlist the help of state, local, and tribal regulatory programs to reduce the occurrence of risk factors within their own jurisdictions. One way these jurisdictions can assist in this effort is by establishing a baseline of risk factor compliance within individual jurisdictions and by evaluating the design, management, and operation of their regulatory food programs through the use of the FDA’s Recommended National Retail Food Regulatory Program Standards (Standards).

The Standards consist of 9 standards that together form the basis for a pro-active food regulatory program. These standards are: Regulatory Foundation, Trained Regulatory Staff, HACCP-based Inspection, Uniform Inspection Program, Foodborne Illness Surveillance, Compliance and Enforcement, Industry Recognition, Program Resources and Program Assessment.

The last of these Standards, Program Assessment, addresses a process for program self-assessment based on the first 8 standards and their impact on reducing the occurrences of risk factors. A protocol to assist regulatory jurisdictions in establishing a baseline on the occurrence of risk factors is provided in the Appendix G of the Standards document.

Managers of regulatory inspection programs are encouraged to review existing practices and procedures to ensure that current program activities target reducing the occurrence of those risk factors identified in Figure 31, Data Reports and Discussion Section IV, E. The focus of the regulatory program should be on the reduction of those risk factors applicable to each establishment type.

Standard Operating Procedures (SOPs) for regulatory programs should include provisions that guide the field inspection process to achieve the desired results.

Regulatory food programs should incorporate practices and procedures that:

• Establish a regulatory foundation that reflects current scientific data and uniform national requirements.
• Provide flexible work schedules for inspectors to accommodate the need to observe critical food preparation practices or operational steps during non-traditional regulatory work hours.
• Include a method for assessing difficult to observe risk factors so appropriate adjustments can be made to inspection schedules.
• Train and appropriately equip regulatory employees to determine the compliance status of the risk factors.
• Use an inspection approach that focuses on the risk factors most often associated with foodborne illness.
• Determine and document the compliance status [IN or OUT OF COMPLIANCE , NOT OBSERVED, NOT APPLICABLE notation] for each risk factor during routine inspections. Inspectors can document cooling times by leaving data loggers at one site while the inspector proceeds to a second inspection site. The inspector can return later to retrieve the data logger at the first site.
• Take appropriate corrective action when risk factors are determined to be OUT OF COMPLIANCE.
• Assist food establishment operators in their efforts to develop SOPs and action plans for attaining managerial control for recurring risk factors. Date marking and temperature logs should be used, whenever possible, to determine if time/temperature requirements are being met.

Program managers are also encouraged to conduct periodic self-assessments of their program to determine effectiveness and establish a compliance baseline for the risk factors. For more information on conducting program self-assessment refer to the FDA’s Recommended National Retail Food Regulatory Program Standards at www.cfsan.fda.gov or contact an FDA Regional Food Specialist in your geographical area.

INDUSTRY RECOMMENDATIONS

The following recommendations are offered to assist industry efforts to establish active managerial control over the most prevalent OUT OF COMPLIANCE risk factors identified by this report.

The management of each food establishment should:

• Develop and implement Standard Operating Procedures (SOPs) that address the risk factors. These SOPs should detail procedures specific to the operation for time/temperature control of potentially hazardous food; personal hygiene, and measures to prevent food from being contaminated.

• Provide all employees with specific training to implement the SOPs.

• Provide the necessary resources, equipment, and supplies to implement the SOPs. Items such as temperature measurement devices, preferably thermocouples, test papers, temperature logs, hand soap, towels, and chemical sanitizers are crucial to the successful control of specific risk factors.

• Assess SOPs to ensure control over all risk factors. Critical limits and measurable standards for control of the risk factors should be incorporated into SOPs. Critical limits provide a means for measuring the effectiveness of an establishment’s food safety procedures.

• Establish monitoring procedures that focus on critical processes and practices. Monitoring procedures will only be effective if employees are given the knowledge, skills, and responsibility for food safety.

• Identify methods to routinely assess the effectiveness of the SOPs. This assessment approach could be based on an internal review, regulatory inspection results, or third party evaluation.

Enhancing industry’s active managerial control of all identified risk factors could have a significant impact on reducing their occurrence. Focusing intervention strategies that are designed to reduce the occurrence of high OUT OF COMPLIANCE data items will have the most significant impact on advancing the development of industry food safety systems. Improvements in industry’s active managerial control of the risk factors will result in a greater level of consumer protection.

Enhancing the effectiveness of the nation’s retail food safety system requires a sound science based approach upon which regulators and industry can base their food safety programs. The provisions of the Food Code provide a solid foundation, for both regulatory and industry managers, to develop food safety systems designed to control these risk factors.

VII. AREAS FOR FUTURE STUDY

The intent of establishing a national baseline of current compliance with Food Code provisions that address the CDC-identified risk factors is to track the change in the occurrence of risk factors through future comparison studies. It is hoped that implementation of this study’s recommended intervention strategies will decrease the incidence of risk factors in retail food settings by 25% by the year 2010. Two more comparison projects are being planned by FDA to gauge the Agency’s progress via a combined federal, state, local, and tribal effort toward meeting this goal. Theoretically, comparison projects using the initial data as the baseline could be repeated every few years until either all establishments within each comparison set had been inspected or had gone out of business.

In considering comparison studies, the Agency will pay particular attention to advances in food safety technology and research and their effect on various critical limits and intervention strategies required to control risk factors. Any future comparison study design will accommodate such changes by adding new data items to the existing collection form. In this way, future data collection will not only allow for comparison with the initial data set, but will establish a basis for future comparison of any new data items as well.

Comparison projects might involve concentrating on specific observations from the initial baseline collection where data were either insufficient or on observations that had a high incidence of not being observed. Such observations might include cooling, cooking, and/or receiving temperatures. Changes in various aspects of the initial project methodology, such as the time and duration of inspections, might facilitate better data collection in these areas.

Since it is hoped that implementing intervention strategies suggested in this report will have a direct, positive impact on the reduction of risk factors and foodborne illness in the 9 facility types tested, inclusion of other facility types in future studies might be suggested. Facility types under consideration for future studies include day care facilities, secondary schools, camps, and/or temporary establishments.

Future studies could also be concentrated on specific foods and the occurrence of specific observations relating to risk factors that lead to foodborne illness in those foods. For example, practices involving the receipt, storage, and preparation of eggs might be specifically addressed as they relate to Salmonella Enterididis infections. Other possibilities include cold holding of raw seed sprouts and/or cold holding and date marking of foods traditionally implicated in Listeriosis outbreaks.

A comparison study to gauge the effectiveness of implementing various intervention strategies on the overall reduction in the occurrence of risk factors is also feasible. These intervention strategies might include implementation of HACCP plans and Standard Operating Procedures to address Food code provisions, such as no bare hand contact with ready-to-eat foods, and Food Protection Manager Certification.

Hypertext updated by bap/dms/cjm 2000-SEP-11