This publication is distributed by the U.S. Department of Transportation, National Highway Traffic Safety Administration, in the interest of information exchange. The opinions, findings, and conclusions expressed in this publication are those of the author(s) and not necessarily those of the Department of Transportation or the National Highway Traffic Safety Administration. The United States Government assumes no liability for its contents or use thereof. If trade or manufacturer's names or products are mentioned, it is only because they are considered essential to the objective of the publication and should not be considered an endorsement. The United States Government does not endorse products or manufacturers.

As NHTSA and the nation's Emergency Medical Services (EMS) family celebrate the first thirty years of organized EMS and prepare for the many challenges of appropriately serving our communities into the Twenty First Century, our continuing goal is to reduce unnecessary death and disability. The recently released EMS Agenda for the Future (NHTSA, Fall, 1996) broadens that goal to protecting the communities' health.

Our continuing partnership with the Health Resources and Services Administration, Maternal and Child Health Bureau, provided us with the opportunity to take an important step in pursuit of this expanded goal, through the development of the "Leadership Guide to Quality Improvement for Emergency Medical Services Systems".

Quality is anything that enhances the product or services from the viewpoint of the customer (patient). In EMS, our customer is not only the individual patient we serve , but the entire community. We need to align our values with community needs.

With the rapidly changing health care environment, EMS must determine how it can best serve community health, while remaining the public's emergency medical safety net. We need to provide for improved health, with improved quality and improved efficiency, while continuously monitoring our progress.

This "Leadership Guide to Quality Improvement for Emergency Medical Services (EMS) Systems" was developed to serve as a template for EMS managers who want to establish and maintain a program for continuously monitoring and improving the quality of patient care and support services in all parts of the EMS system. It encourages EMS leaders to integrate continuous quality improvement practices as essential parts of normal EMS routines.

The Leadership Guide is presented in a loose-leaf format to allow for addition of new materials and notes resulting from continued study and growth in the area of quality improvement.

NHTSA plans to develop additional materials and programs to contribute to continued growth in this important area and we would strongly encourage EMS leaders at all levels to embark on this journey with us. We hope this Leadership Guide will be a useful tool as you and your respective systems shape the future of EMS.

Contract Information

Participants

Expert Writing Panel

National Review Team

Introduction

Quality Improvement Background

The Baldrige Categories

Leadership

Information and Analysis

Strategic Quality Planning

Human Resource Development and Management

EMS Process Management

EMS System Results

Satisfaction of Patients and Other Stakeholders

Assessing Progress

QI Tools and Techniques

Multivoting

Run Chart

Histogram

Cause-and-Effect Diagram

Flowchart

Pareto Diagram

Quality Improvement Terms

Related Literature

Co-Principal Investigators

James N. Eastham, Jr., Sc.D., Associate Professor of Emergency Health Services, University of Maryland - Baltimore County, Baltimore, Maryland.

Howard R. Champion, F.R.C.S, F.A.C.S., Visiting Scholar, National Study Center for Trauma and EMS, University of Maryland at Baltimore, Baltimore, Maryland; Professor of Surgery and Chief of Trauma, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Project Medical Director

Robert R. Bass, M.D., Director, Maryland Institute for Emergency Medical Services Systems, Baltimore, Maryland.

Project Manager

Patricia S. Gainer, J.D., M.P.A., National Study Center for Trauma and EMS, University of Maryland at Baltimore, Baltimore, Maryland.

Research & Technical Assistant

Jason Paluck, Emergency Health Services Department, University of Maryland-Baltimore County, Baltimore, Maryland.

Department of Transportation /National Highway Traffic Safety Administration
Division of Emergency Medical Services

EMS Specialists

Susan McHenry
Valerie Gompf

Directors

Jeffrey P. Michael, Ph.D.
Sue Ryan

Michael F. Altieri, M.D., F.A.A.P., Chief, Pediatric Emergency Medicine, Department of Emergency Medicine, Fairfax Hospital, Fairfax, Virginia; Associate Clinical Professor of Emergency Medicine and Associate Clinical Professor of Pediatrics - Georgetown University and George Washington University. Virginia State Critical Care Committee.

Wayne S. Copes, Ph.D., Vice President, Tri-Analytics, Inc., specializing in Medical Database Management, Research and Software Development. Director, Maryland Cancer Registry; Manager, Pennsylvania Trauma Center Registry; and Technical Director, Project IMPACT (national critical care database jointly sponsored with the Society of Critical Care Medicine).

Steven J. Davidson, M.D., M.B.A., Chairman, Emergency Medicine, Maimonides Medical Center, Brooklyn, New York and Professor of Emergency Medicine at SUNY-HSCB. Past Medical Director, Philadelphia Fire Department (1983-94). Fellow, American College of Emergency Physicians; Senior Director, The American Board of Emergency Medicine; Team Leader, Pennsylvania Interdisciplinary Team; Institute for Healthcare Improvement Professional Education Collaborative (1994-95).

Drew Dawson, Chief, Emergency Medical Services Bureau, Montana Department of Health and Environmental Services, Helena, Montana. President, National Association of State EMS Directors.

Lynn Eastes, R.N., M.S., Trauma Coordinator/Quality Management Coordinator, Trauma Program, Oregon Health Sciences University. Oregon State Trauma Advisory Board, Oregon State Quality Improvement Subcommittee.

John E. Gough, M.D., Assistant Medical Director, Division of EMS, Department of Emergency Medicine, East Carolina University School of Medicine, Greenville, North Carolina.

Lieutenant John Maddox, NREMT-P, ACLS, BTLS, PALS, CPR, HazMat OPS, Fire Officer I, and Instructor III. EMS Supervisor, Prince William County, Virginia Department of Fire and Rescue; CISD Coordinator; and Quality Improvement Director.

David R. Miller, President, Healthspan Transportation Services, a mobile health service providing ambulance, specialized transport and support services; Vice President for Allina Health System, an integrated health system including hospitals, physicians, and health plans, St. Paul, Minnesota.

Robert Swor, D.O., Director, EMS Program, Department of Emergency Medicine, William Beaumont Hospital, Royal Oak, Michigan. President, National Association of EMS Physicians.

Introduction

Since the enactment of the National Highway Safety Act of 1966, and the formal beginning of emergency medical services (EMS), the common goal of EMS systems has been to reduce unnecessary death and disability. While this goal remains constant, we are confronted more than ever before by the public with the demand that EMS provide the highest quality service at the lowest possible cost. There are clear expectations for improved health, improved quality and improved efficiency.

This manual provides a useful guide for EMS system leaders to use to improve quality within their organizations. This manual encourages EMS leaders to integrate continuous quality improvement (QI) practices into EMS operations to the extent that those practices become an essential and seamless part of normal EMS routines. Specific activities are suggested within three developmental stages. While specific activities may differ depending on the jurisdiction of the organization, the developmental stages of QI integration will be the same for local, regional, or statewide EMS organizations. These developmental stages are: 1) building potential for success by developing an awareness and appreciation that QI is a worthwhile endeavor; 2) expanding workforce knowledge of and capability in QI practices and techniques; and 3) fully integrating the strategic quality planning process and related quality improvement actions into the daily EMS operation.

This document uses the Malcolm Baldrige Quality Program as a model to guide your organization's QI efforts and to evaluate your progress. The Baldrige Award was established in 1987 through federal legislation as a way to: 1) promote awareness of the importance of quality improvement; 2) recognize organizations that make substantial improvement in products, services and performance; and 3) foster inter-organizational information sharing on best practices. The Baldrige Award launched a special initiative in 1994 to extend the program to the realm of health care.

The Baldrige program identifies seven key action areas or categories. The QI information in this manual is organized according to those seven categories, as follows:

• Leadership involves efforts by senior leadership and management leading by example to integrate quality improvement into the strategic planning process and throughout the entire organization and to promote quality values and QI techniques in work practices.

• Information & Analysis concerns managing and using the data needed for effective QI. Since quality improvement is based on management by fact, information and analyses are critical to QI success.

• Strategic Quality Planning involves three major components: 1) developing long- and short-term organizational objectives for structural, performance and outcome quality standards; 2) identifying ways to achieve those objectives, and 3) measuring the effectiveness of the system in achieving quality standards.

• Human Resource Development and Management involves working to develop the full potential of the EMS workforce. This effort is guided by the principle that the entire EMS workforce is motivated to achieve new levels of service and value.

• EMS Process Management concerns the creation and maintenance of high quality services. Within the context of quality improvement, process management refers to the improvement of work activities and work flow across functional or department boundaries.

• EMS System Results entails assessing the quality results achieved and examining the organization's success at achieving quality improvement.

• Satisfaction of Patients and Other Stakeholders involves ensuring ongoing satisfaction by those internal and external to the EMS system with the services provided.

Health care organizations that follow the Baldrige program have the option of asking for an external review of their progress. They report benefits gained by simply applying the Baldrige guidelines and recommendations, including improvements in: service and patient care delivery; economic efficiency and/or profitability; patient and community satisfaction and loyalty; and health outcomes.

For more information regarding the Malcolm Baldrige National Quality Award, contact the National Institute of Standards and Technology on the following Homepage: www.quality.nist.gov

Tracking your Progress

It is important to begin with the endpoint in mind. Just as a blueprint is needed to build a house, you should have a mental blueprint of how your EMS organization or system will look and operate once QI strategies and techniques are part of EMS planning and operations.

Experience from other fields shows that integrating QI into an organization or system takes several years. Remember that thousands of organizations around the world in a wide variety of industries, including health care, have been successful in developing a strong focus on quality. Their success should encourage similar efforts in EMS.

There are three significant stages of development as EMS systems begin to implement QI.

Stage I: Building Potential for Success

Your EMS organization or system will begin its QI journey when the leaders begin the process of learning about the theory, techniques and benefits of quality improvement. By the end of Stage I, senior leaders should be able to articulate these benefits, believe that these benefits can be achieved, and have a plan of action for achievement.

Stage I requires strong leadership and commitment at the local, regional and state level to: 1) learn and understand quality improvement strategies; 2) assess thoroughly the present situation of each EMS organization or system regarding quality levels; and 3) establish action plans for training and orientation in quality improvement.

Stage II: Knowledge Expansion

Stage II establishes the structural foundation necessary to fully integrate QI into the strategic planning process. In this stage, emphasis is placed on ensuring that the entire workforce of an EMS organization or system is informed about and participates in the development of the strategic quality improvement plan. Workforce members need a working knowledge of basic QI philosophy, tools, and techniques so that they can be full partners in the strategic quality improvement planning process. At the end of this phase, all EMS workforce members should be able to identify their internal and external customers, how to measure the quality of the services provided or received, and how to identify and resolve quality problems in their own work. There should also be a new sense of openness and partnership between staff and management within organizations and among local, regional and statewide systems.

By the end of Stage II, local, regional and state EMS systems should have in place the structure and process that allows evaluation and comparison of the quality indicators identified in the strategic plan. EMS organizations should be able to take action to attain the quality targets identified in their plans, determine the success of their efforts, and, when negative results occur, revise and restart their action plans for improvement. The efforts of initial QI teams should also begin to show benefits, typically in the areas of measuring and reporting critical indicator quality levels.

Stage III: Integration and Commitment

At the final developmental stage, QI activities clearly impact the organization's work processes, for example, changes in work site management and evaluation of work; use of self-directed teams; and re-alignment of work processes to achieve new quality targets on critical indicators. The workforce is empowered to take action in team settings to identify, deploy and evaluate new production methods. Teams can take self-correcting action by accessing timely information on performance levels for key quality indicators. At all levels, traditional supervision gives way to leadership that helps the workforce maintain and improve the quality of their work. At the state and regional level, there is less emphasis on regulatory inspection of local agencies and more emphasis on providing resources, comparative information and coaching to local agencies that are accountable for developing and following their own strategic QI plans.

Communication among individual members of the EMS workforce, self-directed work teams, organizations and systems speeds adoption of successful innovations. There is increased benchmarking within the organization, as well as with other organizations (inside and outside of EMS ). Regional and state EMS organizations support inter-organizational benchmarking and communication. At all levels, the EMS system promotes and encourages continuous improvement as a fundamental operating strategy.

The most important Stage III milestone is achieving results from QI efforts; for example, increased patient satisfaction and health status from improved EMS services and quality of care. Economic benefits also result, including cost savings, increased profitability or operating surplus, and more efficient use of resources.

The Importance of Results!

As you look at each of the Baldrige categories and their implications for structural and procedural changes in your EMS organization, keep in mind a very important concept: the most important results for achievement are improved health of EMS patients, improved quality of EMS services, and improved efficiency of resource use. Simply completing the steps needed to implement quality management is not the most important measure of progress. As Dr. Donald Berwick, a national leader in health care quality improvement, has aptly pointed out -- we have accomplished little if our efforts to improve quality only result in our ability to collect and analyze more data.

Summary

This chapter explored the need to develop quality improvement capabilities in EMS. The Baldrige Quality Program and its seven categories for action were introduced as a method for guiding EMS quality improvement efforts. The three developmental stages for quality improvement efforts were also presented.

The remainder of this manual covers the application of each of the seven Baldrige categories to EMS. The manual also includes a glossary of QI terms, a review of pertinent QI literature, a set of basic QI tools, and a series of evaluative questions and examples for EMS systems.

Background

Deming's Principles Applied to EMS
EMS can and should be made better Efforts to improve EMS quality should be continuous Every EMS process can yield data and information on how well the process works Data and information are essential to improving EMS quality

There is an increasing focus on "quality" throughout United States. When talking about "Total Quality Management", "Continuous Quality Improvement", or any other name given to the quality movement, the common thread is meeting the needs of those who pay for and use the services and products provided by an organization. All types of industries, including health care, have lowered costs and improved the quality of their operations and products by working to meet the needs of the people they serve.

Many books have been written describing the philosophy and methods used in the quality movement. This section is not intended to be a substitute for those works, but rather provides a brief overview of quality management principles by reviewing the work of three leaders who have shaped the discipline. These three pioneers all stressed the importance of management awareness and leadership in promoting quality.

W. Edwards Deming. Deming began working in Japan in 1950 and was instrumental in building the Japanese industry into an economic world power. His strongly humanistic philosophy is based on the idea that problems in a production process are due to flaws in the design of the system, as opposed to being rooted in the motivation or professional commitment of the workforce. Under Deming's approach, quality is maintained and improved when leaders, managers and the workforce understand and commit to constant customer satisfaction through continuous quality improvement.

Deming and his colleague, Shewhart, promoted the PDCA cycle -- Plan, Do, Check and Act. In an EMS operation, we can

PLAN to implement a policy to improve quality and/or decrease the cost of providing services. After the plan is developed, we DO it by putting the plan into action and then CHECK to see if our plan has worked. Finally, we ACT either to stabilize the improvement that occurred or to determine what went wrong if the gains we planned for did not materialize. PDCA is a continuous cycle; any improvement realized by carrying out one PDCA cycle will become the baseline for an improvement target on the next PDCA cycle. The process of improvement (PDCA) is never ending, although the dramatic improvements of initial PDCA efforts may be hard to sustain.

Deming also developed his famous "14 points" to transform management practices. Those points are applied to EMS and summarized below.

1. Create constancy of purpose.

   An EMS organization's highest priority is to provide the best quality medical care and/or transportation services to its community at the lowest cost possible. An EMS organization is responsible to both its community and its own workforce to maintain a high level of excellence and value. An EMS organization must strive to maximize efficiency and effectiveness through constant improvement.

2. 2. Adopt the new philosophy.

   Everyone working in EMS can find ways to promote quality and efficiency, to improve all aspects of the EMS system, and to promote excellence and personal accountability. Pride of workmanship must be emphasized from recruitment to retirement. By their behavior, leaders set the standard for all workers.

   PDCA In Action

   EMS managers, reviewing ambulance response performance over time, discover that the goal of on-scene arrival within 6 minutes after notification only happens about 20% of the time. In response, a team of managers and medics develop a PLAN to improve this rate to 90% of the time. The plan requires new staffing patterns and a new vehicle deployment strategy. The plan is put into action (DO) on a trial basis during which the teams CHECKS response time information. If the desired result is achieved, the managers ACT to stabilize the new response capability. If the plan does not work, the team again ACTS to understand what went wrong, to learn from mistakes, and to try a new revised plan.

3. Cease dependence on inspection to achieve quality.

   Reliance on routine 100 % inspection to improve quality (i.e., a search for errors, problems, or deficiencies) assumes that human performance error or machine failure is highly likely. Instead, there should be a continuous effort to minimize human error and machine failure. As Deming points out, "Inspection (as the sole means) to improve quality is too late!" Lasting quality comes not from inspection, but from improvements in the system. For example, documenting deficiencies in EMS record-keeping does not, by itself, generate ideas that would make the task of record-keeping less error-prone. A quality-driven approach might, instead, encourage development of clear and simple record-keeping forms that minimize or eliminate the likelihood of mistakes.

4. Do not purchase on the basis of price tag alone.

   Purchasers must account for the quality of the item being purchased, as well as the cost. High quality organizations tend to think of their suppliers as "partners" in their operation. Successful partnerships require clear and specific performance standards and feedback on whether those standards are being met. Supplier performance can also be improved through an understanding of supplier QI efforts; longer-term contracts that include explicit milestones for improvement in key features; joint planning for improvement; and joint improvement activities.

5. Constantly improve the system of production and service.

   Quality can be built into all EMS activities and services and can be assured by continuous examination to identify potential improvements. This requires close cooperation between those who provide services and those who consume services. Improved efficiency and service can result from focusing not only on achieving present performance targets, but more importantly, by breaking through existing performance levels to new, higher levels.

6. Institute QI training on the job.

   On-the-job QI training ensures that every worker has a thorough understanding of: 1) the needs of those who use and/or pay for EMS services; 2) how to meet those needs; and 3) how to improve the system's ability to meet those needs. Incorporating QI into the fabric of each job can speed learning.

7. Institute effective leadership.

   The job of management is leadership. Effective leaders are thoroughly knowledgeable about the work being done and understand the environment and complexities with which their workers must contend.

   Leaders create the opportunity for workers to suggest improvements and act quickly to make needed changes in production process. Leaders are concerned with success as much as with failure and focus not only on understanding "substandard", but also "super-standard" performance. The effective leader also creates opportunities for below- and above-average performers to interact and identify opportunities for improvement.

8. Drive out fear.

   The Japanese have a saying: "Every defect is a treasure", meaning that errors and failures are opportunities for improvement. Errors or problems can help identify more fundamental or systemic root causes and ways to improve the system.

   Yet, fear of identifying problems or needed changes can kill QI programs! Also, some may feel that the idea of making improvements is an admission that the current way of doing things is flawed or that those responsible are poor performers.

   Improved performance cannot occur unless workers feel comfortable that they can speak truthfully and are confident that their suggestions will be taken seriously. Managers and workforce members must assume that everyone in the EMS system is interested in doing his or her best!

9. Break down barriers between departments.

   Barriers between organizations or between departments within one organization are obstacles to effective QI. Inter-departmental or intra-organizational friction or lack of cooperation result in waste, errors, delay, and unnecessary duplication of effort. A continuous and lasting QI program requires teamwork that crosses traditional organizational lines. QI requires that all workforce members, depart-ments, and units share a unified purpose, direction, and commitment to improve the organization. Intra-organizational pathways are developed and cultivated as mechanisms by which to improve performance.

10. Eliminate slogans, exhortations, and targets for the workforce for zero defects and new levels of productivity.

    The problem with such exhortations is that they put the burden for quality on worker performance instead of poor system design. QI requires that the organization focus on improving its work processes. In so doing, service quality will increase, productivity and efficiency will rise, and waste will diminish.

11. Eliminate management by numbers and objective. Substitute leadership!

    For Deming, work production standards and rates, tied to incentive pay, are inappropriate because they burn out the workforce in the long run. Alternatively, a team effort should be marshaled to increase quality, which will lead to increased profits/savings that can then be translated to, for example, higher salaries or better benefits. Improvement efforts should emphasize improving processes; the outcome numbers will change as a consequence.

12. Remove barriers to pride of workmanship.

    The workforce is the most important component of the EMS system. EMS cannot function properly without workers who are proud of their work and who feel respected as individuals and professionals. Managers can help workers be successful by making sure that job responsibilities and performance standards are clearly understood; building strong relationships between management and the workforce; and providing workers with the best tools, instruments, supplies, and information possible.

13. Institute a vigorous program of education and self-improvement.

    EMS workers can improve their lives through education and ever-broadening career and life opportunities. EMS needs not just good people; it needs people who are growing through education and life experiences. Management, as well as members of the workforce, must continue to experience new learning and growth.

14. Put everybody to work to accomplish the transformation.

    The essence of QI is an organization-wide focus on meeting the needs of those who use and/or pay for EMS services. Effective quality management programs go beyond emphasizing one or two efforts or areas to improve performance. Every activity, every process and every job in EMS can be improved. Everyone within the organization can be given an opportunity to understand the QI program and their individual role within that effort. Improvement teams that include broad representation throughout the organization can help ensure success of initial efforts and create opportunities for cross-disciplinary dialogue and information exchange.

Joseph Juran. Juran's approach is based on the idea that the QI program must reflect the strong inter-dependency that exists among all of the operations within an organization's production processes.

According to Juran, Quality Planning is the process of understanding what the customer needs and designing all aspects of a system that is able to meet those needs reliably. Designing an EMS system to do anything less is wasteful because it does not meet patient need. Once the system is put into operation, Quality Control is used to constantly monitor performance for compliance with the original design standards. If performance falls short of the standard, plans are put into action to deal quickly with the problem. Quality control puts the system back into a state of "control", i.e., the way it was designed to operate. Quality Improvement occurs when new, previously unobtained, levels of performance ~ Breakthrough Performance ~ are achieved!

Juran also proposed the idea of the "Vital Few and the Useful Many" that helps prioritize which QI projects should be undertaken. In any organization, there will be a lengthy list of possible ideas for improvement. Since the resources to actually implement new ideas is limited, however, leaders must choose those vital few projects that will have the greatest impact on improving ability to meet customer needs. The criteria for selecting QI projects includes potential impact on meeting customer needs, cutting waste, or marshaling the necessary resources required by the project.

Juran also developed the idea of instituting a leadership group or "Quality Council", consisting of the organization's senior executive staff. The Quality Council is typically charged with the responsibility for designing the overall strategy for quality planning, control and improvement. Senior leadership involvement is a must since QI activities are as important as other management tasks (e.g., budgeting, human resource management, purchasing and training), and leaders can integrate QI into every aspect of EMS operations.

The Juran Trilogy
Quality Planning - initial design of operations based on meeting customer/ consumer needs. Quality Control - continuously monitoring how the system is maintaining its customer/consumer-dictated performance levels, with corrective action when needed. Quality Improvement - creation of special teams to plan, test, and implement new methods to reach unprecedented levels of performance.

Philip Crosby. Crosby coined the phrase "quality is free", meaning that the absence or lack of quality is costly to an organization, e.g., in money spent on doing things wrong, over, or inefficiently. Conversely, spending money to improve quality, e.g., to reduce waste or improve efficiency, saves money in the long run.

According to Crosby, ensuring quality should occur primarily at the design phase. Rather than spending time and money on finding and fixing mistakes and errors, Crosby advocates organizational changes to encourage doing a job right the first time. Crosby challenges organizations to think of how processes can be designed or re-designed to reduce errors and defects to reach a goal of "zero defects".

Crosby believes managers' policies and actions indicate their commitment to quality. He also advocates a step-by-step approach for educating the entire workforce about quality principles, extensive measurement to document system failures, and formal programs to redesign faulty production processes.

The QI principles and methods of Deming, Juran and Crosby provide a basic foundation for most QI efforts. In this document, QI principles and methods are applied to EMS organizations and systems so that EMS can begin the journey into a new era of quality. Reading about the work of others provides a start, but in the long run, it will be unwavering leadership that will provide the most significant ingredient for success. That leadership can be achieved through a personal and professional commitment to learn and apply these principles.

The following "success stories" show how QI principles have been used by EMS systems to improve their services. Each of these projects was developed based on customer needs, used data to drive the QI process, and relied on a collaborative, multi-disciplinary approach to improve quality.

"The Family Safety Program was developed based on direct input from one of our customers, the Public Health Department. We asked them how our ambulance service could better benefit the community in-between the times we were responding on calls. Since injuries to children in the community parks had been increasing, the Department asked for our help.

"With that request, we looked at our ambulance response data and found that most incidences in the parks were biking and in-line skating injuries. We then assembled a multi-disciplinary group to help us define the scope of the problem and identify possible solutions. The group consisted of paramedics, public health professionals, park and recreation staff, police department staff, school teachers and an epidemiologist. Led by the ambulance staff, the group agreed that a training program designed for kids about biking, in-line skating and playground safety might be beneficial. The group also agreed to collect data to determine if injuries were reduced as a result of the program; to reinforce the training in a creative way; and to secure funding to develop the program.

"In cooperation with the school teachers, the paramedics developed and taught a training program in the parks between calls. Each training session took about 15 minutes, after which each child was given a cool-looking reflective sticker for their helmet. The stickers helped us identify the kids who had been trained. Also, kids who were 'caught safe' through the summer received coupons for free ice cream.

A leading health care system provided funds for the program. Many local businesses donated funds, as well as helmets, bikes, and ice cream.

"The epidemiologist developed a survey tool; observers collected data at the beginning and the end of the summer and also collected data from community where there was no such training. When the data were finally compiled, the results demonstrated a statistically significant reduction in biking and in-line skating injuries."

"The goal of the Vehicle Damage Reduction Project was to reduce the costs of damage to the ambulances. Paramedics, mechanics and management formed a process improvement team to review vehicle damage data. The team determined that the greatest cause for damage was backing accidents: ambulances backing into their stalls would bang into the cement walls, damaging the bumpers. The team took two actions. First, they created a policy that required the EMS worker sitting in the passenger seat to get out of the ambulance and spot for the driver during back- up. Second, the team mounted tires on the walls to reduce damage to the bumpers. As a result, we saved money and reduced the anxiety of the ambulance crews and mechanics."

"Understanding the goals of managed care, we envisioned that paramedics could reach beyond their normal emergency medical skills to help senior citizens stay healthy and live independently in our community. We put that vision into action through the Senior Paramedic Assessment and Referral (SPAR) Pilot Project. This project linked our ambulance service, a home care agency, a hospital and a managed care organization.

"The project is centered around the fact that paramedics respond daily to help seniors manage sudden illness and injury. Some of these seniors have underlying health and social problems that are risk factors that may be minimized with appropriate attention or care. Consequently, while providing the needed emergency care, paramedics also assess the senior's living environment and characteristics of their daily lives. We found a way to capture this essential information and to refer the senior to other health care professionals for follow-up assessment and care."

Leadership

The Emergency Medical Service (EMS) leader's role in promoting and developing QI begins with creating and sustaining a personal and an organizational focus on the needs of internal and external customers and consumers. Through their actions, leaders demonstrate a clear commitment to the organizational mission, values, goals and expectations that promote quality emergency medical services and performance excellence. The customer-oriented mission, vision, values, and goals of the EMS organization are best integrated into all aspects of management through effective leadership.

Emergency Medical Services Leadership

Regardless of whether the focus is at the state, regional, or local level, the EMS organization's chief officer or executive must spearhead leadership for the QI program. Under his or her leadership, all other managers or leaders must work together to: 1) set the direction for quality improvement by creating a strong patient focus; 2) create clear and unambiguous statements that define the organization's mission, and values and identify operational objectives and long-term expectations; and 3) demonstrate continuous commitment to achieving the organization's quality improvement goals. (See Strategic Quality Planning.)

Achieving ever higher levels of service performance requires that EMS leaders develop a strategic quality plan (see "Strategic Quality Planning") that integrates QI into their system. The Strategic Quality Plan should:

• identify clear goals that define the expected outcome of the QI effort;

• be fact-based and use indicators to measure progress;

• include systematic cycles of planning, execution, and evaluation;

• concentrate on key processes as the route to better results; and

• focus on patients and other stakeholders.

Patients and Other Stakeholders

EMS leaders must insure that all organizational and system processes focus on the needs of patients and other stakeholders. Within the context of this manual, the term "patient" indicates the person receiving the health care service; "other stakeholders" indicates those, other than the patient, who have an interest in the health care and other services being rendered by the EMS system, e.g., the patient's family, the community in which the EMS system operates; local, county, and state governments that provide resources and/or regulate the operation of the EMS system; insurers and other third-party payers who pay for the care being rendered; and other health care providers that work with the EMS system, e.g., hospitals, physicians, and nurses.

Patients and other stakeholders can also be thought of as customers of the EMS system, and, depending on how they relate to the EMS system, as either internal or external customers of the system. External customers include those outside the actual operation of the EMS system, e.g., patients and their families, governmental entities, the community, and insurance companies and other third party payers. Internal customers, i.e., those who are involved in or with the operation of the EMS system, include the system's employees and volunteers, members of the leadership councils or committees that plan and coordinate the system; the variety of agencies that interact to form the ongoing, functioning EMS system; and other health care providers, including hospitals, that together with the EMS service, provide health care to ill and injured patients.

The idea of internal and external customers applies to all levels of EMS organizations. For example, a statewide EMS system might have as its internal customers, local EMS agencies, regional EMS organizations, county-wide planning bodies, as well as its own state-level employees. Its external customers would include the state legislature and federal regulatory agencies that support and monitor the operation of the EMS system.

Focusing on patients and other stakeholders means first identifying who those individuals and entities are (e.g., by a simple listing of internal and external customers) and then working to understand their needs and expectations. This latter task can be accomplished in two ways. The first way involves contacting customers and asking them about their needs and expectations. For example, conducting patient surveys can provide direct and measurable information on which parts of the EMS service most affect overall patient satisfaction and health status. Alternatively, conducting focus groups of former patients often provides rich detail, although the results of these discussions may be hard to quantify for analysis purposes. It is important to remember that efforts to identify customer needs should not focus solely on patients, but should also include similar efforts with the other external and internal customers of the EMS system, e.g., regional and state level leaders forging and maintaining strong communication links with the leadership of legislative, regulatory, and professional groups.

A second approach to identifying customer needs and expectations is through input from front-line staff members who deal every day with customers, in particular, external customers, e.g., patients or personnel from other agencies involved in the EMS system. EMS workers who have daily contact with these individuals are an invaluable source of information on internal and external customers' needs.

Either of these approaches can be used to produce a list of Key Customer Requirements that can form the basis for the EMS system's mission, vision, and values statements, as well as its strategic planning goals and objectives. For example, patients and their families can identify their expectations or concerns regarding the timeliness of EMS response, the ease of access to the EMS system, or the level of courtesy and caring demonstrated by EMS personnel.

Similarly, internal customers, such as clinical providers, can inform leaders about their needs that would, in turn, lead to improved services, e.g., training programs, protocol modifications, management and human resource issues, or job safety concerns.

Focus Groups
5 to 10 "customers" meet for 1-2 hours discuss pre-identified topics encourage sharing of ideas note-taker records information conduct different focus groups on the same topic until information from the groups becomes repetitive

Leaders of regional and statewide systems interact with local agencies in similar fashion. The role of the state EMS leadership is to meet the needs of regional leaders who in turn meet the needs of local agencies. While local, state and regional leaders are equally interested in identifying customers, there are usually major differences in the scope of the inquiry. While local agencies focus on individual patient provider interactions, regional and state leaders are usually interested in comparisons of how entire agencies or regions are performing to meet patient and community needs. Similarly, state EMS leaders can seek out opportunities for comparisons of their own performance with their peer states as a way of judging the effectiveness of the state system in meeting customer needs.

Once internal and external customer needs are identified and a list of Key Customer Requirements catalogued, the EMS system's mission, vision, and values statements, as well as its strategic planning goals and objectives, can be completed. Prioritization of key customer requirements occurs during this effort. (See Strategic Quality Planning). The creation of those planning documents is an important leadership responsibility because they must reflect the viewpoints of the overall constituency of a local agency, region, or statewide system.

The EMS agency or system mission statement describes the fundamental reason for the existence of the organization. It should describe all the essential components of the organization, such as identification of the system's customers; geographic service area; major services provided; economic goals; and organizational strengths.

The vision statement declares where the organization wants to be in the future and serves as a major focal point of strategic quality planning. The values statement identifies the basic tenets and principles of how people will work together. The values statement covers issues of fairness, honesty, commitment, dependability and expectations.

Mission: Purpose of the organization Vision: Desired future status Values: Beliefs and principles Goals: Proposed accomplishments

Operational goals and objectives are defined within the strategic quality planning process and provide day-to-day direction for system progress.

Empowerment of the EMS Workforce

People perform better and strive harder to succeed when they feel personally invested in their work. All members of the EMS system must feel empowered to make an impact on the quality of their system.

"Empowerment" Every EMS worker has the authority and the ability to solve problems and improve services.

Through careful planning and transition, managers can maintain authority and responsibility while, at the same time, increase the autonomy of and input from staff. This transition requires the creation of new working relationships among staff members. Training in topics like team dynamics and problem-solving may help provide personnel with the skills needed to make the new working relationships successful. For example, teams of pre-hospital care providers can be formed to identify ways to improve the quality of care rendered to patients. Additionally, other personnel, such as dispatchers, fleet maintenance, and data collection personnel can be added to create "Care Improvement Teams" that focus on ways by which the entire EMS response, patient care and transport process can be modified to better meet patient needs. At regional and state levels, regional councils and state advisory boards serve a similar function. These groups provide an excellent forum for the development of leadership expertise and consensus on regional and statewide quality improvement direction and policy.

Senior EMS leadership must also create opportunities for managers to develop and improve their management skills within the context of the QI effort. It is important to clarify managers' QI roles and responsibilities, as well as to ensure that their activities reflect an ongoing commitment to the organization's mission, vision, values and goals. Additionally, managers can be a strong motivating force for the entire organization to be involved in QI, since managers often function as the linchpin between senior leadership and the work-force. Consequently, early manager buy-in to QI activities is crucial.

EMS leaders can use a variety of approaches to increase manager involvement, including: encouraging increased communication among all organizational levels and departments; changing manager responsibilities to include more quality improvement team facilitation/leadership, and less inspection or supervision. Finally, all managers should participate in frequent quality, financial, and strategic performance reviews.

Managers should also be encouraged and supported in their efforts to demonstrate to the entire organization their ongoing commitment to quality improvement. For example, organization leaders should ensure that all managers have the time and incentive to participate actively as instructors and learners in QI educational activities. Additionally, managers can meet frequently with their internal and external customers; make contributions to the organization's newsletter; mentor new employees in QI values and policies; and serve as leaders and facilitators of quality improvement teams. These leadership and management activities should be evaluated periodically to determine whether they are achieving the desired result. Where results fall short of organizational goals, senior leaders need to revise and re-direct efforts into more productive initiatives.

Leadership Accountability

All members of the EMS leadership system should assess how well they each "walk the talk" of quality improvement. This might be accomplished, for example, through internal customer feedback on their leadership ("coaching") performance and how they might improve. Since in the early stages of QI, staff may be apprehensive about evaluating the boss, use of anonymous feedback may be helpful. Eventually, however, anonymity may be unnecessary as fear of reprisals lessens.

Community Citizenship

EMS organizations are part of the communities they serve and can contribute to community well-being in the same sense that every citizen is expected to contribute. For example, EMS leaders can promote community citizenship by setting the highest personal and organizational standards for ethical conduct in business and work practices. Such standards might include, for example, procedures that allow for public accountability and disclosure of performance information. EMS leaders must see to it that their organizations and staff continuously exhibit professional behavior and values.

Summary of Baldridge Program Leadership Objectives
Designate senior executive or cheif to lead the QI Effort Educate Leadership and Management in QI theories, strategies and benefits Initiate strategic quality planning Set leadership/management standards, tasks and procedures Develop policies & actions for community involvement

EMS organizations also have a responsibility to cooperate with other health care, public safety, and private organizations that play a role in the overall EMS system. Participating in community-wide planning for EMS services, as well as disaster planning and response activities, provides the opportunity to forge strong links with these organizations and the community. Liaisons with other EMS-related organizations in the community will help to maintain a customer focus, as well as provide opportunities to learn more about the needs of internal and external customers. EMS organizations can also participate in statewide reporting systems that enable comparison and bench-marking between local agencies or regional systems.

Further, EMS organizations can take a leadership role in educating the public about preventive health activities, environmental protection and other community-wide issues. Education activities, such as injury prevention classes, pre-arrival emergency care and system access training, not only educate the public but also strengthen goodwill. Although the degree of community involvement will depend on available resources, all EMS organizations can make some contributions in this regard. For example, smaller organizations might take part in cooperative activities with other, larger organizations to maximize resources. Regional and state level agencies and leaders can foster these activities by providing multi-jurisdictional coordination and resources.

Finally, EMS leaders can encourage on-the-job or after-hours involvement in rganized community programs, e.g., blood drives, "toys for tots", scouting, or sports programs. Such participation can help the organization maintain its grassroots links to the community, encourage employee leadership within the community, and improve morale.

Information and Analysis

The efficient collection and management of data and its transformation into useful information are fundamental to a successful Quality Improvement program. Data are necessary to describe customer needs, evaluate performance, establish goals for improvement, and monitor progress.

Selection of Data Used for Planning, Management, and Evaluation of Overall Performance

Specific data elements must be linked to key areas of organizational performance. Data elements must also be designed to meet the needs of those who will use the information. Data and information must be reliable, rapidly accessible, standardized, and timely.

Use data to determine performance excellence

Are EMS services timely? Do providers adhere to prescribed protocols? What is the level of patient / stakeholder satisfaction? How does performance compare with similar systems? Are data and information used in planning and operations? Do all workforce members understand and use available data? Have QI efforts been successful at improving performance? Are changes in one critical performance indicator affecting other areas?

• Reliability involves training to ensure that all are knowledgeable about the data being collected; that data collection is automated whenever possible and integrated into work processes; and that there are ongoing assessments of data quality. Reliability is also affected by the motivation of the data collector. Therefore, everyone responsible for collecting EMS data must have an understanding for how the data is used and an appreciation for the benefits that accrue from data collection.

• Rapidly accessible data are those that can be quickly analyzed to answer questions. This requires computerization of data.

• Standardization of data refers to the organization's efforts to make uniform its data sets, data definitions, codes, classifications, and terminology across departments and services, as well as to make them compatible with external data bases.

• Timely data provides accurate, up-to-date information about the performance level of key processes in the EMS agency or system.

Ambulance run form data is essential to an effective QI program. The run form documents the patient encounter and is crucial for evaluating how well an EMS organization fulfills its key performance tasks (e.g., prehospital response, treatment and transport time intervals, adherence to established treatment protocols, and changes in patient health status). The NHTSA recommends a minimum EMS data set that should be included in an EMS organization's information system.⁽¹⁾ EMS systems should adopt these data elements and operational definitions verbatim in order to facilitate benchmarking comparisons. These data elements, however, may not support evaluation of locally-specified treatment, transport, or triage guidelines, or other policies. Consequently, data elements needed for local evaluation should be added, if necessary.

Stakeholder data (e.g., from insurance companies, employers, managed care companies) are used to determine the types of EMS services needed or desired. Such information can be obtained and updated periodically by questionnaire or interview. For example, the geographic area encompassed by an EMS regional system may include a number of industries that could require an EMS response in the event of a hazardous materials incident. By collecting and updating pertinent information from the companies, the EMS system can better ensure its ability to respond to such incidents by, for example, arranging for special training or necessary equipment.

Satisfaction data are used to determine how well the EMS system is meeting the needs of patients and other stakeholders. For example, was the EMS response judged to be timely? Were the providers judged to be efficient, effective, helpful, courteous? Was the necessary equipment available? While it may not be possible or appropriate to collect such data from the patient or his/her family during the initial contact, these data may be collected subsequent to the initial patient encounter.

Process data are important for identifying and managing local needs, such as, vehicle use, age, and mileage, maintenance status, reliability; provider training, education and accreditation data; financial data; other administrative data (for example, personnel hours worked). Process data are also used to determine the root cause of problems and to compare performance against standards or other peer agencies.

Data Management

Data management procedures are used to continuously monitor and improve the usefulness of local or area wide EMS information systems. Effective data management begins during the process of determining what data should be collected. The existing data set elements should first be evaluated and modified if necessary: every currently-collected data element should be reviewed to determine if it is the best possible information source for evaluating the quality, cost or the source of problems for a key organizational process. Next, the process of transforming data into useful information must be evaluated and improved. Careful consideration of the effectiveness of information distribution for the work force must occur. Specifically, managers must determine if the presentation of results effectively supports process management, decision-making and performance improvement efforts.

State Level Data Management Activities. Necessary data management activities by the state EMS office are as follows.

• Define the EMS Data Set: Define the minimum data set needed to accommodate the evaluation of key performance areas. As noted previously, the minimum data set should conform to that recommended by the NHTSA. The data set must allow for careful attention to protocols, policies and evaluation of key process quality as called for in the Strategic Quality Plan.

• Specify Requirements for Registry Participation: Define data collection and submission requirements - including checks for data quality, completeness and timeliness - format of data submission, and operational definitions.

• Provide Registry Software: Software that meets reporting requirements is vital for successful data collection and analysis. Staff training and technical support can increase effective software use. Software can be "homegrown" or commercially available, as long as it meets reporting specifications (format, checks, etc). Procedures can be devised that standardize data set submissions using software other than that provided by the state.

• Support Statewide Programs in Data Quality: Ensuring the integrity of data and data collection procedures is a fundamental component of an EMS data and information system. Personnel who focus on data quality can also provide ongoing data-related training and independently abstract run data to check the accuracy and reliability of field data and data collection procedures.

• Determine the Contents of Data Analysis Reports: The content and frequency of data analysis reports determine how and how well data will be used in QI activities. Irrelevant, incomplete or out-of-date information is of little value. Specifically identifying how the data will be used can help increase data utility.

Comparative reports are highly useful. For example, regional reports can compare data with those from other regions; reports to a provider agency can compare that agency's results with those of other agencies within the region. Such comparisons can: 1) help recipients determine performance areas that need improvement; 2) increase benchmarking as a QI technique; and 3) hasten adoption of "best practices" throughout EMS.

The statewide registry may be the most able (because of state access to databases and more sophisticated computer personnel and equipment) to link the run form database with related database(s), e.g., hospital discharge or medical examiner databases. There are many useful applications for such linked databases, e.g., evaluating trauma triage criteria, where trauma system registry and/or hospital discharge databases are linked with the EMS registry information. Linking databases in this manner can provide a firm cornerstone for effective use of data for QI purposes.

Local Level Data Management Activities. Necessary data management activities at this level are as follows.

Comparative Reports
Data Completeness - by element Timeliness - time between patient contact and record closing Frequency of Overrides - entries outside defined data ranges Prehospital Time Comparisons Compliance with Protocols Patient / Stakeholder Satisfaction Cardiac Arrest Survival (Utstein) Use of Trauma Triage Criteria Variability in Resource Consumption

• Planning: Planning for all the various data management activities is crucial at the local level, since it is here that most of the data is collected.

• Training: Individual providers need training in collection and use of run form data. Providers need a thorough understanding of all operational definitions, the applications and importance of data, those activities necessary to ensure data completeness and quality, and prescribed procedures for data collection. Such training also provides an excellent opportunity to emphasize the crucial importance of field providers conveying reliable information to their internal customers (i.e., medical director, treating physician/nurse, operations managers, etc). Further, field providers and others who collect raw performance data should receive instruction on how the data collection is linked to the system's strategic quality plan. Perhaps most important to stress is the fact that data collection and analysis ultimately affect patient health because the analysis of patterns and trends in patient care and outcome is an important source of new or revised treatment, triage and transfer protocols.

  Procedures for collecting and recording data should be specific, well-defined, and reflect an understanding and appreciation for the working environment of the providers who collect the data. Procedures can include ways to "automate" the data, e.g., direct data entry into a laptop or other data entry device, or onto a scannable form to eliminate keypunching. Direct electronic data entry is most desirable because it permits checks for data quality to be completed near the time of "collection", increases the likelihood of error correction, and reduces data omissions.

• Adopt a Local Registry Software/ Run-Form Database: The local computerized database/registry typically contains data abstracted from completed run forms. Registry software permits initial as well as in-depth review and analysis of the information provided by the registry.

• Check Data: Registry software should perform checks for data quality and require that data anomalies be resolved and records "closed" before they are reported on or transferred to an intermediate or statewide database.

• Reporting: Registry software can allow for standard "built-in" analysis of unit and provider performance (e.g., "time intervals", adherence to policies and protocols, and consumer satisfaction), as well as data quality (e.g., completeness, accuracy, consistency and timeliness of record closing). Locally defined "ad hoc" reports can be prepared for QI activities.

• Data Quality Program: Data quality and timeliness are key to the success of the QI program, and provider training is a significant determinant of data quality. Other important factors that can increase data quality include: evaluating provider knowledge of data collection procedures and operational definitions; early checking (at scene, in transit, in hospital, or at station) and feedback to providers on quality and completeness; continuing education on data quality; and feedback to data collectors on information of particular interest, e.g., charts and graphs showing performance in comparison to standards or other factors. Presentation of data is an important element in increasing interest: pictures that display complex information at a glance are invaluable.

• Local Database Manager: One individual should be responsible for managing the local run-form database/registry. Responsibilities should include helping to assure data quality and completeness; running "built-in" or, where necessary, writing "ad hoc"programs; preparing data analyses to support unit planning, management and evaluations; and submitting data to the state/regional database.

Evaluation and Improvement of Data Management

Provider / Organization Level: Ongoing evaluation of data management activities ensures quality and identifies areas requiring further development, personnel needing additional training, and equipment necessary to improve productivity.

Comparisons from other disciplines can help guide EMS data management effectiveness. For example, data accuracy and completeness rates in established cancer or trauma registries can provide some general guidance. Experiences in other registries, however, may not be directly comparable because of factors unique to those registries, e.g., the amount of data per record; or the time over which those data are collected may extend well beyond hospitalization.

Statewide Level: By virtue of its statewide perspective, the state EMS Office can be particularly useful in evaluating the effectiveness of data management. For example, the state can examine and compare data quality measures across regions, identify variations, and suggest areas and opportunities for improvement. Data management techniques and results can also be compared among similar states. Where areas for improvement are identified, the state can provide or assist with arrangements for necessary training or remedial activities.

Competitive Comparisons/ Benchmarking

Comparisons and benchmarks are important for each key EMS performance area. Comparisons and benchmarks (e.g., the best 5% of performers with respect to a particular measure), based on data from other states, regions or agencies, can put an organization's performance into perspective. Benchmarks from appropriate non-EMS organizations can also be helpful, e.g., police or home protection company response times.

Performance comparisons can occur in two ways: 1) point comparisons can be made of time-specific performance indicators that are compared to established standards; or 2) comparisons can include monitoring of trends over time in key performance areas. The results from comparisons and benchmarking may suggest no action (already among the best performers); a need to review and refine current work processes (performance is "close" to benchmark); or total re-evaluation and search for breakthrough approaches (performance is far below the benchmark).

State Leadership in Benchmark Definition: The state may be in the best position to select relevant databases for comparison and benchmarking and to provide periodic feedback to regions and organizations. Initially, benchmarks could be empirically derived from statewide EMS data. State EMS agencies can spearhead interactions with benchmark-level performers in each area, identify their "best practices" and distribute that information to others while maintaining confidentiality when necessary. State agencies can also lead the search for relevant bench-marks from other states and industries.

Where areas for improvement are identified, the state can help identify: 1) performance with respect to the benchmark; 2) activities stimulated by the shortfall; 3) resulting changes (e.g., improvement) in performance; 4) subsequent initiatives to pursue or define new benchmarks; and 5) individual provider efforts to improve performance by local benchmarking.

Analysis and Use of Organization Level Data

Use of data at the individual performer or department level differs from use at the organization and systems level. At the organizational and systems level, data can be related to quality, customers, medical markets, and operational performance. Together with relevant financial information, these data are integrated and analyzed to support organization-level review, action, and planning.

Understanding customers and markets: EMS serves entire communities and populations-at-risk. Understanding the demo-graphics and socio-cultural features of the EMS service area is important for planning all EMS activities. Access to demographic databases (e.g., the U.S. Census) is helpful, as is partnering with health insurers and managed care organizations in the service area to gather and analyze incidence data for acute illness and injury. Databases developed in cooperation with other provider groups or professional societies can be helpful, particularly those that include data from the entire health care system in the EMS service's medical trade area(s).

Improving customer-related decision making and planning: Understanding the needs of customers (i.e., payers) requires ongoing communication with the employers, private parties and governments that pay for EMS services. Billing information can provide data on payer mix and utilization, augmented with data on prevalence of paying organization, as it is likely that the population overall and the users of the service will be different.

Improving operations-related decision making and planning: Incidence and demand data are critical bits of information. Demand pattern analysis results can be used for refining current operations and long-range planning for future operations. Information from key sources, for example, patients and other stakeholders, health care practitioners, EMS service population area statistics should be obtained and updated periodically. Surrogates should be sought where necessary (e.g., organizations that assist those dependant on home ventilators) and advice obtained on relevant operational management and planning issues.

Understanding organizational capabilities: Operations performance may be evaluated using indicators of operational performance (response-time reliability, etc.) and cost (base charge, per capita annual subsidy, etc.). This allows for comparison across systems and encourages managers to constantly test and answer the question: "Are we doing the best we can with the money we have?"

System-wide efforts are vital to determine if EMS organizations make measurable clinical differences to their communities. Whenever possible, EMS organizations should use standardized methods to evaluate cardiac arrest survival. Use of cardiac arrest survival, the most widely recognized and reliably measured clinical performance indicator, permits comparison of results with other, similar systems. Similar outcome measures are sorely needed for other patients, e.g., trauma or pediatric, that would allow for outcome comparisons across systems.

Understanding competitive performance: Identifying and understanding "the competition" is important to ensure that EMS systems are responsive to the needs of patients and other stakeholders. Answering the following questions can help focus EMS systems on performance improvement:

• Who is the competition?

• What services do they offer at what cost?

• What is their level of performance?

• What are the gaps in their services?

• Can and should EMS realign its services within the framework of managed care needs?

• What other technologies may potentially compete with EMS system components (e.g., telephone advice systems, interactive TV, Internet searching and information exchange as a substitute for 911)?

Conclusion

Information and analysis activities can be daunting to EMS systems that have little experience in data collection, management, analysis, and interpretation. Yet, data collection- and analysis is central to the effective design and implementation of the strategic quality plan. EMS Systems should undertake those data and information activities as their current resources permit, but also seek to expand their capabilities by using the principles described in this chapter.

Strategic Quality Planning

Strategic planning is the process of developing long- and short-term organizational objectives, identifying ways to achieve those objectives, and measuring the effectiveness of these efforts. Quality planning is the successful design or re-design of a system to perform to the quality standards expected by patients or other stakeholders. In the quest for continual improvement, strategic planning can be closely linked with quality planning and combined into a single organization or agency-wide planning process.⁽²⁾ This chapter will focus on an integrated process that incorporates both strategic and quality planning into strategic quality planning.

Overview of Strategic Quality Planning

Strategic quality planning is neither magical nor mysterious; it is simply an organized method of determining where an EMS system or organization wants to be and how it plans to get there. Strategic quality planning is not something separate from the EMS system; rather, it is an integral, ongoing part of the system. It involves the careful integration of all components of the EMS system, including clinical performance, financial support, legal authority, personnel management, education and training, and data collection and analysis. Individual components are mutually interdependent; planning and evaluation of one component cannot occur in isolation from the others.

EMS systems involve many different organizations and individuals with separate authorities, management, and governing bodies, each of which may have its own strategic quality planning process. EMS often involves organizations and individuals not traditionally viewed as health care providers (e.g., law enforcement personnel, dispatchers, and the general public). Yet it is because of the diversity of the organizations involved that strategic quality planning is imperative to the overall improvement and smooth functioning of the entire EMS system.

The activities of each level of EMS (state, regional and local EMS organization) are different, but complimentary. Strategic quality planning, as well as the entire QI process itself, should occur at the local, regional, and state EMS system level. Just as the state EMS system must support local EMS systems, these systems' activities should be compatible with the overall statewide vision and mission.

Strategic Quality Planning at the State and Regional Level

Strategic quality planning at the state EMS level differs from strategic planning at the local or agency level. Unlike the private sector, there are no competitive or entrepreneurial demands placed on a government agency. Instead, the state EMS agency is charged with creating or maintaining public policy and meeting the needs of a different type of "customer": tax payers/citizens and EMS companies/ corporations, as well as the patient, hospitals, and health care providers. The state EMS agency is typically charged with designing or, in most cases, re-designing systems of care that will lead to optimal patient outcomes. The state is also charged with identifying demographic and economic issues that will impact the delivery of EMS care and planning to meet these changing needs. Whether it involves the design of a trauma system from the ground up, or the re-design of a system that has evolved over several decades, strategic quality planning can help the state EMS agency better meet the needs of the EMS community and those they serve.

The goal of strategic planning is not just the plan itself; rather strategic planning encompasses the method of doing the planning.

The state EMS strategic quality plan serves as the roadmap for achieving quality improvement in EMS for the entire state. One key to successful strategic quality planning at the state level is to involve all those individuals and organizations that will be affected by the plan. A strategic quality plan that lacks input and buy-in from those affected will only gather dust on the shelf. Mechanisms for obtaining involvement and buy-in from the various stakeholders will be discussed later in this chapter.

In order for the plan to be a useful tool in measuring performance across the state and improving service to the patient and other stakeholders, it must be:

• monitored and revised frequently according to the results obtained and in response to the changing health care environment in the state;

• simple, easy-to-use, and applicable to both rural and urban settings;

• achieved through consensus-building throughout the state with local and regional EMS agencies, hospitals, EMS providers, patients and other stakeholders;

• conveyed to all members of all involved organizations; and

• easily and quickly changed to reflect new demands or changing conditions.

Strategic Quality Planning at the Local/Agency Level

The nature of strategic quality planning at the local level varies according to whether the organization is in the public sector (e.g., local government agency, fire bureau, or county EMS agency) or the private sector (e.g., "Ambulance Company XYZ, Inc."). In both public and private sectors, however, strategic quality planning still involves planning new or revising existing services based on patients and other stakeholders' needs, expectations and specifications. In both sectors, the design of new processes or services must follow a prescribed quality planning formula to assure that the process is designed correctly the first time.

Strategic quality planning in the private sector is more focused on favorable market positioning in a competitive environment, as well as on financial viability. Strategic quality planning in a public agency that uses volunteers can be a great challenge. The planning process must be user friendly and designed to maintain interest and involvement from start to finish. Enthusiasm can be increased when participants understand the usefulness of strategic quality planning in guiding progress and advancing patient care. Involvement will be minimal, however, if participants view strategic quality planning as only a bureaucratic exercise.

Because each level of an emergency medical service system has different responsibilities and functions, objectives and action plans may be different. A cohesive EMS system requires compatible and complimentary vision statements, consistent key driver identification and uniformity of performance indicator definitions.

An EMS system involved in the strategic quality planning process would complete the following:

• Develop a vision statement

• Define the strategic quality planning structure

• Identify underlying assumptions that affect planning

• Identify the key drivers of the EMS system

• Develop measurable objectives (performance indicators)

• Determine compliance with the performance indicators

• Develop and implement action plans to bring the EMS system into compliance

• Evaluate the impact of the action plans on performance indicators

• Modify the action plans and/or the indicators

Developing the Vision Statement

A vision statement provides a futuristic look at and broad guidance for the EMS agency or system. In simple terms, the vision statement helps to make sure everyone is going in the same direction. Each objective and action plan that is subsequently devised is consistent with the vision statement, thus assuring constancy of purpose and compatibility of actions.

While development of the vision statement is directed by the leaders of the state, regional, or local EMS system, the system or agency "players" should be deeply involved in the development process. People are more likely to help implement what they help to develop. (See "Leadership").

Typically, a vision statement would be a short, motivational description of the EMS system's ideal condition. The vision statement can serve as motivation for those involved in the system and can be a steady guide through the numerous changes necessary to achieve a quality system.

The state's vision statement is broad, encompassing the entire statewide EMS system. The regional system's vision statement should be unique to that system and consistent with the state vision statement. Development of the vision statement for the local agency (e.g., ambulance service) is directed by the service's managers. The vision statement is unique to that service and is consistent with the local system and state vision statements.

Strategic quality planning structure

The right strategic quality planning structure is based upon the organizational characteristics of the EMS system. The structure must account for the fact that strategic quality planning is an ongoing process, based on the principles of quality improvement and involves the EMS system's organizational, financial and clinical aspects.

Strategic quality planning is not something new and different that requires a separate system or a separate process; instead, strategic quality planning is a process that drives all planning and all quality improvement efforts. Thus, the strategic quality planning structure must take into consideration existing planning and quality mechanisms, including state EMS advisory councils, local EMS councils, health care advisory councils and other specific statutory or regulatory requirements.

Assumptions

Underlying any planning process are implicit assumptions that steer the organization and its personnel in certain directions. These assumptions should be identified as part of the strategic quality planning process. Making assumptions explicit allows for discussion and agreement on whether the assumptions remain valid and useful for the future of the organization or whether they need to be changed or discarded.

Some examples of common assumptions regarding EMS include the following:

EMS Vision Statement (Example)

Emergency Medical Services of the future is a community-based health management system that is fully integrated with the overall health care system. It has the ability to identify and modify high risk illness and injury indicators, provide injury prophylaxis, provide acute illness/injury follow-up, and contribute to treatment of chronic conditions and community health monitoring. This new entity is totally integrated with other health care providers and public safety and public health agencies, thereby decreasing utilization of acute health care resources. Emergency medical services serves as the public emergency medical safety net. --U.S. DOT/NHTSA, Washington, D.C.,1996.

1. EMS represents the intersection of public safety, public health, and medical care systems.

2. The public expects that EMS will continue.

3. EMS at the local level will continue to involve diverse organizations and personnel.

4. As one component of a varied and complex health care system, EMS will be significantly impacted by the continuing evolution of health care.

5. There is currently a lack of information regarding EMS systems and outcomes.

EMS Agenda for the Future, U.S. DOT/ NHTSA, Washington, D.C. 1996

Key drivers

Key drivers are those areas most critical to the success of the EMS system and should be consistent with the system's mission and vision. Identification of the key business drivers provides the basis for focusing the EMS system's quality improvement efforts in specific areas. Key drivers include customer-driven quality requirements, as well as the organization's operational requirements.

The key drivers are identified through the strategic planning process and are based on expert opinion, good judgment and common sense. Once identified, the key drivers should be validated by internal customers (employees, staff and volunteers) and external customers (patients and other stakeholders). Examples of key drivers for the EMS system might include, but not be limited to:

• Prevention (Injury prevention and preventative health care)

• Public access to EMS

• Timely scene response times

• Timely and appropriate patient interventions

• Timely arrival at an appropriate medical facility

• Effectiveness of care for pediatric patients

• Customer satisfaction

• Workforce relations

Objectives and Performance Indicators

Objectives are measurable statements that are consistent with the system's mission, vision and key drivers. Clear operational definitions are needed for each objective. When well-defined, these objectives can serve as performance indicators against which system progress toward attainment of the objective can be objectively assessed or compared.

Strategic quality planning will likely involve both procedures-oriented and outcomes-oriented objectives. Procedure-oriented objectives are those that are assumed to facilitate achievement of the broad plan objectives while outcome- oriented objectives are focused on the accomplishment of measurable outcomes.⁽³⁾ Broad organizational goals tend to come from the "top down"; however, quality improvement projects are often "bottom up" and should be consistent with the broad goals. Rather than just monitor the completion of procedures, effectiveness is based on measurably improved outcomes.

Additionally, some objectives will likely be similar or even identical among state, regional, and local agency participants. Other objectives will be unique to each type of EMS structure or will vary among similar structures.

Compliance

Compliance with the objectives simply involves using available sources of data and information to measure compliance with the performance indicators.

Analysis

Where compliance falls short of the performance objective / indicator, an analysis must be completed to determine the cause of the problem. EMS is a highly complex system, and analyses (as well as the action plans that grow from the analyses) must account for this complexity. As an example, an EMS system has as one of its objectives (performance indicators) the following:

"75% of all patients suffering a

witnessed cardiac arrest and who are

in ventricular fibrillation will be defibrillated

within 3 minutes of the arrest".

Compliance is determined to be only 10%, however. A careful analysis of the situation might identify the existence of one or more of the following reasons for the non-compliance, each of which may ultimately require action steps at different levels of the EMS system.

Possible reason: First responders are not permitted to defibrillate.

Local: The local fire chief will not permit the fire service personnel to defibrillate.

Local: The medical director will not authorize first responders to defibrillate.

State: State law or administrative rules prohibit defibrillation by First Responders.

Local: The city council has not provided funding to purchase defibrillators for the first responders.

Local: The fire chief does not include a request for AEDs in the annual budget.

State: The EMS licensing rules do not require an AED to be present in licensed non-transporting units in urban areas.

State: The state grant-in-aid program for local services cannot be used to procure defibrillators in an urban area.

Possible Reason: 9-1-1 coverage is available to only 50% of the population.

Local: The county commissioners have eliminated funding for expanding 9-1-1 coverage throughout the county.

Local: The telephone company cites technical difficulties and antiquated equipment as a barrier to expanding 9-1-1 coverage.

State: There is no state law mandating 9-1-1 coverage.

State: There is no statewide dedicated funding source earmarking revenue to assist with 9-1-1 coverage.

Possible Reason: The general public does not know when and how to call 9-1-1.

Local: There are no 9-1-1 stickers to place on telephones because they were eliminated from the communications budget by the mayor.

Local: There is no on-going effort to educate the public about 9-1-1 coverage in the community.

State: The state telecommunications agency or state EMS agency has not instituted the "Make the Right Call" campaign designed by the National Highway Traffic Safety Administration.

State: The state legislature diverted funding from 9-1-1 public information to the state Medicaid program.

Possible Reason: EMS personnel are not complying with established protocols

Local: There is no ongoing system of quality improvement or retrospective medical direction in the local EMS system.

Local: The system has a "phantom" medical director who only signs re-certification, but is not involved with actual medical direction.

Local: EMS personnel are not familiar with the defibrillation protocols.

Local: There is no ongoing continuing education program for EMS personnel.

State: There is no statewide protocol for early defibrillation.

State: There is no mandated continuing education in the use of the Automated External Defibrillator.

From these limited examples, it should be clear that analyzing the causes of non-compliance will likely involve the entire EMS organization and system (including policy-makers, care providers, medical directors, public safety personnel and others).

Develop and implement action plans

Action plans are where the "rubber meets the road". When the system is out of compliance with the objectives, an action plan will need to be developed to correct the cause of the problem. The action plan should be specific as to the exact steps necessary to assure compliance.

Evaluate impact of action plans

Following implementation of the action plans, there must be a re-evaluation of compliance with the objective. Did the action plan make a difference?

Modify the action plans and/or indicators

If the action plan did not make a difference, it may be necessary to attempt other action plans. The overall strategic plan should be flexible enough to be easily modified based on new information, modified priorities or changed conditions. The planning process is as important as the plan itself.

Evaluating the Effectiveness of the Strategic Quality Plan

The initial strategic quality plan prescribes the key drivers of the EMS service and corresponding measurable objectives (indicators) that operationalize these key drivers. The strategic plan also sets the goals for the level of achievement. Evaluation of the strategic plan compares the goals or desired level of achievement to the actual accomplishments.

As an example, the State Health Division of a rural western state evaluates the success of one component of its 1995 Strategic Plan in the following manner:

Key EMS Driver: Appropriate/timely patient interventions.

Objectives (performance indicators):

Endotracheal Intubations:

Procedure-Oriented Objectives:

1. Finalize the computerized intubation data collection and reporting system by December 1996.

2. Implement a statewide skills requirement for endotracheal intubation.

3. Identify anesthesiologists and hospitals in each region willing to sponsor paramedic intubation experiences in the operating room.

Outcome-Oriented Objectives:

1. Improve the intubation success rate across the entire state to 90%.

Evaluation of Compliance:

1. Statewide skills requirement implemented in March (2 live intubations per year required).

2. Intubation success rate of 90% across entire with exception of Regions 1 and 3.

3. 50% statewide compliance with skills requirement.

4. Four hospitals agreeing to sponsor paramedic intubation training; three others considering it. No hospitals in rural areas have agreed to participate.

Action Plan:

1. Assemble intubation CQI team with members from regional and local EMS agencies, rural, urban, and suburban hospitals, state anesthesia association to work on plan to improve intubation success rates and increase intubation clinical experiences.

2. Continue to hold quarterly training sessions throughout the state for local EMS companies regarding data collection and reporting of intubation attempts and successes.

Other Evaluation Methods

Because the goal of strategic/quality planning is to better meet the needs of internal and external customers, feedback is an important part of evaluating the planning process. In addition to comparison of performance against objectives / indicators, EMS leaders can use direct input from customers to determine if their needs are being met. Evaluation methods will differ slightly according to the jurisdiction involved, though each method can be used at both the state and local level.

Focus groups with consumers and customers are another effective method to use in evaluating the success of the strategic plan. Focus groups can be used at the state and EMS organization level, as well as at the local provider level (see "Leadership"). Similar to focus groups, customer surveys are another way to evaluate the outcomes of the strategic plan. Surveys are generally less expensive than focus groups, but are often limited by poor response rates and variable reliability of the data.

EMS Examples

Following is a limited example of strategic quality planning related to 9-1-1 access:

Key Driver: Prompt Public Access to the EMS system via a 9-1-1 dispatch center staffed by appropriately trained personnel.

Objectives and performance indicators:

Outcomes oriented objectives

1.1	75% of the emergency medical calls to 9-1-1 dispatch should be received by the 9-1-1 center within 5 minutes after the time of onset of symptoms or injury.
1.2	90% of the population should have access to 9-1-1.
1.3	95% of the emergency medical dispatch instructions should be consistent with the approved dispatch protocols.
Procedure oriented objectives
1.4	Dispatchers should be included in the QI activities of the local EMS system.
1.5	There should be EMS dispatch protocols that are coordinated with the EMS system and approved by the system medical director.
1.6	There should be adequate funding for the 9-1-1 dispatch center.
1.7	All 9-1-1 operators should participate in an EMS dispatch training program meeting the standards established by National Highway Traffic Safety Administration.

Compliance

Using the available information and data sources, compliance with the objectives should be determined and reported. (See example above.)

Action Plans

There should be action plans developed to improve compliance. (See examples above.)

Evaluate impact of action plans

Following implementation of the action plans, there must be a re-evaluation of compliance with the objective. Did the action plan make a difference?

Modify the action plans and/or indicators

If the action plan did not make a difference, it may be necessary to attempt other action plans or to modify the indicators.

Human Resource Development and Management

EMS' most important asset is the dedicated people who work throughout the system. These individuals hold the key to successful and lasting quality improvement efforts.

The EMS workforce includes all those who contribute to the delivery of the EMS organization's mission and services, regardless of career or volunteer status. An EMS system's human resource practices affect the EMS workforce and are inextricably connected to EMS performance results. The EMS workforce can be empowered and enabled to develop and use their full potential to achieve their local agency and regional or statewide system vision for the future. For this to occur, the EMS organization must provide opportunities for performance excellence, as well as for personal, professional and organizational growth.

Human Resource Planning and Evaluation

Human resource planning includes all aspects of job design and personnel management of the EMS system and its personnel. Human resource evaluation focuses on assessing and improving human resource planning, practice, and performance.

Once the strategic quality planning process has started, EMS leaders must carefully translate those plans to the realm of human resources. The link between personnel resources and overall performance can be strengthened by relating specific quality goals to specific human resource goals and by identifying what human resources must be in place to help ensure success. For example, targets for morbidity and mortality reduction must consider existing skills and capabilities of the workforce. Similarly, new targets for health status outcome performance will not be achieved simply by expecting workforce members to work harder or more efficiently; new skills, training or technology may be needed. Consequently, EMS leaders may need to consider changes in: (1) work process design to improve flexibility, efficiency, coordination, or response time intervals; (2) workforce development, education, initial and refresher training (including credentialing); (3) compensation, recognition, and benefits; (4) staff composition; or (5) recruiting efforts.

Personnel-related and organizational performance data can be used to analyze personnel needs, assess the links between human resource practices and key performance results, and identify changes needed to achieve EMS system goals. Data elements may include job satisfaction, turnover, absenteeism, safety, grievances, recognition, training, and information from exit interviews, as well as overall system strengths and weaknesses that could affect the agency's ability to fulfill human resource plan requirements.

In order to develop the full potential of EMS personnel, it is critical to evaluate efforts to improve human resource planning, practice, and performance. Evaluations can be augmented by comparative or bench- marking information and used to identify specific personnel needs or new approaches or practices.

Workforce Work Systems

Human Resource Planning

Re-design work processes or jobs to increase opportunity, responsibility, and decision making Promote greater labor-management cooperation Recognize and reward efforts that increase patient and stakeholder satisfaction Survey staff to identify ways to improve performance Prioritize personnel problems based on potential impact on productivity Develop recruitment / re-training strategies and initiatives Form partnerships to increase education, training and job opportunities Address safety factors

Improvement of quality may require that the workforce be re-organized into new, more effective work units. These may include non-traditional work teams, problem-solving teams, or functional units that are formal or informal, temporary or long-term. Units may cut across customary organizational lines and be self-managed or managed by supervisors.

The total quality EMS organization is committed to operating patient care systems and administrative services that can achieve high performance. "High performance" services maximize efficiency and produce the highest level of quality possible.

Job performance can be enhanced where EMS systems: (1) design jobs that ensure that the roles, responsibilities, duties, and tasks of each workforce member are tailored to achieve the system's goals; (2) create opportunities for initiative and self-directed responsibility; (3) foster flexibility, job efficiency, task coordination, and rapid response to changing requirements; and (4) ensure effective communications across traditional units/departments. Enhancing existing jobs might include simplifying job classifications, cross-training, rotating jobs, modifying work layout or work locations, or using new technologies e.g., computer links or conferencing technology.

The EMS organization's compensation and recognition system can also be used to improve the effectiveness of the work and job design. In this manual, compensation and recognition refer to all aspects of pay and reward, including promotion, bonuses, and recognition. Recognition is extremely important in EMS systems and encourages high performance levels and work contributions that are above and beyond past efforts. Organizational reward mechanisms are equally important for both compensated and volunteer EMS organizations and help foster a sense of community in the work environment. There are many formal, informal, individual and group approaches that EMS systems use to recognize and reward performance. New approaches can also be used to strengthen links with patients and other stakeholders, e.g., community-wide recognition of the efforts of all health providers in the EMS system.

Work Force Education, Training, and Development

Education and training empower the workforce to achieve not only the job requirements, but also the goals of the EMS organization, and the vision for the entire EMS system. Most EMS education and training is directed at meeting clinical skill needs and certification requirements. In the context of a strategic quality plan, however, education and training may extend beyond the need for clinical expertise. For example, since managers will need to lead and facilitate QI teams, managers should be trained in the variety of skills relevant to these activities, such as, leadership and team facilitation. The list at right contains some of the topics that team facilitators and leaders should pursue through education and training. State lead EMS agencies can support local systems and organizations by developing curriculum and arranging multi-jurisdictional classes.

Train to Increase EMS Workforce Effectiveness, Efficiency and Safety
Leadership Skills Interpersonal Communications Teamwork Quality Improvement Principles Problem-solving Understanding and Using Data Meeting Patient Needs Practice Guidelines Critical Pathways Process Analysis & Simplification Waste Reduction Cycle Time Reduction Error-Proofing Basic Reading & Writing Skills Continuing Education

State, regional and local agency leadership is needed to develop quality improvement objectives that address how education and training are designed, delivered, reinforced, and evaluated. Improvement areas include: (1) how the EMS workforce is involved in determining specific education and training needs and in designing delivery and evaluation options; (2) how re-credentialing requirements are translated into educational program designs; and (3) how knowledge and skills are reinforced on the job.

On a daily basis, the EMS workforce interacts directly with the system's patients and other stakeholders. Training for patient and stakeholder (front-line) contact should include not only the knowledge and skills necessary to provide effective emergency care and transport, but also those that contribute to customer awareness and needs assessment. These skills include more effective ways of listening to and soliciting input from patients and other stakeholders; managing patient and stakeholder expectations; and anticipating and handling system problems or failures.

In order to determine these non-traditional training needs, the local agency will likely need to conduct an organization-wide staff assessment. A needs determination should analyze job responsibilities and the types and levels of skills required. The actual training might occur within or outside of the EMS organization and involve on-the-job, classroom, or other types of education and training, e.g., developmental assignments within or outside of the organization. With the increasing popularity of the Internet, state agencies can become the focal point for curriculum development using distance education methods. Computer-based education programs provide opportunities for maximum scheduling flexibility to meet the needs of changing personnel schedules.

Workforce Well-Being and Satisfaction

Worker well-being and satisfaction are necessary for the organizational delivery of high quality emergency services. Consequently, leadership must focus on maintaining a work environment where workforce well-being factors (such as health, safety, and ergonomics) are included in quality improvement activities. Managers must determine and understand what the workforce needs to achieve and maintain physical, mental, and social well-being.

EMS agencies and organizations should also determine which services, facilities, activities, and opportunities will be available to the workforce to support their personal development, well-being, and job satisfaction. EMS organizations at every jurisdictional level may want to provide or support the following: personnel and career counseling; career development and employability services; recreational and cultural activities; non-work-related education; day care; and special leave and flexible scheduling for family and community service responsibilities.

Similarly, EMS leaders need to review indicators of workforce morale and motivation. The strategic quality plan should include human resource key indicator analysis and improvement objectives. Valuable sources of information include: grievance proceedings; incidents involving field provider health and safety, including infectious disease exposure; back injuries; assaults; staff evaluations of leadership and management; use of staff development and career opportunities; use of sick-time and workman's compensation; and exit interviews.

EMS Process Management

This chapter examines how key processes are designed, managed, and improved to achieve higher performance. Within the context of this manual, "process management" is used to refer to the improvement of work activities and work flow across functional or department boundaries.

Design and Introduction of EMS Patient Care Services

Community financial constraints and health care reform are challenging EMS leaders to design new services and methods of operating the EMS system, to adapt to new demands for quality and cost efficiency, and to conduct evaluative research to demonstrate the value of EMS.

The QI-oriented EMS organization has in place a well-defined strategy for designing new services and for evaluating and, where necessary, re-designing existing services. Such strategies specify: (1) how decisions are made to launch new preventative, primary or emergency medical care services; (2) how environmental changes are translated into efficient patient care and work processes (e.g., changing patient and stakeholders needs, regulatory or payer requirements; technological innovations); and (3) how the timing and flow of new service proposals occurs so that the operations of all external and internal system organizations are integrated and coordinated in support of the new service.

Further, the measurement plan for quality indicators of new services should specify what variables are to be measured, who is responsible for measurement, and when and where measurement is to occur. Preliminary performance standards can be defined so that results can provide the information needed for the strategic quality improvement process.

Finally, EMS leaders should also develop specific procedures to insure that proposals and planning documents for new services are throughly considered and pre-tested, where necessary, to ensure maximum effectiveness and safety for patients and the workforce. Finally, the design, evaluation and pre-testing process itself should be subject to continuous quality improvement.

Future Trends

EMS of the future may serve a direct or supporting role in the delivery of a wide variety of services. These services may involve such diverse areas as disease and injury prevention, health maintenance and promotion, diagnostic testing and screening, post-discharge home care, and rehabilitation services. Design issues that EMS organizations typically address include:

• modifications of existing patient EMS services, such as, shifting a service from an inpatient to an outpatient setting; introducing a new technology into an existing service; instituting use of critical pathways;

• new ALS services resulting from research;

• new or modified facilities or deployment strategies designed to meet operational or patient health care service requirements; and

• significant re-design of processes to improve productivity or cost efficiency.

Design approaches will differ depending on the nature of the patient service. If several design projects are carried out simultaneously, EMS leaders will need to coordinate resources among the various projects. In service design or evaluation, the key requirements of patients and other stakeholders must be paramount, e.g., safety and risk management; timeliness of care; system access; coordination and continuity of care; patient involvement in care decisions; measurement capability; availability of staff with necessary critical skills; availability of referral sources; use of technology; unit capacity and utilization; supplier capability; and documentation.

Design requirements must also account for the standard EMS processes:

• Recognition of an emergency

• Bystander Intervention

• EMS System Access

• Triage

• Dispatch

• First Responder Response

• First Responder Service

• Transport Vehicle Response

• Scene Triage

• Scene Care

• Patient Transport

• Patient Disposition

• System Recovery

When considering expanding the scope of EMS practice to include prevention or primary care services, leaders must also assess the wide mix of service options and medical professionals available.

Delivery of Patient Health Care

The delivery of key patient health care services can be managed to ensure that design requirements are met and that quality, effectiveness and efficiency are continuously improved. Critical indicators can aid in this process. Critical indicators are clearly defined measurements that compare various input and process characteristics. The regular analysis of critical indicators of quality will yield patterns of performance that will trigger quality improvement projects. Once defined, EMS leaders must select and support improvement projects based upon pre-determined criteria relevant to the importance of the project within the strategic quality objectives of the EMS organization.

Critical indicators must be developed for each key health care service; these indicators exhibit some specified phenomena that can be measured. Typically, an individual EMS field care provider will observe or experience a sentinel event that will trigger a decision to make a correction. For example, a paramedic makes a second attempt to intubate a patient after failing to hear proper breath sounds following the first attempt (sentinel event). When a sentinel event occurs (i.e., failure to get breath sounds), the provider focuses on rapid discovery of the cause(s) and use of a quick remedy of the problem based on predetermined action plans. This is effective quality control -- putting the delivery of patient care back on the right track as soon as possible. Notice the difference between quality control and quality improvement. Quality control is a rapid restoration of the process to its intended quality level. Quality improvement involves action over a longer period of time that results in achievement of new breakthrough levels of performance.

At higher levels of authority, a summary approach to process management is used. For example, for a supervisor or manager, the review of an aggregate measure that indicates a significant change in a rate or trend typically triggers a response. Continuing with the intubation example (above), the manager who has the authority and responsibility to maintain the quality of care provided by all the paramedics would monitor the intubation success rate of all providers as one group and try to find causes for the unacceptable variation in rates. The group would typically be based on a bi-weekly summation of the results of all intubation attempts. The manager would define "a standard" success or failure rate. With regular monitoring and definition of acceptable variation limits, managers can know when to act. Solutions are then designed based on the aggregate analysis of how rates differ when a variety of causal variables are controlled. For example, a sudden and unacceptable increase in the rate of first attempt intubation failures per hundred cases, where intubation was indicated, would be analyzed to determine a cause. Potential causes might include the location of the patient when failure occurred, the time of day, shift on duty, training scores, individual success/failure rates of all medics. (See Cause and Effect Diagram in the QI Tools Appendix).

If the resulting action by the managers stabilizes the success rate to acceptable limits of variation over a defined period of time, then quality control has been achieved. If the result of the intervention causes the intubation success rate to reach a new, previously unachieved level of success that is sustained, then quality improvement has also occurred. Managers who have access to organization-wide, regional and/or state data have the responsibility to identify and lead quality improvement projects.

Support Services Design and Delivery

Managing quality involves every aspect of EMS operations, including services that support the EMS system's delivery of health care. Support services include:

Recruitment

Training

Human Resources

Accounting/ Payroll

Materials Mgmt.

Fleet Maintenance

Information Systems

Purchasing

Medical Control

Communications

Through careful attention to the needs of those who use support services, these support functions can be designed and managed to meet on-going quality standards and to drive continuous improvement. Typically, those who use support services include not only patients and other stakeholders, but also the EMS workforce, departments or other units within the system.

Community Health Services Design and Delivery

Community health services are population-based services that support the general health and well-being of the community served. Such services might include, e.g., CPR, injury prevention education, immunization, population screening (e.g., hypertension, cholesterol), or indigent care. The same strategies that are used in managing and improving the quality and efficiency of direct patient health care and support services can also be applied to community health services programs.

Supplier Performance Management

Key suppliers are those outside providers that supply the goods and services that are most important to the effective functioning of the EMS system, e.g., suppliers of key materials, instruments, vehicles, devices, or services. Requirements for these goods and services typically include defined quality levels, delivery times, and price. Fully apprising key suppliers of the EMS system's ongoing and changing needs, and feedback to suppliers as to whether those needs are being met, are fundamental to ensuring supplier accountability. Developing on-going working relationships and effective communication can be enhanced through joint planning, rapid information and data exchanges, use of benchmarking and comparative information, customer-supplier teams, partnerships, long-term agreements, incentives, and recognition strategies. Results of supplier performance process management may necessitate changes in suppliers.

State/Regional Agency Role in Process Management

The state or regional lead agency plays a direct roll in stimulating productive process management efforts. First, the state or lead agency can support multi-jurisdictional training in process management techniques. Second, a state can manage the quality of their own internal processes and, thereby, set an example for all other organizations in the system. Third, the state or lead agency can provide opportunities for inter-agency or inter-regional comparisons that determine current performance levels and future potential for improvement. The ability to conduct comparisons is very important: local agencies can have considerable difficulty gaining access to comparative data from their peer agencies. Finally, when comparisons are made, the state or lead agency can help create a network of leaders who can share strategies for success. In this way, the most successful organization in a particular process can work closely with those organizations that are struggling with less success in the same process.

EMS System Results

Quality improvement activities are designed to positively effect the key drivers of an EMS agency or system. Key drivers are those performance areas defined in the strategic quality plan that are most critical to the success of the EMS system or agency as it works to achieve its mission. Key drivers include customer-driven quality requirements, as well as the organization's operational requirements.

The key drivers, identified through the strategic planning process, are based on expert opinion, good judgment and common sense. Examples of key drivers are listed in the "Strategic Quality Planning". Failure to maintain quality as evidenced by a key driver performance indicator should trigger action to stabilize and reverse the decline.

Measuring EMS system results serves multiple functions, the over-arching purpose of which is to assess how well the system is doing in its key driver areas, as well as the impact of efforts to improve performance in each key driver. Results encompass three broad areas. Input results focus on the necessary resource components of the system, e.g., leadership, EMS workforce, suppliers, equipment. Process results examine the effectiveness of the design and delivery of work processes, productivity and operational performance. Outcome results look at the effectiveness of patient care, support services, and fulfillment of public responsibilities. Such public responsibilities might include, for example, improvements in disaster response and public health emergencies.

Each of the three results areas uses one or more performance indicators -- sometimes referred to as "critical indicators". Critical indicators are clearly defined measurements, typically reported in tables, charts and graphs, that compare various input and process characteristics. Comparisons are often presented over a specified period of time. Using a comprehensive set of performance indicators provides an effective way to monitor and improve results. These results are important: patients and other stakeholders need assurance that the system has proven effectiveness and efficiency; and system providers (both care givers and support personnel) need reinforcement that their work makes a difference.

The ability to accurately and reliably interpret system results depends primarily on the types of data used and the methods employed to collect the data. (See "Information and Analysis"). Results should be prospectively defined, clear and quantifiable. Current performance levels; trends; comparative performance levels; rate of improvements; and demonstration of sustained improvement can be examined in all key driver areas. Information about the results achieved by the system and quality improvement efforts provide a useful basis for ongoing strategic quality planning and QI project definition. An organizational "scorecard", charting results in key driver areas, can help leaders and workforce members understand current performance levels in relation to target goals and objectives.

Input Results

A fundamental underpinning to effective EMS systems is the performance of personnel, equipment, administration, and finances. Key performance areas include the following:

1. Productivity indicators such as effective use of EMS manpower, materials, energy, information, capital and other assets and resources.

2. General financial performance indicators such as cost of system administration, information systems, and asset utilization; cost comparisons with other similar systems.

3. Human resource indicators such as safety, absenteeism, workforce well-being and job satisfaction.

4. Supplier performance indicators⁽⁴⁾ such as supplier service dependability, and availability, as well as the durability and effectiveness of the goods sold.

Process Results

The assessment of time intervals in the context of the continuum of emergency care is an important process results measure. Such intervals would include: activation time interval (time of call dispatched to time of vehicle response), overall scene response time interval (time of call received at dispatch to arrival at patient); and time and success rate of field interventions (e.g. airway management).

Process of care also includes the delivery of clinical services. Of particular importance are those services that are essential for patient survival. Process results that show trends in success rates, such as airway management, defibrillation, or pharmacological intervention among selected high risk patient populations, demonstrate how well the strategic quality planning and improvement process is working.

Outcome Results

Patient Health Care Results. The outcome that is most important in terms of EMS effectiveness and improvement is patient health results from care rendered. There is little consensus in the literature, however, as to what constitutes "quality" patient care, and it is difficult to pinpoint with assurance the cause of a good patient outcome. The most commonly accepted definition of quality care is an increased rate of survival from a life-threatening event. Other, broader patient health care results are also important and include a variety of changes in the patient's health status. Focusing on the "5 D's" will help EMS systems examine the results of the care they render:

Death: Did the patient survive to hospital discharge?

Disability: Was there an improvement in the patient's functional status as a result of patient care rendered?

Discomfort: Was there improvement in the patient's symptoms (e.g., alleviation of pain; improved breathing)?

Dissatisfaction: Was the patient (and/or family) satisfied with service rendered?

Destitution: Was the treatment provided at lowest cost to the patient, the payor and to society as a whole?

Appropriateness of care, as well as the efficiency of care delivery, are also important to examine. Comparative data from similar systems or "best practices" data, can help provide an objective indication of system effectiveness.

Cardiac Arrest Outcomes. The patient health care result most commonly applied to EMS is survival after an out-of-hospital cardiac arrest. This result is particularly attractive as an outcome measure for a number of reasons. It focuses on a clearly definable clinical entity, i.e., sudden cardiac arrest with a clearly definable outcome (live/death). Treatment is standardized nationwide (i.e., Advanced Cardiac Life Support). EMS Advanced Life Support (ALS) has been shown to impact positively patient outcome. A wealth of literature exists to provide benchmarks for comparative purposes. Survival is time- dependant. Finally, data definitions are fairly standardized (Utstein).

As a result, an EMS system can use cardiac arrest as a measure of the systems' structural components (e.g., response), process (e.g., consistency of ALS care rendered), and outcome (e.g., survival to hospital discharge). Reporting survival rates as a function of the population served per year aids in adjusting for varying survival rates. This also helps the system control for differences in resuscitations initiated, initial rhythm definitions, case exclusions and other "denominator" variables. Factors that may impact survival, such as bystander CPR, system access, system response, and actual scene care rendered can be examined and evaluated. Those results can then be used as input to the strategic quality plan for subsequent improvement action.

Focusing on cardiac arrest has its limitations, however. Typically, cardiac arrest cases comprise only a small percentage of the care provided by an EMS system. An EMS system may not have a large enough number of cardiac arrests from which to derive useful information. Access to hospital outcome data is not consistently available, although death certificates are public documents. Significant survival determinants, such as age and pre-existing health status, are beyond the control of the EMS system. Despite these limitations, cardiac arrest survival data is useful as one of the many parameters with which to assess system performance.

Trauma Outcomes. Another patient health care result important to EMS is outcome from injury. In contrast to cardiac arrest survival, however, field trauma care, in and of itself, has not been shown to impact survival, although trauma systems, of which EMS care is a part, have been shown to improve survival following injury. Although injuries typically comprise more of an EMS system's caseload than sudden cardiac arrest, examining patient care results in this area are more difficult to assess for several reasons. The EMS response to the injured patient at the injury site may vary, depending on available resources and time and distance factors to definitive care. EMS patient management techniques for certain conditions, e.g., shock after injury, may vary among systems. In-hospital treatment, including surgery, and rehabilitative care post-discharge, significantly impact mortality and morbidity, and must be considered. Confounding factors, e.g., age, pre-existing condition or multiple injury, also need to be considered. Severity indices that have high levels of sensitivity and specificity remain under-developed.

As a result, examination of trauma outcomes generally requires a degree of complexity and expense that necessitates close interface with the regional or statewide trauma system. The time, volume, and variety of data required also limits the ability to obtain meaningful results in a relatively short time. Consequently, such results may be more useful for larger regional or statewide issues involving program planning, policy, or patient care.

For smaller systems or individual services, focusing on more basic trauma outcomes may be helpful. Preventable death studies, performed in conjunction with hospital multi-disciplinary committees, as well as field and physician representatives, are important indicators of system performance. The focus, however, should be on improving performance and on what can be learned from the death, rather than on where can blame be assigned.

Satisfaction of Patients and Other Stakeholders

Satisfaction of EMS patients and other stakeholders⁽⁵⁾ is the purpose of EMS systems, yet soliciting and tracking levels of patient and stakeholder satisfaction is typically one of the weakest parts of EMS operations. Patient and stakeholder satisfaction is important to a quality-oriented EMS system. Information on patient satisfaction levels, including singular events, trend analysis and comparisons, is an important ingredient in the strategic quality planning process. Unresolved patient satisfaction problems can threaten the stability of an EMS organization in terms of retaining contracts and market share.

Focus First on the Satisfaction of Patients and Other Stakeholders

Managing Relationships with Patients and Other Stakeholders

Managing the relationship between your EMS organization and its patients and stakeholders requires communication. Patients and stakeholders need easy access to appropriate information and assistance. They also need the opportunity to provide praise or complaints about system performance. Most importantly, formal and informal complaints received at any point in the system should be quickly and effectively resolved.

The quality-oriented EMS system has communication procedures for receiving, reviewing, and responding to praise, complaints, and comments in all the many forms in which they may arrive, e.g., phone calls made to individual providers or departments; comments made to field providers during the course of their work; newspaper articles; and other information-sharing events. The challenge is to process this information in ways that build and preserve relationships and increase knowledge about specific patient and stakeholder needs and expectations.

In some cases, patient and stakeholder satisfaction information provides an immediate opportunity to restore the quality of service that may have abruptly fallen. For example, a complaint received by a patient about a billing error may reveal problems in how bills are produced or processed. Analysis of patterns of complaints and comparisons over time may yield valuable information about targets for satisfaction improvement. For example, discovering socio-demographic or timing patterns of negative feedback may provide clues about potential causes of dissatisfaction.

Comparison to standards, to other similar systems, or to other objective data from independent sources is often fruitful in identifying strategies for satisfaction improvement. For example, if a comparison among several agencies performing identical functions reveals significant differences in patient satisfaction ratings, leaders and managers of the successful organization and struggling agency can meet to discuss experiences and opportunities. State and regional EMS agencies, with their multi-jurisdictional vantage points, play an important role in developing opportunities for comparison and for networking among the agencies they serve.

Often, however, EMS organizations are in no hurry to expose "dirty laundry", i.e., complaints. Emphasizing the perspective that defects, problems and errors are, in fact, opportunities for improvement increases the likelihood that more systems will be interested in sharing comparative data and networking to improve performance.

Implementation Strategies

The following are suggested strategies that may help EMS systems improve communication with and satisfaction of patients and other stakeholders.

• Conduct QI courses for front-line employees that include listening techniques, sensitivity training, and cultural diversity.

• Establish patient-to-provider networks to provide effective, on-going communication for feedback and information gathering, i.e., civic groups, call-in phone line, surveys.

• Solicit feedback through newsletters, Internet home page postings, local cable television spots, and articles in local newspapers.

• Characterize and chart the specific requirements for various groups of patients and stakeholders, using information gathered from market research, complaints, surveys, focus groups, and new customers.

• Determine and chart requirements and deployment strategies, on both short- and long-term bases, using various methodologies, i.e., frequency, sample size, target population, etc., with priority processing of dissatisfaction data.

• Compare customer satisfaction levels with similar EMS providers.

• Use accumulated requirements to determine service features in new or revised products or processes (e.g., development of Family Safety Program based on customer input about the high incidence of childhood bicycle injuries).

• Link patient/stakeholder feedback to workforce, as well as to leadership for strategic quality planning.

• Offer training through local training agencies, mass media, EMS symposia, satellite up-link programs, continuing medical education, or retreats.

• Patient / stakeholder communication access through voice mail number posted on the outside of units with bumper stickers, point-of-service survey card left with each individual contacted with voice mail number, E-mail number, Chief's office number and the provider's name and unit number.

• Local Quality Council examination of data to determine improvement steps necessary to address service quality issues.

• Feedback channels available continuously through as many means as possible, e.g., local newspapers, direct individual feedback (e.g., via letter or phone call to the customer who initiated the call, opening messages on voice mail numbers, frequent update of Internet home page, or by addressing focus or civic groups.)

The Baldrige Quality program uses self-assessment and external review to stimulate action and help participants gauge their progress in quality improvement efforts. In a recent article (Brown 1994), a former Baldrige award examiner suggested a self-assessment tool which we have adapted and modified to be applied to the EMS environment. The tool provides a series of programmatic milestones with which to determine progress and areas for future efforts.

The tool's questions can apply to any type of EMS organization regardless of its size or jurisdiction because every component of the Baldrige program is relevant to all types of EMS organizations. For example, all EMS organizations have leaders who must guide their organization's drive to improved quality; similarly, all EMS organizations must plan how they will achieve improved quality of service and customer satisfaction. In all organizations, improvements in quality and customer satisfaction are highly dependent on the ability to collect and analyze information, as well as a willing workforce that understands and cooperates in achieving these objectives. Organizations that use this tool, however, should first specify the jurisdictional context being examined and identify who their customers are and what product or service they need.

The introduction to this manual identified three stages of organizational development for quality improvement: I) Building Potential for Success, II) Expanding Knowledge, and III) Integration and Commitment. The seven Baldrige action categories that followed discussed QI efforts that encompassed all three developmental stages because QI efforts are multi-dimensional right from the start. Two of the criteria are perhaps most important, for they measure the impact of the other five. As you look through the questions in the tool, you will notice that the "Satisfaction of Patients and Other Stakeholders" and "EMS System Results" are the "proof in the pudding" categories, although improvement in these two areas also requires progress in the other five categories.Remember that progress will take time, often years. It is easy to become side-tracked or discouraged without a clear vision of what you are trying to achieve. These questions can help focus efforts and keep organizational QI efforts on track.

NOTE: Material for this self-assessment was drawn from an article by Brown, Mark G: "Measuring up against the 1995 Baldrige Criteria" The Journal for Quality and Participation, Vol. 7, No. 7, pp. 66-72. Used with the permission of The Association for Quality and Participation, Cincinnati, Ohio.

Instructions: This self-assessment should be completed by the senior leader of your EMS organization or a member of the leadership team responsible for developing your organization's focus on EMS quality. When you can answer "yes" to all of the questions in a particular stage, you can be confident that your organization is ready to move into the next stage of development. You should also strive to move forward stage by stage in all seven Baldrige areas simultaneously. You will notice that action areas in one category reference activity in another category. For example, a human resources section task may impact or emanate from the strategic quality plan section.

Stage I: Building Potential for Success

• Is the senior leader (CEO, Chief, COO, President, etc.) of your EMS organization knowledgeable regarding quality management theory and the benefits for your organization to the point where he/she could effectively explain and endorse these topics to others in your organization or elsewhere in the EMS system?

• Has your senior leader established a new strategic quality planning group within your EMS organization? OR Has an existing group (such as your senior management committee, the executive committee, or quality council) taken on new focus and responsibility with respect to strategic quality planning?

• Does the senior leader (or designee) of your EMS organization lead the meetings of the strategic quality planning group?

• Are all other leaders of your EMS organization knowledgeable about QI theory and the benefits for your organization? Can they effectively explain and endorse QI and its operation to others in your organization or elsewhere in the EMS system?

• Does your EMS organization have a set of documents that describes the EMS mission, vision, and values? Are these posted or distributed in such a way that all can see them?

• Did all the members of your EMS organization have input into the development of the mission, vision and values statement?

• Are the leaders developing a systematic approach for evaluating their own leadership effectiveness and involvement in QI?

• Are the criteria that the leaders use to evaluate their own leadership and involvement compatible with your EMS organization's vision and values statements?

Stage II: Expanding Knowledge

• Do leaders effectively communicate your EMS organization's vision and values to all workforce members? Are most, if not all, leaders directing or participating in educational efforts to increase QI knowledge and awareness throughout the entire organization?

• Have the leaders supported the implementation of programs that demonstrate your EMS organizations community citizenship? For example, are workforce members organizing public CPR courses, injury prevention educational programs including, e.g., violence prevention, bike safety, fire prevention and safety or other EMS related community service programs?

Stage III: Integration and Commitment

• If necessary, has the leadership restructured your EMS operations or organizational to promote a constant focus on efficiency, high performance, and meeting internal and external customers?

• Do leaders take an active role in regularly reviewing all performance measures related to strategic quality planning goals and objectives?

• Is your EMS organization active in general community support activities that go beyond EMS? For example, do your workforce members participate in and/or organize charity fund raisers, newspaper drives, holiday toy collection or repair, housing rehabilitation for the poor and elderly, adult literacy programs, or other charitable or service activities?

Stage I: Developing Potential for Success

• Has your EMS organization designed data collection and reporting systems around the needs of those who use the data to plan and make decisions?

• Does the data collection strategy identified in the strategic quality plan include a broad focus on information needs including: customer satisfaction, employee satisfaction, financial performance (if applicable), service quality, supplier performance, and operational performance?

• Has an assessment been completed of your EMS organization's ability to collect data and process information for each key performance indicator listed in the strategic quality plan?

• As a result of the assessment, have objectives been listed in the strategic quality improvement plan that are directed at improving the availability and reliability of data used in key performance indicators?

Stage II: Expanding Knowledge

• Do all EMS managers, employees or volunteers understand the correlation between different types of measures of key performance objectives and customer satisfaction, financial performance (if applicable), or patient health status?

• Has your EMS organization been successful at collecting information on at least several key performance indicators and successful at processing that data into information and feeding it back to employees, volunteers and managers on a regular basis?

• Has your EMS organization continued to question managers, employees and volunteers about how better to meet their decision making needs with improved data collection and information processing?

• Has your EMS organization made plans to collect data that will facilitate comparisons of performance with other organizations providing similar services, especially in the areas of service quality, patient care, customer satisfaction, supplier performance, employee data and internal operations and support?

Stage III: Integration And Commitment

• Has your EMS organization evaluated and made many major improvements in its measures and data collection and reporting methods over the last few years?

• Does your EMS organization regularly collect competitive (if appropriate) and benchmark data on: 1) service quality, including patient care; 2) customer satisfaction; 3) supplier performance; 4) employee data; 5) internal operations and support functions; and 6)other appropriate processes and functions? Is all bench marking data reliable?

• Does your organization systematically evaluate and improve the scope, sources, and uses of its competitive (if appropriate) and benchmark data?

• Is data from all areas of your EMS organization and on all aspects of performance summarized into a few key indices, and results analyzed to identify trends and opportunities for improvement?

• Is there evidence that all key organization decisions and plans are based upon analysis of performance data?

Stage I: Developing Potential for Success

• Has an initial strategic quality plan for your EMS organization been completed?

• Does the strategic quality plan use as a key reference your EMS organization's mission, vision, and values statements?

• Does the strategic quality plan reflect the opinions and feedback of members of your EMS organization beyond those actually involved in the drafting of the plan?

• Does the strategic quality plan include a list of internal and external customers and their requirements for quality of services?

• Does the strategic quality plan describe 12-month goals and objectives for expanding the knowledge and use of QI techniques throughout your EMS organization?

• Has an initial list of key drivers of your EMS organization been developed and included in the strategic quality plan?

• Does the initial list of key drivers also include at least one key performance indicator for each key driver?

Stage II: Expanding Knowledge

• Has the strategic quality plan been improved over the initial Stage I version?

• Was the revision to the initial strategic quality plan based on a thorough analysis of customer needs, competition (if applicable) and potential risks to your organization if internal and external customer needs were not met?

• Does the revised strategic quality plan describe the needs of internal and external customers? Is there a clear connection between customer needs and your key EMS drivers?

• Does the strategic quality plan include a list of performance measures for each of the key drivers of your EMS organization?

• Does the strategic quality plan identify long and short-term goals, objectives and strategies for each performance measure?

Stage III: Integration and Commitment

• Has your EMS organization evaluated and improved its strategic quality planning process several times over the last several years?

• Has your EMS organization developed and included in the plan specific projections or forecasts illustrating how performance will compare to benchmark EMS organizations? Is performance in key driver areas projected to be superior?

Stage I: Developing Potential for Success

• Has the level of worker satisfaction been determined on multiple dimensions, including compensation, opportunity for self-improvement, work safety, and job satisfaction?

• Has your EMS organization made a review of all its operational goals and strategies to see if adequate human resource support exists to meet these goals?

• Did the review of human resource needs and the worker satisfaction survey include consideration of the need to improve selection, training, involvement, empowerment and recognition plans?

• Within the strategic quality plan, does your EMS organization have specific quality goals and improvement strategies identified for human resource processes, such as hiring, career development including training, education, and recognition programs?

• Does your organization have a structured training/education curriculum for training all levels and functions of workers; is that curriculum based upon a thorough analysis of worker training needs?

• Are training needs derived from an analysis of competencies needed to meet key organizational goals as defined in the strategic quality plan?

• Does your EMS organization employ systematic and effective mechanisms to promote on-the-job reinforcement of skills learned in training?

• Does your EMS organization tailor the message and medium used for training to the audience and content?

Stage II: Expanding Knowledge

• Has your EMS organization begun the process or already implemented a number of innovative approaches to job and work design such as self-directed teams wherever appropriate in your organization?

• Are there new goals and strategies in place for improving worker satisfaction, safety, health, and ergonomics ?

• Has your EMS organization developed a strategy to evaluate the effectiveness of its training programs and has it begun to evaluate at least some of them?

• Has your EMS organization determined the needs for special services to workers, e.g., counseling, recreation, day care, cross-training, re-training, basic education, special benefits, drug/alcohol treatment, etc.?

Stage III: Integration and Commitment

• If your organization is a for-profit EMS provider agency, do employees at all levels have a significant portion of their compensation at risk; is the at-risk pay based on performance measures over which employees have strong influence or control?

• Does your EMS organization use several different approaches to recognizing and rewarding individuals and groups of workers?

• Do the workers feel well-recognized for their accomplishments?

• Does your EMS organization evaluate the effectiveness of all the EMS education and training programs it conducts? Is there evidence of continuous improvements in all EMS education and training programs provided by your organization as a result of the evaluations?

• Does your EMS organization have a well-defined and multi-faceted strategy in place for providing special services to workers such as counseling, recreational programs, day care, cross-training, re-training, basic education, special benefits, drug/alcohol treatment, etc.?

• Are several methods used to measure and improve worker satisfaction; is there evidence that worker satisfaction has improved as a result?

Stage I: Building Potential for Success

• Has your EMS organization developed a strategy to identify and evaluate all key processes that define or support your EMS operations to insure that critical work functions are designed and operate to meet the needs of internal and external customers?

• Has your EMS organization completed identifying and documenting via flow charts some of the key processes that define and support your EMS operations and that must function properly if internal and external customer needs are to be met?

• For documented key processes, has your EMS organization begun to identify process quality measures (key indicators) based on customer requirements and have quality standards been identified for the measures?

Stage II: Expanding Knowledge

• Has your EMS organization completed documenting its key processes and identified process quality measures (key indicators) and standards based on internal and external customer quality requirements?

• Has your EMS organization considered what the future needs of internal and external customers are likely to be and used them as a driver to begin the process of designing new processes to meet new service needs?

• Has your EMS organization thoroughly defined quality requirements for all of your key equipment, materials, and service suppliers? Have those requirements been adequately communicated to the suppliers?

• Does your EMS organization require your suppliers to have preventive and corrective processes in place to ensure that they will be able to consistently meet your equipment, materials and service requirements?

• Are data on key process measures collected on a regular basis? Does your EMS organization use valid control strategies to keep all process measures within standards or acceptable levels?

• Has the documentation of key organizational processes been expanded to include important support functions within your EMS organization? Is data on process measures collected for which specific standards or goals have been set?

Stage III: Integration and Commitment

• Does your EMS organization design new and/or improved EMS services and support processes using an approach that is based upon a thorough analysis of internal or external customer requirements?

• Does the design of new and/or improved EMS services and support processes include the use of key indicator variables that will signal if customer need is being met?

• Does the design of new and/or improved EMS services and support processes include the implementation of strategies, policies, or technology that will keep in control the amount of variation in these new or improved processes, as measured by the key indicator variables?

• Are your existing EMS service and support process designs reviewed, tested and validated by taking into consideration your service performance record, the use of your services, your process capabilities, your supplier capabilities, and the future requirements of your internal or external customers?

• Does your EMS organization systematically appraise its evaluation process? Does your EMS organization implement new policies and procedures to improve the process of evaluation in an effort to shorten the time between evaluation and introduction of improvements?

• Does your EMS organization use research, bench marking, new technology, and information from customers to initiate process improvement efforts?

• Have any of your EMS organization's key production and delivery processes been re-engineered or improved in dramatic ways over the last few years?

• Have any of your organization's key EMS support processes been re-engineered or improved in dramatic ways, resulting in improvements in cycle time, productivity, and customer satisfaction?

• Has your EMS organization implemented cooperative efforts to improve supplier quality such as partnerships, joint training for vendors and buyers, contractual incentives, supplier certification programs, and recognition for exemplary results?

Stage I: Building Potential for Success

• Are active steps underway to help employees or volunteers increase their focus on achieving quality goals?

• Are demonstration projects planned which will show to all personnel the relationship between quality improvement efforts and quality and service improvement outcomes?

• Do all efforts to orient employees and volunteers to achieving quality and operational results emphasize the role of measurement and how these measurements will be used?

Stage II: Expanding Knowledge

• Do all the employees or volunteers in your EMS organization understand the purpose and meaning of the organization's increasing focus on continuous improvement of service quality, and efficiency? Are all personnel aware that these results will be clearly measured for the purpose of demonstrating the impact of quality improvement efforts?

• Within your EMS organization, have there been some successful demonstrations of the impact of quality improvement efforts on any of your internal or external service outcomes?

• Do plans exist to allow comparison of your organization's quality improvement results with other EMS or non-EMS bench mark organization quality efforts in other geographic areas or jurisdictions?

Stage III: Integration and Commitment

• Has your EMS organization shown steady improvements in the quality of your internal and external services over the last three or more years?

• Are improvements in quality results seen on all key indicator variables used to assess product/service quality?

• Do your EMS organization's quality results compare favorably to those of your peer EMS organizations and, if applicable, your major competitors?

• If applicable, do sales, cash flow, operating expenses and other financial results show significant improvement trends over multiple years and levels of performance that are superior to competitors?

• Do the trends indicate excellent gains in reducing cycle time in applicable EMS or support services?

• If applicable to your EMS operation, do profits or retained earnings show clear improvement trends over three or more years and are profit levels better than those of other similar operations?

• Is there evidence over the last three years that your organization has been able to significantly reduce operational costs without damaging quality?

• Do measures of your EMS-related public health performance show excellent improvement trends and levels of performance that are clearly superior to other EMS organizations in your local or regional geographic area?

• Do measures of employee or volunteer satisfaction or morale show excellent improvement trends and levels of performance that are clearly superior to employee satisfaction levels in EMS organizations of similar size?

• Do the measures of employee or volunteer safety show clear and impressive improvement trends and levels of performance that are better than other EMS organizations in your local area or region?

• Does your EMS organization have data to demonstrate a trend of three years or more worth of improvements in quality or service and / or product by all of your major suppliers?

• Is the quality of the your suppliers' products and / or services superior to the quality of all major competitor suppliers?

Stage I: Building Potential for Success

• Has your EMS organization determined how it will continuously evaluate its methods for identifying customer requirements?

• Has your EMS organization identified a set of improvements in the organization's approaches to building positive relationships with customers? Does the information collected on customers and their specific needs appear useful for decision-making on how to increase satisfaction levels?

• Are systems being developed for frequently collecting data on hard measures of customer satisfaction, such as increased public financial support or repeat business, and soft measures such as opinion surveys or focus groups?

• Do plans exist for developing ways of determining levels of customer satisfaction among peer organizations or if applicable, key competitors?

Stage II: Expanding Knowledge

• Do methods exist for determining levels of customer satisfaction among peer organizations, or if applicable, key competitors?

• Does your EMS organization segment your customers according to common needs and characteristics, and use multiple methods to frequently determine customer needs and requirements relating to your EMS products and services?

• Does your EMS organization have many ways to make it easy for customers to seek information, comment, or complain about your EMS products or services?

• Does a formal system exist for tracking and resolving formal and informal complaints in a timely manner?

Stage III: Integration and Commitment

• Does your EMS organization evaluate and show evidence of continuous improvement over the last few years in your approaches to measuring customer satisfaction?

• Is there data to indicate that all major measures of customer satisfaction show a continually improving trend over at least the last three years?

• Have significant improvements been made in the levels of customer satisfaction over the last three years?

• Is there data on all major adverse indicators (e.g., complaints, unpaid bills, legal actions) that show decreasing trends?

• Is research conducted to project future customers and predict what their key requirements are likely to be? Are customers of peer organizations or competitors also studied over at least the last three years?

• Does customer satisfaction data for all your major EMS products and services show continuous improvement over the last three years?

• Is your EMS organization's level of customer satisfaction superior to that of your peer organizations and to EMS industry?

• For commercial EMS organizations, does your organization have data to demonstrate positive trends over the last three or more years in gaining and avoiding losses of customers?

• For commercial EMS organizations, does your organization have data demonstrating gains in market share over the last three or more years relative to major competitors' market share?

Note: Materials in this QI Tools and Techniques section were adapted from materials originally prepared by DOT/NHTSA's Continuous Improvement Team, Office of Strategic Planning and Evaluation. The assistance of NHTSA's Continuous Improvement Team is gratefully acknowledged. For more information on NHTSA's Quality Improvement efforts, contact the Associate Administrator for Plans and Policy (202-366-1574).

Teams and work groups use tools, e.g., brainstorming, to generate lists of process-related problems, potential solutions, approaches or options to address an issue. Once this is done, however, teams are sometimes unable to quickly and easily reduce the items on the list into a few manageable ideas. When team members perceive that more than one item has significant merit, multivoting can be used to quickly identify the most important items on the list. Multivoting is best suited for use in large groups that are reviewing long lists. It is valuable when there is difficulty in reaching a consensus on the highest priority items on a list. It is not used, however, when trying to reach a consensus on a single issue.

Guidelines

Use the following guidelines to conduct a multivoting exercise in a team or work group meeting:

• Display the items under consideration on a flipchart, making sure to eliminate duplicate items
• Number the items on the list to facilitate recordkeeping.
• Give each team member a number of votes equal to approximately one half of the number of items on the list (e.g., 10 votes for a 20-item list).
• Have each team member vote for the items he or she believes are most important. Team members may cast all votes for one item, for several items, or vote for individual items until they use their allotted number of votes.
• Tally the votes.
• Select the four-to-six items that receive the highest number of votes. Discuss and rank order the items. If the team cannot establish the top four-to-six, remove the items having the fewest votes and then conduct another vote.

Types of Voting

Dots - members are each given a sheet of adhesive dots. One dot is provided for each vote the member is allocated. If desired, each member can be given different colored dots. Members stick their dots next to items on the flipchart. As stated earlier, they may cast all votes for one item, or distribute their votes as they choose.

Show of Hands - the team leader or facilitator asks for a show of hands on each item on the list. Members keep track of how many times they've voted and are on the honor system to raise their hands only the prescribed number of times.

Ballots - The list of items is reproduced on a paper ballot, and team members identify their preferences. This method is the most confidential, and ensures that each member votes only the allowed number of times. Depending on the number of items, tabulation can be time consuming and group momentum can be lost while waiting for the count to be completed.

The example below shows the outcome of a volunteer EMS organization's listing of potential quality improvement projects. The checkmarks show the votes each topic received during the multivoting exercise.

Why Use Multivoting?

Multivoting is used to help teams focus on problem-solving and identifying high priority items in an efficient manner. It is particularly valuable in deciding issues because each member has a clear understanding of how the team will reduce the number of items to manageable proportions and how it will identify them in priority order. Multivoting allows for each member to participate equally in the decision making process. This is particularly important in gaining acceptance and buy-in for future actions based on the decision.

When we collect data about a work process, it is often helpful to illustrate the results in a graph. A run chart is one type of graph that is used to see if we perform our work in a consistent way, or if there are obvious changes as the work progresses over the course of time. A run chart can be prepared for any characteristic of a work process that we want to measure and evaluate.

In a run chart, the data for a process measure are plotted either after several batches of work are done or as work is completed over a period of time. In an intermittent or batch process, data are usually obtained in a sequence. For a continuous process, data are usually obtained at set time intervals.

Run charts can be used to monitor characteristics or features in a number of work processes in an EMS organization. Process characteristics that are typically measured include dimensions of quantity, quality, or time. When the data is plotted, the chart can be used to identify trends, shifts, patterns, and outliers that may exist in our work.

How to Construct a Run Chart

Step 1. Determine the problem or question to be studied.

Step 2. Collect the appropriate type and adequate amount of data. (Note: Ask for assistance if you have questions about the sampling plan.)

Step 3. Scale and label the horizontal, or x, axis to describe the process in the batch sequence or time period that was measured.

Step 4. Scale and label the vertical, or y, axis for the characteristic, or variable, that is to be plotted.

Step 5. Plot each data value in the sequence or chronological order that it was obtained.

Step 6. Label the graph, including a description of the process and the sample size.

When interpreting a run chart, the following guidelines apply. A trend is a change in the process where values move in the same direction over time. A shift or run is a process change in which the average or center line shifts. A pattern is any non-random result, such as a cycle that repeats over time. An outlier is a value that lies significantly outside the range of the rest of the data. These four cases are illustrated below.

A run chart can raise question about the work process and its performance. For example, are the results what would be expected? Can the shifts, trends, or outliers be explained? The data might also reveal another common problem. Often a process will be free of trends, shifts, patterns, and outliers, but is still unable to meet specifications. The solution to this problem is to identify improvements that will adjust the process to target or reduce the variability.

Why Use Run Charts?

Run charts are used to determine if a process is performing as expected and whether there are changes in a process characteristic in a sequence or over time. Run charts are also used to identify early patterns and outliers among the observed data. This analysis can be useful for problem solving and for comparing to a process standard or requirement.

As an example, a run chart was prepared to monitor ambulance response reliability during the month of January. In the run chart below the process characteristic Number of Late Arrivals on Priority 1 Calls is plotted for the month of January. The chart illustrates that the number of late arrivals fluctuates considerably, particularly around January 6^th and 26^th when major snow storms occurred. Also During this period the lowest number of late responses was 5 on January 11^th and 12th while the highest number of late responses was 26 on January 7^th.

Before taking steps to improve our work, we often collect data to see how we are doing at the present time. One way to describe and evaluate our performance is to display this data in a chart called a "histogram". In a histogram, data are grouped into defined intervals and displayed according to their frequency of occurrence in each interval. This method provides insights about performance and, in particular, the variation that normally occurs in work.

There are numerous situations where histograms can be used to show how much variation exists in work, e.g., how much time it takes an EMS organization to complete a routine job. If you repeatedly measure the length of time it takes to complete a job, you will observe that the time varies in each instance. You will also, see, however, that all of the measurements fall within a certain range.

How to Construct a Histogram

Once you have collected a set of data for a repeated activity, complete the following steps.

Step 1. Count the number of observations in the data set.

Step 2. Determine the range of the data. This is obtained by subtracting the smallest value from the largest value.

Step 3. Decide the number of intervals, displayed as bars, to use. A good rule is five to seven for 20 to 50 observations, and six to 10 for more than 50 observations.

Step 4. Divide the range by the number of intervals. Round the number to a whole number.

Step 5. Select the boundaries for each interval so that they are not overlapping.

Step 6. Count the number of observations that fall within the boundaries for each interval.

Step 7. Draw, scale, and label the horizontal (x-axis) and vertical (y-axis) axis lines of the chart. Label the x-axis for the intervals that cover the data range. Mark the vertical axis from zero until the highest frequency is included.

Step 8. Draw vertical bars for each interval. The height of the bars equals the number of observations at that interval. The width of each of the bars should be the same.

Step 9. Title the completed chart. The title should describe the nature of the observations being made summarized in the chart and the time frame in which the data was collected.

Why Use Histograms?

A histogram is a picture of the data distribution that includes its spread and shape. This can provide clues about the variation that exists in the work performed. Distributions can be skewed in either a positive (tail of the distribution to the right) or negative (tail of the distribution to the left) direction from the center. By examining the spread and shape of a distribution, the extent of variation in a work process can be determined. This can provoke further discussions to identify the cause of variation and the measures needed to either control or reduce it.

A foundation of our continuous improvement effort is data collection. Data are used to better understand variation in a work process and determine how well we are doing in meeting standards based on patient and other stakeholder expectations. A histogram is a useful tool to display these findings in order to identify our current performance and show how we are improving work processes over time.

In the example in Figure 1, the Histogram shows the response interval performance of an EMS agency to emergencies during one month. This chart shows how well the organization is meeting the needs of its patients.

Sometimes, a problem keeps us from completing a job as well as we would like. The problem may result from long-standing policies and procedures or because of a lack of adequate equipment or facilities. These problems can become more complicated to resolve if several people are working together to complete an assignment. A cause-and-effect diagram is used to show the causes of a problem. Since there is generally more than one cause to any problem, the diagram is used to further divide causes into groups or categories. This approach often uncovers the root causes of our problem. When the root causes are identified, we can evaluate how much each cause contributes to the problem.

Constructing a Cause-and-Effect Diagram

The following steps are used to construct a cause-and-effect diagram. These diagrams are sometimes called "fishbone diagrams", because they resemble a fish skeleton when completed.

Step 1. Develop a statement of the problem. Write it down on the right side of a piece of paper (the fish head). Draw a central arrow across the middle of the page that points to the problem.

Step 2. Brainstorm a list of probable causes of the problem. Write each of these down on another sheet of paper.

Step 3. Review the list of causes and identify the major categories. Write down the names of the categories as main branches (fish bones) off the central arrow.

Step 4. Review the causes and list each under the appropriate category. If necessary, revise or expand the list of categories.

Step 5. Write down each cause as a small branch drawn off the main category branch for the category under which it falls.

Why Use Cause-and-Effect Diagrams?

Cause-and-effect diagrams can help clearly illustrate possible relationships between causes. They can be used to uncover the root causes of problems or specific problem steps or bottlenecks in a work process. By arranging possible causes into categories in a diagram, we can develop a better understanding of problems and the contributing factors. To prepare a diagram, we must expand our original understanding of the problem situation. Our exploration often gives us a look at the underlying assumptions of our work. While a cause-and-effect diagram is an effective analysis tool, it only helps us identify possible causes or categories of problems. Even if everyone agrees on the list, it is important to determine what is not known about each cause and how that information can be uncovered. If necessary, we must collect additional data and analyze it to identify and confirm actual causes.

(Click on the picture to view a larger version)

This cause-and-effect diagram was developed to identify the causes for why ambulance run reports are inaccurate. The central arrow points to the problem statement. Main branches lead to four categories of causes. These are: People, Methods, Machines and Materials. For each major causal category, there are a number of specific causes that are shown as smaller branches. When creating this diagram, a quality improvement team brainstorms a list of category specific causes and marks them on the diagram. In this figure, under the major category of methods, the QI team believes that the education programs used to teach personnel how to use the run forms properly may be ineffective in changing the behaviors of existing personnel or may not be adequate to fully inform new personnel. After listing and developing this list of "suspected" causes, the team would decide which causes are most influential and worth pursuing.

Everyday in EMS systems, hundreds of tasks are completed in order to meet specific objectives. Much of our work flows between departments, offices and other organizations. It is easier to see how specific tasks and activities contribute to our mission if we can picture the whole process. A flowchart illustrates the activities performed and the flow of resources and information in a process. Two types of flowcharts are particularly useful -- high level and detailed.

High Level Flowchart

A high level flowchart illustrates how major groups of related activities, often called "subprocesses", interact in a process. Typically, four to seven subprocesses are shown in a flowchart. By including only basic information, high level flowcharts can readily show an entire process and its key subprocesses. An example of a high level flowchart is shown in Figure 1. The four subprocesses are: EMS system access; information gathering and triage; prearrival instructions; and dispatch.

Detailed Flowchart

A detailed flowchart provides a wealth of information about activities at each step in a subprocess. An example of a detailed flowchart for two of the access and dispatch subprocesses is shown below. It shows the sequence of the work and includes most or all of the steps, including rework steps that may be needed to overcome problems in the process. A quality improvement team can increase the detail to show the individuals performing each activity or the time required to complete each activity. If necessary, the link between various points in the subprocess and other high level flowcharts of the process can also be shown.

How to Draw a Flowchart

Flowcharts are drawn using these symbols as building blocks.

Activity: A square or rectangle identifies a step (task, activity) in the process. The name of the step is written inside.

Decision: A diamond identifies a decision or branch point in the process. Each path emerging from a decision block is labeled with one of the possible answers to a question that is posed at this point in the process.

Flow: An arrow indicates the sequence and direction of flow within the process. This is usually the transfer of an output of one activity to the next (where it becomes an input).

Output/Input: A parallelogram identifies a material or information output or input from an activity. The name of the output (input) is written inside.

Connector: A circle is used to indicate a continuation of the process flow elsewhere on the same page or on another page. The same label written on the connector symbol appears on another connector where the process flow continues.

Activity blocks are the most common elements of a flowchart. They can be arranged in serial or parallel paths depending on how activities are actually performed. Decision blocks indicate conditional situations where the output of an activity needs to meet certain criteria before the process can continue. If the criteria are not met, a different set of activities follow. This is often called "re-work" and is drawn as a feedback loop in the flowchart.

Why Use Flowcharts?

An EMS organization pursuing quality improvement is constantly looking for ways to improve the effectiveness and efficiency of its work. "Effectiveness" means producing the required results or output when needed. "Efficiency" means simply producing those results or outcomes the first time with minimum resources. In order to generate ideas on how to be more efficient and effective, it is helpful to define and document how activities are actually performed Flowcharts are useful for this purpose.

Flowcharts can be useful to identify activities in a process that reduce our effectiveness and efficiency. For example, some activities may be redundant or repeated, others may be unnecessary. Activities may be performed in sequence, when they could be conducted at the same time to reduce the overall time for the process. Flowcharts can be used to identify conditions that cause delays and bottlenecks. This can bring focus to problems at various points within the process that need further evaluation and improvement.

(Click on chart to see a lager version)

There may be many causes for problems or conditions that adversely affect work processes. A pareto diagram is a type of bar chart in which the bars representing each problem cause are arranged, or ranked, by their frequency in descending order. A Pareto diagram is useful in interpreting data and confirming the relationships that are suggested in cause-and-effect studies. This approach is based on the idea that 80% of the problem comes from 20% of the causes; the diagram is used to separate the "vital few" problem causes from the "trivial many". This aids in focusing on correcting or improving the vital few causes that contribute most to the problem.

How to Construct a Pareto Diagram

Step 1:	Decide on the problem, type of data needed and cause categories (see Cause-and-Effect diagram).
Step 2:	Collect or obtain data.
Step 3:	Order the causes or categories.
Step 4:	Calculate the cumulative total and the percentage of the total for each cause in a cumulative percent table.
Step 5:	Draw and label the horizontal, or x axis, including an interval for each cause.
Step 6:	Draw, scale and label the vertical (y) axis on the left side of the diagram. Mark the y axis from 0 through the cumulative total. Draw and label the vertical axis on the right side of the diagram. Mark this axis from 0 to 100% percent, corresponding to the cumulative total.
Step 7:	Draw the vertical bars for each cause, in the order of the highest to lowest frequency (from left to right). The width of each bar should be the same.
Step 8:	Plot a point at the center of each bar equal to the cumulative totals, until the total adds up to 100%. Add a zero point at the left side of the first bar. Connect the points with straight lines.
Step 9:	Title the diagram to describe the nature of the observations and the time frame in which the data was collected.

The figure below shows an example of a Pareto Chart. In this case an ambulance company wanted to identify the leading causes of incomplete data on run reports. Company managers found that about 35% (using the cumulative scale on the right) of the incomplete forms were missing the patient's zip code. Further, 35% of the missing data sheets showed that the ID of other responding agencies was missing. These two data elements accounted for 70% of all run reports with missing data. With this information, managers can work with field personnel to identify potential solutions. The benefit of the Pareto Chart analysis was to isolate the two major sources of missing data so that the most effective solutions can be pursued. Also, examining the specific data elements that are most likely to be missing may point out specific solutions ideas. For example, in this case, zip code data is missing because map books in the ambulance do not show zip codes. Simply adding a zip code map book could solve the problem.

bench marking: The practice of setting operating targets for a particular function by selecting the top performance levels, either within or outside a company's own industry. In a broader sense, bench marking involves searching for and copying new ideas and best practices for the improvement of processes, products and services.

brainstorming: A technique to generate as many thoughts and ideas as possible within a defined time (e.g., 10 - 15 minutes).

cause: The reason for a problem or defect.

common cause: A reason for a problem or defect that is inherent in the production process.

consumer: The recipient of the service provided. (See "customer").

critical indicator: clearly defined measurements that compare various input and process characteristics.

cross-functional teams: Teams composed of workforce members from several different work units.

customer: In the broadest sense, the recipient of the service provided or the purchaser of the services. Generally, however, the term refers to the purchaser of the service. (See "consumer").

cycle time: The amount of time it takes to complete a defined task.

defect: Non-conformance to requirements; the lack of a necessary characteristic.

effectiveness: Conformity to requirements; the degree to which the service is performed in the correct and desired manner.

external customers: Individuals, entities and organizations who are outside of the actual operation of the EMS system and who receive services provided by the EMS system, a component of the system, or an individual working within the system.

internal customers: Individuals, entities and organizations who are involved in or with the operation of the EMS system and who receive services provided by the EMS system, a component of the system, or an individual working within the system. Internal customers include EMS' employees and volunteers, members of the leadership councils or committees that plan and coordinate the system, agencies that interact to form the ongoing system and other health care providers.

key business drivers: Imperatives that are critical to customer satisfaction, competitive effectiveness or goal achievement.

key suppliers: Outside providers that supply the goods and services that are most important to effective functioning of the EMS system.

linkage: Interactions that effect coordination and completion of tasks.

Malcolm Baldrige National Quality Award: National award given to companies and businesses in recognition of their achievements in quality. The Award is managed by the U.S. Department of Commerce's National Institute of Standards and Technology.

mission statement: A statement that describes the fundamental reasons for the existence of the organization.

objectives: Measurable statements that are consistent with the mission, vision and key drivers.

problem: The result of non-conformance to patient and other stakeholder requirements.

process management: Improvement of work activities and work flow across functional or department boundaries.

quality: The extent to which products and services meet or exceed customer requirements.

quality assurance (QA): Retrospective review or inspection of services or processes that is intended to identify problems.

quality care: The extent to which health care services meet the patient's needs and produce the desired health outcome.

quality improvement: The continuous study and improvement of a process, system or organization.

quality indicators: Characteristics of products, services or processes that represent quality.

sentinel event: An undesirable event or phenomenon that triggers further analysis and investigation.

stakeholder: Individuals and organizations, other than the patient who receives the EMS services, that have some interest in the operation of the EMS organization, e.g., the patient's family, the community in which the EMS system operates, government officials, the patient's insurer/third-party payor, and health care providers.

strategic quality planning: An integrated planning process that incorporates strategic planning with quality planning.

value statement: A statement that identifies basic tenets and principles of how people work together.

vision statement: A statement that declares where the organization wants to be in the future. A vision statement serves as a major focal point of an organization's strategic quality plan.

Note: The summaries contained herein are intended to provide stimulus for discussion and consideration of basic quality theories, their practical application to health care in general and EMS in particular, and associated issues and problems with such applications. In summarizing the documents, re-wording was necessary in some instances; in those cases, every attempt was made to retain and convey the authors' original meaning. Also, in some documents, the authors used quality terms interchangeably, e.g., "quality assurance" and "quality improvement". In those instances, no attempt was made to reconcile the use of quality terms, i.e., the summaries reflect the authors' choice of terminology. Finally, the enclosed summaries are organized into sections, e.g., "Theoretical Frameworks". Consideration of the information contained in the documents, however, should not be confined to one particular area, since most articles contain information applicable to a broad range of areas.

Prehospital care, with its emphasis on a systems approach, is fertile soil for quality improvement approaches. There has been a great deal of EMS interest in Quality Assurance (QA) and Quality Improvement (QI).; however, there is scant published research in the area, and no well-defined literature base to EMS quality management.

EMS system structure is important in evaluating and improving care. Because of the way EMS systems developed, there are many different types of EMS systems and administrative structures. The resulting complex (and non-uniform) approach to prehospital patient care throughout the country will require that EMS systems individualize their evaluation and QI strategies.

In developing a QA program, EMS systems are initially faced with problems in defining the scope of the system's authority. Prehospital care, including dispatch, is often rendered by multiple personnel and organizations over which EMS agencies have limited authority. Similarly, EMS agencies may have little ability to influence QA activities with receiving facilities.

Common data elements for QA activities must be defined early. Data collection and handling, including coordination of various data sources, is often burdensome. As there are many sources of patient care data, EMS systems must work with various entities to achieve data access or submission, assure confidentiality, collate individual patient data, and centralize and control record handling responsibilities.

Regarding standards for evaluation, there are only a few well-defined standards in EMS. While certain national standards are applicable across the board, e.g., training performance standards, all standards must address local realities and be developed through broad-based consensus approaches in order to be enforceable. Key considerations include resource availability, medical community support, geographical constraints, and political issues.

Evaluation of EMS system components must occur at both the level of individual service and regionwide. Prospective evaluation is the major means by which EMS quality has been ensured, e.g., credentialling of providers. Systems should consider development of standards to allow evaluation of dispatch, facilities and personnel who provide medical direction and receive patients, and the EMS Medical Director. Concurrent evaluation provided on-scene is the optimal method of ensuring quality. Alternatives include upon patient arrival at the hospital and on-line medical direction. Retrospective review is the most time-consuming and least valuable. Coordinating a comprehensive retrospective review of a system with multiple providers is an enormous task: EMS systems must consider appropriate use of resources and attain a systemwide agreement as to the value of the review. EMS evaluation often focuses on outcome measures; cardiac arrest and trauma have been typically used to examine EMS outcomes.

A successful quality program will identify and quantify areas for improvement. Change can be facilitated in identified areas through provider education, hospital-based clinical rotations, provider QA participation, and physician input. Management support for QA is vital. EMS systems must also work to remove constraints on QA that result from lack of legal protections for QA activities.

In modern American health care, there are two approaches to the problem of improving quality.

The first approach is based on the "Theory of Bad Apples" that holds that quality is best achieved by discovering and removing bad apples. This approach advocates quality by inspection and includes activities such as recertification, establishment of thresholds for acceptability, and requires research into better tools for inspection (e.g., increasing sensitivity and specificity). It involves the search for outliers, examination of mortality data, and vigilant regulation. According to this outlook, one can use deterrence to improve quality and reward / punishment to control people who do not care enough to do what they can do or know is right. This approach leads to a defensive and fearful workforce.

The second approach is based on the "Theory of Continuous Improvement". This approach holds that problems, and therefore opportunities to improve quality, are usually built into the complex production process and that defects in quality are only rarely attributed to lack of will, skill, or benign intention. Even when people are at the root of the defect, the problem is generally not one of motivation or effort, but rather of poor job design, failure of leadership or unclear purpose. According to this outlook, real improvement in quality depends on understanding and revising the production processes on the basis of data about the processes themselves. Continuous improvement is sought throughout the organization through constant effort to reduce waste, rework and complexity. The focus is on the average producer, not the outlier, and on learning, not being defensive.

The steps to be followed in establishing the Theory of Continuous Improvement in health care are:

1. Leaders who speak for the profession must establish and hold to a shared vision of a health care system undergoing continuous improvement.

2. There must be substantial investments in quality improvement, including managerial time, capital and technical expertise, as well as in education and study to understand the complex production processes used in health care.

3. Respect for health care workers must be reestablished; health care workers must be assumed to be trying hard, acting in good faith, and not willfully failing to do what they know to be correct.

4. Dialogue between customers and suppliers of health care must be open and carefully maintained.

5. Modern technical, theoretically grounded tools for improving processes must be used.

6. Health care institutions must organize for quality.

7. Health care regulators must become more sensitive to the cost and ineffectiveness of relying on inspection to improve quality.

8. Professionals must participate in specifying "preferred methods of care", but must avoid minimalist "standards of care".

9. Individual physicians must join the effort for continuous improvement. QI has little chance of success in health care without the understanding, participation and leadership of individual physicians.

The traditional approach to quality of care is too narrow to meet need of modern health care providers; too static in emphasizing conformance to standards instead of continuously improving existing practices; focuses on physician performance while underemphasizing nonphysicians and organizational processes; and stresses technical expertise and interpersonal relations while leaving out other important factors, e.g., ability to mobilize organizational resources to meet patient needs.

Industrial quality experts define "quality" as a continuous effort by all members of an organization to meet needs and expectations of customer. Recognition and analysis of variation is fundamental to quality management and particularly important in medical care where there are multiple sources of variation that combine at random to cause variations in outcome (sources of variation include, e.g., patients, preexisting condition, presentation, provider mix, availability of diagnostic tests, and accuracy of results). When multiple sources of variation are present, isolated observations provide insufficient information for objective decision making. Optimal decision making requires application of basic statistics to series of observations.

Central principles of industrial quality improvement as applied to health care: 1) senior administrative and clinical leaders should explicitly and actively pursue an ethic of continuous improvement in the quality of care and service; and 2) processes, not individuals, should be the objects of quality improvement. Processes are complex and require a systematic approach to their analysis and improvement. Processes are frequently characterized by unnecessary rework and waste: modifications that reduce rework and waste may simultaneously improve quality and reduce cost. Personnel at all level can be trained to use simple analytic techniques and graphical methods for the study of process.

Substantial quality improvement can be achieved by eliminating unnecessary variation in the execution of treatment processes. Elimination of unnecessary variation in clinical practice may improve care. Consensus "best practices" should be developed at the institutional level, based on medical literature and local needs/constraints, and updated as needed. These are vastly different from requiring mandatory adherence to externally imposted, static guidelines/standards. Focusing on process also complements traditional quality assurance reliance on outcome measures: since outcome measures do not identify the causes of defects, they are more useful when examined in conjunction with process.

The routine assessment of quality is rarely linked with the day-to-day management of health care systems or with the decisions made by the individual and aggregate purchasers of health care. New directions in health care delivery are increasing pressures for health care to reevaluate how quality is assessed, to consider how information regarding quality may be used, and to challenge existing notions of definitions of quality.

Modern health care quality assurance has two central strands of inquiry: what is to be studied and by what methods will the study occur. "Structure, process and outcome" are typically offered as the potential objects of investigation. Three methods of investigation are typically promoted: implicit review - uses of groups or experts to judge the how well a system or provider dealt with individuals or groups of patients; explicit review - involves specifying criteria for care and review of records / observations to check conformity to criteria; and use of sentinels - defines classes of unacceptable or red-flag events and then investigates the events, using implicit or explicit reviews. It is noteworthy that implicit and explicit reviews seem to produce different results: in head-to-head comparisons, implicit and explicit reviews of the same cases have yielded significant discrepancies in ratings. Industrial quality measurement techniques, in contrast, are quite different and have benefitted from five decades of trial, development, evaluation. There are three important lessons to be learned from these techniques.

Lesson 1: High quality design is more efficient in the long run than thorough inspection at the end of the production line. Modern quality control engineers try to control quality in the actual design of the product, rather than performing inspection of the end product produced.

Lesson 2: The quality of a product involves a fundamentally multidimensional concept. Emphasizing the structure-process-outcome aspect of quality is too narrow. Much health care effort has been invested in demonstrating the relationships between the elements of care (structure and process) and the outcomes of care. According to this view, the measurement of process is a surrogate for measuring the real goals of health care: improved health status, function, and comfort. Making health care status outcome a central focus, however, has serious limitations. First, it burdens the exploration of quality with the agenda of virtually all clinical and health services research, and too little is known about what in health care actually produces health. If the relationship between process and outcome is unknown, then measuring outcome is not a useful indicator of quality because it is unknown which aspects of the process to preserve and which to change. Second, there is good reason to believe that a great proportion of health care probably does very little to alter the course of illness. Unless outcome is defined broadly to include many elements (e.g., patients' feelings, attitudes, and satisfaction), then outcome-focused research will likely conclude that much of health care is wasteful, useless and scientifically unsupported. The reality is that health care does not deliver outcome, but rather the process, itself.

Lesson 3: There must be an investment in a corporate culture geared towards producing a high quality product. It is not enough to develop new techniques of measurement and control to be implemented at various points of the manufacturing process.

Finally, health care's reliance on the medical record as the primary source of data for quality is flawed since it is widely acknowledge that medical records are impaired by differences in record keeping systems and variations in recording practices. Health care should use other data sources as well, e.g., surveys, observations, simulations. Use of industrial QI's practical statistical methods will greatly assist in health care's movement into QI.

[The remainder of the article described the Harvard Community Health Plan's Quality of Care Measurement Program, not included in this summary.]

For quality to be improved and managed, it must be defined in terms of specific measurable attributes; the level of quality is defined by the level of selected attributes. A system is a sequence of actions by, and interactions between, functional units that bring about the delivery of a service. The nature of the units' actions and interactions can be explicitly defined (e.g., by protocol) or implicitly defined by the norms of the local culture. Systems combine both explicit and implicit elements into a "usual way of doing things". In any system, there will be a certain percentage of bad outcomes (defects) because there is an uncountable number of interconnecting causal pathways by which bad outcomes can occur and for which no system can account for all.

Failure to achieve a desired level of quality stems from two sources: either they are attributable to the system ("systemic") or are attributable to causes external to the system ("extrasystemic"). Systemic problems have three characteristics: 1) every participant in a system is at risk of experiencing the systemic problem; 2) the rate at which systemic problems occur is predictable over the local run, during which the level of both good and bad outcomes will be fairly stable; and 3) the level of systemic problems will fluctuate over the short run. In contrast, extrasystemic problems tend to cluster by person, place and time. The cause of the extrasystemic problem is not part of the system; therefore, the problems occur only when the special cause is operating.

Most current health care quality monitoring focuses only on extrasystemic causes (e.g., search for statistical outliers); focuses exclusively on individuals; and examines individual breaches from accepted practice. Quality programs based on the threat of retribution, however, have a limited ability to improve the quality of health care because the overall level of quality is determined primarily by the system, not its outliers. (However, c.q.i. is poorly suited to detect egregious deviations from acceptable standards that place patients at unacceptable risk and require immediate attention. Thus, a certain amount of record review and surveillance remains necessary.)

Once the attributes used to define the quality of care have been selected, continuous quality improvement can be organized around the following four tasks:

1. Separation of externally caused problems from systemic problems. For individual occurrences, one cannot reliably determine whether an event is due to the system or to a cause external to the system. In the aggregate, however, the characteristics of systemic problems allow identification of when extrasystemic causes are operating. Statistically derived data should be used cautiously as an indicator of the presence of extrasystemic problems--a statistical outlier is a question about quality, not an answer.

2. Monitoring the system to make sure no new extrasystemic problems occur. In monitoring, comparisons are made between historical experience and current experience.

3. Studying the system to identify how problems arise within the system. The goal of this task is to study the system in order to guide system changes to improve the inherent level of quality. The process of system improvement may be difficult and will likely involve a diagnostic phase (requiring a specific understanding of the current system and the major factors involved in failure to achieve a desirable level o quality) and a remedial phase (involving the implementation of changes to improve the system).

4. Evaluating the efficacy of changes to the system. The same techniques used to study the system can be used to evaluate changes. Large improvements are not expected: the ideal is continuous improvement through a series of incremental improvements, not great leaps. Process evaluation verifies that the planned change has actually occurred; outcome evaluation assesses the influence of the change to the system on the occurrence of outcomes. Changes in outcomes measures should be viewed circumspectly; many outcomes, such as mortality, are strongly determined by patient characteristics and thus are largely out of the control of the medical care system. Large numbers of patients may be necessary to evaluate the effectiveness of system changes on patient outcome; where small patient volumes exist, historical data or data from similar settings may be used.

This article discusses challenges that health care organizations will face as they work to integrate TQM and CQI into their systems. The fundamental patient care process is driven by the knowledge and experience of individuals physicians, nurses, and allied health care workers. Therefore, it is a daunting task to evaluate and redesign health care production processes that are oriented to the special needs of individual patients and to the skills of the 50-60 diverse health care workers directly involved in their care. This unique aspect of health care gives rise to the following challenges.

First, physicians, who are critical players in health care production processes, are generally not employees of entities providing care, e.g., hospitals, health systems, and may not care about improving the entity's effectiveness or efficiency. As a group, however, physicians are interested in helping their patients. Therefore, when TQM and CQI activities center on monitoring and improving the clinical quality of patient care, physicians will likely become involved. To facilitate this involvement, the following strategies are suggested: involve physicians in activities that are of direct clinical relevance to patients; target physician involvement to identify and use physicians who will be perceived as leaders among their peers; and present physicians with hard evidence, not opinions, about existing practices.

Second, health care workers sometimes display a greater alliance to a profession's body of knowledge and behavior codes than to the values and goals of the organization for which they work. This profession-over-organization orientation can result in barriers to interdisciplinary communication, compromise and problem-solving techniques and processes. The following strategies are suggested: because inter-professional conflicts often arise over resource constraints, rather than true interprofessional issues, design TQM/CQI implementation to address internal resource allocation issues; present TQM/CQI as a mechanism for developing better group problem-solving skills and more effective policies; build on existing interprofessional cooperation and communication strengths; and identify areas of interprofessional overlap, as well as areas of unique knowledge and skill, and ensure that implementation teams represent all involved areas.

Third, multiple levels of management hierarchy have developed that emphasize "vertical and portional" rather than "horizontal and organization-wide" approaches to problem-solving and tend to push problems that could best be addressed at the immediate patient care delivery level up through bureaucratic structures. Organizations must reduce the number of management levels within the hierarchy while providing mechanisms and incentives for horizontal collaboration. Suggested strategies include: a focus on meeting patient care needs; and redrawing the organizational chart to improve the ways in which patient care needs are met.

Fourth, traditional evaluation and reward systems emphasize individual technical competence, rather than the overall quality of team performance and productivity; these systems have done little to foster interest in broader organizational quality and productivity. Suggested strategies include: explicitly identify as a performance requirement behaviors that facilitate interdepartmental team building, problem solving and cooperation; similar adjustment of employee recruitment, retention and education programs; and active incorporation of effective labor relations practices into TQM initiatives.

Fifth, workers who have advanced to the management level may have done so because of well-developed technical skills, but may lack training in and understanding of basic management, supervisory and problem-solving skills. As a strategy to deal with this, senior management must invest heavily in educating its manager cadre in both TQM and basic management skills.

Sixth, many health care workers view with skepticism the use of industrial statistical quality control methods and processes and believe that they are incompatible with the highly individualized nature of patient needs, health services and delivery mechanisms. Suggested strategies include: use of a targeted (as opposed to a broad) approach to monitor, via statistical control processes, how consistently providers comply with basic care processes; investment in education and information systems to assist in use of specific tools; and creative adaptation of industrial control processes into health care settings.

Quality initiatives are receiving increased attention due to regulatory mandates, e.g. JCAHO; a genuine desire among providers to improve patient care; and need to reduce costs that result from poor quality.

Substantial financial advantages can be gained through improved quality. There are two main components to the costs of quality: 1) the cost of measuring quality and 2) the cost of poor quality. The cost of measuring quality is the cost of the quality assurance program itself, e.g., costs of the quality assurance staff, costs of disseminating and reporting data to management and workers, costs of hardware/software. The cost of poor quality is more pervasive, less easily defined and potentially much larger.

It is difficult to measure the costs of quality in service industries, such as health care. The product delivered is less tangible and more perishable, and waste, inefficiency and lost opportunities are more difficult to identify. Even less quantification of cost reductions from quality improvement has taken place in the health care industry. Early research has suggested that 40% or more of health care operating costs are spent on "nonconformance".

In the area of emergency care, even less research has been conducted. One study that examined the impact of overcrowding and long waits for hospital admission found that ED inefficiencies resulted in more than $2 million per year in Medicare reimbursement to the hospital.

The ideal emergency medicine model for defining costs of poor quality or improvements from quality initiatives would be comprised of a list of quality indicators that are easy to measure and can be tracked on a periodic basis. These could be used to estimate cost shifts from quality programs and the impact that QI strategies have on physicians, providers, and patients. An example of one such item is patient complaints. Quantification of these costs might include personnel time handling the complaints, costs of bills lowered or dropped for good will, and adverse public relations.

The potential fiscal impact of improved quality on health care providers and organizations is substantial. There is a need for increased health care research in this area of quality improvement.

As a group, physicians may be especially problematic when industrial techniques are introduced into medical practice institutions. While based on principles known in traditional quality assurance, modern quality process encompasses major departures from traditional QA and is based on objective statistically-based methods that are not controlled by physicians. For example, retrospective review of individual patient charts is replaced by analysis of statistical data; subjective physician review of questioned cases is augmented by quality process imported from industry, including Pareto charters, scatter diagrams and control charts.

Traditional health care quality review is built on an implicit assumption that clinical personnel generally and physicians most centrally are the sole determinants of patient care quality. Modern quality theory holds that quality is the end result of a complex interaction of people and support systems and that all personnel, working as a coordinated team, are essential to high-level quality process and performance. Modern quality management encompasses a complex paradigm shift in institutional quality culture and practice.

When faced with this shift, physicians may respond negatively. Physicians are unlikely to have been exposed to the concepts of teamwork so essential to quality improvement. Further, physicians tend to oversimplify institutional problem solving and systems improvement approaches. The general lack of trust and, to a certain extent, accountability that exists between physicians and management in traditional medical practices, also can negatively affect physician perception of and involvement in quality management initiatives.

In order to minimize these problems, a shift from traditional quality assurance to modern quality improvement techniques should be framed in a supportive manner for physician participants. Physicians should understand how these techniques can help them gain a greater sense of participation and proactive influence in patient care. Early quality management projects should be structured around the dual goals of improving patient care quality and appropriately empowering physician participants.

Quality assurance as health care policy has been ineffective in enabling good care and functions to retrospectively police the quality of care by identifying offenders who fail to adhere to normative values. Many concepts of health care quality assurance grew from the structure - process - outcome model. Industrial quality management strategies are becoming more common in health care, although EMS systems have long used concepts of quality management to measure and improve clinical service, e.g.., System Status Management.

Industrial quality management theory identifies three universal processes of management for quality: quality planning, quality control and quality improvement (Juran). Regarding leadership for quality, industrial quality theory holds that the key to creating a successful quality management program lies in the consensus support and substantive commitment of leadership to the mission of quality. The Medical Director in the EMS system has a pivotal role in this regard.

1. Structure and Quality Planning - In the structure - process - outcome model, structure is viewed as an vague and "rather blunt instrument" for assessing quality. In contrast, in modern industrial management, structure is of paramount importance. In order to apply industrial management quality planning techniques, EMS systems must:

   • Identify the customers of the EMS system. Patients and their families, their physicians their hospitals, the medical community, other public safety agencies, community at large and the governments of the areas where the system operates.

   • Determine the customers' needs. EMS leaders must seek out and incorporate customers' perspectives as to their needs, which will vary according to type of customer. For example, for customers who are patients, the need is for a high level of reliability for the provision of equal access to definitive care for all customers in an EMS service area.

   • Develop product features that respond to customer's needs. Industrial management theory requires that every product feature should: meet the needs of the customers (e.g., EMS system must meet patient care needs); meet the needs of the system as a supplier, including internal customers (e.g., the needs of government to control necessary aspects of EMS service delivery); meet the demands of competition (e.g., through techniques such as competitive bidding or performance reviews); and should minimize costs to both customers and suppliers (e.g., requirements for cost-effective service).

   • Develop processes that are able to produce the features. A process is a "systematic series of actions directed toward achievement of a goal", examples of which would be demand pattern analysis and peak-load staffing to achieve better EMS response times.

   • Transfer the resulting plans to the operating forces. This typically occur through dry-runs, or pilot tests that involve interaction among all involve personnel.

2. Process and Quality Control. Traditional QA defines process as the characteristics of provider behavior in the management of health and illness that involves a set of activities between practitioners and patients; traditional QA holds that process is best observed by the study of documentation of care. Industrial quality management, however, views process from an operational perspective: a process can be detrimentally affected by many events that can damage its ability to meet operational goals; thus, control must be exercised to minimize damage by either preventing its occurrence or restoring the status quo. EMS systems need strategies to assess and improve their consistency and reliability. Nearly all effective control of EMS systems is exercised by the workers; controllability of processes is largely a function of self control by individual EMS providers.

   Standardizing processes to increase their reliability and productivity is a central concept of quality management. All processes exhibit some degree of variation, an understanding of which is key to quality control. A system that is in statistical control has a definable identity and capability and is one where all special causes of variation have been removed. The processes by which medical care is provided are probably not stable; however, there are instances of processes in EMS that are in statistical control (e.g., systems with 90% reliability of 8-minute response times).

   The benchmarks for QM's concepts of quality control as applied to EMS are as follows:

   • Evaluate actual product performance. The measures and tools for evaluating EMS processes are problematic but improving. Cardiac arrest has been used as a benchmark for EMS system performance. Improvement in EMS systems depends on the definition of essential data for field performance and reliable retrieval methods.

   • Compare actual performance with product goals.

   • Act on the difference. The process of quality control requires application of a feedback loop. Leaders and managers should act on differences between goals and performance through participatory management that focuses attention on the system of work, not on the individual.

3. Outcome and Quality Improvement. Traditional quality assurance defines outcome as changes in health status as a result of structure and process. In EMS, however, direct association between the process of care and outcome is difficult to ascertain. Modern quality improvement as applied to EMS requires:

   • Establish the infrastructure. Quality improvement is subtle and gradual but pervasive in the organizational culture, involving everyone from managers to workers. QI in EMS will fail if it is viewed as a quick fix. Medical directors function best as leaders, role models, political champions, clinical experts and consensus builders. The key to quality management in EMS is for the medical directors to let go. Only where the worker (who has nearly all the ability to control the processes of care) is given the appropriate knowledge to choose, the responsibility for the results of their actions, and share the authority to change the system, will EMS be a quality-driven practice of medicine.

   • Identify the improvement projects and establish project teams. Management should identify its agenda for improvement based on consensus, and with the help of experts, if needed. The initial improvement project should be significant to both management and workers, have a relatively straightforward solution, and provide a quick success to enhance QI momentum. The quality team should reflect a cross section of management and workers.

   • Provide the teams with resources, training, and motivation to: 1) diagnose the causes; 2) stimulate remedies; and 3) establish controls to hold the gains. There are a number of programs that can assist with introducing QI processes to EMS organizations. Resources would include the following statistical tools:

     1. Process flow chart. A graphic representation of the sequential steps in a process. Construction of the chart often reveals hidden aspects of a process, hidden customers, new perspectives of processes, illustrates critical steps, and locates process flaws.

     2. Ishikawa, cause-and-effect, or "fishbone" diagrams. Categorize and display in groups theories about how and why processes fail. These diagrams promote understanding of each member's elements, how individual perceptions are often incomplete and how problems are often interrelated. Elements to be considered are: the 5 "P's": patrons (external customers), people (internal customers), provisions (supplies), places (the work environment), and procedures (policies and protocols).

     3. Check sheets. Organize necessary data elements efficiently to allow for collection.

     4. Histograms. A frequency distribution chart of a process or distribution of continuous data. Usually in the form of a line or bar graph that illustrates the variation of continuous data such as time, weight, size or temperature.

     5. Run or trend chart. Illustrate changes in process over time.

     6. Pareto charts. Graphs the frequency of occurrences of defects by type from greatest to least. Assists in identifying where areas of priority to decrease defects.

     7. Statistical process control chart. The most powerful of the tools, a control chart illustrates trends in control of processes and graphically identifies some common and special causes of variation. A typical chart will show the variability of sample or population data around a mean; a companion chart quantifies the range of variation.

CQI's "system" approach, that improvements in patient care result from improvements in the entire system, rather from better performance by providers, is compatible with the EMS system approach of care. EMTs should be involved with the development and implementation of quality programs. In general, remedial activity should focus on patterns of practice rather than on individual deviations from protocol. Emphasis should be on education and retraining; sanctions should be reserved for situations in which it is necessary to protect the public.

JCAHO's 10-step monitoring and evaluation process serves as the standard model with which to manage quality of care provided:

1. Assign responsibility
2. Delineate scope of care
3. Identify important aspects of care
4. Identify indicators
5. Establish thresholds for evaluation
6. Collect and organize data
7. Evaluate care
8. Take actions to solve problems
9. Assess actions and document improvement
10. Communicate relevant information to the organization wide QA program

A "standard" is a generalized goal that is an achievable model of excellence and is used to define expectations. An "indicator" is an objective behavior or outcome that can be measured to determine compliance with a standard. A "threshold" is an established level or percentage of acceptable compliance that indicates when further evaluation should be initiated.

QA and CQI are essentially widely divergent methods. QA is essentially a problem-identifying mechanism, while CQI is essentially a problem-solving method. CQI focuses on system problems; QA focuses on people problems. QA uses retrospective inspection to enforce minimum standards and identify problems. CQI examines processes to identify areas for improvement; defects are analyzed using statistical principles and, when identified, are considered to be opportunities for improving the process. In QA, standards are generalized goals that are achievable models of excellence used to define expectations; in CQI, standards are based on best-practice models that are emulated throughout the system. The traditional QA practices of retrospective review and inspection should not be discarded, however, but should be used as tools in a strategy of overall quality management and serve as a subset of a much broader CQI program.

When developing a systemwide EMS quality assurance (sic) program, "structure" should be addressed separately from "content". Effective organization of the structure of the program before the content is written will encourage participation by and among all system components. The central lead agency should take ultimate oversight responsibility. An ideal systemwide program would include all components of the EMS system; each component should have a committee to be responsible for activities within that component. Local or regional EMS agencies / committees should provide tiers of oversight and create levels of graded responsibility to supervise individual services or components. Issues that cross geopolitical boundaries should be part of a systemwide review.

The content of the program should be flexible and responsive to changes for continuous system improvement. Content must be dictated by local needs, circumstances, and available resources.

There are several important pitfalls to be avoided in designing EMS QA (sic) programs:

1. The program must be patient-care oriented and must avoid "bean counting". Asking the right questions and avoiding meaningless data collection are important. Data should be collected only if they can be subjected to analysis that will provide significant and timely feedback on actual patient care.

2. The program should avoid being too centralized in a "quality" office since participation from all providers is the only way to obtain timely and meaningful feedback with which to improve patient care.

3. The program's emphasis should be on producing significant improvement of the system, not punishment.

"Closing the loop" is using information gained in studying a system to change behavior within the system. This chapter discusses two approaches to closing the loop: the traditional approach and the continuous improvement approach.

The Traditional Approach. EMS is prone to focusing on outliers and single problem occurrences through case reviews in order to identify workers who need "fixing". Once identified, "closing the loop" is achieved by modifying the behavior of the outliers. Categories of QA actions in closing the loop include: actions on the system; actions on personnel within the system; and continued observation of the system and system personnel. Most medical directors currently use an informal method of positive feedback, usually one-on-one, and a formal method for negative feedback that meet the needs of due process.

The disciplinary focus of a QA program is fraught with problems that are faced by all systems. Focusing on identifying the "bad apple" usually leads to defensive reactions and maintenance of the status quo. Workers view with mistrust increased observation by system managers, and eventually, audits, inspections and other quality activities are viewed with suspicion. Workers consequently try to manipulate the system.

The Continuous Improvement Approach. Health care has begun to recognize the potential of the Deming management method and the theories of continuous improvement. In this approach, EMS workers are impressed with the idea that every defect is a treasure that provides an opportunity to learn more about the EMS system. Workers become empowered and able to participate in system learning and collaborate in QI efforts. Managers have the responsibility, not only to individual patients, but also to ensure that the EMS systems extracts the maximum amount of knowledge from its experiences: every process in every system provides information that can be used to examine if system goals are being met.

Through process examination and use of statistical tools, managers can distinguish between a stable system (i.e., under control, regardless of how variably it performs) and an unstable system; and then identify the special causes of trouble. Once the causes have been removed and the system is performing in a stable manner, managers can focus on improving overall system performance by examining causes of intrinsic system variability, resulting in continuous improvement and system evolution. System managers must determine the control limits of processes in their system in order to minimize the likelihood of confusing special causes of problems with common causes of problems.

High system variability often results from an unclear purpose. Occasionally, poor job design also contributes to poor system performance. Consequently, leaders must clearly articulate a constancy of purpose that is directed to the true organizational mission and ensure that education and job orientation programs are relevant to the real job. Listening to field and system support workers can be they key to uncovering defects. These individuals can identify specific problems, direct management attention to parameters that should be measured, and facilitate identification of root causes of system variability.

In order to achieve continuous improvement in EMS, senior EMS system managers must commit to the concept and practice. Assistance in the form of outside sources may be required. As system managers determine the control limits of various processes, they must not allow the occurrences of bad outcomes that fall within control limits to provide an excuse to return to traditional practices of searching for outliers. Instead, these instances should provide the motivation to eliminate special-cause variability so that common-cause variability can ultimately be effectively addressed.

Each EMS system must devise a strategy for transforming to CQI. A sensible strategy would encourage starting with a small component that can be encompassed within the grasp of a single, committed senior manager. A successful demonstration project, no matter how small, begins the necessary process toward system transformation.

Quality has historically been regarded as an attribute that was "inspected" after a product was produced, the primary responsibility for which rested with the inspection group, not the producers. During WWII, innovative methods of quality control were developed; these methods were subsequently successfully applied by the Japanese. These methods hold that quality is primarily a human process, rather than an end product; and if the processes themselves can be fixed, problems in the outcomes can be prevented, thereby eliminating the need to correct errors.

The quality model generally used in the contemporary health care organization was created by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO). JCAHO emphasis on QA has shifted from peer review to criteria-based audits to ongoing monitoring and evaluation. JCAHO's monitoring and evaluation model emphasized outcomes management through a 10-step process.

Total quality management (TQM) and continuous quality improvement (CQI) offer a scientific approach to quality improvement based on quantitative methods and organizational development principles; emphasize change processes and long-term commitment; and offer participation on decision-making that spurs improvement. Leaders must create the vision; establish objectives; develop a game plan; provide the tools; support the teams; create an environment that fosters quality; be obsessed with customers' needs (including employees); and be results oriented, process driven and fact-based. Constant refinements in processes and performance are needed to consistently achieve better results. Performance standards are raised through improved work processes and reducing "variation" by increasing the consistency of performance.

Quality in health care may be measured and improved in the following six dimensions: 1) competency/ credentialling (professional and/or organizations); 2) appropriateness/accessibility (UR); 3) resource utilization (efficiency/cost); 4) effectiveness (desired outcome, quality assurance); 5) safety/risk management (medical/legal); and 6) customer satisfaction. Results from a survey of focus groups of paramedics suggested the following 15 EMS quality indicators:

Patient satisfaction

Managerial satisfaction

Patient outcomes

Dispatch accuracy

EMS crew satisfaction

Call quality

Partner performance

Response times

Complaints

Paramedic wellness/occupational illness

Equipment practicality

EMS cost-effectiveness

Public confidence

Crew and equipment appearance

Innovations/research

The orderly transition from quality assurance to continuous quality improvement requires understanding and integration of the QA and QI models. A Process Quality Management Improvement process is suggested that revolves around "plan-do-check-act":

Step 1: Ownership of the process.

Step 2: Define the process and identify customer requirements.

Step 3: Define and establish measures.

Step 4: Assess conformance to customer requirements.

Step 5: Outcome evaluation

Step 6: Rank opportunities and set targets for improvements

Step 7: Achieve a new level of process performance by developing action plans, determining root causes, testing and implementing solutions, and maintaining the gains achieved.

The following definitions are contained in this reference book for health care language.

quality 1. A character, characteristic, or property of anything that makes it good or bad, commendable or reprehensible; thus the degree of excellence that a thing possess. 2. The total quality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs. 3. Fitness for use.

quality assurance (QA) 1. All planned or systemic actions necessary to provide adequate confidence that a service or product will satisfy given requirements for quality. 2. Designing a product or service, as well as controlling its production, so that quality is inevitable. 3. In health care, the activities and programs intended to provide adequate confidence that the quality of patient care will satisfy stated or implied requirements or needs.

quality of care The degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.

quality control (QC) 1. The process through which actual performance is measured, the performance is compared with goals, and the difference is acted on. 2. The use of operational techniques and statistical methods to measure and predict quality.

quality improvement (QI) The attainment, or process of attaining, a new level of performance or quality that is superior to any previous level of quality.

continuous quality improvement (CQI) In health care, a management approach to the continuous study and improvement of the processes of providing health care services to meet the needs of patients and other persons. Continuous quality improvement focuses on making an entire system's outcomes better by constantly adjusting and improving the system itself, instead of searching out and getting rid of "bad apples" (outliers). Synonyms and near-synonyms: continuous improvement (CI); hospital quality improvement (HQI); quality improvement (QI); and total quality management (TQM).

total quality management (TQM) A continuous quality improvement management system directed from the top, but empowering employees and focusing on systemic, not individual, employee problems.

It is critical to identify local levels of performance ("standards" or "norms") with which to facilitate ongoing quality improvement (QI) activities and evaluate EMS. Standards can be developed for any objective EMS activity, although some standards that deal with subjective areas, e.g., patient satisfaction with prehospital care, are more difficult to develop.

Standards may take many different forms. A standard may be a baseline or minimal goal that must be met; it may be an optimal, but achievable goal; or it may be a goal that is set intentionally high in order to stimulate participant efforts to excel or increased funding to increase likelihood of achievement. Unachievable standards can serve as lighting rods for criticism and litigation.

Standards are best developed through consensus building: full involvement of as many individuals and organizations is important. Standards should be tailored to individual systems and never simply be copied from a reference book.

The timing of standard development is also important. A given EMS system should neither be the first or last to develop and implement standards: the innovative system will be bogged down by either defending or correcting the new standards; the laggard system will spend months/years languishing in a suboptimal situation.

EMS standards may be developed locally, regionally, at the state level, or nationally. Local standards rarely have the force of law, but are often widely accepted and followed. Regional standards are often weak, since few regional bodies are capable of enforcing such standards and, at a regional level, broad-based consensus is often difficult to obtain. State-level standards are generally very powerful, but may be highly unpopular. Since the state is often the strongest authority for ambulance providers, state-developed standards may be the most useful for local QI programs. National standards promulgated by the federal government are very powerful, although national standards developed by voluntary organizations can often be easily side-stepped. Voluntary standards that become a formal part of a certification process can become powerful.

Once standards are developed and approved, they must be explained and implemented in an understandable manner. Following a period of education and questioning, the standards must be rigorously enforced. Introducing and adopting standards can be painful. When change is necessary, it is helpful to make as many simultaneous changes as possible, to be followed by a period of stability and integration.

This article suggests EMS quality assurance standards from a structure-process-outcome perspective.

EMS System Access. Desirable structural standards for EMS system access and their methods of evaluation are:

1. Structural standards
   1. 9-1-1 in place
   2. Rapid triage to medical dispatch
   3. Translation services
   4. Poison-control interface

2. Methods of evaluation
   1. Time-interval reports
   2. Consumer questionnaire
   3. Online supervision
   4. Audit

EMS System Dispatch. Desirable system dispatch and methods of evaluation are:

1. Structural standards
   1. Dispatch priority algorithms/protocols
   2. Prearrival telephone instructions
   3. Medical control of algorithms/protocols
   4. QA Plan

2. Methods of Evaluation
   1. Online supervision - Medical and Dispatcher
   2. Tape reviews
   3. Feedback reports
   4. Dispatch and field data - Ideal Dispatch and Field Interval Standards:

      1. "Determine time" - the period of time a dispatcher takes to determine the appropriate ambulance response: < 30 sec, 90th percentile

      2. "Queue time" - the period of time a caller waits for an available ambulance, the interval between when the call is received and when the call is determined: 0

      3. "Roll time" - the interval between the time of ambulance dispatch and arrival on scene: 5 min, 90th percentile

      4. "Response time" - the determine time, queue time, and roll time added together: < 8 min, 90th percentile

      5. "Field time" - <20 minutes

      6. "Transport time"

         - Urban: <10 minutes

         - Rural: <30 minutes

      7. "Hospital time" - the time from when the ambulance arrives at the hospital to when it clears the hospital: < 15 min

First Responders and EMT-Basics. Desirable structural standards and methods of evaluation are:

1. Structural standards
   1. Written protocols - Medical and procedural
   2. Level of training > basic EMT
   3. Early defibrillation capability
   4. Scene triage (two-tiered systems)
   5. Medical control and QA plan

2. Methods of Evaluation
   1. Response times and other intervals
   2. Online supervision
   3. Online medical control
   4. Retrospective peer review
   5. Tape review
   6. Medical control module review
   7. Audits

Paramedics. Because paramedics are the backbone of most EMS systems, QA must cover assessment, treatment and skills maintenance. Because assessment and treatment occur simultaneously, they should be evaluated together. Desirable structural standards and methods of evaluation are:

1. Structural standards
   1. Prescribed equipment
   2. Written protocols - Medical; Provider interface; and Destination, to include response times and other intervals
   3. Medical control and QA plan

2. Methods of evaluation - patient assessment and treatment
   1. On line supervision - Medical control; Comparison with ED/Hospital diagnosis; Documentation (explicit vs. implicit criteria) to include computer-assisted audits; Peer review; Concurrent radio skill review.
   2. Tape review
   3. Comparison of clinical data elements with those performed by RNs/MDs to include outcome (survival; LOS); review of unanticipated therapies; and audits

3. Methods of evaluation - skills maintenance
   1. Online supervision
   2. Physician verification
   3. Success rates
   4. Outcome
   5. Audits

Receiving Hospitals. The receiving hospital is a critical component, although it is usually difficult to set standards for receiving hospitals or to incorporate them into the quality plan.

1. Structural standards
   1. Recordkeeping
   2. Personnel - Physicians (board-certified; special-certified; resident coverage); Nurses (certification; continuing education)
   3. Services and equipment - Referral and transfer agreements; Medical equipment; and Communication equipment.

2. Process standards
   1. Compliance with laws, regulations
   2. EMS committee activity
   3. Data collection an analysis
   4. Provision of orientation, training and continuing education in EMS
   5. Participation in EMS QA
   6. Participation in disaster planning
   7. Acceptance of patients

3. Outcome standards
   1. Outcome data - Survival; LOS; Other
   2. Passing score on audit

4. Evaluation methods
   1. Paramedic feedback
   2. Self-assessment tool
   3. Audits
   4. Patient-satisfaction surveys.

Medical Control Hospitals.

1. Structural standards
   1. Recordkeeping
   2. Personnel
   3. Physicians (including medical director, medical control nurses and paramedics) - Board certification; Special certification; Minimum FTEs
   4. Services and equipment (meet or exceed receiving hospital standards) - Clerical support; Computer hardware/software; Adequate office space; Supplies for exchange; Redundant communication equipment; and Recording equipment

2. Process standards
   1. Compliance with laws, regulations
   2. Committee membership
   3. Documentation, data collection and analysis
   4. Training, orientation, and continuing education in EMS issues
   5. Participation in system QA
   6. Participation in disaster planning
   7. Participation in prehospital research
   8. Online medical control - MDs, medical control RNs/paramedics, and staffing; Duties and responsibilities
   9. Field observation for MDs and medical control RNs
   10. Continuing education requirements for MDs and medical control RNs

3. Outcome standards
   1. Outcome - Survival; LOS; Other
   2. Audit performance
   3. Response time to medical control radio

The ideal EMS QA system will be one where managers have developed comprehensive QA standards, criteria, methods of evaluation, and feedback to frontline personnel. The chosen standards should emulate those used in the most renowned systems, or at least reflect the best system possible considering available resources.

Efforts to provide quality assurance in EMS have been inconsistent and sporadic; its focus on retrospective review of documented patient care was viewed as punitive by manner. Unlike traditional QA programs, however, quality improvement programs assume that all EMTs and members of the EMS system want to do the best they can for every patient and, given the proper tools and training, EMS personnel will provide high quality care. The principles of continuous quality improvement (CQI) as applied to EMS are: 1) build quality in to the EMS system; 2) use positive, instead of negative reinforcement of EMS personnel; 3) when a problem occurs, examine the EMS system first, not the individual; 4) clearly define expectations before holding individuals accountable; 5) expect but limit variations in the EMS system; and 6) continuously strive for knowledge that will help the EMT respond effectively to patient needs.

The quality improvement loop refers to the ongoing process of improving the EMS system (see Figure). If standards are not met, the cause of the noncompliance will most often be with the system itself. If the system is not at fault, the standard may be unachievable or may have been improperly implemented. Rarely will the EMS personnel be found to be the outright offender.

EMS standards must be clear, measurable and reflect the expectations of the system. Once standards have been established, all personnel must receive education in the standards, after which the standards may be implemented. There are five (5) types of standards that must be set for an EMS system:

Patient care standards - these define a specific, unified, acceptable approach to each commonly encountered patient problem so that personnel can provide a specific and consistent quality of prehospital care. Thus, protocols must be developed for the patient presentations likely to be encountered by the EMS system and must meet three (3) criteria: 1) the protocol must be medically correct; 2) the protocol must be capable of implementation in the field; and 3) the protocol must be cost-effective.

Time Standards - these standards must be individualized for the system resources. The most important of these standards are response time, on-scene time, and transport time.

Procedural Standards - these standards cover all procedures that will be performed by an EMS system and describe how to do the procedure, where and when it is appropriate, and how much time should be allowed for its performance. Examples include AHA's ACLS for dysrhythmia treatment.

Educational Standards - these standards will ensure that the initial training of EMS personnel corresponds to the expectations of EMS practice and will include minimum standards for continuing education and testing.

Equipment Standards - these cover the type, quality and maintenance of EMS equipment.

After the standards are implemented, information from many sources must be examined to determine whether patient care is being delivered as intended. These sources will include EMS run reports, hospital records, dispatch records, direct observation, coroner's reports. Questions about how patient care is rendered will often take the form of a "performance standard", i.e., a measurable objective that evaluates progress toward achieving a goal established for the system, such as successful IV placement in 90% of cases.

Quality improvement activities may be prospective, concurrent, or retrospective. Prospective QI occurs before the patient receives care and has the potential of preventing mistakes from occurring. Prospective QI activities include primary or initial EMS education, continuing education, periodic skills evaluation, and apprenticeship / preceptor programs. Concurrent QI occurs while the care is being rendered by EMS personnel and includes direct field observation, indirect field observation via radio or telemetry, and observation upon hospital arrival.

Retrospective QI occurs after care has been rendered and is the least beneficial, but most common of the three. Retrospective QI includes debriefing after hospital arrival; critiques that involves the medical director and EMS personnel; chart reviews to determine record completeness, accuracy, logic and correlation with the ED record; chart audits to determine personnel compliance with system policies and protocols; focused audits that review only one or two parameters within a large number of charts; and incident reports that document and evaluate both unexpectedly good and poor outcomes.

This paper reports the results of a national consensus project that resulted in the development of an 81-item uniform EMS data set.

Prehospital data is intended to serve as the legal documentation of the patient encounter and medical record of the prehospital phase of care; provide necessary information for billing of services; serve as the foundation for research, quality improvement programs, and system evaluation and alteration; and assist in the allocation of societal resources. Despite its importance, little progress has been made in development of uniform prehospital data collection and reporting due to three main obstacles: 1) EMS systems are largely under local control and few local administrators or medical directors have a perspective or concern beyond the needs of their own subsystem; 2) limited financial and personnel resources and research expertise; and 3) lack of a lead federal agency to direct a national consensus project.

NHTSA undertook to establish a consensus-based uniform prehospital data set. Data elements, their relative priority ("essential" or "desirable"), definitions and comments were subsequently developed (see attached). The purpose of the data set and definitions is to provide common terminology and definitions to be used in EMS evaluation. Prehospital data should be linked with hospital and autopsy outcome data and efforts should be made that accurate, high-quality information is collected in the pre-hospital setting.

This book describes an explicit and rational approach to interpreting clinical performance data. This information printed below is a shortened version of the end-of-chapter summaries provided in the text.

Clinical performance data are neutral quantitative measurements of important patient care processes or patient health outcomes that are generated by application of performance measures ("indicators"). Interpretation of clinical performance data is a multi-step process by which meaning is assigned to raw clinical performance data. These data quantify important dimensions of clinical performance: i.e., appropriateness; availability; continuity; effectiveness; efficacy; efficiency; respect and caring; safety; and timeliness. The ability to draw accurate conclusions from raw clinical performance data depends on the degree to which its interpretation is conducted in an explicit and rational manner.

There are six tasks to the process of interpreting clinical performance data. First, interpreters must select a relatively modest amount of focused data for interpretation. A useful unit of data for interpretation is an indicator data set (a collection of data generated by a common indicator). Second, in order for data to have meaning, it must be viewed in the context in which the data arose. The indicator information gives users a context for, and the important characteristics of, the data. Third, the strength of the data must be assessed to determine if the performance information generated therefrom is trustworthy. Fourth, raw data must be expressed in a frequency distribution that permits conclusions regarding the relationship between one series of observations and another. Fifth, a certain amount of variation (fluctuation in a series of results) is inevitable; however, unusual, undesirable variation, may detrimentally affect quality. Control charts can be used to examine variation within processes over time; comparison charts aid in the study of variation across different organizations. Sixth, determining the cause of undesirable variation can be assisted by use of flowcharts, cause-and-effect diagrams, and brainstorming. Pareto diagrams and checksheets can identify the "vital few" causes for organizational focus.

The ability to effectively interpret clinical performance data depends on the ability to sustain a sharp focus on a discrete clinical performance issue through the interpretation process; the indicator data set provides a logical and useful focus. An indicator data set is a collection of data generated by applying an indicator that frames and quantifies one or more important dimensions of a performance issue. One limitation of an indicator data set is that it focuses on a discrete dimension of performance related to a clinical performance issue; the data focus may need to be expanded to include other indicator data sets that reflect other dimensions of the performance issue.

Making sense of clinical performance data requires understanding the context for the data, as reflected in the characteristics of the indicator that generated the data. Clinical performance data measure: > 1 dimensions of performance; patient health outcomes or care processes; sentinel events or aggregate performance data; and desirable/undesirable outcomes/processes. Clinical performance data may be either continuous variable data or discrete variable (attribute) data.

Data interpreters must carefully consider the strength of clinical performance data, the attributes of which are as follows. Relevance is the degree to which clinical performance data relate to the functions or process of organizations and the relative importance of these functions or processes. The range of health care processes and outcomes must be addressed to give data users a complete portrait to accurately judge organizational performance and to allow effective placement of resources. Data reliability, the extent to which data results are consistent across repeated measurements, should be present at an acceptable level if data are to provide an accurate representation. Data validity, the extent to which an indicator and its data measure only what they were intended to measure, involves the constructs of face validity, construct validity, convergent validity, and scientific validity. The data's degree of variation influences improvement opportunities. Wider variation in data usually signals more improvement opportunities than narrower variation; similarly, a large discrepancy between a data set's average value and the desired value signals more opportunities than a small discrepancy. Finally, organizations should have control over the processes and outcomes that are undergoing measurement; otherwise, improvement efforts can be futile.

Variation is fluctuation in a series of results over time. Variation is inescapable in any system or process. Common cause variation is due to the process and is inherent in all processes . Special cause variation is due to factors that intermittently and unpredictably induce variation over and above that inherent in a particular system. Two common mistakes made regarding variation are: 1) reacting to an outcome as if it came from a special cause, when it was actually due to a common cause; and 2) treating an outcome as if it came from common causes, when actually it came from a special cause. Tampering is acting on some signal of variation without taking into account the difference between special-cause and common-cause variation.

Whenever data between organization is being compared, two conditions must be met: 1) the process or outcome must demonstrate good control before attempting to understand variation observed among organizations; and 2) confounding factors that differ between comparison groups and influence outcomes of interest must be statistically eliminated or reduced, or comparisons will yield biased results. Risk adjustment is used to overcome the effect of differences that can influence indicator rates and distort comparison.

The major objective of studying variation is to judge whether a process is in good control, and thus, whether future results can be predicted. Stabilizing an out-of-control process achieves and maintains a steady state. Processes that are statistically in good control, but do not adequately meet customer needs/expectations, can be examined to identify opportunities for change that are expected to improve all future results.

Identifying causes of variation can be assisted by use of the following tools and techniques. A flowchart is a pictorial summary that shows the steps, sequence, and relationship of the various operations involved in performing a function or process. A cause-and-effect diagram enables interpreters to identify the causes responsible for the observed variation. Brainstorming elicits large numbers of ideas from a group that is encouraged to use its collective thinking power to generate ideas and unrestrained thoughts in a relatively short period; it is often used to generate the underlying causes for cause-and-effect diagrams. Pareto analysis involves determining which few causes of a process are vital and taking action to address these causes, as opposed to the innumerable other incidental causes; the "80-20 rule" says that approximately 80% of the value comes from 20% of the elements. Thus, the Pareto chart is used to focus attention on the "vital few" areas. A checksheet, used in constructing a Pareto diagram, is a data collection form that summarizes counts for individual cause categories and is used to determine how often certain events are happening.

There is little understanding of the impact of EMS on patient outcome, major reasons for which include a lack of involvement by academic medical institutions in EMS research, the inapplicability of traditional medical research approaches to the highly complex and uncontrolled EMS environment, and the fact that accurate data is difficult to efficiently and reliably collect. Major problems in prehospital data collection are as follows:

1. Assumptions about what data are available may be wrong. Researchers have found a high degree of variability in the types and methods of interventions provided in the field, rendering it difficult to reliably and accurately answer simple research questions.

2. Providing a good data collection system does not guarantee good data collection. No matter how sophisticated or expensive, the best data collection system must be evaluated for data entry compliance and data validity prior to its use for system analysis.

3. Human factors might be more important than structure factors. EMS data are collected in a large number of uncontrolled and highly unusual circumstances by personnel who are under stress, conceivably in danger, and may be near exhaustion. The assumption that reliable data is collected in such a setting is suspect.

4. Researchers may be collecting the wrong data. Careful attention must be given to identifying those data items that reflect outcome alterations, as opposed to data elements that are presumed to be associated with outcome.

5. Data collection methods are suboptimal. Few research studies have involved systematic in-field observations. Any major improvement in understanding EMS systems will require specific, accurate, timed observations, and better use of available technologies.

6. Much of the data relies on perceptions and estimates. Many EMS systems treat as hard data information that is based on perception and estimates by EMS personnel, despite the fact that researchers believe such perceptions/estimates to be unreliable.

7. Different systems may have different problems. The limitless number of regional variations that impact data collection may alter information reliability and vastly complicate efforts to compare outcomes among systems.

Suggested methods of addressing these problems include: 1) medical directors must develop close relationships with key EMS personnel; 2) cultivation of interest in system evaluation and improvement; 3) use of outside experts and/or independent observers; 4) minimize collecting useless data elements and be realistic about system data needs and collection capabilities; and use of new technology that minimizes use of personnel for data collection.

Direct observation of field performance provides the most reliable and accurate method of orienting and evaluating the new paramedic because it provides an immediate critique and maximum educational impact while preserving patient safety. However, it is the most difficult method for the physician advisor to provide personally in a busy prehospital care system. This paper reports the results of a field instructor program designed to provide a system of quality control, performance evaluation, and introduction and orientation to the EMS system for new employees.

The program used 6 - 8 carefully selected senior paramedics as "field instructors" who met strict criteria; these paramedics were supervised by the paramedic education coordinator and physician advisor. These senior paramedics oriented new employees to the system and then rode as observers for a minimum of one week. Subsequently, new employees worked with the field instructors on a rotating basis. The program emphasized continuous verbal and written communication, both between field instructors and their charges and between field instructors, the paramedic education coordinator, and the physician advisor. After initial evaluations, field instructors developed a plan to meet the new employee's training needs. Decisions to promote the paramedic to standard duty were reached only after satisfactory completion of rotations with all field instructors. Those who failed to progress were asked to resign or are terminated.

The authors believe that the program predictably identifies individuals who will and will not perform to required standards. The authors say that the financial costs of the program are minimal, although the stress upon each field instructor is considerable as they are responsible for the education of the new recruit, as well as the safety of the patient being treated by the new recruit.

Users of on-line medical command (OLMC) assume that on-line consultation is an important factor in maintaining the quality of prehospital care [and in conducting concurrent QI efforts]. This assumption has led to the recommendation that the physician should be in direct communication with the paramedic when they provide advanced life support. This study evaluated the efficacy of OLMC use under this broad patient inclusion rule.

The database consisted of 7,862 cases seen during a one-year period. DOA's and all BLS cases were excluded from the study.

Results showed that in 30% of ALS cases, paramedics did not use OLMC. In the 70% of the cases where OLMC was used, OLMC was more likely to be used for cardiac patients and less likely to be used for trauma patients; among cardiac patients, OLMC was more likely to be used for patients suffering from lethal arrhythmias than those with nonlethal arrhythmias; among trauma patients, OLMC was more likely to be used with blunt, as opposed to penetrating trauma patients; and OLMC was more likely to be used with unconscious patients than conscious patients.

Determination of time to call, defined as the elapsed time from arrival on scene to the initiation of on-line communication, was based on 1,767 cases where the information was available. Average time to call was 10.5 minutes, but varied significantly across incident type: time to call was longer for cardiac patients (11 min.), compared with trauma patients (9.8 min.).

The use of OLMC was associated with improved paramedic compliance with prehospital protocols, as determined by the percentage of cases subsequently diverted to medical investigation. Base physicians deviated from protocols in 3.4% of the cases (0.02% of ALS cases). The use of OLMC was also associated with a significantly longer on-scene time of +8.4 minutes (for both trauma and cardiac patients), increasing the systems' average on-scene time with OLMC to 26 minutes.

The technological characteristics of OLMC use and the existing empirical evidence raise the possibility that targeted use of OLMC with explicit use of paramedic discretion could result in more efficacious use of OLMC. The choice between OLMC use under a broad patient inclusion rule and a system that relies on paramedics to initiate communication for a small fraction of patients means that paramedics must correctly triage OLMC use; the authors believe that, with additional training, paramedics could be relied on to correctly triage OLMC use.

The authors conclude that OLMC use cannot be assumed to contribute to improved care for all patients, or that a broad patient inclusion rule with limited paramedic discretion results in efficient use of OLMC. On-line communications should not be viewed primarily as a component of the medical control system, but as a potential aid to the prehospital medical care system.

This study compared the prehospital care delivered by multiple agencies and their paramedics in a suburban EMS system to assess whether a receiving hospital quality assurance audit would improve paramedic and agency performance.

A committee composed of physicians, nurses and paramedics developed performance criteria consistent with EMS policy. Runsheets were reviewed by committee members and compared with a checklist (see below). Deficiencies were categorized and tabulated. A performance profile (average deficiencies per run) was calculated for each agency and each individual paramedic. Reports of individual agency performance were compared with other agencies, and individual paramedic performance was compared with all paramedics. Results were returned to supervisors of each agency with subsequent feedback to paramedics.

During the study period, significant improvement occurred. The audit provided clear, objective, quantified information regarding individual paramedic and overall agency performance; agency supervisors disseminated audit information to their paramedics. The audit was determined to be an effective method of tracking agency and individual performance and resulted in improved compliance with county protocols for patients delivered to the receiving hospital.

There is no widely accepted standard model for the chronologic time sequence of EMS response and care. This study tested a time interval model, developed by the authors, to elucidate more clearly the sequence of EMS response.

The time interval model was evaluated by direct observation of ALS during a one-year period in the four EMS regions in Arizona, involving both urban and rural systems. The patient population was a convenience sample.

The on-scene time interval was composed of patient access, initial assessment, scene treatment, and patient removal, the largest single component of which was patient removal. On average, scene treatment accounted for only 31% of the on-scene interval, while accessing and removing patients accounted for 46%. Rural agencies experience longer response intervals, recovery intervals, out-of-service intervals, and patient access intervals, but shorter initial assessment intervals. For both rural and nonrural, on-scene IV line attempts were associated with longer on-scene and treatment intervals, but were explained almost entirely by associated delays, rather than the procedures themselves.

A significant amount of the on-scene interval is impacted primarily by logistic issues, rather than by medical care: nearly half the on-scene interval was taken up with reaching the patient and removing the patient from the scene. This patient access interval may explain some differences noted in outcome among systems for out-of-hospital cardiac arrest. Nonurban agencies were more likely to experience logistic an operational problems. The existence of at least one logistic problem led to a significant increase in the on-scene, patient removal and recovery intervals. Future investigation will be required to validate this model in other systems and states.

This article discusses the use of Continuous Quality Improvement (CQI) in trauma care to improve quality, efficiency, and patient outcome.

The movement from QA to CQI represents a complete paradigmatic shift and significant cultural change for the health care organization. Traditional QA focuses on individuals, using a reactive process that evaluates issues retrospectively to prevent their reoccurrence. QA decisions are often based on assumptions of causes; therefore, solutions may not completely resolve problems, so problems seem to reoccur. QA tends to encourage organizational status quo and mediocrity. In contrast, CQI mandates a top-down promulgation of quality and a cultural change for the organization. CQI focuses on processes and systems of care, not individuals. CQI requires the organization to constantly evaluate and revise processes to better meet the needs of customers.

Despite the relative weaknesses of QA and strengths of CQI, QA efforts should not be abandoned entirely. Certain QA activities (e.g., indicator monitoring and evaluation) should remain, although their aim should be changed to focus on patterns of performance instead of individual performance. Additionally, the sensitivity and specificity of indicators can be evaluated and refined so that they can be used to identify substantive issues. Once identified, all of the cases that "fall out" under this indicator can be analyzed to determine if their cause is due to a system failure. A monitoring protocol, which lists all of the indicators to be evaluated in a given year, can help organize the indicator workload by categorizing indicators as either sentinel events (having serious patient outcome consequences and warranting case-by-case review) or rate-based events (not requiring immediate action).

Trauma case management and CQI are also integrally related. Trauma case management is a process of providing clinical oversight to the trauma patient's entire hospital course in order to decrease LOS and cost, while maintaining quality and customer satisfaction. Case management is an effective way to identify and quantify systems and processes that increase LOS, cost, customer dissatisfaction, and lack of quality. Critical paths are also useful in CQI. Critical paths are time lines for certain events during the patient's hospitalization, thus standardizing patient care activities and making them more predictable. Variations from the critical path can be analyzed to identify system impediments to quality and efficiency.

Quality planning can be used to set priorities on where to focus CQI activities. Critical areas for initial work can be identified through CQI's techniques of brainstorming, decision matrixes, and Pareto diagrams. Cause-and-effect (fishbone) diagrams display all the possible relationships between an outcome or effect and its cause; these diagrams are helpful in brainstorming. The flow diagram, a picture of a particular process, increases understanding of how processes work and where breakdowns are occurring. Run charts are used to determine whether variation in processes is due to common causes (normal variability) or special causes (abnormal variability). Statistical process control charts, similar to run charts, also have statistically calculated upper and lower control limits. Where all points falling within the control limits, the system is considered "stable" and not requiring further evaluation; where points fall outside the control limits indicate special cause variations.

[The article concludes with a case study of CQI implementation in trauma care, which is not included in this summary.]

EMS performance evaluation demands an efficient, flexible means of data collection. Traditional peer review and retrospective medical record audits are time-consuming, cumbersome, and inefficient for high-volume EMS systems. In this study, the researchers computerized data from ambulance calls and then compared their ability to detect deficiencies in patient care, data collection and reporting prior to and following introduction of the computerized system.

A system of quality assurance was designed and implemented in the following stages:

Phase 1: Standardization of data collection forms and recording by field teams. This involved redesign and testing of a computer form for recording essential patient care items.

Phase 2: Development of protocols. Protocols were developed by the medical director and associate; written standing orders were implemented for ALS cases.

Phase 3: Computer programming for the system of error detection. Errors were divided into 3 categories--Category 1: documentation errors, defined as absent but retrievable; Category 2: documentation errors that result in a permanent loss of data; and Category 3: documentation errors directly affecting patient care, including deviation from protocols. By arbitrary decision, an equation was developed to determine the acceptable number of errors.

Phase 4: Evaluation of the system; comparison of error detection before and after system implementation.

In addition to the statistical information gathered, a "patient-care profile" is generated, depicting the experience and call loads of each paramedic.

Results showed that detection of patient care and documentation errors increased dramatically with the introduction of the system (5 patient care errors / month before implementation vs. 56 patient care errors / after implementation). The authors felt that this increased was not the result of a sudden change in the performance of field teams in a relatively stable personnel pool, but rather due to better error detection. The number of paramedics "flagged" for review decreased significantly over time. The authors conclude that a system of quality assurance using available technology is important for any modern EMS system.

The team approach is integral to the success of any quality improvement project, as is the organization-wide commitment to QI. Initial QI efforts must show concrete results from in-house projects developed by in-house teams. This is difficult, however, when training, knowledge, oversight and resources are scarce or nonexistent at the start-up phase as they often are.

This paper reports the results of an initial QI project at Massachusetts General Hospital that was intended to improve the process of inpatient transport and to provide senior management with measurable evidence of the merits of a QI approach.

Middle and upper management made presentations to senior management on QI and suggested a pilot project to demonstrate effectiveness. Several years earlier, senior management had worked with QI experts from major corporations to successfully improve the Medicare billing process, but had not subsequently applied QI methods.

Since no official QI governing body existed in the hospital at the time of the project, the pilot project was taken on by members of the hospital community who were sincerely interested in promoting QI within the institution. The project team was unique in that its members included no frontline employees, but instead included many managers, most of whom were self-educated in QI methods.

After an initial planning meeting, a team was formed to analyze and make recommendations to improve the transport process. Team members were provided with just-in-time training in specific QI tools. The team then developed a problem statement; identified the "customers" of the patient transport process; developed a flowchart documenting the current process; brainstormed to develop a cause-effect diagram; identified several root causes of problems; and worked toward developing solutions. Team perceptions regarding opportunities for improvement were corroborated through surveys of nurses, support staff, transporters, dispatchers and patients.

By implementing the QI process, communication improved among all groups involved in the process. After implementation, "dramatic" improvements were seen in the incident reports and patient survey data regarding the transport process. Improvements were also realized in transport staff turnover and patient transport turnaround time.

Additionally, the team learned the following lessons regarding QI. First, members learned to analyze all information and potential solutions before determining which solutions are useful and which actions to take. Second, periodic self-assessment of team progress was crucial to ensure that the project stayed on track, to make adjustments in membership, and to reflect on improvements to date. Third, changing the team membership had positive (fresh perspectives) and negative (extra time needed to train and integrate new members) effects on the project.

Finally, senior management determined that the concept of continuous improvement was consistent with hospital goals; examination of methods to implement QI more broadly were being undertaken.

N.Y. law requires that all hospitals with emergency departments develop a quality assurance program "to include prehospital care providers". This paper reports the development of the quality assurance program at one 242-bed community hospital that receives patients from 64 volunteer and 3 paid ambulance services.

The quality assurance approach that was selected was to conduct patient care record reviews (PCR), initially for completeness of documentation and, subsequently, for appropriateness of care. Prehospital providers were involved in the planning process early and were asked to provide input into the development of the reviews. PCR review was conducted on a company-specific, as well as a collective basis. A minimum of 30% per company were randomly selected for review. Determinations of appropriateness of care were made by comparisons of care with the New York State BLS protocols and the Western Regional EMS intermediate life support and ALS protocols.

Randomly selected PCRs that did not meet the pre-set standards were highlighted and returned to the company. Questions of appropriateness of care were referred to the individual fire chief or first aid captain. Collective and company-specific summaries were distributed. Featured speakers, case reviews, and educational offerings based on the Q.A. findings constituted the remainder of the Q.A. Program.

A total of 49% of all ambulance runs were reviewed during a one-year period. The authors state that measurable improvements in overall documentation have resulted. Areas of concern regarding prehospital patient care were identified, and subsequent remedial education was instituted. Communication between ED and prehospital personnel improved.

The authors conclude that PCR reviews provide valuable information and insight about the overall status of EMS. Specific area of concern can be identified and addressed within a reasonable time frame.

Most EMS systems do not establish necessary mechanisms to evaluate prehospital care. This study reports a semi-annual evaluation and audit program of ALS base hospital performance intended to assess the base hospitals' ability and success in providing ongoing and retrospective medical control to field paramedics.

Audits are conducted at least every six months, but no more frequently than every three months. The audit is conducted by an audit team composed of the system medical director and at least one prehospital care coordinator. Examination includes the following areas:

1. Compliance with the EMS system - base hospital contract
2. Compliance with county ALS policies and procedures
3. Assessment of overall medical control
4. Ensuring of due process if subsequent action taken against hospital
5. Documentation
6. Assessment of record keeping and organization of base hospital
7. Observation of on-line base hospital functioning

The authors report that the audits have uncovered problems with provider agencies, base hospitals, system policies, and individual personnel prior to the development of a crisis situation. The audit also serves as the basis for disciplinary action against a base hospital, as well as a measure of progress over time. The authors conclude that the ALS base hospital audit is the single most valuable tool EMS can use to ensure quality medical control.

This article reports the application of quality improvement concepts to a 300-bed nonprofit community hospital. Increased competition in the hospital's service area provided the impetus for the hospital to undertake quality efforts.

The hospital worked with and followed Crosby's quality improvement approaches. According to this approach, all work is an activity or series of activities that result in an output, or if it is service, an outcome. Every process has a supplier and a customer. Outputs of processes have requirements and quality is conformance to those requirements as agreed upon by the customer and the supplier. These requirements must be met each time through prevention of error and by making the processes that create the output appropriate, capable, and reliable. The cost of quality can be identified: it is composed of the price of conformance (POC), what it costs to do things right, which should not exceed 2.5% of costs; and the price of nonconformance (PONC), or what it costs to do things wrong, which has been estimated at a low of 20% of health care costs and a high of 50% of health care costs.

The hospital's executive officers attended quality management seminars and subsequently formed a quality steering committee to execute the quality process and draft a hospital-wide policy for quality improvement. Committee members met monthly to review and support implementation of the policy. Commitment by top hospital personnel was found to be crucial: their participation invigorated QI efforts, and occasional non-participation slowed progress.

A quality improvement team was established to run the QI program and to promote the transition to the QI culture. The committee requested that each of the 63 hospital departments choose three processes that they believed needed improvement. Problems that were too big or extended beyond the boundaries of the department, and required cross-functional teamwork, were referred to a corrective action coordinator. There, the problems were prioritized and assigned to a team composed of those who were involved in and understood the problem. These special teams were charged with: defining the problem; putting a fix in place if necessary; identifying the root cause; taking corrective action; and monitoring the process.

Hospital employees were informed of QI efforts via posters, bulletin boards, articles in the hospital newsletter and special events. Employee education was provided to ensure that employees shared a common QI language and understood how the process worked.

The authors point to major accomplishments from the cross-functional corrective action teams, e.g., earlier patient discharge and improved surgical scheduling from changes in laboratory testing schedules. The authors estimate that the initial PONC of $8.5 million has been reduced by $2.1 million in less than two years.

1. "Uniform Prehospital Data Elements and Definitions: A Report from the NHTSA Uniform Prehospital Emergency Medical Services Data Conference." D Spaite, R Benoit, D Brown, R Cales, D Dawson, C Glass, C Kaufmann, D Pollock, S Ryan, E Yano. Annals of Emergency Medicine, 25(4), April 1995, pp. 525-534.

2. Campbell, A.B. (1993). Strategic planning in health care: Methods and application. Quality Management in Health Care, 1(4), 12-23.

3. Ibid.

4. ⁰ The operation of EMS systems depends on a high quality, low cost supplier network for equipment, information, and services. For example, the timely provision of first response defibrillation, or dispatch services (e.g., police dispatchers) are important components of the system's patient care, but may be performed by agencies with a different organizational structure, lines of authority and competing priorities than the EMS agency. In industries other than EMS, companies are increasingly demanding, as a pre-condition to contracting for services, data from supplier's on their QI activities. Thus, in addition to assessing the outcome of the system's QI activities, the EMS system must also be knowledgeable about the QI activities of their suppliers. Forging a cooperative relationship with suppliers should be a priority, with the goal of examination and improvement of results. Additionally, many new technologies, devices and procedures are introduced into the EMS market based solely on evaluations conducted by the manufacturer. EMS systems need to be more involved in the evaluation and implementation of new technologies to help insure that they meet the needs of their systems, patients and stakeholders.

5. As noted previously, within this manual, the term "stakeholder" includes, but is not limited to, hospitals, physicians, health insurance organizations, health maintenance organizations, local, state and federal government agencies, elected officials, disaster planning groups, and community civic groups.