[House Hearing, 109 Congress]
[From the U.S. Government Publishing Office]
OVERSIGHT HEARING ON THE
TRAINING PROVIDED TO VBA CLAIMS
ADJUDICATORS AND THE STANDARDS
USED TO MEASURE THEIR PROFICIENCY
AND PERFORMANCE
=======================================================================
HEARING
BEFORE THE
COMMITTEE ON VETERANS' AFFAIRS
HOUSE OF REPRESENTATIVES
SUBCOMMITTEE ON DISABILITY ASSISTANCE
AND MEMORIAL AFFAIRS
ONE HUNDRED NINTH CONGRESS
SECOND SESSION
_____________
SEPTEMBER 13, 2006
_____________
Printed for the use of the Committee on Veterans' Affairs
Serial No. 109-62
_____________
U.S. GOVERNMENT PRINTING OFFICE
30-381 PDF WASHINGTON : 2007
---------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government
Printing Office Internet: bookstore.gpo.gov Phone: toll free (866)
512-1800; DC area (202) 512-1800 Fax: (202)512-2250 Mail: Stop SSOP,
Washington, DC 20402-0001
COMMITTEE ON VETERANS' AFFAIRS
STEVE BUYER, Indiana, Chairman
MICHAEL BILIRAKIS, Florida LANE EVANS, Illinois, Ranking
TERRY EVERETT, Alabama BOB FILNER, California
CLIFF STEARNS, Florida LUIS V. GUTIERREZ, Illinois
DAN BURTON, Indiana CORRINE BROWN, Florida
JERRY MORAN, KANSAS VIC SNYDER, Arkansas
RICHARD H. BAKER, Louisiana MICHAEL H. MICHAUD, Maine
HENRY E. BROWN, Jr., South Carolina STEPHANIE HERSETH, South
JEFF MILLER, Florida Dakota
JOHN BOOZMAN, Arkansas TED STRICKLAND, Ohio
JEB BRADLEY, New Hampshire DARLENE HOOLEY, Oregon
GINNY BROWN-WAITE, Florida SILVESTRE REYES, Texas
MICHAEL R. TURNER, Ohio SHELLEY BERKLEY, Nevada
JOHN CAMPBELL, California TOM UDALL, New Mexico
BRIAN P. BILBRY, California JOHN T. SALAZAR, Colorado
JAMES M. LARIVIERE, Staff Director
SUBCOMMITTEE ON DISABILITY ASSISTANCE
AND MEMORIAL AFFAIRS
JEFF MILLER, Florida, Chairman SHELLEY BERKLEY, Nevada,
JERRY MORAN, Kansas Ranking
JEB BRADLEY, New Hampshire, Vice Chairman TOM UDALL, New Mexico
GINNY BROWN-WAITE, Florida LANE EVANS, Illlinois
PAIGE MCMANUS, Subcommittee Staff Director
C O N T E N T S
September 13, 2006
Page
Oversight Hearing on the Training Provided to VBA Claims
Adjudicators and the Standards Used to Measure their Profi-
ciency and Performance .................................... 1
OPENING STATEMENTS
Chairman Miller ............................................ 1
Hon. Shelley Berkley, Ranking Democratic Member ............ 2
Prepared statement of Ms. Berkley .......................... 22
STATEMENTS FOR THE RECORD
Hon. Ginney Brown-Waite .................................... 23
WITNESSES
Walcoff, Michael, Associate Deputy Under Secretary for Field
Operations, Veterans Benefits Administration, U.S. Depart-
ment of Veterans Affairs .................................. 3
Prepared statement of Mr. Walcoff ..........................
Wise, Lauress L., Ph.D., President and Chief Executive Officer,
Human Resources Research Organization ..................... 11
Prepared statement of Dr. Wise ............................. 36
Cox, J. David, R.N., National Secretary-Treasurer, American
Federation of Government Employees, AFL-CIO ............... 13
Prepared statement of Mr. Cox .............................. 45
Smithson, Steve, Deputy Director for Claims Services, Veter-
ans Affairs and Rehabilitation Division American Legion ... 15
Prepared statement of Mr. Smithson ......................... 56
OVERSIGHT HEARING ON THE TRAINING
PROVIDED TO VBA CLAIMS ADJUDICATORS AND
THE STANDARDS USED TO MEASURE THEIR
PROFICIENCY AND PERFORMANCE
_______________
Wednesday, September 13, 2006
U.S. House of Representatives,
Subcommittee on Disability Assistance and
Memorial Affairs,
Committee on Veterans' Affairs,
Washington, D.C.
The Subcommittee met, pursuant to call, at 2:30 p.m., in Room 334,
Cannon House Office Building, Hon. Jeff Miller [Chairman of the
Subcommittee] presiding.
Present: Representatives Miller, Berkley.
Staff Present: Paige McManus, Majority Counsel and Mary Ellen
McCarthy, Minority Counsel.
Mr. Miller. Good afternoon, everybody. Our Ranking Member is here, so
we will bring this meeting to order.
As we all know, the number of veterans filing claims for compensation
has increased every year since 2000, and the claims that are being
filed are becoming much more complex. As we are all very well aware,
many veterans are upset with the time that it takes to adjudicate a
claim for compensation benefits. As of July, the most recent data
available, it is taking the Veterans Benefits Administration an average
of 174 days to adjudicate a claim, with an accuracy rate of 88 percent.
The ability to provide timely and accurate benefits is dependent not
solely on increasing staffing levels, but providing proper and thorough
training.
In recognition of this, the Committee recommended an additional 400,000
above the administration's request in its fiscal year 2007 views and
estimates to fund training and certification initiatives. Congress and
certainly VA must ensure that current and new employees have the skills
and knowledge necessary to render accurate and fair decisions the first
time. VBA has a number of programs and tools that it utilizes to train
new Veteran Services Representatives, and to retrain or refresh more
senior VSRs and Rating Veteran Service Representatives.
The VBA has implemented a skills certification test to assess the
knowledge base of claims adjudicators, and to provide additional
training when necessary. However, it is my understanding that a great
many employees either don't have the time needed to devote to training,
or are failing basic competency tests.
We meet today to receive testimony on the types of training provided to
claims examiners, the standards that are being used to measure
proficiency and performance, and just what VBA is doing to enhance
performance.
I want to take this opportunity to recognize our Ranking Member,
Ms. Berkley, for comments.
Ms. Berkley. I thank you very much, Mr. Chairman, for holding this
hearing, and welcome to all of you.
VA employees who decide critical benefits issues for our nation's
veterans must obviously have proper training in order to fulfill the
VA's mission. I am very pleased that we are giving attention to this
matter today. I am concerned that the majority of experienced VSRs who
have taken open-book, open-resource tests were not successful in
passing these tests. I am also concerned that files reviewed by
Committee staff, and claims brought to the attention of the Committee,
often contain obvious errors, which VA managers quickly agree to
correct.
According to Mr. Walcoff's testimony, the VA provides national
comprehensive training to regional office staff. However, both the
employees union, American Federation of Government Employees, and the
American Legion, report wide regional variations in training. Having a
national plan does not guarantee that a national plan will be
effectively implemented. The union testimony concerning VBA excluding
labor representatives from meetings about training and performance
standards is, quite frankly, very disturbing. Labor and management need
to work together to improve the training provided to VA employees.
Veterans who have honorably served our nation deserve quality decisions
in a timely manner in order to receive the benefits and services they
deserve. I am anxious to hear your testimony, both panels, and I thank
all of you for being here today, and look forward to hearing what you
have to say.
Mr. Miller. Thank you very much, Ms. Berkley.
Our first panel is already seated at the table. Other members will be
coming and going as we have other hearings going on this afternoon.
Testifying on behalf of the Veterans Benefits Administration is Mr.
Michael Walcoff, Associate Deputy Under Secretary for Field Operations.
He is accompanied by Mr. James Whitson, who is Director of the Eastern
Area. Ms. Dorothy MacKay is here today, she is the Director of the
Office of Employee Development and Training, along with Ms. Janice
Jacobs, Deputy Director of Policy and Procedures at the Compensation
and Pension Service.
Mr. Walcoff, you are recognized.
Mr. Walcoff. Thank you, Mr. Chairman, and members of the Subcommittee.
I appreciate the --
Mr. Miller. If you could pull that mic just a little bit closer.
Mr. Walcoff. Sure.
STATEMENT OF MICHAEL WALCOFF, ASSOCIATE DEPUTY UNDER SECRETARY FOR
FIELD OPERATIONS, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF
VETERANS AFFAIRS; ACCOMPANIED BY JAMES A. WHITSON, DIRECTOR, EASTERN
AREA VETERANS BENEFITS ADMINISTRATION; DOROTHY MACKAY, DIRECTOR, OFFICE
OF EMPLOYEE DEVELOPMENT AND TRAINING, VETERANS BENEFITS ADMINISTRATION;
JANICE JACOBS, DEPUTY DIRECTOR, POLICY AND PROCEDURES, COMPENSATION AND
PENSION SERVICE
Mr. Walcoff. I appreciate the opportunity to appear before you today
to address training for Veterans Benefits Administration claims
adjudicators, the standards we use to measure their proficiency and
performance, and how we communicate changes in laws, regulations,
policy, and procedures, to field station staff.
I am being accompanied by my VBA colleagues; Mr. Jim Whitson, who is
the director of our eastern area office and also the Project manager
for C&P certification; Ms. Janice Jacobs, deputy director of
compensation and pension service; and Dorothy MacKay, director of our
office of employee development and training.
Training is essential to every quality organization. VBA is committed
to ensuring all employees have the opportunity to learn and develop
essential knowledge, skills, and abilities. We recognize effective
training as a core element of VBA's infrastructure, and key to our
succession planning efforts, as we hire to replace the many experienced
employees expected to retire over the next few years. VBA has deployed
new training tools and programs to provide consistency in training.
Newly hired veteran service representatives, or VSRs, and newly
appointed rating VSRs participate in consistent, nationally-developed
centralized training, followed by a national standardized training
curriculum at their home offices.
This fiscal year, the undersecretary for benefits, Admiral Daniel L.
Cooper, mandated all claims examiners to have at least 70 hours of
job-specific training. Most other employees must have at least 40
hours of training. In fiscal year 2007, the mandatory training for
claims
adjudicators will increase to 80 hours.
VBA ensures regional offices have timely, accurate, and current
information. C&P service issues guidance letters, called `` fast
letters,'' to advise field employees on policy and procedural changes,
and legal revisions. C&P service issued 23 fast letters in 2005, and
16 to date in 2006.
Decision assessment documents, or DADs, explain the facts, reasoning,
and holding of precedential court decisions, and the opinions of the
office of General Counsel, and explain any impact on VBA. Notification
of fast letters, DADs, and other changes, are e-mailed to field
employees and posted on the C&P service's website.
C&P service uses satellite broadcasts to give in-depth analysis on VBA
procedures and policies by subject matter experts, and hosts monthly
conference calls to discuss current hot topics and give instant
feedback on questions and concerns.
Regarding VBA, VSR, and RVSR national performance standards, a VBA
organizational cornerstone to improve benefit delivery and enhance
accountability is our system of individual performance assessment. All
VSRs and RVSRs are subject to national performance standards that focus
on key elements of quality, productivity, customer service, and
timeliness. Local accuracy reviews are conducted using national
quality review criteria. Agreements were reached with the American
Federation of Government Employees prior to implementation. The
current VSR performance standards have been in effect since October of
2005. The RVSR performance standards have been in effect since
November of 2001. The VBA work group is reviewing these standards for
possible revision.
Regarding certification testing; in conjunction with our national
performance standards, VBA developed a skill certification testing
process to assess job proficiency. A memorandum of understanding was
signed in January 2000 with AFGE for a certification program for VSRs,
RVSRs and decision review officer positions.
In December 2000, VBA contracted with the research group, the Human
Resources Research Organization, or HUMRO, to help develop a
certification instrument with VBA subject matter experts to assess
knowledge and readiness of VSR GS 10 incumbents for promotion to the
GS 11 level.
A 100 question multiple choice open-book test is administered in
morning and afternoon sessions. In August of 2003, 25 percent of the
298 participants passed the first validation test. In April of 2004,
29 percent of the 650 participants passed the test. After the first
two tests, a mandatory 20-hour VSR readiness training curriculum was
implemented.
On May 3rd, 2006, 934 VSRs took the third test. During the afternoon
portion, multiple test items were duplicated from the morning portion.
The correct version of the test was given on June 7th, and the third
test yielded an improved pass rate of 42 percent. To date, VBA has
promoted 633 VSRs to the full-performance GS 11 level, through
certification testing. We are expanding certification testing next, to
the RVSR position, from the GS 11 level to the GS 12 journeyman level.
Mr. Chairman, this concludes my testimony. I greatly appreciate being
here today, and look forward to answering your questions.
[The statement of Michael Walcoff appears on p. ]
**********INSERT**********
Mr. Miller. Thank you very much for your testimony. I think we both
have some questions, and one of the issues I think that we probably
both want to know about is when a case is reprimanded from the Board
for further development, it is generally sent to the Appeals Management
Center in Washington, DC. Is the employee who made the initial
decision - first question - is the employee ever notified of what the
deficiencies were, or what may have happened in the appeal? You know,
I would think that if they are not, then it probably would help the
R.O. directors develop a better understanding of how ratings happen,
and where the issues are. Can you go into that just a little bit?
Mr. Walcoff. Sure. Let me first say that when a case is remanded from
the board, it is very often three, sometimes four years after it was
originally worked, for a lot of different reasons. Very often, the
case that is remanded back to VBA doesn't look like it did when it left
the regional office. Additional information has been sent in by the
veteran. Many, many cases are requiring a new exam, and that is a
reason why a case might be remanded. Sometimes, the rating schedule
may change. There might be a change in regulation that might cause for
the case to be remanded. And certainly, in a small percentage of
cases, a percentage of cases, it is similar to when the VSR finished
it, or the rating specialist finished it, and it is because of an error
that was made at that time. But it is a combination of all those
factors that go into the remand rate.
When we originally set up the AMC, our discussions with the prior
secretary dealt with how many of the remands that came out of the board
should be done at the AMC, and how many should be returned to regional
offices. It was Secretary Principi's opinion that he wanted all the
remands to be processed at the AMC. He felt that this was the most
efficient way to work through the remands. He was concerned about the
fact that there were a lot of older remands that were still out there,
and he felt that this would be the best way to deal with that.
We had some very serious discussions with him about why we felt that,
at minimum, a certain percentage of these cases needed to go back to
regional offices. One of the reasons, quite frankly, was that in
people working these cases, and they have the sense that `` no matter
what I put on this piece of paper, I will never see the case again,''
that there is almost a little bit of a disincentive in terms of being
much more careful about what they are sending to the board, and making
sure that it is ready for the case to be adjudicated by BVA. And we
were concerned about that. We wanted to make it so that some of these
cases do go back to regional offices.
But more importantly, we felt that it was important that some of these
cases go back to R.O.'s for training purposes. It is very important for
people who are working on certifying cases to the board, that they get
to see what types of cases are being remanded, so they can use that
information in doing the work that they are currently doing. So it
doesn't necessary to go back to the original person that prepared the
case, because as I say, it is many years later.
But they do see a good number of the cases. And they particularly see
what we call the egregious cases, the ones where there is really
absolutely no excuse for why this case ever left the regional office.
And we want to make sure that those cases are reviewed by the regional
offices, and by the management at those offices to make sure that it
doesn't happen again.
Mr. Miller. In its 2005 report on state variances in compensation
payments, the VA Inspector General surveyed about a thousand employees
on factors that affect consistency of rating decisions. The results
showed that while 60 percent of the employees had positive opinions
about the quality of the training that they were receiving, many feel
that the training is not a high priority, and some 37 percent or so
indicated that formal classroom training occurs once a quarter, or less
often than that. Can you talk to us a little bit about what is being
done?
Mr. Walcoff. I would say that the Under Secretary, Admiral Cooper, has
I think probably stressed training, training and consistency, probably
more than any other subject since I have been working with him. I have
been in Washington since 1998, and I have served under several under
secretaries, and was here previously, back in the early '90s with two
other under secretaries. I have never worked for anybody that was, I
believe, as serious about training as Admiral Cooper is, and I think it
is because of his background with all the years he spent in the Navy.
I will tell you that it is one thing to say that training is important,
and `` here is a mandatory requirement that you do 70 hours.'' But the
real question is, how do you ensure that is actually happening? And I
will tell you a couple different ways that I think we are going about
doing that. One is I think we are providing very close oversight when
we go to regional offices, to make sure that training is being done
and is being done effectively. The C&P service review goes to about 18
offices a year, and to review the C&P operations at each office. And
one of the main things that they look at is the training that is being
done; whether the curriculum that is specified by the C&P service is
being followed, and whether all employees are going to the training.
In addition to that, our area directors visit their offices two, three,
four times a year. And one of the things that they look at is the same
thing: how is the training being done? And they will not just talk to
the managers. They will actually go and talk to the VSRs, and talk to
the rating specialists, and ask them, you know, `` How much training
have you received,'' you know, `` Did you actually attend this class?
Were you at the satellite broadcast?'' Those types of things, to make
sure that it is happening.
Secondly, we have a log that has been put together by Dorothy's staff
that requires the managers to report every hour of training that is
given to employees, so that we can verify that 70 hours of training
this year, in this case, was actually given to every employee in
regional offices in the service center. So we also review the logs.
And the third thing that we do, and this is something that I haven't
seen in all the years I have been working, is that Admiral Cooper
personally reviews the training report that is done twice a year by
each regional office, that goes into great detail about all the
training that they have done during that previous six months. And this
is not something that comes in and he has Dorothy give him a summary
report. He actually has to see the reports. He reads every one of
them, marks them all up with comments, and sends them out back to her
and to me, you know, with questions, and wanting us to follow up on
things, obviously showing that he is sincerely interested in it.
So I think that aside from having the requirement, there is a lot of
follow-up to make sure that it is actually being done. This is a new
requirement. I mean, this is the first year that we actually had that
mandatory 70 hours. And I think it was a little bit of a shock to our
organization, frankly, that somebody did more than just say, `` Yeah, I
think training is important,'' but actually put out a mandatory
requirement, you know, saying that `` You will do training.'' That has
had an impact on the organization.
Mr. Miller. Ms. Berkley?
Ms. Berkley. Thank you, Mr. Chairman. In reviewing panel two's
testimony, a couple of things struck me, and I would like to ask you
questions based on the testimony that was submitted by the other panel.
According to AFGE's written testimony, it indicated that unrealistic
performance standards and inconsistent training adversely impact the
quality of employees' work. They report also being excluded from the
task force meetings where current training and performance standards
are discussed. Can you share with us why we would exclude these key
stakeholders, and why they are not at the table when discussions of
this magnitude are being -- when they are discussing these issues?
Mr. Walcoff. Well, first of all, I would tell you that I think we have
a very positive relationship with AFGE. The speaker that will be
representing AFGE here in the second panel is somebody that I have
worked personally with for many years, and I think that while we may
disagree on individual issues, I think he would agree that the VBA and
AFGE have had a good relationship.
I think that we certainly accept that there is an appropriate role for
labor, for AFGE. We certainly understand all the things that we are
responsible to do in terms of our relationship with them, and I think
that we very clearly meet every requirement that we have in terms of
our relationship.
We do include them on many of these types of issues that we work on.
The RVSR performance standards, for instance, that are in place right
now, the union had a representative on that team. They participated in
setting up those standards. That is aside from the fact that we then
negotiated an agreement with them about the standards that are in
question right now. The VSR standards, which is I think what David is
referring to in his statement, we negotiated an agreement with them on
the implementation of those standards.
And the disagreement that we have right now with them that he has
mentioned in his testimony has to do with a requirement in the MOU for
us to review the actual performance under those standards, and make
changes, and there is a difference of opinion as to whether we have
done that or not. But we certainly have worked with the union on the
implementation of those standards, and met every requirement that is
there for our labor-management relationship.
In terms of the standards being too difficult, I respectfully disagree.
I think that some of our managers feel that we should have set the
floors higher. We make it clear that it is not just a productivity
requirement; that there is an absolute quality requirements in there,
as well as a timeliness and a customer-service requirement. They are
all considered critical. And if you look at the performance of the
field in those standards, right now we have a pass rate of over -- in
terms of meeting all the requirements of their position -- of over 80
percent within the VSR position. And that goes up every month, as
people become more familiar with the changes that we made almost a year
ago in the way we measure those standards.
Some of the problems that we have had had to do with the tracking
system that is designed to go with those standards, called Aspen.
There has been -- it is a new system, and it is taking quite a while
for some of our employees, as well as managers, to understand how to
use it. And as we reviewed the pass rates in preparation for the
grievance that is mentioned in David's testimony, we found that there
were some R.O.'s that weren't using the tracking system correctly, and
that was why some of the pass rates look lower than they actually are.
But we are very confident that the pass rate is over 80 percent, and
going up every month.
Ms. Berkley. Do you think there is inconsistent training? And does it
adversely impact on the quality of the employees' work?
Mr. Walcoff. You know, inconsistent training is something that we are
always very concerned about, and we have done everything, I think, that
we can to try to deal with that. That is why we have not only -- not
only do we have a national curriculum, starting with the basic
challenge training for all of our new employees, where they go to a
centralized place and spend three weeks there, and then when they go
back to their regional offices, spend 23 weeks taking a national TPSS
curriculum that everybody is required to take.
But for journeyman VSRs and rating specialists, we get the 70-hour
requirement, where C&P service has listed a number of courses that are
mandatory for every individual during that year, and also specifies
`` This is the way you will teach it. Here is a satellite broadcast,''
you know, `` Here is a video,'' that type of a thing. So certainly,
mechanisms are out there to ensure that there is consistent training.
We track the logs to make sure that everybody is doing what they are
supposed to do consistently. You know, in an individual --
Ms. Berkley. There is adequate oversight, in your opinion?
Mr. Walcoff. I believe there is. I believe there is, ma'am.
Ms. Berkley. Let me draw your attention to the American Legion
testimony. In it, they have indicated that the focus on end-product
production quotas in regional offices leads to poor-quality decisions
and appeals. When and how did the VA last evaluate the usefulness of
end-product code measure of performance? And is there perhaps a better
method of measurement?
Mr. Walcoff. I think that everybody, or just about everybody, would
agree that we would like to evolve away from the basic end-product
system, to more of an issues-oriented system, and I have been to
hearings where this has been discussed. And I think you know that up
until about a year ago, we really didn't have the ability to accurately
measure the work that we are doing or receiving on an issues-based
basis. With RBA 2000 now being mandatory, and everybody using it, that
is really a vehicle that really does allow us to be able to count
issues versus cases.
And that is really the first big step toward moving away from an
end-product system. It is certainly a more accurate way to measure
quality. It is a more accurate way to measure our production. And it
would do a lot in terms of some of the concerns even that AFGE has
about how we measure how productive, you know, an individual is. We
have talked a lot about it.
There are some initial steps taking place for us to begin doing that.
It is going to be a process. I mean, it is certainly something that
were going to have to work with, with all of you all on, in terms of
explaining what we are doing, why we are doing it, and how that will
affect our performance, because we want to be able to give you apples
and apples to be able to compare in terms of whether we are improving
or not. Certainly we would work with service organizations, and with
the union, as we migrate from the one system to another. It is not
something that is going to be done over a period of months. But we
have had discussions with Admiral Cooper, and I know that he is
committed that this is something that were going to want to do.
Ms. Berkley. Is there anything that we can do to help implement that,
in getting away from the end-product system? Because I agree with you;
it won't take months. But how do we, as members of this Committee,
enable you to do this and not be here 20 years from now talking about
the same thing?
Mr. Walcoff. One of the things that I think will be helpful is, you
know, any time we make a major change in the way we measure something,
you know, I think that there is always a concern -- and as I say, I
have been around in headquarters for a while, and I know that sometimes
there is a concern, `` Well, they are kind of changing the name of the
game a little bit to maybe just take away attention from what actual
performance is, and that sort of thing.'' And I think certainly that
is a legitimate concern that our stakeholders certainly should be
thinking about.
What I would want to do in terms of working with you is to try to sit
down and talk about, `` Okay, what is it that you would need for
reassurance as we evolve from one system to another?'' So that you
could see not only that we are moving, you know, in a direction that we
both think we need to, but you can also monitor on us in your oversight
capacity as to how we are performing, and be able to do it on some
consistent plane, rather than in two different languages, sort of, you
know, so to speak.
And that is something we will be able to hopefully work with you on,
and you know, I think we can come to some agreement on that.
Ms. Berkley. Okay. Thank you, Mr. Walcoff.
Mr. Miller. Thank you very much. We appreciate your testimony. Thank
you for those that joined you today. If you would, if you could have a
member, you or a member of your staff, if they could remain behind we
would appreciate it, for the next panel. Thank you very much.
Mr. Walcoff. Okay, thank you.
Mr. Miller. And if the second panel will make their way forward.
While you are getting settled I am going to go ahead and introduce
everybody if it is okay.
Dr. Lauress Wise is president of the Human Resources Research
Organization, the group VBA contracted with to develop the skills
certification testing program. Mr. Jeffrey David Cox is the national
secretary-treasurer of the American Federation of Government Employees.
And Mr. Steve Smithson, he is with us, is the deputy director for
claims services at the Veterans Affairs and Rehabilitation Division of
The American Legion.
And each of you we would ask, if you would, hold your testimony to five
minutes, since there are three of you. Your full testimony will be
entered into the record. It will be printed in its entirety, as it was
submitted.
Dr. Wise, welcome, and you may begin.
STATEMENTS OF LAURESS L. WISE, PH.D., PRESIDENT AND CHIEF EXECUTIVE
OFFICER, HUMAN RESOURCES RESEARCH ORGANIZATION; J. DAVID COX, R.N.,
NATIONAL SECRETARY-TREASURER, AMERICAN FEDERATION OF GOVERNMENT
EMPLOYEES, AFL-CIO; STEVE SMITHSON, DEPUTY DIRECTOR FOR CLAIMS
SERVICES, VETERANS AFFAIRS AND REHABILITATION DIVISION, AMERICAN LEGION
STATEMENTS OF DR. LAURESS WISE
Mr. Wise. Thank you. Good afternoon, Mr. Chairman. I am Lauress Wise,
president of the Human Resources Research Organization, known less
formally as HUMRO. HUMRO is a nonprofit 501(c)3 research and
development organization established in 1951 that worked with
government agencies and other organizations to improve their
effectiveness in developing and managing their human resources.
I have been asked to testify today about the work that HUMRO has done
for the Veterans Benefits Administration on their program for
certifying essential skills for veteran service representatives. For
brevity, veteran service representatives are referred to as VSRs.
These service representatives play a key role in seeing that our
veterans receive a full array of benefits to which they are entitled.
Performance at the highest level of this position requires a thorough
understanding of an extensive set of policies and procedures, and skill
in identifying appropriate applications of these procedures to
individual circumstances.
The skill certification program embarked on by the VBA is critical to
ensuring that service representatives have the knowledge and abilities
needed to perform their jobs effectively.
My written testimony, which I ask be entered into the record, describes
in some detail HUMRO's efforts to develop the VSR skill certification
test. I will just cover briefly the essential points here. Our work
began in 2001 with an analysis of critical tasks performed by the VSRs
and the knowledge and skills required to perform these tasks
effectively. HUMRO worked with a design team that included VBA
management, expert job incumbents, union representatives, and members
of the veteran service organizations, to review the job analysis
results, and develop a blueprint for the certification test.
The blueprint identified content areas to be covered by the test and
specified the number of test questions to be included for each of these
content areas. We worked with the design team to develop, review, and
provide high-quality test questions. A pilot test was conducted to
screen individual questions followed by a field test of the entire
certification process.
Two different approaches were taken to accessing the validity of the
resulting test scores for certifying a candidate's qualification for
the GS 11 position. In a content validity study, experts affirmed that
each of the test questions measured knowledge and skills essential for
job performance. We also conducted a criterion-related validity study.
In that study, test scores were found to be significantly correlated
with job performance ratings made by each examinee's supervisors. The
test development and validation process conformed fully to professional
standards for test development and use. The resulting certification
test is a valid and important tool for ensuring essential skills for
those promoted to the highest level VSR job.
I was also asked to comment on the issues surrounding the May 3rd
administration of the skill certification test. Two forms of the test
ere administered to 934 candidates. The reason for administering two
forms was so that examinees sitting next to each other would not get
the same questions in the same order. Each test form included 100
scored items and 20 additional questions being pilot tested for future
use. The two forms had 67 operational, or scored, items in common,
albeit in different locations within each test form.
Because of the length of the test, the test is split across two
sessions; one in the morning and one in the afternoon. Candidates
received a different test booklet for each of these two sessions. Due
to a processing error at HUMRO, some of the questions for the afternoon
booklets were inserted into the incorrect test form. This error
resulted in duplicating some questions from the morning session in the
afternoon session booklets for that corresponding test form. Quality
control procedures in effect at the time included a review of each test
but did not include a thorough comparison of the morning and afternoon
booklets within a test form. Consequently, this processing error was
not caught prior to the test administration.
The week following the May administration, HUMRO Vice President Beverly
Dugan, our project director, Dr. Patricia Keenan, and I met with VBA
leadership to discuss the problem and identify methods for providing
valid scores to the May administration examinees. Our discussion
identified several possibilities, including using some of the pilot
test items to construct an 80-item test; ignoring the redundancy and
scoring each of the duplicate items to provide a 100-item test; and
conducting a supplemental administration, using the items that were
originally intended to be included in each of the afternoon booklets.
The solution selected by the VBA management was to conduct a
supplemental test, and administer the items that were originally
intended to be presented in each of the afternoon tests. This allowed
everyone to be scored on 100 separate items, to keep the test mapped to
the blueprint exactly as designed, and made the May 2006 administration
much more equivalent to the operational field test and validity test,
and to those administrations planned for the future.
The supplemental test administration was held on June 7th, 2006. After
both portions of the test were scored, 42 percent of those who took the
test passed. The supplemental testing did create an inconvenience to
examinees, and additional burden to those who administered the test,
but the end result was an assessment that covered the content framework
as intended, with questions and scores that were psychometrically
sound.
HUMRO's staff have reviewed the factors that led to the error in
assembling the test booklets. One such factor was the limited time
available for assembling and checking booklets. The VSR job continues
to evolve. New types of cases are often added to the caseload. Newly
electronic tools and databases are developed, and more pension cases
are being moved to pension maintenance centers.
A workshop to review test questions, to be sure that they reflected
current policy and processes, was held in April of 2006. Item writers
reviewed all of the items in the bank, revised many of them, and
updated the references. The revisions were more extensive than
originally anticipated, and the work to update the item bank was
completed late the week prior. We had only two days to select the
forms. In retrospect, it was clear that more time is needed.
We have now implemented procedures to prevent future occurrence of
these problems, both by allowing more time, and incorporating more
thorough checks of the test booklets.
In summary, the VSR test is an important tool for improving the
effectiveness of the VSR workshop, and serving the benefit needs of our
veterans.
Mr. Chairman, that concludes my testimony. Thank you for the
opportunity to appear today.
[The statement of Lauress Wise appears on p. ]
**********INSERT**********
Mr. Miller. Doctor, thank you very much for your testimony. What I
would like to do if we could before we go to questions is let the other
panelists go ahead with their remarks.
Mr. Cox, you are recognized.
STATEMENT OF J. DAVID COX
Mr. Cox. Thank you, Mr. Chairman.
Chairman and members of the Subcommittee, good afternoon. I am J.
David Cox, national secretary-treasurer of the American Federation of
Government Employees. Thank you for inviting AFGE to testify today.
I ask that my written statement be submitted for the record.
Mr. Miller. Without objection.
Mr. Cox. As the purse Vice President of AFGE's national VA council and
a local president, I have visited a large number of VBA regional
offices. What I observed from the front lines is that VSRs and rating
specialists are under tremendous pressure to process enough cases to
meet performance standards. They stay late into the evening and work
weekends. They study on their own time because training is often cut
short to make the numbers.
Despite all this, many of them cannot make the numbers. They are
further demoralized by a confusing certification test. VSRs and rating
specialists want to help veterans get an accurate and prompt decision
on their claims. Many are themselves service-connected veterans. VBA
training is clearly an investment worth making. AFGE believes that
genuine collaboration on training and performance measures will make
this critical investment go far. Lawmakers, managers, employee
representatives, and veterans groups, have to work together to come up
with training and standards that are workable, doable, and durable.
Unfortunately, VBA seems less and less open to collaboration than in
the past. After we have worked together to develop the CPI model, we
have now been left out of key task force meetings. After we worked
together on national performance standards, they have refused to
address problems that have been developed. Even though we developed
the VSR certification by agreement, VBA is refusing to work with us to
fix what isn't working. The role of on-the-job training for VSRs and
rating specialists cannot be underestimated.
By VBA's own estimates, it takes at least two to three years for a new
VSR to be fully productive. Yet, what VBA is doing about an impending
shortage of experienced adjudicators due to retirement and high
turnover among frustrated new employees.
AFGE offers the following recommendations: a joint VBA stakeholder team
should develop a national mandatory training plan, with strong
oversight. We need a uniform curriculum, training schedule, and a set
of best practices. The oversight process should be transparent to all
stakeholders, with regular reports to Congress to make sure that VSRs
and rating specialists everywhere get the same quality and amount of
training, and rotate through all adjudication teams.
VBA should develop a cadre of trainers skilled in both teaching and
veterans benefits. Currently, trainers are selected without consistent
criteria. All trainers should be of the caliber one is likely to find
at the VBA Academy. Continuing education should be mandatory and
nationally uniform. Supervisors should be required to hold weekly
meetings to go over new laws, regulations, court cases, and best
practices.
Current performance standards need revision. One third of VSRs and
rating specialists are failing to meet unrealistic production quotas.
CPI specialization should also be factored in. Training for skills and
certification tests should be nationally uniform. The first two
validity tests for VSR certifications had a very low passage rate, and
more than half failed the May 2006 test. I can't help but wonder
whether any managers have ever had to take this test, and what their
pass rate would be.
The test is supposed to be a culmination of the training, but there is
a real disconnect between the two. Trainers do not even know what to
cover. Employees are not told how many questions they need to get
right, which questions they got wrong, and what the correct answer is.
Something is wrong when two employees with identical scores test at
different times, and only one passes.
Finally, we hope the Subcommittee will consider the related issue of
classification of the VSR and rating specialist positions. Unlike
comparable adjudication jobs at other federal agencies, the VSR career
ladder ends at the GS 10. We currently understand that VBA recently
completed a review of these classifications. Once again, we would like
to be part of this process. We look forward to working with Chairman
Miller and Ranking Member Berkeley, to ensure that VBA claimants get
the best services possible. Our veterans deserve nothing less.
Thank you, Mr. Chairman, for allowing me to be here today.
[The statement of J. David Cox appears on p. ]
**********INSERT**********
Mr. Miller. Thank you, Mr. Cox. Mr. Smithson, you are recognized.
STATEMENT OF STEVE SMITHSON
Mr. Smithson. Mr. Chairman and members of the Subcommittee, thank you
for the opportunity to present the American Legion's views on the
training provided to Veterans Benefits Administration claims
adjudicators and the standards used to measure their proficiency and
performance. We commend the Subcommittee for holding this hearing to
discuss these important issues. I am going to limit my remarks to
emphasize major points discussed in my written testimony, and ask that
my written statement be entered into the record.
Mr. Miller. Without objection.
Mr. Smithson. The American Legion appreciates the importance the
Undersecretary for Benefits has placed on training, and the
improvements that have taken place, including centralized training for
regional office staff conducted by the compensation and pension
service. However, in our opinion, the culture of VA's claims
adjudication has been and continues to be production-driven. This
results in everything else, including quality assurance and training,
taking a back seat. If this culture is ever going to change, VA, in
addition to the centralized training currently taking place, must
consistently implement national training requirement or standard in
each regional office for all employees, new hires and experienced
alike.
It must also be implemented with strong oversight directly from the
Under Secretary for Benefits' office, and it must have teeth.
Management stations not in compliance must be held accountable, or the
effort will not be successful.
Additionally, there must be an emphasis placed on using information
obtained from BVA decisions, DRO decisions, and errors noted in the
National Star reviews, and other internal quality reviews. Such data
should be tracked, examined for patterns, and used in formal,
customized training at each R.O.
Although such data is being collected and the American Legion's quality
review visits at VA regional offices have identified some stations that
have been conducting such training, it needs to be done across the
board at each R.O. Unless regional office staff, both managers and
individual adjudicators, learn from their mistakes and take corrective
action, there will continue to be a high rate of improperly adjudicated
claims, resulting in consistently high appeals rate, and a subsequent
high VBA remand and reversal rate.
Performance standards, which tend to go hand-in-hand with the
production-first mentality entrenched in VBA is another topic that
needs to be addressed. Performance standards of claim developers and
raters are centered on productivity as measured by work credits, known
as end products. Both veteran service representatives and rating
service representatives have minimum national productivity requirements
that must be met each day. Unfortunately, the end product work
measurement system, as managed by the VA, does not encourage regional
office managers to ensure that adjudicators do the right thing for the
veteran the first time. For example, denying a claim three or four
times in the course of a year before granting benefits sought allows
for several end-product work credits to be counted just for this one
case, rather than promptly granting the benefit and taking only one
work credit.
In the view of the American Legion, the need for substantial change in
VBA's work measurement system is long overdue. A more accurate,
reliable work measurement system would help to ensure better service to
veterans. Ultimately, this would require the establishment of a work
measurement system that does not allow work credit to be taken until
the decision in the claim becomes final; meaning that no further action
is permitted by statute, whether because the claimant has filed to
initiate a timely appeal, or because BVA rendered a final decision.
Lastly, we are pleased that C&P has begun implementing a job
certification test for VSRs, and we look forward to similar testing
being established for RVSRs and DROs. However, we are concerned that
the current testing taking place is not required as a condition of
employment for the position. although successful completion of the
test is required for promotion or transfer to the rating board, it is
still optional. The ultimate goal of a proficiency or competency
testing should be to ensure that an individual in any given position is
competent, proficient, and otherwise qualified to perform the duties
required of that position. This goal will not be achieved if testing is
not mandatory and remedial training or other corrective action is not
required for those who do not successfully pass the test. Although this
concept may not be embraced by some, the ultimate goal is to have
qualified and competent staff who will provide the best possible
service for our veterans.
In closing, the American Legion realizes that VBA faces many
difficult challenges during the upcoming fiscal year. Although we have
offered our suggestions and comments, we realize that there is no easy
solution, and we will continue to work closely with VA to ensure that
our nation's veterans receive the benefits to which they are entitled.
That concludes my testimony and I wills be happy to answer any
questions.
[The statement of Steve Smithson appears on p. ]
**********INSERT**********
Mr. Miller. Thank you very much, and I will start with you. You
talked in your written testimony about too few experienced supervisors
were available to provide trainees, I guess, with proper mentoring and
quality assurance. Can you give me some type of an indication as to
what you would think an experienced supervisor would be?
Mr. Smithson. An experienced supervisor would be somebody that has
been on the job for a number of years, has worked up through the
system, and has proven themselves to be competent through their work,
the performance standards, and other measures.
Mr. Miller. Mr. Cox, you had talked about VA failing the VSRs by not
adequately preparing them to take the skills certification test. Did
I read that correctly? How much of the onus of passing a test falls on
the employee?
Mr. Cox. There would be a part that would fall on the employee,
certainly, sir. However, much of the training material for the
training for the test, and for the employee to study, is on the
computer, sir. And I wouldn't even raise the issue of an employee
taking a computer home at this point to study for the test. But I mean
those type of things are the things readily available. And at the same
token, this is not a job that an employee comes with a ready set of
skills. I am a registered nurse by profession. I come to the VA with a
set of skills required for that job. With a VSR, that person comes to
the VA, and it is on-the-job training, and it is all done there on the
job.
Mr. Miller. Would AFGE support mandatory testing as a condition of employment?
Mr. Cox. I think AFGE would have to look very, very closely, and be
concerned with the mandatory testing as a condition of employment, sir.
We don't believe that is done at other federal agencies with similar
type jobs, similar to the claimant jobs; Social Security being one of
them.
Mr. Miller. Dr. Wise, you indicated that some of the test questions in
the pilot test were dropped because participants indicated that there
were problems with those questions. Can you give us an idea of what
those might have been?
Mr. Wise. Well, there certainly are instances where the rules may have
evolved since the question was written, so that no longer clear which
is the correct option on the question, would be an example.
Mr. Miller. Ms. Berkley?
Ms. Berkley. Thank you, Mr. Chairman. The State of Nevada has the
fourth highest remand rate in the nation, and I have a lot of very
unhappy veterans, and I have been trying for quite a while to figure
out what we could do to make this work a little bit better. I have
questions for each of you, and again, let me thank you all for being
here and sharing your knowledge with us.
Mr. Smithson, in your experience, do regional offices which provide
consistent, high-quality training have fewer errors and lower remand
rates?
Mr. Smithson. Could you repeat the question please?
Ms. Berkley. Yes, I can. Do regional offices - this seems like a
no-brainer to me, but I am kind of curious, regarding your experience.
Do regional offices that provide consistent, high-quality training have
fewer errors and lower remand rates? Have we been able to track
high-quality training, better service on the job, lower remand rates?
Mr. Smithson. In our experience, there is more satisfaction. Veterans
have more satisfaction from receiving decisions from those offices,
yes.
Ms. Berkley. I listened to your testimony, but I have to tell you I
think you speak faster than I hear. So I would appreciate if perhaps
you spoke a little slower. Do you have any specific recommendations
for revision of VBA's workload management system?
Mr. Smithson. Again, I think we need to -- are you talking about the
end-product, the credit, work credit?
Ms. Berkley. That could be part of it. Sure.
Mr. Smithson. For the work credit, I think we need to look at not
giving credit for a case until it becomes final, whether that be the
one-year period expires and the veteran does not file a timely appeal,
or the BVA renders a final decision.
Ms. Berkley. When we talk about VBA's workload management system, is
there a magic number that -- and I know it can't be the same number
for every employee; everybody works differently. But is there a point
at which our VBA employees are so overloaded that they can't possibly
provide the quality service that our veterans need?
Mr. Smithson. I think some stations, they are definitely overworked,
not enough staff. When we do our quality review visits, not only do we
look at the decisions for quality errors; we talk to the VBA staff, and
a lot of them tell us that, you know, there is not enough people, the
standards are too high, they are not realistic. Some stations, we
encountered in our experience, in addition to the national standard
have their own standards that may be above the production standards,
that may be above what the national standard is, because of their
backlog and their particular situation.
Ms. Berkley. Okay. Mr. Cox, welcome. Are the problems that you
described in your testimony with training widespread, or more
problematic to specific regional offices? Is it just across the board?
Mr. Cox. I think it is across the board. I think the training, that
there is not consistency with the training. And also, dealing with
adult learning is a creature unto itself, as I think we are all aware.
Ms. Berkley. I would hate to have to go to law school again, that is
for sure.
Mr. Cox. Yes, I wouldn't want to go to nursing school again today,
either. But I think people learn by different modalities. Some do
well with online-type training. Others need a standard classroom,
those type things. And so the training that meets the needs of the
individual in how they learn to perform that job -- again, I think one
very specific thing about these jobs in VBA are you don't come with a
ready set of native skills for them. You have to get that training
there at the job site. I mean, there are people in these VSR positions
that come with a high school diploma, and some come with graduate
degrees that do the same work, but they must learn all that work there.
Ms. Berkley. But doesn't learning have to be somewhat standardized,
the methods -- I think in our school systems across the country, there
are kids that learn different ways, but I don't think that we can
provide a specific type of learning path for one student and one for
another and another. Is there a way that we could standardize this
nationally, and still be able to provide the training that our VBA
employees need?
Mr. Cox. I think that you could get the standardized training. I
would look at what is done at the VBA Academy in Baltimore. There are
people -- that is their job, to do nothing but training. It is not a
collateral-type duty or things that they do. They are trainers, and
they do that work consistently over and over.
And I do applaud the VBA. I think the training academy in Baltimore
has done an excellent job, and will continue to do an excellent job,
and that there are good lessons to be learned from the things that are
housed there.
Ms. Berkley. Let me ask you a couple of questions -- part of your
testimony concerned me because I am a strong union supporter, and a
great believer that unions and management working together certainly
provide a much better product. Can you describe the efforts the union
has made to engage VBA in collaboration on training and performance
issues, and has there been a change in behavior? And if so, to what do
you attribute that change of behavior to?
Mr. Cox. I would think, you know, also Mr. Walcoff responded earlier,
and I have always had a great working relationship with Mr. Walcoff and
his colleagues at VBA, as well as many of us at AFGE. However, and I
would, you know, say that in his comments, it is `` We comply with the
law, we comply with the contract, we do the things that meets our
labor-management obligation.'' However, many times we have to do
more. We have a veteran, your constituents, at the end of this
process, waiting for their claims to be processed, and be processed
correctly. So to just comply with the law, with what it says in the
labor-management relationship, I don't believe is enough. There must
be a true partnership, a collaborative relationship of working
together, and meeting the needs, clearly identifying problems, and
arriving at a solution, and sometimes sharing information that either
side kind of holds private.
Ms. Berkley. Thank you very much.
Mr. Wise, how does the pass rate of 25 percent and 42 percent for the
open-book VSR test compare to other similar occupational tests with an
open-book format? That seems extraordinarily low, if you have got the
answers at your fingertips.
Mr. Wise. Well, I would agree. And actually, I am not familiar with
that many instances of open-book tests, which I think is a fairly
unique feature of this assessment, in allowing the examinees access to
the resources, most all the resources they would normally have on the
job.
Ms. Berkley. And you know, I found that kind of interesting because
when I was in college and law school, if I took an open book test, I
always performed worse, because I think I was expecting that I would be
able to get the answers really quickly during the test-taking time, and
it was just a disaster. So I am not sure that is a good idea.
Mr. Wise. Well, it is an important feature of the assessment, that in
order to ensure its validity we try and replicate the job as closely as
possible. So for that reason, open book does seem appropriate to this
exam.
Ms. Berkley. Does the low pass-rate suggest that a number of
experienced employees do not have the requisite knowledge, skills, and
aptitudes to perform their current jobs adequately? And if that is not
the case, to what do you attribute the low test scores?
Mr. Wise. The VBA management and, you know, and the experts that
reviewed it set the pass scores at a level that was adjudged to be the
point at which people that were just sufficiently qualified would need
to score, in order to demonstrate the kind of skill that is required
for this higher-level position. The lower passing rates would indicate
that many of the candidates who took the exam don't yet have those
skills, but they do have the opportunity to go back and study and take
the exam again.
Ms. Berkley. Okay, thank you very much.
Mr. Miller. Thank you very much, Ms. Berkley.
That is all the questions that I have, too. And everybody's testimony
has shown that the training claims examiners receive is vitally
important to achieving VBA's mission. As I said in my opening
statement earlier, the adjudication of claims is a complex and
cumbersome process, and I certainly appreciate the importance that
Admiral Cooper has placed on training by mandating the 80 hours of
training annually, and hope that the regional offices' directors are
taking heed of this mandate.
I look forward to working with VBA and its stakeholders to ensure that
proper training and skills development is at the forefront of claims
development. I appreciate everybody's time for coming and attending
today. Without objection, members will have five legislative days to
submit materials for the record, as well as post-hearing questions to
the witnesses.
With nothing further, this hearing is adjourned.
[Whereupon, at 3:29 p.m. the hearing was adjourned.]