Breakout Session: Developing Standards of Excellence in Vocational Evaluation

Developing Standards of Excellence in Vocational Evaluation was a working session designed to review current best practices in vocational evaluation both at the nine state operated comprehensive rehabilitation centers and in selected state vocational rehabilitation agencies. This was a panel presentation where panel members made presentations regarding the standards currently in use at their facilities. Additionally, the August 2003 30th Institute on Rehabilitation Issues (IRI) publication, A New Paradigm for Vocational Evaluation Chapter 8, "Promising Practices of Vocational Evaluation and Assessment" was overviewed and discussed.

This session was chaired by Dr. Richard S. Luck, Director, Woodrow Wilson Rehabilitation Center, Virginia and recorded by Ms. Susan Howarth, Rehabilitation Technical Specialist, Workforce and Technology Center, Maryland. There was good representation on the panel from both centers and state agencies by the following members:

• Ms. Dale Batten, Department of Rehabilitation Services, Virginia
• Mr. Jim Doyle, Workforce and Technology Center, Maryland
• Ms. Kathy Gates, Roosevelt Warm Springs Institute for Rehabilitation, Georgia
• Ms. Elizabeth Kinne, Roosevelt Warm Springs Institute for Rehabilitation, Georgia
• Mr. Steve Kirby, Woodrow Wilson Rehabilitation Center, Virginia
• Dr. Kim Lawson, Roosevelt Warm Springs Institute for Rehabilitation, Georgia
• Ms. Cindy Martin, Hiram G. Andrews Center, Pennsylvania.
• Ms. Trina Robertson, Workforce and Technology Center, Maryland
• Ms. Janna Simms, Hot Springs Rehabilitation Center Hospital, Arkansas

Additionally we had excellent attendance and participation from the following:

• Janet Zerke, West Virginia Rehabilitation Center, West Virginia
• Tenia Eldridge, Workforce and Technology Center, Maryland
• Joan Patinella, Workforce and Technology Center, Maryland
• Betsy Sims, Workforce and Technology Center, Maryland
• Christine Hudson, Workforce and Technology Center, Maryland
• Greg Schmieg, Roosevelt Warm Springs Institute for Rehabilitation, Georgia
• Bill Norwood, Tennessee Rehabilitation Center, Tennessee
• Evelyn DeBoard Perkins, Carl D. Perkins Comprehensive Rehabilitation Center, Kentucky
• Sam Castiglione, Workforce and Technology Center, Maryland
• Remona Phillips, Workforce and Technology Center, Maryland
• Terry Sims, Hot Springs Rehabilitation Center, Arkansas
• Jeff Knight, Woodrow Wilson Rehabilitation Center, Virginia
• Vincent Downs, Workforce and Technology Center, Maryland

Top of page

A. Discussion and Highlights: Panel and Participants

Panel Presentations

There were 4 presentations by panel members focused on models in place in the several centers and states as follow:

• Chapter 8 of the 30th IRI publication was overviewed by Dr. Richard Luck. This publication made it very clear that vocational evaluation had evolved into a more comprehensive system of not only the traditional vocational evaluation process but one which includes career assessment and career planning. Additionally, this presentation was a summary of the findings of the study group that included a review of the models posed by Kansas, South Carolina, Virginia and Maryland. All of these models had several elements in common. These include:
• Soliciting input from a variety of stakeholders
• A long term implementation process with a built-in review mechanism
• Modification of the vocational evaluation report to be more customer friendly
• Implementation of a continuous quality improvement process
• An inclusion of assistive technology assessments as part of the vocational evaluation and career assessment process and
• Placing reports in the hands of consumers and making the promotion of empowerment and personal responsibility easier to understand.
• The Virginia approach is one in which evaluators from Woodrow Wilson Rehabilitation Center (WWRC) have a high degree of cooperation with the Department of Rehabilitation Services (DRS) field counselors. Evaluations are viewed as an individualized, systematic and comprehensive process in which consumers and staff work together to complete the assessment. It begins with an individualized plan that is carried out in a manner sensitive to cultural, ethnic, and other environmental concerns. The field offices work as a team composed of evaluator, employment specialist and field counselor. They have staffings to identify employment opportunities in the community and all take responsibility for the placement and the success (26) or failure (28) of the case outcome.
• The Hiram G. Andrews approach begins with an intensive screening for assessment. There are no direct referrals into training without the client completion of the screening first taking place. Clients are sorted into three levels of service: high verbal, hands on and individual (one on one). Participation in evening programs while at the center is mandatory and this allows for both assessment and development in the "life skills" areas. All assessments are based upon a 3 week schedule as follows: first week consists of the initial interview, aptitude and achievement assessments and an evaluation of individual service needs; second week involves career exploration and guidance with counselor assistance; and the third week involves work try outs in selected areas. The report is ready within 10 days.
• The Maryland Workforce and Technology Center had just recently undergone a CARF accreditation review and the CARF process according to their experience was overviewed concerning vocational evaluation services. It was reported that the acquisition of accreditation by CARF is greatly improved through advance preparation especially regarding documentation. Some of the CARF standards which were seen as of particular importance involved the following considerations:
• Individual plans for the evaluee
• Functional capacity of individuals' strengths and needs
• Provision of assistive technology to maximize potential
• The integration of resources and support
• Specific questions to be answered including input from the consumer
• Clarity of purpose of initial interview especially concerning accommodation and assistive technology
• The identification of a system either internal or external to the center to provide accommodations once identified.

Top of page

The Workforce and Technology Center employs several guidelines for producing high quality vocational evaluation and career assessments as follows:

• Qualified staff:
• all new staff are mentored for a year by an experienced evaluator
• peer review of reports on a monthly basis
• staff encouraged to develop expertise in specific populations
• on-going staff development and training
• Time measures are in place for length of services as well as timeliness of reports.
• Variety of assessments: comprehensive, exploratory, focused, itemized, job analysis, consultation
• Customer driven services providing an array of services that are consumer driven
• Consumers are empowered as stakeholders, participating in the assessment process to make informed decisions.
• The Georgia experience was somewhat different in that they had contracted out their vocational evaluations in 1992 to the Community Rehabilitation Programs (CRP). In 1996 four of the six CRPs were eliminated which created difficulty in getting vocational evaluations completed in a timely manner. This issue is currently being studied and bringing the evaluation process back within the agency and Center is being considered as a way to save money and increase timeliness of reports. The current emphasis on vocational evaluation needs to be on employment trends specifically as it involves assistive technology, competitive employment, customized employment and life long learning.

  In the area of work samples it is important that the evaluator gives specific instructions for the completion of work samples and that the methods and directions of administration and scoring are clear and understandable by the consumer. Additionally, in the area of psychological testing, APA standards must be employed within the proper settings. In the area of employment exploration safety was sited as of paramount importance as well as a clear understanding of the specific regulations of the worksite and employer relative to both quantity and quality of work performed. Employer expectations and employment opportunities available relative to the individual consumer's needs and abilities were also stressed as important considerations.

Top of page

Participant Discussion

After the panel presentations the workshop participants were divided into small groups for discussion and exchange of ideas. These small group discussions were designed specifically to accomplish the following:

• To give participants opportunities for questions and answers, feedback, and sharing of ideas
• Identify and prioritize existing and new and/or needed Standards that address key challenges and issues regarding Vocational Evaluation and Career Assessment to answer the following questions:
• What is needed that is not currently in place?
• What possible mechanisms could be developed to put standards in place?
• Small Group participants were given the opportunity to discuss and identify possible Standards of Excellence that they believe should be adopted by NCSOCRC.
• In a large group they were given the opportunity to synthesize and recommend agreed upon Standards of Excellence.
• Identify action steps that need to be taken to implement the results of this Track Discussion.
• Who will take the leadership role?
• Identify strategies for addressing recommended action steps.

The resulting discussion was both stimulating and challenging. The small groups worked very diligently and produced many good recommendations and ideas regarding the establishment of standards of excellence in vocational evaluation and career assessment.

B. Current Challenges Facing Rehabilitation Centers

The small groups identified several challenges to the development and adoption of standards of excellence in vocational evaluation at the nine state operated comprehensive rehabilitation centers as follows:

• Reaching consensus on what the standards should be will take considerable time and effort.
• Leadership in continuing the process would be an essential element in reaching agreement on the development of standards.
• CARF standards are in place in several of the centers and potential conflicts between them and any new standards would have to be reconciled.
• At some point a regulatory or compliance monitoring process would need to be put in place to ensure conformity to any standards that could possibly be developed and agreed upon.
• Peer review teams would have to become part of the process to review and monitor compliance and this will take valuable time and resources from the cooperating centers.
• A mechanism will need to be put into place that will sustain the momentum gained at this Forum.

Top of page

C. Effective Practices in Establishing Standards of Excellence in Vocational Rehabilitation

One of the great values of this forum track session was the identification of many similarities that the nine state operated comprehensive rehabilitation centers currently share. Not only were similarities identified but also significant differences were brought to light through the lively interchange that took place. One significant difference is the role that technology plays in the actual production of the evaluation report. Most Center evaluators generate their own reports through word processing capabilities, but on the other hand, in states where there are labor unions that protect clerical workers; reports are still dictated and then transcribed. Another significant difference is the context within which vocational evaluations are produced. While the basic tenets of a vocational evaluation are covered by all Centers, they are done so in very different ways. For example some centers provide evaluation within the context of a center based evaluation while others do so in the context of the workplace.

Small group reports identified several effective practices that need to be considered in generating standards of excellence in vocational evaluation. These are as follow:

• Individual client consideration
• Timeliness of evaluation in the rehabilitation process
• Timely reports
• Qualified staff conducting evaluations and producing reports
• Requirement that the process be consumer driven
• The vocational evaluation process focused upon employment
• Client choice is a necessary emphasis
• Cross validating process skills from pencil and paper to hands on assessments
• Hollistic process including client data from the social, personal, and cultural spheres as well as the clients values, attitudes and abilities
• Emphasis of Vocational Evaluation as a continuous dynamic process

Top of page

D. Recommendations:

1. Utilize the following preliminary standards in further developing and refining Standards of Excellence. The process of career assessment and vocational evaluation can be conceptualized as a dynamic three step information gathering and reporting process: Pre-assessment, concurrent and post.

• Pre Assessment

• Initial preparation should include thorough review of referral information.
• Requests for additional information on previous services or assessments may be helpful to provide additional content for the vocational evaluation.
• In order to maximize the benefit of the evaluation process, every consumer is entitled to:
• A clear understanding of ancillary services required for a successful evaluation experience.
• A well planned individualized evaluation.
• Input throughout the evaluation process.

• Concurrent

• Identifying, respecting and developing a sensitivity of diversity in approach to the evaluation process and recommendations.
• All assessments will be consumer driven for their individualized area/needs.

• Post

• The evaluation process shall identify, recommend and provide follow up consultation on reasonable accommodations and assistive technology specific to recommended employment outcomes.

2. Continue the work begun at this forum through the development of a Steering Committee charged with scheduling follow up activities.

• The following individuals volunteered to form a Steering Committee to continue the work of this Track Session:

• Tenia Eldridge, Workforce and Technology Center, Maryland
• Susan Howarth, Workforce and Technology Center, Maryland
• Janet Zerke, West Virginia Rehabilitation Center, West Virginia
• Steve Kirby, Woodrow Wilson Rehabilitation Center, West Virginia
• Dr. Richard S. Luck, (Convener), Woodrow Wilson Rehabilitation Center, West Virginia

3. Evaluate other existing standards of excellence that may be inherent in certifying and accrediting bodies both in the U.S. and abroad.

4. Promulgate the findings and recommendation of this work group to staff at the comprehensive centers, field offices and community rehabilitation programs and solicit input from as broad a base of stakeholders as possible.

5. Utilize the CERMUSA network of video conferencing and other means of teleconferencing available to continue the work of this work group.

6. Continue the further development of the rudimentary Standards developed in this Program Track through the writing of a position paper that enumerates the identified Standards and identifies the future direction for their universal adoption.

7. Initiate a preliminary design of an organization that will institutionalize the draft Standards and enumerate the process for adopting and adhering to them. Such an organization will necessarily be required to develop a process for voluntary certification or accreditation, peer review, and compliance monitoring.

Top of page

---