Assessment Issues for
America’s Job Center Staff

Welcome to this workshop/course on assessment in job centers. The course is divided into six parts and requires you to answer some questions found on the worksheet which you will download below.

Your task is to move through each part of the course, read the information, watch the video when asked, and answer the questions on your worksheet.  Once you complete the worksheet, mail it to Janet Wall to receive your clock hours for your certification and professional development. You will earn 5 clock hours toward your certification if you are a Certified Workforce Development Professional and 4 clock hours if you are an NCC, LPC, GCDF, or other counselor related certification working in a job center. Please indicate your status on the worksheet. The difference in clock hours is based on the NBCC requirements for counselors.

You have 30 days to complete the course requirements.

View the other offerings and how they relate to the NAWDP competencies. https://careerdevelopmentmusings.wordpress.com/2016/09/03/crosswalk-of-ceuonestop-courses-and-webinars-to-nawdp-competencies/ 

This workshop/course deals primarily with assessment as it relates to the career development and career counseling of clients serviced by America’s Job Centers. The module matches the assessment related competencies as specified by the National Association of Workforce Development Professionals (NADWP) listed below. They are similar to what is expected by a career counselor who belongs to the American Counseling Association or the National Career Development Association.

Regardless of your professional affiliations, using assessment in career exploration, career management, and/or educational or training placement, is important for those assisting customers in finding and succeeding in work that offers a living wage and personal fulfillment. It is also important to the employer and society that deserves a worker who reaches his or her potential and contributes his or her skills and abilities to the organization, the institution, and the common good.

Related NAWDP Competencies

• Administers and interprets a variety of assessment tools.
• Identifies the kinds of information individuals need, including assessment, in order to make realistic career decisions, and where that information can be found.
• Knows what skills are needed to search for, obtain, retain, and change employment.
• Provides customer with career exploration and job development skills.
• Possesses analytical and observation skills that coupled with knowledge and information can lead to effective problem solving assistance.
• Identifies customer needs and expectations to create positive customer satisfaction.

Learning Objectives for this Workshop

At the end of this workshop you will be able to:

• describe the role of assessment in the career development process and how it helps both your clients and employers
• identify the assessments used in your job center and list what they measure
• describe the various criteria for selecting the right assessment for your clientele
• verbalize the meaning of reliability and validity
• state the cautions in interpreting assessment results
• analyze the assessment you use to be sure that you are using solid assessments that can help your clients

In this workshop/course, you will be asked to read some information, watch videos, and complete some exercises. This information is available 24/7/365 so that you can add to your professional qualifications and skills at times that are convenient for you.

As you go through the course, you will complete a worksheet to help you complete various exercises and organize the information your have learned.

DOWNLOAD AND SAVE the worksheet by clicking on the graphic. Save the worksheet to your computer and complete the items as directed throughout this website. 

***** Specific directions to view a video, read an article,
or answer questions are listed in red.*****

You will be asked to complete an evaluation at the end of the course. One of the questions relates to the time it took you to complete the course requirements. 

Completion of this course can assist you in re-certification and certainly will help you do a better job with your clients. You have 30 days to complete the workshop requirements. 

****Just as a reminder, the materials in this course are
copyrighted and are available for your use in the course only. ****

Now, if you are ready, go to Part 1 to start the course.

Why Use Assessment Anyway

If you work at an America’s Job Center, you or one of your colleagues may have given an assessment, interpreted an assessment for a customer, or used the results to help them find a job, training program, or career path. Have you thought about why assessment are given, why they matter to you and also to the employer?

There are a lot of situations in which assessments can help a person on a career path. Some examples are:

1. A person may just be entering the job market and are unsure of the type of training one needs to help get a good job, or even what type of job a person would like. Assessments can help focus a person in a search for training or a career.

2. A person may be receiving some financial assistance for training from an agency. The agency may want to assess the person’s capabilities so they can help provide the right type of training.

3. A person may have been looking for a job, but hasn’t been successful because he or she doesn’t have a diploma or the required certificate. Tests can be taken to get a certificate or a diploma.

4. A person may have a job that he or she doesn’t like, but they don’t know which other careers may be compatible. Assessments can help you identify careers with a future.

5. A person may be in an industry using new technology. His or her job has disappeared, and he or she needs to retrain and upgrade their skills. Tests can help identify your particular training needs for high technology jobs.

6. Layoffs at a company are likely and it is important to explore other career options. Assessments can help direct a person toward other jobs that might be suitable.

7. A person may have a disability, but clearly can be successful in compatible jobs and careers. Assessments can help identify that person’s strengths and abilities.

8. Re-entering the job market after time away is often challenging. Skills need to be upgraded. To match a person’s skills with current requirements in current occupations and jobs is often the job of an assessment. Tests and other assessments can help.

9. A person is up for a promotion. Taking an examination is often a common part of the promotions procedure.

10. A young person in high school is unclear about what career field to pursue. An interest inventory and an ability inventory can help to clarify a compatible career field with the person’s  interests.

11. A college freshman is confused about what major to select, an assessment can help the person clarify what skills and abilities he or she should develop in order to be successful in various possible career fields.

12. A person is unhappy with their current job situation because it doesn’t seem to contribute to society in a meaningful way. A work values inventory might be appropriate.

13. An employer is looking for a person with specific technical skills in order to hire him or her into a particular job critical to the company. Assessments can assist in the identification of skills.

14.  An employer has several qualified candidates for a job. Assessments can help identify the best person for the job.

Hopefully, this workshop module will make you feel more comfortable about tests and assessments, their administration, interpretation, and use with individuals moving through their career. Hopefully it will assist you in your ethical and professional use of assessment in your work.

WATCH this video to see where assessment is in the career planning process.

COMPLETE items 1 and 2 on your worksheet.

Typical Assessments

How do you learn about your customers and clients? Yes, you do an intake interview, meet with the people, ask about their employment history, review their education and training, and the like.  But do you really know them in an objective way? Do you know what they can do? Do you know what they want to do? Do you know what their skills are and how that matches up with employer needs?

Assessments often provide those answers for you.

If you look carefully, your job center probably uses many assessments to help your clientele get the best advice, direction, and services possible. It is very likely many assessments are to be found in the tool kits of each job center.

In this section you will be doing an environmental scan of the assessments used in your job or employment center. You will determine what they measure and why they may be used.

WATCH this short video to learn about the difference between tests and assessments and what the assessment might provide as you help your clients and customers.

ANSWER item 3 on your worksheet before gong on to the next section.

Criteria for Selecting An Assessment

As stated in the previous section, there are many hundreds of career and workforce related instruments that you can select from. How can you make the determination from all those possibilities to choose the one that is best for your clients and customers. It is not a simple task and it requires your purposeful thought.

Giving an assessment just so you can say you gave one is not good professional practice. Maybe you have a favorite instrument and you like to give it to just about everyone that comes into the job center for assistance. That reminds me of the concept that if the only tool you have is a hammer and that all problems look like nails. The one-size fits all approach is not usually appropriate or helpful.

In this section, you will learn about the various criteria you should use in selecting the right assessment for your client. Each of these factors should be very carefully considered before you select an instrument for your use.

Step 1: What Do You Want to Know About the Client and Why?

The A #1 question for you to answer is what do you want to know about the person who has come in for assistance? Clearly you have completed an intake interview and perhaps reviewed some records, but if you are thinking of using an assessment you must first answer this question. What is it you, the employer, or the client wants to know?

Do you want to conduct preliminary discussions about a client’s vocational interest areas? Is it for basic career exploration? Is it to identify a range of occupations or a few specific ones? Is it for selecting a college major or training program? Might it be for job selection? Do you want objective information on the person’s abilities, skills, interests, values, personality? Write down why you want to use an assessment and what you want to measure on your worksheet before you go further in this section.

Step 2: What Does the Assessment Measure?

In the previous section of this course you listed several of the assessments used in your job center. Look at those assessments and identify those which say they measure what you want to know about the client. If one of those assessments doesn’t measure what you want to know about your client, use other resources to find one that fits the bill. What you want to know about the client should match what the publisher says the instrument will measure.

Steps 1 and 2 sound easy, but many counselors and coaches tend to skip over these steps ending in disastrous results. The client isn’t helped and time and resources are wasted.

Step 3: Was the Instrument Developed on People Like Your Client?

The assessment you use should be appropriate for your client and developed on people that have characteristics like your client. This is where you will need to dive into the manual associated with the instrument to find out how the instrument was developed and on whom. If you are helping long term unemployed job seekers who have years of experience in their careers, was the assessment developed on similar kinds of people? Many instruments have been developed on high school or college students and may not be appropriate for the people that you help.

The instrument you select should fit the characteristics of the person or persons who will actually take the instrument.  As an example, an interest inventory developed for adults in transition may not to be appropriate for a middle school or high school student.

Step 4:  Is the Language Level Appropriate?

Do the items have content that is easily understood by your clients?  Does the assessment use vocabulary that is familiar to them? Also, an assessment should have directions and items that can be understood by the same age or grade level of your clients. If you serve minority or non native speaking clients, are the words, phrases, or statements likely to be understood and suitable for those individuals? Do the items resonate with both males and females?

Step 5: Is the Format Appropriate for Your Clientele?

If there is an option of taking the assessment on paper, via computer, or over the Internet, you, as the assessment user, must be sure that the format and delivery platform do not influence the outcome of the results. For example, is the assessment found to be easier when taken on the Internet thus giving some people an advantage?

Will you offer your clients an option of assessment format? If so, you need to be sure that there is evidence showing that the results on a paper/pencil instrument will be the same as the results delivered via computer or over the Internet. This is especially true for assessments that might be high stakes such as those given for entrance into a training or educational program where there is a score cutoff or minimum. Comparability of assessment results is a very important concept as not all assessments give the same results or scores regardless of delivery platform. Good test publishers will provide this information in their technical manuals.

Do your clients prefer a paper/pencil assessment or one delivered via the computer or one administered via the Internet?  Some research shows that younger individuals prefer a computer format for taking an assessment; some older individuals and some ethnic groups may not be so inclined.

Assessment takers should not be intimidated by the format of the instrument; rather they should be concentrating on answering the items to the best of their ability. Directions and format should be easy to follow regardless of the delivery platform. Navigation should be simple and intuitive.

Step 6: Is the Assessment Technically Sound?

The next section will deal with this in more detail, but it is critical that any instrument you use is technically strong, meaning that is has high reliability and established validity through extensive research. You should not put any faith in an instrument that doesn’t have a strong research base. It could be hurtful, counterproductive, and professionally unethical.

Step 7: How is the Assessment Scored?  

A key factor to be considered in the selection of an assessment is the scoring process. Scoring can be performed by the person taking the assessment, by you as the user, or automatically by a scoring program.

Some instruments provide a self-scoring option. One consideration in self-scoring is the complexity of the scoring process. The more complicated the scoring procedures, the more open the process is to error and incorrect interpretations. Simple functions such as counting or adding have been shown to pose fewer problems than scoring procedures which require the transposition of information from one page to another and/or multi-stage processes that include multiplication, division, or weighting in any way. Think about self-scoring requirements versus the capabilities of your clients.

If the instrument is not self-scored, will you need to score the inventory yourself?  If you have just a few instruments, scoring them may not be too onerous. But if you have administered the instrument to a group, this may require a significant amount of your time.

Perhaps the responses to the items are placed on a scannable sheet for scoring by the instrument’s publisher, an external vendor. or your agency. This technique enhances accuracy, but some additional factors must be considered. If scanning answer sheets is an option, you may wish to weigh the benefits of accurate scoring against the degree of delay in receiving the results back in sufficient time to be of use to your clients. It may be worth it to you to purchase or lease a scanner along with the scoring program if you have many instruments to score and you do it frequently.

Scoring of online assessments generally offer the best of both worlds – accuracy and quick turnaround. However, with this option, privacy and security may be issues.

Step 8: How are the Results Reported? 

Another important aspect is the amount of information provided after the assessment is scored. Assessments that are paper-based and self-scored often provide only basic information. That is not necessarily bad.  Automated systems often report larger amounts of information tailored to the assessment taker. Look at the score reports to determine what information is provided and how that compares to how you and your clients hope to use the information.

The amount of information must be right for the age, knowledge, and level of readiness of the persons with whom you are working. It is possible that too much information will overwhelm assessment takers and prevent them from moving forward in their career development or life planning. Too little information is not very helpful either. These are judgment calls you will need to make by knowing your clients. You may need to provide your assistance to the client to sort out what her or she needs to know and act on. That’s what a career counselor and workforce development professional does!

Sometimes graphical or pictorial representations are the quickest to understand and interpret, but they may gloss over more comprehensive information. You may see marketing information about some assessments that brag about providing dozens of pages of interpretive information, seducing you to think that more is better. Generally that is not true.

It’s a lot to think about, but if you want to be professionally responsible and offer a solid assessment to your client to help them in their career journey, you need to follow the steps  outlined above.

Watch this video which will describe some other considerations that you should think about as you establish the use of assessment in your job center.

ANSWER items 4 on your worksheet before moving on to the next section.

Technical Characteristics

There are two very important technical considerations when you are using assessments.  These concepts are reliability and validity.  These will be discussed in turn.

RELIABILITY:  Reliability is essential to testing and a good understanding is required. A general definition of reliability refers to how consistently an assessment measures what it is purported to measure.  In essence, reliability allows the user to depend upon the assessment to measure a trait or variable each time it is used. When evaluating the reliability of an assessment it is important to understand the meaning of the reliability coefficients that are reported. Reliability coefficients are reported as a correlation coefficient with a range from 0.00 to +1.00. When the reliability coefficient is 0.00, there is no relationship or no reliability. A number closer to 1.00 is preferable.

What is an acceptable level for reliability? There is no absolute answer to this question. When selecting an assessment, you need to determine its purpose and implications of the results of the test. If the results will have significant life-altering implications (e.g., placement decisions, admissions, and job selection) then high levels of reliability would be necessary. Things easier to measure, like achievement, skills, work values, should also show high reliability. 

WATCH the video below to obtain additional information on assessment reliability.

ANSWER items #6 on your worksheet and then go to the next section on validity.

VALIDITY: Once a level of reliability is established by research studies, the next most important assessment criteria is validity. Validity relates to how you can use and interpret the results and this is established by research studies. Basically, just because you have a score from an assessment, you cannot use the score in any way you want. There needs to be some research support that supports the specific use.

For example, you are probably familiar with the SAT or ACT which are both well researched and well respected assessments. These assessments were created for a specific purpose. Research studies show that they fulfill their purpose well, which is to predict success or not in first term postsecondary education. As good as those assessments are, they do not predict success in specific subjects, success in one’s career, or success in life.

As for another example, the Armed Services Vocational Aptitude Battery (ASVAB) was created to predict success in various military training programs. Hundreds of research studies show this to be true. However, various studies also suggest that the results can predict school grades, is related to occupational success, and higher pay in civilian occupations, just for starters. Why is this true? Because various research studies have verified these facts. What the ASVAB doesn’t do is measure achievement status in high school or other areas where research has not been done.

The Self-Directed Search (SDS) was designed to determine a person’s dominant interests.  As a result of various research studies, it has been shown to be associated with various characteristics of occupations, job satisfaction, and productivity, if the person is in an occupation or job that satisfies the person’s dominant interest areas.  The SDS does not measure a person’s level of occupational success, achievement, college success or any other aspect for which there is no validity evidence.

Work Keys is used by many staff in workforce development. Work Keys is made up of several tests including Applied Mathematics, Locating Information, and Reading for Information. The publisher has conducted profiles of many occupations and has shown, through research studies, the levels of each of the three skills that is generally needed for various occupations. They can do this because they have studied the requirements of that occupation. As helpful as WorkKeys might be, it does not suggest that a person will do well in college, or training programs, or in all occupations or jobs. It is specific to the occupation and its profile.

WATCH the video below for more insights on assessment validity.

ANSWER item #7 on your worksheet.

If you have completed this section, move on to Part 5.

Interpreting Results

The rubber meets the road, so to speak, when you have to interpret the test or assessment results. This is where your client may take some concrete action based on what you say to him or her. Clearly what you tell them is critically important to the client’s future, so you have to be sure what you say is absolutely correct.

Unfortunately this workshop cannot possible cover all the various assessment you may use and the resulting interpretations. This section will cover the general considerations you must understand and follow in order to make legitimate and ethical use of the test/assessment results.

Dick Bolles of What Color is Your Parachute fame, describes a situation where a counselor/coach gave an interest inventory to a client and told him that the results showed he had no aptitude for mechanical jobs or tasks. For years the client avoided situations and tasks that required some mechanical ability. When a situation arose where he needed to do some construction work on his house, he fully expected to fail and to hate the experience. To his surprise he did well and enjoyed the experience immensely. He felt that the misinterpretation of his assessment hurt his options for satisfying careers.

The counselor misinterpreted the assessment and set the man on a very wrong path for most of his professional life.  I know you don’t want to misinterpret assessment results and hurt your clients so pay attention to the checklist below.

Below is a checklist of things that you must consider before interpreting any test/assessment. If you are taking these items seriously, then you are probably offering solid and helpful interpretations.

What does the test/assessment measure? It should be specifically stated by the publisher. Does the publisher show any research that proves that the test/assessment actually measures those ideas/constructs?  Is that research credible? Scientific? Don’t presume the assessment measure anything beyond what the publisher suggests unless you have done your own research.

Does the publisher offer technical information such as reliability and validity evidence? If not, done bother trying to interpret the test or assessment.

 

How does the publisher specify how the results could be used? Is this backed up by research and specific evidence?

 

Is the suggested use and interpretation overstated, that is does it promise more than what the research evidence suggests? Are you using the test/assessment results within the allowed interpretations?

 

Does the publisher say how the results should NOT be used?

 

Does the publisher consider standard error of measurement that is, that test/assessment scores are not perfect points by rather a range of probable scores?

 

Are the results relevant specifically to your clientele?

 

Is the test/assessment current, that is, has it been updated to reflect current events/conditions?

WATCH this video as each item is explained.

Answer item# 8 on your worksheet.

Putting it All Together

Here is the tough part of the course, putting it all together.  In this section you will investigate in depth, one instrument used or you propose to use in your workforce center.  In this section you will go to your answer sheet and complete several questions.  To answer the questions you will need to look at the instrument and many of its supporting documents such as user manuals, technical manuals, etc.  These documents might be paper documents your agency has on file or they may be on the publisher’s website which you can download and review.

The purpose of this section is to help you dive deeper into the instrument in order to be sure it is good for your clients and that it is a credible instrument.  Remember, your profession and your clients expect you to use tools that are technically solid and appropriate for them so that they can plan their futures, actions, and decision on a solid foundations.

Their futures depend on you and your knowledge and judgement. 

Congratulations on completing the course.  Submit your worksheet to Janet Wall who will review your answers and issue you your certificate of completion.

Please complete this short evaluation.

 

Thanks for your support of CEUonestop.com.