Updated April 28, 2021
I’ve heard from many of you who wonder why so much time passes between taking a beta exam and getting your results. We really appreciate your help with our beta program. I’d like to give you some insight into the beta timeline for creating high-quality exams, so you can understand why it might take as few as 8 or as many as 16 weeks to get your results.
Finding 250 qualified candidates
We conduct beta exams as part of the development process for releasing exams. Beta testing lets us learn how each question (aka ‘item’ in my world) performs in exam conditions. To get the best information about the psychometric performance of the items as well as detailed verbatim feedback, we need at least 250 qualified people to take the beta exam.
We offer the beta exam at a discounted price for a four-week period. If fewer than 250 qualified candidates sign up, we extend the offer for a few more weeks until we meet our threshold of 250 candidates. This pushes out the time it takes before we can move to the next stage of exam development.
This initial process takes from four to eight weeks.
Analyzing and reviewing test results
As soon as at least 250 qualified candidates have taken the test, we pull the beta data and perform a detailed analysis. This includes reviewing a wide variety of statistics that help us understand how each item is performing and how various combinations of items (called forms) are performing. We also review time spent on each item and each form, along with your comments.
I use this information to determine which items we’ll keep on the exam, which items we’ll remove, and which items need further discussion with subject matter experts (SMEs).
SMEs help me finalize the test item pool by identifying items that can be fixed and the items that must be removed. They work with me to set the cut (or passing) score. I consolidate this feedback to finalize the item pool and cut score.
This takes about two weeks.
Assembling the tests
Next, we design the forms assembly specifications (the combination of items that you see when you sit for the exam). We want to ensure that every exam delivery is psychometrically equivalent, while balancing multiple other considerations.
This takes about two weeks.
Building, testing, and publishing the tests
After we complete the forms assembly specifications for the live version of the exam, we’re ready to build, test, and publish the exam. We work with one of our exam delivery partners to build the exam “package.” This also entails setting up the exam in their system and testing it to ensure that everything (including scoring) is functioning as expected. My team also reviews this pre-published version. Then the exam is published by the exam delivery provider into their channel so that you can take the exam either at a test center or through online proctoring.
This takes about two weeks.
Rescoring beta tests after the exam goes live
During the week after the exam is live, we rescore all the beta tests. (We refer to this as “rescore” because each beta candidate is rescored on ONLY the items we keep on the exam or that didn’t undergo significant changes during the process described above.) We validate the rescore exam specs, review them for accuracy, and then apply the specs to each beta delivery. We conduct multiple quality reviews during this process. Every beta candidate’s test is compared to the list of items that were kept in the pool, and the cut score applied to determine pass/fail status. As soon as the steps are complete, beta candidates are notified of their results.
This takes about a week.
Note about certification expiration dates when you pass a beta exam
If you pass a beta exam that leads to a certification, the certification earned date that appears on your transcript will be the date that the exam goes from beta to live. Your certification expiration will be based on the certification earned date. In the past, we awarded certifications based on the beta test date, meaning your certification earned date would be before the exam went live and you would need to renew your certification a few months after getting your result. By using the live date, we are ensuring you get the most time possible before needing to renew. Your transcript will continue to include the date you took the beta exam, so you can still demonstrate that you participated in the beta exam if so desired.
Thanks for your help
We hope this gives you some insight into our beta exam process, and why we take such care (and so much time) to get it right. At every step, we keep our quality bar high. The result is an exam that the industry can rely on. We appreciate your role in helping to make our process successful, and we thank you for your feedback and insight...and your patience in getting your beta exam result.
