1/11/2023 – Version C.23.01
Version C.23.01 is the third version of the Portuguese Speaking Assessment. This version was created following the deprecation of the previously used version of a speech recognition solution and the adoption of a different solution that showed better accuracy following a human rating study.
After this process to confirm the best speech recognition engine, thousands of previously administered Portuguese version C assessments were rescored with the selected model. This score data was used to update the item difficulty parameters for the test form and to create a new scoring algorithm and machine learning model in order to best predict Portuguese speaking ability.
NOTE: Which user confirms the update to this version will be recorded in the version history.
Technical Details
What’s improved
Speech recognition model now uses a new speech recognition solution that over performed the updated version of the previous solution.
Machine learning model shows very high agreement with predicted score of previous version.
8/12/2021 – Version B.21.08
Version B.21.08 consisted of some technical improvements to the assessment including a more direct and stable connection to the most recent version of IBM Watson’s Portuguese ASR for scoring. A psychometric transition from a partial credit model to a graded response model was adopted to better align with other language versions of Emmersion’s speaking tests. Data collected since Version A.19.03’s release was used to update item discrimination and threshold parameters and generate an improved scoring model.
NOTE: Which user confirms the update to this version will be recorded in the version history.
Technical Details
What’s improved
- Updated items in form to optimize scoring using the newest version of IBM Watson.
- Scoring algorithm update
What’s fixed
- Updated to the most recent version of IBM Watson’s Portuguese ASR solution.
- Removed inefficiencies in connecting to IBM Watson for Portuguese scoring.
- Adjusted model to reduce likelihood of misidentifying ability of high ability (mastery) and low ability (beginner) test takers.
7/18/2019 – Version A.19.07
Version A.19.03 was the first version for the Portuguese Speaking Assessment. We conducted a calibration pilot to determine the item and ability parameters to create an assessment form and accompanying scoring algorithm to best predict Portuguese speaking ability.
NOTE: Which user confirms the update to this version will be recorded in the version history.
Technical Details
What’s improved
- Used Rasch infit/outfit statistics to screen calibration data to the most informative items and response patterns
- Incorporated regression model to predict ability level with a high-degree of agreement