Dissertations / Theses on the topic 'Educational testing and measurements'

Schools of nursing around the United States take multiple measures to prepare nursing students for safe practice in today’s complex healthcare system. One area in which schools of nursing continue to struggle is the first-attempt pass rates of NCLEX-RN. Despite various ways of preparing nursing student graduates, the NCLEX-RN first-attempt pass rates for United States’ nursng schools remain sub-optimal. Because many of the efforts to increase first-attempt NCLEX-RN pass rates have been inadequate and new ways of preparing nursing students remain underexplored, the purpose of this study was to explore a computerized adaptive testing program, PassPoint, and identify any predictors for NCLEX-RN success on first attempt. The purpose was also to compare and analyze the computerized adaptive testing program, PassPoint, to a traditional preparatory testing method, Kaplan, in relation to NCLEX-RN first-attempt success in an associate degree nursing program in the midwestern United States. After employing a retrospective correlation design, a number of statistically significant relationships were noted.

This quantitative, ex post facto study examined the impact a districtwide high school one-to-one mobile technology initiative had on the number of 10 ^th grade students in the Forney Independent School District (ISD) (Texas) who passed their standardized TAKS tests during the 2010 academic year. The research study used annual public Forney ISD (Texas) data obtained from the Texas Education Agency (TEA) website. The study compared the 2009 academic year when 10^th grade students (n=520) did not have one-to-one mobile technology and the 2010 academic year when 10 ^th grade students (n=530) were able to use one-to-one mobile technology as their personal school and home device. Findings from the study determined if there was a statistically significant relationship between one-to-one mobile technology and the number of 10^th grade students who passed their standardized TAKS tests during the 2010 academic year. With a chi-square critical value of 3.84 the results of the study showed a statistically significant relationship between the number of 10^th grade students who passed their TAKS tests in 2010 with one-to-one mobile technology. In 2010, the number of 10^th grade students who passed the English Language Arts test increased by 5% (92% to 97%) with a chi-square statistic of 12.86. The number of students who passed the Math test increased by 18% (65% to 83%) with a chi-square statistic of 44.39. The number of students who passed the Science test increased by 12% (70% to 82%) with a chi-square statistic of 21.04. The number of students who passed the Social Studies test increased by 6% (90% to 96%) with a chi-square statistic of 14.79. The number of 10 ^th grade students who passed all of their TAKS tests increased by 18% (57% to 75%) with a chi-square statistic of 38.67. The theoretical framework was constructed by the dual coding theory of Alan Pavio and the multiple intelligence theory of Howard Gardner.

LIU, FU, Ph.D. Comparisons of Subscoring Methods in Computerized Adaptive Testing: A Simulation Study. (2015) Directed by Dr. Terry A. Ackerman. 187 pp. Given the increasing demands of subscore reports, various subscoring methods and augmentation techniques have been developed aiming to improve the subscore estimates, but few studies have been conducted to systematically compare these methods under the framework of computerized adaptive tests (CAT). This research conducts a simulation study, for the purpose of comparing five subscoring methods on score estimation under variable simulated CAT conditions. Among the five subscoring methods, the IND-UCAT scoring ignores the correlations among subtests, whereas the other four correlation-based scoring methods (SEQ-CAT, PC-MCAT, reSEQ-CAT, and AUG-CAT) capitalize on the correlation information in the scoring procedure. By manipulating the sublengths, the correlation structures, and the item selection algorithms, more comparable, pragmatic, and systematic testing scenarios are created for comparison purposes. Also, to make the best of the sources underlying the assessments, the study proposes a successive scoring procedure according to the structure of the higher-order IRT model, in which the test total score of individual examinees can be calculated after the subscore estimation procedure is conducted. Through the successive scoring procedure, the subscores and the total score of an examinee can be sequentially derived from one test. The results of the study indicate that in the low correlation structure, the original IND-CAT is suggested for subscore estimation considering the ease of implementation in practice, while the suggested total score estimation procedure is not recommended given the large divergences from the true total scores. For the mixed correlation structure with two moderate correlations and one strong correlation, the original SEQ-CAT or the combination of the SEQ-CAT item selection and the PC-MCAT scoring should be considered not only for subscore estimation but also for total score estimation. If the post-hoc estimation procedure is allowed, the original SEQ-CAT and the reSEQ-CAT scoring could be jointly conducted for the best score estimates. In the high correlation structure, the original PC-MCAT and the combination of the PC-MCAT scoring and the SEQ-CAT item selection are suggested for both the subscore estimation and the total score estimation. In terms of the post-hoc score estimation, the reSEQ-CAT scoring in conjunction with the original SEQ-CAT is strongly recommended. If the complexity of the implementation is an issue in practice, the reSEQ-CAT scoring jointly conducted with the original IND-UCAT could be considered for reasonable score estimates. Additionally, to compensate for the constrained use of item pools in PC-MCAT, the PC-MCAT with adaptively sequencing subtests (SEQ-MCAT) is proposed for future investigations. The simplifications of item and/or subtest selection criteria in a simple-structure MCAT, PC-MCAT, and SEQ-MCAT are also pointed out for the convenience of their applications in practice. Last, the limitations of the study are discussed and the directions for future studies are also provided.

The purpose of the current study was to test the theory of action hypothesized for the Mathematics Formative Assessment System (MFAS) based on results from a large-scale randomized field trial. Using a multilevel structural equation modeling analytic approach with multiple latent response variables decomposed across student, teacher, and school levels of clustering, the current study found evidence of effects of MFAS that were consistent with the MFAS theory of action. First, assignment to the treatment condition was associated with higher mean student mathematics performance and a higher prevalence of small group instruction compared to schools assigned to the control condition—both of which are outcomes hypothesized to result from MFAS use. Also, a positive association between teacher-level mathematics knowledge for teaching and student mathematics performance was found in the current study, which is consistent with the interrelation of constructs specified in the MFAS theory of action. However, evidence of the particular linkages of MFAS use→teacher knowledge→classroom practice→student mathematics performance and the putative cascade of effects that would substantiate the mechanisms of change posited in the MFAS theory of action were not detected in the current study. Thus, positive effects of MFAS on teacher and student outcomes were substantiated; however, as to how the effects of MFAS on teachers transfer to improved outcomes for students remains to be empirically demonstrated. Based on my review of the results from the current study and consideration of the literature on formative assessment as it relates to the design of MFAS tasks and rubrics, I discuss a proposed modification to the theory of action that specifies the addition of a direct path from MFAS use to student mathematics performance, in addition to the indirect path currently specified.

The college admission systems of the United States require the Scholastic Aptitude Test (SAT) and American College Testing (ACT) examinations. Although, some resources suggest that SAT and ACT scores give some meaningful information about academic success, others disagree. The objective of this study was to determine whether there is significant predictive validity of SAT and ACT exams for college success. This study examined the effectiveness of SAT and ACT scores for predicting college students’ first year GPA scores with a meta-analytic approach. Most of the studies were retrieved from Academic Search Complete and ERIC databases, published between 1990 and 2016. In total, 60 effect sizes were obtained from 48 studies. The average correlation between test score and college GPA was 0.36 (95% confidence interval: .32, .39) using a random effects model. There was a significant positive relationship between exam score and college success. Moderators examined were publication status and exam type with no effect found for publication status. A significant effect of exam type was found, with a slightly higher average correlation for SAT compared to ACT score and college GPA. No publication bias was found in the study.

Much research has been done on the correlations between SAT and ACT scores and student performance in college. However, research into independent school practices reveals little has been done at this level. The test many independent schools use is the ISEE (Independent School Entrance Exam). To date, no local or national research has been done on the correlation of ISEE scores and student performance. The purpose of this study was to analyze the school's historical entrance test scores and conduct research to determine how well they correlate with students' success at the school, specifically with freshmen end of year weighted GPA.

Data were gathered for all current, graduated, withdrawn, and expelled students from 2005-2012. For each student, the data included the ISEE scores in verbal reasoning, quantitative reasoning, reading comprehension, and mathematics achievement. Also gathered was each student's 9^th grade weighted cumulative GPA. Seven extracts were performed to identify students who were new freshmen to the high school beginning with 2005.

The study concluded that the test scores are not the best predictor of student success, but they are a factor to be taken into consideration. Regression analysis revealed a weak positive relationship among the four ISEE sub scores and end of year freshman weighted GPA. Combined, the four showed that 30% of the variability of GPA can be attributed to the testing results.

Not easily measured, admission decisions are a delicate balance of qualitative and quantitative measures.

The school will use the research as a guideline for admission decisions and use caution in proceeding in the overuse of test scores as a large mitigating factor. It is known that there is a relationship but decision-makers must take a holistic approach.

The No Child Left Behind Act requires all students be proficient in reading and mathematics by 2014, and students in subgroups to make Adequate Yearly Progress. One of these groups is students with disabilities, who continue to score well below their general education peers. This quantitative study identified scoring differences between disability groups on the Florida Comprehensive Assessment Test (FCAT) over a 6-year period. The percentages of students who scored at the proficient level in reading, mathematics, and writing in the fourth grade, and reading, mathematics, and science in the fifth grade were used to identify differences in 12 disability groups. All students with disabilities are combined into one category for reporting purposes and assigning school grades. Disaggregation of the special education categories revealed scoring differences between groups in all subjects and both grades. Students with speech impairments had the highest number of students scoring at the proficient level in all subjects, while students with intellectual disabilities had the fewest. The categorical rank order was identical for reading in both grades and similar in the other subjects. Students with specific learning disabilities, who constitute approximately 50% of all students with disabilities in these grades, were in the lowest five categories for both grades in reading and in fourth grade mathematics, and in the lower 50% in fifth grade mathematics and science. Recommendations included the need for alternate measures of student achievement; specifically, modified assessments, in addition to teacher evaluations and the impact on the Florida Flexibility Waiver's achievement goals.

Computer Adaptive Tests (CATs) have been used successfully with standardized tests. However, CATs are rarely practical for assessment in instructional contexts, because large numbers of examinees are required a priori to calibrate items using item response theory (IRT). Computerized Classification Tests (CCTs) provide a practical alternative to IRT-based CATs. CCTs show promise for instructional contexts, since many fewer examinees are required for item parameter estimation. However, there is a paucity of clear guidelines indicating when items are sufficiently calibrated in CCTs.

Is there an efficient and accurate CCT algorithm which can estimate item parameters adaptively? Automatic Racing Calibration Heuristics (ARCH) was invented as a new CCT method and was empirically evaluated in two studies.

Monte Carlo simulations were run on previous administrations of a computer literacy test, consisting of 85 items answered by 104 examinees. Simulations resulted in determination of thresholds needed by the ARCH method for parameter estimates. These thresholds were subsequently used in 50 sets of computer simulations in order to compare accuracy and efficiency of ARCH with the sequential probability ratio test (SPRT) and with an enhanced method called EXSPRT. In the second study, 5,729 examinees took an online plagiarism test, where ARCH was implemented in parallel with SPRT and EXSPRT for comparison.

Results indicated that new statistics were needed by ARCH to establish thresholds and to determine when ARCH could begin. The ARCH method resulted in test lengths significantly shorter than SPRT, and slightly longer than EXSPRT without sacrificing accuracy of classification of examinees as masters and nonmasters.

This research was the first of its kind in evaluating the ARCH method. ARCH appears to be a viable CCT method, which could be particularly useful in massively open online courses (MOOCs). Additional studies with different test content and contexts are needed.

An increasing number of parents are opting-out their children from high-stakes. Accountability systems in education have used students? test scores to measure student learning, teacher effectiveness, and school district performance. Students who are opted-out of high-stakes tests are not being evaluated by the state tests, making their level of achievement or proficiency unknown by the state government. The purpose of this basic interpretive qualitative study was to gain an understanding of the various reasons, factors, experiences, and personal events that led parents to opt-out their children from at least one 3rd through 8th grade high-stakes test. Data were collected using a researcher-designed semi-structured interview protocol developed using ecological approaches to systems theories and critical pedagogy theories. The study was set in New York and 10 participants were interviewed, all from different rural or small suburban school districts throughout the state. Five themes and 12 subthemes emerged from first and second cycle coding. Key findings indicated that parents decided to opt-out their children from high-stakes tests because they felt high-stakes were inappropriate and unfair. Further, parents were dissatisfied with current high-stakes testing practices. Previous 3rd through 8th grade testing procedures that allowed teachers to make and grade the state tests were seen as acceptable. Parents indicated no issue with testing. However, from a social change perspective they felt the current system of high stakes testing was used improperly to rate students, teachers, programs, and school districts, and that testing should be used to drive instruction and help struggling students. This study is beneficial for school personnel and policy makers because it provides different ways to assess student achievement.