Date of Award

Summer 8-10-2017

Document Type

Dissertation

Degree Name

Doctor of Education (EdD)

Department

Organizational Leadership

First Advisor

Keith Larick, Ed.D.

Second Advisor

Carol Anderson-Woo, Ed.D.

Third Advisor

Carlos V. Guzman, Ph.D.

Abstract

Purpose: This study explored a technological contribution to education made by the

Defense Language Institute Foreign Language Center (DLIFLC) in the formative assessment field. The purpose of this quantitative correlational study was to identify the relationship between online formative (Online Diagnostic Assessment; ODA) and summative (Defense Language Proficiency Test 5; DLPT5) assessments in foreign language instruction in Spanish, Korean, Chinese Mandarin, and Standard Arabic to determine their relationship to student success in a basic course program for adult students at the DLIFLC.

Methodology: This nonexperimental correlational study included a standard regression model to determine correlations between ODA scores and DLPT5 final scores through a Pearson product–moment correlation.

Findings: Findings were as follows: (a) Category IV languages showed higher discrimination across levels than did a Category I language; (b) the ODA has a closer relationship to the DLPT5 for reading than for listening; (c) listening scores tend to consistently fall one to two levels lower than DLPT5 at Interagency Language Roundtable (ILR) Levels 3 and 2+; and (d) both reading and listening tend to have a consistent moderate relationship between the ODA and the DLPT5 at ILR Level 2.

Conclusion: Because the literature review revealed a disconnect between theory and practice when looking at formative and summative assessments, and because research results showed that at least one ODA assessment demonstrated a higher degree of correlation (and score differentiation across ILR levels), the conclusion was that it is possible to devise assessments with dissimilar design constructs—formative and summative—but with common ILR requirements that, if designed appropriately, lead to comparable ILR results. Therefore, DLIFLC leaders are highly encouraged to devise similar ODA–DLPT5 correlations and benefit from the results of this research.

Recommendations: ODA developers and research experts need to study reasons for variance in correlation at upper ILR levels for listening as well as the differences between Category I and Category IV languages while considering (a) open-ended responses written in the English language, (b) the ODA semiadaptive features, (c) testing times, (d) differences between formative and summative assessments constructs, and (e) unique idiosyncrasies for assessing listening.

Download

Share

COinS