Empirical Reasearch Project
- 1. 1 Academic Proficiency Rates in Minnesota Emerald J. Peltier Undergrad Research, St. Cloud State University, St. Cloud, MN 56301-4498, United States Abstract: The purpose of this paper is to show which factors affect academic proficiency rates in Minnesota public schools. It measures four exogenous variables against one endogenous variable, math proficiency rates1, in 325 public school districts between the years 2012-2016. The paper will determine what each variable’s effect is on students’ proficiency and whether or not that variable is significant in determining math proficiency rates. It should be noted that this paper will focus on third grader’s test results as the measurement. Introduction Academic proficiency compels me not only because I have young children who attend elementary school in Minnesota, but also because I feel it is highly important for all children to receive the very best education that we, as a society, can provide to them. This topic is important to understand in its entirety so that we may ensure that we are doing everything possible to maximize our children’s full academic potential. Gathering and reporting data has been a mandated process for schools for many years but many superintendents have concluded that gathering and reporting is typically as far as it goes. What good is gathering data if we don’t analyze it? (American Association of School Administrators). My ambitions are to analyze some of the data reported by public schools in order to see what the effects are on student’s success rates, as measured by the Minnesota 1 I use math proficiency rates as a measureof overall proficiency because math is one of the corecurriculums.High competency in math is correlated with success notonly academically,butpersonally as well,such as higher levels of employability (Child Trends Databank).
- 2. 2 Comprehensive Assessment (MCA)2 math tests. I will observe how different economic backgrounds affect a student’s performance as well as how well the school is utilizing its state funds to improve student’s comprehensive math scores. I will also look at the relationship between enrollment and math proficiency and also between reading proficiency and math proficiency. My goal is to employ the data derived from each school district and conclude whether or not those school districts, overall, are operating at or close to their maximum potential. I will also extrapolate which factors have a positive effect on proficiency rates and which have a negative effect as well as how great those effects may be. Literature Review Papke (2005) evaluates the effects that per student spending has on student performance. Papke does this through examining fourth grader’s test rates through the Michigan Educational Assessment Program (MEAP)3 exams. In her model, she also uses factors that account for economic gaps and enrollment gaps in order to overly identify her model. She concludes that there is approximately a one to two percent increase in proficiency per ten percent increase in spending. Furthermore the increase in proficiency is even greater for those districts that were initially underperforming. Papke focuses her research on the pass rates of fourth grade math tests. While my research is focused on pass rates of third grade math tests because that is the more consistent and prevalent data for Minnesota. Using math proficiency rates as a measure of overall proficiency is acceptable because proficiency in math is reciprocal with overall academic success as well as life-long success such 2 The MCA tests are a comprehensive set of tests administered every year to measure whether students meet the academic standards setfor their particular grade.These tests not only measure student’s performance, but schools and districts as well which helps indicatehowwell a school has aligned their curriculumwith their students. 3 Exams that are very similar to those of the MCA tests.
- 3. 3 as high employability rates, according to the Child Trends Databank (2015). I retrieve these math test results on the Minnesota Department of Education’s (2016) website. According to the Minnesota Department of Education (MDE), the MCA tests are administered every year as a measurement of student comprehension as compared to the Minnesota Academic Standards that specify the benchmarks children should be achieving at their specific age. Studies have shown that children who are raised in poverty emerge from school with a lower level of academic attainment than those children not raised in poverty. This is modeled and explained in a report by Goodman et al. (2010). Goodman describes the findings unveiled about children raised in poverty and how that upbringing affects their academic attainment. Kids who grow up with this background tend to get less medical care which leads to being sick and missing school more often than the average student. These children have limited access to educational material at home and they do not receive the academic support at home that children raised in higher level income homes acquire. Behr et al. (2004) also examines the mean income level as a dependent variable and studies the levels of education in each quintile. Their findings were that as median incomes increased the level of education also tended to increase. Murray et al. (1998) investigates how school finance systems affect academics and what the best methods of funding are. They concluded that court-induced reforms reduced the achievement gap and increased academic proficiency overall. Thus, increased spending – such as adding an extra teacher to each grade level, more up to date materials, and better technologies – within each school district produces better academic performance. They also concluded that by
- 4. 4 allowing courts to control the distribution of financial resources, the states could better control the achievement gap4 by allocating their resources where needed most. According to Rios (1998) enrollment can play a major factor in student proficiency. He has concluded that higher enrollment suggests a higher student to teacher ratio. The ideal class size would be no more than 18 students per teacher and any more than this reduces student’s progress academically and behaviorally. Mosteller (1995) analyzes a rare cross-sectional data set, the Tennessee Student-Teacher Achievement Ratio (STAR)5 class size experiment, to measure student’s proficiency through time. Mosteller concluded that the schools who were given a smaller class size ended up with higher comprehension rates than other schools with higher class sizes. He also concluded that decreasing class sizes in schools which had a lower socioeconomic status (and also had previously fared much lower than their counterparts) improved their rank immensely. In the short literature review by Fite (2002), he assessed many literature works that gauged how reading affected math comprehension. Through his research, Fite noted that many arithmetic and algebraic equations involve a quantitative amount of comprehensive reading. This inferred that a student who has a low reading comprehension would have difficulty doing well on arithmetic and algebraic math. He also concluded that students who do well in reading also tend to do well in math. This further promotes the need for comprehensive reading in order to better grasp mathematical concepts. 4 The achievement gap is the observed disparity of academic performance among groups defined be socioeconomic status. 5 This was a revolutionary study conducted in Tennessee that was created to measure how the student-teacher ratio affected student’s comprehension.
- 5. 5 The Model The empirical model is represented below: MATH = c + b1 FRL + b2 EXPEND/ENROL + b3 ENROL + b4 READ + e The data used in this model is derived from the time period of 2012-2016 from the Minnesota Department of Education. The endogenous variable (MATH) in the model is the percentage of students meeting or exceeding standards in 3rd grade math as demonstrated through the MCA test. MATH is represented by the term ‘c’ and with all else equal to zero, that is where MATH crosses the axis; constituting what percent of students were found to be proficient in the math tests given in Minnesota from 2012 to 2016. The term ‘e’ represents the residual6. We measure the endogenous variable by four exogenous variables. The first variable being free and reduced lunch participation percentages (FRL). This variable is representative of the economic dissimilarities that exist between school districts. Measuring FRL against MATH illustrates how economic disparities affect academic proficiencies in third graders. We would expect that FRL would have a negative relationship with MATH. A negative relationship would indicate that a greater percentage of people eligible for FRL in a school district would cause the proficiency rate to decrease. Theoretically, we can conclude this expectation because generally children raised in lower income homes do not have access to the same amount of academic resources as students raised in higher income homes. These children also do not typically come from homes of parents with higher educations and therefore they do not receive the same psychological support as other students. 6 The residual,also known as the error term, represents the pieces of a model that cannotaccurately be measured or accounted for. The valueof the error term represents the margin of error that explains thevariation between the model results and the actual results.
- 6. 6 The second variable we measure is expenditures per student (EXPEND/ENROL). Expenditure per student is measured through data derived from the MDE in a spreadsheet of state funding granted per district divided by enrollment. We expect that this variable will be negatively related to our endogenous variable. Expenditure is divided by enrollment which indicates that as enrollment increases, the amount of expenditure allocated to each category decreases. This means a decreased amount of funding to allocate on resources which results in outdated materials and technology. This is easily comprehendible if you think about a school having $100 of funding to spend on resources for 10 students, this averages $10 per student. If you have that same $100 to spend on resources for 20 students, you now have half as much funding to allocate on resources per student. Our next variable is enrollment (ENROL). Enrollment is measured through the data from the MDE website and is measured in total students enrolled per district. Enrollment should have a negative relationship with math proficiency because the more students a district has, the higher student to teacher ratio it will have. Larger class sizes have been proven to result in lower academic proficiency by creating more opportunities for student distractions and reducing the one on one learning time between students and educators. Reading proficiency rates is the fourth variable measured in my model. Reading proficiency is measured through MCA reading tests and is recorded by schools to the MDE. Students who are proficient in reading tend to also succeed in math as well. It has been proven that students who succeed in reading are able to better comprehend algebraic, arithmetic, and other math problems that include word problems. Therefore, we can expect this variable to be positively related to math proficiency.
- 7. 7 Results While running this model for the first time, I measured it over one year only using the ordinary least squares (OLS) method7. After running the white test8, I discovered there was evidence of heteroscedasticity9 and tried to treat the model by weighing it using math proficiency for all grades. The model and variables were all significant but weighing the model had no effect on the heteroscedasticity that existed. Unfortunately, as long as heteroscedasticity still existed, the model was unreliable and could not be used. Another way to treat heteroscedasticity is to run the data over time. Treating the data with this method will eliminate noise and make the variance consistent within the variables and in between years. After running the model with five years of data, heteroscedasticity no longer exists, however two of the variables are not significant. When using panel data10 we need to conduct a test to discover whether there is fixed effect or random effect11. By running the Hausman12 test, we conclude that there is heterogeneity meaning we need to have a fixed effect model. After running the model using the OLS fixed effect method, the heteroscedasticity is now treated but the expenditure variable is insignificant. We then weigh expenditure by enrollment in 7 OLS is a method used to estimate the unknown parameters in a model whileminimizingthe sum of the squared residualsbetween the observed responses. 8 A test proposed by Halbert White in 1980 that tests for constantvarianceusingthe auxiliary model. If the coefficients in the auxiliary model areinsignificant,than the model is heteroscedastic. 9 Heteroscedasticity exists when the varianceof the errors is not constant; i.e. either increasingor decreasingover time. 10 Panel data, also known as cross-sectional time-series data,entails a datasetin which the behavior of the variablesareobserved across time. 11 Fixed effect allows oneintersect for each cross sectional units through the use of dummies. Random effect allows for only one intersect that is typically themean of the datasets or at leastcloseto it. 12 A test used to differentiate between fixed and random effects models in panel data.
- 8. 8 order to reduce the variation in the data. Below is the output of this model derived from the eviews program13. Table 1 OLS Estimation Output – Fixed Effect Variable Coefficient Std. Error t-Statistic Prob. C 26.0196 1.7001 15.3050 0.0000 FRL -0.0843 0.0194 -4.3559 0.0000 EXPEND/(ENROL) -0.0029 0.0023 -1.2688 0.2047 ENROL -0.0001 0.0002 -2.5159 0.0120 READ 0.8187 0.0185 44.1916 0.0000 R-squared: 0.7551 F-statistic: 12.1355 Prob(F-statistic): 0.0000 In Table 1 above, we see the results of the estimation output by way of the OLS fixed effect method. Originally, our model tested high for heteroscedasticity. In order to treat the model, we needed to pool the data and run cross sectional data over time. After doing so, there is no longer evidence of heteroscedasticity and we can now interpret the findings. The model has an R-squared of 0.7551 which indicates that 75% of the variability in the data is explained by these four variables. This is quite impressive for panel data with only four explanatory variables. The F-statistic is significant at any level meaning the results of the model did not happen by chance and all of the variables are significant and necessary to be included in the model. Our constant variable is representative of the intercept of a regression and signifies what MATH equals when all of the exogenous variables are equal to zero. So we can interpret this as, with no other factors included, 26% of third grade students are proficient in math as shown through the MCA test results. 13 Eviews is a statistical,forecasting,and modelingtool.
- 9. 9 As expected, the FRL variable has a negative effect on math proficiency rates. The coefficient is -0.08 meaning that for every 1% increase in free and reduced lunch in a school district, the proficiency rates in math decrease by 0.08%. This variable had the second biggest effect on math proficiency and is significant at any level. Expenditures per student also has a negative effect on math proficiency, reinforcing our previous expectations. The effect that expenditures has on proficiency is quite small; for every 1% increase in expenditures weighed by enrollment, there is a 0.003% decreases in math proficiency. The surprising result about this variable is how minor the effect is and the fact that it is only significant at the 20% level. This can be interpreted as an inadequate use of funding by school districts. Contrary to what both Papke and I expected, this variable has a minor effect on math proficiency rates. The variable with the smallest effect on math proficiency rates is enrollment. The effect is negative, as expected; for every 1% increase in enrollment, we see a 0.001% decrease in math proficiency. Enrollment is significant at the 1% significance level, indicating its importance in being included in the model. Though the variable had the expected effect on the model, it was not to the extent I would have expected. Reading proficiency has by far the biggest effect on math proficiency. Its effect is positive, as expected, and is significant at any level as indicated by the p-value. For every 1% increase in reading proficiency there is a 0.82% increase in math proficiency. As mentioned earlier, an increased comprehension in reading will induce an increased level of comprehension in math.
- 10. 10 Conclusion Using the ordinary least squares fixed effect method was sufficient for this model. Papke does a good job analyzing her data but her focus was to show the different effects that funding has on schools with different economic backgrounds. My study focuses on an additional factor, reading proficiency, which could also attribute to academic proficiency and better identify the model. There are definitely more variables that could be taken into account but unfortunately, while performing any study the biggest obstacle faced is finding sufficient data and in many cases, the data does not exist. Adding reading proficiency into the model proved to be crucial as the effect on math proficiency was the largest. Free and reduced lunch also had a large effect on math proficiency with enrollment and expenditures being significant but weakly effective. There is no perfect solution to increase math proficiency but it would be beneficial to look at how districts could make better use of funding in order to increase reading and math proficiency. The reason expenditures has such a minor effect could very possibly be due to the fact that the funding is not being allocated properly. Acknowledgements I would like to thank all the educators who take the time to teach our young children, you are key to these children’s futures. Thanks is also due to those involved in reporting the data, gathering the data, and organizing the data to/at the Minnesota Department of Education. Special thanks to my boyfriend, Kurt Kleespies, for helpful editorial comments on my earlier drafts and to the Economics Department at St. Cloud State University. Special thanks also to Dr. Masoud Moghaddam. Your support and guidance is indispensable and I cannot begin to thank you enough.
- 11. 11 References American Association of School Administration. 2004. Using Data to Improve Schools -What’s Working. http://www.aasa.org/uploadedFiles/Policy_and_Advocacy/ files/UsingDataToImproveSchools.pdf Behr et al. 2004. The Effects of State Public K-12 Education Expenditures on Income Distribution. National Education Association. Washington D.C. http://www.nea.org/assets/docs/HE/expenditures.pdf Child Trends Databank. 2015. Mathematics proficiency. http://www.childtrends.org/?indicators=mathematics-proficiency Fite, Gene. 2002. Reading and Math: What is the Connection? Kansas Science Teacher, Volume 14. Kansas City, Kansas. https://www.emporia.edu/dotAsset/9acbacde-104d-4b37-b13a- ffc1ec7885cb.pdf Goodman et al. 2010. Poorer Children’s Educational Attainment: How Important Are Attitudes and Behavior? https://www.jrf.org.uk/sites/default/files/jrf/migrated/files/poorer- children-education-full.pdf Minnesota Department of Education. 2016. Data Center. Roseville, Minnesota. http://education.state.mn.us/MDE/Data/index.html Mosteller, Frederick. 1995. The Tennessee Study of Class Size in the Early School Grades. The Future of Children, Volume 5. http://www.princeton.edu/futureofchildren/publications/docs/05_02_08.pdf Murray et al. 1998. Education-Finance Reform and the Distribution of Education Resources. American Economic Review 789-812. Papke, Leslie E. 2005. The Effects of Spending on Test Pass Rates: Evidence from Michigan. Journal of Public Economics 821-839. Rios, Robert J. 1998. Class Size: Does It Really Matter? John Hopkins University. http://education.jhu.edu/PD/newhorizons/Transforming%20Education/Articles/Class%20 Size/