Equation | (# participants) x (% of children who would have not enrolled in any other ECE program of similar quality) x ($ total value of ECE per child) |
Explanation | This metric estimates the impact of high-quality, classroom-based ECE programs for children from birth through age 5 on lifetime earnings. Number of participants: Reported by program. Percentage of participants who would not have enrolled in any other ECE program of similar quality: [47%]. Estimated as the inverse of the share of the population ages 3 to 5 and under 185% of the federal poverty line who are enrolled in pre-K programs in the Twin Cities metropolitan area. The Constellation Fund staff computes this rate at [53%] using ACS data (U.S. Census Bureau, 2016). Total value of ECE per child: [$43,014]. Estimated by the Constellation Fund staff. Benefits per child include additional benefits to participants from the following outcomes: A. Academic progress: [$18,550] B. Health: [$12,179] C. Child welfare: [$12,285]. Benefits are discounted to present value based on the average age of participation through age 65. These benefits come from the following estimations: A. Benefits from academic progress A.1. High school: First, we estimate the impact of high-quality pre-kindergarten programs on high school graduation rates among students who do not pursue further higher education. We assume that high-quality pre-kindergarten programs boost the odds that students eventually graduate from high school by [18%]. This impact emerges from sophisticated meta-analysis conducted by McCoy et al. (2017). We assume a counterfactual high school graduation rate for low-income children of [65%] (Minnesota Compass, 2018). Thus, the net effect of ECE on high school graduation is [12%] (65% x 18% = 12%). We multiply this net effect by the percentage of students who do not enroll in college [57%]. In Minnesota, 51 percent of high school graduates enroll in college (Minnesota Department of Education, 2018). The Constellation Fund staff adjusts this number by a factor of 0.85 to account for the reduced rate of college enrollment of low-income students targeted by programs funded by the Constellation Fund. Since we do not know the difference in college enrollment rates by income in Minnesota we use national data from the National Student Clearing House (2016), that indicates that high school students from low-income schools are only 0.85 times as likely to enroll in college as students from an average school. (Low-income schools have more than 50% of students eligible for free or reduced priced lunch). The resulting percentage of low-income youth who graduate from high school but do not enroll in college is [57%] ((100% – (51% x 0.85)) = 57%). We then multiply this percentage by the difference in lifetime earnings between individuals in the Twin Cities with a high school diploma and those without a diploma [$198,729]. Finally, we multiply the resulting benefit by a high school graduation causal factor, which measures the degree to which the observed difference in earnings between high school graduates and individuals who have not graduated from high school is causal [.5] (WSIPP, 2019). (12% x 57% x $198,729 x 0.5 = $6,797). A.2. Some college: First, we estimate the impact of high-quality pre-kindergarten programs on high school graduation rates as estimated above. We multiply this net effect by the percentage of students who enroll in college [43%], as explained above. Since we do not know the difference in college graduation rates by income in Minnesota, we use the national 6-year graduation rate for student from low-income schools of 48% (National Student Clearing House, 2016). This is the average rate for 2 and 4-year colleges. The percentage of students who enroll in college but do not graduate is [52%] (100% – 48% = 52%). We then multiply this percent by the difference in lifetime earnings between individuals with some college and those who do not have a high school diploma in the Twin Cities [$311,199]. Finally, we multiply the resulting benefit by a college participation causal factor, which measures the degree to which the observed difference in earnings between individuals with some college and high school graduates with no further education is causal [.56] (WSIPP, 2019). (12% x 43% x 52% x $311,199 x 0.56 = $4,676). A.3. Associate degree: First, we estimate the impact of high-quality pre-kindergarten programs on high school graduation rates as estimated above. We multiply this net effect by the percentage of students who enroll in college [43%], as explained above. We then multiply by the proportion of low-income students who enroll in 2-year institutions [30%] (National Student Clearing House, 2016). We then multiply this by the percentage of low-income students who earn a 2-year college degree [29%] (National Student Clearing House, 2016). We then multiply this by the difference in lifetime earnings between individuals with an associate degree and those who do not have a degree in the Twin Cities [$311,051]. Finally, we multiply the resulting benefit by a college graduation causal factor, which measures the degree to which the observed difference in earnings between college graduates and high school graduates with no further education is causal [.56] (WSIPP, 2019). (12% x 43% x 30% x 29% x $311,051 x 0.56= $782) A.4. Bachelor’s degree: First, we estimate the impact of high-quality pre-kindergarten programs on high school graduation rates as estimated above. We multiply this net effect by the percentage of students who enroll in college [43%], as explained above. We then multiply by the proportion of low-income students who enroll in 4-year institutions [70%], (National Student Clearing House, 2016). We then multiply by the percent of low-income students who earn a 4-year college degree [57%], (National Student Clearing House, 2016). We then multiply this by the difference in lifetime earnings between individuals with a bachelor’s degree and those who have only completed high school in the Twin Cities [$664,609]. Finally, we multiply the resulting benefit by a college graduation causal factor, which measures the degree to which the observed difference in earnings between bachelor’s graduates and high school graduates with no further education is causal [.46] (WSIPP, 2019). (12% x 43% x 70% x 57% x $664,609 x 0.46 = $6,295) The total expected benefits from academic progress associated with ECE is: $6,797 + $4,676 + $782 + $6,295 = $18,550 B. Benefits from improved health We assume that high-quality ECE results in 0.6 additional quality-adjusted years of life (QALY) to participants (Garcia et al., 2016). Constellation assigns a value of $50,000 per QALY. The undiscounted added value of ECE from improved health is estimated as [$29,000] (0.6 QALY x $50,000 = $29,000). We then discount this benefit to present value from the average age of participation through life expectancy. C. Benefits from reduced child abuse and neglect We estimate a 39 percent reduction in out of home placements due to high-quality preschool, based on Reynolds, Rolnick, Englund & Temple (2010). We estimate that the value of preventing child abuse (in terms of QALY losses) is about $350,000, based on the findings of Peterson, C., et al. (2018). To complete the calculation below, we assume a counterfactual rate of identified child abuse of [9%], based on incidence rate of 3% reported by the Minnesota Department of Human Services and adjusted for populations of color by a factor of three. (Minnesota Department of Human Services, 2018) The benefit from reduced child abuse and neglect is [$12,285] (39% x $350,000 x 9% = $12,285). |
References | García, J. L., Heckman, J. J., Leaf, D. E., & Prados, M. J. (2016). The life-cycle benefits of an influential early childhood program (Working Paper No. 22993). National Bureau of Economic Research. doi.org/10.3386/w22993 Kent, A. (2009). Vulnerable youth and the transition to adulthood: Youth from low-income families. ASPE Research Brief. United States Department of Health and Human Services. Retrieved from https://aspe.hhs.gov/basic-report/vulnerable-youth-and-transition-adulthood-youth-low-income-families McCoy, D. C., Yoshikawa, H., Ziol-Guest, K. M., Duncan, G. J., Schindler, H. S., Magnuson, K., … Shonkoff, J. P. (2017). Impacts of early childhood education on medium- and long-term educational outcomes. Educational Researcher, 46(8), 474–487. doi.org/10.3102/0013189X17737739 Minnesota Compass (2018). Education: High school graduation. High school students graduating on time by income. Retrieved from http://www.mncompass.org/education/high-school-graduation#7-6108-d Minnesota Department of Education. (2018). Minnesota report card. Retrieved from http://rc.education.state.mn.us/# Minnesota Department of Human Services (2018). Child protection in Minnesota: Keeping children safe. Retrieved from https://edocs.dhs.state.mn.us/lfserver/Public/DHS-4735-ENG Peterson, C., Florence, C., & Klevens, J. (2018). The economic burden of child maltreatment in the United States, 2015. Child Abuse and Neglect, 86, 178–1. https://doi.org/10.1016/j.chiabu.2018.09.018 Reynolds, A., Rolnick, A., Englund, M. & Temple, J. (2010). Early childhood development and human capital. In A. Rolnick, A. Reynolds, M. Englund & J. Temple (Eds.), Childhood programs and practices in the first decade of life: A human capital integration. (pp. 1-27). U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-Cost Technical Documentation. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
CONSTELLATION METRICS DATABASE
Constellation is in the process of finalizing its Technical Methodology Report for publication and, once completed, it will be posted here. In the meantime, if you have any questions about the technical aspects of our Metrics Evaluation Framework, please email them to metrics@constellationfund.org
EDU001 High-quality early childhood education (ECE) programs leading to lifetime earnings
EDU002 Child care leading to increased earnings of parents
Equation | (# parents with participating children) x (% parents using child care solely because of program) x (Q: increase in employment due to child care) x ($ average earnings for a low-income population) x (# years of participation) |
Explanation | This metric estimates the impact of child care use on increased earnings for parents. Number of parents with participating children: Reported by program. Percentage of parents using child care solely because of program: Estimated by Constellation staff. Q: Increase in employment due to program: [0.09]. Estimated using the formula: Average annual earnings of employed low-income individuals: [$13,500]. Estimated using American Community Survey (ACS) 5-year estimates (U.S. Census Bureau, 2016) for the Twin Cities metropolitan area. If program data on the number of children and their ages per household is available, this can be refined to reflect the actual program population with average annual earnings of low-income population with children in specific age groups using ACS 5-year estimate Census data. Number of years of participation: Estimated by Constellation staff based on available program data. In the absence of program data, we assume one year of additional income. |
References | Blau, D. M., & Tekin, E. (2007). The determinants and consequences of child care subsidies for single mothers in the U.S.A. Journal of Population Economics, 20, 719–741. Retrieved from: https://link.springer.com/article/10.1007/s00148-005-0022-2 Cannon, J. S., Jacknowitz, A., & Painter, G. (2006). Is full better than half? Examining the longitudinal effects of full-day kindergarten attendance. Journal of Policy Analysis and Management, 25(2), 299–321. doi.org/10.1002/pam.20174 Cascio, E. U. (2009). Maternal labor supply and the introduction of kindergartens into American public schools. Journal of Human Resources, 44(1), 140–170. doi.org/10.3368/jhr.44.1.140 Matthews, H. (2006). Child care assistance helps families work: A review of the effects of subsidy receipt on employment. Center for Law and Social Policy. Retrieved from: https://www.clasp.org/publications/report/brief/child-care-assistance-helps-families-work-review-effects-subsidy-receipt Minnesota Compass. (2018). Proportion of Adults Working. Adults (16-64) below poverty level by employment status, Twin Cities 7-County Region, 1990-2018. Retrieved from: http://www.mncompass.org/workforce/proportion-of-adults-working#7-11327-d U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. |
EDU004 High school equivalence leading to lifetime earnings
Equation | (# participants) x [(% participants who pass the high school equivalence test) – (% counterfactual rate of passing high school equivalence test in a comparable population)] x {($ difference in lifetime earnings of an individual with a high school equivalence vs. no high school completion) x (% causation factor of high school on earnings) + [(% counterfactual rate of college progress of high school equivalence) x ($ difference in lifetime earnings of individuals with some college vs. a high school equivalence with no further education) x (% causation factor of some college on earnings)] + [(% high school equivalence holders who obtain an associate degree) x ($ difference in lifetime earnings of individuals with an associate degree vs. a high school equivalence with no further education) x (% causation factor of college on earnings)] + [(% high school equivalence holders who obtain a bachelor’s degree) x ($ difference in lifetime earnings of individuals with a bachelor’s degree vs. a high school equivalence with no further education) x (% causation factor of college on earnings)]} |
Explanation | This metric estimates the impact of receiving a GED diploma on lifetime earnings. It also allows for the estimation of benefits from the subsequent increased chance of enrolling in college or earning a higher educational degree. Number of participants: Reported by program. GED Percentage of participants who pass the high school equivalence test: Reported by program. Counterfactual rate of passing high school equivalence test in a comparable population: [75%]. This is based on the GED passing rate of African Americans in Minnesota in 2013 (GED Testing Services, 2014). Difference in lifetime earnings between an individual with a high school equivalence vs. no high school completion: [$125,000]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of high school on earnings: [0.5]. This is the percentage of observed earnings gains that are caused by high school graduation. This factor measures the degree to which the observed difference in earnings between individuals with a high school equivalent and those without a high school equivalent is causal (WSIPP, 2019). Some college Counterfactual rate of college enrollment without completion for individuals with a high school equivalent: [34%]. This is the proportion of GED graduates who once enrolled but were no longer enrolled at the time of the survey (Heller & Mumma, 2010). Additional lifetime earnings of individuals with some college experience vs a high school equivalent with no further education: [$186,500]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of some college on earnings: [0.56]. This is the percentage of observed earnings gains that are caused by “some college” experience. This factor measures the degree to which the observed difference in earnings between individuals with “some college” experience and those with a high school equivalent is causal (WSIPP, 2019). Associate degree Counterfactual rate of college graduation for individuals with a high school equivalent: [3%]. This is the average proportion of GED graduates who earn a 2-year degree or a certificate estimated from Heller & Mumma (2010) and Tyler & Lofstrom (2008). Difference in lifetime earnings between individuals with an associate degree vs. high school equivalence and no further education: [$186,400]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of some college on earnings: [0.56]. This is the percentage of observed earnings gains that are caused by having a two-year postsecondary degree, approximated using the causation factor from “some college” experience. This factor measures the degree to which the observed difference in earnings between individuals with an associate degree and those with a high school equivalent is causal (WSIPP, 2019). Bachelor’s degree Counterfactual rate of college graduation for individuals with a high school equivalent: [3%]. This is the average proportion of GED graduates who earn a 4-year degree or a certificate estimated from Heller & Mumma (2010) and Tyler & Lofstrom (2008). Difference in lifetime earnings between individuals with a bachelor’s degree vs. high school equivalence and no further education: [$540,000]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of college on earnings: [0.46]. This is the percentage of observed earnings gains caused by having a four-year postsecondary degree. This factor measures the degree to which the observed difference in earnings between individuals with a bachelor’s degree and those with a high school equivalent is causal (WSIPP, 2019). |
References | GED Testing Service. (2014). 2013 annual statistical report on the GED test. Retrieved from: https://docplayer.net/62213985-2012-annual-statistical-report-on-the-ged-test.html Heller, B., & Mumma, K. S. (2010). Is the GED a Viable Pathway to College for Adult Students? New Regression Discontinuity Evidence From Massachusetts. Tyler, J. H. & Lofstrom, M. (2008). Is the GED an effective route to postsecondary education for school dropouts? (Working Paper No. 13816). Cambridge, MA: National Bureau of Economic Research. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-Cost Technical Documentation. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU005 High school equivalence leading to improved health
Equation | (# participants) x (% participants who pass the high school equivalence test) – (% counterfactual rate of passing high school equivalence test in a comparable population)] x ($ per participant present discounted benefits) |
Explanation | This metric estimates the impacts of obtaining a high school diploma equivalent on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of participants: Reported by program. Percentage of participants who pass the high school equivalence test: Reported by program. Counterfactual rate of passing high school equivalence test in a comparable population: [75%]. This is the GED passing rate of African Americans in Minnesota in 2013 (GED Testing Services, 2014). QALY increase: [5.1]. We estimate that high school graduation boosts the future health status of students by 5.1 QALYs at age 85, based on the work of Muennig, et al. (2010) $ value per QALY: [$50,000]. Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | GED Testing Service. (2014). 2013 annual statistical report on the GED test. Retrieved from: https://www.gedtestingservice.com/uploads/files/5b49fc887db0c075da20a68b17d313cd.pdf Muennig, P., Fiscella, K., Tancredl, D., & Franks, P. (2010). The relative health burden of selected social and behavioral risk factors in the united states: Implications for policy. American Journal of Public Health, 100(9), 1758–1764. https://doi.org/10.2105/AJPH.2009.165019 |
EDU006 High school leading to lifetime earnings
Equation | (# participants) x [(% participants who graduate) – (% counterfactual rate of high school graduation)] x {[($ difference in lifetime earnings for high school graduates vs. no high school completion) x (% causation factor of high school on earnings)] + [(% counterfactual rate of low-income high schoolers who enroll in college but do not graduate) x ($ difference in lifetime earnings of individuals with some college vs. high school with no further education) x (% causation factor of some college on earnings)] + [(% counterfactual rate of college progress – associate degree) x ($difference in lifetime earnings of individuals with an associate degree vs. high school with no further education) x (% causation factor of college on earnings)] + [(% counterfactual rate of college progress – bachelor’s degree) x ($difference in lifetime earnings of individuals with a bachelor’s degree vs. high school with no further education) x (% causation factor of college on earnings)]} |
Explanation | This metric estimates the additional earnings associated with receiving a high school diploma. It also estimates benefits from the subsequent increase in the chance of enrolling or earning a higher educational degree. Number of participants: Reported by program. High school Percentage of participants who graduate from high school: Reported by program. Counterfactual rate of high school graduation: [65%]. This is the graduation rate of low-income students in the Twin-Cities (Minnesota Compass, 2018). Whenever appropriate, we use the graduation rate of low-income students (approximated by eligibility for free- or reduced-price lunch) in specific school districts. Difference in lifetime earnings between high school graduates vs. individuals with no high school completion: [$198,700]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of high school on earnings: [0.5]. This is the percentage of observed earnings gains are caused by high school graduation. This factor measures the degree to which the observed difference in earnings between high school graduates and non-high school graduates is causal (WSIPP, 2019). Some college Counterfactual rate of college enrollment without completion for individuals with a high school diploma: [25%]. We use national enrollment data for students from low-income schools to estimate college enrollment in Minnesota. In Minnesota, [51%] of high school graduates enroll in college (Minnesota Department of Education, 2018). Data from the National Student Clearing House (2016) indicates that low-income students enroll in college [15%] less than higher-income students. Thus, we estimate that [43%] of low-income students in Minnesota enroll in college. We subtract the percent of students who graduate (average of the 2- and 4-year degree program graduation rate) [48%] to obtain an estimate of the percentage of students who enroll in college but do not graduate [25%]. Difference in lifetime earnings between some college vs high school: [$99,500]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of some college on earnings: [0.56]. This is the percentage of observed earnings gains are caused by the impact of some college experience (“some college”) on earnings. This factor measures the degree to which the observed difference in earnings between individuals with some college experience and those with only a high school diploma is causal (WSIPP, 2019). Associate degree Counterfactual rate of college graduation for individuals with a high school equivalent: [6%]. We estimate this rate as follows: As shown above, [43%] of low-income students in Minnesota enroll in college, [46%] of these students enroll in 2-year institutions, and [29%] of them graduate (National Student Clearing House, 2016). Difference in lifetime earnings for individuals with an associate degree vs. a high school diploma: [$112,300]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of some college on earnings: [0.56]. This is the percentage of observed earnings gains are caused by an associate degree, which is approximated using the causation factor from “some college”. This factor measures the degree to which the observed difference in earnings between individuals with some college and those with only a high school diploma is causal (WSIPP, 2019). Bachelor’s degree Counterfactual rate of college graduation for individuals with a high school equivalent: [9%]. We estimate this rate as follows: As shown above, [43%] of low-income students in Minnesota enroll in college, [35%] of these students enroll in 4-year institutions, and [57%] of them graduate (National Student Clearing House, 2016). Difference in lifetime earnings for individuals with a bachelor’s degree vs. a high school diploma: [$465,800]. This is computed using ACS data (U.S. Census Bureau, 2016). Causation factor of college on earnings: [0.42]. This is the percentage of observed earnings gains are caused by a four-year college degree. This factor measures the degree to which the observed difference in earnings between graduates and those with only a high school diploma is causal (WSIPP, 2019). |
References | Minnesota Compass (2018). Education: High school graduation. High school students graduating on time by income. Retrieved from http://www.mncompass.org/education/high-school-graduation#7-6108-d Minnesota Department of Education. (2018). Minnesota report card. http://rc.education.state.mn.us/# Minnesota Office of Higher Education. (2017). Graduation rates: Graduation and retention rates of undergraduates in Minnesota’s postsecondary institutions. Retrieved from https://www.ohe.state.mn.us/mPg.cfm?pageID=754 National Student Clearinghouse (2016). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-Cost Technical Documentation. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU007 High school leading to improved health
Equation | (# participants) x [(% participants who graduate) – (% counterfactual rate of high school graduation)] x ($ per participant present discounted benefits) |
Explanation | This metric estimates the impacts of obtaining a high school diploma on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of participants: Reported by program. Percentage of participants who graduate from high school: Reported by program. Counterfactual rate of high school graduation: [65%]. This is the graduation rate of low-income students in the Twin-Cities (Minnesota Compass, 2018). Whenever appropriate, we use the graduation rate of low-income students (approximated by eligibility for free- or reduced-price lunch) in specific school districts. QALY increase: [5.1]. We estimate that high school graduation boosts the future health status of students by 5.1 QALYs at age 85 based on the work of Muennig, et al. (2010). $ value per QALY: [$50,000]. Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Minnesota Compass (2018). Education: High school graduation. High school students graduating on time by income. Retrieved from: http://www.mncompass.org/education/high-school-graduation#7-6108-d Muennig, P., Fiscella, K., Tancredl, D., & Franks, P. (2010). The relative health burden of selected social and behavioral risk factors in the united states: Implications for policy. American Journal of Public Health, 100(9), 1758–1764. https://doi.org/10.2105/AJPH.2009.165019 |
EDU008 Increased test scores leading to future earnings
Equation | (# participants) x (% impact of program on test scores) x ($ average lifetime earnings) |
Explanation | This metric estimates the impact of different types of educational interventions that impact test scores on lifetime earnings. Number of participating children: Reported by program. Impact of program on test scores: Estimated by Constellation’s staff using program data on improvement in test scores. If no outcome data is available, we look for evidence of the effectiveness of the specific program model. Impact of improved test scores on earnings: [10% per 1.0 effect size increase in test score] from Krueger, A.B. (2003); Levin, H., et al. (2007). We use the standard deviation of the average score on the Minnesota Comprehensive Assessment Series 3 reading test (Reading MCA-III Test) of low-income children in Minneapolis [1.9] as a proxy for effect size (Minnesota Department of Education, 2018). The estimated impact of parenting on test scores and earnings is: High-impact programs |
References | Levin, H. M., Belfield, C., Muennig, P. A., & Rouse, C. (2007). The costs and benefits of an excellent education for all of America’s children. Krueger, A. B. (2003). Economic considerations and class size. The Economic Journal, 113(485), F34-F63. Minnesota Department of Education. (2018). Subscore report. Retrieved from http://w20.education.state.mn.us/MDEAnalytics/DataTopic.jsp?TOPICID=31 using the following search criteria: District: Minneapolis School District; School: All Schools; Test: MCA-III; Year: 2017; Grade: 3; Gender: All Students; Race/Ethnicity: All Students; Category: Free/Reduced Priced Lunch. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. |
EDU009 Improved non-cognitive skills of youths leading to increased earnings
Equation | (# participants) x (% counterfactual rate of service) x (% increase in annual earnings) x (Effectiveness of program) x ($ average annual earnings for high school graduates) x (# working years) |
Explanation | This metric estimates the impact of programs that improve non-cognitive skills for youth on lifetime earnings. Non-cognitive skills include intra- and inter-personal competencies in human development related to personality, emotions, and social skills. These may include self-esteem, locus of control, and motivation, etc. Most programs focused on these types of skills also impact academic, mental health, and substance abuse outcomes. We determine all these potential outcomes and the availability of data to estimate them during the assessment process. We then apply trumping rules to avoid double counting. Number of participants who receive services: Reported by program. Counterfactual rate of non-cognitive services for youth: Determined by Constellation staff based on a landscape analysis of the program. Increase in annual earnings: [0.03]. This is the average increase in annual earnings for black and Latino youth per standard deviation change in self-esteem, locus of control, or motivation towards academic goals. This is estimated using summary data from Jones et al. (2015). For programs serving Latina females, we use [0.09]. Effectiveness of program: Estimated standard deviation using program data on pre-post assessments or the percent of participants who successfully complete the program. Average annual earnings for high school graduates: [$24,700]. This is estimated using ACS data (U.S. Census Bureau, 2016). Number of working years: Estimate from participation age to age 65. These benefits are then discounted to present value. |
References | Jones, D. E., Karoly, L. A., Max Crowley, D., & Greenberg, M. T. (2015). Considering Valuation of Noncognitive Skills in Benefit-Cost Analysis of Programs for Children. Journal of Benefit-Cost Analysis, 6(3), 471–507. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. |
EDU010 Low literacy programs for adults leading to increased annual earnings
Equation | (# participants who achieve seventh grade reading level) x (% participants who receive services solely because of program) x (% increase in earnings due to improved literacy) x ($ average annual earnings of individuals with less than 8th grade) x (# working years) |
Explanation | This metric estimates the impact of a program for adults that increases English literacy up to at least a seventh grade reading level on lifetime earnings. Participants who achieve seventh grade reading level: Reported by program. Percentage of these participants who receive services solely because of the program: Estimated by Constellation Fund staff. Average annual earnings of individuals with less than 8th grade: [$13,875]. The is the average annual earnings of individuals with less than an 8th grade education level, a proxy for low English literacy skills computed using ACS data (U.S. Census Bureau, 2016). Percentage increase in earnings due to improved literacy: [10%]. We estimate a 10 percent average earnings boost due to improved literacy based on Sum, Kirsch & Yamamoto (2004). This research indicates that individuals who improved from very low literacy levels (second- to seventh-grade equivalency) to more moderate literacy levels (seventh- to tenth-grade equivalency) experienced higher earnings. Number of working years: Estimate from participation age to age 65. These benefits are then discounted to present value. |
References | Sum, A., Kirsch, I. & Yamamoto, K. (2004). Pathways to labor market success: The literacy proficiency of U.S. adults. Princeton, NJ: Educational Testing Service, Policy Information Center. Retrieved from https://www.ets.org/research/policy_research_reports/publications/report/2004/idwc U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov |
EDU011 English as a Second Language (ESL) education leading to increased annual earnings
Equation | (# participants completing at least a year of ESL) x (% participants who get ESL classes solely because of program) x (% impact of ESL programs on annual earnings) x ($ average annual earnings of individuals who don’t speak English) x (# working years) |
Explanation | This metric the impact of a program that increases spoken English skills for individuals for whom English is not their first language (ESL) on lifetime earnings. Number of participants completing a year of ESL: Reported by program. Percentage of participants who get ESL classes solely because of program: Computed by the Constellation Fund staff. Impact of ESL programs on annual earnings: [15%] for high-quality programs and [6%] for low-quality programs. The Constellation Fund staff estimate these parameters using an ordinary least square regression with the following parameters:
The high impact of 15% is the percentage change in average annual earnings associated with going from “not speaking English at all” to “speaking well.” The low impact of 6% is the percentage change in average annual earnings associated with going from “not speaking English at all” to “speaking not well.” Average annual earnings of individuals who do not speak English: [$23,662], The is the average annual earnings of individuals who don’t speak English, are not citizens, and are under 185% of the federal poverty level computed using ACS data (U.S. Census Bureau, 2016). Number of working years: Estimate from participation age to age 65. These benefits are then discounted to present value. |
References | U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov |
EDU012 Home visiting programs leading to increased academic achievement via improved externalized behavior
Equation | (# children) x (Q1: Improvement in externalizing behavior due to the intervention) x (Q2: Impact of externalizing behavior on high school graduation) x ($ difference in lifetime earnings for high school graduates vs. no high school completion) |
Explanation | This metric estimates the impact of home visiting programs on improved externalized behavior leading to increases in academic achievement on increased lifetime earnings It should only be used for programs with a specific or primary goal of improving the behavior or mental health of children. This metric is based on a meta-analysis of the impact of a wide array of home visiting programs including the following: Healthy Families America (HFA), Family Check Up for Children, Nurse-Family Partnership (NFP), Parents as Teachers (PAT), Triple P – Positive Parenting Program®—Variants suitable for home visiting, Family Spirit, Child First, Home Instruction for Parents of Preschool Youngsters (HIPPY), Early Head Start–Home-Based Option (EHS-HBO), Play and Learning Strategies (PALS). These programs serve children from birth through age 17, and mothers or expectant mothers. If the organization works with a specific home-visiting model, the particular effect sizes of that program model will be used. Number of children: Reported by program. Q1: Improvement in externalizing behavior due to the intervention: [-0.022 preschool children; -0.038 school-age children]. This is estimated by Constellation staff using the following formula: Q2: Impact of externalizing behavior on high school graduation: [-0.079]. This is estimated by Constellation staff using the following formula: Difference in lifetime earnings between high school graduates vs. no high school completion: [$198,700]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. |
References | Baker, A. J. L., & Piotrkowski, C. S. (1996). Parents and children through the school years: The effects of the home instruction program for preschool youngsters. New York: National Council of Jewish Women, Center for the Child. Barlow, A., Mullany, B., Neault, N., Compton, S., Carter, A., Hastings, R., Billy, T., CohoMescal, V., Lorenzo, S., & Walkup, J. T. (Jan 2013). Effect of a paraprofessional home-visiting intervention on American Indian teen mothers’ and infants’ behavioral risks: A randomized controlled trial. The American Journal of Psychiatry, 170(1), 83-93. Barlow, A., Mullany, B., Neault, N., Goklish, N., Billy, T., Hastings, R., … Walkup, J. T. (2015). Paraprofessional-delivered home-visiting intervention for American Indian teen mothers and children: 3-Year outcomes from a randomized controlled trial. American Journal of Psychiatry, 172(2), 154-162. Caldera, D., Burrell, L., Rodriguez, K., Crowne, S. S., Rohde, C., & Duggan, A. (2007). Impact of a statewide home visiting program on parenting and on child health and development. Child Abuse & Neglect, 31(8), 829–852. doi:10.1016/j.chiabu.2007.02.010 Enoch, M. A., Kitzman, H., Smith, J. A., Anson, E., Hodgkinson, C. A., Goldman, D., & Olds, D. L. (2016). A prospective cohort study of influences on externalizing behaviors across childhood: Results from a nurse home visiting randomized controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 55(5), 376–382. Gardner, F., Connell, A., Trentacosta, C. J., Shaw, D. S., Dishion, T. J., & Wilson, M. N. (2009). Moderators of outcome in a brief family-centered intervention for preventing early problem behavior. Journal of Consulting and Clinical Psychology, 77(3), 543–553. Landry, S. H., Smith, K. E., & Swank, P. R. (2006). Responsive parenting: Establishing early foundations for social, communication, and independent problem-solving skills. Developmental Psychology, 42(4), 627-42. Landsverk, J., Carrilio, T., Connelly, C. D., Ganger, W., Slymen, D., Newton, R., et al. (2002). Healthy Families San Diego clinical trial: Technical report. San Diego, CA: The Stuart Foundation, California Wellness Foundation, State of California Department of Social Services: Office of Child Abuse Prevention. Lowell, D. I., Carter, A. S., Godoy, L., Paulicin, B., & Briggs‐Gowan, M. J. (2011). A randomized controlled trial of Child FIRST: A comprehensive home‐based intervention translating research into early childhood practice. Child development, 82(1), 193-208. Minnesota Department of Human Services. (2018). Children’s mental health: Transforming services and supports to better meet children’s needs (No. DHS-5051-ENG) (p. 2). Minnesota Office of Higher Education. (2017). Graduation rates: Graduation and retention rates of undergraduates in Minnesota’s postsecondary institutions. Retrieved from https://www.ohe.state.mn.us/mPg.cfm?pageID=754 Olds, D. L., Kitzman, H., Cole, R., Robinson, J., Sidora, K., Luckey, D. W., et al. (2004). Effects of nurse home-visiting on maternal life course and child development: Age 6 follow-up results of a randomized trial. Pediatrics, 114(6), 1550–1561 Pfannenstiel, J. (2015). Evaluation of the i3 validation of improving education outcomes for American Indian children. Unpublished manuscript. Overland Park, KS: Research & Training Associates, Inc. Roggman, L., Boyce, L. K., & Cook, G. (2009). Keeping kids on track: Impacts of a parenting-focused Early Head Start program on attachment security and cognitive development. Early Education & Development, 20(6) 920-942 Sanders, M. R. & Glynn, T. (1981). Training parents in behavioral self-management: An analysis of generalization and maintenance. Journal of Applied Behavior Analysis, 14(3), 223-223. Shaw, D. S., Connell, A., Dishion, T. J., Wilson, M. N., & Gardner, F. (2009). Improvements in maternal depression as a mediator of intervention effects on early childhood problem behavior. Development and Psychopathology, 21, 417–439. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Walkup, J. T., Barlow, A., Mullany, B. C., Pan, W., Goklish, N., Hasting, R., Cowboy, B., Fields, P., Baker, E. V., Speakman, K., Ginsburg, G., Reid, R. (2009). Randomized controlled trial of a paraprofessional-delivered in-home intervention for young reservation-based American Indian mothers. Journal of the American Academy of Child & Adolescent Psychiatry, 48(6), 591-601. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU013 Home visiting programs leading to increased lifetime earnings via academic achievement (High school graduation)
Equation | (# children) x (Q: Improved probability of graduating from high school due to the intervention) x ($ difference in lifetime earnings for high school graduates vs. no high school completion) |
Explanation
| This metric estimates the direct impact of home visiting leading to increased academic achievement on lifetime earnings. It is based on a meta-analysis of the impact of a wide array of home visiting programs including the following: Healthy Families America (HFA), Family Check Up for Children, Nurse-Family Partnership (NFP), Parents as Teachers (PAT), Triple P – Positive Parenting Program®—variants suitable for home visiting, Family Spirit, Child First, Home Instruction for Parents of Preschool Youngsters (HIPPY), Early Head Start–Home-Based Option (EHS-HBO), and Play and Learning Strategies (PALS). These programs serve children from birth through age 17, as well as mothers and expectant mothers. Number of children: The number of participants reported by program. Q: Improved chances of high school graduation due to the intervention: [0.11]. Constellation staff arrived at this estimate using the following formula: In this formula, ES is the effect size from a meta-analysis of prenatal care programs on low-weight births [0.63] (Constellation computations using 2 studies, one of them (Manning, et al.2010) is a meta-analysis). The base percentage is [0.28], which is the standard deviation of the graduation rate of low-income students in the Twin-Cities. Constellation staff calculations are based on data from Minnesota Compass (2018). Difference in lifetime earnings for high school graduates vs. no high school completion: [$238,200]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. This lifetime value includes $39,500 from potential higher education achievement beyond high school ($198,700 from high school + $39,500 from higher education). See Metric EDU006 for details on assumptions about estimates of higher education achievement and subsequent earnings increases as a result of increased high school completion. |
References | Levenstein, P., Levenstein, S., Shiminski, J. A., & Stolzberg, J. E. (1998). Long-term impact of a verbal interaction program for at-risk toddlers: An exploratory study of high school outcomes in a replication of the mother-child home program. Journal of Applied Developmental Psychology, 19(2), 267–285. https://doi.org/10.1016/S0193-3973(99)80040-9 Manning, M., Homel, R., & Smith, C. (2010). A meta-analysis of the effects of early developmental prevention programs in at-risk populations on non-health outcomes in adolescence. Children and Youth Services Review, 32(4), 506–519. https://doi.org/10.1016/J.CHILDYOUTH.2009.11.003 Minnesota Compass (2018). Education: High school graduation. High school students graduating on time by income. Retrieved from http://www.mncompass.org/education/high-school-graduation#7-6108-d U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU014 Home visiting programs leading to reduced child abuse and neglect and increased lifetime benefits
Equation | (# children) x (Q: Reduction in child abuse and neglect due to the intervention) x ($ benefit from reduced out of home placement) |
Explanation
| This metric estimates the impact of reduced child abuse and neglect on lifetime health, estimated in terms of quality-adjusted life years (QALY). It is based on a meta-analysis of the impact of a wide array of home visiting programs including the following: Healthy Families America (HFA), Family Check Up for Children, Nurse-Family Partnership (NFP), Parents as Teachers (PAT), Triple P – Positive Parenting Program®—Variants suitable for home visiting, Family Spirit, Child First, Home Instruction for Parents of Preschool Youngsters (HIPPY), Early Head Start–Home-Based Option (EHS-HBO), Play and Learning Strategies (PALS). These programs serve children from birth through age 17, as well as mothers and expectant mothers. If the program is specifically using the Parents as Teachers (PAT) model, EDU024 should be used instead. Note: Constellation treats child abuse/neglect and out of home placements as equivalent outcomes. In the case of this metric, we link the home visiting intervention to a reduction of child abuse and then monetize this outcome using the value of out of home placement. We don’t know the monetary cost of child abuse, this cost could be larger or smaller than the cost of out of home placement, our estimation would underestimate the effect of the program if the former is true or overestimated otherwise. Number of children: The number of participants reported by program. Q: Reduced child abuse and neglect due to the intervention: [1×10-4]. This is estimated by Constellation staff using the following formula:
Benefit from reduced child abuse and neglect: [$350,000]. This is an estimate of the value of preventing child abuse in terms of lost quality-adjusted life years, based on the findings of Peterson, et al (2015). These benefits are already discounted to present value. |
References | Gray JD, Cutler CA, Dean JG, Kempe CH. Prediction and prevention of child abuse and neglect. J Social Issues 1979;35:127–39. Honig AS, Morin C. When should programs for teen parents and babies begin? Longitudinal evaluation of a teen parents and babies program. J Primary Prev 2001;21:447–54. Huxley P, Warner R. Primary prevention of parenting dysfunction in high risk cases. Am J Orthopsychiatry 1993;63:582–8. Minnesota Compass (2018) “Wilder Research analysis of data from the Minnesota Department of Human Services, and Bridged-Race Population Estimates, U.S. Centers for Disease Control and Prevention. Retrieved from: https://www.mncompass.org/early-childhood/risk-and-reach/maltreatment-reports-filed Minnesota Department of Human Services (2018). Child protection in Minnesota: Keeping children safe. DHS-4735-ENG. Retrieved from: https://edocs.dhs.state.mn.us/lfserver/Public/DHS-4735-ENG. Olds, D. L., Eckenrode, J., Henderson, C. R., Kitzman, H., Powers, J., Cole, R., Sidora, K., Morris, P., Pettitt, L. M., Luckey, D., York, N., Eckenrode, N., & Kitzman, N. (n.d.). Long-term Effects of Home Visitation on Maternal Life Course and Child Abuse and Neglect Fifteen-Year Follow-up of a Randomized Trial. Peterson, C., Florence, C., & Klevens, J. (2018). The economic burden of child maltreatment in the United States, 2015. Child Abuse and Neglect, 86, 178–183. https://doi.org/10.1016/j.chiabu.2018.09.018 Velasquez J, Christensen M, Schommer B. Intensive services help prevent child abuse. Am J Maternal Child Nurs 1984;9:113–7. Wagner MM, Clayton SL. The Parents as Teachers program: results from two demonstrations. Future Child 1999;9:91–115. Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU015 Home visiting programs leading to increased primary care and quality-adjusted life years (QALY)
Equation | (# children) x (Q: % increase in primary care due to the intervention) x (# QALY increase) x ($ QALY) | ||||||||||||||||
Explanation | This metric estimates the impact of home visiting programs leading to increased primary care access and use on lifetime health, estimated in terms of quality-adjusted life years (QALY). This metric is based on a meta-analysis of the impact of a wide array of home visiting programs including the following: Healthy Families America (HFA), Family Check Up for Children, Nurse-Family Partnership (NFP), Parents as Teachers (PAT), Triple P – Positive Parenting Program®—Variants suitable for home visiting, Family Spirit, Child First, Home Instruction for Parents of Preschool Youngsters (HIPPY), Early Head Start–Home-Based Option (EHS-HBO), Play and Learning Strategies (PALS). These programs serve children from birth through age 17, as well as mothers and expectant mothers. Number of children: Reported by program. Q: Percentage increase in primary care due to the intervention: [1×10-4]. This is estimated by Constellation staff using the following formula: QALY increase due to treatment: [See table below]. We assume that home visiting may increase the chances of receiving primary care from the age of participation to age 18. The value of one year of access to health care is [0.07 QALY] based in the work of Muennig (2005) and Muennig, Glied & Simon (2005). The following table contains the number of QALYs for assumed or observed average ages of participating children.
$ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. | ||||||||||||||||
References | The Commonwealth Fund. (2018). Health Center Data Center. Retrieved from: https://datacenter.commonwealthfund.org/topics/children-medical-and-dental-preventive-care-visit-past-year Fergusson, D. M., Grant, H., Horwood, L. J., & Ridder, E. M. (2005). Randomized trial of the early start program of home visitation. Pediatrics, 116(6). https://doi.org/10.1542/peds.2005-0948 Muennig, P. (2005). The cost effectiveness of health insurance. American Journal of Preventive Medicine, 28(1), 59–64. Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. New York, NY: Robin Hood. Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU016 Home visiting programs leading to increased academic achievement (high school) and lifetime earnings of mothers
Equation | (# mothers) x (Q: % increased chances of graduating from high school due to the intervention) x ($ additional lifetime earnings between high school vs. no high school) |
Explanation | This metric estimates the impact of home visiting programs leading to increased maternal academic achievement on maternal lifetime earnings. It is based on a meta-analysis of the impact of a wide array of home visiting programs including the following: Healthy Families America (HFA), Family Check Up for Children, Nurse-Family Partnership (NFP), Parents as Teachers (PAT), Triple P – Positive Parenting Program®—Variants suitable for home visiting, Family Spirit, Child First, Home Instruction for Parents of Preschool Youngsters (HIPPY), Early Head Start–Home-Based Option (EHS-HBO), Play and Learning Strategies (PALS). These programs serve children from birth through age 17, as well as mothers and expectant mothers. Number of mothers: Reported by program. Q: Increased chances of graduating from high school due to the intervention: [0.13]. This is estimated by Constellation staff using the following formula: In this formula, ES is the effect size from a meta-analysis of home visiting programs on mothers’ education [0.74] (Constellation computations using 2 studies). The base percentage is [0.18], which is the standard deviation (SD) of the proportion of low-income students graduating from high school in Minnesota. We estimate the SD using a graduation rate of [65%] (Minnesota Compass, 2018) and student data from the Minnesota Department of Education (2019). Additional lifetime earnings between High school vs. no high school: [$238,200]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. This lifetime value includes $39,500 from potential higher education achievement beyond high school ($198,700 from high school + $39,500 from higher education). See Metric EDU006 for details on assumptions about estimates of higher education achievement and subsequent earnings increases as a result of increased high school completion. |
References | The Commonwealth Fund. (2018). Health Center Data Center. Retrieved from: https://datacenter.commonwealthfund.org/topics/children-medical-and-dental-preventive-care-visit-past-year Minnesota Department of Education (2019). Student graduation data retrieved from: https://public.education.mn.gov/MDEAnalytics/DataTopic.jsp?TOPICID=2 Fergusson, D. M., Grant, H., Horwood, L. J., & Ridder, E. M. (2005). Randomized trial of the early start program of home visitation. Pediatrics, 116(6). https://doi.org/10.1542/peds.2005-0948 Barnet, B., Liu, J., DeVoe, M., Alperovitz-Bichell, K., & Duggan, A. K. (2007). Home Visiting for Adolescent Mothers: Effects on Parenting, Maternal Life Course, and Primary Care Linkage. The Annals of Family Medicine, 5(3), 224–232. https://doi.org/10.1370/afm.629 Sweet MA, Appelbaum MI. Is home visiting an effective strategy? A meta-analytic review of home visiting programs for families with young children. Child Dev. 2004;75(5):1435–1456 U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU017 Home visiting programs leading to increased use of contraceptives and quality-adjusted life years (QALY) of mothers
Equation | (# mothers) x (Q: Increased use of contraceptive due to the intervention) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of home visiting programs on the increased use of contraceptives by mothers and subsequent increases in maternal lifetime health, estimated in terms of quality-adjusted life years (QALY). It is based on a meta-analysis of the impact of a wide array of home visiting programs including the following: Healthy Families America (HFA), Family Check Up for Children, Nurse-Family Partnership (NFP), Parents as Teachers (PAT), Triple P – Positive Parenting Program®—Variants suitable for home visiting, Family Spirit, Child First, Home Instruction for Parents of Preschool Youngsters (HIPPY), Early Head Start–Home-Based Option (EHS-HBO), Play and Learning Strategies (PALS). These programs serve children from birth through age 17, as well as mothers and expectant mothers. Number of mothers: Reported by program. Q: increased use of contraceptive (condoms) due to the intervention: [0.17]. This is estimated by Constellation staff using the following formula: QALY increase: [0.13]. This is the average gain in QALY from 13 contraceptive methods reported by Sonnenberg, et al, (2004). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Barnet, B., Liu, J., DeVoe, M., Alperovitz-Bichell, K., & Duggan, A. K. (2007). Home Visiting for Adolescent Mothers: Effects on Parenting, Maternal Life Course, and Primary Care Linkage. The Annals of Family Medicine, 5(3), 224–232. https://doi.org/10.1370/afm.629 Guttmacher Institute (2018). Retrieved from: https://www.guttmacher.org/sites/default/files/report_downloads/table_3_state_level_estimates_of_contra_use.pdf Sonnenberg, Frank A., et al. (2004). Costs and net health effects of contraceptive methods. Contraception, 69(6), 447-459. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU024 Home visiting programs leading to reduced child abuse and neglect and improved lifetime health – Parents as Teachers
Equation | (# children) x (Q: Reduction in child abuse and neglect due to the intervention) x ($ benefit from reduced foster care placement) |
Explanation | This metric estimates the impact of reduced child abuse and neglect on lifetime health, estimated in terms of quality-adjusted life years (QALY). It is based on a meta-analysis of the impact of the Parents as Teachers (PAT) home visiting model. Note: Constellation treats child abuse/neglect and out of home placements as equivalent outcomes. In the case of this metric, we link the home visiting intervention to a reduction of child abuse and then monetize this outcome using the value of out of home placement. We don’t know the monetary cost of child abuse, this cost could be larger or smaller than the cost of out of home placement, our estimation would underestimate the effect of the program if the former is true or overestimated otherwise. Number of children: The number of participants reported by program. Q: Reduced child abuse and neglect due to the intervention: [2×10-5]. This is estimated by Constellation’s staff using the following formula: In this formula, ES is the effect size from a meta-analysis of Parents as Teachers on child abuse and neglect is [0.061] (WSIPP, 2019). The base percentage [0.0003] is the standard deviation of the counterfactual rate of child abuse. We assume a rate of child abuse of [9%], based on an incidence rate of 3% reported by the MN Department of Human Services and adjusted for population of color by a factor of 3. (Minnesota Department of Human Services, 2018). Benefit from reduced child abuse and neglect: [$350,000]. This is an estimate of the value of preventing child abuse in terms of lost quality-adjusted life years, based on the findings of Peterson, et al (2015). These benefits are already discounted to present value. |
References | Gray JD, Cutler CA, Dean JG, Kempe CH. Prediction and prevention of child abuse and neglect. J Social Issues 1979;35:127–39. 13. Honig AS, Morin C. When should programs for teen parents and babies begin? Longitudinal evaluation of a teen parents and babies program. J Primary Prev 2001;21:447–54. Huxley P, Warner R. Primary prevention of parenting dysfunction in high risk cases. Am J Orthopsychiatry 1993;63:582–8. Minnesota Compass (2018) “Wilder Research analysis of data from the Minnesota Department of Human Services, and Bridged-Race Population Estimates, U.S. Centers for Disease Control and Prevention. https://www.mncompass.org/early-childhood/risk-and-reach/maltreatment-reports-filed Minnesota Department of Human Services (2018). Child protection in Minnesota: Keeping children safe. DHS-4735-ENG. Retrieved from https://edocs.dhs.state.mn.us/lfserver/Public/DHS-4735-ENG. Olds, D. L., Eckenrode, J., Henderson, C. R., Kitzman, H., Powers, J., Cole, R., Sidora, K., Morris, P., Pettitt, L. M., Luckey, D., York, N., Eckenrode, N., & Kitzman, N. (n.d.). Long-term Effects of Home Visitation on Maternal Life Course and Child Abuse and Neglect Fifteen-Year Follow-up of a Randomized Trial. Peterson, C., Florence, C., & Klevens, J. (2018). The economic burden of child maltreatment in the United States, 2015. Child Abuse and Neglect, 86, 178–183. https://doi.org/10.1016/j.chiabu.2018.09.018 Velasquez J, Christensen M, Schommer B. Intensive services help prevent child abuse. Am J Maternal Child Nurs 1984;9:113–7. Wagner MM, Clayton SL. The Parents as Teachers program: results from two demonstrations. Future Child 1999;9:91–115. Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf Washington State Institute for Public Policy. (2019). Retrieved from: http://www.wsipp.wa.gov/BenefitCost/Program/118 |
EDU018 Scholarships leading to academic credential (2-year/Associate’s degree)
Equation | (# students receiving a scholarship) x (Q: % earning a higher education degree due to the intervention) x ($ additional lifetime earnings from a 2-year degree vs. high school completion) x (% causation factor of a college degree on earnings) |
Explanation | This metric estimates the impact of education scholarships on the likelihood of receiving an associate degree, leading to increased lifetime earnings. Number of students receiving scholarship: Reported by program. Q: Percentage earning a higher education degree due to the intervention: [0.05]. This is estimated by Constellation Fund staff using the following formula: In this formula, ES [0.17] is the effect size from a meta-analysis of higher education scholarship programs on the rate of graduation at 2-year higher education institutions. The effect size is measured as a percent increase. The base percentage [29%] is the average graduation rate for 2-year institutions estimated using data from the National Student Clearinghouse (2016). Additional lifetime earnings from a 2-year degree vs. high school completion: [$112,300]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of a college degree on earnings: [0.56]. This is the percentage of observed earnings gains caused by an associate degree, which is approximated using the causation factor for some college experience (“some college”). This factor measures the degree to which the observed difference in earnings between types of individuals with an associate degree and those with only a high school diploma is causal (WSIPP, 2019). |
References | Bartik, T. J., Hershbein, B., & Lachowska, M. (2019). The Effects of the Kalamazoo Promise Scholarship on College Enrollment and Completion. Journal of Human Resources, 0416-7824R4. https://doi.org/10.3368/jhr.56.1.0416-7824r4 Cohodes, S. R., & Goodman, J. S. (2014). Merit Aid, College Quality, and College Completion: Massachusetts’ Adams Scholarship as an In-Kind Subsidy. American Economic Journal: Applied Economics, 6(4), 251–285. https://doi.org/10.1257/app.6.4.251 National Student Clearinghouse (2016). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU019 Scholarships leading to academic credential (4-year/Bachelor’s degree)
Equation | (# students receiving a scholarship) x (Q: % earning a higher education degree due to the intervention) x ($ additional lifetime earnings from a 4-year degree vs. high school completion) x (Causation factor of a college degree on earnings) |
Explanation | This metric estimates the impact of education scholarships on the likelihood of receiving a bachelor’s degree, leading to increased lifetime earnings. Number of students receiving scholarships: Reported by program. Q: Percentage earning a higher education degree due to the intervention: [0.14]. This is estimated by Constellation Fund staff using the following formula: In this formula, ES [0.25] is the effect size from a meta-analysis of higher education scholarship programs on the rate of graduation at 4-year higher education institutions. The effect size is measured in percent increases. The base percentage [0.57] is the average graduation rate at 4-year postsecondary institutions estimated using data from the National Student Clearinghouse (2016). Additional lifetime earnings from a 4-year degree vs. high school completion: [$465,900]. This is computed using ACS data (U.S. Census Bureau, 2016). These benefits are already discounted to present value. Causation factor of college on earnings: [0.46]. This is the percentage of observed earnings gains caused by a four-year college degree. This factor measures the degree to which the observed difference in earnings between types of graduates and non-graduates is causal (WSIPP, 2019). |
References | Bartik, T. J., Hershbein, B., & Lachowska, M. (2019). The Effects of the Kalamazoo Promise Scholarship on College Enrollment and Completion. Journal of Human Resources, 0416-7824R4. https://doi.org/10.3368/jhr.56.1.0416-7824r5 Benjamin L. Castleman, & Long, B. T. (2016). Looking Beyond Enrollment : The Causal Effect Of Need-Based Grants On College Access, Persistence, And Graduation. Benjamin L . Castleman Harvard Graduate School of Education Bridget Terry Long Harvard Graduate School of Education and NBER. Journal of Labor Economics, 34(4), 1023–1073. Cohodes, S. R., & Goodman, J. S. (2014). Merit Aid, College Quality, and College Completion: Massachusetts’ Adams Scholarship as an In-Kind Subsidy. American Economic Journal: Applied Economics, 6(4), 251–285. https://doi.org/10.1257/app.6.4.251 Goldrick-Rab, S., Kelchen, R., Harris, D. N., & Benson, J. (2016). Reducing Income Inequality in Educational Attainment: Experimental Evidence on the Impact of Financial Aid on College Completion. American Journal of Sociology, 121(6), 1762–1817. https://doi.org/10.1086/685442 National Student Clearinghouse (2016). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ Scott-Clayton, J. (2009). On Money and Motivation: A Quasi-Experimental Analysis of Financial Incentives for College Achievement. 60. http://files/1612/Scott-Clayton – On Money and Motivation A Quasi-Experimental Anal.pdf U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU020 Counseling or Coaching on campus leading to a higher education credential
Equation | (# participants) x (Q: % earning a higher education degree due to the intervention) x ($ additional lifetime earnings from a postsecondary degree vs. high school completion) x (Causation factor of a college degree on earnings) |
Explanation | This metric estimates the impact of counseling or coaching on postsecondary campuses on the likelihood of postsecondary persistence or receiving a postsecondary degree, leading to increased lifetime earnings. Counseling and coaching include, but are not limited to orientation attendance, intensive advising, help in defining clear academic plans, and one-on-one coaching. Number of participants: Reported by program. Q: Percentage earning a higher education degree due to the intervention: [0.05]. This is Estimated by Constellation Fund staff using the following formula: In this formula, ES [0.10] is the effect size from a meta-analysis of counseling or coaching on the rate of graduation at any level of higher education. The effect size is measured in percent increase. The base percentage [48%] is the average graduation rate for 2 and 4-year institutions for low-income students estimated using data from National Student Clearinghouse (2016). Additional lifetime earnings from a postsecondary degree vs. high school completion: [$217,000]. This is the weighted average of additional lifetime earnings of associate and bachelor’s degree holders computed using ACS data (U.S. Census, 2017). These benefits are already discounted to present value. Causation factor of college on earnings: [0.46]. This is the percentage of observed earnings gains caused by a higher education degree, based on an average of individuals who some college experience and a 4-year degree. This factor measures the degree to which the observed difference in earnings between college graduates and individuals with a high school diploma is causal (WSIPP, 2019). |
References | Barkley, A. P. (2010). “Academic Coaching” for Enhanced Learning, Higher Levels of Student Responsibility, and Greater Retention Producer Expectations and the Extensive Margin in Grain Supply Response View project. https://www.researchgate.net/publication/254384174 Bettinger, E. P., & Baker, R. B. (2014). The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Advising. Educational Evaluation and Policy Analysis, 36(1), 3–19. https://doi.org/10.3102/0162373713500523 Hatch, D. K., & Garcia, C. E. (2017). Academic advising and the persistence intentions of community college students in their first weeks in college. Review of Higher Education, 40(3), 353–390. https://doi.org/10.1353/rhe.2017.0014 National Student Clearinghouse (2016). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ Pechac, S., & Slantcheva-Durst, S. (2019). Coaching Toward Completion: Academic Coaching Factors Influencing Community College Student Success. Journal of College Student Retention: Research, Theory and Practice. https://doi.org/10.1177/1521025119869849 Sneyers, E., & De Witte, K. (2018). Interventions in higher education and their effect on student success: a meta-analysis. Educational Review, 70(2), 208–228. https://doi.org/10.1080/00131911.2017.1300874 U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU021 Counseling or coaching for high school students leading to a higher education credential
Equation | (# participants) x (Q: % earning a higher education degree due to the intervention) x ($ additional lifetime earnings from a higher education degree vs. high school completion) x (Causation factor of a college degree on earnings) |
Explanation | This metric estimates the impact of counseling or coaching for high school students on the likelihood of postsecondary persistence or receiving a postsecondary degree, leading to increased lifetime earnings. Number of participants: Reported by program. Q: Percentage earning a higher education degree due to the intervention: [0.05]. This is estimated by Constellation Fund staff using the following formula: In this formula, ES [0.10] is the effect size from a meta-analysis of availability and expenditures in student services (including academic and non-academic supports) on the rate of graduation at any level of higher education. The effect size is measured in percent increase. The base percentage [48%] is the average graduation rate at 2- and 4-year postsecondary institutions estimated using data from the National Student Clearinghouse (2016). Additional lifetime earnings from a higher education degree vs. high school diploma: [$217,000]. This is the weighted average of additional lifetime earnings of associate and bachelor’s degree holders computed using ACS data (U.S. Census, 2017). These benefits are already discounted to present value. Causation factor of college on earnings: [0.56]. This is the percentage of observed earnings gains caused by a higher education degree. This factor measures the degree to which the observed difference in earnings between college graduates and individuals with only a high school completion is causal (WSIPP, 2019). |
References | Bettinger, E. P., Long, B. T., Oreopoulos, P., & Sanbonmatsu, L. (2012). The role of application assistance and information in college decisions: Results from the h&r block fafsa experiment. Quarterly Journal of Economics, 127(3), 1205–1242. https://doi.org/10.1093/qje/qjs017 Carruthers, C. K., & Fox, W. F. (2016). Aid for all: College coaching, financial aid, and post-secondary persistence in Tennessee. Economics of Education Review, 51, 97–112. National Student Clearinghouse (2016). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ Rodríguez-Planas, N. (2012). Longer-Term Impacts of Mentoring, Educational Services, and Learning Incentives: Evidence from a Randomized Trial in the United States. American Economic Journal: Applied Economics, 4(4), 121–139. https://doi.org/10.1257/app.4.4.121 Stephan, J. L., & Rosenbaum, J. E. (2013). Can High Schools Reduce College Enrollment Gaps with a New Counseling Model? Educational Evaluation and Policy Analysis, 35(2), 200–219. https://doi.org/10.3102/0162373712462624 U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
EDU022 Non-academic student support services leading to a higher educational credential (2-year/Associate degree)
Equation | (# participants) x (Q: % earning a higher education degree due to the intervention) x ($ additional lifetime earnings of higher education degree vs. high school) x (Causation factor of a college degree on earnings) |
Explanation | This metric estimates the impact of non-academic student support services for students in postsecondary programs on the likelihood of receiving a postsecondary degree, leading to increased lifetime earnings. This metric is based on multifaceted, integrated programs that directly address many of the barriers to academic success faced by low-income and minority students. The program may include the following components: frequent and comprehensive advising, active messaging to students to encourage enrollment following college acceptance, frequent and comprehensive career and employment services, and financial support. Number of participants: Reported by program. Q: Percentage earning an associate degree due to the intervention: [0.10] This is estimated by Constellation staff using the following formula: Student service expenditures per student: Reported by program. Difference in lifetime earnings between a higher education degree vs. high school equivalence: [$217,000]. This is the weighted average of additional lifetime earnings of associate and bachelor’s degree holders computed using ACS data (U.S. Census, 2017). These benefits are already discounted to present value. Causation factor of college on earnings: [0.56]. This is the percentage of observed earnings gains caused by a higher education degree. This factor measures the degree to which the observed difference in earnings between types of graduates and non-graduates is causal (WSIPP, 2019). |
References | National Student Clearinghouse (2016). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ U.S. Census Bureau. (2017). American Community Survey 5-year estimates – public use microdata sample, 2013-2017. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2019). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf Weiss, M. J., Ratledge, A., Sommo, C., & Gupta, H. (2019). Supporting community college students from start to degree completion: Long-term evidence from a randomized trial of CUNY’s ASAP. American Economic Journal: Applied Economics, 11(3), 253–297. https://doi.org/10.1257/app.20170430 |
EDU023 Non-academic student support services leading to a higher educational credential (4-year/Bachelor’s degree)
Equation | (# participants) x (Q: % earning a higher education degree due to the intervention) x ($ additional lifetime earnings of higher education degree vs. high school) x (Causation factor of a college degree on earnings) |
Explanation | This metric estimates the impact of non-academic, student support services for students in postsecondary programs on the likelihood of postsecondary persistence or receiving a postsecondary degree, leading to increased lifetime earnings. This metric is based on multifaceted, integrated programs that directly address many of the barriers to academic success faced by low-income and minority students. The program may include the following components: frequent and comprehensive advising, active messaging to students to encourage enrollment following college acceptance, frequent and comprehensive career and employment services, and financial support. Number of participants: Reported by program. Q: Percentage earning an associate degree due to the intervention: [0.02] This estimated by Constellation staff using the following formula: Student service expenditures per student: Reported by program. Difference in lifetime earnings between associate degree vs. high school equivalence: [$465,900] This is estimated using ACS data (U.S. Census, 2017). These benefits are already discounted to present value. Causation factor of college on earnings: [0.46]. This is the percentage of observed earnings gains caused by a four-year college degree. This factor measures the degree to which the observed difference in earnings between college graduates and individuals who only completed high school is causal (WSIPP, 2019). |
References | National Student Clearinghouse (2019). National College Progression Rates. Retrieved from: https://nscresearchcenter.org/hsbenchmarks2016/ U.S. Census Bureau. (2017). American Community Survey 5-year estimates – public use microdata sample, 2013-2017. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2016). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf Weiss, M. J., Ratledge, A., Sommo, C., & Gupta, H. (2019). Supporting community college students from start to degree completion: Long-term evidence from a randomized trial of CUNY’s ASAP. American Economic Journal: Applied Economics, 11(3), 253–297. https://doi.org/10.1257/app.20170430 |
HEA001 Prenatal programs leading to reduced low-weight births (program data on low-weight births available)
Equation | (# pregnant women) x (% mothers getting assistance solely because of the program) x [(% participants giving birth to low-weight babies) / (% mothers who typically give birth to a low-weight baby)-1] x (# QALY increase) x ($ QALY)) |
Explanation | Enhanced prenatal care programs delivered through Medicaid provide non-clinical services that support maternal wellness and infant health during the prenatal period, such as care coordination, health education, risk assessment, psychosocial support, or nutritional counseling. These programs are delivered in a primary healthcare setting and provided by either a nurse or a social worker. Women are eligible for these programs during pregnancy, with some benefits continuing through the first twelve months postpartum. Participants typically receive program benefits for three to sixteen months, including both prenatal and postpartum services. Number of pregnant women: Reported by program. Percentage of women getting assistance solely because of the program: Estimated by Constellation Fund staff. Percentage of participants giving birth to low-weight babies: Reported by program. Percentage of women who will typically give birth to low-weight baby: [See below]. The appropriate counterfactual is selected by Constellation staff based on reported characteristics of program participants and their community. Babies born at low birth weight in single births (Minnesota Compass, 2018): · [5.2%] All mothers in the Twin-Cities (2016) · [6.8%] Mothers of color in the Twin Cities (2016) · [7.7%] Black mothers in the Twin Cities (2016) · [20%] Poor neighborhoods Other possible sources to estimate baseline rates range from [7%-20%] based on several studies. The rate of low birth weight is about 15 percent across all education levels for black mothers, rising to 20 percent in poor neighborhoods, and is about 7 percent among all Latina mothers (Collins, Wambach, David & Rankin, 2009; Elo et al., 2009; Hamilton, Martin & Ventura, 2010). QALY increase: [2.8]. This is the estimate for the value of avoidance of low birth weight based on the work of Johnson & Shoeni (2007), which finds that children born at less than 5.5 pounds are more likely to experience poor health in childhood and adulthood even after accounting for many covariates. Technical note: QALY impact estimated by Constellation staff using results from Johnson & Shoeni (2007) as follows: the authors estimate the impact of LBW on health quality index (100pts) to be 3.77. We assume a life expectancy of 75 years, resulting in an impact of 2.8 QALYs [3.77 x 75/100]. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Minnesota Department of Health (2017). Center for Health Statistics. Retrieved from https://pqc.health.state.mn.us/mhsq/index.jsp Collins, J. W., Jr., Wambach, J., David, R. J. & Rankin, K. M. (2009). Women’s lifelong exposure to neighborhood poverty and low birth weight: A population study. Maternal and Child Health Journal, 13(3), 326–333. Elo, I., Culhane, J., Kohler, I., O’Campo, P., Burke, J., Messer, L., Kaufman, J., Laraia, B., Eyster, J., and Holzman, C. (2009). Neighbourhood deprivation and small-for-gestational term births in the United States. Paediatric and Perinatal Epidemiology, 23(1), 87–96. Hamilton, B. E., Martin, J. A., & Ventura, S. J. (2010). Births: preliminary data for 2009. National Vital Statistics Reports: From the Centers for Disease Control and Prevention, National Center for Health Statistics. 59(3), 1–19. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/25073731 Johnson, R. C., & Schoeni, R. F. (2011)., and The influence of early life events on human capital, health status, and labor market outcomes over the life course. The B.E. Journal of Economic Analysis & Policy, 11(3). |
HEA003 Prenatal programs leading to reduced infant deaths
Equation | (# pregnant women) x (% mothers getting assistance solely because of the program) x (Q: % avoided infant deaths due to the intervention) x ($ value per life saved) |
Explanation | Enhanced prenatal care programs delivered through Medicaid provide non-clinical services that support maternal wellness and infant health during the prenatal period, such as care coordination, health education, risk assessment, psychosocial support, or nutritional counseling. These programs are delivered in a primary healthcare setting and provided by either a nurse or a social worker. Women are eligible for these programs during pregnancy, with some benefits continuing through the first twelve months postpartum. Participants typically receive program benefits for three to sixteen months, including both prenatal and postpartum services. Number of pregnant women: Reported by program. Percentage of women getting assistance solely because of the program: Estimated by Constellation Fund staff. Q: Percentage of avoided infant death due to the intervention: [0.00067]. This is estimated by Constellation staff using the following formula: In this formula, ES is the effect size from meta-analysis of prenatal care programs on infant mortality [-0.088] (WSIPP, 2017). The base percentage is [0.5%], which is the infant mortality rate in Minnesota (Minnesota Department of Health, 2017). $ value per life saved: We estimate the value of a life based on a [$50,000] QALY. This value varies by the age and expected years of life of each participant. Thus, we compute total benefits of a program based on specific program data on participants age and discount the annual value to present value using Constellation’s standard discounting method. |
References | Arima, Y., Guthrie, B.L., Rhew, I.C., & De Roos, A.J. (2009). The impact of the First Steps prenatal care program on birth outcomes among women receiving Medicaid in Washington State. Health Policy (Amsterdam, Netherlands), 92(1), 49-54. Buescher, P.A., Roth, M.S., Williams, D., & Goforth, C.M. (1991). An evaluation of the impact of maternity care coordination on Medicaid birth outcomes in North Carolina. American Journal of Public Health, 81(12), 1625-9. Hillemeier, M.M., et al. (2015). Effects of maternity care coordination on pregnancy outcomes: propensity-weighted analyses. Maternal and Child Health Journal, 19(1), 121-7 Korenbrot, C., & Patterson, E. (1995). Evaluation of California’s Statewide Implementation of Enhanced Perinatal Services as Medicaid Benefits. Public Health Reports, 110(2), 125-133. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1382091/ Minnesota Compass (2010). https://www.mncompass.org/_pdfs/presentations/BlueCross_HealthInequities_10-10.pdf Minnesota Department of Health (2017). Vital Statistics – Infant/Fetal Mortality. Minnesota Center for Health Statistics. Retrieved from: https://www.health.state.mn.us/data/mchs/vitalstats/infant.html Washington State Institute of Public Policy – WSIPP (2017). Retrieved from http://www.wsipp.wa.gov/BenefitCost/Program/680 |
HEA004 Primary care leading to increased quality-adjusted life years (QALY)
Equation | (# individuals receiving primary care) x (% participants getting medical services solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impacts of one year of increased access to and use of primary care on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of individuals receiving primary care: Reported by program. Percentage of participants getting medical services solely because of the program:
QALY increase due to treatment: [0.07]. This is the QALY value of one year of access to health care based in the work of Muennig (2005) and Muennig, Glied & Simon (2005). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Muennig, P. (2005). The cost effectiveness of health insurance. American Journal of Preventive Medicine, 28(1), 59–64. Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. New York, NY: Robin Hood. The Commonwealth Fund. (2018). Minnesota State Health System Ranking. Health Center Data Center. Retrieved from: http://datacenter.commonwealthfund.org/scorecard/state/25/minnesota/ Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. . Retrieved from: http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf
|
HEA002 Prenatal programs leading to reduced low-weight births (no program data on low-weight births available)
Equation | (# pregnant women) x (% mothers getting assistance solely because of the program) x (Q1: % avoided low-birthweight or preterm births due to the intervention) x (# QALY increase) x ($ QALY) |
Explanation | Enhanced prenatal care programs delivered through Medicaid provide non-clinical services that support maternal wellness and infant health during the prenatal period, such as care coordination, health education, risk assessment, psychosocial support, or nutritional counseling. These programs are delivered in a primary healthcare setting and provided by either a nurse or a social worker. Women are eligible for these programs during pregnancy, with some benefits continuing through the first twelve months postpartum. Participants typically receive program benefits for three to sixteen months, including both prenatal and postpartum services. Number of pregnant women: Reported by program. Percentage of women getting assistance solely because of the program: Estimated by Constellation Fund staff. Q1: Percentage of avoided preterm births due to the intervention: [0.05]. Estimated by Constellation’s staff using the following formula: In this formula, ES is the effect size from meta-analysis of prenatal care programs on low-weight births [-0.087] (WSIPP, 2017). The base percentage is [7%-20%], which is based on the incidence rate ranging from 7% to 20% from several studies. The rate of low birth weight is about 15% across all education levels for black mothers, rising to 20% in poor neighborhoods, and is about 7% among all Latina mothers (Collins, Wambach, David & Rankin, 2009; Elo et al., 2009; Hamilton, Martin & Ventura, 2010). QALY increase: [2.8]. This is the estimate for the value of avoidance of low birth weight based on the work of Johnson & Shoeni (2007), which finds that children born at less than 5.5 pounds are more likely to experience poor health in childhood and adulthood even after accounting for many covariates. Technical note: QALY impact estimated by Constellation staff using results from Johnson & Shoeni (2007) as follows: the authors estimate the impact of LBW on health quality index (100pts) to be 3.77. We assume a life expectancy of 75 years, resulting in an impact of 2.8 QALYs [3.77 x 75/100]. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Arima, Y., Guthrie, B.L., Rhew, I.C., & De Roos, A.J. (2009). The impact of the First Steps prenatal care program on birth outcomes among women receiving Medicaid in Washington State. Health Policy (Amsterdam, Netherlands), 92(1), 49-54. Buescher, P.A., Roth, M.S., Williams, D., & Goforth, C.M. (1991). An evaluation of the impact of maternity care coordination on Medicaid birth outcomes in North Carolina. American Journal of Public Health, 81(12), 1625-9. Collins, J. W., Jr., Wambach, J., David, R. J. & Rankin, K. M. (2009). Women’s lifelong exposure to neighborhood poverty and low birth weight: A population study. Maternal and Child Health Journal, 13(3), 326–333. Elo, I., Culhane, J., Kohler, I., O’Campo, P., Burke, J., Messer, L., Kaufman, J., Laraia, B., Eyster, J., and Holzman, C. (2009). Neighbourhood deprivation and small-for-gestational term births in the United States. Paediatric and Perinatal Epidemiology, 23(1), 87–96. Hamilton, B. E., Martin, J. A., & Ventura, S. J. (2010). Births: preliminary data for 2009. National Vital Statistics Reports: From the Centers for Disease Control and Prevention, National Center for Health Statistics. 59(3), 1–19. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/25073731 Hillemeier, M.M., et al. (2015). Effects of maternity care coordination on pregnancy outcomes: propensity-weighted analyses. Maternal and Child Health Journal, 19(1), 121-7. Johnson, R. C., & Schoeni, R. F. (2011)., and The influence of early life events on human capital, health status, and labor market outcomes over the life course. The B.E. Journal of Economic Analysis & Policy, 11(3). Korenbrot, C., & Patterson, E. (1995). Evaluation of California’s Statewide Implementation of Enhanced Perinatal Services as Medicaid Benefits. Public Health Reports, 110(2), 125-133. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1382091/ Washington State Institute of Public Policy -WSIPP (2017). Retrieved from http://www.wsipp.wa.gov/BenefitCost/Program/680 |
HEA005 Hepatitis C testing of high-risk population leading to increased quality-adjusted life years (QALY)
Equation | (# high-risk individuals tested) x (% individuals tested solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of hepatitis C testing on high-risk populations leading to improved health, estimated in terms of quality-adjusted life years (QALY). Number of individuals tested: Reported by program. Percentage of participants who are treated solely because of the program: [7.5%]. There is no high-quality proxy for a counterfactual rate of screening of hepatitis C, so we use the testing rates among commercially insured people who inject drugs (Bull-Otterson, et al., 2020). QALY increase: [0.4]. This is the average impact of screenings computed using summary results reported by Hahné et al (2013) and similar results from Nayagam et al (2017). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention. (2018). Vaccination Coverage Among Adults in the United States, National Health Interview Survey, 2016. Retrieved from: https://www.cdc.gov/vaccines/imz-managers/coverage/adultvaxview/pubs-resources/NHIS-2016.html Hahné, S. J., Veldhuijzen, I. K., Wiessing, L., Lim, T.-A., Salminen, M., & Laar, M. van de. (2013). Infection with hepatitis B and C virus in Europe: a systematic review of prevalence and cost-effectiveness of screening. BMC Infectious Diseases, 13(1). Bull-Otterson, L., Huang, Y.-L. A., Zhu, W., King, H., Edlin, B. R., & Hoover, K. W. (2020). Human Immunodeficiency Virus and Hepatitis C Virus Infection Testing Among Commercially Insured Persons Who Inject Drugs, United States, 2010–2017. The Journal of Infectious Diseases. https://doi.org/10.1093/infdis/jiaa017 Nayagam, S., Sicuri, E., Lemoine, M., Easterbrook, P., Conteh, L., Hallett, T. B., & Thursz, M. (2017). Economic evaluations of HBV testing and treatment strategies and applicability to low and middle-income countries. BMC Infectious Diseases, 17(Suppl 1). |
HEA006 Hepatitis B screening, prevention, and vaccinations leading to increased quality-adjusted life years (QALY)
Equation | (# individuals tested) x (% individuals tested/vaccinated solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of hepatitis B screening, prevention, and vaccinations leading to improved health, estimated in terms of quality-adjusted life years (QALY). Number of individuals vaccinated or tested: Reported by program. Percentage of participants who are vaccinated solely because of the program:
QALY increase: [0.66]. This is the average impact of screenings computed using summary results reported by Hahné et al (2013). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention. (2018). Vaccination Coverage Among Adults in the United States, National Health Interview Survey, 2016. Retrieved from https://www.cdc.gov/vaccines/imz-managers/coverage/adultvaxview/pubs-resources/NHIS2017.html#:~:text=In%202017%2C%20reported%20hepatitis%20B,to%20the%20estimates%20for%202016. Hahné, S. J., Veldhuijzen, I. K., Wiessing, L., Lim, T.-A., Salminen, M., & Laar, M. van de. (2013). Infection with hepatitis B and C virus in Europe: a systematic review of prevalence and cost-effectiveness of screening. BMC Infectious Diseases, 13(1). Nayagam, S., Sicuri, E., Lemoine, M., Easterbrook, P., Conteh, L., Hallett, T. B., & Thursz, M. (2017). Economic evaluations of HBV testing and treatment strategies and applicability to low and middle-income countries. BMC Infectious Diseases, 17(Suppl 1). |
HEA007 Hepatitis B/C treatment leading to increased quality-adjusted life years (QALY)
Equation | (# individuals treated) x (% individuals treated solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of hepatitis B or C treatment leading to improved health, estimated in terms of quality-adjusted life years (QALY). Number of individuals treated: Reported by program. Percentage of participants who are treated solely because of the program: [16%]. Toy (2017) reports that about 35% of individuals with hepatitis B are diagnosed in the U.S. and 45% of these diagnosed individuals are treated, resulting in a 16% treatment rate. Hepatitis B – QALY increase: [2]. This is based on an average over several different types of treatments, populations, and studies (Dakin, Bentley & Dusheiko, 2010; Kanwal et al., 2005; Veenstra et al., 2008). Hepatitis C – QALY increase: [1.9]. This is based on findings from Siebert & Sroczynski (2005) and Deniz et al. (2011). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Dakin, H., Bentley, A. & Dusheiko, G. (2010). Cost-utility analysis of tenofovir disoproxil fumarate in the treatment of chronic hepatitis B. Value in Health: The Journal of the International Society for Pharmacoeconomics and Outcomes Research, 13(8), 922–933. Deniz, B., Brogan, A. J., Miller, J. D., Talbird, S. E, Thompson, J. R., 2RTI Health Solutions & 3RTI Health Solutions. (2011). Cost-effectiveness of telaprevir combination treatment compared to pegylated- interferon + ribavirin alone in the management of chronic hepatitis C in patients who failed a prior pegylated-interferon + ribavirin treatment. Paper presented at the 62nd Annual Meeting of the American Association for the Study of Liver Diseases, San Francisco, CA. Kanwal, F., Gralnek, I. M., Martin, P., Dulai, G. S., Farid, M., & Spiegel, B. M. R. (2005). Treatment alternatives for chronic hepatitis B virus infection: a cost-effectiveness analysis. Annals of Internal Medicine, 142(10), 821–831. Siebert, U., & Sroczynski, G., German Hepatitis C Model GEHMO Group, & HTA Expert Panel on Hepatitis C. (2005). Effectiveness and cost-effectiveness of initial combination therapy with interferon/peginterferon plus ribavirin in patients with chronic hepatitis C in Germany: a health technology assessment commissioned by the German Federal Ministry of Health and Social Security. International Journal of Technology Assessment in Health Care, 21(1), 55–65. Toy, M. (2017). Population health impact and cost-effectiveness of chronic hepatitis B diagnosis, care, and treatment in the United States. Appendix A in G. J. Buckley, & B. L. Strom (Eds.) A national strategy for the elimination of Hepatitis B and C, Phase two report. Washington, D.C.: National Academies Press. Retrieved from: https://www.nap.edu/read/24731/chapter/10#204 Veenstra, D. L., Sullivan, S. D., Lai, M., Lee, C., Tsai, C. & Patel, K. K. (2008). HBeAg-negative chronic hepatitis B: Cost-effectiveness of peginterferon alfa-2a compared to lamivudine in Taiwan. Value in Health: The Journal of the International Society for Pharmacoeconomics and Outcomes Research, 11(2), 131–138. |
HEA008 Cancer (all types) screenings leading to increased quality-adjusted life years (QALY)
Equation | (# individuals screened) x (% individuals getting screened solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of cancer screening leading to improved health, estimated in terms of quality-adjusted life years (QALY). Number of individuals screened: Reported by program. Percentage of participants who are screened solely because of the program: We use screening rates from the Minnesota Health Care programs for low-income as counterfactual baselines for different types of cancer (MHCP, n/d):
QALY increase:
$ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Center for the Evaluation of Value and Risk in Health, Tufts Medical Center (n.d.). Cost Effectiveness Analysis Registry. Retrieved from: https://research.tufts- nemc.org/cear4/SearchingtheCEARegistry/SearchtheCEARegistry.aspx Heijnsdijk, E. a. M., de Carvalho, T. M., Auvinen, A., Zappa, M., Nelen, V., Kwiatkowski, M., … de Koning, H. J. (2015). Cost-effectiveness of prostate cancer screening: a simulation study based on ERSPC data. Journal of the National Cancer Institute, 107(1), 366. Mandelblatt, M., Lawrence, W., Womack, S., Yi, B., Jacobsen, D., Hwang, Y., Gold, K., Barter, J. & Shah, K. (2002). Benefits and costs of using HPV testing to screen for cervical cancer. Journal of the American Medical Association, 287(18), 2372-2381. Minnesota Health Care Programs (MHCP), (n/d) Retrieved from: https://www.health.state.mn.us/diseases/cancer/sage/about/facts.html Mittmann, N., Stout, N. K., Tosteson, A. N. A., Trentham-Dietz, A., Alagoz, O., & Yaffe, M. J. (2018). Cost-effectiveness of mammography from a publicly funded health care system perspective. CMAJ Open, 6(1), E77–E86. Tafazzoli, A., Roberts, S., Ness, R. & Dittus, R. (2005). A comparison of screening methods for colorectal cancer using simulation modeling. In M. E. Kuhl, N. M. Steiger, F. B. Armstrong & J. A. Jones (Eds.), Proceedings of the 2005 Winter Simulation Conference. Piscataway, NJ: Institute of Electrical and Electronics Engineers. Minnesota Department of Health (2017). Quick facts: Cancer screening in Minnesota. Retrieved from: http://www.health.state.mn.us/divs/healthimprovement/data/quick-facts/screening.html |
HEA009 Diabetes screenings leading to increased quality-adjusted life years (QALY)
Equation | (# individuals screened) x (% individuals getting screened solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of Type 2 diabetes screening leading to improved health, estimated in terms of quality-adjusted life years (QALY). Number of individuals screened: Reported by program. Percentage of participants screened solely because of the program: [0.1]. This is based on the Minnesota Department of Health estimate that around 1 in 10 individuals with diabetes are not diagnosed (Minnesota Department of Health, 2016). QALY increase: [0.02]. This estimate is for the screening of patients for Type 2 diabetes from the Centers for Disease Control and Prevention, Diabetes Cost-Effectiveness Group (1998) and Engelau, Narayan & Herman (2000), which indicate that the QALY for diabetes screening can, on average, be estimated at about 0.02. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention. (2017). Diabetes report card 2017. Atlanta, GA: Centers for Disease Control and Prevention, U.S. Department of Health and Human Services. Retrieved from: https://www.cdc.gov/diabetes/pdfs/library/diabetesreportcard2017-508.pdf Centers for Disease Control and Prevention, Diabetes Cost-Effectiveness Group. (1998). The cost-effectiveness of screening for Type 2 Diabetes. Journal of the American Medical Association, 280, 1757- 1763. Engelau, M., Narayan, K. & Herman, W. (2000). Screening for Type 2 Diabetes. Diabetes Care, 23(10), 1563-1580. Minnesota Department of Health (2016). Diabetes in Minnesota. Retrieved from: http://www.health.state.mn.us/divs/healthimprovement/content/documents-diabetes/2016DiabetesMN.pdf |
HEA010 Diabetes treatment leading to increased quality-adjusted life years (QALY)
Equation | (# individuals treated) x (% individuals getting treated solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of diabetes treatment leading to improved health, estimated in terms of quality-adjusted life years (QALY). Number of individuals treated: Reported by program. Percentage of participants treated solely because of the program: [0.27]. The percentage of diagnosed individuals receiving treatment in Minnesota is 73%, thus the percent not treated is 27% (CDC, 2017). QALY increase: [0.14]. This is the average QALY for best-practice diabetes treatment based on meta-analysis from Tice, et al (2016). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention. (2017). Diabetes report card 2017. Atlanta, GA: Centers for Disease Control and Prevention, U.S. Department of Health and Human Services.https://www.cdc.gov/diabetes/pdfs/library/diabetesreportcard2017-508.pdf Tice, J., Chapman, R., Shore, K., Seidner, M., Ollendorf, D., Weissberg, J. & Pearson, S. (2016). Diabetes prevention programs: Effectiveness and value. Final evidence report and meeting summary. Institute for Clinical and Economic Review. Retrieved from https://icer-review.org/material/final-report-dpp/. |
HEA011 Fluoride varnish for children leading to improved test scores and lifetime earnings
Equation | (# children) x (% children receiving care due to program) x (Q1: Impact of program on reducing caries) x (Q2: Improvement in English proficiency due to avoided caries) x (Q3: Impact on lifetime earnings from increased academic performance) x ($ lifetime earnings) |
Explanation | Fluoride varnish is a form of fluoride that temporarily adheres to the tooth in order to maintain contact between the fluoride and the tooth for several hours. In the studies we reviewed, fluoride varnish was applied every three to six months over a 12- to 36-month time period. Benefits for permanent and primary teeth are computed separately if program data for each is available. Number of participants: Reported by program. Percentage of children receiving care due to program: [39%]. 61% percent of Minnesota children age 1 through17 living in low-income households (100% of the federal poverty level) had a dental visit in the last year ([1] Minnesota Department of Health, n.d.). Q1: Impact of program on reducing cavities: [0.08 primary, 0.11 permanent]. This is estimated using the formula: In this formula, ES is the effect size from the WSIPP meta-analysis results, which is [-0.198] for primary teeth and [-0.267] for permanent teeth (WSIPP, 2014) The base percentage is [0.63], which is the share of third grade students in public schools with high population of students who are eligible for free- or reduced-price lunch who have untreated or treated tooth decay ([2] Minnesota Department of Health, n.d.). Q2: Improvement in English proficiency due to avoided cavities: [0.13]. This is the effect size estimated from an odds ratio of 1.5 from Seirawan, et al. (2012) using the formula: These authors find significant results only in the case of English proficiency. To establish a relationship between academic proficiency (test scores) and earnings we expect to observe a stronger impact on test scores. For instance, this could be a significant impact on math AND reading scores. Due to the weak evidence found on the impact of cavities on academic performance, we discount the estimated effect size by 50%. We used the adjusted formula as follows: (ln(1.5)/1.65)) x 0.5 = 0.13 Q3: Impact of improved test scores on earnings: [0.013]. This is estimated based on a 10% per 1 effect size increase in test scores from Krueger, A. B. (2003) and Levin, H., et al. (2007). We use the standard deviation of the average score on the Minnesota Comprehensive Assessment Series 3 reading test (Reading MCA-III Test) of low-income children in Minneapolis [1.9] as the benchmark effect size. Thus, the impact of improvement in English reading proficiency (test scores) is estimated as [(0.13 x 10%) /1.9] = 0.013. Average lifetime earnings of low-income individuals: [$514,000]. This is the average earnings of individuals with a high school diploma to approximate the lifetime earnings of low-income individuals (U.S. Census Bureau, 2016). These benefits are already discounted to present value. |
References | Krueger, A. B. (2003). Economic considerations and class size. The Economic Journal, 113(485), F34–F63. Levin, H. M., Belfield, C., Muennig, P. A., & Rouse, C. (2007). The costs and benefits of an excellent education for all of America’s children. Retrieved from Columbia University Academic Commons: [1] Minnesota Department of Health (n.d.) Quick Facts: Oral health in Minnesota. Retrieved from https://www.health.state.mn.us/people/oralhealth/data/oralhealthmn.html [2] Minnesota Department of Health (n.d.) Quick Facts: Oral health in Minnesota. Retrieved from https://data.web.health.state.mn.us/tooth-decaySeirawan, H., Faust, S., & Mulligan, R. (2012). The impact of oral health on the academic performance of disadvantaged children. American Journal of Public Health, 102(9), 1729–1734. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov Washington State Institute for Public Policy (2014). Oral health: Fluoride varnish treatment for permanent teeth. Retrieved from http://www.wsipp.wa.gov/BenefitCost/Program/490 (Permanent teeth) Washington State Institute for Public Policy (2014). Oral health: Fluoride varnish treatment for primary teeth. Retrieved from http://www.wsipp.wa.gov/BenefitCost/Program/491 (Primary teeth). |
HEA012 Sealants leading to increased quality-adjusted life years (QALY)
Equation | (# individuals treated) x (Q: % patients who avoid tooth decay with treatment solely because of program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of resin sealants on future health, estimated in terms of quality-adjusted life years (QALY). Sealants are plastic films applied to the biting surfaces of molars to prevent decay. This analysis focuses on the effect of resin sealants compared to no treatment. Number of patients: Reported by program. Q: Percentage of patients who avoid tooth decay with treatment: [0.19 general population; 0.31 Latinx individuals; 0.32 Low-income individuals]. This is estimated using the following formula: In this formula, ES is the effect size [0.973] indicating the reduction in tooth decay due to program (WSIPP, 2014). The base percentage is the share of patients receiving care solely because of the program, which we approximate based on the estimate that 74 percent of Minnesota adults age 18 and older report having had at least one past year dental visit. (Minnesota Department of Health, n.d.). If program targets Latinx individuals, we approximate the rate of use of dental service using the Crozier (2011) estimate of the percent of Latinx Individuals who lack dental insurance [45%]. For low-income population, bout 35% of Medicaid enrollees report receiving oral care (MN Department of Health, 2017) QALY increase: [0.2]. This estimate of the value of stopping dental pain is based on an average of QALY values found in the literature for chronic oral pain and its control [0.39] (Thomsen, Gundgaard, Sorenson, Sjogren & Eriksen, 2000; Schmeir, Palmer, Flood & Gourlay, 2002). We discount the average from these studies by 50% since not every person will develop tooth decay even without sealants and not all tooth decay results in chronic pain. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Minnesota Department of Health (n.d.) Quick Facts: Oral health in Minnesota. Retrieved from http://www.health.state.mn.us/divs/healthimprovement/data/quick-facts/oralhealth.html Minnesota Department of Health (2017). Retrieved from: https://data.web.health.state.mn.us/service-use-medicaid Schmeir, J., Palmer, C., Flood, E. & Gourlay, G. (2002). Utility assessment of opioid treatment for pain. Pain Medicine, 3(3), 218-230. Thomsen, A., Gundgaard, J., Sorenson, J., Sjogren, P. & Eriksen, J. (2000). Cost-effectiveness of multidisciplinary treatment of patients with chronic non-malignant pain. Copenhagen, Denmark: Multidisciplinary Pain Centre, Danish National Hospital. Retrieved from https://www.researchgate.net/scientific-contributions/5872408_Annemarie_Bondegaard_Thomsen Crozier, S. (2011, November). Insights gained in Hispanic survey. ADA News. American Dental Association. Retrieved January 25, 2011. |
HEA013 Tooth decay treatment leading to increased quality-adjusted life years (QALY)
Equation | (# individuals treated) x (% treated patients treated solely because of program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of programs providing tooth decay treatment on future health, estimated in terms of quality-adjusted life years (QALY). We compute benefits for patients identified with chronic pain and for regular cases without a chronic pain diagnosis. Number of patients: Reported by program. Percentage of treated patients treated solely because of program: [64%]. About 36% of adult Medicaid enrollees in the 7-county metro area received a dental service within the last year (MN Department of Health, 2018). If program targets Latinx individuals, we approximate the rate of use of dental service using the Crozier (2011) estimate of the percent of Latinx individuals who lack dental insurance [45%]. QALY increase for chronic pain patients: [0.39]. This estimate of the value of stopping dental pain is a rough average of QALY values found in the literature for chronic pain and its control (Thomsen, Gundgaard, Sorenson, Sjogren & Eriksen, 2000; Schmeir, Palmer, Flood & Gourlay, 2002). QALY increase regular patients: [0.2]. This estimate of the value of stopping dental pain is a rough average of QALY values found in the literature for chronic pain and its control (Thomsen, Gundgaard, Sorenson, Sjogren & Eriksen, 2000; Schmeir, Palmer, Flood & Gourlay, 2002). We discount the average from these studies by 50% since not all tooth decay results in chronic pain. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Minnesota Department of Health. (2018). Medicaid Dental Service Use. Minnesota Public Health Access Data. Retrieved on June 2020 from: https://data.web.health.state.mn.us/medicaid-dental-service-use-query# Schmeir, J., Palmer, C., Flood, E. & Gourlay, G. (2002). Utility assessment of opioid treatment for pain. Pain Medicine, 3(3), 218-230. Thomsen, A., Gundgaard, J., Sorenson, J., Sjogren, P. & Eriksen, J. (2000). Cost-effectiveness of multidisciplinary treatment of patients with chronic non-malignant pain. Copenhagen, Denmark: Multidisciplinary Pain Centre, Danish National Hospital. Retrieved from https://www.researchgate.net/publication/228477510_Cost-effectiveness_of_multidisciplinary_treatment_of_patients_with_chronic_non-malignant_pain Crozier, S. (2011, November). Insights gained in Hispanic survey. ADA News. American Dental Association. Retrieved January 25, 2011. |
HEA014 Oral cancer screening of high-risk population leading to increased quality-adjusted life years (QALY)
Equation | (# individuals screened) x (% screened solely because of program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of yearly oral cancer screening on future health, estimated in terms of quality-adjusted life years (QALY). This metric is specifically for annual community outreach screening programs for males over 40 years of age who are regularly using tobacco and/or alcohol. Number of participants: Reported by program. Percentage of adults receiving care due to program: [0.65]. About 35% of Medicaid enrollees report receiving oral care (MN Department of Health, 2017). If the program targets Latinx individuals, we approximate the rate of use of dental service using the Crozier (2011) estimate of the percent of Latinx individuals who lack dental insurance [45%]. QALY increase: [0.04]. This is based on an estimate that screenings could result in a 0.04 QALY increase per patient (Dedhia et al., 2011). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Dedhia, R., Smith, K., Johnson, J., & Roberts, M. (2011). The cost-effectiveness of community-based screening for oral cancer in high-risk males in the United States: A Markov decision analysis approach. The Laryngoscope, 121(5), 952-960. Minnesota Department of Health (n.d.) Quick Facts: Oral health in Minnesota. Retrieved from https://www.health.state.mn.us/people/oralhealth/data/oralhealthmn.html Crozier, S. (2011, November). Insights gained in Hispanic survey. ADA News. American Dental Association. Retrieved January 25, 2011. |
HEA015 Asthma treatment leading to increased quality-adjusted life years (QALY)
Equation | (# participants with asthma) x (% individuals getting treatment solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of asthma treatment on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of participants: Adults and children reported separately by program. Percentage of participants getting services solely because of the program:
QALY increase: [0.05]. This estimate is the QALY value for a year of comprehensive asthma intervention based on research findings from Muennig, Glied & Simon (2005) and Schermer et al. (2002). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Minnesota Department of Health (2017). Asthma program: Quick facts – asthma in Minnesota. Retrieved from http://www.health.state.mn.us/divs/healthimprovement/content/documents-asthma/11.28.2017AsthmaFactSheet.pdf Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. New York, NY: Robin Hood. Schermer, T. R., Thoonen, B. P, van den Boom, G., Akkermans, R. P., Grol, R. P., Folgering, H. T., van Weel, C., & van Schayck, C. P. (2002). Randomized controlled economic evaluation of asthma self-management in primary health care. American Journal of Respiratory and Critical Care Medicine, 166(8), 1062-1072. |
HEA016 H.I.V. screening leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% individuals getting screened solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of H.I.V. screening on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of individuals screened: Reported by program. Percentage of participants who are screened solely because of the program: In the absence of program-specific data, we use the percent of low-income female and male individuals who are tested for sexually transmitted illnesses in the U.S.A. as a proxy for HIV screening. An estimated 80% of women are tested and 7% of men (Cuffe, et al.,2016).
QALY increase: [0.03]. This estimate is the QALY value for H.I.V. testing in a high-risk population (Muennig, Glied & Simon, 2005). This estimate includes the benefits of improved quality of life and longer survival for the person tested due to timely treatment, as well as the reduction of transmission of H.I.V. to others. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. New York, NY: Robin Hood. |
HEA017 Birth control leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% individuals getting treatment solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of birth control access and use on lifetime health, estimated in terms of quality-adjusted life years (QALY). The benchmark study evaluated 13 methods of contraception among women aged 15 to 50 years with respect to differences in health gains among other health outcomes. The study compared these methods with a hypothetical reference case of nonuse of contraception. The reversible contraceptive methods evaluated were: combination oral contraceptives (OCs); transdermal contraceptive patch (patch); vaginal ring; male condom (condom); diaphragm; copper intrauterine device (IUD); levonorgestrel-releasing IUD; depot medroxyprogesterone acetate (DMPA); estrogen–progestin monthly injectable; and two behavioral methods, periodic abstinence and withdrawal; as well as two permanent methods, tubal sterilization, and vasectomy. The general metric is based on the average gain in QALY from the 13 contraceptive methods in the study however, the metric can be modified to estimate the benefits of a specific method. Number of participants: Reported by program. Percentage of individuals receiving care due to program: [26%]. This is based on the percentage of women aged 18–49 in Minnesota who are at risk of unintended pregnancy and do not use contraceptives (Douglass, et al., 2017). QALY increase: [0.13]. This is based on the average gain in QALY from 13 contraceptive methods reported by Sonnenberg, et al. (2004). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Douglas-Hall, A., Kost, K., and Kavanaugh M. (2017) State-Level Estimates of Contraceptive Use in the United States, 2017. New York: Guttmacher Institute, 2018. Sonnenberg, Frank A., et al. (2004). Costs and net health effects of contraceptive methods. Contraception, 69(6), 447-459. |
HEA018 Sex education leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% individuals getting training solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of intensive sexual education on lifetime health, estimated in terms of quality-adjusted life years (QALY). The benchmark QALY increase estimated in this metric is based off of two, school-based sexual education models described below, with adjustments made based on the actual intensity of the program model being assessed. Number of participants: Reported by program. Percent of individuals receiving sex education solely because of the program: Estimated by the Constellation Fund. QALY increase: [0.19]. These added QALY are with respect to traditional sex education at schools and are based on two intensive models (Cooper, et al., 2012). Model 1 is a teacher-based model comprised of a 20-session classroom-based program over 2 years (10 sessions at age 13–14 years, and 10 at age 14–15 years). Teachers are taught in groups of thirteen during a 5-day training course run by a health promotion practitioner. Model 2 is a peer-led model comprised of three sessions during one school term. Training is undertaken in groups of twelve peer educators per training session over a 2-day intensive course led by a health promotion practitioner. The impact on QALY from these programs is high (0.39) but, note that the interventions are significantly more intensive than regular sexual education. We compare the length and content of the grantee’s program to these two programs and determine whether to apply the full impact on QALYs from the literature or discount the QALY value if the grantee’s program does not match the dosage, intensity, and quality of the programs in the literature. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Douglas-Hall, A., Kost, K., and Kavanaugh M. (2017) State-Level Estimates of Contraceptive Use in the United States, 2017. New York: Guttmacher Institute, 2018. Sonnenberg, Frank A., et al. (2004). Costs and net health effects of contraceptive methods. Contraception, 69(6), 447-459. |
HEA019 STI screening and treatment leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% individuals screened/treated solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of screening for sexually transmitted diseases (STI) on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of participants: Reported by program. Percentage of participants receiving test/treatment due to program: The counterfactual is based on the percent of low-income female and male individuals who are tested for sexually transmitted illnesses in the U.S.A. An estimated 80% of women are tested and 7% of men (Cuffe, et al.,2016).
QALY increase: [0.4]. Based on findings from Beck, et al. (2016). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Beck, E., Armbruster, B., Birkett, M., & Mustanski, B. (2016). PRM51 – The Benefits of A Holistic View: Considering Multiple Health Outcomes for HIV and STI Testing. Value in Health, 19(7), A366. Cuffe, K. M., Newton-Levinson, A., Gift, T. L., McFarlane, M., & Leichliter, J. S. (2016). Sexually Transmitted Infection Testing Among Adolescents and Young Adults in the United States. Journal of Adolescent Health, 58(5), 512–519. |
HEA020 Mental health care leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% of participants who receive treatment solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of mental health care access and treatment on lifetime health, estimated in terms of quality-adjusted live years (QALY). This metric should be used in conjunction with HEA021. Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [54%]. This is estimated by Constellation Fund staff based on data from SAMHSA (2017), which shows that 46% of individuals under 200% of the federal poverty level with any mental illness receive mental health care. QALY increase: [0.16]. This is based on findings from Ising, et al. (2016) which estimated the impact of treatment on QALY over a 4-year period. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Ising, H. K., Lokkerbol, J., Rietdijk, J., Dragt, S., Klaassen, R. M., Kraan, T., Boonstra, N., Nieman, D. H., van den Berg, D. P., Linszen, D. H., Wunderink, L., Veling, W., Smit, F., & van der Gaag, M. (2017). Four-Year Cost-effectiveness of Cognitive Behavior Therapy for Preventing First-episode Psychosis: The Dutch Early Detection Intervention Evaluation (EDIE-NL) Trial. Schizophrenia bulletin, 43(2), 365–374. https://doi.org/10.1093/schbul/sbw084 Substance Abuse and Mental Health Services Administration (SAMHSA). (2017). Key substance use and mental health indicators in the United States: Results from the 2016 National Survey on Drug Use and Health (HHS Publication No. SMA 17-5044, NSDUH Series H-52). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. Retrieved from: https://www.samhsa.gov/data/ |
HEA021 Mental health care leading to increased earnings
Equation | (# participants) x (% participants who receive treatment solely because of the program) x (% patients who respond to treatment) x (% increased earnings from avoided days of work lost as a result of the treatment) x ($ Average annual earnings of low-income population) |
Explanation | This metric estimates the impact of mental health care access and treatment on increased earnings. This metric should be used in conjunction with HEA020. Number of participants: Reported by program. Percentage of participants who receive treatment solely because of the program: [54%]. This is estimated by Constellation Fund staff based on data from SAMHSA (2017) which shows that 46% of individuals under 200% of the federal poverty level with any mental illness receive mental health care. Thus, we estimate that about 54% of participants would have not received treatment in the absence of the program. Percentage of patients who respond to treatment: [60%]. We base this estimate on a wide reading of the research literature, including Berndt et al. (2000) and Kazak et al. (2010). Increased earnings from avoided days of work lost as a result of the treatment: [60%]. Banerjee, Chatterji & Lahiri (2017) report that individuals with mental illness lose, on average, 6.6 weeks of work during the previous year or 13% of worktime. Average annual earnings of low-income population: [$8,600 – $24,300], counterfactual earnings calculated from ACS 5-year estimates (U.S Census Bureau, 2016). $8,600 is the average annual earnings of the population in the Twin Cities under 180% of poverty for all earners and non-earners. $24,300 is the average annual earnings for individuals with high school diplomas. The Constellation Fund staff will determine which counterfactual is appropriate for the grantee. Example: Assuming average annual earnings of $24,300, the value of the lost work time is approximately $3,000 or about 13% of the average annual income. Other researchers have found similar results (Kaya & Chan, 2017; McIntyre, Liauw, & Valerie, 2011). These benefits are already discounted to present value. Note that this effect does not include potential job loss due to illness or as a consequence of work absences. |
References | Banerjee, S., Chatterji, P., & Lahiri, K. (2017). Effects of psychiatric disorders on labor market outcomes: A latent variable approach using multiple clinical indicators. Health Economics. 26(2): 184–205. Berndt, E., Koran, L., Finkelstein, S., Gelenberg, A., Kornstein, S., Miller, I., Thase, M., Trapp, G. & Keller, M. (2000). Lost human capital from early-onset chronic depression. The American Journal of Psychiatry, 157(6), 940–947. Kaya, C., & Chan, F. (2017). Vocational rehabilitation services and outcomes for working age people with depression and other mood disorders. Journal of Rehabilitation, 83(3), 44–52. Kazak, A., Hoagwood, K., Weisz, K., Hood, J.R., Kratochwill, K., Vargas, L.A. & Banez, G.A. (2010). A meta- systems approach to evidence-based practice for children and adolescents. The American Psychologist, 65(2), 85–97. McIntyre, R., Liauw, S., & Taylor, V.H. (2011). Depression in the workforce: the intermediary effect of medical comorbidity. Journal of Affective Disorders, 128 Suppl 1, S29-36. Substance Abuse and Mental Health Services Administration (SAMHSA). (2017). Key substance use and mental health indicators in the United States: Results from the 2016 National Survey on Drug Use and Health (HHS Publication No. SMA 17-5044, NSDUH Series H-52). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. Retrieved from: https://www.samhsa.gov/data/ |
HEA023 Post-traumatic stress disorder/depression treatment leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% participants who receive treatment solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of post-traumatic stress disorder or depression treatment on lifetime health, estimated in terms of quality-adjusted life years (QALY). This metric should only be used for programs using best-practice treatments. This metric should be used in conjunction with HEA023. Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [54%]. This is estimated by Constellation Fund staff based on data from SAMHSA (2017) which shows that 46% of individuals under 200% of the federal poverty level with any mental illness receive mental health care. Thus, we estimate that about 54% of participants would have not received treatment in the absence of the program. QALY increase: [0.15]. This is the value of the relief symptoms of PTSD or depression due to best practice therapeutic or pharmacological care (Revicki et al., 2005; Rost, Pyne, Dickinson & LoSasso, 2005). Note that this QALY value already accounts for the probabilities of treatment response. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Johar, M. & Truong, J. (2014). Direct and indirect effect of depression in adolescence on adult wages. Applied Economics, 46(36), 4431–4444. Revicki, D., Siddique, J., Frank, L., Chung, J., Green, B., Krupnick, J., Prasad, M. & Miranda, J. (2005). Cost- effectiveness of evidence-based pharmacotherapy or cognitive behavioral therapy compared with community referral for major depression in predominantly low-income minority women. Archives of General Psychiatry, 62(8), 868-875. Rost, K., Pyne, J., Dickinson, L. M. & LoSasso, A. T. (2005). Cost-effectiveness of enhancing primary care depression management on an ongoing basis. Annals of Family Medicine, 3(1), 7-14. Substance Abuse and Mental Health Services Administration (SAMHSA). (2017). Key substance use and mental health indicators in the United States: Results from the 2016 National Survey on Drug Use and Health (HHS Publication No. SMA 17-5044, NSDUH Series H-52). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. Retrieved from: https://www.samhsa.gov/data/ |
HEA023 Post-traumatic stress disorder/depression treatment leading to increased earnings
Equation | (# participants treated for PTSD or depression) x (% participants who receive treatment solely because of the program) x (% participants who respond to treatment) x (% decrease in earnings prevented as a result of the treatment) x ($ average annual earnings for low-income population) |
Explanation | This metric estimates the impact of post-traumatic stress disorder or depression treatment on increased earnings. This metric should only be used for programs using best-practice treatments. This metric should be used in conjunction with HEA022. Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [54%]. This is estimated by Constellation Fund staff based on data from SAMHSA (2017) which shows that 46% of individuals under 200% of the federal poverty level with any mental illness receive mental health care. Thus, we estimate that about 54% of participants would have not received treatment in the absence of the program. Percentage of participants who respond to treatment: [60%] We base this estimate on a wide reading of the research literature, including Berndt et al. (2000) and Kazak et al. (2010). Percentage decrease in earnings prevented as a result of the treatment: [20%], the estimated 20 percent increase in earnings as a result of PTSD treatment is based on the work of Berndt et al. (2000) and Kessler (2000), which shows that PTSD and depression both reduce days worked per month by about 3.6 days, or about 43 days per year, representing about 17 percent of the work year. We round up to 20 percent. This estimate of lost wages is very conservative because it does not consider the more structural aspects of lost opportunity and unstable employment. Moreover, PTSD typically lasts three years for those who get treatment (Kessler, 2000). We do not extend this cost over the lifetime but conservatively apply the cost only to the current year. Average annual earnings of low-income population: [$8,600 – $24,300], counterfactual earnings calculated from ACS 5-year estimates (U.S Census Bureau, 2016). $8,600 is the average annual earnings of the population in the Twin Cities under 180% of poverty for all earners and non-earners. $24,300 is the average annual earnings for individuals with high school diplomas. The Constellation Fund staff will determine which counterfactual is appropriate for the grantee. Example: Assuming average annual earnings of $24,300, the value of the lost work time is approximately $3,000 or about 13% of the average annual income. Other researchers have found similar results (Kaya & Chan, 2017; McIntyre, Liauw, & Valerie, 2011). These benefits are already discounted to present value. Note that this effect does not include potential job loss due to illness or as a consequence of work absences. |
References | Berndt, E., Koran, L., Finkelstein, S., Gelenberg, A., Kornstein, S., Miller, I., Thase, M., Trapp, G. & Keller, M. (2000). Lost human capital from early-onset chronic depression. The American Journal of Psychiatry, 157(6), 940–947. Kazak, A., Hoagwood, K., Weisz, K., Hood, J.R., Kratochwill, K., Vargas, L.A. & Banez, G.A. (2010). A meta- systems approach to evidence-based practice for children and adolescents. The American Psychologist, 65(2), 85–97. Kessler, R. C. (2000). Posttraumatic stress disorder: The burden to the individual and to society. Journal of Clinical Psychiatry, 61(5). U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. |
HEA024 Severe mental illness (Schizophrenia treatment) leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% participants who receive treatment solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of post-traumatic stress disorder or depression treatment on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [54%]. This is estimated by Constellation Fund staff based on data from SAMHSA (2017) which shows that 46% of individuals under 200% of the federal poverty level with any mental illness receive mental health care. Thus, we estimate that about 54% of participants would have not received treatment in the absence of the program. QALY increase: [0.15]. This is the value of the relief symptoms of severe mental illness from treatment of schizophrenia based on Andrews, Issakidis, C., Sanderson, S., Corry, J. & Lapsley (2004) and Carr, Lewin & Neil, 2006. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Andrews, G., Issakidis, C., Sanderson, S., Corry, J. & Lapsley, H. (2004). Utilising survey data to inform public policy: Comparison of the cost-effectiveness of treatment of ten mental disorders. British Journal of Psychiatry, 184(6), 526-533. Carr, V. J., Lewin, T. J. & Neil, A. L. (2006). What is the value of treating schizophrenia? Australian and New Zealand Journal of Psychiatry, 40(11-12), 963-971. Substance Abuse and Mental Health Services Administration (SAMHSA). (2017). Key substance use and mental health indicators in the United States: Results from the 2016 National Survey on Drug Use and Health (HHS Publication No. SMA 17-5044, NSDUH Series H-52). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. Retrieved from: https://www.samhsa.gov/data/ |
HEA025 Multisystemic Therapy for children and families leading to reduced lifetime costs to children associated with out of home placement
Equation | (# children receiving treatment) x (Q: Avoided out of home placements due to the intervention) x ($ benefits from avoided out of home placement) |
Explanation | This metric estimates the impact of Multisystemic Therapy for children and families leading to reduced out-of-home placement and subsequent reductions in costs to children associated with out-of-home placement. Number of children receiving treatment: Reported by program. Q: Avoided out-of-home placements due to the intervention: [0.134]. This is estimated by Constellation staff using the following formula: In this formula, ES is the effect size from a meta-analysis of Multisystemic Therapy programs on out of home placements [-0.226] (WSIPP, 2017). The base percentage is the assumed counterfactual rate of observed child abuse [9%] based on an incidence rate of 3% reported by the MN Department of Human Services (2018) and adjusted for by a factor of 3. Benefit from reduced out of home placement: [$350,000]. This estimate of the value of preventing child abuse (in terms of lost QALYs) is based on the findings of Peterson, et al (2015). These benefits are already discounted to present value. |
References | Minnesota Department of Human Services (2018). Child protection in Minnesota: Keeping children safe. DHS-4735-ENG. Retrieved from https://edocs.dhs.state.mn.us/lfserver/Public/DHS-4735-ENG. Peterson, C., Florence, C., & Klevens, J. (2018). The economic burden of child maltreatment in the United States, 2015. Child Abuse and Neglect, 86, 178–183. https://doi.org/10.1016/j.chiabu.2018.09.018 Swenson, C.C., et al. (2010). Multisystemic Therapy for child abuse and neglect: A randomized effectiveness trial. Journal of Family Psychology, 24(4), 497-507. Washington State Institute for Public Policy, WSIPP (2017). Multisystemic Therapy (MST) for child abuse and neglect. Retrieved from http://www.wsipp.wa.gov/BenefitCost/Program/266 |
HEA026 Mental health care or depression treatment for transgender population leading to increased earnings
Equation | (# participants) x (% participants receiving care due to program) x (Q: Impact of program on depression) x (% decrease in earnings prevented as a result of the treatment) x ($ average annual earnings for low-income population) |
Explanation | This metric estimates the impact of mental health care treatment for transgender individuals leading on increased earnings. Number of participants: Reported by program. Percentage of individuals who receive mental health care due to program: [0.52]. This is the percent of transgender patients, reported in a sample survey, who showed evidence of psychological distress but had not received mental health services in the past year (Shiperd et al., 2010). Q: Impact of program on depression: [0.27]. First, we estimate the mean effect size using the formula: In this formula, OR is the odds ratio showing the impact of receiving timely transgender-specific mental health on depression [3.08] (Seelman, et al., 2017). This results in an effect size of ln(3.08)/1.65 = 0.68. Then, we estimate the number of impacted participants using the formula: In this formula, ES is the effect size estimated from Seelman, et al. (2017) [0.68] and the base percentage is the percent of transgender individuals with depression [44%] (Seelman, et al., 2017). Percentage decrease in earnings prevented as a result of the treatment: [20%], the estimated 20 percent increase in earnings as a result of PTSD treatment is based on the work of Berndt et al. (2000) and Kessler (2000), which shows that PTSD and depression both reduce days worked per month by about 3.6 days, or about 43 days per year, representing about 17 percent of the work year. We round up to 20 percent. This estimate of lost wages is very conservative because it does not consider the more structural aspects of lost opportunity and unstable employment. Moreover, PTSD typically lasts three years for those who get treatment (Kessler, 2000). We do not extend this cost over the lifetime but conservatively apply the cost only to the current year. Average annual earnings of low-income population: [$8,600 – $24,300], counterfactual earnings calculated from ACS 5-year estimates (U.S Census Bureau, 2016). $8,600 is the average annual earnings of the population in the Twin Cities under 180% of poverty for all earners and non-earners. $24,300 is the average annual earnings for individuals with high school diplomas. The Constellation Fund staff will determine which counterfactual is appropriate for the grantee. Example: Assuming average annual earnings of $24,300, the value of the lost work time is approximately $3,000 or about 13% of the average annual income. Other researchers have found similar results (Kaya & Chan, 2017; McIntyre, Liauw, & Valerie, 2011). These benefits are already discounted to present value. Note that this effect does not include potential job loss due to illness or as consequence of work absences. |
References | Berndt, E., et al. (2000). Lost human capital from early-onset chronic depression. The American Journal of Psychiatry, 157(6), 940–947. Shipherd, J. C., Green, K. E., & Abramovitz, S. (2010). Transgender clients: Identifying and minimizing barriers to mental health treatment. Journal of Gay & Lesbian Mental Health, 14(2), 94-108. Kessler, R. C. (2000). Posttraumatic stress disorder: The burden to the individual and to society. Journal of Clinical Psychiatry, 61(5). Seelman, K., Colón-Diaz, M., Lecroix, R., Xavier-Brier, M., & Kattari, L. (2017). Transgender Noninclusive Healthcare and Delaying Care Because of Fear: Connections to General Health and Mental Health Among Transgender Adults. Transgender Health, 2(1), 17-28. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. |
HEA027 Mental health care or depression treatment for transgender population leading to avoided suicide deaths
Equation | (# participants) x (% individuals receiving mental health care due to program) x (Q: Impact of program on suicide attempts) x ($ value per life saved) |
Explanation | This metric estimates the impact of transgender-specific mental health care on the incidences of suicide. Number of participants: Reported by program. Percentage of individuals who receive mental health care due to program: [0.52]. This is the percent of transgender patients, reported in a sample survey, who showed evidence of psychological distress but had not received mental health services in the past year (Shiperd et al., 2010). Q: Impact of program on suicide deaths: [0.01]. To calculate this, we first estimate the mean effect size using the formula: In this formula, OR is the odds ratio showing the impact of receiving timely transgender-specific mental health on suicide attempts [3.81] (Seelman, et al., 2017). This results in an effect size of: ln(3.81)/1.65 = 0.81. Then, we estimate the number of impacted participants is estimated using the formula: Where ES is the effect size estimated from Seelman, et al. (2017) [0.81] the base percentage is the estimated percent of transgender individuals with successful suicide attempts [0.0036]. We estimate this by adjusting the percent of transgender individuals with suicide attempts [10%] (Seelman, et al., 2017) by the estimated rate of successful suicide attempts [3.6%]. We estimate this rate from the number of suicide attempts in the U.S. and the number of deaths by suicide in 2017 (NIH, 2017) $ value per life saved: We estimate the value of a life based on a [$50,000] QALY. This value varies by the age and expected years of life of each participant. Thus, we compute total benefits of a program based on specific program data on participants age and discount the annual value to present value using Constellation’s standard discounting method. |
References | Berndt, E., et al. (2000). Lost human capital from early-onset chronic depression. The American Journal of Psychiatry, 157(6), 940–947. Shipherd, J. C., Green, K. E., & Abramovitz, S. (2010). Transgender clients: Identifying and minimizing barriers to mental health treatment. Journal of Gay & Lesbian Mental Health, 14(2), 94-108. Kessler, R. C. (2000). Posttraumatic stress disorder: The burden to the individual and to society. Journal of Clinical Psychiatry, 61(5). Seelman, K., Colón-Diaz, M., Lecroix, R., Xavier-Brier, M., & Kattari, L. (2017). Transgender Noninclusive Healthcare and Delaying Care Because of Fear: Connections to General Health and Mental Health Among Transgender Adults. Transgender Health, 2(1), 17-28. National Institute of Mental Health (2017). https://www.nimh.nih.gov/health/statistics/suicide.shtml#part_154968 U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Seelman, K., Colón-Diaz, M., Lecroix, R., Xavier-Brier, M., & Kattari, L. (2017). Transgender Noninclusive Healthcare and Delaying Care Because of Fear: Connections to General Health and Mental Health Among Transgender Adults. Transgender Health, 2(1), 17-28. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. |
HEA028 Substance abuse treatment leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% participants who receive treatment solely because of the program) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of substance abuse treatment on lifetime health, estimated in terms of quality-adjusted life years (QALY). This metric should be used in conjunction with HEA029. This metric is used to estimate benefits for any addiction treatment program based on the average impact of all treatment methods. We determine in a case-by-case basis whether a specific metric can be developed based on the program’s methodology. Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [0.9]. This estimate for those who enroll in the substance abuse treatment program solely due to our grant is based on findings that only about 10 percent of people who need substance abuse treatment actually receive it (Substance Abuse and Mental Health Services Administration, 2012). QALY increase: [0.03]. Research indicates that a currently active substance abuse disorder reduces the quality of life between about 0.13 and 0.20 QALY (Kilmer, 2009), with the higher range reserved for heroin users (Nicosia, Pacula, Kilmer, Lundberg & Chiesa, 2009). Aos, Mayfield, Miller & Yen (2006) report a 22 percent reduction in symptoms based on an average drug treatment program. Applying these findings together, we find a 0.03 QALY improvement due to a typical substance abuse program (0.13 reduction in quality of life x 0.22 expected improvement due to intervention = 0.03 improvement in QALY due to program). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Aos, S., Mayfield, J., Miller, M. & Yen, W. (2006). Evidence-based treatment of alcohol, drug, and mental health disorders: Potential benefits, costs, and fiscal impacts for Washington State. Olympia, WA: Washington State Institute for Public Policy. Retrieved from http://www.wsipp.wa.gov/rptfiles/06-06- 3901.pdf Kilmer, B. (2009). Substance use and treatment in NYC: Cost, benefits, and opportunities. Annotated presentation to Robin Hood. New York, NY: Robin Hood. Nicosia, N., Pacula, R., Kilmer, B., Lundberg, R. & Chiesa, J. (2009). The economic cost of methamphetamine use in the United States, 2005 (MG-829). Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/monographs/MG829.html Substance Abuse and Mental Health Services Administration (2012). Results |
HEA029 Substance abuse treatment leading to increased earnings
Equation | (# participants) x (% patients enrolled solely due to the program) x (Q1:Impact of program on substance abuse) x (Q2: Impact of substance abuse on employment) x ($ average annual earnings low-income) |
Explanation | This metric estimates the impact of substance abuse treatment on increased earnings. This metric should be used in conjunction with HEA028. This metric is used to estimate benefits for any addiction treatment program based on the average impact of all treatment methods. We determine in a case-by-case basis whether a specific metric can be developed based on the program’s methodology. Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [0.9], estimate for those who enroll in the substance abuse treatment program solely due to our grant is based on findings that only about 10 percent of people who need substance abuse treatment actually receive it (Substance Abuse and Mental Health Services Administration, 2012). Q1: Impact of program on substance abuse: [0.22]. The program impact is the reduction in symptoms based on an average drug treatment program [-22%] (Aos, Mayfield, Miller & Yen, 2006). Q2: Impact of substance abuse on employment: [0.14]. Q2 is estimated as the (percentage impact of substance abuse on probability of employment) x (employment rate of population of interest), where the percentage impact of substance abuse on employment is [-0.29] (WSIPP, 2017). The employment rate of low-income individuals is [47%] (U.S. Census Bureau, 2018). The resulting impact is (0.29 x 0.47 = 0.14). Average annual earnings of employed low-income individuals: [$13,500]. This is estimated using ACS data for the Twin Cities metropolitan area (U.S. Census Bureau, 2016). We assume one year of additional income. In this metric, we use earnings of employed low-income individuals instead of all low-income individuals since the effect size refers to the impact on employment. |
References | Aos, S., Mayfield, J., Miller, M. & Yen, W. (2006). Evidence-based treatment of alcohol, drug, and mental health disorders: Potential benefits, costs, and fiscal impacts for Washington State. Olympia, WA: Washington State Institute for Public Policy. Minnesota Compass. (2018). Twin Cities region neighborhood profile data: Phillips community. Retrieved from: http://www.mncompass.org/profiles/communities/minneapolis/phillips Substance Abuse and Mental Health Services Administration (2012). Results from the 2011 National Survey on Drug Use and Health: Mental Health Findings, NSDUH Series H-45, HHS Publication No. (SMA) 12-4725. Retrieved from: https://www.samhsa.gov/data/sites/default/files/2011MHFDT/2k11MHFR/Web/NSDUHmhfr2011.htm U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf |
HEA030 Smoking cessation leading to increased quality-adjusted life years (QALY)
Equation | (# participants) x (% participants who receive treatment solely because of the program) x (% impact of program on smoking session) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of one year of smoking cessation on lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of participants: Reported by program. Percentage of participants receiving treatment solely because of program: [1.0]. Only 0.4% of all tobacco users in the state received telephone counseling, cessation medications, or both from the state Quitline (Centers for Disease Control and Prevention, 2012). Percentage impact of program on smoking session: [0.2]. Reported twelve-month abstinence rates for persons who used smoking-cessation clinics have ranged from 20% to 40%. We apply the lowest value from this range (CDC, 1994). QALY increase: [1.2]. This is the QALY estimate for the benefit to adult’s smoking cessation for one year based on the work of the New Zealand Ministry of Health (2004). $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention. (1994). Retrieved from: https://www.cdc.gov/mmwr/preview/mmwrhtml/00017511.htm Centers for Disease Control and Prevention. (2012). Minnesota Monitor. Retrieved from: https://www.cdc.gov/tobacco/data_statistics/state_data/state_highlights/2012/pdfs/states/minnesota.pdf New Zealand Ministry of Health. (2004). An economic evaluation of the Quitline Nicotine Replacement Therapy (NRT) Service. Wellington, New Zealand: Author. Retrieved from: https://quit.org.nz/-/media/Images/Quitline/PDFs-and-Docs/Full-index-page/FINAL—NRT-economic-evaluation-Aug-04.pdf?la=en |
HEA031 Peer support for individuals with substance abuse disorder leading to increased earnings
Equation | (# patients enrolled in program) x (Q1:Impact of program on substance abuse) x (Q2: Impact of substance abuse on employment) x ($ average annual earnings employed low-income) |
Explanation
| This metric estimates the impact of peer support programs for individuals living with a substance abuse disorder on increased earnings. This analysis examined interventions provided by a peer specialist to individuals with substance abuse disorders. One study was included in this analysis. This study examined the impact of a brief motivational intervention provided by a peer specialist for individuals using heroin and cocaine. The study participants were screened and identified at walk-in general health clinics. Number of patients enrolled: Reported by program. Q1: Impact of program on substance abuse: [0.23]. Q1 is estimated as the (percentage enrolled because of the program) x (percentage effect of program on substance abuse), where the percentage enrolled because of the program is based on findings that only about 10 percent of people who need substance abuse treatment actually receive it (Substance Abuse and Mental Health Services Administration, 2012). The percentage effect of program on substance abuse (program impact) is [-0.25] (WSIPP, 2018). The resulting impact is (0.9 x 0.25 = 0.23). Q2: Impact of substance abuse on employment: [0.14]. Q1 is estimated as the (percentage impact of substance abuse on the probability of employment) x (employment rate of population of interest), where the percentage impact of substance abuse on employment is [-0.29] (WSIPP, 2017). The employment rate of low-income individuals is [47%] (U.S. Census Bureau, 2018). The resulting impact is (0.29 x 0.47 = 0.14). Average annual earnings of employed low-income individuals: [$13,500]. This is estimated using ACS data for the Twin Cities metropolitan area (U.S. Census Bureau, 2016). We assume one year of additional income. In this metric, we use earnings of employed low-income individuals instead of all low-income individuals since the effect size refers to the impact on employment. |
References | Substance Abuse and Mental Health Services Administration (2012). Results from the 2011 National Survey on Drug Use and Health: Mental Health Findings, NSDUH Series H-45, HHS Publication No. (SMA) 12-4725. Retrieved from: https://www.samhsa.gov/data/sites/default/files/2011MHFDT/2k11MHFR/Web/NSDUHmhfr2011.htm U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. U.S. Census Bureau. (2018). American Community Survey 5-year estimates – public use microdata sample, 2013-2017. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from: http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf Washington State Institute for Public Policy. (2018). Retrieved from: http://www.wsipp.wa.gov/BenefitCost/Program/336 |
HEA032 Domestic violence assistance leading to reduced health care cost (adults)
Equation | (# participants) x (% of participants getting assistance solely because of the program) x (Q: Impact of the program on health outcomes) x (% Health care costs paid out of pocket by victims) x ($ Health care costs associated with domestic abuse) |
Explanation | This metric estimates the impact of support and assistance for adults related to domestic violence on lifetime mental and physical health care expenses. Based on the evidence reviewed, intensive advocacy may improve short-term quality of life and reduce physical abuse one to two years after the intervention for women recruited from domestic violence shelters or refuges. Brief advocacy may provide small short-term mental health benefits and reduce abuse. Number of participants: Reported by program. Percentage of participants getting assistance solely because of the program: [79%, female]. This is based on reporting that 21% of female victims and 6% of male victims disclosed their victimization to a doctor or nurse at some point in their lifetime (Black et al, 2010). Q: Impact of the program on health outcomes: [0.14]. The program’s impact on participant health outcomes is estimated using the following formula: In this formula, ES [0.37] is the average odds ratio of the impact of interventions on physical abuse and mental health. (Rivas et al, 2015). The base percentage [26%] is the average incidence rate of negative health conditions among individuals with a history of abuse (Black et al, 2011) Percentage of health care costs paid out of pocket by victims: [30%]. This is the reported average of medical and mental health costs paid out of pocket by victims (Theresa’s Fund, 2018). Health care costs associated with domestic abuse: [$11,500]. This is the estimated present discounted lifetime health care costs associated with domestic abuse (2018 dollars). |
References | Black, M.C., Basile, K.C., Breiding, M.J., Smith, S.G., Walters, M.L., Merrick, M.T., Chen, J., & Stevens, M.R. (2011). The National Intimate Partner and Sexual Violence Survey (NISVS): 2010 Summary Report. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. Retrieved from: https://www.cdc.gov/violenceprevention/pdf/NISVS-2010SummaryReport-508.pdf Holmes, M.R., Richter, F.G.C., Votruba, M.E., Berg, K. A., & Bender, A. E.. (2018). Economic Burden of Child Exposure to Intimate Partner Violence in the United States. Journal of Family Violence 33(239). Rivas, C., Ramsay, J., Sadowski, L., Davidson, L. L., Dunne, D., Eldridge, S., …Feder, G. (2015). Advocacy interventions to reduce or eliminate violence and promote the physical and psychosocial well-being of women who experience intimate partner abuse. Cochrane Database of Systematic Reviews (12). Theresa’s Fund, Domestic Shelters. (2015). Economic Impact of Domestic Violence. Retrieved from: https://www.domesticshelters.org/domestic-violence-statistics/economic-impact-of-domestic-violence |
HEA033 Domestic violence assistance leading to improved employment
Equation | (# participants) x (% participants get assistance solely because of the program) x (Q: Impact of the program on employment outcomes) x ($ average annual earnings of employed low-income individuals) x (# working years) |
Explanation | This metric estimates the impact of assistance and support related to domestic violence on employment outcomes. Number of participants: Reported by program. Percentage of participants getting assistance solely because of the program: [79%, female]. This is based on reporting that 21% of female victims and 6% of male victims disclosed their victimization to a doctor or nurse at some point in their lifetime (Black et al, 2010). Q: Impact of the program on employment outcomes: [0.26]. The program’s impact on participant employment outcomes is estimated using the following formula: In this formula, ES [0.348] is the impact of the program on employment (WSIPP, 2017). The base percentage [0.463] is the proportion of low-income individuals employed in Minnesota (Minnesota Compass, 2018). Note that the proportion of adults working will be lower than the employment rate, which is calculated by dividing the number of employed people by the total number of people in the labor force. The labor force is made up of employed and unemployed people who are currently looking for work. The employment rate does not account for discouraged workers who are no longer seeking employment, nor for people who do not participate in the labor force for any number of reasons. Average annual earnings of employed low-income individuals: [$13,500]. This is estimated using ACS data for the Twin Cities metropolitan area (U.S. Census Bureau, 2016). In this metric, we use earnings of employed low-income individuals instead of all low-income individuals since the effect size refers to the impact on employment. Number of working years: Estimated from average participation age through age 65. Benefits are then discounted to present value based on the average age of participation to through age 65 (estimated working years). |
References | Black, M.C., Basile, K.C., Breiding, M.J., Smith, S.G., Walters, M.L., Merrick, M.T., Chen, J., & Stevens, M.R. (2011). The National Intimate Partner and Sexual Violence Survey (NISVS): 2010 Summary Report. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. Retrieved from: https://www.cdc.gov/violenceprevention/pdf/NISVS-2010SummaryReport-508.pdf Minnesota Compass (2018) Proportion of adults (age 16-64) working by poverty status, Minnesota, 1990-2017. Retrieved from: http://www.mncompass.org/workforce/proportion-of-adults-working#1-11324-d U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Washington State Institute for Public Policy. (2017). Cognitive behavioral therapy (CBT) for adult posttraumatic stress disorder (PTSD). Retrieved from: http://www.wsipp.wa.gov/BenefitCost/Program/241 |
HEA034 Domestic violence assistance leading to avoided homicides
Equation | (# participants) x (% of participants get assistance solely because of the program) x (Q: Impact of the program on the likelihood of death) x ($ value per life saved) |
Explanation | This metric estimates the impact of assistance and support related to domestic violence on homicide rates, estimated in terms of quality-adjusted life years (QALY). Number of participants: Reported by program. Percentage of participants getting assistance solely because of the program: [79%, female]. This is based on reporting that 21% of female victims and 6% of male victims disclosed their victimization to a doctor or nurse at some point in their lifetime (Black et al, 2010). Q: Impact of the program on the likelihood of death: [0.26]. This is the number of impacted participants estimated using the formula: In this formula, ES [0.348] is the percentage of abused women saved from death due to the program. We use the effect size of interventions on reducing severe physical abuse in women leaving a shelter at 24 months to proxy the effectiveness of the program on preventing death. This effect size is an odds ratio reported by Rivas et al (2015). The base percentage [0.001] is the percent of women suffering from abuse who would be killed. This is an estimate for the homicide rate of women who are abused from the National Center for Injury Prevention and Control (2003). However, for women who leave their homes because they are afraid and seek help from our grantee, the odds of death are probably much higher because prior interpersonal violence increases the risk of domestic homicide 15-fold (Centers for Disease Control and Prevention, 2003). We leave the increase in the risk aside and conservatively use the one-tenth of one percent estimate. $ value per life saved: We estimate the value of a life based on a [$50,000] QALY. This value varies by the age and expected years of life of each participant. Thus, we compute the total benefits of a program based on specific program data on participants’ age and discount the annual value to present value using Constellation’s standard discounting method. |
References | Black, M.C., Basile, K.C., Breiding, M.J., Smith, S.G., Walters, M.L., Merrick, M.T., Chen, J., & Stevens, M.R. (2011). The National Intimate Partner and Sexual Violence Survey (NISVS): 2010 Summary Report. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. Retrieved from: https://www.cdc.gov/violenceprevention/pdf/NISVS-2010SummaryReport-508.pdf Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. (2003). Costs of intimate partner violence against women in the United States. Atlanta, GA: Centers for Disease Control and Prevention. Retrieved from: https://www.cdc.gov/violenceprevention/pub/IPV_cost.html. McFarlane, J., Malecha, A., Watson, K., Gist, J., Batten, E., Hall, I. & Smith, S. (2005). Intimate partner sexual assault against women: Frequency, health consequences, and treatment outcomes. Obstetrics and Gynecology, 105(1), 99-108. Rivas, C., Ramsay, J., Sadowski, L., Davidson, L. L., Dunne, D., Eldridge, S., Feder, G. (2015). I. Cochrane Database of Systematic Reviews (12). |
HOU001 Homelessness reduction leading to increased earnings for adults
Equation | (# adults receiving services) x (% adults reduce homelessness solely because of this program) x ($ average increase in wages) |
Explanation | This metric estimates the impact of avoided homelessness or new permanent housing on increased wages for adults. Number of adults receiving services: Reported by program Percent of these adults who avoid or are no longer homeless due to the program: [0.52]. For programs serving individuals at imminent risk of homelessness or are currently homeless (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. From this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Net increase in wages one year after entering supportive housing by population: [$4,093], Wilder Research, 2017). We assume one year of additional income. |
References | Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. . Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf Wilder Research. (2017). Homelessness in Minnesota: Youth on their own. Findings from the 2015 Minnesota Homeless Study. Retrieved from: http://mnhomeless.org/minnesota-homeless-study/reports-and-fact-sheets/2015/2015-homeless-youth-4-17.pdf |
HOU002 Homelessness reduction leading to increased cash assistance for adults
Equation | (# adults receiving services) x (% adults reduce homelessness solely because of this program) x ($ net increase in cash assistance) |
Explanation
| This metric estimates the impact of avoided homelessness or new permanent housing on increased access to cash through public assistance programs. Number of adults receiving services: Reported by program Percentage of these adults who avoid homelessness due the program: [0.52]. For programs serving individuals at imminent risk of homelessness or are currently homeless (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. From this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Net increase in cash assistance (public programs) one year after entering supportive housing by population: [$79]. (Wilder Research, 2017). We assume one year of additional income |
References | Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. . Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf Wilder Research. (2017). Homelessness in Minnesota: Youth on their own. Findings from the 2015 Minnesota Homeless Study. Retrieved from: http://mnhomeless.org/minnesota-homeless-study/reports-and-fact-sheets/2015/2015-homeless-youth-4-17.pdf |
HOU003 Homelessness reduction leading to avoided deaths of unsheltered adults
Equation | (# unsheltered adults) x (% unsheltered adults who get assistance solely because of the program) x (% unsheltered adults would are likely to die without intervention) x (% unsheltered adults saved from death due to the program) x ($ value per life saved) |
Explanation | This metric estimates the impact of new permanent housing for unsheltered adults on the likelihood of death, estimated in terms of quality-adjusted life years (QALY). Number of unsheltered adults experiencing homelessness: Reported by program. Percentage of unsheltered adults experiencing homelessness who get assistance solely because of the program: [45%]. This is the percentage of unsheltered homeless who would stay outside (Wilder Research, 2016). Percentage of unsheltered adults who would likely die without intervention: [6%]. Our 6% estimate is an approximation based on evidence suggesting that homelessness was associated with an all-cause mortality hazard ratio of 1.6 compared to adults not experiencing homelessness (Morrison, 2009). We estimate a counterfactual probability of death between ages 15-64 to be 3.7% based on mortality statistics from Minnesota Department of Health (2016). $ value per life saved: We estimate the value of a life based on a [$50,000] QALY. This value varies by the age and expected years of life of each participant. Thus, we compute total benefits of a program based on specific program data on participants age and discount the annual value to present value using Constellation’s standard discounting method. |
References | Center for Disease Control and Prevention (2017). Retrieved from https://www.cdc.gov/features/homelessness/index.html Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. Report to Robin Hood. New York, NY: Robin Hood. Morrison, D.S. (2009). Homelessness as an independent risk factor for mortality: results from a retrospective cohort study. International Journal of Epidemiology, 38 (3), 877-83. MN Compass (2010). Retrieved from https://www.mncompass.org/_pdfs/presentations/BlueCross_HealthInequities_10-10.pdf Minnesota Department of Health, Center for Health Statistics. Retrieved from: https://mhsq.web.health.state.mn.us/birth/queryFrontPage.jsp?goTo=deathInterfaceSas.jsp&queryPage=deathInterfaceSas.jsp&startQuery=true Schermer, T. R., Thoonen, B. P, van den Boom, G., Akkermans, R. P., Grol, R. P., Folgering, H. T., van Weel, C. & van Schayck, C. P. (2002). Randomized controlled economic evaluation of asthma self-management in primary health care. American Journal of Respiratory and Critical Care Medicine, 166, 1062-1072. Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. . . Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf |
HOU004 Homelessness reduction leading to increased academic achievement for youths under 5 years of age
Equation | (# children under age 5 receiving services) x (% families receiving housing solely because of this program) x (% impact of supportive housing on educational outcome) x ($ increase in lifetime earnings from educational outcome) |
Explanation | This metric estimates the impact of avoided homelessness or new permanent housing on academic achievement and subsequent increases in lifetime earnings for children under 5 years of age. Number of children under 5 receiving housing: Reported by program Percent of these adults who avoid homelessness due the program: [0.52]. For programs serving individuals at imminent risk of homelessness (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. To this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Impact of housing on educational outcomes: [0.056 SD]. We found evidence that children who experienced homelessness for the first time as toddlers or younger are 0.6 times as likely to be proficient when tested in reading in math at third grade as non-homeless children. However, results from other studies in this area are mixed or have found no statistically significant effects of homelessness on academic performance. To account for this low level of evidence, we average the effect sizes from five studies including four who did not find significant results. To compute the effect-size we use the following equation for dichotomous results (WSIPP, 2017): where ES is the effect size and OR is the odds ratio from the revised studies. The resulting average effect size is -0.056 measured in standard deviations. Increase in lifetime earnings from educational outcomes: [$31,000], the gain in initial wage rate for one standard deviation increase in standardized test score is estimated at [0.05]. While the gain in the growth rate of wage for one standard deviation increase in standardized test score is [0.003] (Hall & Farkas, 2011). Based on these two results, and the average lifetime earnings of individuals with high school diplomas, we estimate that an increase in one standard deviation in standardized test scores is associated with $31,000 in additional lifetime earnings. Benefits already discounted to present value. |
References | Brumley, B., Fantuzzo, J., Perlman, S., & Zager, M. L. (2015). The unique relations between early homelessness and educational well-being: An empirical test of the continuum of risk hypothesis. Children and Youth Services Review, 48, 31–37. Coulton, C. J., Richter, F., Kim, S.-J., Fischer, R., & Cho, Y. (2016). Temporal effects of distressed housing on early childhood risk factors and kindergarten readiness. Children and Youth Services Review, 68, 59–72. Fantuzzo, J., LeBoeuf, W., Brumley, B., & Perlman, S. (2013). A population-based inquiry of homeless episode characteristics and early educational well-being. Children and Youth Services Review, 35(6), 966–972. Hall, M., & Farkas, G. (2011). Adolescent Cognitive Skills, Attitudinal/Behavioral Traits and Career Wages. Social Forces, 89(4), 26. Obradović, J., Long, J. D., Cutuli, J. J., Chan, C.-K., Hinz, E., Heistad, D., & Masten, A. S. (2009). Academic achievement of homeless and highly mobile children in an urban school district: Longitudinal evidence on risk, growth, and resilience. Development & Psychopathology, 21, 493–518. Rafferty, Y., Shinn, M., & Weitzman, B. C. (2004). Academic achievement among formerly homeless adolescents and their continuously housed peers. Journal of School Psychology, 42, 179–199. U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA: Author. Retrieved from http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBenefitCostTechnicalDocumentation.pdf Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf |
HOU005 Homelessness reduction leading to reduced chronic illness for children
Equation | (# children receiving services) x (% families reduce homelessness solely because of this program) x (Percentage point decrease in the probability of chronic illness) x (QALY increase) x ($ QALY) |
Explanation | The metric estimates the impact of avoided or reduced homelessness for children on the likelihood of chronic illness, estimated in quality-adjusted life years (QALY). Number of children in families receiving housing: Reported by program Percent of these adults who avoid homelessness due the program: [0.52], for programs serving individuals at imminent risk of homelessness (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. To this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Percentage point decrease in the probability of chronic illness: [0.064], we estimate a 6 percentage point difference in the probability of chronic illness for homeless individuals compared to an individual who are not homeless, based on findings from the National Center for Family Homelessness (1999) and the Family Housing Fund (1999) that, controlling for important covariates, approximately 16 percent of poor children in poverty who are homeless suffer chronic illness, whereas only 9 percent of poor children who are not homeless suffer chronic illness. QALY increase: [0.1], we estimate a 0.10 QALY value for the avoidance of chronic illness based on the average difference in QALY between those with totally controlled versus not well-controlled asthma (Briggs, Wallace, Clark & Bateman, 2006). Asthma is the most prevalent chronic illness afflicting poor children, so it provides an appropriate yet conservative guess for the cost of chronic illness—conservative because homeless children are twice as likely to suffer from at least one chronic illness. $ value per QALY: [$50,000]. Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Briggs, A., Wallace, M., Clark, T. & Bateman, E. (2006). Cost-effectiveness of asthma control: An economic appraisal of the GOAL study. Allergy, 61, 531-536. National Center for Family Homelessness. (1999). Homeless Children: America’s new outcasts. Newton Centre, MA: Author. Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf Wilder Research. (2017). Homelessness in Minnesota: Youth on their own. Findings from the 2015 Minnesota Homeless Study. Retrieved from: http://mnhomeless.org/minnesota-homeless-study/reports-and-fact-sheets/2015/2015-homeless-youth-4-17.pdf |
HOU006 Homelessness reduction leading to reduced lifetime costs to children associated with out of home placement
Equation | (# children receiving services) x (% families reduce homelessness solely because of this program) x (% of children who avoid foster care solely because of the program) x (Counterfactual rate of child abuse) x ($ benefit from reduced out of home placement) |
Explanation | This metric estimates the impact of avoided or reduced homelessness leading to reduced out-of-home placements and subsequent improved lifetime health, estimated in terms of quality-adjusted life years (QALY). Number of children in families receiving housing: Reported by program Percent of these adults who avoid homelessness due to the program: [0.52], for programs serving individuals at imminent risk of homelessness (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. To this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Percentage of children who avoid foster care solely because of the program: [49%], we estimate the average impact of housing on out of home placement from Lenz-Rashid (2013) and Fowler & Schoeny (2015). Benefit from reduced out of home placement: [$350,000], estimate of the value of preventing child abuse (in terms of lost QALYs) is based on the findings of Peterson, et al (2015). Benefits already discounted to present value. |
References | Fowler, P. J., & Schoeny, M. (2015). The Family Unification Program: A randomized controlled trial of housing stability. Child Welfare, 94(1), 167–187. Lenz-Rashid, S. (2013). Supportive housing for homeless families: Foster care outcomes and best practices. Sierra Health Foundation. Retrieved from https://www.sierrahealth.org/assets/pubs/Cottage_Housing_Report_May_2013_Web.pdf. Minnesota Department of Human Services (2018). Child protection in Minnesota: Keeping children safe. https://edocs.dhs.state.mn.us/lfserver/Public/DHS-4735-ENG Peterson, C., Florence, C., & Klevens, J. (2018). The economic burden of child maltreatment in the United States, 2015. Child Abuse and Neglect, 86, 178–183. https://doi.org/10.1016/j.chiabu.2018.09.018 Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf Wilder Research. (2017). Homelessness in Minnesota: Youth on their own. Findings from the 2015 Minnesota Homeless Study. Retrieved from: http://mnhomeless.org/minnesota-homeless-study/reports-and-fact-sheets/2015/2015-homeless-youth-4-17.pdf |
HOU007 Homelessness reduction leading to reduced hospitalizations
Equation | (# participants) x (% participants getting supportive housing solely because of the program) x (% decrease in hospitalizations due to program) x (% participants hospitalized as a result of a physical illness) x (QALY increase) x ($QALY) |
Explanation | This metric estimates the impact of avoided or reduced homelessness on the rate of hospitalization, estimated in quality-adjusted life years (QALY). Number of participants: Reported by program. Percent of these adults who avoid homelessness due the program: [0.52], for programs serving individuals at imminent risk of homelessness (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. To this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Percentage of decrease in hospitalizations due to program: [0.30], based on Culhane, Metreaux & Hadley (2001); Martinez & Burt (2006); and Sadowski, Kee, VanderWeele & Buchanan (2009) Percentage of patients hospitalized as a result of physical illness: [0.20], for those who are housed in supportive housing and who avoided hospitalization or would have been hospitalized due to some general diagnosis, based on the findings of Salit, Kuhn, Hartz, Vu & Mosso (1998). QALY increase: [0.025], estimate for the value of avoiding hospitalization due to general illness (Lavelle, Meltzer, Gebremarian, Lamarand, Fiore & Prosser, 2011). $ value per QALY: [$50,000]. Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Culhane, D. P., Metreaux, S. & Hadley, T. (2001). The impact of supportive housing for homeless people with severe mental illness on the utilization of the public health, correcting, and emergency shelter systems: The New York-New York Initiative. Washington, DC: Fannie Mae Foundation. Retrieved from https://www.researchgate.net/publication/228638508_The_Impact_of_Supportive_ Housing_for_Homeless_People_with_Severe_Mental_Illness_on_the_Utilization_of_ the_Public_Health_Corrections_and_Emergency_Shelter_Systems_The_New_York- New_York_Initiative Lavelle, T.A., Meltzer, M. I., Gebremariam, A., Lamarand, K., Fiore, A.E. & Prosser, L.A. (2011). Community-based values for 2009 pandemic influenza A H1N1 illnesses and vaccination-related adverse events. PLoS One, 6(12). E27777. Martinez, T. E. & Burt, M. (2006). Impact of permanent supportive housing on the use of acute care health services by homeless adults. Psychiatric Services: A Journal of the American Psychiatric Association, 57(7), 992-999. Sadowski, L., Kee, R., VanderWeele, T. & Buchanan, D. (2009). Effect of a housing and case management program on emergency department visits and hospitalizations among chronically ill homeless adults: A randomized trial. Journal of the American Medical Association, 301(17), 1771-1778. Salit, S., Kuhn, E., Hartz, A., Vu, J. & Mosso, A. (1998). Hospitalization costs associated with homelessness in New York City. New England Journal of Medicine, 338(24), 1734-1740. Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf |
HOU008 Crisis apartments for individuals experiencing domestic violence leading to savings to renters
Equation | (# families placed in crisis housing) x (% of these renters obtain their apartment solely because of this program) x (# length of crisis housing stay) x ($ value of an efficiency apartment) |
Explanation | This metric estimates the impact of crisis apartments on avoided rental costs for individuals experiencing domestic violence. This metric measures the value of the housing only; other benefits generated by these programs are captured in other metrics. Crisis apartments, for the purpose of this metric, include relocation services that result in housing or direct housing support. Number of families placed in crisis housing: Reported by program Percentage of these renters who obtain their apartment solely because of this program: [% of participants who obtained an apartment], estimated by the Constellation Fund staff for each program. We usually use 100% for programs serving individuals at imminent risk or after identified as victims. Domestic violence is the third leading cause of homelessness among families. 50% of women who are homeless report that domestic violence was the immediate cause of their homelessness, according to Theresa’s Fund, Domestic Shelters. (2014). Length of housing stay: [# Months], estimated using data from program participants $ value of an efficiency apartment: [$1,161], average monthly rent for apartments in the Twin Cities metropolitan area (U.S. Department of Housing and Urban Development, 2017). Use actual program data whenever available. Note: discount to present value if length of stay is more than 3 years. |
References | Theresa’s Fund, Domestic Shelters. (2014). Domestic Violence Statistics. Retrieved from https://www.domesticshelters.org/domestic-violence-articles-information/faq/domestic-violence-statistics U.S. Department of Housing and Urban Development. (2017). Comprehensive housing market analysis. Minneapolis-St. Paul-Bloomington, Minnesota-Wisconsin. Retrieved from https://www.huduser.gov/portal/publications/pdf/MinneapolisMN-comp-17.pdf |
HOU009 Transitional apartments leading to savings to renters
Equation | (# families placed in transitional housing) x (% of these renters who obtain their apartment solely because of this program) x (# length of transitional housing stay) x ($ value of an efficiency apartment) |
Explanation | This metric estimates the impact of transitional housing on avoided rental costs for homeless or housing unstable individuals. Transitional housing can be described as a home that assists people in transitioning from homelessness, substance abuse addiction, abuse in the home, or other types of poor living environments to stable housing. This metric measures only the value of the housing provided. Other benefits from reduced substance abuse or homelessness are estimated using other metrics. Number of individuals placed in transitional housing: Reported by program Percent of these adults who avoid homelessness due to the program: [0.52], for programs serving individuals at imminent risk of homelessness (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. To this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Number of transitional housing stays: [# Months], estimated using data from program participants $ value of an efficiency apartment: [$1,161], average monthly rent for apartments in the Twin C metropolitan area (U.S. Department of Housing and Urban Development, 2017). Use program data whenever available. Note: discount to present value if length of stay is more than 3 years. |
References | U.S. Department of Housing and Urban Development. (2017). Comprehensive housing market analysis. Minneapolis-St. Paul-Bloomington, Minnesota-Wisconsin. Retrieved from https://www.huduser.gov/portal/publications/pdf/MinneapolisMN-comp-17.pdf. Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf |
HOU010 Homelessness reduction leading to avoided deaths of unsheltered adults living with H.I.V.
Equation | (# unsheltered homeless adults) x (% of unsheltered homeless who get assistance solely because of the program) x (% unsheltered homeless would die without intervention) x (% unsheltered homeless saved from death due to the program) x ($ value per life saved) |
Explanation | This metric estimates the impact of reduced homelessness for previously unsheltered adults living with H.I.V. on life expectancy, estimated in quality-adjusted life years (QALY). Number of unsheltered adults experiencing homelessness: Reported by program. Percentage of participants who get assistance solely because of the program: [45%], about 45% of unsheltered homeless would stay outside (Wilder Research, 2016). Percentage of unsheltered adults who would likely die without intervention: [14%], the odds of death of HIV positive individuals who are homeless are 27 times higher than individuals with housing Spinelli, et al. (2019). We estimate the probability of death between ages 15-64 to be 3.7% based on mortality statistics reported for 2016 by the Minnesota Department of Health. Thus, the estimated likelihood of death of percent of HIV-homeless is: (0.037/0.27 = 0.14) $ value per life saved: We estimate the value of a life based on a [$50,000] QALY. This value varies by the age and expected years of life of each participant. Thus, we compute the total benefits of a program based on specific program data on participant’s age and discount the annual value to present value using Constellation’s standard discounting method. |
References | Center for Disease Control and Prevention (2017). Retrieved from https://www.cdc.gov/features/homelessness/index.html Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. Report to Robin Hood. New York, NY: Robin Hood. Morrison, D.S. (2009). Homelessness as an independent risk factor for mortality: results from a retrospective cohort study. International Journal of Epidemiology, 38 (3), 877-83. MN Compass (2010). Retrieved from https://www.mncompass.org/_pdfs/presentations/BlueCross_HealthInequities_10-10.pdf Minnesota Department of Health, Center for Health Statistics. Retrieved from: https://mhsq.web.health.state.mn.us/birth/queryFrontPage.jsp?goTo=deathInterfaceSas.jsp&queryPage=deathInterfaceSas.jsp&startQuery=true Schermer, T. R., Thoonen, B. P, van den Boom, G., Akkermans, R. P., Grol, R. P., Folgering, H. T., van Weel, C. & van Schayck, C. P. (2002). Randomized controlled economic evaluation of asthma self-management in primary health care. American Journal of Respiratory and Critical Care Medicine, 166, 1062-1072. Spinelli, M. et al. (2019) Homelessness at diagnosis is associated with death among people with HIV in a population-based study of a US city. AIDS: September 1, 2019 – Volume 33 – Issue 11 – p 1789–1794 Wilder Research. (2016). 2015 homeless adults and children: Minnesota statewide survey data. . . Retrieved from http://mnhomeless.org/minnesota-homeless-study/detailed-data-interviews/2015/HennepinCountyMN_Adult2015_Tables51-67.pdf |
HOU011 Stable housing for individuals living with H.I.V. leading to improved health
Equation | (Number of H.I.V. diagnosed participants) x (Q1: Reduced probability of adequate CD4 cell count due to the intervention) x (Q2: Additional years of life expectancy from improved H.I.V. biomarkers, CD4) x ($ QALY) |
Explanation | This metric estimates the impact of stable housing on improved health for individuals living with H.I.V., estimated in terms of quality-adjusted life years (QALY). Number of H.I.V. diagnosed participants: Reported by program. Q1: Reduced probability of adequate CD4 cell count due to the intervention: [28%], Galarraga, et al, (2018) this is the reduction in the probability of showing adequate CD4 markers for individuals with stable housing vs. those with unstable housing situations such as living in shelters. Q2: # additional years of life expectancy from improved H.I.V. biomarkers, CD4: [1.1], based on the findings of Walenski, et al. (2010), we estimated that H.I.V. infected individuals with higher levels of CD4 live 1.1 more years than individuals with lower levels of the biomarker. $ value per QALY: [$50,000]. Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Galárraga, O., et al. (2018). The effect of unstable housing on HIV treatment biomarkers: An instrumental variables approach. Social Science & Medicine, 214, 70–82. Walensky, R., et al. (2010). Forecasting the Impact of a Comprehensive HIV Strategy in Washington DC. Clinical Infectious Diseases, 51(4), 392-400. |
HOU012 Stable housing for mentally ill individuals leading to improved health
Equation | (# participants hospitalized) x (% participants get assistance solely because of the program) x (% participants hospitalized as a result of a mental illness) x (% decrease in hospitalizations) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of stable housing on decreased hospitalizations as a result of mental illness, estimated in terms of quality-adjusted life years (QALY). Number of participants hospitalized during the year: Reported by program. Percent of these adults receiving treatment due to the program: [0.52], in absence of program relevant data, we use the counterfactual used for homelessness reduction interventions. That is, for programs serving individuals at imminent risk of homelessness (e.g. individuals coming from shelters, or with eviction notices and no feasible housing alternative), we assume a 100% rate of effectiveness. To this number, we subtract the percentage of homeless individuals in the Twin Cities metropolitan area who are on a waiting list for any public housing, Section 8 housing, or some other type of housing that offers financial assistance as a counterfactual [48%] (Wilder Research, 2016). Percentage of participants hospitalized as a result of a mental illness: [80%], we estimate that 80 percent of those who are housed in supportive housing and who avoided hospitalization would have been hospitalized due to mental illness or substance abuse conditions based on research indicating that approximately 80 percent of homeless people have primary or secondary mental illness/substance abuse conditions (Salit, Kuhn, Hartz, Vu & Mosso, 1998). Percentage decrease in hospitalizations: [30%], decrease in hospitalizations for people receiving supportive housing (Culhane, Metreaux & Hadley, 2001; Martinez & Burt, 2006; Sadowski, Kee, VanderWeele & Buchanan, 2009). If research studies find a range of results, we will apply the lowest estimate. QALY increase: [0.33], value of avoiding hospitalization for mental illness/substance abuse conditions at 0.33 QALY, by averaging the QALY values for the avoidance of depression, estimated at 0.30 QALY (especially Frank, McGuire, Normand & Goldman, 1999; Schoenbaum, Sherbourne & Wells, 2005), and avoiding relapse of schizophrenia, estimated at 0.36 QALY (Davies et al., 2008). $ value per QALY: [$50,000]. Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Culhane, D. P., Metreaux, S. & Hadley, T. (2001). The impact of supportive housing for homeless people with severe mental illness on the utilization of the public health, correcting, and emergency shelter systems: The New York-New York Initiative. Washington, DC: Fannie Mae Foundation. Davies, A., Vardeva, K., Loze, J., L’Italien, G., Sennfalt, K. & van Baardewijk, M. (2008). Cost-effectiveness of atypical antipsychotics of the management of schizophrenia in the UK. Current Medical Research and Opinion, 24(11), 3275–3285. Frank, R., McGuire, T., Normand, S. & Goldman, H. (1999). The value of mental health care at the system level: The case of treating depression. Health Affairs, 18(5), 71–88. Martinez, T. E. & Burt, M. (2006). Impact of permanent supportive housing on the use of acute care health services by homeless adults. Psychiatric Services (Washington, D.C.), 57(7), 992–999 |
HOU013 Housing quality improvements leading to health benefits for adults
Equation | (# adults receiving services) x (% improved houses solely because of this program) x (% health and safety problems present in the home are resolved) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of housing quality improvements on improved health for adults, specifically reduced asthma triggers, estimated in terms of quality-adjusted life years (QALY). Percentage of adults receiving housing: Reported by program % of improved houses solely because of this program: [100%], the percentage of houses that receive services helpful enough to improve health conditions solely because of this program is estimated by Constellation Fund staff using program data. We assume a 100% rate of service for programs covering 100% of the cost of the renovation and serving individuals who would have not been able to do improvements without the support of the program. If the program covers a fraction of the improvements, we use the percent of costs paid by the program. Percentage of health and safety problems present in the home which are resolved: [20%], the estimated benefits in this metric depend on the extent to which the health and safety problems are present in the homes of participants. We use program data to determine this incidence. If no program data is available, we use 20% as the prevalence rate. Adult Current Asthma Prevalence by Race/Ethnicity, BRFSS, 2008 = 11.3% (CDC, 2015). The prevalence of asthma among black individuals is twice the rate in the general population (CDC, 2008). Thus, we approximate the rate of low-income individuals to be at least 20% (1.75 x 11.3). (Since we do not know rates of asthma by income in Minnesota, we use the rates for black adults as a proxy for low-income adults.) QALY increase: [0.02], we estimate a 0.02 QALY value for the remediation of asthma triggers subsequent to housing quality improvement based on the findings of Muennig, Glied & Simon (2005), who report that a comprehensive asthma intervention of medical, education and self-help support produces a 0.05 QALY improvement in the lives of the patients, while medical-only care produces on average 0.03 QALY improvement (Schermer et al, 2002). We apply the 0.02 QALY difference (0.05 – 0.03 = 0.02) as the value of the decrease in home-based asthma triggers through the improvement in quality of the home environment. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Division of Population Health. BRFSS Prevalence & Trends Data [online]. 2015. [accessed Sep 24, 2018]. URL: https://www.cdc.gov/brfss/brfssprevalence/ Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion (2008). Asthma in MN. Retrieved from: https://www.cdc.gov/asthma/stateprofiles/Asthma_in_MN.pdf Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. Report to Robin Hood. New York, NY: Robin Hood. Schermer, T. R., Thoonen, B. P, van den Boom, G., Akkermans, R. P., Grol, R. P., Folgering, H. T., van Weel,C. & van Schayck, C. P. (2002). Randomized controlled economic evaluation of asthma self-management in primary health care. American Journal of Respiratory and Critical Care Medicine, 166, 1062-1072. |
HOU014 Housing quality improvements leading to health benefits for children
Equation | (# adults receiving services) x (% improved houses solely because of this program) x (% health and safety problems present in the home are resolved) x (# QALY increase) x ($ QALY) |
Explanation | This metric estimates the impact of housing quality improvements on improved health for children, specifically reduced asthma triggers, estimated in terms of quality-adjusted life years (QALY). Percentage of adults receiving housing: Reported by program % of improved houses solely because of this program: [100%], the percentage of houses that receive services helpful enough to improve health conditions solely because of this program is estimated by Constellation Fund staff using program data. We assume a 100% rate of service for programs covering 100% of the cost of the renovation and serving individuals who would have not been able to do improvements without the support of the program. If the program covers a fraction of the improvements, we use the percent of costs paid by the program. Percentage of health and safety problems present in the home which are resolved: [8.2%], the estimated benefits in this metric depend on the extent to which the health and safety problems are present in the homes of participants. We use program data to determine this incidence. If no program data is available, we use the following prevalence rates for the Twin Cities metropolitan area: We use 8.2% Estimated based on the Adult Self-Reported Current Asthma Prevalence Rate (Percent) and Prevalence (Number) by Income and State or Territory, BRFSS 2010 = 9.6%. This is the rate for individuals in Minnesota with an income less than $25,000/year. (Centers for Disease Control and Prevention, 2015). The prevalence rate for children is not available at the MSA level. However, at the state level we know that children’s rate is 14 percent lower than adult’s rate of current asthma. We adjust the adult rate by this percentage to approximate the prevalence among children at 8.2%. QALY increase: [0.02], we estimate a 0.02 QALY value for the remediation of asthma triggers subsequent to housing quality improvement based on the findings of Muennig, Glied & Simon (2005), who report that a comprehensive asthma intervention of medical, education and self-help support produces a 0.05 QALY improvement in the lives of the patients, while medical-only care produces on average 0.03 QALY improvement (Schermer et al, 2002). We apply the 0.02 QALY difference (0.05 – 0.03 = 0.02) as the value of the decrease in home-based asthma triggers through the improvement in quality of the home environment. $ value per QALY: [$50,000] Benefits are then discounted to present value based on the average age of participation to life expectancy. |
References | Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Division of Population Health. BRFSS Prevalence & Trends Data [online]. 2015. [accessed Sep 24, 2018]. URL: https://www.cdc.gov/brfss/brfssprevalence/ Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion (2008). Asthma in MN. Retrieved from: https://www.cdc.gov/asthma/stateprofiles/Asthma_in_MN.pdf Muennig, P., Glied, S. & Simon, J. (2005). Estimation of the health benefits produced by Robin Hood Foundation grant recipients. Report to Robin Hood. New York, NY: Robin Hood. Schermer, T. R., Thoonen, B. P, van den Boom, G., Akkermans, R. P., Grol, R. P., Folgering, H. T., van Weel, C. & van Schayck, C. P. (2002). Randomized controlled economic evaluation of asthma self-management in primary health care. American Journal of Respiratory and Critical Care Medicine, 166, 1062-1072. |
ECO001 Employment programs leading to increased earnings
Equation | (# participants who find employment due to the program) x (# total time of paid work) x ($ net increase in earnings) |
Explanation | This is a generic metric. The actual estimation depends on the availability of outcome data from the program. Employment programs may include: Job training programs, job placement programs, programs that provide direct employment to participants. Data requests for programs seeking to improve employment should include the following items.
Use the following examples of procedures and assumptions as starting point:
|
References | Card, D., Kluve, J., & Weber, A. (2017). What works? A meta-analysis of recent active labor market program evaluations. National Bureau of Economic Research. Working Paper 21431. Retrieved from: http://www.nber.org/papers/w21431 Council of Economic Advisers. (2016). Active labor market policies: Theory and evidence for what works. [Issue Brief]. Retrieved from https://obamawhitehouse.archives.gov/sites/default/files/page/files/20161220_active_labor_market_policies_issue_brief_cea.pdf Heinrich, C. J., Mueser, P. R., Troske, K. R., Jeon, K.S., & Kahvecioglu, D. C. (2013). Do public employment and training programs work? IZA Journal of Labor Economics, 2(1), 6. Minnesota Compass. (2018). Twin Cities region neighborhood profile data: Phillips community. Retrieved from http://www.mncompass.org/profiles/communities/minneapolis/phillips U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov |
ECO002 Reduced recidivism leading to increased earnings
Equation | (# participants) x [(% participants who are re-incarcerated after program) – (counterfactual recidivism rate)] x (Impact of re-incarceration on earnings) x ($ average annual earnings of formerly incarcerated individuals) |
Explanation | This metric estimates the impact of reduced recidivism on short-term earnings. To use this metric, the program must capture and be able to report a 3-year recidivism rate. Number of participants: Provide by grantee. Percentage of participants who are re-incarcerated after program: Provide by grantee. Counterfactual recidivism rate: [0.25]. 3-year recidivism rate. (Minnesota Department of Corrections, 2016). [CFT18]. Impact of re-incarceration on earnings: [0.98]. Re-incarcerated individuals earn just 2% of what formerly incarcerated individuals earn during the year of re-incarceration. Estimated from Western and Sirois (2017). Average annual earnings of formerly incarcerated individuals: Data provide by program. Otherwise, use the average annual earnings of formerly incarcerated individuals [$2,000], computed using ACS data (U.S. Census Bureau, 2016). Benefits are computed for one year of additional earnings after program participation. |
References | Minnesota Department of Corrections. (2016). Performance report, 2016. Retrieved from https://mn.gov/doc/assets/2016-DOC-Performance-Report-for-web_tcm1089-299033.pdf Western, B., & Sirois, C. (2017). Racial inequality in employment and earnings after incarceration. Harvard University. Retrieved from https://www.semanticscholar.org/paper/Racial-Inequality-in-Employment-and-Earnings-after-Western-Sirois/4a382dfc2efc093c85274edb81957b59a0eec6b1 U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the Seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov |
ECO003 Microenterprise programs leading to increased earnings of entrepreneurs
Equation | (# firms or entrepreneurs) x ($ Additional average annual entrepreneur’s earnings) x (# total firm-years of operation after loan) |
Explanation | Estimates for start-up and existing businesses should be conducted separately. Start-up and existing businesses have different survival rates and may also show different sales and earnings profiles. In the case of microcredit programs, the estimation of benefits do not include any assumption about the effectiveness of the lending model used by the microcredit institution. For example, microcredit models may include different types of loans (individual or group lending), loan caps, the relative importance of loans to business scale, or underwriting policies, among many other lending practices and policies. The Constellation Fund collects information on these characteristics but does not conduct new evaluations to generate new knowledge about the effectiveness of lending technologies. Additional average annual entrepreneur’s earnings: This is the difference in the average annual earnings before and after the microloan. This information is provided by the grantee. In some cases, business sales or revenues are the only data available (as opposed to profits or net earnings). In those cases, we suggest using a 10 percent margin of profit over gross revenues to proxy net personal earnings. All dollar values should be discounted to present value. Technical note on estimation of earnings: Pre-loan earnings are used as the counterfactual state. This assumption may cause some estimation bias for repeat borrowers. First-time borrowers tend to experience greater relative increases in profits, especially for newer businesses. We suggest computing benefits by type of borrower (new or repeat borrowers), or by including a number of previous loans in a regression estimation. See Diaz (2013) for an econometric application. Total number of firm-years of operation after loan: This is the sum of the years of operation of all firms. We include up to six years of benefits after loan is received. The six-year timeframe is based on survival rates data for firms in Minnesota that show that after six years the survival rate falls below 50% (The Constellation Fund’s own estimations based on data from the U.S. Bureau of Labor Statistics, 2013).For example, if the microcredit program has given loans to three firms with the following number of years operations after the loan: Note: discount to present value if benefits are assumed to last for more than 3 years. |
References | Diaz, J. Y. (2013). Impact of technical assistance and microcredit among rural households in El Salvador. Retrieved from the University of Minnesota Digital Conservancy, http://hdl.handle.net/11299/148729. Rossman, S., Theodos, B., Brash, R., Gallagher, M., Hayes, C., & Temkin, K. (2008). Key findings from the evaluation of the Small Business Administration’s Loan and Investment Programs. Urban Institute. Retrieved from https://www.urban.org/research/publication/key-findings-evaluation-small-business-administrations-loan-and-investment-programs. The U.S. Bureau of Labor Statistics. (2013). Business employment dynamics: Establishment age and survival data, Minnesota, Table 7: Survival of private sector establishments by opening year. Retrieved from https://www.bls.gov/bdm/mn_age_total_table7.txt |
ECO004 Microenterprise programs leading to increased earnings of employees
Equation | (# employees) x ($ Additional average annual entrepreneur’s earnings) x (# total employee-years after loan) |
Explanation | Metric can be used for microlending or microenterprise support. Additional average annual employees’ earnings: This is the difference in the average annual earnings before and after the microloan. Employees’ average annual wages post-loan are provided by the grantee. Employees’ average annual wages pre-loan would depend on the demographic characteristics and educational attainment of employees and the industry and type of jobs. The Constellation Funds staff will determine what the appropriate counterfactual wage for the grantee is. Most counterfactual wages used by the Constellation Fund are estimated using American Community Survey (ACS) 5-year estimates from the U.S Census Bureau, (2016). Total number of years-employees after loan: This is the sum of all employees that worked during the years of operation of all firms. We include up to six years of operations after loan is received. The six-year timeframe is based on survival rates data for firms in Minnesota that show that after six years the survival rates fall below 50%. For example, if the program provides loans to three firms with the following number of years operations and employees after the loan: Firm 1 was open for eight years after receiving the loan and employs an average of five workers. The total number of employee-years is: (6 years x 5 workers) + (4 years x 7 workers) + (4.2 years x 6 workers) = 83.2 Note that we only count the first six years of operation of Firm 1. For Firm 2, we only count the four years that it was open. Finally, Firm 3 has been open for three years at the time of the evaluation, but it is possible that it will remain open for three more years. We want to include the potential future earnings that the entrepreneur could earn if the business remains open for a total of six years after the loan. To estimate these potential future benefits, we use survival rates from Rossman, et al (2008) as proxy for the probability that firms currently open but with less than six years of operation will remaining open for six years. These are rates for firms receiving loans from the Small Investment Company Loan Program (SBIC) from the Small Business Administration. This program provides loans to small businesses through community investment intermediaries. We use the lower bound rates to account for smaller, more vulnerable businesses likely to be served by grantees. For example, Firm 3 has been open for three years and the survival rate at year four of start-up firms is 45%, the rate at year five and six are 44% and 35% respectively. Thus, the total number of years we expect Firm 3 to be open is: 3 + 0.45 + 0.44 + 0.35 = 4.2 Note: Discount to present value if benefits are assumed to last for more than 3 years. |
References | Diaz, J. Y. (2013). Impact of technical assistance and microcredit among rural households in El Salvador. Retrieved from the University of Minnesota Digital Conservancy, http://hdl.handle.net/11299/148729. Rossman, S., Theodos, B., Brash, R., Gallagher, M., Hayes, C., & Temkin, K. (2008). Key findings from the evaluation of the Small Business Administration’s Loan and Investment Programs. Urban Institute. Retrieved from https://www.urban.org/research/publication/key-findings-evaluation-small-business-administrations-loan-and-investment-programs. The U.S. Bureau of Labor Statistics. (2013). Business employment dynamics: Establishment age and survival data, Minnesota, Table 7: Survival of private sector establishments by opening year. Retrieved from https://www.bls.gov/bdm/mn_age_total_table7.txt |