METRICS DEEP DIVE
Units of Impact: Expected average impact per-participant on an outcome in t-years after the program begins equals the difference in average outcome among potential participants between two possibilities:
- If all participants get the program due to a decision to fund the proposal; or,
- If no participants get the program due to a decision not to fund the proposal.
Value Per Unit: Monetary value of a unit change in outcome at year-t as valued by low-income individuals.
Discount Rate: Size ranges depending on factors such as the level risk that the program would not produce the expected benefits.
T: Number of years that program effects are expected.
- The Constellation Fund calculates private monetary benefits for program participants at or below 185% of the federal poverty guideline.
- All monetary values are converted to constant dollars.
- Present values are calculated using discount rates of 2%-4% of future values, depending on various programmatic factors.
- All earnings are net of taxes. For state taxes, Constellation uses effective tax rates by income percentile from the Minnesota 2017 Tax Incidence Study (Minnesota Department of Revenue, 2017). For federal taxes, Constellation uses federal effective tax rates by expanded cash income percentile reported by The Tax Policy Center (Urban Institute and Brookings Institution, 2016).
- As a general rule, Constellation checks that the nonprofit’s intervention matches as closely as possible with the program models observed in the literature. Other matching factors may include characteristics of the populations served, the intensity or dosage of program, and the quality of program.
- Constellation’s quantitative evaluation does not include any differential weights on dollar gains for participants based on income or other factors. The Robin Hood Foundation experimented with using weights on gains but found no basis for choosing the weights, and, regardless, their weighting methods did not affect the relative ranking of grant proposals (Weinstein and Bradburd, 2013).
- Constellation’s quantitative evaluation does not include the value of any cash and noncash public transfer payments or benefits received or potentially lost due to increases in income.
- In evaluating average annual post-program earnings, Constellation considers potential issues with data reported by a program, including bias/error from self-reported earnings and unreliable or missing data. Ideally, data comes from tax returns, verified income from employers, the Department of Employment and Economic Development, or other government agencies.
- Most subpopulations based on demographics are computed by Constellation using American Community Survey (ACS) micro data for the Twin Cities or smaller regions. If the sample size is too small for specific subgroups (e.g. race, immigrant status, or level of education), Constellation computes marginal effects by population group from the Twin Cities or larger regional samples. If the population of interest is not found in Census data, Constellation uses third party reports.
- In determining the counterfactual state of earnings or other outcomes in the absence of any program, Constellation uses average earnings or other status factors of the target population in the Twin Cities metropolitan area based on Census data. This average value or rate serves as an ad-hoc threshold for program impact.
- Minnesota Department of Employment and Economic Development. (2017). Minnesota unemployment statistics. Retrieved from https://apps.deed.state.mn.us/lmi/laus/Default.aspx
- Minnesota Department of Revenue. (2017). 2017 Minnesota tax incidence study. Retrieved from http://www.revenue.state.mn.us/research_stats/research_reports/2017/2017_tax_incidence_study_links.pdf
- Urban Institute and Brookings Institution Tax Policy Center. (2016). Average effective federal tax rates – all tax units by expanded cash income percentile, 2016 [Model estimates – Distribution tables by percentile]. Retrieved from http://www.taxpolicycenter.org/model-estimates/baseline-average-effective-tax-rates-july-2016/t16-0092-average-effective-federal
- U.S. Census Bureau. (2016). American Community Survey 5-year estimates – public use microdata sample, 2012-2016. Generated using Public Use Microdata Area (PUMA) in the seven-county Twin Cities Metropolitan Area. Retrieved from http://factfinder.census.gov
- Weinstein, M. & Bradburd, R. (2013). The Robin Hood Rules for Smart Giving. New York: Columbia University Business School Press.