«April 2007 Federal Deposit Insurance Corporation Abstract: Using data collected from a pre-training survey, post-training survey, and telephone ...»
A Longitudinal Evaluation
of the Intermediate-term Impact
of the Money Smart
A Longitudinal Evaluation
of the Intermediate-term Impact of the
Money Smart Financial Education Curriculum
upon Consumers’ Behavior and Confidence
Federal Deposit Insurance Corporation
Abstract: Using data collected from a pre-training survey, post-training survey, and telephone
follow-up survey, this study analyzes the impact of the Money Smart financial education curriculum upon the financial opinions and behaviors of course participants during the survey period. The data indicate that Money Smart financial education training positively affected consumer behaviors and that behavior changes were demonstrated many months after completing the training. Among the significant findings were that participants were more likely to open deposit accounts, save money in a mainstream deposit product, use and adhere to a budget, and have increased confidence in their financial abilities when contacted six to twelve months after completing the Money Smart course than they were before taking the course.
Comments or questions about this study should be directed to firstname.lastname@example.org.
Table of Contents Acknowledgements
The Money Smart Financial Education Curriculum
Discussion of Findings
A. Results and observations from respondents starting Money Smart classes
1. Checking and Savings Accounts
B. Results and Observations from the Post-Training Survey
1. Checking and Savings Accounts
C. Results and Observations from the Telephone Follow-up Survey and Repeated Measurements Tests
1. Some Basic Results from the Telephone Follow-up Survey
2. Longitudinal Results from the Telephone Follow-up Survey and Six Repeated.............. 23 Measurement Tests
3. Longitudinal Analysis for Opinion Statements Regarding Financial Confidence............ 36 D. Comments with respect to demographic variables
Appendix A: The FDIC’s Implementation of Money Smart
Appendix B: Survey Instruments
Appendix C: Development of the Survey Instruments
Appendix D: Procedures for Implementing the Surveys
Appendix E: Summary of Survey Sites
Appendix F: Call Design for Telephone Follow-up Survey
Appendix G: Statistical Details
Appendix H: Research Limitations
Appendix I: References
Appendix J: Statistical Tables
Acknowledgements This report represents the culmination of the hard work and cooperation of many people both
inside and outside the FDIC. Those outside the FDIC include:
The instructors, staff, and volunteers working at all of the Money Smart survey sites. Many of these individuals participated in training, conducted Money Smart training sessions, administered pre- and post-training surveys, and forwarded surveys to the Gallup Organization.
Survey respondents. Their participation in Money Smart courses and their willingness to complete the pre-training, post-training, and follow-up surveys provided the base for the analysis contained in this report.
NeighborWorks America and its staff, especially Milton Sharp, for their efforts in assisting in the creation of the survey instruments, testing the instruments, selection of and coordination with survey sites, and for their feedback throughout the survey.
The Gallup Organization for their expert contributions to creating the survey instruments;
planning, monitoring, and controlling the administration of the pre- and post-training surveys; conducting the telephone follow-up survey; collecting the raw data and preparing response data files for all three surveys; and providing assistance and guidance to FDIC staff throughout the study.
Within the FDIC, this survey was the result of an interdivisional team from the FDIC’s Division of Supervision and Consumer Protection (DSC) and Division of Insurance and Research (DIR),
The study management team: Elaine Drapeau, Fair Lending Specialist, DSC; Bobbie Jean Norris, Special Assistant to the Deputy Director for Examination Policy and Oversight, DSC; and Dr. David Chapman, Chief Statistician, DIR. Ms. Drapeau initiated the study while a member of the Community Affairs staff. As the initial Project Manager, she was instrumental in the development of the pre-training and post-training surveys, the design of the survey process, and establishing the parameters for site selection and course requirements. Ms. Drapeau assumed a new position prior to the start of the survey. Ms.
Norris assumed responsibility for the management of the study during the site selection process, and served as the Project Manager for the remainder of the study. She was responsible for coordinating study activities with NeighborWorks America, the Gallup Organization, and all FDIC parties. Dr. Chapman served as special advisor for the duration of the study, providing expert technical advice and counsel on the development of survey instruments and the administration of the survey. Ms. Norris and Dr. Chapman provided detailed reviews of all project reports, and collaborated with the contractor on the development of the follow-up survey and on a multitude of decisions made throughout the study to ensure the integrity of the study.
Introduction Prior research in the area of basic financial education has not considered the longer-term impact on consumer behavior, or results, from a comprehensive adult financial education curriculum such as Money Smart. Therefore, this study was designed to explore whether training in financial education classes using the curriculum resulted in positive changes in respondent money management skills and behavior over several months.
This study, which the FDIC conducted in cooperation with NeighborWorks America (NWA), 1 consisted of a three-part survey of participants in Money Smart courses 2 across the country. The FDIC engaged The Gallup Organization to assist with the development of the survey questions and to administer the survey. The assessment used a pre-training survey to gather baseline data on students’ knowledge, behaviors and confidence, a post-training survey to gather data on changes in students’ knowledge, behaviors, confidence and their future intentions, and a telephone follow-up survey six to twelve months 3 after the conclusion of the training to identify changes in the students’ financial behaviors over the six to twelve month period from completion of the training.
The survey results 4 suggest that Money Smart financial education training covering the basics of checking, savings, budgeting, and credit can positively change consumer behavior and improve financial confidence during a six to twelve month time period following the course. For example, the rate at which respondents regularly saved money increased from before the course, and respondents were very likely to follow through on their goals of saving money in a savings account. Respondents were also much more likely to use a budget, and regularly keep to it.
Overall, consumer confidence in financial matters substantially increased after completing the Money Smart course and was sustained over the survey period.
The Neighborhood Reinvestment Corporation, created by Congress in 1978 as a national nonprofit corporation, has a mission of creating “opportunities for people to live in affordable homes, improve their lives and strengthen their communities.” Governed by a board of directors consisting of the leaders of the five financial institution regulatory agencies and the U.S. Department of Housing and Urban Development (HUD), Neighborhood Reinvestment Corporation began operating under the name of NeighborWorks America in 2005.
Throughout the remainder of this report, the term Money Smart course or classes will mean the Money Smart modules required to be taught in order to be considered in this survey. The survey required that at least three modules covering the topics of checking accounts, savings accounts, budgeting, and credit be taught. The modules could be any three of the following five: Check it Out, Money Matters, Pay Yourself First, To Your Credit or Charge it Right.
Training classes ended as early as November 2004 and as late as September 2005. The telephone follow-up survey began in February of 2006. Accordingly, a small number of respondents may have completed the training less than six months or more than twelve months before the telephone follow-up survey.
Because of the volume of the survey data results, data tables are presented in the report in three ways.
Illustration refers to tables presented in the body of the report. Exhibit refers to tables presented in the Exhibits section of this report. Table refers to tables in Appendix J.
Study Goals The impact of a financial education curriculum such as Money Smart is seldom tracked and measured. 5 There is much anecdotal evidence to indicate at least short term changes in financial management behaviors such as budgeting and bill payment after financial education training.
For example, according to one Money Smart instructor, referring to Money Smart class participants: “They can now live on a budget, work, and pay their bills.” 6 However, anecdotal accounts do not provide objective measures of program success. Financial institutions involved in financial education initiatives typically consider the classes useful for ancillary purposes such as employee recruitment, community goodwill, or generating a long-term demand for financial services. Rarely do financial institutions track the number of new accounts opened or other objective outcomes associated with their financial education efforts. 7 Despite these challenges, data on financial education course outcomes are essential in order to ascertain whether time and other resources spent on financial education are worthwhile. 8 The U.S. Department of Treasury’s Office of Financial Education has also identified the use of “performance measures to track progress” as one of the eight elements of a successful financial education program. 9 This measurement can be of short-term outcomes, such as whether the respondent intended to open a new bank account, or longer-term changes in behavior and practices, such as obtaining and using financial skills to independently make sound financial decisions. 10
Prior research assessing the effectiveness of Money Smart has examined only the former:
changes in the intentions of participants as measured by comparing pre-training survey to posttraining data. Lyons and Scherpf concluded that completing Money Smart increased a participant’s probability of planning to open a new account. 11 The researchers recognized they had no way of determining whether these intentions translated into actual conduct because their data were based solely on surveying participants immediately before the first Money Smart class taken and immediately after the last Money Smart class taken.
Lyons, A. C., Palmer, L, Jayarantne, K. S. U., and Scherpf, E. (2003). Are we making the grade: a
national overview of financial education and program evaluation, Journal of Consumer Affairs, 40 (2):
Keenan, C. (2004). Financial Literacy: How It Adds Up to Good Business, Community Banker, 13 (3):
“Without a way to measure progress, it is easy to question whether all the effort at financial education is worthwhile.” Shankar, V. (2004). Finding a Way to Measure Financial Literacy Efforts, American Banker, 169 (209): 3-3.
U.S. Department of the Treasury, Office of Financial Education, Eight Elements of a Successful Financial Education Program (2004). http://www.treas.gov/offices/domestic-finance/financialinstitution/fin-education/support-docs/eight-elements.pdf.
Lyons, A. C., and Scherpf, E. (2004). Moving from unbanked to banked: evidence from the Money
Smart program, Financial Services Review, 13 (3): 228-229. Ibid. at 215-231.
Other studies that have analyzed the longer-term impact of financial education have only done so with respect to limited types of groups. 12 For example, a Freddie Mac study of mortgage borrowers over a five-year period found that between two pools of similarly situated borrowers, the ninety-day delinquency rate for the group that received pre-purchase home ownership counseling was 19 percentage points lower than the group that did not receive counseling. 13 The National Endowment for Financial Education found that three months after completing a financial education course, high school students changed their spending and savings patterns from before the course. 14 None of the available research has considered the longer-term impact of financial education upon a broad audience. Thus, this study was intended to fill a research gap by determining the effect of Money Smart financial education covering basic banking, savings, budgeting, and credit, upon participant behavior six to twelve months after the conclusion of the training.
U.S. Government Financial Literacy and Education Commission, Taking Ownership of the Future: The National Strategy for Financial Education (2006). Pages 98-100.
Hirad, A. and Zorn, P. (2001). A Little Knowledge is a Good Thing: Empirical Evidence of the Effectiveness of Pre-Purchase Homeownership Counseling, Freddie Mac Working Paper.
Danes, S. M. 2003-2004 Evaluation of the NEFE HSFPP, http://www.nefe.org/hsfppportal/includes/main/home.asp?portal=4&page=4000#TheImpactoftheHSFPPo nStudents The Money Smart Financial Education Curriculum The FDIC launched Money Smart as a nationwide initiative in September of 2001. The curriculum was designed to help adults enhance their money management skills, understand basic financial services offered by the financial mainstream, and build financial confidence to use banking services effectively. Money Smart was also designed to provide financial institutions with a tool to assist in community outreach and economic development.
The Money Smart curriculum consists of ten modules: