Evidence, Monitoring and Evaluation
Together with our partners, we are shaping an outcomes-focused service system. This involves building an evidence base of services that are proven to work, as well as monitoring and evaluating solutions to ensure they are improving outcomes.
Shared Outcomes Framework
A comprehensive monitoring and evaluation framework is used to identify the best-practice data collection required to measure benefits and evaluate outcomes consistently across service solutions.
TFM’s Shared Outcomes Framework translates our vision into a quantifiable set of outcomes and indicators drawn from multiple sources. This helps TFM and other agencies to track whether their combined efforts are improving these outcomes for vulnerable children and families in NSW over time.
This framework builds upon the NSW Human Services Outcomes Framework, observing the same outcomes domains of Economic, Education & Skills, Empowerment, Health, Home, Safety, and Social & Community. Using these domains ensures consistency in outcomes measurement across NSW agencies.
This framework enables us to take a client centric view to identify the effectiveness of programs to ensure we make a positive long-term difference to people’s lives.
Monitoring and evaluation
Programs and integrated solutions are evaluated continually, with data collection and performance monitoring systems embedded in regular service delivery. Ongoing monitoring will help us test how well supports are performing, if the outcomes anticipated are being achieved, and whether readjustment is necessary.
Programs we are evaluating
Evaluation has commenced or is planned for 2019, for the following programs implemented as part of Their Futures Matter:
- Functional Family Therapy Child Welfare (FFT-CW)
- Multi-systemic Therapy for Child Abuse and Neglect (MST-CAN)
- Under 12s cohort service solution
- LINKS Trauma Healing Service
- Futures Planning and Support
Their Futures Matter is working collaboratively with partner agencies and non-government organisations to design practical and effective solutions to improve individual outcomes for vulnerable children and families.
Using the best available evidence from government agencies, the service sector and academic literature, we are able to identify:
- what solutions have demonstrated success
- where more evidence is needed
- service solutions that need to be designed to support vulnerable children, young people, and families
For example, our Aboriginal Evidence Building in Partnership Project is building the local evidence base for Aboriginal programs in NSW. Our aim is to work with our community partners to develop a stronger evidence base of programs and services that work best for Aboriginal children, young people, families, and communities.
Independent Evaluation of MST-CAN® and FFT-CW®
The National Drug and Alcohol Research Centre (NDARC) has delivered their final evaluation of Multisystemic Therapy for Child Abuse and Neglect (MST-CAN®) and Functional Family Therapy – Child Welfare (FFT-CW®) programs. This marks the finalisation of the initial evaluation of these programs since their implementation in NSW in 2017.
The evaluation is a significant contribution to the growing NSW and international evidence base for preservation and restoration therapeutic home-based programs.
This report includes a process, outcomes and economic evaluation of the FFT-CW® and MST-CAN® programs. The key findings of the evaluation were:
- Although the programs are in the early stages of their life cycle, the completion rates across both MST-CAN® and FFT-CW® are positive.
- Entries to OOHC are substantially lower than control groups for families who have successfully completed other programs.
- ROSH re-report rates for FFT-CW® and MST-CAN® are lower than control groups for families who successfully completed other programs.
- Referrals of Aboriginal families were lower than expected and the number of Aboriginal families accessing services could be improved.
- Families and District Directors are enthusiastic about the therapeutic and practical value of both programs.
- It is too early to quantify the economic benefits of the programs however the evaluation methodology will be re-run regularly.
- The acceptance rate for families offered a program placement is relatively high (76 per cent of Aboriginal families and 77 per cent of all families).
The Program Highlights infographic provides a summary of service delivery statistics, implementation initiatives, and planned improvements to MST-CAN® and FFT-CW®.
For any enquiries, please contact the program team by emailing MST-CANandFFT-CWSupport@facs.nsw.gov.au.