Project conducted by: Auckland University of Technology, Gambling and Addictions Research Centre
Principal researchers: AUT: Komathi Kolandai-Matchett, Jason Landon, Maria Bellringer, Nick Garrett, Stuart Mundy-McPherson, Max Abbott. KPMG: Brylee Haapu, Souella Cumming
Background
The Ministry of Health’s implementation of an integrated problem gambling public health strategy includes psychosocial intervention services and primary prevention public health services contracted to providers located throughout New Zealand.
Aim
To evaluate the effectiveness of the Brief, Full, Workshop-based, Facilitation and Follow-up intervention services and the Policy Development and Implementation, Safe Gambling Environments, Supportive Communities, Aware Communities, and Effective Screening Environments public health services.
Method
The evaluation employed a mixed-methods approach guided by a logical framework and evaluation criteria agreed with MoH. Content and thematic analysis of over 100 progress reports from 2010 to 2013 for all providers was used to identify outputs, outcomes, best practice and challenges for public health services. An analysis of the Client Information Collection (CLIC) database from 2010 to 2013 determined key trends in intervention services. Views about service effectiveness were obtained through surveys of staff (n=64), clients (n=148) and allied agencies (n=42) of eight providers.
Additional perspectives and clarifications were obtained from three focus group interviews with staff and managers of the eight providers. KPMG was subcontracted to conduct a clinical audit of the intervention services delivered by the eight providers, based on providers’ contracts with MoH, and other guidelines. This included review of documentation, and staff and client interviews. A triangulation process was used to compare and contrast findings from the various evaluation data sources and clinical audit observations. Several factors limited the generalisability of findings.
Key findings
Intervention services
- Providers effectively ensured clients’ access to information, met clients’ expectations in terms of service quality and cultural appropriateness, reached out to targeted at-risk populations, and facilitated clients’ access to other support services.
- Some providers did not meet the minimum number of client sessions agreed with MoH and a greater level of clinician involvement in delivering Follow-up services appeared to be required. New Brief Intervention client numbers increased in 2010-2013 particularly for significant other clients. This service was delivered primarily in public settings.
- Full and Workshop-based Intervention services were delivered primarily through face-to-face sessions. Limited CLIC screen records, and staff and client views suggested positive client outcomes.
- Facilitation services were delivered mainly for gambler clients. Enabling a seamless referral process was easier for organisations with multiple support services. Perceived positive client outcomes included improved gambling behaviour, fiscal management abilities and financial situations. Allied services developed awareness and ability to identify problem gambling symptoms. Areas for improvement included provider-allied service relationships and joint client management protocols.
- Follow-up services were delivered either by clinicians or by support staff. While a clinician’s role was regarded as vital for relationship maintenance, time constraints were an issue. Follow-up at one, three, six and 12 months following treatment was regarded as effective. Client reported outcomes included reassurance and encouragement gained from a sense of ongoing support and care.
- Although providers are not mandated to record scores for all recommended screens in the CLIC database, changes to provider practice in reporting pre- and post- screen scores could enable reliable measurement of client outcomes.
Public health services
- Providers reported policy outcomes, impacts on host responsibility practices, and enhanced public awareness.
- Providers were effective in ensuring appropriate public health resources for community members and in delivering public health activities using culturally appropriate approaches.
- All providers successfully collaborated with a broad range of stakeholder groups. Community engagement led to community partnerships in public health programmes as well as community ownership over initiatives.
- Most Māori and Pacific providers reported explicit examples of cultural approaches incorporated in public health services that they believed were effective for at-risk groups and for encouraging involvement in various activities. More detailed reporting is required to understand public health approaches that meet unique needs of at-risk Asian communities.
- The Policy Development and Implementation service resulted in some organisational gambling policies, public policy support, and influence on Class 4 venue policies.
- The Safe Gambling Environments service resulted in improvements to multi-venue exclusion processes, and development of some gambling venues’ host responsibility measures.
- In delivering Supportive Communities, providers enabled resiliency building through community partnerships, and delivered activities enabling community connectedness, healthier alternatives, and public debate on gambling harms, resulting in community involvement and knowledge increase.
- For the Aware Communities service, providers delivered awareness-raising presentations, carried out training on brief screening, supported community and youth-led awareness programmes, and engaged the media, resulting in increased public understanding.
- The Effective Screening Environments service resulted in screening and referral practices among some stakeholder groups, and increased awareness of the availability of intervention services.
- Areas for improvement included staff knowledge development and clarity in public health work plans, progress reports and description of aspects regarded to be innovative.
Other findings and recommendations
- Programme sustainability was noted (eg, some communities continued to carry on provider-initiated projects). Programme sustainability could be included in service specifications to enhance long-term effectiveness of public health services.
- Effective planning to maximise efficiency is required where different services overlap (ie, where there are connections between services and/or similarities in services’ objectives).
- Staff appeared to focus on single service success indicators (eg, activity, output or outcome). Use of multiple success indicators within a logical framework of service delivery could improve service planning and guide self-monitoring processes.