Evaluation Framework for New Income Management (NIM)

Attachments

• Evaluation Framework for New Income Management [946.2 kB]
  • Listen to PDF

HTML Version

Date:  1 December 2010

December 2010 

This outlines a framework for the evaluation of the NIM model in the NT. The framework is intended to provide a broad structure for the evaluation. It addresses the scope of assessment, high-level research questions, study design, methodologies and proposed data and sources.

Executive summary 

In May 2010, a consortium of experts from the Social Policy Research Centre (SPRC), the Australian Institute of Family Studies (AIFS) and the Australian National University (ANU) was engaged by the Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) to develop an overarching framework for the evaluation of new income management (NIM) to guide evaluation activities over the period 2010-14.

The terms of reference for developing the evaluation framework are that the evaluation activities:

• be completed and reported by December 2014
• provide information on the implementation of the NIM in the Northern Territory by the end of 2011 in order to inform decisions about an expansion of the model beyond the Northern Territory
• result in data being collected that can be used to evaluate short, medium and, where possible, longer-term impacts/outcomes of new income management
• include a set of ethics guidelines and an ethical clearance strategy relevant to this evaluation project.

This document outlines a framework for the evaluation of the NIM model in the NT. The framework is intended to provide a broad structure for the evaluation. It addresses the scope of assessment, high-level research questions, study design, methodologies and proposed data and sources. (See Section 4 for more detail.)

Undertaking the evaluation will require an iterative approach and, as such, it is expected that some elements of this framework (including the program logic, the design of research instruments and detailed questions) will need further development as the project progresses.

The development of the evaluation framework involved extensive consultations with both government and non government sectors. The evaluation framework takes account of:

• the way in which NIM is being implemented, especially that it has been rolled out to the whole of the Northern Territory in a short period of time
• the previous income management policy which has been operating in the Northern Territory since 2007
• the fact that other policies may change at the same time as NIM is being implemented
• practical challenges involved in collecting data in remote areas of the Northern Territory
• particular ethical issues involved in collecting data from vulnerable groups including: children, women, the elderly, Indigenous Australians, and people from culturally and linguistically diverse backgrounds.

It takes a multi-method approach (also called triangulation) as it is not appropriate to use an experimental design on a complex social policy such as this. This means collecting different types of primary data (qualitative and quantitative), using secondary administrative and survey data, and collecting information on the same questions from multiple informants (e.g. those being income managed, Centrelink staff, financial counsellors, child protection workers).

As a first step, the framework involves collecting data for an early implementation snapshot, which will also provide benchmarking data for the evaluation. The consortium of researchers who developed this evaluation framework has been commissioned to collect the majority of the early implementation data.

The development of the evaluation framework and collection of the early implementation snapshot data constitutes Phase 1 of the evaluation. The conduct of a comprehensive and independent evaluation of the NIM constitutes Phase 2.

Phase 2 involves evaluating:

• the effectiveness of the program's implementation
• whether the program was delivered as intended to the target population in a fair and equitable manner (including access to necessary services)
• an assessment of the impacts of NIM on individuals, families and communities in the Northern Territory and
• an analysis of value for money (to the extent that this is achievable within the timeframe of this evaluation).

The evaluation is designed to produce the data necessary to evaluate short, medium and, where possible, the longer-term outcomes of NIM. The short term outcomes are foundations and enablers for the measure, and should be evident within the early years of the evaluation project. Behavioural changes/outcomes could occur in the medium term and be evident within the life of this evaluation project.  However, some of the longer-term and sustained changes in behaviour are likely to take a number of years to become evident.

All evaluation activities will need to continue to be undertaken in close consultation with both government and non-government stakeholders.

A requirement of the evaluation framework was that it allow the data to be produced to enable reports to be delivered to FaHCSIA by end of each calendar year from 2011-2014, including:

• a substantial progress report addressing implementation issues and early progress in achieving short term outcomes, by the end of 2011
• annual intermediate evaluation reports that synthesise results to date and inform future analysis, by the end of 2012 and 2013 and
• a final outcome evaluation report  by the end of 2014.

1. Introduction 

Legislation for the model of new income management was passed on 1 July 2010 and was introduced by the Australian Government from 9 August 2010. The model first commenced in the Northern Territory (NT) in urban, regional and remote areas. Over time, and drawing on evidence from implementation experience in the NT, it may progressively be rolled out more broadly across Australia.

The Australian Government Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) has commissioned a consortium of researchers from the Social Policy Research Centre, the Australian Institute of Family Studies and the Australian National University (Centre for Aboriginal Economic Policy Research and the Research School of Economics) to assist in the development of a framework for the evaluation of the new model of income management.

A transparent and objective evaluation process increases the credibility of the assessment of the new measure of income management. The involvement of an external and independent evaluator will provide legitimacy to both the evaluation process and the assessment of the measure.

Ideally, in evaluating social policies data would be collected prior to the implementation of the policy (baseline data). Data collected after the implementation of the policy would then be compared to the baseline data to track changes over time.

However, in the case of NIM, income management has been implemented in 73 discrete communities since 2008 as part of the Northern Territory Emergency Response, which makes collecting ‘pure' baseline data (ie to identify the circumstances and beliefs of people before the implementation of policy) impossible. Even if it were possible to have ‘pure' baseline data, other policies that may also affect people in the Northern Territory who are subject to income management have also been subject to change.  Thus, it would not be possible to attribute, confidently, pre- and post-implementation differences in data collected to the NIM measure. 

Given that it is not possible to collect ‘pure' baseline data in this context, the evaluation framework includes an early implementation snapshot study. This should provide information to complement administrative baseline data generated prior to implementation of income management policy. Together, these will provide benchmark data for the assessment of NIM. 

The implementation snapshot should include collection and analysis of primary data from a wide range of people affected by NIM or involved in administering or implementing the model. This would include people who are income managed, community leaders, Centrelink staff, merchants, money management and financial counselling service providers, and child protection workers.

The evaluation framework is designed to ensure the key evaluation questions will be able to be answered using data collected from multiple sources, using both quantitative and qualitative research methods. This will provide "triangulation" for the key findings of the evaluation.

The terms of reference for developing the evaluation framework are that the evaluation:

• be completed by December 2014
• provide information on the implementation of the NIM in the Northern Territory by the end of 2011 in order to inform decisions about an expansion of the model beyond the Northern Territory
• result in data being collected that can be used to evaluate short, medium and, where possible, longer-term impacts/outcomes of new income management, and
• include a set of ethics guidelines and an ethical clearance strategy relevant to this evaluation project.

This document outlines a framework for the evaluation of the NIM model in the NT and should not be read as a detailed evaluation work plan. The evaluation itself will have to establish the practicality of the approach proposed here, in particular the availability and quality of the various secondary datasets, and the specific issues involved in engaging with individuals, families and communities affected by NIM. The conceptual basis for the program, including the description of the new income management policy and the program logic, is described in detail in Sections 2 and 3. Sections 4-6 outline the consortium's proposed evaluation framework including challenges in evaluating this measure, the study design and methodology and proposed reporting timelines.

Background information to the development of NIM can be found at Appendix A. Appendix B discusses current income management evaluation activities undertaken by FaHCSIA. Appendix C provides FaHCSIA's program logic model for the NIM model. Appendix D shows the hierarchy of outcomes and associated data.

A comprehensive English language literature reviewon evaluation methodologies used to evaluate relevant international programs such as conditional cash transfers and financial counselling programs is at Appendix E. A list of data sources needed to support the evaluation is at Appendix F, while Appendix G describes other funded initiatives in the NT that support vulnerable children and families.

In developing the framework, extensive consultations have been undertaken with staff from the Northern Territory Department of Health and Families, the Western Australian Department of Child Protection, Centrelink, the Department of Finance and Deregulation, Government and Non-Government Think Tank Reference Groups, and the Australian Government Department of Families, Housing, Community Services and Indigenous Affairs.

2. The new model of income management 

Income management was first introduced in 2007 as part of the Northern Territory Emergency Response (NTER). The initial rollout of income management only affected people who received income support or family assistance payments and who lived in the 73 prescribed communities, their associated outstations and the 10 town camp regions of the Northern Territory.

The new model of income management was introduced on 1 July 2010. The new program differs from the previous model, in particular with regard to the targeting of particular groups of income support recipients. Importantly NIM applies to people who meet criteria independent of their race or ethnicity—this is consistent with the Racial Discrimination Act of 1975 (RDA). Information on earlier initiatives and background to the development of new income management is at Appendix A.

Implementation of new income management commenced on 9 August 2010 in the Barkly region (Zone 1).  Implementation in Zone 2 – Alice Springs, Katherine, East Arnhem Land and other outback areas – commenced on 30 August. Implementation in the remaining zones commenced on 20 September (outback areas) and 4 October (Darwin and Palmerston). It is expected that most people will have been transitioned from the old scheme to new income management by 31 December 2010.

2.1 What is income management?

Under income management, a percentage of a person’s welfare payments is set aside for their priority needs and those of their children and families. This helps to ensure that:

• money paid by the government for the benefit of children is directed to the priority needs of children
• women, the elderly and other vulnerable community members are provided with better financial security, and
• the amount of cash in communities is reduced to help counter substance abuse, gambling and other anti-social behaviours that can lead to child abuse and community dysfunction.

Income managed funds cannot be used to purchase excluded goods, including alcohol, home brew kits, home brew concentrates, tobacco products, pornographic material and gambling goods and activities.

Income managed funds must be directed towards agreed priority needs and services such as food, rent and utilities. This process assists families to meet essential household needs and expenses.

2.2 Rates of income managed funds

An individual's income support and family assistance payments are income managed at 50 per cent for the participation/parenting (mainstream), vulnerable and voluntary streams and 70 per cent for the child protection stream. Any lump sums (e.g. Baby Bonus) and advance payments are income managed at 100 per cent. The portion of an individual’s regular fortnightly payments that is not subject to income management (i.e. the discretionary funds) is paid in the usual way.

Income management does not reduce the total amount of payment an individual receives from Centrelink. It only changes the way in which they receive their payments. Individuals can spend their income managed money by using the BasicsCard at approved stores, or by arranging direct payments to organisations such as community stores, landlords, or utility providers.

2.3 Categories or streams of new model of income management

The NIM model is more targeted in its approach than the previous income management measure under the NTER.  For the purposes of this framework, NIM has been described using four broad streams or categories:

2.3.1 Participation/parenting (mainstream) stream

• For disengaged youth—people aged 15 to 24 years who have been in receipt of specified welfare payments (Youth Allowance, Newstart Allowance, Special Benefit or Parenting Payment) for more than three of the last six months.
• For long-term welfare recipients—people aged 25 years and above who are on specified welfare payments such as Newstart Allowance and Parenting Payment for more than one year in the last two years.

2.3.2 Child protection stream

• For parents and/or carers referred for income management by a child protection worker. Child protection authorities will refer people for compulsory income management if the child protection worker deems that income management might contribute to improved outcomes for children at risk. This measure will apply at the discretion of a State or Territory child protection worker.

2.3.3. Vulnerable stream

• For vulnerable welfare payments recipients who would benefit from income management in order to meet their social and parental responsibilities, to manage their money responsibly, and to build and maintain reasonable self-care. This stream provides Centrelink Social Workers with an additional tool to help individuals who are vulnerable and/or at risk (e.g. Individuals on Age pension or Disability Support Pension and those subject to financial harassment). It can only be applied following an assessment by a Centrelink Social Worker.

2.3.4 Voluntary stream

• For people on income support who wish to volunteer for income management to assist them to meet their priority needs and to learn how to manage their finances for themselves and/or their family in the long term. 

The pathways into the new income management measure are shown in Figure 1, below:

Figure 1. From old to new: an illustration of major pathways for the new income management measure1

Figure 1 represents the pathways for customers to experience the new model of income management.

Previous income management customers under the Northern Territory Emergency response and new customers to income management can transition on to new income management through three compulsory measures or through voluntary income management. Each compulsory measure has specific eligibility factors.

Individuals moving off income management can be out of scope, exempt from income management or otherwise not be subject to income management due to moving to a non-income management area.

2.4 Additional features including incentives for people on income management

The new model of income management has a number of additional features.

1. The Matched Savings Payment is an incentive payment to encourage people on income management to develop a savings pattern and increase their capacity to manage their money. If eligible, a person can receive $1 for every $1 they save, up to a maximum of $500. A person can only receive a Matched Savings Payment once. The Matched Savings Payment is paid directly into the person’s income management account.
   
   To receive a Matched Savings payment an individual must:
   • be income managed (excluding VIM and Cape York income management)
   • complete an approved money management course
   • maintain a pattern of savings from their discretionary funds for 13 weeks or longer after the commencement of the approved money management course  and
   • not have previously received a Matched Savings Payment.
2. The Voluntary Income Management Incentive Payment is a payment to encourage people who are not income managed but who might benefit from it to volunteer for income management and to continue to participate in it long enough to recognise its benefits. Individuals who voluntarily participate in income management are eligible for an incentive of $250 for every six months they remain on VIM.
3. Income management is supported by financial counselling and money management services, totalling $53 million over four years. Income management arrangements introduced under the NTER will operate until 30 June 2011, while people are transitioned to the new model.

2.5 Exemptions

The new model of income management provides pathways to evidence-based exemptions for people under the participation/parenting (mainstream) stream of income management. For people with children, exemptions will be based on a financial vulnerability assessment and demonstrated evidence of responsible parenting activities for their children. These include regular child health checks and immunisations, and participation by the child in age appropriate, social, learning or physical activities.

Individuals referred to the child protection or the vulnerable streams are not eligible for exemption pathways. However, appeal processes are available through Centrelink, the Social Security Appeals Tribunal and the Northern Territory Government. See Section 2.6 below for more detail.

For people without children, exemptions will be based on a demonstrated record of or participation in employment and study.

2.6 Appeal rights

Under the new model of income management people will have access to the full range of review and appeal rights through Centrelink’s Authorised Review Officers and the Social Security Appeals Tribunal. Additionally reviews and appeals processes will be available through the Northern Territory Government specifically for the Child Protection Measure

1. As this figure only represents the major pathways for the new income management measure, it may not capture the nuances of individual circumstances.

3. Program logic 

Program logic (also referred to as theory of change), has been used to develop and evaluate programs and initiatives since the early 1970s. It improves the quality and focus of evaluation advice to government.

This process is used to ‘surface the implicit theory of action inherent in the proposed intervention in order to delineate what should happen if the theory is correct and to identify short medium and long term indicators of changes which can provide evidence on which to base evaluations' (London et al, 1996).

In order to evaluate NIM it is important to clearly articulate the objectives of the policy and the criteria by which the success or failure of the policy in meeting the objectives set for it are to be evaluated. It is also important for the evaluation methodology to be able to identify any unintended impacts, both positive and negative.

This section outlines the concept map for the NIM model (see Figure 2). A high-level concept map sets out the most important components of the program logic:

• the knowledge base underlying the program
• the strategies to be adopted as part of the program
• the problems to be addressed
• the needs and assets of the communities and
• the influential factors that have an impact on these problems and also on needs and assets, with the desired results to be achieved by the program.2

FaHCSIA has developed its own program logic, and that can be found at Appendix C.

Figure 2. High-level concept map

Description of High-level concept map.

Influential Factors

• Extensive Review of the Northern Territory Emergency Response.
• Experience and feedback from people on other income management schemes (e.g. VIM, CPSIM, CY).
• Access to services.
• Knowledge of the range of policy levers that influence behaviour and outcomes for welfare recipients.

Knowledge

• Income management is one tool that may assist people to allocate their income more responsibly, and reduce their vulnerability and that of their dependents.
• Passive welfare creates a cycle of dependency.
• Income management can provide a gateway into other initiatives including building money management skills, and leverage for responsible parenting and participation.
• Focusing on priority needs allows issues such as food security, housing and children’s needs to be addressed.
• Previous income management arrangements were discriminatory and required the suspension of the Racial Discrimination Act.

Strategies

Implement income management to:

• ensure appropriate expenditure of income support payments on priority goods
• reduce the amount of income support that is spent on excluded goods
• reduce the amount of harassment, financial abuse and related pressures in relation to welfare payments and
• provide incentives and support for improved financial management skills and practices, workforce and educational participation, and responsible parenting.

Problems/issues for consideration

• Variation in opportunities for economic participation within communities.
• Entrenched welfare dependency and associated issues.
• Low levels of positive social participation and high degree of vulnerability.
• Inadequate spending on priority needs and other poor financial management outcomes including debt and arrears.
• Breakdown in community values and solidarity.
• Need to provide support for the most vulnerable in communities.
• Acknowledging the potential strength of some particular communities, including remote Indigenous Communities.

Desired results

• More spending on priority goods.
• Less spending on excluded goods.
• Reduction in immediate hardship of individuals and families.
• More responsible parenting and better outcomes for children, including nutrition and education.
• Increased financial management skills and capacity to save.
• Greater levels of positive economic and social participation and responsibility.

[ top ]

3.1 Objectives/rationale of NIM

Objectives for NIM have broadly been taken from Minister Macklin's second reading speech, and the Government's Policy Statement "Landmark Reform to the Welfare System, Reinstatement of the Racial Discrimination Act and Strengthening of the Northern Territory Emergency Response".

The Australian Government's Policy Statement (2009, p. 1) identifies the aims of new income management as to:

The policy statement also identifies income management as a key tool in the Government's broader welfare reforms to promote responsibility and strengthen families by ensuring that income support payments are spent where they are intended.

Income management limits expenditure of income support payments on excluded items, including alcohol, tobacco, pornography, gambling goods and activities. It ensures that money is available for life essentials, and provides a tool to stabilise people's circumstances, easing immediate financial stress.

According to the program logic, this change in expenditure patterns is expected to result in a number of other benefits for children, parents and the broader community. A reduction in negative expenditures may result in reductions in alcohol fuelled violence, substance abuse and risky behaviour. The promotion of positive expenditure patterns may result in more effective meeting of children's needs including improved nutrition and increased spending on children's clothing and school-related expenses.

As a consequence of better spending on children, there may be improvements in positive health behaviours and improved educational attendance, which in turn could lead to improved educational outcomes.

In addition, the exemption criteria are intended to reinforce some of these positive outcomes. For example, families may be able to secure exemptions if their children are immunised and if they are attending school. Additionally, exemptions are available on demonstrating that children are engaged in activities such as structured, age appropriate social, learning or physical activities.

Overall, the program is intended to reinforce responsible parenting and more generally promote principles of engagement, participation and personal responsibility.

The Australian Government funds financial counselling services through the Financial Management Program (FMP), which was established to build financial resilience and wellbeing among those most at risk of financial and social exclusion and disadvantage. The Program helps vulnerable people across a range of income and financial literacy levels to manage their money, overcome financial adversity, participate in their communities and plan for the medium to long term.

This Program contributes to improved outcomes for vulnerable people, families, and communities by:

• fostering the improved use and management of money
• helping people address immediate needs in times of financial crisis and
• undertaking research to inform policies to reduce the impact of problem gambling.

The provision of financial counselling through FMP and the new incentives for saving could lead to improved savings and household budgeting, which in turn could result, for example, in the ability to purchase needed consumer durables. Better financial management is intended to assist families in meeting important bills such as rent and utilities, which could in turn stabilise housing and reduce risks of eviction and homelessness or sleeping rough.

It is also expected that certain sorts of child neglect could well be exacerbated by poor household financial management, for example, poor management may mean that children are hungry. It is also anticipated that income management will assist some individuals in resisting undesirable behaviour by their relatives and kin (i.e. harassment for money).

In broad terms, new income management is intended to set in motion a series of positive behaviours that will be mutually reinforcing. Outcomes are therefore expected to be:

• short-term (e.g. changed expenditure patterns—less expenditure on excluded goods, more expenditure on priority items)
• medium-term (e.g. take-up of referrals to money management and financial counselling service providers, improved educational attendance), and
• long-term (e.g. acquisition of money management skills, improved employment opportunities and improved educational attainment).

The potential impacts of the model are not only expected to be felt by the individuals directly affected, but the communities in which these individuals live are also expected to be affected.

As there will be movement onto and off the program, it will be necessary to consider outcomes not only for those who are currently participating in the model, but also for those who have left the model and are no longer having their benefits managed.

2. This approach to the logic model reflects the Logic Model approach developed by the WK Kellogg foundation. This structural approach is further developed in the next section with regard to the identification of the key evaluation questions.

4. New income management evaluation framework 

This section outlines the consortium's proposed evaluation framework. It is expected that some elements of this framework will be further developed by the independent evaluator as the project progresses.

This framework addresses scope of assessment, high-level research questions, study design, methodologies, proposed data (and sources) and an indicative timetable. Research Instruments and more detailed evaluation questions will be developed in Phase 2 of the evaluation project.

The main purposes of the evaluation activities to be conducted over the 2010-2014 financial years are to:

• provide evidence on the impact on NIM on those who are affected
• assess whether the reforms were implemented effectively, and
• understand whether NIM is a cost-effective model so as to inform future government decision making and social policy formulation for both the wider and the Indigenous communities.

4.1 Scope of assessment

The evaluation should contain a process evaluation, an outcome evaluation and a value for money analysis. This evaluation strategy has been informed by a comprehensive literature review of evaluations of similar conditional welfare programs internationally (see Appendix E).

4.2 Process evaluation

The process evaluation should examine how NIM was implemented and should report on the barriers and enablers which affected its implementation. (See Section 5 for more detail).

4.3 Outcome evaluation

The outcome evaluation should assess the impact of the model at the individual, family/household and community level in the Northern Territory. The evaluation should draw on administrative data, survey data, longitudinal interviews and case studies to determine the extent of individual, family/household and community level changes over 2010-14 financial years.

Regular reports should be delivered to FaHCSIA by end of each calendar year from 2011-2014, including:

• a substantial progress report addressing implementation issues and early progress in achieving short term outcomes, by the end of 2011
• annual intermediate evaluation reports that synthesise results to date and inform future analysis, by the end of 2012 and 2013, and
• a final outcome evaluation report  by the end of 2014 (See Section 5).

Interim evaluation reports (short-medium term outcome evaluations) should be conducted on an annual basis and be delivered by end December 2011-2013 – see Sections 5 and 6 for more detail. The final evaluation report will be delivered by the end of December 2014 and contain information on the short-medium, and wherever possible longer-term, outcomes along with value for money analysis3. See Section 4.4.2 for more detail.

With regard to the evaluation questions and indicators, the evaluation will seek to provide answers to the following high level questions. At this stage, the questions are broad and there is a need to further develop specific research questions in Phase 2 of this project.

[ top ]

4.4 Broad overarching questions across all four streams of new income management (ie participation/parenting, child protection, voluntary and vulnerable streams)

4.4.1 Process evaluation

1. How effectively has NIM been administered and implemented?

• What have been the resource implications of implementing the program?
• Have suitable individuals and groups been targeted by NIM?
• Have people been able to transfer into and out of NIM appropriately (e.g. choosing to transfer from income management under NTER to VIM)?
• What has been the effect of the introduction of NIM on service providers?

2. What is the profile of people on the different income management streams?
3. Have there been any initial process 'teething issues' that need to be addressed?
4. What are the views of participants in the NIM model and their families on the implementation of the program? 
5. Has the measure been implemented in a non-discriminatory manner?

4.4.2 Outcome evaluation

1. What are the short, medium and longer-term impacts of income management on individuals, their families and communities?

• How do these effects differ for the various streams of the program (mainstream, voluntary, child protection, vulnerable)?
• Have there been changes in spending patterns, food and alcohol consumption, school attendance and harassment?
• What impact does NIM have on movement in and out of NT among people on the measure?
• Has NIM contributed to changes in financial management, child health, alcohol abuse, violence and parenting (ie reduced neglect)?
• Do the four streams achieve appropriate outcomes for their participants?
• Has NIM had any unintended consequences (positive or negative)?
• Are there differential effects for different groups? (including—if sufficient data is available—by Indigeneity, gender, location, age, educational status, work status, income, length of time on income support, marital status, family composition and diverse cultural and linguistic background)
• Does IM provide value for money by comparison with other interventions?
• Does NIM provide any benefits over and above targeted service provision? 

[ top ]

4.5 Research questions for specific streams of the NIM model

4.5.1 Questions specific to the participation/parenting stream

1. Has NIM helped to facilitate better management of finances in the short, medium and long term for people on income management and their families?
2. Has access to services or interventions improved for those families?
3. Have other changes in the wellbeing and capabilities of the individuals and families occurred?

4.5.2 Questions specific to the child protection stream

1. What has been the impact of income management on child neglect?
2. What has been the impact on child wellbeing in those families referred to the child protection measure (CPIM)?
3. What are the barriers and facilitating factors for child protection workers to use income management as a casework tool?
4. What (if any) service delivery gaps have impacted on the usefulness of the CPIM?

4.5.3 Questions specific to the vulnerable stream

1. Are vulnerable people appropriately targeted by this measure?
2. How does income management impact on the vulnerability of individuals?
3. Have people on this stream experienced changes in the level of harassment (e.g. humbugging)?

4.5.4 Questions specific to the voluntary stream

1. Have people who volunteered for income management been able to make an informed choice?
2. How long do voluntary income management recipients stay on the measure?
3. What are the key motivations for people who voluntarily access income management, and why do they stop?

3. An analysis of value for money should assess the cost effectiveness of new income management. The cost effectiveness analysis will use the quantitative data from the outcome evaluation and financial data from programs to assess the extent to which the costs produced tangible benefits.

5. Study design and methodology 

5.1 Challenges in evaluating new income management

There are a number of conceptual and practical challenges to evaluating this measure.

5.1.1 Conceptual challenges

Attribution

Separating the impacts of NIM from those of other policies and programs is challenging. NIM is being implemented as part of a range of intersecting Commonwealth and Territory social policy initiatives which will have an impact on individuals and communities. Examples include policies related to alcohol restrictions, school nutrition and attendance programs and a range of other health initiatives. There may also be changes to other policies during the evaluation period.

From an evaluation perspective a central challenge will be to differentiate the impact of income management from these other programs and interventions, as well as identifying both positive and negative interdependencies between these.

Nature of expected changes

NIM has a number of short, medium and long-term objectives at the individual and population levels and it will be very difficult to disaggregate these different outcomes at different levels.

Service availability and quality

NIM is predicated on the assumption that participants will use their income to benefit their children and improve their lives.  In order to do so they must avail themselves of a range of services and opportunities including purchasing healthy food, sending their children to school and having their health checked, attending financial counselling, attending TAFE and seeking work.  However if there is limited or no availability of some of these services or opportunities then this will undermine the effectiveness of the model, and will greatly reduce the likelihood of positive outcomes. The quality of services, including the skills and qualifications of workers and level of adherence to policy guidelines, may also impact on service outcomes. The identification of service delivery gaps, however, may prove to be a useful finding within itself, in terms of informing future policy development. The evaluation will therefore need to separate the effects of income management itself from the effects of the services associated with the model, in particular financial counselling.

5.1.2 Practical challenges

A significant number of individuals being income managed will be vulnerable and it may be challenging to engage them in the evaluation.

Data will need to be collected from income management participants from diverse backgrounds and living in very different areas (e.g. cities, town and remote communities) and the data collection instruments will need to be able to cope with the diversity of those being income managed.

• Data will need to be collected from those living in remote communities and those delivering services in remote communities. This creates logistical challenges.
• A substantial proportion of those being income managed (particularly in the Northern Territory) will be Indigenous. There are particular issues and challenges in collecting information from Indigenous Australians. The research ethical issues involved are discussed in Section 7.

A number of further complexities should be noted. As described in Section 2, there are four major groups directly affected by the new income management:

• those who are compulsorily managed because of the type of benefit they are receiving and their duration of benefit receipt (participation/parenting stream)
• those who are managed because a Centrelink social worker believes them to be particularly vulnerable (vulnerable stream)
• those who are referred by child protection authorities (child protection stream), and
• those who volunteer for income management (voluntary stream).

There are a number of different pathways people can take between the different types of income management. For example, people could move from Child Protection of Income Management (CPIM) to Voluntary Income Management (VIM) and VIM to CPIM. See Figure 1 on page 7 for more detail.

In addition, those affected can be categorised in relation to their previous exposure to income management. There will be people who were income managed under the NTER, because of the location in which they live, and those who are being income managed for the first time. Additionally, in the non-income managed population there will be people who have never had their income managed, as well as those who were previously managed but are no longer subject to the measure, such as Age Pension and Disability Support Pension recipients living in locations where income management was previously applied.

It is also important to note that the schematic program logic shown above could be expected to differ between the three main groups identified earlier. That is, the program logic for people who are income managed because of concerns about child neglect will differ from the logic for people who are income managed because of the type of benefit that they receive and their duration on that benefit. Both of these program logics will differ from that for people who volunteer for the program. It seems plausible that people who volunteer for the program will be more motivated to engage with income management and, therefore, are likely to have better outcomes than people whose incomes are compulsorily managed. Similarly, it might be anticipated that people who are income-managed due to referrals from child protection authorities may well have less favourable outcomes.

In considering the program logic outlined above it is also essential to bear in mind that outcomes for those who are income managed will reflect much more than the effects of the program. The overall context in which income management occurs is of crucial importance. General economic circumstance such as changes in unemployment can have major effects on outcomes for income-managed individuals. Similarly, if fresh fruit and vegetables are simply too expensive to be met out of current benefit levels for NT residents living in remote areas, or if housing costs are too high in some locations to be adequately met with existing rent assistance, then outcomes could well appear negative even if the program itself actually had a positive impact. In addition, there are likely to be other changes in policy from the Commonwealth or the State or Territory government that could impact on outcomes for people who are income managed and the communities in which they live, for example, unrelated changes in benefit policy, health policy, education policy, housing policy or child protection policy.

A further complicating factor is that individuals who are income managed under the new model will have a range of differing family circumstances that will affect the outcomes of the model. In some cases families or wider kinship networks may pool resources to assist those who are income-managed, which could either reinforce or undercut the objectives of the model, while other individuals may not have this form of family support.

These considerations suggest that it will be necessary to take a broad approach to defining outcome variables – that is, the preferred evaluation approach should identify both the key outcomes that are intended by the new income management but place these in a broader context that can capture the impact of changes in the general environment and other policy changes.

The design will use both qualitative and quantitative methods to answer the research questions. It is not possible to use an experimental design, and therefore the outcomes will have to be determined by triangulating data from different sources.

5.2 Data sources

An important and necessary component will be the collection of data needed to support the evaluation. A thorough data audit, conducted by the consortium, indicates that significant data gaps exist.  Many of those datasets that do exist are either not very reliable, not easily available for small geographic areas or are difficult to access for various reasons. 

The evaluation should utilise a variety of data sources. This reflects the particular characteristics of the program and the outcomes that are being measured. The use of a diverse set of data sources also allows the evaluation to be conducted in a multi-layered way, taking account of the reported experience of individuals, administrative information on this, and the perspective of those involved in the implementation of the program. It also permits the ‘triangulation' of particular outcomes which may be difficult to measure. A mixed-methods evaluation is proposed that draws together information from multiple sources.

The evaluation of NIM will need to draw on data from a number of sources. This section provides an overview of these sources. The data that are needed fall into four types:

• administrative by-product data (also termed system data) from governments
• system data from private enterprises (eg. retail sales)
• purpose-designed data collected from people on NIM, those involved in implementing the model and the broader community,  comprising both:
  • individual surveys utilising a range of approaches appropriate to the circumstances of different groups, and
  • qualitative data collection through interviews, focus groups and other mechanisms.
• existing survey data.

5.3 Timing of different components of the evaluation

The evaluation should be undertaken in two stages. Stage one is the development of the evaluation framework including the scope and methodology of the evaluation, and establishment of the methods and data collection. Stage one includes an early implementation snapshot study of service providers in the NT to establish their readiness to implement NIM, as well as surveys of income managed clients to capture benchmark data that reflects circumstances of individuals, families and communities soon after the implementation of NIM.

Stage two will include two sub stages; the first stage will involve providing a process evaluation report to FaHCSIA by December 2011.  This report will focus on the implementation of NIM and the barriers and facilitating factors to implementation.  It will also include the views of a range of stakeholders and indications of short term outcomes including transitions to IM, exemptions, service availability and store data. 

The second sub phase will focus on the short, medium and, where possible, longer  term impacts of NIM on people, their families and communities. Intermediate evaluation reports that synthesise results to date and inform future analysis should be delivered to FaHCSIA by the end of 2012 and 2013. The final outcome evaluation report should be provided by end of 2014.

This graphic shows the deliverables due throughout the project, along with the activities that will occur at each stage.  The deliverables are:

• December 2011 = Intermediate Evaluation Report (process)
• December 2012 = Intermediate Evaluation Report (Process & short term outcomes)
• December 2013 = Intermediate Evaluation Report (Short & medium term outcomes)
• December 2014 = Intermediate Evaluation Report (Final Outcome Evaluation Report

Ongoing data collection/information gathering will occur throughou the project.

Implementation Snapshop study will occur between August 2010 and December 2011.

Other inputs as required will occur from December 2011 until Decenber 2014.

5.4 Geographic analysis

We recommend that a key component of the impact evaluation be an ecological analysis describing the association between the prevalence of income management and key outcome variables aggregated across small to medium geographic areas.

Whether this analysis will be suitable for the examination of long-term outcomes will depend upon the extent to which people move between different locations. An examination of Centrelink administrative (and possibly Census) data on mobility patterns will thus need to be undertaken as a complementary component to this analysis.

The main motivation for this is that information on many of the key outcome variables such as expenditure patterns are difficult to collect at the individual level for those participating in the program and even more difficult to collect for comparable people not participating (or for participants prior to their participation). Moreover, there is intrinsic interest in community level outcomes.

The methodology proposes that aggregate information be collected for regions across the NT (and possibly other States) around the implementation period, including on:

• the proportion of the population (or some relevant sub-population) participating in NIM (or some aspect of NIM)
• outcome variables (e.g. expenditures, crime rates, child wellbeing outcomes) and
• confounding variables (such as the operation of other programs).

The correlation between the NIM participation rate and the outcome variables is examined while controlling for confounding variables. If data over time are available, fixed-effect models can be used which examine the changes in NIM and outcome measures in each region.

The main threat to the validity for this analysis, as with all non-experimental analyses, is that there may be unobserved differences across regions which are correlated with both the outcome variables and NIM participation. For example, areas which have a high proportion of the population moving onto NIM might also experience a large increase in other interventions at the same time. If this is not also measured and controlled for the observed association will be a biased estimate of the impact of NIM on outcomes.

Similarly, the cessation of the old model of income management will need to be controlled for (or might form an intervention variable to be analysed in its own right). The associated threat to the reliability (or precision) of such an analysis is that, once all these potential confounders are controlled for, there may be insufficient independent variation in the NIM participation variables to enable comparison of different levels of NIM. Whether this will be the case is difficult to ascertain prior to data collection. A necessary requirement for the geographic analysis described here to be informative is that there be sufficient geographic variation in the changes in NIM participation over time. The first exploratory stage of the geographic analysis would thus be to analyse the geographic spread of income management participation patterns using the Centrelink administrative data.

For example, even if the NIM is rolled out at the same time to all regions of the NT, there will be some regions where a large proportion of the population is subject to this program, and other areas where the proportion subject in the population is small. If favourable changes in the outcome variables are observed in the former areas but not the latter, then this will provide strong evidence on the efficacy of the model.

Because most components of the NIM will vary together at the regional level, this impact analysis will be most suited to measurement of the overall impact of the NIM program, rather than particular components.

Note that it is intrinsically impossible in this analysis to separate the impact of the NIM model from other variables which vary closely along with it. For example, NIM is targeted at particular disadvantaged groups. In the case described in the previous paragraph, one cannot rule out that the observed association will be due to these groups doing better for some unexplained reason or because of another intervention. The research can only be made more robust by seeking to understand the impact of all the potential confounding factors.

The following considerations should guide the collection of data for this exercise.

5.5 Geographic scope

Ideally, the scope for this exercise should include comparable areas outside of the NT which have not been subject to IM. Data extraction from Commonwealth data collections should be designed with this intention in mind. However, many of the key outcome variables are only available via State government departments. These variables may be both defined differently and available for analysis in different forms in different States – which might thus require a restriction to the NT. Nonetheless, if it is envisaged that NIM will be generalised to other states and territories, collecting data from these jurisdictions now may form the grounding for future evaluations of those programs.

5.6 Time scope

The data should preferably cover the time period starting several years before the NIM implementation to a period after NIM is well-established and bedded down.

5.7 Regional aggregation

The data for the outcome variables should be collected at as small a regional level as possible, consistent with the relevant catchment areas for the different outcomes. For example, for alcohol expenditure, a suitable unit might be a township or a suburb. In both cases there might be spill-over effects into adjoining regions, but this can be modelled in the data analysis if the initial data collection is at a suitably low level of aggregation.

Similarly, data on migration between the regions can be incorporated into the modelling exercise. The most useful data for this will be Centrelink data on location patterns of the whole client base – not just those involved in NIM.

The different outcome variables will generally be available at different levels of aggregation. It is therefore important that the geocoding in the NIM data be as detailed as possible so as to permit the creation of NIM participation estimates at levels of aggregation that match the different outcome variables.

5.8 Take-up and participation in new income management

If geo-coded data on NIM participation is available, this can then be compared with Census and other estimates of small area populations to estimate the NIM participation rate in each area over time.

5.9 Confounding factors

The key confounding factors to be considered will be the presence of other policy interventions in the different areas. Detailed information on these will need to be included in the modelling. See Appendix G: List of funded initiatives in the NT that support vulnerable children and families.

5.10 Sub components

As NIM contains four types of participants (mainstream, vulnerable, voluntary and child protection), the framework will seek to address each group separately.  This is particularly true for the child protection component which not only potentially serves a different group of people, but also has very different entry and exit processes.  Although the data for the sub-components will overlap, it is important to disaggregate these components in the analysis to ensure that the appropriate processes and outcomes for each group are treated separately.

5.11 Evaluation of the child protection measure in NT

Evaluation of CPIM will require some specific data collection. Data will need to be collected from child protection workers in the form of either in-depth interviews or focus groups.

It is also proposed that a case file review be undertaken and coded according to a pro forma. This methodology is particularly effective for creating de-identified data from confidential client files.4 It is proposed that a case file review be conducted to evaluate the impact of CPIM on child protection outcomes (e.g. re-notification, re-substantiation, substantiated type abuse and primary presenting problems).

The evaluation will examine data provided by the NT Government, which will track those families who have been referred to CPIM.  The NT Department of Health and Families will provide information on:

• incidents of child protection notifications
• category of child protection notification (e.g.  neglect, physical abuse, sexual abuse or emotional abuse)
• whether or not the notification resulted in investigation by a caseworker
• whether or not reports were substantiated
• broad identifier of the reporter type (e.g.  hospital, family, policy, school) and
• number of the child's encounters with youth justice system.

The Child Protection records of each child whose family is referred will also be examined for up to a 5-year period preceding the referral, in order to establish whether income management results in changes in re-notification rates for families. Information will also be collected about the service use of these families including:

• What other support services has the family been referred to?
• What other services has the family accessed or failed to access?

The NT Department of Health and Families (DHF) caseworkers will also be surveyed via online electronic surveys which can be completed in various sessions over a period of time (e.g. several weeks) to allow for workload management. These surveys will be similar to those of Centrelink workers and other stakeholders, and will cover their attitudes to IM, training, relationships with other agencies and barriers and facilitating factors to helping families who neglect their children.

In addition to these data about the families, more qualitative information will also be sought via the child protection caseworkers, including:

• children's access to adequate food, clothing, education, health services, notifications and stability of living arrangements and
• parents' attitudes, financial management skills and confidence, levels of stress, knowledge of IM, exposure to harassment.

4. AIFS has successfully used this approach in evaluating the Magellan program and the 2006 changes to the family law system.

6. Ethics guidelines and ethical clearances for the evaluation of new income management 

6.1 Ethics guidelines

The National Health and Medical Research Council (NHMRC) has a set of advice and guidelines on ethics and related issues in the fields of health and human research. The evaluation should be undertaken in accordance with these guidelines and should, in our view, be approved by a Human Research Ethics Committee which is registered with the Australian Health Ethics Committee. In addition, when working in the NT approval may be required from the relevant NT ethics committees.

A significant portion of evaluation participants are likely to be of Indigenous background. The NHMRC guidelines include guidance for conducting research with Aboriginal and Torres Strait Islander people.5

In undertaking research with Indigenous people, particular attention needs to be paid to ensuring that participation is both informed and voluntary. Consideration needs to be given to the values underlying ethical research with Indigenous people - reciprocity, respect, equality, responsibility, survival and protection, spirit and integrity-across all aspects of the research process, including:

• consultation and negotiation
• mutual understanding of the purpose of the research
• the use of culturally appropriate instruments
• use of and access to research results, and
• communication of findings at a local level.

Consultation should begin prior to the commencement of the research and occur as an ongoing process throughout the evaluation. Consultation should be premised on respect, negotiation, and informed consent. Individuals and communities may need time to consider a proposed research project and to discuss its implications. Consultation and negotiation should achieve mutual understanding and agreement about the research’s aims, methodology, and implementation, as well as the use of the results it produces.

It is important that the consultation and negotiation process is not considered merely an opportunity for researchers to tell the community what they, the researchers, want. Indigenous knowledge systems and processes must be respected and acknowledged. Research in Indigenous studies must show an appreciation of the diversity and uniqueness of people and individuals. The intellectual and cultural property rights of Indigenous people must be respected, preserved, and acknowledged. Indigenous researchers, individuals and communities should be invited to be involved in research directly and as collaborators.

Appendix A: Earlier initiatives, background to the development of new income management 

Since 2007, the Australian Government has been progressively developing a national reform agenda in relation to welfare recipients in disadvantaged regions and in relation to dysfunctional families and communities. Measures implemented include:

• Income Management in the Northern Territory Emergency Response (NTER)
• Child Protection Scheme of Income Management (CPSIM)6
• Voluntary Income Management (VIM)
• Cape York Welfare Reform (CYWR)
• Improving School Enrolment and Attendance through Welfare Reform Measure (SEAM).

Each measure uses a combination of different tools to achieve its goals, including income management or increased conditionality on the receipt of income support. These measures are briefly described in the remainder of this section.

A1. Income management in the Northern Territory

Income management was first introduced as in the NT as part of the Northern Territory Emergency Response (NTER), announced in June 2007, to promote socially responsible behaviour and help protect children. Legislation was passed in August 2007 to enable income management.

The initial roll-out of income management only affected people (Indigenous and non-Indigenous) who received income-support payments and who lived in 73 prescribed communities, their associated outstations and 10 town camp regions of the Northern Territory.

Under the NTER model of income management, half of people’s welfare payments were set aside for the priority needs of individuals, children and their families. Income-managed funds must have been directed towards agreed priority needs and services such as food, rent and utilities. Income-managed funds could not be used to purchase excluded items such as alcohol, tobacco, pornography or gambling products.

The objective of the NTER income management measure was to ensure that:

• money paid by the government for the benefit of children is directed to the priority needs of children
• women, the elderly and other vulnerable community members are provided with better financial security and
• the amount of cash in communities is reduced to help counter substance abuse, gambling and other anti-social behaviours that can lead to child abuse and community dysfunction.

A2. Child protection scheme of income management (CPSIM)

The Commonwealth and Western Australian Governments are working together to implement a trial of CPSIM in Western Australia. A bilateral agreement supports this trial. Under this initiative, the Western Australian Department of Child Protection has the option of requesting that Centrelink manage an individual's income support and family payments in cases where poor use of existing financial resources is wholly or partially contributing to child neglect or other barriers the individual may be facing.

The Commonwealth Government has responsibility for income support and family payments and is therefore able to link certain conditions to these payments; however broader responsibility for child protection remains with the Western Australian Government.

CPSIM was implemented in specific Western Australian locations from November 2008 and has been progressively rolled out to the Kimberley region, and particular Department for Child Protection districts of metropolitan Perth.

As in the Northern Territory, income management involves Centrelink directing income support and family payments to meet priority needs such as food, clothing and housing. Income managed funds cannot be used to purchase alcohol, tobacco, pornography or gambling products.

Support services are offered to assist those on income management and include financial management services provided through FaHCSIA and Parent Support services provided through the Department of Child Protection. Appendix B provides an overview of current evaluation activities of this measure.

A3. Voluntary income management measure (VIM)

VIM was implemented in conjunction with CPSIM in Western Australia in late 2008. This initiative allows income support recipients in particular districts in metropolitan Perth and the Kimberly region to volunteer for income management to assist them to meet their priority needs and learn tools to help manage their finances for themselves and/or their family in the long term.

Individuals who are placed on VIM also receive a referral to financial counselling or financial education services funded by FaHCSIA. Appendix B provides an overview of current evaluation activities of this measure.

A4. Cape York welfare reform (CYWR)

A different approach to welfare is being trialled in the Cape York communities of Aurukun, Coen, Hope Vale, and Mossman Gorge and associated outstations. Cape York Welfare Reform is a partnership between the four communities, the Australian Government, the Queensland Government and the Cape York Institute for Policy and Leadership. The reforms, which will run from 1 July 2008 to 31 December 2011, aim to create incentives for individuals to engage in the real economy, reduce passivity and re-establish positive social norms.

Under the reforms, an independent statutory body called the Family Responsibilities Commission (FRC) has been established to help rebuild social norms in the four Cape York Welfare Reform communities. Components include referring individuals to support services and possibly to income management. Fifteen programs covering housing, education, social responsibility and economic opportunity are being rolled out as part of the reforms. Appendix B provides an overview of current evaluation activities of this measure.

A5. Improving school enrolment and attendance through welfare reform measure (SEAM)

SEAM aims to increase the enrolment and regular attendance at school of school-age children whose parents are receiving a schooling requirement payment7 by placing conditions on the parents' receipt of such payments to ensure their children are enrolled at and attending school on a regular basis. Parents who do not comply may have their schooling requirement payment suspended until they do comply. Other payments such as FTB and CCB continue to be paid during a SEAM suspension period.

SEAM does not reduce the primary responsibility of state and territory education authorities to respond to truancy issues. Rather, it is intended to provide an additional tool to help resolve intractable cases of no enrolment or poor attendance. SEAM includes a pathway to parenting resources for those parents who need assistance through Centrelink support and referrals to relevant services.

SEAM responds to the finding that an estimated 20,000 Australian children of compulsory school age are not enrolled in school, with many others not attending regularly. SEAM encourages parents to ensure that their children are enrolled at and regularly attending school by linking enrolment and attendance to parents' welfare payments. SEAM uses possible suspension of income support payments, supported by a case management approach, to encourage responsible parental behaviour. In extreme cases income support payments may be cancelled, but this is expected to be a very rare occurrence.

SEAM trials commenced from the beginning of the 2009 school year in six locations in the Northern Territory: Katherine, Katherine Town Camps, Tiwi Islands, Hermannsburg, Wadeye and Wallace Rockhole. The Department of Education, Employment and Workplace Relations (DEEWR) has the responsibility for evaluating this measure.

6. As part of the Australian Government’s commitment to improving outcomes for vulnerable and disadvantaged Australians, the Child Protection Scheme for Income Management (CPSIM) in Western Australia has been extended for a further year. Voluntary Income Management (VIM) is also offered in WA. Under the new income management measure in NT, the CPSIM is referred to as the Child Protection Measure (CPM). This reflects the subtle differences between the two compulsory measures designed to help Child Protection Authorities in WA and NT to help vulnerable families.
7. Schooling requirement payments are defined in section 124D of the SS(Admin)Act. They include social security benefits and pensions as defined in section 23 of the SSAct as well as 3 payments under the Veteran’s Entitlement Act 1986 (VEA).

Appendix B: Current income management evaluation activities 8

B1. Findings from the evaluation of the child protection scheme of income management and voluntary income management measures in Western Australia

The Australian Government implemented a trial of two separate models of income management in the Kimberley and Cannington regions in WA starting in November 2008. The evaluation of these trials, led by FaHCSIA and supported by WA Department of Child Protection (DCP), was undertaken by ORIMA Research Pty Ltd. The report was publicly released by the Minister on 8 October 2010 and can be found can be found on the FaHCSIA website.

The evaluation was designed to assess:

• the impact of income management in improving child wellbeing
• the impact of income management on the financial capability of individuals and
• the effectiveness of the implementation of income management.

The evaluation findings are based on various data sources including:

• quantitative data collected from surveys of people on VIM and CPSIM, and the comparison group9
• online surveys of Centrelink and DCP staff, as well as financial counsellors, money management advisers, WA peak welfare sector bodies and community organisations with an interest in income management
• administrative data from Centrelink, WA DCP and financial management service providers and
• qualitative data based on focus groups and interviews conducted with community leaders in the Kimberley area.

Since the start of trials of income management in WA in November 2008 to 30 April 2010, there has been a total of 1,131 people in receipt of Centrelink payments who have participated—328 who have been referred to income management by the WA DCP and 803 who have volunteered for income management. At 30 April 2010, there were 598 people on income management—226 people on CPSIM and 372 people on VIM. See Figure 1, below.

Figure 1: Number of current CPSIM and VIM participants (28 November 2008 to 30 April 2010)

Figure 1 identifies the number of current child protection income management and voluntary income management customers as of 28 November 2008 to 30 April 2010.

The graph illustrates that the number of voluntary income management customers is larger than those on the child protection measure, with both measures experiencing steady increases in customer’s number from 28 November 2008 to 30 April 2010.

B2. General findings

Overall the evaluation report suggests that income management had made a positive impact on the wellbeing of individuals, children and families. Moreover, since the introduction of income management, CPSIM respondents reported decreases in a range of negative behaviours in their communities.

There was an initial perception that compulsory income management (including CPSIM) would not be as well received or as able to achieve the best outcomes for individuals as would a voluntary scheme of income management (VIM). However, contrary to the common view that compulsory schemes, such as the Child Protection Scheme of Income Management (CPSIM), are not as effective as voluntary schemes, the evaluation findings showed that that both voluntary and child protection measures are having equally positive effects on the wellbeing of families in WA.  Around 62 per cent of people on CPSIM and 60 per cent of people on VIM thought that income management had made their life better.

A further breakdown reveals that more people on the VIM scheme (51 per cent of people surveyed) than the CPSIM (34 per cent) felt that income management had made their lives “a lot better”.  The remainder felt that income management had made their lives “a bit better” (9 per cent for VIM and 28 per cent for CPSIM). 

This finding is reinforced by the willingness of both people on CPSIM and VIM to recommend income management to others. Sixty-five per cent of CPSIM and 82 per cent of people on VIM reported that they had already recommended income management to someone else or planned to do so in the future. (A similar sentiment appears evident in the NT where early indications in the Barkly area show that 75 per cent of those who could have exited from the new income management scheme chose to remain on VIM).

There were three main reasons why people on income management had recommended or intended to recommend the scheme:

• income management can have a positive impact on people’s lives
• income management helps improve budgeting skills and saving money and
• due to the benefits associated with the BasicsCard.

The report also identified areas for improvement in terms of the income management schemes.  There was a low level of awareness among people on income management as to the proportion of funds that was being managed and the reasons they were subject to income management.  The report also revealed a need to increase the number of merchants that are approved to accept the BasicsCard and a need to improve communications to people on income management. The most commonly reported potential negative outcome was that some people might become dependent on income management.

B3.  Specific findings

There was an increase in the percentage of people able to buy essential items and meet priority needs, and an increased ability to save money on a regular basis.  Around 74 per cent of respondents reported that they had been unable to pay for at least one essential item (such as food, utilities, rent, bills and so on) in the 12 months prior to commencing income management.  However, during income management the proportion of people unable to pay for such items decreased by 25 per cent.

After exiting income management, many people reported that their increased ability to meet their own, and their children’s, priority needs that occurred whilst they were on the scheme continued and, in some cases, improved. Figure 2 below shows the proportion of people on CPSIM who ran out of money to pay for an essential item category before, during and after (in the case of people formerly on CPSIM) income management participation. During the income management period, both current and people who were formerly on CPSIM were less likely to run out of money for food, utilities, rent and other bills.

Whilst on income management (and compared with when they were not on income management), CPSIM respondents were most likely to report that the following positive impacts had emerged:

• they and their children had eaten less takeaway food (56 per cent and 55 per cent respectively)
• their children had eaten more food (54 per cent)
• they and their children had eaten more fresh food (53 per cent and 48 per cent respectively)
• they had purchased more clothes for their children (53 per cent) and
• they had purchased more toys for their children (48 per cent).

Figure 2: Proportion (per cent) of people running out of money for essential items, by item type: before, during, and after income management

Figure 2 illustrates the proportion of people running out of money for essential items before, during and after income management

Before income management, 59% of respondents were running out of money for food. During income management, this percentage dropped to 29%. After income management only 16% of people were running out of money for food.

Before income management, 40% of respondents were running out of money to pay utility expenses. During income management, this dropped to 9% of respondents.

Before income management, 31% of respondents were running out of money for the children’s education. During income management, this dropped to 14%. After income management, the percentage of people running out of money for children’s education increased to 21%. 

In terms of their ability to save money, around 70 per cent of CPSIM and 80 per cent of VIM respondents reported that they were regularly able to save money when they were on income management.  This is a significant increase from 51 per cent and 54 per cent respectively before being involved in the income management scheme.

The evaluation report indicates that the take up rates for money management and financial counselling services are low.  Around 31 per cent of all CPSIM respondents and around 14 per cent of all VIM respondents reported that they had attended financial counselling or money management services while they were on income management. Around 87 per cent of VIM respondents who accessed a financial management service thought that this service had provided them with skills to manage their money better. The introduction of the matched savings scheme and incentive payments on 9 August 2010 is expected to improve the take up of financial counselling services in the future.

Despite the low take up rates, over two-thirds of respondents across the four key stakeholder groups believed that income management had led to at least a moderate improvement in financial management among people on CPSIM, and around one-third of respondents across three key stakeholder groups believed that the trial had led to a large improvement in financial management among those people.

About 55 per cent of financial counsellors and money management advisers, and 52 per cent of Centrelink staff believed that the VIM trial had led to at least some decrease in the incidence of trial participants seeking emergency relief, emergency payments or other financial crisis support.

At a persons’ mid-term review, WA DCP case workers indicated that for 46 per cent of CPSIM cases people were ‘very likely’ or ‘fairly likely’ to manage their income to prevent neglect occurring in the future. At their final review, this had increased to 61 per cent of CPSIM cases.

Since the introduction of income management, CPSIM respondents reported decreases in a range of negative behaviours in their communities, including:

• 70 per cent reported less drinking
• 67 per cent reported less violence
• 62 per cent reported less gambling and
• 60 per cent reported less humbugging.

In addition, 56 per cent of CPSIM respondents felt that there had been an increased participation in cultural activities since the implementation of income management.

B4.    Potentially negative outcomes from income management

The most commonly reported potential negative outcome that stakeholders thought could emerge from income management measures in WA was that some people might become dependent on the system and not be able to manage their finances without remaining on income management. 

From July 2010, the income managed funds ratio was reduced from 70 per cent managed/30 per cent not managed to 50 per cent managed/50 per cent not managed for people on VIM.  The CPSIM ratio remained unchanged.  Although a large number of service providers (Centrelink [68 per cent] and financial services providers [56 per cent]) believed that the change had an adverse impact on the effectiveness of VIM, a few respondents commented that it was useful in promoting VIM to potential clients.

B5.    ORIMA Research Pty Ltd recommendations

ORIMA recommended the Government undertake the following actions.

• Develop a communications campaign that positively promotes the benefits of income management.
• Improve communications with people on income management about how the program operates.
• Provide targeted training about income management to DCP and Centrelink staff.
• Increase the number and variety of merchants accepting BasicsCard.
• Improve communications with people on CPSIM about why they are going on income management and how it will help them.
• Encourage participation in VIM upon completion of CPSIM.
• Give people on VIM a choice of the ratio of funds subject to income management.

B6. Cape York welfare reform

The evaluation is being conducted over three years with 3 parts.

• An implementation review of the FRC.
• A progress report covering implementation issues and early progress in achieving outcomes.
• An outcome report summarizing reports and providing evidence about whether change occurred against the four objectives of the reform:
  • rebuilding social norms
  • restoring Indigenous authority
  • increasing individual engagement in the real economy and
  • transitioning people to home ownership.

Key evaluation questions include:

• Are social norms and behaviours changing?
• Was the CYWR implemented as agreed?
• Have governance arrangements supported changes in the service provision, social norms and behaviours?
• Has service provision changed in a way that supports norm and behaviour change?

A comprehensiveevaluation framework and program theory for the Cape York Welfare Reform Trial

8. The information in this section was provided by FaHCSIA.
9. Table 2 presents the research design/number of interviews.
10. Number of people who participated in the survey as a percentage of all people in the sample who were contacted.

Appendix C: FaHCSIA's program logic for new income management 

Internal / External Factors

Internal and external factors include:

• Ministerial support
• government support and agreement (Commonwealth and Territory and State) and

other support services available in NT (including remote service delivery).

Potential Unintended Consequences

Introducing new policy/program may result in a number of potential unintended consequences. These may include:

• increased customer vulnerability
• priority needs not being met
• NIM model may ‘undo' the benefits of current IM model
• impact on stores (e.g. due to change in customer numbers subject to NIM or use of BasicsCards)
• impact on domestic violence
• impact on mobility (e.g. lack of cash, availability of BasicsCard and other services or to avoid IM) and
• access to/take up of other services.

Appendix D: Two representations of hierarchy of outcomes 

* While Centrelink does not provide a break down BC data by each stream of IM, it may be possible to obtain approximate distribution by stream by using 'imputation' – a statistical technique used in such cases.

Greater Self-Reliance And Economic, Social And Community Engagement And Increased Choices

This chart outlines how greater self-reliance and economic, social and community engagement and increased choices result from:

• increased financial literacy,
• Improved Children’s wellbeing (in relation to priority needs)
• Income management is no longer needed

These result from

• Ability to set aside money for unexpected expenses (e.g. illness, car repairs) and to develop basic household budgeting practices
• Increased take up of MM/FC services.
• Reduction in harmful behaviours and pressure to give money to family and friends (humbugging).
• Increased take up of education services, including school, early childhood education and age appropriate, social, learning or physical activities.
• Responsible parenting activities undertaken.
• Improved food and housing security
• Widespread ability to use BasicsCard (stores are available, understanding operation of card)
• Money is spent on priority needs

Appendix E: Literature review: an international literature review addressing research designs and outcomes in income management research 

E1  Summary

In recent years many governments around the world have introduced conditional welfare programs, targeting particular groups within society. The philosophical basis and justification lies in the welfare contract between a society that provides financial and other support to those people who need it, and the recipients themselves who must meet certain obligations and responsibilities in return for that support from the state. The new approach to welfare rights and responsibilities is not without critics, but many commentators who have written about conditionality conclude that it can be philosophically and morally justified provided that considerable care is taken to avoid burdening those people who are already unjustly disadvantaged.

The purpose of this literature review is to prepare the ground for developing a robust mixed methods evaluation of the new income management model in the NT of Australia. Any evaluation will need to be designed to be able to monitor the progress of the new income management model and be able to provide a robust overall assessment. The evaluation should also be designed to ensure that less tangible goals such as child, family and community wellbeing are captured in the evaluation. Here we review the international literature addressing research designs and outcomes in income management research.

E2 Conditional welfare and income management

E2.1 Introduction to conditional welfare

Income management is part of a larger category of welfare programs which are generally referred to as 'conditional welfare'.  The basis of conditional welfare is to use the benefit system or cash transfers as a way of changing behaviour. Internationally the majority of conditional welfare policies and programs have been targeted at the most disadvantaged members of society and are aimed at encouraging families to send children to school and ensure they are vaccinated.  In the broadest sense most government transfers are conditional in that payment is dependent on some conditions (e.g. seeking a job, undergoing a medical examination) being met by the benefit recipient.  However in the narrower sense discussed here, conditional welfare refers to policies which use the benefit system directly to influence behaviour.  Internationally the majority of conditional welfare programs involve 'Conditional Cash Transfers'  (CCT), in which welfare recipients are given a supplement when they behave in certain prescribed ways.

In recent years many governments around the world have introduced conditional welfare programs to influence a wide range of practices within specified populations. Broadly speaking, program components are either 'punitive', with negative consequences attached to undesirable behaviour and spending on 'excluded' items such as tobacco, alcohol, gambling, pornography; or program components are 'rehabilitative' which sees desirable behaviour incentivised and 'priority' spending, on healthy nutritious food, education, housing, clothing, transport and health, rewarded. Conditional welfare programs have been widely adopted in many countries as a new approach in social assistance for combating poverty and fostering social inclusion (de la Brière and Rawlings 2006).

Conditional welfare programs differ from standard welfare programs in that they require certain income or expenditure behaviour, or a change in that behaviour, before participants receive benefits (or in order for recipients to continue to receive benefits). Changes may include increasing or decreasing expenditure, or demonstrating other behavioural change. Conditional welfare programs tend to target families with children, particularly in low and middle income families. The focus is usually on managing expenditure on behalf of children. In practice, the position of the government usually appears as though it knows best about how money should be spent on children, and certainly better than many parents within the program. Government views are usually expressed through a set of formal conditions operating within the program. Parental views and preferences are often captured using social surveys, often expenditure survey, and other forms of evaluation and reporting undertaken by researchers and public service workers who are in regular contact with families and children in the program. Populations selected as the subjects of conditional welfare programs are almost always those considered 'vulnerable', either because they are poor (low socio-economic position) or because of some other form of social disadvantage. Specific sub-groups of the population are sometimes selected as subjects of conditional welfare programs. Examples include single parent families, recent migrants, and people living in particular geographical regions or within particular jurisdictions.

The conditional welfare philosophical framework views the welfare contract in terms of a society that provides financial and other support to those people who need it and the recipients themselves who must meet certain obligations and responsibilities in return for that support from the state (Dwyer 2000). There are also mutual obligations between citizens irrespective of claims on the state. However, some critics suggest that conditionality is contrary to liberal principles; while others argue that it burdens those who are already unjustly disadvantaged (Elizabeth and Larner 2009). In many modern high income societies such as Australia and the UK conditionality has already been firmly embedded in the welfare contract between the citizen and the state. Social security and unemployment benefits, for example, are usually paid only on condition that welfare recipients are actively seeking to enter the workforce (ACOSS 2005). Many who have written about conditionality, whether in its favour or not, conclude that conditionality can be philosophically and morally justified provided that considerable care is taken to avoid burdening those people who are already unjustly disadvantaged (Deacon 2004).

2.2  A note on conditional cash transfers (CCTs) programs

As discussed above, cash transfers programs can be targeted (unconditionally) at specific vulnerable groups, or distributed universally. Another form of cash transfer program has an element of conditionality. Conditional cash transfers (CCTs) programs link the provision of cash or grants to the behaviour of the target population, who are required to perform certain verifiable actions such as securing minimum investments in children's education and health. Payment is therefore dependent on regular school attendance, or use of preventive health services or some other specified condition. Pioneered in Latin America by Brazil and Mexico during the 1990s, CCTs have spread across Latin America and many other countries around the world. CCTs are seen as a promising solution to some of the toughest challenges facing social policy. In low and middle income settings, investment in CTTs is seen particularly important in relation to meeting the Millennium Development Goals (MDGs) of reducing hunger and tackling extreme poverty and vulnerability (DFID 2006). Efforts to help poor families to invest in children's health and education will ultimately help prevent transmission of poverty from one generation to the next. Improved nutrition in young children will, in turn, benefit their health and education, lead to healthier children, and these benefits will be passed on to the next generation.

E3 Purpose of the review

The purpose of this international literature review is to prepare the ground for developing a robust mixed methods evaluation of the new model of income management in the NT of Australia.

The evaluation methodology must be designed to be able to assess and monitor the progress or 'health' of the new income management measure in the Northern Territory. FaHCSIA has also made it clear that the evaluation should be designed to ensure that less tangible goals such as child, family and community wellbeing are captured in the evaluation. Here we review the international literature addressing research designs and outcomes in income management research.

E3.1  A note on the literature review

The international review of literature was exploratory rather than systematic and confined to the English-language literature. The starting point was the authors' existing knowledge of the literature and included searches in Google Scholar, followed by more specialised databases such as IngentaConnect, Project Muse and Cambridge Journals Online and the Medline to cover the health journals. Other sources were identified and included as their relevance became clear during the research process. Key words for the search were: income management, conditional cash transfers, conditionality, social protection, welfare, family income and expenditure. The key inclusion criterion for the literature review was the relevance to understanding research designs and outcomes in income management research.

E4 Conditional welfare programs

There are a number of conditional welfare programs. These are described below.

1. Opportunity NYC–Family Rewards (USA)
   In 2007, New York City's Center for Economic Opportunity launched Opportunity NYC–Family Rewards, an experimental, privately funded CCT program to help families break the cycle of poverty. Family Rewards is the first comprehensive CCT program in a developed country. Aimed at low-income families in six of New York City's highest-poverty communities, Family Rewards ties cash rewards to pre-specified activities and outcomes in children's education, families' preventive health care, and parents' employment. The three-year program is being operated by Seedco (a private, non-profit intermediary organization) in partnership with six community-based organizations. The program is being evaluated by MDRC (a non-profit, nonpartisan social policy research organization) through a randomised control trial involving approximately 4,800 families and 11,000 children, half of whom can receive the cash incentives if they meet the required conditions, and half who have been assigned to a control group that cannot receive the incentives.
   Despite initial challenges in understanding the program's large number of incentives and related payment requirements, nearly all families eventually earned rewards, more than $6,000, on average, over the first two years (Riccio, Dechausay et al. 2010). In addition, the program resulted in a number of other outcome measures:
   • evidence of reduced poverty and hardship, including reduced hunger, housing and health care hardships
   • increased savings and the likelihood that parents would have bank accounts
   • increased school attendance, but did not improve school outcomes or grades overall
   • increased use of health insurance coverage and use of health services and dental care services, and
   • increased employment levels.
2. Food Stamp Program (USA)
   The United States Supplemental Nutrition Assistance Program (SNAP), historically and commonly known as the Food Stamp Program, is a federal-assistance program that provides assistance to low- and no-income families. Though the program is administered by the U.S. Department of Agriculture, benefits are distributed by individual States. The program provides families with stamps that can be used to purchase eligible foods from eligible retailers. Stamps are provided to families on a means-tested basis, and are supplemental to other forms of welfare, rather than replacements. During the 1990s the program was revamped and actual stamps were phased out in favour of a specialised debit-card system, the Electronic Benefit Transfer. Evaluation of the program suggests that food stamps help to reduce poverty in part by freeing up other income without reducing the quality of food available in the household (Jolliffe, Gundersen et al. 2003).
3. Temporary Assistance to Needy Families programs (USA)
   The Temporary Assistance to Needy Families (TANF) program covers a range of programs in the United States which differ between States. In some States, individual Counties (similar to Australia's Local Government tier) may implement customised programs. The main aim is to provide temporary assistance for low income families with children under the age of 18. Programs include income support while looking for work, special payments and intervention programs for children and work placement schemes. Access to programs is usually dependent on child enrolment in school and adults actively seeking employment. Different evaluations have been carried out in different jurisdictions; however the general consensus is that the program has some positive effects on employment rates, but impacts on the lived experience of poverty and family incomes remain less clear (Moffitt 2003).
4. Self Sufficiency Project (Canada)
   The Self Sufficiency Project (SSP) in Canada was conceived and funded by Human Resources Development Canada. The project offered a temporary earnings supplement to selected single-parent families receiving Income Assistance (welfare) in British Columbia and New Brunswick. Between November 1992 and March 1995, more than 6,000 single parents who were long-term Income Assistance (welfare) recipients were invited to join the SSP research study. To collect the supplement (a monthly cash payment based on actual earnings), a single parent had to work full-time and leave Income Assistance. They were then eligible to receive the supplement for up to three years, as long as the single parent continued to work full-time and remained off Income Assistance. The supplement roughly doubled the earnings of many low-wage workers (before taxes and work-related expenses). The Self Sufficiency Project found positive outcomes for employment for adults and school performance for children. Poverty reduction impacts only persisted for the time earnings supplements were provided, but child education impacts persisted (Michaelopoulos, Robins et al. 2005).
5. Progresa (now Oportunidades) program (Mexico)
   In Mexico, the seminal Progresa program (now Oportunidades) began in 1997 and aimed to improve the health and education outcomes of low-income children. Households, selected on socioeconomic criteria, were given special cash payments provided that children regularly attended both school and appointments for preventive health care. Participating children aged 4 to 23 months were also given food supplements. Nutrition: research suggests that 70 percent of households participating in Progresa in Mexico have shown improved nutritional status. There have been reductions in stunting and the growth rate among infants has increased (Skoufias and McClafferty 2000; IFPRI 2002). Health: research on the Progresa program operating in Mexico suggests that the scheme boosted demand among women for antenatal care by 8 percent, which contributed to a 25 percent drop in the incidence of illness in newborns and 12 percent lower incidence of ill-health among children compared with non-Progresa children (Skoufias and McClafferty 2000). Immunization against measles increased by a mean of 3 percentage points 6 months after the beginning of the Progresa program and tuberculosis vaccination was 5 percentage points greater for infants than at baseline. However, there was less evidence of improvement in school outcomes or post-school employment opportunities.
6. Red de Proteccio´n Social (Nicaragua)
   In Nicaragua, the Red de Proteccio´n Social program was designed in a similar manner to the program in Mexico. Disadvantaged households in low-income areas received a cash transfer provided they brought their children who were younger than 5 years to preventive health examinations (where they received antiparasitic drugs, vitamins, iron supplements and vaccinations) and attended health education workshops. An additional cash transfer was contingent on enrolment and regular attendance at primary school. In Nicaragua's Red de Protección Social program, immunisation levels among recipient infants increased by 18 percent. Vouchers provided to sex workers and their clients for the treatment of sexually transmitted infections led to large declines in reported rates of syphilis (9%) and gonorrhoea (5.25%) (Sandiford, Gorter et al. 2002; Ensor 2004).
7. Familias en Acción (Colombia)
   In Colombia, the program Familias en Acción targetted the poorest households in disadvantaged municipalities. The program provided monetary transfers to mothers on the condition that their children who were younger than aged 7 years attended preventive health examinations, and another transfer if their children aged 7 to 17 years attended school regularly. Mothers were also encouraged to attend health education courses. Evidence from the Colombian program suggests an increase in children's preventive health care visits by 23 percentage points for children younger than aged 2 years and 33 percentage points for children aged 2 to 4 years (Attanasio, Battistin et al. 2005;
8. Programa de Asignación Familiar (Honduras)
   In Honduras, any household in municipalities with high prevalence of malnutrition and benefiting from the Programa de Asignación Familiar had access to two types of monetary incentives: one conditional on school attendance of children aged 6 to 12 years, and the other conditional on undergoing monthly health examinations for children and prenatal care attendance for pregnant women (Morris, Flores et al. 2004). The evaluation of the Honduras program Programa de Asignación Familiar showed a mean increase of 6.9 percentage points in the coverage of the first dose of diphtheria, tetanus, pertussis vaccine among children younger than 3 years.
9. Bolsa Escola (now Bolsa Familia) (Brazil)
   In Brazil, Bolsa Escola (now Bolsa Familia) is a national conditional welfare program that transfers $6-19 a month to an estimated 5 million poor families, at a cost of 0.15 percent of gross domestic product (Rawlings 2004). The program aims to improve child and maternal health among low income populations. Mothers received capped transfers based on the number of beneficiaries (either children younger than aged 7 years or pregnant or lactating woman) in the household. Transfers are conditional on attendance at health check-ups and nutrition workshops for the women and adherence to vaccination schedules for children. Education Brazil's program makes cash transfers to mothers with school-age children on the condition that school attendance rates of 90% are achieved. Research shows falls in school drop-out rates and higher enrolment rates in education (UNESCO 2003).
10. Mi Familia Progresa (My Family is Moving forward) (Guatemala)
    In Guatemala, the program Mi Familia Progresa (My Family is Moving forward) was introduced relatively recently in 2008. This program provides cash payments to poor mothers, conditional upon them sending their children to school and for health check-ups. Two types of monetary transfers, both targeted to women. A monthly health and nutrition transfer of around US$ 18 (Quetzales 150) is paid to mothers of children under the age of six, to pregnant women and to breast-feeding mothers, under the condition that they attend health centres to receive a basic package of nutritional and preventive maternal-child health care services. An education transfer of US$ 18 (Q 150) is paid to families with at least one child between the ages of 6 and 15 attending primary school or preschool. If more than one child in the family is within the prescribed age, all of them must show proof of school attendance for the mother to receive the monthly transfer. The two types of transfer are not mutually exclusive and they can be cumulated provided that the family qualifies for both and complies with the conditions. The amount provided is not adjusted to the number of children, size of the family or other special conditions, probably to avoid perverse fertility incentives but also to maximise the number of beneficiaries across the country. Given the short life of Mi Familia Progresa, there is little in the way of preliminary results to-date (Gaia 2010).
11. National Family Welfare Program (India)
    India launched the National Family Welfare Program in 1951 with the objective of reducing the birth rate to the extent necessary to stabilise the population at a level consistent with the requirement of the national economy. The program provides incentives for women to spend money on health related activities for themselves and their children. There is a strong focus on family planning including subsidising contraception and reproductive health. The program has been linked to reduced rates of fertility and infant mortality (World Bank 1995), for example, the achievements of the program at the end of the 7th five year plan (1985-90) were:
    • reduction in crude birth rate from 41.7 (1951-61) to 30.2 (1990)
    • reduction in total fertility rate from 5.97 (1950-55) to 3.8 (1990)
    • reduction in infant mortality rate from 146 (1970-71) to 80 (1990)
    • increase in couple protection rate from 10.4% (1970-71) to 43.3% (1990)
    • over 118 million births were averted by the end of march, 1990.
    Disentangling the distinct impacts of the project from other impacts such as increasing education and living standards, and increasing employment rates among women, remains an issue, particularly given the length of time the program has been running (for over 50 years). The National Family Welfare program continues to evolve and continues to target child and maternal health issues.
12. The Female Secondary School Stipend Project (Bangladesh)
    The Female Secondary School Stipend Project in Bangladesh was established to increase the enrolment of girls in secondary schools, thereby delaying marriage and childbearing. The program paid school fees and transferred an incentive payment direct into girls' bank accounts on condition of at least 85% school attendance. Recorded enrolments in program schools increased by 105% between 1994 and 1999 while aggregate enrolments of girls increased by 111%. In rural areas, enrolment rates increased by 12 percentage points per year. Unfortunately, accurate data to measure the specific impact of the program on enrolment do not exist. The School Stipend was one among many other interventions pushing for increased enrolment of girls, including increased focus on non-formal education and tuition waivers, and it is difficult to extract the impact of the stipend from these other causal factors (Raynor and Wesson 2006; Schurmann 2009).
13. The Zomba Cash Transfer Program (Malawi)
    The Zomba Cash Transfer Program (ZCTP) in Malawi was a randomised conditional cash transfer intervention program that provided incentives – in the form of school fees and cash transfers – to schoolgirls and young women who have recently dropped out of school to stay in or return to school. Baseline surveys were conducted with 3,805 girls and young women between October 2007 and January 2008. The study reported that the percentage of initial dropouts who returned to school and were in school at the end of the 2008 school year was 17.2% among the control group compared with 61.4% among the treatment group. Thus, program beneficiaries were 3-4 times more likely to be in school at the end of the 2008 school year than the control group. After one year, the program also reported declines in early marriage, teenage pregnancy, sexual activity, and risky sexual behaviour (Baird, Chirwa et al. 2009).

E5 Assessment of conditional welfare programs

E5.1 Impact assessment

Overall, the evidence suggests that conditional welfare programs are effective in increasing the use of health and education services, which essentially means they have had demonstrable impacts on the health and education related behaviours in the target populations. There is also good evidence that conditional welfare programs can improve nutrition and health status among target populations. Many issues remain however and conditional welfare programs continue to provoke much debate within social policy. As conditional welfare programs are a relatively recent contribution to social policy, longer-term impacts are less clear. The cost-effectiveness of many conditional welfare programs is often uncertain, particularly when compared with supply-side interventions such as improving infrastructure and services. While evidence indicates that even when cash transfers are not tied to service use, the additional income is often used for health, education and nutrition priorities. A recent evaluation of the impact of an emergency unconditional cash transfer scheme in Somalia, for example, showed that the provision of cash grants to women not only helped the poor repay debts and improve their food intake, but also empowered them to invest in their own health care (Ali, Fanta et al. 2005). Unconditional cash transfers are usually seen as being more appropriate where service coverage is poor.

There are also structural issues to consider. Social transfers are not a panacea to compensate for a lack of investment in equitable education and health services. Conditional welfare programs, to stand any chance of success, will need to operate in those areas where there are adequate services, schools, clinics and the like. In settings where access to public services is low, expanding capacity may be a preliminary step before the introduction of conditional welfare programs. User fees for health and education services may need to be abolished, waivered or reduced. Fee waivers are often an important complement to social transfers, provided adequate measures are put in place to meet the costs. This will likely increase the impact of both the social transfer and the fee waiver or free service policy although further research is needed on the combination of social transfers and supply side interventions that are most effective in different settings.

There may be issues relating to the size of the transfers needed to change behaviour in different settings; there may even be inefficiencies in conditional welfare programs. A program may yield quite a high marginal cost per outcome, particularly if money goes to all individuals in a targeted population regardless of compliance with the conditionality of a program. Ideally, positive outcomes of conditional welfare programs should be weighed against their cost effectiveness, particularly when conducting robust evaluations, but cost effectiveness is an area which is underdeveloped in this field as Lagarde et al. (2007) observe.

E5.2  Issues in the design and evaluation of conditional welfare programs

Decisions about the type and value of social transfers are extremely context-specific. Very often, the choice and design of social transfer programs reflects the priorities and political realities of policymakers, as much as technical feasibility, institutional capacity and affordability. There are a number of issues to consider in the design and evaluation of conditional welfare programs:

E5.3  Policy objectives

Is improving human development an explicit policy objective or an indirect objective of a program to reduce and poverty? When deciding which transfer is more appropriate in certain conditions, it is important to keep in mind that governments have multiple objectives. If a government wants, for example, to provide income support to tackle poverty and improve human development outcomes, then both conditional and unconditional transfers are likely to be the most appropriate choice. There may be broader social policy considerations to think about. There may be merit, for example, in combining transfers with other policy options, such as the possible removal of a service user fee. The conditional welfare program can provide a platform for coordinating overall social protection strategy, social policies and sector strategies that together contribute to improved education and health goals.

E5.4  Understanding demand

What types of financial and non-financial barriers to access need to be addressed? What is the optimal value of transfer required to overcome barriers for vulnerable groups? Who in the household should receive transfers to maximise impact? In some contexts it may be a good idea to conduct pilot work, before rolling out the conditional welfare scheme.

E5.6  Cash or restricted spending choice

Is there adequate provision of quality services to tie transfers to service use? Do vulnerable people have sufficient information to help make informed choices, if not additional efforts may be required to ensure information needs are met. To what extent are there political pressures or impediments to restricting spending choices through conditions or vouchers; is there the capacity to monitor compliance with conditions? Income management programs could also be complemented with a fee waiver scheme or, where appropriate, entitlement to free service provision. The transaction costs of alternative policy options will need to be considered and estimated. Vouchers work tend to be more successful when targeted to easily defined vulnerable groups and they often work best when the main cost involved is paying for the service, and there are no additional out of pocket expenses involved, or people are not giving up work time to access services.

E6. Targeted and universal approaches

What are the trade-offs between accuracy of targeting and the politics and cost of targeting? Are there existing programs that use effective approaches to targeting, e.g. geographical, community-based or proxy means-testing scheme that a social transfer scheme can also use?

E6.1 Cost-effectiveness

Will the benefits of social transfers – in terms of human development outcomes – exceed the costs of administering them?

E6.2 Political feasibility

What is the constituency for social transfer programs? Will financing be sufficiently predictable and long-term to sustain social transfer programs?

E6.3 Capacity, governance and accountability

What are the options for delivering cash and vouchers directly to people (e.g.  using banks, post offices or health centres); which different institutions and levels of government administration need to cooperate to implement social transfers effectively and in coordination with education and health service provision?

E6.4 Evaluation

Usually there is an imperative to monitor and evaluate social transfer schemes. Many cash management programs have been the subject of strict randomised controlled trials which help to ensure that the income management program effects are captured (see the review of evaluation methods, below). There may be opportunities to collaborate with other agencies on rigorous evaluations of social transfer programs to ensure that human development and poverty reduction objectives are addressed in order to build the evidence base.

E7  Review of methodologies for income management programs

E7.1  Hierarchy of evidence

The strength of evidence provided by research is dependent on both the methodology used and the quality of the particular study. Evidence hierarchies have been developed to help researchers to rank different research methods according to the validity of their findings, based on the risk of error or bias in their results. Figure 10.1 shows the conventional hierarchy of research evidence which puts randomised techniques above controlled and controlled above quasi experimental techniques which tend to rely on evaluation before and after an intervention. We have used the hierarchy of evidence to guide our thinking for this review; the hierarchy of evidence has also been used to structure our literature review of research designs and the results are summarised in Table 10.2.

A randomised control trial would be the gold standard for any aspiring new study looking to investigate any cash transfer or income management programs. The vast majority of studies reviewed in Figure 10.1 have adapted either a randomised controlled or a controlled study design (without randomisation of participants to treatment or intervention groups). Often researchers may rule out the gold standard of random allocation for political reasons or because it is unethical in community evaluations, and in this context the next best approach would be a 'stratified random' study which randomly allocated areas for introduction of the policy, or more commonly a controlled study in which matched locations or towns as similar as possible to treatment towns are identified, in order to try and identify any effects and outcomes of the income management program. A before and after quasi-experimental design would be the least desirable form of evaluation simply because it is hard to attribute change to an intervention. As shown in Figure 10.1, a systematic review of the results from randomised controlled trials and studies is usually considered the pinnacle of the research hierarchy of evidence.

Figure 10.1 Hierarchy of evidence

Figure 10.1 illustrates the hierarchy of evidence used to guide the researchers conducting the evaluation to rank different research methods according to the validity of their findings, based on the risk of error or bias in their results. Figure 10.1 shows the conventional hierarchy of research evidence which puts randomised techniques above controlled and controlled above quasi experimental techniques which tend to rely on evaluation before and after an intervention.

E7.2  Methodologies, randomisation and controls in income management trials

A good example of a randomised control study design to investigate the impact of an income management program is provided by the recent experience in Malawi as discussed above. Here, 3,821 young women living in villages in Zomba, a district in Southern Malawi, were allocated to receive the intervention (treatment), half received the intervention and the rest serve as the control group. The following schematic in Table 10.1 captures the research design. The program provided incentives – in the form of school fees and cash transfers – to schoolgirls and young women who have recently dropped out of school to stay in or return to school. Within each treatment community or village, all never-married 13-22 year-old recent dropoutswho are eligible to return to primary or secondary school were identified and were always treated (with conditional cash transfers).

Table 10.1 Malawi research design

The schematic in Table 10.1 captures the research design. The specific example used provided incentives – in the form of school fees and cash transfers – to schoolgirls and young women who have recently dropped out of school to stay in or return to school. Within each treatment community or village, all never-married 13-22 year-old recent dropoutswho are eligible to return to primary or secondary school were identified and were always treated with conditional cash transfers.

Source: Baird, Chirwa et al. (2009; 2009).

The study reported that the percentage of initial dropouts who returned to school and were in school at the end of the 2008 school year was 17.2% among the control group compared with 61.4% among the treatment group. Given the strict randomised control design, we can be fairly confident that the outcomes observed resulted from interventions in the treatment group; girls in the treatment group were 3-4 times more likely to be in school at the end of the 2008 school year compared to girls in the control group. After one year, the program also reported declines in early marriage, teenage pregnancy, sexual activity, and risky sexual behaviour (Baird, Chirwa et al. 2009).

As noted above, a randomised controlled study design is the gold standard for evaluating a new income management program. Accordingly, many of the studies reviewed in Table 10.2 have adapted such a design. Often researchers may rule out random allocation for political reasons or because it is unethical in community evaluations. In this context, the next best approach would be a matched study in which control locations or towns are chosen to be as similar as possible to treatment towns in order to try and identify any effects and outcomes of any new policy. In some instances a 'before and after' quasi- experimental design may be the only option. However, this design would be the least desirable form of evaluation simply because it is hard to attribute change to the intervention as other things that we cannot control, such as a rise in living standards or a general education campaigns, may play a role in changing people's behaviour. There may also be other factors which we are not able to observe. In a before and after design the outcomes of the intervention group are compared before and after the intervention. This type of evaluation relies on the availability of good baseline data on the variables of interest, including expenditure patterns, people's money management skills, and indicators and measures of the wellbeing of children, families and communities. However, realistic evaluation, particularly in the social sciences, must observe research principles, consider levels of funding, observe ethics, and politics, time constraints and routine datasets that may already be available to assist with any planned inquiry. In all research studies there are trade-offs to be made.

E8 Evaluation instruments used in income management program evaluations

In Table 10.2 we have also considered the range of instruments that researchers have used in their evaluations, we have summarised the results under broad categories including social surveys (which includes inquiries where questionnaires and/or personal consumption diaries are used), routine administrative data collected by schools and hospitals and other public bodies, face-to-face interviews and focus groups. It is clear from the review that researchers often make use of multiple instruments to evaluate the impact of income management, often involving mixed methods of qualitative and quantitative techniques.

Actual expenditure behaviour changes may be considered the primary outcome in the income management program. Second level outcomes relate to policy objectives that go beyond the immediate program. These may include increased school attendance or educational attainment, improved public safety, health and economic outcomes such as entry in the labour market. As mentioned above, outcomes arise from complex systems of which specific programs are one usually only component, causal relationships are often difficult to establish and usually require randomised controlled study designs. There is good evidence, as we saw above, to support a link between conditional welfare programs and increased school attendance of children, better use of health services and evidence of adults returning to work. In any evaluation great care needs to be taken to avoid overburdening the families involved in the programs that are likely to be disadvantaged already.

[ top ]

This table shows a review of methodologies and evaluation instruments of international conditional cash transfer schemes. Each study and its origins are assessed based on the nature of the evaluation design and its study group analysis. Evaluation instruments highlight the different methodologies and information sources for each individual study of conditional cash transfer schemes.

*Indicates use of MIS IDA Data – a software program developed by the Center for Social Development for use in the management of IDA programs. The program assists organisations in managing IDA accounts, program administration and evaluating programs (Carpenter, 2008: 5)

Appendix F: Data sources to support the evaluation 

F1 Administrative data sources
 

F1.1  Centrelink administrative data

The main source of administrative data which will be used in evaluating NIM is Centrelink Management Information System (MIS) data. This will enable a comparison of individuals who are income managed with individuals who are not income managed, including in areas of Australia not subject to income management.

This data will provide information on:

• the implementation of NIM
• reasons for being subject to income management (disengaged youth; long-term welfare recipients; referred by child protection authorities; vulnerable welfare payment recipients)
• reasons for exemptions from income management
• the allocation of income managed income by areas of spending
• attendance at a money management and financial counselling service
• receipt of matched savings program
• movements into and out of income management by reason
• whether those being income managed continue to receive income support payments 
• for those who continue to receive some Centrelink payments, whether they are have income from paid employment and
• geographic mobility (for example if the introduction of NIM results in persons potentially impacted moving from the NT).

The Centrelink administrative data can provide longitudinal data on the experience of those being income managed and how they compare to those of people not being income managed. It also allows the experience and outcomes of different groups to be compared, including:

• Indigenous status
• geographic location and
• whether income management is voluntary or compulsory.

F2  Northern Territory administrative data

Consultations with NT government departments indicates that generally administrative data produced by the NT government does not provide information on whether people are receiving particular types of Centrelink payments or whether they are being income managed.

While this data cannot be linked to the specific individuals affected by NIM it can be used to evaluate the impact of NIM collectively.11 The key areas for which it appears that NT administrative data can contribute to the evaluation of NIM are:

• child health and development (low birth weight; stunting, wasting, underweight, anaemia in children under 5 12, child protection substantiations)
• education (school enrolment and attendance data13 ; National Assessment Program – Literacy and Numeracy (NAPLAN) data14 )
• sales of alcohol15 and tobacco16
• crime data17
• housing data and
• incidence of late payment/non-payment of utilities bills.

To the extent possible data is sought on a detailed geographic basis, and where possible as a time series. This reflects the analytical needs of the evaluation. Utilising the ecological approach, and comparing the outcomes for regions with different proportions of the population subject to income management, permits the isolation of the impact of the program, relative to general trends and other universal changes. Similarly the availability of time series data permits trends to be observed, as well a 'before' and 'after' data.  It is important to note that the evaluators will require permission to use all these datasets, and that some data may not be available (or the evaluation may not be given permission to use the data) for small communities.

F3 Store based data

There are sources of administratively based store data that will be of value to the evaluation. These are:

• information derived from the licensing arrangements for stores in remote communities and
• point of sale/scanner data on the composition of spending.

In addition, as detailed below it is proposed to utilise surveys to obtain more specific information from retail and other outlets.

F4 Evaluation specific data collections

Ideally, in evaluating social policies data would be collected prior to the implementation of the policy (baseline data). Data collected after the implementation of the policy would then be compared to the baseline data to track changes over time.

However, in the case of NIM, the fact that income management has already been happening in 73 discrete communities since 2008 makes collecting baseline data that that have not been tainted by the earlier, more localised, income management scheme difficult, if not impossible.

In order to evaluate NIM, primary data will need to be collected from:

• people subject to income management
• Community Elders
• Centrelink staff
• merchants
• money management and financial counselling service providers and
• child protection case workers.

A combination of quantitative and qualitative data will be needed. While NIM is being implemented across the NT it will be necessary to understand the extent to which the implementation and impacts of income management vary by gender, location, age, educational status, work status, income, length of time on income support, marital status, family composition, and diverse cultural and linguistic background.

F5 Survey of clients subject to income management

This component involves conducting randomly-selected representative quantitative surveys of people being income managed and those who had been income managed in the past (under the NTER). These surveys would be supplemented by a qualitative study which would provide a rich description of their attitudes towards and experiences of income management and the impact it had on them and their families.

The first survey would be of people being income managed at the time of the interview or likely to commence on income management in the near future. The second would be of those who had been income managed in the past but were no longer being income managed at the time of the survey.18

This would allow data to be collected from people who are subject to income management for an extended period (or who have returned to income management after a period of not being subject to income management) and those who have moved off income management.

While ultimately the sample size that can be achieved will depend upon cost, the sample size for each survey should ideally include at least 500 people. The sample needs to be randomly selected, and it should be stratified in order to ensure that data are collected from individuals of various ages, from men and women, and from a range of different geographical locations that represent the diversity of the NT. The areas selected should include major town/cities, regional centres and remote communities.

The survey will need to collect information from a representative sample of people subject to the measure in order to assess whether experiences and outcomes differ. The number of geographic areas from which data is collected will depend upon cost. The greater the number of areas from which data is collected the greater the cost. The sample should be clustered both to contain cost and to allow community level effects to be estimated. The sample should include those who are being compulsorily income managed and those who are being voluntarily income managed. The sampling frame will be from Centrelink administrative data on people on income management.

This kind of sample is very challenging to administer, particularly in remote areas, and the evaluators will need to test out appropriate methodologies for engaging evaluation participants and facilitating accurate data collection.

Attempts should be made to link the survey data of participants to other data sources including administrative data. This data linkage can only happen with consent and approval would be required from data custodians. Possible sources of data are enrolment and attendance at school for children, NAPLAN and data on child development and health.

The in-depth interviews will ask participants to consider the impact of income management on their families. Family impacts will be important from a number of perspectives; for example, it is possible that some individuals will be income-managed while others in their family will not, and alternatively, some income-managed individuals will be part of families where most family members are also covered by the scheme, with potential implications for outcomes. Family behaviours will also be of interest, since outcomes could be affected by whether people tend to make “whole of family” shopping trips or purchase items together or in bulk.

This component would collect information on a range of issues including:

• expenditure on and consumption of alcohol, tobacco and gambling
• parenting skills
• understanding of income management
• experience of using the BasicsCard and other methods that can be used to spend income managed funds and whether they had any problems
• perceived impact of income management on health of children, their own health and family relationships
• self reported impact of income management on the extent to which they experience harassment of financial exploitation
• experience of money management and financial counselling and whether the respondent thinks this helped
• housing stability and adequacy
• whether the respondent was granted a matched savings scheme payment and, if so, what the money was spent on and
• issues to do with stigma associated with being income managed.

F5.1 In-depth interviews/focus groups with Indigenous community leaders

In order to assess the impacts of the NIM on Indigenous people it is proposed to conduct in-depth interviews and/or focus groups with community leaders to discuss the implementation of income management on the community. Community Leaders will be asked about the impact of income management on children's health and wellbeing, alcohol consumption, gambling and substance misuse, and violence in the community.

F5.2 Quantitative survey of merchants accepting BasicsCards

In order to understand the impact of store licensing and income management on stores, income managed customers and the broader community it is proposed to conduct a quantitative study of merchants who are registered with Centrelink for income management.19 This survey would need to be conducted in several waves with the first during the second half of 2010 or the first half of 2011 and second wave as late as possible in the period over which the evaluation of NIM is being conducted. A small number of qualitative interviews could be conducted with merchants.

The information from the quantitative survey of merchants would be combined with data on the level of sales and the nature of goods purchased from licensed merchants and how this had changed over time. The survey could include questions about the impact of NIM on their store, what is bought using income managed money and whether income management has changed this, whether income management creates stigma, whether the BasicsCard works effectively, whether other payment methods work effectively, other effects of NIM on the community, customers understanding of income management, impact of income management on the store, and whether income management has increased their costs.

The quantitative survey of merchants could be conducted by telephone (with an option for online completion) perhaps combined with a follow-up mail out survey depending upon the type of information being sought. A further expansion would be to cover the possibility of getting scanner data from some of the major retailers.

F5.3  Survey of Centrelink staff

The principal aim of this component of the evaluation would be to provide information on: (a) implementation issues; (b) how the exemptions system works and how they assess whether an exemption to being income managed should be given and their perceptions of how well this works; (c) the interaction between Centrelink and those providing money management and financial counselling; and (d) the perceptions of staff concerning the impact that NIM is having on those being income managed and their children and dependents. A further component would be a more specific survey of the staff involved in the debriefing interviews with those coming off the old NTER scheme of income management and considering whether or not to go onto VIM, as it would appear that this would be the best way of getting data about the motivations of the groups opting into or out of VIM.

This survey would need to be conducted in several waves with the first during the first half of 2011 and the second wave as late as possible in the period over which the evaluation of NIM is being conducted. In addition, a small number of qualitative interviews or focus groups could be conducted with Centrelink staff.

It is recommended that this survey be conducted on-line given that Centrelink staff will have access to an internet connection, it allows for the survey to be completed anonymously and is the most cost-effective means of collecting quantitative data from Centrelink staff.

F5.4  Survey of money management and financial counselling support services staff

The money management and financial counselling support services will be delivered by community organisations. The services provided will include crisis support, financial counselling, advocacy and budgeting and money management education.

This component would involve collecting information from the CEO or a senior manager from the organisations providing these services and staff. The interviews with the CEO or a senior manager would collect detailed information on the type of services their organization provides, whether they were able to recruit and retain suitable staff and their views about the impact of the income management scheme and how it is being implemented. The survey of staff (generally professional staff) would collect the following information:

• professional background information about the respondent, including the job they performed
• respondents' perceptions about the efficacy of the service, including:
  • efficacy of the assessment of people's needs
  • networks/working relationships with the community
  • networks/working relationships (including referrals) with other services
  • the ability of the service to respond to people's needs
• range and nature of services provided by their organisation, including the duration and content of money management courses
• geographic scope of the services provided by their organization and
• whether there are any unintended consequences.

F6 Existing survey sources

F6.1  The National Aboriginal and Torres Strait Islander Social Survey (NATSISS)

The NATSISS was conducted in 2002 and 2008 (from August 2008 to April 2009) and the next wave is scheduled to be collected in 2013 with data not becoming available until 2014.

F6.2  Footprints in Time: the Longitudinal Study of Indigenous Children (LSIC)

LSIC aims to improve the understanding of, and policy response to, the diverse circumstances faced by Aboriginal and Torres Strait Islander children, their families and communities.

The study collects information about the lives of Aboriginal and Torres Strait Islander children, covering areas including:

• the children—their physical and mental health, how they develop socially and cognitively, their place in their family and community, and significant events in their life
• the children's families—their health, work, lifestyle, and family and community connectedness
• the children's communities—facilities, services, and social and community issues, and
• services—child care, education, health and other services used by the child and the child's family.

The first wave of LSIC, conducted in 2008, collected information from the parents and carers of two groups of children, mostly aged between 6 to 18 months and 3½ - 4½ years. A second wave was conducted between March and December 2009 and the third wave of data is being collected in 2010.

In the first wave of the Longitudinal Study of Indigenous Children, parents affected by income management/quarantining through Centrelink were asked “How is it going?” and were given the opportunity for an open-ended response. Wave 2 included an identical question. In Wave 3 study participants are being asked whether they are being income managed, whether they think income management/quarantining is a good or bad thing for their family, whether since income management/quarantining they have noticed any changes (either positive or negative) in their community and, if so, the nature of the changes they have noticed. LSIC also provides some information on the family's financial situation and how they manage their money.

The survey collected information about the lives of Aboriginal and Torres Strait Islander children in areas such as children's health, learning and development, family and community relationships, culture and identity, and community issues and services. For Wave 1, the Footprints in Time team interviewed parents and carers of 1,687 Indigenous children aged between six months and five years, from 11 sites around Australia; from Adelaide to Broome, Dubbo to Darwin, Nowra to Thursday Island.20

11. For example, if the NIM is expected to have an impact on, say, the level of alcohol consumption, while this may be difficult to measure at the individual level it would be expected that any effect should be apparent in the relative levels (or changes in the relative level) of consumption of a community where a large number of people are subject to the NIM compared areas where relatively few are impacted on by the program.
12. Data available from the Northern Territory Growth Assessment and Action (GAA) report.
13. Student Enrolment and Attendance Collection 8 November - 2008, 2009 & 2010
14. Wholesale Alcohol Supply 2001 - 2008
15.  Dependent on whether there is any NT data on tobacco sales & if this available over time and by location
16. The Northern Territory Department of Justice publishes quarterly data on the incidence of selected crimes (Murder; Attempted Murder; Manslaughter; Robbery; Assault; Sexual Assault; Other Offences Against the Person; House Break-ins; Break-ins to Commercial or Other Premises; Motor Vehicle Theft and Related Offences; Other Theft; Property Damage; Other Property Offences) <Research and Statistics Unit>
17. Ideally a longitudinal study which collects information shortly after commencing income management and then approximately 18-24 months later would be conducted as part of the evaluation. However, the budget for the evaluation is likely to not be sufficient for this and so it is proposed that two cross-sectional surveys of clients of income management be conducted.
18. Under the NIM a person being income managed will be able to use their income managed funds to purchase priority goods and services at stores that are not registered for income management of the BasicsCard by organising for Centrelink's Income Management Payment and Contact team to pay the store by cheque or electronically.
19. The final study data included information from:
    • 1,687 parents or primary carers of an Indigenous child.
    • 960 Study children were in the Baby cohort, approximately 6–18 months old
    • 727 were in the Child cohort, approximately 3 years 6 months to 4 years 6 months old
    • approximately 1,500 Study child questionnaires, which included information on language, height and weight
    • approximately 240 interviews with the secondary carer (Parent 2) and
    • approximately 45 questionnaires collected from teachers, child care centres or family day care providers.

Appendix G: List of funded initiatives in the NT that support vulnerable children and families 

With regards to the evaluation of the Australian Government's trial of new income management (NIM) in the NT, it is important to be aware of the suite of other government measures, initiatives and programs that may have direct or indirect implications on the outcomes presented in the report.

It should be highlighted that any observations attributable to the difference between the implementation snapshot study and follow up data need to be considered in the context of these other initiatives, rendering clear causality statements about the effectiveness of NIM a more difficult prospect. In the research context, these other programs can be considered as representing 'confounding variables'. Nevertheless, it is clearly valid to consider the effects of income management policy in any holistic examination of peoples' changed circumstances.

This paper briefly outlines a number of programs and initiatives aimed at vulnerable members of the community in the NT.

Firstly, there are a number of other welfare conditionality reforms currently occurring, some of which may exist in the same locations to be observed for the purpose of this evaluation. These include:

• 'Learn or Earn' - an initiative encouraging Australian youth aged under 21 who have not completed year 12 or equivalent to engage in an appropriate level of study or training in order to be eligible for Youth Allowance payments, and
• the School Enrolment and Attendance Measure (SEAM) - aims to decrease the number of eligible school-age children not attending school by suspending the income support payments of parents of truant children or those not enrolled at school. The actual suspension of income support payments is expected to be a rare punishment; warnings, reminders and offers of support will be the first priorities in educating parents about the need for their children to attend school.

Associated somewhat with these measures—in the area of employment, education and training—is the Keep Australia Working initiative, designed to limit in Australia the negative effects of the Global Financial Crisis. This initiative includes the Centrelink Jobs Expos which, along with FaHCSIA-funded Commonwealth financial counsellors, aim to maximise employment and training opportunities in local communities.

Within the policy area of child protection and family support, initiatives include the Family Support Program and the REACh (Responding Early Assisting Children) program directory of services. The first of these, the Family Support Program, incorporates three core streams, covering community and family partnerships, family and parenting services, and family law services. REACh is intended to improve the capacity of families and caregivers to respond appropriately to children's needs for care, development and safety through timely access to community resources that can support them in their parenting role. It aims to improve access to support services and encourage at risk and vulnerable families to engage with their community through collaborative approaches. Other programs and initiatives include the National Framework for Child Protection, the Register of Harm Prevention Charities, the National Plan for Foster Children, Young People and their Carers, the 'Keeping Safe; Child Protection Curriculum' operating in the Northern Territory, and Early Childhood Initiatives, such as the 'Every Child is Important' project, Parenting Information Projectand Child Care Links Projects. Attached to the end of this document is an exhaustive list, courtesy of the NT Government, of currently funded programs and initiatives in the NT that support vulnerable children and families.

Indigenous-specific programs include a large range of initiatives under the Closing the Gap banner, related to the improvement of Indigenous life expectancy, infant mortality, early childhood development, education and employment. Associated programs include the Indigenous Children Program and Indigenous Parenting Support Services.

In measuring the success of NIM in terms of increased community safety and the reduction of addictive and damaging behaviours, such as gambling and drug-taking, a number of other initiatives need to be considered as playing a role. These include the Family Violence Prevention Strategy, local targeted initiatives such as dry bans and those associated with drinking and gambling licenses, and the National Drug Strategy.

Incorporated within the actual model of NIM are several programs and initiatives that may be accessed in order to complement the efforts of the model itself and therefore potentially have an impact on the results. The aims and objectives of such initiatives are integral to the policy direction of new IM; the model's overall effectiveness depends upon the optimal uptake of such services by people on income management. The provision of these services are a core component of the new income management model and are not seen as peripheral or extra services.  

Money management and financial counselling support services will help people who are income managed and others in the community to develop the skills to appropriately use their income support payments. It is not compulsory for people to go to one of these services, but it is encouraged because they provide a range of support that can help people avoid or resolve financial difficulties and put in place good ways to manage their money. Services are delivered by community organisations and include crisis support, financial counselling, advocacy and budgeting and money management education.

The Matched Savings Payment is an incentive payment to encourage people on income management to develop a savings pattern and increase their capacity to manage their money. If eligible, a person can receive $1 for every $1 they save, up to a maximum of $500. A person can only receive a Matched Savings Payment once.

The Matched Savings Payment is paid directly into the person's income management account.  To receive a Matched Savings payment an individual must:

• be income managed (excluding VIM and Cape York income management)
• complete an approved money management course
• maintain a pattern of savings from their discretionary funds for 13 weeks or longer after the commencement of the approved money management course and
• not have previously received a Matched Savings Payment.

Individuals who voluntarily participate in income management are eligible for an incentive of $250 for every six months they remain on VIM. Income management is supported by financial counselling and money management services, totalling $53 million over four years.

The following table gives an indication of the range of initiatives in the NT which support vulnerable children and their families, and which could influence outcomes for people subject to NIM.

References 

References Framework

Australian Government (2009), Policy Statement, Landmark Reform to the Welfare System, Reinstatement of the Racial Discrimination Act and Strengthening of the Northern Territory Emergency Response.

Australian Institute of Health and Welfare (2009), Report on Evaluation of IncomeManagement in the Northern Territory, 20 August. accessed: 15/12/09

Centrelink: Voluntary Income Management Information Sheet, accessed: 10/12/09

Department of Families and Housing, Community Services and Indigenous Affairs: Child Protection Measures Information Sheet, accessed: 10/12/09

Department of Families and Housing, Community Services and Indigenous Affairs: Income Management Debit Card Information Sheet, accessed: 10/12/09

Department of Families and Housing, Community Services and Indigenous Affairs: Voluntary Income Management Information Sheet, accessed: 10/12/09

The Hon. Jenny Macklin: Media Release Major Welfare Reforms to Protect Children and Strengthen Families, 25/11/2009 accessed: 10/12/09

London, Scott (1996) Understanding Change: the Dynamics of Social Transformation.  

References: Literature Review

ACOSS (2005). Welfare Reform: Participation or Punishment? Strawberry Hills, NSW, Australian Council of Social Service.

Ali, D., T. Fanta, et al. (2005). Cash relief in a contested area: lessons from Somalia. Humanitarian Practice Network Paper Number 50. London, Overseas Development Institute.

Attanasio, O., E. Battistin, et al. (2005). How effective are conditional cash transfers? Evidence from Colombia. Briefing Note No. 54. London, Institute for Fiscal Studies.

Attanasio, O., et al. (2004). Baseline Report on the Evaluation of Familias en Accion. London, Institute for Fiscal Studies.

Attanasio, O., E. Fitzsimons, et al. (2006). Child Education and Work Choices in the Presence of a Conditional Cash Transfer Programme in Rural Colombia. WP06/01. London, Institute for Fiscal Studies.

Attanasio, O., L. C. Gomez, et al. (2005). The Short-Term Impact of a Conditional Cash Subsidy on Child Health and Nutrition in Colombia. London, The Institute of Fiscal Studies.

Baird, S., E. Chirwa, et al. (2009). Schooling, Income, and Sexual Behavior: The Design, Implementation, and Short-Term Impacts of a CCT Program for Schooling in Malawi. Washington, DC, The World Bank.

Baird, S., E. Chirwa, et al. (2009). The Short-Term Impacts of a Schooling Conditional Cash Transfer Program on the Sexual Behavior of Young Women. Washington, DC, World Bank.

Barrera-Osorio, F., M. Bertrand, et al. (2008). Conditional cash transfers in education: design features, peer and sibling effects evidence from a randomized experiment in Colombia. Policy Research working paper ; no. WPS 4580.

Carrillo, P. E. and J. Ponce Jarrín (2009). "Efficient delivery of subsidies to the poor: Improving the design of a cash transfer program in Ecuador." Journal of Development Economics 90(2): 276-284.

de la Brière, B. and L. B. Rawlings (2006). Examining Conditional Cash Transfer Programs: A Role for Increased Social Inclusion? Washington, DC, Social Protection, World Bank.

Deacon, A. (2004). Can conditionality be justified? On condition. P. Collins and A. Rossiter. London, Social Market Foundation.

DFID (2006). Using social transfers to improve human development. SOCIAL PROTECTION BRIEFING NOTE SERIES, NUMBER 3. London, Department for International Development.

Dwyer, P. (2000). Welfare rights and responsibilities: contesting social citizenship. Bristol, Policy Press.

Elizabeth, V. and W. Larner (2009). "Racializing the “Social Development” State: Investing in Children in Aotearoa/New Zealand." Social Politics: International Studies in Gender, State & Society 16(1): 132-158.

Ensor, T. (2004). "Consumer-led demand side financing in health and education and its relevance for low and middle income countries." The International Journal of Health Planning and Management 19(3): 267-285.

Gaia, E. (2010). Mi Familia Progresa: Change and Continuity in Guatemala’s Social Policy. Social Policy Review 22: Analysis and Debate in Social Policy. I. Greener, C. Holden and M. Kilkey. Bristol, Policy Press: 199-223.

Glewwe, P., P. Olinto, et al. (2003). Evaluating the Impact of Conditional Cash Transfers on Schooling in Honduras: An experimental approach, Final Report for USAID.

IFPRI (2000). SECOND REPORT: Implementation Proposal for the PRAF/ IDB Project – Phase II PROJECT COORDINATING UNIT (UCP).

IFPRI (2001). Fourth Report of the PRAF Series PRAF/ IDB Phase II: Analysis of the Situation Before the Beginning of Distribution of Vouchers and Project Implementation. Washington, DC.

IFPRI (2002). PROGRESA: Breaking the Cycle of Poverty. Washington, DC, International Food Policy Research Institute.

Jolliffe, D., C. Gundersen, et al. (2003). Food Stamp Benefits and Child Poverty in the 1990s, United States Department of Agriculture.

Lagarde, M., A. Haines, et al. (2007). "Conditional Cash Transfers for Improving Uptake of Health Interventions in Low- and Middle-Income Countries: A Systematic Review." JAMA 298(16): 1900-1910.

Michaelopoulos, C., P. K. Robins, et al. (2005). "When financial work incentives pay for themselves: evidence from a randomized social experiment for welfare recipients." Journal of Public Economics 89: 5-29.

Moffitt, R. A., Ed. (2003). Means-Tested Transfer Programs in the United States. Chicago, University of Chicago Press.

Moore, C. (2008). Assessing Honduras’ CCT Programme PRAF, Programa de Asignación Familiar: Expected and Unexpected Realities. Country Study no. 15. Brasilia, Brazil, International Poverty Center.

Morris, S. S., R. Flores, et al. (2004). "Monetary incentives in primary health care and effects on use and coverage of preventive health care interventions in rural Honduras: cluster randomised trial." The Lancet 364(9450): 2030-2037.

Ponce, J. (2006). The Impact of a Conditional Cash Transfer Program on Students’ Cognitive Achievements: The Case of the “Bono de Desarrollo Humano” in Ecuador, Documento de Trabajo 06/301. Facultad Latinoamericana de Ciencias Sociales - Sede Ecuador.

Ponce, J. and A. S. Bedi (2008). The Impact of a Cash Transfer Program on Cognitive Achievement: The Bono de Desarrollo Humano of Ecuador, IZA Discussion Paper No. 3658, Institute for the Study of Labor.

Rawlings, L. (2004). A new approach to social assistance: Latin America’s experience with conditional cash transfer programs. Social Protection Discussion Paper Series No 0416, World Bank.

Raynor, J. and K. Wesson (2006). "The Girls’ Stipend Program in Bangladesh." Journal of Education for International Development 2(2).

Regalia, F. and M. Robles (2005). SOCIAL ASSISTANCE, POVERTY AND EQUITY IN THE DOMINICAN REPUBLIC. Inter-American Development Bank.

Riccio, J., N. Dechausay, et al. (2010). TOWARD REDUCED POVERTY ACROSS GENERATIONS: Early Findings from New York City’s Conditional Cash Transfer Program. New York, MDRC.

Sandiford, P., A. Gorter, et al. (2002). Vouchers for health: using schemes for output-based aid. Public Policy for the Private Sector Note Number 243. Washington, DC, World Bank.

Schady, N. R. and M. C. Araujo (2006). Cash Transfers, Conditions, School Enrollment, and Child Work: Evidence from a Randomized Experiment in Ecuador. World Bank Policy Research Working Paper 3930. Washington, DC, World Bank.

Schurmann, A. T. (2009). "Review of the Bangladesh Female Secondary School Stipend Project Using a Social Exclusion Framework." J HEALTH POPUL NUTR 27(4): 505-517.

Skoufias, E. and B. McClafferty (2000). Is PROGRESA working? Summary of the results of an evaluation by IFPRI. Discussion Paper 118. Washington, DC, International Food Policy Research Institute.

UNESCO (2003). Education for All Global Monitoring Report 2003/04. Paris, United Nations Educational, Scientific and Cultural Organization.

Velásquez-Pinto, M. D. (2004). The Bono Solidario in Ecuador: an exercise in targeting, ESS paper 17, Social Security Policy and Development Branch, International Labour Office.

World Bank (1995). India’s Family Welfare Program: Toward a Reproductive and Child Health Approach. Washington, DC, World Bank.

Abbreviations and glossary of terms 

Last updated: 13 April 2016 - 3:29pm