Mission Australia Centre - Outcome Project Final Report

Table of Contents

Executive Summary

This document is the final report for a project funded by the then Department of Family and Community Services (FaCS) and now Family and Community Services and Indigenous Affairs (FaCSIA).

The project aimed to:

  • Present best practice evidence for preventing homelessness in men and specifically indigenous men;
  • Identify the best strategies for implementing outcomes measurement;
  • Use currently available outcomes and client satisfaction measurement tools to record client outcomes during the implementation of the new service model; and
  • Make recommendations for the use of outcomes tools in other SAAP services.

Additionally, this project intended to address the following research questions:

  1. Has the introduction of a best practice service model for Campbell House resulted in more positive outcomes for clients?
  2. What are the most useful outcomes and client satisfactions measurement tools and delivery methods for SAAP accommodation clients and staff?
  3. What are likely to be the most effective implementation strategies to roll-out the routine use of standardised outcomes measurement tools in Mission Australia SAAP?
  4. What are the costs associated with outcomes and client satisfaction measurement in SAAP services?

The project, conducted in three phases, ran between September 2004 and July 2006. The first phase involved a retrospective audit of client outcomes for those accommodated at the original service between January 2003 and September 2004. Phase 2 recorded client outcomes and satisfaction between December 2004 and June 2005 while the service was located at temporary premises. The third phase of the project commenced in July 2005 and measured client outcomes and satisfaction at the redeveloped Campbell House, renamed the Mission Australia Centre. The purpose in recording outcomes at the different service models was to ascertain whether outcomes from these three periods of service delivery highlighted any changes.

The original service, Campbell House, had been operating for 30 years as a men's crisis accommodation service with shared rooms, provision of meals and laundry. The service provided few day time activities, and a worker client case load of 1:19. Following research into best practice models, an evaluation of the service, and consultation with community partners, the decision was made to rejuvenate the service. This change was required to reflect current best practice in service provision and to break the 'revolving door' cycle of entrenched homelessness.

To implement the new service delivery model the building required extensive renovation. During this time the service was located temporarily at St Vincent's Hospital.

Following renovation, the Mission Australia Centre (MAC) now houses:

  • Spectrum Apartments - 5 X 8 bedroom apartments for men who have been homeless for up to four years, intensive case management, worker client case loads of 1:8, living skills support and access to the;
  • Cooinda Centre - Services and activities for people who are homeless or at risk of homelessness;
  • Urban Apartments - Nine affordable housing units for singles or couples who work in the inner city but who struggle with the affordability of private rental; and
  • Creative Youth Initiatives - An art, music and photography program for young people aged between 16 and 24 years.

The project's literature review identified that effective programs are a combination of supportive community based services and adequate affordable permanent housing. While this form of accommodation is not provided at the MAC, it does provide a 'home-like' environment with intensive support, living skills education and access to a wide range of supportive community based services, both on and offsite. Further recommendations from the research indicate that assisting clients at first point of contact; smaller client worker case loads; connections with health and employment services; consumer decision making and involvement; and the move away from institutional care are conducive to best practice service delivery. These recommendations are consistent with the Mission Australia Centre practices.

To specifically address the needs of Indigenous homeless men, the Indigenous Homeless Services Improvement Strategy was developed. The strategy comprises Indigenous Homeless Awareness Training for staff; provision of outreach services; partnership with Indigenous Organisations; Indigenous Voices Initiative; and an Indigenous Employment and Personal Support Initiative. These services were integrated into MAC.

Further literature was reviewed to identify the most effective strategies for implementing change, and in particular, the measurement of outcomes. The key strategies identified included:

  • Agency commitment;
  • Integration into case management practice;
  • Combining information systems that support data input and report generation;
  • The purposes and limitations of outcomes measurement are fully understood by key stakeholders;
  • Training and support during implementation;
  • Multifaceted interventions such as reminders, audit and feedback, opinion leaders;
  • Positive organisational, economic and community environments;
  • Early identification and intervention to resolve any barriers to change;
  • Adequate resourcing; and
  • Reinforcement of change.

The strategies above, were combined with others identified during the project and literature review. These were:

  • A planned and systematic approach to outcomes measurement;
  • Raising both internal and external awareness of outcomes;
  • Refining outcomes tools and practices to meet client, staff and service needs;
  • Goal selection that is client driven and worker supported;
  • Consultative, collaborative and consensual decision making with case workers;
  • Ongoing group and one on one education, training and support; and
  • Support from senior management.

Key results from the project showed that the client profile across the three phases was similar. Due to the subjective nature of the assessment of outcomes in Phase 1, the outcome results for this phase may not be a true reflection of actual outcomes. Phase 2 was not conducive to the introduction of outcomes measurement due to a range of issues likely to have impacted on outcomes results.

Significant differences were found between Phase 2 and Phase 3 results in a number of areas. A larger proportion of clients in Phase 3 selected goals and were much more likely to be successful in achieving them. In addition, the numbers of clients achieving goals in Phase 3 for Accommodation, Drug and Alcohol and Gambling are significantly higher. Whilst the reasons for the improvement in outcomes between Phase 2 and Phase 3 cannot be confirmed, it is very likely that the improved physical environment, greater availability of services on site and a systematic approach to outcomes measurement played a key role. Phase 3 staff were enthusiastic about the process, were well supported and had a client ratio of 1:8. These factors most likely contributed to improved outcomes and outcomes measurement practices.

An analysis of costs associated with implementation and ongoing use over the 42 week period in Phase 3 was undertaken. The cost of implementing outcomes measurement across 130 clients by 5 staff members over this time was approximately $10,000. This cost includes training, development of tools and ongoing support to staff. Case management and other operational costs were not included.

Recommendations for use of outcomes measurement tools in other SAAP services include a planned and systematic approach; training and ongoing support for staff; resources dedicated to the facilitation of the implementation; tools that provide visual achievement of client goals; tools that are adaptable to meet client needs and fit with staff and service delivery; and development of electronic tools to assist in collating and analysing outcomes data.

Further recommendations for research with the MAC include trialling goal weighting; verifying whether services received by clients are linked with goals achieved; determining the most effective interventions to assist clients to achieve their goals; and a longitudinal project to determine whether crisis accommodation service interventions have long term impact on client outcomes.

The project concluded that the MAC now offered a best practice service delivery model for men and indigenous men and that the measurement of outcomes in SAAP services is certainly feasible and effective if approached in a systematic and supported way using appropriate tools.

Abbreviations Used
Abbreviation Meaning
DOCS Department of Community Services - State N.S.W.
FACS Department of Family and Community Services - Federal
FACSIA Department of Family and Community Services and Indigenous Affairs - Federal
MA Mission Australia
CH Campbell House
ICHRG Inner City Homeless Reference Group
SNDS Special Needs Dental Service
SAAP Supported Accommodation Assistance Program
OCH Office of Community Housing
IPU Intoxicated Persons Unit
NDCA National Data Collection Agency
SMART SAAP Management and Reporting Tool
RAT Retrospective Audit Tool
MAC Mission Australia Centre

Return to Top

Background

Introduction

The redevelopment of the Mission Australia Centre cost $7m in capital funding. These funds were provided by donors, State Government, and a generous bequest. In addition, State Government funded a position to oversee the project and transitional arrangements. Funding for the Outcomes Measurement Project and the Indigenous Homeless Services Improvement Strategy research position was provided by FaCSIA.

As a result, Campbell House underwent a physical transformation and was renamed the Mission Australia Centre. The MAC building now provides an intensive support accommodation service; a range of on-site services and programs for people who are homeless or at risk of homelessness; a creative arts, photography and music program; and low cost affordable housing. The public is able to access many of the on-site services which will help integrate homeless people into the local community and assist in deinstitutionalising the service.

Due to the introduction of a new and innovative service delivery model the Outcomes Measurement Project was undertaken to:

  • Present best practice evidence for preventing homelessness in men and specifically indigenous men;
  • Identify the best strategies for implementing outcomes measurement;
  • Use currently available outcomes and client satisfaction measurement tools to record client outcomes during the implementation of the new service model; and
  • Make recommendations for the use of outcomes tools in other SAAP services.

In addition, this project intended to address the following research questions:

  1. Has the introduction of a best practice service model for Campbell House resulted in more positive outcomes for clients?
  2. What are the most useful outcomes and client satisfactions measurement tools and delivery methods for SAAP accommodation clients and staff?
  3. What are likely to be the most effective implementation strategies to roll-out the routine use of standardised outcomes measurement tools in Mission Australia SAAP?
  4. What are the costs associated with outcomes and client satisfaction measurement in SAAP services?

Service History

Campbell House

Campbell House was an established homeless men's service that operated in Surry Hills, Sydney for 30 years with the purpose of providing crisis accommodation. Historically, Campbell House provided three types of accommodation, Residential Accommodation, Proclaimed Place or Intoxicated Persons Unit and a Nursing Home/Detox Unit. The Residential Accommodation Service was provided for homeless men in crisis. The men were over the age of 18 and able to stay for a period of up to three months. The residential service had 57 beds and was open 24 hours a day, seven days a week. The IPU had 28 beds and the nursing home, 25. The Intoxicated Persons Unit (IPU) operated as a safe sobering up unit where people could receive clean clothes, a shower, food, crisis intervention and a bed.

Clients were able to access the service directly, via Missionbeat or referral from other agencies, community services or government departments. Support options include intensive individualised assistance, advocacy, referral, meals, clothing, laundry and dental services. Clients ate in a communal dining room with meals prepared by kitchen staff. Clients' laundry was also done by staff. Case management was provided by three case workers with a worker to client ratio of 1:19. Two welfare support staff per shift assisted clients with primary care needs.

A Special Needs Dental Service (SNDS) was also on site, available for use by the clients. SNDS provided a multi-disciplinary approach to oral health care for those with exceptional, rarely addressed needs. Dental services were offered to homeless men and women between the hours of 8am to 4.30pm Monday to Thursday.

Services for the clients of Campbell House were visits from Centrelink and Mental Health crisis teams. Sports activities were provided by South Sydney Council every Monday. Books and reading material were also available for clients.

As each client was referred to the service, an initial assessment was undertaken. This determined whether the client was appropriate for the service and whether their needs could be met. At this point, accommodation was either allocated or the client was referred to a more appropriate service. Case management could commence once the accommodation was allocated. In ongoing case management a case plan was set between the case worker and client. The client exited the service once alternative accommodation was found.

Clients accessing Campbell House may also have been using a variety of other crisis accommodation services in the inner city. These services may have included:

  • Foster House;
  • Matthew Talbot Hostel;
  • Albion Street Lodge; and
  • Edward Eagar Lodge.

In 2001 Mission Australia undertook an internal evaluation of Campbell House including the three service delivery areas, IPU, Nursing Home and residential accommodation. The evaluation key findings relating to the residential accommodation identified:

  • The service was 'evening focused' with very few activities provided for the clients in the day;
  • No living skills or employment programs available for the clients;
  • No specialist counselling available within the service;
  • Little community integration;
  • No formal structure for transferring clients into medium term accommodation;
  • Case workers had case loads of 1:19 which were considered to be high;
  • Staff had differing understandings of case management principles and that the case management needed to be more proactive; and
  • Staff required case management and computer skills training.

The evaluation also highlighted issues with the nursing home and IPU being on the same site as the residential accommodation. Those using the hostel were a different group of homeless men from those utilising the nursing home and IPU. An aspect of the evaluation was to determine whether the service was meeting the changing needs of the current client group which appeared more complex than clients of 30 years ago. Research was also undertaken into homeless people residing in the inner city.

A consultative reference group of service providers within the inner city was established consisting of representatives from The Salvation Army, St Vincent de Paul Society, The Haymarket Foundation, The Wesley Mission, Department of Community Services and Department of Housing. This group evolved into the Inner City Homeless Reference Group (ICHRG) which became the Campbell House working party that oversaw the development of the new model. The Reference Group also discussed the planning of services for homeless people in the inner city.

The ICHRG made the decision to provide more specialised service to inner city homeless men. As a result Campbell House IPU closed in October 2003, with funding going to The Salvation Army's Foster House and the Haymarket Foundation's Albion Street Lodge. The Campbell House Nursing Home funding was taken up by the Salvation Army's, Foster House. Matthew Talbot Hostel and St Vincent de Paul, continued with the older and more entrenched homeless men. A specialist case worker was employed for Campbell House to assist in finding longer term accommodation and support for Campbell House clients.

Mission Australia worked extensively with inner city homeless agencies, local residents and state and local governments to ensure that services continued to be available at alternative locations and that the Mission Australia Centre satisfied the needs of clients and the local community (Mission Australia, 2004). Network meetings were held with Health, Housing, Police and the local Surry Hills community.

Campbell House - transitional service at St Vincent's Hospital

The development of a new service delivery model in Campbell House required extensive building renovations. In September 2004 the Residential Accommodation service of Campbell House relocated to temporary premises at St Vincent's Hospital in Darlinghurst. The service model was consistent with that provided at the Campbell House location although the change of locality enforced some changes such as:

  • The number of beds decreased to 29;
  • Bedrooms consisted of three single, two double, one triple, two with four beds, one with five beds, one with six beds; and
  • Client case loads initially dropped to 1:9 or 1:10 per case worker then increased to 1:29.

Mission Australia Centre (MAC)

The new MAC opened on 1 July 2005 with no break in service.

The MAC was developed to target men who are homeless for the first time. The model has a preventative approach with a focus on early intervention and provision of a holistic response to the many issues faced by the client group. By providing intensive support and opportunities to learn skills at the initial crisis stage clients should be more prepared and skilled in moving into more permanent accommodation that will lead to reintegration into the community. Evidence suggests that these changes will result in improved outcomes for clients.

The environment of the Mission Australia Centre is warm and inviting with culturally appropriate décor. The public is able to access some of the services, which will help integrate homeless people into the local community and society. The building is structured to maximise on-site opportunities for clients and provide normalisation through assimilation with the surrounding community.

The site now houses four components of service delivery:

  • Spectrum Apartments

The intensive support accommodation service targets homeless men in crisis who have been homeless for a period of up to four years. The service is available for clients for three month periods at any one time. Residential clients accommodated in Spectrum Apartments have their own bedroom within one of the five apartments. The apartments consist of eight bedrooms, one lounge room and dining area, kitchen, laundry facilities, bathroom, computer area/meeting area, outdoor balcony, and office for a Unit Coordinator. Each apartment is assigned a Unit Coordinator who provides clients with intensive case management, living skills development including cooking and cleaning, advice and information, emotional support, advocacy and support, and referral. The worker client case load ratio is 1:8. Support is provided 24 hours a day, 7 days per week. Clients are able to have supervised meetings with children and families within the units.

The following diagram highlights the current service delivery model for clients residing at Spectrum Apartments.

Diagram highlights the current service delivery model for clients residing at Spectrum Apartments

  • Cooinda Centre

Clients are able to access on-site facilities in the Cooinda Centre located in the ground floor of the building. Services offered in the Cooinda Centre include the Special Needs Dental Service; a General Practitioner; Optometry; Chiropractic; Gambling counselling; generalist counselling; library; Centrelink; Mental Health Clinic; Legal Clinic; Barber; Alcohol Anonymous (AA), Gamblers Anonymous (GA) and Crystal Meth Anonymous (CMA - Ice) Support Groups.

Educational programs include Dealing with Conflict and Improving Relationships; Indigenous Voices Initiative; Computer classes; art classes; drama classes; guitar tuition; keyboard tuition, oral health information sessions; financial; literacy awareness; tax clinic; and the Catalyst University Project.

Importantly, the Cooinda Centre offers a range of training and education programs and is also able to support families to access pre-schools and schools for their children. The Catalyst Program is offered in partnership with the Australian Catholic University and enables people experiencing homelessness to attain a Certificate of Liberal Arts. In addition the Centre also offers workshops and training focused on:

  • Effective Communication;
  • Urban Living Skills;
  • Computer Training;
  • Literacy Skills;
  • Anger Management;
  • Oral History/Storytelling;
  • Performing Art; and
  • Cooking Classes.
  • Urban Apartments

Nine low cost affordable housing units including five studios and four one bedroom units are available on site for low income earners working close to the city. These units provide affordable accommodation for men/women or couples who do not require support but allow them to avoid the crisis accommodation system. Whilst support such as case management is not available to this group, the facilities provided within the building will be available to them. Rent is calculated at 30% of tenants' gross weekly income. Applicants must require a minimum of 12 months tenure. Mission Australia Centre Staff manage the rental and maintenance.

  • Creative Youth Initiatives

Creative Youth Initiatives is an established Mission Australia service providing visual arts and music programs for youth. The service assists young people aged 16-25 years who are homeless, at risk of becoming homeless or are socially disadvantaged. The service has been relocated to the basement of the Mission Australia Centre. 'Sounds of the Street' is an accredited TAFE course in which participants learn to compose, perform and record music, producing commercial and professional CDs. Artworks is a visual arts program which provides materials, skilled tuition and a safe place in which to create and explore personal issues.

Indigenous Homeless Services Improvement Strategy

The Indigenous Homeless Services Improvement Strategy, developed as part of this project, aims to inform and guide the delivery of culturally sensitive, appropriate and accessible services for Indigenous people experiencing homelessness. The strategy reflects a commitment to partnership between the service and the Indigenous community which aims to build systemic and sustained outcomes to deliver both reduced rates of Indigenous homelessness and improved life long outcomes.

Mission Australia employed an Indigenous researcher to consult with the aboriginal community in the inner city of Sydney, and to advise Mission Australia on the type of programs which would allow greater access for aboriginal homeless people to a range of services which assist in the prevention of homelessness within this community.

To implement and evaluate this strategy, Mission Australia has been successful in obtaining further demonstration project funding from FaCSIA through the National Homelessness Strategy.

The Strategy currently comprises five key program initiatives including:

  • Indigenous Homeless Awareness Training for Staff;
  • Provision of Outreach Services;
  • Partnership with Indigenous Organisations;
  • Indigenous Voices Initiative; and
  • Indigenous Employment & Personal Support Initiative.
  • Indigenous Homeless Awareness Training for Staff

Recognised across the service system and also by the Indigenous Homeless Services Improvement Strategy, is that many services currently available for people experiencing homelessness are culturally inappropriate for the Indigenous community and are therefore poorly utilised.

Thus while Mission Australia recognises that the employment of Indigenous support workers is the most effective strategy to improve the service accessibility, staff training and skill development is also essential.

The Indigenous Homeless Awareness Training Program1 will be refined to address the specific issues of Indigenous homelessness including:

  • Racism;
  • Key social and cultural issues affecting the Indigenous community;
  • The pathways in and out of homelessness and the cycle of homelessness within the Indigenous community; and
  • Hidden homelessness and overcrowding.
  • Provision of Outreach Services

The outreach service will set up regular contact with Indigenous organisations that are first contact with homeless Indigenous people such as the Aboriginal Housing Company and the Aboriginal Medical Service. The service will also establish a support network and referral system with agencies considered relevant to the Indigenous homeless population of inner city Sydney including health, welfare, employment, education and legal services, housing organisations and hospital emergency departments.

  • Partnership with Indigenous Organisations

The Partnership with Indigenous Organisations program aims to achieve partnership across Indigenous organisations that are in first contact with homeless Indigenous people (such as the Aboriginal Housing Company and the Aboriginal Medical Service Outreach Service). The components of the program are as follows:

  • The establishment of a system of referral to locally based support agencies;
  • Identify existing Indigenous organisations in the inner-city, including Marrickville
  • and Newtown;
  • Develop a contact list/personal advisors from each organisation in contact with Indigenous homeless people;
  • Set up regular Koori BBQ's and Pizza Days;
  • Conduct an Open Day/Information Day;
  • Involvement of the Aboriginal community in the process of developing programs/services; and
  • Membership of the Aboriginal Homelessness Network Group.
  • Indigenous Voices Initiative

The Indigenous Voices Initiative offers an innovative combination of education and arts training in four disciplines - oral history/storytelling, drama, music, and local Aboriginal linguistics (Darug).

The program is designed to give the Indigenous homeless population a voice to share their experiences of inner city homelessness, and reflects an increased commitment to better understand Indigenous homelessness. It is intended that the oral histories program will break down the one-dimensional perspective of Indigenous homelessness and reveal its complex realities.

It is our vision that the Indigenous Voices Initiative will instill a strong cultural identity, improved self-esteem and a sense of belonging that together will support Indigenous people to gain greater control of their lives.

The program is sensitive to the ownership of cultural knowledge and copyright issues.

  • Indigenous Employment & Personal Support Initiative

This program is designed to increase the health, well-being and self-sufficiency of Indigenous people experiencing homelessness by increasing the number of employed Indigenous people.

Temporary placements and trainee placements within various services will be encouraged through the Indigenous Employment Initiative. This will be coordinated in conjunction with the Personal Support & Employment Centre Program planned for future implementation at Mission Australia Centre.

In order to support the education and training needs of the Indigenous people, liaison will be made with Centrelink, TAFE, Eora and Tranby Colleges, University of Sydney and University of NSW.

In addition a Youth Employment Initiative is proposed to target Indigenous homeless youth aged 1625, who left school prior to the completion of year 12. This will be coordinated in conjunction with the Creative Youth Initiatives and the Employment Centre provided at the Mission Australia Centre.

Other service initiatives prioritised for action at the Mission Australia Centre include:

  • The provision of a shower facility in case of great need (i.e. job interview, getting ready for work);
  • The provision of lockers for short term storage;
  • The availability of a washing machine and dryer;
  • The acceptance and storage of mail; and
  • The provision of a transport service for visits to health, welfare and housing services.
  1. For further information on course outline, see the Aboriginal Homeless Research Report produced by Angie Pitts for the Mission Australia Centre in October 2005.

Return to Top

Literature review

The literature review is split into two sections. The first section, preventing homelessness, provides a cross section of aspects pertaining to homelessness issues such as, a definition of homelessness, the changing face of homelessness, the relationship between homelessness, housing and employment, and the cost of homelessness. This section also covers specific literature on delivering homeless services to Indigenous People. Literature on best practice models is also incorporated. The second section addresses strategies for implementing outcomes measurement.

Preventing homelessness

Definition of Homelessness

Homeless people are those that do not have access to safe, secure and adequate housing. The factors leading to homelessness are diverse but can include long term unemployment, inadequate weekly income, lack of exit points from crisis services, inaccessible public housing, mental health and inadequate health and disability services for homeless people (Robinson, 1998).

Chamberlain & MacKenzie classify homelessness in the following categories (Chamberlain & MacKenzie, 2001, p.12):

Culturally recognised exceptions: Categories
where it is inappropriate to apply the minimun standard - eg seminaries, gaols, student halls of residence Marginally housed: people in housing situations close to the minimum standard
Tertiary homelessness: people living in single rooms in private boarding houses - without their own bathroom, kitchen or security of tenure
Secondary homelessness: people moving between various forms of temporary shelter including: friends, emergency accommodation, youth refuges, hostels and boarding houses
Primary homelessness: people without conventional accommodation (living on the streets, in deserted buildings, improvised dwellings, under bridges, in parks etc.

This is the definition used in this report.

Based on the Australian 2001 Census night, 100,000 people were counted as homeless, almost half (48,000) were staying in temporary accommodation with family members or friends. The next largest group were those living in boarding houses, 22,887 whilst those sleeping rough were counted at 14,158 with those in crisis accommodation at 14,251 (Chamberlain and MacKenzie, 2003). The number of people identified as homeless in NSW on Census night 2001 were 26,676 (Chamberlain and MacKenzie, 2003).

The changing face of homeless services

Research by the 1998 National Symposium on Homelessness in the United States of America (USA) indicates that the nature and characteristics of homelessness changes over time (Fosburg and Dennis, 1998). Mission Australia has seen change in Campbell House's 30 year history with male clients presenting with more complex issues, as well as a younger age group, increased drug usage, mental illness and family breakdown and an increase in demand for homeless services.

A growing understanding of homelessness and a realisation that increasing numbers of Mission Australia clients are young men with complex needs, has led Mission Australia to change the delivery of its services for homeless people (Mission Australia, 2004). There have been concerns that some service models may, in fact, have institutionalised homeless people and kept them apart from the community. These factors support the remodelling of the Campbell House.

Research undertaken in Scotland indicates that service users at institutional hostels and facilities were believed to be segregated and stigmatised (Rosengard Associates and Scottish Health Feedback, 2000). Large scale hostels were seen to be inappropriate due to the lack of security, stability, and opportunity for homeless people to develop independent living skills (Communities Scotland, 2001).

Where homeless services once provided hot meals and a bed for the night, services are moving towards the reintegration of clients into the community by providing skills and support (Fosburg & Dennis, 1998). In both the UK and USA, large institutionalised hostels which once provided accommodation for homeless people are being phased out and replaced with smaller units or self contained furnished flats (Rosengard Associates and Scottish Health Feedback, 2000).

Given that homelessness has changed in the past we can also expect that homelessness will continue to change in the future. Flexible service delivery, which caters to clients' individual needs, a wide range of support services to accommodate those needs, innovative new models, assistance at the first point of contact, services targeting particular client groups are key aspects in reducing homelessness.

Homelessness and Housing

The lack of affordable housing for low-income households in Australia is a contributing factor to homelessness. There has been an increasing demand for affordable housing due to population growth and decreases in average household size. Over the past 15 years, the supply of housing for low-income households has not met the increasing demand (Australian Institute of Health and Welfare (AIHW), 2003, p.159). Home ownership is not a feasible option for many low-income earners, particularly young people, due to high indirect taxes and land shortages (AIHW, 2003, p.174). The availability of rental property for low income earners decreased by 28% between 1986 and 1996 (AIHW, 2003, p.174). In Sydney, there is a shortage of affordable housing particularly in the inner city. This is evidenced by over 20% of Rental Assistance recipients spending over half their income on rent (Commonwealth Advisory Committee on Homelessness, 2001, pp. 13-43). The lack of affordable housing contributes to homelessness because people who are marginally housed may have difficulties finding alternative accommodation if needed, and people who become homeless may find it increasingly difficult to secure any accommodation.

Homelessness and Employment

In Australia in 2001, approximately 60% of homeless people aged between 18-35 were either working or looking for work (Commonwealth Advisory Committee on Homelessness, 2001, pp.13-43). In 1997-98, 62% of SAAP clients indicated that one of the three most important issues for them was the lack of employment opportunities (Commonwealth Advisory Committee on Homelessness, 2001, pp.13-43). Mission Australia believes that accessing regular and stable employment plays an important role in the prevention and reduction of homelessness (Mission Australia, 2004). The Commonwealth Advisory Committee on Homelessness recommends integrating homeless services with employment programs tailored for homeless people in order to increase clients' employment opportunities (Commonwealth Advisory Committee on Homelessness, 2001, pp.13-43).

Cost of homelessness

Due to the complexity of homelessness experiences, it is difficult to assess the actual financial costs to the community. There is general agreement that homelessness incurs higher social and economic costs to the community in contrast to someone who is not homeless. Economic costs of homelessness include the use of government and non-government resources, personal financial costs to individuals and families, increased use of health services and lost potential economic productivity in other sectors (Berry, Chamberlain, Dalton, Horn & Berman, 2003, p.1). The social cost of homelessness includes the reduction of social cohesion by increasing the strain on social networks (Berry et. al, 2003, p.1). This may involve the dissolving of relationships and the isolation of individuals from the community.

Delivering homeless services to Indigenous People2

It is broadly recognised that Indigenous Australians are more likely to be homeless and to live in overcrowded conditions than non-Indigenous Australians. In 2005 the Australian Institute of Health and Welfare reported that the rate of homelessness in Indigenous populations was 18 per 1,000, or

3.5 times higher than non-Indigenous Australians (AIHW, 2005). Similarly, the rate of overcrowding among Indigenous people is six times the rate of non-Indigenous people. These issues are compounded by other measures of disadvantage such as unemployment, low educational attainment, poor health and insecure housing tenure (Steering Committee for the Review of Government Service Provision (SCRGSP), 2005).

In his description of Indigenous homelessness, Paul Memmot recognises the quantitative and qualitative differences between Indigenous and non-Indigenous homelessness, and uses five categories to illustrate the experience of homelessness for Indigenous people. These are:

  1. Lack of access to any stable shelter, accommodation or housing, living in public places. This category includes those who consider these spaces as their home, currently live in the city but intend to return home and people who no longer maintain contact with home communities, and live permanently in a public space;
  2. Transient homelessness which involves people living itinerant lifestyles and moving between temporary and insecure accommodation;
  3. Spiritual forms of homelessness in which people are separated from traditional lands and from family and kinship networks (AIHW, 2005);
  4. Overcrowding; and
  5. Women and young people leaving unsafe or unstable family circumstances, including family and sexual violence, alcohol and drug abuse and poor health care.

Pathways into Homelessness

Recent urban Indigenous homelessness studies have considered in detail the factors that most commonly precede Indigenous homelessness and have included alcohol and drug abuse, mental illness, risk of violence, crime, unemployment, discrimination and issues of insecure tenure and overcrowded or substandard housing stock.

Also understood is that the migration of Indigenous people from rural to urban environments places them at risk of homelessness, as highlighted in an Adelaide based study which explains:

'This displacement takes Traditional living people away from family, friends and connections placing them in situations of compound disadvantage socially, financially, culturally and environmentally. Language barriers further compound these issues, as does their interaction with mainstream services which have little or no understanding of the lifestyle they have come from and the transition they are required to undertake' (Walker & Ireland, 2003).

Thus clearly acknowledged is that the issues of inner city Indigenous homelessness are complex and influenced by the underlying social and economic position of Indigenous Australians within our community. Therefore action to successfully address the issues of Indigenous homelessness relies on collaboration and commitment to cooperative partnerships across the service system.

As part of its commitment to achieving real change for Indigenous people experiencing homelessness, Mission Australia has developed the Indigenous Homeless Services Improvement Strategy. The strategy will be used to guide the implementation of a range of initiatives and programs specifically designed to address the needs of the Indigenous community, and deliver sustained and systemic change for those experiencing homelessness within the inner city.

Mission Australia recognises that Indigenous organisations are the preferred provider of programs and services which directly impact the Indigenous community. However, where it is appropriate to deliver services and programs through non-Indigenous agencies, the Indigenous Homeless Services

Improvement Strategy aims to ensure the provision of culturally appropriate and sensitive services. Again partnership with and support from the Indigenous community, Indigenous organisations, government and non-government agencies is fundamental for the Strategy's successful implementation.

While the Indigenous Homeless Services Improvement Strategy will build upon the range of services currently available at Mission Australia's Cooinda Centre, the strategy also delivers specific and targeted services for the Indigenous community.

Best Practice Service Delivery

Fosburg and Dennis claim that effective programs for homeless people are a combination of supportive community based services and adequate affordable permanent housing (Fosburg & Dennis, 1998). Whilst Mission Australia will not be providing permanent housing to clients in crisis at the remodelled service, supportive community based services such as employment services, dental and health care, training and education, and counselling etc. will be provided to clients.

Recommendations from research conducted by Communities Scotland into effective intermediate accommodation services include (Communities Scotland, 2001):

  • Conducting initial assessments with clients to identify accommodation and support needs;
  • Provision of appropriate support to enable people to sustain housing; and
  • Flexible intermediate accommodation that is as close as possible to 'ordinary' housing which assists people into sustainable housing.

Other service delivery recommendations include:

  • Assisting clients at the first point of contact (Newman, 2003);
  • Smaller case loads for case managers that prioritise the client's self determining needs (Fosburg & Dennis, 1998);
  • Case management is effective with a load of no more than 20 clients per case manager (Rickwood, 2004);
  • Strong connections between service providers and health and employment services (Newman, 2003);
  • Involving consumers in making decisions about programs;
  • Involving consumers in community meetings, advisory boards, interviewing consumer staff assist in empowering clients (Fosburg & Dennis, 1998); and
  • Moving away from institutionalised care.

The Mission Australia Centre's new service delivery model is unique in the delivery of crisis accommodation. While there does not appear to be any identical crisis models, other similar models are described below.

  • Urban Living Project

The Urban Living Project is currently being developed in the UK. This project plans to house 400 homeless people by creating a community with support services such as counselling, employment, training and income benefit advice on-site. The service aims to assist homeless people who are in the hostel, bed and breakfast cycle by offering stability and support services to integrate back into mainstream society. Companies are being invited to run franchised businesses in the building such as cafes or other services and provide training and job opportunities. Affordable housing will also be offered to key workers priced out of the London housing market (Crisis, 2005).

  • Pathways to Housing Program - New York

The Pathways to Housing program in New York assists people living on the streets with mental illness and addiction. Clients are moved directly from street life into permanent private market housing. Tenants pay 30% of their income towards rent whilst Pathways to Housing pays the remainder. Assertive Community Treatment (ACT) teams deliver services directly to the clients in their new homes. These services include helping clients meet basic needs, improving quality of life and social skills and increasing their employment skills. The teams are on call 24 hours a day, 7 days a week.

The client determines the type and regularity of the services although is required to meet twice a month and participate in a money management program. After rental costs are paid monthly budgets are developed by the client and coordinator with the goal being that the clients learn to manage their own money. Data collected in 2000 showed that 88% of the clients had remained housed after five years and that clients were satisfied with their housing. The client's psychological well-being had also improved which was related to having been given the choice of what activities to engage in and where to live (National Alliance to End Homelessness, 2001).

  • Common Ground, New York

The very successful Common Ground offers accommodation to homeless people with rental prices set at 30% of the client's income. The goal of Common Ground is to keep clients housed in a well maintained and safe community with staff located in the apartment buildings. A support system is provided through access to medical and mental health care, job training and job placement and one on one relationships with case workers and vocational counselors. Libraries, clinics, computer centre, art studios and building lobbies are part of the services provided to assist in integrating clients with neighbours and tenants to increase contacts and integration with the community.

  • Homes for the Homeless

Homes for the Homeless provides transitional accommodation and support services for homeless children and their families in New York. There are four buildings which house approximately 530 families. The accommodation is comprised of large rooms and apartments. Case management is provided although is voluntary. Programs include education and recreation, family health and preservation, substance abuse, treatment, independent living skills, employment training, and follow up support after placement in permanent housing. A database tracks former residents for statistical research purposes. There is no rent collection as the resident's rents are 100% subsidised. Residents are involved in the development of management policies. The properties are self managed by the organisation and have security access. The buildings are well maintained, with the community involved in the planning processes for each of the centres. The organisation believes its success is attributed to non-traditional approaches, including developing and implementing innovative programs supported by the research that it undertakes (The Enterprise Foundation, 2004).

Implementing Outcomes Measurement

Definition of outcomes

The definition of outcomes as proposed by Baulderstone and Talbot (2004, p.3), is:

'An outcome is a change or an absence of change in an identified state. Additionally, this change or absence of change is the intended or unanticipated result of an action or set of actions carried out by a program.'

For this project, outcomes related to any change identified for the client whilst accommodated at the service. These changes may not necessarily have been due to any intervention provided by the service. Change may also have occurred as a consequence of other support services or circumstances the client was involved in at the time. The change could either be intentional or could occur unintentionally, or be either positive or negative.

Therefore in this report the term 'outcomes' will be used to identify any change for the client.

Conditions for measuring outcomes

The Outcome Measurement in SAAP Funded Services Report identifies key conditions that are required for the successful implementation of outcomes within SAAP services (Baulderstone & Talbot, 2004, pp.37-43):

  • Agency commitment;
  • Outcome measurement requires integration with a clear case management process;
  • Outcome measurement be integrated with information systems which support data input and report generation;
  • That the purposes and limitations of outcome measurement is fully understood by key stakeholders; and
  • Training and support is provided during implementation.

In determining that outcomes have occurred, Baulderstone and Talbot, (2004, p.3) claim that three conditions need to be met:

  • Change or absence of change can be clearly demonstrated;
  • That the change was planned or can be shown to be the unintended result of an action; and
  • That the change did not occur as the result of some other action or occurrence outside of the program.

Four criteria that have been defined by Baulderstone and Talbot (2004, pp.6-7) as necessary in selecting outcomes tools are that they are easy to use; time and resource effective; encourage client involvement; and are culturally appropriate. The Outcome Measurement in SAAP Funded Services Report also states that no one tool suits all services and may have to be adapted to suit the needs of the clients and service (Baulderstone & Talbot, 2004, pp. vii & 19).

Research suggests that clients who have trust in the outcome measurement process have positive views of outcome measurement (Baulderstone & Talbot, 2004, Vol.17, p.33). Contributing factors to a positive view include the client's view of the workers and agency, understanding of the tools used to collect the data, as well as the purpose and rationale for measuring outcomes. Understanding the benefit in supplying the information to the various stakeholders also contributes to clients' positive views of outcome measurement (Baulderstone & Talbot, 2004, p.33). Baulderstone and Talbot (2004, p.40) indicate that the provision of outcome measurement training and support for workers assists in staff embracing the process, which in turn assists clients. Clients who participated in the outcomes measurement process experienced positive benefits from their involvement (Baulderstone & Talbot, 2004, p.33).

With the implementation of outcome measurement into routine practice, further literature was reviewed to determine effective strategies. The Diffusion of Innovation theory reinforces the need for continued support and encouragement for those staff and clients less willing or able to uptake the new approach (Rogers, 1995).

Literature relating to the implementation of evidence based medicine highlighted the most effective strategies to influence professional behaviour change (NHS Centre for Reviews and Dissemination, 1999). These are:

  • Multifaceted interventions, including reminders, audit & feedback, education, opinion leaders, use of computerised treatment planners & systems shown to be more effective than single interventions - but are more expensive;
  • Organisational, economic and community environments influence uptake;
  • Barriers to change need to be identified early and interventions designed to overcome them;
  • Any strategies need to be adequately resourced; and
  • Reinforcement of any change is required e.g. specific recommendations to individuals or educational outreach.

Such approaches can affect self-efficacy (or the belief in one's ability to perform a particular behaviour) and perceptions about the attitudes of important others to the new behaviour. These strategies need reinforcement through the stages of change.

Non-effective strategies included:

  • Merely making new methods or approaches available, or passive dissemination alone; and
  • Those that took no account of the local context.

Summary

The review of literature suggests that the change in service delivery model at the MAC is consistent with best practice and aligned with current homelessness issues. Research and experience indicates that the nature and characteristics of homelessness has changed considerably over the past thirty years. Therefore, in order to meet the needs of the changing client group, services also requires change in moving to an independent, supportive, community integrated, and flexible service model.

A number of options should be available for providing assistance to homeless people through this type of service model, including:

  • supportive community based services;
  • housing which is similar to 'ordinary' housing;
  • links with health and employment service providers;
  • assisting clients at the first point of contact;
  • life skills programs to assist homeless people sustain independent housing and become independent;
  • smaller case loads;
  • case management which prioritises client needs;
  • independent businesses offering training and job opportunities;
  • 24 hour a day, 7 day a week support;
  • follow up support once the client has exited the service;
  • services for Indigenous people need to be culturally sensitive and substantial liaison work; and
  • needs to be undertaken with the Indigenous community

The dominant learning from the review of strategies to implement outcomes measurement is that a planned and systematic approach to bringing about change is essential. In addition, it was recognised that programs undergoing major change are not good candidates for introducing new measurement concepts.

  1. This is a summary of the Aboriginal Homeless Research Report produced by Angie Pitts for the Mission Australia Centre in October 2005

Return to Top

Methodology

The methodology has been divided into three sections. The first section relates to the Personnel and related tasks and roles of those who were involved in the project. Section two incorporates the methodology used in the implementation of outcomes measurement. The third section covers the methodology for data collection and analysis in Phases 1, 2 and 3. The phases are presented separately as the methodology used in each differs considerably due to the differing circumstances of each phase.

Personnel

The Outcomes Project Team included the Operations Manager, Service Development Manager, and Service Development Project Officer.

  • Operations Manager - Kay Elson/Kerry Edgecombe
  • Service Development Manager - Anne Chamberlain
  • Service Development Project Officer - Maggie Pressnell

The main tasks of this team were to:

  • Finalise the research design and methodology for the project;
  • Organise and deliver training to case workers;
  • Collect data throughout the project stages;
  • Analyse data and write up results;
  • Provide ongoing support to staff;
  • Devise strategies to implement any changes in practice required as a result of this research and report to stakeholders;
  • Assist with the writing of any resulting publications and presentations from the project; and
  • Present the recommendations to stakeholders.

Key stakeholders in the project were:

  • Campbell House/Mission Australia Centre staff;
  • Clients;
  • Funders;
  • Mission Australia NSW/ACT Community Services; and
  • service network representatives/researchers/those with expertise in the field.

Campbell House/Mission Australia Centre staff was involved in the project in the following ways:

  • through information sessions on the data collection process;
  • training in outcomes measurement tools;
  • involvement in the selection and adaptation of the outcomes tool;
  • data collection;
  • advice and expertise in measurement methodology;
  • action research activities; and
  • through participation in implementation of any recommendations.

Implementing Outcomes Measurement

The literature review identified the most effective strategies for implementing outcomes measurement. The following is an outline of the strategies used in the project.

Agency commitment

Strong support for outcomes measurement was given by the senior management of NSW/ACT Community Services in a variety of ways. For example, an outcomes project was included in the performance objectives of all service managers across NSW/ACT.

Outcomes measurement training was provided to all services who wished it with additional training sessions subsequently held in NSW/ACT State Office, Wagga Wagga, Newcastle, Dubbo, all women's services, Child Family & Migrant Services and externally to The Parramatta Men's Homelessness Coalition and Mission Australia's South Australian Community Services.

Conference presentations on outcomes were given by NSW/ACT Community Services and Campbell House staff in 2005 and 2006. Each Division at NSW/ACT Community Services had at least one outcomes presentation by the Campbell House Project Officer in 2005/6.

Resources

An Outcomes Toolkit developed by the Service Development Unit was released on CD in 2005 and distributed to all NSW/ACT Community Services. This Toolkit contained a range of outcomes measurement tools in electronic format as well as comprehensive instructions on their use and information on the theory of outcomes measurement.

Without the resource of the Campbell House Project Officer it would have been extremely difficult, if not impossible, to introduce outcomes measurement at Campbell House or in other services across the State. Adequate resourcing is essential for getting evidence into practice

Information systems and report generation

The majority of community services across Mission Australia NSW/ACT were required to report on service outcomes in 2005 and 2006. A template and sample report was developed for the services by the Service Development Unit to assist staff in reporting on their service outcomes.

In addition, electronic tools were developed to assist services in recording individual goals and outcomes for three outcomes measurement tools, Standard Goal Scaling, BT Generic Scale and Goal Attainment Scale. These were included in the Outcomes Toolkit.

Feedback

Informal feedback was provided to staff on an ongoing basis at fortnightly meetings.

Further feedback on overall outcomes from Phase 3 will be reported to staff in the near future.

MAC staff also provided feedback on their experience of outcomes measurement to other Mission Australia NSW/ACT staff at the Community Services Forum in July 2006.

Further strategies

The following strategies are discussed further in the methodology for Phases 2 and 3 below:

  • Engaging an opinion leader, (Jo Baulderstone) to train staff on outcomes measurement;
  • Consultation with staff on appropriate outcomes measurement tools;
  • Integration of outcomes measurement as a routine part of case management practices as in Phase 3;
  • Staff training and ongoing support in Phases 2 and 3;
  • Purposes and limitations of outcomes measurement discussed in training and in ongoing support in Phases 2 and 3;
  • Customisation of outcomes measurement tools to meet client needs and fit staff practices and service delivery model; and
  • Appropriate selection of tools that are easy to use, resource and time effective and encourage client involvement.

Client satisfaction

The preferred method for collecting client satisfaction data was through client focus groups. Three focus groups were held in both Phase 2 and Phase 3. Focus groups were organised by the Service Manager and Project Officer. Two members of the research team always attended the focus groups - one to take notes and one to conduct the group. Notes were transcribed following the focus groups and key findings noted. The groups were limited to 8 participants and refreshments were provided.

Costs

The costs associated with the roll-out of outcomes and client satisfaction measurement were calculated for Phase 3 only. The general operating costs of the service and costs associated with case management were excluded as both were considered routine. The costs of developing the excel tools are excluded as these were outside the project scope.

Phase One

Audit methodology

A retrospective audit of Campbell House client records was carried out to measure outcomes of clients using the service between January 2003 and September 2004. Auditing client records was considered the most accurate and valid means of gathering past client demographics, interventions provided and outcomes. Recording information from client records was based on the golden rule, 'If it's not written down, it didn't happen'.

The sample of client records selected was based on the following criteria:

  • Clients had undertaken case management at Campbell House;
  • Clients had stayed at Campbell House for a continual period of 6 or more weeks;
  • The client had utilised Campbell House between January 2003 and September 2004; and
  • That only the client's most recent admission be used for assessing any outcomes.

The minimum six week period allowed for clients settling within the service and establishing relationships with case workers. Case workers stated that by this period of time, issues were generally identified with the client and readiness for change more likely.

The 21 month period was chosen as a reasonable period of time to provide a wide range of client records for the audit. These client records were also likely to be on-site rather than having been archived.

In September 2004 a review of a small number of the client records was undertaken to determine whether the records held data that could be used for outcomes measurement and to aid the development of an audit tool. An audit tool was specifically developed to capture information on client outcomes or any changes that may have occurred for the client whilst residing at Campbell House. This tool was piloted in October 2004 and amendments, such as the inclusion of a data collection protocol, were made at this stage. Auditors were trained in the use of the tool at this time. A copy of the final audit tool is available in Appendix 1.

Another validation study of 15 records was conducted in March 2005 to assess variation between the recording practices of auditors. The recording was found to be consistent.

Auditing began in November 2004 by the Service Development Project Officer and an intern from Macquarie University.

Data collected via the audit included:

  • The clients age;
  • Former accommodation type or homelessness grouping (adapted from Chamberlain and MacKenzie's classification) (Chamberlain & MacKenzie, 2003);
  • The number of occasions the client had accessed services at Campbell House;
  • The length of time the client had stayed at Campbell House on the last occasion;
  • Whether the client was Indigenous;
  • The client's source of income;
  • Whether the client has a disability;
  • What the client's presenting issues were at the most recent admission;
  • Changes in the client's presenting issues (i.e. outcomes);
  • What services were provided to the client;
  • Where the client exited to;
  • Who made the decision for the client to exit; and
  • Why the client exited the service.

The audits took between 10 minutes and an hour, with the majority of files audited within 35 minutes.

The results of this audit were reported to FACS through the Campbell House Outcomes Project Interim Report, internally to staff and management both at Campbell House and more broadly via internal and external discussions, publications and presentations. This served to raise awareness of the importance of recording outcomes to confirm anecdotal reports that good outcomes were being achieved. It also reinforced the need for a systematic approach to collecting and reporting on outcomes.

Assigning ratings to client outcomes

In Phase 1 there was no system in place to routinely record client outcomes. Outcomes had to be assessed by auditors from reading client files.

Any changes in circumstances or outcomes noted in client files were recorded by the auditors on the Retrospective Client File Audit Form. These data were input into an excel spreadsheet and assigned ratings according to the following scale.

Scaling Category
Very Positive
Positive
Neutral
Negative
Very Negative

This rating scale was developed by Jo Baulderstone from Flinders University, SA.

There were a number of difficulties attributing a rating to client outcomes interpreted from case notes. Primarily the difficulties related to the lack of outcomes detail in the case notes and in scaling client changes. Auditors were required to make judgements about the degree and direction of change. Tighter coding instructions and peer review assisted in this process but it was still open to error.

As a result, ratings were based on whether the client was perceived to be in a more or less independent situation after accessing Campbell House. Neutral ratings were recorded when it was impossible to assess whether the clients change had been either positive or negative or the situation had not changed for the client.

Very positive ratings for accommodation outcomes included exiting to:

  • Private accommodation;
  • Department of Housing;
  • Community Housing;
  • Shared accommodation;
  • Rehab; and
  • Psychiatric care.

Referrals to rehabilitation and psychiatric care were rated as very positive based on an appropriate referral based on client needs.

Positive ratings included exiting to:

  • Medium term accommodation;
  • Boarding house or caravan parks; and
  • Family.

These forms of accommodation are included in Chamberlain and MacKenzie's (2001, p.12) tertiary homelessness classification. Attributing a positive rating to them was based on the assumed level of independence required to maintain this type of accommodation such as paying rent, managing own meals and daily living skills. Moving back to family was attributed a positive rating. An assumption was made that exiting to family is a positive outcome although without further detail this may not always have been the case.

Neutral ratings were attributed to accommodation including friends, hospital, and other forms of emergency accommodation. This is due to the assumed temporary or transitional nature of these situations. Clients who may have moved to another crisis accommodation service were not rated as having changed their accommodation type. Two further outcomes, putting a deposit on a room and contacting accommodation providers, were attributed neutral ratings.

A negative rating was attributed in one instance where the client exited Campbell House without accommodation to go to. A very negative rating was attributed to one client who exited Campbell House to gaol.

The following table summarises the ratings for accommodation type.

Rating Category Accommodation Type
Very Positive Private accommodation, Department of Housing, Community Housing or shared accommodation, rehabilitation, psychiatric care
Positive Medium term accommodation, boarding house, caravan park, family
Neutral Other crisis accommodation, hospital, friends
Negative No accommodation
Very Negative Gaol, streets, deserted buildings

No client satisfaction data were collected for Phase 1. All audit results were transferred to an excel spreadsheet for analysis.

Phase Two

Phase 2 data were collected between December 2004 and May 2005. The Standard Goal Scaling tool was used for outcomes measurement and focus groups for client satisfaction. In Phase 2 the service delivery was based on the Phase 1 service model, but with less clients and reduced case management staff. Case loads fluctuated from between 1:9 to 1:29 during Phase 2. In total there were 160 clients accommodated at the service within this period.

Outcomes measurement tool selection and implementation

In September 2004 meetings took place with key Campbell House service staff to notify and consult with them in relation to the aims of the Outcomes Project.

A 'Measuring Outcomes' Workshop took place on 2nd November for three case work staff and Team Leader. The Workshop provided training in relation to outcomes hierarchy, terminology, program logic and the use of specific outcomes measurement tools. Other SAAP service staff also undertook the training. For some staff, this was their first exposure to outcomes measurement. The measurement tools included in the Workshop included the Rosenberg Self-Esteem Scale, Goal Attainment Scale, Standard Goal Scaling and the BT Scale Generic Outcome Measurement Instrument. Training was provided by Jo Baulderstone, Lecturer in Public Policy and Management at Flinders University.

The Outcomes Project Team and Campbell House staff worked together to select a measurement tool appropriate to the clients, staff and service model. The Standard Goal Scale was chosen. The tool was seen as flexible for clients and the service model, easy to use by clients and case workers, included client participation in setting goals, easy to incorporate into case planning and review, and appropriate for clients with varying levels of literacy. Goal Scaling was seen to be easily integrated into case management at both the case planning and review stages. The Standard Goal Scale was also identified as one of the two most appropriate tools by participants in the Outcomes Measurement Pilot Project for SAAP services (Baulderstone & Talbot, 2004, p.14).

Meetings took place with case workers and Team Leader to fine tune the tool and incorporate unique goals of the services' clients. The Campbell House Standard Goal List was extended, and goal domains reprioritised. The Standard Goal List remained a working document that could be updated to incorporate newly identified goals. The Standard Goal Lists were laminated to be shown to the client to outline steps to achieving the goal. An initial criteria and process for using the Standard Goal Scale was also established in consultation with the Campbell House staff.

Use of Standard Goal Scaling commenced within case management sessions in December 2004. Due to difficulties in the pre-Christmas period a decision was made to restart Standard Goal Scaling in January 2005.

Consultation with case workers in January 2005 determined that the criteria and process for using the Standard Goal Scaling was too case worker directed and goal setting should be more client driven. The criteria and process were updated allowing the client to determine their goals with the case worker taking a supportive role. The updated criteria and process are included as Appendix 2.

Follow-up discussions identified that the client driven approach was more successful. While some clients did not want to actively participate in the Standard Goal Scaling process, the case worker could record non-participatory clients' goals utilising the Standard Goal Scaling Tool. Case workers utilised a paper based form to record client goal setting and review. These forms were stored in the client file.

Due to service changes in Phase 2 and the irregularity of case management sessions, the collection and recording of outcomes data fluctuated. It was suggested to case workers at this time that ideally case management and goal review should occur weekly. This highlighted that a stable environment and staffing is necessary in conducting outcomes measurement. This is also supported by evidence (The Urban Institute, 2003).

Data collection

Data collection commenced in September 2005 taking one project team member seven days. Client files had to be reviewed to collect essential demographic and situational data as well as capture outcomes from paper based outcomes sheets kept in the client files. All data was input directly from client files into an excel spread sheet that was then used to analyse the data. The outcomes ratings provided by workers or client and worker at the time of case management were input as shown in Figure 1. To ensure confidentiality, client names were scrambled.

Client Number 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 1.11
HILUR 1                                         5                 5        
BTEUT         1                                                                        
SETIN                                                                 1                
MOSAT                 2 2                                         1        
PEVEA                                                                                        
FATEL                                                                         3        
DANAW                                                                                        
AARLT                                                                         1        
CODAM                                                                                        
METCC                                                                         2        
GHARA                                                                                        
TITHU                                                                         5        
FEAAU 3 2 2                                                 1        
HASUC                                                                                        
MANUL                                                                                        

Statistical analysis

The statistical analysis for Phase 2 was undertaken in Microsoft Excel. Counts of clients and their demographic and profile data were used to produce descriptive statistics and graphs. These data were compared with data from Phase 1 and 3.

Analysis of outcomes data included:

  • A count of all clients who selected goals by goal domain;
  • Calculation of the average number of goals selected;
  • Count clients' minimum and maximum number of goals;
  • A count of all goals by goal domain; and
  • The percentage of goals in each rating category by goal domain.

These data were compared with data from Phase 3. To determine whether there was a significant difference between Phases 2 and 3, a comparative analysis using Chi Square tests was conducted on both the number of goals selected for each goal domain with ratings of 5, 4 and 2 (Fully Achieved, Partly Achieved and Not Achieved). Goals with a rating of 3 and 1could not always be determined as positive or negative so were excluded from this analysis.

Phase Three

Overview

In July 2005 the MAC was opened with the provision of the new service delivery model. The period for collection of Phase 3 outcomes and satisfaction was July 2005 to May 2006. This period includes clients that transitioned from the temporary service to MAC. Clients had to have exited by 19 May to be included in Phase 3. One hundred and seventy-nine clients met this criteria, 2 files were not located, leaving 177 client records for outcomes analysis.

Outcomes measurement implementation

In the first few weeks of operation the service had a lower occupancy. This allowed time for the first of the outcomes training with four new and one remaining case workers with new titles of Unit Coordinator. Initial training involved outcomes terminology and hierarchy, program logic and key factors for successful outcomes measurement. Specific training on the use of the Standard Goal Scaling tool was provided at this time for its immediate use within case management practice.

Outcome measurement was officially implemented within case management practice as of August 2005 although in reality, routine incorporation into practice varied between Unit Coordinators.

Initial regular fortnightly meetings between the Unit Coordinators and Outcomes Project Officer involved customising the Standard Goal List to better reflect the service delivery model. These meetings gained feedback from staff, provided support and monitored the use of the outcomes tool. Team meetings were replaced by fortnightly individual meetings with Unit Coordinators, as well as weekly telephone and email support to review their individual use of the tools.

The Standard Goal List was reorganised with the goal domains re-prioritised, spiritual health was added as a goal domain, gambling intervention became an independent goal domain, and goals relating to programs and services offered at MAC were included. The Standard Goal List continues as a working document to incorporate newly identified goals.

Changes to practices included discontinuation of ratings 1 (no longer relevant/no longer pursued) and 3 (the same/maintenance) in December 2005. These two ratings were open to misinterpretation i.e. rating 1 could be used when the client had achieved a goal; or rating 3 could be used to indicate they had either maintained or achieved their goal e.g. maintaining abstinence from alcohol could be both achievement and maintenance. It was agreed by Unit Coordinators and the Outcomes Project Officer that only the ratings of 2 (not achieved), 4 (partly achieved), and 5 (fully achieved) would be used.

To aid all Mission Australia NSW/ACT Community Services staff in recording and measuring individual client outcomes, the Mission Australia Outcomes Measurement Toolkit and electronic tools were developed. The Outcomes Measurement Toolkit is included as Appendix 3. The electronic tools replaced the paper version for recording client goals and review sessions. An electronic version of the Standard Goal Proforma/Excel Aggregation Sheet is included as Appendix 4. Printable graphs provided staff and clients with a visual aid of the extent to which goals had or had not been met. Sample graphs are shown in Appendix 5 and Appendix 6.

Data collection

Data collection commenced in May 2006 taking approximately 13 days to complete. Data took longer to collect than in Phase 2 due to an increased number of clients involved in outcomes measurement and increased case note content. As in Phase 2, data was input directly from client files into an excel spread sheet that was then used to analyse the data. Client files still had to be reviewed to collect essential demographic and situational data as well as capture outcomes from paper based outcomes sheets kept in the client files.

Ratings attributed to outcomes were provided by workers or client and worker at the time of case management and recorded in client files. Project Team members then input outcomes data (the scores for each goal in each goal domain) into an excel spreadsheet as shown in Figure 1.

Statistical analysis

The statistical analysis for Phase 3 was the same as that undertaken for Phase 2.

Return to Top

Results

The results are presented in three sections. The first section reports on the effectiveness of implementing outcomes measurement and associated costs, the second section reports on client profiles between the three phases, and the third section reports on outcomes and client satisfaction.

Where appropriate case studies have been utilised to highlight results.

Table 1 shows the timeframes, sampling and response rates for each of the three project phases.

Table 1 -Timeframes, sampling and response rates
Phase Sampling and response rates
Phase 1 - Campbell House January 2003 to September 2004 (21 months, clients who resided at the service for a minimum of 6 weeks were eligible for client record audit). N = 197 client files audited out of the 882 clients who used the service during that time
Phase 2 - Campbell House -Transitional Service at St. Vincent's Hospital October 2004 to June 2005 (9 months), N = 160 with 73 clients choosing at least one goal
Phase 3 - Mission Australia Centre July 2005 to May 19th 2006 (11 months and clients needed to have left the service by May 19th 2006) N = 177 clients with 130 clients choosing at least one goal

Return to Top

Implementing outcomes measurement

Literature reviewed on implementing outcomes measurement stated that programs undergoing major change are not good candidates for introducing new measurement concepts (The Urban Institute, 2003). Our experience of implementing outcomes in Phase 2 confirmed this. Phase 2 was a period of great change where worker case loads were high, there was client dissatisfaction, workers had difficultly providing regular case management sessions, and there was resistance to the new concept of outcomes measurement. During this time the complement of case workers changed from three to one.

Rolling out outcomes measurement was significantly less complicated in Phase 3. Goal identification in Phase 3 was predominantly client driven with encouragement, support and review regularly taking place within case management. Case management, where possible, tended to be provided weekly. The ability to review client's ongoing goal achievement was further aided by a lower worker:client ratio of 1:8. Staff was open to new processes and practices within their new environment, and outcomes measurement was fully integrated as standard case management practice.

Staff stated that the electronic excel tools provided a quick overview of where the client was at, could be given to clients as a reminder of the direction and tasks needed to achieve their goals, and clearly identified both the nominated goals and the steps to achieving larger goals. Graphs proved a useful tool for indicating to clients how much change had occurred and the achievements clients had made. Clients reported they 'loved' the graphs, and had used them as evidence of achievement and reliability to show referral agencies and other service providers.

Staff reported that the outcomes measurement tool improved their case management skills and aided them in reflecting on their practices. Setting and reviewing client goals became a key focus of case management and was assisted by the visual layout of client goals on the spreadsheet. Phase 3 case notes showed significant improvement in comparison to Phases 1 and 2 in terms of the quality of documentation, the depth of recording, readability of case notes and a focus on support and change. A comment from a Phase 3 staff member below highlights the effectiveness of the implementation of outcomes measurement.

'Initially I thought Standard Goal Scaling and outcomes measurement was confusing, painful and was going to be a waste of time. What I came to find was that clients became more goal focused and motivated, that it stimulated conversation between myself and clients and I became more focused on client's strengths. Clients enjoy seeing the progress they are making as we track the changes and they see the evidence of their achievements. Measuring outcomes has added structure to case management sessions, has helped record the changes and shows an instant summary of what the client is working on. I feel more motivated and I reflect more on my skills and practices as a whole as well as with individual clients and issues. We either give clients diaries so they can record their goals and tasks in them, or give them a weekly plan of their goals and then review them the next time we meet. The recording does take a bit more time, but that's getting better.' (Staff member, Mission Australia Centre, July 2006)

To effectively implement outcomes measurement this project successfully utilised all the following change management strategies:

  • Awareness raising internally and externally about the need for change and the benefits e.g. having evidenced to show that services achieve good outcomes;
  • Audit and feedback on what staff are and are not doing well;
  • Emphasising the focus on clients by refining tools to meet client needs and expectations and involving clients as much as possible in the setting of goals;
  • Ensuring that the information on outcomes and evidence of effectiveness is supported by consensus from staff and is not seen as taking over, but supporting, their practice;
  • Making contextual changes to the tools and system;
  • Education and training both group and one on one;
  • Support from senior management;
  • Use of opinion leaders; and
  • Ongoing support.

These were all supported by the literature review.

Costs associated with implementing outcomes measurement

The costs of implementing outcomes measurement for the 130 clients who participated in goal setting for Phase 3 are calculated below. Staff estimated that each client who participated in goal setting took up to an additional half hour of case worker time each week. Each case worker also received one on one consultation for 15 minutes each fortnight.

Calculated for 130 clients for 42 weeks of Phase 3
Outcomes Measurement Hours Cost per hour Total cost
Staff
0.5 hrs per client (8) per week to record outcomes per staff member = .5x8x42x5 0.5 $21* $840
Training - 5 staff for one off 3 hour training 15 $21 $315
Customising the goal list - 5 staff for two one hour meetings 10 $21 $210
Ongoing support 15 minutes per fortnight with Project Officer = .25x5x21 26.25 $21 $551
Project Officer
Training delivery and receipt 5 $25** $125
Ongoing support 15 minutes per fortnight for each staff member = .25x5x21 26.25 $25 $631
Preparation for staff training 6 $25 $150
Customising the goal list 2 $25 $50
Initial consultation with staff 1 $25 $25
Travel time 21 $25 $525
Collection of data 98 $25 $2450
Analysis of data 75 $25 $1875
Change Management
Opinion leader, travel, accommodation and fees 8   $1936
Total costs     $9683

* CSW 3 hourly rate ** Project Officer hourly rate

Return to Top

Client profiles

A comparison of the client demographics and admission/exit status for each phase was conducted using the following variables.

  • Age groups;
  • Presenting issues;
  • Previous accommodation;
  • Income sources;
  • Disability status;
  • Indigenous status;
  • Length of stay;
  • Frequency of admission;
  • Services received;
  • Exit points;
  • Exit reason; and
  • Exit reason by decision maker.

The results of the analysis are presented below.

Age

Figure 2 shows that the most common age group in each of the three phases was 35-44 years. In each Phase, 71%, 76% and 82% of clients respectively were aged less than 45 years. Clients least likely to use the service were aged 55+. Age groupings were similar across the three phases and is skewed to the right.

Figure 2 - Comparison of age groups - Phase 1, 2 and 3


Figure 2 - Comparison of age groups - Phase 1, 2 and 3

Presenting issues

Figure 3 shows that the most common presenting issues for each phase were drug & alcohol use, financial problems (financial difficulty, loss of income), history of homelessness, relationship breakdown (family breakdown/relationship breakdown) and psychiatric illness.

Figure 3- Clients Presenting Issues - Phase 1, 2 and 3


Figure 3- Clients Presenting Issues - Phase 1, 2 and 3

Previous Accommodation

Primary and Secondary Homelessness combined were the top two previous accommodation categories in all three phases (70%, 91% and 90% respectively), followed by rental accommodation. Primary homelessness is characterised by people living on the streets, in deserted buildings, improvised dwellings, in parks etc. Secondary homelessness is characterised by people moving between various forms of temporary shelter. The clients in all phases were therefore definitely in need of crisis accommodation.

Income Sources

Income sources for all phases were predominantly Newstart Allowance and Disability Support Pension (DSP) with clients with these two income sources making up approximately 80% of clients in all phases. The large majority of clients entering the service therefore had some form of income.

Disability Status

Of those clients with a disability recorded (60 or 30% for Phase 1, 44 or 27% for Phase 2, and 42 or 24% for Phase 3) most were recorded as Mental Illness. In Phase 2 approximately 28% of clients had Disability Status as 'Not Stated' with this category approximately 55% in Phase 3. The higher percentage of unknown disabilities in Phase 3 is due to non-recording of this variable in client files.

Figure 4 - Disability Status - Phase 1, 2, and 3


Figure 4 - Disability Status - Phase 1, 2, and 3

Indigenous Status

In each Phase the majority of clients were recorded as not being Indigenous at 91% in Phase 1, 86% in Phase 2, and 72% in Phase 3. However, in Phase 3 a much larger proportion of clients were recorded as having unknown Indigenous status than in any other Phase.

Length of Stay

Figure 5 shows the percentage of clients who were accommodated over the three phases for six weeks or less, and then on a weekly basis until 12 plus weeks. Although all clients in Phase 1 who stayed six weeks or less were not included in the audit, they were counted in the length of stay to compare across the 3 phases, as shown in Figure 5. Most clients over the three phases stayed six weeks or less although Phase 3 shows a lower percentage of clients in this group.

Figure 5 - Length of Stay in Weeks - Phase 1, 2 and 3


Figure 5 - Length of Stay in Weeks - Phase 1, 2 and 3

Number of admissions

Most clients attended the service once only (60%, 72% and 75% respectively) in all three Phases. Although the number of client admissions was counted in each phase, outcomes were recorded for clients' last admission only.

Services received by clients

Across all three phases, it was assumed that most clients received meals, shower and laundry as client files did not record this detail. Services received by clients over the three phases are shown in Table 2.

A higher proportion of clients appeared to received Emotional Support, Referral, Advice/information, Advocacy/liaison and Drug & Alcohol support in Phase 3 than in the other phases. This could be due to more engagement with staff in Phase 3, increased awareness of client needs, a greater ability to meet with clients due to a lower client:staff ratio, greater access/availability of services and/or better recording.

Apart from Transport, Interpreter Services, Housing and Identification Assistance, clients in Phase 3 appeared to receive more services than the other two phases. From the audit conducted in Phase 1, it was sometimes difficult to assess what services had been received, so the true level of services received in this phase may be misrepresented.


Table 2 - Services Received - Phase 1, 2 and 3
Services Received Phase 1 Phase 2 hase 3
No. of clients % of clients No. of clients % of clients No. of clients % of clients
Emotional Support 30 15% 3 2% 127 72%
Referral 80 41% 23 14% 106 60%
Advice/Information 103 52% 33 21% 123 70%
Use of office facilities 15 8% 4 3% 18 10%
Advocacy/Liaison 89 45% 29 18% 95 54%
ID assistance - - 3 2% - -
Housing assistance 92 47% 2 1% 46 26%
Govt Allowance assistance 25 13% 3 2% 30 17%
Training assistance - - 1 1% 25 14%
Employment assistance 13 7% 1 1% 33 19%
Financial assistance 5 3% 4 3% 11 6%
Drug/alcohol support 21 11% 3 2% 47 27%
Cultural support - - - - 2 1%
Legal issues assistance 8 4% - - 19 11%
Court Support - - - - 4 2%
Belongings storage 9 5% - - 11 6%
Recreation 5 3% - - 9 5%
Interpreter services 1 1% - - - -
Transport 19 10% 4 3% 10 6%
Living skills 5 3% - - 36 20%
Personal development - - - - 18 10%
Cooking lessons N/A - N/A - 45 25%
Cleaning lessons N/A - N/A - 20 11%
PSP Program - - - - 10 6%
Catalyst Program - - - - 8 5%
Indigenous Group N/A - N/A - - -
Oral Health Care 18 9% - - 27 15%
GP/Medical Services 73 37% 1 3% 119 67%
Anxiety Management N/A - N/A - 11 6%
Gambling assistance 4 2% - - 17 10%
Immigration assistance 1 1% - - 1 1%
Computer skills/lessons N/A - N/A - 16 9%
Behaviour/anger management 2 1% - - 15 8%
Child related support 3 2% - - 4 2%
Interview preparation - - - - - -
Self advocacy skills 1 1% - - 2 1%
Budgeting 6 3% - - 13 7%
Other - - - - 1 1%
Total No. of services by phase 628   114   1783  
Total clients 197   160   177  

Client exits

The majority of exit points were unknown in all Phases. This was mainly due to many clients leaving without notice or involuntarily because of poor behaviour e.g. Drug or alcohol use. In Phase 1, 89/197 (45%) clients exited to 'Unknown', in Phase 2 113/160 (71%) exited to 'Unknown' and in Phase 3 this figure was 106/177 (60%). Encouragingly, the percentage exiting to 'Unknown' was less in Phase 3 than in Phase 2. Most exit decisions were made by clients in all three Phases.

For those clients for whom the exit point was known, most were recorded as exiting to rental accommodation (13%, 4%, and 11% respectively), crisis/short term accommodation (11%, 3%, 7% respectively), or medium/long term accommodation (7%, 1%, 7% respectively) in all three phases.

Return to Top

Outcomes and client satisfaction

Phase 1 - Outcomes

Table 3 shows that the most commonly recorded outcomes for Phase 1 were Accommodation, Employment and Health. Seventeen clients had outcomes recorded in the Daily Living Activities, Education and Training, Financial Assistance and Gambling areas.

The proportion of clients who had Very Positive or Positive outcomes for Accommodation, Employment, and Health was very high.

Despite the use of a standardised auditing tool, there were difficulties in assigning ratings using limited case notes alone. For these reasons Phase 1 outcomes data was not compared with Phases 2 and 3.


Table 3 - Change in Outcomes by Domain - Phase 1
  Rating (%)
Goal Domain No. of client outcomes by Domain % of outcomes by Domain Very positive Positive Neutral Negative Very negative
Accommodation 74 37.6 54% 30% 14% 1% 1%
Employment 27 13.7 67% 22%   11%  
Health 26 13.2 69% 19% 4% 4% 4%
Drug and Alcohol 15 7.6 7% 60% 7% 20% 7%
Income Support 14 7.1 43% 29% 29%    
Family Relationships 13 6.6 38% 46% 15%    
Legal and Identification 11 5.6 55% 27% 17%    
Daily Living Activities 7 3.6   71%   14% 14%
Education and Training 5 2.5 80% 20%      
Financial              
Assistance 3 1.5 100%        
Gambling 2 1.0   100%      
  197            

Return to Top

Phase 2 - Outcomes

In Phase 2, 73 of the 160 clients were involved in goal setting. These 73 clients selected 321 goals in total. As in Phase 1, the most common outcomes were in the areas of Accommodation, Employment, Health and Drug and Alcohol. No Domestic Violence goals were selected. Goals in Phase 2 most likely to be Fully or Partly Achieved were Legal Issues, Daily Living, Financial Matters and Employment. Only very small numbers of clients selected Daily Living, Legal and Financial Matters goals.

Table 4 shows the results of the:

  • Number of clients who selected goals in each goal domain (count of all);
  • Average number of goals selected by goal domain (count divided by number of clients);
  • Minimum and maximum number of goals selected by clients by domain;
  • Total number of goals selected in each goal domain (count of all goals by goal domain); and
  • Percentage of ratings attributed to each goal by goal domain (percentage of 5,4,3,2,1 ratings by goal domain).

Table 4- Change in Outcomes by Domain - Phase 2
  Achievement as Percentage of total goals
Selected in domain
Goal Domain No. of clients selectin g goals within domain Average no of goals selecte d by client in domain Min & Max no. of goals selecte d in domain % of total goals selecte d Total number of goals selecte d in domain Fully Achieved (5) Partly Achieved (4) Unchange d (3) Not Achieved (2) No longer relevant (1)
Accommodation 57 2.4 1-8 goals 43.3% 139 38% 4% 4% 12% 42%
Drug/Alcohol/Gamblin g Support & Intervention 24 1.5 1-4 goals 11.8% 38 39% 0% 0% 22% 39%
Financial Matters 9 1.3 1-2 goals 3.7% 12 58% 0% 17% 8% 17%
Legal Issues 8 1.6 1-4 goals 4.0% 13 46% 23% 0% 15% 15%
Identification 3 1 1 goals .9% 3 33% 0% 0% 0% 67%
Health 29 1.6 1-4 goals 14.6% 47 55% 2% 15% 2% 26%
Relationships 6 1 1 goal 1.5% 5 0% 20% 40% 20% 20%
Family 9 1.4 1-2 goals 4.0% 13 31% 8% 8% 15% 38%
Employment 20 1.8 1-4 goals 11.5% 37 27% 3% 5% 14% 51%
Education and Training 7 1.4 1-2 goals 3.1% 10 10% 0% 10% 20% 60%
Domestic Violence 0 0 0 0% 0 0% 0% 0% 0% 0%
Daily Living 2 1.5 1-2 goals .9% 3 67% 0% 0% 33% 0%
Personal Skills and Attributes 1 1 1 goal .3% 1 0% 0% 0% 0% 100%

Change by goal domain was analysed and is presented in Figure 6. For this analysis all goals within a domain with a rating of Fully or Partly Achieved (rating of 5 or 4) were classified as Change. Goals within a domain with a rating of Not Achieved (rating 2) were classified as No Change. Goals with a rating of 3 and 1 could not always be determined as positive or negative so were excluded from this analysis.3 These categories can be estimated by the residual percentage in each goal domain. i.e. each column will not add up to 100%.

As indicated in the figure below, the most common goal domains selected were Accommodation and Health. Those goal domains showing the most positive percentage change were Legal, Financial, and Health. Activities of Daily Living showed a high percentage of change although only 3 goals were selected. Fifty-eight (42%) goals selected in the Accommodation domain were Fully or Partly Achieved. The graphs shows the number of goals on the apex of each column.

Figure 6 - Change in goals by Goal Domain - Phase 2


Figure 6 - Change in goals by Goal Domain - Phase 2

At 22 years of age, Tim came to Campbell House with a long history of alcohol and heroin abuse, having used heroin since 13 years old. His parents had an AVO against him which he had breached twice, and he had been in and out of prison for the past six years. Tim could not read or write properly. While at Campbell House Tim accessed a Drug and Alcohol Program, maintained probation and parole appointments, sought legal advice, maintained sobriety, moved beyond his current issues and looked ahead to a bright future. Tim's self esteem improved and he was accepted for a place at South Sydney TAFE college to study English.

Case Study, Campbell House temporary location, May 2005

Return to Top

Phase 2 - Client Satisfaction

Three focus groups were conducted between March and June 2005 at the transitional service. Six to eight clients participated in each of the focus groups. Those clients' length of stay ranged from one week to three months. Focus group discussions were collated and summarized to capture the main themes of discussion and important points raised by the clients within the groups.

Accommodation: While clients valued the accommodation and understood that it was a temporary service, concerns were raised regarding its cleanliness and pest control. Clients reported the facility lacked privacy in the dormitory accommodation and security for storage of personal belongings. Clients suggested more comfortable furniture and additional recreational facilities as improvements to the accommodation.

The menu and food was criticized, being described as monotonous and limited in variety. At this time food was brought in daily due to the lack of adequate kitchen facilities. Clients reported the laundry service lacked flexibility, wanting to manage their personal laundry that would provide valuable daily living skills and build independence.

Staff and Case Management: A majority of clients reported that staff were approachable, respectful, empathetic and easy to talk to. Clients emphasized that these qualities were highly valued and compared positively to other services. While some clients reported that they had had contact with staff, others spoke of very little contact or support and requested that more support, information and services be available on site. Those who had been involved in case management reported that staff had a good knowledge of the service system and were strongly supportive of people motivated to achieve change within their lives. Clients considered the accessibility of staff and individual case management important in their capacity to make positive changes.

Rules and Regulations: While the clients generally acknowledged that service rules were important and designed to support and protect the group as a whole, discussion highlighted that clients considered the rules too restrictive (eg. not allowing others into rooms) while others were not restrictive enough (eg. T.V use at night).

Summary: While clients generally described use of the service as a positive experience, they also discussed the shame and stigma associated with use of crisis accommodation. Clients reflected on the monotony and boredom associated with this temporary service and emphasized the value of planned activity and support to access training, employment and recreation.

Return to Top

Phase 3 - Outcomes

As outcomes measurement was fully implemented within case management practice in this phase, all clients participated. Staff stated that a small number of clients were interested and were actively involved in the rating system. Unit Coordinators consulted with clients on whether goals had been fully achieved, partly achieved or not achieved and rated the client goals accordingly.

The number of clients selecting at least one goal in Phase 3 was 130/177 (73.4%). The outcomes results in Table 6 refer to these 130 clients only. In total there were 1,563 these 130 goals selected by clients.

Table 6 shows the most commonly selected goal domains in Phase 3 were Activities of Daily Living, Accommodation, Drug & Alcohol Support, Employment, Mental Health, Financial Matters, Physical Health and Education and Training. This compares with the most commonly selected goals in Phase 2 which were Accommodation, Health, Drug & Alcohol, and Employment.

Those domains where goals were most likely to be Fully or Partially Achieved were Activities of Daily Living, Physical Health, Accommodation, Gambling Support, Employment, Education and Training, D&A Support, Mental Health, and Relationships. Spiritual Health and Legal Issues domains were less likely to be selected but those clients that did select them were extremely successful in achieving them. Selection of goals and achievement were quite well aligned meaning that clients were generally selecting goals that they were very likely to achieve. Those domains where goals were least likely achieved were Identification, Family and Financial Matters. No clients selected goals in the Domestic Violence domain.

John at 35 years of age, had had a long history of imprisonment due to his drug and alcohol addiction, was disconnected from family and had been long term unemployed. When John came to MAC he received daily intensive case management which assisted him in identifying his issues and setting goals for his future. He was involved in weekly unit meetings that assisted in his integration within a larger group. John was encouraged to write to his family and let them know that he was working things through. While at MAC John utilised a range of services including the doctor, chiropractor, dentist and lawyer as well as attending the Anger Management Course and AA and enrolled in the Personal Support Program through Centrelink to find employment. John maintained abstinence from drugs and alcohol, reestablished ongoing contact with his family, found casual employment which led to full time permanent employment due to his outstanding work performance and secured independent affordable accommodation at MAC. Probation and Parole Services called to congratulate the MAC team for the outstanding service and support which has resulted in such positive and major outcomes for John.

Case Study, MAC, July 2006

Further analysis of exit points and goal selection indicated that those who selected goals were significantly more likely to have known exit points as shown Table 5. The X² result for this analysis is 6.98 p > .01.


Table 5 - Phase 3 exit points by goals chosen
Goals Chosen Exit Point Known Exit Point Not Known Total
Yes 59 71 130
No 11 36 47
Total 70 107 177
Table 6 - Change in Outcomes by Domain - Phase 3
Goal Domain No. of clients selecting goals by Domain Average No. of goals selected by Domain Min/Max of number of goals selected by domain % of all goals selected by domain Total No. goals selected by Domain Fully Achieved (5) Partly Achieved (4) The same (3) Not Achieved (2) No longer relevant (1)
Accommodation 111 2.6 1-7 19% 297 56% 24% 5% 13% 2%
Drug/Alcohol Support and Intervention 82 2.2 1-8 11% 181 46% 28% 10% 14% 2%
Gambling Support and Intervention 42 1.5 1-4 4% 63 62% 17% 6% 14% 0%
Financial Matters 67 1.5 1-4 6% 101 50% 13% 12% 25% 0%
Legal Issues 25 2 1-5 3% 50 70% 28% 2% 0% 0%
Identification 19 2.4 1-5 2% 30 23% 13% 20% 40% 3%
Physical Health 43 2.1 1-4 6% 92 70% 12% 9% 5% 4%
Mental Health 41 2.6 1-8 7% 109 46% 27% 13% 13% 2%
Spiritual/Cultural Health 2 1 1 .1% 2 50% 50% 0% 0% 0%
Relationships 8 1.2 1-2 .6% 10 10% 60% 20% 10% 0%
Family 19 1.1 1-2 1% 21 19% 33% 19% 24% 5%
Employment 73 2.3 1-9 11% 171 55% 20% 2% 21% 2%
Education, Training and MAC Programs 47 2 1-7 6% 98 35% 40% 11% 9% 5%
Domestic Violence 0 0 0 0% 0 0% 0% 0% 0% 0%
Activities of Daily Living 73 4.4 1-9 21% 322 75% 21% 1% 2% 1%
Personal Skills and Attributes 8 1.3 1-3 1% 11 55% 0% 45% 0% 0%
General 3 1.6 1-3 .3% 5 0% 20% 60% 20% 0%

Change by goal domain was analysed and is presented in Figure 7. For this analysis all goals within a domain with a rating of Fully or Partly Achieved (rating of 5 or 4) were classified as Change. Goals within a domain with a rating of Not Achieved (rating 2) were classified as No Change. Goals rated as 3 and 1 were excluded from this analysis.4 The proportion of goals in this category can be estimated by the residual percentage in each goal domain i.e. each goal domain will not add up to 100%.

Those goal domains where goals were most likely to have a high percentage Change were Legal, Activities of Daily Living, Physical Health, and Accommodation. Although Spiritual Health showed a high percentage of Change, only 2 clients chose goals in this domain. The graph shows the number of goals on the apex of each column.

Figure 7 - Change in goals by Goal Domain - Phase 3


Figure 7 - Change in goals by Goal Domain - Phase 3

Return to Top

Phase 3 - Client Satisfaction

Three focus groups were conducted between February and June 2006. Five to eight clients participated in each focus group. Participants' length of stay ranged from approximately one week to four and half months. The discussion from each focus group was collated and summarized to capture the main themes and important points shared by the clients.

Accommodation and Services: Clients reported that the Mission Australia Centre provided clean, comfortable and safe accommodation. Clients reported that the individual bedrooms provided offered privacy and security which was greatly appreciated by men using the service. Clients also believed the accommodation service had been enhanced with on-site access to the wide range of services and programs available at the Cooinda Centre. The development of a 'one stop shop' approach to care and service delivery was highly valued by clients, especially those transferring to the service from institutional care. Clients reported that the structured environment with a focus on independent living was valuable and assisted people to achieve their goals.

A number of clients suggested that the '3 month stay' limit currently in place be extended to four months, as this additional time would enable clients to consolidate positive changes made within their lives.

While clients generally described the food as 'good', they also reported it was sometimes limited in variety especially at lunch time. Significantly, men discussed their enjoyment and participation in the cooking classes which had enabled them to gain valuable daily living skills.

A number of clients commented on their participation in a range of courses and activities including a computer course, an Australian literature course and art classes. Other clients discussed their use of the dentist and optometry services and reported that the ability to access these services with good outcomes had had a positive impact on their self esteem. Some clients described that they had 'grown a lot' since coming to the service and that the time spent at the Mission Australia Centre had been a 'turning point' in their lives.

Staff and Case Management: Clients overwhelmingly described staff within the centre as helpful, nurturing, understanding and tolerant. Clients reported to highly value the case management and support role fulfilled by Unit Coordinators, especially the staff's knowledge of services and their capacity to access and provide real assistance for clients to complete their studies and training.

Clients reported that Unit Coordinators had provided them with regular support and valued the specific and individual support to establish goals, monitor progress and achieve positive outcomes. Clients reported that the focus on goals assisted their motivation; they had achieved many goals in comparison to their stay at other services; and enjoyed seeing their goals and achievements on graphs. While some clients described a sense of obligation to participate in the programs and services available, it was also recognized that 'clients got out of the service what they put in'.

Rules and Regulations: Clients acknowledged the importance of the services' rules intended to protect all clients and assist people to achieve change within their lives. While some clients reported that the rules concerning the curfew were too restrictive especially on weekends, others felt that the curfew was appropriate.

Most clients reported valuing the independent living culture within the units and described the expectation that everyone contributed to the cleanliness and smooth functioning of the units.

Summary: Within the final discussion of each focus group, clients emphasized the value and importance of providing a service that strengthened an individual's skill and capacity to successfully participate in community life and the workforce. Clients emphasized that this outcome was strongly supported by the Mission Australia Centre and its focus on independence within the accommodation, the establishment of individual goals, the range of programs and services available and the supportive case management provided.

Finally the clients spoke positively of the 'new benchmark' that had been achieved by Mission Australia House and urged this to be maintained and extended across other services.

Return to Top

Outcomes - Comparative Results Phase 2 & 3

A comparative analysis between Phases 2 & 3 outcomes results was undertaken. Table 7 shows outcomes by goal domain and phase and the significance of any change. In analysing these results the number of Partly and Fully Achieved goals were added together, and compared with the Not Achieved goals for each goal domain in each phase.

In statistical terms Accommodation, Drug, Alcohol and Gambling, Legal Issues, Education and Training and Daily Living goal domains show a significant increase in positive achievement of goals.

The results of the X² tests in Table 7 show that:

  • The achievement of Accommodation goals is significantly different between Phases 2 and 3 with 42% of goals Partly or Fully Achieved in Phase 2 versus 75% in Phase 3; and
  • The achievement of Drug, Alcohol and Gambling goals is significantly different between Phases 2 and 3 with 39% of goals Partly or Fully Achieved in Phase 2 versus 75% in Phase 3.

Although some of the results of the X² tests were significant, it should be noted that the number of goals in the some of the cells for Phase 2 were very small and therefore these results should be viewed with caution. The goal domains are:

  • Legal Issues;
  • Education and Training;
  • Activities of Daily Living;
  • Employment; and
  • Health.

Table 7 - Comparative outcomes results - Phase 2 and 3
Goal Domain

Phase 2

Phase 3

 
No of Goals No of Goals Fully or Partly Achieved Percentag e of Goals Fully or Partly Achieved No of Goals Not Achieved Percentag e of Goals Not Achieved No of Goals No of Goals Fully or Partly Achieved Percentag e of Goals Fully or Partly Achieved No of Goals Not Achieved Percentag e of Goals Not Achieved Probability
Accommodation 139 58 42% 17 12% 297 238 80% 31 13% X²= 6.064634 and p < .05
Drug, Alcohol and Gambling* 38 15 39% 8 22% 244 184 75% 34 14% X²= 5.322069 and p < .05
Financial Matters 12 7 58% 1 8% 101 64 63% 25 25% X²= 0.909298 and p > .05
Legal Issues 13 9 69% 2 15% 50 49 98% 0 0% X²= 9.216301 and p < .01
Identification 3 1 33% 0 0% 30 11 36% 12 40% X²= 1.043478 and p > .05
Health** 47 27 57% 1 2% 203 157 77% 19 9% X²= 1.425646 and p > 0.5
Relationships 5 1 20% 1 20% 10 7 70% 1 1% X²= 1.40625 and p > .05
Family Relationships 13 5 39% 2 15% 21 11 52% 5 24% X²= 0.016502 and p > 0.5
Employment 37 11 30% 5 14% 171 128 75% 36 21% X²= 0.716621 and p > .05
Education and Training 10 1 10% 2 20% 98 73 75% 9 9% X²= 7.967105 and p < .01
Domestic Violence 0 0 0% 0 0% 0 0 0% 0 0%  
Daily Living 3 2 67% 1 33% 322 309 86% 6 2% X²= 13.63483 and p < .001
Personal Skills and Attributes 1 0 0% 0 0% 11 6 55% 0 0%  
General 0 0 0% 0 0% 5 1 20% 0 20%  

Return to Top

Discussion

Demographics, Admission and Exit Status and Services Received

The percentage of clients in each age grouping in all three phases was similar with clients predominantly aged less than 44 years of age. Between January 2003 and May 2006, clients least likely to use the service were aged 55+. This may be a reflection of the general age groupings of the homeless male population.

Presenting issues are similar across all three phases with Drug and Alcohol, History of Homelessness, Financial Issues, Psychiatric Illness and Family/Relationship Breakdown predominant. Phase 1 showed a higher proportion of History of Homelessness and Financial Issues while Phase 3 shows a higher proportion of Drug and Alcohol Use. Knowing the predominant presenting issues can be used to inform the service of the interventions most needed by clients.

Status prior to entering the service indicates clients were mostly classified as primary or secondary homeless. The length of time clients had been homeless or the number of homelessness episodes was not recorded, although clients' presenting issues indicate that some clients had a history of homelessness (45%, 27%, 19% respectively) in all three phases. Generally clients using the service in all three phases had significant homelessness issues, either living on the streets or in unstable accommodation.

Although the majority of clients had some form of income, mostly Newstart Allowance or Disability Support Pension, their presenting issues indicate that financial difficulty was a key feature of their homelessness. Clients' other admission status criteria indicates that they would have high and complex needs, including approximately 30% of clients who had a recorded mental or physical illness.

The proportion of clients acknowledging indigenous status for Phase 1 and 3 was similar but with a much higher proportion identified in Phase 2 at 11%. Although indigenous status is a question asked at admission, staff report not all clients acknowledge their status.

In all three phases the majority of clients stayed 6 weeks or less. Further analysis of outcomes to determine whether length of stay has any influence on outcomes would be valuable. Not only did clients in the three phases stay for a short period of time but the majority attended the service only once. The window of opportunity for outcomes achievement therefore is very narrow with this particular client group. In analysing admission frequency, it appears that in Phases 2 & 3 clients were less likely to be repeat users than in Phase 1.

Whilst Phase 1 shows a comparatively high percentage of clients staying more than 12 weeks, this may be due to the longer timeframe of the sample i.e. 21 months vs 9 and 11 months in Phases 2 and 3 respectively. However, as clients in Phases 2 and 3 would also have had the opportunity to stay up to 9 or 11 months should the service have allowed it, the difference in length of stay between the Phases could be real.

All clients had access to basic support services such as meals, shower and laundry although these were not recorded in client files. The most common services received, especially in Phase 3, were emotional support, referral, advice and information and advocacy and liaison. Drug and alcohol support was more commonly recorded in Phase 3 than in the other phases. The higher levels of support across all services in Phase 3 may be due to the improved client staff ratio, or greater access/availability and also better recording practices. A further analysis of outcomes against support received may also be useful. In Phase 2, the client case worker ratio increased to 1:29 and very little support appears to have either been provided or recorded in the transitional accommodation.

In all phases the majority of clients had 'Unknown' exit points. Exit decisions were mostly made by the client. The second highest category of exit decision making was by staff where clients have left involuntarily. Drug use was a predominant reason for involuntary exit in Phase 3.

Summary of demographics, admission and exit status and services received.

In all three phases clients were young, classified as primary or secondary homeless, had drug and alcohol problems and a history of homelessness. They stayed short periods of time, generally only used the service once and often left without the exit point being known. This would suggest that the achievement of outcomes would be difficult due to the complex nature of the clients.

Caveats for interpretation of outcomes data

Goal Weighting

Goals vary in terms of difficulty of achievement, for instance, goal 1.1 Develop a list of accommodation options appears less difficult than goal 1.23 Move into independent accommodation. Goals were not weighted, but will be considered for further outcomes measurement.

Subjectivity/Validation

Goal ratings are susceptible to support worker subjectivity due to the inability to validate all ratings with all clients. Ideally case worker and client should work together to validate goal ratings. This is not always feasible particularly when clients do not want to be formally involved.

Outcomes

In Phase 1 outcomes were assessed through an audit of case notes. It became apparent during this audit that although anecdotally clients were achieving good outcomes, these were not recorded systematically in the case notes. Achievement of outcomes in Phase 1 is therefore only an estimate of the true outcomes of the service at that time. It is also impossible to determine how many clients set or did not set goals.

In Phase 2, 73/160 (45.6%) of clients selected at least one goal with an average of 3.2 goals per client. In Phase 3, 130/177 (73.4%) of clients selected at least one goal with an average of 11.9 goals per client. Therefore there was a substantial increase in the proportion of clients selecting goals between Phase 2 and Phase 3. This difference appears significant with X2 = 27.15676 and p <.001.

For those clients who did select goals in Phases 2 and 3 significant changes were recorded. In Phase 2 Fully or Partly achieved goals were predominantly in the Legal, Daily Living, Financial and Health areas with 69%, 67%, 58% and 57% of them being either Fully or Partly Achieved. Goals least likely to be achieved in Phase 2 were Daily Living (33%), Drug & Alcohol/Gambling (22%), Relationships (20%), Education and Training (20%) and Family Relationships and Legal Issues (15% each).

In Phase 3 those goals most likely to be achieved were Legal Issues (98%), Daily Living (96%), Physical Health (82%), Accommodation (80%), Gambling (79%), Employment (75%), and Education and Training (75%). Goals in the Identification goal domain showed as the least likely in being achieved (36%).

Phase 2 clients Partly or Fully achieved 43% of goals in comparison to 78% of Phase 3 clients.

The outcomes achieved in Phase 3 appear to be significantly different to Phase 2, with a higher proportion of clients nominating goals and being successful in their achievement. This may be due to a variety of factors including:

  • A variety of evidence based strategies for implementing outcomes;
  • Improved service availability for clients;
  • A lower worker:client ratio of 1:8 in Phase 3 vs 1:29 in Phase 2 and 1:19 in Phase 1;
  • Improved recording methods by staff;
  • An increased focus on outcomes throughout NSW/ACT Community Services;
  • Integrating goal setting and outcomes measurement into case management;
  • Providing case workers with training and tools for outcomes measurement;
  • Providing case workers with continued support in the measurement of outcomes;
  • An outcomes measurement tool that was customised for the service; and
  • A more positive physical environment.

Longitudinal assessment of individual outcome achievement may be more able to determine overall success of SAAP crisis interventions.

Client satisfaction

Client satisfaction with the services was assessed in Phases 2 and 3.

In Phase 2 clients were located in temporary accommodation at St.Vincent's Hospital. This environment, whilst a necessary transition venue, was not liked by clients. They reported dissatisfaction with the physical facilities, the meals and the lack of contact with staff. This would generally not have promoted an environment conducive to intensive goal setting and outcomes achievement. However, clients acknowledged that staff were open, non-judgemental and supportive.

With the refurbished building and improved infrastructure, service model and human resources in Phase 3, clients gave much more positive feedback about the service. In particular they mentioned the improved physical surroundings including the privacy allowed with private rooms, the array of services and facilities available, the helpfulness of the staff and the focus on providing a service that strengthens an individual's skill and capacity to successfully participate in community life and the workforce. Clients made positive comments on goal setting, planning and achievement.

Clients reported that Mission Australia Centre was a 'new benchmark' that had been achieved and urged this to be maintained and extended across other services.

Return to Top

Conclusion

Research Questions

1. Has the introduction of a best practice service model for Campbell House improved outcomes for clients?

The results suggest that there has been a positive change in outcomes for clients using the service in Phase 3. More clients in Phase 3 than Phase 2 selected at least one goal (73.4% vs 45.6%) This difference in proportion of clients choosing at least one goal between the two phases is significant.

In terms of outcomes achieved, clients in Phase 3 selected more goals and had a higher percentage of Fully or Partly Achieved goals than the clients in Phase 2 in all the key goal domains. Determining what actually contributed to these changes is less certain with a range of factors contributing such as:

  • The improved physical surroundings of the new service;
  • New staff focussed on goal setting and outcomes measurement from the start;
  • The availability of tools and supports to facilitate outcomes measurement;
  • Ongoing training and support from a dedicated project officer; and
  • The increased availability of on-site services, programs and facilities for clients.

2. What are the most useful outcomes and client satisfaction measurement tools and measurement methods for SAAP accommodation clients and staff?

In consultation with staff at the original and new service, there was overall agreement that the Standard Goal Scale was the most appropriate outcomes measurement tool. This was customised by the staff to suit the service and the clients. Client Satisfaction was best measured through structured focus groups as clients were more likely to attend, particularly with incentives such as cake! Questions were devised in consultation with staff but clients were given plenty of opportunity to make further suggestions and comments. The use of questionnaires to gain client feedback had not been successful at the service in the past.

3. What are likely to be the most effective implementation strategies to roll-out the routine use of standardised outcomes measurement tools in Mission Australia SAAP services?

A range of strategies were employed to implement outcomes measurement not only for Campbell House/MAC but throughout NSW/ACT Community Services.

At Campbell House/MAC, intensive support for staff to implement outcomes measurement was necessary. The staff was trained initially in groups then had individual support from the project officer. This approach ensured that each staff member had the opportunity to discuss any difficulties or concerns confidentially as well as receiving standardised instruction in outcomes theory and the use of the tools. The outcomes toolkit was devised separately to this project but proved invaluable in embedding the implementation process.

In addition to this project, Mission Australia NSW/ACT Community Services increased its focus on outcomes through the period of the project and beyond. All staff received training in outcomes measurement and a copy of the toolkit. In addition, all Service Managers were required, as part of their performance agreements, to conduct at least two outcomes projects over the period of the study.

The best practice implementation strategies identified in the literature review provided very effective, particularly:

  • Agency commitment;
  • Integration into case management practice;
  • Combining information systems that support data input and report generation;
  • Training and support during implementation;
  • Positive organisational, economic and community environments;
  • Adequate resourcing;
  • Reinforcement of change;
  • A planned and systematic approach;
  • Raising both internal and external awareness;
  • Refining tools and practices to meet client, staff and service needs;
  • Goal selection be client driven and worker supported;
  • Consultative, collaborative and consensual decision making with case workers; and
  • Group and one on one education, training and support.

4. What are the costs associated with outcomes and client satisfaction measurement in SAAP services?

The cost of implementing outcomes measurement across 130 clients by 5 staff members over a period of 42 weeks is approximately $10,000. This cost includes training, development of tools and ongoing support to staff. Case management is not included in this cost.

Return to Top

Recommendations

Recommendations from the project include:

  • That the implementation of outcomes measurement be planned and systematic to aid its success;
  • Staff be provided with training and ongoing support both on an individual and team basis;
  • A staff member/resource be appointed to facilitate and manage the implementation of outcomes measurement;
  • Use flexible and consultative approaches with staff to improve outcomes measurement practices and promote practice and skill development;
  • Use feedback from staff to improve outcomes measurement implementation based on their knowledge and experience of clients;
  • Use a goal focused approach to encourage clients' motivation to achieve goals;
  • Use visual tools and graphs that outline goals and achievement to motivate clients and acknowledge and provide evidence of change;
  • Adapt outcomes tools and practices to meet client needs including level of involvement and interest;
  • Adapt outcomes tools and practices to fit with staff practices and service delivery model;
  • Standard Goal Scaling can be adapted for use across a range of service delivery models;
  • Standard Goal Scaling can strengthen case management practices by providing structure; goal setting, planning and review; stimulate conversation between worker and client; provide an instant summary of where the client is at; and can aid case workers to reflect on case management practices;
  • Trial goal weighting to address the differing degrees of difficulty in achieving some goals;
  • Develop electronic tools to assist in collating and analysing outcomes data;
  • Undertake further analysis of services received by MAC clients to determine whether services received are linked with goals achieved;
  • Undertake further research to verify the most effective interventions in assisting clients to achieve their goals and move out of homelessness; and
  • Undertake a longitudinal research project to determine whether crisis accommodation service interventions have longer term impact on client outcomes.

Return to Top

Acknowledgements

We would like to thank the following people for their assistance in the development of this project and report.

  • Family and Community Services and Indigenous Affairs (FaCSIA) for funding for the outcomes project;
  • Janis Redford, Manager NSW/ACT Community Services for her leadership and encouragement;
  • Kay Elson, Campbell House Project Manager, for her depth of knowledge and experience of Campbell House and the homeless sector;
  • Kerry Edgecombe for her support and assistance whilst she was Operations Manager for the Mission Australia Centre;
  • Service Manager, Diana Jazic for her support, experience and insight into the clients of Campbell House/Mission Australia Centre;
  • Team Leader and case worker, Meffan Kawai, for his assistance with collecting data for the report, knowledge and experience, and insight into the clients of Campbell House/Mission Australia Centre;
  • Intern student from Macquarie University, Adam Stebbing, for his assistance with collecting and preparing Phase 1 data for the report;
  • The staff of the original Campbell House who have assisted with a number of queries on the running of Campbell House;
  • Unit co-ordinators and staff of the new Mission Australia centre for their enthusiastic approach to goal setting and outcomes measurement and 'setting the agenda'; and
  • Alison Boylan and Skye MacDonald for their meticulous data collection for Phase 3.

Return to Top

Glossary

Outcomes
The impact or consequences or results of the program for the service user. A change or an absence of change in an identified state which is the intended or unanticipated result of an action or set of actions carried out by a program (Baulderstone, 2004).

Homelessness
Homelessness refers to the condition of inadequate access to secure and safe housing. Standards of what constitutes secure and safe housing are culturally relative; they are developed by a community in a given place at a certain time (Chamberlain & Johnson, 2001).

Action Research
Action research is a research method that allows for continuous evaluation and development of existing practices. Action research entails an ongoing cycle of analysis that involves the examination of existing practices, evaluating the ability of existing practies to meet a desired goal or need, and enacting changes to existing practices based on evaluation (Punch, 2001, p.143). Action and evaluation occur simultaneously in projects utilising action research (Punch, 2001, p.143).


Definitions of Homelessness
Culturally recognised exceptions: Categories
where it is inappropriate to apply the minimun standard - eg seminaries, gaols, student halls of residence Marginally housed: people in housing situations close to the minimum standard
Tertiary homelessness: people living in single rooms in private boarding houses - without their own bathroom, kitchen or security of tenure
Secondary homelessness: people moving between various forms of temporary shelter including: friends, emergency accommodation, youth refuges, hostels and boarding houses
Primary homelessness: people without conventional accommodation (living on the streets, in deserted buildings, improvised dwellings, under bridges, in parks etc.

Return to Top

Appendices

  • Appendix 1: Closing the Revolving Door
  • Appendix 2: Campbell House Retrospective Client Audit Form
  • Appendix 3: Campbell House Goal Scaling Criteria and Guidelines
  • Appendix 4: Mission Australia NSW/ACT Outcomes Measurement Toolkit Version 1
  • Appendix 5: Excel Aggregation Sheet
  • Appendix 6: SGS - Goal Scores Graph
  • Appendix 7: SGS - Individual Goal Graph

Return to Top

References

  • Australian Institute of Health and Welfare (AIHW), (2003) Australia's welfare 2003 Canberra: AIHW, p.159-174
  • Australian Institute of Health and Welfare (AIHW) (2005) Indigenous Housing Needs 2005, a Multi-measure Needs Model, Canberra: AIHW
  • Baulderstone, J. (2004) Measuring Outcomes Workshop, 1 & 2 November 2004, Flinders University
  • Baulderstone, J. and Talbot, C. (2004) Clients and outcome management: Counting by numbers: Homeless data, Parity, Vol.17, Issue 6, Melbourne
  • Baulderstone, J. and Talbot, C. (2004) Outcome Measurement in SAAP Funded Services, Flinders Institute of Public Policy and Management, p.vii - 40
  • Berry, M., Chamberlain, C., Dalton, T., Horn, M. and Berman, G. (2003) Counting the Costs of Homelessness: A Systematic Review of Cost Effectiveness and Cost Benefit Studies of Homelessness, Canberra: Australian Housing and Research Institute
  • Chamberlain, C., and Johnson, G. (2001) The debate about homelessness: Australian Journal of Social Issues, 36(1), pp.35-50
  • Chamberlain, C., and MacKenzie, D. (2001) Counting the Homeless 2001, Australian Census Analytic Program, Australian Bureau of Statistics
  • Chamberlain, C., and MacKenzie, D. (2003) Youth Homelessness: Early Intervention and Prevention, Sydney, Australian Centre for Equity through Education
  • Commonwealth Advisory Committee on Homelessness (2001) Working Towards a National Homelessness Strategy: Commonwealth Advisory Committee on Homelessness: Consultation Paper, Department of Family and Community Services, http://www.facs.gov.au/internet/facsinternet.nsf/vIA/nat_homeless/$file/total.pdf, accessed 30 October 2004
  • SCRGSP (Steering Committee for the Review of Government Service Provision) (2005) Overcoming Indigenous Disadvantage: Key Indicators 2005, Productivity Commission, Canberra
  • Communities Scotland (2001) Models of Intermediate Accommodation, Precis No. 4, http://www.communitiesscotland.gov.uk/web/FILES/CSPrecis4.pdf, accessed 20 October 2004
  • Crisis (2004) Urban Living Project Scotland, http://www.crisis.org.uk/projects/crisis_urbanlivin.php, accessed 15 September 2004
  • The Enterprise Foundation (2004) Beyond Housing: Homes for the Homeless, New York, NY, http://www.enterprisefoundation.org/model%20documents/e458.htm, accessed 25 October 2004
  • Fosburg, L.B., and Dennis, D.L. (1998) The 1998 National Symposium on Homelessness Research, United States Department of Health and Human Services, http://aspe.hhs.gov/progsys/homeless/symposium/overview.htm, accessed 14 September 2004
  • Mission Australia (2004) Homelessness: New understanding, new responses, Mission Australia Snapshot
  • National Alliance to End Homelessness (2001) Best Practices - Pathways to Housing, New York, NY,
  • http://www.endhomelessness.org/best/pathways.htm
  • Newman, T. (2003) The Strategic Thrust of the Victorian Homelessness Strategy, Department of Human Services, Victoria
  • Punch, K. F. (2001) Introduction to Social Research: Quantitative and Qualitative Approaches, London, Thousand Oaks and New Delhi, Sage Publications, p.143
  • Rickwood, D. (2004) Pathways of Recovery: Preventing Relapse, Commonwealth of Australia, Department of Health and Ageing, National Mental Health Promotion and Prevention Working Party
  • Robinson C. (1998) Living at risk: Breaking the cycle of homelessness, A report from the St.Vincent de Paul Society 1998
  • Rogers, E. (1995) Diffusion of Innovation, Fourth Edition. New York, NY, The Free Press
  • Rosengard, A., & Associates, and Scottish Health Feedback (2000) Hostels for Homeless People in the Future, Scottish Executive Unit, http://www.scotland.gov.uk/cru/kd01/red/fhhp-03.asp, accessed 13 September 2004
  • NHS Centre for Reviews and Dissemination (1999) Effective Health Care: Getting Evidence into Practice, Royal Society of Medicine Press Limited, Vol. 5, No. 1, ISSN: 0965-0288, http://www.york.ac.uk/inst/crd/ehc51.pdf, accessed 12 June 2006
  • Sanson-Fisher, R., Grimshaw, J., and Eccles, M. (2004) The science of changing providers' behaviour: the missing link in evidence-based practice, The Medical Journal of Australia, MJA 180 (5): 205-206, http://www.mja.com.au/public/issues/180_05_010304/san10013_fm.html , accessed 12 June 2006
  • The Urban Institute (2003) Key Steps in Outcome Measurement: Series on Outcome Management for Nonprofit Organizations, http://www.urban.org/uploadedPDF/310776_KeySteps.pdf, accessed 12 June 2006
  • Walker, V. and Ireland, M. (2003) Sustainable Housing for Traditional Living Aboriginal People moving to Adelaide: Malpa-Kutjara, prepared for the Department of Family and Community Service (SA)

Return to Top

Last updated: