ACF - Administration for Children and Families

09/22/2022 | Press release | Distributed by Public on 09/22/2022 15:43

Applying Human-Centered Design to Human Services: Pilot Study Findings

Applying Human-Centered Design to Human Services: Pilot Study Findings

Publication Date:September 22, 2022

Download Report

Download Report PDF (2,829.43 KB)
  • File Size: 2,829.43 KB
  • Pages: 131
  • Published: 2022

Introduction

Expand

Research Questions

  1. What types of challenges within ACF programs are best suited for an HCD approach?
  2. What resources are required to implement HCD approaches in ACF programs?
  3. What systemic or cultural barriers may make implementation a challenge, and can those be mitigated?
  4. What does HCD implementation look like?
  5. How can the HCD approach be evaluated in order to better understand outcomes of interest to ACF? Can HCD be evaluated to determine whether or not this approach is more or less successful than traditional approaches?
  6. What criteria are defined as successful outcomes when evaluating this process?
  7. Were improvements observed on outcomes of interest for end users?
  8. Were improvements observed within the organization?

Human-Centered Design (HCD) is a process and a mindset for addressing complex problems by designing solutions with those who will ultimately use the solution (i.e., end users). Because of its inherent focus on end users or recipients of services, HCD appears to have potential for promoting effective, efficient, and compassionate service delivery that is aligned with the mission of the U.S. Department of Health and Human Services' Administration for Children and Families (ACF). Despite this appeal, the implementation of HCD within human services is relatively novel. There has been little empirical work to date on how HCD might be used to improve outcomes of interest to ACF programs.

In 2018, ACF's Office of Planning, Research, and Evaluation (OPRE) initiated the Human-Centered Design for Human Services (HCD4HS) project to explore the viability of HCD in addressing the complex problems facing public sector human services programs. The HCD4HS project team was comprised of Child Trends, Anthro-Tech, and MEF Associates. This project included a review of the knowledge base to define HCD and describe how it has been used and evaluated in the human services context, and a pilot study to evaluate the implementation of HCD, with a focus on assessing its evaluability.

Purpose

Expand

Human services programs address complex social issues ranging from supporting healthy relationships, child welfare, and economic mobility, to providing high-quality, accessible early childhood programs. HCD can be used to spark innovation to address the myriad challenges faced by human services programs.

Key Findings and Highlights

Expand
  • All three pilot sites appeared to make good progress in addressing very disparate challenges: Temporary Assistance for Needy Families (TANF) cliff effect, staff engagement, and completion of child support order modifications. However, given that there were only three sites, we cannot fully address which types of challenges within the broad range of ACF programs may be best suited for HCD.
  • Design teams (team of staff at each pilot agency) identified project management, leadership support, and incentives for end user participation as critical resources for implementation of HCD, similar to implementation of other change initiatives.
  • Design teams identified primary barriers as time and capacity, recruitment of end users, getting incentives in place to encourage end user participation, and organizational structures and processes.
  • Using a capacity building approach, all design teams participated in an initial HCD Primer workshop (24 hours) followed by 11 months of training and coaching from an HCD consultant (average of 5 hours/week) and content expert (average of 2-4 hours/week). All training and coaching were provided virtually.
  • Design teams clearly demonstrated HCD principles related to empathy and collaboration. Demonstration of some principles varied across site, time, and reporter. The evaluation timeline precluded full assessment of all principles.
  • Given limitations of existing measures, this project developed tools including weekly and monthly logs of HCD activities, interviews, and an Implementation Assessment assessing HCD principles, process, and mindset. However, the project timeline precluded evaluation of solutions, including outcomes for end users. In order to compare HCD to other approaches, different tools would likely be needed.
  • Given that design teams did not get to the HCD phase where they would implement their solution, outcome improvements for end users could not be assessed.
  • All design teams demonstrated an HCD mindset by demonstrating empathy; openness to the opinions and perspectives of end users and others; and adopting new ways of identifying challenges, brainstorming, and trying different ideas. They also demonstrated a bias toward action.
  • HCD can be evaluated systematically in human services programs with a variety of theoretically-driven data collection tools, although more work is needed in measure development.
  • With expert training and coaching, design teams demonstrated HCD principles and implemented a range of HCD techniques with different challenges, end users, and contexts.
  • From early in the evaluation, design team members demonstrated an HCD mindset.
  • Design teams demonstrated capacity for HCD through using strategies competently, building confidence, developing processes to support sustainability, and addressing challenges that arose.
  • HCD was found to be useful and relevant in addressing disparate challenges across three sites, and each site had interest in continuing to use HCD in some ways.

Methods

Expand

The HCD4HS project selected three human services agencies that administer different ACF-funded programs, have different end users, and are likely to successfully implement HCD based upon their readiness, including need, fit, resources, and capacity: Denver Human Services, Santa Clara County Social Services Agency, and Washington State Division of Child Support. These sites were identified through a structured recruitment process and were selected from 32 human services agencies or programs across the country that self-nominated.

The pilot study implemented a capacity-building approach in which HCD consultants from Anthro-Tech provided ongoing HCD training and coaching to a design team who carried out the HCD activities. Training began with an introductory workshop in January 2021; Design Thinking Workshops were held in the summer of 2021; and coaching occurred from February through December 2021. Design teams received an average of 7-9 hours of coaching support per week. Evaluation data were collected throughout most of the implementation period, beginning in February 2021 and ending in November 2021.

The HCD4HS pilot study used a mixed methods evaluation approach in which similar questions were asked of different individuals participating in the pilot study, in different formats. For example, some data were collected as weekly and monthly logs to assess the consistency of implementation experiences across time. To address the research questions, the project developed of a suite of measures for use in this evaluation based on the literature, which included:

  • Design Team Logs,
  • HCD Consultant Logs,
  • Content Expert Logs,
  • Interview protocols for design teams and HCD consultants, and an
  • Implementation Assessment tool (a semi-structured group interview assessing key aspects of implementation in a manner that is quantifiable).

Citation

Expand

Murray, D.W., Rosinsky, K., Haas, M., Glosser, A., & Boyd, S. (2022). Applying Human-Centered Design to Human Services: Pilot Study Findings, OPRE Report 2022-214, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.