Download REA Impact Study: Implementation Report PDF

TitleREA Impact Study: Implementation Report
LanguageEnglish
File Size3.1 MB
Total Pages161
Table of Contents
                            REA Implact Study: Implementation Report Cover
Acknowledgements
Table of Contents
List of Exhibits
1. Introduction
	1.1 Policy Context and Overview of the REA Program
		1.1.1 REA’s “Assistance” Components
		1.1.2 REA’s “Enforcement” Components
	1.2 Overview of the Research Design
		1.2.1 Five Study Hypotheses
		1.2.2 Four Treatment Conditions
		1.2.3 Two Complementary Evaluation Studies
	1.3 Overview and Organization of the Implementation Evaluation
	1.4 About This Report
		1.4.1 Terminology
		1.4.2 Disclaimers
2. Overview of the Four Participating State Grantees
	2.1 State Selection
	2.2 Pre-Study Program Models
		2.2.1 Indiana
			Integration of REA into the State’s Workforce Development System
			UI Involvement in and Support of the REA Program
			State-Specific Implementation Issues
		2.2.2 New York
			Integration of REA into the State’s Workforce Development System
			UI Involvement in and Support of the REA Program
			State-Specific Implementation Issues
		2.2.3 Washington
			Integration of REA into the State’s Workforce Development System
			UI Involvement in and Support of the REA Program
			State-Specific Implementation Issues
		2.2.4 Wisconsin
			Integration of REA into the State’s Workforce Development System
			UI Involvement in and Support of the REA Program
			State-Specific Implementation Issues
	2.3 Economic Backdrop for Implementing the Impact Study
3. Implementing the Research Design across Four States
	3.1 Integrating State Variation into a Single Evaluation Design
		3.1.1 Refinement of States’ Treatment Conditions
		3.1.2 Overview of States’ Program Eligibility Rules
		3.1.2 Adaptation of States’ Random Assignment Processes
	3.2 Implementing Random Assignment during the Evaluation
		3.2.1 Random Assignment Procedures across States
		3.2.2 Timing of Random Assignment
		3.2.3 Size and Composition of the Sample
	3.3 Monitoring REA Service Receipt
	3.4 Benefits of Technical Assistance and Monitoring
	3.5 Conclusion
4. The Initial REA Meeting
	4.1 Scheduling and Timing Policies
		4.1.1 Scheduling Process
			Indiana
			New York
			Washington
			Wisconsin
		4.1.2 Timing of the Initial REA Meeting
	4.2 Content of Initial Meeting—Full REA
		4.2.1 Federal Guidance to States
		4.2.2 Cross-State Summaries
			Pre-Orientation
			Orientation
			One-on-One Assessment Meeting
			UI Eligibility Assessment
	4.3 Content of the Initial Meeting—Partial REA
		4.3.1 Study Guidance to States
		4.3.2 Cross-State Summaries
			Pre-Orientation
			Orientation
			One-on-One Assessment Meeting
			UI Eligibility Assessment
	4.4 Attendance Rates
		4.4.1 Full REA (Single and Multiple) Initial REA Meetings
		4.4.2 Partial REA Initial REA Meetings
	4.5 Conclusion
5. Subsequent REA Meetings
	5.1 Cross-State Summaries
		5.1.1 Eligibility Criteria
		5.1.2 Description of the Second REA Meeting
			Timing and Duration
			Staffing and Delivery
			Content of Meetings
		5.1.3 Description of the Third REA Meeting
	5.2 Attendance Rates—Multiple REA
	5.3 Overall Completion Rates in Indiana
	5.4 Conclusion
6. Receipt of Reemployment Services
	6.1 Overview of Federal Guidance
	6.2 Variation in REA-Related Reemployment Service Referrals
		6.2.1 Referral Policies
		6.2.2 Compliance Monitoring
		6.2.3 Timing of Service Receipt
			Policies on Timing
			Actual Timing of Service Receipt
	6.3 Receipt of Reemployment Services
		6.3.1 Indiana
		6.3.2 New York
		6.3.3 Washington
		6.3.4 Wisconsin
	6.4 Key Observations
7. Noncompliance with REA and UI Requirements
	7.1 Federal Guidance on Consequences of Noncompliance
	7.2 Potential Repercussions: Penalizing Noncompliance
	7.3 The Complexities of Implementing a Noncompliance Policy
	7.4 Noncompliance—Failure to Report
		7.4.1 Initial REA Meeting
			Procedures for Noncompliance
			Penalty for Noncompliance
				Creation of a Pending Issue
				Investigation of a Conduct Issue
		7.4.2 Subsequent REA Meetings
	7.5 Noncompliance—Reemployment Plan Activities
		7.5.1 Procedures for Assessing Compliance
		7.5.2 Penalty for Noncompliance
	7.6 Issues Discovered during Eligibility Review
		7.6.1 Review of Claimants’ Eligibility
		7.6.2 Review of Weekly Work Search Activities
	7.7 Discussion
8. Conclusion
	8.1 Cross-State Overview
		8.1.1 Indiana
		8.1.2 New York
		8.1.3 Washington
		8.1.4 Wisconsin
	8.2 Final Thoughts
References
Appendix A: Implementation Study Research Questions
Appendix B: Qualitative Data Collection
Appendix C: Random Assignment Processes by State
Appendix D: Data and Samples
Appendix E: Detailed Descriptions—Initial REA Meeting Content and Procedures
Appendix F: Detailed Service Receipt Exhibits
Appendix G: Compliance and Duration-Related Text Used in Notification Letters
                        
Document Text Contents
Page 1

REA Impact Study:
Implementation

Report

Contract # DOLQ123633231
Order # DOL-OPS-14-R-00012 (REA2)

Revised Draft

January, 2017

Prepared for:
Molly Irwin

Chief Evaluation Office
U.S. Department of Labor
Frances Perkins Building

200 Constitution Ave. NW
Washington, DC 20210

Submitted by:
Abt Associates

4550 Montgomery Avenue
Suite 800 North

Bethesda, MD 20814

In Partnership with:
George Washington University

Capital Research Partners

Page 2

This project has been funded, either wholly or in part, with federal funds from the U.S. Department of
Labor, Chief Evaluation Office under Task Order DOL-OPS-14-U-00072. This report does not
necessarily reflect the views or policies of the Department of Labor, nor does mention of trade names,
commercial products, or organizations imply endorsement of same by the U.S. Government.

The suggested citation is: Minzner, A., Klerman, J., Epstein, Z., Savidge-Wilkins, G., Benson, V.,
Saunders, C., Cristobal, C., and Mills, S. (2017). REA Impact Study: Implementation Report.
Prepared for the U.S. Department of Labor. Cambridge, MA: Abt Associates.

Abt Associates REA2 Impact Study: Implementation Report

Page 80

REEMPLOYMENT SERVICES


Using these five categories, this section examines participation patterns in all reemployment services
received starting the day after claimants’ random assignment to a treatment group (as recorded in the
state’s data system). The analysis does not consider the source of the referral for the services (i.e., from an
REA meeting, self-referral, or something else). This analysis provides insights into the net increase in
service receipt that can be associated with assignment to a particular treatment condition.64

Exhibit 6.3: Common Service Type Categories
Service Type Examples of Services

Staff-Assisted Services • Career guidance and counseling
• Job search assistance
• Assessment of skills and/or needs
• Orientation, outreach, and general case management
• Provision of labor market information

Training Services and
Supports

• Engaged in adult basic education activities
• Engaged in occupational skills and/or on-the-job training
• Engaged in short-term pre-vocational services

Workshop Attendance • Workshops with topics such as:
o Job search strategies
o Work readiness
o Financial literacy

• Specialized assessment/diagnostic testing proctored in group setting

Referrals to Employment,
Training, and Services

• Referral to job search workshops
• Referral to additional supportive services
• Referral to employment

Self-Service and Facilitated
Self-Help

• Accessed information for job search
• Accessed labor market information
• Completed skills/career interest assessment (not in group setting)

6.3.1 Indiana

Indiana’s Single REA treatment group had a relatively high rate of service receipt; 72 percent of
participants received at least one service (Exhibit 6.4). The most common service received was self-
services and facilitated self-help.

64 For the purposes of this implementation analysis, knowing whether service receipt was directly tied to REA
meetings would be desirable but is not crucial. Unfortunately, the nature of the states’ data systems made it
difficult to definitively identify what services were received as referrals from the REA meetings.

Abt Associates REA2 Impact Study: Implementation Report ▌pg. 72

Page 81

REEMPLOYMENT SERVICES


Exhibit 6.4: Service Receipt among Indiana Participants during Study Period, by Treatment
Condition

Comparing service receipt by Single REA members with that of Partial REA members, the former
received considerably more services in two of the most time-intensive categories: staff-assisted services
and workshop attendance. In neither group did many claimants receive training services and supports.
Additionally, though service receipt appeared more similar than expected in the categories of referrals and
self-services, taken together, these results suggest substantively important differences in services received
by the two treatment groups (see Appendix F, Exhibit F.2).65 Partial REA participants almost exclusively
received auto-generated employment referrals, whereas the Single REA participants received both auto-
generated referrals and customized employment referrals from staff plus pre-vocational services,
assessments, and other supportive services.

Similarly, the most common self-services activities for Partial REA group members were accessing job
search information, assessing labor market information, and completing a skills assessment, all of which
occur in the Indiana Career Connect (ICC) system.

Though the percentages of Partial REA group and Control group members receiving any service were
higher than expected, these are primarily limited to self-services and auto-generated referrals. The more
nuanced service-level data (shown in Appendix F, Exhibit F.2) show that less than 5 percent of either of
these groups received a staff-assisted service during the study period, and that the state honored its
commitment to the study by not providing one-on-one assistance to these participants on a routine basis.

65 All UI claimants are required to register with ICC. When registering, claimants can sign up for emails that
provide customized, available job listings. Claimants select the frequency of these emails (e.g., daily, weekly).

Abt Associates REA2 Impact Study: Implementation Report ▌pg. 73

Page 160

APPENDIX G


Exhibit G.3: Washington’s Compliance-Related Text Used in Notification Letters
Group Text

Control [n/a; no notification sent]

Partial REA “If you miss your appointment, Employment Security will send a questionnaire that you must
complete to explain why you did not attend. We will decide whether you had good cause for
missing the orientation, as well as review your availability for work and your job-search activities.
This may result in your benefits being denied and you may have to repay some or all of any
benefits you’ve received.”

Single REA “If you miss your appointment, Employment Security will send a questionnaire that you must
complete to explain why you did not attend. We will decide whether you had good cause for
missing the orientation, as well as review your availability for work and your job-search activities.
This may result in your benefits being denied and you may have to repay some or all of any
benefits you’ve received.”

Multiple REA “If you miss your appointment, Employment Security will send a questionnaire that you must
complete to explain why you did not attend. We will decide whether you had good cause for
missing the orientation, as well as review your availability for work and your job-search activities.
This may result in your benefits being denied and you may have to repay some or all of any
benefits you’ve received.”

Exhibit G.4: Wisconsin’s Compliance-Related Text Used in Notification Letters
Group Text

Control “Initial Mailed Notice: “Your deadline to complete both the online orientation and assessment is
[INSERT DATE]. Note: After the deadline, you will not receive unemployment benefits until you
complete both the online orientation and assessment elements.”

Partial REA “Initial Mailed Notice: “Your deadline to complete both the online orientation and assessment is
[INSERT DATE]. Note: After the deadline, you will not receive unemployment benefits until you
complete both the online orientation and assessment elements.”
Online Meeting Requirement: “If you fail to schedule, fail to attend a scheduled session or fail to
complete any follow-up requirements, your Unemployment Insurance benefits may be denied until
you comply with all requirements.”

Single REA [n/a; no single REA treatment group]

Multiple REA “Initial Mailed Notice: “Your deadline to complete both the online orientation and assessment is
[INSERT DATE]. Note: After the deadline, you will not receive unemployment benefits until you
complete both the online orientation and assessment elements.”
Online Meeting Requirement: “If you fail to schedule, fail to attend a scheduled session or fail to
complete any follow-up requirements, your Unemployment Insurance benefits may be denied until
you comply with all requirements.”

Abt Associates REA2 Impact Study: Implementation Report ▌pg. 152

Page 161

APPENDIX G


Exhibit G.5: Indiana’s Duration-Related Text Used in Notification Letters
Group Appointment Duration Text

Control [n/a; no appointment held]

Partial REA No text about appointment duration included

Single REA “The Orientation and Assessment Interview process may take < > hours.”a

Multiple REA [n/a; no Multiple REA treatment group]
a The duration included in the letter varies from office to office, as each office inserts the duration into the bracketed
space. The state standard is 45 minutes.

Exhibit G.6: New York’s Duration-Related Text Used in Notification Letters
Group Appointment Duration Text

Control [n/a; no appointment held]

Partial REA “Your visit may take up to 1 hour.”

Single REA “Your visit can take up to two hours.”

Multiple REA “Your visit can take up to two hours.”

Exhibit G.7: Washington’s Duration-Related Text Used in Notification Letters
Group Appointment Duration Text

Control [n/a; no appointment held]

Partial REA “DURATION: 2 hours”

Single REA “DURATION: 3 hours”

Multiple REA “DURATION: 3 hours”

Exhibit G.8: Wisconsin’s Duration-Related Text Used in Notification Letters
Group Appointment Duration Text

Control [n/a; no appointment held]

Partial REA “Each REA-Partial Session will last approximately 15 minutes and will include a 1:1 meeting with
Job Service staff.”

Single REA [n/a; no single REA treatment group]

Multiple REA “Each REA-Full session will last approximately three (3) hours and will include a group
presentation, including a short presentation from a Ul adjudicator, and 1:1:1 where each
participant receives 5-10 minutes of individual assistance from Job Service and Unemployment
Insurance staff”

“Each 1:1:1 session should last approximately 5-10 minutes. lf additional time is needed to fully
assist the participant, a Continuing Eligibility Assessment (CEA) should be scheduled using the
Re-employment Appointment Notice.”

“The CEA appointment should be scheduled for 15-30 minutes (at the discretion of local office
staff.)”

“The FEA appointment should be scheduled for 15-30 minutes (at the discretion of local office
staff.)”

Abt Associates REA2 Impact Study: Implementation Report ▌pg. 153

Similer Documents