Leisure Works! - Phase II Report
Martin R. Sheehan, Ph.D., Principal Investigator
R44 AG17003-02 Phase II
Project Period: 4/01/2001 - 5/31/2003
Program Title: Leisure Works! Expanding Options for People with Developmental Disabilities
Summary
The idea behind Leisure Works! was to increase leisure time options for adults with developmental disabilities by providing a curriculum that would walk them through the process of identifying options, researching them, and trying them out.
Through a grant from the National Institute on Aging, an IRIS Media research and development team, along with expert consultants, gathered data, developed a video and print materials based on the data, and tested the effectiveness of the program.
The Leisure Works! video and workbook program were used by adults living in group homes. These residents and the direct care staff in their homes evaluated the materials.
Compared to a control group "which did not show an increase in frequency or variety of leisure activities" study participants increased the number of times each month they participated in leisure activities and tried out new leisure activities that were unfamiliar to them. The program and materials were also highly rated by staff. Click here for a description of the intervention program .
Project Aims
The goal of the project was to develop, produce, and test an interactive multimedia (IMM) training program on providing quality retirement and leisure services to older adults with developmental disabilities (DD). The IMM program was designed primarily for use by direct support staff who work with these older adults in residential and day programs. Other users who would benefit included parents, advocates, case managers from the DD service network, and community program staff from the aging service network.
The complete program addressed five broad content areas in the area of retirement and leisure:
1) Understanding the leisure experience
2) Leisure awareness and appreciation
3) Community opportunities and resources
4) Assessment and skill development
5) Choice making and decision making about leisure activities.
In the course of developing the program, the scope and sequence of the content were modified and expanded. The delivery format was also somewhat modified. These modifications are covered in the task descriptions. Once completed, the program was evaluated in a randomized controlled study assessing knowledge, skills, and behavioral outcomes. Project activities were completed in a timely manner.
Research Results and Significance
The Phase II product incorporated a modified and expanded version of the Phase I product, adding four program modules. There were eight product development tasks: (1) to undertake a content analysis for the remaining sub-topic in Module 1 and for the remaining four modules; (2) to conduct reviews of the proposed content with a panel of experts and to revise based upon their input; (3) to conduct focus groups to evaluate content; (4) to design and develop instructional materials (scripts, navigational flowcharts, and screen layouts) and to collaborate with SONY Digital Authoring Services on the design of the graphic user interface (GUI); (5) to conduct a review of instructional materials by the panel of experts and to revise according to their input; (6) to produce the media elements and to finalize the design of the IMM GUI with SONY; (7) to author, test and revise the Alpha program integrating media assets and GUI design; (8) to Beta test the CD-ROM version of the IMM program using the in-house team and the expert panel.
Formative evaluation occurred throughout the development process by means of focus groups and a panel of experts. Summative evaluations occurred during the last year of the project. The overall project timeline is shown in the below. There were two evaluation tasks: (9) to design and evaluate measures using face validity, item difficulty, and test-retest instrument development procedures; and (10) to conduct a randomized controlled study (with a sample of 100 direct-support staff members and 100 residents) that evaluates knowledge, skills, and behavioral outcomes.
|
Project Timeline |
|||||||||||||||||||||||||
|
|
Project Task Activities |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
|
1 |
Content Analysis |
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
2 |
Expert Review/Content |
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
3 |
Focus Groups |
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
4 |
Develop Instructional Material |
|
|
|
X |
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
Expert Review/Scripts |
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
Produce Media |
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
Author Program |
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
Beta Test |
|
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
9 |
Develop Instruments |
|
|
|
X |
X |
X |
X |
X |
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
||
|
10.1 |
Subject Recruitment & Baseline |
|
|
|
|
|
|
|
|
|
X |
X |
X |
X |
|
|
|
|
|
|
|
|
|
|
|
|
10.2 |
Intervention & Posttest |
|
|
|
|
|
|
|
|
|
|
|
|
|
X |
X |
X |
X |
X |
X |
X |
X |
X |
|
|
|
10.3 |
Data Analyses & Final Report |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
X |
X |
Tasks
- To undertake a content analysis for the remaining sub-topic in Module 1 and for the remaining four modules.
- To conduct reviews of the proposed content with a panel of experts and to revise according to their input.
- To conduct focus groups to evaluate content and provide feedback.
- To design and develop instructional materials (scripts, navigational flowcharts, and screen layouts) and to collaborate with SONY Digital Authoring Services on design of graphic user interface.
- To conduct a review of instructional materials by the panel of experts and to revise according to their input.
- To produce the media elements and to finalize the design of the IMM GUI with SONY.
- To author, test, and revise the Alpha program integrating media assets and GUI design.
- To beta test CD-ROM version of the IMM program by the in-house team and the expert panel.
- To design and evaluate measures using face validity, item difficulty, and test-retest instrument development procedures.
- To conduct a randomized controlled study (with a sample of 100 direct support staff and 100 residents) that evaluates knowledge, skills and behavioral outcomes.
Description of Task Activities.
Undertake a content analysis for the remaining sub-topic in Module 1 and for the remaining four modules.
The development team consisted of the Principal Investigator, the Research Associate, the Media Producer, the Scriptwriter, and the Project Coordinator. As the development team analyzed the remaining undeveloped content, it became apparent that the users (direct support staff) would greatly benefit in their understanding and application of the material if they could view and work on the program alongside the people they support. It was also determined that including people with DD as program users would not negatively affect any of the content. Therefore the program content and delivery were modified to be used by dyads consisting of a staff person (or "learning partner") and an adult with DD. Since this approach required delivering the medium on a more accessible platform than that of computer-based CD-ROM, the programs were produced for the DVD format. This format is widely available in group homes and other settings and maintains the interactive properties of CD-ROM. In addition, the program was made available on video for consumers who do not have DVD.
The proposed content covered the following topic areas:
1: Understanding the Leisure Experience
- The Value of Leisure (Phase I)
- Facts about Older Adults with Developmental Disabilities (Phase I)
- Barriers to Leisure for Older Adults with Developmental Disabilities (Phase I)
- Expanding Leisure Options
2: Leisure Awareness and Appreciation
- Understanding the Meaning of Leisure
- How Leisure is Different from Other Life Domains
- Identifying a Range of Possible Leisure Pursuits
- Potential Leisure Awareness Activities
3: Community Opportunities and Resources
- The Value of Community Leisure Experiences
- Conducting a Community Survey
- Overcoming Barriers in the Community - Physical and Attitudinal
- Developing Strategies for Inclusion
4: Assessment and Skill Development
- Conducting an Individualized Leisure Assessment
- Basic Teaching Skills: Model, Assist, Practice
- Teaching New Leisure Skills
- Developing Participation Skill
5: Choice-making and Decision-making about Leisure Activities
- Personal Competency and Self-determination
- Teaching Choice-making and Decision-making
- Identifying Needed Resources
- The Leisure Action Planning Process
These topic areas referenced were reframed as 12 lessons on expanding leisure options for people with DD. Each lesson consists of audiovisual material and an interactive workbook:
Lesson 1: Let's Do Leisure
- Video presents the meaning of leisure and varieties of leisure activities.
- Workbook activity consists of signing a leisure learning agreement by staff person and person with DD.
Lesson 2: What's My Leisure?
- Video demonstrates that leisure is an activity that involves un-obligated time, personal choice, and enjoyment: My time! My choice! My thing!
- Workbook activity consists of listing the leisure activities one likes to do.
Lesson 3: What I Get from Leisure
- Video shows that leisure activities have benefits.
- Workbook activity consists of listing benefits that you get from your favorite leisure activities.
Lesson 4: Is My Leisure Balanced
- Video encourages the notion of adding new leisure activities in order to have a less one-sided and more balanced leisure life.
- Workbook activities consist of analyzing current activities and then determining whether one needs to do more inside or outside, at home or in the community, high energy or low energy, and solo or group activities in order to have a more balanced leisure life.
Lesson 5: Magic Boxes
- Video demonstrates how easy it is to sort the leisure activities one does into seven categories (magic boxes: caring, learning, sports and games, socializing, creating, trips and outings, and entertainment activities.
- Workbook activity consists of categorizing one's current activities and then thinking up new leisure ideas to fill empty boxes.
Lesson 6: Brainstorming New Activities
- Video introduces ways to brainstorm new activities and then shows people using brainstorming techniques to brainstorm activities that have a benefit they like.
- Workbook activity consists of listing benefits one wants and then brainstorming activities that provide that benefit.
Lesson 7: Halftime
- Video provides a review of the material covered so far.
- Worksheet activity consists of completing previous leisure generating exercises and picking out two activities that one most wants to try out.
Lesson 8: My Leisure Resources
- Video shows how to pick two new activities.
- Worksheet helps then identify places and agencies that may have these activities.
Lesson 9: Asking Around
- Video provides information on how to call and find out about the new activities.
- Worksheet activity consists of seven steps for getting key information about the first activity.
Lesson 10: What You Found Out
- Video reviews the things that were found out about the activity.
- Worksheet activity consists of the seven steps for getting key information about the second activity.
Lesson 11: Don't Say No
- Video show how to brainstorm and overcome barriers.
- Worksheet activity consists of problem solving.
Lesson 12: Graduation
- Video summarizes the successes people have had in experiencing a new variety of leisure activities.
- No worksheet for this lesson.
Conduct Reviews of the Proposed Content With a Panel of Experts and Revise Based Upon Their Input.
The expert review team consisted of four outside consultants. Dr. Cohen is an internationally recognized expertise on aging in the general population. Dr. Heller is an expert in the area of aging and individuals with DD. Ms. Elliott, a recreation specialist, brings skills and knowledge in the area of recreation and individuals with DD. Dr. Baker is widely published in the area of staff training. This panel of experts reviewed both the content outline and learning objectives and provided input that helped simplify content and break down activities into more achievable units.
Conduct Focus Groups to Evaluate Content and Provide Feedback.
Two focus groups (one with self-advocates and one with support staff) were held on June 28, 2001 at the Statewide DD Inservice in Corvallis , Oregon . Self-advocates were recruited from participants at a day-long session that occurred the day before the inservice started and included people from throughout the state, who lived in a variety of settings and who had a range of cognitive and physical disabilities. The questions asked of this group included their favorite things to do for fun and why, what they got out of it, what they disliked, who chooses, and what they would like to do but have not and why. We found that people enjoyed things that got them out of the house; things anybody can do; and things that were challenging, relaxing, or improved skills. One took it as a challenge if someone said she could not do something; she wanted to prove them wrong. People reported that they did not try new things because they were not brave enough, it was too expensive, or it did not seem interesting.
The support staff in the other focus group worked in a variety of primarily residential settings with people of varying abilities. They named a variety of leisure activities that people participated in and reasons why: socializing/meeting new people, getting out in the community, and physical release. Things that the staff saw as barriers to trying new things included: finances, too much stimulation or unpredictability, potential client/resident embarrassment because they do not think they are good at the activity, staff embarrassment at being seen with someone who drools or has "behaviors," issues of sexuality (hugging strangers), low staffing, people stuck in routines, lack of transportation, lack of assertiveness when want to do something, and staff create expectations' "you don't like that." When asked about examples of creative leisure activities, staff primarily discussed ways to deal with staff and organizational barriers.
Design and Develop Instructional Materials (Scripts, Navigational Flowcharts, and Screen Layouts) and Collaborate with SONY Digital Authoring Services on the Design of the GUI.
The PI, the Media Producer, the Scriptwriter, and the Research Associate collaborated on the expansion and refinement of the instructional modules. The scope and sequence of each module was worked out and drafts of scripts and interactive lessons were written. Concurrently, rough graphic mock-ups for the media and workbook materials were developed. By this time, IRIS Media had developed its own DVD authoring capability and there was no need to out-source this part of the project to Sony Digital Authoring Services as originally envisioned. Therefore all work the DVD GUI was completed in house.
Conduct a Review of the Instructional Materials by the Panel of Experts and to Revise According to their Input.
The developed materials (scripts and workbook drafts) were sent to the panel of experts for final review. Ms. Elliot met with the development team in July, 2001 to give suggestions on the proposed content. She gave some general suggestions like planning for the unforeseen and to emphasize the creative aspects. She also had specific suggestions on the proposed script particularly in the area of "what do I get out of leisure?" She suggested to simplify some of the work sheets and shared a "benefits of leisure worksheet."
The other consultants were contacted by email and provided concrete suggestions for both the script and work sheets. Some of the suggestions that were included in the revised materials were:
- Simplify some of the wording on the worksheets
- Emphasize that people can try out different activities to see if they like them
- Use "my time, my choice, my thing" throughout all the lessons as an anthem
- Do not present long lists of things on the video screen
- Use multiple examples to make your points more clear to the intended audience
- Use the word "fun" a lot to emphasize doing something that I really enjoy.
Produce the Media Elements and Finalize Design of the IMM GUI.
The Media Producer directed the video shooting and managed the post-production, which included editing the materials, finalization of graphics and animations, recording the narration and the other audio, and the development of the music and sound effects. The professional media staff that included a Camera Operator, a Lighting Gaffer, a Location Sound person, a Production Coordinator, a Video Editor, a Sound Designer, a Graphic Artist, an Animator, and a Production Assistant.
Video sequences were shot on the professional DVCAM digital format. As often as possible, people with DD worked as actors and models in the program. Actors were recruited through the local DD network and through Phame Inc. (Pacific Handicapped Artists, Musicians, and Entertainers Inc.) in Portland , Oregon . Actors were also balanced for ethnicity and gender. Video sequences were edited on a Fast 601 digital nonlinear editing system at IRIS. Graphics and animations were produced on Adobe Illustrator, PhotoShop, AfterEffects, and other leading graphic design programs. Narration will be recorded at a professional recording studio. The Sound Designer was responsible for composing music, and mixing all the audio elements. Audio post-production took place at IRIS Media.
Author, Test, and Revise Alpha Program Integrating Media Assets and GUI Design.
The Media Producer encoded media assets to DVD and created navigational links between audio, video, graphics, menu assets, and the GUI.
Beta Test the Prototype Version using the In-house Team, the Expert Panel, and the Focus Group Members.
Under the direction of the PI, development team members conducted tests of the DVD program and minor modifications were made to the interface. Since the simpler DVD interface was used in place of the CD-ROM one, extensive Beta testing became unnecessary.
Design and Evaluate Measures Using Face Validity, Item Difficulty, and Test-Retest Instrument Development Procedure.
Face Validity. Project staff developed 99 items (8 multiple choice, 91 True/False) for the first stage of instrument development. Twelve group home staff participated in an examination of face validity. These people met as a group, answered the questionnaire, and took part in a group discussion of the items.
Item Difficulty. Based on the results and discussion from the face validity group, 45 items were dropped and 55 carried forward for a test of item difficulty (47 True/False, 8 multiple choice). Thirty group home staff participated in a test of item difficulty.
Test-Retest. Based on the results of the item difficulty activities, 36 items were carried forward into the test-retest examination (2 multiple choice, 34 True/False). Fifty-three group home staff took part in test-retest activities with 3 days between test administrations. The test-retest correlation was .74 and a paired-samples t-test revealed no significant change in mean from Time 1 to Time 2.
Conduct a Randomized Controlled Study that Evaluates Knowledge, Skills, and Behavioral Outcomes.
The evaluation involved an experimental test of this product with three main objectives: (a) to test the product's effectiveness at increasing leisure knowledge and skills for the staff to increase leisure opportunities for the residents; (b) to demonstrate that the followed curriculum leads to an expansion of the leisure experiences of older adults with DD living in group home facilities; and (c) to demonstrate that expanding leisure experiences for older adults with DD reduces leisure constraints and increases leisure and life satisfaction. To test these hypotheses, we used a two-group (Intervention vs. No-Treatment Control) pre/post-test design.
Subjects.
A total of 73 direct care group home staff and 75 residents of group homes initially participated as dyads in the evaluation. Two group home staff worked with two residents each. These subjects were from 17 different residential programs throughout the state of Oregon . Thirty-seven dyads were assigned to the Intervention group, and 38 to the Control group. Five residents dropped out of the study between the baseline and final assessment; the final N was 70 dyads, 38 in the Control Group and 32 in the Intervention group.
Staff demographics. 80% of the 73 staff were female; that distribution held across groups (81% of the Intervention group; 77% of the control group). Only 14% of the Intervention group and 19% of the Control group had any educational background in special education. However, over 80% of the staff in both groups reported having over 24 hours of in-service training. 13% of the Intervention group had previous education or training on leisure, while 30% of the Control group had received such training; this difference was not statistically significant. Staff in both groups was primarily non-Hispanic (92% Intervention, 94% Control) and Caucasian (78% Intervention, 83% Control). 45% of these subjects had a high school degree (43% Intervention; 47% Control), and 30% had 1-3 years of college work (35% Intervention; 25% Control).
Resident demographics. 61% of the participating residents were men, 39% were women, and this distribution held within groups. None of the residents were Hispanic and 92% were Caucasian. The average age of participating residents was 39 (s.d. 12.1). Residents had lived in their current placements an average of 5.1 years (s.d. 4.5).
Procedures.
All dyads were assessed at Baseline. The Baseline assessment consisted of a demographics questionnaire (completed by staff for the participating resident), the Test of Leisure Knowledge (completed by staff), the Five Dimensional Life Satisfaction Index (FDLSI) and the Leisure Assessment Inventory (LAI) (completed by resident interview), and the Support Staff Leisure Inventory (SSLI) (completed by staff). Upon completion of the Baseline assessment, dyads in the Intervention Group were given the Leisure Works curriculum, consisting of a video tape and workbook. Approximately 3 weeks after Baseline, a probe of user satisfaction was administered to staff in the Intervention group. Approximately 6 weeks after Baseline, an additional probe of user satisfaction was administered to staff in the Intervention Group and the SSLI was administered to staff in both groups. Approximately 3 months after Baseline, a final assessment was done with both members of the dyad; this assessment consisted of the same instruments administered at Baseline and an additional user satisfaction questionnaire for staff.
Results.
There were no significant differences on any of the demographic variables (age, education, gender, time in residence, etc.) between the two groups.
Although there was a strong trend for staff in the Intervention Group to improve their scores on the Test of Leisure Knowledge more than staff in the Control Group, this difference was not statistically significant. Table 1 presents the means and standard deviations for the two groups.
Table 1 - Test of Leisure Knowledge
|
Pretest
|
Posttest
|
||||
| Mean | s.d. | Mean | s.d. | N | |
| Knowledge about Leisure | |||||
| Intervention | 6.69 | 2.87 | 7.45 | 3.12 | 29 |
| Control | 7.24 | 2.68 | 6.96 | 2.81 | 29 |
|
Leisure Assessment Inventory |
|||||
| Participation | |||||
| Intervention | 20.52 | 6.82 | 19.62 | 10.19 | 21 |
| Control | 19.78 | 9.19 | 21.68 | 10.84 | 28 |
| Preference | |||||
| Intervention | 10.48 | 6.02 | 11.38 | 9.04 | 21 |
| Control | 11.61 | 9.73 | 11.92 | 10.94 | 28 |
| Interest | |||||
| Intervention | 7.95 | 7.21 | 5.38 | 3.41 | 21 |
| Control | 5.75 | 5.63 | 6.07 | 6.53 | 28 |
| Constraints | |||||
| Intervention | 6.31 | 3.01 | 5.94 | 3.07 | 21 |
| Control | 7.04 | 2.49 | 6.65 | 3.45 | 28 |
Table 2 - Five Dimensional Life Satisfaction Index
|
Baseline
|
3-month Followup
|
||||
| Mean | s.d. | Mean | s.d. | N | |
| Living Arrangements Satisfaction | |||||
| Intervention | 1.65 | .42 | 1.701 | .401 | 21 |
| Control | 1.70 | .42 | 1.665 | .429 | 30 |
| Relationship Satisfaction | |||||
| Intervention | 1.41 | .47 | 1.48 | .47 | 21 |
| Control | 1.32 | .41 | 1.42 | .33 | 30 |
| Job Status Satisfaction | |||||
| Intervention | 1.32 | .55 | 1.49 | .42 | 21 |
| Control | 1.36 | .44 | 1.42 | .43 | 30 |
| Health Satisfaction | |||||
| Intervention | 1.11 | .33 | 1.18 | .32 | 21 |
| Control | 1.16 | .51 | 1.12 | .42 | 30 |
| General Happiness | |||||
| Intervention | 1.27 | .45 | 1.29 | .38 | 21 |
| Control | 1.12 | .47 | 1.14 | .48 | 30 |
None of the four scores (Leisure Participation, Preference, Interest and Constraints) from the Leisure Assessment Inventory demonstrated any differential change over time between the two groups. Table 1 presents the means and standard deviations of these scores.
None of the five scores (Living Arrangements, Relationships, Job Status, Health, General Happiness) showed any differential change over time between the two groups. Table 2 presents the means and standard deviations of these scores.
The simple number of leisure activities engaged in by residents in the two groups (as reported on the Support Staff Leisure Inventory) did not show a differential change over time. However, two scores from that instrument did demonstrate significant changes over time between the two groups. The overall frequency of engaging in leisure activities from Baseline to the 3-month follow up increased from 2-3 times a month (2.75) to slightly over once a week (3.10) for residents in the Intervention group, while residents in the Control Group showed no change (Baseline mean 2.99; 3 month mean 2.83 (F(1,64=4.06, p=.04). Similarly, the percentage of new leisure activities engaged in increased from Baseline (11%) to the 3-month follow up (15%) for residents in the Intervention group while residents in the Control group showed a decrease (Baseline 19%; 3 month follow up 9%) (F(1,64)=5.20, p=.026). Table 3 presents the means and standard deviations of these scores.
Table 3 - Materials Use
Staff who used the curriculum reported a high level of satisfaction with the materials. At the 3-month follow up assessment, 88% had used the materials, 79% reported liking the workbook "mostly" or "a lot," and on a scale of 1 to 10, rated the videos an average of 8.2 for realism. They also reported that 68% of the residents they worked with had researched new leisure activities and 50% had started a new leisure activity. On a scale of 1 to 10, they rated the materials 8.2 on ease of use and 7.8 on their relevance to their work with adults with DD. All (100%) said they would recommend the materials to other direct care staff.
Discussion.
The first objective, to test whether the product was effective at increasing staff knowledge about leisure and skills related to increasing leisure opportunities, was not significant. There was a strong trend for the staff in the intervention group in terms of increased knowledge of leisure. Given the size of the sample and length of the study, this is an encouraging result. The second objective was to demonstrate that using the curriculum leads to an expansion of the leisure experiences of older adults with DD living in group home facilities. While the overall number of leisure activities did not change over time, both the frequency and variety of leisure activities increased for the intervention group. We found that residents in group homes were doing a variety of leisure activities even before the intervention began. After using the curriculum, the intervention group went out more frequently for leisure activities and did more new activities. Therefore we accomplished one of the major objectives of the curriculum. The third objective was to demonstrate that expanding leisure experiences for older adults with DD reduces leisure constraints and increases leisure and life satisfaction. We had hoped to see some changes in constraint reduction and leisure/life satisfaction. This was not the case in the present study. Perhaps tracking residents for a longer period of time would have shown some changes on these variables.
One of the limitations of this study has to do with the sample population. This curriculum was designed for adults with mild and moderate levels of retardation. There was no formal screening mechanism in place to insure that residents were of a cognitive level to benefit from the program. We did a rough screen by asking staff to report on the functioning level of the referred clients. Our experiences in the group homes leads us to believe tha
