Promoting Social Competence in Family Child Care Settings

Period covered : 9-1-04 to 8-31-07

Background: Phase I
In Phase I, the training module on Setting up to Support Social Development was developed. We developed an instructional videotape that illustrated concepts with video-vignettes demonstrating examples and non-examples of children and caregivers in family child care settings. The manual provided the caregiver with a summary of concepts demonstrated on the videotape. For the first module we also developed handouts to distribute to parents as well as activity ideas for promoting specific children's social skills. The content of this first module covered: a) description and rationale for an ecological and proactive approach, b) overview of social development and typical social skills for preschool-age children, c) arrangement of physical space, d) selection of materials and toys, and e) setting schedules and routines. The workshop was conducted in an interactive format that included discussions on implementing the ideas presented in the participants' child care homes and activities that provided caregivers with practice of the ideas presented (e.g., designing play areas to facilitate social development, making effective daily schedules). For Phase I, we evaluated the effectiveness of the training module on the child care environment and caregiver practices. Phase I demonstrated feasibility and promising trends for enhancement of the child care environment and increased positive child behavior.

Specific Aims of Phase II
In Phase II, we proposed to develop two more training modules: (a) Proactive Approaches for Guiding Children's Behavior, and (b) Understanding and Dealing with Challenging Behavior. The content of the second module on proactive management practices includes: (a) developing positive relationships with children, (b) developing and teaching clear rules and expectations, (c) gaining children's attention, (d) providing clear directions before an activity begins, (e) monitoring, and (f) providing positive attention and encouragement. The content of the third module on understanding and dealing with problem behavior includes: (a) description of common challenging behaviors, (b) effective strategies for dealing with challenging behavior, (c) understanding the function of recurring problem behavior, and (d) building a support plan for individual children who display ongoing behavioral challenges. These last two modules were also designed for an interactive workshop format with an instructional DVD illustrating concepts with video-vignettes of children and caregivers in family child care settings. A manual was developed for all three modules. This manual includes summary of program content, practice activities, activity ideas for children, and handouts for parents. The last module also includes a step-by-step form for assessing the function of problem behavior and developing an individual behavior support plan.

The objectives for Phase II were to:
1. Develop content material for Module 2: Proactive Approaches for Guiding Children's Behavior and 3: Understanding and Dealing with Challenging Behavior for the video-based instructional curriculum.
2. Get feedback on content material from a focus group of family child care providers.
3. Get expert input from consultants on the content of all materials, revise content of materials, and finalize video scripts, manuals, and handouts.
4. Produce training videos and manual.
5. Evaluate the entire training program (all 3 modules) in a randomized design with two groups: (a) the program delivered in a series of 3 workshops and (b) waitlist control group. We proposed to analyze the effects on:

a. The child care environment (i.e., physical space, materials and toys)
b. Caregiver practices (i.e., promotion of social skills, scheduling, behavior management practices)
c. Overall child social skills and problem behavior as rated by child care providers and observed in the child care setting
d. Consumer satisfaction (i.e., ratings by caregivers of program usefulness).

Product Development
Development of modules 2 and 3. Similar procedures that were used for the development of module 1 during Phase I were used for the development of modules 2 and 3 video-based training program and workbook. This involved collaboration of script development between the Principal Investigator, Julie C. Rusby, Ph.D. and the media developer, Brion Marquez; and focus group and expert consultant feedback.

For module 2: Approaches for Guiding Children's Behavior, the Principal Investigator reviewed and synthesized topic information on effective proactive behavior management practices. In collaboration with the Media and Print Developers Dr. Rusby developed preliminary drafts of the script, manual, and child activity ideas. For module 3: Understanding and Dealing with Challenging Behavior, the Principal Investigator reviewed and synthesized topic information on challenging behavior. She and the media developer collaborated on the development of the Behavior Support Guide: a tool to help family child care providers identify why the challenging behavior may be occurring and how to develop a behavior support plan to encourage alternative positive behavior. Before finalizing the draft of Module 3 content, the Principal Investigator met with, Ph.D. from the University Oregon, an expert on functional assessment and intervention, to discuss preliminary ideas on the content and approach for the topic. Revisions and additions to the draft of Module 3 content were made based on this meeting.

The following guidelines for the video development were established: a) the video presentation would feature scripted vignettes demonstrating examples in family child care settings (documentary footage was used less often in these modules due to the need for specific behavioral examples), b) the video would avoid using a didactic lecture approach, c) the visual material would consist of a variety of home child care set ups with child care providers and children representing diverse ethnic and racial backgrounds, e) the video and manual would be designed for interactive use, providing a number of breaks for interactive opportunities (a viewer pauses the DVD to complete an activity described in the manual), and f) the explanatory language in the narration would be kept simple, but would avoid patronizing care providers. The developers prepared a 30-minute video script for each module and initial drafts of the manual, child activities, workshop activities, and the Behavior Support Guide for focus group review.

Focus group feedback. Feedback was provided on the drafts of the scripts and practice activities for each intervention module in separate focus groups. The Media Developer and Principal Investigator led the focus groups. The Print Developer and a research assistant also attended and took notes.

For module 2 the focus group consisted of 8 family child care providers; 2 were Hispanic, 1 Asian, 1 African American, and 4 Caucasian. One of the focus group participants was male and the rest were female. This focus group gave feedback on the video script and child activities. They also tried out the activity and worksheet on Planning for a New Activity or Toy. For module 3 the focus group consisted of 7 family child care providers; 1 was Hispanic, 1 Asian, and 5 Caucasian; 6 were female and 1 male. During this focus group feedback was given on the script, participants provided examples of challenging behavior problems that they have encountered, and caregivers tried filling out the Behavior Support Guide form and gave feedback on it.

Generally, the feedback on the video script and child activities was positive. Caregivers felt that it was clear, they understood it, and they learned something from it. Suggestions for improvements of the language and simplification of procedures for the activity sheet on "Planning for a New Activity or Toy" and on the Behavior Support Guide were provided. Changes to the scripts and materials were made based on the feedback from these focus groups.

Expert consultant feedback. After each focus group, consultants and the Co-Investigator were sent the revised video-script and draft of the manual (including child activities, workshop activities, and the Behavior Support Guide) for their feedback. Feedback was attained on Module 2 from consultant Sue Norton, co-director of Lane Family Connections (expertise in training and resource development for child care providers) and from Co-Investigator, Ted Taylor, Ph.D. Feedback was attained on Module 3 from consultants Robert Horner, Ph.D. (expert on functional assessment and intervention planning for children with challenging behavior) and Sue Norton, and from co-Investigator Ted Taylor, Ph.D. The helpful suggestions for improvements were made and incorporated into the script, manual, workshop activities, and the Behavior Support Guide.

Production of the video-based training program and manual.
In module 1 (Phase I), we had primarily used a documentary approach to acquire footage that would demonstrate targeted learning objectives (organizing the physical childcare environment, using toys and materials appropriately, developing schedules and routines). For instructional purposes, we supplemented this documentary material with scripted vignettes that provided examples and non-examples of the recommended practices. Because the content of modules 2 and 3 required showing multiple examples of children behavior, we chose to script and direct the action rather than hope to capture it in vivo. To do this, we set up three different childcare settings, cast children who were able to take direction, and patiently filmed the vignettes as scripted. The Principal Investigator was involved in approving the sets, props and talent selection.

The production team, headed by the Media Developer, filmed for five days with a cast that had good ethnic representation. We hired experienced caregivers to assist in caring for the child actors and seeing that they were well fed and occupied with interesting activities. Children worked in four hour blocks so that they wouldn't become exhausted or stressed.

Post-production was carried out at IRIS Media's studio by an Editor and Graphic Animator under the direction of the Media Producer. In addition to dramatized vignettes, program elements included graphics, animations, illustrations, original music, narration, and sound effects. Narration and original music was recorded at an audio studio. The program was edited in-house on a Liquid Silver 601 digital editing system. After final revisions by Dr. Rusby, the program was mastered on the DVCPRO format. A professional duplicator created VHS copies. We encoded a DVD master of the program to make the 30-minute program available on DVD and VHS.

Graphics and illustrations from the program were also used in the manual and collateral materials. These were formatted by the Graphic Designer, then reviewed and revised by the Principal Investigator before printing. The finished materials were transferred to the PDF format for printing using the DocuTech process. The manual, with parent handouts inserted, was printed as a 75-page 8 ½ x 11 bound booklet.

Evaluation Study
Participating family child care providers. Family child care providers were recruited from Lane and Marion Counties in Oregon with collaboration from the local the Child Care Resource Referral Network agencies. A total of 70 caregivers agreed to participate in the study. By the end of the 2-year study, 57 caregivers remained in the study (81% retention). All the participants were female, 73% were Caucasian, 10% were Hispanic/Latino, 4% Asian or Hawaiian/Pacific Islander, 3% American Indian, 1% African-American, and 9% were of other or unknown ethnicity/race. Participants represented a variety of age ranges; 17% were under 30 years old, 32% were between 30 and 39, 26% between 40-49, and 25% were 50 or older.

The education level of participants also varied; 29% only had a High School diploma or GED, 43% had some college, 6% had an AA degree, 14% had a BA or BS degree, and 8% had some graduate level education. The average number of years of experience was 9, ranging from 6 months to 30 years. Seventy-three percent provided child care in their home by themselves. The average number of children in their care was 8, with an average of 4 being preschool-age.

Assessment procedures and completion. Assessments included pre and post caregiver questionnaires in which child care providers answered questions about demographics and the strategies they used to promote child development and manage children's behavior in their child care homes. Three site visits per phase also occurred in which research assistants conducted direct observations of caregiver practices and children's behavior and completed ratings of the child care environment, caregiver practices and child behavior. By baseline data collection 65 caregivers continued participation in the study and 100% completed the baseline questionnaires. The first site visit assessments were completed on 98% of participating child care homes in baseline, the second on 94%, and the third on 92%. By the post assessment phase, 63 caregivers were participating in the study (2 caregivers had dropped out of the study because they were closing down their child care business). 89% of participating caregivers completed the post questionnaire and site visit assessments were conducted in 83% of participating child care homes.

Reliability of measures. During each site visit caregiver and children's behavior were observed for 20 minutes during free play. The frequency of caregiver's positive attention and negative attention for children's behavior were tallied and a "rate per minute" was calculated (total frequency/# of minutes). Also a rate-per-minute composite score of children's overall problem behavior (noncompliance, physical aggression, and potentially dangerous behavior) was computed. Good inter-rater reliability was achieved for these observation measures; the intraclass correlation coefficient (ICC) for caregiver positive attention was .94 at baseline and .95 at post, for caregiver negative attention was .86 at baseline and .91 at post, and for child problem behavior was .90 at baseline and .94 at post.

Observer ratings of child care providers' use of effective behavior management practices and of children's positive behavior (e.g., compliance, cooperation, appropriately expresses feelings, etc.) were assessed using a 5-point Likert scale and composite scores were computed by averaging the items. The reliability (Cronbach's Alpha) for the observer rating of caregivers' effective behavior management was .85 at baseline and .86 at post, and of children's positive behavior was .95 at baseline and .85 at post. Observers also completed a checklist rating of the child care environment, the Child Care Ecology Checklist (CCEC). The reliability of composite scores for the CCEC were as follows; .91 baseline and .89 post for Enriched Environment, .88 baseline and .90 post for Organized Environment, .85 and .86 for Planned Activities, and .82 baseline and .84 post for Caregiver Monitoring.

Associations of baseline measures of child care and child behavior. The extent to which the child care environment and caregiver practices were correlated with child behavior at baseline was tested. The ratio of observed positive attention to negative attention and observer ratings of effective behavior management were negatively associated with children's problem behavior and positively associated with children's positive behavior. Measures of the child care environment and caregiver monitoring were related to children's positive behavior, but not significantly related to problem behavior (See Table 1).

Intervention procedures and participation. Caregivers were randomized into the intervention or waitlist control groups after the collection of baseline data. Caregivers in the immediate intervention group were invited to participate in the set of three 3-hour workshops that were scheduled about 2 weeks apart. Workshops were conducted in the caregivers' local county. Caregivers who missed a given workshop were invited to attend the workshop on the alternative day in their community or to view the video and do the practice activities in the manual at home before the next workshop session. For caregivers who did a module through home study, the interventionist checked their activity work in their manual, answered questions, and checked for understanding of the content at the next workshop.

Good participation rates were achieved; 90% of the caregivers were exposed to module 1 (71% attended the workshop and 19% completed the home study), 90% received module 2 (81% attended the workshop and 9% did home study), and 87% received module 3 (80% attended the workshop and 7% did home study). A total of 97% of the caregivers in the intervention group received some exposure to intervention and 90% received at least two modules of content.

Consumer satisfaction: Caregiver reports of program usefulness and barriers. A large majority of the child care providers reported that the workshops were useful. For modules 1 and 2, 90% of the caregivers said that they learned "some" to "very many" new things and practical skills. For module 3, 100% of the caregivers said that they learned "some" to "very many" new things and practical skills. A large majority of caregivers reported that they planned to make "some" to "very many" new changes or try new things based on the workshops (86% for module 1, 82% for module 2, and 92% for module 3). Caregiver reports of program usefulness are shown in Table 2.

Knowledge test on the Behavior Support Guide. This knowledge test was developed to assess whether caregivers understood the concepts about the function of problem behavior and developing effective behavior support plans presented in the third workshop module. During the workshop caregivers completed a sample Behavior Support Guide based on a video example of a child's problem behavior in a child care home. These were scored for how well each caregiver defined the problem behavior and its function, and developed a support plan. The average score on this knowledge test was 80%, the median was 83% and the range was 46% to 96%. Only one caregiver scored below 60% and four scored between 60% and 70%.

Procedures for analyzing intervention outcomes. For data that were collected once at pre- and once at post-intervention (i.e., the caregiver reports and the CCEC), multivariate analysis of variance to identify effects by group (intervention vs. control), time (baseline vs. post), and interaction between group and time were conducted. For data in which three data points were collected at each phase (the direct observations and observer ratings) random coefficients analysis (RCA) were conducted providing estimates of normative intercept (level) and slope across all time points for control participants and pre for those in intervention. The RCA then estimated the change in level and in slope associated with intervention across post assessments. RCA models with the same slope for both intervention and control group data and models with separate slopes for intervention and control groups at baseline were tested, and the best fitting model was selected using the Akaike Information Criterion (AIC).

Outcomes on the child care environment. No significant pre-post differences were found between the intervention and the wait-list control groups on the Child Care Ecology Checklist measures of enriched environment, organized environment, or planned activities. Pre-post increases in self-reported improvements in environmental enrichment (physical layout and toy variety) were found in both groups (p = .01); however, greater increases were found in the intervention group (although not statistically significant a small-to-medium effect size was found, d = .38).

Outcomes on caregiver monitoring and effective behavior management practices. Caregiver monitoring (from the pre-post measure of the CCEC) was found to decrease in both groups over time; however, the decrease was smaller for the intervention group (group X time F = 3.06, p = .09). A medium effect size was found for this difference (Cohen's d = .51).

Random coefficients analyses were performed on observer ratings of caregiver's effective behavior management and on observed positive attention. Significant improvements were found for observer ratings of caregiver's effective behavior management for the intervention group (intercept t = 2.82, p = .006). The level-change represents a medium-to-large effect size (partial r = .34, d = .72). Although the intervention group showed greater decreasing trends in effective behavior management over time than the control group, the intervention and control group slopes at post did not significantly differ (see figure 1). Rates of observed positive attention especially increased at immediate post; however, decreases in slope for positive attention were greater at post for the intervention group compared to the wait-list control (slope t = -1.68, p = .094). Figure 2 depicts the observed rates of positive attention.

Effects of intervention exposure and of years of experience providing child care.
The hours of intervention exposure were calculated (3 hours for each workshop attended and 2 hours for each home study completed) ranging from 0 hours (no exposure) to 9 hours. Intervention exposure had no mediating effect on caregivers' behavior management practices or children's problem behavior. Mediating effects of years of experience providing child care were also examined. Experience was associated with effective behavior management practices (the greater experience the more effective practices were used; partial r = .25, t = 2.04, p = .046), but experience had no moderator effect. Child care experience also had no moderating effect on children's problem behavior.

Summary of results.
In summary, this Phase II study demonstrated that the intervention was highly appealing to family child care providers. This is demonstrated both by the attendance rates and by their feedback on usefulness. The most salient findings of the study were that caregivers in the intervention group improved their observed effective behavior management practices and those improvements were associated with reductions in children's problem behavior. In addition, a medium effect size was found for caregiver monitoring and a small effect size for positive attention.

This study also detected evidence that the improvements in caregiver practices and decreased child problem behavior began to fade over the time of the follow-up assessments. This suggests that further research is warranted to evaluate strategies to maintain intervention gains, such as providing individualized consultation and/or subsequent "booster" sessions. This research highlights the importance of specific, proactive caregiver practices in reducing problem behaviors in young children, and that these practices can be taught and carried out by family child care providers. A large-scale efficacy study that evaluates longitudinal outcomes for children is also needed to substantiate these preliminary findings. See Table 3, Inclusion Enrollment Report.

Publications

The paper depicting the Phase II outcomes is currently under review.

Rusby, J.C., Smolkowski, K., Marquez, B., & Taylor, T.K. (submitted May 2007) Proactive approaches for enhancing children?s social development in child care homes. Submitted to Early Childhood Research Quarterly.

Patents/Copyrights and Materials
The program, CARESCAPES: Promoting Children?s Social Competence, has been copyrighted in 2007 by IRIS Media, Inc.

Product and Intended Use
The 3-part Carescapes DVD and book show family child care providers how to nurture healthy social skills and how to provide children with opportunities that enhance social development. IRIS is taking steps to market the product to family child care providers, as well as early childhood teachers, consultants and researchers. It also will be of interest to those who train early interventionists and child care providers, such as child resource and referral agencies and college faculty.

Section 1: Setting up the Environment to Support Social Development (23 minutes)

• Arranging physical space
• Choosing toys and materials
• Scheduling the day's activities

Section 2: Guiding Children's Behavior (23 minutes)

• Creating a positive relationship with children
• Planningahead to promote positive behavior
• Developing and teaching rules and expectations
• Providing clear directions
• Monitoring behavior
• Providingpositive attention and encouragement

Section 3: Understanding and Dealing with Challenging Behavior (26 minutes)

• Responding appropriately to challenging behavior
• Knowing the three key goals of behavior
• Teaching positive behavior
• Using the Behavior Support Guide to understand challenging behavior and to encourage positive behavior

The 75-page, spiral-bound book includes content covered on the DVD, ideas for cooperative play activities, parent handouts on social development, and the Behavior Support Guide forms.

Current status of product.
Product has been recently commercialized. The product has been commercially packaged. The product is listed on IRIS Media's e-commerce site (www.lookiris.com), and has been included in IRIS Media's product catalog.

Commercial name of product.
CARESCAPES: Promoting Children's Social Competence -A Guide for Family Child Care Providers.