Research

• Current Projects
• Past Projects
• Principal Investigators

IRIS Media Products

• K-12 Professional Development
• Developmental Disabilities
• Parenting & Parent Involvement

Research > Projects

Incredible Infants - Phase I Final Report

Increasing Access to Early Parenting Education and Support
Phase I, Grant #1R41HD051184-01A1
Carol W. Metzler, Ph.D., Principal Investigator - Phase I Project Period: 5/5/06 - 4/30/08

Specific Aims of Phase I
The central aim of this Phase I project was to create a program that would reach out to parents of infants via an email delivery system with small manageable nuggets of parenting information and support. This effort had the ultimate goal of achieving an engaging self-directed learning experience that imparts knowledge, motivation, and skills using compelling storylines, modeling situations, small self-contained lessons, interactive learning exercises (quizzes, branching situational vignettes, etc.), attractive presentation, and intuitive organization.

Our specific aims for Phase I were to: (1) develop a 3-month prototype series for parents of infants, (2) conduct a formative evaluation of the prototype with parents of infants, and (3) conduct a feasibility test of the program through evaluating efficacy, consumer satisfaction, and usage among a sample of parents of infants. These aims were accomplished. In Phase I, we developed and produced the Incredible Infants Online prototype for parents of infants, conducted a formative evaluation and feasibility test, and established reasonable proof of concept of the entire program's design. We accomplished this in six steps: (1) gathered source material and created the program outline, (2) conducted focus groups of parents of infants, (3) designed the program in collaboration with Content and Scientific Consultants, (4) produced media assets and program web applica-tions, (5) implemented the technology and tested its usability, and (6) conducted an outcome evaluation. In the following sections, steps in the development and evaluation of the Phase I prototype are described.

Step 1 (Aim 1): Gathering of Source Material and Creation of Program Outline
The Development Team (Principal Investigator, Media Developer, Content Developer, Instructional Designer, and Content Consultants) gathered all source materials from the Parenting: The First 3 Years, Incredible Infants series group-based curriculum, determined learning objectives for the Incredible Infants Online program, re-organized the source materials to make them suitable for self-administered online delivery, drafted a program outline, and created a design for program delivery. Although the program is primarily directed at first-time parents of young infants (under 4 months old), the goal was to create a program that would be applicable and useful to parents at anytime in the first year of their baby's life.

The team concluded that an effective adaptation of the F3Y curriculum for online application would need to be series of online lessons that would include: (a) originally produced video segments presenting in-depth content and skill demonstrations, (b) interactive exercises to practice and develop skills, (c) printable resources for practicing and personalizing the content, (d) knowledge checks that allow parents to self-assess knowledge gains, (e) an online "community of support" with other parents of infants, and (f) access to an archive of the online lessons. An important feature of the program would be the email-based instructional tool (EdClips) that would deliver a series of automatically generated unique email messages to the parents over the length of the 11½-week program. The Development Team's goal was that each EdClip email message (34 total) would be an attractively presented, manageable dose of parent education material, and that embedded in the body of each message would be links to additional content on the Incredible Infants Online website. Together, the EdClips service and the Incredible Infants Online website would offer a complete package for self-directed learning and social support. The goal was also that all program components would be usable by anyone with access to a computer, Internet service (including those with low bandwidth), and a valid email address.

Step 2 (Aim 2): Focus Groups of Parents of Infants
After the structure and format of the program had been outlined, 17 parents of infants (9 mothers, 7 fathers) were brought in for a focus group to gather feedback about the general framework, structure, and content for the program. Focus group participants were recruited from the Eugene/Springfield community and represented a range of income and educational levels; 13 (76%) were Caucasian, 2 (12%) were Pacific Islander, 1 (6%) was American Indian, and 1 (6%) was "other." Parents' ages ranged from 22 to 40, and babies' ages ranged from 1 month to 12 months old. Most parents were technologically sophisticated; 82% used email frequently either at home or at work and 82% frequently used the internet to gather information. These parents reported that they used email and the internet at all times of the day or evening.

The parents in the focus group had very favorable responses to the notion of email-driven online delivery of parenting education and support. The group agreed that parents receive quite a bit of support during pregnancy, but that much less support is provided for the transition to the "reality of having a baby," and that email and web delivery would be very convenient and accessible. Focus group parents expressed enthusiasm for being able to access convenient, up-to-date, and research-based information on common topics, developmental information, good exemplars (e.g. of a good routine or schedule for a baby), practical tips (e.g. for coping with feelings of isolation, finding child care, adjusting to a new family structure), and for being able to compare experiences and build a sense of community with other parents. The focus group also gave valuable feedback regarding specific technical features, such as manageable size of email files, ability to access all program materials at any time (looking forward and backward in the curriculum to suit their needs), download times, and issues of content blocking by some internet service providers. Finally, the focus group gave input on the general look and feel of the website and the materials. Participants were paid $50.

Because of the importance of the EdClips in engaging the parents in the program, we also obtained feedback from parents on two prototype program EdClips, in order to get input from target parents on desirable format and structure for the emails. Eighteen parents were recruited through postings to online classifieds and parenting forums and then emailed sample EdClips; they were then asked to complete a comment form which they emailed back to us. These 18 individuals (17 mothers, 1 father) were selectively chosen to represent a range of occupations and races/ethnicities (44% Caucasian, 28% Asian, 17% Latino, and 11% African-American). They viewed four different prototype versions of each of two different EdClips, ranked these messages by appeal, motivational value, usefulness, and informational value, and provided comments on the clarity, value, and appeal of the EdClips. Parents indicated a strong preference for relatively short, attractive, bulleted EdClips that provided specific information (as opposed to longer, more story-oriented messages). These parents were paid $30 for participating in this electronic focus group.

Step 3 (Aim 1): Design program in collaboration with Content and Scientific Consultants
Overview of the Program
Table 1 lists the themes and topics addressed in the Incredible Infants Online program, distributed across 34 EdClip emails in seven modules, as well as each EdClip's link to the online videos, interactive exercises, and/or printable resources on the website. Every EdClip has a link to the discussion forum. As described above in the Significance section, the content for the program is built upon existing research on the importance of maternal attentiveness, responsiveness, and reciprocal and mutually rewarding interactions for optimal infant development (e.g., Kaye, 1982; Shaw et al., in press; Shaw et al., 2001). The messages focus on the importance of being responsive to the infants' signals and attentive to their needs, building and nurturing adult relationships, and establishing a safe, secure, and stimulating home environment.

Using the F3Y Incredible Infants group-based curriculum as a guide, the Media Developer, Content Developer, and Principal Investigator created drafts of all program elements. These drafts were reviewed by the Content Consultants and Scientific Consultants, and their feedback was incorporated before finalization.

Designing the User Experience
The Development Team's goal was to provide substantive, evidence-based parenting information as well as social support to first-time parents, in an engaging, convenient, and easily-accessible fashion. As a result, we took care in considering the demands faced by the typical user in accessing the program. Focus group input was instrumental in guiding the design of the user experience. The look and feel of the EdClips and the web resources were designed to be friendly, relaxing, attractive, and relatively uncluttered, with engaging photos of babies and families. In following the focus group recommendations, the Development Team structured the website resources to allow access to all program materials at any time (both previous material and future material).

To participate in the program, parents subscribe to Incredible Infants Online through a standard online registration and login process, involving a username, password, email account, and alias (to ensure confidentiality on the discussion forums). Upon registration, participants begin receiving the automated EdClips three times per week. They are also provided full access to the Program Dashboard - the page where all program components reside. Participants are free to choose the pace at which they want to explore the materials. Some may choose to wait for the EdClips to arrive and explore only the resources linked to each EdClip; others may choose to explore content on the Program Dashboard independently of the EdClips.

The instructional design of this online training is based on evidence-based applications of (a) media engagement techniques (e.g., realism, humor, drama) to bring impact and distinctiveness to the content (e.g., Gordon, 2000; Fletcher, 1989; Harwood & Weissberg, 1987; Olson & Bruner, 1974); (b) modeling and observational learning techniques (e.g., Bandura, 1986; 1977) and interactivity (Joyce & Showers, 1995) to enhance self-efficacy; and (c) principles of effective learning to drive the delivery of instruction (e.g., Engelmann & Colvin, 2006).

Program Components
EdClips. The EdClip, or automated email message, is the "portal" through which users access all program materials on the Incredible Infants Online site (see Figure 1). Upon setting up the user account and activating the program license, the EdClip service automatically begins sending the user's email account a series of email messages at regular intervals. The messages are generated asynchronously so that over time each user receives the complete array of messages in the Infants Online curriculum. Each EdClip email targets an F3Y topic related to infant development (milestones, brain development, and temperament), parenting practices (e.g. responding to infants' cues, promoting brain development, safe environment), parents' self-care, and obtaining social support. Each topic is addressed by 1-;3 emails, for a total of 34 messages. Within each of the modules, parents receive a number of EdClip messages that break down these topics into manageable subtopics which progressively build on each other, providing (a) clear, simple presentation of information, (b) specific suggestions for targeted parenting practices, with clear rationale; and (c) suggestions for practicing that skill to help them apply it in their own family. An important goal of message design was to recognize the unique stresses and demands that new parents face by providing convenient access, absence of jargon, appreciation for their experiences, and opportunities to develop a social support network. Each EdClip is linked to (a) a video exemplar (n= 17), interactive exercise (n= 7), or printable tip sheet (n=11) and/or worksheet (n= 2) and (b) the expert-moderated discussion forum (n= all 34). Sample EdClips are provided in Appendix A-1.

Video. Video presentations
Figure 1. Sample EdClip and web-based program components from Incredible Infants Online were used to explore topics and to model key skills, often providing examples of right and wrong approaches. These video presentations were de-signed to be of relatively short duration (2-4 minutes), but to contain key information and have high emotional appeal to parents. Key goals of the video presentations are to hold parents' attention and motivate them to generalize the content to their situations. The efficacy of video as an observational learning tool has been established (Bandura, 1986; Gordon, 2000), and the website provided a new way of delivering video-based learning to an audience. For low-band-width users wishing to avoid slow download times, a DVD was created that contained the entire video collection in an easily-accessible manner. A DVD of all videos can be found in Appendix A-2.

Interactive exercises. The interactive exercises offer program participants opportunities to rehearse newly learned skills through interactive learning and structured practice activities that required decision-making, branching, and application of concepts. For example, one interactive exercise asks participants to determine the safety of individual toys; another prompts the viewer to interpret babies' communication cues. In addition, an interactive self-assessment provides parents with instant feedback for content review. Sample screen shots of the interactive exercises can be found in Appendix A-3.

Printable tip sheets and worksheets. Printable informational materials were created, such as tip sheets and instructional worksheets that require user interactivity. These printable materials are available online as Adobe Portable Document Files (PDFs). This allows participants to view the materials online or print them out. All printable materials are provided in Appendix A-3.

Discussion forum. The discussion forum was designed to allow new parents to interact with other parents on a variety of topics. A series of 20 topic postings specific to the Infants Online themes and messages provide stimulus for discussion among parents. These topic postings can be found in Appendix A-4. For example, the topic posting for "Soothing Secrets" states "A lot of new parents have a hard time soothing their baby at certain times. What soothing secrets can you share that might help others?" Participants in the discussion forum are also free to post their own topics to create open forum discussion threads or specific topic categories. The discussion forum is "hosted" by the trained parent education staff at Birth to Three's "helpline" call center, who monitor the postings daily for inappropriate content and safety concerns, prompt discussion and interaction, and model reinforcing responses.

Step 4: Produce Media AssetsProduction of media components for the proposed program was realized along separate but coordinated tracks. An iterative design approach that used a "test and make changes" process was used for developing all media assets. The production team, headed by the Media Developer, was composed of a Media Production Manager and Technology Coordinator. The production of media assets (e.g., real-time video, graphics, and animation sequences) wasthe responsibility of the Media Production Manager, who led a team that included a Graphic/Animator Artist, a Video Editor, an Audio Engineer, as well as other production staff (Grips, Production Assistants, etc.). Video footage was filmed in real homes on location. The cast was selected to reflect a balance of ethnic and racial diversity, and was recruited from a talent pool of new parents in Eugene and Portland. Post-production took place at IRIS Media's studio. The Video Editor and Graphic Artist/ Animator handled key post-production tasks using a digital nonlinear workstation to edit video and animated segments. A layered soundtrack of dialog and natural sounds, narration, music, and sound effects was produced. The Principal Investigator and Development Team staff viewed a digital offline version and provided feedback on the presentation prior to finalization. To prepare the interactive web application, video sequences were encoded as Flash Video (FLV) file formats. The interactive componentswere the responsibility of the Instructional Designer and

Technology Coordinator and used standard web design tools. Interactivity was handled through Adobe Flash Action Script programming.

Text materials such as the EdClips and the printables underwent a similar iterative development process. Once the content was assured, these materials were formatted by the Graphic Artist and design elements (banners, photos, illustrations, etc.) were added. Prior to finalization, these were reviewed by the Development Team and final corrections were made. At this point, the EdClips and printables were uploaded to IRIS Media's IRISedOnline instructional website platform using the program's authoring tools.

Step 5: Technology Implementation and Usability Testing
Technology implementation. Program components (EdClips, video, interactive flash, static HTML, printables, discussion forums, etc.) were loaded into the IRISedOnline website platform using a custom-designed content management system (CMS). This custom CMS allows project staff to easily update program components during usability, track user access, and manage user licenses. The CMS also allows research staff to directly access the session reports for users and track progress by users through the program components.

Usability testing. Once program prototypes were completed, project staff conducted a usability test of the online program and assessment materials to ensure user-friendly navigation. Virzi (1990, 1992) has shown that five participants can uncover approximately 80% of an application's usability problems. We recruited five participants in the Eugene-Springfield area and exposed them to a condensed version of the program over the course of two weeks. These usability testers were asked to fully explore the program and all of its features. They then were asked to complete the Questionnaire for User Interface Satisfaction (QUIS) 7.0 (Chin, Diehl, & Norman, 1988) and a detailed consumer satisfaction questionnaire, and were asked to comment freely about their experience. Usability testers provided extensive feedback about program use, and were paid $50.

The usability testing was successful in identifying usability issues that were corrected before intervention began. Usability testers identified website login complexity as an issue. To fix this, we programmed the application to remember user passwords so that the user could easily return to the site through the links on the EdClip emails. Additionally, we changed the required password from 8 characters to 6 characters minimum. Site clutter was noted by some testers, so initial view and layout of program components on the Program Dashboard was modified to present modules and components more clearly based on the activity the user was currently involved in. Some users were using wireless internet connections and experienced delays in starting and stopping video playback. To correct this we made adjustments to Flash Video Playback by creating a bandwidth checker in the code for the files that calculates how long the video needs to buffer based on the user's internet connection speed and the length of the video segment. This allowed for seamless playback and increased usability for video playback on slower internet connections.

Step 6 (Aim 3) Phase I Outcome Evaluation - Methods
Design. The Incredible Infants Online program was evaluated in a randomized controlled pretest-posttest design. Parents meeting eligibility criteria (see below) were assessed at baseline (T1) and randomized to immediate exposure to Incredible Infants Online (Treatment Condition; TX) or to a Waitlist Control (WC) condition. All parents were then assessed again approximately 3 months later at an immediate post-test (T2). After WC parents completed their post-test assessments, they received access to the Incredible Infants Online program. Due to the brief Phase I timeline and budget constraints, the WC group did not complete an additional assessment after their participation in the program.

Participants and procedures. A number of pediatricians in Eugene, Oregon, were provided with study recruitment materials and were asked to refer to the study mothers of age-eligible infants who they believed would benefit from the Incredible Infants Online program. Despite repeated follow ups, recruitment from pediatric clinics was very slow. Because of time and budget constraints, the clinic recruitment strategy was supplemented with an online recruitment strategy. Advertisements were posted on online parenting forums and on online classifieds in several metropolitan areas around the country (e.g. Atlanta, Phoenix, Washington DC, Los Angeles). Interested parents were directed to the study website, which provided detailed information about the study and a link to the online screener for the study (provided through an online assessment tool, QuestionPro). To be eligible for the study, participants needed to: (a) be the biological or adoptive mother of a first-born infant age 0-4 months; (b) have custody of that infant, (c) be English-speaking; (d) have high-speed access to the Internet; (e) report on the screener that they are experiencing feelings of stress, uncertainty, lack of self-confidence in their parenting, and/or social isolation at least "sometimes"; and (f) not otherwise be receiving parenting support services (home visitation, parenting groups). Those who qualified for the study were invited to participate and provided a link to the study consent. Consenting mothers then completed the T1 questionnaire online using the QuestionPro online assessment tool through a secure link. Following completion of their T1, mothers were randomized to the TX or WC condition.

The final N for the study was 65 mothers. In all, 667 individuals completed the screener. A total of 73 were qualified for the study (11% of those screened), and 65 of these (89% of those qualified) enrolled in the study and completed T1. They were then randomized to TX (n= 34) or WC (n= 31).

Participating mothers ranged in age from 18 to 41 years old. Sixty-six percent were Caucasian, 9% Hispanic, 9% African-American, and 15% Asian. They represented a range of employment status (42% worked full-time, 32% worked part-time, and 22% were not working by choice), and educational levels (14% had completed high school, 42% had completed some college, and 44% had completed college or beyond). With regard to annual income, 12% earned less than $30,000 per year, 20% between $30,000 and $50,000, 38% between $50,000 and $70,000, and 30% over $70,000. Most mothers were married (86%) or living with their partner (8%); 6% were single parents. The babies of participating mothers ranged in age at T1 from 1 week to 4 months old at T1; 46% were boys and 54% were girls.

Following completion of the 11½-week program (TX) or 3 months after their T1 (WC), mothers were sent an email asking them to complete the online T2 questionnaire, and were provided a link to the questionnaire. The T2 questionnaire was completed by 56 (86%) of these 65 participants. Questionnaires took approximately 30 minutes to complete, for which participants were paid $30 at each timepoint (those who had not completed the T2 questionnaire after several reminders, however, were offered an additional $5.00 as an extra incentive.).

Because the Incredible Infants Online program was designed for parents of babies at any time in their first year and we tested it only on a narrower sample of parents of infants 0-4 months old, we wished to obtain consumer satisfaction data from parents of older babies as well, in order to learn about their experience with the program. Thus, we recruited 7 mothers of babies 4-12 months old, and asked them to participate in the program and complete only the T2 consumer satisfaction and demographic questionnaires.

In addition, because we had created a DVD option for those with low-bandwidth internet access, we were prepared to allow five individuals who reported on the screener that they had low-bandwidth internet access, but met all other eligibility criteria, to participate in the program and then ask them to complete the T2 consumer satisfaction and demographic questionnaires. No one met these qualifications, however.

Measures. Parenting skills were measured with a Video Analog Test (VAT) created in Phase I of this project; parents viewed a series of 10 video vignettes depicting common parenting situations and were asked to identify the problem and the appropriate strategy for handling that situation, among multiple choices. Responses were scored on percent correct. In developing the VAT measure, a draft of the VAT was put before a focus group of 18 mothers of infants to gauge clarity and difficultness of the items, and the measure was refined on the basis of that information. Social support was measured with a subset of items from the Social Support for Parenting Scale (adapted from Barrera et al., 1981 and Telleen, 1985), in which mothers reported, on a 7-point scale, the degree to which they (a) had desired advice and information about raising a baby in the past month, (b) received helpful advice and information on raising a baby during that time, and (c) were satisfied with that advice and information. "Met need for advice" was measured by calculating the difference between advice desired and helpful advice received (lower score= met need). Parenting self-efficacy was measured by asking parents to rate, on a 7-point scale, their confidence in their knowledge of how children grow and develop. Pleasure in parenting was measured with the 10-item Pleasure in Parenting Schedule (Fagot, 1995), which asks parents to rate, on a 5-point scale, their like or dislike of a variety of routine childrearing tasks.

Results
Program Use. Of those randomized to TX, 82% (n= 28) activated their IRISedOnline license to receive the program; these individuals received all program EdClips at a minimum. Of these, 93% (n= 26) visited the program website, and 14 (50%) engaged in the program for at least 20 minutes. The mean number of visits to the website was 6.1 (range= 1-31), and the mean time of program use across all visits was 41.8 minutes (range= 5-145). With respect to the components viewed, 88% viewed at least one video (M= 4.5, range= 0-15), 73% engaged in at least one interactive exercise (M= 2.0, range = 0-7), and 62% viewed at least one printable form (M= 2.2, range= 0-10). Seventy-seven percent of the participants reported visiting the discussion forum, although many fewer were active in actually posting comments (23%). Thirty-one percent of parents reported that they accessed the program exclusively through the EdClips, 48% mostly through the EdClips but some directly from the website, 17% equally via EdClips and the website, and 4% directly from the website only.

Outcomes. To examine the efficacy of the program, the TX participants who engaged in the program for 20 minutes or more and completed the T2 assessment (n= 11) were compared to the control participants who completed the T2 assessment (n= 26). The two groups did not differ with respect to pretest characteristics or exposure to other media-related parenting information or professional parenting services received. Given the limited power of this small feasibility trial, effect sizes rather than significance levels are used to evaluate the program effects. Analysis of covariance (ANCOVA) was conducted on the following posttest outcome measures in which the pretest outcome measures were included as covariates: VAT parenting skills, supportive advice received, satisfaction with supportive advice, met need for advice, parenting self-efficacy, and pleasure in parenting. An overall multivariate model was tested, followed by six univariate models. The multivariate effect size was large; the active TX participants were found to have greater gains compared to the WC participants, F (6, 19)= 2.29, p= .078, eta-square= .420, Cohen's d= 1.70. As can been seen in Table 2, large effect sizes were obtained for the VAT parenting skills (eta-square= .130, d= .83) and parenting support received (eta-square= .145, d= .91). A moderately large effect was obtained for satisfaction with parenting support (eta-square=.117, d=.77). Medium-sized effects were obtained for parenting self-efficacy (eta-square= .064, d= .57), pleasure in childrearing (eta-square= .058, d= .54), and met need for advice (eta-square= .044, d= .47). Thus, these results provide support for the efficacy of the Incredible Infants Online program.

continued on page 2 ...

Related Principal Investigators

• Brion Marquez