continued from previous page...

 

Increasing Access to Early Parenting Education and Support
Phase I, Grant #1R41HD051184-01A1
Carol W. Metzler, Ph.D., Principal Investigator - Phase I Project Period: 5/5/06 -  4/30/08

Results
Program Use. Of those randomized to TX, 82% (n= 28) activated their IRISedOnline license to receive the program; these individuals received all program EdClips at a minimum. Of these, 93% (n= 26) visited the program website, and 14 (50%) engaged in the program for at least 20 minutes. The mean number of visits to the website was 6.1 (range= 1-31), and the mean time of program use across all visits was 41.8 minutes (range= 5-145). With respect to the components viewed, 88% viewed at least one video (M= 4.5, range= 0-15), 73% engaged in at least one interactive exercise (M= 2.0, range = 0-7), and 62% viewed at least one printable form (M= 2.2, range= 0-10). Seventy-seven percent of the participants reported visiting the discussion forum, although many fewer were active in actually posting comments (23%). Thirty-one percent of parents reported that they accessed the program exclusively through the EdClips, 48% mostly through the EdClips but some directly from the website, 17% equally via EdClips and the website, and 4% directly from the website only.

Outcomes. To examine the efficacy of the program, the TX participants who engaged in the program for 20 minutes or more and completed the T2 assessment (n= 11) were compared to the control participants who completed the T2 assessment (n= 26). The two groups did not differ with respect to pretest characteristics or exposure to other media-related parenting information or professional parenting services received. Given the limited power of this small feasibility trial, effect sizes rather than significance levels are used to evaluate the program effects. Analysis of covariance (ANCOVA) was conducted on the following posttest outcome measures in which the pretest outcome measures were included as covariates: VAT parenting skills, supportive advice received, satisfaction with supportive advice, met need for advice, parenting self-efficacy, and pleasure in parenting. An overall multivariate model was tested, followed by six univariate models. The multivariate effect size was large; the active TX participants were found to have greater gains compared to the WC participants, F (6, 19)= 2.29, p= .078, eta-square= .420, Cohen's d= 1.70. As can been seen in Table 2, large effect sizes were obtained for the VAT parenting skills (eta-square= .130, d= .83) and parenting support received (eta-square= .145, d= .91). A moderately large effect was obtained for satisfaction with parenting support (eta-square=.117, d=.77). Medium-sized effects were obtained for parenting self-efficacy (eta-square= .064, d= .57), pleasure in childrearing (eta-square= .058, d= .54), and met need for advice (eta-square= .044, d= .47). Thus, these results provide support for the efficacy of the Incredible Infants Online program.

Table 2. Pre-Post Descriptive Statistics and ANCOVA Results for the Outcome Measures
Outcome measure/ Condition	Pretest		Posttest			Condition Effect
	M	SD	M	SD	Adj M	F-test	p-value	Eta2
VAT parenting skills						5.07	.031*	.130
Treatment	0.47	0.11	0.66	0.16	0.66
Control	0.49	0.13	0.56	0.14	0.56
Supportive advice received						5.62	.024*	.145
Treatment	3.40	1.17	3.70	1.34	3.69
Control	3.31	1.62	2.69	1.09	2.69
Satisfaction with supportive advice						4.24	.048*	.117
Treatment	5.46	1.70	6.46	1.21	6.54
Control	5.71	1.37	5.71	1.63	5.63
Met need for advice						1.51	.228	.044
Treatment	0.50	1.18	-0.10	0.74	-0.09
Control	0.58	1.03	0.23	0.71	0.22
Parenting self-efficacy						2.34	.135	.064
Treatment	5.00	1.67	6.09	0.83	6.09
Control	4.96	1.00	5.62	0.90	5.62
Pleasure in parenting						1.96	.171	.058
Treatment	4.43	0.46	4.49	0.37	4.42
Control	4.18	0.48	4.17	0.45	4.24
Note. M = Mean, SD = Standard Deviation, Adj = Adjusted. * p<.05

Consumer Satisfaction. Consumer satisfaction data for all 28 participants who activated their program license suggests that the Incredible Infants Online program was enthusiastically received by these mothers. The majority (87%) reported that the Incredible Infants Online program "quite a bit" to "very much" met their needs, and 83% indicated that the program addressed topics important to themselves and their families. Ninety-one percent of mothers found the program "quite a bit" to "very" interesting and entertaining, and 87% would recommend it to other parents. Comments included, "It was a very good resource for the first-time parent who is all about learning and becoming the best they can be," "it's a wonderful program," and "loved it." A number of parents also expressed that the program helped them see that they are not alone: "No matter how alone you may feel there are plenty of parents out there feeling the exact same way."

Because the EdClips are designed to be the portal through which parents are drawn into the web-based program components, it is important that the EdClips be engaging and interesting. Eighty-seven percent of mothers were "quite" to "very" satisfied with the EdClips; 87% found them interesting and useful, and 96% said that they trusted the parenting advice given in the EdClips. Comments included, "The emails... got me fired up to view the next segment," and "I really loved the emails and looked forward to getting them."

It is also critically important that the video sequences are realistic, containing situations that parents can relate to. Of those who viewed videos, 80% were "quite a bit" to "very" satisfied with the videos, 85% found them useful and interesting, and 85% said that they were realistic. Comments included that the videos were helpful and informative, that they contained "realistic situations with real people... relevant to me and similar to situations I encounter," and that "they gave me a new perspective on how to deal with things."

Additional program components were also well-received: Of those who viewed interactive exercises and printable materials, 88% and 77% were "quite" to "very" satisfied with those components, respectively. Many parents commented that the interactive exercises were fun and gave them good skill practice, and that the printables were quite informative. Seventy-eight percent of mothers who visited the discussion forum said that they found the interactions "quite" to "very" interesting and useful. A number of parents noted that fewer people actually posted in the forum than they would have liked, but no one commented negatively about the forum itself. The limited number of people who posted in the forum may reflect the limitations that come with a small sample size. One mother commented that she felt satisfied with the level of involvement, however: "I was expecting not very many people to be involved, but was rather happy when I saw many parents like myself. It felt good to have a way to talk, and voice my feelings and opinions to others like myself."

Parents were also impressed with the convenience of the program. They appreciated "the flexibility of being able to do it when time permitted," that "it was great and convenient to receive information in my inbox," and that the program was "something I could do at my leisure, in my own time, in my own home."

Consumer satisfaction among mothers of older babies. A group of 7 parents with babies 4-12 months old also participated in the Infants Online program and provided consumer satisfaction data to allow us to gauge the extent to which the program also appealed to parents of older babies. This group was also enthusiastic about the program: 86% reported that the program "quite a bit" to "very much" met their needs, was interesting and entertaining, and addressed topics that were important to them and their family. One parent commented, "The program was awesome and so easy to understand. I'm sad to see it go." All (100%) of these mothers were "quite a bit" to "very" satisfied with the videos, with 100% reporting that the videos were useful and interesting, and 80% reporting that they were realistic. Moreover, 86% of mothers were "quite a bit" to "very" satisfied with the EdClips, 100% with the interactive exercises, and 80% with the printable information. Older baby mothers noted that the discussion forum was "a great way to communicate with others," and "interesting, but... wished more would have participated."

Conclusions and Lessons Learned
Overall Conclusions
In Phase I, we accomplished our two primary goals: (a) we developed an online parenting education and support program that combines automated email, video streaming, and interactive web resources to reach out to new parents with high-quality, science-based parenting education and support; and (b) we evaluated the feasibility of the program on a sample of parents. We demonstrated that we could produce materials suitable and attractive to parents of infants; in fact, we obtained very positive consumer satisfaction feedback from parents of both younger and older babies. Parents expressed enthusiasm for the attractiveness, realism, relevance, and usefulness of the program. Furthermore, although our sample size was relatively small, we obtained large effects among active users of the program on parenting skills and supportive advice received; moderately large effects on satisfaction with that advice; and moderate effects on parenting self-efficacy, pleasure in parenting, and met need for supportive advice. Parents indicated that they learned "valuable tools on parenting," "how to interact with my child better," "how to be a better and more patient mom," that "giving babies attention is not going to spoil them," and that "my baby isn't crying to annoy me, she is crying cause she needs something." Indeed, the development of these materials has motivated Kaiser Permanante Northwest, Providence Health and Services, and Oregon

Healthy Start to get involved in a large-scale implementation and test of the F3Y Online program (described in detail in Methods).

We believe that these results demonstrate feasibility of the concept and provide ample justification for continuing with development. Our Phase I experience points to a number of ways in which we can strengthen the program and the evaluation strategy for Phase II, however, with regard to increasing participation in the program, enhancing program features, and improving recruitment and engagement in the study. These lessons learned are described below and are all incorporated into the Phase II workplan and described in greater detail in the Methods section.

Participation in the Program
Improve the navigability of the website. Although users reported high satisfaction with the program elements, program participation was clearly less than hoped for. Because some participants spent little time on the website at all, and also because a proportion of participants accessed the program materials by navigating around the website rather than following email links to specific features of the site, we will take a number of steps to improve the engagement value and navigability of the website, and give participants more guidance in their experience of the website. These steps include providing more general orientation to the site; providing more explanatory and engaging labels for the features; providing a more structured, guided, and easily understood menu of features; and conducting a more rigorous usability test in Phase II than in Phase I.

Maximize the value of the EdClips for engaging the participant. Although we are confident that, based on focus group input and experimental group feedback, we created EdClips that are relatively short, attractive, informative, and easy to read, it is possible that some parents felt satisfied with the amount of information in the emails and did not feel the need to follow the link to the program website to obtain more information. Thus, we plan to maximize the motivational value of the prompts associated with the links, take steps to ensure that the EdClips do not get flagged as Junk or Spam, and, again, conduct a more rigorous usability test in Phase II than in Phase I to ensure that the EdClips are serving their function as the portal through which participants enter the website.

Maximize participation in discussion forum. Participation in the discussion forum was not as active as we had hoped for. We are confident that having large group of simultaneous users will create a critical mass and boost participation. But we have identified room for improvement nonetheless, including clearer links between the EdClips and the relevant discussion forum topics, clearer order and organization of the discussion forum topics, fewer topics posted so as not to dilute participation; and enabling of custom "avatars" (personal icons) and photos to be uploaded.

Refine and Improve Program Structure and Features
Based on focus group input, consumer satisfaction feedback from participants, and our own experience with the program, there are a number of refinements that we have identified for Phase II for all three programs (Infants Online, Ones Online, and Twos Online). First, the focus group participants agreed that including a feature that enabled parents to fill out a journal for baby/toddler memories and milestones and parents' own self-reflection and goal-setting would be highly desirable. We see this as quite compatible with the program structure and content, and although not possible within the resource limitations of Phase I, we have planned to include this feature in Phase II. Second, based on feedback from some participants in the evaluation that the program seemed rather long, we plan to reduce the length of program somewhat (from 34 EdClips to 30) so that it does not extend over quite so long a period of time. We will still include roughly the same number of web-based components; we will simply have somewhat fewer EdClips with more web-based resources per EdClip. In addition, we will also include more content regarding the stresses and issues faced by working parents, and more emphasis on early literacy and the link between early learning and later success in school.

Evaluation Procedures

Improve recruitment through establishing collaborative relationships with healthcare organizations. In Phase I, we encountered recruitment challenges. We were challenged by pediatric clinic staff who were not particularly motivated to recruit for us within our limited resources and time frame. It became clear to us that in Phase II, we would need larger, more active, and more invested recruitment partners. As a result, we have formed an alliance with three different large-scale organizations providing services to families of young children (two health maintenance organizations and one statewide home visitation service) to recruit eligible families to our Phase II evaluation (see letters of support). These partners are described in detail in the Methods section. Each of these organizations sees a great deal of benefit for their target audience in providing parenting education and support services online, and is motivated to assist us in recruiting families to the Phase II evaluation. Our plan to reimburse them for their costs also increases the extent to which they will be willing and able to assist us. We are confident that these alliances will increase the ease of participant recruitment.

Ensure engagement and participation in the program in order to ensure dosage. In Phase I, 18% of individuals assigned to the TX condition never activated their program license despite repeated reminders, and thus received no program dosage at all. In Phase II, we will call those who have not activated their license and offer to assist them or actually activate their license for them. In addition, because a number of users only received the EdClips and did not engage in the website-based resources, we will also call all participants approximately two to three weeks after program activation to check in and make sure that they are receiving the EdClips, orient them to the program and the website, and answer any questions they might have.

 

Summary of Phase I Activities
This Phase I project developed and evaluated a unique, innovative, and effective program for providing online parenting education and support to new parents. The Incredible Infants Online program served as the prototype for the 3-part First 3 Years Online program and was evaluated in a small-scale randomized trial with 65 mothers of infants aged 0-4 months. It was shown to have positive effects and was well-received by users. Results from the evaluation suggest that the program had positive effects on mothers' parenting skills, perceptions of supportive advice, self-efficacy, and pleasure in parenting. In comments, parents expressed enthusiasm for the "wealth of information," the sense that "I'm not alone," helpfulness in "dealing with my daily issues," and the convenience and flexibility of the program. Participation was less than hoped for, however. The proposed Phase II project will include refinement of the Infants Online program as well as development and evaluation of the remainder of the First 3 Years Online series: the Wonderful Ones Online and Terrific Twos Online programs. Following the conclusion of the evaluation, the program will be prepared for commercialization.