Alcohol Servers Applying Psychology - Phase I Report
Colvin, Geoff, Ph.D., Principal Investigator
1R43AA017560-01
Project Period: 8/01/2008 - 4/30/2009
Brief Project Summary
The goal of this Phase I project was to develop and establish the feasibility of an online professional development program for alcohol servers. The primary aim was to train servers to apply psychologically sound methods of behavior management to the potentially volatile environment of bars, taverns, and other establishments that serve alcohol. An introductory module of this program was developed, produced, and evaluated. The program consisted of an interactive, media-rich Internet-based training program.The Phase I effort comprised three broad interdependent activities: (1) content development based on formative evaluation (focus groups, key informant interviews, expert consultation), (2) program production and testing (usability evaluations with representative members of the target audience), and (3) a summative evaluation with 42 alcohol servers.
All three principal activities were satisfactorily completed and the evaluation found significant and large effects across the four outcome measures [behavioral intentions, self-efficacy, knowledge, and Video Situation Tests (VSTs)]. The results of the satisfaction measure and the acceptance of technology measure showed that participants felt very satisfied with the program and that it was easy to use, accessible, and useful. Specifically, 95% reported they were satisfied with the quality of the training, 93% would use this method of training again, 93% said that the training content was well organized, 93% found the materials to be engaging, 95% found the ideas presented easy to understand, 90% would recommend the training to other servers, 88% of the participants reported that the materials gave them new ideas about maintaining a positive environment. 93% said that this kind of training will help in their job, and 95% said that they viewed the whole training. Computer system log files showed that the participants watched the program for an average of .5 hours.
Proposed Phase II plans are predicated on results and lessons learned from this Phase I effort, which are discussed in more detail at the conclusion of this Preliminary Study section. Table 2 below provides an overview of Phase I activities.
| Activity | Purpose | Participants | Outcomes |
|---|---|---|---|
| CONTENT DEVELOPMENT | |||
| Focus group | Formative evaluation & input | 9 servers |
|
| Key informant interviews | Formative evaluation & input | 7 managers, bartenders, & consumers |
|
| Expert consultation | Formative evaluation & input | Project consultant |
|
| Media assets design & development | Formative design & development | Project development staff |
|
| PROGRAM PRODUCTION | |||
| Media assets production | Produce program | Project production staff |
|
| Upload to IEO (web training platform) | Prepare program for online use | Project production staff |
|
| Usability test | Uncover navigation problems & design errors |
8 servers |
|
| FEASIBILITY EVALUATION | |||
| Summative evaluation | Pre-post test design | 42 servers |
|
| Post-test only: multivariate survey analysis |
|
||
Content Development
At the project's inception, the Principal Investigator (PI) and key project staff identified a set of learning objectives for the introductory module. With the learning objectives as a basis for discussion and development, they created a preliminary program matrix and instructional flowchart that established the architecture of the program module and indicated how learning objectives would be met. They then solicited critical feedback through formative development activities, and used this input to inform revisions to this preliminary material.Focus group
We conducted a formative focus group with nine alcohol servers. Participants were recruited by distributing posters and flyers to local establishments, through contacts made by the PI during research for his book on bar safety, and by a project consultant who had operated a bartender training school. The focus group helped us examine issues of social validity, providing information on ways to optimize server acceptance of the program. Participants were heterogeneous across age, experience, ethnicity and gender. All identified themselves as bartenders, servers, or security. All participants had a computer at home, used the Internet for personal research, and used email. Participants were 45% male and 55% female; none were Hispanic, and 13% chose not to respond. In terms of race, 80% were White, 7% were more than one race, 3% were Asian, 2% were Black/African American, and 8% chose not to respond. In terms of education, 35% had an undergraduate degree, and 27% had some college experience.The PI and key project staff followed focus group guidelines developed by Morgan (1998). The session was audiotaped, results were analyzed to identify themes in participants' responses, and data was collected for use in program development. The PI began with a brief description of the ASAP program and provided audio/video examples of similar programs. He then described the project training objectives and led participants in a structured discussion about key topics to elicit personal experiences.
Participants recommended that in order for the training to be successful, it should: (1) target content to servers, (2) provide tips on making more money, (3) perhaps offer a certificate for completing the training, and (4) make the training available online as opposed to at a facility. The group felt that the sense of being safe tended to depend on the establishment and the atmosphere, although there was general agreement that the atmosphere depends a great deal on how staff acts towards the clientele. Surprisingly, most were unaware of a formal set of procedures their establishment had for dealing with emergencies. They reported that owners and managers tend to discourage calling 911 because they do not want to deal with increased liability insurance, liquor control commission reviews, or bad publicity. Nevertheless, participants admitted that they had been in situations when the police were called because it felt like the safest thing to do.
Participants reported that they typically receive an hour of training from the Oregon Liquor Control Commission on procedures for carding individuals, basic information on over-serving, and crisis intervention. This training is renewed periodically. They were aware that people often come to their establishment under the influence of a drug other than alcohol, but were not aware of any particular strategy for handling problems that might arise as a result. Some, particularly female participants, felt that there needed to be training on how to deal with sexual aggressiveness from patrons toward servers and between patrons.
Participants also indicated a willingness to learn a set of psychologically sound approaches for managing the behavior of clients, and they were very receptive to the idea of proactive problem prevention rather than responding after the fact. They expressed interest in relaxation techniques that would help them keep their cool in volatile situations. While the group supported the idea of training in psychological methods, their overwhelming recommendation was that the training must be realistic, down-to-earth, and interesting in order to be viewed as desirable or relevant. They indicated a strong preference for practical rather than conceptual training, and recommended the use of humor instead of a dry and overly serious approach.
Participants reported several areas in which they had received no training: getting along with customers, setting the tone for a location, recognizing signals that there could be a problem in the making, and managing conflicts between patrons or between patrons and servers. The group was quite aware of laws governing serving minors and over-serving. A portion of the meeting was spent sharing anecdotes about experiences in problem situations. As hoped, this discussion shed light on the standard terms and colloquial language used by servers, and provided useful anecdotes that were incorporated into the script for the video program's situational vignettes. Focus group participants were paid $75.
Key informants
Seven key informants representing stakeholders from each of the following categories were interviewed: (1) leadership: business owners, managers, and personnel preparation instructors; (2) direct service providers: alcohol servers in bars, restaurants, public facilities, and those in pre-service training; and (3) customers or consumers. The goal of these interviews was to solicit in-depth feedback on effective, pragmatic approaches that could optimize systemic fit and server participation in the online training program and to gather suggestions on ways to make establishments a less threatening, more inclusive environment.When asked what makes a successful server, leadership mentioned the ability to get along with customers and provide good service, trustworthiness, and a good work ethic. Employees who have difficulties getting along with or causing problems with other staff and customers were listed as likely to get fired along with those who steal money or supplies, and those who are often late for work. Leadership admitted that there is no official training available for dealing with people, but that management provides some sort of on-the-job training. One manager mentioned that training in this area is critical because while some employees are naturals in this area, others need a lot of work, and some just never get it.
When asked whether they or others they knew had received training in people management skills, servers reported having received little or no training. One server mentioned that servers are expected to either know what to do or learn as they go. Another server mentioned that some owner/managers provide some training, but usually after mistakes have been made. Servers mentioned that while solid training in this area is needed, they had never seen written material or Internet training on the topic, but would accept opportunities for training if it were available and inexpensive. Expense was mentioned as an important factor if they were required by management to pay for the training and do it on their own time. Regarding emergency procedures, servers concurred that they are told to call the police, but that when they do, they are often blamed for not being able to handle the problem on their own and for giving the establishment a bad name. They mentioned managers' lack of consistent procedures and requests. One server admitted feeling quite unsafe, especially around closing in the early hours of the morning.
Customer key informants described what they look for in a drinking establishment: a safe environment where people aren't picked on or get into fights, a place to socialize, a place to listen to music or watch sports, good service, and clean setting and restrooms. Among the attributes of a good server, they listed being friendly without being too engaging, being on top of things (e.g. serving promptly, cleaning up, accommodating larger groups), being welcoming, giving customers a sense of being important, and catching problems before they become major. Most of these characteristics depend on behavior skills that can be learned systematically. Regarding safety, the following comments were made: they avoid places with a reputation for being dangerous, they do not go to a place if it tends to "get ugly," they believe that servers can keep the lid on most situations if they are doing their job, and owners need to have clear policies and enforce them.
There was very strong consensus across all groups that alcohol servers play a key role in creating a safe and relaxed environment, and that it is important for servers to have skills based on psychological principles. Training in these areas, however, was not usually available except in a haphazard manner. There was also consensus that emergency procedures should be more consistent. Finally, all groups thought that problems can be prevented if servers have good management skills and training.
Expert input
We sought input from our expert consultants, Ronald Schwerzler, M.D., an expert on the effects of alcohol and other drugs on human behavior and medical Director of Serenity Lane, a statewide alcohol recovery program; and from Zorah M. Rose, former owner and director of Premium Pour Bartending School, who provided instruction, consultation, and job placement opportunities in the food and beverage industry. Both consultants provided feedback on program goals, participated in discussions about presentation strategies, and reviewed drafts of script materials.Dr. Schwerzler advised project staff on the behavioral implications of alcohol as a mood enhancing drug. He advised us to include material and strategies for handling patrons affected by drugs other than alcohol, and to provide information to servers about the interaction of alcohol with other drugs, including psychotropics, narcotics, and prescribed medications.
Zorah Rose provided insights into the culture and work environment at bars and taverns and assisted in recruitment efforts. A proponent of creating friendly environments in drinking establishments, Ms. Rose advocated motivating workers to use proactive strategies by appealing to their sense of professionalism and job satisfaction, and by reminding them that they would benefit economically through greater tips and greater job security, common concerns for workers in this field.
A literature review of current research on aggression in drinking establishments, alcohol/drugs and behavior, and violence prevention efforts was also very helpful. Project staff discussed the need for incorporating key information from this literature review into the training program. They also discussed the importance of validating the beneficial aspects of social drinking, which include social recreation for patrons and the economic livelihood of workers, managers, and owners in the industry.
Media assets development
Data and impressions were gathered from formative activities that were conducted to help shape the development of media assets. The PI and a team of specialists including an instructional media designer, scriptwriter, and a graphic artist created scripts for a video presentation, practice vignettes and self-assessments, and text and graphics for downloadable print materials. The graphic designer also created a uniform "look and feel" that would provide visual consistency in all program components, and was animated by the program's training goals. This development effort was an iterative one and relied on consultation and feedback with consultants and key informants.Program Production
Once final drafts of the instructional materials and designs were created and approved by the PI, the media producer initiated and supervised production of the program's multimedia, text and web assets. The media producer was assisted by team of production specialists (graphic designer, motion graphics Flash animator, technology coordinator responsible for web and interactive programming, video director, video editor, sound designer, production coordinator, lighting gaffer and production grip. All video and multimedia sequences used a racially and ethnically diverse cast of talent (actors, extras), recruited from a professional talent pool in Eugene and Portland. Video footage was filmed in a local tavern using a professional digital video format, DVCAM, and professional lighting and grip equipment. Digital post-production was accomplished at IRIS Media's studio professional using state-of-the-art video and audio equipment. The PI and other team staff reviewed digital offline versions of video sequences and provided feedback prior to finalization.Once program assets were completed, the technology coordinator uploaded them to IRIS Ed Online (IEO), the training delivery interface, using IRIS Media's content management system (CMS). Project staff then conducted a usability test of the online program and assessment materials to ensure user-friendly navigation. Following Virzi (1990; 1992), we conducted a usability test with eight Eugene-Springfield alcohol servers judged to be reasonably representative of our target market in terms of age, ethnicity, and gender. Participants were asked to use the study website as a typical user would: access all program components, use the interactive exercises, and complete assessments. Using IEO web logs, we mapped participants' navigation through the site, noting the order in which portions of the site were accessed. We then used these observations to revise our final system implementation.
Participants were also asked to complete a questionnaire following recommendations from Tullis and Stetson's (2004) research. We used their 10-item adaptation of Brooke's (1996) widely-used System Usability Scale (SUS) to rate usability of the program website. Finally, at the outcome evaluation, participants were interviewed to gather tips on improving delivery and removing barriers to access. Participants in this usability evaluation were paid $75. Usability tests revealed only two minor navigational errors. These were corrected and the program was released for feasibility evaluation.
Feasibility Evaluation
A two panel, pre-and post-training design was employed to assess the feasibility of the ASAP program. Change in knowledge, skills, self-efficacy and behavioral intention to use were measured pre-post and satisfaction, technology acceptance and program usage were measured post training.Participants
Forty-two alcohol servers were recruited to participate in the Phase I feasibility study, following procedures that IRIS Media has developed and used successfully for online recruiting. Participants were paid $50 for their participation in completing the pre-and post-training assessments. We actively recruited in areas with high minority representation (Oregon is over 90% white). Recruitment materials were designed to be representative in regards to race, ethnicity, and gender.Procedures
In the process of conducting previous online studies, we have developed successful retention and reminder procedures using the study website and email prompts. IRIS Media staff created a study website that managed participation in the research study. Recruitment flyers were created and sent to professional contacts for distribution to servers in Brooklyn, NY; Eugene, OR; Hollywood, CA; Portland, OR, San Francisco, CA, and other locations. The recruitment flyers provided the URL for the study website. Participants who visited the study website were able to read a brief explanation of the project and were directed to a series of screening questions to determine eligibility. Screening criteria excluded any individual who had never been a server, was not in training to become a server, or was not currently a server. Participants were also required to be 18 years or older and have access to email and a high-speed Internet connection. Eligible participants were then directed to read a detailed written description of the study procedures and were invited to complete an online informed consent form. Participants who signed and submitted the informed consent form were invited to create a user name and password, with which they could access the IEO site and the pre-training assessments. Upon completing the pre-training assessments, participants were given immediate access to the online training program (intervention materials).Participants were allowed two weeks to access the program before completing the post-training assessments. After the two-week trial period, participants who had not completed the surveys were emailed and asked to complete the post-training assessment. Email reminder messages were sent to those who did not respond within three days. The IEO site has a built-in management system that allows qualified research staff to access the database in order to collect user activity data (participant responses, number of user attempts, date and time of access, and percentage of assessment completion). This encrypted data was downloaded for analysis by research personnel.
Measures
The evaluation plan was based on the Theory of Reasoned Action (TRA; Ajzen & Fishbein, 1973) which serves as the theoretical underpinning of the Technology Acceptance Model (TAM; Davis, Bagozzi, & Warshaw, 1989). The TAM centers on two specific beliefs that have been shown to influence acceptance of technology: perceived ease of use and perceived usefulness. Thus, TAM suggests that perceived usefulness and ease of use influence behavior (i.e., use of the program). Furthermore, Bandura's Social Cognitive Theory (Bandura, 1977) suggests a relationship between self-efficacy and outcome behavior. Thus, the program evaluation measures were based on these theoretical frameworks.Basic demographic variables that were collected included age, gender, ethnicity, years of experience, position (owner, manager, server, security), and educational background.
Satisfaction and acceptance of technology measures were used to assess perceived ease of use, perceived usefulness, engagement, comprehension, and acceptability of the ASAP program. Likert scales were used to measure these constructs. Open-ended questions solicited responses not covered by the Likert scales.
Attitudes and behavioral intention items were based on TAM items developed by Gardner and Amoroso (2004) and used a 6-point Likert scale to assess the likelihood of using a web-based training program and applying the ASAP concepts to reduce aggressive behaviors. A knowledge test was based on the content covered in the Phase I program (i.e., understanding aggression, using non-verbal techniques, and responding to problem situations). To assess self-efficacy toward performing the recommended behaviors, items were constructed to document changes in participants' perceptions of their mastery of the skills presented in the program and self-efficacy to perform them as a function of this training. Lastly, computer system log files assessed amount of program use (e.g., time spent, section visited, videos viewed). In the process of developing each scale, we generated an item pool (e.g., 15-20 items). The battery of measures was piloted during the usability testing. Items that were difficult to understand or had limited variability were dropped or revised. Item analytic procedures (e.g., frequency distributions and plots, internal consistency analysis, components analysis) were used during the program evaluation phase to create psychometrically sound scale scores.
Video Situations Test (VST)-Skills test
Participant skills in handling problem behaviors were measured using four real-life video vignettes. The VST consisted of short (20-45 second) video vignettes in which an actor plays the role of an aggressive patron. Vignettes represented situations closely related to the ASAP program, and showed either correct or incorrect response by servers. Participants were asked if they agreed with how a problem was handled in the vignette, or how they would handle the problem. Scenarios were balanced on gender, ethnicity, and race. Video vignettes as assessments of skill acquisition have been successfully used at IRIS Media with similar protocols, showing evidence of test-retest reliability (r=.77) and sensitivity to change.Data analysis
Prior to analysis, all variables were checked for out-of-range values and inter-and intrameasure consistency; frequency distributions and plots were examined for unusual data distributions or data points. Any necessary data transformations were employed. The main analysis tested the hypothesis that participants would show significant improvement in knowledge, skills, attitudes, behavioral intentions, and self-efficacy in applying ASAP training concepts. A within-subject, repeated measures approach was used to test for change in the measures by comparing baseline scores obtained at the pre-training assessments to scores obtained during the post-training assessments. User engagement was measured using analytic web logs.Forty-two participants started the program. Thirty-seven had usable data for analysis. Demographic and satisfaction data were analyzed for all 42 subjects who started the study. Participants were 55% male. Consumer Satisfaction data showed that the program was well received and culturally on target: 95% were satisfied with the quality of the training, 90% would recommend the training to other alcohol servers, 93% would use this method of training again, and that it is a type of training that helps with their job, 93% found the program engaging and well-organized, and 88% said that the materials gave them new ideas about how to maintain a happy and controlled environment (See Appendix B for Demographics and Consumer Satisfaction results tables.)
When asked how they would have liked the training to be different, individuals responded with the following: "Can't think of how it could have been made better," "I thought it was a great program," "I wouldn't have changed it," "Better quality (than) the OLCC videos I had to watch to get my servers license." Requesting additional input, individuals responded: "Fine work. Better than the Washington state standard training by far," "Great job," "I had a situation last Saturday night, and now realized some things my manager and myself (sic) could have done better," "This is a much better form of training for the alcohol license and I wish that this would be offered for the OLCC license training," "This was very easy to understand and it makes me feel confident that I can apply these principles. They seem simple, easy and effective," and "Videos were much better quality (than) the OLCC videos I had to watch to get my servers license."
Program efficacy
To test the efficacy of the training program, pre-post paired t-tests were conducted on the four outcome measures: behavioral intentions, self-efficacy, knowledge, and VST skill application. Table 3 presents the pretest and posttest means and standard deviations as well as the paired t -test results and effect sizes for the outcome measures. As can be seen, significant and large effects were obtained on all four outcome measures in the predicted direction. The largest effect was obtained for the knowledge test (partial pointbiserial r = .88), followed by behavioral intentions (r = .70), self-efficacy(r = .70), and the VST skill application (r = .67). Hence, as a result of the on-line training program, alcohol severs improved their knowledge about reducing and diffusing patron aggressive behaviors, self-confidence in their ability to apply the psychological skills taught, behavioral intentions to apply the skills, and ability to apply the skills based on video scenarios presented to them.| Outcome Measure | Pretest | Posttest | T | p-value | Effect Size | ||
|---|---|---|---|---|---|---|---|
|
M
|
SD
|
M
|
SD
|
||||
| Behavioral intentions | 3.17 | 0.37 | 3.50 | 0.44 | 5.89 | <0.001 | 0.70 |
| Self-efficacy | 3.12 | 0.51 | 3.55 | 0.41 | 5.84 | <0.001 | 0.70 |
| Knowledge | 74.1 | 8.0 | 87.9 | 6.7 | 10.98 | <0.001 | 0.88 |
| VST: Skill application | 78.4 | 21.4 | 98.0 | 6.9 | 5.38 | <0.001 | 0.67 |
| Note. N = 37. Effect size is the partial point-biserial correlation (Rosnow & Rosenthal, 2008, p.391). | |||||||
Feasibility benchmarks
Program feasibility was demonstrated by the achievement of the following benchmarks: (a) a high level of program usage (e.g., 100% of participants accessed and completed the program content); (b) a high degree of consumer satisfaction (e.g., mean ratings of 5 or greater a 6-point rating scale across satisfaction domains); and (c) significant and large effects were obtained on knowledge, skills, attitudes, behavioral intentions, and self-efficacy.Phase II Efficacy Trial
A large scale randomized controlled trial will be conducted in Phase II to evaluate the efficacy of the interactive training program compared to a "business-as-usual" control condition. The outcome assessment will be expanded to include behavioral observation data based on a validated protocol for collecting data within drinking establishments (Graham et al., 2004) in order to document pre-and post-training rates of prosocial and aggressive behaviors. The Phase II trial will have sufficient statistical power to test for potential moderating effects (e.g., years of experience) and putative mediating mechanisms (e.g., proactive management skills). In addition, the efficacy trial will include a substantial follow-up period (e.g., 12 weeks) in order to examine the maintenance of program effects.Summary and Conclusions
In Phase I, we accomplished our primary goals: (1) conducting formative research to inform program content, objectives, methods, implementation and systemic fit, (2) producing content, media assets and technology for the program, and conducted a usability test, and (3) conducting a feasibility evaluation of the ASAP program with a representative group of servers. The project demonstrated that we could produce a relevant professional development e-learning program that was suitable for alcohol servers. Program development was an iterative process informed by our expert consultants. The PI and the Research Manager sought input and feedback from focus groups and key informants who were knowledgeable about the hospitality industry. We exceeded our recruitment goals and successfully conducted the evaluation.Although cautious in our interpretation of Phase I results due to the relatively small sample size, we are pleased with the ratings from the evaluation sample (treatment group) on matters of knowledge, self-efficacy, consumer satisfaction, and program usage (In Phase II, we will conduct a field trial with a large sample of alcohol servers using a randomized treatment control design). The feasibility study showed that, as a result of the on-line training program, alcohol severs improved their knowledge about reducing and diffusing patron aggressive behaviors, increased their self-confidence in their ability to apply the psychological skills taught, increased their behavioral intentions to apply the skills, and increased their ability to apply the skills based on video scenarios presented to them. This is noteworthy, given that this Phase I project was meant to be introductory and only a portion of the entire training proposed for Phase II.
References
Abrams, D.B., Orleans, C.T., & Niaura, R..S. (1996). Integrating individual and public health perspectives for treatment of tobacco dependence under managed health care: A combined stepped care and matching model. Annals of Internal Medicine, 18, 290-304.
Ajzen, I., & Fishbein, M. (1973). Attitudinal and normative variables as predictors of specific behaviors. Journal of Personality and Social Psychology, 27, 41‐57.
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91‐97.
Bandura, A. (1977). Self‐efficacy: Toward a unifying theory of behavior change. Psychological Review, 84, 192‐215.
Barrera, M Jr., Glasgow, R. E., McKay, H. G., Boles, S. M., & Feil, E. G. (2002). Do internet‐based support interventions change perceptions of social support?: An experimental trial of approaches for supporting diabetes self‐management. American Journal of Community Psychology, 30 (5), 637‐654.
Baum, L. (2002). Factors related to use of Internet parent support groups by primary caregivers of a child with special health care needs (Doctoral dissertation, University of Utah, 2002). Dissertation Abstracts International, DAI‐B 63/02, 734, Publication #AAT 3041616.
Brooke, J. 1996. SUS: A quick and dirty usability scale. Earley, United Kingdom : Redhatch Consulting Ltd.
Brown, G. L., & Linnoila, M. I. (1990). CSF serotonin metabolite (5‐HIAA) studies in depression, impulsivity, and violence. Journal of Clinical Psychiatry, 51(4)(suppl), 31‐43.
Brown, M., Yando, R., & Rainforth, M. (2000). Effects of an at‐home video course on maternal learning, infant care and infant health. Early Child Development and Care, 160, 47?65.
Bureau of Labor Statistics, U. S. Department of Labor. (2005, May 13) Women in the labor force: A databook. Retrieved August 3, 2007, from http://www.bls.gov/bls/databooknews2005.pdf
Bureau of Labor Statistics, U.S. Department of Labor. (n.d.) Food services and drinking places. Retrieved August 3, 2007, from http://www.bls.gov/oco/cg/cgs023.htm
Bushman, B.J. (1997). Effects of alcohol on human aggression: Validity of proposed explanations. Recent Developments in Alcoholism, 13, 227‐243.
Carr, E. G., Dunlap, G., Horner, R. H., Koegel, R. L., Turnbull, A. P., Sailor, W., Anderson, J., Albin, R. W., Koegel, L. K., & Fox, L. (2002). Positive behavior support: Evolution of an applied science. Journal of Positive Behavior Interventions, 4, 4‐16.
Children of Alcoholics Foundation, Inc. (1996). Collaboration, coordination and cooperation: helping children affected by parental addiction and family violence. New York: Children of Alcoholics Foundation.
Coleman, P. K., & Karraker, K. H. (2000). Parenting self‐efficacy among mothers of school‐age children: Conceptualization, measurement, and correlates. Family Relations, 49 (1), 13?24.
Collins, J. J. (1989). Alcohol and interpersonal violence: Less than meets the eye. In N. A. Weiner & M. E. Wolfgang, (Eds.). Pathways to Criminal Violence. (pp. 49‐67). Newbury Park, CA: Sage Publications.
Collins, R.L., Quigley, B, & Leonard, K.E. (2006). Womenʹs physical aggression in bars: an event‐based examination of precipitants and predictors of severity. Aggressive Behavior, 33, 304‐313.
Colvin, G. (1992). Managing acting‐out behavior: A staff development program to prevent and manage acting-out behavior [Videotape]. Eugene, OR: Behavior Associates.
Colvin, G., & Battistella, P. (2007). The successful bartender: Putting people skills to work. Eugene, OR: Behavior Associates.
Colvin, G., Kame?enui, E., & Sugai, G. (1993). School‐wide and classroom management: Reconceptualizing the integration and management of students with behavior problems in general education. Education and Treatment of Children, 16, 361‐381.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35, 982‐1003.
Dumka, L. E., Stoerzinger, H. D., Jackson, K. M., & Roosa, M. W. (1996). Examination of the cross‐cultural and cross‐language equivalence of the parenting self‐agency measure. Family Relations, 45 (2), 216‐222.
Engelmann, S., & Carnine, D. W. (1982). Theory of instruction: Principles and applications. New York: Irvington.
Engelmann, S., & Colvin, G. (2006). Rubric for identifying authentic Direct Instruction programs. Eugene, OR: Engelmann Foundation.
Fagan, J. (1990). Intoxication and aggression. Crime and Justice, 13, 241‐320.
Fletcher, D. (1989). The effectiveness and cost of interactive videodisc instruction. Machine‐Mediated Learning, 3, 361‐385.
Foster, S. L., & Mash, E. J. (1999). Assessing social validity in clinical treatment research issues and procedures. Journal of Consulting and Clinical Psychology, 67(3), 308‐319.
Gardner, C. and Amoroso, D. (2004). Development of an instrument to measure the acceptance of internet technology by consumers. IEEE: Hawaii International Conference on Systems Sciences ‐ 37 Proceedings, Kona‐Kailua, Hawaii, U.S.A., January 4‐8, 2004.
Gettinger, M. (1988). Methods of proactive classroom management. The School Psychology Review, 17(2) 227‐242.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: the RE‐AIM framework. American Journal of Public Health, 89, 1322?27.
Gordon, D. A. (2000). Parent training via CD?ROM: Using technology to disseminate effective prevention practices. Journal of Primary Prevention, 21(2), 227?251.
Graham, K., LaRoque, L., Yetman, R., Ross, T. J. & Guistra, E. (1980) Aggression and barroom environments. Journal of Studies on Alcohol, 41, 277‐292
Graham, K., Bernards, S., Osgood, D., Homel, R., and Purcell, J. (2005). Guardians and handlers: The role of bar staff in preventing and managing aggression. Addiction, 100 (6), 755?766.
Graham, K., Osgood, D.W., Zibrowski, E., Purcell, J., Gliksman, L., Leonard, K., Pernanen, K., Saltz, R. F., & Toomey, T.L. (2004). The effect of a Safer Bars programme on physical aggression in bars: Results of a randomized controlled trial. Drug and Alcohol Review, 23, 31‐41.
Grant, T. R., Hawkins, A. J., & Dollahite, D. C. (2001). Web‐based education and support for fathers: Remote but promising. In J. Fagan & A. J. Hawkins (Eds.), Clinical and education interventions with fathers (pp. 143?170). New York: Haworth.
Graves, T.D., Graves, N.B., Semu, V., & Sam, I.A. (1981). The social context of drinking and violence in New Zealandʹs multi‐ethnic pub settings. In T.C. Harford & L.S. Gaines (Eds.) Social Drinking Contexts. (pp. 103‐120). Rockville, MD: NIAAA Research Monograph No. 7.
Gustafson, R. (1994). Alcohol and aggression. Journal of Offender Rehabilitation, 21(3/4), 41‐80.
Haine, R. A., Sandler, I. N., Wolchik, S. A., Tein, J.‐Y., & Dawson‐McClure, S. R. (2003). Changing the legacy of divorce: Evidence from prevention programs and future directions. Family Relations, 52, 397‐405.
Harwood, R. L., & Weissberg, R. P. (1987). The potential of video in the promotion of social competence in children and adolescents. Journal of Early Adolescence, 7, 345‐363.
Hawkins, J. D., Von Cleve, E., & Catalano, R. F. (1991). Reducing early childhood aggression: Results of a primary prevention program. Journal of the American Academy of Child & Adolescent Psychiatry, 30(2), 208‐217.
Hingson, R. W., Heeren, T., Zakocs, R. C., Kopstein, A., & Wechsler, H. Magnitude of alcohol‐related mortality and morbidity among U.S. college students ages 18‐24. Journal of Studies on Alcohol, 63(2):136‐144, 2002.
Homel, R., & Clark, J. (1994). The prediction and prevention of violence in pubs and clubs. Brisbane, Australia: School of Justice Administration.
Homel, R., Tomsen, S. & Thommeny, J. (1992). Public drinking and violence: not just an alcohol problem. Journal of Drug Issues, 22 (3), 679?97.
Horner, R., OʹNeill, R., & Flannery, K. (1993). Building effective behavior support plans from functional assessment information. In M. Snell (Ed.), Instruction of persons with severe handicaps, (4th ed.) (pp. 184‐214). Columbus, OH: Charles E. Merrill.
Horner, R., Vaughn, B., Day, H. & Ard, F. (1996). The relationship between setting events and problem behavior. In L. K. Koegel, R. Koegel & G. Dunlap (Eds.). Positive behavioral support: Including people with difficult behavior in the community (pp. 381 ‐ 402). Baltimore, MD: Paul Brookes.
Imel, S. (1995). Teaching adults: Is it different? Myths and realities. Columbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education.
Ireland, C.S. , & Thommeny, J.L. (1993) The crime cocktail: licensed premises, alcohol and street offences. Drug and Alcohol Review, 12 (2), 143‐150.
Kauffman, J. M., Mostert, M. P., Trent, S. C., & Hallahan, D. P. (1998). Managing classroom behavior: A reflective case‐based approach (2nd ed.). Boston: Allyn & Bacon.
Klesges, L.M, Estabrooks, P. A., Dzewaltowski, D. A., Bull, S. S., & Glasgow, R. E. (2005). Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination. Annals of Behavioral Medicine, 29 (Suppl.), 66‐75.
Krueger, R. A. (1994). Focus groups: a practical guide for applied research. Newbury Park , CA: Sage.
Lagges, A. M. & Gordon, D. A. (1999). Use of an interactive laserdisc parent training program with teenage parents. Child & Family Behavior Therapy, 21 (1), 19‐37.
Lang, A.R. (1993). Alcohol‐related violence: Psychological perspectives. Alcohol and Interpersonal Violence. NIAAA Research Monograph No. 24. (NIH Pub. No. 93‐3496), 121‐148.
Langan‐Fox, J., Armstrong, K., Balvin, N., & Anglim, J. (2002). Process in skill acquisition: Motivation, interruptions, memory, affective states and metacognition. Australian Psychologist, 37(2): 104‐117.
Leonard, K. E., Quigley, B. M., Collins, R. L. (2003). Drinking, personality, and bar environmental characteristics as predictors of involvement in barroom aggression. Addictive Behaviors, 28(9), 1681‐1700.
Lipsey, M. W., Wilson, D. B., Cohen, M. A., & Derzon, J. H. (1997). Is there a causal relationship between alcohol use and violence? A synthesis of evidence. Recent Developments in Alcoholism, 13, 245‐282.
MacAndrew, C., & Edgerton, R.B. (1969). Drunken comportment. Chicago: Aldine Publishing.
Merriam, S. B., & Cafferella, R. S. (1999). Learning in adulthood (2nd ed.). San Francisco, CA: Jossey‐Bass.
Miczek, K.A., DeBold, J. F., van Erp, A. M., & Tornatzky, W. (1997). Alcohol, GABAA‐benzodiazepine receptor complex, and aggression. Recent Developments in Alcoholism, 13, 139‐171.
Morgan, D. L. (1996). Focus groups. Annual Review of Sociology, 22, 129‐152.
Morgan, D. L. (1998). Planning focus groups. Thousand Oaks, CA: Sage.
Nafpaktitis, M., Mayer, G. R., & Butterworth, T. (1985). Natural rates of teacher approval and disapproval and their relation to student behavior in intermediate school classrooms. Journal of Educational Psychology, 77(3), 362‐367.
National Highway Traffic Safety Administration. (n.d.) Impaired driving in the United States. Retrieved August 3, 2007, from http://www.nhtsa.dot.gov/people/injury/alcohol/impaired-drivingusa/US.pdf
National Institute for Occupational Safety and Health (NIOSH). 1996. Current Intelligence Bulletin 57. Violence in the workplace: Risk factors and prevention strategies. Cincinnati, OH: NIOSH.
National Restaurant Association. (n.d.) Restaurant industry: Facts at a glance. Retrieved August 3, 2007, from http://restaurant.org/research/ind_glance.cfm
Nurko, C. & Proffit, W. R. (2005). Acceptability and perceived effectiveness of web‐based self‐instruction in clinical orthodontics. Angle Orthodontist, 75(4), 521‐525.
Oldenburg, B., & Parcel, G. (2002) Diffusion of health promotion and health education innovations. In K. Glanz, B. Rimer, & F. Lewis (Eds.). Health Behaviour and Health Education: Theory, research, and practice. (pp. 312?334) San Francisco: Jossey‐Bass.
Olson, D.R. and Bruner, J.S. (1974) Learning through experience and learning through media. In D. R. Olson (Ed.) Media and symbols: The form of expression, communication, and education. The Seventy‐third Yearbook of the National Society for the Study of Education. Chicago: University of Chicago Press.
Peek‐Asa, C., Casteel, C., Kraus, J.F., Whitten, P. (2006) Employee and customer injury during violent crimes in retail and service businesses. American Journal of Public Health, 96 (10), 1867‐72.
Pernanen, K. (1991). Alcohol in human violence. New York: Guilford Press.
Quigley, B. M., & Leonard, K. E. (2005). Alcohol use and violence among young adults. Alcohol Research and Health, 28(4), 191‐194.
Robins, L.N. (1996). Deviant children grown up. Baltimore: Williams & Wilkins.
Rosenshine, B. (1983). Teaching functions in instructional programs. Elementary School Journal, 83, 335‐352.
Saltz, R.F. (1986). Server intervention: Will it work? Alcohol, Health & Research World, 10, 12‐19.
Saltz, R.F. (1987). The roles of bars and restaurants in preventing alcohol‐impaired driving: An evaluation of server intervention. Evaluation and the Health Professions, 10, 5‐27.
School of Criminal Justice at Rutgers University?s Crime Prevention Service (n.d.). Preventing bar violence. Retrieved August 4, 2007, from http://crimeprevention.rutgers.edu/crime/barviolence/preventingbarviolence.htm.
Scott, K. D., Schafer, J., & Greenfield, T. K. (1999). The role of alcohol in physical assault perpetration and victimization. Journal of Studies on Alcohol, 60 (4), 528‐536.
Scribner, R.H., MacKinnon, D.P., & Dwyer, J.H. (1995). The risk of assaultive violence and alcohol availability in Los Angeles County. American Journal of Public Health, 85(3), 335‐340.
Shults, R. A., Elder, R. W., Sleet, D. A., Nichols, J. L., Alao, M. O., Carande‐Kulis, V. G., Zaza, S., Sosin, D. M., Thompson, R. S., & the Task Force on Community Preventive Services. (2001). Reviews of evidence regarding interventions to reduce alcohol‐impaired driving. American Journal of Preventive Medicine, 21(4 Suppl S):66‐88.
Single, E., & Storm, T. (Eds.). (1985) Public drinking and public policy: Proceedings of a symposium on observational studies. Toronto: Addiction Research Foundation.
Solomon, R., & Payne, J. (1997). Alcohol liability in Canada and Australia: Sell, serve and be sued. The Tort Review, 4(3), 232‐233.
Sprague, J. R. & Golly, A. (2004). Best Behavior: Building positive behavior support in schools. Longmont, CO: Sopris West Educational Services.
Stockwell, T., Lang, E., & Rydon, P. (1993). High risk drinking settings: the association of serving and promotional practices with harmful drinking. Addiction, 88 (11), 1519‐26.
Sulzer‐Azaroff, B., & Mayer, R. G. (1991). Behavior analysis for lasting change. New York: Holt, Rinehart, Winston.
Toomey, T. L., Kilian, G. R., Gehan, J. P., Perry, C. L., Jones‐Webb, R., & Wagenaar, A. C. (1998). Qualitative assessment of training programs for alcohol servers and establishment managers. Public Health Reports, 113(2), 162‐9.
Toomey, T. L., Wagenaar, A. C., Gehan, J. P., Kilian, G., Murray, D. M., & Perry, C. L. (2001). Project ARM: Alcohol risk management to prevent sales to underage and intoxicated patrons. Health Education and Behavior. 28 (2), 186‐99.
Tullis, T.S., & Stetson, J.N. (2004). A comparison of questionnaires for assessing website usability. Proceedings of the Usability Professionals Association (UPA) 2004. Minneapolis, MN.
U.S. Department of Veteran Affairs. (2007, March 23) Eating and drinking places. Retrieved August 3, 2007, from http://www.vetsuccess.gov/resources/industries/profile?id=30
University of Minnesota. (2006). What is Responsible Beverage Service Training (RBST) and how does it work? Retrieved August 4, 2007, from http://www.epi.umn.edu/alcohol/policy/rbst.shtm#9
Vella, J. (2000). Taking learning to task: Creative strategies for teaching adults. San Francisco: Jossey‐Bass, Inc.
Virzi, R. A. (1990). Streamlining the design process: Running fewer subjects. Proceedings of the Human Factors Society, 34th Annual Meeting, 291‐294.
Virzi, R. A. (1992). Refining the test phase of usability evaluations: How many subjects is enough? Human Factors: The Journal of the Human Factors Society, 34(4), 457‐468.
Walker, H. M., Colvin, G., & Ramsey, E. (1995). Antisocial behavior in school: Strategies and best practices. Pacific Grove, CA: Brooks/Cole Publishing Co.
Walker, H. M., Schwarz, I. E., Nippold, M. A., Irvin, L. K., & Noell, J. W. (1994). Social skills in school‐age children and youth: Issues and best practices in assessment and intervention. Topics in Language Disorders, 14 (3), 70?82.
Webster‐Stratton, C., Hollinsworth, T., & Kolpacoff, M. (1989). The long‐term effectiveness and clinical significance of three cost‐effective training programs for families with conduct‐problem children. Journal of Consulting and Clinical Psychology, 57(4), 550‐553.
Webster‐Stratton, C., & Reid, M. J. (1999). Treating children with early‐onset conduct problems: The importance of teacher training. Paper presented at the annual meeting of the Association for the Advancement of Behavior Therapy, Toronto, Ontario, Canada.
Wells, S., Graham, K., and West, P. (1998). The good, the bad, and the ugly?: Responses by security staff to aggressive incidents in public drinking settings. Journal of Drug Issues, 28 (4), 817‐836.
Wendling, T. (2005). Injury update: A report to the Oklahoma injury surveillance participants. OK: Injury Prevention Service.
Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203‐214.
Zack, M., & Vogel‐Sprott, M. (1997). Drunk or sober? Learned conformity to a behavioral standard. Journal of Studies on Alcohol, 58(5), 495‐501.
Zahn, M.A. (2003) Intimate partner homicide: An overview (NCJ 196544). National Institute of Justice Journal, 250, 2 ‐3.
