Case Study

Research-Informed Development for Interactive Media:
Enhancing Learning by Engaging Students with Users

Michael Hanley, Jennifer George-Palilonis, Vinayak Tanksale

Ball State University

Abstract 

This case study investigates how the research-informed development model affects the pedagogical learning outcomes and design solutions of university students responsible for creating interactive advertising and news content for television and the iPhone mobile device. An interdisciplinary group of three professors and 31 undergraduates from advertising, computer science, journalism graphics, and telecommunications employed a research-informed development process to create interactive design products and collect feedback from target users about the interactive advertising, news content, and interface designs and functionality. Students used the feedback from user focus groups to revise and improve the design work before each of three rounds of usability tests. 

Keywords: Interactive, media, pedagogy.

Background 

The idea that communications professionals should periodically survey users to collect feedback related to how well the media serve the public’s needs is not new. Print and broadcast news organizations have been conducting research for years to gather feedback about design and content. Advertisers and ad agencies are acutely aware of the necessity to connect with users. And usability tests are essential for developing Web sites that users find relevant, useful, and functional. Likewise, more than ever before, consumers are bombarded with hundreds of options for obtaining news and information, and users expect to have greater control over how the create and consume content (Online Media and Search 2007).

All of these factors combined have led many educators to begin to reconsider their traditional assumptions about how to create the most successful teaching and learning environments. In a rapidly changing and relatively tumultuous media environment, college graduates must understand media users intimately. At Ball State University, an 18,000-student, Midwestern university, a group of students and professors have begun to address this key issue through an interdisciplinary course focused on interactive advertising and news for television and the iPhone mobile device. Using a research-informed development (RID) model, the course enables students to put their work in front of real interactive device users throughout the semester and respond to feedback through frequent required revisions.

When the course was created in 2006, the focus was on applying live news content to designs in a daily television news broadcast to prove that interactive television content could be produced on a tight deadline. With each iteration of the project in 2006 and 2007, the faculty and student teams strived to improve the interactive interface and enhance the collaborative teaching and learning experience. The interactive interface for television consists of a remote-control device programmed via Microsoft’s Windows Media Center software to interact with advertising, entertainment, and news content on screen.

During the spring 2008 semester, faculty continued the evolution of the course by providing students with an RID process that focused more on device interface functionality and usability than on creating content for live TV news deadlines. Three professors collaborated with an interdisciplinary team of 31 undergraduate students to develop four interactive advertising and three news products for use on a TV news broadcast and a small screen mobile device (the Apple iPhone and iPod Touch). The group consisted of students and faculty from advertising, computer science, journalism graphics, and telecommunications, representing two colleges and three departments.  

The Challenge 

Increasing recognition notes the educational value provided by design activities in which students create external products that they share and discuss with others (Papert 1993; Resnick 1998; Soloway, Guzdial, and Hay 1994). Existing literature reveals that pedagogically solid design projects involve authentic, hands-on tasks; use familiar and easy-to-work-with materials; possess clearly defined outcomes that allow for multiple solutions; promote student-centered, collaborative work and higher order thinking; allow for multiple design iterations to improve the product; and have clear links to science and engineering concepts (Crismond 1997).

The highly interactive and nonlinear nature of interactive advertising and news for television and the iPhone necessitates a significant amount of usability testing to ensure that the products students develop are intuitive, engaging, attractive, and enticing for a general user. Although this testing concept is nothing new to industry, its application is far less the norm for college classrooms. Traditional pedagogy, which includes students completing assignments prescribed and judged solely by an instructor, is not only more common in design classrooms, but it is much easier. Yet this one-to-one process is not what students will encounter when they enter the professional world of media, communications, and design. Rather, their work ultimately will be judged by the users who receive it, and this user group is as diverse as it is large. For students, regardless of their areas of expertise, the RID process for this course therefore represents as “real-world” an experience as possible in the context of a classroom environment.

The four key challenges facing the team in spring 2008 were as follows:

The Solution 

Most problem solving in classrooms is directed toward a given set of possible answers and does not challenge students to solve open-ended problems. And yet, most technological design problems are not well defined and often are difficult to solve. This situation provides an opportunity for teachers to introduce constructivist pedagogical practices to engage and challenge students through their own learning. The RID model provides a way to use design and research as a pedagogical strategy.

Informed design is a pedagogical approach developed and validated through the National Science Foundation-funded New York State Curriculum for Advanced Technological Education Project (Burghardt and Hacker 2003). Informed design activities require that “students reach design solutions informed by prior knowledge and research, as opposed to trial-and-error problem solving where conceptual closure is often not attained” (Center for Technological Literacy 2008).

The interactive advertising and news course adopted this design ideology and expanded on it by integrating focus group sessions into the development process so that students could use research to apply incrementally the perspective of the target user to the development process. During the RID process, students presented their interactive design outcomes via focus groups and one-on-one interviews to a sample of respondents, aged 18-65 years, who provided immediate feedback about the usability, quality, and overall value of the work. Students took that information back to the classroom to revise and improve their design work before the next round of usability tests. As a pedagogical tool, the RID model provides an engaging and innovative teaching method that uses actual usability data that will provide the interactive advertising and television and mobile device industries with valuable insights and tools for the future of interactive television advertising and news.

Collaborative Teaching and Learning

It is important to note that in addition to the RID model used by students and faculty engaged in the course, a collaborative teaching and learning philosophy was equally important and effective in engaging students. Collaborative teaching and learning may be the most significant and critical concept related to how such projects fit into the higher education curriculum. More than 150 studies since 1924 have provided clear evidence that cooperative learning experiences promote higher achievement than do their more traditional counterparts. Studies reveal that cooperative activities improve student motivation, build essential communication skills, develop higher-order levels of thinking, and increase social awareness and tolerance for differences (Dishon and O’Leary 1984). Recent research specific to teaching media convergence also suggests that team teaching is the key to successful projects, assuming administrative support exists (Auman and Lillie 2008).

The Process

After spending two weeks at the beginning of the semester studying interactive advertising and interaction design concepts, the 31 students divided into four teams, each responsible for the design and functionality of one of three interface projects:

  1. Interactive advertising content design; nine students (six advertising, one journalism graphics, two telecommunications multimedia).
  2. Television user interface; five students (two journalism graphics, three telecommunications multimedia).
  3. Small screen (iPhone) user interface; 10 students divided into two teams (four journalism graphics, six telecommunications multimedia).

The eight computer science students responsible for programming the design modification into the TV and iPhone interface software did not join a specific team. Rather, they worked as a core programming team on an as-needed basis for the advertising and news teams.

Interactive Advertising

The interactive advertising student team assigned advertising students the responsibility of identifying and contacting three local businesses to participate in the iTV project. A pub-style restaurant near campus, a local Toyota car dealership, and a gardening center agreed to participate and provide advertising materials. Students met with each business and developed a campaign strategy.

The interactive advertising had three primary goals:

  1. Promote the businesses on the main screen of the TV and iPhone interface.
  2. Create interactive advertising content that would enable viewers to interact with the advertising content via the controller, including the ability to request information or order products.
  3. Integrate advertising with news content in a way that does not interfere with the editorial delivery of the news content.

The television news user interface team began the project using the TV interface developed by the 2007 iTV course students; it chose three news feature stories developed by students in a fall 2007 journalism informational graphics class for use as the interactive news content. The news feature stories, each with multiple types of graphics, textual, and video content, dealt with the following subjects: (1) “Aging America,” or how aging baby boomers are looking for alternatives to nursing homes and assisted care facilities and often hiring services that offer companion care; (2) “Foster Care,” or what happens to foster children as they turn 18 years and age out of the system; and (3) “Milk,” or how the rising cost of milk is an economic indicator that affects businesses and consumers.

The small screen (iPhone/iTouch) user interface team used the same advertising content and the same three news stories. The interfaces were designed for use on the Apple iPhone and iTouch, which share the same screen size and navigation tools.

After a month-long development phase, the teams engaged in the first round of usability studies for the television interface through one-on-one interview sessions with 11 target subjects (week 6 of the semester). Students attended the sessions and observed the interviews behind one-way glass in a communications building at Ball State University. The subjects were audio and video recorded as they engaged with the TV interface via a remote control, and their comments and feedback was recorded for evaluation. The following week, students and professors met to discuss specific feedback and overall trends from the sessions, and the teams made decisions regarding which interface functions to redesign for the next round of testing. Three weeks later (week 10 of the semester), usability testing for the television interface was repeated with 8 target subjects, and the first studies of advertising content and the iPhone prototypes were conducted with 8 additional subjects. Again, students observed the sessions and made changes to their work on the basis of user feedback. Usability tests were conducted two more times (weeks 12 and 14) on all three interfaces before students presented their final projects in late April. The number of research subjects varied in each session, but a total of 66 subjects provided feedback during the four usability testing sessions, divided as follows:

The Results 

The course resulted in the completion of four interactive television advertisements and three news broadcasts. Students, in their end-of-course analyses, reported how challenging and liberating the process was for them, mainly due to the intellectual and technical challenges they faced and overcame during the process.

As noted, students used feedback from the research subjects as a guide for making changes to their work to improve the usability and design and thus better serve the advertising and news content. This progressive endeavor involved substantive changes to the design and functionality, as well as many small, barely noticeable changes. Between the second and third round of usability tests for the television platform, for example, the students completely redesigned the interface to make better use of space on the screen and enhance the order of different content areas. 

TV Interface Usability Testing: Version 1

Onscreen icons, ticker, and screen navigation. In the television screen research sessions using the Version 1 interface, subjects noted, among other things, that though onscreen icons provided text descriptions, it was difficult to determine what would happen if they actually selected the icons (i.e., clicked on them with the remote controller) (Figure 1). Likewise, most research subjects never realized the ticker that ran constantly across the bottom of the TV screen was interactive and anticipated a higher level of customization in the onscreen text and images. Perhaps most significant, the TV interface users criticized the left-to-right navigation pattern required by the remote controller, noting that it was “clunky,” “very annoying,” and “the most frustrating aspect” of the interface. (To navigate across the TV screen, a user needed to push an up, down, left, or right arrow on the controller.) As one subject said, “It’s like trying to change lanes on a multilane freeway during rush hour. You want to get over but it’s very difficult.” Most subjects also mentioned that after a few minutes of navigating with the controller though, it became easier to use and interact. 

Figure 1. Onscreen Icons and Ticker, Version 1

Figure 1. Onscreen Icons and Ticker, Version 1

Customizing the TV screen. Television viewers could customize the screen using the controller and navigate to four widgets-Sports, Stocks, Calendar, and Weather-or the Customize button, then click the OK button on the remote control. Doing so would activate the video screen in the center of the screen to reveal a Customize panel on the other side of the video screen. On this Customize panel, the viewer could select scores from different sports leagues, major market indices, or enter a zip code. Upon clicking the Save button on the Customize panel, the user would see a display of all four widgets with the requested content. Specifically, the Weather widget displayed weather and forecasts for the entered zip code, the Calendar widget displayed local events based on the entered zip code, and the Sports and Stocks widgets displayed selected scores and market index values.

Participants believed that the customization screen for each feature would also include actual stories focused on that specific feature; for example, most expected a weather report after selecting the weather feature or specific information about sporting events. A few participants also thought they had selected the wrong item because they saw the same customization screen each time. Some did not like that after they selected “Save” or “Cancel,” the interface automatically took them back to the Sports feature, even though they had originally selected Stocks or Weather (Figure 2). 

Figure 2. Customizing the TV Screen, Version 1

Figure 2: Customizing the TV Screen, Version 1

Icons within story packages. The blue square icons and the oval icons at the bottom of the TV screen appeared confusing to most research participants, because they could not translate what the icons represented. Although the blue icons provided a visual representation of the available content (i.e., a camera indicated available photos or video, a clock represented a timeline, and a chart icon showed that a pie, fever, or bar chart was available), it was difficult to know what would happen if a respondent clicked on these oval icons. The ambiguous oval icons did not inspire viewers to explore them completely (Figure 3).

Figure 3. Icons within Story Packages, Version 1

Figure 3. Icons within Story Packages, Version 1

Advertisement icon. Advertising on the screen showed the logo of a local company as a clickable icon (Figure 4). In the example in Figure 4, the logo of a local restaurant appears. Most participants thought they would be able to interact with the advertisement, such that if they selected it, they expected to see contact information, menus, and specials. Several participants mentioned that it was difficult to see the advertisement on the screen and that they were not initially sure what it was. 

Figure 4. Advertisement Icon, Version 1

Figure 4. Advertisement Icon, Version 1

TV Interface Usability Redesign and Testing: Version 2

The early television research sessions set the stage for a dramatic overhaul of the TV interface that the students redesigned. It is worth noting that watching the faces of students as they observed the early research sessions was both painful and mildly amusing for the professors. Students were clearly disappointed that their first attempts at interface functionality had been less than well received by users. The true test of functionality occurs when users with no prior experience with a device are forced to give it a test drive, and it became equally clear that students quickly began to realize the impact of their decisions and design work on others. The time set aside for students to observe their work in use was perhaps the best spent time of the semester. The number of negative comments about functionality and ease of use dramatically decreased in subsequent rounds of testing, a trend of which the design and multimedia students were most proud. In an end-of-semester review, one student wrote, “Hearing what [research subjects] had to say about our stuff was tough sometimes. But, getting to put our work in front of them for their feedback was invaluable. Everything we learned in those sessions was used to make our work better. And everything we learned in general . . . about ‘the audience’ will also be something I take with me into my career after this course is finished.” 

Figure 5. Original Version 1 TV Interface (left) and Redesigned Version 2 TV Interface.

 

Figure 5. Original Version 1 TV Interface (left) and Redesigned Version 2 TV Interface.

Onscreen icons, ticker, and screen navigation. On the basis of feedback received during research rounds 1-3, the students redesigned the TV interface (Figures 5 and 6). The overall screen color changed from green and turquoise, which research participants described as dark and uninviting, to light grey and blue. The Sports, Stocks, and Weather onscreen icons moved from the left to the right side of the screen, which allowed the icons to increase in size and anchored the active TV screen in the upper-left corner, which in turn gave the overall screen a more balanced design and enabled changes to the icons on the right side and bottom without affecting the active TV screen. It also simplified the screen navigation by eliminating one of the three directions a user needed to follow to move from the left side of the screen to the right. With the new screen design, a user could navigate right across the bottom and up to reach the icon in the top right corner of the screen, whereas with the original design, a user had to navigate down the left side of the screen, across the bottom, and up the right side to reach the icon.

The Sports, Stocks, and Weather content received enhancements through more customizable choices. When clicked, one of the three icons would enlarge and be displayed in color, while the other two icons would appear muted in light grey. A “Customize” button above the icons allowed for customization at any time. The “Calendar” and “Poll” icon areas were deleted, based on suggestions from the focus group participants.

The students also revamped the design of the three news story icons at the bottom of the screen by deleting the small blue square and grey oval icons from the original design, which research participants had deemed confusing. The new, larger icons showed a scene from each of the three news feature stories. When a user clicked an icon, a new window would open that gives the user the option to view a video of the news story or access graphic, textual, or visual images from the story.

The constant ticker at the bottom of the screen received a visual enhancement; students changed the color of the type to blue on a light grey background rather than grey on a dark green background. The word “News” was added at the left end of the ticker to indicate the content. 

Figure 6. Redesigned Television Interface, Version 2

Figure 6. Redesigned Television Interface, Version 2

Advertisement icon. The advertising icon moved to the bottom-right corner of the screen to give it more prominence, help distinguish it from editorial content, and establish a permanent location that users would recognize over time as the location for advertisements. The size of the advertising icon was substantially enlarged compared with that in Version 1 (see Figure 7). 

Figure 7. Advertisement Icon, Version 2

Figure 7. Advertisement Icon, Version 2

iPhone Interface Usability Testing

The early sessions for testing the iPhone interface applications provided surprisingly positive feedback. We expected users to be more critical, particularly considering the newness of the device and the relative newness of the general concept of an interactive advertising and news interface for a small-screen device. Yet subjects found the interfaces intuitive and the interactivity logical and easy to follow, as Figure 8 shows. Likewise, users were generally positive about the organization of advertising and news content and excited that they had a great deal of control over the experience, because they could use their iPhones to explore content in whatever order they chose. Aside from a few minor aesthetic tweaks and organizational changes to make some content (such as the news ticker) more accessible, the iPhone interfaces changed little during the course of the semester.

Certain technological limitations presented unique challenges to the advertising, news design, and telecommunications students. For example, when the iPhone plays a video, nothing other than the video can appear on the screen at the same time. Another challenge pertains to the limited set of tools and technologies supported by the iPhone. The students overcame similar challenges by designing and implementing creative and innovative solutions.  

Figure 8. iPhone Screen Navigation

Figure 8. iPhone Screen Navigation Part 1

Figure 8. iPhone Screen Navigation Part 2

 

Figure 8. iPhone Screen Navigation Part 3

Student Feedback About the Development Process

As part of the RID process, students provided formal feedback through a questionnaire about their participation in the interface development process and offered suggestions about how to improve the process. Although students on each team developed different types of interfaces, their feedback remained consistent regarding the enhancements recommended for the course and development process. The key recommendations included:

  1. Make the course two semesters to allow students more time to develop the interactive interfaces.
  2. Provide more one-on-one coaching by professors and critiques with small groups and professors during the semester.
  3. Offer more instructor interaction during team collaboration to help focus the vision and direction of the teams.
  4. Include fewer telecommunication multimedia students so team roles can be more clearly defined. (There were 11 telecommunication multimedia students during spring 2008; during 2008-2009 there will be 4.)
  5. Include fewer advertising students to allow for more in-depth understanding of all phases of the interactive advertising development process. (There were six advertising students during spring 2008; there will be four during 2008-2009.)
  6. Focus on one advertising client instead of three.
  7. Add more interactive functions to the advertisements beyond links to videos, text, and graphics.
  8. Start implementing prototypes using the required technologies during the first week of the class rather than waiting five weeks.

These recommendations all have been incorporated into the planning for the 2008-2009 course. 

Pedagogical Implications

Research-informed development models can be adopted for any course that requires students to develop content or create designs and use research to gather immediate feedback from users for project improvements. In addition, RID models may be integrated into introductory, intermediate, or advanced courses effectively, as long as professors and students together identify, at the outset of the course, the user demographics and research processes and participants. In our case, participants were members of the university community and residents of Muncie, Indiana, and ranged in age from 18 to 65 years. The more diverse the user base, the more realistically varied the feedback will be. As with any field of study, students also need to understand the basic standards for the task at hand. The first month of the “Interactive Media Design” course involved exploring interaction design, advertising fundamentals, and basic design principles, as well as bringing computer science students up to speed on the technical aspects of what would become the interface infrastructure.

To achieve any level of success with a RID classroom model, students must understand their roles and what skill sets their peers bring to the table. Professors must understand how to manage students from a variety of different disciplines. Finally, university administrators must begin to reconsider traditional course models and be flexible in determining what is now appropriate for immersive courses that are intended to facilitate a higher degree of collaboration than is standard. Recognizing the interdisciplinary nature of this project, as well as understanding how to bring together different departments and colleges effectively in the traditionally siloed university environment, sits essentially at the heart of building an exciting, engaging, and relevant course.

Perhaps the most important and difficult obstacle to overcome in preparing students for the RID model was getting them more focused on the “journey” of the course and less on the success or failure of the discreet design elements, advertising content, news content, or technology. By nature, college students spend much of their academic lives trying to figure out how to get better grades in courses and organizing their lives around syllabi that prescribe each class task in detail. In this course, students knew from the first day that though there was a plan for the course and an outline of expectations for “progress,” the professors would always be open to letting students and the course take different directions. Students were told not to worry about grades but rather to be concerned with completing the work outlined as we progressed through the course, engaging with their partners in active problem solving, and working to overcome their fear of failure. These concepts are very difficult for students to understand, because they are often uncomfortable with such unstructured concepts and, in some cases, seem suspicious of a professor’s motives. Educators who wish to experiment with the RID model should be prepared to nurture student fears and provide constant responses to “how are we doing” questions from students. Likewise, when individual students fail to pull their weight, it is critically important to make them aware of faculty concerns and help them understand what they need to do to improve.

In the case of “Interactive Media Design,” the professors took a very holistic approach to student evaluations. Attempting to evaluate each students’ quality of work, commitment, and engagement would not do justice to the overall assessment of the course and its participants. Therefore, though we outlined for students a number of standards for “A-work,” “B-work,” and so on, and kept students appraised of their progress along the way, we did not provide grades for assignments at specific points during the semester. Rather, we focused on the journey. We focused on collaboration. And we insisted that every student contribute equally at all times. In the end, a student’s practical failures often led to high marks, because the journey they took resulted in true learning.

Granted, these activities and processes are not appropriate for all types of courses, and they are admittedly difficult to organize, manage, maintain, and complete. However, in our experiences, the extra hours, complicated procedures, and sometimes nightmarish organizational obstacles have been worth it. We are constantly amazed at the commitment and dedication of the students involved, as well as their ability to collaborate across disciplines and the quality of the end project. There is no greater thrill for an educator than to see students excel by taking ownership and pride in their work and doing so with a real-world objective in mind.

Finally, it is important to note that Ball State University’s “Interactive Media Design” course is a special topics, three-credit-hour course offered outside the regular curricula for the departments involved, and students are recruited to participate. Thus, in this case, we have the advantage of choosing undergraduate students advanced in their studies who have proven, in other courses and departmental activities, to be leaders and high achievers.  

The Future: Next Steps

During the 2008-2009 school year, the iTV advertising and news design course will be renamed the iMedia Course and extended to a two-semester format. The name change reflects the increasing focus on iPhone interface development. Students and faculty will again focus on developing interactive advertising and news platforms for television and the iPhone. Our objective for the iPhone is to recruit students who own an iPhone to become participants in our interface usability testing via their own cell phones. To test the iPhone interfaces wirelessly, we will partner with a mobile content delivery company to send, monitor, and conduct research via the iPhone.  

References 

Auman, A. and J. Lillie (2008), “An Evaluation of Team-Teaching Models in a Media Convergence Curriculum,” Journalism & Mass Communication Educator, 62 (4), 360-375. 

Burghardt, M.D. and M. Hacker (2003), The New York State Curriculum for Advanced Technological Education, http://www.nyscate.net. 

Center for Technological Literacy (2008), “The ABCs of Creating Informed Design Activities,” Hofstra University, http://www.hofstra.edu/academics/soeahs/TEC/tec_informed_design_resources.cfm (accessed May 21, 2008). 

Crismond, D. (1997), “Investigate-and-Redesign Tasks as a Context for Learning and Doing Science and Technology: A Study of Naive, Novice and Expert High School and Adult Designers Doing Product Comparisons and Redesign Tasks,” Unpublished doctoral thesis. Cambridge, MA: Harvard Graduate School of Education. 

Dishon, D. and P.W. O’Leary (1984), A Guidebook for Cooperative Learning: A Technique for Creating More Effective Schools. Holmes Beach, FL: Learning Publications, Inc. 

Online Media and Search: The New Opportunity (2007), “Fast Search and Transfer,” http://www.fastsearch.com/Online_Media_and_Search_The_New_Opportunity_BlbJf.pdf.file (accessed July 24, 2008). 

Papert, S. (1993), The Children’s Machine. New York: Basic Books. 

Resnick, M. (1998), “Technologies for Lifelong Kindergarten,” Educational Technology Research and Development, 46 (4), 43-55.. 

Soloway, E., M. Guzdial, and K. Hay (1994), “Learner-Centered Design,” Interactions, 1 (2), 36-48.

About the Authors

Michael Hanley (M.A., Ball State University) is an Assistant Professor of Advertising in the Department of Journalism at Ball State University. His research interests include interactive advertising, mobile advertising and marketing, new media, and advertising and gender issues. His work has appeared in the International Journal of Mobile Marketing, Journal of Human Subjectivity, and the Journal of Advertising Education.

Jennifer Palilonis (M.A., Ball State University) is an Assistant Professor in the Department of Journalism at Ball State University. Her research interests include design and content development for interactive news and advertising, research-informed development, multimedia storytelling, multimedia as a teaching and learning
tool, print and online design and blended learning. She is the author of “A Practical Guide to Graphics Reporting” and a media design consultant.

Vinayak Tanksale (M.S., Purdue University) is an instructor in the Department of Computer Science and a Faculty Fellow in the Insight and Research group of the Center for Media Design at Ball State University. His research interests include interactive news and advertising on television and mobile devices, Internet Protocol television (IPTV), convergent news, mobile technologies, and emerging media technologies.

Acknowledgments 

The authors thank Jackie Martinsen and Bill Vaughn of the Center for Media Design at Ball State University for their invaluable assistance with focus group research.