We recently sat down with one of our Launch teams to discuss what they have learned so far about market testing from the Launch faculty, as well as their plans for future market research. Reina, a Research Scientist at iRT (innovation Research & Training) is working on an innovation to supplement iRT’s existing program: Media Aware for Young Adults. She shares how her team’s research background provides a strong foundation for learning new skills for market testing.
Recently our team completed a first round of market testing that will be used to create a prototype of our new product as part of our Launch experience. Market testing is a critical, new method for us to learn and use as research scientists who are interested in wide adoption of new interventions. It is also an important method for other teams in our cohort to learn, as we all work to refine and advance our respective innovations.
Our Innovation
My teammates and I are working on developing an innovative resource to complement an existing online program called Media Aware, which was created by researchers at iRT and is grounded in media literacy and aimed at promoting sexual health and preventing sexual assault. Media Aware’s goal is to teach young adults to think critically about media messages about sex and relationships, and internalize those skills so that they use them when they encounter media messages regularly in their everyday lives. The program also provides medically-accurate information, fills in gaps where information is needed, and debunks misinformation about sex and relationships.
Through previous rigorous testing, which was conducted in a randomized controlled trial, we know Media Aware works—it is efficacious in decreasing sexual risk behaviors in community college students.
During our journey with Launch, we are creating an online library of media examples to supplement the Media Aware program that can be used by college health educators interested in sparking discussions with college students. The library will contain media examples that are timely and relevant to a wide range of youth, as well as educators’ guides to support in-classroom discussions. We want to include this library of continuously updated media examples, because we understand that students learn better when they see people and examples they can relate to in a curriculum.
Both from our initial evaluation of Media Aware and from other studies we have pursued over our careers, my teammates and I have a deep background in conducting scientific research. When Launch required that we conduct market testing, we were excited to do so, as the idea of conducting tests to find answers to important questions was nothing new to us; however, market testing required us to apply our expertise in conducting research to a new context. We have noticed some distinct differences between the scientific research studies that we typically conduct and this newer method of market research. It’s been interesting and gratifying to learn these new processes, while also flexing familiar skills.
Methods and Instruments
Traditionally in our research on program implementation, we use qualitative interviews and web-based instruments such as online surveys to gather feedback. We assess constructs such as program acceptability, usability, and feasibility; barriers and facilitators to implementation; and whether respondents could actually see themselves implementing a program. We also ask about whether there are parts of a program they liked as well as parts they would like to see improved.
We leveraged our interviewing skills for our initial round of market testing, during which time we conducted focus groups with college students and educators to inform the development of an initial prototype of the online library. We aimed to better understand how these key stakeholders envisioned themselves interacting with the online library and what factors may be most effective at motivating educators and students to engage with the library in the future.
For our upcoming round of market research, we have been considering new types of testing. There are a variety of instruments available to use to gather feedback from end users, which we have been learning about during our meetings with our Launch faculty. Our team has chosen to embark on a “pretend to own” task, in which we will ask college educators and college students to use the program prototype as if it was part of their daily lives. As users complete this task, we will observe them and ask them to think aloud as they are using it. While these live listening sessions are different from the formalized surveys and interviews we typically use, we are hoping to gain great insights from observing what’s working smoothly, identifying potential pitfalls, and integrating these findings together to inform the iterative development of a new prototype of our product.
Speed
One thing that differs greatly between scientific research and market research is the speed at which it is conducted. A scientific research study could take months or even years to design and complete. But when you’re building an innovation, you have to move much more quickly. For example, in our market testing, we collected data and began to use the information to refine designs for our innovation over the course of a couple of weeks.
Analysis
In a typical study, we would analyze data once all of the data from the entire sample was collected. However, in market research, we analyzed the data in real time and made changes to the prototype as we heard new and critical feedback. We’ve found that market research is less about synthesis and more about continued and iterative revision—and, as researchers who are used to rigorous evaluation, we’ve had to learn to be comfortable pivoting quickly without using rigorous experimental methods.
Looking Ahead
While it has been nerve-wracking, at times, to deviate from our training in standard rigorous evaluation methods, it has also been very exciting and rewarding. We have been able to see our progress in real time, as we make changes based upon what we learned. During the second half of the Launch program, we will conduct our “pretend to own” testing, continue learning from potential end users, and address challenges by tweaking our prototype or by pivoting. We look forward to growing new skills as scientists, gaining valuable insights, and continuing to practice our new market research skills!