What kind of data do I need?

Wed Apr 25th 2012

In many academic disciplines, whether it be Sociology, Psychology, Linguistics, Marketing, or Human-Computer Interaction, researchers are often faced with a problem or a specific research question that requires empirical evidence to support their claims. In other words, rather than using theoretical arguments and analyses to solve that problem, these researchers must obtain actual data observed from a real-world situation that mimics the problem of interest and use data from that situation to address the research problem.

In exploratory studies, researchers could use the observed data to help formulate a hypothesis or a solution to the problem. In confirmatory studies on the other hand, researchers typically have a hypothesis to the problem that they want to test, and the observed data is used to support whether the hypothesis is correct or not.

Experiment design is a research area that prescribes the procedures for testing hypotheses in confirmatory studies. It is an area which is typically studied under the broader umbrella of research methods. In order to gather the necessary data to address a problem, researchers need to first determine the type of data that is needed. For example, is it qualitative or quantitative data? How will people be involved in the data collection process, is it better to have many participants once (e.g., cross subject or between subjects) or fewer participants in multiple sessions (e.g., within subject)? Is it necessary to conduct the same procedure on the same participants over time (e.g., longitudinal studies)?

Once the type of data has been determined, what methodology should be used to collect the information? Should interviews or focus groups be used? What about controlled laboratory settings in comparison to field studies? What are the tradeoffs in administering the different methods and the quality of the data obtained?

For the specific data needs and the method used for collecting data, what are the requirements on the amount of data that is needed (e.g., sample requirements)?

Finally, once the data has been collected, which analysis should be used to find the answers needed for the original research problem? Were there assumptions made about the situation and did the observed data meet those assumptions? Should a different analysis procedure be used if certain assumptions are not met? These are all questions that are addressed in identifying the appropriate research method to address a problem.

Rather than the question of having a problem and not knowing how to gather data for it, a variant of the question one might have is: "I have specific research questions that I need answers to. How do I go about collecting the data I need to answer these questions?" This situation is very similar to the original one, except that the researcher is already one step ahead into identifying specific research questions of interest. Furthermore, if these research questions are formulated in very precise terms, they are often easy to translate into hypotheses. In other words, researchers who are at this point would already know they want to do a confirmatory study (rather than an exploratory one), and the remaining steps are to identify the data collection needs, carry out it out, and conduct the analyses.

"Management wants a better understanding of employee efficiencies … Is Statistics applicable here?"

In the business world, the types of questions that arise tend to be of a different nature. Typically, the situation involves someone in a managerial role wanting to understand why something is happening (or not happening). For example, a commonly asked question is: "Certain staff members seem to be very efficient at some tasks but not others. Management wants a better understanding of employee efficiencies in order to better plan and prioritize their tasks. Is Statistics applicable here?"

The simple answer is "Yes". However, if this is a problem you would like answers to, don’t feel that you have to learn all the relevant Statistics inside and out, pick up Excel, and analyze everything from scratch. There is a wide range of project management tools available that are tailored to accommodate to businesses of different sizes and, in some cases, different industries.

The long answer involves understanding the business processes in practice and designing (or purchasing) the appropriate tools for project management, time tracking, project reporting, resource allocation, utilization projections, and potentially scenario analysis. Many software tools that have embedded Statistics functionality for these purposes are available for purchase or are free to use on the Web.

Statistics is a powerful tool that can provide summaries of what happened in the past as well as predict future performance based on history

While many organizations adopt technology to tackle this commonly arising problem, some managers fail to recognize that the success of the solution relies on organizational support and policies as well as the statistical output from these tools. It is crucial to realize that Statistics is a powerful tool that can provide summaries of what happened in the past as well as predict future performance based on history, but it is up to the organizations to decide how they want to use that information at the end of the day and implement suitable procedures that work for their business environment.

"Why aren’t users coming to my website?"

With the advent of the Web, many companies now have an online presence and make use of online marketing tactics to drive traffic away from their competitors and toward their site. In these cases, popular questions one might ask are: “Why aren’t users coming to my website?” or “Why aren’t users discovering these features of my website?” Many companies nowadays use Web analytics to address these questions. This means that different aspects of a website that indicate website performance may be monitored and measured. Simple examples include monitoring the number of unique users coming to the site on a daily basis. This information gives an indication of website traffic. If the company had carried out an online marketing campaign, this metric can be used to measure the success of that campaign.

Another metric is to measure the time each user spends on the site. This metric is used so often that it has a name – called stickiness. Did the user accidentally come to the site and end up leaving it right away? If so, why? Is it because they clicked on an ad and the initial impression of the site did not fulfill what the ad promised? Related to this metric is to monitor where users are coming from. What directed the user to this site? Is it a search engine – if so, which keywords were used? Is it an affiliate partner – if so, which one(s)? One might also be interested in knowing more specifically if certain parts of the site were useful to customers. In that case, rather than monitoring the time spent on the entire site, we can focus on monitoring the time spent on portions of the site. This type of metric is very useful for companies that use a subscription model, because it helps them understand the utility of the content that their customers subscribe to. For example, if certain content is underutilized, the company may decide to make it freely accessible, and to avoid publishing that type of content in future development.

Web analytics can be used to help understand user behavior on the Web and their preferences in website usage.

Furthermore, Web analytics can be used to help understand user behavior on the Web and their preferences in website usage. The examples above discussed only simple metrics, such as the number of users or the time spent per webpage. More complex analytics can be used to look at sequences of events that users do on the website in order to better understand the tasks they were engaged in as well as the context of those tasks. This kind of information can provide valuable feedback for companies and help them improve website usability as well as traffic. Taking this one step further would also enable companies to develop websites that adapt to the individual user’s experience, allowing the user to establish a more personal relationship with the company.


More Blogs

Recovering lost local git commits

A New Client: Thomas Hinds Tobacconist

The Choice Matrix - Official Launch!

A New Client: Manitoba Health

Manitoba Venture Challenge