Month: January 2018

How do I tell if the model is good enough if I only have R-squared to base it on? What’s a good value for  R-squared? How large does an R-squared need to be for the model to be valid? I am aware that there is the understanding among analysts that a model is not useful unless R-squared is at least some fraction greater than 50%. Is this the standard to be followed?

Generally, it is better to look at adjusted R-squared as these are unbiased estimators that correct for sample size and number of parameters estimated.

In marketing science, particularly in the study of customer behavior,  no hard cut-off has ever become the accepted norm for the value of R-squared (or adjusted R-squared for that matter). In marketing we are making predictions about human behavior and not the workings for a physical system, so to get to any level of insight from nothing is a big win.

Thus I would advise that in this case R² should be “large’, meaning, large enough in the eye of the experimenter, and what is large varies a great deal according to the type of experiment being conducted. For example in social science experiments I would be delighted to get a statistically significant R² value as low as 0.20. But in natural sciences and engineering experiments, I wouldn’t be happy with R² values that are lower than 0.90.

As a general rule in experimental designs, the more the researcher knows about the science, the better controlled the experiment can be. Hence, the expectation for R² increases. Obviously, human behavior is poorly understood so R² values that physicists and engineers would outright dismiss are acceptable. I wouldn’t even go to the topic of over-fitting.

A good rule of thumb is to consider the context of the experiment. If our projects involve customer response, then statistically significant R² values around 0.50 can be good enough to give us a basis for improvement. But if we are looking at cycle time through process, we would want a higher threshold, say 0.70. Still, you can tell these are arbitrary. As with every statistical approach, you need to have a fairly good idea of the underlying phenomena that you are trying to model.

The most important thing to remember is that we want our data to point us in the right direction for making improvements. What R² will do for us varies depending on the context of the experiment. I wouldn’t put too much stock on a single metric like R² if there are other available metric we can use to validate the model.

Why come up with this website? What is the purpose of this website?

During the course of my marketing science career, too many times what I do has been called magic – MackyMagic – as its own brand of effectiveness in producing results. However, as what Arthur C. Clarke has said, “Magic is just science we don’t understand yet”. 

With this page I want contribute to a better understanding of marketing science by de-mystifying it for audiences. How? Letting the audience ask the big questions.

Marketing Science tends to be off-putting with all the snobbish buzzwords. Not only that, marketing scientists tends to be preachy when explaining a concept, usually with an air of arrogance and exclusivity, which further alienates even the most experienced marketers. I find that an effective way to make marketing science relatable is to let the audience ask the questions. This not only sets the context as to how advanced the language of the reply should be, it also mitigates being preachy, and most importantly, it enforces me to put myself in the shoes of the marketer that leads to a better understanding of the issue at hand. Along these lines, if I get I question asking about R-squared (i.e. what’s an acceptable value for r-squared), unless it is actually a question about what it is, I would assume that the asker already knows what R-squared is.

In recognizing that the skills needed to be an effective marketing scientist is not monopolized by only those with advance analytics skills; a marketer is best placed to be a marketing scientist by having the right tools, support, and the right information.

I want to be able to help propagate a culture of evidence-based decision-making, assuring us that product and marketing decisions will be more effective in achieving desired outcomes when based on accurate and meaningful information.