Often variables and tests seem straightforward, even obvious, that is what we call face validity. The problem with face validity is that it is too often not supported by real relationships. The bigger the dataset, the more likely minor relationships seem real.
The old reliance on statistically significant results falls away as we use ever-larger datasets. Many statistical tests, especially those used on descriptive data are sensitive to large numbers. Thus, you are more likely to find relationships that are not important. In addition, bias built into the variables can lead you to generalize in a way that is neither fair nor profitable.
In smaller surveys and samples, careful wording of questions and variable choices become even more important. It is more difficult to find relationships. If the subject does not understand the question, or there is bias in the measure, a true relationship can be missed or misinterpreted.
The data do not always tell you what you think. The first step of understanding the variables is far too easy to rush through. The excitement of collecting data or the pressure to produce quick and dirty results works against the quality of the research. This is where professional researchers help the most.
Modern Media Barn 