Proposed Validation Requirements for Research Studies
The article by Straub, Boudreau, and Gefen (2004) highlights the continuing issues that Information System researchers have in ensuring the validity of their studies. The authors identify various validation requirements and propose a set of guidelines to assist researchers in the future. The guidelines for validation of research are categorized into mandatory, highly recommended, and optional.
The authors suggest that a heuristic approach is taken to validate research. Heuristics refers to problem-solving by taking a practical approach based on real life experiences (Harvey, 2007).
Validation Methods
The validation methods the authors mention in their literature review are instrumentation validity, internal validity, statistical conclusion, and external validity. Instrumentation validation is the most critical type of validation and ensures that the measurement tool is measuring the correct content. Internal validation refers to ruling out rival hypotheses. Statistical conclusion refers to the method used to demonstrate a certain degree of confidence that the data and testing of the data are correct. The authors do not fully describe external validity.
Key Validation Requirements
The key validation requirements tabled by the authors in order of priority are construct validity, reliability, manipulation validity, content validity, and predictive validity. Construct validity examines the measures to determine if they are capturing the intent of the construct. Reliability means the measures are accurate. Manipulation validity demonstrates that participants are aware that the variables have been manipulated. Predictive validity means the outcomes of the variables can be predicted. Content validity is determined through methods such as examination of the literature review and review by a panel of judges.
Quantitative Study - Review for Validity
The study by Szerb, Terjesen, and Rappai, (2007) explored how environmental country economics and individual characteristics affected the decision to invest in four different groups of informal investors. The authors conducted a quantitative study with the dependent variable in the study being the informal investor, and the independent variables being ten types of environmental data. The control variables were the individual demographics. The data collected covered 31 countries over a three-year period. There were 257,793 interviews conducted, which included 5,960 informal investors that were then categorized into four groups. The four groups were classified as classic love money, outsider, kin owner, and classic business angel.
There is no mention of any validation methods conducted in the study. One component I could test for validity in this study include content validity and for this, I would need a panel of experts. Statistical conclusion validity is available by review of each hypothesis result. I would need further information on the measures to be able to test construct validity or manipulation validity.
The positivist design contingency in the study by Szerb and Terjesen et al. (2007) was one of exploratory work. There is limited research available that explores the measures outlined in the study by Szerb and Terjesen et al. (2001&). Probing in new ways is apparent in the study.
References
Harvey, N (2007). Use of heuristics: Insights from forecasting research. Thinking & Reasoning, 13(1), 5-24. Doi: 10.1080/13546780600872502
Straub, D., Boudreau, M.-C., & Gefen, D. (2004). Validation guidelines for IS positivist research. Communications of the Association for Information Systems, 2004(13), 380-427. Retrieved from http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3243&context=cais
Szerb, L., Terjesen, S., & Rappai, G. (2007). Seeding new ventures-green thumbs and fertile fields: Individual and environmental drivers of informal investment. Venture Capital, 9(4), 257-284. doi:10.1080/13691060701414949
Article Review and Analysis
In this course, you have learned methods for statistically manipulating research data and explored the importance of ensuring validity in the methods that are used to collect research data. Consider the various aspects of validity you have explored. How might you prioritize their importance?
With these thoughts in mind:
Review the article "Validation Guidelines for IS Positivist Research," by Straub, Boudreau, & Gefen and focus your analysis on each of the following guiding questions. You should address each set of questions in a short paragraph. Be sure to refer directly to the article where appropriate.
- Briefly describe the types of validity presented. What are the critical differences among them? Assess the authors' performance in explaining them.
- Select a previous article from this course that presents a quantitative study. How would you assess the study's validity? What information would you need in order to be able to do so, and is that information present in the article?