The Use of Online Panel Data in Management Research: A Review and Recommendations



Aguinis, H., Lawal, S. O. 2013. eLancing: A review and research agenda for bridging the science–practice gap. Human Resource Management Review, 23: 617.
Google Scholar | ISI


Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., Rao, S. M. 2018. Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board Task Force report. American Psychologist, 73: 325.
Google Scholar | Medline


Aust, F., Diedenhofen, B., Ullrich, S., Musch, J. 2013. Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45: 527535.
Google Scholar | Medline | ISI


Baer, M. D., Bundy, J., Garud, N., Kim, J. K. 2018. The benefits and burdens of organizational reputation for employee well-being: A conservation of resource approach. Personnel Psychology, 71: 571595.
Google Scholar


Behrend, T. S., Sharek, D. J., Meade, A. W., Wiebe, E. N. 2011. The viability of crowdsourcing for survey research. Behavior Research Methods, 43: 800813.
Google Scholar | Medline | ISI


Bono, J. E., Glomb, T. M., Shen, W., Kim, E., Koch, A. J. 2013. Building positive resources: Effects of positive events and positive reflection on work stress and health. Academy of Management Journal, 56: 16011627.
Google Scholar | ISI


Boswell, W. R., Olson-Buchanan, J. B., Harris, T. B. 2014. I cannot afford to have a life: Employee adaptation to feelings of job insecurity. Personnel Psychology, 67: 887915.
Google Scholar | ISI


Brawley, A. M., Pury, C. L. 2016. Work experiences on MTurk: Job satisfaction, turnover, and information sharing. Computers in Human Behavior, 54: 531546.
Google Scholar | ISI


Buhrmester, M., Kwang, T., Gosling, S. D. 2011. Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6: 35.
Google Scholar | SAGE Journals | ISI


Callegaro, M., Baker, R. P., Bethlehem, J., Göritz, A. S., Krosnick, J. A., Lavrakas, P. J. 2014. Online panel research: A data quality perspective. New York: Wiley.
Google Scholar


Chandler, J., Mueller, P., Paolacci, G. 2014. Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46: 112130.
Google Scholar | Medline | ISI


Chandler, J., Paolacci, G., Peer, E., Mueller, P., Ratliff, K. A. 2015. Using nonnaive participants can reduce effect sizes. Psychological Science, 26: 11311139.
Google Scholar | SAGE Journals | ISI


Chandler, J., Shapiro, D. 2016. Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12: 5381.
Google Scholar | Medline


Cheung, J. H., Burns, D. K., Sinclair, R. R., Sliter, M. 2017. Amazon Mechanical Turk in organizational psychology: An evaluation and practical recommendations. Journal of Business Psychology, 32: 347361.
Google Scholar


Chua, R. Y. J., Morris, M. W., Ingram, P. 2009. Guanxi vs. networking: Distinctive configurations of affect- and cognition-based trust in the networks of Chinese vs. American managers. Journal of International Business Studies, 40: 490509.
Google Scholar | ISI


Clark, T. 2011. Gaining and maintaining access: Exploring the mechanisms that support and challenge the relationship between gatekeepers and researchers. Qualitative Social Work, 10: 485502.
Google Scholar | SAGE Journals | ISI


Cohen, J. 1992. A power primer. Psychological Bulletin, 112: 155159.
Google Scholar | Medline | ISI


Crilly, D., Ni, N., Jiang, Y. 2016. Do-no-harm versus do-good social responsibility: Attributional thinking and the liability of foreignness. Strategic Management Journal, 37: 13161329.
Google Scholar


Crone, D. L., Williams, L. A. 2017. Crowdsourcing participants for psychological research in Australia: A test of microworkers. Australian Journal of Psychology, 69: 3947.
Google Scholar


Crump, M. J., McDonnell, J. V., Gureckis, T. M. 2013. Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One, 8: e57410.
Google Scholar


Cunliffe, A. L., Alcadipani, R. 2016. The politics of access in fieldwork: Immersion, backstage dramas, and deception. Organizational Research Methods, 19: 535561.
Google Scholar | SAGE Journals | ISI


DeVoe, S. E., House, J. 2016. Replications with MTurkers who are naïve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015). Journal of Experimental Social Psychology, 67: 6567.
Google Scholar


Downs, J. S., Holbrook, M. B., Peel, E. 2012. Screening participants on Mechanical Turk: Techniques and justifications. Advances in Consumer Research, 40: 113114.
Google Scholar


Elenkov, D. S., Judge, W., Wright, P. 2005. Strategic leadership and executive innovation influence: An international multi-cluster comparative study. Strategic Management Journal, 26: 665682.
Google Scholar | ISI


Feitosa, J., Joseph, D. L., Newman, D. A. 2015. Crowdsourcing and personality measurement equivalence: A warning about countries whose primary language is not English. Personality and Individual Differences, 75: 4752.
Google Scholar | ISI


Fowler, F. J. 2014. Survey research methods. Thousand Oaks, CA: Sage.
Google Scholar


Gamache, D. L., McNamara, G., Mannor, M. J., Johnson, R. E. 2015. Motivated to acquire? The impact of CEO regulatory focus on firm acquisitions. Academy of Management Journal, 58: 12611282.
Google Scholar | ISI


Gleibs, I. H. 2017. Are all “research fields” equal? Rethinking practice for the use of data from crowdsourcing market places. Behavior Research Methods, 49: 13331342.
Google Scholar | Medline


Goodman, J. K., Cryder, C. E., Cheema, A. 2013. Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26: 213224.
Google Scholar | ISI


Goodman, J. K., Paolacci, G. 2017. Crowdsourcing consumer research. Journal of Consumer Research, 44: 196210.
Google Scholar | ISI


Göritz, A. S. 2007. Using online panels in psychological research. In Joinson, A. N., McKenna, K. Y. A., Postmes, T., Reips, U.-D. (Eds.), The Oxford handbook of Internet psychology: 473485. Oxford, UK: Oxford University Press.
Google Scholar


Harmon, D. J., Kim, P. H., Mayer, K. J. 2015. Breaking the letter vs. spirit of the law: How the interpretation of contract violations affects trust and the management of relationships. Strategic Management Journal, 36: 497517.
Google Scholar


Hauser, D. J., Schwarz, N. 2016. Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48: 400407.
Google Scholar | Medline | ISI


Huang, J. L., Bowling, N. A., Liu, M., Li, Y. 2015. Detecting insufficient effort responding with an infrequency scale: Evaluating validity and participant reactions. Journal of Business and Psychology, 30: 299311.
Google Scholar | ISI


Ilgen, D. R. 1986. Laboratory research: A question of when, not if. In Locke, E. A. (Ed.), Generalizing from laboratory to field settings: 257267. Lexington, MA: Heath.
Google Scholar


Insko, C. A., Wildschut, T., Cohen, T. R. 2013. Interindividual–intergroup discontinuity in the prisoner’s dilemma game: How common fate, proximity, and similarity affect intergroup competition. Organizational Behavior and Human Decision Processes, 120: 168180.
Google Scholar | ISI


Judge, T. A., Cable, D. M., Colbert, A. E., Rynes, S. L. 2007. What causes a management article to be cited—article, author, or journal? Academy of Management Journal, 50: 491506.
Google Scholar | ISI


Judge, T. A., Ilies, R., Scott, B. A. 2006. Work–family conflict and emotions: Effects at work and at home. Personnel Psychology, 59: 779814.
Google Scholar | ISI


Keith, M. G., Harms, P. D. 2016. Is Mechanical Turk the answer to our sampling woes? Industrial and Organizational Psychology, 9: 162167.
Google Scholar


Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahnik, S., Bernstein, M. J., . . . Nosek, B. A. 2014. Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45: 142152.
Google Scholar | ISI


Landers, R. N., Behrend, T. S. 2015. An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Industrial and Organizational Psychology, 8: 142164.
Google Scholar | ISI


Li, H., Kuo, C., Russell, M. G. 1999. The impact of perceived channel utilities, shopping orientations, and demographics on the consumer’s online buying behavior. Journal of Computer-Mediated Communication, 5: 123.
Google Scholar


Litman, L., Robinson, J., Rosenzweig, C. 2015. The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behavior Research Methods, 47: 519528.
Google Scholar | Medline | ISI


Little, L. M., Major, V. S., Hinojosa, A. S., Nelson, D. L. 2015. Professional image maintenance: How women navigate pregnancy in the workplace. Academy of Management Journal, 58: 837.
Google Scholar | ISI


Lovett, M., Bajaba, S., Lovett, M., Simmering, M. J. 2018. Data quality from crowdsourced surveys: A mixed method inquiry into perceptions of Amazon’s Mechanical Turk masters. Applied Psychology: An International Review, 67: 339366.
Google Scholar


Mason, W., Suri, S. 2012. Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44: 123.
Google Scholar | Medline | ISI


Meade, A. W., Craig, S. B. 2012. Identifying careless responses in survey data. Psychological Methods, 17: 437455.
Google Scholar | Medline | ISI


Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., Lynam, D. R. 2017. Using online, crowdsourcing platforms for data collection in personality disorder research: The example of Amazon’s Mechanical Turk. Personality Disorders: Theory, Research, and Treatment, 8: 2634.
Google Scholar | Medline


Mochon, D., Frederick, S. 2013. Anchoring in sequential judgments. Organizational Behavior and Human Decision Processes, 122: 6973.
Google Scholar


Ng, T. W., Feldman, D. C. 2012. The effects of organizational and community embeddedness on work-to-family and family-to-work conflict. Journal of Applied Psychology, 97: 12331251.
Google Scholar | Medline | ISI


Ng, T. W., Feldman, D. C. 2015. Idiosyncratic deals and voice behavior. Journal of Management, 41: 893928.
Google Scholar | SAGE Journals | ISI


Paolacci, G., Chandler, J. 2014. Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23: 184188.
Google Scholar | SAGE Journals | ISI


Paolacci, G., Chandler, J., Ipeirotis, P. G. 2010. Running experiments on Amazon Mechanical Turk. Judgement and Decision Making, 5: 411419.
Google Scholar | ISI


Peer, E., Brandimarte, L., Samat, S., Acquisti, A. 2017. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70: 153163.
Google Scholar | ISI


Peer, E., Vosgerau, J., Acquisti, A. 2014. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46: 10231031.
Google Scholar | Medline | ISI


Peterson, R. A., Merunka, D. R. 2014. Convenience samples of college students and research reproducibility. Journal of Business Research, 67: 10351041.
Google Scholar


Phillips, J. M., Gully, S. M., McCarthy, J. E., Castellano, W. G., Kim, M. S. 2014. Recruiting global travelers: The role of global travel recruitment messages and individual differences in perceived fit, attraction, and job pursuit intentions. Personnel Psychology, 67, 153201.
Google Scholar | ISI


Piccolo, R. F., Colquitt, J. A. 2006. Transformational leadership and job behaviors: The mediating role of core job characteristics. Academy of Management Journal, 49: 327340.
Google Scholar | ISI


Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., Podsakoff, N. P. 2003. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88: 879903.
Google Scholar | Medline | ISI


Ramsey, S. R., Thompson, K. L., McKenzie, M., Rosenbaum, A. 2016. Psychological research in the internet age: The quality of web-based data. Computers in Human Behavior, 58: 354360.
Google Scholar


Rouse, S. V. 2015. A reliability analysis of Mechanical Turk data. Computers in Human Behavior, 43: 304307.
Google Scholar | ISI


Shadish, W. R., Cook, T. D., Campbell, D. T. 2002. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin.
Google Scholar


Shapira, Z., Shaver, J. M. 2014. Confounding changes in averages with marginal effects: How anchoring can destroy economic value in strategic investment assessments. Strategic Management Journal, 35: 14141426.
Google Scholar


Simmons, J. P., Nelson, L. D., Simonsohn, U. 2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22: 13591366.
Google Scholar | SAGE Journals | ISI


Smith, N. A., Sabat, I. E., Martinez, L. R., Weaver, K., Xu, S. 2015. A convenient solution: Using MTurk to sample from hard-to-reach populations. Industrial and Organizational Psychology, 8: 220228.
Google Scholar | ISI


Stritch, J. M., Pedersen, M. J., Taggart, G. 2017. The opportunities and limitations of using Mechanical Turk (Mturk) in public administration and management scholarship. International Public Management Journal, 20: 489511.
Google Scholar


Swaab, R. I., Phillips, K. W., Schaerer, M. 2016. Secret conversation opportunities facilitate minority influence in virtual groups: The influence on majority power information processing, and decision quality. Organizational Behavior and Human Decision Processes, 133, 1722.
Google Scholar


Texas A&M University & University of Georgia . 2018. Texas A&M/University of Georgia rankings of management department research productivity. Retrieved from http://www.tamugarankings.com/rankings/2017-2/
Google Scholar


Tosti-Kharas, J., Conley, C. 2016. Coding psychological constructs in text using Mechanical Turk: A reliable, accurate, and efficient alternative. Frontiers in Psychology, 7: 741.
Google Scholar | Medline


Tracy, S. J. 2013. Qualitative research methods: Collecting evidence, crafting analysis, communicating impact. New York: Wiley-Blackwell.
Google Scholar


University of Texas at Dallas . 2018. The UTD top 100 business school research rankings. Retrieved from: http://jindal.utdallas.edu/the-utd-top-100-business-school-research-rankings/index.php
Google Scholar


Walter, S., Seibert, S., Goering, D. D., O’Boyle, E. H. 2018. A tale of two sample sources: Do results from online panel data and conventional data converge? Journal of Business and Psychology. Advance online publication. doi:10.1007/s10869-018-9552-y
Google Scholar


Woo, S. E., Keith, M., Thornton, M. A. 2015. Amazon Mechanical Turk for industrial and organizational psychology: Advantages, challenges, and practical recommendations. Industrial and Organizational Psychology, 8: 171179.
Google Scholar | ISI


Wowak, A. J., Mannor, M. J., Arrfelt, M., McNamara, G. 2016. Earthquake or glacier? How CEO charisma manifests in firm strategy over time. Strategic Management Journal, 37: 586603.
Google Scholar


Yamagishi, T., Mifune, N., Li, Y., Shinada, M., Hashimoto, H., Horita, Y., . . . Simunovic, D. 2013. Is behavioral pro-sociality game-specific? Pro-social preference and expectations of pro-sociality. Organizational Behavior and Human Decision Processes, 120: 260271.
Google Scholar | ISI



Source link

Telefono: 081 18676363
Fax: 081 18676362
80133, Napoli, Italy
Piazza Bovio 33