RESEARCH ARTICLE
The Motivation Competencies That Count Most: An Online International Study
Robert Epstein1, *, Megan E. Ho1, Zoë Scandalis1, Anna Ginther1
Article Information
Identifiers and Pagination:
Year: 2022Volume: 15
E-location ID: e187435012202030
Publisher ID: e187435012202030
DOI: 10.2174/18743501-v15-e2202030
Article History:
Received Date: 8/8/2021Revision Received Date: 22/10/2021
Acceptance Date: 9/12/2021
Electronic publication date: 08/03/2022
Collection year: 2022
open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Abstract
Background:
With an online sample of 8,349 people from 123 countries (74.9% from the U.S., Canada, and India), a new test was used to rank eight motivation-related competencies according to how well they predicted desirable, self-reported outcomes. Each of the competencies was derived from empirical studies showing that such competencies were associated with higher levels of motivation. The competencies were: Maintains Healthy Lifestyle, Makes Commitments, Manages Environment, Manages Rewards, Manages Stress, Manages Thoughts, Monitors Behavior, and Sets Goals.
Objective:
The study was conducted to identify and prioritize competencies that are associated with higher levels of motivation.
Methods:
A “concurrent study design” was used to assess predictive validity, which was suggested by a strong association between test scores and self-reported answers to criterion questions about levels of motivation, life satisfaction, and professional success. Regression analyses were conducted to prioritize the competencies. Demographic analyses were also conducted.
Results:
The findings support the value of motivation training; test scores were higher for people who had received such training and were positively correlated with the number of training hours accrued. Effects were found for education, race and age, but no male/female difference was found. Regression analyses pointed to the importance of two of the eight competencies in particular: Sets Goals and Manages Thoughts.
Conclusion:
The study supports the view that motivation competencies can be measured and trained and that they are predictive of desirable motivational outcomes.
1. INTRODUCTION
Motivation has been discussed from many different perspectives since it was first introduced as a useful research construct in the early 1900s [1-4]. Generally speaking, the term is said to refer to hypothetical internal states of an organism that are believed to somehow energize the organism to behave in certain ways [5-7]. The Oxford English Dictionary includes modern-sounding uses of the language of motivation as early as the late-1300s, by which time “motive” was already referred to as “an inward prompting or impulse” [8].
In early formulations of the term in psychology, inner motivational states were often described as aversive – that is, as unpleasant states that an organism sought to reduce or eliminate [1, 9]. The internal states are normally inferred from behavior, but behavior alone is not always a good indicator; one can eat, for example, without being hungry. Context is important in making inferences about motivation [10, 11], and so are aspects of an organism’s history that produce a motivational state – what behavioral psychologists call “establishing operations” [12]. Contemporary discussions about motivation focus on a number of different issues, such as the relative contributions of intrinsic versus extrinsic motivation [13-19], theories of motivation [20-22], environmental and personality factors affecting motivation [23-27], and how people become motivated in particular settings or situations, such as in the work environment [28-31], in sports and exercise activities [32-37] and educational settings [38-41].
A comprehensive review of different approaches to the study of motivation is beyond the scope of this paper. Instead, we will focus on just one approach to motivation that has received relatively little attention, and that is the competencies approach. What types of knowledge and skills – preferably that can be both measured and trained – help to boost motivation? David McClelland’s work is relevant here, given that his early research and theoretical work focused on motivation [42, 43] and given that an article he published in the American Psychologist in 1973 helped propel a competencies movement that continues to impact business, education, government, and other domains in which human performance is important [44]. Much of the research McClelland and his colleagues conducted on competence was performed through his company, McBer, in the 1970s and 80s [45] and was eventually summarized in books published in 1982 and 1993 [46, 47]. The McBer organization focused on measuring and training competencies to improve leadership performance, often by using structured interviews (“behavioral event interviews”) to determine what competencies were common among top performing leaders [47]. McClelland and his associates did not, however, look generally at which measurable and trainable competencies underlie motivation per se. Nor did they develop written tests to assess such competencies [48, 49].
One advantage of a competencies approach is that it avoids labeling. Because competencies can be improved, even low scores on a competencies test can give people hope. A competencies approach breaks complex and sometimes mysterious looking abilities like “leadership” into a set of measurable, trainable skills [50]. Generally, when people know it is possible for them to improve, they feel and perform better [40, 51].
A number of tests have been developed over the years to measure how motivated people are, such as the Motivation Analysis Test [52, 53] and the Managing by Motivation Questionnaire [54], but no tests that we are aware of look at the broad range of competencies that might energize people to behave. Andersen [49] described a new test that specifically measures three important types of motivation – power, achievement, and affiliation – that McClelland himself showed were important in leadership and management.
Other tests that measure level of motivation include the Achievement Motivation Inventory [55], the Athletic Motivation Inventory [56], the Miller Motivation Scale [57], the Sales Motivation Inventory [58], the Work Motivation Inventory [59], and the Volunteer Motivation Inventory [60]. At first glance, the Motivational Sources Inventory [61] would appear to look at competencies of some sort, but in fact, it purports to measure only internal mental components of motivation. The 20th edition of the Mental Measurements Yearbook (MMY) [62] lists 26 validated tests that measure levels of or various aspects of motivation, but none are listed that measure competencies that might boost motivation.
The internet has complicated matters, allowing any individual or group to post any sort of test, with users generally unaware that no attempt has been made to evaluate the test. Non-validated tests of motivation appear on websites such as https://www.psychologytoday.com, https://www.queendom. com/, https://www.seemypersonality.com. Sometimes these tests include disclaimers such as, “this report is intended for personal growth purposes only,” and sometimes they offer minimal results and request a fee to see more detailed results. No regulations exist to limit the posting of such tests, so they will inevitably grow in number over time. With more and more people going first to the internet to get information about physical and mental health [63], the spread of non-validated tests is problematic, and there is no easy solution to this problem.
Several tests (none of which is listed in the MMY) measure competencies that managers or teachers need to motivate other people. Ololube [64] described such a test designed for use with teachers in Nigeria, for example, and in 2008 the first author of the present paper posted a managerial version of the test described herein (the EMCI-m, accessible at https://mymotivationskills.com/managers/). In addition, business training programs have relied on tests that measure a range of managerial competencies, including competencies that increase employee motivation [65]1. Breaking down human performance into measurable, trainable competencies has proved to be valuable in areas such as education [66-68], business [69-71], the military [72], sports [73], and even, more recently, in areas such as creativity [74, 75], stress management [76, 77], and relationships [78-80].
The present paper investigates a competencies approach to the study of motivation, posing the question: Could acquiring certain types of knowledge and skills increase levels of motivation and perhaps even improve life outcomes? We use a “concurrent study design” to evaluate some aspects of the predictive validity of a new test that measures such competencies, and we rank order those competencies according to how well they predict self-reported levels of motivation, professional success, and life satisfaction – desirable outcomes that might conceivably be more likely if people have high levels of motivation competencies. We also look at demographic differences in test scores, specifically by age, gender, race, education level, and location.
2. METHODS
2.1. Participants
The present study analyzed data obtained from a convenience sample of 8,349 individuals in 123 countries who took the EMCI-i at the website https://MyMoti vationSkills.com/ between May 20, 2007 and June 25, 2020. Participants were not actively recruited. They presumably found the test through search engines or through links to the test posted at various times on a variety of different websites, among them: https://motivationshow.com, https://itsallabout focus.com, and https://motivation-goals.com.
Our original sample contained 9,123 cases before cleaning, 8,349 cases after cleaning. The 8,349 cases make up our final sample and are the subject of our statistical analyses. Cases were removed when participants answered fewer than half the questions, reported English fluency levels under 6 (on a scale from 1 to 10, where 10 was the highest possible fluency), or appeared to be under age 13 (see below). When people took the test more than once, we included only data obtained on the first trial in which they answered at least half the questions. Regarding location, 5,084 (63.9% of the number of cases for which location was known) of our participants were from the U.S. or Canada, 874 (11.0%) were from India, 2,000 (25.1%) were from 120 other countries, and location was unknown for 391 (4.7%) people. In total, therefore, the location was known for 7,958 (95.3%) of our participants, and 74.9% of those individuals were from the U.S., Canada, or India.
The mean age of the participants was 29.3 (SD = 12.8, mode = 18, range 13 to 90). A minimum-age cutoff of 13, which was approved by our Institutional Review Board (IRB), was used given that the Flesch-Kincaid reading level of our test questions was 7.1 and given that 13 is the typical age at which students in the U.S. have completed the 7th grade. (We acknowledge that many people at this age read below their grade level.) In all, 658 (7.9%) of our participants (after data cleaning) were under age 18.
3,448 (41.3%) of our participants identified themselves as male, 4,846 (58.0%) as female, and 55 (0.7%) as other. 407 (4.9%) said they lacked a high school degree, 3,042 (36.4%) said they had received a high school degree, 518 (6.2%) said they had received a 2-year associates degree, 2,790 (33.4%) said they had received a college degree, 1,324 (15.9%) said they had received a master’s degree, and 212 (2.5%) said they had received a doctoral degree; educational level was unknown for 56 (0.7%) participants.
4,104 (49.2%) identified themselves as white, 1,481 (17.7%) as Asian, 1,749 (20.9%) as Hispanic, 497 (6.0%) as black, 37 (0.4%) as Native American, and 358 (4.3%) as other; race/ethnicity was unknown for 121 (1.4%) participants. Finally, after cleaning, the average English fluency reported by participants who had answered our fluency question was 9.1 (SD = 1.2).
1,324 (15.9%) of our participants reported that they had training in motivation, and 6,959 (83.4%) said they had not; training history was unknown for 66 (0.8%) participants. When asked how motivated they were (on a scale from 1 to 10 where 10 was the highest possible answer), participants reported an average score of 6.8 (SD = 2.1). When asked how satisfied they were with their life, participants reported an average score of 6.5 (2.3). When asked how much success they had had in their professional life, participants reported an average score of 6.3 (2.3).
2.2. Materials and Design
Our investigation employed a “concurrent study design” that provided convergent validity evidence with related measures, following the most recent guidelines of Standards for Educational and Psychological Testing [79, cf. 80], co-published by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. Specifically, we sought to measure the strength of the relationships between our test scores and the scores on our self-reported criterion questions. This design is called “concurrent” because we obtained test scores and criterion measures at the same time, a strategy that avoids possible temporal confounds. Results from studies employing this design are considered especially robust when the pattern of relationships between test scores and criterion measures proves to be consistent across different demographic groups.
The test instrument employed in this study is called the Epstein Motivation Competencies Inventory for Individuals (EMCI-i) and was developed in 2007 and posted online that year. The EMCI-i has 40-items, each of which requires an answer on a 5-point Likert scale, with points labeled Agree and Disagree at the extremes. It assesses eight competencies that both research and theory suggest are important for motivation. Definitions and relevant references, which have been updated since the test was first posted, are given in Table 1.
Table 1. Eight motivation competencies.
|
Maintains a healthy lifestyle: You maintain a healthy lifestyle and good habits. You take good care of your body and brain. Sample item: “I avoid substances that might damage my body or brain.” References: [81–86] |
|
Makes commitments: You make commitments to others in a way that boosts your performance. Sample item: “I always keep my promises.” References: [87 – 94] |
|
Manages environment: You alter and maintain both your social and physical environments to keep yourself motivated. Sample item: “I use ergonomic devices to help me perform at my peak.” References: [95 – 100] |
|
Manages rewards: You manage rewards in a way that boosts your motivation. Sample item: “I often ask others for encouragement and support.” References: [85, 101–106] |
|
Manages stress: You take steps to reduce stress for yourself, practice relaxation techniques, and also take a proactive approach to stress management. You practice techniques that help you avoid or minimize stress. Sample item:“I often try to identify and remove sources of stress in my life.” References: [107–110] |
|
Manages thoughts: You manage your thinking to keep yourself motivated. Sample item: “I frequently try to picture a bright future for myself.” References: [107, 110-119] |
|
Monitors behavior: You monitor your behavior and progress. In almost all situations, keeping records of performance boosts performance. Sample item: “I use many different methods to monitor my behavior.” References: [109, 120 – 127] |
|
Sets goals: You set short and long term goals for yourself. Sample item: “I typically spend a few minutes each day to plan the day.” References: [91, 101, 109, 128-149] |
2.3. Procedure
Participants were first given basic information about motivation and then informed that they would be taking “an inventory of eight types of skills and knowledge that help people boost their motivation or keep themselves motivated.” They were also told: “Most people are able to complete the test in less than 10 minutes, and there are no right or wrong answers. Just select the response that seems best. After you answer the questions, you will be given a detailed report that will give you your scores and explain what they mean.”
Participants were then asked basic demographic questions, along with three criterion questions that they answered on a 10-point Likert scale, as follows: (1) “Generally speaking, how motivated are you?” (scale from Not at all to Highly motivated). (2) “How satisfied are you with your life?” (scale from Not at all to Extremely satisfied). (3) “How much success have you had in your professional life?” (scale from Low to High). Participants were also asked whether they had ever received any training intended to boost their motivation and how many hours of training they had received. The test itself followed.
After completing the test, the test taker clicked on a “Submit” button to generate a detailed report defining the eight competencies and showing his or her total score and the score in each competency area. It also gave people an email address where they could get additional information about the ongoing study or, consistent with standards of the American Psychological Association (APA) and the United States Department of Health and Human Services (HHS), could request that their data be deleted.
Each competency is measured by five items, one of which is a dummy item – a slightly reworded version of another item in the same competency area – that can be used to assess the consistency of responding as soon as an individual completes the test [75]. None of the eight dummy items was scored or included in any of the statistical analyses in the present study.
3. RESULTS
3.1. Reliability and Validity Evidence
Internal-consistency reliability as measured by traditional reliability coefficients was relatively high for total scores: Cronbach’s alpha was 0.88, and the Guttman split-half measure was 0.84. For the eight individual competencies, Cronbach’s alpha ranged from 0.32 (for Maintains Healthy Lifestyle) to 0.72 (for Sets Goals), and the Guttman split-half measure ranged from 0.23 (for Maintains Healthy Lifestyle) to 0.74 (for Manages Thoughts). Test-retest reliability was not measured (see Discussion). We also did not develop an alternate form of the test, so alternate-form reliability could not be estimated.
Regarding evidence of predictive validity, test scores were positively correlated with participants’ answers to all three of our criterion questions: their estimates of their own level of motivation (Spearman’s ρ = 0.55, P < 0.001)2; their self-reported life satisfaction (ρ = 0.46, P < 0.001); and their self-reported professional success (ρ = 0.37, P < 0.001). In addition, test scores were substantially higher among the 15.9% of our participants who reported having had motivation training (Mann-Whitney U = 3,581,047.5, P < 0.001, Myes = 66.9 [SD = 12.7], Mno = 61.6 [13.7]) and were positively correlated with the number of training hours (Spearman’s ρ = 0.10, P < 0.001).
3.2. Demographic Differences
Demographic differences are summarized in Table 2. Small but significant effects were found for the level of education completed with total scores generally increasing with educational level. Test scores were correlated with age, but the correlation was small and probably of little practical significance (Spearman’s ρ = 0.05, P < 0.001). An effect was also found for country (H = 161.2, P < 0.001, MUnitedStates/Canada = 62.0 [13.5], MIndia = 68.1 [11.9], MOther = 61.5 [14.2]), with participants from India substantially outscoring people from all other countries combined (U = 2,415,341.5, P < 0.001, MIndia = 68.1 [11.9], MAllOtherCountries = 61.8 [13.7]) – perhaps indicating that our sample was drawn mainly from urban areas in India, where people have tended since the late 1990s to adhere to a “work hard and get rich” ethos [150, 151].
An effect was found for gender, but the effect disappeared when only males and females were compared (U = 8,198,300.0, P = 0.15, MMale = 62.2 [14.3], MFemale = 62.8 [13.3]). A relatively large difference was also found between males and females combined and those who chose “other” as a designation for their gender (U = 148,329.0, P < 0.001, MMale/Female = 62.5 [13.9], MOther = 52.8 [16.0]).
| Education Level Completed | ||||
|---|---|---|---|---|
| Demographic | Mean Score | SD | H | P |
| None | 54.1 | 15.8 | 250.8 | < 0.001 |
| High School | 60.9 | 13.7 | ||
| Associates | 62.4 | 13.6 | ||
| College | 63.8 | 13.3 | ||
| Masters | 65.7 | 12.5 | ||
| Doctorate | 64.0 | 12.6 | ||
| Race | ||||
| Demographic | Mean Score | SD | H | P |
| American Indian | 65.1 | 18.5 | 161.6 | < 0.001 |
| Asian | 65.9 | 13.4 | ||
| Black | 63.7 | 14.7 | ||
| Hispanic | 63.1 | 12.3 | ||
| White | 61.0 | 13.9 | ||
| Other | 59.2 | 14.4 | ||
| Location | ||||
| Demographic | Mean Score | SD | H | P |
| India | 68.1 | 11.9 | 161.2 | < 0.001 |
| United States / Canada | 62.0 | 13.5 | ||
| Other | 61.5 | 14.2 | ||
| Gender | ||||
| Demographic | Mean Score | SD | H | P |
| Female | 62.8 | 13.3 | 22.2 | < 0.001 |
| Male | 62.2 | 14.3 | ||
| Other | 52.8 | 16.0 | ||
| Criterion | Variable | β | B (95% CI) | SE B | t | Sig. t | r2 |
|---|---|---|---|---|---|---|---|
|
Motivation Level |
Sets Goals | 0.311 | 0.177 (0.164 to 0.191) | 0.007 | 25.956 | < 0.001 | 0.379 |
| Manages Thoughts | 0.248 | 0.162 (0.148 to 0.176) | 0.007 | 22.452 | < 0.001 | ||
| Maintains a Healthy Lifestyle | 0.107 | 0.071 (0.058 to 0.085) | 0.007 | 10.668 | < 0.001 | ||
| Manages Environment | 0.110 | 0.080 (0.064 to 0.097) | 0.008 | 9.564 | < 0.001 | ||
| Manages Stress | -0.094 | -0.068 (-0.085 to -0.052) | 0.008 | -8.231 | < 0.001 | ||
| Monitors Behavior | 0.085 | 0.058 (0.043 to 0.073) | 0.008 | 7.581 | < 0.001 | ||
| Excluded variables: Makes Commitments, Manages Rewards (Model 6) | |||||||
| Life Satisfaction | Sets Goals | 0.235 | 0.143 (0.128 to 0.157) | 0.008 | 18.925 | < 0.001 | 0.283 |
| Manages Thoughts | 0.259 | 0.180 (0.164 to 0.196) | 0.008 | 21.871 | < 0.001 | ||
| Maintains a Healthy Lifestyle | 0.134 | 0.095 (0.080 to 0.110) | 0.008 | 12.500 | < 0.001 | ||
| Manages Environment | 0.109 | 0.084 (0.066 to 0.103) | 0.009 | 8.889 | < 0.001 | ||
| Manages Stress | -0.060 | -0.047 (-0.065 to -0.028) | 0.009 | -4.980 | < 0.001 | ||
| Makes Commitments | -0.023 | -0.018 (-0.033 to -0.002) | 0.008 | -2.229 | 0.026 | ||
| Excluded variables: Monitors Behavior, Manages Rewards (Model 6) | |||||||
| Professional Success | Sets Goals | 0.250 | 0.153 (0.137 to 0.169) | 0.008 | 18.902 | < 0.001 | 0.187 |
| Manages Thoughts | 0.165 | 0.116 (0.098 to 0.133) | 0.009 | 13.053 | < 0.001 | ||
| Manages Environment | 0.129 | 0.101 (0.081 to 0.121) | 0.010 | 10.040 | < 0.001 | ||
| Manages Stress | -0.093 | -0.072 (-0.092 to -0.053) | 0.010 | -7.170 | < 0.001 | ||
| Maintains a Healthy Lifestyle | 0.064 | 0.046 (0.030 to 0.062) | 0.008 | 5.579 | < 0.001 | ||
| Excluded variables: Makes Commitments, Monitors Behavior, Manages Rewards (Model 5) | |||||||
3.3. Competency Differences
The average total score on the test was 62.5% (13.7). Average scores on each of the eight competencies, from highest to lowest, were as follows: Manages Thoughts (M = 73.3 [20.7]), Manages Stress (M = 70.3 [18.6]), Manages Rewards (M = 66.3 [18.0]), Manages Environment (M = 64.9 [18.5]), Sets Goals (M = 64.0 [23.7]), Monitors Behavior (M = 57.5 [19.7]), Makes Commitments (M = 55.3 [18.5]), and Maintains a Healthy Lifestyle (M = 49.1 [20.3]).
3.4. Regressions and Factor Analysis
Linear regression was used to determine which of the eight competencies best predicted the three self-reported criterion measures. These analyses pointed strongly and consistently towards the importance of two of the eight competencies: Sets Goals and Manages Thoughts (Table 3). The ranking of the eight competencies according to their overall predictive value (based on the regression coefficients) was poorly correlated with the ranking of these competencies according to how highly participants scored on each one (ρ = 0.0, r = 0.0). Sets Goals, for example, which ranked highest in predictive value, ranked fifth in competency scores (see Competency Differences, above).
To examine patterns of responding on the test, we performed an exploratory principal components factor analysis that included all 32 scored items. The appropriateness of our data for factor analysis was confirmed by a high Kaiser-Meyer-Olkin measure of sampling adequacy (0.93) and a significant Bartlett’s test of sphericity (P < 0.001). Overall, the analysis yielded seven distinct, interpretable, and statistically sound components, all of which corresponded fairly closely to the competency areas that were used to formulate the test: (1) Improves Performance, (2) Monitors Behavior, (3) Manages Goals and Rewards, (4) Maintains Healthy Lifestyle, (5) Manages Thoughts, (6) Manages Commitments, and (7) Manages Stress (Table 4). The percentage of variance accounted for by each of these components varied from 5.4% to 9.7%. The cumulative total percentage accounted for was 49.2%.
A statistician might argue that we should eliminate test items with low factor loadings, or that we should even revise our competencies model to be consistent with the principal components that emerged in the analysis. We agree that this can be done, but we have reservations about proceeding that way in the present instance. Many psychological tests attempt to measure hypothetical constructs such as self-esteem or idealism. Factor or item reduction is often helpful in the development of such tests because these techniques can yield more stable measures of the construct. The EMCI-i, however, measures trainable skill sets that were derived from published studies. The consistency of items within each competency area is greater than the consistency of items within each factor-analytic component. This is not surprising, given that factor-analytical computations are not based on the actual content of the test items. If only for training purposes, therefore, we believe that our original item clusters have more value than the principal components. That said, the results of our factor analysis could in fact be used to streamline the EMCI-i – eliminating less informative items (such as items 9, 11, 16, 30, and 38) (Table 4) and using principal components to provide guidelines for redefining our original components [152].
| Rotated Component Matrix | |||||||
|---|---|---|---|---|---|---|---|
| Component | |||||||
| Item | Improves Performance | Monitors Behavior | Manages Goals and Rewards | Manages Thoughts | Manages Commitments | Maintains Healthy Lifestyle | Manages Stress |
| i32 | .698 | ||||||
| i35 | .682 | ||||||
| i28 | .678 | ||||||
| i40 | .668 | ||||||
| i17 | .638 | ||||||
| i16 | |||||||
| i18 | .667 | ||||||
| i19 | .620 | ||||||
| i5 | .560 | ||||||
| i22 | .521 | ||||||
| i27 | .510 | ||||||
| i11 | |||||||
| i36 | .723 | ||||||
| i2 | .650 | ||||||
| i25 | .604 | ||||||
| i29 | .504 | ||||||
| i9 | |||||||
| i24 | .724 | ||||||
| i6 | .681 | ||||||
| i10 | .493 | ||||||
| i30 | |||||||
| i13 | .648 | ||||||
| i14 | .568 | ||||||
| i1 | .545 | ||||||
| i20 | .478 | ||||||
| i31 | .591 | ||||||
| i34 | .523 | ||||||
| i39 | .514 | ||||||
| i38 | |||||||
| i8 | .626 | ||||||
| i7 | .567 | ||||||
| i21 | .526 | ||||||
| Year | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 |
|---|---|---|---|---|---|---|---|---|---|
| n | 1143 | 733 | 870 | 698 | 532 | 318 | 1257 | 337 | 455 |
| Mean Score (SD) | 61.3 (13.4) |
62.7 (12.3) | 61.6 (12.5) | 61.8 (13.3) | 63.7 (13.1) | 64.6 (12.7) | 66.7 (12.4) | 57.9 (16.6) | 58.3 (15.9) |
| Mean Age (SD) | 35.2 (12.6) |
29.2 (13.3) | 28.6 (13.3) | 25.9 (11.8) | 25.5 (11.5) | 23.6 (10.7) | 34.1 (13.4) | 25.8 (11.4) | 25.1 (11.4) |
| Gender (% Fem.) | 60.9 | 61.7 | 56.8 | 54.0 | 57.3 | 58.5 | 61.9 | 63.2 | 58.9 |
| Race (% White) | 59.7 | 51.4 | 52.0 | 40.7 | 39.5 | 30.8 | 60.1 | 56.7 | 57.8 |
| Year | 2017 | 2018 | 2019 | Significance Test† |
|---|---|---|---|---|
| n | 191 | 187 | 238 | 6,959 |
| Mean Score (SD) | 60.7(15.1) | 59.4 (14.6) | 56.3 (15.7) | 256.7*** (H) |
| Mean Age (SD) | 25.5(10.9) | 26.0 (11.7) | 27.3 (13.3) | 65.6*** (F) |
| Gender (% Fem.) | 59.7 | 47.1 | 55.0 | 32.8** (Χ2) |
| Race (% White) | 44.5 | 54.0 | 55.5 | 201.6*** (Χ2) |
3.5. Changes Over Time
Because the data were collected over a period of 13 years, we also looked for changes over time. Mean total scores per year did change significantly over time, and so did important demographic characteristics of the sample (Table 5).
We found no evidence, however, of significant trends in our data. When we divided our data into two segments – those collected before December 6, 2013 – the midpoint of the time range over which our data were collected – and those collected on or after that date, we found no significant difference between the mean total scores for those two periods (U = 7,708,321.5, P = 0.54, MPreMidpoint = 62.7 [13.1], MPostMidpoint = 62.1 [14.9]).
4. DISCUSSION
We believe that the EMCI-i is the first test that looks relatively comprehensively at trainable competencies that are associated with high levels of motivation. Moreover, by employing a concurrent study design, we were able to produce converging evidence to suggest that the EMCI-i is a valid measuring instrument [79].
Although we relied on a convenience sample (see Limitations, below), it is unlikely that this sample could have systematically affected our main findings, which are as follows: First, goal setting appears to be the most valuable of the eight competency areas we evaluated. This is good news given that goal setting is a particularly simple concept that is relatively easy to train [153, 154]. Second, there appears to be no relationship between the relative value of our eight motivation competencies (apparent from our regression analyses) and the motivation skills people actually have, which suggests a strong need for training motivation competencies. Third, although our study was correlational, not experimental, our results are consistent with the view that motivation training has value – the more training, the better.
5. LIMITATIONS
The greatest limitation in this study might also be considered a strength – namely, that we had no control over the composition of the sample. Our convenience sample was not necessarily representative of the populations of the countries with the most participants in the study (U.S., Canada, and India), nor, for that matter, of any other country. People drawn to a test of motivation skills might be different in nontrivial ways from people who are not drawn to such a test; at the very least, they are probably more interested in boosting their motivational level. Because our data were collected over a period of 13 years, the sample also varied over time in ways we could not control (Table 4); this occurred, presumably, because the list of websites linking to the test kept changing and also because the ranking of the test in search engines was likely shifting. Our sample was non-representative in one obvious respect: 51.8% of our participants had at least a 4-year college degree. In the U.S., only 33.3% of the adult population has earned such a degree [155].
These limitations notwithstanding, a long-term internet study yields a large and diverse sample – in this case, a cleaned sample of 8,349 people from 123 countries. We suggest that such a sample is more valid than the one that has long dominated the social sciences – namely, a few hundred first- or second-year students from a single university [156-158, cf. 159]. We also suggest that the self-selection that undoubtedly occurred in our sample might have been advantageous: Our ideal sample might indeed consist of people who are interested in ways to increase their motivation.
Because we collected data online without identifying our participants (which we did in order to be granted exempt status by our IRB under HHS and APA ethical rules), we were also limited in how we could assess evidence of reliability and validity. We could not measure test-retest reliability, for example. In theory, we could have administered multiple versions of our test to assess alternate-form reliability, but we did not do so. We also had no way to compare EMCI-i scores to those our participants might have achieved on previously validated tests.
Also of concern: Our study relied entirely on self-reported data, and because responses were collected online from unknown individuals, we had no way to confirm the accuracy of those responses or of the demographic information that was provided. That said, a growing body of evidence suggests that people are especially honest when they take online tests anonymously, especially when they are being asked about socially sensitive matters [160-163]. Self-reported data have also been shown to be good predictors of a variety of different behaviors [164, 165]. Other research has also demonstrated the predictive value of self-reported data, especially when test items pinpoint specific behaviors [46, 47, 166, 167]. All the items on the EMCI-i pinpoint behavior in varying degrees. Thus, instead of saying something vague such as, “I’m very reliable,” the EMCI-i includes items such as “I always keep my promises” and “I always record my progress when working on a tough project.”
5.1. Managerial Implications
Because the EMCI-i could conceivably be used in business, we offer a brief analysis of the test’s possible “adverse impact” under the current standards of the U.S. Equal Employment Opportunity Commission (EEOC), which provides statistical standards that must be met by test instruments that could be used to hire, fire, or promote employees [168-170]. The relevant standard that must be met by such tests is called the “four-fifths rule,” according to which “a disparity is actionable when one group’s pass rate is less than four-fifths (80%) of another group’s pass rate” [171]. Our results do not violate this rule for any of the groups we measured by race, ethnicity, or gender.
5.2. Future Research
Our list of competencies was based on the empirical literature that was available when the test was developed (Table 1); it is not exhaustive. It has long been known, for example, that workplace performance (a concept related to motivation) is tied to factors such as autonomy, task variety, quality of personal relationships, opportunities for advancement and other factors [172-175]. Energy level – another concept related to motivation – is affected by diet, disease, and other factors [176-181]. Future revisions of the EMCI-i could be expanded to include additional competencies derived from literatures that were not reviewed when the original test was developed. Both items and competencies could also be improved using a principal components analysis and other statistical techniques.
Because the present study relied on self-report of criterion measures, future research could also look at the extent to which EMCI-i scores are associated with a variety of different outcome measures that should, in theory, be associated with motivation: evaluations by peers or supervisors, for example, or performance measures in the workplace. The value of the EMCI-i could also be assessed in a training context. The first author of this study has developed and evaluated specific training methods that can boost the strength of the eight competencies evaluated herein [153].
CONCLUSION
Our study presents credible evidence showing the value of using a competencies approach to understanding and boosting motivation, with Goal Setting emerging as the most valuable competency among the eight we compared. As David McClelland and his associates showed decades ago, there are distinct advantages to breaking down complex performances implied by vague concepts such as “motivation” into measurable, trainable competencies [44, 45].
NOTES
1 For examples of such training programs, see: the International Board of Standards of Training, Performance, and Instruction (http://ibstpi. org/training-manager-competencies/), the American Management Association (https://www.amanet.org/training/seminars/management-and-supervisory-skills-training.aspx), or the Oxford Management Centre (http://oxford- management.com/training-courses/certificate-series/). Also see the test at https://www.mindtools.com/pages/article/newTMM_28.htm.
2 Nonparametric statistical tests such as Spearman’s rho, the Mann-Whitney U, and the Kruskal-Wallis H are used throughout this study because scores on the EMCI-i lie on an ordinal scale [181]. Unless otherwise indicated, all test scores are reported as a percentage of total correct rather than as raw scores.
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
Exempt approval was granted by the Institutional Review Board (IRB) of the American Institute for Behavioral Research and Technology (AIBRT). AIBRT is registered with the HHS Office for Human Research Protections (OHRP) under IORG0007755. Our IRB is registered with OHRP under number IRB00009303, and the Federalwide Assurance number for our IRB is FWA00021545.
HUMAN AND ANIMAL RIGHTS
No animals were used in this research. All human research procedures were followed in accordance with the ethical standards of the committee responsible for human experimentation (institutional and national), and with the Helsinki Declaration of 1975, as revised in 2013.
CONSENT FOR PUBLICATION
Informed consent from the participants was obtained.
AVAILABILITY OF DATA AND MATERIALS
Anonymized data are available upon requests sent to info@aibrt.org. The data have not been publicly posted because, under the terms of the exempt approval granted by our IRB, we need to take special precautions to protect the identity of our study participants.
FUNDING
None.
CONFLICT OF INTEREST
The authors declare no conflict of interest, financial or otherwise.
ACKNOWLEDGEMENTS
This paper is based on part on a paper presented at the April 2016 meeting of the Western Psychological Association in Long Beach, California. We thank Philip Cheung, Ronald Robertson, Chelsea Song, and Kim Vu for help during the early stages of this project.
REFERENCES
| [1] | Hull CL. Principles of Behavior 1943. |
| [2] | Hull CL. A behavior system: An introduction to behavior theory concerning the individual organism 1952. |
| [3] | Lewin K. A dynamic theory of personality: Selected papers 1935. |
| [4] | Zeigarnik B. Studies concerning action and affect psychology, III. The retaining of completed and uncompleted actions. Psychol Forsch 1927; 9: 1-85. |
| [5] | Gorman P. Motivation and emotion 2014. |
| [6] | Kleinginna PR Jr, Kleinginna AM. A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv Emot 1981; 5(4): 345-79. |
| [7] | Elliot AJ. Competition and achievement outcomes: A hierarchical motivational analysis. Motiv Sci 2020; 6(1): 3-1. |
| [8] | Motive.The compact edition of the oxford english dictionary 1971; 1859. |
| [9] | Freud S. Instincts and their vicissitudes.Collected Papers of Sigmund Freud 1915; 60-83. |
| [10] | Geller ES, Casali JG, Johnson RP. Seat belt usage: A potential target for applied behavior analysis. J Appl Behav Anal 1980; 13(4): 669-75. |
| [11] | Klein G. Nebraska. Lincoln, NE: University of Nebraska Press 1954; pp. 1954; Nebraska Symposium on Motivation 224-74. |
| [12] | Keller FS, Schoenfeld WN. Principles of psychology: a systematic text in the science of behavior 1950. |
| [13] | Areepattamannil S. Relationship between academic motivation and mathematics achievement among Indian adolescents in Canada and India. J Gen Psychol 2014; 141(3): 247-62. |
| [14] | Kasser T, Ryan RM. Further examining the american dream: Differential correlates of intrinsic and extrinsic goals. Pers Soc Psychol Bull 1996; 22(3): 280-7. |
| [15] | Kasser T, Ryan RM. Be careful what you wish for: Optimal functioning and the relative attainment of intrinsic and extrinsic goals.Life goals and well-being: towards a positive psychology of human striving 2001. |
| [16] | Murayama K, Kitagami S, Tanaka A, Raw JA. People’s naiveté about how extrinsic rewards influence intrinsic motivation. Motiv Sci 2016; 2(3): 138-42. |
| [17] | Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000; 55(1): 68-78. |
| [18] | Wan CS, Chiou WB. The motivations of adolescents who are addicted to online games: A cognitive perspective. Adolescence 2007; 42(165): 179-97. |
| [19] | Warneken F, Tomasello M. Extrinsic rewards undermine altruistic tendencies in 20-month-olds. Motiv Sci 2014; 1(S): 43-8. |
| [20] | Dunning D. Motivational theories.Theory and explanation in social psychology 2015; 108-31. |
| [21] | Pardee RL. Motivation theories of Maslow, Herzberg, McGregor & McClelland. A literature review of selected theories dealing with job satisfaction and motivation. ERIC Document Reproduction Service 1990; ED316767. |
| [22] | Weiner B. An Attributional Theory of Motivation and Emotion 1986; 159-90. |
| [23] | Baldwin JD, Baldwin JI. Behavior principles in everyday life 2nd ed. 1986. |
| [24] | Bashshar CE. Virtual learning environments' impact on adult learners' motivation in the workplace [dissertation]. [Minneapolis, MN]: Walden Dissertations and Doctoral Studies 2017; 1-159. |
| [25] | Engelmann JB, Pessoa L. Motivation sharpens exogenous spatial attention. Motiv Sci 2014; 1(S): 64-72. |
| [26] | Kuzucu Y, Şimşek ÖF. Self-determined choices and consequences: The relationship between basic psychological needs satisfactions and aggression in late adolescents. J Gen Psychol 2013; 140(2): 110-29. |
| [27] | Zeigler-Hill V, Hobbs KA. The darker aspects of motivation: Pathological personality traits and the fundamental social motives. J Soc Clin Psychol 2017; 36(2): 87-107. |
| [28] | Gkorezis P, Kastritsi A. Employee expectations and intrinsic motivation: Work-related boredom as a mediator. Employee Relat 2017; 39(1): 100-11. |
| [29] | Jungert T, Van den Broeck A, Schreurs B, Osterman U. How colleagues can support each other’s needs and motivation: An intervention on employee work motivation. Appl Psychol 2017; 67(1): 3-29. |
| [30] | Locke EA, Latham GP. Goal setting: A motivational technique that works! 1984. |
| [31] | Locke EA, Latham GP. A theory of goal setting and task performance 1990. |
| [32] | Brown TC, Fry MD, Moore EW. A motivational climate intervention and exercise-related outcomes: A longitudinal perspective. Motiv Sci 2017; 3(4): 337-53. |
| [33] | Ghorbani S. Motivational effects of enhancing expectancies and autonomy for motor learning: An examination of the OPTIMAL theory. J Gen Psychol 2019; 146(1): 79-92. |
| [34] | Miquelon P, Castonguay A. Integrated regulation, behavior consistency, and physical activity maintenance. Motiv Sci 2017; 3(1): 76-90. |
| [35] | Roberts GC. Motivation in sport and exercise 1992. |
| [36] | Standage M, Ryan RM. Self-determination theory in sport and exercise. Handbook of sport psychology 4th ed. 2020; 352-78. |
| [37] | Straub WF, Williams JM, Eds. Cognitive sport psychology 1984. |
| [38] | Canning EA, Harackiewicz JM. Teach it, don’t preach it: The differential effects of directly-communicated and self-generated utility–value information. Motiv Sci 2015; 1(1): 47-71. |
| [39] | Job V, Friese M, Bernecker K. Effects of practicing self-control on academic performance. Motiv Sci 2015; 1(4): 219-32. |
| [40] | Lin-Siegler X, Ahn JN, Chen J, Fang F-FA, Luna-Lucero M. Even Einstein struggled: Effects of learning about great scientists’ struggles on high school students’ motivation to learn science. J Educ Psychol 2016; 108(3): 314-28. |
| [41] | Weiner B. A theory of motivation for some classroom experiences. J Educ Psychol 1979; 71(1): 3-25. |
| [42] | McClelland DC. The achieving society 1961. |
| [43] | McClelland DC. Studies in motivation 1987. |
| [44] | Chouhan VS, Srivastava S. Understanding Competencies and Competency Modeling — A Literature Survey. IOSR J Bus Manag 2014; 16(1): 14-22. |
| [45] | Chen DW, McClelland DC. Pioneer In Work on Human Motivation [Internet]. The New York Times. The New York Times 1998. Available from: https://www.nytimes.com/1998/04/05/us/d-c-mcclelland-80-pioneer-in-work-on-human-motivation.html |
| [46] | Boyatzis RE. The competent manager: A model for effective performance 1982. |
| [47] | Spencer LM, Spencer SM. Competence at Work: Models for Superior Performance 1993. |
| [48] | Hoffarth MJ. From achievement to power: David C. McClelland, McBer & Company, and the business of the Thematic Apperception Test (TAT), 1962-1985. J Hist Behav Sci 2020; 56(3): 153-68. |
| [49] | Andersen JA. Managers’ motivation profiles: Measurement and application. SAGE Open 2018; 8(2) |
| [50] | Markus LH, Cooper-Thomas HD, Allpress KN. Confounded by competencies? An evaluation of the evolution and use of competency models. N Z J Psychol 2005; 34(2): 117-26. |
| [51] | Zhao Q, Wichman A, Frishberg E. Self-doubt effects depend on beliefs about ability: Experimental evidence. J Gen Psychol 2019; 146(3): 299-324. |
| [52] | Bernard LC, Walsh RP, Mills M. The Motivation Analysis Test: An historical and contemporary evaluation. Psychol Rep 2005; 96(2): 464-92. |
| [53] | Cattell RB, Sweney JL, Radcliffe JA. Handbook for the Motivation Analysis Test 1964. |
| [54] | Sashkin M. The MbM Questionnaire: Managing by motivation 3rd ed. 1996. |
| [55] | Schuler H, Thornton G, Frintrup A, Mueller-Hanson R. AMI: Achievement Motivation Inventory: Technical and user’s manual 2004. |
| [56] | Tutko TA, Lyon LP, Ogilvie BC. Athletic motivation inventory manual 1975. |
| [57] | Miller HJ. The Miller Motivation Scale: A New Counselling and Research Tool. ERIC 1987. Available from: https://eric.ed.gov/?q= The%2BMiller%2BMotivation%2BScale%3A%2BA%2BNew%2BCounselling%2Band%2BResearch%2BTool.&id=ED280879 |
| [58] | Murray LE, Bruce MM. A study of the validity of the Sales Comprehension Test and Sales Motivation Inventory in differentiating high and low production in life insurance selling. J Appl Psychol 1959; 43(4): 246-8. |
| [59] | Hall J, Williams MS. Work motivation inventory 1967. |
| [60] | Esmond J, Dunlop P. Developing the volunteer motivation inventory to assess the underlying motivational drives of volunteers in Western Australia. CLAN WA Inc 2004. Available from: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.734.3555&rep=rep1&type=pdf |
| [61] | Barbuto JE Jr, Scholl RW. Motivation sources inventory: Development and validation of new scales to measure an integrative taxonomy of motivation. Psychol Rep 1998; 82(3): 1011-22. |
| [62] | Carlson JF, Geisinger KF, Jonson JL, Eds. The twentieth mental measurements yearbook 2017. |
| [63] | The Internet and Health. Pew Research Center: Internet, Science & Tech Pew Research Center 2020. Available from: https://www.pewresearch.org/internet/2013/02/12/the-internet-and-health/ |
| [64] | Ololube NP. Benchmarking the motivational competencies of academically qualified teachers and professionally qualified teachers in Nigerian secondary schools. Afr Symp 2005; 17-37. |
| [65] | Klein JD, Spector JM, Grabowski BL, de la Teja I. Instructor Competencies: Standards for face-to-face, online & blended settings 3rd ed. 2004. |
| [66] | Blašková M, Blaško R, Kucharčíková A. Competences and competence model of university teachers. Procedia Soc Behav Sci 2014; 159: 457-67. |
| [67] | Gómez-Arizaga MP, Conejeros-Solar ML, Martin A. How good is good enough? A community-based assessment of teacher competencies for gifted students. SAGE Open 2016; 6(4): 1-14. |
| [68] | Warn J, Tranter P. Measuring quality in higher education: A competency approach. Qual High Educ 2001; 7(3): 191-8. |
| [69] | Albino G. Technical and behavioral competencies on performance evaluation: Petrek leaders’ perspectives. SAGE Open 2018; 8(2) |
| [70] | Shippmann JS, Ash RA, Battista M, et al. The practice of competency modeling. Person Psychol 2000; 53(3): 703-40. |
| [71] | Hindiarto F, Hidayat R, Riyono B. Is training effective to develop individual’s soft skills in organizations? emphatic communication training on friendly and helpful behavior in private hospital. Open Psychol J 2020; 13(1): 5-13. |
| [72] | Zook AM. Military competency-based human capital management: A step toward the future 2006. |
| [73] | Komarudin K, Mulyana B, Novian G. The effect of life kinetik training models to improve self-confidence in team and individuals athletes. Open Psychol J 2021; 14(1): 220-6. |
| [74] | Epstein R, Phan V. Which competencies are most important for creative expression? Creat Res J 2012; 24(4): 278-82. |
| [75] | Epstein R, Schmidt SM, Warfel R. Measuring and training creativity competencies: Validation of a new test. Creat Res J 2008; 20(1): 7-12. |
| [76] | Epstein R. The big book of stress relief games quick, fun activities for feeling better 2000. |
| [77] | Momtaz YA, Ansari M, Foroughan M. Humor training program on sense of humor among older adults: A randomized controlled trial. Open Psychol J 2020; 13(1): 333-40. |
| [78] | Epstein R, Robertson RE, Smith R, Vasconcellos T, Lao M. Which relationship skills count most? A large-scale replication. J Couple Relationship Ther 2016; 15(4): 341-56. |
| [79] | Standards for educational and psychological testing 2014. |
| [80] | Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao SM. Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. Am Psychol 2018; 73(1): 3-25. |
| [81] | Cassidy T. Social background, achievement motivation, optimism and health: A longitudinal study. Couns Psychol Q 2000; 13(4): 399-412. |
| [82] | Festini SB, Hertzog C, McDonough IM, Park DC. What makes us busy? Predictors of perceived busyness across the adult lifespan. J Gen Psychol 2019; 146(2): 111-33. |
| [83] | Halbesleben JR, Bowler WM. Emotional exhaustion and job performance: The mediating role of motivation. J Appl Psychol 2007; 92(1): 93-106. |
| [84] | Milyavskaya M, Koestner R. Psychological needs, motivation, and well-being: A test of self-determination theory across multiple domains. Pers Individ Dif 2011; 50(3): 387-91. |
| [85] | Murray G, Nicholas CL, Kleiman J, et al. Nature’s clocks and human mood: The circadian system modulates reward motivation. Emotion 2009; 9(5): 705-16. |
| [86] | Zhang T. Relations among school students’ self-determined motivation, perceived enjoyment, effort, and physical activity behaviors. Percept Mot Skills 2009; 109(3): 783-90. |
| [87] | Dawes NP, Larson R. How youth get engaged: Grounded-theory research on motivational development in organized youth programs. Dev Psychol 2011; 47(1): 259-69. |
| [88] | Diefendorff JM, Chandler MM. Motivating employees. In: Zedeck S, Ed. APA Handbook of Industrial and Organizational Psychology 2011; 65-135. |
| [89] | Grant AM. Does intrinsic motivation fuel the prosocial fire? Motivational synergy in predicting persistence, performance, and productivity. J Appl Psychol 2008; 93(1): 48-58. |
| [90] | Klein HJ, Wesson MJ, Hollenbeck JR, Alge BJ. Goal commitment and the goal-setting process: Conceptual clarification and empirical synthesis. J Appl Psychol 1999; 84(6): 885-96. |
| [91] | Myer JP, Becker TE, Vandenberghe C. Employee commitment and motivation: A conceptual analysis and integrative model. J Appl Psychol 2004; 89(6): 991-1007. |
| [92] | Naquin SS, Holton EF. The effects of personality, affectivity, and work commitment on motivation to improve work through learning. Hum Resour Dev Q 2002; 13(4): 357-76. |
| [93] | Wallace JC, Johnson PD, Mathe K, Paul J. Structural and psychological empowerment climates, performance, and the moderating role of shared felt accountability: A managerial perspective. J Appl Psychol 2011; 96(4): 840-50. |
| [94] | Yun S, Takeuchi R, Liu W. Employee self-enhancement motives and job performance behaviors: Investigating the moderating effects of employee role ambiguity and managerial perceptions of employee commitment. J Appl Psychol 2007; 92(3): 745-56. |
| [95] | Judd JS. The role of learning environment on high school chemistry students’ motivation and self-regulatory processes. Diss Abstr Int 2010; 70: 2385. |
| [96] | Molloy LE, Gest SD, Rulison KL. Peer influences on academic motivation: Exploring multiple methods of assessing youths’ most “influential” peer relationships. J Early Adolesc 2010; 31(1): 13-40. |
| [97] | Moreno JA, González-Cutre D, Sicilia Á, Spray CM. Motivation in the exercise setting: Integrating constructs from the approach–avoidance achievement goal framework and self-determination theory. Psychol Sport Exerc 2010; 11(6): 542-50. |
| [98] | Nanda R, Sørensen JB. Workplace Peers and Entrepreneurship. Manage Sci 2010; 56(7): 1116-26. |
| [99] | Parsons J, Midgley C, Adler T. Grade-related changes in the school environment: Effects on achievement motivation. Adv Motiv Achiev 1984; 4: 283-331. |
| [100] | Syed F, Naseer S, Bouckenooghe D. Unfairness in stressful job environments: The contingent effects of perceived organizational injustice on the relationships between job stress and employee behaviors. J Gen Psychol 2021; 148(2): 168-91. |
| [101] | Ambrose ML, Kulik CT. Old Friends, New Faces: Motivation Research in the 1990s. J Manage 1999; 25(3): 231-92. |
| [102] | Deci EL, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull 1999; 125(6): 627-68. |
| [103] | Eisenberger R, Rhoades L. Incremental effects of reward on creativity. J Pers Soc Psychol 2001; 81(4): 728-41. |
| [104] | Frey BS, Neckermann S. Awards: A view from psychological economics. J Psychol 2008; 216: 198-208. |
| [105] | Gottfried AE, Marcoulides GA, Gottfried AW, Oliver PH. A latent curve model of parental motivational practices and developmental decline in math and science academic intrinsic motivation. J Educ Psychol 2009; 101(3): 729-39. |
| [106] | Pearsall MJ, Christian MS, Ellis AP. Motivating interdependent teams: Individual rewards, shared rewards, or something in between? J Appl Psychol 2010; 95(1): 183-91. |
| [107] | Keogh E, Bond FW, Flaxman PE. Improving academic performance and mental health through a stress management intervention: Outcomes and mediators of change. Behav Res Ther 2006; 44(3): 339-57. |
| [108] | LePine JA, LePine MA, Jackson CL. Challenge and hindrance stress: Relationships with exhaustion, motivation to learn, and learning performance. J Appl Psychol 2004; 89(5): 883-91. |
| [109] | Ning HK, Downing K. The reciprocal relationship between motivation and self-regulation: A longitudinal study on academic performance. Learn Individ Differ 2010; 20(6): 682-6. |
| [110] | Thompson A, Gaudreau P. From optimism and pessimism to coping: The mediating role of academic motivation. Int J Stress Manag 2008; 15(3): 269-88. |
| [111] | Bostan CM, Maricuțoiu LP, Constantin T. Successful management of automatic dysfunctional thoughts in the context of negative emotions. Anxiety Stress Coping 2018; 31(6): 686-701. |
| [112] | Bandura A, Locke EA. Negative self-efficacy and goal effects revisited. J Appl Psychol 2003; 88(1): 87-99. |
| [113] | Bartels RD. Strategic evaluations of goal pursuit: The power of positive and negative thinking. Diss Abstr Int 2010; 71: 705. |
| [114] | Baum JR, Locke EA. The relationship of entrepreneurial traits, skill, and motivation to subsequent venture growth. J Appl Psychol 2004; 89(4): 587-98. |
| [115] | Judge TA, Bono JE, Erez A, Locke EA. Core self-evaluations and job and life satisfaction: The role of self-concordance and goal attainment. J Appl Psychol 2005; 90(2): 257-68. |
| [116] | Koizumi R. The relationship between perceived attainment and optimism, and academic achievement and motivation. Jpn Psychol Res 1992; 34(1): 1-9. |
| [117] | Oyserman D, Destin M. Identity-based motivation: Implications for intervention. Couns Psychol 2010; 38(7): 1001-43. |
| [118] | Strauss K, Griffin MA, Parker SK. Future work selves: How salient hoped-for identities motivate proactive career behaviors. J Appl Psychol 2012; 97(3): 580-98. |
| [119] | Zhang Y. The optimism in goal pursuit. Diss Abstr Int 2007; 68: 3384. |
| [120] | Ellis S, Davidi I. After-event reviews: Drawing lessons from successful and failed experience. J Appl Psychol 2005; 90(5): 857-71. |
| [121] | Judge TA, Hurst C. Capitalizing on one’s advantages: Role of core self-evaluations. J Appl Psychol 2007; 92(5): 1212-27. |
| [122] | Kann R. Increasing motivation and reading comprehension of exceptional learners: Three modeling techniques. Pointer 1984; 29(1): 20-2. |
| [123] | Koo M, Fishbach A. Climbing the goal ladder: How upcoming actions increase level of aspiration. J Pers Soc Psychol 2010; 99(1): 1-13. |
| [124] | Lee FK, Sheldon KM, Turban DB. Personality and the goal-striving process: The influence of achievement goal patterns, goal level, and mental focus on performance and enjoyment. J Appl Psychol 2003; 88(2): 256-65. |
| [125] | Porath CL, Bateman TS. Self-regulation: From goal orientation to job performance. J Appl Psychol 2006; 91(1): 185-92. |
| [126] | Tolli AP, Schmidt AM. The role of feedback, casual attributions, and self-efficacy in goal revision. J Appl Psychol 2008; 93(3): 692-701. |
| [127] | Vancouver JB, Kendall LN. When self-efficacy negatively relates to motivation and performance in a learning context. J Appl Psychol 2006; 91(5): 1146-53. |
| [128] | Audia G, Kristof-Brown A, Brown KG, Locke EA. Relationship of goals and microlevel work processes to performance on a multipath manual task. J Appl Psychol 1996; 81(5): 483-97. |
| [129] | Bandura A, Schunk DH. Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. J Pers Soc Psychol 1981; 41(3): 586-98. |
| [130] | Brown SP, Jones E, Leigh TW. The attenuating effect of role overload on relationships linking self-efficacy and goal level to work performance. J Appl Psychol 2005; 90(5): 972-9. |
| [131] | Bryan JF, Locke EA. Goal setting as a means of increasing motivation. J Appl Psychol 1967; 51(3): 274-7. |
| [132] | DeShon RP, Gillespie JZ. A motivated action theory account of goal orientation. J Appl Psychol 2005; 90(6): 1096-127. |
| [133] | Dierdorff EC, Surface EA, Brown KG. Frame-of-reference training effectiveness: Effects of goal orientation and self-efficacy on affective, cognitive, skill-based, and transfer outcomes. J Appl Psychol 2010; 95(6): 1181-91. |
| [134] | Gong Y, Fan J. Longitudinal examination of the role of goal orientation in cross-cultural adjustment. J Appl Psychol 2006; 91(1): 176-84. |
| [135] | Ilies R, Judge TA. Goal regulation across time: The effects of feedback and affect. J Appl Psychol 2005; 90(3): 453-67. |
| [136] | Kleingeld A, van Mierlo H, Arends L. The effect of goal setting on group performance: A meta-analysis. J Appl Psychol 2011; 96(6): 1289-304. |
| [137] | Kozlowski SW, Bell BS. Disentangling achievement orientation and goal setting: Effects on self-regulatory processes. J Appl Psychol 2006; 91(4): 900-16. |
| [138] | LePine JA. Adaptation of teams in response to unforeseen change: Effects of goal difficulty and team composition in terms of cognitive ability and goal orientation. J Appl Psychol 2005; 90(6): 1153-67. |
| [139] | Locke EA. Motivation through conscious goal setting. Appl Prev Psychol 1996; 5(2): 117-24. |
| [140] | Morisano D, Hirsh JB, Peterson JB, Pihl RO, Shore BM. Setting, elaborating, and reflecting on personal goals improves academic performance. J Appl Psychol 2010; 95(2): 255-64. |
| [141] | Mossholder KW. Effects of externally mediated goal setting on intrinsic motivation: A laboratory experiment. J Appl Psychol 1980; 65(2): 202-10. |
| [142] | Payne SC, Youngcourt SS, Beaubien JM. A meta-analytic examination of the goal orientation nomological net. J Appl Psychol 2007; 92(1): 128-50. |
| [143] | Porter CO. Goal orientation: Effects on backing up behavior, performance, efficacy, and commitment in teams. J Appl Psychol 2005; 90(4): 811-8. |
| [144] | Rheinberg F, Engeser S. Motivational competence: The joint effect of implicit and explicit motives on self-regulation and flow experience.Motivation, consciousness and self-regulation 2012; 79-89. |
| [145] | Terborg JR. The motivational components of goal setting. J Appl Psychol 1976; 61(5): 613-21. |
| [146] | Terborg JR, Miller HE. Motivation, behavior, and performance: A closer examination of goal setting and monetary incentives. J Appl Psychol 1978; 63(1): 29-39. |
| [147] | van Hooft EA, Noordzij G. The effects of goal orientation on job search and reemployment: A field experiment among unemployed job seekers. J Appl Psychol 2009; 94(6): 1581-90. |
| [148] | Vancouver JB, Weinhardt JM, Schmidt AM. A formal, computational theory of multiple-goal pursuit: Integrating goal-choice and goal-striving processes. J Appl Psychol 2010; 95(6): 985-1008. |
| [149] | Wang M, Takeuchi R. The role of goal orientation during expatriation: A cross-sectional and longitudinal investigation. J Appl Psychol 2007; 92(5): 1437-45. |
| [150] | Gopal A, Srinivasan R. The new indian consumer. Harvard Business Review 2006. Available from: https://hbr.org/2006/10/the-new-indian-consumer |
| [151] | Kumar A. . Indian women are among the most highly motivated in the world. OneWorld South Asia 2013. Available from: http://southasia.oneworld.net/peoplespeak/indian-women-are-among-the-most-highly-motivated-in-the-world |
| [152] | Cahyani AD, Iskandarsyah A, Cahyadi S, Srisayekti W. Using confirmatory factor analysis to evaluate construct validity of the Indonesian palatable eating motives scale (I-PEMS). Open Psychol J 2020; 13(1): 1-4. |
| [153] | Epstein R, Rogers J. The big book of motivation games 2001. |
| [154] | Worden JM. An analysis of training focused on improving SMART goal setting for specific employee groups. Diss Abstr Int 2015; 75: 1-128. |
| [155] | Educational Attainment in the United States. The United States Census Bureau. The United States Census Bureau 2019. Available from: https://www.census.gov/data/tables/2019/demo/educational-attainment/cps-detailed-tables.html |
| [156] | Anderson J, Burks SV, Carpenter J, et al. Self-selection and variations in the laboratory measurement of other-regarding preferences across subject pools: Evidence from one college student and two adult samples. Exp Econ 2012; 16(2): 170-89. |
| [157] | Goodwin KA, Goodwin CJ. Research in psychology: Methods and design 8th ed. 2018. |
| [158] | Henrich J, Heine SJ, Norenzayan A. The weirdest people in the world? Behav Brain Sci 2010; 33(2-3): 61-83. |
| [159] | Kühberger A, Fritz A, Scherndl T. Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS ONE 2014; 9(9): e105825. |
| [160] | Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: The tailored design method 2014. |
| [161] | Kaplan RM, Saccuzzo DP. Psychological testing: Principles, applications, and issues 2009. |
| [162] | Robertson RE, Tran FW, Lewark LN, Epstein R. Estimates of non-heterosexual prevalence: The roles of anonymity and privacy in survey methodology. Arch Sex Behav 2018; 47(4): 1069-84. |
| [163] | Trau RN, Härtel CE, Härtel GF. Reaching and hearing the invisible: Organizational research on invisible stigmatized groups via web surveys. Br J Manage 2012; 24(4): 532-41. |
| [164] | Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: A meta-analysis. Behav Res Methods 2015; 47(4): 1237-59. |
| [165] | Krumpal I. Determinants of social desirability bias in sensitive surveys: A literature review. Qual Quant 2013; 47(4): 2025-47. |
| [166] | Smith JM, Smith P. Testing people at work: Competencies in psychometric testing. Blackwell 2005. |
| [167] | Wood R, Payne T. Competency-based recruitment and selection 1998. |
| [168] | EEOC issues fact sheet on employment tests and selection procedures to screen applicants, workers: Agency highlights ‘best practices’ for employers to prevent job discrimination 2007. Available from: United States Equal Employment Opportunity Commission https://www.eeoc.gov/newsroom/eeoc-issues-fact-sheet-employment-tests-and-selection-procedures-screen-applicants-workers |
| [169] | Employment Tests and Selection Procedures 2010. Available from: https://www.eeoc.gov/laws/guidance/employment-tests-and-selection-procedures |
| [170] | Avoiding adverse impact in employment practices 2010. Available from: https://www.shrm.org/ResourcesAndTools/tools-and samples/ toolkits/Pages/avoidingadverseimpact.aspx |
| [171] | Peresie JL. Toward a coherent test for disparate impact discrimination. Indiana Law J 2009; 84(3): 773-802. |
| [172] | Hoppock R. Job satisfaction 1935. |
| [173] | Houser JD. What people want from business 1938. |
| [174] | Schwartz B. Why we work 2015. |
| [175] | Slemp GR, Kern ML, Patrick KJ, Ryan RM. Leader autonomy support in the workplace: A meta-analytic review. Motiv Emot 2018; 42(5): 706-24. |
| [176] | Calder RK, Mussap AJ. Factors influencing women’s choice of weight-loss diet. J Health Psychol 2015; 20(5): 612-24. |
| [177] | Frank L, Matza LS, Hanlon J, et al. The patient experience of depression and remission: Focus group results. J Nerv Ment Dis 2007; 195(8): 647-54. |
| [178] | Hartmann C, Dohle S, Siegrist M. A self-determination theory approach to adults’ healthy body weight motivation: A longitudinal study focussing on food choices and recreational physical activity. Psychol Health 2015; 30(8): 924-48. |
| [179] | Somer E. Food & mood: The complete guide to eating well and feeling your best 2nd ed. 1999. |
| [180] | Teitelbaum J. From fatigued to fantastic! 2007. |
| [181] | Gravetter FJ, Wallnau LB. Essentials of statistics for the behavioral sciences 8th ed. 2013. |
