Full Length ArticleStatesmen or cheerleaders? Using topic modeling to examine gendered messages in narrative developmental feedback for leaders
Introduction
Gender inequalities persist both in leadership representation and in pathways to leadership roles. In business, women comprise a mere 5% of CEOs and 10.6% of board directors among US Fortune 500 corporations (Catalyst, 2018), and 7% of CEOs and 29% of boards members in UK FTSE 100 corporations (Vinnicombe, Doldor, & Sealy, 2018). Likewise, in politics, just 24% of national parliamentarian roles and 18% of ministerial positions across the world are held by women (UNWomen, 2018). Although many factors contribute to these inequalities, from persistent gender stereotyping and perceived role incongruity between leadership and ‘female’ attributes (Eagly & Heilman, 2016) to organizational barriers stemming from non-inclusive cultures and processes (Eagly & Carli, 2007), surprisingly little attention has been paid to how developmental feedback may contribute to women's under-representation in senior leadership roles. Unlike performance feedback, which is typically retrospective and comparative, developmental feedback is forward looking and individualized (Zhou, 2003); often in the form of a narrative it sends messages about how individuals need to act and, potentially, change in order to succeed and progress as leaders.
The current research extends scholarship on gender, feedback and leadership by examining whether narrative feedback pertaining to leadership development is gender biased in sending differential messages to male and female leaders. We do this through an innovative combination of automated topic modeling and traditional inductive analysis of narrative feedback. We utilize a large naturalistic data set comprising written comments about development needs (N = 1057) that were provided anonymously to male (n = 98) and female (n = 48) elected leaders in the United Kingdom (UK) as part of a national leadership development program for local government. Leaders requested feedback from colleagues and employed officials with whom they worked closely, and trusted to provide accurate and helpful developmental feedback about their day-to-day performance as a leader. To analyze these feedback comments, we combined topic modeling, a novel inductive software-automated text mining technique, with more traditional qualitative coding in a four-step procedure to identify differential salient messages about how men and women needed to develop in their leadership roles.
Our findings extend scholarship in two ways. First, we reveal how gender bias operates in the specific context of leadership development, through messages embedded in feedback for male and female leaders that reflect implicit stereotypes and recommend alternative developmental paths for leadership progression. We demonstrate that leadership development feedback is biased in ways that go beyond the tension posed by agency [male] and communion [female], as currently theorized in psychological literature; and that developmental feedback is less likely to direct women towards nurturing the visionary and political skills required for senior leadership roles. We integrate literatures on gender bias (Bear, Cushenbery, London, & Sherman, 2017), developmental feedback (Zhou, 2003) and leadership development (Lord & Hall, 2005) to conceptualize when and how narrative developmental feedback can support women's leadership progression. Second, we establish the utility of topic modeling for analyzing naturally occurring narrative data routinely captured and stored by organizations, but neglected by leadership scholars as too difficult or resource-intensive to analyze (Schmiedel, Muller, & vom Brocke, 2018). Importantly, we show how topic modeling can complement traditional qualitative methods such as thematic analysis, and thus expand the methodological toolbox leadership researchers can use in producing much needed exploratory work in the field (Antonakis, 2017). In particular, we demonstrate the added value of this method in exploring traditionally ‘hard-to-reach’ HR/OB topics for leadership scholars (Hannigan et al., 2019).
Section snippets
The role of feedback in leadership development
Rigorous evaluative feedback is critical both to adult learning in general (DeRue & Wellman, 2009; Kanfer & Ackerman, 1989), and to leadership development more specifically (Avolio, 2005; McCauley & Van Velsor, 2004; Morrison & Brantner, 1992; Steffens et al., 2018). Feedback leads to increased self-awareness and a more accurate perception of one's performance and ability compared to others (Bollich, Johannet, & Vazire, 2011; Kluger & DeNisi, 1996; Maki, 1998), and in the case of leaders, the
Gender bias in feedback processes
Gender complicates the developmental role of feedback for leaders. As an inherently social process, leadership development takes place in an environment suffused with ideologies about what it means to be a leader and a woman (Ely et al., 2011). Despite claims that developmental feedback challenges self-perceptions, helps individuals locate areas for development, and contributes to leadership identity construction (Ely et al., 2011), remarkably little is understood about the mechanisms by which
Research context
The context of this study is data collected in a developmental feedback process, which formed part of a formal leadership development program, established to provide local politicians across the UK with access to political learning. Although there are differences between political leaders and those in traditional business contexts in terms of the power that politicians hold to shape their roles and define performance, the characteristics and behaviors required in both forms of leadership are
Data analysis and findings
Our analytical approach was inductive and entailed four main steps. Fig. 1 provides an overview of the data analysis process. Inductive approaches are methodologically fit for underexamined and less theoretically mature research areas (Edmondson & McManus, 2007; Neuman, 2006) such as qualitative leadership development feedback (Speer, 2018). In contrast to top-down hypothesis testing characteristic to a deductive approach, an inductive approach allows for bottom-up identification of key themes
Discussion
In this research we sought to understand the nature of gender bias in developmental leadership feedback. Drawing on an inductive analysis of a large naturalistic qualitative data set, we examined narrative feedback from 1057 raters to 146 male and female political leaders, drawing out salient messages received by men and women. Using software-based topic modeling analysis, we identified and interpreted 12 topics capturing key areas of advice for leadership development that were discussed across
Conclusion
Cheryl Sandberg's (2013) now famous advice for working women to ‘lean in’ has attracted criticism for its emphasis on women's individual agency and its relative neglect of structural gender inequalities. ‘Leaning in’ is a series of choices shaped, at least partially, by the organizational landscape women navigate. Endemic organizational processes such as feedback can insidiously send different messages to men and women about what they should lean towards and how they should do it. Our findings
References (120)
On doing better science: From thrill of discovery to policy implications
The Leadership Quarterly
(2017)- et al.
Feedback as an individual resource: Personal strategies of creating information
Organizational Behavior and Human Performance
(1983) - et al.
Performance feedback, power retention, and the gender gap in leadership
The Leadership Quarterly
(2017) Words versus numbers: A theoretical exploration of giving and receiving narrative comments in performance appraisal
Human Resource Management Review
(2010)Leadership development: A review in context
The Leadership Quarterly
(2000)- et al.
Advances in leader and leadership development: A review of 25 years of research and theory
The Leadership Quarterly
(2014) - et al.
Gender and leadership: Introduction to the special issue
The Leadership Quarterly
(2016) - et al.
Leadership and followership identity processes: A multilevel review
The Leadership Quarterly
(2017) - et al.
A political conceptualization of managerial behaviour
Human Resource Management Review
(1994) - et al.
Identity, deep structure, and the development of leadership skill
The Leadership Quarterly
(2005)
On breaking the glass ceiling: the political seasoning of powerful women executives
Organizational Dynamics
The relational nature of leadership identity construction: How and when it influences perceived leadership and decision-making
The Leadership Quarterly
At the crux of dyadic leadership: Self–other agreement of leaders and direct reports—Analyzing 360-degree feedback
The Leadership Quarterly
Qualitative and historiometric methods in leadership research: A review of the first 25 years of the leadership quarterly
The Leadership Quarterly
Gender and career success: The facilitative role of political skill
Organizational Dynamics
Female first, leader second? Gender bias in the encoding of leadership behavior
Organizational Behavior and Human Decision Processes
Leader vision formation and forecasting: The effects of forecasting extent, resources, and timeframe
The Leadership Quarterly
Getting rid of performance ratings: Genius or folly? A debate
Industrial and Organizational Psychology
Transparency and replicability in qualitative research: The case of interviews with elite informants
Strategic Management Journal
Improving and evaluating topic models and other models of text
Journal of the American Statistical Association
Toward a political theory of leadership
Leadership Quarterly
Feedback-seeking in individual adaptation: A resource perspective
Academy of Management Journal
Self-regulation for managerial effectiveness: The role of active feedback seeking
Academy of Management Journal
Unconscious bias training: An assessment of the evidence for effectiveness
Men's and Women's Perceptions of the Gender Typing of Management Subroles
Sex Roles
Leadership development in balance: Made/born
A review of best practice recommendations for text analysis in R (and a user-friendly app)
Journal of Business and Psychology
Stata tip 63: Modeling proportions
Stata Journal
Shifting standards and the inference of incompetence: Effects of formal and informal evaluation tools
Personality and Social Psychology Bulletin
The language of performance evaluations: Gender-based shifts in content and consistency of judgment
Social Psychological and Personality Science
Critical perspectives on men and masculinities in politics: Introduction
Politics & Gender
Surveying a suite of algorithms that offer a solution to managing large document archives
Communication of the ACM
Latent Dirichlet allocation
Journal of Machine Learning Research
In search of our true selves: Feedback as a path to self-knowledge
Frontiers in Psychology
The evolution and devolution of 360 feedback
Industrial and Organizational Psychology
Managerial tactics for communicating negative performance feedback
Personnel Review
You stab my back, I’ll stab yours: Management experience and perceptions of organization political behaviour
British Journal of Management
Rats in the shadows: Researching organizational politics
Women CEOs of the S&P 500
Reading tea leaves: How humans interpret topic models
“Which men?” How an intersectional perspective on men and masculinities helps explain women’s political underrepresentation
Politics & Gender
Identity ambiguity and change in the wake of a corporate spin-off
Administrative Science Quarterly
Formal dining at Cambridge colleges: Linking ritual performance and institutional maintenance
Academy of Management Journal
Leadership development: An outcome-oriented review based on time and levels of analyses
Annual Review of Organizational Psychology and Organizational Behavior
Who will lead and who will follow? A social process of leadership identity construction in organizations
Academy of Management Review
A quasi-experimental study of after-event reviews and leadership development
Journal of Applied Psychology
Developing leaders via experience: The role of developmental challenge, learning orientation, and feedback availability
Journal of Applied Psychology
From politically naïve to politically mature: Conceptualizing leaders’ political maturation journey
British Journal of Management
Refining the concept of political will: A gender perspective
British Journal of Management
Cited by (24)
The impact of forum content on data science open innovation performance: A system dynamics-based causal machine learning approach
2024, Technological Forecasting and Social ChangeWhen subordinates do not follow: A typology of subordinate resistance as perceived by leaders
2023, Leadership QuarterlyUnconscious bias in the HRM literature: Towards a critical-reflexive approach
2023, Human Resource Management ReviewIn support of slow science: Robust, open, and multidisciplinary
2023, Leadership QuarterlyCommon methodological mistakes
2023, Leadership QuarterlyUsing structural topic modeling to gain insight into challenges faced by leaders
2022, Leadership QuarterlyCitation Excerpt :In contrast, topic models, by examining the word’s co-occurrence with other words, can differentiate between meanings and this identical word can be correctly classified into different topics depending on its particular context. Multiple examples of this approach have begun to appear in the organizational science literature and topic models have been shown to classify documents reliably and accurately to topics (Banks et al., 2018; Doldor et al., 2019; Schmiedel et al., 2019). Topic models can be used to answer a series of critical questions about a corpus of text.
- 1
The first and second author have contributed equally to the paper.