Full Length Article
Statesmen or cheerleaders? Using topic modeling to examine gendered messages in narrative developmental feedback for leaders

https://doi.org/10.1016/j.leaqua.2019.101308Get rights and content

Abstract

This inductive study extends scholarship on gender, feedback and leadership by drawing on a large naturalistic data set of 1057 narrative developmental feedback comments to 146 political leaders in the UK. We used automated topic modeling, a novel methodology, to identify 12 underlying topics within developmental feedback, and complemented this with an in-depth qualitative analyses of feedback content for male and female political leaders across the topics. This resulted in four aggregate theoretical dimensions: 1) strategic focus 2) political influence 3) confidence and 4) agency and communion. Our findings chart novel dimensions of gender bias that go beyond the widely theorized tension posed by agency [male] and communion [female]. These new dimensions are pertinent to developmental, rather than performance feedback processes, and provide male and female leaders with different developmental roadmaps. We outline the value of our novel methodology to leadership scholarship and discuss implications for future research and practice.

Introduction

Gender inequalities persist both in leadership representation and in pathways to leadership roles. In business, women comprise a mere 5% of CEOs and 10.6% of board directors among US Fortune 500 corporations (Catalyst, 2018), and 7% of CEOs and 29% of boards members in UK FTSE 100 corporations (Vinnicombe, Doldor, & Sealy, 2018). Likewise, in politics, just 24% of national parliamentarian roles and 18% of ministerial positions across the world are held by women (UNWomen, 2018). Although many factors contribute to these inequalities, from persistent gender stereotyping and perceived role incongruity between leadership and ‘female’ attributes (Eagly & Heilman, 2016) to organizational barriers stemming from non-inclusive cultures and processes (Eagly & Carli, 2007), surprisingly little attention has been paid to how developmental feedback may contribute to women's under-representation in senior leadership roles. Unlike performance feedback, which is typically retrospective and comparative, developmental feedback is forward looking and individualized (Zhou, 2003); often in the form of a narrative it sends messages about how individuals need to act and, potentially, change in order to succeed and progress as leaders.

The current research extends scholarship on gender, feedback and leadership by examining whether narrative feedback pertaining to leadership development is gender biased in sending differential messages to male and female leaders. We do this through an innovative combination of automated topic modeling and traditional inductive analysis of narrative feedback. We utilize a large naturalistic data set comprising written comments about development needs (N = 1057) that were provided anonymously to male (n = 98) and female (n = 48) elected leaders in the United Kingdom (UK) as part of a national leadership development program for local government. Leaders requested feedback from colleagues and employed officials with whom they worked closely, and trusted to provide accurate and helpful developmental feedback about their day-to-day performance as a leader. To analyze these feedback comments, we combined topic modeling, a novel inductive software-automated text mining technique, with more traditional qualitative coding in a four-step procedure to identify differential salient messages about how men and women needed to develop in their leadership roles.

Our findings extend scholarship in two ways. First, we reveal how gender bias operates in the specific context of leadership development, through messages embedded in feedback for male and female leaders that reflect implicit stereotypes and recommend alternative developmental paths for leadership progression. We demonstrate that leadership development feedback is biased in ways that go beyond the tension posed by agency [male] and communion [female], as currently theorized in psychological literature; and that developmental feedback is less likely to direct women towards nurturing the visionary and political skills required for senior leadership roles. We integrate literatures on gender bias (Bear, Cushenbery, London, & Sherman, 2017), developmental feedback (Zhou, 2003) and leadership development (Lord & Hall, 2005) to conceptualize when and how narrative developmental feedback can support women's leadership progression. Second, we establish the utility of topic modeling for analyzing naturally occurring narrative data routinely captured and stored by organizations, but neglected by leadership scholars as too difficult or resource-intensive to analyze (Schmiedel, Muller, & vom Brocke, 2018). Importantly, we show how topic modeling can complement traditional qualitative methods such as thematic analysis, and thus expand the methodological toolbox leadership researchers can use in producing much needed exploratory work in the field (Antonakis, 2017). In particular, we demonstrate the added value of this method in exploring traditionally ‘hard-to-reach’ HR/OB topics for leadership scholars (Hannigan et al., 2019).

Section snippets

The role of feedback in leadership development

Rigorous evaluative feedback is critical both to adult learning in general (DeRue & Wellman, 2009; Kanfer & Ackerman, 1989), and to leadership development more specifically (Avolio, 2005; McCauley & Van Velsor, 2004; Morrison & Brantner, 1992; Steffens et al., 2018). Feedback leads to increased self-awareness and a more accurate perception of one's performance and ability compared to others (Bollich, Johannet, & Vazire, 2011; Kluger & DeNisi, 1996; Maki, 1998), and in the case of leaders, the

Gender bias in feedback processes

Gender complicates the developmental role of feedback for leaders. As an inherently social process, leadership development takes place in an environment suffused with ideologies about what it means to be a leader and a woman (Ely et al., 2011). Despite claims that developmental feedback challenges self-perceptions, helps individuals locate areas for development, and contributes to leadership identity construction (Ely et al., 2011), remarkably little is understood about the mechanisms by which

Research context

The context of this study is data collected in a developmental feedback process, which formed part of a formal leadership development program, established to provide local politicians across the UK with access to political learning. Although there are differences between political leaders and those in traditional business contexts in terms of the power that politicians hold to shape their roles and define performance, the characteristics and behaviors required in both forms of leadership are

Data analysis and findings

Our analytical approach was inductive and entailed four main steps. Fig. 1 provides an overview of the data analysis process. Inductive approaches are methodologically fit for underexamined and less theoretically mature research areas (Edmondson & McManus, 2007; Neuman, 2006) such as qualitative leadership development feedback (Speer, 2018). In contrast to top-down hypothesis testing characteristic to a deductive approach, an inductive approach allows for bottom-up identification of key themes

Discussion

In this research we sought to understand the nature of gender bias in developmental leadership feedback. Drawing on an inductive analysis of a large naturalistic qualitative data set, we examined narrative feedback from 1057 raters to 146 male and female political leaders, drawing out salient messages received by men and women. Using software-based topic modeling analysis, we identified and interpreted 12 topics capturing key areas of advice for leadership development that were discussed across

Conclusion

Cheryl Sandberg's (2013) now famous advice for working women to ‘lean in’ has attracted criticism for its emphasis on women's individual agency and its relative neglect of structural gender inequalities. ‘Leaning in’ is a series of choices shaped, at least partially, by the organizational landscape women navigate. Endemic organizational processes such as feedback can insidiously send different messages to men and women about what they should lean towards and how they should do it. Our findings

References (120)

  • L.A Mainiero

    On breaking the glass ceiling: the political seasoning of powerful women executives

    Organizational Dynamics

    (1994)
  • L.A. Marchiondo et al.

    The relational nature of leadership identity construction: How and when it influences perceived leadership and decision-making

    The Leadership Quarterly

    (2015)
  • S.E. Markham et al.

    At the crux of dyadic leadership: Self–other agreement of leaders and direct reports—Analyzing 360-degree feedback

    The Leadership Quarterly

    (2015)
  • K. Parry et al.

    Qualitative and historiometric methods in leadership research: A review of the first 25 years of the leadership quarterly

    The Leadership Quarterly

    (2014)
  • P.L. Perrewe et al.

    Gender and career success: The facilitative role of political skill

    Organizational Dynamics

    (2004)
  • K.A. Scott et al.

    Female first, leader second? Gender bias in the encoding of leadership behavior

    Organizational Behavior and Human Decision Processes

    (2006)
  • A.S. Shipman et al.

    Leader vision formation and forecasting: The effects of forecasting extent, resources, and timeframe

    The Leadership Quarterly

    (2010)
  • S. Adler et al.

    Getting rid of performance ratings: Genius or folly? A debate

    Industrial and Organizational Psychology

    (2016)
  • H. Aguinis et al.

    Transparency and replicability in qualitative research: The case of interviews with elite informants

    Strategic Management Journal

    (2019)
  • E.M. Airoldi et al.

    Improving and evaluating topic models and other models of text

    Journal of the American Statistical Association

    (2016)
  • A.P. Ammeter et al.

    Toward a political theory of leadership

    Leadership Quarterly

    (2002)
  • S.J. Ashford

    Feedback-seeking in individual adaptation: A resource perspective

    Academy of Management Journal

    (1986)
  • S.J. Ashford et al.

    Self-regulation for managerial effectiveness: The role of active feedback seeking

    Academy of Management Journal

    (1991)
  • D. Atewologun et al.

    Unconscious bias training: An assessment of the evidence for effectiveness

    (2018)
  • L.E. Atwater et al.

    Men's and Women's Perceptions of the Gender Typing of Management Subroles

    Sex Roles

    (2004)
  • B.J. Avolio

    Leadership development in balance: Made/born

    (2005)
  • G.C. Banks et al.

    A review of best practice recommendations for text analysis in R (and a user-friendly app)

    Journal of Business and Psychology

    (2018)
  • C.F. Baum

    Stata tip 63: Modeling proportions

    Stata Journal

    (2008)
  • M. Biernat et al.

    Shifting standards and the inference of incompetence: Effects of formal and informal evaluation tools

    Personality and Social Psychology Bulletin

    (2010)
  • M. Biernat et al.

    The language of performance evaluations: Gender-based shifts in content and consistency of judgment

    Social Psychological and Personality Science

    (2012)
  • E. Bjarnegård et al.

    Critical perspectives on men and masculinities in politics: Introduction

    Politics & Gender

    (2018)
  • D.M. Blei

    Surveying a suite of algorithms that offer a solution to managing large document archives

    Communication of the ACM

    (2012)
  • D.M. Blei et al.

    Latent Dirichlet allocation

    Journal of Machine Learning Research

    (2003)
  • K.L. Bollich et al.

    In search of our true selves: Feedback as a path to self-knowledge

    Frontiers in Psychology

    (2011)
  • D.W. Bracken et al.

    The evolution and devolution of 360 feedback

    Industrial and Organizational Psychology

    (2016)
  • M. Brown et al.

    Managerial tactics for communicating negative performance feedback

    Personnel Review

    (2016)
  • D. Buchanan

    You stab my back, I’ll stab yours: Management experience and perceptions of organization political behaviour

    British Journal of Management

    (2008)
  • D. Buchanan

    Rats in the shadows: Researching organizational politics

  • Catalyst

    Women CEOs of the S&P 500

  • J. Chang et al.

    Reading tea leaves: How humans interpret topic models

  • S. Childs et al.

    “Which men?” How an intersectional perspective on men and masculinities helps explain women’s political underrepresentation

    Politics & Gender

    (2018)
  • K.G. Corley et al.

    Identity ambiguity and change in the wake of a corporate spin-off

    Administrative Science Quarterly

    (2004)
  • M.T. Dacin et al.

    Formal dining at Cambridge colleges: Linking ritual performance and institutional maintenance

    Academy of Management Journal

    (2010)
  • D.V. Day et al.

    Leadership development: An outcome-oriented review based on time and levels of analyses

    Annual Review of Organizational Psychology and Organizational Behavior

    (2015)
  • D.S. DeRue et al.

    Who will lead and who will follow? A social process of leadership identity construction in organizations

    Academy of Management Review

    (2010)
  • D.S. DeRue et al.

    A quasi-experimental study of after-event reviews and leadership development

    Journal of Applied Psychology

    (2012)
  • D.S. DeRue et al.

    Developing leaders via experience: The role of developmental challenge, learning orientation, and feedback availability

    Journal of Applied Psychology

    (2009)
  • E. Doldor

    From politically naïve to politically mature: Conceptualizing leaders’ political maturation journey

    British Journal of Management

    (2017)
  • E. Doldor et al.

    Refining the concept of political will: A gender perspective

    British Journal of Management

    (2013)
  • Cited by (24)

    • Common methodological mistakes

      2023, Leadership Quarterly
    • Using structural topic modeling to gain insight into challenges faced by leaders

      2022, Leadership Quarterly
      Citation Excerpt :

      In contrast, topic models, by examining the word’s co-occurrence with other words, can differentiate between meanings and this identical word can be correctly classified into different topics depending on its particular context. Multiple examples of this approach have begun to appear in the organizational science literature and topic models have been shown to classify documents reliably and accurately to topics (Banks et al., 2018; Doldor et al., 2019; Schmiedel et al., 2019). Topic models can be used to answer a series of critical questions about a corpus of text.

    View all citing articles on Scopus
    1

    The first and second author have contributed equally to the paper.

    View full text