Managerial Ethics

You are viewing an old revision of this post, from December 5, 2020 @ 14:56:50. See below for differences between this version and the current revision.

Recently, Google fired leading ethical AI research Timnit Gebru. Timnit was a manager at Google’s Ethical AI Research division, having wrote a scathing email to an internal listserv (Google Brain Women and Allies) as well as sent an ultimatum to her grandboss (boss’s boss) Megan Kacholia. Megan (as far as I can tell) reports to Jeff Dean, head of research.

Timnit was fired for behavior “inconsistent with the expectations of a Google manager.”

Let’s talk about that, as well as about Jeff Dean’s public responses.

What’s Happened

Timnit, one of the brightest voices in the Ethical AI field, joined Google Research. Google hired her as a manager; we can speculate that because Timnit has such a towering reputation in the field, Google wanted her to build out a team and hire the best people in the field.

Google’s behavior reflects an enlightened self-interest. It is surprisingly easy to build machine learning models that are not ethical – reinforcing systemic biases – even for experts in machine learning. This is not a new problem (advertising has known of the dangers of e.g. including zip code in regression models for decades, due to redlining) but as models get more complex, harder to debug, and more widely used having models that are “ethical” is fantastically important. That’s both for an ethical reason and also for a liability reason – it is not difficult to believe that, quite soon, different regulatory environments across the world will start taking a skeptical look at these (extremely profitable) models.

Once inside, as a manager, Timnit had two responsibilities: first, to do excellent research (the same as being on the outside) and second, to work in Google’s interest as a representative of the company.

This all came to a head when:

  1. Her management (this appears to be Megan, her grandboss) blocked a paper from being submitted. Despite what Jeff Dean says, this kind of thing has been rare, andwould have been extremely surprising to everyone involved.
  2. Timnit got really pissed off – and quite rightfully so. Blocking this kind of submission threatens Job Number #1: doing excellent research, that Google hired her for.
  3. Timnit escalated all the way: in order to indicate how important this was to her management (we can assume they weren’t understanding why it was a big deal) she threatened to resign. This is usually a signaling thing, i.e. I can no longer do my job effectively if this stands.
  4. Timnit vents to an “ally” group, gripes that she doesn’t seem to have the influence to do Job Number 2: work in Google’s interests (as defined by getting Google to adopt ethical/diversity goals), and floats the idea of external pressure.
  5. Google (Megan) fired her.

The paper under discussion has leaked and the abstract is here. It appears to be normal, even boring, but provides some additional insight. Basically, the paper points out that modern ML systems have a non-trivial environmental impact (this is known) and that ingesting random web data can encode unknown bias into the model (this is known). More research on this is good and expected.

It also suggests something about what happened internally, e.g. why the paper was blocked. Let’s speculate that everyone was operating in good faith.

  1. Legal and PR review the paper and find that it points out specific flaws in models that Google is running.
  2. PR has nightmares of headlines after it comes out – “Google Research says its own AI models have problematic bias, environmental impact” and says “Absolutely not!”
  3. They inform Megan, who says something like “OK, good to call out that risk, let’s make sure we add things to the paper about how Google is ahead of these problems – we’ll miss this deadline, but OK” (Jeff Dean cites this in his public response)
  4. Timnit is furious (as above; plus, specific conferences only happen once a year) and things go sideways.

Why did Megan fire Timnit?

Timnit broke two cardinal rules – rules of managerial ethics:

  1. Don’t increase the risk the company faces: By threating external pressure (legal action) and promoting the idea to other employees, Timnit is (by one interpretation) operating against the best interests of Google.
  2. Don’t mark yourself as outside the team: By threatening to resign, Timnit put herself on the other side of Google – i.e. less interested in working to make Google successful, and more interested in her own agenda. It is unacceptable to have a manager that does not subordinate their own interests to the company.

It’s easy (as a manager) to look at the email sent and say “Yep, that’s a firable offense. Completely unacceptable for an authority figure to say those things, and deeply unethical to promote anti-Google actions.”

Should Megan have fired Timnit?

My personal read is that while firing Timnit is understandable, it was a mistake. It’s a sub-optimal management decision, and feels more like a knee-jerk reaction to someone not accepting Megan’s power and authority and daring to challenge her. People who have power sometimes get a little too used to that power.

What should have happened?

  • PR had valid concerns of the headlines, but the decision to block the paper from being submitted – especially since papers could be edited after submission – was bad. PR should not get to block scientific research: it violates a separation of concerns, and diminishes the value of the Research department (other researchers like your research less if they know PR and Legal have first say).
  • PR should have waited for paper to be accepted, and then actively reached out to reporters to cover the paper – promote it! – and then point to additional research and work that Google has been doing to mitigate all of this. PR’s job is to manage the flow of information from Google to the public – it should have managed that.
  • Megan should have made that happen. Allowing PR/Legal to block the paper is a bad decision for someone to whom the Research department reports to.
  • Timnit signaled that this was very important: Megan should have taken that as a signal for how extreme a step blocking the paper is, and not as a threat to her own power (or to Google).
  • By escalating this, ironically, it is easy to argue that Timnit was operating in Google’s best interests. If people know that PR/Legal can block your papers (especially ethical AI papers!) the value of the Research you do degrades. That is, the research team turns into an expensive extension of the PR department.
  • Megan should have consulted Timnit’s manager. Remember, Megan is Timnit’s boss’s boss. The firing should at least have come from Timnit’s manage, and it’s possible Timnit’s manager could have de-escalated more easily.
  • Timnit should not have sent that email to the internal Women and Allies listserv; however, absent this other stuff that doesn’t quite seem fireable. It’s more an opportunity for internal coaching and training – why you shouldn’t do that, how to channel that frustration, and how to be more effective internally. Sending such an email to an “ally” group is much more acceptable than sending it to, say, your direct reports or your broader team. In the first one, you are speaking in a “safe space”; in the other, you are exerting your role as a manager and infusing communication with corporate authority.  

Finally, firing Timnit raised the stakes and enabled a much larger public blowback than allowing the paper to be published would have. To put it another way, the paper represented a small PR threat: firing Timnit was a much larger PR threat. I would call that “a mistake.”

Management holds all the power here: continuing to escalate as a manager is not a good look. Part of managements’ job is de-escalation.

Jeff Dean inherits the mess

It’s quite likely that Jeff Dean knew none of this before Timnit was fired, and there was a public backlash.

He now has multiple problems:

  1. It’s really, really important that researchers still at Google don’t take this the wrong way – that Google is not the place to do research. Google wants to hire the best people, and do the best research. It can’t do that if this has a chilling effect on the work people are doing.
  2. It’s really important that other people don’t believe that PR and Legal can block Google’s research papers, because that would undermine the value of the enterprise.
  3. Jeff Dean wants to protect the team he has – which includes Megan. I don’t know for sure, but we might believe that Megan has worked for Jeff for a long time and he values her tremendously. Even if this was a mistake, everyone makes mistakes.

Jeff Dean’s public response tries to thread that needle without admitting wrongdoing. Admission of wrongdoing would be easy, but it might not protect his team. So he tries to reframe the review process; reframe why the paper was blocked; and put it in context of the wide work that the entire team is doing. These are all good things to do.

Good for Jeff Dean: as expected, he’s doing a great job.

Where do people go from here?

I’m not really sure. I hope Timnit continues to do great work, and I hope she can find a way to be more effective.

My own personal view is that there needs to be a really big shift, something similar to the shift that Control Systems research went through in the 1960s – 1980s around safety. Right now people (engineering, product managers, etc) tend to look at “incidence” (oh this only happens 3% of the time, NOT A BIG DEAL); rarely at conditional incidence (this happens 90% of the time for this subpopulation, which is 3.5%), and almost never safety (this should NEVER happen).

I’m a big proponent of AI/ML – I think that automatic decision systems are as transformative to society as the invention of things like writing, the printing press, and engines. Figuring out how to do this well is hard, and we need people to tackle these hard problems. Timnit has been tackling some of them: work like that needs to continue.

Appendix: A Cry for Help

As a minor note, the email that Timnit did send to Women And Allies listserv sounds a bit like a cry for help. She points out that she’s found it hard to be effective at Job Number 2: work in Google’s best interests. Let’s look at this one line:

I understand that the only things that mean anything at Google are levels, I’ve seen how my expertise has been completely dismissed.

That implies that when Timnit took research or questions to other parts of the organization (i.e. talking to Directors to enable Google to tackle ethical/diversity problems before) she was dismissed because she was “only” a manager. This is despite her comparative expertise: she had expertise they did not.

The other complaints (difficulty hiring women; ignoring opinions from marginalized groups; inability to get Google management to care about these issues) all suggests a role mismatch: Google was asking her to leverage her expertise (worldwide leading ethical AI research) but had not given her organizational remit/power to do that work. Obviously, this is a hard needle to thread: the right answer is probably some kind of change management.

Some obvious ideas:

  • Change through organizational hierarchy: If researchers cannot effectively drive change on implementation/engineering teams, then create a mechanism where research highlights issues, and then push those up-and-down (up through Research management, down through Eng).
  • Change through process:If other teams (e.g. Eng) ignore researchers that don’t have the right title (e.g. “Director” or “VP”) then given researchers something like a veto power. That’s the nuclear option – an easier thing would be to just allow research some amount of time (3 months?) to be able to hold things “under review.” Effective teams would hate that and probably work with researchers in advance to prevent that from happening.
  • Change through incentives: Change the promotion system / bonus review system to include “reviews” of the work that engineering teams have done; doing something that ignores research could then “neuter” the value of the work. People not getting promoted changes incentive systems very quickly indeed.
  • Senior-ize some researchers: Just give some researchers (or ethical AI people specifically) “undeserved” organizational power.

These solutions all assume that Google really cares about having its deployments on the cutting edge, or at least not doing obviously flawed things. That’s not guaranteed! They could, for interest, be more interested in making more, protecting their monopoly, defending from regulatory options, etc.

Appendix: A Lesson for PR Departments

It sounds like Legal/PR review used here is basically identical to the review process used for general publication that employees write, e.g. if an employee is writing a piece to appear in a magazine, or is doing some document they want to publish.

In those cases, there is often no real external deadline; or PR might be in control (wanting to “time” the publication). Plus, the work being done is done for the sake of the organization — perhaps a fringe benefit is employee reputation. Moreover, the expected audience is large, and general (PR employees know how reporters and the public will respond a lot better that most employees).

These assumptions are not true for research papers. The public does not read research papers: they are read by other experts in the field. Reporters are unlikely to read the paper – especially if you give them a nice 3-paragraph explanation.

That means that editing or blocking the paper doesn’t really serve the same purpose as “normal” review. Rather, rapid response in parallel with paper submission would be more effective.

Post Revisions:

Changes:

December 5, 2020 @ 14:56:50Current Revision
Content
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Recently, Google fired leading ethical AI research <a href="https:/ /twitter.com/ timnitGebru">Timnit Gebru</a>. Timnit was a manager <p>Recently, Google fired leading ethical AI research <a href="https:/ /twitter.com/ timnitGebru">Timnit Gebru</a>. Timnit was a manager
at Google’s Ethical AI Research division, having wrote a <a href="https:/ /www.platformer.news/p/the- withering-email-that-got- an-ethical">scathing at Google’s Ethical AI Research division, having wrote a <a href="https:/ /www.platformer.news/p/the- withering-email-that-got- an-ethical">scathing
email</a> to an internal listserv (Google Brain Women and Allies) as well as email</a> to an internal listserv (Google Brain Women and Allies) as well as
sent an ultimatum to her grandboss (boss’s boss) Megan Kacholia. Megan (<em>as sent an ultimatum to her grandboss (boss’s boss) Megan Kacholia. Megan (<em>as
far as I can tell</em>) reports to Jeff Dean, head of research.</p> far as I can tell</em>) reports to Jeff Dean, head of research.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Timnit was fired for behavior “inconsistent with the <p>Timnit was fired for behavior “inconsistent with the
expectations of a Google manager.”</p> expectations of a Google manager.”</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Let’s talk about that, as well as about Jeff Dean’s public <p>Let’s talk about that, as well as about Jeff Dean’s public
responses.</p> responses.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:heading {"level":1} --> <!-- wp:heading {"level":1} -->
<h1>What's Happened</h1> <h1>What's Happened</h1>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Timnit, one of the brightest voices in the Ethical AI field, <p>Timnit, one of the brightest voices in the Ethical AI field,
joined Google Research. Google hired her as a manager; we can speculate that <em>because</em> joined Google Research. Google hired her as a manager; we can speculate that <em>because</em>
Timnit has such a towering reputation in the field, Google wanted her to build Timnit has such a towering reputation in the field, Google wanted her to build
out a team and hire the best people in the field.</p> out a team and hire the best people in the field.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Google’s behavior reflects an enlightened self-interest. It <p>Google’s behavior reflects an enlightened self-interest. It
is <em>surprisingly easy</em> to build machine learning models that are not is <em>surprisingly easy</em> to build machine learning models that are not
ethical – reinforcing systemic biases – even for experts in machine learning. ethical – reinforcing systemic biases – even for experts in machine learning.
This is not a new problem (<em>advertising has known of the dangers of e.g. This is not a new problem (<em>advertising has known of the dangers of e.g.
including zip code in regression models for decades, due to redlining</em>) but including zip code in regression models for decades, due to redlining</em>) but
as models get more complex, harder to debug, and more widely used having models as models get more complex, harder to debug, and more widely used having models
that are “ethical” is fantastically important. That’s both for an <em>ethical </em>reason that are “ethical” is fantastically important. That’s both for an <em>ethical </em>reason
and also for a <em>liability</em> reason – it is not difficult to believe that, quite and also for a <em>liability</em> reason – it is not difficult to believe that, quite
soon, different regulatory environments across the world will start taking a soon, different regulatory environments across the world will start taking a
skeptical look at these (<em>extremely profitable</em>) models. </p> skeptical look at these (<em>extremely profitable</em>) models. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Once inside, as a manager, Timnit had two responsibilities: <p>Once inside, as a manager, Timnit had two responsibilities:
first, to do excellent research (<em>the same as being on the outside</em>) and first, to do excellent research (<em>the same as being on the outside</em>) and
second, to work in Google’s interest as a representative of the company.</p> second, to work in Google’s interest as a representative of the company.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>This all came to a head when:</p> <p>This all came to a head when:</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} --> <!-- wp:list {"ordered":true} -->
<ol><li>Her management (<em>this appears to be Megan</em>, <ol><li>Her management (<em>this appears to be Megan</em>,
<em>her grandboss</em>) blocked a paper from being submitted. Despite what Jeff <em>her grandboss</em>) blocked a paper from being submitted. Despite what Jeff
Dean says, this kind of thing has been <strong>rare</strong>, andwould have been Dean says, this kind of thing has been <strong>rare</strong>, andwould have been
extremely surprising to everyone involved.</li><li>Timnit got really pissed off – and <em>quite extremely surprising to everyone involved.</li><li>Timnit got really pissed off – and <em>quite
rightfully so</em>. Blocking this kind of submission threatens Job Number #1: rightfully so</em>. Blocking this kind of submission threatens Job Number #1:
doing excellent research, that Google hired her for. </li><li>Timnit escalated <em>all the way</em>: in order to doing excellent research, that Google hired her for. </li><li>Timnit escalated <em>all the way</em>: in order to
indicate how important this was to her management (<em>we can assume they weren’t indicate how important this was to her management (<em>we can assume they weren’t
understanding why it was a big deal</em>) she threatened to resign. This is understanding why it was a big deal</em>) she threatened to resign. This is
usually a signaling thing, i.e. I can no longer do my job effectively if this usually a signaling thing, i.e. I can no longer do my job effectively if this
stands.</li><li>Timnit vents to an “ally” group, gripes that she stands.</li><li>Timnit vents to an “ally” group, gripes that she
doesn’t seem to have the influence to do Job Number 2: work in Google’s doesn’t seem to have the influence to do Job Number 2: work in Google’s
interests (<em>as defined by getting Google to adopt ethical/diversity goals</em>), interests (<em>as defined by getting Google to adopt ethical/diversity goals</em>),
and floats the idea of external pressure.</li><li>Google (Megan) fired her.</li></ol> and floats the idea of external pressure.</li><li>Google (Megan) fired her.</li></ol>
<!-- /wp:list --> <!-- /wp:list -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>The paper under discussion <a href="https:/ /www.technologyreview.com/ 2020/12/04/1013294/google- ai-ethics-research-paper- forced-out-timnit-gebru">has leaked</a> and the <a href="https:/ /www.reddit.com/ r/MachineLearning/ comments/k69eq0/ n_the_abstract_ of_the_paper_ that_led_to_ timnit/">abstract is here</a>. It appears to be normal, even boring, but provides some additional insight. Basically, the paper points out that modern ML systems have a non-trivial environmental impact (this is known) and that ingesting random web data can encode unknown bias into the model (this is known). More research on this is good and expected.</p> <p>The paper under discussion <a href="https:/ /www.technologyreview.com/ 2020/12/04/1013294/google- ai-ethics-research-paper- forced-out-timnit-gebru">has leaked</a> and the <a href="https:/ /www.reddit.com/ r/MachineLearning/ comments/k69eq0/ n_the_abstract_ of_the_paper_ that_led_to_ timnit/">abstract is here</a>. It appears to be normal, even boring, but provides some additional insight. Basically, the paper points out that modern ML systems have a non-trivial environmental impact (this is known) and that ingesting random web data can encode unknown bias into the model (this is known). More research on this is good and expected.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>It also <em>suggests</em> something about what happened <p>It also <em>suggests</em> something about what happened
internally, e.g. why the paper was blocked. Let’s speculate that everyone was internally, e.g. why the paper was blocked. Let’s speculate that everyone was
operating in good faith.</p> operating in good faith.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} --> <!-- wp:list {"ordered":true} -->
<ol><li>Legal and PR review the paper and find that it <ol><li>Legal and PR review the paper and find that it
points out specific flaws in models that Google is running.</li><li>PR has nightmares of headlines after it comes points out specific flaws in models that Google is running.</li><li>PR has nightmares of headlines after it comes
out – “Google Research says its own AI models have problematic bias, out – “Google Research says its own AI models have problematic bias,
environmental impact” and says “Absolutely not!”</li><li>They inform Megan, who says something like “OK, good environmental impact” and says “Absolutely not!”</li><li>They inform Megan, who says something like “OK, good
to call out that risk, let’s make sure we add things to the paper about how to call out that risk, let’s make sure we add things to the paper about how
Google is ahead of these problems – we’ll miss this deadline, but OK” (<em>Jeff Google is ahead of these problems – we’ll miss this deadline, but OK” (<em>Jeff
Dean cites this in <a href="https:/ /docs.google.com/document/ d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/ preview?pru=AAABdlOOKBs*gTzLnuI53B2IS2BISVcgAQ">his Dean cites this in <a href="https:/ /docs.google.com/document/ d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/ preview?pru=AAABdlOOKBs*gTzLnuI53B2IS2BISVcgAQ">his
public response</a>< /em>)</li><li>Timnit is furious (<em>as above; plus, specific public response</a>< /em>)</li><li>Timnit is furious (<em>as above; plus, specific
conferences only happen once a year</em>) and things go sideways.</li></ol> conferences only happen once a year</em>) and things go sideways.</li></ol>
<!-- /wp:list --> <!-- /wp:list -->
<!-- wp:heading --> <!-- wp:heading -->
<h2>Why did Megan fire Timnit?</h2> <h2>Why did Megan fire Timnit?</h2>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Timnit broke two cardinal rules – rules of managerial ethics:</p> <p>Timnit broke two cardinal rules – rules of managerial ethics:</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} --> <!-- wp:list {"ordered":true} -->
<ol><li><strong>Don’t increase the risk the company faces</strong>: <ol><li><strong>Don’t increase the risk the company faces</strong>:
By threating external pressure (legal action) and promoting the idea to other By threating external pressure (legal action) and promoting the idea to other
employees, Timnit is (<em>by one interpretation</em>) operating against the best employees, Timnit is (<em>by one interpretation</em>) operating against the best
interests of Google.</li>< li><strong>Don’t mark yourself as outside the team</strong>: interests of Google.</li>< li><strong>Don’t mark yourself as outside the team</strong>:
By threatening to resign, Timnit put herself on the other side of Google – i.e. By threatening to resign, Timnit put herself on the other side of Google – i.e.
less interested in working to make Google successful, and more interested in less interested in working to make Google successful, and more interested in
her own agenda. It is <em>unacceptable</em> to have a manager that does not subordinate her own agenda. It is <em>unacceptable</em> to have a manager that does not subordinate
their own interests to the company.</li></ol> their own interests to the company.</li></ol>
<!-- /wp:list --> <!-- /wp:list -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>It’s easy (<em>as a manager</em>) to look at the email sent <p>It’s easy (<em>as a manager</em>) to look at the email sent
and say “Yep, that’s a firable offense. Completely unacceptable for an authority and say “Yep, that’s a firable offense. Completely unacceptable for an authority
figure to say those things, and deeply unethical to promote anti-Google figure to say those things, and deeply unethical to promote anti-Google
actions.”</p> actions.”</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:heading --> <!-- wp:heading -->
<h2>Should Megan have fired Timnit?</h2> <h2>Should Megan have fired Timnit?</h2>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>My personal read is that while firing Timnit is <p>My personal read is that while firing Timnit is
understandable, it was a <em>mistake</em>. It’s a sub-optimal management decision, understandable, it was a <em>mistake</em>. It’s a sub-optimal management decision,
and feels more like a knee-jerk reaction to someone not accepting Megan’s power and feels more like a knee-jerk reaction to someone not accepting Megan’s power
and authority and daring to challenge her. People who have power sometimes get and authority and daring to challenge her. People who have power sometimes get
a little too used to that power.</p> a little too used to that power.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>What should have happened?</p> <p>What should have happened?</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:list --> <!-- wp:list -->
<ul><li>PR had valid concerns of the headlines, but the decision to block the paper from being submitted – especially since papers could be edited after submission – was <strong>bad</strong>. PR <em>should not</em> get to block scientific research: it violates a separation of concerns, and diminishes the value of the Research department (other researchers like your research less if they know PR and Legal have first say).</li><li>PR should have waited for paper to be accepted, and then <strong>actively reached out to reporters</strong> to cover the paper – promote it! – and then point to <em>additional research and work</em> that Google has been doing to mitigate all of this. PR’s job is to manage the flow of information from Google to the public – it should have managed that. </li><li>Megan should have made that happen. Allowing PR/Legal to block the paper is a bad decision for someone to whom the Research department reports to. </li><li>Timnit signaled that this was <em>very important</em>: Megan should have taken that as a signal for how extreme a step blocking the paper is, and not as a threat to her own power (or to Google).</li><li>By escalating this, <em>ironically</em>, it is easy to argue that Timnit was operating in Google’s best interests. If people know that PR/Legal can block your papers (especially ethical AI papers!) the value of the Research you do degrades. That is, the research team turns into an expensive extension of the PR department.</ li><li>Megan should have <em>consulted Timnit's manager</em>. Remember, Megan is Timnit's boss's boss. The firing should at least <em>have come from</em> Timnit's manage, and it's possible Timnit's manager could have de-escalated more easily. </li><li>Timnit <strong>should not</strong> have sent that email to the internal Women and Allies listserv; however, absent this other stuff that doesn’t quite seem fireable. It’s more an opportunity for internal coaching and training – why you shouldn’t do that, how to channel that frustration, and how to be more effective internally. Sending such an email to an “ally” group is <em>much more acceptable</em> than sending it to, say, your direct reports or your broader team. In the first one, you are speaking in a “safe space”; in the other, you are exerting your role as a manager and infusing communication with corporate authority. &nbsp;</li></ul> <ul><li>PR had valid concerns of the headlines, but the decision to block the paper from being submitted – especially since papers could be edited after submission – was <strong>bad</strong>. PR <em>should not</em> get to block scientific research: it violates a separation of concerns, and diminishes the value of the Research department (other researchers like your research less if they know PR and Legal have first say).</li><li>PR should have waited for paper to be accepted, and then <strong>actively reached out to reporters</strong> to cover the paper – promote it! – and then point to <em>additional research and work</em> that Google has been doing to mitigate all of this. PR’s job is to manage the flow of information from Google to the public – it should have managed that. </li><li>Megan should have made that happen. Allowing PR/Legal to block the paper is a bad decision for someone to whom the Research department reports to. </li><li>Timnit signaled that this was <em>very important</em>: Megan should have taken that as a signal for how extreme a step blocking the paper is, and not as a threat to her own power (or to Google).</li><li>By escalating this, <em>ironically</em>, it is easy to argue that Timnit was operating in Google’s best interests. If people know that PR/Legal can block your papers (especially ethical AI papers!) the value of the Research you do degrades. That is, the research team turns into an expensive extension of the PR department.</ li><li>Megan should have <em>consulted Timnit's manager</em>. Remember, Megan is Timnit's boss's boss. The firing should at least <em>have come from</em> Timnit's manage, and it's possible Timnit's manager could have de-escalated more easily. </li><li>Timnit <strong>should not</strong> have sent that email to the internal Women and Allies listserv; however, absent this other stuff that doesn’t quite seem fireable. It’s more an opportunity for internal coaching and training – why you shouldn’t do that, how to channel that frustration, and how to be more effective internally. Sending such an email to an “ally” group is <em>much more acceptable</em> than sending it to, say, your direct reports or your broader team. In the first one, you are speaking in a “safe space”; in the other, you are exerting your role as a manager and infusing communication with corporate authority. &nbsp;</li></ul>
<!-- /wp:list --> <!-- /wp:list -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Finally, firing Timnit raised the stakes and enabled a <em>much larger public blowback</em> than allowing the paper to be published would have. To put it another way, the paper represented a small PR threat: firing Timnit was a <em>much larger PR threat</em>. I would call that "a mistake."</p> <p>Finally, firing Timnit raised the stakes and enabled a <em>much larger public blowback</em> than allowing the paper to be published would have. To put it another way, the paper represented a small PR threat: firing Timnit was a <em>much larger PR threat</em>. I would call that "a mistake."</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Management holds all the power here: <em>continuing to escalate</em> as a manager is not a good look. Part of managements' job is de-escalation.</p> <p>Management holds all the power here: <em>continuing to escalate</em> as a manager is not a good look. Part of managements' job is de-escalation.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:heading {"level":1} --> <!-- wp:heading {"level":1} -->
<h1>Jeff Dean inherits the mess</h1> <h1>Jeff Dean inherits the mess</h1>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>It’s quite likely that Jeff Dean knew none of this before Timnit <p>It’s quite likely that Jeff Dean knew none of this before Timnit
was fired, and there was a public backlash. </p> was fired, and there was a public backlash. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>He now has multiple problems:</p> <p>He now has multiple problems:</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} --> <!-- wp:list {"ordered":true} -->
<ol><li>It’s really, really important that researchers <ol><li>It’s really, really important that researchers
still at Google don’t take this the wrong way – that Google is not the place to still at Google don’t take this the wrong way – that Google is not the place to
do research. Google wants to hire the best people, and do the best research. It do research. Google wants to hire the best people, and do the best research. It
can’t do that if this has a chilling effect on the work people are doing.</li><li>It’s really important that other people don’t can’t do that if this has a chilling effect on the work people are doing.</li><li>It’s really important that other people don’t
believe that PR and Legal can block Google’s research papers, because that believe that PR and Legal can block Google’s research papers, because that
would undermine the value of the enterprise.</li><li>Jeff Dean wants to protect the <em>team he has</em> would undermine the value of the enterprise.</li><li>Jeff Dean wants to protect the <em>team he has</em>
– which includes Megan. I don’t know for sure, but we might believe that Megan – which includes Megan. I don’t know for sure, but we might believe that Megan
has worked for Jeff for a long time and he values her tremendously. Even if has worked for Jeff for a long time and he values her tremendously. Even if
this was a mistake, <em>everyone makes mistakes</em>.</li></ol> this was a mistake, <em>everyone makes mistakes</em>.</li></ol>
<!-- /wp:list --> <!-- /wp:list -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Jeff Dean’s <a href="https:/ /docs.google.com/document/ d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/ preview?pru=AAABdlOOKBs*gTzLnuI53B2IS2BISVcgAQ">public <p>Jeff Dean’s <a href="https:/ /docs.google.com/document/ d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/ preview?pru=AAABdlOOKBs*gTzLnuI53B2IS2BISVcgAQ">public
response</a> tries to thread that needle without admitting wrongdoing. response</a> tries to thread that needle without admitting wrongdoing.
Admission of wrongdoing would be easy, but it might not <em>protect his team</em>. Admission of wrongdoing would be easy, but it might not <em>protect his team</em>.
So he tries to reframe the review process; reframe why the paper was blocked; So he tries to reframe the review process; reframe why the paper was blocked;
and put it in context of the wide work that the entire team is doing. These are and put it in context of the wide work that the entire team is doing. These are
all good things to do.</p> all good things to do.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Good for Jeff Dean: <em>as expected</em>, he’s doing a great <p>Good for Jeff Dean: <em>as expected</em>, he’s doing a great
job.</p> job.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:heading {"level":1} --> <!-- wp:heading {"level":1} -->
<h1>Where do people go from here?</h1> <h1>Where do people go from here?</h1>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>I’m not really sure. I hope Timnit continues to do great <p>I’m not really sure. I hope Timnit continues to do great
work, and I hope she can find a way to be more effective. </p> work, and I hope she can find a way to be more effective. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>My own personal view is that there needs to be a really big <p>My own personal view is that there needs to be a really big
shift, something similar to the shift that Control Systems research went shift, something similar to the shift that Control Systems research went
through in the 1960s – 1980s around <strong>safety</strong>. Right now people through in the 1960s – 1980s around <strong>safety</strong>. Right now people
(engineering, product managers, etc) tend to look at “incidence” (<em>oh this (engineering, product managers, etc) tend to look at “incidence” (<em>oh this
only happens 3% of the time, NOT A BIG DEAL</em>); rarely at conditional only happens 3% of the time, NOT A BIG DEAL</em>); rarely at conditional
incidence (<em>this happens 90% of the time for this subpopulation, which is incidence (<em>this happens 90% of the time for this subpopulation, which is
3.5%</em>), and almost never safety (<em>this should NEVER happen</em>).</p> 3.5%</em>), and almost never safety (<em>this should NEVER happen</em>).</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>I’m a big proponent of AI/ML – I think that <em>automatic decision <p>I’m a big proponent of AI/ML – I think that <em>automatic decision
systems</em> are as transformative to society as the invention of things like writing, systems</em> are as transformative to society as the invention of things like writing,
the printing press, and engines. Figuring out how to do this well is hard, and the printing press, and engines. Figuring out how to do this well is hard, and
we need people to tackle these hard problems. Timnit has been tackling some of we need people to tackle these hard problems. Timnit has been tackling some of
them: work like that needs to continue. </p> them: work like that needs to continue. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:heading {"level":1} --> <!-- wp:heading {"level":1} -->
<h1>Appendix: A Cry for Help</h1> <h1>Appendix: A Cry for Help</h1>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>As a minor note, the email that Timnit did send to Women And <p>As a minor note, the email that Timnit did send to Women And
Allies listserv sounds a bit like a cry for help. She points out that she’s Allies listserv sounds a bit like a cry for help. She points out that she’s
found it hard to be effective at <strong>Job Number 2</strong>: work in Google’s best found it hard to be effective at <strong>Job Number 2</strong>: work in Google’s best
interests. Let’s look at this one line:</p> interests. Let’s look at this one line:</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:quote --> <!-- wp:quote -->
<blockquote class="wp-block-quote"><p>I understand that the only things that mean anything at Google are levels, I’ve seen how my expertise has been completely dismissed.</p> </blockquote> <blockquote class="wp-block-quote"><p>I understand that the only things that mean anything at Google are levels, I’ve seen how my expertise has been completely dismissed.</p> </blockquote>
<!-- /wp:quote --> <!-- /wp:quote -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>That <em>implies </em>that when Timnit took research or <p>That <em>implies </em>that when Timnit took research or
questions to other parts of the organization (i.e. talking to Directors to questions to other parts of the organization (i.e. talking to Directors to
enable Google to tackle ethical/diversity problems before) she was dismissed because enable Google to tackle ethical/diversity problems before) she was dismissed because
she was “only” a manager. This is despite her <em>comparative expertise</em>: she she was “only” a manager. This is despite her <em>comparative expertise</em>: she
had expertise they did not. </p> had expertise they did not. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>The other complaints (<em>difficulty hiring women; ignoring opinions <p>The other complaints (<em>difficulty hiring women; ignoring opinions
from marginalized groups; inability to get Google management to care about from marginalized groups; inability to get Google management to care about
these issues</em>) all suggests a role mismatch: Google was <em>asking her</em> to these issues</em>) all suggests a role mismatch: Google was <em>asking her</em> to
leverage her expertise (worldwide leading ethical AI research) but had not leverage her expertise (worldwide leading ethical AI research) but had not
given her organizational remit/power to do that work. Obviously, this is a hard given her organizational remit/power to do that work. Obviously, this is a hard
needle to thread: the right answer is probably some kind of change management.</p> needle to thread: the right answer is probably some kind of change management.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>Some obvious ideas:</p> <p>Some obvious ideas:</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:list --> <!-- wp:list -->
<ul><li><strong>Change through organizational hierarchy</strong>: If researchers cannot effectively drive change on implementation/engineering teams, then create a mechanism where research highlights issues, and then push those up-and-down (up through Research management, down through Eng).</li><li> <strong>Change through process</strong>:If other teams (e.g. Eng) ignore researchers that don’t have the right title (e.g. “Director” or “VP”) then given researchers something like a veto power. That’s the nuclear option – an easier thing would be to just allow research some amount of time (3 months?) to be able to hold things “under review.” Effective teams would <em>hate</em> that and probably work with researchers in advance to prevent that from happening.</li> <li><strong>Change through incentives</strong>: Change the promotion system / bonus review system to include “reviews” of the work that engineering teams have done; doing something that ignores research could then “neuter” the value of the work. People not getting promoted changes incentive systems very quickly indeed.</li>< li><strong>Senior-ize some researchers</strong>: Just give some researchers (or ethical AI people specifically) "undeserved" organizational power. </li></ul> <ul><li><strong>Change through organizational hierarchy</strong>: If researchers cannot effectively drive change on implementation/engineering teams, then create a mechanism where research highlights issues, and then push those up-and-down (up through Research management, down through Eng).</li><li> <strong>Change through process</strong>:If other teams (e.g. Eng) ignore researchers that don’t have the right title (e.g. “Director” or “VP”) then given researchers something like a veto power. That’s the nuclear option – an easier thing would be to just allow research some amount of time (3 months?) to be able to hold things “under review.” Effective teams would <em>hate</em> that and probably work with researchers in advance to prevent that from happening.</li> <li><strong>Change through incentives</strong>: Change the promotion system / bonus review system to include “reviews” of the work that engineering teams have done; doing something that ignores research could then “neuter” the value of the work. People not getting promoted changes incentive systems very quickly indeed.</li>< li><strong>Senior-ize some researchers</strong>: Just give some researchers (or ethical AI people specifically) "undeserved" organizational power. </li></ul>
<!-- /wp:list --> <!-- /wp:list -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>These solutions all assume that Google really cares about having its deployments on the cutting edge, or at least not doing obviously flawed things. That’s not guaranteed! They could, for interest, be more interested in making more, protecting their monopoly, defending from regulatory options, etc. </p> <p>These solutions all assume that Google really cares about having its deployments on the cutting edge, or at least not doing obviously flawed things. That’s not guaranteed! They could, for interest, be more interested in making more, protecting their monopoly, defending from regulatory options, etc. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:heading --> <!-- wp:heading -->
<h2>Appendix: A Lesson for PR Departments</h2> <h2>Appendix: A Lesson for PR Departments</h2>
<!-- /wp:heading --> <!-- /wp:heading -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>It sounds like Legal/PR review used here is basically identical to the review process used for general publication that employees write, e.g. if an employee is writing a piece to appear in a magazine, or is doing some document they want to publish.</p> <p>It sounds like Legal/PR review used here is basically identical to the review process used for general publication that employees write, e.g. if an employee is writing a piece to appear in a magazine, or is doing some document they want to publish.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>In those cases, there is often no real external deadline; or PR might be in control (wanting to "time" the publication). Plus, the work being done is done for the sake of the organization -- perhaps a fringe benefit is employee reputation. Moreover, the expected audience is <em>large</em>, and <em>general</em> (PR employees know how reporters and the public will respond a lot better that most employees).</p> <p>In those cases, there is often no real external deadline; or PR might be in control (wanting to "time" the publication). Plus, the work being done is done for the sake of the organization -- perhaps a fringe benefit is employee reputation. Moreover, the expected audience is <em>large</em>, and <em>general</em> (PR employees know how reporters and the public will respond a lot better that most employees).</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>These assumptions are <strong>not true</strong> for research papers. The public <em>does not read</em> research papers: they are read by other experts in the field. Reporters are unlikely to read the paper - especially if you give them a nice 3-paragraph explanation. </p> <p>These assumptions are <strong>not true</strong> for research papers. The public <em>does not read</em> research papers: they are read by other experts in the field. Reporters are unlikely to read the paper - especially if you give them a nice 3-paragraph explanation. </p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->
<!-- wp:paragraph --> <!-- wp:paragraph -->
<p>That means that editing or blocking the paper doesn't really serve the same purpose as "normal" review. Rather, rapid response in parallel with paper submission would be more effective. </p> <p>That means that editing or blocking the paper doesn't really serve the same purpose as "normal" review. Rather, rapid response in parallel with paper submission would be more effective. </p>
  <!-- /wp:paragraph -->
  <!-- wp:paragraph -->
  <p>The exception is work that directly damages the company, e.g. estimating some negative effects that current company policies have. Having that in a journal article or conference paper would make mitigation (in court or the news) nearly impossible. That implies that blocking or delaying papers should have a <strong>much higher bar</strong> in research papers than for other public communications.</p>
<!-- /wp:paragraph --> <!-- /wp:paragraph -->

Note: Spaces may be added to comparison text to allow better line wrapping.