This is the 6th release of Twitter internal discussions, done by Matt Taibbi.
https://twitter.com/mtaibbi/status/1603857534737072128
- mtaibbi
For part 1~5, check my previous posts:
How Twitter suppressed Hunter Biden laptop story
Twitter censorship - part 3 - removal of Trump
Twitter censorship - part 4 - removal of Trump (continued)
Twitter censorship - part 5 - removal of Trump (continued)
The Twitter Files, Part Six
TWITTER, THE FBI SUBSIDIARY
The #TwitterFiles are revealing more every day about how the government collects, analyzes, and flags your social media content.
Twitter’s contact with the FBI was constant and pervasive, as if it were a subsidiary.
Between January 2020 and November 2022, there were over 150 emails between the FBI and former Twitter Trust and Safety chief Yoel Roth.
Some are mundane, like San Francisco agent Elvis Chan wishing Roth a Happy New Year along with a reminder to attend “our quarterly call next week.” Others are requests for information into Twitter users related to active investigations.
But a surprisingly high number are requests by the FBI for Twitter to take action on election misinformation, even involving joke tweets from low-follower accounts.
The FBI’s social media-focused task force, known as FTIF, created in the wake of the 2016 election, swelled to 80 agents and corresponded with Twitter to identify alleged foreign influence and election tampering of all kinds.
Federal intelligence and law enforcement reach into Twitter included the Department of Homeland Security, which partnered with security contractors and think tanks to pressure Twitter to moderate content.
It’s no secret the government analyzes bulk data for all sorts of purposes, everything from tracking terror suspects to making economic forecasts.
The #TwitterFiles show something new: agencies like the FBI and DHS regularly sending social media content to Twitter through multiple entry points, pre-flagged for moderation.
What stands out is the sheer quantity of reports from the government. Some are aggregated from public hotlines:
An unanswered question: do agencies like FBI and DHS do in-house flagging work themselves, or farm it out? “You have to prove to me that inside the fucking government you can do any kind of massive data or AI search,” says one former intelligence officer.
“HELLO TWITTER CONTACTS”: The master-canine quality of the FBI’s relationship to Twitter comes through in this November 2022 email, in which “FBI San Francisco is notifying you” it wants action on four accounts:
On Thu, Nov 10, 2022 at 10:53 PM (redacted)@fbi.gov> wrote:
Hello Twitter contacts,
FBI San Francisco is notifying you of the below accounts which may potentially constitute violations of Twitter's Terms of Service for any action or inaction deemed appropriate within Twitter policy.
@joahnathan1wade
@fromMA
@madandpissedoff
@mault_thomasBest Regards,
Fred
FBI SF
Twitter personnel in that case went on to look for reasons to suspend all four accounts, including @fromma, whose tweets are almost all jokes (see sample below), including his “civic misinformation” of Nov. 8:
From: (redacted)twitter.com
Subject Re: election related potential TOS violations
Date: November 10, 2022 at 3:12 PM
(redacted)@twitter.com
(redacted)@twitter.com, (redacted)@twitter.com, (redacted)@twitter.com
Thanks Patrick. Ive escalated to GET for a first pass.
On Thu, Nov 10, 2022 at 6:03 PM (redacted)twitter.com> wrote:-FBI folks
I've reviewed this already from the TD perspective and suspended three of the accounts for multi-account abuse and ban evasion violation.Rodrigo - could you please review @fromMA for possible civic misinformation or direct to the appropriate part of GET for their review?
Thanks,
- Patrick
Just to show the FBI can be hyper-intrusive in both directions, they also asked Twitter to review a blue-leaning account for a different joke, except here it was even more obvious that @clairefosterPHD, who kids a lot, was kidding:
Hello Twitter contacts,
FBI San Francisco is notifying you of the below accounts which may potentially constitute violations of Twitter's Terms of Service for any action or inaction deemed appropriate within Twitter policy.
Thank you!
Katherine
- Twitter post by user @byrum_wade, display name "ULTRA MAGA", stating the following: "Americans, Vote today. Democrats you vote Wednesday 9th." The tweet was posted on 8 November 2022 at 2:10 AM CST.
- Twitter account "@ClairFosterPHD" claimed in her posts that she is a ballot counter in her state and, in additional posts, states "For every negative comment on this post, I'm adding another vote for the democrats" and "If you're not wearing a mask, I'm not counting your vote."
Private Sector Engagement Squad
FBI San Francisco
“Anyone who cannot discern obvious satire from reality has no place making decisions for others or working for the feds,” said @ClaireFosterPHD, when told about the flagging.
Of the six accounts mentioned in the previous two emails, all but two - @ClaireFosterPHD and @FromMa - were suspended.
In an internal email from November 5, 2022, the FBI’s National Election Command Post, which compiles and sends on complaints, sent the SF field office a long list of accounts that “may warrant additional action”:
From: (redacted)@fbi.gov>
Sent: Sunday, November 6, 2022, 12:56 PM
(redacted)@fbi.gov
Cc: (redacted)@fbi.gov, (redacted)@fbi.gov, (redacted)@fbi.gov, (redacted)@fbi.gov
Subject: Request for Coordination with TwitterASAC Chan,
The National Election Command Post(NECP) is requesting assistance from SF regarding coordination with Twitter. Specifically, NECP has been made aware of Tweets by certain accounts that may warrant additional action due to the accounts being utilized to spread misinformation about the upcoming election. Specifically, NECP is requesting the following:1. Coordination between SF and Twitter to determine whether the accounts identified below have violated Twitter's terms of service and may be subject any actions deemed appropriate by Twitter.
2. The issuance of preservation letters regarding the accounts identified below in order to preserve subscriber information and content information pending the issuance of legal process.
3. Any location information associated with the accounts that Twitter will voluntarily provide to aid the FBI in assigning any follow-up deemed necessary to the appropriate FBI field office.Twitter Acconts
1. @DartfulCodger
2. @DrAndrewJackson
3. @DanDuryeas
4. @2020_mtb
5. @JeanneGary13
6. @RSBNetwork
7. @Davidkloy
8. @Ronsmit49336969
9. @AScottHoneycut
10. @thearmogidaship
11. @Davidkloy
12. @lexitollah
13. @Tiboron11
14. @wisefrog57
15. @hfsboatr5
16. @chriswest567
17. @Trump2017847791
18. @kag2020_2024
19. @Tiberius444
20. @BillyBaldwin
21. @chrisfig33ad
22. @michaelp4283
23. @BrettBarker22
24. @EPaul9
25. @warrenintronnoPlease let us know if you need additional information to process this request by replying all to this email.
Thanks,
- Michael
Agent Chan passed the list on to his "Twitter folks":
On Sun, Nov 6, 2022 at 9:10 PM Elvis Chan (redacted)@fbi.gov> wrote:
Twitter folks,
Please see below list of Twitter accounts which we believe are violating your terms of service by disseminating false information about the time, place, or manner of the upcoming elections.
Let us know if you decide to take any actions against these accounts based on our tipper to you. Also let us know if we need to issue a preservation letter as we intend to serve legal process for these accounts. Thanks for your consideration.
Regards, Elvis
Elvis M. Chan
Asst. Special Agent in Charge
FBI San Francisco, Cyber Branch
Work: (redacted)
Cell: (redacted)
Email: (redacted)@fbi.gov
Pronouns: he, him, his
Twitter then replied with its list of actions taken. Note mercy shown to actor Billy Baldwin:
From: Patrick Conlon (redacted)
Sent: Tuesday, November 8, 2022 09:41 AM
To: Chan, Elvis M. (SF)(FBI)(redacted)@fbi.gov
Cc: Yoel Roth (redacted)(CID)(FBI)(redacted)@fbi.gov>;(redacted)(CID)(FBI) (redacted)@fbi.gov>;(redacted)(CID)(FBI)(redacted)@fbi.gov>;(redacted)(CID)(FBI)(redacted)@fbi.gov>;(redacted(CID)(FB) (redacted)@fbi.gov>;(redacted)(SF)(FBI)(redacted)@fbi.gov>;(redacted)(SF)(FBI)(redacted)@fbi.gov>;(redacted)(SF)(FBI)(redacted)@fbi.gov>
Subject: [EXTERNAL EMAIL] - [SOCIAL NETWORK] Re: Request for Coordination with TwitterHi Elvis,
Thank you for your patience as our team assessed the accounts that you flagged.
We've completed our review and taken the following actions on some of the accounts:
Permanently suspended for policy violations (ban evasion, platform manipulation, excessive misinformation strikes) -Temporarily susbended for spam behaviors -
Had Tweets bounced for civic misminformation policy violations -
- @Ronsmit49446969
- @thearmogidaship
- @lexitollah
- @Tiboron11
- @kag2020_2024
- @kag2020_2024
- @chrisfig33ad
- @michaelp4283
- @warrenintronno
In regards to your question about a preservation letter, it is a good way to ensure that the data hasn't been purged from our systems before legal process if filed and processed. Externally the contact for submitting those is the same.
Thank you,
- Patrick
Many of the above accounts were satirical in nature, nearly all (with the exceptions of Baldwin and @RSBNetwork) were relatively low engagement, and some were suspended, most with a generic, “Thanks, Twitter” letter:
When told of the FBI flagging, @Lexitollah replied: “My thoughts initially include 1. Seems like prima facie 1A violation 2. Holy cow, me, an account with the reach of an amoeba 3. What else are they looking at?”
“I can't believe the FBI is policing jokes on Twitter. That's crazy,” said @Tiberius444.
In a letter to former Deputy General Counsel (and former top FBI lawyer) Jim Baker on Sep. 16, 2022, legal exec Stacia Cardille outlines results from her “soon to be weekly” meeting with DHS, DOJ, FBI, and the Office of the Director of National Intelligence:
Stacia Cardille
Elections Work: Wednesday, September 16
To: Jim BakerPlease see below for a summary of elections-related work I completed today, Wednesday, September 16:
(1) Government-Industry Sync. I participated in our monthly (soon to be weekly) 90-minute meeting with FBI, DOJ, DHS, ODNI, and industry peers on election threats. A few items to note:
- Foreign adversaries are amplifying themes being advanced by domestic actors to undermine the legitimacy of the election. USG specifically referenced vote by mail.
- I explicitly asked if there were any impediments with the ability of the government to share classified information or other relevant information with industry. FBI was adamant that no impediments to information sharing exist.
- I asked if USG was tracking foreign threats related to non-presidential races. Long silence. The government is not tracking foregn interference or threats related to down ballot races.
- DHS has created a chart of key dates in the election process at the state level. They are confirming information with the states this week, and should get us a summary chart of key dates next week.
- @Policy tweeted about the meeting without Legal review. As the only Twitter representative to speak, I raised my disappointment in the lack of my involvement directly to Policy Comms. It was a missed opportunity to document some key points of the meeting from our perspective.
(2) Project Heron. I met with Legal, Trust & Safety, and Public Policy stakeholders to discuss an intervention under consideration regarding labelling election results. I raised concerns about feasibility and I relayed first-hand experience with news media mistakes during a challenging primary season. I separately met with Sam and Matt to discuss worst-case scenario planning. I will meet with them again tomorrow on the project.
(3) Civic Integrity Labelling. Enforcement on our expanded Civic Integrity Policy begins tomorrow, Thursday, September 17. I provided feedback on a Scenario Planning Document that includes approving the language of the labels. I also reviewed a Moment that will link to Vote by Mail claims, consulted with outside counsel on the inclusion of one Tweet, and provided content-related advice to the Curation team.
(4) Account Security Upgrades. On Thursday, September 17, over 2,300 high-risk politician and journalist accounts will be prompted to upgrade their security settings. (See description from yesterday.) I reviewed and approved (1) the Comms plan and (2) the email to external stakeholders. I also told Public Policy of the 14 accounts that needed additional attention due to missing email addresses.
(5) Board Update. I worked closely with Mathew on additional edits to the Board narrative and the accompanying chart.
(6) Escalations. I handed the following escalations:
- Responded to DHS regarding information they provided on a Facebook operation - we found no analogous activity.
- Worked with Angela to try to get this terrible impersonation account spewing 9/11 conspiracy theories impersonating a DCCC staffer whose dad died in 9/11. Pending.
- Flagged a specific Tweet on Illinois use of modems to transmit election results in potential violation of the civic integrity policy (except they do use that tech in limited circumstances).
- Scheduling meeting with OH Secretary of State media director.
- Working with @gov to ensure we handle the verification of the OH Speaker of the House.
- Followup on Dubuque County verification request with Lisa.
- Solicited additional information from Yoel on product functionality and limitations around retweeting labeled content so we can explain to DNC.
- Lincoln Project is not pleased their video was labeled under SAMM. Bridget is driving that interaction.
- Allowlisted @DonWinslow and @Springsteen.
(7) Policy Comms. In addition to the Comms work on the security upgrades, I ...
The Twitter exec writes she explicitly asked if there were “impediments” to the sharing of classified information “with industry.” The answer? “FBI was adamant no impediments to sharing exist.”
This passage underscores the unique one-big-happy-family vibe between Twitter and the FBI. With what other firm would the FBI blithely agree to “no impediments” to classified information?
At the bottom of that letter, she lists a series of “escalations” apparently raised at the meeting, which were already “handled.”
About one, she writes: “Flagged a specific Tweet on Illinois use of modems to transmit election results in possible violation of the civic integrity policy (except they do use that tech in limited circumstances).”
Another internal letter from January, 2021 shows Twitter execs processing an FBI list of “possible violative content” tweets:
From: Unified Escalations ue@twitter.com
Subject: [JIRA] UE-49841 Report by the FBI on Possible Violative Content
Date: January 5, 2021 at 2:26 PM
To: us-elections-escalations@twitter.comYou have been added as a participant.
You can track your ticket here - https://jira.twitter.biz/servicedesk/customer/portal/500/UE-49841
DESCRIPTION:
Hi GETSupport, please see these Tweets reported by the FBI as possible violations of our policies.
https://twitter.com/comicbooknerd64/status/1346521521737650185?s=10
- comicbooknerd64
https://twitter.com/comicbooknerd64/status/1346523888352686080?s=10
- comicbooknerd64
https://twitter.com/lonireeder/status/1346498820494905344?s=10
- lonireeder
https://twitter.com/plf28807288/status/1346478916639211522?s=10
- plf28807288
https://twitter.com/meezplz/status/1346245697230823426?s=10
- meezplz
https://twitter.com/meezplz/status/1345886920979738626?s=10
- meezplz
https://twitter.com/bmwyates/status/1345379162769543169?s=10
- bmwyates
Thank you,
Stacia
Here, too, most tweets contained the same, “Get out there and vote Wednesday!” trope and had low engagement. This is what the FBI spends its time on:
In this March, 2021 email, an FBI liaison thanks a senior Twitter exec for the chance to speak to “you and the team,” then delivers a packet of “products”:
----- Forwarded message -----
From: (redacted)(IB)(FBI)(redacted)@fbi.gov>
Date: Wed, Mar 3, 2021 at 11:38 AM
Subject: DSAC Products - 3/1
To: Stacia Cardille (redacted)Hi Stacia,
It was great speaking with you and the team at Twitter last week. I wanted to flag a few products that were released today, and earlier in the week, that may be of assistance to Twitter.
Let us know if you have any questions.Kindly,
(redacted)
The executive circulates the “products,” which are really DHS bulletins stressing the need for greater collaboration between law enforcement and “private sector partners.”
From: Stacia Cardille (redacted)
Subject: Fwd: DSAC Products - 3/1
Date: March 3, 2021 at 8:46 AM
To: (redacted) Yoel Roth (redacted) Patrick Conlon (redacted)
Cc: (redacted) Jim Baker (redacted)Please see the products the FBI Office of Public Sector just provided us. Please feel free to share with your teams.
- (U) Russian Malign Influence Use of Permissive Social Media Platforms
- (U//FOUO) Heightened Domestic Violent Extremist Threat to Persist in 2021
- (U) Iranian Influence Efforts Primarily use Online Tools to Target US Audiences, Remain Easily Detectable for Now
Thanks,
Stacia
The ubiquity of the 2016 Russian interference story as stated pretext for building out the censorship machine can’t be overstated. It’s analogous to how 9/11 inspired the expansion of the security state.
UNCLASSIFIED//FOR OFFICIAL USE ONLY
U.S. DEPARTMENT OF HOMELAND SECURITY
OFFICE of INTELLIGENCE and ANALYSIS
INTELLIGENCE IN BRIEF
3 MARCH 2021 IA-48020-21FOREIGN INFLUENCE
(U) Russian Malign Influence Use of Permissive Social Media Platforms(U//FOUO) We assess that Russian malign influencers probably will increasingly use US social media platforms that offer more permissive operating environments. We base this assessment on the reduced effectiveness of Russian influence operations on established US social media platforms and current Russian proxy activity on these growing US platforms. Our assessment also is based on the assumption that Russian malign influences see operations advantages in sites with less active effort to ban false information, offensive language, and inauthentic behavior.
While the DHS in its “products” pans “permissive” social media for offering “operational advantages” to Russians, it also explains that the “Domestic Violent Extremist Threat” requires addressing “information gaps”:
- (U//FOUO) Information gaps and challenges associated with the individualized nature of radicalization could be partially mitigated with increased collaboration between law enforcement, terrorism prevention efforts, and private sector partners. We judge these partnerships would improve our ability to detect changes in DVE trends and provide early warning of potential attacks.
FBI in one case sent over so many “possible violative content” reports, Twitter personnel congratulated each other in Slack for the “monumental undertaking” of reviewing them:
(redacted) 17:29:28
Anyone need help reviewing the tweets forwarded in *FBI Report on Possible Violative Content?*
replies to threadTuesday, November 3rd 2020 17.52.17 by (redacted)
+1, we can help on SI
Tuesday, November 3rd 2020 19.39.35 by (redacted)
Thank you all so much for your help. A monumental undertaking!
There were multiple points of entry into Twitter for government-flagged reports. This letter from Agent Chan to Roth references Teleporter, a platform through which Twitter could receive reports from the FBI:
From: Yoel Roth (redacted)
Subject: Re: More Information Sharing
Date: October 16, 2020 at 1:53 PM
To: Chan, Elvis M. (SF)(FBI)(redacted)@fbi.gov
Cc: Yoel Roth (redacted) Stacia Cardille (redacted)Received.
On Fri, Oct 16, 2020 at 1:44 PM Chan, Elvis M. (SF)(FBI) (redacted)@fbi.gov> wrote:
Twitter folks,
I just got something hot off the presses today. Please be on the lookout for a Teleporter message from me with two documents to download. Thanks!
Regards,
ElvisElvis M. Chan
Supervisory Special Agent
Squad CY-1, National Security
FBI San Francisco
Work: (redacted)
Cell: (redacted)
Email: (redacted)@fbi.govThis communication contains neither recommendations nor conclusions of the FBI. It is the property of the FBI and is loaned to your agency; it and its contents or attachments are no to be distributed outside your agency.
Reports also came from different agencies. Here, an employee recommends “bouncing” content based on evidence from “DHS etc”:
(redacted) 17:18:38
Resharing this as it doesn't look like they were actioned yesterday. Given the evidence we received from DHS etc I'd lean towards BADing the URL and bouncing the videos tbh given the accusations but "relatively" low visibility
State governments also flagged content.
Twitter for instance received reports via the Partner Support Portal, an outlet created by the Center for Internet Security, a partner organization to the DHS.
“WHY WAS NO ACTION TAKEN?” Below, Twitter execs – receiving an alert from California officials, by way of “our partner support portal” – debate whether to act on a Trump tweet:
(redacted) 04:02:36 Relevant ECs 15005, 15104, 15360 re public observation From our Voting Law Compliance Handbook: : Jurisdictions count ballots through a detailed process in EC 15100 et seq. "Hey Team, the @CASOSvote reported a {https://sos.ca.gov/elections/publications-and-resourced/voting-law-compliance-handbook}|sos.ca.gov/elections/publications-and-resources/voting-law-compliance-handbook by Donald Trump. I have pasted their ask below. They are requesting feedback on the tweet and why Twitter didn't take any action. "Ballots in California are only counted by local elections officials in an open and transparent process. This tweet undermines voters confidence that their votes will be fairly and properly counted. Please see relevant election codes: Relevant ECs 15004, 15104, 15360 re public observation From our Voting Law Compliance Handbook:
https://sos.ca.gov/elections/publications-and-resourced/voting-law-compliance-handbook}|sos.ca.gov/elections/publications-and-resources/voting-law-compliance-handbook
I would like to watch how ballots get counted on election night to see how it works. Is this process open to the public? [Please contact your county elections official to ask if there are COVID-19 specific instructions related to elections observers.[ Yes. The entire process, from the opening of ballots on election night is open to the public. (Elections Code 15004, 15104) Contact your local election official for more information on observing the process on election night. Additionally, to test the accuracy of the counting machines prior to the official certification of election results, each county election official must conduct a public manual count of the ballots cast in one percent of the precincts or a two-part public manual count, the ballots counted are chosen at random by the election official. (Elections Code 15360) Relevant ECs 15101 re county's as entities who country: Jurisdictions count ballots through a detailed process in EC 15110 et seq."
replies to threadWednesday, October 14th 2020 04.16.39 by (redacted)
To clarify, is their ask for Twitter to review, or to explain why no action was taken upon a previous review?Wednesday October 14th 2020 04.18.47 by (redacted)
Why no action was taken? This report came in through our partner support portal
Here, a video was reported by the Election Integrity Project (EIP) at Stanford, apparently on the strength of information from the Center for Internet Security (CIS):
(redacted) 16:39:43
This is a very long and legal-heavy video, but essentially it claims that PA election workers opened the inner envelopes on ballots before election day and called people to correct their ballots prior to election day. According to CIS(escalated via EIP), the video misrepresents 1) PA law, 2) the PA Supreme Court decision, and 3) the affidavits shown in the video (details in the thread). Thoughts on how to handle this one? {}https://twitter.com/tracybeanz/status/1326981600689602561?s=19
- tracybeanzFriday, November 13th 2020 16.39.55 by (redacted)
Details from CIS: "The law doesn't prohibit curing, it prohibits curing prior to 7a on Election Day. The author misrepresents this by stating that the officials can't contact voters whereas they can't do so until pre-canvassing starts. The author misrepresents the PASC decision by stating voters are not forbid from curing ballots; PASC simply stated the law didn't require officials to offer an opportunity to cure. On the affidavits, these folks were contacted about having an opportunity to cure a ballot. In none of the examples do the affidavits state that the election official called them and said the voter's specific ballot needed curing. The author of the video misrepresented this and claimed otherwise."
If that’s confusing, it’s because the CIS is a DHS contractor, describes itself as “partners” with the Cyber and Internet Security Agency (CISA) at the DHS:
The EIP is one of a series of government-affiliated think tanks that mass-review content, a list that also includes the Atlantic Council’s Digital Forensics Research Laboratory, and the University of Washington’s Center for Informed Policy.
The takeaway: what most people think of as the “deep state” is really a tangled collaboration of state agencies, private contractors, and (sometimes state-funded) NGOs. The lines become so blurred as to be meaningless.