Last week’s extreme heat event in California caused an outage at a key Twitter data center in Sacramento, CNN reported Monday, leaving the social media platform vulnerable in the event of another outage at a different data center.
On Monday, September 5, the platform “experienced the loss of its Sacramento data center (SMF) due to extreme weather,” said an internal message to Twitter engineers sent on Friday, according to the article published by CNN.
September 5 marked the beginning of the most intense leg of one of the largest heat waves in Northern California history: Downtown Sacramento reached 113 degrees, followed by a high of 116 degrees that turned last Tuesday on the hottest day in the city’s history.
Data centers are warehouse-like sites with large volumes of computers, servers, and other equipment, much of which must be temperature-controlled or risk overheating.
The internal memo, sent by VP of Engineering Carrie Fernandez, says the Sacramento outage left Twitter in a “non-redundant” state.
This means that if one of the other two sites, located in Portland and Atlanta, experiences an outage, the platform may “not be able to deliver traffic to all Twitter users.”
In a statement to CNN, Twitter said “there have been no outages impacting people’s ability to access and use the social network at this time.”
Peiter Zatko, Twitter’s former chief of security, complained last month to several federal government agencies, including the Securities and Exchange Commission, about Twitter’s “insufficient data center redundancy,” CNN reported.
Zatko’s complaint noted that in the period from 2020 to 2021, Twitter had data centers in Sacramento and Atlanta, with plans beginning in 2020 to add a new data center, according to a recent Washington Post report, which obtained a copy of the complaint.
This complaint cites a February 2020 post from a blog called Data Center Dynamics, which said that Twitter leases space in a data center in Sacramento operated by a subsidiary of the .
The exact location of Twitter’s data center in Sacramento has not been publicly disclosed. Technology companies often keep that information protected for security reasons; likewise, data centers that lease space tend not to disclose their customers.
NTT has three data centers in the Natomas area of Sacramento. The newest of the three is a 180,000-square-foot wholesale data center, according to the company’s website.
But in an emailed statement Tuesday morning, NTT said it did not experience an outage last week at its Sacramento hub or any of its California sites.
Rather, to reduce the load on the grid, the company said it switched to generator power during the height of the heat wave.
“Throughout last week’s extreme heat weather conditions in California, NTT’s data centers remained operational, with 100% uptime and no interruptions or outages for any of our customers,” the statement said. of the company. “As the power situation became more urgent, NTT worked closely with SMUD and the energy commission to move load from its data center to backup generators to reduce demand on the grid, freeing up critical power resources for others”.
It’s unclear if Twitter currently leases space in various data centers in Sacramento.
Twitter has data ties to NTT America dating back to at least 2009, according to media reports and archived Twitter blog posts from the time, the latter of which said the social media giant would work with NTT to keep the “footprint” of your data center.
Twitter around 2010 announced plans to build its own dedicated data center in Salt Lake City, separating itself at least in part from NTT, but the project fell through for numerous reasons, according to a 2011 Reuters story.
Instead, Twitter moved its data center, previously in Santa Clara, to a Sacramento facility owned by a company called RagingWire, Reuters reported at the time.
NTT acquired a majority stake in RagingWire in 2014 and 100% of the company in 2018, according to a company statement. NTT on its website describes Sacramento as an optimal location for the data center due to its proximity to the Bay Area and Silicon Valley, while offering less risk of earthquake damage.
Several other large data centers are located in the California capital, including QTS, Natomas, and Prime Data Centers in McClellan Park.
Zatko was due to testify on Twitter security before the US Senate Judiciary Committee on Tuesday.
