跳到内容
现在

科技 · Microsoft 365故障持续报道 · 第 2 篇,共 5 篇

Microsoft 365故障扰乱政府服务,恢复工作遇阻

微软难以维持已恢复服务器的响应能力,新加坡提醒政府回复可能延迟,澳大利亚交易受到影响。

文字 阅读需 7 分钟 · 124 个来源

原语言视频

译自英文原文。

自上次报道以来的变化

3 小时后

现在

微软难以维持已恢复服务器的响应能力,新加坡提醒政府回复可能延迟,澳大利亚交易受到影响。

等待新加坡政府机构回复的人们面临延误。在新南威尔士州,有些交易无法完成,因此客户被要求推迟前往服务中心。微软全球故障已将电子邮件故障演变成公共服务问题——即使办事处仍在开放、紧急系统仍在运行也是如此。⁠4⁠11

这场于11月12日开始的中断持续到了11月13日,波及亚太地区。微软已恢复部分服务器容量,但事实反复证明,让机器重新上线与用户重新连接后保持响应,是两回事。微软记录在案的06:12 UTC更新仍在描述服务器恢复和流量调整,而非全面恢复。⁠16⁠81

在我们于11月13日05:05 UTC发布的报道中,我们描述了服务器意外不可用、恢复措施未奏效,以及恢复时间尚未确认。微软随后发布的声明延续了这一恢复工作。更多地区的情况说明了后果:政府通信中断、行政交易无法办理,金融从业人员无法使用电子邮件。

我们正继续努力恢复前端服务器,同时实施更多流量优化措施,以提高服务稳定性并增加容量。

Microsoft 365状态 81

公共服务,存在重要例外

新加坡政府科技局GovTech表示,包括电子邮件在内的一些政府系统受到影响,因此对咨询的回复可能延迟。该机构建议,有紧急事务的人通过其他渠道联系相关部门,包括致电其呼叫中心。⁠4

该机构的保证有明确范围:紧急服务仍全面运行,可拨打995和999联系;国家数字身份服务Singpass未受影响;公务人员仍可在内部使用Microsoft Teams。这些例外情况很重要。政府电子邮件中断并不意味着应急响应、数字身份服务或所有通信渠道都停止了。⁠4

在澳大利亚,新南威尔士州服务局建议客户推迟前往服务中心,并尽可能在线办理交易。服务中心仍在开放,但部分服务中心和呼叫中心的交易无法办理。无法完成的紧急交易可申请优先回电。北领地当局也报告了服务中断,但现有说法并未列出所有受影响服务。⁠11⁠7

医疗机构方面的说法提到了行政事务上的困难,包括改用纸质处方。昆士兰卫生厅表示,目前没有已知的临床影响,并称在需要时正采用业务连续性安排来维持服务。这是对该部门服务情况的有限保证,并不能证明澳大利亚任何地方都没有出现临床后果。在维多利亚州,Triple Zero表示,人们仍可拨打电话,其服务未受到直接影响。⁠7⁠11

大学和航空业的情况也呈现出同样的区别:通信中断不等于运营停止。查尔斯·斯特特大学确认Outlook和其他微软服务出现问题,但Teams和Zoom仍可使用。员工无法收发电子邮件,而需要通过电子邮件进行身份验证的系统也难以使用,导致员工和学生的请求受阻。墨尔本机场的航班据称未受影响,维珍航空则警告称确认邮件可能延迟送达。⁠11

香港银行从业者遇到了又一个电子邮件瓶颈。《南华早报》报道了银行业消息人士的说法:员工只能离线工作,高盛和野村的银行家称工作受到干扰。一名野村消息人士称,邮件收发从周五清晨起便无法使用,他们接受采访时仍未恢复。这些是员工的说法,并非正式声明称两家银行的所有系统都已故障、交易已经停止,或已经造成可量化的经济损失。⁠10

服务器恢复运行,随后再次故障

此次事件的核心是Exchange Online,这是支撑Microsoft 365邮箱及许多关联功能的云平台。用户无法打开收件箱、无法可靠地检索邮件,也无法收发电子邮件。一些日历功能也发生故障。Teams、Copilot和SharePoint也有问题报告,但影响因功能、组织和地点而异。新加坡的Teams内部服务仍在运行,这一情况有助于纠正“所有微软应用在所有地方都发生故障”的说法。⁠1⁠4⁠11

微软的解释变得更具体,但仍不是确定的根因结论。服务器意外变得不可用,未能通过自动健康检查,导致其余基础设施不得不在多个地区承担更多工作。微软称,负载升高是可用服务器减少所致,而非客户需求激增。该公司还指出,受影响服务器上的一个核心进程不堪重负,正在降低容量。⁠6⁠15⁠11

工程师暂停部署、重定向并重新分配流量,还重启了受影响的服务器。一种替代健康检查方法使部分服务器恢复服务,微软随后开始更广泛地采用这一方法。该公司还准备了自动化系统,用于检测并重启停止响应的服务器。⁠5⁠13⁠15

但部分重启后的服务器在流量恢复后再次停止响应。其他尝试包括在特定地点屏蔽选定类型的流量、使用此前的配置重建部分服务器,以及重启彼此隔离的部分基础设施,再重新引入请求以测试稳定性。这些都是恢复尝试,并不能证明根本问题已被消除。⁠8⁠11⁠1

微软承认部分地区有所恢复,但表示采取的措施未能带来持续恢复。这是迄今报告的进展所面临的核心局限:部分用户的访问可能有所改善,但在常规请求下,基础设施仍可能再次故障。⁠4

一个负责处理服务器健康状态的组件正作为健康检查失败的潜在原因接受调查。微软尚未确定确切的触发因素。澳大利亚内政部长托尼·伯克转述了微软的判断,称此次中断并非网络事件,并将可能的解释比作一次出错的软件更新。但这种类比,以及使用较旧配置重建服务器,都不能证明此次故障是由错误更新造成的。⁠12⁠11

波及范围比损失情况更明确

74提交问题报告的国家和地区 8
>5,500澳大利亚用户报告访问问题的时间,约为09:30 AEDT 11

据德国之声报道,Downdetector在约一小时内收到来自74个国家和地区的报告。这表明事件波及地域广泛,但不代表每个地方都普遍发生故障。目前仍无法可靠统计实际受影响的个人和组织数量。⁠8

故障追踪器上的数字反映的是用户投诉,而非经审计统计得出的无服务人数。11月12日美国东部时间14:07的一次特定服务快照记录了超过97,000条Outlook报告、超过38,000条Microsoft 365报告、约4,500条Teams报告,以及近6,000条Copilot报告。这些报告可能涉及多个服务中的同一用户,因此不能相加得出受影响人数。⁠14

其他快照的时间和地理范围各不相同。德国之声报道称,格林尼治标准时间18:00刚过,美国Outlook报告接近130,000条;BBC Mundo报道的一次全球快照接近140,000条。半岛电视台记录到,格林尼治标准时间20:00前,Outlook报告约有77,000条。这些数据并非按统一标准测得的全球序列,投诉数量下降也不能证明服务已稳定恢复。同样,称中断持续了9或10小时,是报道发布时的快照,并非此次事件的最终持续时间。⁠8⁠17⁠19⁠1⁠2

此次故障也影响了与Microsoft 365连接的AI工具,但这并不能证明ChatGPT或Claude全部停摆。OpenAI报告称,微软连接器出现问题,可能导致新连接失败,或使工具无法按预期运行。OpenAI现有的说明没有指出具体哪些工具受到影响。⁠5

Anthropic记录在案的11月12日22:23 UTC更新提供了更具体的信息:邮件、日历和搜索工具在大多数请求中仍无法正常工作,Claude for Outlook也仍受影响。两家供应商受影响的连接器均未确认全面恢复时间。⁠100

与我们先前的报道相比,目前的实际运营后果更加明确,但最重要的几个问题仍未解决:最初的原因、实际受影响用户总数、全部损失,以及恢复状况何时能在各地区保持稳定。对微软而言,真正的终点不是又一次成功重启,而是人们恢复工作时,基础设施仍能持续响应。

时间线

第 4 天 · 5 篇报道

事件时间线
  1. 我们的播报 · 第 2 次更新当前位置

    Microsoft 365故障扰乱政府服务,恢复工作遇阻
  2. 微软所称客户受到实际影响的结束时间。

  3. 微软表示,所有仍受影响的客户和地区服务均已恢复,并继续监控延迟邮件。

  4. 初步报告发布;拉曼纳在 X 上致歉。

来源98

在此处引用的来源中,Claude最先发布,比本报道播出早3天。 随后还有68家媒体发布。

29家媒体未注明日期:reddit.comcloud.microsoftnacionale.comRed Hot Singaporethers.ac.jpBenzinga

持续报道 · 第 2 篇,共 5 篇

按媒体查看所有来源

来自 98 家媒体的 124 个来源· 引用最多的优先

播出原貌
  1. Microsoft is still recovering servers as its outage disrupts government services. Singapore warns that email replies may be delayed, and some Australian government transactions are unavailable. Earlier on November 13, we reported unsuccessful recovery changes and no confirmed restoration time. Can Microsoft bring its servers back and keep them responding when users reconnect?
  2. Microsoft's latest update still describes work to recover servers and improve capacity. That continues the recovery effort we described earlier. The additional consequences now stretch across government correspondence, public transactions and financial-sector work in Asia-Pacific.
  3. Problem reports arrived from 74 countries and territories within about an hour. The effects varied sharply between organizations. Singapore's government email, Australian administrative services and Hong Kong bankers' communications all encountered problems, while some other services kept working.
  4. The disruption centers on Exchange Online, the cloud system behind Microsoft 365 mailboxes and many connected functions. Users couldn't open inboxes, reliably retrieve messages, or send and receive email. Some calendar functions failed too. Teams, Copilot and SharePoint also had reported problems, with different functions affected in different places.
  5. Microsoft says it's recovering the servers that handle incoming requests and adjusting traffic to improve stability and capacity. Earlier measures produced partial recovery in some regions, but Microsoft said that improvement wasn't sustained. Some restarted servers stopped responding again when traffic returned.
  6. Microsoft says servers unexpectedly became unavailable and failed automatic health checks. That left the remaining infrastructure carrying more work across several regions. The company says the extra load came from losing usable servers. Customer demand hadn't surged. It also found an overwhelmed core process that was reducing capacity.
  7. An alternative health-check method brought some servers back, and Microsoft began applying it more widely. But returning traffic exposed continuing failures. Engineers also restarted isolated portions of infrastructure, then reintroduced requests to test whether those portions could stay responsive.
  8. Microsoft paused deployments, redirected and redistributed traffic, and restarted affected servers. It prepared automatic systems to detect and restart servers that stopped responding. Other experiments included blocking selected traffic types in particular locations and rebuilding some servers with a previous configuration.
  9. Microsoft is investigating a component that processes server health as a possible origin of the failed checks. It hasn't identified the definitive trigger. Rebuilding servers with an older configuration is one attempted remedy, and doesn't tell us which change caused the failure.
  10. Australia's Home Affairs Minister Tony Burke relayed Microsoft's belief that the disruption wasn't a cyber event. He compared the possible explanation with an update going wrong. Microsoft is still investigating the cause, so that comparison remains tentative.
  11. Subscribe to this channel as we follow Microsoft's recovery and the services still affected.
  12. Singapore's emergency services remain operational on 995 and 999, GovTech says. Singpass, the national digital-identity service, is unaffected. Government officers can still use Teams internally, even as disrupted email threatens to delay replies to the public.
  13. GovTech advises people with urgent enquiries to contact agencies through other channels, including call centres. The continuing internal Teams service shows how uneven this outage is. Microsoft had also announced recovery of a separate North American Teams issue while broader Microsoft 365 problems continued.
  14. Service NSW asked customers to postpone service-centre visits and complete transactions online where possible. Some transactions at service centres and call centres were unavailable, although the centres stayed open. Urgent transactions that couldn't be completed could receive a priority callback.
  15. Northern Territory authorities also reported service disruption. Healthcare organizations described administrative difficulties, including paper-prescription workarounds. Queensland Health said there were no known current clinical impacts and that it was using business-continuity arrangements where needed to maintain services.
  16. Victoria's Triple Zero said people could still call and that its services faced no direct impact. Charles Sturt University confirmed Outlook and other Microsoft service problems, while Teams and Zoom were functioning. Email-dependent work still encountered obstacles.
  17. At Charles Sturt, staff couldn't send and receive email, and systems requiring email authentication became difficult to use. That obstructed requests from staff and students even where workarounds existed. An Adelaide University employee also described widespread effects on staff and students.
  18. Flights at Melbourne Airport were described as unaffected. Virgin warned that confirmation emails could be delayed. Across these institutions, losing a message or an authentication step could interrupt administrative work even when the service behind it kept operating.
  19. Bank employees in Hong Kong described working offline on Friday morning. Bankers at Goldman Sachs and Nomura reported disruption. A Nomura source said staff had been unable to send or receive email since early morning, with the problem continuing when they spoke.
  20. Microsoft employees also struggled with email, alongside customers in business and government. The worldwide number of individuals affected hasn't been counted reliably. More than nine hours, and about ten hours, were descriptions of the continuing disruption at different points, rather than its final duration.
  21. One snapshot recorded more than 97,000 Outlook problem reports and more than 38,000 for Microsoft 365. Teams had about 4,500, and Copilot nearly 6,000. These submissions can overlap across services, so adding them wouldn't produce a count of affected people.
  22. The US reporting peak exceeded 130,000 in one snapshot. A separate worldwide Outlook snapshot approached 140,000. Those figures cover different geographic areas. Complaints later declined, including an Outlook snapshot of about 77,000, but people reporting fewer problems doesn't demonstrate stable recovery.
  23. Australia recorded more than 5,500 access complaints in its morning snapshot. A Brazilian snapshot counted 30,073 Outlook notifications. Another observation put the later decline at more than half. The differing reporting windows prevent a reliable, single worldwide rise-and-fall curve.
  24. Indonesian complaints describe what people encountered as well as where they encountered it. Website and app access each made up 31 percent of submitted Outlook complaints, while login problems represented 20 percent. Those listed categories don't cover every Outlook complaint.
  25. For Microsoft 365, app issues accounted for 42 percent, login 39 percent and website access 15 percent. For Teams, those shares were 82 percent, 12 percent and 6 percent. Each measures submitted complaints for that service, rather than all its users.
  26. The outage also reached AI tools that connect to Microsoft 365. OpenAI said new Microsoft connections might break and tools might fail. Anthropic described possible failures involving SharePoint and Teams chat, connections that might not complete, and Outlook add-in actions that could fail.
  27. Anthropic's later update said mail, calendar and search tools were still failing for most requests. Claude for Outlook remained affected. These connections depend on Microsoft's infrastructure. A confirmed restoration time for either company's affected connector still hasn't been announced.
  28. Microsoft has brought some capacity back, but returning traffic has repeatedly exposed instability. Its latest statement still describes server recovery and traffic adjustments. The company hasn't given a full-restoration time or explained definitively what triggered the server failures.

译文更新于

近期播出