跳到内容
现在

科技 · Microsoft 365 服务中断持续报道 · 第 1 篇,共 5 篇

微软 Outlook 恢复受挫:恢复的服务器再次在流量冲击下故障

微软的电子邮件服务在11月13日凌晨仍未稳定恢复,影响波及医院、公共机构以及相关人工智能工具。让服务器重新上线只是工作的一半。

文字 阅读需 7 分钟 · 51 个来源

原语言视频

译自英文原文。

如果人们回来使用服务器时,服务器再次发生故障,什么才算恢复?这成为微软11月12日服务中断的核心问题:恢复容量还不够,如果这些容量无法可靠承载重新流回的流量,就谈不上恢复。

11月13日凌晨,恢复工作仍未完成。RR Tech Pulse转载的一则微软更新标注时间为格林尼治标准时间03:13,称已实施的更改未能达到预期效果;在流量管理仍不平衡的情况下,微软正寻求其他增加容量的方法。这则据称的更新并未经过独立验证,无法确认是微软发布的原文。当时没有经确认的全面恢复时间。⁠38

影响远不止收件箱无法访问。工作邮件和日历失灵,多伦多一家医院网络启动了基础设施故障应对机制,公共机构警告通信可能延迟,与微软相连的人工智能工具也开始返回错误。不过,这并不是所有微软应用程序都一律停摆:微软表示,尽管其他功能仍受影响,消息传递和在线会议基本正常。⁠1⁠2⁠45

服务器恢复了,但运行仍不稳定

负责处理工作电子邮件的 Exchange Online 是此次服务中断的核心。用户无法稳定加载收件箱、发送或接收邮件,或调取日历信息。微软最初警告称,美洲地区通过多种连接方式访问邮箱时会遇到问题;随后,警告范围扩大至任何地区都可能出现 Microsoft 365 连接问题。美国、加拿大、爱尔兰、法国、日本和新加坡等国家的用户报告了服务中断。⁠3⁠14⁠7⁠6⁠16⁠1

随着事件持续,微软的解释逐渐具体。服务器未能通过自动健康检查,意外变得不可用,导致剩余基础设施承受更大压力。至关重要的是,微软表示,负载升高并非由流量增加造成。问题在于容量减少,而非已确认的客户需求激增。⁠3⁠8

这描述的是故障机制,而不是确凿的根本原因。CBC 报道称,超过八小时后,微软仍在努力查明原因。检查为何失败,以及恢复的服务器为何无法可靠处理流量,仍未得到解释。微软表示,没有迹象表明发生了网络攻击。RMIT 大学副教授 Mark Gregory 推测问题可能出在网络层,但那是他的判断,并非经确认的结论。⁠4⁠5

我们已暂停所有部署,正在将更多流量从受影响的基础设施上转移,并重启受影响的服务器,使其恢复服务。

微软,ABC 新闻引述 2

该公司还测试了负载再平衡配置和另一种健康检查方法。后者使部分受影响的服务器得以恢复,微软随后开始扩大其应用范围。据报道,其他措施还包括在部分地区限制特定类型的流量、测试先前的服务器配置,以及为停止响应的服务器准备自动检测和重启机制。这些都是恢复尝试,并非恢复已完成的证明。⁠13⁠11⁠38

11月12日美国东部时间下午6点左右,微软表示,部分受影响的服务器已恢复,看起来运行稳定。随后,微软警告称,一旦流量到达这些服务器,它们可能再次无响应。CBC 报道称,微软发现先前被认为已恢复的服务器仍然脆弱,因此限制了流向这些服务器的流量。后来的警告取代了早先的安抚说法。⁠2⁠4

微软随后表示,重启基础设施并单独恢复服务器可用性取得了“积极成果”,同时会监控稳定性,再扩大处理范围。但部分措施未能带来预期的缓解,据称在格林尼治标准时间03:13发布的更新仍称容量问题尚未解决。真正的考验已不再是服务器能否重启,而是服务能否在流量回涌时保持可用。⁠2⁠38

当电子邮件成为公共基础设施

在多伦多 Unity Health,团队无法可靠地收发电子邮件。该医疗网络宣布进入应对基础设施丢失或故障的 Code Grey 状态。院方明确表示,患者和住院者的护理不受影响,医院仍安全开放。通信应急响应不应被误认为医院关闭或患者受到伤害的证据。⁠45

对于即将到来的预约有紧急疑问的患者,被告知可通过医院总机拨打诊所或科室电话。非紧急的电子邮件事项可以等到服务恢复后再处理。这是应对影响人们联系医疗服务方式的故障的务实变通办法,尽管护理工作本身仍在继续。⁠45

VIA Rail 警告称,乘客可能收不到与行程有关的电子邮件。多伦多大学、Seneca Polytechnic 和 George Brown Polytechnic 报告电子邮件服务中断,但课程似乎仍照常进行。密歇根州技术、管理与预算部报告称,州政府电子邮件和应用程序受到影响。这些说法都不能证明所有相关服务均已停止运行。⁠7⁠8

纽约市应急管理部门警告称,市政府机构的电子邮件回复以及依赖微软的在线服务可能延迟或暂时中断。该部门建议有紧急需求的居民改用其他联系方式;非紧急市政服务请拨打311,紧急情况请拨打911。共享技术供应商发生故障,已演变为公共服务获取问题。⁠2

投诉数量不等于客户人数

影响规模显而易见,但实际受影响人数并不清楚。BBC 援引 Downdetector 数据报道称,11月12日格林尼治标准时间18:00左右(美国东部时间下午1点),Outlook 全球问题报告数峰值接近140,000。这些是用户提交的投诉,并非经审计确认的失去访问权限的个人或组织数量。⁠1

半岛电视台报道称,美国东部时间下午1点刚过,Outlook 投诉约有130,000条;下午3点前不久,约有77,000条。《卫报》报道称,午后早些时候达到峰值五小时后,投诉约为13,000条。投诉减少只表明新增报告变少,并不能证明每位受影响客户都已恢复服务。⁠13⁠5

另一组加拿大—美国数据仍无定论:CTV 的录制报道给出的数据是美国东部时间下午1:30时,Outlook 报告接近150,000条;而 Canada News Media 刊载的一则注明来源为 CTV 的可访问报道则给出接近140,000条。两者都不应被悄然替换成 BBC 的全球数据,后者的地域范围描述并不相同。⁠7⁠32

其他服务也出现了各自的阶段性数据。Euronews 在11月12日的报道中援引了超过46,000条 Microsoft 365 投诉和约3,000条 Teams 投诉;《独立报》在11月12日稍晚的报道中则分别援引了超过38,000条和约4,500条。这些是不同时间的测量,并非同步汇总,也不一定相互矛盾。将它们相加也无法得出独立客户的数量。⁠15⁠11

故障沿着互联工具蔓延

微软另行宣布,在此前北美地区 Teams 出现问题后,缓解措施已恢复服务健康状况和功能,尽管部分用户仍报告遇到困难。这项公告并不能证明 Outlook 或 Exchange Online 已恢复。微软表示,不依赖 Exchange Online 的服务应可正常运行;依赖关系比产品名称更重要。⁠14⁠15⁠4

这一差别也解释了为何其他公司的人工智能工具受到影响。OpenAI 报告称,其 Microsoft 365 连接器出现大量错误,并警告新的连接可能无法建立,工具也可能无法按预期运行。该公司可查阅的官方状态信息摘录没有点名具体工具,也没有确认服务已恢复。这并不意味着 ChatGPT 的所有功能都无法使用。⁠19

Anthropic 的官方状态更新在11月12日22:23 UTC 表示,其 Microsoft 365 连接器中的邮件、日历和搜索工具对大多数请求仍无法正常工作,Claude for Outlook 也仍受影响。Anthropic 将问题归因于微软365服务事件。连接器恢复时间尚未得到确认。⁠20

这次事件带来的一项商业后果如今有了新日期:电动卡车制造商 Workhorse Group 以微软服务中断造成影响为由,推迟了第三季度业绩电话会议。会议原定于11月12日举行,现改至美国东部时间11月16日下午4:30。这是重新安排的商业活动,并非微软恢复服务的期限。⁠29

事件仍有重大问题悬而未决:全面恢复、正常流量下能否稳定运行、明确的技术解释,以及实际受影响的人群规模。重启服务器和投诉减少都是进展指标,但两者单独来看,都无法回答依赖这项服务的人最关心的问题:他们需要时,服务能否正常运行?

时间线

第 4 天 · 5 篇报道

事件时间线
  1. 《爱尔兰时报》报道称,夜间多次恢复运行的服务器再次崩溃。此前关于稳定性的说法已被新一轮故障所取代。

  2. 微软称,在优化流量后,服务稳定性和容量有所改善,同时将相关措施扩大到受影响地区,以争取实现持续恢复——而非宣称已经恢复。

  3. 我们的播报 · 第 2 次更新较新

    Microsoft 365故障扰乱政府服务,恢复工作遇阻
  4. 微软所称客户受到实际影响的结束时间。

  5. 微软表示,所有仍受影响的客户和地区服务均已恢复,并继续监控延迟邮件。

  6. 初步报告发布;拉曼纳在 X 上致歉。

来源49

在此处引用的来源中,Windows Forum最先发布,比本报道播出早11小时。 随后还有32家媒体发布。

16家媒体未注明日期:Claudecloud.microsoftFlowboticsGreek City Timesisdown.appkdnoito.me

持续报道 · 第 1 篇,共 5 篇

按媒体查看所有来源

来自 49 家媒体的 51 个来源· 引用最多的优先

播出原貌
  1. Microsoft's email servers failed again when traffic returned.
  2. The disruption reaches workplace email, hospital communications and tools connected to Microsoft's systems. Restarts have brought some capacity back, but recovery hasn't held reliably. Can Microsoft restore service and keep it working under normal traffic?
  3. Users in the United States, Canada, Ireland, France, Japan and Singapore reported problems. Microsoft initially warned about mailbox access across the Americas, then said Microsoft 365 connections could be affected in any region.
  4. The outage began on November 12 and continued into November 13. People couldn't reliably open inboxes, send or receive email, or retrieve calendar information. Microsoft's Exchange Online service, which handles workplace email, was at the center of the disruption.
  5. Outlook problem reports peaked globally at nearly 140,000 around one p.m. Eastern. Those are complaints submitted about service problems. They can't tell us how many people or organizations actually lost access.
  6. A separate Canada-U.S. Outlook count ranges from nearly 140,000 to nearly 150,000, with that discrepancy still unresolved. Its geographic scope also differs from the global peak.
  7. Other snapshots put Outlook complaints at about 130,000 around one p.m., then about 77,000 before three. Five hours after the early-afternoon peak, the count was about 13,000. Fewer incoming complaints didn't mean everyone had recovered.
  8. Late that evening, the Canada-U.S. count still exceeded 35,000 across Microsoft services, including nearly 6,700 in Canada. That broader measure includes multiple services, so it can't be directly compared with an Outlook-only peak.
  9. Microsoft 365 complaints moved from more than 46,000 to more than 38,000 across two snapshots. Teams reports moved from around 3,000 to roughly 4,500. These separate service counts can't be added into a total of unique customers.
  10. The failures also reached OneDrive and Copilot, Microsoft's AI assistant. Users could have trouble viewing particular files or calendar details. But Microsoft said messaging and online meetings were largely functional.
  11. Microsoft said servers failed automated health checks and became unexpectedly unavailable. The remaining systems then carried more pressure. The company traced the elevated load to lost capacity. Servers returned to service could fail again when traffic reached them.
  12. More than eight hours into the investigation, Microsoft still hadn't explained the underlying cause. Why the checks failed, and why the servers couldn't reliably handle returning traffic, remained unresolved. The company said it had no indication of a cyberattack.
  13. RMIT University associate professor Mark Gregory suggested a problem in the network layer, the systems that carry data between services. That was his assessment. Microsoft hadn't confirmed it as the cause.
  14. Microsoft paused deployments and shifted traffic away from the affected infrastructure. It also redistributed traffic across its worldwide systems, restarted servers and tested a configuration change to rebalance the load.
  15. An alternative health-check method allowed a subset of servers to return. Microsoft began extending that change and preparing automated detection and restarts for servers that stopped responding. Those measures still left it managing an unstable recovery.
  16. Microsoft also limited certain traffic types in some locations and rebuilt some servers using a previous configuration. It isolated infrastructure again to restart components suspected of contributing to the problem.
  17. Around six p.m. Eastern, Microsoft said a portion of returned servers appeared stable. Later, it warned they could become unresponsive under traffic. It restricted traffic to servers previously thought restored after finding they were still vulnerable.
  18. Some attempts to open more capacity failed to deliver the expected relief. Microsoft later described positive results from restarting isolated infrastructure, but said it was monitoring stability before expanding that recovery.
  19. Subscribe to this channel to follow Microsoft's recovery and whether restored services stay reliable.
  20. Unity Health Toronto activated a Code Grey, its response to infrastructure loss or failure. Teams couldn't reliably send and receive email. The network said patient and resident care was unaffected, and its hospitals remained safe and open.
  21. Patients with urgent questions about upcoming appointments were directed to call their clinic or department through the hospital's main telephone line. Non-urgent email matters could wait until service returned.
  22. VIA Rail warned passengers they might not receive trip-related emails. The University of Toronto, Seneca Polytechnic and George Brown Polytechnic also reported email disruption. Classes appeared to continue without disruption.
  23. Michigan's Department of Technology, Management and Budget reported effects on state email and applications. Individual workers also missed meetings. A Vancouver animation worker said calendar failures caused her to miss meetings on a Disney film project with a major deadline the next day.
  24. New York City Emergency Management warned that agency email replies and Microsoft-dependent online services could be delayed or temporarily disrupted. It advised residents with urgent requests to use another contact method, with 311 for non-emergency city services and 911 for emergencies.
  25. Microsoft separately said it had restored service health and functionality after an earlier North American Teams issue. Some users still reported connection problems. Outlook and Exchange Online recovery continued, while services without an Exchange Online dependency were expected to function normally.
  26. OpenAI said new Microsoft connections could be broken and some connected tools weren't working as expected. Anthropic reported errors and slow responses across its Microsoft 365 connector, including possible failures involving mail, SharePoint, Teams chat and calendars.
  27. Anthropic initially flagged the problem at 17:52 UTC on November 12 and identified an upstream provider issue later that evening. New connections could fail to complete.
  28. At 22:23 UTC, Anthropic said mail, calendar and search tools were still failing for most requests. Claude for Outlook remained affected. Its upstream provider was restarting servers as part of the recovery effort.
  29. Microsoft's recovery was still incomplete early on November 13. In its 03:13 GMT update, the company said applied changes weren't taking as expected. It was pursuing alternative ways to increase capacity while traffic-management imbalances continued. Microsoft hadn't given a full-restoration time.
  30. Electric truck maker Workhorse Group postponed its third-quarter results event, citing business disruption from the Microsoft outage. Its earnings conference call, originally scheduled for November 12, was moved to November 16 at four-thirty p.m. Eastern.

译文更新于

近期播出