CLOSED: [2024-07-17 Wed 17:55] :PROPERTIES: :CREATED: [2024-05-24 Fri 15:51] :ID: 2024-05-24-Microsoft-compromised :END: :LOGBOOK: - State "DONE" from "DONE" [2025-07-21 Mon 17:41] - State "DONE" from "DONE" [2024-11-12 Tue 18:27] - State "DONE" from "STARTED" [2024-07-17 Wed 17:55] :END: - Updates - 2024-11-12: "Die Lage der IT-Sicherheit in Deutschland 2024" - 2025-07-21: Microsoft outsorced DoD administration to China This is huge. This is important. In July 2023 [[https://www.microsoft.com/en-us/security/blog/2023/07/14/analysis-of-storm-0558-techniques-for-unauthorized-email-access/][Microsoft had to announce a security incident]] which impact is more or less a total desaster for Microsoft and its trustworthyness. This was only the beginning of the story of the biggest security catastrophy I have heard so far. When I tell people about that incident, they usually don't seem to believe. I blame the general media for not having properly covered the incident. This should have been in the headlines for weeks and Microsoft should be history by now. To my astonishment, this was not the case. Not even close. [[https://en.wikipedia.org/wiki/Microsoft_Azure#Security][Even Wikipedia is not as alarming]] as the incident should indicate. Why do I think that way? In short: basically *any service provided by Microsoft and at least all Windows hosts need to be considered hacked beyond repair*. This is the mother of worse case scenarios when it comes to security. Let me explain - using sources we trust. - What Happened - What Should Have Been Instead in an Alternative Time-Line - What the Attackers Are Able to Do With Microsoft - What Microsoft Would Need to Do to Fix This Properly - What Now? - Bonus: CISA Report Findings *** What Happened I started to collect reports on this IT security incident on [[id:2016-11-12-cloud][my cloud article]]. However, the huge importance of the incident calls for more attention and more context for people not deeply familiar with IT security and how IT security incidents are handled - usually. - 2023-07-14: Hackers stole a Microsoft Azure Active Directory certificate which gave them *full access to basically all Microsoft cloud services* including Outlook, Office, SharePoint, Teams, "Login with Microsoft", and so forth. ([[https://www.microsoft.com/en-us/security/blog/2023/07/14/analysis-of-storm-0558-techniques-for-unauthorized-email-access/][MS blog entry]], [[https://www.wiz.io/blog/storm-0558-compromised-microsoft-key-enables-authentication-of-countless-micr][wiz.io]], [[https://www.heise.de/news/Neue-Erkenntnisse-Microsofts-Cloud-Luecken-viel-groesser-als-angenommen-9224640.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][German heise]]) - [[https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf][It was the State Department who informed Microsoft about malicious actions observed]] on 2023-06-16. *Microsoft did not even discover the hack themselves*. - With the default logs, customers of Microsoft Azure could not even theoretically detect intruders as you would need to pay extra to get access to those log files. This is a dubious business decision that clearly prioritizes profit against security. - (Besides: [[https://m365admin.handsontek.net/multiple-services-partially-incomplete-log-data-due-to-monitoring-agent-issue/][Microsoft doesn't necessarily handle log files good enough]].) - Microsoft did *not* communicate a detailed list of services that were affected. Ever. - 2023-07-25: Security experts like [[https://social.tchncs.de/@kuketzblog/110773607467923832][Mike Kuketz]] think that most probably we need to consider all Microsoft systems as compromised (hacked) that are using their cloud authentication including *all Windows hosts* doing so *and O365*. - 2023-07-28: According to [[https://www.heise.de/news/Gestohlener-Cloud-Master-Key-Microsoft-schweigt-so-fragen-Sie-selber-9229395.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][this German source]], Microsoft is still refusing to tell what happened and which systems are affected to what extend. - 2023-08-07: [[https://en.wikipedia.org/wiki/Bruce_schneier][Bruce Schneier]], one of the most respected security professionals, wrote [[https://www.schneier.com/blog/archives/2023/08/microsoft-signing-key-stolen-by-chinese.html][in an article]]: - "Actually, two things went badly wrong here. The first is that Azure accepted an expired signing key, implying a vulnerability in whatever is supposed to check key validity. The second is that this key was supposed to remain in the the system’s Hardware Security Module—and not be in software. This implies a really serious breach of good security practice. The fact that Microsoft has not been forthcoming about the details of what happened tell me that the details are really bad." - [[https://www.heise.de/meinung/Kommentar-Microsoft-provoziert-den-Cloud-GAU-und-reagiert-dann-katastrophal-9258697.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][2023-08-18: German comment]] as one example of many: Many similar comments like that underline the growing conclusion that Microsoft disqualifies as a trustworthy partner because of the clear mismatch of the incident impact and the poor communication by Microsoft. - 2023-09-06: first public explanation by MS: [[https://msrc.microsoft.com/blog/2023/09/results-of-major-technical-investigations-for-storm-0558-key-acquisition/][Microsoft: Results of Major Technical Investigations for Storm-0558 Key Acquisition]] - Press reactions: [[https://www.heise.de/news/Gestohlener-Microsoft-Schluessel-stammte-aus-einem-Crash-Dump-9297240.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][heise (German)]], [[https://blog.fefe.de/?ts=9a075ef7][fefe (German)]] - 2023-09-29: Due to the missing containment by Microsoft, stolen data will face the day of light: 60,000 emails were stolen from 10 USA State Department accounts. ([[https://www.reuters.com/world/us/chinese-hackers-stole-60000-emails-us-state-department-microsoft-hack-senate-2023-09-27/][reuters.com]], [[https://www.heise.de/news/60-000-geklaute-Regierungsmails-Erste-Zahlen-nach-Microsofts-Cloud-Key-Debakel-9321044.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][German heise]]) - (Side note from 2023-09-29: [[https://graz.social/@publicvoit/111147782761723981][My Mastodon message about the latest news]] was posted on [[https://news.ycombinator.com/][Hacker News]] and [[https://news.ycombinator.com/item?id=37702095][its discussion reached number one on HN worldwide]].) - 2024-01: "Microsoft says state-backed Russian hackers accessed emails of senior leadership team members". ([[https://apnews.com/article/microsoft-russian-hackers-email-breach-sec-rule-84610492e56778767116a3f89f7ff658][AP]], [[https://edition.cnn.com/2024/01/19/tech/microsoft-russian-hacking-executives/index.html][CNN]]) - Therefore, the hackers were able to at least read the most important emails exchanged with the top management. This would have been a very serious incident by itself. - 2024-03-20: "Review of the Summer 2023 Microsoft Exchange Online Intrusion" by the [[https://www.cisa.gov/resources-tools/groups/cyber-safety-review-board-csrb][CISA Cyber Safety Review Board]] with dramatic findings such as "The Board concludes that *Microsoft’s security culture was inadequate*" which indicates not singular failure but many severe issues with the security culture at Microsoft. ([[https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf][Online PDF]]) - This is a quite remarkable message using harsh words from an important source. - see also [[https://www.heise.de/news/Klatsche-fuer-Microsoft-US-Behoerde-wirft-MS-Sicherheitsversagen-vor-9674431.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][heise (German)]]: "Microsoft's security failure is now officially" - *Read some spicy quotes from that report at the bottom of this article* - 2024-04-12: The attackers are still active in Microsoft's networks and intensified their action *tenfold*. Notice that this is many months after Microsoft got notified about the ongoing attacks. They are simply unable to take back control. ([[https://www.heise.de/news/Nach-Microsoft-Hack-muessen-US-Behoerden-gross-aufraeumen-9682556.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][German heise]]) - 2024-04-21: "*Microsoft is a national security threat*, says ex-White House cyber policy director" ([[https://www.theregister.com/2024/04/21/microsoft_national_security_risk/][The Register]]) - This is yet another quite remarkable message using harsh words from an important source. In such a context, this is unprecedented to my knowledge. - 2024-05-17: The German [[https://en.wikipedia.org/wiki/Federal_Office_for_Information_Security][Federal Office for Information Security (BSI)]] is forced to sue Microsoft because Microsoft did not appropriately cooperate on communicating the impact of the incident. ([[https://www.heise.de/news/BSI-verklagt-Microsoft-auf-Herausgabe-von-Informationen-zu-Security-Desaster-9721245.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][German heise]], [[https://www.heise.de/en/news/BSI-verklagt-Microsoft-auf-Herausgabe-von-Informationen-zu-Security-Desaster-9722507.html][auto-translate of the article]]) - 2024-04-26: "Microsoft CEO Satya Nadella told analysts on an earnings call Thursday that the company is increasing its focus on cybersecurity." ([[https://www.axios.com/2024/04/26/microsoft-earnings-cybersecurity-hacks][AXIOS]]) - Unfortunately, [[https://www.microsoft.com/en-us/security/blog/2022/01/21/celebrating-20-years-of-trustworthy-computing/][this happened already at least once]] and had not the desired effect when you take a look at the typical entry points of hackers and ransomware. This list will be extended with selected articles that are relevant to this incident. The list is and will never be an exhaustive one. Drop me a line below if you think there is something missing here. **** Remark: On Tainted Hosts, Services and Networks In IT security, if host/network A is compromised by hackers and if host/network B is connected to host/network A, you need to consider host/network B as compromised as well unless you can prove otherwise. This prove can be accomplished via uncompromised technical barriers between hosts/networks and deep analysis of, e.g., logging files. As long as you can't prove that, you need to consider the whole set of hosts/networks as compromised. [[https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf][The CISA report mentions:]] #+BEGIN_QUOTE Microsoft found no evidence of a breach in the perimeter of the signing system #+END_QUOTE This is a phrase which is often used in incidents where there is simply not enough information available. Not finding any evidence of a breach can also mean that they simply don't have the logs any more that are able to help them to analyse the situation. So we do have an organization, that failed in protecting their crown jewels and was hacked for at least three years without noticing. The same organization needs to find out what exactly happened during those three years where the attackers were in their systems. Since Microsoft did not provide details on that, we must assume the worst case which is that *any Microsoft service* connected to the hacked hosts/services *is potentially compromised*. And this is basically *everything* related to Microsoft because even Microsoft reported that the hackers were able to access the email system. I need to emphasize that if you fully trust the statements by Microsoft, the problem got analyzed and all negative impact got fixed in the aftermath. The thing is, I don't trust them any more. This is backed by Microsoft's reluctant communication in each step of the incident. A quote from the CISA report: #+BEGIN_QUOTE Victims found it difficult to investigate these intrusions after initial detection because Microsoft could not, or in some cases did not, provide victim organizations with holistic visibility into all necessary data. Although Microsoft activated enhanced logging for identified victims who did not have the appropriate license, Microsoft could not give historical logs to customers unless they already had the premium licenses at the time of the intrusion. Thus, customers could capture data from the time that Microsoft enabled additional logging capabilities but were unable to view past intrusion activity. #+END_QUOTE In other words: for unclear reasons, Microsoft did not assist victims during their own analysis of the situation. The hackers did get deep down into the crown jewels of Microsoft and there are many indicators that point in the direction that Microsoft is not in the situation to completely recover control. *** What Should Have Been Instead in an Alternative Time-Line This would have been one of the best case scenarios. Given enough resources, APT hackers are able to infiltrate any network. So the success of the initial hack is not the most important thing of this story. In an ideal world, Microsoft would have had intrusion detection mechanisms in place where they would have detected the attackers within a very short of time. Furthermore, I would assume that the crown jewels of Microsoft are much better protected in the first place. A series of borders and gateways should be put around those precious assets. Not a single weak point should make it possible to get to the target, only a series of mistakes or bugs. And attacking those different gateways should ring a few alarms before intruders reach their goals. After detecting the attackers within minutes or hours, Microsoft should have contained the attack as well as start an in-depth analysis on what happened. After closing the original weak points, all hacked systems would have been set up from a trustworthy installation source from scratch to ensure that any backdoor, malware, remaining artifacts are gone. Appropriate information should have been published to ensure a proper level of trust to its partners and clients. Mitigation measures would have improved the overall (network) security even more. *** What the Attackers Are Able to Do With Microsoft Since the situation was very different from the best case scenario described in the previous section and the group of attackers basically got everything they could have hoped for, Microsoft should have handled this IT security worst case with extra care. Unfortunately, Microsoft decided otherwise. Hackers that pull off such an attack are usually called [[https://en.wikipedia.org/wiki/Advanced_persistent_threat][advanced persistent threat (APT)]]. Such a group is not only in the position to get into a network. They are able to further attack systems with [[https://en.wikipedia.org/wiki/Advanced_persistent_threat#Life_cycle][state-of-the-art actions of advanced attacks]] within the target network to achieve, e.g., "lateral movements". This also includes setting up [[https://en.wikipedia.org/wiki/Backdoor_(computing)][backdoors]] so that they are able to re-enter the compromised network even when the original security holes get fixed. Good hackers do need seconds or minutes for that since they are supported by easy-to-use tools that automate most of the required steps. In this case, the attackers did not have seconds or minutes. They had several years. With access to basically the whole Microsoft services and infrastructure, the attackers were able to do all sorts of things. They could possibly implant their own security certificates for intercepting network traffic related to Microsoft internal traffic as well as all customer traffic. Hackers could potentially also modify the source code of Microsoft products, introducing backdoors or malicious functionality. Typical Microsoft software services and products consist of Millions of lines of code. This source code contains very complex dependency relations between different modules, accomplished by seperate programming teams and departments. Advanced hackers do have more than enough possibilities to inject arbitrary malicious code that gets deployed to services and clients. Since the vast majority of source code by Microsoft is [[https://en.wikipedia.org/wiki/Closed_source][proprietary software]], no independent expert is able to check or detect any of that [[https://en.wikipedia.org/wiki/XZ_Utils_backdoor][as this is possible with free and open software]]. And Microsoft itself did not recognize the attack for multiple years themselves. This list of possible malicious actions is far from exhaustive: stock market manipulation, insider trading, CEO fraud or direct money transfers, you name it. I think that you already got the idea: *Microsoft and all of its hosts, networks, services and products is considered hacked* by all means unless they can prove the opposite - which they didn't and most probably they can't, given the long years of the attack. *** What Microsoft Would Need to Do to Fix This Properly Most probably, the usual "any compromised system needs to be thrown away and re-created from scratch" will not be applied here. Microsoft can't affort this effort. Microsoft can't affort the impact in terms of downtimes and similar. Since the attack did go unnoticed for so many years, it's most likely impossible to analyse the original incident any more. Log files are not kept that long and even if they still have them, the nature of the attack suggests that the attackers might as well have cleaned up their traces in the log files as long as Microsoft is not [[https://en.wikipedia.org/wiki/Write_once_read_many][using read-only media]] for storing log files. Therefore, for making sure that outsiders don't have (full) control over the systems run by Microsoft, *Microsoft would need to shut down everything and start from scratch*. Alternatively, Microsoft could decide to closely separate and monitor all of their systems in order to determine malicious behavior that happens now. Of course, those systems need to be 100 percent trustworthy so that no manipulation could happen in those monitoring systems. I can think of many issues with that - even with the "from scratch"-approach. For example, how do you determine which installation sources are trustworthy any more? What happens with all the functionality that got implemented since then? How to deal with the risk that you still overlooked a backdoor? Even ignoring those thoughts for now, starting from scratch is not that easy. This does not necessarily happen with a big bang. This could be split up in separate projects, running over multiple months or years. The important thing here would be that engineers make sure that hacked systems can't access the newly set up systems. [[https://en.wikipedia.org/wiki/Airgapped][Air gapping]] is the method of choice here. Of course, this would mean some impact on the running business. Networks, hosts, services, customers would be divided into at least the tainted part of Microsoft and the freshly started part of Microsoft. The amount of negative press about that would have an enourmous impact not only on the stock price. However, deducting from the publicly available information, it would be the required thing to do in order to get back control. This most probably won't happen. As a consequence, *you can't trust any data from Microsoft services any more.* Following that rationale, [[https://en.wikipedia.org/wiki/List_of_Microsoft_software][you can not trust many things our digital world relies on]]. Just a few examples: - Azure and anything that is connected to it - Outlook - Exchange - Microsoft Office (cloud or on-premise) - Windows desktop systems - any company networks/hosts that are run by Microsoft products - GitHub - Visual Studio *** What Now? From my point of view, if you are serious about IT security and trust, Microsoft is done. I don't see any other alternative from a security point of view. So far, the world does seem to have ignored the impact of this incident. The castatrophic security situations don't end with the total loss of Azure control from above: - Long before the Azure incident, Microsoft produced a very long list of fails. [[https://en.wikipedia.org/wiki/Criticism_of_Microsoft][Wikipedia has its own page for "Criticism of Microsoft"]]. Do your own research with your favourite consumer rights organization and read why they recommend using alternatives. - Microsoft got famous for the [[https://en.wikipedia.org/wiki/Criticism_of_Microsoft#Privacy_issues][constant violation of privacy]], e.g., by [[https://www.heise.de/en/news/Microsoft-lays-hands-on-login-data-Beware-of-the-new-Outlook-9608798.html][stealing your passwords on the desktop]] or [[https://4sysops.com/archives/is-microsofts-outlook-app-for-ios-and-android-insecure/][using their mobile apps]] or [[https://www.neowin.net/news/windows-11-is-now-automatically-enabling-onedrive-folder-backup-without-asking-permission/][all of your digital content without even asking]]. [[https://www.schneier.com/blog/archives/2024/04/surveillance-by-the-new-microsoft-outlook-app.html][The New Microsoft Outlook app is a particular interesting spyware]]. - 2024-04-09: "Microsoft has resolved a security lapse that exposed internal company files and credentials to the open internet." "The Azure storage server housed code, scripts and configuration files containing passwords, keys and credentials used by the Microsoft employees for accessing other internal databases and systems." "But the storage server itself was not protected with a password and could be accessed by anyone on the internet." ([[https://techcrunch.com/2024/04/09/microsoft-employees-exposed-internal-passwords-security-lapse/?guccounter=1][techcrunch]], [[https://www.heise.de/news/Microsoft-Code-und-Passwoerter-standen-frei-im-Netz-9681295.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][heise German]]) - It took Microsoft four weeks(!) to get it off the public network. - 2024-04-11: Microsoft lost 1,138,558 files which also included internal passwords to important internal services as well as source code files ([[https://www.heise.de/news/Details-zum-Microsoft-Leak-Ueber-eine-Million-interne-Dateien-waren-oeffentlich-9681391.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][heise German]]). - 2024-07-01: Microsoft admitted that the digital Russian break-in earlier this year was even more severe: "Kremlin spies make off with source code, executive emails, and sensitive US government data. Reports last week revealed that the issue was even larger than initially believed and additional customers' data has been stolen." ([[https://www.theregister.com/2024/07/01/infosec_in_brief/][TheRegister]]) - It's really hard not to mix up all the major hacks of Microsoft central infrastructure that happened just within a few months. I'm puzzled by the fact that Microsoft still tries to sell Azure services with the argument of being secure and trustworthy. According to the impression of all the IT security sources related to that incident, this can't be even more far from the truth. For example, Microsoft tried to [[https://www.heise.de/news/Gesundheitswesen-ueberlastet-Microsoft-will-mit-Cloud-und-KI-helfen-9684821.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag][sell Azure as a trustworthy platform for the German health services]]. *** Bonus: CISA Report Findings (With yet Another Severe Incident) :PROPERTIES: :END: Emphasizing by me: #+BEGIN_QUOTE The Board concludes that Microsoft’s security culture was inadequate. The Board reaches this conclusion based on: 1. the cascade of Microsoft’s avoidable errors that allowed this intrusion to succeed; 2. Microsoft’s failure to detect the compromise of its cryptographic crown jewels on its own, relying instead on a customer to reach out to identify anomalies the customer had observed; 3. the Board’s assessment of security practices at other CSPs, which maintained security controls that Microsoft did not; 4. Microsoft’s failure to detect a compromise of an employee’s laptop from a recently acquired company prior to allowing it to connect to Microsoft’s corporate network in 2021; 5. Microsoft’s decision not to correct, in a timely manner, its inaccurate public statements about this incident, including a corporate statement that Microsoft believed it had determined the likely root cause of the intrusion when in fact, it still has not; even though Microsoft acknowledged to the Board in November 2023 that its September 6, 2023 blog post about the root cause was inaccurate, it did not update that post until March 12, 2024, as the Board was concluding its review and only after the Board’s repeated questioning about Microsoft’s plans to issue a correction; 6. the Board’s observation of a separate incident, disclosed by Microsoft in January 2024, the investigation of which was not in the purview of the Board’s review, which revealed a compromise that allowed a different nation-state actor to access highly-sensitive Microsoft corporate email accounts, source code repositories, and internal systems; and 7. how Microsoft’s ubiquitous and critical products, which underpin essential services that support national security, the foundations of our economy, and public health and safety, require the company to demonstrate the highest standards of security, accountability, and transparency. If Microsoft had not paused manual rotation of keys; if it had completed the migration of its MSA environment to rotate keys automatically; if it had put in place a technical or other control to generate alerts for aging keys, the 2016 MSA key would not have been valid in 2023. Further, if Microsoft had not made the error that allowed consumer keys to authenticate to enterprise customer data (or, alternatively, if it had detected and addressed this flaw), the scope of the intrusion would have been far narrower and would not have impacted the State Department, Commerce Department, or any other enterprise customers. If Microsoft had deployed alerting or prevention to detect forged tokens that do not conform to Microsoft’s own token generation algorithms, this incident likely could also have been stopped or detected by Microsoft all on its own. Even after all this, if Microsoft had other security controls in place for its digital identity system—as the Board finds other CSPs had in place at the time—this intrusion vector could have been blocked or detected. Finally, once State Department alerted Microsoft to the intrusion, Microsoft did not have the logs or other forensic data to determine how or when Storm-0558 had stolen the key. The decision to completely stop manual rotation of signing keys in 2021 after a large cloud outage, along with failing to prioritize the development of an automated key rotation solution, are troubling examples of decision-making processes within the company that did not prioritize security risk management at a level commensurate with the threat and with Microsoft technology’s vital importance to more than one billion of its customers worldwide. Taken together with the inadequate controls in the authentication system to detect and mitigate key theft after multiple attempts by the threat actor to compromise identity and authentication systems, including in Operation Aurora in 2009 and RSA SecureID in 2011—something that all other major CSPs have worked to address in their systems’ architectures—the Board finds that Microsoft had not sufficiently prioritized rearchitecting its legacy infrastructure to address the current threat landscape. In addition, the failure to detect the compromise of an employee’s laptop in an acquired company in 2021, prior to allowing it to connect to Microsoft’s corporate network, raises questions about the robustness of Microsoft’s M&A compromise assessment program. The Board is also concerned with Microsoft’s public communications after the incident. In its September 6, 2023 blog post entitled “Results of Major Technical Investigations for Storm-0558 Key Acquisition,” Microsoft explained that Storm-0558 likely stole the 2016 MSA key in the “crash dump” scenario described above. However, soon after publishing that blog, Microsoft determined it did not have any evidence showing that the crash dump contained the 2016 MSA key. This led Microsoft to assess that the crash dump theory was no longer any more probable than other theories as the mechanism by which the actor had acquired the key, which Microsoft chose to leave uncorrected for more than six months after publishing its September 6 blog. The Board is troubled that Microsoft neglected to publicly correct this known error for many months. Customers (private sector and government) relied on these public representations in Microsoft’s blogs. The loss of a signing key is a serious problem, but the loss of a signing key through unknown means is far more significant because it means that the victim company does not know how its systems were infiltrated and whether the relevant vulnerabilities have been closed off. Left with the mistaken impression that Microsoft has conclusively identified the root cause of this incident, Microsoft’s customers did not have essential facts needed to make their own risk assessments about the security of Microsoft cloud environments in the wake of this intrusion. Microsoft told the Board early in this review that it believed that the errors in the blog were “not material.” The Board disagrees. After several written follow up questions from the Board regarding the blog, Microsoft informed the Board on March 5, 2024, that it would be updating the blog in the “near future.” One week following this communication, and more than six months after its publication of the September 6 blog, Microsoft corrected its mistaken assertions through an addendum to the blog’s existing webpage. The Board also takes note of *a separate incident that Microsoft disclosed in January 2024.* This disclosure revealed a compromise that allowed *a different nation-state actor*, which Microsoft calls Midnight Blizzard and the U.S. government has previously attributed to the Russian Foreign Intelligence Service (SVR), to *access highly-sensitive Microsoft corporate email accounts*. Nearly two months later, Microsoft published a new blog post stating that Midnight Blizzard had *also gained unauthorized access to some of Microsoft’s source code repositories and internal systems*. While this second intrusion was outside of the scope of the Board’s current review, the Board is troubled that this new incident occurred months after the Exchange Online compromise covered in this review. *This additional intrusion highlights the Board’s concern that* *Microsoft has not yet implemented the necessary governance or prioritization of security to address the apparent security weaknesses and control failures within its environment* and to prevent similar incidents in the future. Individually, any one of the failings described above might be understandable. Taken together, they point to a failure of Microsoft’s organizational controls and governance, and of its corporate culture around security. Microsoft’s products and services are ubiquitous. It is one of the most important technology companies in the world, if not the most important. This position brings with it utmost and global responsibilities. It requires a security-focused corporate culture of accountability, which starts with the CEO, to ensure that financial or other go-to-market factors do not undermine cybersecurity and the protection of Microsoft’s customers. Unfortunately, throughout this review, the Board identified a series of operational and strategic decisions that collectively point to *a corporate culture in Microsoft that deprioritized both enterprise security investments and rigorous risk management*. These decisions resulted in significant costs and harm for Microsoft customers around the world. The Board is convinced that Microsoft should address its security culture. In 2002, Microsoft’s founder and then-CEO, Bill Gates, wrote an email to the entire Microsoft workforce on the importance of prioritizing security in product development. He wrote (quote) So now, when we face a choice between adding features and resolving security issues, we need to choose security. Our products should emphasize security right out of the box, and we must constantly refine and improve that security as threats evolve. A good example of this is the changes we made in Outlook to avoid e-mail-borne viruses. If we discover a risk that a feature could compromise someone's privacy, that problem gets solved first. If there is any way we can better protect important data and minimize downtime, we should focus on this. These principles should apply at every stage of the development cycle of every kind of software we create, from operating systems and desktop applications to global Web services. (end of quote) The Board concludes that *Microsoft has drifted away from this ethos and needs to restore it immediately as a top corporate priority*. The Board is aware of Microsoft’s recent changes to its security leadership and the “Secure Future Initiative” that it announced in November 2023. 171 The Board believes that these and other security-related efforts should be overseen directly and closely by Microsoft’s CEO and its Board of Directors, and that all senior leaders should be held accountable for implementing all necessary changes with utmost urgency. #+END_QUOTE #+BEGIN_QUOTE RECOMMENDATION 6: CSPs should engineer their digital identity and credential systems in such a way that substantially reduces the risk of complete system-level compromise. This should be an overriding, top-priority, design goal in the engineering process and be informed by a rigorous threat model developed by the CSP in response to its understanding of the threat landscape. The Board spoke with all major U.S.-based CSPs to gain an understanding of their existing practices and develop a set of recommended baseline best practices. While the specific practices implemented may vary for different use cases and situations, the Board believes technical mechanisms exist today across the industry that can, if broadly implemented, significantly reduce the likelihood of complete system-level compromise. Each of these practices is implemented by at least one major CSP, demonstrating their technical feasibility. Some of these practices, while compatible with accepted industry standards, would also benefit from additional standards development, which is discussed in another recommendation. #+END_QUOTE So yes, if you've read carefully, you found out that Microsoft did not learn from the first incident and got severely hacked yet another time by a different group of hackers. Again, the hackers got access to the innermost Microsoft systems. That incident alone would have been very alarming as well if it did not happen in the aftermath of the other hack. It's a very bad company culture, as stated in the report. Would you like to trust Microsoft with all of your important data? [[https://en.wikipedia.org/wiki/Linux][GNU/Linux]] and other [[https://en.wikipedia.org/wiki/Free_and_open-source_software][FOSS]] software products are more than ready for you to use. I'm using GNU/Linux systems in my personal life since the late 90s. The company I'm working for is almost exclusively run by FOSS products, reducing our dependency and lock-in effects to a minimum. Therefore, it's just a matter of will to do the switch. *** Update 2024-11-12 "Die Lage der IT-Sicherheit in Deutschland 2024" :PROPERTIES: :END: The German [[https://en.wikipedia.org/wiki/Federal_Office_for_Information_Security][Federal Office for Information Security]] (BSI) published a [[https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Publikationen/Lageberichte/Lagebericht2024.html?nn=129410][report on the state of IT security in Germany 2024]] on 2024-11-12. It features a whole page covering the Microsoft incident. Here is a direct quote: #+BEGIN_QUOTE Laut Microsoft wurde der Zugriff, den die Angreifer sich verschafft haben, nur bei OWA und Outlook.com ausgenutzt, mutmaßlich hätten jedoch weitere Services betroffen sein können. Weiterhin war der Betroffenenkreis laut Microsoft auf circa 25 tatsächlich betroffene Organisationen beschränkt. Das tatsächliche Schadensausmaß war damit deutlich geringer als das Schadenspotenzial. #+END_QUOTE Here's a translation from that passage: #+BEGIN_QUOTE According to Microsoft, the access that the attackers gained was only exploited in OWA and Outlook.com, but other services could presumably have been affected. Furthermore, according to Microsoft, the group of those affected was limited to around 25 organizations that were actually affected. The actual extent of the damage was therefore significantly less than the potential damage. #+END_QUOTE According to this, the BSI also confirms that the attackers had the key to "other services". However, they trust the public annoucements of Microsoft that the damage was contained. Furthermore, the report does not mention that the attackers have been in the systems for several years. I don't know how you would interpret that but to me, the BSI report is not convincing enough to me, given the severity of the incident. *** Update 2025-07-21 Microsoft outsorced DoD administration to China :PROPERTIES: :END: Well, it sometimes doesn't even require sophisticated hacking in order to get access to secret networks and classified data of a country. According to [[https://www.propublica.org/article/microsoft-digital-escorts-pentagon-defense-department-china-hackers][this report by ProPublica]] ([[https://www.heise.de/news/Microsoft-Techniker-aus-China-betreuten-Cloud-des-US-Verteidigungsministeriums-10494416.html][German heise]]), Microsoft outsourced the administration of classified data (“Impact Level” 4 and 5 and includes materials that directly support military operations) to cheap Chinese workers "who are paid barely more than minimum wage for the work". In my opinion, it's not this particular incident that should worry us. It's the company culture and mangement decisions that made that incident possible. Check carefully who you trust with your data.