Hardcore Software by Steven Sinofsky (Audio Edition)

The last months of a product development cycle the scale and length of Office10 are moments of calm punctuated by moments of terror. The calm comes from the lack of code changes as thousands of people test the product while hoping not to find anything requiring code changes. The terror comes from issues that arise when you have little time to change them because we’re still dealing with the physical world. Something that seems so trivial like a product name requires a month of more of work to get right, then rolled out around the world. Office10 did not have a name just three months before we would finish. Office10 was also one of the first product releases for Microsoft where when the product made it into customer’s hands there also needed to be live and working web sites that extended the product, for over one hundred million people on day one. Other moments of terror come just after the product ships and those first reviews—it is not just if they are good or bad, but will there be a recall class issue, be it a design choice or product defect. Such an issue could tarnish the entire product cycle. What if our marquee feature—the one in all the printed brochures and print ads—ends up part of a scandal rapidly spreading to Windows XP?

This post brings you inside the last months of a project that was the first Office product to ever finish on time, and perhaps one of the first Microsoft products ever to do so. As it would turn out 2001 might be viewed as peak execution for the middle-age of the PC journey. The Windows release under development which would become Windows XP would also ship on time, a first for Windows. It makes sense that the major products would learn to ship on time just as the world was moving on to the internet.

Back to 067: MYR-CDG: Product Meets Sales

In early 2000, branding and naming the next release of Windows was all gummed up and that slowed down Office, which needed a name in order to release on time in March. Whistler, the code name for Windows, was scheduled to ship about six months after Office10. An August ship date for Windows was achievable, and no one wanted to mess that up. The name, perhaps to the surprise of many readers today, was a long pole in the release as it touched code in many places and could impact localization, manufacturing, printing, and packaging. We needed a name. How hard could that be?

Windows trying to settle on some variant of “experience” because the main thrust of the release was to finally bring the Windows 95 experience of peripherals and consumer software to the enterprise-focused NT code base of Windows 2000. In addition, we collectively concluded the year names were not working (hence Windows Me) because of the difficulty of keeping ship schedules and customer confusion over what products worked with each other, both issues were entirely predictable and frequently discussed at the time. The branding team was thinking of something like Windows EXP or Windows XPR, and finally Windows XP. My concern in an endless email chain was not this product but the next one. Would the next name have a roman numeral, go back to adding version numbers, a superscript, or add descriptor Edition branding? At one point, someone mocked up Windows XP ² which made me wonder if I was being made fun of, especially in the Spanish release.

The corporate teams settled on and cleared the name Windows XP, and what immediately followed was the question as to whether Office10 should be named be Office XP. The corporate branding team and sales teams were in favor. The Windows team, however, did not intend for other products to add an XP suffix. This seemed to be another .NET in the making, which was still nowhere close to being resolved. The branding powers demanded evidence in the product that supported co-branding.

We held many meetings during development about aligning the releases of Whistler and Office10, but the products were on wildly different schedules. From our perspective, Whistler was about finishing Windows 2000 without much new for Office. Besides, Office was going to run perfectly well on the new corporate desktop of Windows 2000 anyway. Windows 2000 had been years late, so it was not unreasonable early in the process to doubt the next Windows schedule. Whistler was making a ton of progress and we were self-hosted, so any worst thoughts were not going to happen. This was a first for Windows! Still, there was no significant release synergy. Office ran on Windows XP, and realistically not even as well as it ran on Windows 2000 PCs with the same amount of memory.

At one of these meetings the topic of naming them the same became heated, if not awkward. BillG wanted to know what was interesting in Office10 and what was unique when it ran on Whistler. Baked into this assault was the belief that Office was not exciting on its own and more importantly that Office failed at exploiting the latest Windows release. This was the view that came from the perspective that innovation from Microsoft emerged first from Windows.

My answers were not satisfying as I restated the series of meetings, lack of APIs, and our system requirements. I chose to highlight the work we did to align with Windows Server for SharePoint, totally irrelevant to the conversation in this context. BillG expressed frustration at “another release of Office that relies on Windows to make it exciting,” which honestly didn’t make much sense and bordered on insulting. By and large we were not looking to the Windows desktop to innovate for Office. In reality, many innovations in user-interface flowed from Office to Windows for years already. That wasn’t interesting to Bill in this moment.

Windows really wanted the XP name to be unique to Windows. Office really didn’t want to look like it required or was matched to Windows XP. We learned that lesson when Office 97 arrived and many consumers at retail were wondering where the matching Windows 97 was or even if Office 97 ran on Windows 95. After more time back and forth, Office marketing and corporate branding agreed to, no surprise, a compromise of a name—Office XP with a “Version 2002” prominently displayed on the box. The apps were called Word 2002, Excel 2002, etc., not Word XP or Excel XP because branding didn’t want to overuse the XP moniker. Windows did not have such a version on the box or in the software. Also, the boxes didn’t look at all similar. This meant customers calling product support or using the web site would need to search for “2002” and not XP, unless they used Google which got it right. I can’t make this stuff up, but just wait until the next release of Office.

We had a name. We were on track for March 2 and we had a date for launch (and boxes for Germany). We signed off on 3/2/01 just like we planned. It was magical to do that. We beat the Excel record from more than a dozen years ago and hit our planned ship date. I know this sounds ridiculous—product development schedules are supposed to work, but it simply did not happen with software projects. We were so happy.

For launch, marketing planned one main US event, which we expected to be covered globally. BillG headlined the event at New York City’s Manhattan Center Ballroom aimed at getting broad media coverage, while around the country and world there were hundreds of local events targeting enterprise IT professionals. There was no effort to get people lined up at stores or any sort of midnight madness, though local offices around the world did some of that and we did have BillG greet the early buyers in New York. A fixed launch date is a great forcing function (there’s that phrase again) to get everything ready for press tours, reviewer workshops, and enterprise product information.

For enterprise customers the main features of the release going back to the original product plan were collaboration and integration with the just-shipped Exchange 2000. SharePoint Portal Server 2002 (there’s that naming again) released a few weeks after Office XP which included SharePoint Team Services and also served as a collaboration platform. Our collaboration story was as complex as predicted in our vision statement, a result of targeting the same scenarios to two different back-end infrastructure products.

For system administrators, we enabled new scenarios, such as installing Office XP from a website and ever-more controls and customizations for deployment. There was a lot there and it was all new for enterprise customers. Industry analysts were having a great time digging into the idea of Office shipping servers.

Through some incredible outreach efforts by marketing and the field, more than 500,000 enterprise customers used the product in pre-release. The internet made it easy to distribute the product, and because of Watson we not only knew (anonymously) that people were using the product, but we were fixing bugs based on their usage and knew the release was high quality. We really knew that, not just hoped. Shipping software had radically changed using Watson. The availability of data forever changed what we worried about when shipping.

These are the Office XP consumer data sheets. These were used at retail outlets, tradeshows, and by sales people making calls on potential resellers.

The traditional tech press and mainstream media continued to struggle with explaining, or making broadly interesting, heavy enterprise features like collaboration. I was still smarting from the reviews of Office 2000 and was determined for us to get credit for the personal productivity features. We built a great set of capabilities that worked with or without servers, and as was common at the time with or without a broadband internet connection, though by that point most every customer was connected.

Office XP introduced several novel features we believed gained notice including the first features that worked seamlessly using an internet connection from within the apps. Building such features was a fascinating lesson in team transformation. With many stories from the early days of the internet about traditional print-based offerings unable to transition to the web, we set out to do something new and innovative with the thousands of pages of training materials and the vast library of content we shipped on CDROM. Jeff Olund (JeffO) began his career in “user education”, creating the written materials that accompany a product. He became a leader in building reference and training materials for Office and then led the worldwide localization team based in Dublin, Ireland. With JeffO’s leadership, content went from a cost center to an asset for the business. The Dublin team went from taking almost a year to localize Office into a dozen languages to localizing Office into 100 different languages in just a month or so.

The next step in content to help customers was to use the internet to provide endlessly growing features such as adding images or clip art to a document, templates, how-to, and more. As an example, before Google pioneered image search, it was luck that enabled an author to find suitable clip art. Especially in large corporations, most of the clip art we shipped with the product was eliminated to save disk space. In Office XP we introduced an Insert ClipArt feature that used an ever-expanding and easily searchable collection of internet images much larger than anything on a CD. The idea of tens of thousands of images available for free was quite cool for business users of PowerPoint tired of angry person and idea lightbulb over head person.

In hindsight, a product having a website seems obviously trivial. But at the time, we were deeply concerned about adding a website to a product used (and liked) by hundreds of millions of people. The web was still flakey (and slow) and while novel, sites routinely did not work. We did so much work in quality, the thought of having toolbar buttons or menu commands leading to strange errors from unavailable sites was horrifying. Websites needed to be new and fresh all the time, and always work.

The help system was no longer limited to what was written and shipped with the product but could also search a large and growing library of how-to articles on the web, including an under-the-radar hit called Crabby Office Lady who offered tips with an irreverent tone (“advice with attitude”), authored by a professional writer on the team Annik Stahl (AnnikS). Annik’s column was wildly popular with hundreds of millions of views. She was interviewed by local newspaper tech columns (often writing similar content) and even had television appearances (as herself, not the character). The character and personality Annik created even drew some inquiries from the VP of Human Resources over concerns of stereotyping. Annik’s goal was to take aim at the perceptions of the Office product, not the Office customer and she had full control over the project she initiated. We saw this approach used to great effect with a series of books such as Word Annoyances, that had become one of the most popular books on using Office products. So popular was the Crabby column on the site that Annik also went on to write a book and was part of several behind the scenes stories.

We created a new online services team under JeffO’s leadership. AndrewK moved from OPU to lead program management to help focus on content working for JeffO, who reported to me. Seasoned managers Mike Kelly (MikeKell) and Randall Boseman (RandallB) had to create new processes and tools to go from zero to a one hundred million web visitors seemingly overnight. The Content team developed schedules for creating, localizing, and releasing content at regular intervals, something we had only previously done on multi-year release boundaries.

Between online content, Crabby Office Lady, and the introduction of a new sidepane user-interface, plus the ongoing ridicule aimed at the Assistant we also made some big changes to Clippy. The plan for the product cycle was to provide even more options and administrator controls to reduce Clippy’s visibility, including making sure that by default Clippy no longer appeared, though it could be summoned if desired. Importantly, the message for Office XP was that it was so easy to use that “ . . . Clippy is no longer necessary, or useful.” That might have been spin, perhaps.

Lisa Gurry (LisaGu) in marketing thought up a clever idea to make the most of this change, and to embrace the opportunity to be self-deprecating in the process. She planned a formal retirement for Clippy. Instead of tacking the feature change on to the press tour in April, she planned a web-based celebration. Gilbert Gottfried, comedian and voice of dozens of animated characters, was enlisted in a set of internet Flash videos (these were all the rage then) of Clippy trying to insert himself into the daily lives of people. On retirement day, Lisa issued a press release and dressed another member of the team in a giant Clippy suit as we strolled Union Square in San Francisco, complete with a cable-car ride.

While this was the least corporate approach to PR, it set a high-water mark for intentionally cool viral marketing. To this day, the “retirement” still gets a laugh.

Laughs, however, were not part of the marquee feature, known as Smart Tags. Smart Tags were a set of buttons shared across Office applications appearing when and where the user needed them (such as when a user made an error in an Excel formula, when Word automatically corrected a user’s action, or when a user pasted some data), providing options to adjust the chosen action or fix an error. Smart Tags appeared when pasting text into Word, giving a choice of whether to match formatting or not, a common need with the rise of taking text from web pages. Smart Tags made it easy to undo (or never do again) when Office autocorrected something incorrectly, like convert a row of dashes to a typeset line.

While there were many different features across the product, Smart Tags provided a single and consistent unified interface. From a marketing perspective, Smart Tags felt like toolbars in their ability to be a visual symbol of innovation in the release. The screen shot was used all over the place.

It was Office at its best. We thought.

In press tours and reviews, Smart Tags proved a novel interface approach and were demonstrably innovative. Surprisingly, it took us several years to achieve—this idea was in the works going back to the original Office interoperability work in 1994, but before we could execute new ideas we needed to clean up the old implementations of menus and toolbars, which we did.

Smart Tags were also extensible. A corporation could recognize an order number or a shipping code and make it easy to link directly to those websites or systems. Browsers, especially Internet Explorer, and tools like free email programs or websites, were not yet universally inserting automatic links by recognizing the text of http://, phone numbers, dates, or other common strings. The most well-known was the use of 1Z as a prefix for a United Parcel Service tracking code (tracking was a new thing for consumers), which with a Smart Tag enabled that text to act like a link to UPS, if Internet Explorer or Outlook used Smart Tags extensibility. Phone numbers, addresses, dates, and more were all candidates for Smart Tag actions. We computed the time saved by the use of Smart Tags to be an insane number of millions of hours—pure marketing.

We thought this such a clever idea and an opportunity to show off synergy with Internet Explorer that we created a Smart Tag add-in for IE that recognized many common potential links. It seemed useful and synergistic and was included in the Internet Explorer 6 beta test. The IE team thought this was clever too—this was before browsers could be customized with add-ins and web pages were mostly static HTML (except for the occasional blinking or marquee text).

It wasn’t long before JimAll (the leader of Windows) called to tell me that people were freaking out. By this he meant that the press tour for IE was not going well. Concerns about Smart Tags were expressed within the context of IE. While there was some hyperbole, the concerns boiled down to feedback expressed stridently by Walt Mossberg, who wrote in the Wall Street Journal, “In effect, Microsoft will be able, through the browser, to re-edit anybody’s site, without the owner’s knowledge or permission, in a way that tempts users to leave and go to a Microsoft-chosen site—whether or not that site offers better information.” The terminology isn’t aging well, but the idea was simply that links would appear on HTML pages that were not authored by the site owner. This in effect could be viewed as editing the site, where editing means adding new links to a page authored by a third party.

This notion of “re-edit” went beyond what we considered or even thought of the feature. There’s some irony in that today browsers and mail clients (and many products) do this for a whole host of special strings and even Apple computers provide reference lookup to Apple sources. At the time, it was routinely pointed out that the climate of 2001 was filled with concerns that Microsoft might leverage one part of Microsoft to benefit another poorly performing or just poor part of Microsoft. Smart Tags in IE surfaced those concerns. Perhaps exactly the right feature, at exactly the wrong time, from exactly the wrong company. Conspiracy theories felt real to many perfectly rational people in the industry.

To make that point, Mossberg concluded, “Microsoft’s Internet Explorer Smart Tags are something new and dangerous. They mean that the company that controls the Web browser is using that power to alter others’ Web sites to its own advantage. Microsoft has a perfect right to sell services. But by using its dominant software to do so, it will be tilting the playing field and threatening editorial integrity.”

We did not think through the potential abuse of the feature, though even in the worst case it was not precisely what was suggested by some. Sites were not being rewritten and links were not getting replaced. Neither was it benign nor free from potential exploitation. We did not consider how someone with bad intentions might develop a Smart Tag add-in that could at best be annoying and at worst recognize text and offer a link to a nefarious web site. As a result, IE removed the feature, which spared us the challenges. It appeared as capitulation, and the press was not shy about expressing that. As a broader point, the company, particularly under the new leadership in our corporate legal group, preferred and encouraged capitulation, especially for anything that might catch the attention of its constituents, the regulators.

Office XP and the press materials still had this marquee feature, using the name Smart Tag everywhere for consistency. Was the name tarnished, though? There wasn’t anything we could do other than downplay the name and issue Q&A and FAQ documents to the marketing teams around the world. This was an unfortunate side-effect of not having thought through the implications in the browser.

The Smart Tag incident took place at what was perceived to be the height of Microsoft’s power and potentially negative influence on the industry. The fact that we relatively quickly backed down has been viewed as a milestone moment. The incident was even portrayed in a widely read profile of Walt in WIRED, May 2004, titled “Kingmaker.” It was an example of “dozens of companies that have redesigned products in response to Walt’s unsparing criticism.” Walt was right. I viewed this incident as the system working—the reviewers acting as a check on poor product choices.

Finally, we always reminded people we made the product more stable each release. With Office XP, however, this was provable. We equipped the press and field with statistics and descriptions of the Watson curves and buckets so they could understand our new approach to improving product quality. Watson was industry-leading and pioneering. Every time I represented this work I am sure I beamed with pride, even though showing it off meant using a tool we wrote for demos that forced the product to crash (again).

The reviews of Office XP generally reflected the positives of the product for both consumers and businesses. The industry was going through a post-dotcom consolidation and, while tech was still enormously popular, the slow but certain decline of print journalism (and the dotcom bubble) affected reviews. First, the budgets in time, people, and hardware for the kind of testing we used to see at Byte and PC Magazine were reduced or gone. This led to a decline in reviews based on usage of new parts of Office, especially the hard-to-test parts like SharePoint. Second, the web itself favored shorter and faster takes on what was new. People wanted reviews at the time of release. While we gave ample lead-time and supported embargoes, other pressing demands made it difficult to spend time in research mode prior to a product release. With the PC deep in middle age, in-depth reviews of technology products were replaced by more instant analysis of meta topics, such as the impact of the product to the company or a take on whether the product should be deployed or not. (Hint, they always said to stick with a current version unless that was old.)

In that context, Office XP provided a glimpse of what was in store for the business. Every review evaluated the product in the context of the value of the upgrade. The assumption for Office was that everyone with a PC owned it and, while this was far from true, it might have been true among those reading tech press with their up-to-date PCs at work, but not all PCs by any measure. Enterprise customers already owned Office XP and were by all accounts only upgrading with a PC refresh cycle. Reviews almost always included benign comments about the product from IT managers along with a quote about not being clear if there was enough value to upgrade at the time, but certainly the future looked good.

The decline of awards like Editor’s Choice and in-depth 10-page reviews with benchmarks was, honestly, sad. It was as though the world cared less about what we did. The internet was the new focus for consumers and businesses. It was no longer the PC and new desktop software.

The enterprise field fully engaged on an XP desktop motion, meaning Windows XP plus Office XP, even with Windows months from finishing (as an aside, Windows XP was the first Windows release to have a plan and schedule that remained steady and the product finished on plan late summer). Office XP and Windows XP were better together for the enterprise. That reduced the sales surface area for the field from two products to one, so to speak, which was greatly preferred. In that sense, the timing of the two products together reinforced a core belief of BillG’s, that new applications provided excitement for a new OS, which created demand for new applications—a virtuous cycle.

Our glitzy launch event in New York at the end of May 2001 featured BillG leading an enterprise-focused event but with enough consumer demonstrations to capture headlines. The focus on productivity was illustrated by a broad though relatively unsupported statement, that by “making Office just 10 percent better we can save hundreds of millions of man-hours.”

We were joined by some special guests that day. Chief among them was the founder and CEO of a relatively new Seattle-based company called Amazon.com. Jeff Bezos joined BillG on stage to demonstrate searching for Office XP on Amazon and buying it with Amazon’s new One-Click ordering feature. A box even arrived on stage for Jeff and BillG to open.

But my mind was already deep in navigating what should come next for the product and the team.

On to 069: Mega-Scale, Mega-Complexity (Chapter X)

Microsoft was often viewed as a Borg-like structure though we’ve already seen when it came to product development it was decidedly of two cultures. The huge and growing global sales force so dominated by SteveB (now CEO) represented a third, one completely defined by the unique combination of massive global scale and local empowerment and respect for culture. What happens when a Redmond-based software engineer-turned product group executive meets this culture head-on? What about when that happens in front of his new boss who previously ran the field and created the process we’re about to see unfold?

This is a story of culture. It is also a story of growing and maturing as an exec. Most of all it is a story of how a product leader is not really complete until they’ve lived and breathed the efforts of the sales people that connect with customers and bring in the money. Below is an early experience. A few years later I would pack up and live in China to experience firsthand on a daily basis what it was like to sell every day.

Back to 066. Killing a Killer Feature (In Outlook, Again)

In the classic book Flow: The Psychology of Optimal Experience, Mihaly Csikszentmihalyi demonstrates that a genuinely satisfying experience takes place in a state of consciousness called flow. During flow, people typically experience deep enjoyment, creativity, and a total involvement with life. While the book uses many examples, the concept of getting in a groove resonated with me. Flow is how programmers spend hours writing code and how our best customers spend hours creating satisfying documents with Office.

Reading Flow helped me, but my job was changing. . .a lot. The demands to be present in the newly added building 34 executive suite overwhelmed me at times. The pressure to spend so much time ruminating with other executives ran counter to what I came to believe and what I experienced as flow. It didn’t used to be like that. I know because I used to see the execs I worked for most every day, casually.

Flow was how I felt writing a vision or the interactions I had as I walked the halls. This was especially important at Microsoft where everyone had a door. The criticism of private offices was that people get shut out of collaboration and spontaneous discussions (in favor of concentration and quiet focused work, apparently). I never believed that to be the case, because we had a culture of door open/door closed that implied a willingness to chat spontaneously or a need to focus. Since my first days at work, the richness of hallway and drive-by discussions were most useful and memorable.

The idea of connecting spontaneously and learning what was on the mind of a member of the team was something I viewed as my job—so much more than sitting in a conference room with other executives or reading slides presented by managers but prepared by people not even in the room. Making eye contact, not in a hurry, discussing someone’s work and seeing if I could put it either into context or simply offer perspective on something going on in a bigger picture—that was flow for me. Everyone was always wrestling with something—bugs, recruiting, implementation choices, and more—and while I didn’t want to walk around fixing things, I could support, learn, and connect. Most importantly, this was a way to connect with less senior members of the team 1:1. Even today, I can recount some of my favorite moments and best connections coming from people I was able to learn from by walking the halls, a phrase made famous at least in the tech world by Intel’s Andy Grove.

Doing so was not the norm for many execs, especially outside Apps or the product groups. The vast majority kept to a Microsoft exec workflow of a reserved conference room across from their office and a steady stream of meetings all day. Even 1:1s were often held in bigger conference rooms.

The 8 a.m. meetings, the meetings to prep for meetings, the emails written by staff to communicate the meetings, the rescheduling of the same, were all starting to pile up. I felt I was becoming detached from the team—I was losing flow. I was uncertain of whether I was a poor fit for the role or if my role was defined wrong. I struggled trying to understand what changed in the landscape that required such a change in how to manage. We weren’t facing new problems or new concerns—there were no new competitors or competing technologies. We had ideas to grow to new areas and were executing, and the enterprise sales process was yielding absurdly good numbers. What was I missing? Or, how could I convince people that managing 2,500 people was an investment in time that might be different from how a huge subsidiary GM or marketing VP ran things? Others in similar positions felt differently. They seemed to gravitate towards this new Microsoft culture. Perhaps I was wrong or underestimated what was needed to operate at scale and that was the driving force behind this perceived shift.

The field’s annual Mid-Year Reviews (MYR), the process JeffR pioneered, were the most coveted meetings for field teams to attend. If there was a culture-defining element to the growing and incredibly successful enterprise field it was MYR, and it could not be more different than anything we did in the Redmond BGs (Business Groups, as we were called to imply we were separate businesses rather than mere product teams). BG was the new term for the teams in Redmond and everyone loved to say it as a differentiator to the field. Everything seemed to come down to the BG versus the field. It was as if Microsoft was a constant struggle between a BG brain and a field brain.

To say MYR was a production would be like calling the Olympics “some people playing sports.” Over the course of three weeks or more in January, the company’s collective leadership focused entirely on syncing up at various locations around the world and presenting mid–fiscal year results. Each year the process became more elaborate, took longer, involved more people, and became more all-consuming with ever-increasing import. Every country, though some were presented in regions, marched through a standardized slide deck slicing and dicing sales budgets and forecasts, finding holes, ferreting opportunities, and sharing best practices. These meetings were tense, terrifying, and a test of thinking on your feet under the harshest of business conditions.

To be extremely clear, this seemed an absolutely fantastic way to run the sales force. To me, this was their version of vision meetings, bug triage, and daily builds. The field was scaling up to tens of thousands of people pushing with a relentless focus. The military precision and standardization were obviously brilliant. My own love of flow was exactly what made me appreciate this process. The scale was mind-blowing.

But I didn’t need to see it to appreciate and understand it, any more than I thought the field needed to attend spec reviews or triage meetings. Product groups saw each other every day and maintained a tightly integrated effort in continuous adjustment starting and ending with Redmond. The product teams integrated across many levels almost constantly.

The field was a geographically, linguistically, and culturally dispersed organization of massive scale tuned to be with customers operating on its own, focused on goals decided at the start of a twelve month execution process. The field was the definition of an organization tuned to thinking and acting locally. Each fiscal year, the global team regrouped and realigned for the next. MYR was the halfway point, checking in on all the assumptions and results that defined the fiscal year and fixing what needed fixing.

The connection between corporate and field was essential. There are many ways to connect—MYR was not the only time to see the field. I visited subsidiaries, connected with PSS routinely, (still) attended executive briefings, and we engaged marketing and planning essentially full time with counterparts in headquarters and subsidiaries, and directly with customers in forums like the OAC. The field headquarters staff was where the planning and coordination of the subsidiaries and regions coordinated full-time. Issues and resolutions bubbled up from the field to HQ for solutions in real-time with the BGs. Every year since becoming a vice president in 1998, I made a point of attending some MYR reviews, just not all of them. In the latter part of the 2000s, attendance became tightly controlled and required an invitation and resulting assigned seat. Failing to show (thus wasting a slot) was deeply frowned upon and clearly visible by a name placard and empty chair. For many, attendance became mandatory, and despite the protests, attendance was viewed as career progression.

What was an MYR like?

Meetings were usually held offsite, often in an airport hotel meeting room, and started at 8 a.m. Wherever we were, the room was always filled to the brim and oxygen was soon replaced by stale air, usually extremely cold air with that breeze of hotel air conditioning. Seating was assigned around a big rectangle of worktables. SteveB sat at the head of the table and fanned out from him on either side by rank were the field executives, followed by leaders from sales segments such as enterprise, education, small business, OEM, retail, and others, along with an army of finance people from corporate and the subsidiary and the region. Across the room from SteveB sat the subsidiary leadership guiding the meeting, with the GM sitting in the center and then a mirror of subordinates on their side, also by rank with the exception of a chief of staff or business manager right next to the GM as owner of the MYR Deck of slides. There was a gallery of the HQ business divisions on one side—usually the head of marketing and the senior product leader (such as, me). Outside sat (or loitered) everyone else who could not fit into the room but was on hand ready to provide backup materials should they be urgently summoned.

There was a slide deck, but it was not made of slides so much, but rather posters printed in 8- to 10- point type on 17 x 22-inch paper, spiral bound. The pages were fully packed with rows, columns, bullets, numbered callouts (“as you can see at callout number 7. . .”), and a lot of heat maps (mini spreadsheets with cells colored coded red, yellow, and green to indicate severity of issues). Every year brought innovations to the decks from better data integration on the back end to the use of color. A most prized possession was the databook, which was essentially a cheat sheet for the whole process that was available to the field executive leadership and SteveB—it contained global and regional summary tables, making it possible for comparisons to be discussed mid-meeting.

Each page was easily eight slides worth of data. Deck preparation began in the fall and was practically a full-time job for a team of people—in the big subsidiaries, dozens were deep in preparation. The slides were almost all data—data pulled from sales systems and reconciled across, and up and down, all the subs. Once all the numbers were right, the teams went through the decks and compared the actual numbers with the start-of-year budgets. Preparation was knowing exactly why for every variance, positive or negative. Outlying data was identified with a callout and a concise explanation was at the ready. Why did you sell fewer education PCs this year? Why are enterprise renewals behind budget? Why have you not met your hiring goals? Why did you rent such an expensive venue?

Every. Single. Number.

Intellectually, I knew for me to sit in this meeting watching what was going on was akin to a salesperson pondering the endless discussion about a single code change in a bug triage meeting. Emotionally, it was another story.

SteveB and the sales leaders were relentless and disciplined about accountability. An inability to explain a number credibly, or worse missing that a number was off, was bad, really bad. Every GM knew that this was not a meeting, but a performance review. Meetings that went poorly were legendary. Everyone was aware of the story of that time a meeting went so badly that when it was finally time for a 15-minute lunch break at 3 p.m., upon returning the seat the GM held before the break was vacant and his name placard was gone. Did that happen? Not quite. But the legend was all that mattered.

When a page turned in the book and the speaker changed, on to OEM sales for example, everyone briefly looked at the page (with a ruler and magnifying glass) and noted the circled numbers and carefully placed arrows. Then suddenly, usually, SteveB pointed to some specific number amid a giant grid of numbers and asked, for example, why business PC laptops were so low last quarter. Missing an outlying number was a crime, and it was shocking to watch what happened as a result. Panic ensued. An important skill among sales leaders is the kind of inherited capability the cheetah has to pick out the camouflaged prey from among all the small creatures in the veldt. Picking those outliers are the small prey of sales executives. SteveB was Jedi master of finding out what was important or determining the opportunity in a grid of otherwise indecipherable numbers.

This, unlike a standard meeting, was an ultramarathon. For a major subsidiary, like the United Kingdom, France, Germany, or Japan, or an important growing one like Korea or Russia, a MYR meeting could go on for 10 or even 12 hours, even though they were only scheduled for half workdays. And when it finally seemed over, the next team showed up. Jetlagged, suited up, hungry, and tired, they could have been waiting half the day in the lobby, but then they were on. We sat there the entire time, and while we broke for a meal, we usually brought food back to the table (which made the whole room smell of food and tired humans). We finished most days past midnight in the early hours of the morning and were right back hours later for 8 a.m. or earlier if the next meeting was already looking like it would go long.

An MYR was to me like what being on a space flight must be—long stretches of nothing and then a moment of terror when some instrument buzzer went off, warning lights strobed, and then the everything turned a red hue of emergency situation. That was these meetings. The buzzer was a subsidiary saying something about Office, the product or business, that was either negative or non-supportive. Then all heads turned and looked right at me. If my head was down in a laptop (or flat on the table asleep, “Bueller . . . Bueller”) then everything about that moment of terror was amplified. The cardinal sin: “I’m sorry. I did not hear what you said.” Leaving to go to the bathroom or to get a drink at the wrong time escalated into a “bad MYR”.

While most of the slides were financial and focused on clear, sales-oriented accountabilities, for many years there was a qualitative section called Feedback to Corp. Usually offered by a GM, the feedback was candid, addressing areas that needed improvement that only BGs in Redmond could fix. Topics ranged from a single product bug that cost a big EA contract, to marketing materials not appropriate to the sales motion, to resource allocation guidelines that were not right, to broad product feedback usually connected to a big and challenging customer. The field was run diplomatically, so ambushes weren’t supposed to happen.

The months-long preparation provided time to socialize the feedback so the BG could have a properly prepared response to recite at that point. These meetings were, in many ways, theater—for both sides. As an example, at the end of a meeting SteveB and the execs offered feedback, which was usually a series of messages about how to approach the second half of the year. By the third major subsidiary, messages were not only honed, but the subsidiary staffs shared them with the downstream subs and everyone knew what to expect and was on the lookout for even the slightest variations (variation was the real feedback). Dedicated staff judiciously tracked every question and potential follow-up, entering information into a tracking system that later pinged the relevant parties with email until an issue was resolved. Over the years this tracking system was automated and targets of feedback received reminder mails for months until an issue was marked as resolved.

MYR 2001 for major EU subs was at the Hilton at Charles de Gaulle Airport. By day three, I had my fill of French hotel food and was longing for the McDonalds that I knew was in Terminal 1. I knew I needed to find time to hop on the shuttle to get there. The shuttle circled from the hotel to the gates and back at regular intervals, taunting me each time I imagined it passed. It was cold and dreary outside. I had no idea what time it was as we’d been in meetings for days on end. Modafinil wasn’t in use yet (Provigil a non-narcotic prescription drug originally used to treat narcolepsy now a favored pharmaceutical for traveling executives). We’d already gone through the United Kingdom and France. Germany was up. Germany and a dozen other countries loitered anxiously in the lobby for two days, in the hopes of picking up some G2 about questions and drill-downs.

The NASDAQ crashed one year earlier, and many countries were in rough shape. MYRs reinforced the adage that “when the United States sneezes the world catches cold.” Still, PBS was generally doing well, though there were concerns that growth was slowing. In reality, we were going through a transition from choppy retail licensing to greater and smoother revenue through Enterprise Agreements. We were riding the year of deploying an enterprise-standard desktop of Windows 2000 with Office 2000, and Exchange 2000 just released. There was plenty of exciting enterprise software and strong initiatives.

The other thing that became clear was that changing the whole world market takes time. While the United States moved to enterprise licenses, the rest of the world was lagging. At the extreme, another decade would pass before Japan was fully enterprise licensed. Germany was in the middle, especially because its market dominated by giant manufacturing companies moved deliberately, meaning slowly. The implications were twofold. First, the enterprise section of the review was tense—HQ was putting pressure on sales to get big customers signed to EAs. Second, most of the Office teams in subs still viewed a retail launch as the big driver, something HQ made second fiddle for Office 2000 and which for Office10 was even less of a priority. While Office planned a single big worldwide launch event, the primary effort was on 100 local North American enterprise events. The strategy was to replicate this in each major market at appropriate scale.

When it finally came time for the PBS section of the meeting, I tried to perk up, still thinking how much I wanted to get on the shuttle. There was some back and forth conversation happening over enterprise selling and the difficulties of customers in the manufacturing and autos sector. There was some friction over the ever-troubling state governments who wanted lower prices on Office and perennially threated to switch to Linux and open source if they didn’t get it.

Everything seemed normal. Then it was time for Feedback to Corp and, with it, an ambush.

Germany decided to make a big deal out of an Office10 retail launch. Our BG guidance was to spend marketing dollars on the enterprise launch (with the large number of local events to drive IT awareness of SharePoint and business value). We planned a global launch event in New York, with BillG in attendance. While the event proposed by Germany was different, if the field made its numbers (all of them) then it was empowered to do what it believed was right. For all the top-down planning, execution was remarkably empowered with the field.

The rub was the German team was in contract negotiations with an expensive venue for the retail event and needed a solid commitment for software availability. They literally needed an immediate certain date for when volume quantities of German Office10 could be available through all German retailers. This was January 2001 and we were scheduled to RTM three months later. They knew we could commit—how could we not? They didn’t like our scheduled date, though, and wanted an earlier one. Our actual boxes in stores date was May 31, which gave us buffer and was the plan.

We scheduled down to the day for everything, but we were not operating the team as though this was a date–driven release at the level of manufacturing. I was 90 percent certain of RTM on March 2, 2001 (3/2/01). Our worst case, we thought no later than March 16 which was only two extra weeks. We could slip because of a bad bug that took time to track down, or if there was a production hiccup (a bad master CD, a virus in an image, a delay in collateral for the box). Beyond that there was time needed to complete localization. We were not yet to the point where English and German versions were done on the same day, but we compressed that difference down to a week or so, especially for German (we always did German early because the words are really long and that was a good test for the product visuals, plus GrantG leading testing was a native German speaker). The final step was manufacturing the CDs, which happened in Japan, and then getting those assembled into boxes and distributed. Any time lost on this step could be made up by spending money on air freight and expedited shipping if really needed.

Such logistics steps and concerns raced through my head in the fraction of a second it took the entire room to turn and look at me like I was the last human in Invasion of the Body Snatchers. Then I committed a fatal error. I did not say, “Yes, book the venue for that date and we’ll make it.” Instead I said, “I can’t be certain of that [earlier] date. We have a schedule, but we are not operating to guarantee retail boxes in stores in Germany on that date. Given what I know and how we’re working, I would do this at the end of May as planned.” I suggested we might finish early (that was the single worst thing I could have done on top of my first worst thing). We already communicated our end of May date in the leadup/socialization phase MYR, but they wanted an earlier date for local reasons and chose to make that case in the MYR forum—it was that important to them. I said a lot of words I should not have bothered saying.

My words were met with silence. I felt like that moment in outer space disaster movies when the alarm is off and all that remains is the silence of space and the red emergency lights. I’m certain I was perspiring.

Because of their venue choice, none of what I said was “acceptable,” I was later told. They wanted that venue, on that date, and were forcing the issue—managing me in front of JeffR (whom they knew super well) and SteveB (whom they knew sided with them). The back-and-forth continued and I kept digging myself a deeper hole—at some point my spirit rose above the room and I watched the mess unfold.

Most of all, what was on my mind (that I could not share) was that we were still in the middle of figuring out a product name. The corporate branding team was working to come up with a name that spanned Office10 and the next Windows release. Without a name we could not finish the software and we could not make boxes, marketing materials, or even announce the product to retail partners. The lack of a name was becoming a sore spot with the engineering and localization teams. We were three months out and literally did not have a product name. Shocking for Microsoft, I know.

As of MYR, there was no name and corporate was suggesting we slip the Office product so they could have time to come up with one. They were working on a Windows timeline, with scheduled end of summer ship date. It bugged me that they were nonchalantly considering a slip of Office so Windows could have more time to come up with a name. As if I needed a reminder that Windows was the lead dog. I was being a good citizen and not throwing that team under the bus at MYR. In the process, I stepped in front of or under the bus myself. Heck, I threw myself off the bridge at the bus.

Neither of us could see the other’s point of view. They were asking me for a favor: Make it happen. Germany could not see that I was trying to avoid an embarrassing event with no software and had no idea that I was working to avoid embarrassing corporate branding or shifting blame to them. I kept thinking that the team back in Redmond was doing so well, the last thing I should do as the manager was come back from a three-week hiatus to create a fire drill over the retail launch in Germany when we prioritized the worldwide enterprise launch for two years, especially when they were in a holding pattern on naming.

Germany and HQ called a truce and accepted the feedback, and marketing agreed to work out the details. The body language directed at me from the SteveB side of the room was painfully clear.

Following the conclusion of the meeting, JeffR pulled me out into the lobby of the hotel where I stood longingly watching the shuttle bus to Terminal 1 roll by every 15 minutes of this ad hoc 1:1. He was obviously livid though maintained a calm delivery—we had only been working together for a few months. Without asking any questions he said, “That cannot happen.” He said we were obliged to deliver. He told me I handled it wrong. This went on for what felt like an eternity—look, there’s the shuttle again. The entire time I assumed my name placard was being removed.

In that moment, I welcomed being put out of my misery.

When we returned to the main room, my name card was still there. I sent some mail back to the United States to say we needed to make Germany happen. Grant believed we could do it, but he could not guarantee that date. We told them to go ahead. Everything was fine.

Except me.

The biggest cultural difference internally was that MYR was the field’s shining moment. It was a chance for them to overtly manage the HQ organizations, so long as they had the facts straight. The field took direction from HQ on good days and put up with shoddy work and poorly executed marketing and product on bad ones. For 50 weeks of the year, they absorbed everything. Each country wanted its chance to be in charge and their meeting was when they had the upper hand over the BG.

Until this MYR in 2001, I hadn’t grokked the power dynamic. The idea of being managed in front of a room was so opposite the old Apps product group culture that for the longest time I simply thought I was being bullied. JeffR taking the side of Germany in the meeting was him taking part in the ritual of the culture when he sided against me in front of everyone. It is always weird to experience your new manager not supporting you in a big public meeting. I wondered if this dynamic was different or crazy or a bit of both?

One thing was clear: My job was different. Same software. Different job. I had some learning to do.

On to 068. The XP eXPerience

Enterprise software customers learn about roadmaps and plans long before the development team has robust execution plans. That’s part of the business. No matter how much these discussions with customers and partners are caveated, a failure to deliver is a big deal. Customers, partners, and salespeople all have a vested interest in those slides (promises!) coming to life when we said they would. When a project is spread across two separate and large development teams, failing to deliver or “cutting a feature” as we called it as if to minimize accountability, the cross-team dynamic is brutal. When BillG gets involved, the temperature rises even more.

Thank you again subscribers. For many readers, you will be receiving a renewal notice about a week before your subscription is automatically renewed. I am incredibly appreciative of your support this past year and look forward to another year of amazing stories: Office user interface, Windows 7, Windows 8, internet services, Surface, and more. So much to dive into. Again, thank you for your ongoing support. It means the world.

Back to 065. SharePoint: Office Builds Our Own Server

JeffR, my new manager and executive vice president, send me a note “LIS: Wow, the story just gets worse and worse. . .Let’s kill it.” He pushed to resolve this festering issue and suggested a meeting with BillG. This would be the second time we cut this feature, having shipped Office 2000 without it.

It was December 2000. In a hastily arranged lunchtime meeting (a BillG must was to always have hot lunch at noon), we sat in the Board Room where we normally met with him. It was a deeply technical meeting, the kind he liked. There were senior people from Exchange, Outlook, and the company experts on data storage that have been meeting as part of Bill’s project to improve storage. The meeting was to decide whether or not to abandon the work on LIS, or the Local Information Store, feature of Outlook and Exchange. The meeting was very tense. The broad view was that this feature was absolutely key to competing with IBM/Lotus Notes. The Exchange Platinum release was late with an uncertain ship date. Office10 was less than three months from being complete, and the code was essentially frozen.

After a tense hour that ran over, BillG said he would think about it, leaving everyone in a state of limbo. It was unusual for him to not reach a conclusion immediately. That meant he was either going to take an unpopular stance, perhaps try to keep the feature alive, or he knew it was a pretty grim situation and was going to avoid confrontation in the room. Counter to what many might have believed, Bill did not like to get involved in binary decisions that leave little room for optionality or win-win outcomes.

An hour later he sent mail saying it was a difficult decision but “I am for killing it because less than 5% of customers would end up using it.” That conclusion was based on the state of the feature as presented.

What followed was the rollout of a brutal cut, which included drafting mails for another month, communication with the field sales group, and removing dependencies on the code.

This recounting made the process seem straight forward, but in fact it was one of the most challenging last-minute product changes I experienced. Going through this surfaced many of the most difficult product strategy and cross-company execution challenges Microsoft exhibited.

During the meeting with Bill, I was not worried or concerned. I was frustrated (and it almost certainly showed). There was no need to have this meeting. The feature basically cut itself. There were no options other than delaying products for perhaps a year to complete this feature. This was the “physics of shipping software”.

Even though the classic by Frederick P. Brooks, Jr., Mythical Man-Month, was over 25 years old and found on every Microsoft engineer’s bookshelf, when decisions made it to the executive ranks, especially when they involved cross-group and strategic initiatives, the lessons from the book were forgotten.

BillG would ask if we could allocate more resources from either team to speed things up. We all knew that there was neither the expertise nor the available resources to do that. Plus, we all knew what the mythical man-month said about that, “The bearing of a child takes nine months, no matter how many women are assigned.”

BillG would offer suggestions on how we could scale back on some of the features or scenarios in order to require less work. We had been doing that for months. The feature had been scaled back so far that even if we shipped what we discussed at the meeting, it would not have moved the needle on competing with Notes. Quite the contrary, it would have disappointed.

BillG would say it would be acceptable to simply add some more time to the schedule, perhaps three months. Physics of shipping this amount of software meant that three months was hardly any time at all. Given all the work to test, stabilize, and for servers getting feedback from customers, adding three months was about the same as adding a few weeks of engineering at best. Besides, even without this feature Exchange Platinum would likely not ship until the end of 2000. Office10 was all but complete and “re-opening the patient” as we said was not an option. It was just physics.

There was something about how the company was working that we permitted ourselves to go through this sort of exercise when there really weren’t any options. Generously it was a process to come to grips with a difficult failure to deliver. Alternatively, it was a way of spending energy exploring options that didn’t really exist anyway. That’s why I was frustrated. It was decision theater. The bottom line was we didn’t deliver and there was no rescue mission.

There are times when these meetings can yield a new outcome. Projects have constraints and if BillG (or anyone in a position to do so) could relax some constraints—the broadly-defined trinity of ship date, features, or quality—then there is a new path to take. Usually, teams that forgo examining and changing these assumptions on their own tend to have other problems as well. Shipping software is managing the trinity and adjusting along the way, not blindly following constraints that aren’t working.

What was this feature and why was it so important? LIS was a new way to store all the email on a PC. This of course seems crazy today because no one wants email on their PC where it could be lost or stolen or worse. LIS was a new model of email where the PC would maintain a copy of the email that was on the server and keep the copies in sync. This made it possible to work without an internet connection while also enabling a rich level of capabilities to build apps on top of email that ran on the PC. This replicated storage was a key feature of Notes.

Underlying the feature was a new data storage architecture. Here’s the challenge. This model of software requires the code on the PC with the exact same capabilities as the code on the server in order to realize the benefits. Notes accomplished this with a smart architecture designed from the start. Exchange and Outlook evolved without such an architecture, and we were trying to retrofit a much more elegant architecture on top of Exchange. To do so would have required building a PC-based system with the same capabilities as the one running on big servers, but able to run with PC-level hardware not the much faster and more capable server hardware. The result: even by December 2000, LIS in Office10 was very best case 20% or more slower than Outlook 2000 and required a high-end PC. No one was accusing Outlook 2000 of being speedy, so this was a significant negative.

There were many other features that were slated to be delivered. Some of them were highly requested by customers. One was the ability to store email using the new UNICODE characters so one email storage file could easily have mail from any language. One feature that might be strange (or poorly architected) was that LIS was going to make it possible to connect from Outlook to the mail server using the web protocol HTTP and not the Windows networking protocol, which was really important for scalability and security. LIS aimed to provide a badly needed search capability that Outlook completely lacked. Another was the ability to store more than 2GB of email on a PC. This might sound crazy, but the cost of storing email on servers was so expensive that most Exchange customers were limiting email to 25-100MB (megabytes!) The rest of email could be stored in a separate file that existed only on a PC. The implication of such an architecture was that Outlook needed to be unbelievably rock solid and never ever damage that mail storage file. Any bugs or fragility in the code might mean a customer would lose all their email, permanently. The idea of inserting an entire new data storage format into the product at this late hour bordered on crazy.

Developing this feature was never going to be easy. It was another case of two major products with different processes, approaches to work, and schedules trying to align. Kurt DelBene (KurtD) was always calm and a great partner with the Exchange team leader Gord Mangione (GordM), but the tension over this work was palpable the entire release. The two products, while built by separate teams, were inescapably linked. Microsoft’s email strategy relied completely on both teams delivering an integrated product, while also serving another larger strategy (Exchange working with Active Directory, Outlook as part of Office).

The bet was even bigger for Office beyond Outlook. We had made a major bet on delivering “Office and Exchange for Corporate Groupware” as a significant pillar of the vision. While our vision process was still new (this was the second time we used it), the idea of losing a whole top-level focus area really hurt. In addition to Outlook, Office created a new tool called Designer, staffing an entire team of experienced engineers specifically for end-users to create applications like those in Notes. It was to be the cornerstone of Notes compete. Without LIS, there was no Designer—the work of that team would not ship at all.

The marketing team briefed important customers about the whole set of LIS deliverables including the features, Notes compete, and the new Designer product. There was a lot of excitement. It was easy to generate excitement with slides and mockups, especially when it made closing a big Exchange deal easier. Unwinding that excitement was brutally painful. Each customer meeting is incredibly difficult for the account and sets the relationship and business back. Immediately discussions turn to potentially turning to IBM for a solution to collaboration. Customers were tuned to escalate these failings straight to SteveB who in turn would again ask if there was anything to do or what he could say or offer. There was nothing. It was physics. The field hated physics. Customers did not understand physics. The press equated a missing feature with vaporware—software that never really existed except to muddy the competitive waters. How could Microsoft, the largest company in the world with the best software engineers anywhere not deliver? What did it mean for the future?

There were no answers, easy or otherwise. One way to say this was we failed to deliver. The lesson is not as simple as a failure to execute. Israeli military postmortems remind soldiers that there are no failures in battle only failures in intelligence. In software, failing to deliver was not a failure in writing code, but a failure in planning what code to write and how to write it. We were still planning products like the primary audience was retail customers or hobbyists who were more than happy to work through messy details, wait a little longer, or have some bugs as long as there was new stuff. Enterprise customers with their huge spend, multi-year planning horizons, and 5-10 year usage plans, were in no position to absorb this sort of attitude. We failed to plan, so our plans failed.

It took most of holiday season 2000 to make rounds with customers and all the teams to let them know we had cut the feature. The reactions across the company were varied depending on the team. The different cultures make quite an appearance at times like this.

The Office team already knew the feature would be cut by the time we told them it was official. More than anything they wondered why it took so long for Kurt and me to admit, finally, what they knew to be the case. Even our marketing team was somewhat relieved as this simplified our collaboration message to SharePoint and our email message to Exchange, without any redundancy. The Designer team, a whole new team, took it in stride as they knew it wasn’t coming together. I do not want to downplay the stress and strain on the individuals who committed a product cycle to the work. It was not their fault—they were at the receiving end of this failure.

The Exchange team was in a different state of mind. They tended to see things through the lens of Office decommitting at best or at worst Office was never committed. When a consumer of a dependency cuts a feature, it was often perceived as though they never really believed or somehow did not try, any evidence to the contrary. The presence of what was seen as a backup plan (SharePoint for collaboration) only made it seem as though there was a plan all along to “fail.” Exchange had a right to be anxious because they owned the Notes compete story and this was a big blow. It would take another couple of years, but Exchange would handily win in the market. The idea of building applications moved to the browser as quickly as customers decided they simply needed great email and scheduling more than applications, and Exchange plus Outlook was superior there. Competing with Notes, by 2000, was about skating to where the puck was as hockey legend Wayne Gretzky might say.

There was no escaping that the wounds were deeper on the Exchange team. Cutting LIS surfaced years later in a story highlighting Microsoft’s perceived morale difficulties. In the story in Forbes Magazine, “Microsoft’s Midlife Crisis”, a former Exchange engineer was quoted "They sent me a 200-page document that said our technology had to be 100% better than the current stuff. Then it failed, of course, so they did it themselves." The Outlook team would say (and did) that it just needed to be the same, not 4-5 times slower and bigger. Nothing was easy and when it doesn’t work and when failure is poorly managed, people can remember the worst parts in the worst way as they seek accountability.

There were hundreds of technical account managers who were anxious to begin to build Notes-like applications who reacted horribly to the cut. To them this was another case of Redmond not understanding what customers needed and worse failing to deliver what we said we would deliver. In the field, where salespeople have quota that they make or not, where general managers either make their numbers or find a new company to work for, failing to deliver what was promised was a first order failure. I attributed much of the enthusiasm and support of SharePoint for collaboration and Notes compete to the lack of an Exchange story. The enterprise team in marketing spent the better part of the next six months smoothing over each country and market, one at a time.

BillG was disappointed of course. Something he always did extremely well was bounce back from these setbacks and not put the team or leaders in any sort of penalty box. I previously shared the story of AFX, my first project, and how we wasted a year and got nothing done while Steve Jobs’s NeXT was on the rise. Bill was as anxious then as he was now and in a similar way—was there anything we could do sooner, more people, or a little more time? I bounced back. We bounced back. In this case, it is fair to say Bill re-doubled his efforts on the major project of the early 2000’s, data storage. In his eyes the failure of LIS only made it more critical to solve the company’s “storage problems.”

We held a big meeting in the atrium and announced the cut to the whole Outlook team. Everyone knew. This was a formality. As we were easing the team into the final decision, I thought of Andy Grove’s Only the Paranoid Survive. In the book he recounts the long process it took to come to grips with the need for Intel to exit the DRAM business only to come to understand the middle managers had long ago realized what was inevitable and made resource allocation choices in that direction. It was as if Grove was the last to really know. The team knew. We all knew. It just took a while to get there. It was the physics of cross-group collaboration, not just the physics of software.

A final reminder to the team was that we don’t market the features that didn’t make it into the release. No one was going to know there was something we did not do. In fact, the list of things we did not do was infinite. We did what we did, and it was going to be great.

I learned a lesson (again) about pre-committing to customers when basic engineering prudence said otherwise. With LIS out of the way, and Watson streamlining development, we were on a path to ship. We were feeling good. Like the end of every project, we were in the period where most people came to work and did nothing but make sure no one else did something rash. For a change, this holiday was going to be enjoyable and free from any project time crunch.

Cutting is shipping. We proved that once again.

Countdown to RTM blast-off and 3/2/01. At least I thought so.

Subscribers, share your story of the most difficult “cut”. In hindsight, was all the pain to make the cut worth it?

On to 067. MYR-CDG: Product Meets Sales

I admit up front this will be one of my favorite sections to offer. SharePoint was a remarkable point in the history of Office as we expanded the product line from desktop Win32 applications to include servers, a prelude to services. Within Microsoft this was by many accounts not only heretical, but also impossible. How could a team made up of “UI programmers” develop a server? Strategically, the inherent conflict between a server tuned for information workers and the actual server business was intense and fraught with difficulties. I would learn another lesson in bundling versus stand-alone product, and endless lessons in just how much the analyst world struggled to make sense of Microsoft’s product line, even when it just didn’t matter.

While I could spend many pages on the features and my love of SharePoint as a product, the transition Microsoft was going through while we developed SharePoint is equally important to the overall story being told. We will start there.

Back to 064. The Start of Office v. NetDocs

“We need an ‘Office’ server” was another one of those driveway lunchroom conversations with SteveB before he became CEO. It was a concrete expression of an abstraction. What he was really saying was that Office needed to think broadly about how to solve the problems information workers were having. That business card he used to make a note of “find me all the stuff about France” was now looking like a product issue for Office. We were all-in on the opportunity and were well ahead of Steve, having been thinking about this from the moment we saw FrontPage. We made it through the 97 and 2000 product cycles and FrontPage had established itself as a favorite tool of Internet Service Providers. We were ready to build on that foundation and expand the little web server we had been using to share product documents on the team. An Office server was a key part of the Office10 vision. The path from vision to RTM was not going to be a straight line.

The first half of the year 2000 was nothing short of eventful:

* Microsoft and customers survived the looming Y2K apocalypse. Despite dystopian fears, except for a few trivial and humorous problems, nothing went wrong at midnight.

* Windows 2000 shipped.

* Microsoft rose to an astronomical $500 billion market cap.

* Then the NASDAQ dropped more than 2,000 points with the Dot Com Bubble becoming a defining event of the rise of the internet.

* Judge Jackson declared Microsoft a monopoly that violated the Sherman Act (causing a 15 percent post-bubble stock price drop), and then later ordered the split of Microsoft.

* Office was attacked by a massive virus, resulting in the disabling of core product features.

* Capping this off, Forum 2000 was a landmark event in the evolution of Windows Server and introduction of what would become Hailstorm in an effort to rebuild Microsoft as an innovator in the internet era.

* PaulMa retired from Microsoft after having had an incredible influence on the company’s operating systems, platforms, and enterprise transformation.

It was also the start of SteveB’s tenure as CEO. Many believed this would mean little or no change given how SteveB was known widely as the “third founder.” SteveB would lead sales and marketing leadership and overall company execution. BillG would lead on technology vision and strategy. This seemed a formalization of what we always felt was the case. On the other hand, how could someone so different keep doing things the same way?

Earlier in the Spring of 1999, there was a BusinessWeek cover story about the remaking of Microsoft. It featured a photo of Bill and Steve together and the subheading “While Bill Gates plots strategy. . .Steve Ballmer shakes up the culture.”

With so much going on, Steve’s first reorganization as CEO seemed relatively minor even expected given how the sales force he created reorganized almost yearly. It didn’t seem like much of a “remaking of Microsoft” as the headlines touted months ago. There were some new people, but the big changes were in financial reporting.

Jeff Raikes (JeffR) returned to Office in the role of group vice president of the soon-to-be-renamed Productivity and Business Services (PBS) division. It was noteworthy that the name included services as per the end of 1999 strategy mail from BillG. JeffR was among the most tenured executives and led the original “Office” organization, the Office Business Unit, OBU, home to Word and Mail. Jeff joined Microsoft in 1981 when the company was 100 or so people, recruited from Apple by SteveB for a product role in Apps, where he led marketing for the original wave of Macintosh applications (and also shared a house with Steve for several months). On his PocketPC he used to keep a “days at Microsoft” Pocket Excel sheet he would open up when talking about how long he’d been at the company. Jeff was frequently mistaken for BillG, as both sported the requisite Microsoft uniform of loafers, khakis, and collared shirts, but it was the ’80s plastic-rimmed glasses that sealed the deal. He grew up in Nebraska, often describing his beloved family farm, an aerial photo of which hung in his office. Before Apple, Jeff attended Stanford. Jeff was the earliest of advocates of pen and tablet computing, leading those as part of Microsoft’s first foray into what was then called Windows for Pen Computing, which was also my first booth duty demonstration using C++ to code for a pen. Jeff also wrote an early memo describing the scenario that would become Excel pivot tables.

Prior to leading PBS, JeffR was the leader of the global sales force, having taken over for SteveB upon his transition to president. He pioneered the mid-year sales review process, which became a staple of the fiscal year planning process with its all-day meetings held through the entire month of January, all around the world. Among the sales team, Jeff was a legendary leader and stood shoulder-to-shoulder with SteveB in the evolution of sales at the company.

With his success in the field, Jeff was more a product of the SteveB perspective on problems and solutions owing to his most recent experience scaling the enterprise field sales organization. As Bill’s longtime friend, he was probably the only executive that was equally well-versed in both BillG and SteveB.

Jeff brought with him a team of support personnel. There was even a chief of staff, a role standard in the field but foreign to the product groups. This was hard to escape notice as we had quite a few offices and a big conference room taken offline (reserved) when in 2004 we eventually moved into the new building 36 that we had fought hard to occupy. Part of assembling the staff ended up taking a good chunk of the top floor of Building 36, which was designed to maximize space for the team, including window offices and already maxed out with the dev team. Space was always the subject of intramural skirmishes.

As part of the staffing of the PBS organization, for the first time, the Office group had substantial and dedicated finance and HR teams front and center of the organization—a seat at JeffR’s leadership team meeting. Some little things that were normal in the field seemed awkward or at least different in the product groups—like routine emails drafted by a staff member to be sent by Jeff for various planning or process tasks or scheduling meetings months in advance. JeffR scheduled everything, so spontaneous chats or walking by the office to talk became less the norm. Instead of PeteH stopping by my office, CollJ (our executive assistant) would get an email from JeffR’s assistant “Can you make sure your exec is available for . . . ”

It was natural though not necessarily mature of me to be skeptical of seemingly trivial changes, but these took place in the larger context of SteveB changes. I was not sure the new processes were consistent with a product-development team culture that was casual, ad hoc, and direct, but perhaps that was the point. During the transition I recalled a story that a former SteveB field staffer told me after he moved to Office marketing. He described how when SteveB moved from the Windows HQ product group to lead Sales, SteveB shifted his hardcore perspective on budget accountability. He changed to believe the field should be responsible for marketing dollars rather than the headquarters (product) group that needed additional discipline, a complete reversal from his days in HQ. As SteveB changed perspective, the problem area moved too. I started to feel we were a problem and Jeff’s role was to fix something. I wasn’t sure what or who was the problem.

Almost right away, JeffR pushed us to better equip the field to sell a big vision for the future of knowledge workers. This had been a long simmering tension point around Office, where I always felt squeezed between making sure we did not cause problems with customers timing Enterprise Agreement (EA) sales and customers wanting to know the future so they might sign up for an Enterprise Agreement. There was always that looming problem of future releases sounding so exciting, customers would choose to wait. Office was under a lot of pressure to get customers to deploy the current version and not wait. It seemed logical then talking about the future would only slow down deployment. This was clearly something that needed to be fixed. In fact, there were echoes of the SteveB 100 one-on-one meetings where he pushed me on this topic. Overall products were still late to market, and we’d just put ourselves out there massively with the expectations set by Forum 2000.

What the field wanted were more materials like Forum 2000, not better demos and whitepapers about Office 2000. The new head of Office marketing explained to me “when I used to work at IBM, we sold more products today based on slide decks about the future, then on any demonstrations of what today’s products actually did.”

I still admit I have a difficult time with that logic, but I am grown up enough now to know it is the reality of enterprise selling. I was uncomfortable selling the future. I was forever hung up on making promises we could not keep, or at least were not sure we could keep. I also felt selling the future was fraught with opportunities for misunderstanding. I’ve seen customers extrapolate features in directions we’d never go. It is obvious in hindsight I was just an engineer and naïve to the ways of sales. I was stuck feeling too high integrity to sell. I was silly.

There was one exception. We found a way to craft our own version of the future that was so far removed from product specifics, that it could not be mistaken for future product versus a futuristic vision.

To show customers (and JeffR) what Office was going to be like, we created a whole experience, almost a Disney-esque Future World of Productivity. MikeAng (now leading product planning) and the design team set up an Office of the Future. The team literally built an entire onsite immersive experience called the Center for Information Work. It occupied several large rooms (2600 sq ft) in a space next door to the Executive Briefing Center. Inspired by the soon-to-be blockbuster film Minority Report starring Tom Cruise (dystopian vision aside), the CIW offered guests a giant wall projecting status of the business—metrics for manufacturing, orders, and more. By a careful combination of scripting and clicking of wireless remotes, the demo revealed wall-size status reports, alerts, problems, and resolutions.

The CIW had a mock-up of an airplane cabin, long before there was Wi-Fi. There was a car of the future, where work and important decision-making continued en route, something that was just starting to take hold with wireless headsets, BlackBerry phones, and PDAs. Mike had listened to me go on and on about my new Toyota Prius hybrid and how futuristic it felt with an oddly mounted gear shift and big touch screen, so he secured a Prius dashboard and front seat, making the auto portion of the exhibit forward-thinking (and for me, super cool). Mike briefly regretted humoring me when SteveB’s main account, Ford, showed up and gave us grief over the car choice.

There were several workstations at the CIW with a wraparound, curved, 180-degree-view display, with the screens partitioned for monitoring activities, work, and collaboration. Curved displays weren’t available to buy but, working with display manufacturers, a curved perspective was created by stitching together flat screens with no bezels—innovative at the time.

The centerpiece was a Microsoft Research project called RingCam developed by Anoop Gupta’s team (AnoopG, who was also a BillG technical assistant), which was a combination of hardware and software for conducting remote meetings, in 2000! The camera sat in the middle of the table seamlessly capturing the audio and video of all attendees. It featured a fancy array microphone that combined with software to detect the speaker and switch the video image to them. At each chair in the conference room there was a Wi-Fi-connected Tablet PC. The PC was an integrated part of the demonstration—documents were signed, spreadsheets were examined, and notes were taken on this device of the future. As if to emphasize the permanence and paramount importance of tablets, the conference room table featured integrated, angled stands for the tablets designed for hands-free viewing and notetaking.

For the first time, I felt like we had an opportunity to get ahead of the constant demand for the future of Office. We revealed no dates, no specific features, but successfully put forth a vision for the seamless use of technology to enhance collaboration, improve decision-making, and integrate data into daily work.

Over the next months, CIW became a meeting place for CEOs, heads of state, VIPs, press, and customers considering Enterprise Agreements visiting Microsoft. Soon there was a team that ran tours nonstop. Over 1000 customers per month made their way through the exhibit.

We found success for the field by creating a story about what we might build one day presented like a World’s Fair exhibit that in no way could be confused with a product roadmap or plans.

JeffR’s new leadership team embarked on its first group project—to identify areas for growth in the business (the business was growing high double digits, but we were forever paranoid about peaking and the bottom falling out). JeffR and the new finance and business development team engaged a major consulting firm to develop a market map for the whole of the business productivity space. Where was all the money on business software going? This project became known as the opportunity map as we crafted a vision for PBS—not a vision like Office or CIW but a plan for determining what new categories of software to enter or serve.

Outside of Microsoft broad horizontal tools like word processing and spreadsheets, the software industry was in an expansion phase driven by the rise of server computing and the web. Businesses were building out data centers and adding server-based applications for sales, marketing, finance, human resources, and more. These tools were domain-specific capabilities that often featured integration with Office, particularly Outlook and Excel, and were generally much more expensive per user than Office, though used by fewer people. The opportunity map was designed to capture the amount of revenue flowing to the rest of the productivity software space.

The map, unsurprisingly, looked as much like a set of opportunities as it did like a set of all the tools that integrated with Office or all the tools that weren’t Office. Categories like Customer Relationship Management (CRM) were highly dependent on using email and integrating with Outlook as the user interface. Business intelligence for finance analyzed vast amounts of transactional data from SAP, or Oracle products to be sliced and diced in Excel, even if they worked to provide analysis in a web browser. While the market need existed to solve these product areas and scenarios, we always struggled with bringing to market domain-focused products while also running the risk of competing with partners who were telling their customers to buy Office.

The question arose as to how we would sell any new products. Would we add new dedicated salespeople for new products, bundle those products with the existing EA, or just add features to Office? We could add features to Office, but charging more for them rather than leveraging those features to drive EA renewal and upgrades was something we faced many times. The pull of bundling more and more into the core value proposition was relentless and the path of least resistance. Even creating an upsell SKU by holding back specific features was a battle against just making sure the customer renewed their EA. Equally difficult was marshaling a new field sales process should we have an entirely new product to sell. Everything was a tradeoff against either finding new EAs or ensuring renewals. I had run into this buzzsaw before and was skeptical, even with Jeff’s command of the field, that we would be able to find ways to sell whole new products.

It also didn’t help that most of these market map businesses relied heavily on consulting and professional services, something Microsoft steadfastly avoided. BillG was not a fan of consulting just as he long disliked product support or anything that could scale by only adding more people. As we kept learning, most of the integration with Office was less about the strengths of Office and more about trying to be part of the sales momentum and ubiquity of Office—using Microsoft’s distribution channel potentially afforded by Office. The makers of these tools did not necessarily want more software integration from Microsoft. Rather they wanted to find a way to insert their products into the massive Microsoft sales engine. If this sounds at all familiar, it is the dynamic we hear today time and again from SaaS companies.

Document Management was a market map opportunity that served customers such as law firms and pharmaceutical companies producing huge numbers of documents, requiring a detailed history of changes to those documents. Customers were always clamoring for a solution from Microsoft in the hopes it would be cheaper, or even free with Office. The existing market was a typical domain-specific product with high prices, low volume, requiring consulting or value-added resellers.

Aiming to enter this market while also broadening or redefining it, a server product under development in a different organization since 1998, Tahoe, planned to cater to the new space of knowledge management, a growing category of software for white-collar workers. The capabilities planned for the first version included managing a company’s Office documents, collaboration, versioning, profiling, and security. These features were to compete with products used in document-heavy professions such as legal and research. Tahoe was also going to be a server for an intranet that supported professional content management tools, web search (like Yahoo but for your own information), and more. Tahoe was going to do quite a bit, perhaps too much. Jeff Teper (JeffTe) led Tahoe and this new product team.

Tahoe, a codename, was a good example of a product arguably designed from the field sales organization out. As an example, at one tumultuous presentation to sales leaders I attended, Steve was anxious backstage to deliver a message to the field that Redmond had heard his calls for an Office server. The work we had done with Office Server Extensions and FrontPage in Office 2000 was not enough. We needed a product for the new generation of Chief Knowledge Officers (a new job big companies were creating) to compete with Lotus/IBM Notes (now called Domino). He knew of the Tahoe plans, but it didn’t seem like enough. There were existing plans for other products with codenames (Grizzly, Polar) and even a side project known as “Digital Dashboard”. In a classic reaction, he was saying we needed more, and sooner.

In what I can only remember as a blur of a tension-filled few moments, all the roadmaps for those products were realigned with components moved from one to another. It was confusing and I had only the vaguest idea of what would get produced. I just knew deciding backstage at a sales meeting probably would not stick. There was a confusing clarification issued after the meeting which remains difficult to parse. I struggled quite a bit because I knew we were in the space with code, dates, and a plan for Office10, but it lacked a certain strategic glow that the other products had. Our execution focused plan could not compete with big visions to dominate an entire ambiguous category of knowledge management. We just wanted to help people share and collaborate with Office. Our products were simple and, effectively, non-strategic.

That glow was that the new product relied on and connected our entire server product line: Windows Server, Active Directory, Exchange (including the new release codenamed Platinum), and SQL Server. What the field and strategic people loved was that the knowledge management from Microsoft used (or required) all the servers too. Such strategic connections were great for efficiency of sales resources and messaging. The whole package could be bundled up as knowledge management and the sales process focused on that high-level message, and not product-focused messages across five different areas. This approach spoke to CIOs and their strategy, not to line executives and execution. In a sense, this approach solved for knowledge management by creating another bundle, but not one designed and built together.

That such a product would be next to impossible to deliver and almost certainly fail to work well, was unfortunately a secondary concern.

In parallel with Tahoe (and the other products), the Office server we were building for Office10 had already survived a tumultuous run-up to get to solid plans. We based the project on FrontPage and the solid market foundation we had built. To handle key scenarios, we needed an additional place to store data beyond standard Windows file servers. For example, customers might want to label a document for marketing and another for sales and then quickly sort or select documents for only one category. The obvious solution to this was SQL but we were organizationally much closer to Exchange, the mail server, especially because of Outlook. Selling Exchange was slightly more important than selling SQL, owing to the battle with IBM/Lotus and the long-term advantage we would gain from an Exchange win.

We spent months going around in circles trying to make Exchange work. At one point I had a showdown with the development manager leading the incubation, MikeKo (an original Excel developer working in the FrontPage team), who was dead set against using Exchange. He would have known, because he was also the original development manager on Outlook. I couldn’t even get him to experiment with Exchange, which left me to defend our non-use of Exchange to the various executives involved. It was part of my job but not pleasant. Of course, I knew he was right, and I was trying to at least bring data to our decision. He thought I was foolish for even considering pushing Exchange on him.

The innovation underpinning our server product was a simple database, a list. Everything was going to be a list. There were lists of people, lists of dates, lists of announcements, lists of documents, lists of comments about documents, even a survey/poll feature for developing custom questionnaires to use, and more. Each list had all the same capabilities in that lists could be customized with different columns, sorted, filtered, and in general treated just like one would treat a list in Excel (or the Access database) while doing all of this from the comfort of Internet Explorer or Netscape Navigator.

We also had features being developed separately to publish office documents to a web server and maintain discussions about them, receive email notice when documents changed or were added (today we call these notifications), and more.

Though we started off with multiple teams, we reconciled the different approaches and arrived at one single plan. We called the product OWS, for Office Web Server (no exciting code name). We were deeply concerned about cost and complexity to deploy it, so we also made sure it worked with the free version of SQL, thus maintaining the free distribution of the Office Server Extensions from Office9. Backed by a full server and the full version of SQL, the product could support hundreds of users (including the entire Office team) but was easy and lightweight enough that groups of 5-10 could easily use it.

OWS was incredibly simple, yet wildly powerful. Even today in researching this section, I ran “SETUPSE.EXE” from the Office XP (“Office10”) CDROM on a standard Windows XP computer. With just a few clicks I had a full team web site up and running. Playing around with the resulting product brought waves of great feelings for all that we had built. Office built a server, a really good one.

Much of this simplicity obscures a remarkable program management effort in addition to the engineering work to build OWS. Program management started in the FrontPage team where JulieLar led the creation and incubation of the original efforts. In order to achieve the breadth of features and tight integration with Office, she partnered with another team in Office (the Office Product Unit) that delivered on integration with applications (for example opening and saving files to OWS) and other features already planned by them—this team was led by Richard Wolf (RWolf), an original champion of the concepts behind an Office server going back to the FrontPage acquisition. The connections across the team also included features built in Word, Excel, and PowerPoint. Creating such a highly collaborative PM environment to deliver integrated products was a hallmark of the new Office team, and OWS exemplified the work.

The full set of features would fill a whole other post. The full set of features would fill a whole other post. Even during pre-release press briefings, we would show up with a single laptop to show off a server at a time when server demos required multiple hefty computers. Many demos started in the server and flowed to Word and Excel then back to the browser. The excitement was such for the product that we ended up (through no small effort) connecting Word, Excel, PowerPoint, and Outlook to the server in a variety of novel ways—no one had really connected desktop productivity tools to a web server. When it came to mail, document authoring, presentations, and spreadsheets—the core of business productivity—those desktop files were like islands the internet could not reach. To the readers that wonder why we did not finish the job and do everything in the browser, I would note were still more than 7 years before Google would acquire its first web-based editing tool (Writely) and the Office team starting the web browser versions of Word, Excel, and PowerPoint. The web browser was not yet up to the task.

Apologies for a bit of indulgence by including the following list. Marketing’s final product guide for the release listed four pages of features including:

* Team Web Site

* Lists with sorting, filtering, custom views, notifications via email, import from Excel, and support column types including single line of text, multiple line of text, numbers, currency, date/time, multiple choice, checkbox Yes/No, internet link, picture, and the ability to choose from an item in a separate list

* Announcements

* Events including Outlook integration

* Contacts including Outlook integration

* Tasks

* Links

* Surveys

* Discussion Boards

* Document and Web Page Discussions (discussions within Office documents or about any web page)

* Document Libraries

* Save/Open dialog integration with Office

* Search using Windows web server built-in Search

* Full customization with the new FrontPage

OWS worked easily through a web interface. It even worked with Netscape Navigator, which drove a lot of Microsoft people kind of crazy.

Below is a demonstration I put together. It is running on a Windows XP Dell Latitude laptop from the same era (with typical specs). Everything was installed using the DVD that shipped with Office XP Special Edition (Office + FrontPage). Keep in mind this is state of the art HTML user experience from 1999-2000.

There was a big problem though, particularly with the document library feature. OWS and Tahoe overlapped quite a bit. Where Tahoe targeted IT managers with enterprise infrastructure, Office10’s OWS targeted teams and individuals, though required IT to set up a server and deploy it.

While Tahoe was progressing through its development cycle, Office10 was finishing up. We began deploying OWS to teams and the feedback was incredibly positive. We knew we were on to something.

Despite being asked to, there was no way to reconcile the strategic overlap. Office10’s server was small, lightweight, and had few requirements. It was designed like an Office product in that it implemented a focused set of features, simply. Tahoe was big, had significant infrastructure requirements, and required a lot of work to set up, customize, and integrate with the rest of the infrastructure. It was complex and not particularly suited to end-users. That was exactly perfect for what the field wanted—a big investment to get something working and integration with all the other servers they were selling seemed far more strategic than Office’s previously free server download.

During a one-on-one with BillG he really let me know how he felt. The meeting was set up to push me to reconcile the Exchange versus SQL storage debate. As CSA, Bill’s most critical project was achieving what he called “storage unification” across our products. He wanted to get everyone to use a single data storage engine, eventually called WinFS (a story for another section). In the meantime, deliberately shipping collaboration that did not fully utilize one storage system was a high crime. In the heat of this discussion, Bill said that it wasn’t that OWS lacked strategy, but rather it was “anti-strategic”. It was as if OWS was a net negative for what we were trying to accomplish.

Ugh.

The more we talked things through, the more we found an opportunity—and by opportunity, I mean opportunity for me to avoid a drawn-out battle between conflicting products that created nothing more than grief and a lot of meetings and no happy customers. The more we found an opportunity to avoid simply shutting down a project for lack of strategy and leaving us exposed in the market for what (at least I believed to be) an enormous space in web-based collaboration for regular information workers. The most useful parts of Tahoe, from my perspective, were the ability to search across a company’s disparate information sources, and to manage published intranet content sites and dashboards. We could thread a needle between otherwise overlapping products.

At the market map offsite, having had several heated email exchanges prior, Tahoe and Office (JeffTe and I) crafted a strategy, a napkin strategy—literally. Every company should deploy Office10 OWS for any team to use. All IT needed to do was set up a small server or two, and people in a company could create a team site—inside a company someone could visit a web server and, in a few clicks, have a custom team site up and running in a self-service manner. We started using this internally and it was instantly a hit, so much so, Microsoft IT became worried about the proliferation of all these sites. Just a year or so after shipping, we had over 3,000 SharePoint Team sites inside of Microsoft.

Enter Tahoe.

Every company could buy Tahoe and use it as the portal to all the OWS team sites that would proliferate around a company. The Portal category was gaining steam and came to represent the implementation of knowledge management. The more OWS sites a company created, the more valuable Tahoe became in order to search across them, keep a directory of them, and so on. For huge companies, Tahoe had other capabilities such as published sites for the company with news, information, the ability to search other important corporate information, even emails stored in Exchange, and an architecture to build corporate dashboards that were all the rage with IT. We navigated our way to a strategy of freemium with an enterprise upsell, like so many products in today’s SaaS era. Looked at another way, we created a classic enterprise arsonist-firefighter strategy whereby we at once distribute a free product that proliferates and another product, a revenue positive one, to manage that proliferation.

The SharePoint name, created and secured by the Tahoe team, proved easy to use for both Tahoe, now SharePoint Portal Server (SPS), and SharePoint Team Services (STS), formerly OWS. SPS provided synergy with the server strategy and hot features like dashboards that CIOs wanted. STS provided the departmental and end-user appeal where work happened. I thought this naming to be particularly clever—SPS and STS. I was horribly wrong and when combined with the full name was unwieldy (Microsoft SharePoint Portal Server 2001 and Microsoft SharePoint Team Services, where legally we could not use SPS and STS). This was a classic example of Microsoft product naming. My apologies.

We agreed that SPS would ship at the same time as Office10 as well, which was exceedingly important. Throughout the release we worked to have a unified experience to the degree we could. SharePoint would become a symbol of JeffR’s new organization and the broader value of Office products to corporate customers.

SPS was a perfect product for an era of complex server products. The industry was blossoming with products from companies such as SAP, Siebel, Cognos, and more. Microsoft spent tens of millions of dollars and hundreds of person-years, including consultants, to roll out SAP. The same with Siebel. Such heavyweight products were state of the art in enterprise software. SPS embraced and reflected those attributes.

STS was an emotional product for me. It was the end of a journey that started with making a web page with our specifications for Office9 and using FrontPage running on a server under my desk. The FrontPage team enhanced the server extensions to create the foundation for STS. The product was pulled in many directions for strategic reasons, but we stuck to it. More than once, I had to go to meetings where we debated killing STS because it conflicted with some strategy (Windows Server, Exchange, bCentral, etc.).

The idea of Office extended by a website for each Office user and team was incredibly important simply because it made using Office better. It was also a vision we had from the time we acquired FrontPage—everyone should have their own place on the web where it is easy to keep their work and share it with others. We were clearly too early. As we will see it was not just that the world was not ready, the world was anti-ready. SPS fit with the products of the era that remained top-down, complex, and under the full control of IT.

We struggled to turn what we built into an asset—the company, or mostly the field, thought of Office as the desktop EA. Getting the right skills to communicate and sell a server was not part of the Office sales motion. It was frustrating to watch companies introduce products that did the things we did and receive credit with analysts and press, or even customers asking if they should use them. We were inundated with requests to do business partnerships with team collaboration products all while, essentially, building a good one in relative silence.

The expression my former managed ChrisP used was “magic beans.” It was a way of describing how some people in the company, no matter what they were doing, seemed to be able to summon magic beans and make products look much larger than life. There was something about STS that lacked magic beans.

We had big plans for STS down the road in the next release. STS was the foundation for extending Office to subscriptions and software as a service.

STS was not without controversy on its own. The company had not yet warmed up to web pages, especially as user experience. I realize how crazy this must sound. In 2000, the company was still of the view that the web is the type of experience for “reach” and the “rich” experience is what happened on the desktop. BillG especially was still hoping for a Win32-like experience for everything we did that used the internet. As an example, the discussions feature of STS also worked from directly inside the applications. It was a ridiculous amount of work we took on just to reinforce this view. Other features such as a Tasks list and Contacts List, which were simply trivial lists in STS, were viewed harshly by BillG because those were supposed to be in Exchange and Outlook (we also did the work so they could be accessed from Outlook).

During the traditional demos with the team, Bill got to see SharePoint (SPS and STS) of course as it was a pillar of the release. We left that section of the demo with him grumbling to me in person, “I hate that UI”. Years went by with him saying “SharePoint? That thing I hate.” Every time someone mentioned SharePoint or sent him a link to a SharePoint document (oh, the links did have horrible URLs that were crazy long and meaningless) he would grumble if I was there or forward me the mail complaining about the link or the UI. I concluded this was much more about the idea of a commoditized HTML-based interface than any specific choices for STS, that even today are fairly benign.

Regretfully, STS was oddly lost in the shuffle. CIOs and thus our field were far more excited by the prospects of enterprise-wide dashboards and other SPS scenarios. The team collaboration was almost backburnered. Where SharePoint really made a name was with an army of consultants who saw SPS as a big opportunity for value-added reselling. The fancy web interface for digital dashboards was both slick and required custom programming for every customer. Combine that with the purchase and management of server hardware and there was a great business for consultants. JeffTe and the new SPS marketing team started a SharePoint conference which turned out to be a cornerstone event for the PBS organization and the field loved everything SharePoint.

Internally during the beta of Office10 we worked with Microsoft IT and created a self-service STS. With a single click, any person in the company could create their own STS site. IT would manage it, back up the data, and keep it all running. It was an enormous hit. Every time a team had a project or an event they would create and STS site. Teams that were already managing their own internal web sites with custom hardware and web development were switching. STS was a huge win for IT, who had a new product they could offer their internal Microsoft customers. We had hoped to replicate this at every big customer. Why not? The rollout of SPS, while exciting to MSIT, did not go as smoothly. The search capabilities were slow, limited, and frustrating to use. The dashboards were difficult to create and slow. It would take some time to transition content publishing sites to the SPS model. The high-end document management features did not replace the domain solutions in place. It was tough.

Even as we came close to shipping, all was not so great. Sometimes it can be difficult to really understand success and what was actually achieved.

Around the internet today, Google Drive, Box, and Dropbox are all products in which I see STS (not literally of course). There are a dozen start-ups building products today with the features like those STS had in 2000, even using the “everything is a list” metaphor such as Notion. The foundation of STS remains a key part of Office 365 today, but still more than likely underutilized by customers given the rise of so many viable alternatives or even the difficulty of finding SharePoint capabilities. Of course, a great many people and organizations rely on modern SharePoint as a key place for storing documents and files. As BillG used to say in exasperated moments, “yet another place to put files.” The potential for tools to improve team collaboration beyond file sharing was mostly obscured by the complexity and weight of the whole offering.

Being early is sometimes the same as being wrong. Being in a bundle is sometimes…not being at all.

A bit after launching the whole wave of products, at the annual sales mid-year reviews SteveB was pushing one country on their Office numbers. As I recall, the country manager began to complain that they pushed SharePoint, but it wasn’t landing. The manager said they are doing a bang-up business and the channel (those value-added resellers) and CIOs love SharePoint, but customers aren’t using the product. I froze. I knew for sure I was about to get pounced on by SteveB in front of everyone. Instead, Steve looked at the me then the room, and agreed. He said, “I get it, no one anywhere is using SharePoint.” Others in the room disputed that, perhaps out of concern for leaving a bad impression for the SharePoint brand they loved.

The conversation continued and Steve turned to me and mouthed again “no one is using it”.

He knew. He was right. As popular as the product was, the part we were selling and what led the conferences, the excitement from industry analysts, and reseller activity, just wasn’t what people were using inside of companies. The team collaboration parts of the product, STS, didn’t receive the attention or visibility so those were under-utilized as well. Was the product a failure?

No, the business results were clear. SharePoint all up had succeeded competitively and by expanding the overall push of servers into the enterprise. This wasn’t the only time customers were buying what we used to call shelfware. The Office business benefitted enormously from the upsell of Office Standard to Office Professional. Office Pro added the Access database, another wonderful product with an incredibly fierce and loyal following. The only problem was that it was used by a single-digit percentage of users, even though half and soon nearly all customers purchased it. We didn’t intend for that to be. It was a win, but bittersweet. It can also confuse what success is in a big company.

Microsoft had a talent for creating a success out of a product that still wasn’t quite a success, as if through sheer force of will and the power of distribution. We issued momentum press releases citing statistics in the millions. A single major company logo deploys a mission critical application which we reference relentlessly. We can even garner support of resellers around the world who will gladly take the business we send to them, even if the result is simply a showcase application. SharePoint was even touted as the fastest product to one billion dollars in revenue, which given Microsoft’s revenue mechanisms is an awkward datapoint. These were the tools of the era. Today, the techniques of growth hacking are the norm such as user-interface that triggers use of a feature that may or may not be intended, or defaults that drive apparent usage of features. All of these are used with bundled products, because absent the specifics of customers directly acquiring a product, Microsoft had no genuine indication of market success. The nature of the big enterprise bundle makes knowing reality difficult.

What really matters is if people are using the product naturally (organically) and the usage is improving (not necessarily more, but decidedly more valuable). We lacked the tools in the early 2000s to really know, but SteveB’s intuition was never doubted. Today we think of this as lacking a clear understanding of product-market fit, a state of existence when the market literally pulls the product out of the company.

We did have exciting moments with STS to be sure. We had an inbound support incident from Japan where a customer was hitting limits for how many documents could be added to a single folder. We had not tested it with more than thousands and learned the customer was unable to add over 100,000 documents while converting all their file servers to SharePoint. Yikes.

The lesson of the bundling things together (STS bundled with SPS, SPS effectively bundled with Exchange and SQL) should have been clear to me by now, having lost the battle over Outlook and soon some other products. I should have admitted defeat and recognized that new things go in the existing product bundle and the economics always make sense—meaning that not charging more doesn’t matter because customers continue to value what they paid for already. We just really dislike building shelfware. Winning with shelfware doesn’t feel like winning.

Every time we were deficient competitively it was to an unbundled product. Even our own unbundled businesses of Visio and Project were almost one billion dollars. Even worse, the unwieldy size and scope of the Office bundle all but guaranteed most customers would never know or experience most of what we do. That’s the fate of most every successful bundle in business software from Microsoft, Oracle, Adobe, SAP, and more. Any success, or defeat of a competitor, comes from an overwhelming mass of software. I was simply naïve to think customers don’t appreciate free, which after all is the same as adding more to the bundle means. I eventually came to think about this as customers paying $300/year for Word, Excel, PowerPoint, and Outlook and everything else was just free and of little ascribed value. The very reason new or startup competitors can win is because there is a price attached so customers know about a product.

With a bundle as features are added and customers keep buying, the value of the bundle appears to be reinforced. The act of bundling is correlated with growth, but there’s no causal link. Conversely, when something is not bundled but the entirety of the sales motion is with the bundle, then the company concludes the unbundled product was a failure on its merits. Again, that is only a correlation not a causation. The presence of growth hacking and so-called vanity metrics distort available metrics, causing even more clouded judgements. Bundles make for great value and efficient sales engine but make it extremely difficult to know if you’re doing the right thing or building a good product. Bundles make it easy to be lazy when it comes to building a fantastic product. Bundled products don’t have to be great to win, just good enough.

I once vented to BillG about this, and his view was more sanguine. He said to think of the whole as a portfolio and to just make sure each release that the total value was what it needed to be. Bill was big on the portfolio approach to management.

In a similar conversation, SteveB was also right about the fact that sometimes the bundle wins, even with an inferior product, because the winning product is not just a product but the value of the product, distribution, and ecosystem around it.

The product confusion did not end. The Windows Server team that sold file sharing servers was deeply concerned they were being pushed out by STS, which did provide a much better way to share files. After many rounds of discussion, the best course to solve this strategy problem was to include STS in Windows as well. More distribution is better I thought. Due to antitrust scrutiny of any bundling choices, we ended up renaming STS to WSS, calling it Windows SharePoint Services. What seemed very clever only further served to marginalize or fracture the product.

This was killing me. I could not figure out what was going wrong or what I was doing wrong. It seemed to be so hard to get traction bringing this to market. I know not everyone shared my view of how cool STS was. What was cool was being measured differently. It was like we valued momentum or strategy more than usage. It couldn’t be that building more complex products that were harder to use and more difficult to deploy and manage was the right answer?

At times I wondered if I personally lacked magic beans, but I didn’t obsess about it. I was able to guide large products, lead teams where people were generally happy (as measured by the ever-present MS Poll survey on employee satisfaction, which SteveB re-emphasized significantly), and to get done what was promised without much dirt flying. None of my managers ever commented on the innovative work the team accomplished—things like SharePoint—or the risks we took, like how we shifted the team to build Office, or even the Office Assistant (“Failure is good,” BillG said often). Rather, the conversation was always thanking me for keeping the trains running on time. Yes, it was great Office shipped on time when almost nothing else did, but it was so much more. The team deserved credit for more than just good at shipping.

My mentor Jeff Harbers and I were once talking and he offered me a way to think about this. Leonard Nimoy’s autobiography, I Am Not Spock, detailed all the things he did besides that one role—that one role everyone knew about. Many thought he begrudged Spock and was not appreciative of the role like fans were. I felt as though shipping was cool, but the team, and I, were so much more. A decade after his original book was published, Nimoy wrote I Am Spock, in which he embraced the role as part of him and explained how people misunderstood the first book. Later Jeff Raikes even lifted my Spock analogy in one of my performance reviews. It resonated. I reached peace thinking about that duality.

Later, working on Windows, I completely embraced it.

On to 066. Killing a Killer Feature (In Outlook, Again)

Microsoft went through so much in the first year of the millennium. It began with SteveB taking on the role of CEO and BillG taking on a new role as Chief Software Architect. Over the first months of the year a new set of technical leaders convened under the direction of Paul Maritz leading all of the product groups to define essentially the next Windows. The group was producing the plans for NGWS, next generation Windows services as outlined in memos from BillG and SteveB. The DOJ trial was complete, and we awaited the verdict, but everything we did was looked at through the lens of the implications of the trial. At every step we were asked if what was going on was a result of the trial, anticipating an outcome, or designed to work around what comes next. Personally, I was just getting my footing as an executive and the leader of Office and just figuring out what it means to be leading such a big part of Microsoft. I still had so much to figure out and was definitely worried about leading the train off the tracks.

NOTE: I really want to offer a big “Thank You” to all the paid subscribers. It takes a lot to take the time to support the work materially and while the proceeds of this work go to registered non-profits in the US, it still means a great deal that you’re paying for the work. Many subscribers are about to hit their one-year renewal and I wish to thank you in advance (and welcome new subscribers). As planned, there is almost exactly another year of posts planned—and for many this is the extra-exciting stuff including what’s coming with SharePoint, Outlook, the Ribbon, Windows 7, Windows 8, Surface, and many controversial (at the time) topics like Courier, including today’s section on NetDocs. THANK YOU!

To celebrate the anniversary, please consider sharing this link for a discount on the yearly subscription. https://hardcoresoftware.learningbyshipping.com/anniversary

Back to 062. Antitrust: Split up Microsoft and 063. Managing a Verdict

The demonstration within the multi-hour series of keynotes was billed as a “sneak preview of something that hadn’t been demoed before. . .technology that embodied the dot net user experience. . .this is real code. . .this technology will apply very broadly in the future across Microsoft products such as Windows.NET [pronounced Windows dot net], Office.NET as well as the consumer subscription service.” What followed was a 15-minute demonstration of word processing, spreadsheet, email, calendaring that looked like Office. The demonstration was easier to use, sleeker, and more connected. It featured enthralling technologies like “universal canvas” and “XML”. What’s not to like?

Within a news cycle the technology demonstration had ballooned to Office.NET and was the future of Office. Within Microsoft, especially in Systems, nothing was higher praise than being the future project and conversely nothing was worse than being the past. The world of dueling code names had been brought to Office, except now it was Office.NET and whatever I was working on, aka Office10. What had been shown was being built by a separate team, an organizational peer of the newly christened old Office.

What was this and where did it come from? Was anyone building a product called Office.NET? Was this planned? I certainly knew the code being demonstrated, but the idea that it was presumed to be the future of Office was newsworthy, even though we did not say that directly and had not intended to leave that impression as far as I knew. Nobody wants to Osborne the most profitable part of Microsoft (a reference to a well-known microcomputer company that went bankrupt preannouncing a next generation product). Weird.

Starting in early January of 2000 coinciding with SteveB’s promotion to CEO and BillG assuming his new role of Chief Software Architect, BillG and PaulMa began working on a series of strategy offsites, meetings, brainstorms, memos, and more called Next Generation Windows Services, or NGWS. There was even a new leadership team formed called the TLT, Technology Leadership Team. Everything was kicked off with memos from both BillG and SteveB highlighting NGWS. BillG explained in a memo Opportunities in the “Software Decade” that NGWS was a bet on par with the graphical interface in both the transformation and opportunity. By now, I was seeing this as a familiar playbook. If you want to say something is big, then compare it to the graphical interface. SteveB also had a memo, Changes and Opportunities.

These memos, Bill’s detailing the technology at a very high level and Steve’s articulating a customer and business focus put forward an innovation agenda for the company. The goal was to let Microsofties know the company was committed to innovation, especially with the rise of “dot com” companies and the huge valuation of anything internet in the public markets. It would be a few months until the market corrected itself, but Microsoft at this juncture was generally in a defensive posture. A broad re-branding was intended to create a new narrative for the company, supported by a next generation of technologies designed for the internet from the ground up, at least starting from Windows.

Just days after Bill’s memo, there was a call for participation in the NGWS planning sessions. A detailed series of offsites and meetings were scheduled. Literally, the invitation stated the work was to figure out the details of NGWS as introduced in the memos.

Please consider subscribing. New subscribers can use this link for an anniversary promotion discount.

In other words, we had a brand before we knew its meaning. That is not entirely fair because there were at least two key strategies under development. A series of projects defining consumer internet services such as email, calendaring, and identity were being developed by a group of the most senior leaders previously of Windows NT and others from around the company, an effort that would later become known as Hailstorm—consumer-focused experiences that also provide a platform for developers. A second effort encompassed the creation of a new programming platform for building internet applications on the server, which was just becoming known as .NET (dot net). This initiative included a variety of tools and platform work that built on the lessons from the first generation of internet applications.

To confuse matters, the .NET branding was starting to be picked up by a range of groups and products, including the next release of Windows NT Server which was sometimes referred to as Windows.NET (originally codenamed Whistler, then later Windows Server 2002, then finally Windows Server 2003). The .NET name was also used for the APIs and programming tools which would collectively be called the .NET Framework, for programming both servers and desktops. If you’re confused reading this, then you were not alone. NGWS was an umbrella term for everything, and .NET was intended to be a technology term, but we sort of ended up with two umbrella terms. The .NET branding was one of the more chaotic and self-inflicted product naming efforts (Was it .net, .NET, or .Net? Before or after the product name? With a space or without?).

Like BillG’s previous memo on software as a service, this memo also lacked any mention of Office. I was beginning to see a pattern. Only now that I was managing Office I started to wonder if I was somehow contributing to this. Whenever I reviewed drafts of these memos, I did not seek to include Office out of a reflex to fly under the radar and to avoid making promises for work that was not even underway, a strategy reinforced by my time working as BillG’s technical assistant. The team always noticed, and I found myself doing my best to explain the virtues of being left out of the strategic fray. Should I have lobbied or been more forceful about including Office? Many would be naturally inclined to run towards the limelight, but so far in Microsoft’s history that proved to be less than helpful. The Cairo OS project was a top-of-mind example.

PaulMa and the platform teams planned a strategy presentation for the press and industry analysts detailing Microsoft’s internet-centric developer strategy. Originally scheduled for early June 2000, the event was delayed several times because of the looming court ruling in the antitrust trial. The NGWS working groups welcomed the extra time. The evangelism efforts did not slow, however, and the first half of 2000 was a steady stream of stories about what NGWS might be along with the implications of the looming court ruling on NGWS, or perhaps speculation that NGWS was an effort to end-run the potential court ruling.

The Windows team generally loved these strategy days as a key part of the culture. PaulMa would often describe them to me as a “forcing function,” which meant a way to coalesce disparate groups into a shared plan. In this case, the planned event would force upon us a collective definition of NGWS.

The industry loved these events too. These were made-for-press events. Stories would run describing what could be announced in the lead-up (called curtain-raisers) and after the event there would be ample analysis. As was almost always the case, the event would gain a nickname or acronym. The event almost always included a new strategy with its own name or acronym. On the heels of Internet Strategy Day and Windows DNA (Distributed internet Architecture), the press was anxious to learn what else was on the way. Frequently, such event days would get scheduled with only a vague idea of what would get talked about and shown. This was one of those events.

The weeks leading up to the event were chaotic and high stress. While the goal was to present a coherent strategy, the process of creating the strategy was more important—the forcing function. This was how Platforms came together as a team. Whether a PDC (Professional Developer Conference), a workshop, or a strategy event, Platforms used the process of creating the presentations the same way Office used memos and the vision planning process. Only Office spent months and involved a broad cross-section of the team across disciplines whereas Windows spent weeks and usually involved the key people, however that might be defined. Instead of detailed plans and schedules like we created in Office, the output consisted primarily of bullet points and architectural diagrams in PowerPoint.

Dubbed Forum 2000, the event proved a seminal moment in the evolution of Microsoft’s platform and quickly came to be known as .NET Strategy Day or .NET Day, and SteveB would refer to it as “most ambitious undertaking since Internet Strategy Day in 1995”. The event aimed to be almost a “mother of all demos”, in reference to the legendary 1968 demonstration of the first graphical interface, hyperlinks, video conferencing, and mouse.

At this point, Microsoft’s approach to strategy presentations was adept at mixing a combination of BillG-style architecture slides with short and slickly produced video vignettes complete with keyed in screen mockups. The series of scenarios envisioning the future of software enabled by .NET formed the heart of the strategy articulated at Forum 2000. They featured the gamut of nascent technologies that were frequently talked about including tablet/pen computers, handwriting, wireless, voice control, video chat, location awareness, presence awareness, notifications, mobile devices, and so much more. The scenarios and designs had a decidedly consumer feel including the bubbly buttons and logotype used in MSN.

There was a slight problem in that the gap between what the audience saw in those demos and what any team might have been working on was, well, significant. It was not that many technologies were decades away from possibility, but only bits and pieces of a product were being worked. The role of a platform strategy is to inspire, however, not necessarily detail everything that is available in short order. Like the 1994 Information at Your Fingertips keynote, these sketches of the future were prescient and designed to create a north star for the company (a favorite expression) and even the industry. So while it was a challenge to be so far out, it was intentional. Unlike previous visionary presentations, Forum 2000 was far more specific in terms of product and roadmap. That was the problem.

Today many look at these presentations and the underlying products that emerged as evidence of many ideas where Microsoft was early, but for one reason or another fumbled in the transition to the modern world. It would be fair to say Microsoft was early to many shifts over the years, but as was so often the case being early ends up being wrong as well. When one is early and fails, the problem was usually that the culture or technology underpinnings are not yet mature enough to support the vision or the market was simply not ready. One could go through each of the technologies shown to realize the decade that would be required to bring them to market. Tablet PCs required screens and processors that did not exist. Handwriting recognition had been stuck at a level of reliability that was more frustrating than useful. Mobile devices would undergo a huge transformation with touchscreens and ubiquitous data connectivity. The services talked about would ultimately arrive with an entirely different architecture than Microsoft was building out at the time. Technologies such as XML would be widely used, but as commoditized as plain text files have always been and confer no real proprietary advantage as Microsoft hoped. Other technologies, such as virtualization that were key to the early cloud era had been rejected and would not play a part in Microsoft server strategy for another 5 years. Early efforts tend to be pointed in the right general direction, but the small errors or incorrect initial assumptions compound over time and ultimately diverge far too much from what eventually makes it to market.

Strategically, the comparison to the transition to the graphical interface was front and center and was used several times throughout the presentation. Our shared desire to repeat that transition and the success that followed provides evidence that being early is good. Windows arrived before the computational power and memory capacity could run the software we built. Taking time for technology to catch up did not deter us. Perhaps what would ultimately trip us up, however, was the grandeur and interconnectedness of our collective plans that left little room for execution error or for influences from the outside world and what was transpiring on the internet at a rapid pace. As many would note critically following the event, it was still about Windows at the center and to truly be a new strategy the conventional wisdom held that Windows needed to be abandoned. It is not at all clear to me that was the mistake, though it does make a simple narrative.

This was the peak moment for the catch phrase developed in response to the antitrust complaint of integrating software into Windows—integrated innovation. We overachieved on integrated innovation in that everything was integrated with Windows as we thought we should be permitted to do. Our defense was also our strategy, and also our technology foundation.

Nevertheless, the industry was excited by all this big talk. If there was a theme to the day, it was innovation. Every section of the day featured an explicit slide calling out innovation. This was a subtle jab at the critics and regulators who felt Microsoft achieved a dominant position and grown subsequently complacent. Innovation highlights were provided for .NET Services, .NET User Experience, .NET Programming, Small Business, and Business Users. That’s a lot of innovation! In addition to the videos, we showed live demonstrations of code: the earliest TabletPC prototype and handwriting recognition, a new browser-based service for small businesses, and the technology demonstration described earlier.

Perhaps more abstractly, the day was about a new era for Microsoft. There were the existing products and from this point forward there were new products built in new ways that would solve the problems the old products had built up. Everything was going to be faster, take less memory, reduce administrative burdens, and provide new levels of capability and convenience for customers. Microsoft clearly divided the world into old and new. That was bold and companies almost always lack the fortitude to make such statements clearly.

Competitively and concretely, .NET (using the term broadly as everyone did) was Microsoft’s answer to Java for server programmers. That was the big battle driving the platforms strategy. Java had captured the hearts and minds of developers building web server applications. The .NET technologies for enterprise software development would go on to create the platform that dominated in-house enterprise IT software, creating a generation of .NET programmers who today are more than comfortable with Microsoft as a provider of cloud infrastructure, even if it is Linux and not Windows. While the .NET programming tools would launch over the next year, this was the first real stake in the ground. On the PC desktop and client, we had won with Internet Explorer, which allowed the vision for the user experience portions of the presentations to move forward with fewer constraints and a focus on what was done on the PC but also supported in the browser—a desktop-first strategy.

It took 18 months before the first product release with the .NET architecture, Visual Studio .NET which was the first product to use the .NET name. The server product line underwent a pivot to support the new capabilities. Ultimately, .NET and its companion and proprietary programming language C# were enormously successful for Servers and Tools and came to define the era of enterprise client/server computing—so much so that most of today’s leaders in IT were products of the .NET era and, as a result, Microsoft created a generation of business IT leaders strongly connected to the company. Much of Microsoft’s strength today in enterprise accounts can be directly tied to IT leaders that rose up the ranks by betting on .NET.

Closer to home, the session on “User Experience” which was really about Office featured a presentation by a group building what instantly became known as Office.NET even though there was a clear demarcation of “technology demonstration”—we loved to think these small changes in wording brought us air cover or permitted distinction between products and directional demonstrations. To clarify, the technology demonstration did not claim to be Office.NET but the roadmap slide of product releases we presented at the time used the name and provided a “2002+” ship date.

The technology came from BrianMac, creator of Outlook, who upon leaving Outlook started up a new team called NetDocs, for network documents. NetDocs reported to BobMu, my manager at the time as well, though Brian and I had not crossed paths all that much since Outlook. We were both focused on what we needed to get done separately.

BrianMac formed the NetDocs team much the same way he built the Outlook team, growing the team to over one hundred in short order. The vision for the product was expansive and included many hot, new technologies. It was also being written in the latest technologies, including the latest magic technology XML (eXtensible Markup Language, which was becoming increasingly popular as part of programming for the browser) and, more importantly, it was using many of the new capabilities in Internet Explorer. XML was also the latest magic beans technology that took on capabilities much greater than reality. Brian had a knack for constructing expansive visions assembled with the strategic technologies as we saw with the creation of Outlook. As with Outlook these technologies were new, unproven, and unfinished. Outlook did quite well.

The scenarios enabled by the NetDocs vision subsumed Office, particularly Outlook, Word, Excel, and more, but with a decidedly modern take. By modern, the implication was that people no longer needed to worry about which Office app to use as there was one single document type, the universal canvas, that worked equally well with words, numbers, graphics, and email, and was easier to use because of that. This was not a new vision and in fact the idea of integrated packages had a history of attempts from both Lotus and Ashton Tate in the pre-Windows era, as well as Microsoft’s Works app (a modest success for price sensitive customers). The all-in-one application was a favorite among the first generation of PC users and BillG in particular who routinely complained about overlap and redundancy across the various “modules” in Office, modules being his favorite way to describe an app in the suite. Would this time be different? Did the processing power and memory finally enable this? NetDocs set out to prove it could.

I was skeptical, but from my position in Office skepticism was viewed by others as defensive and territorial. I wasn’t being defensive. I just didn’t think it could work. Others projected it could be a $1 billion business within three years.

A running joke at the time was that every new product idea somehow included digital photos, and Forum 2000 was no exception. Every demo included digital photos in some form. Digital cameras were the hot consumer item and sharing photos on the internet was becoming mainstream. NetDocs not only included photos, but to illustrate the importance of photos, a product that was extremely innovative but languishing at retail without sales and marketing support, Microsoft PhotoDraw, was reorganized into the NetDocs team. This method of building a new team by acquiring other internal teams and jettisoning their existing product was a strategy employed with Outlook as well. I was a big fan of PhotoDraw and to me it was one of many examples of innovative tools Microsoft created that we were unable to capitalize on because it was too small on its own and too niche to be part of Office—this will become a familiar theme shortly.

There was another running joke that every new product idea being dreamed up somehow also included electronic mail—email was the anchor of the internet and became a big deal for AOL, Yahoo, and MSN. Microsoft was a clear email leader with Exchange, but that was for business. For consumers, Microsoft’s MSN division acquired Hotmail in 1997, the first web-based, viral, and free internet mail. The number of email users on that service was approaching 100 million, when the entire internet population was roughly 300 million. NetDocs also became email. That should not be a surprise given the roots of the team and leaders.

Photos, email, calendaring, XML, word processing, spreadsheets. . .that’s a lot, a lot to like.

NetDocs also enabled a new subscription business model. There was nothing particularly technical about doing this work, though convincing customers to rent software (as people thought of it at the time) was new. The team was working on a new technology to provide seamless updating of the NetDocs Windows desktop application over the internet. Seamless updates might convince customers of the benefits of rental over ownership as the product could be enhanced without purchasing anything new. Customers were struggling to deal with updates, most of whom were not yet able to use the Windows Update service that is now standard on every PC.

Given my early experience getting the first version of Outlook to customers, I remained skeptical of NetDocs achieving all that was sketched out, especially without the kind of constraints being part of the Office release imposed on Outlook originally. The amount of code to write, the ever-changing scope (and resistance to constraints), the huge challenges of building some compatibility and interoperability with Office, as well as the fragility of the technology foundation—as the latest and greatest always seem to be—were not usually a recipe for success and seemed familiar. The success Outlook achieved, due in no small part to being free and bundled with Office and more importantly the only client for Exchange email, provided a halo of sorts for NetDocs. This is a good lesson in how success in a big company can take many forms beyond customers laying out their cash for a product.

I had not paid attention to NetDocs (nor NetDocs to Office) and now suddenly and without warning, NetDocs was front and center strategically for the company. NetDocs was filling a void in the strategy, at least internally which was that for the .NET platform to be successful it needed a killer Office application. I should have internalized that strategic point going to the NGWS meetings, but I did not. I managed the Office10 project aware of the costs of choosing to lay low—that people would view Office as failing to support the platform or even to acknowledge the future. In an industry (and especially a company) where the next version is always way better than the current version and new platforms always require the leading apps to support them, it was challenging to take this approach.

This old versus new dynamic always creates tensions in a large company. Echoing Innovator’s Dilemma (again), a series of press stories played out over several months. While there were grumblings, overwhelmingly people on the respective product teams were not consumed with potential overlap—Microsoft had a long history of next generation projects that fizzled. When will NetDocs replace Office? Will Office stand by and allow NetDocs to replace it? Will customers be confused? How will the market deal with two kinds of Office products? This was a far cry from the Cairo versus Windows NT, or the Windows NT versus Windows 95 battles that played out over years, at least I thought that to be the case.

During these times the negatives the market perceives of the incumbent are amplified irrationally—software bloat, nothing left to add, slowing growth in the business, and more. Simultaneously, the perceived positives of the new product are amplified irrationally: sleek, modern, simpler, faster, lighter weight, innovative, and new. Microsoft was great at setting up this dynamic. I had been the poster child for old technology and resistance to change more than once (Java Office, component Office, web Office), and while I could brush it off, in the case of NetDocs and Office there was quite a bit of bashing externally of a product that was half of the company’s profits. The internal tension was significant, but not because of a deliberate product competition or organizational competition for resources, but because there’s no way to constructively align the past Office with an ever-expanding vision. Regardless of the strategy, NetDocs could have laid low and first spent a couple of years building a product. It just wasn’t in Microsoft’s culture to do so at this time given the demands to put a big vision out there.

Nothing could stop the Forum 2000 train. This was exactly what NGWS needed. There was broad satisfaction with the event, even though ongoing legal challenges clouded the strategic presentations and strategy. The future was .NET everything, including Office.NET.

Deciding to show NetDocs at Forum 2000 was controversial, at least with me, and probably not many others. I was usually the most conservative about showing products or features with an uncertain path to shipping, let alone version 1.0 products built on version 1.0 technologies accomplishing new scenarios that often didn’t pan out. There were also legitimate concerns that word of a modern Office.NET could slow or halt progress on enterprise agreements, in an extremely touchy post-dotcom bubble business environment.

After many email threads prior to the event, NetDocs ended up showing some basic features, such as typing into a word processor-like screen, summing a column of numbers without launching a spreadsheet, and a calendar scenario that used XML technology to merge a personal calendar with a Seattle Mariners calendar. The session painfully reiterated the “technology demonstration” aspects of the demo and never used the phrase Office.NET, though the prominence of Office.NET on the roadmap left few dots to connect.

To mitigate the risk to enterprise agreements, the demo was said to be relevant to Microsoft’s new small business offering, briefly called bCentral. For almost another decade, the Office strategy for the web and internet targeted small business starting with bCentral and always using branding to show a distinct separation from Office for enterprise. This compartmentalized a new approach to the less risky market segment where Microsoft had more upside than downside. For big business, they were pushed to see things through the lens of Windows Server and the software housed in company data centers along with desktop Office, all available with an Enterprise Agreement. This was a defensive approach but was consistent with how customers thought about Microsoft products. Word and Excel were indispensable tools for small business, and increasingly Outlook, especially with many add-ins, was the preferred tool for small businesses to manage sales and customers. Customers could not buy Exchange and set up Windows Active Directory and file/print servers fast enough. In practice, operationalizing the transition outlined was more practical and somewhat defensive, echoing the Innovator’s Dilemma.

The Wall Street Journal was quick to pick up on the potential challenges as well in a story by Rebecca Buckman shortly after the event “Microsoft Readies a New `Office' While Renovating the Old Standby” where she wrote:

What does a company do when its single-biggest product is in danger of being eclipsed by new technologies?

If that company is Microsoft Corp., and the product is Office, it sets up a stealth team of crack engineers to dream up a brand-new version of the software suite -- while continuing to crank out the old standby. It's a tough, two-track strategy that has pitted a "today" development team against a "tomorrow" team, as one person close to the teams puts it -- and it's still unclear, he says, "how today and tomorrow will meet."

As we know now, the danger of being eclipsed did not materialize. We deliberately and somewhat nervously made that bet, as previously described. The teams were not pitted against each other, not yet anyway.

PCWeek reporter Mary Jo Foley loved the NetDocs story and wrote about it many times. By the end of the year, a few months after Forum 2000 she said in a story “Netdocs: Microsoft's .Net poster child?” where she described the product as:

Netdocs is a single, integrated application that will include a full suite of functions, including e-mail, personal information management, document-authoring tools, digital-media management, and instant messaging. Microsoft (msft) will make Netdocs available only as a hosted service over the Internet, not as a shrink-wrapped application or software that's preloaded on the PC.

Netdocs will feature a new user interface that looks nothing like Internet Explorer or Windows Explorer. Instead, Netdocs will deliver an integrated workspace based on the Extensible Markup Language (XML), where all of its application modules are available simultaneously. This interface is based on .Net technology that Microsoft, in the past, has referred as "Universal Canvas."

There was nothing sinister or even that playful about what was going on. It was just. . .going on.

We were busy, and well into building Office10. At the very least, until NetDocs was usable (self-hostable) by more people, the best thing to do was let them keep writing code and hope they stopped trying to recruit people from Outlook.

NetDocs versus Office was just going to simmer for a while. There was no way around that. Because we were in the midst shipping (eight months to go) and NetDocs now had a deadline of sorts, there wasn’t room to attempt to reconcile the products, have them relate strategically, or really do much of anything. I was happy to work heads down and not worry about it.

Putting Forum 2000 in perspective, SteveB sent an all-company follow-up email briefly laying out the strategy and timeline we were undertaking. He reinforced the huge change by referring to the strategy as “Microsoft .NET” (the space after Microsoft was important), coming very close to rebranding or even renaming the company around this new strategy:

Microsoft .NET will be delivered in three forms: a new user experience; infrastructure and tools; and a set of programmable .NET Building Blocks. This is a long-term strategy, one that will take years to execute fully, so it is critical that we all stay focused - not only on our goal but also on the daily steps it will take to achieve it.

The Office team was focused on Office10. We would worry about NetDocs later.

On to 065. SharePoint: Office Builds Our Own Server

PS: Many readers lived through Forum 2000. Some have shared their own experiences from the event, like this wonderful post from Charles Fitzgerald, Exploring Alternative History. Please share your experiences on twitter or in the comments, especially if your personal experiences bring a different perspective as they well might.

Some additional moments from the Forum 2000 video:

Back to 062. Split Up Microsoft

We received little guidance regarding how to talk about legal matters. I was never under orders to avoid speaking about the trial, though that seemed like common sense. Once the verdict came down, teammates were starting to ask questions, wondering what the case meant for Office. I knew enough to know that absent anything official, people made up their own reality. I was worried that this could become a local press issue, with people talking to friends and friends talking to friends, ending up in the Seattle Times.

I organized an impromptu all-hands in the atrium of building 17. Anyone who wanted could attend. This was the largest space we had without going off campus (also where we presented the Office10 vision). Using a single speaker audio system, I spoke into a handheld corded microphone like a lounge singer. I walked the team through the trial and what had happened, not adding anything that was not already available to the press and public, but simply tried to casually explain the facts. What was Microsoft accused of? What was a monopoly? What does a breakup order mean? The trial team was so focused on the external press that we did not have an internal process, so I did the best I could.

I had little to offer by way of details. I took a lesson from a former test leader on the Windows team—a management lesson that permeated Microsoft, perhaps to the point of becoming apocryphal. David Maritz (DavidMa) was formerly an Israeli tank commander during his army service. His unit of tanks out in the desert would sit there in a defensive posture in the dark of night. If the radio was silent for too long, each of the tanks started to worry something was wrong with the other. Panic might sweep across the unit. David said the way they avoided this was for him to check in with the other tanks and periodically let them know that everything was okay—even though he didn’t know himself. He taught us with that anecdote that even when leaders have no information, communicating something was better than nothing.

In between describing the intricacies of the legal process that would play out over years, people were worried that we were being immediately broken up, as in over the course of the coming weeks a spouse, partner, or roommate might work at “the other Microsoft.” I reiterated that there were still many things that could happen before this order could become a reality, and that much was still unclear.

At least there was humor in the situation. No one in the atrium was clear on the legal goal of splitting up Microsoft between Windows and Office. As engineers and employees on the ground, it seemed kind of nuts. Presumably, the issue was that Windows and Office were working too closely, even illegally, together and that needed to stop.

In reality Office and Windows could barely get anything done together. That situation was literally the topic of every meeting across the executive team. Different schedules, different customers, different system requirements, and more reinforced how far-fetched this idea was. More than crazy, by some measures this could have the potential to be a huge relief. Office might finally be treated as a vendor, like Lotus, which we always believed received better placement at Windows developer conferences!

For a decade there were rumors that the Office team accessed secret Windows source code that no one outside of Microsoft could see and that somehow that was an advantage. There were rumors of APIs in Windows that were secretly used by Microsoft to make Office better than competitors. There was no proof of any of this, though it made for a conspiracy theory. Back in the earliest days of a tiny Microsoft, with just tens of developers on big projects, we didn’t even have the technology to secure code from each other even if we wanted to. Ironically, many on the Office team remember diving in and trying to make Windows products work, not the other way around; whether it was Windows graphics for charts in Excel or printing in OS/2, it seemed that the advantage flowed to Windows. In the atrium, people were asking about this topic, and it brought a sense of levity to an otherwise unique situation because most were not around for the early days of Windows 2 and 3, or even Windows 95.

After a brutal series of motions, briefs, and other legal warfare, a year later on June 28, 2001, a federal appeals court reversed the breakup order, reprimanding and removing Judge Jackson and appointing a new judge. As often happens in these complex cases, the judge, Colleen Kollar-Kotelly, pushed to have the parties resolve their differences outside the court. By September 2001, the plaintiffs withdrew their effort to seek the breakup of Microsoft. By November, the case worked out a settlement, which Judge Kollar-Kotelly ruled served the public interest. There were no issues in the settlement regarding Office directly, though later when I moved to Windows in early 2006 some of my immediate responsibilities included complying with the terms of the settlement, which was scheduled to end in November 2007. We voluntarily extended that by two years, which meant the first release of Windows that I worked on included making sure it followed the consent decree.

While much speculation has gone into how the legal issues impacted Microsoft execution and product strategy, my view, even on the front lines back then, was that by far the biggest issue was not in the workplace specifically, but outside of it. Even though they had nothing to do with them, everyone on the team endured the negative comments about the company and its business practices. That’s where the litigation and scrutiny truly caused difficulty. Consider those holiday dinners and family gatherings where an engineer on the team was called to the carpet to explain or defend Microsoft. It was those endless news magazines that piled up in every household. Similarly, when recruiting college students, I frequently found myself on the phone with parents of candidates walking them through the case and the culture of Microsoft while also defending us.

Those side effects of litigation were more difficult than the specific structural and regulatory remedies.

In just a few years I would find myself on Microsoft’s other side of this case, working on Windows. I would manage the last years of the consent decree, but the real challenge was cultural and bringing us back to the days of doing what was best for customers and not pre-judging every action through a legal process we on the development team were hardly expert in.

On to 064. The Start of NetDocs v. Office

Writing about the antitrust court case and the final judgement can be difficult. The topic has been covered extensively and by my own count, of the dozen or so books about Microsoft almost all of them are primarily focused on the trial years and Microsoft achieving monopoly status. If you’re interested in the legal details or stories from competitors those sources are all better. These two sections are about the most dramatic years from the time of the initial Findings of Fact to the resolution. In all the years Microsoft was involved in litigation, before and after, this time was the most challenging. The uncertainty was high and the external forces pushing for the most dramatic outcome—splitting Microsoft—were intense. I wanted to write about the way I felt and the impact to the team.

Note: This mailing delivers two sections at once, 062. and 063.

Back to 061. BSoD to Watson: The Reliability Journey

On June 7, 2000, the verdict, known as the Final Judgment, was delivered. I read the PDF (itself a scanned copy of a fax from the legal team) on my Blackberry flying back from a Windows conference. In-flight connectivity didn’t exist, but the Blackberry magically worked over the pager network, downloading a few sentences at a time while I avoided looking like I was using prohibited electronics in the air.

The Plan shall provide for the completion, within 12 months of the expiration of the stay pending appeal set forth in section 6.a., of the following steps:

Judge Thomas Penfield Jackson ordered the breakup of Microsoft into two companies, though there was a debate over whether it should be two or three companies, after listing all the federal and state laws Microsoft violated. The punditry (and press) all but declared victory. The dragon had been slayed. Magazine covers across mainstream and industry press featured all varieties of busted and gotcha.

The litigation began way back in July 1994 when I was working for BillG as his technical assistant with a lawsuit by the Department of Justice, DOJ. Microsoft and DOJ entered a consent decree to resolve the case, but then in 1998 the DOJ sued Microsoft in civil court for violating the terms of that agreement as it pertained to how Microsoft licensed Windows to PC makers. Microsoft initially lost the case, but on appeal it was ruled that Windows 95 bundling Internet Explorer did not violate the agreement. There was a catch though. The resolution of this case did not preclude further action for violating antitrust law.

Filed May 18, 1998, the US Justice Department and 20 state attorneys general sued Microsoft for violations of the Sherman Antitrust Act. The suit charged the company with abusing its market power to impede competition, especially Netscape. Running over fifty pages, the initial complaint read like a greatest hits of emails, comments, and things we probably should not have said. All the classics were there from "We are going to cut off their air supply. Everything they're selling, we're going to give away for free” to “You see browser share as job 1 . . . . I do not feel we are going to win on our current path. We are not leveraging Windows from a marketing perspective” to “Integrate with Windows [to] increase IE share”.

The trial and subsequent rulings were low points for Microsoft. While the Office team was not part of the offending acts it was very much part of remedy being tossed about. From BillG’s deposition performance to the botched courtroom exhibits to the lack of voices of support from so many that benefitted from Windows, there were plenty of moments to feel awful about. The industry tracked the trial, but the pace of coverage was nothing like we see today with instant commentary and analysis at the speed of Twitter. By and large most employees did not follow the trial day to day and even the daily summaries that went out to some execs were not the most important thing. Even with all these negatives, the team of people at the trial were working incredibly difficult and long hours with a strong sense of purpose and pride. At an exec staff meeting, a Windows executive returning from the trial said to me they genuinely believed it was Microsoft’s “best people doing some of their best work”. There was optimism throughout the trial, until we lost.

Some aspects of the case stuck with me more than others. One in particular was the finding about bundling Internet Explorer with Windows. The judge wrote in the Conclusions of Law, April 3, 2000, that “Microsoft's decision to tie Internet Explorer to Windows cannot truly be explained as an attempt to benefit consumers and improve the efficiency of the software market generally, but rather as part of a larger campaign to quash innovation that threatened its monopoly position.”

I felt that being explicitly called out for building products to “quash innovation” was particularly brutal. With the passage of time, I have come to recognize that if you have faith in the system that governs us, it is fine to disagree with a particular ruling, but one must accept it as a fact because the system does. Even if I disagree, most everyone else will go by what the court held to be the facts determined through the process. I held (and continued to hold) a product person’s view of product development, which is that the work of product development is somehow a higher calling and done for the benefit of customers, partners, and the market. It is fair to say this is a horribly naïve view that doesn’t consider the realities of running a business in a brutally competitive market. This belief of mine would be put to the test later in my career when I found myself managing Windows.

In the Findings of Fact in November 1999, the judge found that the company held a monopoly—an important finding that forever changed how Microsoft was viewed. Following that was a lot of back and forth about the penalties, but once the company is labeled a monopoly, something was going to happen.

Viewed together, three main facts indicate that Microsoft enjoys monopoly power. First, Microsoft's share of the market for Intel-compatible PC operating systems is extremely large and stable. Second, Microsoft's dominant market share is protected by a high barrier to entry. Third, and largely as a result of that barrier, Microsoft's customers lack a commercially viable alternative. (US v Microsoft, Findings of Fact November 5, 1999)

It was in that Final Judgment in June 2000 that the judge ordered a structural remedy and the splitting of Microsoft into two companies. One company was to be the Windows company, and the other was to be made up of the rest of Microsoft including Office. The case had finally hit close to home.

Looking back this was a very long road. The investigation started more than six years earlier, after the Federal Trade Commission (FTC) dropped its case in a deadlocked vote and passed the authority to DOJ. I remember the early meetings from when I was working for BillG and how “crazy” all this felt at the time. While perhaps at the highest level the complaints did not change, the details and reasoning changed as the company saw more success. There was a subsequent case focused on violating the original settlement that lasted well into 1998. There was even a moment of daylight in that case when an appeals court ruled in May of that year that Microsoft could indeed integrate any software it would like into Windows so long as consumers benefitted. Then came this massive antitrust lawsuit on the heels of that small victory, often referred to internally as “the big day”.

It was amazing to think how much the industry changed over this time—Windows 95 and the internet came to be—and many said the industry shifts were just starting, yet the case was still there. The arguments put forth by Microsoft insisting that market forces were already at work to “disrupt” Microsoft fell on deaf ears and were viewed as self-serving. The consensus was that Microsoft had reached an invincible, all-powerful stature that needed to be corrected.

As a practical matter, once a trial started little would change for Microsoft unless, well, we lost. Losing took a much longer time (for both sides). Plus, there was always an appeal. Litigation at this level is a slog and a true test of patience. In high school we once had a guest speaker in social studies class who had a role in the AT&T breakup lawsuit which had just concluded with the breakup of AT&T. He told us he had worked his entire legal career on that case. We were dumbfounded. I now know plenty of lawyers who worked nearly their entire professional career on the Microsoft case.  

That day in June, it obviously felt like we lost . . . badly.

Microsoft had always been comfortable in the context of litigation, perhaps owing to BillG’s upbringing as the son of a prominent Seattle attorney. The earliest days of the company were characterized by a lawyerly Open Letter to Hobbyists, penned by BillG in 1976. In the letter, he argued that software should be a royalty-based product like music. The letter was controversial in a world where all the money was in hardware with freely bundled and shared software but ushered in the pure-play software company we now know. In all fairness to Bill, the hardware side of the industry was characterized by secrecy, patents, and its own litigation.

The early software industry wrestled with how law applied to this new type of product, a product required for hardware, dreamed up like art, and manifested in a proprietary digital encoding.

In 1988 (a decade before the antitrust suit), Microsoft found itself in what it would describe as a straightforward contract dispute, and what Apple would characterize more broadly as an intellectual property dispute in Apple versus Microsoft. Apple agreed to license elements of the Macintosh software for use in Windows 1.0, partially in exchange for an effort to secure Microsoft applications for Macintosh (Excel in particular). The case was front and center of the industry as Apple claimed a right to the “look and feel” of the Macintosh, which seemed to many rather unbounded, though obviously their product was unique on many levels. In a key ruling for all of software the court stated that, "Apple cannot get patent-like protection for the idea of a graphical user interface, or the idea of a desktop metaphor." While ultimately resolved in Microsoft’s favor in 1996, based on contractual terms, the litigation served to condition employees to the hurry up and wait, and the ups and downs, of the winding nature of the US legal system. For years at the annual company meeting someone inevitably submitted a question for BillG about the case and every year he would say there is nothing new but that we felt good on the merits. In between those times, the various motions and courtroom events were rather baffling to non-lawyers, somewhat like trying to watch a cricket match for the first time and not being sure if something good was happening or for which team.

Litigation was a significant part of the industry in the early days of software as the rules of the road were established for software patents, copyright, and contracts. Another closely watched case was Lotus Corporation, a giant, suing Borland International, an upstart, for copyright violation in 1990. Borland had essentially cloned the interface of Lotus 1-2-3 and expanded upon it in its Quattro Pro product, to smooth the transition from 1-2-3 to Quattro by providing a compatibility mode. This case had profound impact on the ability for upstarts to enter an existing market because whether it was user-interface or API, providing compatibility by reverse-engineering (without having access to source code or trade secrets) was key to expanding the industry. This case was decided in Borland’s favor, allowing for the copyright of the Lotus implementation but not the expression of user interface in Borland’s product.

These as with other legal matters were often discussed more as curiosities than existential risks to the company, at least among us less senior people that had no inside scoop on the matters. Even when working with BillG, a time when many of these issues were front and center for the company, he did a remarkable job of compartmentalizing the challenges. Importantly, except for the yearly question at the all-company meeting, these topics were hardly discussed within product groups or large forums and we were always cautioned to do what we believed was in the best interests of the product and not to try to think like lawyers (a cultural challenge I would face when I moved to the post-antitrust Windows team years later).

As these suits were winding their way through the system, Microsoft’s rise to the largest software company and its new power position as an unabated leader continued. From the outside, Microsoft had all the appearances of a growing software empire. From the inside, Microsoft was paranoid and felt everything was fragile and could evaporate at any moment—just as we had seen happen to the fortunes of nearly every technology company before us.

I can’t emphasize this point enough. Microsoft saw all the previous microcomputer companies, many application companies, stand-alone word-processing companies, and of course the mainframe and mini companies all but vanish in the blink of an eye, falling victim to a new generation of technology. I mean Marc Andreessen himself had predicted that Netscape would render Windows a “poorly debugged set of device drivers” (he later attributed the statement to Bob Metcalf) and Microsoft’s nemesis Scott McNealy at Sun Microsystems never missed an opportunity to ridicule the quality and utility of Office. Disappearing was one thing, but from a business strategy perspective, Microsoft was deeply concerned about having our competitive advantage removed by non-market forces. We’d seen what happens when a company like IBM or Intel are made to surrender their earned advantage (or in business school terms, their moat). Somewhere between fragile upstart and unstoppable force was the truth. It would take more than a decade from the first regulatory inquiries until resolution reaching some sort of détente with regulators around the world.

In hindsight, it shouldn’t have been a surprise that a company could become the most well-capitalized company in the world and as a result be subject to regulation. Microsoft’s views that we were just selling software at very low prices that customers and partners put to good use seemed rather quaint and naïve. The government was struggling to wrap itself around how such a huge success could come to exist without any involvement of regulators. The rise of the internet, originally funded by government research, only served as a reminder that something huge was shaping our economy and was essentially free of any government oversight. Normal issues that governments oversee such as product quality and safety, sales and marketing practices, even employment procedures had all gone unchecked. That a company maintained unfettered influence over massive societal changes was basically unacceptable.

It was always difficult to separate out the problem needing to be solved. Was it the problem of what Microsoft did? How Microsoft did that? Or was it simply the scale of success Microsoft achieved?

This mismatch of perspectives—Microsoft as a paranoid upstart just trying to keep up with the popularity of its products and a government blindsided by unregulated corporate growth and power—created a difficult situation, which required the legal and regulatory systems to resolve. Analysts, pundits, former regulators, and competitors can propose “remedies” (as if the success of Microsoft was an affliction) faster than the system can understand the problem (few in government had any expertise in software) and address it in the context of the existing laws.

Competitors complained about one set of problems. Consumers complained about another. Partners had their own issues. Economists and academics had views too. The law had its own definitions of problems. Two things were notable about this early time in Microsoft’s massive success and “power”.

First, parties were seeking a remedy for this problem that was not yet defined as we often liked to say. We lacked specifics, even with the 50-page complaint. Was it simply the scale of Microsoft? Was it that Windows had come to dominate the operating system market for PCs? Was Windows a monopoly? Were PCs to be treated like common utilities? Was Microsoft’s business model of low-price, high-volume problematic? Was it unacceptable for one company to sell operating systems and to sell applications? Or was this about some other type of product integration such as browsers and media players? These questions did not have obvious or consistent answers back then, even among third parties. The complaint said Microsoft could not integrate a browser into Windows, but few complained when Windows added networking, file management, or game graphics. There were examples of common business practices to counter every complaint.

Second, assuming agreement was reached on the problems being solved, what would the right regulatory framework be? How do you solve the problems identified? The experts in regulation (and antitrust) were themselves products of the incredibly long-running cases of IBM, AT&T, and others. In the technology industry we looked at the IBM case and saw litigation solving the problem long after it mattered—the whole industry moved on to mini-computers, workstations, and then PCs from mainframes and it seemed this case was still going on, a condition that created a view that regulating the fast-moving technology industry did not make sense the way it might for the industrial economy. The AT&T case seemed remote as it was created by the government as a monopoly and primarily involved physical cables, and much of the unleashing of competition that took place came about not because of the new regulatory framework as much as what AT&T fought for (for example, they quickly sold off the cellphone operation for a small amount to focus their win on long distance lines). But the breakup of AT&T was on everyone’s mind and that led to calls to breakup Microsoft—it seemed clear if there’s a monopoly it needs to be broken up into pieces. The debate over whether regulation simply stifled one of the most inventive and successful companies in US history continued.

This set up for a confrontation as the process wound through, with each side articulating extremes, and neither side particularly good at stating problems or matching problems and remedies. Microsoft, especially from its paranoid mindset as an upstart, insisted that it had done nothing wrong but make products people bought and so any interference was paramount to killing off innovation just as had happened to IBM. The punditry would opine about the need for choice and alternatives in products and suggest that Microsoft was itself already stifling innovation.

All of this activity changed the company’s narrative. A few years earlier, BillG was a boy wonder, the under 30 founder who had grown a new industry for the world through the magic of software. By the mid-1990s, Bill and the company were ruthless competitors who rolled over every other entity, dictated terms for the industry, and above all could enter any market and dominate. It was this fear of what Microsoft might choose to do next that drove the most extreme views of regulatory remedies—the government needed to do something to prevent Microsoft from becoming a real-world RAMJAC, from Kurt Vonnegut novels.

Regulatory norms over a new industry were unavoidable. Governments are empowered to provide oversight and there was simply no way the newest and seemingly largest and most important industry would escape regulation. It did not matter how we thought of the fragility of our industry or even how much evidence the IBM case offered as to the futility of regulating technology.

We generally learned what little we knew about antitrust in school as it seemed to originate, in industries where a physical lock on a limited supply existed. Microsoft was a company that created an industry with none of those physical barriers, pioneering a licensing and business approach never used before (open licensing versus closed integration), even among the first to charge for software. There were many times we thought it odd that laws written for an entirely different set of circumstances would simply apply. To legal scholars and regulators, such a view was naïve and self-serving. Of course, laws could apply the lawyers would tell me.

We wished someone could have made a good argument as to why technology was different than say banks, telephones, farms, oil, autos, theaters, or shoes. But technology was not different. We quickly learned that trying to tell the regulators or those immersed in the legal system that technology was different was poorly received at best, and destructive to the dialog at worst. Simply being new was not a free pass through the system, not matter how techno-optimistic we might have been. Rather, we were naïve.

On the other hand, it would have been equally fair to have asked for those calling for remedies to do a better job articulating the problem being solved. And therein lies the challenge. Jumping to remedies that so clearly did not address the problem only made one question motives and create the appearance that parties were further apart. Championing remedies that seem to be designed to simply kneecap a single company don’t serve an industry, or an economy. The political (versus legal) nature of proposed remedies only gets worse when we experienced the grandstanding in the halls of the Capitol or in endless quotes and op-eds in national publications.

Given the inevitable, but also the wide gap between parties, the process took a long time. It wasn’t debilitating as some might have suggested. There was learning, discovery, and socialization. The process was more like having a chronic condition with relapsing-remitting pathology. Long periods of time went by without symptoms, then suddenly and unexpectedly there was a flare up, like the opening of new complaint by a regulator, a country getting involved for the first time, a legal filing, or even a dramatic courtroom moment. We were in an ongoing state of “hurry up and wait” as Bill Neukom (BillN), Microsoft’s chief legal strategist during this time would tell me. The process often felt like those NASA drills where you knew at any moment the lights would turn red, steam would fly out of pipes, and a siren would sound signifying a crisis, but you just never knew when that would happen.

Even after all that, the conclusion of the case felt anti-climactic.

With these kinds of cases, the results are rarely as dramatic as early predictions and tend to be far more specific and, well, rational solutions to identified problems. Regulation does work when it is eventually created through the system. It might not be ideal for a newly formed company or for one hoping for more extreme remedies, but it ends up designed to solve the problems that regulators can solve. The problems Microsoft ultimately showed to have pertained to how business was conducted, in a sense these were understood to be problems of monopoly maintenance. Being declared a monopoly was certainly not fun, but in many ways, it was reality—Windows had, in fact, won. It was time for Microsoft to admit that.

Time would show that Microsoft’s argument that technology winning in one era will have a hard time winning in the next was decidedly true. Continuing to debate the end-state wasn’t only futile, but not done. Once a company loses in these cases, it becomes necessary to make way for the winners to own the new narrative. This might be the most difficult part to live through in the near term, and over the long term these same patterns will again play out because to the victors go the spoils, or the narrative.

On to 063. Managing the Antitrust Verdict

Happy New Year! I want to offer a short but sincere thank you to all the subscribers, readers, and sharers who have made the past eleven months of Hardcore Software an incredible experience in sharing, learning, and remembering. It is an honor to continue to share the stories and more importantly the lessons of the PC revolution as experienced in the early days.

This is a free post for the new year as a thank you but also because so many have struggled with the topic over the years. Sometimes when talking about PC crashes, I feel like offering an apology on behalf of all the engineers out there who really were doing their best.

Note: This post is best read via the link due to length and images.

Back to 060. ILOVEYOU

PCs used to crash a lot, a whole lot. PCs routinely crashing, freezing, hanging (various ways to describe a computer that has ceased to function) and losing work were the norm. Over about twenty years of engineering and iteration, the PC experience changed dramatically for the better, with vastly more reliability and higher quality. Now I recognize even typing that should make for a protracted thread on Hacker News or Reddit where everyone shared the crashes that just happened today or happen “constantly”. This is the story of going from a world of nearly universal quality and reliability problems to a literal world-changing innovation that dramatically altered the path of PC quality.

My first semester in college staffing the shared computer facilities where nearly everyone used minis, mainframes, terminals, and card readers. If there were problems, it was almost never something IBM did and almost always some form of “error between the chair and keyboard” as we used to say. When I returned to the Spring semester, Macintosh invaded the terminal rooms. My job dramatically changed. Now I was full time on Friday nights helping people to recover corrupt files from floppy disks after the new Apple MacWrite crashed and “ate” their work. After a few weeks, our team of operators started to share best practices: save your work every hour or so, save to a new file, keep papers under about 10 pages, print drafts if possible, don’t use too many fonts and sizes, and finally if you’re doing a big restructuring then save those deleted sections to another file to reuse. Using MacWrite to write a term paper at the end of a semester was, quite honestly, a risky proposition. I dealt with more than a few classmates who lost their 10-page papers hours before deadline. Lost. Gone. Evaporated. The only thing to show for the work was a useless file and an error message on the screen “Sorry, a system error occurred” with a little cartoon bomb as if humor was appropriate.

That was state of the art. Windows wasn’t far behind. By 1990 with the release of Windows 3.0, Microsoft would introduce its own brand of crashes to the world. Given the rapid rise of PC sales, it was the PC that assumed the mantle of king of crashes. Frustration with PCs crashing, losing work, or just being hard to use was entirely the norm. As we learned from the Stanford researchers who provided the inspiration for Clippy, the precision and exactness of the PC led PC users to assume when something went wrong it was their fault. The PC itself was not the problem.

It was certainly our fault. We were making the software, but we were also making crashes seemingly as fast as we were making features.

Any visit to watch someone use Microsoft Office illuminated the nail-biting, edge-of-seat, stress-inducing experience of using a PC. Our difficult to understand user-interface and faulty-software engrained a generation with defensive usage patterns: save, copy, backup, print, and so on. Even the most basic operations such as reorganizing a long memo or rearranging slides came with a preamble that involved saving the file “94MEMO2.ORI” or some other equally obscure name. Using a command that you weren’t sure of, then of course save your work first because you had no idea what might happen.

A series of changes in how we designed interface and engineered products led to a markedly improved experience and a step-function improvement in product quality.

The journey starts with the most simple and obvious command: Undo.

Most software in the ’90s only worked in one direction, making changes, or destructive changes as we called them. To revert back to what was there previously, an opposite command had to be applied. Clicking again un-bolded a word and it went back to normal, for example. Same with text pasted in one spot: delete and paste again. As programs became increasingly complex, operations were becoming more destructive. Importantly, reversing an operation could be entirely unintuitive, such as changing a chart, a notoriously complicated task or simply moving text with the mouse instead of copy and paste.

People developed ways to cope with this complexity. Prime among them was the use of saving a copy of a file before embarking on big changes and learning how to hit save often. This too had drawbacks. Keeping track of copies of files or saving and then losing old changes that might be useful—it all added to the mental overhead of defending yourself against the whims of software.

Inventing Undo seems lost to history. There were many approaches over many years including Microsoft’s CharlesS when he was at Xerox PARC and even earlier Andries van Dam at Brown University, pioneer in hyperlinks and world-renowned teacher to many and original advisory board member to Microsoft Research). Many specialized products such as Adobe Photoshop and Autodesk AutoCAD introduced undo relatively early. In Office, many people developed Undo—a feature that differentiated Office from most all other software especially considering the complexity of Office across words, pictures, and numbers. Not only did Office create Undo and implement it across products, but for each release it improved.

Office10 introduced multilevel undo across products, extending Undo to a nearly unlimited number of commands—like opening a file, making many changes, and then reverting back to the original just by clicking Undo. All of which could be undone with Redo. Undo and Redo, two simple buttons, represented thousands of hours of work, reducing untold amounts of stress and angst. Office was not the first with the capability, but it was the most widely used, most broadly implemented, and perhaps the most thorough.

Some of the best innovations, like Undo/Redo, while undramatic are both obvious and seamless and often taken for granted, missed only when absent. The early web browsers, though they touted ease and simplicity over Windows and Office, lacked the kind of safeguards being built into Office like Undo/Redo. The analogous buttons in a browser, Back and Forward, failed to work correctly most of the time, and still often don’t.

Undo/Redo also reduced phone calls to corporate help desks. As PCs were being deployed across industries and jobs, companies were pushed to provide support, and that meant on-call telephone support for employees. Windows and Office were sold to LORGs in such a way that the support burden was maintained by the customer, not by Microsoft. Much to the surprise of most individuals at big companies, they could not call Microsoft for help, and if they did they were routed back to their own company or offered a paid incident. Microsoft created a large support staff, but it was only for retail customers. Undo/Redo changed the paradigm of learning how to use Office. Instead of fear, people learned they could try something and if it worked, great, and if it didn’t work it could be undone, or redone. We reduced the risk of using features and the need to ask another person for help. Just try something and if it didn’t work, undo it. The oft-repeated sequence of undo/redo would become an substantial blip in our instrumented studies and when watching people use Office in our usability labs.

Undo/redo did not, however, change the scariest part about using a computer: a crash. Crashes happened at any time, leaving a user staring hopelessly at the screen, often a blue one, hours of work lost to the ether. The lucky person didn’t lose much if they happened to have just hit the magical Save button, but nobody ever expected a crash. The worst type of crash lost the entire file, not just the changes since the last save. A so-called corrupt file became the worst of PC nightmares—you know your work is in there somewhere in the file but you can’t get it out because Office just fails at trying to open the file. A whole cottage industry of file recovery services grew up around PCs.

The loss of work was so profound and such a part of the fabric of using a PC at work that “my computer crashed” replaced “my dog ate it” as an excuse. Crashing computers and lost files were the subject of internet jokes (we didn’t call them memes yet), newspaper cartoons, and an all-too-common film and TV plot device. Who among us has not stopped to snap a photo of a crashed kiosk at the airport or supermarket? Crashes were also the leading single subject of calls to Microsoft’s Product Support and a major cost to customers. These calls were futile at best and there was little a support engineer could offer. Whether senior government officials, expensive lawyers facing court deadlines, or famous authors escalating their way through support, there was almost nothing we had to offer them, VIP or not.

As if crashing weren’t bad enough, the way software handled a crash was, well, laughable, especially in hindsight. I don’t know how many years it took for carmakers to give up and create a red Check Engine light, but the first two decades of PC software were a journey of absurdity, making every crash a bit of a mystery.

When software tries to do something that is literally impossible the processor simply stops and the whole PC ceases to work. That’s a bug, a crash. It is the most severe kind of bug and it comes from programmers writing incorrect code. Technically a bug is any time anyone believes the software behaves differently than expected, even if it does not cause a crash. There are as many ways to crash as there are programmers. The program is trying to do something that doesn’t make sense, such as fetch some data from a location in memory that does not exist or invalid math such as divide by zero. These failures are routine, but not all programs handle them gracefully. Good programmers write defensive code. That means they always check to make sure operations make sense before trying them and after they execute. Even with the best intentions, not every line of code is defensively programmed as DougK ingrained in us in Applications Developer College—it isn’t always practical, and it doesn’t always come for free.

Crashing bugs can be difficult to find and fix. Many times, a crash happens intermittently or appears because a different series of steps are used. The bug may depend on the information being processed—how big a document is being edited or maybe the series of formatting commands, how much free memory the computer had, what else is running at the same time, or even what type of printer or display is in use. Bugs appear anywhere, not just in the code in an application. They could be in the Operating System code, such as MS-DOS or Windows, in the application, like Word, or even in the code that makes a certain model of printer or video display work, but what the end user sees might be totally unrelated to where the coding mistake happens to be.

These conditions make finding bugs an enormous and time-consuming challenge. One of the greatest programmer skills is finding bugs in other people’s code. Legendary programmers in Apps such as JonDe, DuaneC, JodiG, RickP, ScottRa, and DougK were held in high esteem, not only because of the bug-free code they wrote but also for the bugs they diagnosed in other’s code.

I created many bugs on my own before Microsoft and learned how to find bugs planted in Microsoft code during my training in ADC, but I learned about my first commercial bug during my first summer at Microsoft. DanN my lead in ADC shared (JonDe refreshed my memory of the specifics) the story of the infamous “Sindogs” bug in Excel 2.0, which was the first Windows version that shipped with Windows 2.0 about 18 months before I arrived. The bug manifested itself when an important part of Windows, a plain text file with all the system settings, was corrupted—in the file where it was supposed to say “[Windows]” it somehow was changed to “[Sindogs].” Neither the word Sindogs appeared in any code nor did any code write that string, so its appearance was rather mysterious. The bug took days to materialize and was only discovered after Excel testers ran an automated test to create and print charts over and over, for many hours. Eventually, through a significant amount of sleuthing, the team narrowed it down to a bug in drawing code in Windows, which was called when adding arrows to charts then printing them on old-school dot-matrix printers. There was a memory corruption, which changed the contents of the settings file that was in memory before it was saved to the disk. Stories were told about it for years. The Excel team even renamed their file server after the bug, and through Office 97 we connected to the server “\SINDOGS\REL” (REL was short for release) for release builds of Excel.

Imagine tracking down a crazy bug after the product was in market and trying to figure out what caused it, then multiplying that by all the possible printers, video cards, and programs involved. Looking back, it was an engineering marvel that anything worked at all. In moments of frustration, or desperation, that is what we told ourselves.

In the early days of PCs before Windows, crashes froze the computer—nothing worked, not even banging on the keyboard. The only recourse was to turn the computer off and start over, losing unsaved work and causing a potentially extreme emotional moment. In the earliest days of automobiles, drivers had to be mechanics for fear of getting stranded by flaky engines—PCs were sort of like that.

As PCs evolved, so did crashing. Windows developed a new way of dealing with crashes. Rather than freezing the computer and doing nothing, Windows 3.0 offered the first crash-handling experience known by the most friendly of names: Unrecoverable Application Error, or UAE. Instead of freezing, a crash offered a big white message box that read:

UNRECOVERABLE APPLICATION ERROR

The message offered a single “OK” button, which was ironic because nothing was actually OK.

Not only was this not helpful, it offered no solutions to fixing the problem. Useless, yes, but Macintosh did not do much better, offering a similarly useless message, albeit one with a nice sound and a newly famous graphical bomb exploding. The text of the message apologized, “Sorry a system error occurred” and the one button offered not OK but “Restart” as a worse reminder of the state of affairs. In either case, there was effectively nothing to do other than “don’t do that again,” even though no one was ever sure what they did to cause the crash.

That was the state-of-the-art PC experience until Windows 3.1 in 1991, which introduced an innovation that began a 10-year journey into making software more robust in the face of crashes. While a little nicer and equally useless to end-users, the new UAE message was at least useful to software developers. Though, in hindsight, it was laughably hostile given customers were about to lose work:

Application Error

This message had a single Close button. The sequence was meaningless to anyone who did not design microprocessors for a living. Who was this “general” and from what army? As expected, the company quickly abbreviated this as GPF and we entered a new era of tracking these GPFs and certainly talking about them in the cafeteria all the time as a new Micro-speak term.

Over time there were many variations of these crash messages. None were particularly helpful. In fact, they became more techie and contained a broader array of techno-language. Meanwhile, Apple stuck with their exceedingly simple and apologetic system bomb.

In Product Support Services (PSS) and in our bug databases (called RAID) we tracked these snippets of data. When customers called, they read this screen and jargon to the support engineer who then entered it into a tracking system. PSS would diligently record all the numbers and produce a monthly report detailing all the crashes. After a while they could talk about some of the crashes happening more frequently than others because of the similarity in the memory locations of the crash. Because so many crashes were due to settings and configurations unique to a customer environment, PSS became adept at walking through a whole series of potential changes in an effort to simply alter something in the environment to remove the crash. All of this, from the lists of crashes to the sorcery of changing settings, was entirely inadequate but it was the best people doing the best they could. There was almost nothing we could do on the development team with these mere nuggets of data as we searched tirelessly for more information and steps to reproduce crashes.

The primary problem was a lack of information, such as what steps preceded the crash or what else was running. We needed the full state of the computer at the time of the crash, not just the place it crashed. What else is in PC memory, and what else was going on at the time of the crash?

Windows needed a flight data recorder (a.k.a. black box), like on an airplane. A member of the Windows team developed a tool called Sherlock, which was just that, a flight data recorder for PC crashes. Right away, companies ran the program, eventually renamed Dr. Watson due to the discovery of a naming conflict with a commercial product. Shipping with Windows 3.1 Dr. Watson featured an icon of a doctor and a magnifying glass, cementing a new level of approachability for Windows crashes. If customers called PSS with a crash, PSS directed them to restart their PC with Watson running (which could be downloaded from AOL or CompuServe) and try to crash again intentionally to gather additional information to email to Microsoft.

The information was entirely gibberish to customers but super helpful to developers. After the crash, a file was left on the PC and could be sent to Microsoft. The internet was still not in widespread use, particularly with LORG customers, but enough people had email. From there developers and testers could combine that with some information about what a customer was doing at the time of the crash. This helped PSS to help product teams fix real-world crashes.

The relentless march of non-actionable and awkwardly worded crash messages from Windows continued with Windows 95. This revolutionary product aimed to make PCs easier to use, but it did not put a stop to crashes. Windows 95 did update the experience to put much more information in front of the user:

This program has performed an illegal operation

Following this screen was what could only be defined as a wall of numbers and letters, which a user could select and copy to email to Microsoft, after they hung up with PSS because they had dial-up.

Unlike General Protection Fault, which was funny, Illegal Operation was scary. We were telling people that their computer did something illegal. I was recruiting at a university outside the United States when a bilingual student asked me if anyone reviewed the translation of this message, because in her native language the translation sounded like the authorities were on their way to confiscate the computer or at least issue a fine.

Over time to become more friendly and perhaps offer a choice, the blue screen was, by some measures, improved as it became the primary crash experience:

A fatal exception 0E has occurred at 0028:CD0034B23. The current application will be terminated.

The modern 32-bit Windows NT product took the new blue screen to a whole new level. When Windows itself crashed the screen would be filled completely with the numeric contents of memory. The only indication that nothing good was about to happen was the “*** STOP:” that appeared at the top of the screen indicating that the computer needed to be restarted. This is by most accounts the original Blue Screen of Death (BSoD).

While an improved Dr. Watson tool was available and helpful, most customers came to loathe the BSoD experience, which became a meme for PCs. In hindsight, this was a particularly hostile design. BSoD also became Micro-speak but rose to a higher level of pop culture, appearing as the de facto method to represent a crashed computer on TV and in movies.

Windows NT, with its modern operating system design, did not remove crashing from the PC experience, but at least crashes no longer required a full computer restart. That was good. Users still lost their work but only for the program that crashed (as if that was any consolation). For IT professionals, the Dr. Watson information was always saved on the PC and could be fetched remotely and shared with Microsoft.

We were making progress. The additional information and the inclusion of Dr. Watson technology with the broad use of email and new online support meant that development teams received detailed information about crashes. Tracking down a single crash was time-consuming, but we gained an understanding of the real-world product experience.

We increased our level of commitment to eliminating crashes but were only making marginal progress. As Office usage grew, the absolute number of crashes also grew, and the sheer number of them was increasingly noticeable. We continued to double down on taking reports from PSS of the top crashes and fixing them, proudly announcing in a service pack that we removed top crashes. Still, LORGs were complaining and sharing stories of those mission critical documents that were lost in the wee hours of the morning before the big meeting or contracts that were lost just as the final changes were made, and even within Microsoft the stories of lost documents and spreadsheets were too numerous to count. Yet we also knew Office was among the highest quality—least crashing—software on the market.

We desperately needed a breakthrough.

Over the holidays in December of 1998, as we were in the final bug-fixing stages of Office 2000, KirkG (one of my first Microsoft friends) sent me a note saying he wrote up an idea. He banged out the note on his preferred 83-key Compaq keyboard from the 1980s. He had posted it on http://office10 in the total cost of ownership team section. It was shocking because I had known Kirk a decade and could not recall him writing a memo or even a long email about anything. He was a hacker’s hacker who preferred low-level assembly language whenever possible.

Two pages, the memo featured a graphic at the top of D. W. (Dora Winifred) from the animated TV series Arthur. Kirk and Melissa (who recently retired as MelBG) gave birth to their first child and thus were steeped in the children’s culture. The use of D. W. was a play on Dr. Watson (eventually only Watson). Kirk wrote:

DW is an update of Dr. Watson. Its purpose is to extract information about a crash, and establish communication with Microsoft.com. If the bug is known to have been fixed in a service release, DW will assist in installing the SR. If the bug has not been found or fixed, DW will transmit necessary information (stack trace, etc.) to Microsoft.com such that we can fix it.

Why. Customers hate crashes. Of all the things wrong using PCs, nothing is more in-your-face frustrating than a crash. Microsoft has a reputation – rightly or wrongly – for shipping buggy software, and to a large extent, buggy == crashing. We should make every effort to find and fix crashing bugs, and we don’t. We make every effort before shipping [emphasis in original], but once out the door it drops precipitously. With web-based communication, this needn’t be.

Kirk was clear, to the point, and he was right. What Kirk proposed was a sweeping change in how we handled crashes. Using the web, all of about three years old, to create a closed loop from the moment a PC crashed until the bug was fixed. Updated software could be downloaded after we diagnosed and fixed the problem.

Kirk built his idea from an architectural feature in PowerPoint 2000, an attempt to more gracefully handle crashes by giving users a chance to save a file when a crash occurred. While a huge improvement, it did not address the root cause. In a few sentences, Kirk extended PowerPoint’s idea of handling a crash straight from the customer’s PC into the debugger at a developer’s desk.

Instantly, this was a profound change in software. While excited, everyone underestimated exactly how this changed software development. For years to follow, I gave a recruiting talk to college students, detailing the innovation in Watson as among the biggest changes to programming and computer science I experienced.

It truly was.

Like any feature, going from spec (generously calling Kirk’s two-pager a spec) to a full-fledged feature was a journey. In this case, DW was the first time Office connected from a PC to the internet, to Microsoft specifically, and that had repercussions. During the late 1990s, trust in Microsoft was not exactly in abundance (trials, viruses, GUIDs, Y2K, and so on). We were gaining traction with LORG customers who would raise deep concerns about PCs “phoning home.”

Microsoft designed a feature that automatically sent information back to Microsoft, which seemed scary on the face of it. The world was starting to realize the implications of the internet and how it could be misused even while serving so many positives.

Unlike many features of Office, this feature had little by way of user experience but did require a great deal on the back end in Microsoft’s new data centers. The sheer force of will needed to stand up a set of servers and connect them to the internet was incredible—the whole company seemed to fight against it. Changing assumptions of what and how teams operated within a big company was a lot of work.

Watson was a small bit of code that was always running in every Office application. When an app crashed (it wasn’t supposed to, but if it did) Watson gathered the state of the program (what was going on in memory) at the time and packaged this up into a small minidump (also called a CAB, compressed cabinet file). In contrast to a fulldump of everything in the system, it was much smaller and could be sent to Microsoft when and if the PC was connected to the internet. The first step was to make sure CAB files were anonymous, containing no identifying information. This sounded easy. Thinking back to GUIDs and metadata covered by the New York Times, there were challenges. Basic items like the serial number of Office or the hardware address of the network card were omitted. Identifying the PC or human submitting a crash was meaningless to us, but we needed to find a way to convince people that was the case.

The memory and state DW gathered might have contents of a document entirely private to the user or information like a name and address or worse. Even though we could not trace back to a person or PC, the mere presence of this information could be perceived as troubling. EricLev, who moved to OPU after working on Word HTML, designed a user interface that allowed customers to see every single byte of information transmitted to Microsoft. It was one click away from the crash dialog. We appreciated designing for full transparency, but we knew people would be reluctant or even creeped out.

The combination of the location in memory of the crash, the program, and a few other items made for a unique crash signature, or as it was called a Watson Bucket. We enabled Watson early in the development cycle for testing. There were tons of crashes that happened then and mostly we were exercising the system, trying to understand how the flow from crash to bucket to debugger to fix worked.

We learned how quickly crash reports fell into buckets representing a single bug. The more hits a bucket received the more frequent the crash. We began to see that while there were many different crashes, the majority of them could be attributed to a small number of buckets. In other words, if we fixed a few bugs we eliminated a huge number of crashes, dramatically improving the reliability of the product for everyone.

Watson buckets were such that the 20 percent or so of most frequently occurring crashes accounted for more than 80 percent of all experienced crashes. This 80/20 rule is known mathematically as a Pareto distribution, but we lovingly called it the Watson Curve.

Soon in the development of Office10, thousands of CAB files were uploaded. Watson upended our development process. Testers saw crashing bugs in real-world experiences and, as a result, directed testing efforts to features causing the most buckets. Development managers were looking at bugs in the bug database and trying to understand the source. Was the bug found by testing, a person elsewhere, or Watson? Watson streamlined our own bug workflow so that engineers could go straight from the crash to the CAB file details to the debugger in one step. The internal website http://watson became a major part of the engineering process—anyone could visit the site and see the details on a bug (all information was unidentifiable, and the site was secured to members of the development team).

During early beta testing of Office10, Adobe released an update to their popular Acrobat product. In the update they added a toolbar to Office apps to make it easy to create PDF files. Unfortunately, there was a crash in their toolbar for those running the beta of Office10. Fortunately, this crash happened so frequently, and Acrobat was so popular, we immediately saw the Watson bucket and got in touch with Adobe. We knew about the crash because of Watson before Adobe even heard of it. Watson soon expanded so independent software makers could easily see how their software was performing in the real world as well.

Watson continued to evolve after finishing Office10, and somewhat in parallel the Windows team developed a companion service for diagnosing bugs in Windows. These systems were combined in a one-plus-one is greater than two combination to become Windows Error Reporting. The Office team continued to operate the internet service and soon became somewhat of a locus for the product groups running full-scale web services. I found myself signing off on huge purchase orders for servers and storage as we were receiving tens of millions of crashes. All of this starting from KirkG’s idea and a server under his desk.

In 2011, the results of this cross-company work received one of the first Engineering Excellence Awards, created by JonDe, to reward significant milestones. Raising the visibility even more, the work received a Chairman’s Innovation and Excellence Award from Bill Gates. Finally, in 2011, the work was published in the Association of Computing Machinery journal Communications of the ACM, as Debugging in the (Very) Large: Ten Years of Implementation and Experience, with nine authors across the company including KirkG as a top author, with OPU program manager Steve Greenberg (SteveGr) and others listed. (EricLev left Microsoft and was by then a successful founder of CellarTacker, an oenology website he started as a hobby.)

People used to ask if clicking on that “Send Error Report” button did any good. It absolutely did.

While having a flight data recorder was helpful to the product team, customers were still losing data when Office crashed. EricLev’s team designed Office10’s Document Recovery, extending PowerPoint’s innovative crash recovery to Word and Excel. When a crash happened Office automatically saved the file to a new location and automatically restarted showing the last version of the original file as well as the file just before crash. This lifesaving feature was dubbed “airbags for Office” by the marketing team when describing it to the press.

The period through building Office10 and the following releases saw an unprecedented pivot to building enterprise class software. While we started selling enterprise software with Office 97, it took time to catch up in the product team. We changed our engineering practices and built out engineering processes that were as mature as anything IBM might have used for mainframes. A decade earlier, if someone suggested we might become more like IBM, I would have been insulted.

The response to Y2K, viruses and malware, crashes, and long-term support were some of our enterprise trials. On the heels of these, Microsoft built out the internet infrastructure to deliver product updates to a billion PCs around the world. This was known as Windows Update. Since that time, everyone has taken the ability to update devices (and machines) for granted, but it was a project years in the making, designed on the heels of scaling to an enterprise company.

Having passed these tests, we had, in the eyes of customers, moved much closer to the coveted trusted enterprise-ready product organization. The industry took note. I could feel the difference with customers and industry analysts and see the difference in how Microsoft was portrayed in trade press when it came to product quality. Expectations rose, but so did our ability to deliver, and to do so proactively. Office markedly improved product quality and we could quantify the improvements with the number of bugs fixed before shipping and with the real-world crashes experienced by customers declining.

Over future releases the role of telemetry would expand dramatically, first to better creating help and how-to content and then to measuring the usage of the product at an extremely granular level (commands, keyboard shortcuts, toolbar buttons, etc.). Watson was even used to further our efforts at securing the PC by tracking crashes that were used as vulnerabilities by bad actors. Through this evolution we maintained a rock-solid privacy approach and by and large the role of this telemetry was accepted. We had gone from essentially guessing about product quality to reacting to being proactive and understanding at a very deep level how our products were used by nearly everyone. While it might sound like hyperbole today, I stand by the language I used on college campuses and this work was a huge step in applied computer science.

We executed well through Office10 M1, M2 approaching the tail of the release. We were a team of 1,500 full-time engineers at that point. Gradually, code stopped changing, and bugs triaged. Execution and precision were at all-time highs. The product was stable. Everyone was using it all the time. This felt great.

I still worried that we could spin out of control—projects at scale do that. Or maybe forces outside the company work to spin us out of control?

On to 062-063. Antitrust: Split Up Microsoft / Managing The Verdict

Viruses were nothing new in the late twentieth century, but we were about to cross a line where they were far more than annoyances. This is the story leading up to and crossing that line and the very difficult decisions we had to make relative to the value propositions in our products and that customers appreciated. It sounds easy today, but at the time it was enormously difficult. Breaking your own code is never a trivial matter. This story is also a bit of a sleuthing adventure as tracking down this virus is an important part of understanding the dynamics and context of making difficult changes—there’s always a conspiracy theory. We will follow the story through John Markoff’s excellent NY Times reporting, which was rather frustrating to me at the time.

It is somewhat odd to be writing about viruses for computers when we face such a tragic biological virus. At the same time, we are still unwinding from an incredible zero-day exploit so the lessons herein seem quite relevant as well.

This post, slightly edited, appeared as an excerpt in Fast Company in May 2020, on the 20th anniversary of ILOVEYOU with a related Q&A Steven Sinofsky lived Microsoft history. Now he’s writing it. Many thanks to Harry McCracken (@harrymccracken), global technology editor at Fast Company.

Back to 059. Scaling…Everything

One morning during the first week of May of the new millennium, I received a call at my apartment while I was getting ready for work. I heard a reporter tell me their name, and then listened to hyperventilating and (apparently) proclaiming their love for me repeatedly, “I love you. I love you.”

That’s what I heard, anyway. The call. A reporter. Early morning. It was all weird.

Unfortunately, LOVE broke out all over the internet. Over the span of a weekend, inboxes around the world of Outlook and Exchange email users were inundated with dozens of copies of email messages with the subject line, “ILOVEYOU.”

I learned from the reporter that the LOVE email incident was deemed so serious that the PR lead gave them my home number and simultaneously sent me a briefing via email. In the era of dial-up, I could not read the email and talk on the phone because I only had one analog phone line at home. I had no idea what was going on, so I agreed to return the call after I dialed up and downloaded my email.

That’s when I realized the magnitude of the issue.

For all the positives of the PC in business, IT professionals still wrestled with the freedom of PCs, not only the freedom to create presentations and spreadsheets but the freedom to potentially wreak havoc on networks of connected PCs because of computer viruses. Viruses were hardly new, part of PCs from the earliest days. As a new hire in the Apps Development College training, I completed a unit on early MS-DOS viruses. The combination of many more PCs in the workplace, networking, and then email created a new opportunity for those wishing to do harm with viruses. By their nature, and by analogy to the word virus, most viruses are not fatal to a PC, but they could cause significant damage, loss of time, and take a good deal of effort to clean up.

By the late 1990s even amongst government and academia, the risk posed by viruses to the nation’s infrastructure were front and center. Fred B. Schneider, a Cornell faculty member (and former sponsor of our chapter of Association of Computer Science Undergraduates), chaired a working committee going back to 1996 on the topic of trustworthy information systems (the word trustworthy will make an appearance in much of Microsoft’s reaction to the stories shared here). The committee included faculty from many universities and representatives from across industry, including Microsoft’s George Spix (GSpix). The work was convened by Computer Science and Technical Communications Board of the National Research Council whose members included Butler Lampson (BLampson), Jim Gray (Gray) and Ray Ozzie. The effort resulted in a 300-page report that was widely influential, once the commercial world caught up with these challenges.

Enterprise IT was increasingly uneasy about viruses and had the expectation that Microsoft must do something. The openness of the PC platform was a hallmark feature responsible for the utility and breadth of the PC ecosystem, even if some bad actors (as they were called) might exploit that openness. Microsoft took a laissez-faire approach to this annoyance.

Office shared this permissive attitude until the mid-1990s and the rise of networks when sharing files became common. With everyone connected, an annoyance morphed into a virus that shared itself automatically between computers. A virus infected Microsoft Word 6.0 called WM/Concept.A—viruses often had cryptic names. In this case the WM stood for Word Macro, and presumably Concept referred to the fact that this was testing a new concept for viruses.

This was a new type of virus. It did not exploit bugs or programming mistakes in Word. Concept used Word macros as they were designed. Macros were present in software for ages, going as far back to the MS-DOS days of WordPerfect and Lotus 1-2-3. Macros, called programmability or extensibility in Microsoft lingo, were a prized accomplishment and something BillG believed in pushing extensibility for all products. Programmability made a product sticky when customers invested time and effort to write macros. Macros were used to automate repetitive tasks, for example, if a company wrote similar letters to customers for past due notices, one could write a macro, which generated letters with all the right address fields and salutations. Another example might be to automate collecting sources in a document and creating a bibliography. The uses for macros were endless and an entire industry developed consulting and training in Word automation. Word macros used a derivative of BASIC.

Some macros were written to automatically run as soon as a file was opened, making systems appear more automatic to end-users. The WM/Concept.A virus took advantage of a combination of features to create an exceedingly simple and yet maximally annoying experience including: macros, automatic start-up, and networked file sharing.

The original author of WM/Concept.A probably crafted a tantalizing document with the virus and shared it on a network knowing others would open the document, sort of a patient zero. Once anyone opened that document, every document they created or opened became infected. Any document they shared became infected and infected every PC that opened that document. Like the old shampoo commercial, “ . . . and they told two friends, and they told two friends, and so on, and so on.” Well, that is exactly what happened. WM/Concept.A circled the globe relentlessly.

Contrary to what was commonly believed, successful viruses didn’t usually do anything harmful like delete all your files or format your hard drive. If they did, the viruses failed to spread and serve their purpose. WM/Concept.A did only one annoying thing other than propagate, and that was to display a message that looked like a broken Word error message—a simple window with the character “1” and button labeled “OK.” That was it. The message showed up once during the initial infection.

While there was no direct harm to PCs or any documents, the obvious implications of this Concept virus were that viruses could easily spread and, if the author wished, could do significant harm or at the very least truly interrupt workflow, and at most, do heinous things like delete text at random times or worse. Removing WM/Concept.A from an infected system was a chore. A cottage industry of virus removal and disinfecting was born, as was a cottage industry of using the Concept techniques to do more harm.

With this virus, General Manager of Word, PPathe (Blue), decided to take the first steps on Microsoft’s antivirus crusade. Blue and team changed the way macros worked. They added a warning noting that a document contained a macro installed, and also made it more difficult to run macros automatically when simply opening documents. These were small steps and were, in spite of the awfulness of WM/Concept.A, greeted with much pushback by fans of Word and IT managers because these changes broke business systems and workflows. Blue stood his ground and Microsoft’s PSS team proactively worked with customers to get the word out, so to speak.

WM/Concept.A was our first lesson in the incredible balance between building an extensible and customizable system and the need to maintain security and reliability on PCs, and how customers push back when making products more secure means making changes in how they work. But the virus scared me in a much broader way. I wrote another frantic twenty-page memo, Unsafe at Any Megahertz. The title was a reference to the Ralph Nader book that shook up the auto industry in the 60s. It was meant to be a call to action for our product engineering. The software industry grew up from the counter-culture 1960’s. Steve Jobs wore no shoes. Bill Gates hacked his high school computer system. Both were college drop-out. What the PC industry lacked was any formal notion of what it meant to be a software engineer. My clarion call was that our lack of formalism, reproducibility, and external validation would only result in heavy-handed government regulation. I pondered a sort-of Underwriters Laboratory for software. That would be scary. I never sent the memo for fear it would be viewed as just too controversial in a company and industry that was the epitome of informal. Instead, the memo was a good exercise for me in writing is thinking and it made me double-down on how Office would operate with respect to product quality.

The ability for bad actors or even pranksters to wreak havoc on the growing and newly connected PC infrastructure became a major liability for Microsoft. Yet our products were behaving exactly as designed and customers appreciated those design patterns—extensibility was a major selling point of Office and a major part of our product and engineering efforts. As soon as we introduced the functionality changes, new viruses were created that circumvented what protections were in place.

While in the midst of creating the vision for Office10 in early 1999, our PR firm, Waggener Edstrom (often called WaggEd), received an inbound request from John Markoff of the New York Times. Markoff was one of the most respected reporters in technology with a deep history in reporting on all aspects of the industry, especially on intensely technical topics. He received broad acclaim for two books on hacker culture and in particular the effort that led to the identification and capture of Kevin Mitnick, who was convicted of several computer-related crimes.

In our case, Markoff inquired about a feature in Office 97, Office 2000, and a related feature in Windows 98. He was following a tip he received, which we later learned was from a programmer, well known to Microsoft, who built tools for MS-DOS. He was told that documents created with those versions of Office seemed to have been stamped with what the tipster referred to as a “digital fingerprint.” Related to this, it appeared as though the programmer discovered that Windows 98 was also creating what amounted to a fingerprint for an entire PC using similar technology. The combination meant both documents and PCs seemed to have fingerprints.

Kim Bouic (w-kimb, now Barsi) of Waggener Edstrom called me right away. Kim was the PR executive leading the Office business and was exceptional at handling crisis situations like this. On the one hand, she talked me down from lecturing reporters about how they didn’t understand, and on the other gently reminded reporters that things might be more complex than they seemed. Her skills were needed more than ever as “digital fingerprint” was rapidly becoming a crisis.

Markoff was chasing two lines of inquiry and they were leading to the same conclusion, which was that Microsoft created some sort of fingerprint or serial number in Windows and Office—if so, this constituted a major risk to privacy because documents and computers could be traced using this technology. As if this weren’t enough, Intel announced the new Pentium III chip and it contained a unique serial number, which Intel said was for security use but fed right into the narrative of serial numbers for tracking PCs. To make this all the more ominous, in one of the early discussions with Markoff on Windows someone referred to the technology as a GUID (pronounced goo-id), an acronym for globally unique identifier.

Big Brother was suddenly part of the story.

GUID was the name of the Windows functionality that did, in fact, create what was intended to be a unique number—something useful for a broad range of programming tasks. The origins of Microsoft GUIDs were buried deep within the system but the value of a number that was for all practical purposes unique beat people trying to create a number on their own, something that eluded computer scientists for years—in fact, the origins of GUIDs went back at least to early 1980s Open Software Foundation work and were originally called UUIDs, for universally unique identifiers. To create the unique number, the GUID creation function combined several pieces of information. One of them was the serial number of a network card called the MAC address, which was relatively unique and required for networking. That serial number remained visible in the GUID. So, someone with a GUID and access somehow to serial numbers of network cards could identify a computer. The fact that there was no database of MAC addresses or even that anyone kept track of them, or that MAC addresses were part of the lowest level of how the internet worked, were all facts lost in the moment. Much to Kim’s frustration, I continued to try to explain.

Pro tip: If in a PR crisis you find yourself explaining some deeply technical thing, stop.

I could not stop. Kim was annoyed.

The specifics of how Microsoft ended up using GUID technology made this look bad, and that was what Markoff was on to.

This conspiracy theory ran deep.

In Office 97 we introduced hyperlinks as a native feature inside documents. It was part of our push to make Office great for the web. From within any Office document, clicking a link opened the browser. We were especially interested in links between Office documents on a web server. One problem with Office documents is that if files are renamed or moved then the link breaks. The WWW was already well known for broken links. In the corporate world, with reorgs and project name changes, files moved around a lot. We decided that by using FrontPage on the server we could keep track of links used in Office documents and detect when a file was moved, and then repair the links. We thought this was a great way to prevent what was becoming a huge web problem of broken links. We needed something more than a file name, since in a company files might frequently have the same name. A clever idea was to use the new feature of Windows to create GUIDs, and when a link was created to a document a GUID was also recorded. Links in HTML used the file and folder name, so if the file was moved or renamed the link broke. Having a GUID also gave us a chance to fix the link and find the file using FrontPage server. We thought this was a useful and solid plan.

Will Kennedy (WillK) was the development manager on the feature in Office 97, though he moved to Outlook shortly after we began Office10. An Alabama native, college hire, standing 6-foot, 6-inches tall, Will was the epitome of a calm development manager. I forwarded him some of the mail from Markoff describing the feature. He walked through all the Office code with the test team and ascertained we did store the GUID in a document (to further the conspiracy, the GUID was not visible to end-users in any way and was hidden in the file). However, as Office 97 progressed, albeit late, we never implemented the fix-up feature but the GUIDs remained. That seemed benign at the time. Will said it was trivial to remove the code that wrote the GUID to files. He prepared a quick fix for Office 97.

Simultaneously, but coincidentally, Windows 98 implemented a product registration tool that, in the process of registering the PC, collected a set of information about the hardware (how much memory, disk space, CPU, etc.). It was also optionally collecting personal information like any registration process. At Microsoft, the hardware information went to the development team and the optional personal information went to marketing’s customer database. As it turned out, the bits of hardware information needed a unique identifier. Windows chose to use a GUID.

GUIDs contained that network MAC number, which could link the Windows 98 registration and documents created with Office, no matter where those documents ended up being distributed.

The tipster, and Markoff, came up with a scenario by which Microsoft could, if it wanted, maintain the capability of knowing if a document was created on a PC and who registered that PC. In fact, the theory implied that given a random document, it might be possible for Microsoft to determine who created it.

All Microsoft needed to do was connect each of the new databases, and no one could stop us.

Kim needed me to get on the phone with Markoff and explain this theory away. I told her the theory was baseless, and therefore harmless, plus, we would never do what was being suggested.

But it was a conspiracy theory and those can’t be explained away.

Microsoft at the time (early 1999) was not exactly the most loved company and certainly not the most trusted, especially by those outside of technology. Combined with a lack of trust was a perception of power that rivaled governments.

On the other hand, the idea that somehow the Windows and Office teams could connect their databases and execute this scenario seemed laughable to me. We had enough difficulty connecting our bug databases and sharing code, even though we fully understood and had a use for those things!

Kim scheduled a call with Markoff. She reminded me once again to take Markoff seriously, and that I could not dismiss his concerns no matter how wild they were. On the call, I walked through the feature in Office, but there was no way to deny what Markoff was asserting. Microsoft did have these databases. There was a serial number of a network card in a GUID. Files had GUIDs. The story was ultimately filed and was, according to Kim, factual and accurate. As was almost always the case, I took the story personally. Kim reminded me that it was a win, considering where the story started and where it could have gone.

We issued patches to Office. We changed Office so that it did not create GUIDs (that we never used) and we also released a tool to remove GUIDs from existing documents. Windows also changed the product registration tool and the way the GUID creation capability worked. GUIDs are widely used today on the internet in browsers, websites, and mobile phones in almost every application.

The GUID story was also the introduction of the word metadata to the general public, data about data, not generally seen by end-users, but used to describe data. The GUID in an Office document was an example of metadata. With the rise of web browsing, web browser cookies, and mobile phone records, public awareness was just being raised. Privacy advocates were in force challenging metadata collection and analysis. We were truly entering a new era of privacy.

Whereas WM/Concept.A was the Office team’s first experience in dealing with networked viruses, GUID was our first experience dealing with privacy. These both came as we were planning Office10 and changed the way we thought about security and privacy. In fact, security and privacy moved from defensive capabilities to main tenets in our product vision.

We weren’t finished learning.

Just a few days after the New York Times ran Markoff’s story on GUIDs, I received an interesting message in Outlook with the subject line “Important Message from Jon.” And another interesting message with the subject line “Important Message from EJ.” And another. And another. In fact, my inbox was filled with “Important Message from . . .” messages.

That was no good.

Each message contained only the text:

Here is that document you asked for...don't show anyone else ;-)

and a file attachment called LIST.DOC.

I wasn’t the only person getting these emails. It felt as though everyone with Office running Outlook was receiving them, seemingly at the same time.

Then, suddenly, I received no more messages. I couldn’t send messages either. Email was down.

Microsoft shut down email service as did companies and email providers around the world. The internet was under attack by a virus, a replicating email virus. This virus was quickly analyzed by many across the internet—it was named W97M.Melissa.A as it left a signature containing the name Melissa on an infected PC. Melissa was a Word macro virus much like WM/Concept.A but one extended to use Outlook in order to replicate.

Office had been weaponized. This virus also did not harm the infected PCs but was generating so much email that servers were getting gummed up in what is called a denial of service attack, or DoS. The system administrators in charge of mail servers were angry. More importantly, for the first time perhaps hundreds of thousands or millions of white-collar workers were without email all at once, heading into a Monday morning of a work week.

If there had been any doubt, we immediately learned how important email was to the workplace.

Our customers and offices around the world were angry.

News reports were everywhere (though reporters resorted to phone calls to report) and the number of mail servers impacted was in the tens of thousands, which was hundreds of thousands if not more PCs. This was huge.

What was going on? This was a new type of virus. It introduced the term worm to the general population, named such because it could automatically spread itself to other PC users without any action, worming its way around the internet. In the press the terms worm and virus were used interchangeably. Once released into the wild via a deliberate simple email, Melissa virus spread when a recipient opened the file attached to the “Important Messages.” The message was designed to look familiar and so the file was almost always opened.

That was social engineering.

The attachment contained a Word macro and was written in a way that not only bypassed any protections added to Word after WM/Concept.A but immediately disabled the macro protection features. In this sense, it was the same as the Concept. When using any mail program but Outlook, Melissa was like Concept both in how it spread and its annoyance level (high).

If a user was running Outlook, then Melissa went one step further. The code in the Melissa virus used the macro capabilities of Outlook—yes, Outlook had those too, and customers really loved them—to automatically send the same “Important Message” to the first 50 people in the Outlook address book. Instead of telling “two friends,” each infected PC was telling 50 and each of those who opened the attachment told 50 more. That’s how the virus spread across the whole planet in a weekend.

If there was any humor to it (there was not), after mailing 50 people the virus, the code checked if the current day of the month was the same as the minute. If by chance it was, it added the following text to the currently open Word: “Twenty-two points, plus triple-word-score, plus fifty points for using all my letters. Game’s over. I’m outta here.” This made no sense until there was a deep dive into where Windows kept program settings (called the registry) where it recorded that Melissa infected the PC. There, text could be found reading “Kwyjibo.” Together those were a reference to an episode of The Simpsons, “Bart the Genius,” where Bart tries to cheat at Scrabble with that word.

Hunting down the propagators of viruses and worms was an internet hobby and a profession going as far back as Robert Morris and the infamous worm unleashed from Cornell in 1988 (while I was there for my first homecoming!) and sleuthed and then documented by Clifford Stoll in the incredible book, The Cuckoo’s Egg: Tracking a Spy Through the Maze of Computer Espionage. Immediately the internet tried to find clues to the origin of Melissa. In some ways this was similar to the Centers for Disease Control trying to find patient zero. With email this was possible because of the accurate time stamps and journey information in every message—the digital DNA of a computer virus.

Melissa offered up one other clue, unbeknownst to its creator. The LIST.DOC file had a GUID, the kind John Markoff wrote about (the same feature that we had just removed from Office). The tipster assisting Markoff, who we then knew to be Richard Smith, the CEO of Phar Lap, found the GUID and posted it online with the virus code. A graduate student in Sweden said the code looked familiar and pointed Smith to a user named VicodenES. Smith then began to connect the network card address (the same one he complained about as part of a GUID) and records of internet servers. Eventually, Smith was able to connect the whole trail of network card addresses to the actual creator of the virus. He was arrested in New Jersey at his parent’s home a week after releasing the virus.

The metadata in Office 2000 made that possible.

I wish I made that up.

We assisted IT managers and sysadmins in getting their systems back online and removing the virus. This was a criminal investigation, and for the Office team this was the first time we were involved in one at this scale and in real time. The week was filled with remorse, grief, anger, and finally, when we heard that metadata was involved in finding the criminal, pure disbelief.

The macro capability of Outlook, as used by Melissa, developed a huge following and was a strategic aspect of the product. We were in the same spot as we were a few years earlier with Word—a key feature enterprises valued was being weaponized by virus creators. For Word, we added a series of warnings and administrative controls. Our first steps in dealing with Melissa did the same.

As soon as we implemented these and put out updates, we heard from enterprise customers and the community of consultants, authors, and the Microsoft Most Valued Professionals (the group selected by Microsoft’s support professionals to represent the broad external community). We “broke” their solutions built on top of Outlook. Whether these were time management, scheduling assistants, customer relationship management tools for salespeople, or email automation (to name a few), the user was prompted by Outlook every time macros ran. It was annoying, but it needed to be that way because we did not have another readily available solution.

When people using software are in a flow going through some task such as from opening email, to booking tickets, to opening a program, to browsing web pages any warning messages that pop up are essentially ignored, and therefore meaningless.

This lesson was learned repeatedly by every generation of software. A warning message simply got in the way. No one reads text when there is an OK button right there.

As with Word, there was a gradual chipping away of the extensibility of Office. Our products (and customers) were put at risk due to an increasingly connected world. The design approaches we took that worked fine for tech enthusiasts no longer worked for typical office workers with relatively limited knowledge of the inner workings of PCs, and especially not for the administrators who supported them.

Once a virus is thwarted by any means, the community of bad actors works to find similar patterns to exploit, but ones that work around the fixes. In the meantime, an even larger community of copycats duplicate the existing exploit and take advantage of unpatched software or software protected by relatively unsophisticated antivirus tools that did not pick up on small changes to the pattern used. That’s how viruses work.

We briefly secured the product.

The press in the first week in May 2000 when I received that exhilarating call at home on Sunday morning were reporting billions of dollars in damage, computer users around the world blocked from using email or even working. The reports cited the previous Melissa exploit and the much earlier Concept virus in Word, and many in-between.

That’s when I realized the magnitude of the issue.

In reporting, once there are three points of evidence then there is a trend. In this case, the trend was the escalating risk of using Office and the escalating costs to business IT professionals maintaining corporate desktops—TCO, our old friend total cost of ownership, was again a critical issue.

In an online story posted the first evening of the spread, industry “experts” anticipated that by morning half of all PCs in North America became infected and more than 100,000 mail servers in Europe were infected or taken offline as a precaution. The United States Senate was infected, as were important news outlets such as Dow Jones. The infections reached worldwide telecom and television outlets in Denmark and employees of Compaq Computer as far away as Malaysia.

The impact was profound. Billions of dollars in immediate lost productivity and money spent to eradicate the virus. Customers were livid. If and how we responded to this was clearly going to be a test of the empathy for the pain customers were experiencing.

The creators of the ILOVEYOU worm, dubbed “Love Bug” in the widespread press, exploited a hole in the warning messages that ran when Outlook’s data (such as contacts) was accessed. As with Melissa, the worm used the contact list and replicated itself, but instead of just the first 50 contacts it automatically (and silently) sent mail to all the contacts in an address book. This infection also installed itself on the computer so it continued to run all the time and did damage by deleting files and replacing them with copies of the virus. The infection was started by an email attachment with the name “Love Letter,” which was a hidden program and not a letter at all. Any email program would have been vulnerable to this method of transmission, which simply required the user to open the file on their PC, but Outlook was not only the most prominent, it was also the most easily programmable.

It was bad. Really bad.

The team was trying to figure out what to do and began exploring options.

Microsoft Office was having a Tylenol moment. While the human suffering of the computer virus was dramatically less than that of the 1982 product tampering that left seven people dead from poisoning, the brand suffering was comparable. Could Outlook be trusted? Could Microsoft? Tylenol’s parent company, Johnson & Johnson, took unprecedented and drastic measures to save lives and rebuild consumer confidence in the brand. They removed all medicine from distribution and encouraged the destruction of all pills. In addition, the company diagnosed and improved their systems, including the development of tamper-resistant packaging. In doing so, they developed the modern playbook for crisis management.

Acting with uncharacteristic haste, the US Congress House Science Committee, Technology Subcommittee held hearings on May 10, 2000. Witnesses testified about the “love bug” computer virus that infected over 10 million computers worldwide, shutting down Internet servers and corrupting files. Testimony centered around how the virus spread so quickly, the impacts and damages, and what steps could be taken to prevent similar attacks in the future. The witnesses were third party experts, and not from Microsoft. The testimony, I believe, stands in contrast to the hearings in the present day on social networks in how relatively calm and rational, even in the midst of a crisis, the dialog was. Still, it is worth noting that the hearings made numerous references to the ongoing investigations of Microsoft and the global market "power" the company maintained.

Like so many crisis situations in management, at first managers (like me) think they will show up and save the day with some brilliant idea that no one thought of. Failing that (as is almost always the case), the next approach is to take several options and combine them into what seems brilliant but is ultimately unworkable. That’s assuming you don’t show up and just wish the whole thing wasn’t happening. That too is never the case.

Rob Price (RobPr), Outlook’s PM leader, and WillK led the discussion. Their view was clear. First, we would disable sending a bunch of file types of attachments that people routinely sent or opened if they showed up in an inbox. Essentially, this meant not sending executable code or files that ran when opened. Second, we would guard Outlook such that any programmatic access to the address book or attempt to send email silently generated a warning and disabled access. Finally, although wonky, we would treat all email as untrusted, which basically meant no matter how code was snuck into email it did not run without a lot of warnings. We would effectively quarantine email messages and isolate the user’s important Outlook data from any code.

These actions or guards could be enforced and customized by administrators in large companies.

The team wanted to talk about exactly how much “stuff” would break in the process. KurtD and Martin Staley (MartinSt), Outlook’s test manager, said that no matter how much or how little we broke customers would complain we broke either too much or too little. Some things were super simple. For example, large companies compressed their files before emailing them as attachments to save storage and bandwidth. A common way to send them without requiring a separate program to decompress them was to have the compressed file sent as an executable file, which was automatically decompressed and saved on the local hard drive when opened as an attachment. This extension to Outlook was popular. And it would be totally broken, rendering attachments invisible to recipients.

The debate was not whether to make these changes as soon as possible, but whether we should even enable companies to turn them off or somehow reduce the scope of protection. As veterans of the past few rounds of viruses, there was reluctance to enable IT pros to reduce the protections on a PC. Their assessments weighed the risk of an important boss or stakeholder not getting work done with the support costs across an organization if a virus were to surface. We knew with any option that some percentage of customers would side with more pain in after-the-fact remediation than the pain of prevention. It was the wrong tradeoff, but the kind that is often made in IT organizations when they are put on the defensive because of weaknesses of the products they are supporting.

What the team really wanted was permission to cause the pain. They knew what needed to be done but knew there would be pushback. They wanted to know I would support them. For that, I did not hesitate given how much pain we had already caused and how much they clearly understood about the problem and solution. There was so much going on to suggest this significantly undermined the whole value proposition of Office that I wondered if we would have to introduce Outlook-free versions of Office (and reduced pricing—everything always came back to pricing).

In four weeks, on June 8, 2000, the team completed the patch and made it available for download—the now infamous Outlook Email Security Update. It went out for Office 97 and 2000, Outlook 98, and Outlook 2000 and for all sub-versions of those products all around the world.

Four weeks might seem like a long time, but cleansing PCs of the problem was time-consuming and occupied IT. The antivirus vendors and email security products did their part. We notified PSS and field sales. Marketing prepared a library of materials, as did Support, who wrote detailed technical articles for the Microsoft Knowledge Base. We issued a long “interview” with me, as a news release, detailing all the fixes. We did calls with the major press outlets.

The rollout was bumpy. With every virus, the knowledgeable PC enthusiasts tend to take a blame the users stance when faced with an update that diminishes PC capabilities. We saw this with both Concept and Melissa. In the case of LOVE, features like mailing around code were precisely what enthusiasts did frequently, so they were rather irate. In forums they complained, “Who opens attachments from people you don’t know?” People are busy and expect PCs to work. They don’t view using a PC in the same vein as walking down a dark alley in a strange city. Quickly, the community took to trying to find workarounds for the security changes, but to no avail. Several declared that end-users should change a few IT settings we did provide to return to “normal.”

Enterprise admins behaved as expected. Some optimized for the near term. Others took the pain in changing workflow and incompatibilities. Outlook, with the email security update, was the new normal and eventually accepted. While in hindsight it all seemed easy, the idea of breaking an important ecosystem, for a new product especially, was antithetical to Microsoft’s focus on compatibility. What the team proposed and then delivered was gutsy.

Office continued to have fewer and mostly less severe viruses for decades to come. At least a part of that was due to the Outlook Email Security Update.

Tech enthusiasts, IT Pros, and even our beloved MVPs complained for years and wrote many articles pejoratively referring to the Email Security Update as they adjusted to a new normal for Outlook.

The world moved on. And it was a bit safer using Office and Outlook.

On to 061. BSoD to Watson: The Reliability Journey

It is one thing to change a product in order to meet market needs, but entirely another to change the culture. Scaling the teams and processes to meet the needs of our high-paying enterprise customers was another effort, and one that came right when most external indicators made it seem like we were doing everything right, thus making change more difficult. In practice, we had significant challenges meeting the needs of enterprise customers—product, support, quality, and overall enterprise-ness. We needed to bring not just our software to enterprise readiness, but our organizations. The best way to do that is to live through a few crisis moments, whether self-inflicted or not. Microsoft never met a crisis it didn’t enjoy to some degree.

Back to 058. That Dreaded Word: Unification

Note: This post is trying out the new free preview feature.

Increasingly, our newly minted enterprise customers grew frustrated with Microsoft’s readiness as an enterprise partner. When an enterprise customer is frustrated, they describe the company as a vendor rather than a partner. A vendor is what we used to be, or so we thought. We had to up our game.

Customers were sitting on a mismatched pile of software from Microsoft, some of which was by all accounts being ignored by us in the product groups. There were ATMs running OS/2, which we long ago turned over to IBM. Banks in Europe were running Word and Excel on OS/2, which we made as essentially a one-off. One of the leading business magazines had a publishing system that used Word 2.0 and Windows 3.0, from the early 1990s. That was 8 years ago, an infinity in software years. We had moved on from those products. Our customers had not.

Business is business and Microsoft needed to change. While it took us more than a year of meetings, in 2002 we finally announced a Support Lifecycle Policy. To much press and customer outreach we announced that Microsoft products would now have a minimum of five years of product support. In the Microsoft blog post describing the new policy, the CVP Product Support Services, Lori Moore (LoriM), explained that we “worked closely with customers, business and industry partners, leading analysts, and research firms”. Noticeably absent from that were the product groups that would be on the hook to deliver bug fixes and updates to customers covered by Software Assurance as part of this new policy.

It should be no surprise then that Microsoft’s fully distributed and empowered product groups interpreted this policy with differing levels of enthusiasm. Did it apply retroactively? What about products designed for consumers? What if we have multiple releases over five years? What if product releases took more than five years? What if Exchange has one interpretation and Outlook another? The intended effect of this effort was to do good by enterprise customers.

Instead, it was just an early step in making the transition to a new operating model. Customers interpreted the Lifecycle as a license to deploy what they could or would and then freeze the infrastructure for at least five years. Imagine in our fast-changing technology world, just freezing a company’s information infrastructure. Five years was the minimum. Customers could even buy longer support contracts and they did so in droves. It meant that even five years from now, product groups were on the hook to support products that no one was actively working on.

That said, in Office we created a team that grew significantly over the years of full-time engineers dedicated to what we termed Sustaining Engineering. The team, Microsoft Office Sustaining Engineering (MOSE), originally envisioned by Excel test manager Carl Tastevin (CarlT) and then led for many years by former Word test manager Jeanne Sheldon (JeanneS), prided itself on being a direct connection to customers. They would spend the time to understand the context and customer environment driving problems and were reliably the best source of information anywhere for how the product was performing in market with customers. The team was not an outsource model for quality and fixes because the product team developers that created the problems were accountable for fixes. It was a fantastic model for us and one we would later replicate in Windows which had gone full outsourcing after Windows XP shipped, which turned out to be less than ideal.

It would not take long, not even eighteen months, and the policy would again be updated. The feedback was less than positive on the first policy, mostly because it was uneven across the company and hardly long enough compared to traditional enterprise partners. This time the product groups were deeply involved in the process, in what grew to a major corporate undertaking. Since the first announcement, most teams released major new products, and all had built out engineering teams that were able to handle the new volume of support issues from enterprise customers. I attended many of the meetings of the product leaders who were trying to agree on a more uniform policy. The group was working well together but not converging. Microsoft’s multiple billion-dollar businesses each serving distinct customer types and each with substantially different release schedules were struggling to arrive at a more uniform policy that was also longer. Some product groups aimed for very long support terms and others wanted nothing to do with previously released products. Office was in the middle. The position depended entirely on the primary buyer such as IT professionals, OEMs, or general corporate buyers.

In a moment of frustration for how long this was dragging out, I, probably the most senior product group person in the room, suggested (with some force) we should all settle on a ten-year support offering. This would make customers happy and would put this issue to rest. Our competition, such as IBM, supported products for decades. How much support would customers really expect nine or ten years from now for a product long forgotten? What a huge mistake I made. In hindsight, I was intoxicated with the idea of making sure the field (and SteveB) knew we “got it” in the product groups and also over-confident with the idea that we could execute on such a commitment. I really can’t believe I thought this was a good idea—not only for Microsoft but for customers to rely on it. Think of all that changed ten years prior or all that would change from 2000 to 2010, especially considering how most customers were still far away from deploying email and the internet. Yet, SteveB and the field support and sales teams were incredibly happy and thankful for the teamwork and support from the product groups it took to make the updated policy a reality. Ten years.

You could feel the bubbles and the froth everywhere as the end of the millennium approached in the Fall of 1999.

Microsoft was in the midst of an all-out frenzy of activity in late 1999. First and foremost, the company’s main business drivers were firing on all cylinders. Combined, Windows and Office were on pace for almost $18 billion in revenue and enormously profitable. Microsoft was growing at a rate of nearly 30 percent per year in revenue and would end the year with over $20 billion in cash on hand—truly unfathomable numbers. Wall Street responded with an unheard-of market valuation of over $600 billion and a position as one of the (if not the) world’s largest and most successful companies. BillG’s wealth was valued at over $100 billion dollars, personally.

The Microsoft vision was expansive. No software escaped Microsoft’s attempts to enter and redefine a market with PCs and software. It was a crazy time. Anything that came up in the news or could be read about had a project and team somewhere at Microsoft working on it.

As if that wasn’t enough, the burgeoning online division spun out home-grown Expedia as a separate company with an initial public offering that leaped 282 percent on the first day of trading. Microsoft acquired LinkExchange for $250 million to bolster online advertising. The Applications and Tools Group, of which Office was a part, acquired Seattle neighbor Visio, makers of a business drawing application, for one point five billion dollars, making the previously huge Vermeer acquisition look trivial. When we weren’t making money, we were spending it. New buildings were going up. Morale events, offsites, and endless ship meals (for all those late products) were getting more elaborate. Unopened PCs laid in hallways waiting for new hires to show up. Servers piled up in our ever-expanding online data centers waiting to be racked and stacked.

The focus on the enterprise was paying off handsomely and we were in middle of a profitable and maturing PC era. Growth in units and revenue for Windows and Office were not slowing, but the revenue growth outside of those was increasing faster. BillG recognized this, and in the early fall 1999 authored a widely distributed memo with the priorities for the following year. Windows 2000 was the priority, and it was going to ship real soon (it shipped to customers six months later). The other priority was what Bill was referring to software as a service in a September 1999 email to all employees with the subject “Changing the World Together”:

The other very significant focus for us in the years ahead will be providing software as a service. This won’t in any way change our commitment to the PC. But adding to that focus, we will be developing exciting new technologies to provide software as a service across a range of devices in the enterprise, to small business and to consumers. The goal is to maximize simplicity, flexibility, manageability and responsiveness to customer needs and interests. Software as a service reflects a fundamental change in the way we will be designing, integrating, deploying, licensing and supporting technologies, so there’s lots of exciting work ahead.

In terms of history, while IBM leased software for years, the phrase was first used by BillG in reference to the company strategy and quickly followed up by articles describing this across major news outlets. Silicon Valley was using the phrase Application Service Provider or On-demand Software.

It was not lost on me that Bill’s memo did not mention Office. Office was fading into the background as senior executive leadership was increasingly formed from Windows and Server alumni. In a big company when the most profitable and highest revenue team doesn’t get a shout-out, people notice. The team noticed. I do not think the omission was intentional, but rather a byproduct of where he was focused and was spending time.

The early projects for software as a service were incubated by a cross-section of systems and online services senior architects and engineers. Across the company we had countless offsites, meetings, memos, slide decks, and more working to develop a plan, all while the Office team was busy planning and executing a new release of Office while most of the company was just finishing releases. In many ways the first software as a service efforts followed the Innovator’s Dilemma playbook by not sharing connections to the main product groups. For time being these incubations were under the radar, but that would change soon enough—Microsoft couldn’t help itself.

Notably absent from the flurry of activity were competitive concerns. Microsoft appeared to have won the battles that a just couple of years earlier seemed existential. The internet did not end up dominating Microsoft’s traditional business, but rather Microsoft’s traditional business morphed into internet businesses; Wall Street certainly thought so. Windows laptops became the preferred gateway to the internet for consumers, with AOL purchasing Netscape for $4.2 billion on the heels of poorly received products, such as Netscape 4.0 compared to a winning Microsoft Internet Explorer 4.0. Windows NT Servers (eventually updated to Windows 2000) were becoming a corporate standard. Nobody else was making much of a dent in the Office business—not Office competitors, Java, components, or in the browser space. Sun even acquired a clone of Office, a Win32 suite of desktop apps, and then subsequently made it available for free, yet even that was not having an impact.

The antitrust cases continued, but day to day this was a matter for the legal team and not the product groups. The business press heralded Microsoft’s success as one of the great turnarounds of business history, a pivot, as in Microsoft pivoted to an internet focus.

One layer below all of this was a significantly different story.

Microsoft’s success in transitioning to internet technologies was financially successful but strategically weak. Microsoft lost the mindshare of developers—while Windows was a great place to run a browser, it was no longer the preferred platform for writing software. Microsoft was using many internet standards, such as HTML and HTTP, but our influence on those standards was minimal.

It is abundantly clear today as I write this, that the decline of relevance of Win32 as a developer strategy began as early as the early 2000s. Except for the big software companies like Adobe and Autodesk, and the vast array of line of business client-server applications written by in-house IT, there just weren’t any new and exciting applications written for Windows natively. Windows Server was new, but again never really saw a level of investment in unique apps on par with expectations, especially compared to the internet. The most exciting developments going on were browsers, web servers, and software built on those, and there was little at all specific to Windows. It was just that at the time and for the next five years or so most everyone happened to run Windows and Windows Server, giving the appearance of a vibrant Windows ecosystem. There is a subtle but important difference between success and relevancy in the technology world.

Windows Server was great for sharing files and printers inside the confines of a company, but the product was failing in the world of web sites and hosted services, even with significant marketing and programmatic support—today we call this cloud computing, but back then we called it ASPs or ISPs (internet service providers) or just hosters. The challenges didn’t stop with the operating system. The server programming models used on any of the exciting new consumer web services were not using Windows. Our own HotMail continued to resist the Windows platform. Even among the best LORG customers, Windows Servers were mostly used internally, and Linux, a free implementation of the Unix operating system from another era, was the preferred operating system for corporate internet sites, especially commerce and commercial sites. There were strong architectural and technical reasons, limitations, and costs associated with Windows, that facilitated the rise of Linux. Microsoft’s existing business made it impossible to compete with free without taking a major hit to revenue, or so the logic went.

MSN (short for the Microsoft Network), a collection of web-based information services emanating from RedWest (Redmond West Campus), was front and center of the industry, but the financial losses kept mounting with no end in sight. In this sense, Microsoft was consistent with the rest of the internet world. Internet competitors weren’t making a lot of money, especially in comparison to Windows, Server, and Office. For example, market-leading and era-defining Yahoo booked about $600 million in revenue in 1999 while losing money to being marginally profitable—it was unclear whether that concerned or excited us.

Microsoft’s enormous success selling a business vision was hiding the fact that these core products were failing in equipping the new internet platform. While Microsoft was making the most money, the internet was increasingly being run and programmed using non-Microsoft and, worse, free technologies.

Nothing made these aggregate challenges clearer than the enormous COMDEX tradeshow (historically an abbreviation of COMputer Dealers' EXhibition) in November 1999. It was perhaps the largest tradeshow event ever at the time and still a record-holder in the United States. The show had come a long way since the Halt and Catch Fire era just 15 years earlier. Every hotel room, convention space, and taxi were fully occupied for the week. The convention was electric, and massively, painfully crowded.

A funny thing happened as over 220,000 people made their way there that year: PCs became somewhat irrelevant. If this sounds frustrating, surprising, or even puzzling it was. Those very feelings consumed me as I definitely did not think we had ceased to be interesting or that the business we were in was boring.

The consumer internet took over COMDEX. Historically, the PC had been the heart and soul of COMDEX, but it suddenly faded in importance, and with it so did COMDEX. This marked the biggest and, effectively, the last of the show, as all the major PC makers left the following year. So it goes. The Consumer Electronics Show, CES, would become the go to show as COMDEX sputtered.

I saw the rise of the consumer internet during frequent trips to Silicon Valley and to PowerPoint, the 1987 acquisition that was based in Silicon Valley (with offices on Sand Hill Road no less) and remained there, never relocating to Redmond. The billboards on Silicon Valley’s highway 101 shifted away from networking and databases, the foundations of client-server, to the likes of Flooz, Pets.com, and Excite. These were the precursors to the dotcom bubble and the stock market crash that was still months away. Seattle wasn’t immune. Seattle’s newly hip Belltown neighborhood (awhere I had moved to a new apartment) was constantly circled by Webvan.com grocery delivery vehicles and one building was painted with what became an infamous logo for MyLackey.com, a concierge service summoned by a website with a retro-butler graphic.

The dotcom era was not about enterprise software but about consumer services delivered over the web. PCs were too complicated, and the industry, as I witnessed at COMDEX, was responding with a sea of internet appliances that were simpler, easier, and more reliable to use than PCs. The market spoke. PCs were too complex and bloated for the new e-conomy.

The result was that BillG’s keynote at COMDEX that year was all the more divergent. He spoke at length about Windows 2000 and Office 2000 improving reliability and getting work done, topics for the most part lost on the audience and even the press. Microsoft was not immune from the consumer internet though, building on WebTV, an acquisition Microsoft made in the appliance category (I was a huge champion of it, having been an early adopter and buying them for family). Microsoft introduced MSN Companion, an internet appliance for connecting to the internet over a phone line much like the dozens of devices on the tradeshow floor. The device ran Windows CE, a subset of 16-bit Windows that was about to become legacy code that ran on microprocessors from ARM. Compaq, eMachines, and others manufactured the devices, licensing the software from Microsoft in a business model (and code base) that presaged how Microsoft would enter the mobile phone business. These non-PC devices as we called them were big news.

MSN Companion came from the Online Services division. Microsoft, historically two cultures of Apps and Systems, spawned a third—Online Services, Microsoft’s foray into the consumer internet. As if to further emphasize the cultural divide, the online groups were located in a new campus, RedWest, about a mile down the street from the main campus. The brick buildings, identified by letters and surrounded by streams and pathways, were in contrast with the utilitarian stucco facades of the main campus with the lone Lake Bill in a near random series of numbered buildings.

The Online world was indeed a new culture, even though so many on the team came from Apps and Systems. There was arguably a degree of superiority emanating from RedWest, or perhaps inferiority from main campus. Everything seemed more exciting. Everything seemed more relatable to friends and family. Celebrities routinely visited RedWest and these visits were dutifully covered in MicroNews, the employee newsletter that eventually became an online intranet site. RedWest was to old-timers everything that the previous Consumer Division hoped to be, but instead of CD-ROMs it was web sites.

No surprise, the PC presence at COMDEX was brutal. I walked the show floor, systematically as usual. It was much more crowded than expected though the contrast from the previous year could not have been more dramatic. The effort to equip booths with Windows 2000 Ready signage was hopeless because the booths running Windows were on variants of Windows 95 for compatibility or simply because of laziness, as the much beefier Windows 2000 was still in pre-release and had been under development for so long most stopped paying attention. In fairness, Windows 2000 was not aimed at consumers, but at businesses—a necessary narrowing of focus due to the consumer hardware ecosystem that remained unsupported. Productivity tools such as printers were repurposed to show printing photos or CD labels. Email was shown but only when running on new internet appliances. Even digital signs that used to be PowerPoint running in loops were replaced by a new favorite gadget, digital picture frames (all running Linux and the latest ones natively connected to the internet).

Back in Redmond, after COMDEX, Microsoft was better able to compartmentalize the ongoing challenges. External stimuli can be disorienting if you’re a giant, successful company that has just successfully transitioned to a new era. At least that was what the press was telling us. And we believed them.

A favorite saying of mine is from Hemingway’s The Sun Also Rises:

How did you go bankrupt? Two ways. Gradually, and then suddenly.

This is how technology change (or bankruptcy or disruption) happens to the successful. At first, changes take place slowly and mostly go unnoticed. In reality, they do not go unnoticed, but rather, they seem insignificant—especially in a large company where there is always someone shouting or emailing that “the sky is falling.” To paraphrase a line from the 1984 film The Adventures of Buckaroo Banzai Across the 8th Dimension: “No matter what happens, someone always said it would.” It seemed there were quite a few people readying themselves to be able to say, “I told you so.” Memos were flying around, and choice quotes from Innovator’s Dilemma made their way into every slide deck. Every week someone would pull something out of their Sent Items folder to remind others of their predictive wisdom.

Then much further out in time than anyone thought, the accumulation of these small, unnoticed changes pile up and compound into a huge, seemingly unstoppable wave. When faced with these small changes, the natural reaction is to look inward, brushing them off as . . . small and insignificant. There’s always plenty to do, and usually that seems really important and customers are demanding it. The outside world is shut out and the focus turns to self-determined goals and activities. You enter a bubble. It is a really big $500 billion bubble, but a bubble.

I constantly worried about the increasing distance between what we were doing and what seemed interesting to the world beyond Microsoft and business customers.

For us, it was the constant drumbeat of enterprise customers that was difficult to match. Microsoft expanded to tens of thousands of enterprise sales and support personnel in a strong, aggressive, and empowered field organization, a number that greatly outnumbered the product team. On the Office team, about 200 people were in the position to interact effectively with sales and customers, but they were busy designing, building, and marketing Office and customer demands greatly exceeded our capacity to interact.

It is a cliché to read business books and hear of success stories about product teams that get close to customers. I dove headfirst into every book I could find on learning from customers—from In Search of Excellence and Crossing the Chasm to books about customer-obsessed companies like Wal-Mart and General Electric. I quickly learned that those were almost always stories that did not have a natural analog with Microsoft’s products. Our products were complicated, like IBM or GE products, but they were used directly and differently by hundreds of millions of people. Even if a large company used Office, within the company Office was used as though each customer was unique.

The challenge we faced was that the Systems approach much more closely resembled the IBM or GE model of listening to customers, and the field loved that. While each MRI machine might be used and configured differently, there are only thousands of them in the United States. Each email server installation of Exchange had many unique elements, but there were only hundreds of them (at the time). Product groups like Exchange were the same size or bigger than Office but could easily develop high-touch relationships with the largest customers, and further support those with the dedicated field support being built up around the world. Even Windows had a level of indirection in that by and large the view of being close to customers from the Windows team meant being close to the 10 major computer makers, not to hundreds of millions of new PC buyers. The PC makers even handled the cost burden of offering customer support for Windows.

By contrast, Office was not only unique at the company level, but each company might have dozens of configurations, plus each individual might have a different type of PC, and certainly might have different software, printers, or other peripherals. It was the use cases that grew rapidly—companies generally converged on best practices for using and managing servers, but when it came to Office, diversity was the rule and hands-off was the support policy.

That was our strength. The Office team embraced that. The rest of Microsoft, especially field sales and the server products, did not see things the same way.

The lack of direct enterprise engagement looked like Office not listening to its customers, and that was a constant tension on the team and between organizations. We (specifically, me) were not quick to jump to the customer crisis of the moment or to dedicate resources to understanding the latest hot customer scenario or problem. There was always a desire to dedicate resources as a show of customer love. At any given time, there were thousands of these that we knew about, and countless individuals struggling somehow with the product.

By listening to what percolated through support or executive escalation, we risked having the product team be driven by anecdote and squeaky wheel. Customers and partners were rewarded if they escalated, no matter what connection they would use. Every time SteveB or a senior exec returned from visits with the field, the escalations would follow. I always knew who was visiting customers as a trail of emails would land in my inbox.

Office continued to maintain the highest satisfaction ratings among products (and among product development teams), a fact I resorted to sharing. The brand consistently stood for “ease of use” in the eyes of customers. We continued to win head-to-head reviews. The Office Advisory Council, OAC, was our key enterprise customer learning tool and our OAC members loved the depth of engagement.

Becoming an enterprise company was a journey. In the late 1990s, the enterprise companies were Netscape, IBM, Oracle, Sun, HP, and SAP, and in Japan companies like Fujitsu, NEC, and Hitachi ruled the landscape. Wall Street made up fun names for these two groups. NOISE referred to the US companies Netscape, Oracle, IBM, Sun, and everybody else. The Japanese tech industry referred to FNH for Fujitsu, NEC, and Hitachi. Take that FAANG. Microsoft was not a consumer company but more of a vendor to enterprise companies than a partner, especially Office. In a sense, Office became part of the enterprise fabric or infrastructure but not part of enterprise decision-making.

One view was that we were not invested in the dialogue. We were, though not with every customer and that was a problem.

Another view, and what I believed, was that the dialogue was about server infrastructure and Office was personal productivity. The IT world was about servers, just as they were about mainframes. They viewed the desktop as far less strategic. Servers were strategic buys. Office was a must-have, but also a transactional purchase, except for Outlook which was strongly connected to Exchange Server.  IT provided email as a service to employees, so it was held in high regard.

The question was whether my view was limiting, convenient, or expansive.

Starting with Y2K planning early in 1999, company-wide efforts on building enterprise trust dramatically increased. The company was rapidly shifting. In SteveB’s words, we were out there “selling what we built” as enterprise salespeople, but we needed to “build what we could sell” as enterprise product makers.

With SteveB’s appointment to CEO in 2000, Microsoft was, as he said, “all in” on enterprise. BillG was completely and deeply engaged in the design of future products as chief software architect (CSA), his newly created title. Our company-wide processes from planning to budgeting, and especially marketing, changed dramatically with the shift to enterprise leadership. SteveB was the sales leader and that permeated company operations.

The implications were broad. Aside from choosing features that enterprise customers valued, several separate yet significant enterprise-level initiatives were in flight, and each required cross-company product team collaboration and a connection to the global sales and support teams. While any might have taken place before (and many did), with SteveB as CEO these had sales urgency and top-down processes. I could see in all the communications an element of “as chartered by SteveB” or “we committed to get back to SteveB in 120 days” as, let’s say, reinforcement. Taken together they presented an opportunity for Office (and me) to be seen as an equal partner to enterprise customers along with Server teams.

These initiatives included getting ready for the changeover to Y2K, improving PC security, privacy, viruses, malware, crashes, and reliability, and the previously described support policy. These special projects were happening in parallel with building Office10 and stretched through the next release as well.

The Y2K changeover mobilized just about everyone at Microsoft as we saw across the entire tech landscape. Microsoft was almost giddy in preparation. We seized on the opportunity to have a crisis that seemed like a fun technical puzzle. Every product we released going as far back as you could imagine and every company system were exhaustively tested and verified. We had a Y2K operations center, complete with generators and food supplies. Every communication system was made redundant for the rollover. What seemed at first to be a big headache was kind of fun for everyone that spent a year in preparation. The virtual team took great pride in tracking down obscure products and finding compliance issues, documenting them as part of the elaborate classification of Y2K readiness. Microsoft was as surprised as anyone that the new millennium got off to a glitch-free start.

Microsoft already had a few run-ins with regulators regarding privacy of early online services, but the growing internet was creating a whole new set of concerns especially when considering some of the services we were running at massive scale such as HotMail. Similarly, computer viruses that were once nothing more than an annoyance had become business critical problems. The next section is dedicated to the history and building a company-wide response to this challenge.

Y2K was one class of potential bug, but in practice our products had not made much progress in terms of quality over the past ten years. Office was incredibly solid, but if there was a problem then invariably all the customer’s work was lost. Such catastrophic loss was very bad for one consumer but entirely unacceptable to enterprise customers. Server products had their challenges with scale, configuration, and network management. Windows made significant improvements going from the 16-bit to the new 32-bit code base, but for new computers Windows 2000 lacked support for most consumer software and required more memory and disk space than PC price points could support. In a follow-on section, the innovations Microsoft introduced that radically changed not just our products but how developers thought of using the internet to improve software quality and reliability will be detailed.

Working on each of these brought a level of cooperation and collaboration across the company that we had not seen previously. Soon we were sharing best practices and thinking more broadly about how to be proper engineers more than hackers. As one indication of the importance of this change, JonDe, who had led Office for so long, would take on a broad role of creating an Engineering Excellence organization to formalize this type of cross-company maturing.

Individually, each of these initiatives seemed, especially in hindsight, seemed trivial or even obvious and basic work we should have been doing all along. It wasn’t so simple. So much was going on and everything was happening so fast, with so much potential for failure on much larger issues, that it took time to catch up to understanding the situation. From a team perspective, these were milestones or perhaps tests on the journey moving from products for tech enthusiasts to a platform for enterprise computing. Taken together, these represented a step-function change in the perception and reality of product quality and business readiness. Tech enthusiasts were not only more tolerant of these product quality issues but often embraced the foibles and prided themselves on knowing the ins and outs of products. On the other hand, LORGs were not only less appreciative of what they termed product defects but pointed to the very existence of them as proof that Microsoft was not ready, or even capable, of serving the enterprise. Industry analysts chided Microsoft for a lack of enterprise chops and often scored products lower because of this.

The special cross-company projects were meant to serve customers and were decidedly strategic for our existing customers. At the same time, they were not going to address the potential disruption from the changes we saw at COMDEX. It was easy to look inward, but I tended to view these as essentially taxes on the team. All “just work,” as we said. Others with a more near-term focus viewed these as innovation in the context of enterprise.

We were both right. We were both also wrong.

We needed to walk and chew gum at the same time. That need was not always clear. But more often, the whole of Microsoft was still more focused on these activities rather than what might happen in a few years.

On to 060. ILOVEYOU