FreakTakes

I made my first podcast appearance on the Foresight Institute’s podcast! The episode turned out to be a phenomenal one-hour summary of the most important ideas FreakTakes has covered. YouTube and Spotify embeddings are below.

In the episode, I discuss:

• FreakTakes as an applied research shop, and how I choose what to work on

• The role Bell Labs-style systems engineers can play at new science orgs

• How I might give away $100M, inspired by Warren Weaver and the Rockefeller Foundation

• BBNs at early ARPA, the upside of building more BBNs today, and why I joined RenPhil to work on this challenge

• The effect of events like Watergate and Vietnam on scientific bureaucracies

• What Gerald Holton and certain scientists’ oral histories have to say on the ‘burden of knowledge’ as a human systems problem

• Tons of R&D history anecdotes, including:

  • The complicated takeaways of Claude Shannon’s time at Bell Labs

  • Wilbur Wright’s taste

  • Irving Langmuir at GE Research

  • Praise for early MIT and the role of market signals in shaping their research agenda

As always, just email me ([email protected]) to discuss any ideas discussed in the episode! I’m particularly eager to spend time with those who are curious about founding for funding BBNs.

SpotifyYouTube

Subscribe now

Share

Thanks for listening:)

As some readers know, I’ve recently joined Renaissance Philanthropy (RenPhil). While I hope to put many ideas from FreakTakes into practice at RenPhil, I’ll be particularly focused on one goal: building more BBNs.

For those who have not read or do not remember prior pieces on BBN-model orgs (BBNs), the biggest takeaway from my series of ARPA project histories is that, for many of ARPA’s all-time great projects, exceptional projects were the result of exceptional contractors. By “exceptional,” I mean contractor groups who were not just staffed with elite talent, but uniquely aligned with ARPA’s mission. Early ARPA success stories like stealth aircraft, early autonomous vehicles, and the ARPAnet had contractors that shared three distinct traits, rarely found together. Each:

1. Was novelty-seeking, with a strong preference for projects that pushed the technological frontier forward substantially.

2. Built useful technology for actual users. This entailed professional contract management and a willingness to focus on difficult systems engineering tasks.

3. Used more flexible team structures than academia. When compared to academia, they more effectively hired, organized, and incentivized researchers, engineers, and other experts to collaborate on applied projects in a common-sense fashion.

While recent pieces have explored how great BBNs from history mixed contracts, grants, and hybrid team structures to pursue an ambitious technical vision, today’s piece will kick off a series of pieces profiling BBNs and BBN-shaped opportunities that exist today.

The subject of today’s piece is the Topos Institute, a BBN-shaped organization staffed with PhDs from top math and computer science departments. The CEO, Brendan Fong, has a computer science PhD from Oxford, specializing in category theory and its applications. The unifying technical ambition that motivates the Topos team is the goal of building technologies to enable scientists and engineers from adjacent fields to more effectively combine and reason with overlapping models from distinct domains. Topos pursues this technical vision with a mix of carefully selected R&D contracts and traditional research grants. In today’s piece, I interview Brendan to learn more about:

• Topos’ technical vision.

• How Topos balances contracts and grants to fuel their work.

• Topos’ comparative advantages when compared to university departments, startups, etc.

• How Topos is/is not like an FRO.

• What differentiates Topos from other AI for science groups.

Enjoy!

Subscribe now

Share

For those interested, Topos is looking to hire for a crucial position in its Oxford, UK office — discussed later in the piece. It’s an ideal position for someone with a nerdy fascination with the operational history of R&D and excited to contribute to managing a real applied metascience experiment. (I have also been meeting other BBN founders looking to fill similar roles. If that’s exciting to you, please see the end of the piece to learn more.)

In putting together this piece, Brendan and I had over three hours of messy, wide-ranging conversations and interviews. We’ve distilled them into this final, more readable, interview format. From here onward, the italic + bold text denotes my words and the standard text Brendan’s.

Topos, In Short

Brendan, what is Topos from a very high level?

Topos is an experiment in a new structure for doing research for the public benefit.  In pursuit of this goal, we look to combine a focus on fundamental, long-term research similar to a university with a theory of change similar to a tech startup.

The technical area we work in is mathematics and computer science — particularly in collective modeling, collective inquiry, and cooperating in situations that are incredibly complex and multidisciplinary. We do this as a nonprofit whose work is supported by grants and contracts from groups like DARPA, ARIA, philanthropic foundations, technical firms, and gifts from high-net-worth individuals.

Topos’ North Star Technical Ambition

Can you talk more about collective modeling etc. and how that drives Topos’ grand technical vision?

Ultimately, the sort of technology we want to develop and produce is, in some sense, a new form of programming language. As we’ll talk about later in the interview, Topos works on projects in areas like climate modeling, the systems engineering of large airplanes, ensuring AI systems are resistant to catastrophic risks, etc. The communities contributing to each of these technical systems are made up of a variety of technical experts navigating different aspects of the system with their own overlapping models — which often have their own merits and their own limitations.

Topos works on projects with specific user communities. That keeps us grounded in the practicalities, ensuring we don’t get too caught up solving problems we find beautiful but are only modestly connected to practical needs. The grand goal the organization has when working on these specific problems with users is a General Theory of the Specific. We want to build a tool that, like the spreadsheet, helps a variety of users reason across situations with the richness and complexity that comes with combining and interpreting a mix of overlapping models of the world. The term I often use to describe the goal of this general programming language or spreadsheet to combine ideas from overlapping domains is “collective sense-making.”

One difference between this general modeling environment we hope to build and the spreadsheet is that we want to empower our users to represent the world in the conceptual language they use in their work, not force them to translate those representations into numbers and formal logic. A key difference between our modeling environment and even high-level programming languages is that we are more focused on declarative representations of the world, not procedural actions in the world.

As transformative as spreadsheets have been, I think we could make a similarly significant advance if this plays out right.

Do you intend for this to be a tool for mixing models in the worlds of science and engineering, in particular?

Scientists and engineers are the target user groups in the vast majority of our applied contracts. While we think that one day these tools can be helpful to someone running a coffee shop, planning a wedding, or simply just communicating carefully with friends, we see the ‘edge of the wedge’ as working in areas where people already have a rich tradition of articulating the conceptual foundation of their work. And that tends to be in the sciences.

Where Topos Fits in the R&D Ecosystem

Would you like to say anything about the distinct advantages Topos has in tackling certain technical problems — when compared to universities, startups, etc.?

We’ve chatted for a while outside this interview, and I find you very perceptive about these topics. Can you give me a quick rundown of what you think the strongest case is? Like what three things would you name? And I can see if I disagree or would like to add.

Yeah. When we had a discussion surrounding the question, “Do you consider Topos to be a ‘center of excellence outside the university?’” you pushed back. That wasn’t due to a lack of talent. You believe a huge proportion of your staff could have gone on to have successful academic careers. You pushed back because you felt the term ‘center of excellence’ was often associated with centers of excellence for ideas, not technology. You want to be a grounded org. University departments and places committed to ideas, first and foremost, tend to not respond to user needs very dynamically. So, in terms of differentiating yourself from an academic department, I’d say being staffed with similarly talented people dedicated to building technology for actual users with pressing needs is a huge differentiator.

In terms of what differentiates you from startups…VC-funded startups are often limited to areas with a market cap of ~$1 billion+. That’s a big constraint. And in most cases, they don’t really embrace hard technical risks. They often prefer engineering risks to scientific ones, etc. They embrace risks that they hope can be mostly mitigated within a couple of years…a lot of the time, at least. You recruit as ambitiously as a university department or a top startup. But you go where the hard problems are as long as you can find funding from a DARPA, ARIA, or some open-minded grant funder in Berkeley. So, to be clear, you also have an arbitrary constraint. But the arbitrary constraint is “What’s DARPA funding?” or “What’s ARIA funding?” But if you can find that funding, you don’t have to clip the wings of your ambition. It allows you to tackle all sorts of areas VC-funded firms can’t. You can let me know if you agree with all that.

Yeah, I think you pretty much got it. For groundbreaking, public good technologies, we need tight integration between research — the source of deep new ideas — and product — the source of deep new questions. Our core advantage over a university is that we can be user- and impact-driven. Our core advantage over a start-up is that we can tackle deep technical risk. Let me add a bit of color to both points though.

First, academia. One thing that cannot be done in a university setting is, I think, build and maintain technologies for the long term. And that is a core part of our theory of change and how we assess our success in basic research. We want to do basic inquiry around foundational questions — and we win enough grants that allow us to do that — but we don’t want to be measured by the publications we produce: the publications are instrumental. This contrasts with the known incentives against maintaining software and a user base in academia.

Relatedly, another way we differ from a university is being intensely collaborative and team-driven. To me, the incentive structures of academia are entirely individual. Postdocs, tenure, awards, all go to individuals for individual success. One of the reasons for starting Topos was that it was the only way we could find ourselves collaborating day-in-day-out with the top people in our field: university departments often want breadth, rather than aligned faculty well-suited to teaming up. With that system, it can be difficult to find and maintain generative collaborations that lead to core breakthroughs.

What separates us from start-ups? We’re driven by producing public goods — as long as we can find the funding. We don’t orient ourselves around what’s best for investors.

Since I know you want to move on, the last thing I’d like to add is that Topos is a lot of fun! We’re a collaborative, creative, and compassionate place with a great sense of collective purpose. I’m inspired by our team every day.

Specific Technical Projects

Topos’ technical agenda is driven, in no small part, by pursuing a General Theory of the Specific. Could you talk about some specific projects you’ve 1) already worked on or 2) would love to pursue?

One very illustrative example is a systems engineering example. Imagine you’re Boeing or Airbus and you’re constructing a passenger jet. There are thousands of people involved in the process of this construction. How do you ensure it all comes together into a vehicle that people can trust with their travel and lives for decades to come?

Some of that is scientific or engineering modeling. “What is the weight of the engine?” “What are the properties of the materials going into the airplane wing?” “How do the material properties relate to the aerodynamic properties?” Etc. But in bringing the parts together, very significant social processes are also going on. Initially, this group is responsible for this piece of the puzzle, specified in this way; that group is responsible for another piece specified in that way. There are also a lot of initial guesses that get refined over time, through new insights and changes in responsibility. This is the evolution of a process, a highly dynamic process of knowledge discovery.  It’s really important to keep track of that history and understand how different decisions have been made in response to different sorts of problems, insights, and constraints as the work of various groups converges into an interoperable design.

Passenger jets like the Boeing 737 have been flying for 50 years or more…

And, at this point, these firms often need to keep long-retired engineers on retainer, sometimes until they die, in case they’re needed to answer a question. This need often derives from this issue of organizational memory — not remembering why you did what all those years ago, how that learning should apply to the present situation, etc.

Right. Exactly. That sort of knowledge, that sort of understanding of the product, is very difficult to institutionalize with existing technologies. But if you do study the processes of collective sense-making carefully, there is reason to suspect — and it’s Topos’ ambition to build — knowledge management software, structures, techniques, and cultures so you don’t have to keep someone on retainer…unless they want to be on retainer and you want them to be on retainer.

The folks at ARIA connected us. I’d love to hear about your current AI Safety project with them.

What we’re doing with ARIA is a project for Davidad’s Safeguarded AI program. There is this idea that AI systems will increasingly drive actions in the world. This will sometimes extend to critical or complex situations. For example, it would be great if we could use AI systems to optimize the power grid of the UK — payload distribution, inform the construction of new power generation sites, and so on. But there are huge possibilities for catastrophic risks to a country. The power grid is literally the backbone of getting work, as in physical force, done in society. Power grid failures, whether accidental, due to natural disasters, or from cyber and physical attacks, could cause widespread economic and physical harm, and even deaths.

How do you negotiate over the constraints of what you want — in an accessible, high-level way — so your team can use AI tools to generate solutions that you know to be safe and respect those constraints? One thing you need in these situations, where you have all these social and physical factors, is an appropriately precise model of the world. This model should be transparent, interpretable, and verifiable in certain ways. By providing a way for people — in a collaborative and interdisciplinary way — to construct subtle, formal models of the world, you can enable them to run these AI systems inside sandboxes. In these sandboxes, you might have tools helping you to combine techniques that allow you to validate your model and constraints — e.g. automated reasoning, formal verification, theorem proving, etc. Doing this, a team can generate very strong pieces of evidence (almost guarantees) that the system will be safe enough for public deployment.

Do you have any examples of ARIA projects you’d jump at the opportunity to work on? This can either be from their existing program areas or areas you suspect they’ll pursue in the future. ARIA’s program areas are all scientifically-motivated and seem to have an AI for science component. And, as we discussed, there are difficulties that come with pursuing this sort of work entirely within academia. There’s a lot of room for exciting Topos collaborations.

What would we love to do with ARIA? Another program that has conceptual similarities to the Safeguarded AI program is Sarah and Gemma’s program on Forecasting Climate Tipping Points. Their program deals with an extremely complex system — understood only through integrating perspectives from multiple disciplines, including both the physical and social sciences — with deep impacts on society. Their goal is to produce an early warning system for climate catastrophes that is precise, trustworthy, and actionable.

As the program thesis outlines, “our best climate models are computationally expensive and do not capture all the physical processes we need.” Moreover, we want to avoid tipping points, so we don’t and hopefully won’t have too much data on them! And the complexity of the system requires the integration of information at a wide range of spatial and temporal scales. We’d love to undertake a project on mathematical and software development of climate modeling tools that respond to these challenges. Similarly to our ARIA project with Davidad, we could work with ARIA creators (contractors) to figure out things like, “What are the bounds of the system?” and “Under what circumstances will it go into a catastrophic region of behavior?” We’d try to develop tools that enabled domain scientists to construct a shared, interdisciplinary understanding of the climate system that is responsive to new information from sensing systems and research.1

Of course, what that hypothetical project would look like specifically depends on many things — the needs of the program and other creators, etc. I’d be interested in exploring what you’ve mentioned, which is the idea of Topos being a sort of consulting AI for science shop that comes in and assists with tooling to solve problems in and between particular domain areas. Academia is not set up to support cross-disciplinary tooling and workflows, and we’d love to complement the academic system in this way. But…I’d like to be very clear and say that domains are very important. When talking about providing AI tools for other scientific disciplines, I want to do it from a place of humility…

If I can interrupt, do you mind talking about why climate science is such an interesting area for Topos first? And then we can re-visit your point on bringing a level of humility to AI for science work.

Yes, of course. Climate science is a great example of where our work can add value. We build computational tools for cooperating across the different conceptual frameworks and disciplines required to navigate complex systems. One basic, almost too-obvious-to-state idea underlying our work is that a good way of making concrete progress on difficult topics is writing things down. You want to write down models; you want these written models to precisely reflect your understanding of the world.

To adequately describe complex systems, we must use models from many different disciplines simultaneously. The problem, however, is that disciplines often create silos and it becomes difficult to collaborate across these different perspectives. So, we need a way of allowing conceptual-specific language that also enables interoperability between the various languages. A major focus of our tools is this process of translation, mapping between perspectives, identifying overlap, and identifying difference.

In tackling this work, you really reveal the complexity of the situation. It’s a challenge. Things become very messy, very fast. For some arbitrary climate science problem, an oceanographer might be trying to construct maps of the Pacific Ocean, how it works, the concepts involved, etc. The oceanographer will have a particular idea about what water is and what the important features of it are. This is very useful, but when building a climate model you’ll also want to have insights from an atmospheric physicist (etc.). Depending on the situation, an atmospheric physicist will have their own model with different primitives. Factors like solar radiation or windspeed might be more or less important, or interpreted in a different way, than in the oceanographer’s model.

There’s an overlap between these concepts: both fields are talking about water. But an atmospheric physicist is talking much less about ice and much more about water vapor, clouds, etc. But they’re still talking about water. How do you translate between what water means to an oceanographer and what water means to an atmospheric physicist?

One approach we use to tackle problems like this — drawing on our mathematics and computer science backgrounds — is, as you collect data, setting up database ontologies and more sophisticated logical tools to help specify what water is from various points of view. From one point of view, water is some combination of precipitation, cloud cover, snow, etc. But the relational aspects are also very important in forming a more full worldview. From this POV, water doesn’t mean anything in and of itself; it’s something like a database column heading. What reveals the quality of water could be how it relates to what the plankton need — or other parts of the ecosystem.

Having to translate what water means in one field vs. another can be incredibly complex. A single concept here can turn into a web of relationships over there. When dealing with problems like how water is witnessed through its relationships to animal life in the oceans, you might even have to translate between webs of relationships. In trying to create interoperability between frameworks, there are a whole lot of interdependencies that are incredibly difficult to track. You want sophisticated, formal computational tools to help you track that. That’s what Topos tries to build.

With our approach, the scope for ambition in climate science is immense. Consider the current state-of-the-art in the field. There are these gigantic climatic models that groups like the IPCC coordinate. With these models you’re talking about millions of lines of code, often in low-level languages like FORTRAN, written over the past half-century by thousands of scientists across many disciplines. There are good reasons for using FORTRAN, but there’s a recognized need for better tools too. They’re trying to weave all these models together, but they’re presented in ways that are so procedural — not conceptual — that you cannot get at the emergent behavior as these disciplines interact. To tackle this particular application would require succeeding on many smaller, shorter-term problems first — like the systems engineering problem above. But it’s the exact kind of problem we hope to build towards.

Humility, Sourcing Contracts, and the Prospect of Being an FRO

Now, if we can, I’d love to get back to the idea of pursuing this interdisciplinary work with humility. It’s very important to us. Our work relies on deep, respectful relationships with domain experts, our collaborators. To establish such relationships, it’s critical we acknowledge that it’s very rare for an outside group to come in and do things that are groundbreaking in the minds of people who have been working in that area…

If I can interject, it sounds to me like the Topos approach comes in with more humility than is typical from many AI for science groups. As I understand them, your proposals for applied projects are akin to what Blackboard would have been in the early CMU Robotics Institute’s autonomous vehicle work. You’re not coming in and saying, “Let’s build up more vision data and we’re going to run it through an existing set of models. Scale is going to solve it.” Your pitch, to me, sounds something like: “This is a really complex system with a lot of disciplines that have something useful to say about it. We’d like to come in and build you tools that can be something like a synthesized reasoning engine.” Your proposal attempts to take in all the existing models and output something very interpretable, while very overtly attempting to preserve what specialists would consider useful about their fields’ models. You’re not looking to supplant them with any kind of neural net or random forest.

Yes, exactly.

Now, there probably should be some contractor that comes in and tries to get people to build up more data, train neural nets, and things like that. But it sounds like Topos has a pretty differentiated offering — one that has a lot of humility.

Right. We’re not the kind of AI for science shop that seeks to combine clever algorithms, a wealth of data, and compute to yield breakthrough insights. That certainly has its place, and we’ve seen some stunning successes from this approach, including the Nobel Prize-winning work on AlphaFold. But that’s not our comparative advantage. What we want to offer is a much more bespoke product. We hope that through our applied projects we can assist users in eliciting the structures of and clarify the thinking in their domain. Over time, we hope to build up a software stack that helps reify some of that knowledge that has been clarified. If all goes well, our tools will enable them to more easily build up or utilize that knowledge.

If you could raise the funds, would becoming an FRO be an ideal scenario? That’s not easy. But let’s say someone offered you $50 million over 7 years to pursue a big FRO-style project, what would you do with it?

That’s a very interesting question. I guess the answer to your question is: we’d love a modest amount of FRO-style funding — say $10 million — but even with this funding we’d continue to seek out contracts too. It’s a bit too early for us to need something like $50 million over seven years. With the more modest $10 million, we could do a lot of research and engineering groundwork to develop fundamental aspects of our modeling environment. (See CatColab if curious.)

But for us to do good quality work, we need to remain grounded. General research funds are great, but another key resource for us is pressing problems from domain partners invested in the challenges of collaborating, articulating questions carefully, and working together toward solutions. Being resource-constrained, as we are, incentivizes us to maintain close engagements with contract partners with concrete needs.

We’re still figuring out the resource balance for Topos. I’ve found your work on BBNs as an alternative to FROs a very helpful framework for thinking this through. For example, it’s helped me find some questions that explore whether or not we’re on the right path. On one end, I ask questions like, “Are we becoming unmoored and doing too much research for research/philosophy’s sake?” or “Are we responding to actual problems in the world?” On the other end, I ask questions like, “Do we have the freedom to do really ambitious thinking and reset paradigms where we need to reset them?” or “Are we so constrained by the needs of particular contracts that we cannot, where we feel we have an edge on a new foundational approach, pursue that edge as necessary?”

We speak of Topos as vertically integrating foundational research, technology development, and public service. Navigating these questions, and thinking about the balance between FROs, BBNs, and other organizational models, is part of the process of making this idea reality.

Can you say a bit about how you obtain applied contracts?

Our most productive applied projects are driven by shared visions and strong relationships with program managers, whether at DARPA, ARIA, or in private industry. They’ll have a particular concrete problem space they’re exploring. Over the years, through our publications and through conversations, they develop an understanding of what we do and how we can help their programs. That often turns into them saying something like, “Okay, we want a proof of concept. My program has a need in [simulated multiphysics, systems biology, epidemiology, or something like that]. Can you propose a way to demonstrate your ideas in this particular domain?” We then go do a bunch of research on the problem area, often collaborating with domain experts, and construct a bunch of software around this problem — built on top of some basic core libraries we build and maintain.

How about undirected funds?

Undirected funds and, just as important, a strong network of visionary supporters, have been critical in getting Topos off the ground. We are an organizational experiment. Experimenting, exploring, and learning what does and doesn’t work for us: this takes real resources — it’s been around a third of our total funding so far.

Our founding board — particularly our founding chair, Ilyas Khan — provided not just resources but also mentorship, courage, and connections. I’ll be forever grateful to early major donors like Jaan Tallinn and Jed McCaleb for their trust, even despite our naivety and inexperience. Berkeley and Oxford have both been great communities for finding collaboration with open-minded funders who both appreciate our technical ambition and that breakthroughs in some of our approaches can have exciting spillovers for the wider science and technology ecosystem.

A Possible UK Expansion

You’re looking to make a big hire. As I understand it, you’re looking for someone to be your key operational person as you expand your Oxford, UK team. Finding the right person might even make the difference in whether or not you expand. It sounds like they would be both an operational hire and a strategy hire, interfacing between ARIA and your technical team as well as helping the team come up with new project areas. Can you talk more about what this person would do and why it’s such an exciting position?

I’ll preface this by emphasizing that Topos is an ongoing experiment and we’re still seeing it through. So it’s hard to speak on precisely what this person would do. But maybe that’s the point: the right person will find space for innovation exciting!

What is my best first guess? I think we need someone who sees the importance of Topos’ vision — humanity cooperating around issues of complexity. They should believe this technology can be produced by our team, which is (in my opinion) one of the most impressive technology teams around at the moment. And this person should want it to be produced as a public good.

This person is meant to help us continually find answers to practical questions like, “How do we organize this team of technical talent, fund them, and motivate them to ensure we have a sustainable approach to building technologies that move humanity forward?” A mix of philanthropic grants, contracts, and maybe even investments (for certain products) might be involved. Beyond helping us make these decisions, they’re meant to take on a lot of the day-to-day responsibilities necessary to ensure our collaborations run smoothly. This first guess will be refined in concert with the applicant, the wider team, and ultimately practical experience.

As far as the type of person we’re looking for? They don’t need to be good at mathematics. It might even be good if they weren’t. But they should appreciate mathematical ideas, be open-minded and willing to learn how researchers think, and be able to get along well with our technical staff and staff at partner organizations like ARIA.

Many backgrounds could be well-suited to the role. These include:

• An operator with the ability to pitch and manage — who is also driven by public service. This person should also be excited by the experimental nature of our organizational model and the prospect of shaping it.

• A mathematician/computer scientist who gets people and business…and is willing to not do the mathematics. While this person wouldn’t be doing research, having an appreciation for mathematics might give them a deep understanding of what this org is about.

• A domain scientist who, beyond being good with people and business, sees the particular problems of their field as a lens onto a general problem. We’ve had some collaborations with domain-specific scientists flounder a bit because they wanted to start over-tuning for that particular field. That is not a terrible instinct when working on a specific applied contract, but a bad one for someone helping steer our UK operation. We are looking for a General Theory of the Specific, after all.

• A software engineer whose first instinct is not to respond to things by building systems. Software engineers often respond to things by building rules-based systems; that is the nature of the craft. We’ve had only ten contracts throughout the history of the organization. The wrong software engineer might be tempted to immediately begin building databases, search tools, etc. around those learnings. Those approaches can be powerful, but I think what is needed is a bit of patience. This person needs to embrace that this is an ongoing organizational experiment in how to do science better; we’re still learning. We only have an inkling of what’s best. Formalizing too soon is not respecting the problem.

Lastly, I’d like this person to have a really creative spirit around how to build a team that does ambitious basic research that can also build. As we’ve discussed outside this interview, our early DARPA contracts taught us a lot of lessons we hope to incorporate moving forward. I’d estimate that at least half of the funds from those contracts directly funded work we’d have pursued anyway with an undirected NSF grant. This new hire should be excited about helping us figure out a revenue strategy that gets us as close to 100% as possible, and can find exciting partners/contracts to make it happen.

Conclusion

Any final words?

We’re something like a BBN-style org, right now. We have a clear technical vision that we pursue with a mix of contracts with carefully selected partners and grants. These bespoke projects are key to what we do and create deep insights into the nature of what the general problem actually is. Balancing vision and contracts enables us to build technologies that enable cooperation across different perspectives. Parts of our fundraising and governance approaches may change over time — there are many upsides to the FRO governance model, raising a basic research endowment, consulting using internal tools, etc. But our general goals of enabling cooperation in areas of complexity through a balance of basic research and applied projects will not.

Finally, thank you, Eric, so much for this conversation, and for the many leading up to it. It’s been a real privilege and, as always, you’ve helped push my understanding and articulation of Topos deeper.

It’s been a pleasure. I learned a ton through our conversations. I really hope one of my readers (or someone they know!) finds this piece and reaches out. It’s the kind of job I’d have dreamed of before FreakTakes, when I spent my nights reading about scientific history with no clue how to get hired and make a difference at any of the new, exciting organizational experiments I’d heard were being founded in places like Berkeley.

Thanks for reading:) If you’re interested in Brendan’s position, please reach out to him (first [email protected]) or me via email ([email protected]) or Twitter. And if you’d love to fill a similar role at a BBN in a different technical area, please reach out! I might be able to connect you with a researcher in need of a systems engineer/operator. Some technical areas in which I already know a BBN founder open to hiring (at least part-time) for similar work include:

• Non-Model Organisms

• Economic Complexity

• Ocean Water Sensors

• Bespoke EE and Biotech Hardware

• Computational Chemistry

The Topos team, March 2022. Left to right: Tish Tanski, Juliet Szatko, David Spivak, Valeria de Paiva, Evan Patterson, Toby Smithe, Brandon Shapiro, Brendan Fong, Tim Hosgood, Sophie Libkind.

Subscribe now

Share

1

It would mean a lot to me personally to note that climate change, via John Baez’s Azimuth Project, was, in fact, the problem that motivated me to pursue graduate studies in the first place!

I’ve always viewed FreakTakes as a small applied research shop. Discontented geniuses set out to run new science orgs or ambitious scientific philanthropies; I produce pieces that help some of them make decisions that determine parts of how to structure their orgs. I can’t do what they do — I wish I could. They do not have the time to look into the minutiae of the R&D operations from history that inspire their work. So, the task of researching topics like how Bell Labs chose research questions, how the Rockefeller Foundation helped fund the field of molecular biology into existence, and the vital role of BBN-model R&D orgs (BBNs) in ARPA history falls to me.

Before starting FreakTakes, I did not know whether people like Tom Kalil, Adam Marblestone, or Patrick Collison would find anything I might write useful in their work. But I figured it was worth spending a few hundred hours of my time finding out. I already spent most of my spare time reading about 20th C. R&D history and the economics of innovation. So, in January 2022, I started FreakTakes. I’d write five pieces and see how things went. On a trial basis, I’d turn my obsession into a blog.

That was the best decision of my professional life. Over the past three years, the influence of the blog has steadily grown — at least with the small number of key individuals who I see as my target audience. The research quality has steadily improved as I’ve accrued experience. My “taste” regarding which questions are most useful to look into has drastically improved as I’ve gotten to meet with those who run new science orgs and fund ambitious R&D. The experiment has worked out far better than I hoped. The success of the experiment is, in no small part, a result of Stuart Buck making two separate bets on me — the first in April 2022 to bring me on as a part-time Fellow at the Good Science Project, and later that year bringing me on full-time. For that, I will forever be indebted to him.

This year marks a new chapter at FreakTakes. I’ll be joining Renaissance Philanthropy (RenPhil) full-time — while remaining a Fellow at the Good Science Project. Working with Tom Kalil, Kumar Garg, and the rest of the RenPhil team, I hope to translate the lessons of R&D history I’ve written about for three years into action.

My Role at RenPhil

At RenPhil, I’ll be ramping up the amount of time I spend as a do-er. While I’ll be spending time on all sorts of projects at RenPhil, the biggest one will be helping build and grow more BBNs — organizations like 1960s BBN or the early CMU autonomous vehicles group. R&D orgs like these — novelty-seeking, well-equipped to build useful technology for actual users, with more flexible team structures than academic labs — were the driving force behind many of early ARPA’s most exceptional projects. As the number of ARPA-inspired grant funders grows, the need for exceptional contractors like these will also grow.

Building a new BBN requires founders who are world-class researchers or engineers, have an ambitious technical vision, and have a strong sense of why the existing R&D ecosystem cannot pursue the vision. If that sounds like you or someone you know, please reach out. I’d like to help hone your idea and try to make it a reality. Building a new BBN also requires R&D funders with ambitious projects. So, I’m also eager to meet with R&D funders (philanthropies, ARPAs, etc.) to discuss projects they are eager to pursue, but for which they feel ideal contractors do not exist.

Operationally, I’ll also be helping RenPhil execute its ARIA work in the UK — and a variety of other activities. In terms of research, I will continue to produce my long operational histories of R&D. But some things will change.

What is changing?

As I step into this new role, I hope to expand the scope of the Substack a bit. FreakTakes will continue to primarily serve as the home for my operational histories of R&D orgs, but I hope to make FreakTakes into a Substack ambitious researchers and ARPA-like funders eagerly follow for information, talent, and opportunities related to their work. For pieces in this category, I’ll be creating a “RenPhil” tab to file the pieces under.

Some pieces that would fall into this category include:

• Interviews with the founders of existing BBNs — describing the vision for their organization, their founding story, and upcoming projects.

• Short pieces from ARPA-like philanthropists who would love to fund a certain kind of project, but for which the ideal contractor does not exist.

• Proposals for BBNs in particular areas that, with the right founder and team, could tackle ambitious areas of R&D and have a generational impact on a field.

As I join RenPhil, there is one notable tradeoff for FreakTakes. As I spend more time on operational work, I’ll only be able to produce about one-third as many long (7,000+ words) operational histories. But there are two upsides regarding classic FreakTakes content:

1. For the operational histories that I do produce, I’ll be ideally placed to translate the learnings into action.

2. I will also write more FreakTakes Shorts — often around 2,500 words — reflecting on events from the history of R&D, sharing interesting excerpts from researchers’ memoirs, etc.

In general, FreakTakes will remain mostly the same. I love it too much to change it too much. The new pieces I produce in the RenPhil section will be pieces that allow the blog to translate ideas I encounter into real-world projects.

The Surprising Effectiveness of Blogging

Before moving on to my pithy conclusion, I’d like to take a moment to express my appreciation for this community. The open-mindedness and curiosity of the researchers and funders who make up the “new science” community made it possible for a (more or less) random guy in Chicago — who reads about the history of R&D instead of going to parties — to throw his hat in the ring and make something useful of his obsession. Given I have no credentials, none of you had to give my pieces a chance. The fact that so many of you did is a testament to your open-mindedness.

When I first started this blog, I dreamed that maybe someone at a place like the Arc Institute might find it interesting and help me get a job there — not even as a full-time researcher, just as a budget guy who got to do research assignments as they came up. Hoping for more than that felt unreasonable. But sometimes, if you work at it, unreasonable things happen. I was more grateful than words could express when Stuart Buck gave me the opportunity to be a researcher at his think tank. And I’m similarly grateful to have the opportunity to turn so many FreakTakes’ ideas into action at RenPhil. This blog has gone weirdly well, so far.

My Pithy Conclusion: Embracing Restlessness

For a very excitable person, I’ve settled into the life of a history researcher quite comfortably. The decision to reduce the amount of time I spend peacefully finding answers in old books is bittersweet, but not difficult. The reason the decision is easy, despite the pleasure I get from doing research, is probably due to the Midwesterner in me. Even with my passions, I always try to make them practical. I never hoped to become a writer. I began FreakTakes because I had practical goals; I choose what details to include in my pieces based on what would be useful to the CEO of an R&D org, not because I think most people want a 14,000-word piece on Warren Weaver’s work at the Rockefeller Foundation.

I hope I’ll have the excuse to become a full-time researcher again, at some point. But for now, it’s time to translate many of FreakTakes’ learnings into action at RenPhil. An excerpt from a 1954 Warren Weaver essay provides me with a fair bit of comfort in this decision. The excerpt comes in a chapter in which Weaver — the greatest scientific grant funder of all time, a fellow Midwesterner, and a good man — rails against those in the 1950s who sought peace of mind, first and foremost, in their spiritual endeavors. Weaver writes:

Peace of mind is not what I ask from religion. To pray for peace of mind appeals to me as a rather unpleasant insult to the God of the restless cell, of the blazing novae, of the swirling nebulae. The god, in short, of progress, not of stagnation.

...God pity me on the day when I have lost my restlessness! God forgive me on the day when I am satisfied! God rouse me up if ever I am so dull, insensitive, lazy, complacent, phlegmatic, and apathetic as to be at peace!12

I look forward to embracing this new chapter of professional restlessness. Please reach out to me if you’d like to found, fund, work at, or somehow help R&D orgs like the ones I’ve described in this piece. And if you have any ideas for me as I take this next step, I’m all ears!

FreakTakes HQ, January 2022 to August 2024. The desk in my Hyde Park, Chicago apartment. Sketch by my girlfriend. HQ has since moved, so the author could move in with the illustrator

Thanks for reading:)

I’ll be visiting both Cambridge, MA and London (and maybe SF) soon, so please reach out to grab some time while I’m visiting!

Please comment, email or DM me on Twitter me any ideas you think I should consider for the next chapter of FreakTakes ([email protected] or [email protected]).

Subscribe now

Share

1

Notably, Weaver and his wife left what they felt was a nearly idyllic life in Madison, where Weaver was a math professor, to pursue his work at the Rockefeller foundation.

2

Excerpt from Weaver’s book Science and Imagination, Chapter 7, Peace of Mind and Other Semantic Problems

Unlike the rest of the FreakTakes ARPA series, today’s piece does not have the usual narrative structure. It is, instead, a set of personal notes cleaned up and re-structured as a post. While the post is atypical, the ARPA series would be incomplete if I did not outline the factors that mark key regime changes at the agency over time. The piece is not an exhaustive account of every regime change in DARPA history, but details three major variables that mark regime changes and provide some illustrative examples. Most of the examples are drawn from the early history of computing at DARPA because it is both well-known and well-documented. Enjoy!

Subscribe now

Share

DARPA doesn’t exist in a vacuum. It is enmeshed in both the military ecosystem and the D.C. ecosystem. DARPA also has no labs; it is dependent on the community of performers (contractors) available to it — academic labs, companies, and others. Major changes in the politics, economics, incentives, or preferences of any of the above can substantially change what a DARPA office is at any given time.

It is a testament to DARPA’s resilience that it has been able to withstand so many changes throughout the decades, continuing to serve its mission in a less bureaucratic and more entrepreneurial fashion than the vast majority of the government R&D ecosystem. However, it is still essential that those seeking to understand historic DARPA successes understand the “regime” — a DARPA office’s political environment, performer ecosystem, etc. at a given moment — in which particular projects existed.

Through my research on DARPA history thus far, I have come across many small changes that have impacted what DARPA is at a given moment. But three factors have jumped off the page and proven salient enough that I will not sit down to write up a DARPA project from history without understanding them. They are:

1. The level of organizational oversight in a given period

2. The extent to which project “visions” came from office directors vs. PMs

3. The timeline in which projects were expected to pay off and how exactly a given office interpreted DARPA’s mandate that projects serve the military in some way

Today’s piece will not cover any specific project or contractor from DARPA history. Instead, I will detail how changes in these three factors alter how DARPA operates. I will provide examples from oral histories of long-time DARPA employees and performers as well as DARPA historians describing the specific effects of changes in these factors. Many of the specific examples will draw on DARPA computing history and the Information Processing Technology Office (IPTO) —responsible for many or ARPA’s now-famous early computing projects.

I’ll begin by exploring what might be the most substantial shift DARPA has ever experienced: the increase in bureaucracy and procurement rules following the Vietnam/Watergate era. These shifts fundamentally changed how DARPA PMs were allowed to operate.

Image Courtesy of a 2018 tweet from DARPA celebrating the 46th Anniversary of the agency’s 1972 re-brand from ARPA to DARPA.1. From a Small World to a Bureaucratic One

The post-Vietnam and Watergate shifts in Washington D.C. mark a clear “before and after” in DARPA history. Regulatory crackdowns and increased oversight became typical in this period. Much of this oversight impacted the entire government or the entire DoD at once. Regardless of whether or not DARPA had done anything wrong in the public’s eye, the government had. The sweeping reforms left DARPA a very different agency — from the perspective of both its PMs and its performers. While DARPA remained a more fast-acting and less bureaucratic place than many other agencies in D.C., it became noticeably more bureaucratic than it once was.

Procedural Changes (e.g. 1984 Competition in Contracting Act)

This general change in political attitudes had begun to steadily impact DARPA operations throughout the 1970s, but among the years of steady changes there were also abrupt shifts. One example was the implementation of the 1984 Competition in Contracting Act. To DARPA PMs and longtime performers who served both before and after the changes, this shift is often seen as demarcating where an old DARPA stopped and a new DARPA began. While it is reasonable for one to believe that the ramping up of oversight in how government funds were spent was, in principle, good. It’s more difficult to argue that the changes did not substantially impact the speed and freedom with which DARPA PMs could source, iterate upon, and fund ideas.

Even the procedural changes that seemed, on the surface, very reasonable had a noticeable impact on the speed and fluidity of DARPA’s operations. One example of this was the new requirement that agencies publish a notice for all procurements above $10,000 in the Commerce Business Daily. Once published, PMs would only have to wait thirty days for bids to be solicited and could then begin negotiations. Roland and Shiman — who wrote the book on DARPA’s Strategic Computing initiative and extensively interviewed many of the major staff and performers of 1980s DARPA IPTO — wrote the following on the impacts of this change:

Paired with the Competition in Contracting Act, this legislation transformed the old IPTO-ARPA style of funding research.

In the salad days of IPTO during the 1960s and 1970s, program managers solicited proposals in the research community, often by direct contact with the researchers. An office director such as J.C.R. Licklider, or a program manager such as Larry Roberts, for example, might call up a researcher to ask if he or she would be interested in doing a project for ARPA. If there was a meeting of the minds, the researcher would submit a proposal that was essentially approved before it came through the door. Program managers achieved results by knowing what was going on in the research community and who could deliver. They also accepted unsolicited proposals coming over the transom if they were good and suited ARPA’s purposes. And they put some contracts up for competition, as they did the ARPAnet contract that BBN won with Robert Kahn’s help. The style, however, was still less formal than at agencies such as NASA or even elsewhere within the DoD.

With the implementation of the Competition in Contracting Act, far more DARPA contracts were now formally competed. As a result, a wave of new federal procurement laws now applied to these contracts. This particularly impacted an organization like DARPA because DARPA staff not only fund work on the technological frontier, but come up with plans to shape it. DARPA PMs had previously relied on their ability to work out what a contract might look like with the performer community before a formal contract was ever written. Now, all the performers were legally competitors; how DARPA PMs could interact with them once a bid was out had changed. Roland and Shiman describe how this change in the legal relationship between PMs and the performer community impacted project creation, writing:

Particularly crippling to DARPA was the requirement in the new law limiting contact between procurement officers and potential contractors. Such contact was the taproot of the PM’s success. It was by staying in touch with the community, by letting the community know what was needed at DARPA, by finding out what the community was capable of doing that the PM was able to map out a reasonable research agenda in his field and sign up the best workers to make it happen. In the new dispensation, such networking could be viewed as collusion. PMs in the future would simply have to be much more careful, much more circumspect, much more correct when dealing with researchers who were potential contractors. This did not make their jobs impossible, but it did make them harder and less enjoyable.

“Harder and less enjoyable” is a succinct way to describe the differences in the lives of a DARPA PM of the 1980s vs. a DARPA PM of the late-1960s. If it were truly critical, there were procurement levers DARPA directors, office directors, and PMs could pull to make DARPA work the way they needed it to work. As one example, directors could use an OTA-like authority to get around particular pieces of red tape if it was truly important. But directors were also aware that this should only be done sparingly. Also, if a technicality got in the way of common-sense project management — like the above regulation which negatively impacted PMs’ ability to brainstorm with performers when putting together contracts — entrepreneurial individuals at DARPA often found ways to at least approximate their old common-sense workflows in legal ways. In fact, this is how DARPA’s Broad Agency Announcements (BAAs) came to exist. To get around the aforementioned issue, Stephen Squires came up with a technique for soliciting proposals — once described as “a cross between an RFP and a fishing expedition” — which took hold at DARPA.1 It eventually evolved into what is now the BAA. While the BAA was a wonderful organizational innovation, it was only necessary because of the aforementioned changes in procedure.

There was now an added step to the game of being a DARPA PM. PMs and office directors could still be proactive and ambitious, but they often had to devise strategies to make their projects work within the confines of the new legal procedures and red tape. By the late 1980s, many longtime DARPA affiliates felt the PMs could no longer do precisely what they wanted exactly when they wanted to do it. These shifts in procurement laws and contract management contributed to DARPA becoming a place in which funding could easily be given to promising ideas in days or weeks, to one where months was increasingly becoming the norm.

Of course, there were also games to be played in the prior DARPA regime. In the case of IPTO, for example, many researchers at computer science departments that did not receive IPTO’s Centers of Excellence grants — which Licklider’s IPTO office had used to foster top-tier computing departments when the field was in its nascent stages — complained that they were unfairly overlooked. To overcome being outside of the network was also a game, to be sure. A researcher might need to go further out of their way to put themselves on a PM’s radar. But the new procedures ushered in a game that was quite different in nature. The following subsections detail some of the performers’ and PMs’ perspectives on this new game and how it had to be played.

The Performer’s Perspective On These Changes

In their oral histories, often recorded once they had retired, long-time DARPA performers often listed this change in procurement bureaucracy as the single factor that most threatened DARPA’s future effectiveness.

In a series of interviews with those who played major roles in BBN’s work on the famous ARPAnet contract, two separate BBNers used the free air the interviewer provided them to put on the record how bad they felt these changes were for DARPA. In his 1990 oral history interview, Frank Heart, BBN’s project leader for the ARPAnet contract, described what he believed to be the increasing silliness of the government’s approach to technology contracting:

[Interviewer]: Okay, that covers the questions I had, unless you have any general comments you would like to add to the record.

HEART: As I say, the only thing I would emphasize is that if people try to look at all the projects that this country does in the computer world, and all the failures there have been, all the large military computer systems that have fallen on their face, or had a factor of ten overruns in time or money...If you look at all those things, and then you ask, "Why do these few make it and be successful, and why do all those not be successful?" That's an important thing, if one could somehow get that lesson across to Congress and others, it would be kind of nice. It's very hard apparently because they keep doing it the wrong way, mostly. What happens is that people who know nothing about the technology write long lists of requirements. Then they go out to industry and ask industry to bid on these silly requirements, and they get back silly bids. Then they pick the low bidder of the silly bids, which often is the one who understands the technology least; and then ten years later they have a disaster. Instead of people who understand the technology very, very well in the government who are able to drive it to the groups that are the right groups, even if it isn't the low-price bid, and who take close control of it, who don't have an enormous list of specs, and then other people who never wrote the specs, but who have a participatory prototyping kind of relationship with the thing where it grows. If one could get across that lesson, it would be kind of nice. But I don't hold much hope for that.

[Interviewer]: Can you give me any examples of what you consider big failures, using this kind of method?

HEART: Well, gee, there are probably... No, I probably can't do that, just because I wouldn't want to be quoted. But, I mean, they're legion. The number of defense systems where they've tried to build computer systems, and they've been horrible, there are probably lists twenty or so long. I probably wouldn't want to be quoted on naming them, but it's many, many, many subsystems that have been ten years late at ten times the cost — things like that. The ARPANET went from a contract award to an installed operating system in nine months. It went from a contract award to equipment being delivered, on site and running, in nine months.

Heart also believed that working with DARPA, in 1990, was still quite a positive experience and not nearly the headache it had become to work with other pieces of the government that supported research. But the following quote from Alexander McKenzie’s 1990 oral history — Mackenzie was BBN’s so-called “ARPAnet generalist” — makes it clear that these procurement changes had pronounced negative effects on DARPA performers, compared to how things used to be. McKenzie’s interview reads:

[Interviewer]: We have covered what I wanted to ask you about. I want to open it up and ask if there's anything you want to add, any general comments you would like to make about the ARPANET, or your involvement in it, or ARPA in general.

MCKENZIE: I think that the example of the way ARPA worked in those days, in the 1970s — not only in communication but in a lot of other programs — was pretty effective in terms of cost and return on investment. That the model of some really bright people in the ARPA office, with some particular goal in mind, something they thought would help the country, or the DoD, being given the authority to go out and find the smartest people they could find who were interested in working on that problem, and then giving them relatively free rein to do some research or some development, produced pretty remarkable results. And I know that there has been fraud and abuse in government and in contracting and so forth, but it seems to me that the kind of rules and regulations that there are now, that are attempting to prevent that, really make it very difficult for the government to get the same kind of power out of its research dollars these days as it was able to then. I know it's hard to find a balance between accountability and free rein, and these days the government approach seems to be more on the side of accountability and less on the side of free rein. But I think that a lot is being lost.

Mckenzie continues by sharing a quite specific example that he felt characterized the kind of thing DARPA eventually began to worry about that it hadn’t in the 1960s:

I remember a discussion that I had with Jon Postel back in, I don't know, the early 1970s. We were talking about travel to a Network Working Group [which helped come up with the early internet protocols] meeting. And he was saying that he really would like it — maybe it was a subgroup meeting — if it would be some place rather than some other place, because the government accountants were getting university accountants to not want them to spend so much on travel. And his comment was, "I know that in my contract there's only a fixed amount of money. I want to use that money the most effective way possible. I don't know why they think that, when my salary is paid out of this, that I'm going to waste money on unnecessary travel. I want the money for my salary; I want to keep having a job, but sometimes the travel is necessary. They don't seem to understand that that's the case. They think it's just a boondoggle." I think there's a lot to that attitude. That, yes, we spent a lot of time in Network Working Group meetings hammering out ideas, but something really good came out of it.

Mckenzie closes by explaining how, in his eyes, the previous DARPA regime was ideally set up to keep costs down in its own way, while staying ruthlessly focused on outcomes and not process:

BBN wasn't required to file monthly fund expenditures reports and justify every decision on the basis of cost effectiveness. We could pick the Honeywell 516 because it was OK — it was an OK processor for this. It would do the job, and it wasn't, maybe, the most cost effective and we didn't have a subcontract bidding plan in effect to give everybody the opportunity to bid, and thereby put another three months of delay into the schedule - we could just decide. I think that was pretty valuable. If you're talking about multi-million dollar projects, it's probably more important to pay attention to those things than it is in tens-of-thousand or hundreds-of-thousand dollar projects. But, even there, there's probably things where better results would come to the government if they could find a way to shift back a little bit more towards accountability for the results, rather than accountability for the pennies. I think that DARPA in the 1970s did a really good job for the country in that way. It was a joy to be associated with the ARPANET project. It was fun. It was challenging. And I think it was good for the country. It's not so easy to find that mix now, and I think regulation is a big part of it.2

The negative opinions on these bureaucratic changes at DARPA were not limited to its performers.

The PM’s Perspective

As these procedural changes began to take effect, the life of a PM became “harder and less enjoyable” in many ways. Opinions varied on how annoying the changes were. Many PMs thought the changes were horrendous for the pace and cost-efficiency of projects. Some, such as IPTO’s Ronald Ohlander, expressed less polarizingly negative opinions. Ohlander — whose budget more than tripled from $14 million to $55 million as DARPA’s computing applications budget grew — felt this increase in resources granted him many more opportunities than the increased paperwork and bureaucracy took away. But even after taking moderate views like Ohlander’s into account, it is clear that the DARPA of the mid-1980s operated with the increased computing budget much differently than the DARPA of 1970 would have.

PM Paul Losleben’s thoughts on the new mix of resources and bureaucracy were clear:

I…had twice the budget that I had had before…and I was accomplishing less because I had lost the freedom to build the program, to interact, to define direction, to work with the community.3

Ohlander, on the other hand, merely complained that he had to hire more staff than he otherwise would have to keep up with his budget increase. But, with that being said, some of the difficulties he experienced managing the ill-fated Autonomous Land Vehicle (ALV) project may have been mitigated if he could operate more like a ~1970 ARPA PM. The original ALV project had to be scrapped largely because of the mismanaged expectations and misaligned incentives of both Martin Marietta and the university vision researchers. As I covered in a prior FreakTakes piece, Martin Marietta turned out to be a prime systems integration contractor with little interest in implementing cutting-edge component technology. Martin pressed DARPA to specifically define intermediate benchmarks for the technology. Martin would then find ways to hit these benchmarks using minimal cutting-edge technology, not trying to satisfy the spirit of the contract. Meanwhile, many of the university-based computer vision researchers cared more about developing algorithms that were interesting in a basic research sense than they did about developing algorithms to make the vehicle work. Their algorithms often did not perform when plugged directly into machines like the NavLab at CMU.

Whether or not these failures would have happened absent the new procedures, at the minimum the new regulations exacerbated these issues. Roland and Shiman describe how these procurement changes negatively impacted Ohlander’s ALV project:

The new methods of procurement…made Ohlander’s task doubly difficult. In a previous era, he would have held informal discussions with the researchers at an early stage in the planning process, so that contractors understood what was expected of them and what they would contribute. Such a casual approach was no longer possible. The potential contractors were all competitors now. Ohlander could gather the researchers together to help write the [project] plan, but he could not discuss specifics with them about who would do what. The researchers themselves would have been very reluctant to talk, especially with their potential competitors. Ohlander could not be certain that the successful competitors would work well together. He had never worked at all with some of them, especially those in industry, such as GE and ADS. All he could do was to specify in their contracts that they were required to participate in open discussions and exchange information freely with the other contractors. They [DARPA] would have to trust in his [Ohlander’s] ability, as the source of funds, to twist arms if necessary. Running SCVision in two phases made this easier, as it allowed DARPA to cut off recalcitrant or underperforming contractors after only two years.

The multi-phase structure of the project proved to be essential. At the end of the first phase, DARPA would have to fire the prime contractor, Martin Marietta. While many of the academics continued to receive their basic research funding, they gradually became only indirect contributors to DARPA’s main autonomous vehicle project. Their research was often not as applied as DARPA needed it to be. The spirit of the ALV project primarily lived on in the autonomous vehicle teams Chuck Thorpe and others were building at the CMU Robotics Institute. In the CMU Robotics Institute, Raj Reddy had built up a research operation that valued what DARPA wanted it to value: cutting-edge research alongside practical systems engineering work. In CMU, DARPA’s autonomous vehicle staff eventually found a one-stop shop that could manage the practical aspects of the project and many of the research portions. That was great. But more time and money likely had to be spent on the wrong performers than may have been necessary under DARPA’s old procedures.

In 1984, as the Competition in Contracting Act was being implemented, a Science reporter named Mitchell Waldrop — now of Dream Machine fame — wrote an article that characterized the feeling of DARPA PMs and their performers as “confusion and heated tempers.” One side was frustrated as they figured out how to write bids for government contracts while the other tried to figure out how to evaluate them in a legal and auditable fashion. ARPA’s office staff and performers had begun to spend far more time checking compliance boxes than they ever had before. Bureaucratic obstacles stood between the PMs and their beloved performer communities. Things had changed.

To learn from any project from ARPA history, it is crucial to understand whether the project took place before or after these procedural shift and how that affected a PM’s project management.

2. Office Director’s Visions vs. PM’s Visions

DARPA has become quite famous for the freedom of its PMs. Many advocates of the ARPA model believe a PM’s freedom to pursue their own unique vision is an emphasis that DARPA has always had. However, in DARPA’s early decades, in particular, the freedom to decide which project areas to pursue often fell to its office directors, not individual PMs. But even in cases when specific goals were dictated to PMs by office directors, the PMs still generally had the freedom to decide exactly how to go after those goals.

There are pros and cons to leaving the job of “visionary” to office directors vs. the distributed cohort of PMs. The former may provide an office with fewer unique opinions but ensure that the whole office is working in a coordinated fashion towards a few primary goals. The latter ensures that a given cohort of PMs is free to pursue as many unique ideas as possible. Many offices from DARPA history tend to fall somewhere in between the two extreme ends of the spectrum.

A prominent example of a relatively director-driven office was IPTO in its early decades. Early IPTO office directors were picked because they could both provide a strong vision for the office as well as effective leadership of the PMs. Roland and Shiman describe the different directions in which early IPTO directors steered the office, writing:

J. C. R. Licklider revolutionized computing by aggressively supporting his twin hobby horses of time-sharing and man-machine symbiosis, and by proselytizing for his vision of an “intergalactic computer network.” He recruited Ivan Sutherland to do the same with graphics. Strong successors such as Taylor and Roberts ensured that artificial intelligence (AI) received the support it needed well into the 1970s. The payoff from AI was slower to come than in time-sharing, graphics, and networking, but IPTO kept faith with the field nonetheless.

These IPTO directors often fostered beneath them a few academically-decorated PMs with visions of their own which they were green-lit to pursue. But, oftentimes, there were even more PMs stewarding the visions of others than were encouraged/expected to have visions of their own.

Three examples of projects that were dreamt up by someone other than the PMs who ran them were the Pilot’s Associate, Battle Management, and Autonomous Land Vehicle projects — all covered in a prior FreakTakes piece. Several individuals across IPTO, in consultation with the armed forces, had a hand in dreaming up these projects. The most important of these individuals was probably Clinton Kelly. Despite this, Kelly became the PM of none of them. He helped guide them, in a way, but was not the official PM. Despite the fact PMs like John Retelle (Pilot’s Associate), John Flynn (ship-level Battle Management), Al Brandensteein (fleet-level Battle Management), and William Isler (ALV) did not come up with the ideas themselves, they still had relatively long leashes to execute their projects as they saw fit. Despite not having the freedom to come up with projects from scratch, their freedom was more similar to that of a SEAL team commander who might not get to choose which objective to take, but often has the freedom to decide how to take the objective. In fact, PMs from the armed forces found themselves in the position of stewarding others’ visions relatively frequently.

A major upside of office directors coming up with a vision and several PMs being expected to execute that one vision is that PMs could be assigned to projects that they might not 100% believe in. This can be useful. There exist cases that might draw appropriate skepticism from all individual PMs, but which it is optimal for the office portfolio to pursue nonetheless. I’ll provide an example of a program which was roughly this shape — one in which the right course of action required PMs to pursue all possibilities, however unlikely.

In the early 1980s, IPTO director and chief visionary Robert Kahn thought that computers needed about a three order of magnitude increase in performance before they could truly approach AI-type applications. There were three paths that he felt could make this happen. To be thorough, he felt it made sense to fund all of them — even though some seemed far more likely to work than others. The goal was important enough that he felt no stone should be left unturned. Option 1 was to improve chip performance, funded through the VLSI program. Option 2 was to speculatively design new software approaches that could be run on later generations of machines more suited to the approaches than present machines. Option 3 was considered the most promising option. Option 3 was to design and build parallel architectures. With the office largely following the vision of one or a few people, and not every single PM, Kahn was easily able to ensure each of these approaches was pursued thoroughly — even if options like speculative software design felt risky and unlikely to pay off.4

In IPTOs early decades, such as in the 1980s, it was also not uncommon for PMs to be called in to manage projects that were the vision of another PM. PM Ronald Ohlander’s portfolio of AI projects was one example of this strategy. In the following excerpt, Roland and Shiman describe what was expected of the non-visionary PMs brought in to help Ohlander manage his then-swelling AI portfolio:

All three men were typical of the active-duty military officers who did one or more tours at DARPA on assignment from their services. They served beside civilian program managers who came to DARPA from industry or academia…The civilians usually came with more powerful credentials in their technical fields; this often meant that they had technical agendas of their own and attempted to shape the field while at DARPA. The military officers came with a better understanding of the clientele for DARPA’s products…They could translate service needs for the computer science community and translate technical capabilities to serving officers. Sears played the latter role. He took over programs for which he had little formal training, but he educated himself on the job and relied on his common sense and management experience to keep the programs on track. Most importantly, he had to recognize good excursions from the plan, to know when a project was making better-than-expected progress and needed more resources to exploit the opportunity. Just as importantly, he had to know when a project was failing and had to be invigorated, redirected, or abandoned.

While many seem to intuitively understand the pros of PMs’ freedom to have visions of their own throughout DARPA history, it’s important to understand that this is not something that has been true of every office throughout DARPA history. For example, DARPA’s golden goose, early IPTO, did not always work this way. The individual freedom of PMs has huge upsides, but so does the entirety of an office’s PMs working in a coordinated fashion to accomplish a few goals. There is room for both approaches in any ARPA-like organization; the state of a field, an office’s goals, the personnel available, and the set of institutional circumstances an office finds itself in will determine which approach is optimal.

3. Payoff Timelines and DARPA’s Military-Focused Mandate

The final section details a variable that heavily dictates which projects are in scope for a PM and how a project is pursued: a DARPA office's attitudes towards payoff timelines and dual-use technology. This is often dictated by things like the preferences of DARPA directors and the political moment an office finds itself in.

Changes in acceptable payoff timelines for a project can come about for a variety of reasons. For example, wartime or a change in political winds can lead administrations to push for more “fiscal responsibility.” A common response to this is for DARPA directors to emphasize project plans that can produce applications quickly. On the flip side, certain directors and presidential administrations have been known to openly embrace dual-use technologies and provide generous payoff timelines. DARPA’s exceptional track record often earns the agency a longer leash than is typical. There is also a lot of middle ground between these two extremes. It is not uncommon for certain offices within DARPA to maintain quite short leashes and keep a clear focus on military practicality while, simultaneously, other offices exercise long leashes and focus on more dual-use technology with unclear time horizons for payoffs. This was the case in the 1970s in which the applied Engineering Applications Office (EAO) coexisted with the risk-embracing IPTO.

Early IPTO’s Dual-Use Ethos, the Mansfield Amendment, and George Heilmeier

Obvious shifts in timelines and attitudes towards dual-use technology might abruptly follow a change in the President and/or DARPA director. One example of an abrupt shift in how military-centric DARPA was expected to be came in the early 1970s. In 1970, with the Vietnam War waging and amid a generally troubled political environment, the Mansfield Amendment was passed as an addendum to the Defense Appropriation Act of 1970. The amendment proposed that the DoD should not be allowed to conduct any basic research. While the amendment was repealed within the year, the point Congress intended to make was clear. In response, ARPA soon changed its name to DARPA. The D in DARPA was no longer silent.5

Throughout the 1960s, Licklider and prominent IPTO PMs, such as Larry Roberts, had openly funded the most promising and useful research they could find, often without too much thought to how military-focused it was. In general, the IPTO office visionaries were true believers in the technology and its eventual widespread usefulness in all areas of society. So, they were generally happy to grant funds to any researchers who would push the computing field forward. The general, dual-use ethos that drove these visionaries was something along the lines of, “What’s good for the country will be good for the military.”6

As we now know, this attitude proved to be spot on in the case of computing. When the field of computing was in its nascent stages and much still needed to be worked out, IPTO gave massive block grants to computing “centers of excellence” like MIT, Stanford, CMU, University of Utah, and others. While many somewhat applied projects and direct asks were made of the centers of excellence which were often funded by the block grants, there was a lot of excess funding from the grants which the departments were free to use to pursue their own interests. Notably, the ARPAnet contract itself was pursued by Larry Roberts largely with general use cases in mind. The driving force behind the project was not exactly a military-specific need, but the idea that a broader communication breakthrough like the ARPAnet must improve military operations somehow. Despite the success IPTO had with this approach, this style of operating soon became more difficult.

While the Mansfield Amendment was repealed, it succeeded in forcing changes on DARPA beyond making the D in the name no longer silent. At the minimum, it was now clear to DARPA PMs and office directors that they were being watched. They were now careful to justify projects with specific military applications in mind and find more development projects. Larry Roberts described some of the changes as follows:

The Mansfield Amendment…forced us to generate considerable paperwork and to have to defend things on a different basis. It made us have more development work compared to the research work in order to get a mix such that we could defend it…The formal submissions to Congress for AI were written so that the possible impact was emphasized, not the theoretical considerations.7

However, Roberts did note in his oral history that while the mix of funds did shift to become more applied, for his personal budget DARPA found additional money for applications and that his basic research budget was not cut. While this applied shift felt abrupt to PMs at the time, a modern PM would probably classify this era of DARPA history as relatively exploratory.

This applied shift, which came in the wake of the Masnfield Amendment, ratcheted up a level in 1975. This is when DARPA appointed its most well-known director: George Heilmeier. Heilmeier would become famous for exercising a very heavy hand in deciding which projects would be pursued at DARPA along with his famous Heilmeier Catechism. Office directors and PMs now needed to have satisfying answers to Heilmeier’s six questions related to things like current technological limitations, applications, and intermediate measures of success if they wanted funding for a project. While none of the principles in the Heilmeier Catechism were new to DARPA, the expectation that each of the six questions had a thorough answer was much stricter. Roland and Shiman attempted to describe many PMs’ views on the change, writing:

Heilmeier’s catechism was not unreasonable, but he reportedly applied it with a draconian enthusiasm that reflected a chilling political climate settling over Washington in the first half of the 1970s.

While Heilmeir’s DARPA may not look too strict to modern eyes, it felt very strict compared to prior ARPA regimes. Heilmeier believed DARPA should look much more like mission-driven agencies, such as NASA, than a basic research funder like the NSF. He thought DARPA had unreasonably broadened its scope. He even mocked what had come of DARPA, derisively calling it “NSF West.” Under Heilmeier, PMs were pushed harder to get their exploratory research out into the field as applications. Funding exploratory research with only a hazy idea of what the applications could look like became a taller task than before. For those in D.C. who felt DARPA could be perceived as spending recklessly, Heilmeier was just the man to rein it in.

But Heilmeier’s tenure at DARPA should not be viewed as an event that marked some monotonic decrease in DARPA’s embrace of exploratory, dual-use technology. While the federal government norm is often for things to grow only more onerous over time, in DARPA history it is quite common for operating rules mandating a certain level of strictness to abruptly become less onerous.8 The period coinciding with the departure of Heilmeier as DARPA director and the prospective appointment of Robert Kahn as IPTO Director brought about one of these shifts on an office level.

Under Heilmeier, basic research funding from DARPA had become insufficient by the standards of Kahn and the computing community. New DARPA Director Robert Fossum was sympathetic to this viewpoint and was happy to bring Kahn on as IPTO director and honor his one condition to take the job: that DARPA increase basic research funding in computing. Roland and Shiman described how this shift in how applications-focused IPTO was occurred, writing:

In congressional testimony in 1981, Fossum reported that DARPA funding in constant dollars had fallen steadily from 1970 to 1976; it had not yet recovered its 1970 level when Kahn became IPTO director in 1979. The post-Vietnam heat of the mid-1970s was abating, Fossum did not share Heilmeier’s catechism or his pessimism about artificial intelligence, and Fossum wanted Kahn to take the job. By 1982 the IPTO budget had climbed from its 1979 low back up to a respectable $82 million.

These changes in an office’s focus, which are often the result of non-technological factors, not only impact which projects enter the DARPA funnel and continue to receive funding, but also the life cycle that a given project takes. In the case of DARPA’s 1980 Strategic Computing Initiative and the underlying pressure to have clear applications for the program, this pressure likely had both positive and negative impacts. When assessing DARPA’s impact on the development of autonomous vehicles, the pressure may have been for the best. While the Autonomous Land Vehicle project did come up short, the push to get vision applications into the real vehicles quickly allowed CMU to prove itself to be DARPA’s great systems integration contractor in the field of autonomous vehicles. On the flip side, in the case of the Pilot’s Associate program, the PM in charge of the program thought the premature push for a mature prototype took a program that was making solid technological leaps and forced it to abruptly implement a system that was not as cutting-edge or intelligent as it could have been otherwise.

In very obvious ways, variations in DARPA’s attitudes towards dual-use technology and the pursuit of projects with long-run or unclear payoffs impact which projects get chosen and how they are pursued. But one would be incorrect to view some eras of DARPA as simply better or worse places for the most ambitious research. For example, while some (too crudely) might view Tony Tether’s wartime DARPA agency in the 2000s as a more difficult place to pursue ambitious programs — given its use of procedures like budget wedges and emphasis on programs that might pay off in the current war — an entrepreneurial PM like Geoff Ling proved this was an ideal time to pursue one of DARPA’s most ambitious programs ever: the Revolutionizing Prosthetics Program. Ling’s program made major breakthroughs in neurally integrated upper limb prosthetics and brain computer interfaces (BCIs) that helped lay the base for some of the ambitious BCI projects of today.

While the impact of an office’s attitude on dual-use technology and payoff timelines is somewhat intuitive, one important wrinkle to consider is whether the customer (the military) must actually agree an application is desirable before a project is funded. Whether or not an office can pursue a project despite the military being lukewarm or cold on an idea has a massive impact on what sorts of projects get pursued.

A military-focused office can often say “no” to the military

Military-focused project selection and listening to the military are not always the same thing. Different DARPA regimes and offices can have varying abilities to say “no” to the armed services. In the case of the Pilot’s Associate, the Air Force did want it and indicated they‘d be willing to actively participate in its development. In the case of the ALV, the Army did not express much desire for an autonomous recon vehicle — at least not strong enough to warrant DARPA’s spending on the vehicle — but IPTO staff had a strong belief that the Army would want the ALV once they saw it in action. So, DARPA was comfortable making the ALV application the centerpiece of its SCVision portfolio. This ability to say no to the army in the 1980s DARPA computing portfolio stands in stark contrast to the accounts of some PMs who worked under directors like Tether. Tether often mandated that PMs obtain a budget wedge — a formal promise by an armed service to continue funding a particular project if the R&D benchmarks reached a certain point — before beginning to fund many projects.

In Tether’s case, the recent 9/11 attacks and the country transitioning into wartime allegedly had a strong impact on this decision. Ling’s prosthetics program was the right kind of ambitious for the moment; meanwhile, overriding the no of the military to build seemingly futurist technology the Army said it did not want likely would not have been. One cannot truly understand and apply the lessons of any project’s success or failure without understanding the office’s attitudes towards dual-use technology, payoff timelines, and saying no to the customer in a given era.

Concluding Thoughts

DARPA is no longer a young government agency. In a difficult environment, it has managed to remain a place where bright minds take risks, act (relatively) nimbly, and get things done. The agency continues to specialize in pursuing projects that are big-if-true, fall through the cracks of the standard R&D ecosystem, or might succumb to the valley of death without DARPA’s intervention. The agency’s history is one we would be foolish not to learn from.

To adequately learn from this history, it’s crucial we understand what changes and what stays the same about the agency over time. Certain key factors have remained constant throughout (D)ARPA’s life. The twin constants that, to me, define the ARPA model over time are the use of a clear customer relationship as the agency’s North Star and the use visionary PMs (or office directors) with agency instead of passive funding panels. But it is possible that there are even more key variables across DARPA history than there are constants. Today’s piece outlines what I believe to be three of the most important ones those. Those looking to learn applied lessons from the agency’s history should keep them in mind.

Thanks for reading:) As always, I’d be eager to discuss/assist PMs or PM-like individuals in any way I can. Just reach out to me on Twitter (eric_is_weird) or via email ([email protected]) to chat.

Announcements

An Announcement on RenPhil’s AI For Math Fund. Talented and curious readers should eagerly apply for grant funding from RenPhil and XTX Markets’ AI for Math Fund. The funders are particularly interested in those working on software, datasets, field building, and big-if-true projects to build AI technologies useful to mathematicians.

A Coming Piece Will Cover Constants Across DARPA History. While the focus of this piece was on how DARPA has changed over time, there are also important constants throughout DARPA history. I will say more about these in a brief follow-up piece on FreakTakes. For now, I’ll simply say that the two most notable constants are probably:

1. DARPA’s continued commitment to replacing the panels typical in government R&D grant-making with PMs with agency

2. Its use of a clear customer relationship as a strong North Star for its projects

Subscribe now

Share

Appendix: A Note on Other Headwinds In The 1980s

Regulations are not solely to blame for the initial negative results of the ALV project, and projects like it. In the mid-1980s, Ohlander was fighting against two other sweeping trends that surely made the life of a PM for a project like the ALV harder. Many groups that would once have been more aligned with DARPA’s incentives were gradually drifting in scope.

The first of these headwinds was the gradual drift of university labs away from applied research contracts. With post-war federal basic research funding continuing its explosion, universities were leaning more and more into basic research funding and further away from the applied research contracts that were a key source of funding in prior decades. University labs were gradually growing less well-suited to DARPA-style contracts that required a very applied technological component — like operating a working vehicle in which to plug academic vision algorithms.

The second headwind PMs like Ohlander were fighting against in the 1980s was an increasingly myopic sense of what projects corporate R&D should pursue. A 1970s recession, the 1980s “financialization” of management in engineering organizations, and likely other causes saw corporate research drift in its ambition. My prior FreakTakes piece on Lockheed’s Skunk Works provides an interesting case study of what these pressures looked like from within a defense prime. It was even difficult for Kelly Johnson and the famous Skunk Works to fight against them as the decades wore on.

Both of these trends happening simultaneously contributed to an organization like BBN’s ability to recruit top academic engineers from places like Lincoln Labs so convincingly. BBN’s recruits simultaneously felt that Lincoln Labs did not work on truly useful technology, but also that their minds would be wasted at standard corporate R&D departments like Honeywell’s. BBN was one of a small number of organizations that still existed to work on truly novel, truly useful applied research contracts. These two simultaneous trends likely had their own negative impacts on DARPA on top of the negative impacts brought on by the new procurement rules. But, at the minimum, the growing apart of industry and academic research made it even more vital that DARPA keep a strong hand in coordinating — and when needed, twisting the arms of — performers on an ongoing basis. The new procedures made this much more difficult.

General Links

Beyond the specific citations above, much of the general knowledge in the piece is indebted to the great books and oral histories on early IPTO conducted/written by Judy O’Neill, Arthur Norberg, William Aspray, Alex Roland, and Philip Shiman:

• University of Minnesota IPTO oral histories can be found here

• Transforming Computer Technology: Information Processing for the Pentagon, 1962-1986

• Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983-1993

1

Roland and Shiman wrote this, but it’s unclear if the quote came directly from a PM they interviewed or if it was their distilled explanation of BAA descriptions given to them.

2

Emphasis my own

3

Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983-1993

4

I simply use this example to describe a shape of a situation from history where the office director as chief visionary could be advantageous. I do not know whether or not the PMs who pursued these projects were, in fact, working on their first choice of project or not.

5

This is a reference to a joke made by Robert Kahn in his oral history.

6

This is a general takeaway distilled by Roland and Shiman from the O’Neill, Aspray, Roland, and Shiman interviews.

7

Larry Roberts, “Expanding AI Research,” in Expert Systems and Artificial Intelligence: Applications and Management, ed. by Thomas C. Bartee (Indianapolis, IN: Howard W. Sams & Co., 1988), 229–230. I am indebted to Roland and Shiman for this citation, who are, in turn, indebted to the great David Hounshell for it.

8

This is, to me, one of the most impressive things about the agency.

ARPA’s early decades of success have made the ARPA model iconic. Big wins from ARPA’s early history include autonomous vehicles, the internet, and stealth aircraft technology. Inspired by these successes, scientific grant funders are increasingly emulating the ARPA PM approach to R&D funding. While I wholly support the proliferation of the ARPA model, it's important to not overlook a key lesson of ARPA history: many exceptional ARPA projects resulted from exceptional contractors.

By "exceptional," I mean contractor groups who were not just staffed with elite talent, but uniquely aligned with ARPA’s mission. All three aforementioned ARPA success stories had contractors — which DARPA calls performers — that shared three distinct traits, rarely found together. Each performer:

1. Was novelty-seeking, with a strong preference for projects that pushed the technological frontier forward substantially.

2. Built useful technology for actual users. This entailed professional contract management and a willingness to focus on difficult systems engineering tasks.

3. Used more flexible team structures than academia. When compared to academia, they more effectively hired, organized, and incentivized researchers, engineers, and other experts to collaborate on applied projects in a common-sense fashion.

I call orgs that check all three boxes BBN-model orgs — named after Bolt, Beranek & Newman (the ARPAnet contractor). In this piece, I make the case for the BBN Model and why it can be a fantastic complement to the FRO Model for the R&D community.

In the first section, I outline how BBNs compare to FROs and the bottleneck they address. In the second and third sections, I'll summarize how historically-great DARPA performers like BBN and CMU's early autonomous vehicle teams used the approach to set themselves apart from other academic departments and firms. In the fourth section, I'll list examples of problems this model is optimized to pursue and briefly discuss what is needed to get BBNs off the ground.

Subscribe now

Share

I've limited the scope of this piece to simply introducing the concept of BBN-model orgs. While this approach does not apply to every area of R&D, the idea is already finding traction with a small number of founders and funders. Just last week, a bootstrapped BBN, Amodo Design, was made one of nine ARIA activation partners — alongside groups like DeepMind and Convergent Research. I’ll share more details about Amodo and other aspiring BBNs in coming pieces. In the meantime, if you are an entrepreneurial scientist or philanthropist who would like to discuss this topic, please reach out ([email protected]).

Thanks to Thomas Milton, Corin Wagen, and Adam Marblestone for their comments and ideas on the draft.

BBNs: A Complement to FROs

In 2022, Adam Marblestone et al. published their now well-known Nature piece detailing the vision for Focused Research Organizations (FROs). The team made clear the problem they planned to address:

Our goal is to create a model to support an ecosystem of small-to-mid scale projects that fall between the cracks of what start-ups, academia and other organizations do.

The authors paint a picture of time-bound, nonprofit R&D startups wholly funded to pursue a well-defined goal — usually to build a specific thing like a dataset or tool. They point to "grand projects" like the Human Genome Project, Large Hadron Collider, Hubble Telescope, and Human Cell Atlas as having similar goals. Less than three years on from that piece, Convergent Research has brought six FROs into existence (and counting). Adam and the team have proven that you can recruit teams of the very best researchers to join these orgs, make them high-status, and get the operational scaffolding in place to begin doing great work with minimal delay. Convergent’s experiment in scientific structures is off to a great start.

But fundraising is a clear bottleneck to the creation of new FROs — often costing $20 to $100 million. There is, of course, an upside to this tradeoff. FROs raise so much money upfront to ensure they will have all the money they need to pursue their vision from day one. But until groups like universities or the NIH are convinced to set aside substantial war chests to fund and run FROs, fundraising will continue to be a bottleneck to FRO creation. In the meantime, many with the talent and technical vision to found and run FROs will not be able to do so if they are not one of the lucky few to complete a successful FRO fundraise.

BBNs offer an alternative path for these talented individuals. BBNs raise less money upfront than FROs. Instead, they heavily incorporate a contract research approach to build towards the ambitious technical vision of their founding field strategist. They can operate as firms or nonprofits, drawing on a mix of R&D grants and contracts to fund their technical agenda. Project by project, BBNs can make strides in key areas of R&D — as BBN did with real-time computing and the CMU teams did with autonomous vehicles.1 The approach limits BBN founding areas to those in which funders are eager to pay for contracts aligned with some ambitious technical vision. But where those markets do exist, BBNs can be spun up for an order of magnitude less upfront capital than FROs in many cases — some might even be bootstrapped.

The downside is that this introduces a level of ongoing financial risk that is not present in FROs. Founding a BBN is not advisable if you can successfully fundraise for an FRO or have an offer to run a team at the Arc Institute. Those are more stable options. BBN founders may need to be a bit scrappy or creative in their revenue strategies to survive and expand. They might have grand ambitions on the scale of FROs or the Arc Institute, but they must begin with plans that, in the beginning, operate on the scale of an NIH grant, philanthropic grant, or DARPA contract. But if done well, as with FROs, the BBN model can enable a “critical mass of scientists, researchers, and managers” to assemble and pursue ambitious R&D agendas outside of the university.

BBNs offer ambitious field strategists a chance. With BBNs, the chance offered by some contracts and a bright team can be leveraged to world-changing effect. While BBNs might even compete for the same contracts as groups like Raytheon, what differentiates the two is that BBNs are steered in ways that prioritize an ambitious technical agenda over revenue-maximization. This can be tough, but it can be done. The early career of the original BBN’s first computing field strategist, J.C.R. Licklider, demonstrates this. His steering of the BBN computing portfolio provides a clear example of how the BBN approach — with the right field strategist, team, and contracts — can drive an exceptionally ambitious R&D agenda.

BBN’s Computing Agenda: A Mix of Grants, Contracts, Talent, and Vision

BBN was founded in the 1940s. It began as a psychoacoustics R&D firm spun out of MIT by two professors. The firm began as a way for the professors to take on large contracts that were cutting-edge, but too involved to pursue in their own labs — concert hall design for the UN, cockpit design R&D for the Navy, etc. Over the next decade, the firm opportunistically expanded. Hiring for their firm as if they were staffing an academic department, the rule of thumb was to only make hires that would “raise the average level of competence of the firm.” In the late 1950s, the BBN partners looked to expand into computing. They believed J.C.R. Licklider was the best person to lead this effort. Licklider had a clear vision for the near future of computing. It was a vision that might have been too engineering-heavy to be optimally pursued at MIT.

BBN offered an interesting alternative. The firm had carved out a useful niche for itself, with a comparative advantage in projects that required both:

1. Cutting-edge academic knowledge

2. Professional engineering work or contract management

Licklider, a man of impressive vision, saw the administrative scaffolding BBN provided and believed it could be used to pursue his technical vision of real-time personal computing. This technical agenda — quite different from the batch-processing paradigm then in vogue — would require substantial engineering improvements in computing interfaces and real-time computing technology. Licklider left his tenured position at MIT to join BBN. Under his leadership, BBN’s computing efforts would go on to earn BBN praise from many former MIT researchers, later being described as “the third great university of Cambridge,” “the cognac of the research business,” and the true “middle ground between academia and the commercial world.”

A field strategist with an infectious sales pitch, Lick would gradually convince many MIT, Harvard, and Lincoln Labs researchers to defect and join BBN. This included a core of rare real-time computing experts, many of whom came from Lincoln Labs. While MIT and Lincoln Labs were great research environments, many who defected to BBN felt the nature of academia forced them to leave their best ideas as under-developed prototypes or “applied” ideas in papers. Many of these individuals — more engineers than pure researchers — would have felt their minds were wasted at a large firm like Honeywell. But BBN offered an interesting alternative to them as well. They could simultaneously work on the hardest problems and build useful technology.

Licklider convinced BBN to provide funds for computers that were relatively generous by the firm’s standards, but quite small given the scope of his ambition. The work began with a $30,000 Royal McBee ($300,000 today). Licklider soon convinced the BBN partners to buy another, more powerful machine that he felt would be more useful. After this second purchase, Licklider immediately proved the partners’ confidence in him was well-placed. One BBN founder described how seamlessly contracts fell into place following the second purchase, writing:

Lick and I took off for Washington D.C. to seek research contracts that would make use of this machine, which carried a price tag of $150,000 (~$1.5 million today). Our visits to the Department of Education, National Institutes of Health, National Science Foundation, NASA, and the Department of Defense proved Lick’s convictions correct, and we soon secured several important contracts.

Licklider’s computing group routinely found creative ways to offset the costs of investigations aligned with their technical agenda. Some of the group’s revenue came from relatively typical academic funding sources — federal research grants, a NASA contract to use the group’s computer to produce a textbook, etc. But BBN also took on much more applied contracts, well-suited to the group’s embrace of engineering and project management tasks. I’ll briefly describe two of these applied projects that took advantage of BBN’s comparative advantages. Both helped lay the groundwork for now-famous technical breakthroughs.

The first of these projects was BBN's Libraries of the Future Project. The project, indirectly funded by the Ford Foundation, sought to understand how computing might impact libraries moving forward. It was a contract for a philanthropy that, today, might go to a consulting firm like the Bridgespan Group. But in this unassuming contract, Lick saw an opportunity. The BBN team won the contract and used it as an excuse to explore seemingly futurist technology and make practical engineering improvements. As one example, the second half of the group’s book-length report for this unassuming contract contains technical explorations that attempt to emulate a future in which users interact with the library as a store of knowledge with a question-answering front-end, not as a way to find books. The final report also contained early work related to PC file systems, improved information retrieval methods, mathematical representations of information, and associated hardware improvements — all paid for by this unassuming contract. The opportunities to further explore and translate academic ideas via this contract proved attractive to other researchers in Cambridge. On this project alone, Marvin Minsky, John McCarthy, and Fischer Black all spent time working with the BBN group.

The second of these projects was built on a contract to develop a usable real-time computing system in a presciently chosen application area: hospital administration. The contract came about because a BBN partner made a compelling pitch to the NIH Clinical Center’s Director that hospital administration would be done using computers at some point. The director was convinced and had the partner write up a proposal for BBN to begin developing and building a computing system to pilot with a research hospital. The BBN hospital time-sharing computer project provided the firm with three years of funding, worth ~$10 million in total today. The contracting relationship would continue for several additional years, growing into an actual system deployed in the operations of Massachusetts General Hospital. With the contract, BBN was able to fund years of work a bit too far along for academia but not close to private-market-readiness. The contract required far more contract management and engineering work than would have been reasonable for a university team — they had to continually reduce machine error rates, install and maintain terminals at the hospital, coordinate with hospital staff, train doctors, provide customer service, make UX changes, etc. But with this applied work came a sizable budget. BBN’s small team of real-time computing experts used it to push the engineering frontier forward.

By the time Larry Roberts began putting out feelers for the ARPAnet contract in the late-1960s, BBN had been solving all sorts of related problems for years — similar to the ones that had stumped Roberts while he was at MIT. BBN won the contract. The first four nodes of the ARPAnet would be delivered within a year, on time and on budget. But it was BBN’s years of work on related contracts that paved the way for this overnight success.

Nonprofit BBNs: The Case of CMU's Early Autonomous Vehicle Teams

The BBN model can also be operated with a nonprofit structure. The early CMU autonomous vehicle teams — which helped pave the way for the autonomous vehicle revolution — demonstrate this. In another world, these teams probably could have operated with an FRO structure. But in practice, they operated using a model more similar to BBN’s contract research model.

The CMU Robotics Institute was primarily established to build ambitious, practical computing systems. While housed at a university, these CMU teams are more aptly compared to a group like BBN than a group like Marvin Minsky's computer science department at MIT. Contracts were a large part of the Institute’s plans from its inception. The Institute’s early autonomous vehicle group successfully deployed a BBN-style approach on DARPA’s 1980s autonomous vehicle contracts, achieving clear counterfactual impact and developing systems DARPA’s primes and academic labs under contract would not.2

The CMU group had an operational structure and incentives that enabled it to excel on these ambitious, applied contracts. As one example, the Robotics Institute created suitable tracks through which engineering and project management-focused individuals could be hired and tenured just as CMU's best professors would be. Instead of researching and teaching classes, these individuals would research and operate projects. A CMU Ph.D. and one-time project manager for the group’s early autonomous vehicle projects, Chuck Thorpe described the incentives of his role, recounting:

Raj [Reddy] took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

The CMU teams also found ways to effectively use the grad students as both project staff and researchers. Requiring dissertation problems of their own they could “sole author,” grad students were assigned components of the system for their dissertation to improve upon. Each project had to easily plug into the larger system. High-risk, high-reward projects still had a place in this structure. If projects were high-risk, project managers like Chuck Thorpe might simply assign more grad students to improve a component to ensure at least one succeeded. A good project might reduce the processing time for data from some sensor 10X. The best grad student project would prove to be world-changing.

In 1988, a sharp grad student named Dean Pommerleau combined his knowledge of CMU’s systolic array machine and neural nets to build a neural net-based steering system for the vehicle — ALVINN. In the following years, the team built this neural net steering system into a new vehicle system. In 1995, with this new system, the CMU vehicle would drive cross-country 98.5% autonomously on its “No Hands Across America” tour.

Getting BBNs Off The Ground

Attempting to found and operate a BBN is not easy. You might fail to raise a required seed grant or win contracts to get off the ground. You might sell a contract funder on your idea but need a partnership with an existing org or philanthropy to get the deal over the finish line. You might win an initial batch of revenue and still have to shut down after a year due to a lack of funds. You might even succeed in raising funds, but raise them from misaligned funders and become a myopic R&D contractor. This is a failure mode all the same; becoming Battelle or Raytheon is not the goal of a BBN. But great field strategists can use the BBN model to turn revenue, people, and vision into what Patrick Collison recently called a “great scene of discovery.” This is what BBN made itself into for real-time computing and CMU for autonomous vehicles.

Building a BBN is a viable, alternative path for those field strategists and scientific entrepreneurs eager to do work in the cracks of the R&D ecosystem. Done well, BBNs can be a force multiplier to investments made by scientific funders, enabling them to undertake projects that are cheaper, better, or different than they would otherwise. This “create the performer” approach still has its uses in the context of modern philanthropy.3 As Jacob Trefethen recently discussed on his blog, the Gates Foundation funds “half or more” of many global health product development partnerships (PDP) budgets because they usefully fill cracks vital to Gate’s global health mission.

But founding BBNs can require overcoming a somewhat delicate coordination problem. While some BBNs can be bootstrapped, many BBNs will require some combination of a low-seven-figure seed grant to get started, contracts, and (maybe) NSF/NIH grants to optimally pursue their research agenda. It takes a bit of talent to overcome the associated chicken-and-egg problems that might arise. Philanthropies might want to provide seed funds to spin up a BBN, but only if they can be assured contracts for R&D projects will materialize. A contract funder like DARPA might require the org actually exist before engaging in some negotiations. OpenPhil might happily provide 60% of the budget for ~2 years for some BBN useful to its mission but require the group to source additional funds and contracts to cover the rest. The list goes on. It’s not an easy path. But if the crack is big enough and a field strategist is proactive enough, these challenges can be overcome.4

But none of this can happen without an ambitious, open-minded contract funder. Funders with the level of vision and ambition of ARIA are ideal. I would encourage funders reading this to ask themselves the question, “Could I take my portfolio to another level if I was able to create a single performer to work in a targeted area?” If the answer is “yes” or “maybe,” you might benefit from a BBN.

BBNs might apply to many areas. Some areas in which BBNs might have natural comparative advantages include:

• Development of new instruments

• Producing bespoke hardware

• Engineering new pipelines to serve a group of researchers

• Building and maintaining research software or datasets

• Pilot-plant style work with the intention to:

  • Scale promising lab bench ideas in manufacturable processes

  • Produce a key research material for labs

• Translation of academic ideas to non-venture markets.

  • One example could be a CivTech BBN that wins city government contracts to both translate the best social science insights into practice and conducts research projects on the org’s internal data.

It’s hard to predict how many areas might benefit from a BBN, but the model offers opportunities for the ideas of special individuals to surprise you. Progress studies funder and Stripe CEO Patrick Collison recently spoke of a similar phenomenon regarding a new Stripe product, tweeting:

We launched Stripe Issuing a few years ago. I wasn't sure how many use-cases there would be — like, issuing a card is surely a pretty niche thing to do. As usual, developer platforms surprise, and customers have now issued a cumulative 200 million cards.

BBN’s idea to bid on the Libraries of the Future project is one example of this cleverness at work. The BBN model has the potential to not just plug obvious cracks in the R&D ecosystem, but create great scenes of discovery in their place. In which cracks these scenes will take hold is hard to predict.

The fact that FROs now exist and are a fixture of the new science lexicon makes the task of coming up with ideas for BBNs easier for field strategists. A good FRO problem is, in many cases, rather similar to a problem for a BBN. Adam Marblestone recently shared one example, explaining that an FRO might arise from a problem like neuroscientists requiring a different kind of microchip. One complex chip solving many life sciences problems is a great example of an FRO problem. But what if many neuroscience applications required many, simpler microchips? That’s a problem well-suited to a BBN.5

Tyler Cowen points to the history of Hollywood as proof that exciting young people can find ways to create exceptional things if given an opportunity to take charge and figure things out. FROs, with their time-bound nature, remind him of Hollywood movies. BBNs, with the scrappy nature of their origins, have the chance to become a vehicle for unproven R&D entrepreneurs to prove they are masters of their craft — just as Tyler believes Paul McCartney did with the Beatles.

It is an exciting proposition. But founders and funders with vision are required to make it a reality.

Thanks for reading.

Please reach out if…

I am already working with a few field strategists who believe they might have ideas for BBNs in areas like automated material discovery, bespoke semiconductor production as test bed hardware for certain areas of semiconductor design, and modifying instruments to enable a wider range of life science experiments. I will release those pieces in the coming weeks. Please reach out if you are a funder or interested field strategist to discuss ideas ([email protected] or on Twitter). I’d love to help!

I am setting aside as much time as necessary to assist would-be BBN founders and funders. I believe we have J.C.R. Lickliders sitting on the shelf with vision, philanthropists with exciting ideas that are best served by a new contractor, and outside funders who might happily provide the seed funding to catalyze this change. BBNs can be a vehicle to turn this potential energy into action.

Share

Various ideas and details included in this piece have been explored in FreakTakes pieces on BBN, DARPA’s Autonomous Land Vehicle and NavLab projects, an interview with CMU’s Chuck Thorpe, and how BBNs might apply to new era semiconductor research.

I’d also like to mention that Adam Marblestone listed OtherLab as an org that does great work with a BBN-type model.

1

The model offers other advantages in relation to FROs. BBNs are well-suited to projects/areas where you need to work closely with users to define your problem over time, gradually build up your team in response to user needs, or find it advantageous to not be overly influenced by a single funder’s wishes.

2

In the early years of DARPA's first autonomous recon vehicle project, while many academic DARPA fundees plead for data from DARPA’s vehicle to train academic vision algorithms, the CMU team plead for $1.2 million ($3.5 million today) to build a pair of their own test bed vehicles. In 1986, DARPA agreed. This filled a key crack in DARPA’s performer ecosystem. To that point, the prime R&D contractor, Martin Marietta (the “Martin” in Lockheed Martin), had shown little enthusiasm for testing cutting-edge academic ideas. This test bed work entailed piles of engineering and administrative work academic departments did not want to touch. This niche of “test bed contractor for cutting-edge ideas” was largely unwanted. But CMU, with its systems engineering focus, wanted it. 

3

I credit Adam Marblestone with this phrasing. He used it in an email exchange.

4

In many areas, the market for BBNs might be growing. The more difficult it becomes to pursue applied contracts at universities, the less common it becomes to do ambitious research at R&D firms like Raytheon, and the more field strategists take positions of authority as scientific funders, the more cracks form which BBNs might reasonably fill. 

5

In fact, in the 1970s DARPA created its own performer to make bespoke chips for its MOSIS program — supplying DARPA researchers, professors, and EE grad students with cheap, fabricated chips of their own design.

Bottom Line Up Front: Recently, large pots of federal funds have been set aside for chip research. A significant portion of these funds should find their way to research teams that operate like the best research groups from the prior, vertically-integrated era. Two ideal teams to learn from are BBN — the prime ARPAnet contractor — and CMU’s early autonomous vehicle teams, who laid much of the groundwork for the autonomous vehicle revolution. Both groups were ideally structured to solve ambitious, vertically-integrated computing problems with small teams of people with broad technical knowledge. Each org was aggressively novelty-seeking, solved practical problems for real users, and was managed in a firm-like fashion. Groups with this older operating model possess key comparative advantages that make them superior to academic labs and VC-funded firms in many applied research areas. This includes somewhat speculative projects in the early proof-of-concept stage, a key focus area for the NSTC.

Introduction

Many experts believe we are living in a post-Moore’s law world. NVIDIA’s Jensen Huang and computing legends like John Hennessy and David Patterson count themselves among them. But that does not mean these predictions spell doom for the computing field. Their predictions simply acknowledge that we are approaching the end of the “there’s plenty of room at the bottom” era of general-purpose computing — and moving on to another.

As Hennessy and Patterson point out, this shift hearkens the semiconductor ecosystem back to a style of work that was common before the 1980s — an era FreakTakes has covered extensively. In this piece, I’ll detail how the best teams from that period operated and why it’s necessary to bring those operating models back to thrive in this new era of semiconductor design.

The piece is broken up into three sections.

• In the first section, I briefly cover the history of the “there’s plenty of room at the bottom” era.

• In the next section, I explain what experts believe a “there’s plenty of room at the top” era could look like. Current researchers and veterans like Hennessy, Patterson, and Leiserson et al. believe the shift to looking for progress at the top of the computing stack will involve the vertical integration of research teams and a focus on domain-specialized hardware.

• In many ways, this is a shift back to how computing research groups used to operate before the 1980s. So, in the third section, I will summarize key management decisions that characterized some of the best old school computing organizations — such as BBN and the CMU autonomous vehicle groups.

With that, let’s get into it.

Subscribe now

Share

Early ARPAnet IMP Contributors (Left to Right): Truett Thatch, Bill Bartell (Honeywell), Dave Walden, Jim Geisman, Robert Kahn, Frank Heart, Ben Barker, Marty Thorpe, Will Crowther, Severo Ornstein. Not pictured: Bernie Cosell. Photo courtesy of Wikimedia Commons

This piece heavily draws on two prior FreakTakes pieces on the BBN and CMU teams:

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

Readers familiar with the basics of semiconductor history may skip the first section. Readers familiar with Leiserson et al.’s “There’s plenty of room at the Top” paper may skip all but the final three paragraphs of the second section.

There was plenty of room at the bottom

The optimism that drove the early Moore’s Law era of general-purpose computing is well-characterized by Richard Feynman’s “There’s Plenty of Room at the Bottom,” speech to the American Physical Society in 1959. In his speech, Feynman implored scientists and engineers to focus their efforts on the exciting opportunity of engineering biological, electrical, and physical systems on exceedingly small scales. He believed we were nowhere near the limitations of physics when it came to engineering systems on these scales. So, he asserted that the area was ripe for talented engineers and scientists to pile in and build a world of systems whose possibilities were orders of magnitude beyond the status quo in 1959.

For decades, the semiconductor field did just that, doubling performance every 18-24 months until the early 2000s. It still continues to excel at a remarkable rate, simply with more tradeoffs than we have become accustomed to seeing. The practical foresight of individuals like Gordon Moore in his 1965 Moore’s Law paper, Robert Dennard et al. in the more technically-detailed 1974 Dennard Scaling paper, and Gordon Moore again in his 1975 paper — summarizing of the field’s progress and providing updated predictions for the coming decades — largely held true until the early 2000s.

As the early 2000s approached, it was becoming clear to many in the industry that the general approaches outlined in the Dennard Scaling paper were not going to continue to provide the same improvements. The aptly named 1999 Paul Packan paper, Pushing the Limits, outlined several physical limits the field was approaching with the current approaches — including thermal limit challenges. Many of the specific challenges Packan outlined in the paper had no known path of attack that did not entail major trade-offs. Industry attempted to take alternative paths of attack to continue to scale computing power — such as the increased emphasis on multi-core processors — but the limitations researchers like Packan outlined have proven to be quite difficult. Since the mid-2000s progress in the field has continued, but it has not been Moore’s Law-level progress.

So, if there is not much cost-efficient room to be found at the bottom, where should the field look? Well, many researchers believe the natural place to look is: the top.

There’s plenty of room at the top, but…

In Leiserson et al.’s 2020 paper, There’s plenty of room at the Top, a group of researchers from MIT outlines a vision for the potential gains in computing power by focusing on the top of the computing stack — software, algorithms, and hardware architecture. They are quick to point out that this approach will require divorcing one's thinking from the “rising tide lifts all boats” approach which many became accustomed to under Moore’s Law. The authors write:

Unlike the historical gains at the Bottom…gains at the Top will be opportunistic, uneven, and sporadic…they will be subject to diminishing returns as specific computations become better explored.

The authors paint a broad picture of what this integrated approach will look like, writing:

Architectures are likely to become increasingly heterogeneous, incorporating both general-purpose and special-purpose circuitry. To improve performance, programs will need to expose more parallelism and locality for the hardware to exploit. In addition, software performance engineers will need to collaborate with hardware architects so that new processors present simple and compelling abstractions that make it as easy as possible to exploit the hardware.

The authors then go on to demonstrate a toy example in the area of software performance engineering. They use a simple baseline problem as an example, multiplying two 4096-by-4096 matrices. To start, they implement this in the Python code that minimizes programmer time spent. They then proceed to do performance engineering until the same function eventually runs 60,000 times faster — first by changing to more efficient coding languages, then introducing parallelization, and beyond.

At the end of the performance engineering section, they reflect on what it will take for teams of researchers and engineers to effectively take advantage of opportunities analogous to this one in the future:

Because of the accumulated bloat created by years of reductionist design during the Moore era, there are great opportunities to make programs run faster. Unfortunately, directly solving [some arbitrary, domain-specific] problem A using specialized software requires expertise both in the domain of A and in performance engineering, which makes the process more costly and risky than simply using reductions. The resulting specialized software to solve A is often more complex than the software that reduces A to B. For example, the fully optimized code in Table 1 (version 7) is more than 20 times longer than the source code for the original Python version (version 1).

The authors’s next section explores a few major successes in the history of algorithmically-driven computing performance. They use the history of algorithms for matrix multiplication as a key example to note that while the history of algorithm improvements has been a bit uneven in terms of rate of discovery, in certain cases it has rivaled Moore’s Law-level progress.

The authors highlight algorithms as another area that, while only semi-reliable, holds extreme promise for continuing to push computing progress forward. Among other reasons, they think there might be promise in this approach because many commonly used algorithms were originally designed to utilize random access memory on the serial/sequential processors of the early 1960s and 1970s.1 They believe many algorithms that take this approach use modern hardware inefficiently because they under-utilize the parallel cores and vector units in modern machines.

But, at the end of the algorithms section, the authors once again note that there is a clear difficulty in pursuing problems of this form in the modern, rather modularized research ecosystem. They write:

These approaches actually make algorithm design harder, because the designer cannot easily understand the ramifications of a design choice. In the post-Moore era, it will be essential for algorithm designers and hardware architects to work together to find simple abstractions that designers can understand and that architects can implement efficiently.

In the final section, on hardware, the authors outline the general characteristics of computing areas which they believe can continue to experience extreme performance improvements — and which will curtail. Areas that benefit from parallelism and streamlining are notable examples with a lot of potential for progress to continue. The authors use the following graphic to highlight the continued progress in SPECint rate performance — a processing performance metric which increasingly benefits from parallelism as more cores are used — as an archetypal example of a metric that they expect to continue improving at a high rate.

As designers increasingly embrace parallelism, Leiserson et al. believe “the main question will be how to streamline processors to exploit application parallelism.” In tackling this challenge, they expect two strategies to dominate. The first is that they expect designers to increasingly use simpler cores that require fewer transistors in chip designs. These simpler cores will be specifically chosen to efficiently accomplish the task at hand. For this to be worthwhile, designers must find ways to ensure two simplified cores incorporated in a specialized design can accomplish more than a single complex core.

On the second approach to tackling the challenge of streamlining processors, the authors write:

Domain specialization may be even more important than simplification. Hardware that is customized for an application domain can be much more streamlined and use many fewer transistors, enabling applications to run tens to hundreds of times faster. Perhaps the best example today is the graphics-processing unit (GPU), which contains many parallel “lanes” with streamlined processors specialized to computer graphics. GPUs deliver much more performance on graphics computations, even though their clocks are slower, because they can exploit much more parallelism. GPU logic integrated into laptop microprocessors grew from 15 to 25% of chip area in 2010 to more than 40% by 2017…Hennessy and Patterson foresee a move from general-purpose toward domain-specific architectures that run small compute-intensive kernels of larger systems for tasks such as object recognition or speech understanding.

While many interpret the belief that we have seen the end of Moore’s law as lacking in technological optimism, that does not have to be so. Hennessy and Patterson’s 2017 Turing Award Lecture on this topic was titled, “A New Golden Age for Computer Architecture.” There’s no reason this shift in approach must result in disappointment.

The following excerpt from a version of the Turing Lecture Hennessy gave at Stanford in 2019 briefly characterizes their thoughts on what the near future of semiconductor design will look like.

I think Dave [Patterson] and I both believe that the only path forward is to go to domain-specific architectures that dramatically improve the environment and the application for that particular environment in the performance that’s achievable as well as the energy consumption. Just do a few tasks, but do them extremely well. And then, of course, combination — putting together languages which help specify applications for these particular architectures and combining it with the particular architectures.

Hennessy concludes the talk by specifically emphasizing the implications of the new domain-specific design approaches on scientific team structures, saying:

This creates a new Renaissance-like opportunity — because now unlike what’s happened for many years in the computer industry. Now people who write applications need to work with people who write compiler technology and languages; they need to work with people who design hardware. So, we’re re-verticalizing the computer industry as it was before the 1980s. And it provides an opportunity for a kind of interdisciplinary work in the industry that I find really exciting. It’s sort of like this great quote, “Everything old is new again.” It’s back to the future. And I think this will provide interesting opportunities for new researchers as well as deliver incredible advances for people who use our technology.

What did the old models look like?

Hennessy has as much of a right as anybody to be excited about the re-verticalization of hardware design and the re-introduction of the old models. This “old” era is one that Hennessy understands well; it is the era in which he came up. In the following excerpt from his oral history, Hennessy describes the integrated nature of his PhD thesis — which he called “a combination of language design and compiler technology” — on a domain-specific problem at Stony Brook:

My thesis work was on real time programming…[I worked with] a guy who was a researcher at Brookhaven National Lab [that] came along with an interesting problem. He was trying to build a device that would do bone density measurements to check for long term, low level radiation exposure among people who were working at Brookhaven. What he wanted to do was build a very finely controlled x-ray scanning device that would scan an x-ray across an arm or so and get a bone density measurement. The problem is you had to control it pretty accurately because, of course, you don’t want to increase the radiation exposure to people. And micro processors at that time were extremely primitive so the real question is could we build a…programming language that could do this kind of real time control in which you could express the real time constraints. You could then use compiler technology to ensure that they were satisfied.

After joining the Stanford department in the late 1970s, Hennessy continued to partake in projects that were structured something like the one above — such as Jim Clark’s Geometry Engine project. As Hennessy mentioned, it was normal in that era. Several historically-great computing research groups that utilized this general operating structure have already been covered on FreakTakes. Early BBN and the early CMU autonomous vehicle teams are two great examples of the dynamism of this approach at work.

While BBN was a private research firm and the CMU Robotics Institute was technically an academic group, when it came to operations and incentives, the two had more in common with each other than they did with many fellow companies or academic researchers. Both were:

• Novelty-seeking

• Committed to building useful prototypes and technology for users, not doing mere “paper studies”

• Utilized firm-like management structures

These traits led each to great success in their respective fields. The following two subsections will briefly summarize the relevant operational strategies each used in their work that the new-era semiconductor ecosystem can learn from.

Early BBN

BBN had an exceptional talent for finding ways to generate revenue through contract research and research grants that allowed the firm to build academic ideas into real technologies. In many areas — such as real-time computing in the late 1960s — BBN was the go to group for projects that required both academic research toolkits and the kind of cutting-edge engineering which academic departments are rarely structured to pursue. J.C.R. Licklider’s Libraries of the Future project, BBN’s work as the prime systems integrator and IMP designer on the ARPAnet contract, and BBN building a time-sharing computer for the Massachusetts General Hospital in the early 1960s are just a few examples. BBN’s projects often simultaneously required bleeding-edge knowledge of computing, performed a practical service for a paying customer, and could result in technology that pushed the technical frontier forward.

BBN often did all of this with relatively small, vertically integrated teams of talented people. The firm was able to recruit many of MIT’s best researchers because of its unique model of both seeking novel technical problems and commitment to building practical technology. At the time, researchers who did “paper studies” and built basic prototypes at places like Lincoln Labs often grew disenchanted; they felt their work might not be built into industry products for a decade or more. But these disenchanted individuals still felt they would be wasted and bored if they joined a company like Honeywell. But BBN presented an exciting third option. With BBN, there were no such fears. The firm earned nicknames from MIT affiliates like “the third great university of Cambridge,” “the cognac of the research business,” and the true “middle ground between academia and the commercial world.”

With this reputation and the ability to recruit top talent that came with it, small teams at BBN could do big things. The BBN team primarily responsible for building the first ARPAnet IMPs and managing the first year of the project was only about eight people. Small is how ARPAnet team leader Frank Heart preferred it. As he saw it, that small, vertically integrated team of elite talent in the relevant areas was all BBN needed. In his oral history, he described the project’s management style, saying:

I mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

The approach Frank Heart describes is not complex, but it’s also not how most universities handle similar projects. The more vertically integrated semiconductor design becomes, the more common it will become for BBN-model groups to be the ideal team structure for research projects. If veterans like John Hennessy are right and the computing industry is re-integrating and re-verticalizing, semiconductor research funders should ensure that groups with this uncommon structure can win funding to work on the research areas to which they are ideally suited.

CMU’s Early Autonomous Vehicle Teams

There are successful historical examples of universities operating in a very BBN-like fashion, as well. For example, much of MIT’s early history as an institute of technology dedicated to serving industry above all else demonstrates this point. In the early 1900s, the Institute placed such a heavy emphasis on contract research that, at one point, its applied chemistry laboratory was over 80% industry-funded. However, a much more recent example of this mode of operating is CMU’s 1980s autonomous vehicle research groups. CMU succeeded in maintaining a vertically integrated, applications-focused ethos in a period when most universities were taking a step back from this approach.

CMU’s autonomous vehicle vision research teams — and many other computing teams at the university — had a talent for operating in a way that could only be described as “firm-like.” Many grad students who worked on projects like the 1980s NavLab project were as much project employees as they were graduate students. They generated theses they could be sole authors on, but their work often had to be a part of a team effort on a single piece of integrated technology. Also, many staff were solely incentivized to build great technology, not rack up citations and build useful technology when they could find the spare time. In his FreakTakes interview, Chuck Thorpe recalled the following conversation he had with Raj Reddy about his incentive structure as he was being hired to manage the autonomous vehicle project:

Raj took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

Aligning the incentives of key staff and organizing grad students in a way that enables the group to build the most ambitious, useful system it can makes sense. But this is also not how university’s tend to manage applied projects. Few universities succeed in doing things like creating tenured research tracks that are truly equal to traditional professorships. The best staff scientists and engineers I meet from places like MIT feel there is an obvious ceiling on how high they can rise. By getting many of these key incentive alignment, strategy, and team structure questions right, CMU built up a powerhouse research team that, in the 1980s, laid the base for the autonomous vehicle revolution.

The CMU team’s status as a sort of middle ground between academia and the commercial world proved to be a clear comparative advantage for them in the early days of DARPA’s first autonomous vehicle project, the Autonomous Land Vehicle (ALV) — detailed in a prior FreakTakes piece. At the end of the project’s first stage, DARPA had to fire the prime contractor, Martin Marietta — the “Martin” in Lockheed Martin. Martin had continually pressured DARPA to give them concrete, intermediate benchmarks to hit. When DARPA did just that, Martin found ways to hit the benchmarks without incorporating the best tech from DARPA’s basic vision researchers. The basic researchers also fell short of DARPA’s expectations. Many of DARPA’s SCVision researchers seemed more interested in collecting data from the machines’ cameras and sensors to plug into their academic projects. Oftentimes, their models would not work well when plugged into real-world machines. The academics knew this not because Martin was diligently testing each of their ideas, but because the CMU research team had pushed DARPA for modest funds to build their own vehicles. Using these vehicles, they tested and iterated upon cutting-edge vision models in a practical setting. Unlike most academic departments that did not want the massive systems integration and logistical headaches that came with this engineering-heavy task, CMU wanted to do it. DARPA was impressed by the productivity of the scrappy CMU team and continued to fund them even after the ALV project ended.

To the modern eye, the most striking early success from the CMU group might be the group’s neural net-based wheel-turning system. In my introduction to the Chuck Thorpe interview, I summarized the breakthrough:

In 1988, then-CMU grad student Dean Pomerleau successfully integrated the first neural net-based steering system into a vehicle — his ALVINN system. In 1995, Dean and fellow grad student Todd Jochem drove the successor to Dean’s ALVINN — the more complex RALPH — across the country on their “No Hands Across America” tour. The RALPH system employed neural nets but also put much more effort into model building and sensor processing. Holding Dean and Todd, the upgraded NavLab was able to autonomously drive 98.5% of the way on its successful cross-country journey.

According to Chuck Thorpe, there was something specific about Dean that enabled him to make this breakthrough with such limited compute. The trait Thorpe pointed out is one very relevant to this piece. According to Thorpe, what made Dean special was that he closely worked with the hardware developers at CMU and was one of the only people who understood how to get the most out of the group’s systolic array machine. Thorpe described Pomerleau’s extreme effectiveness in getting the most out of the Warp machine:

One of the supercomputers [DARPA] built was at Carnegie, and it was a thing called the Warp machine. The Warp machine had ten pipelined units, each of which could do ten million adds and multiplies at once.

H. T. Kung was working on systolic computing, where things just sort of chunk through the pipeline. And we said, “Here's this big box. Because we're sponsored by Strategic Computing, we'll put it on the NavLab. Can anybody figure out what to do with it?” “Well, it's a really complicated thing and really hard to process.” But Dean could figure out what to do with it. Partly because Dean is a really, really smart guy.

Partly because the inner structure of a neural net is, “Multiply, add, multiply, add.” You take these inputs, you multiply them by a weight, you add them together, you sum it up, run it through your sigmoid function. So, Dean was, as far as I know, the only person…well, that's maybe a bit of an exaggeration. Some of the speech people were able to use the Warp also, because they were doing similar kinds of things…But Dean got the full hundred megaflops of processing out of the Warp machine! And then he put it onto the NavLab and we got the whole thing to work.

It was not random that CMU was working on a systolic array computer. DARPA had contracted CMU to build the systolic array machines with computer vision as an early use case in mind. DARPA hoped that the two CMU projects would prove to be complementary. While the Warp team and the autonomous vehicle teams were not working together as one at this point, Pomerleau’s hard work ensured that they complemented each other to world-changing effect.

As Hennessy pointed out in his Turing Lecture at Stanford, Google’s now-famous TPUs with systolic array processors — optimized for machine learning applications — are a key example of the coming trend in domain-specialized hardware. Research groups that operate like BBNs ARPAnet team or CMU’s early autonomous vehicle teams seem ideally suited to help build this future.2

Everything old is new again

I am not qualified to make predictions on the future of Moore’s Law or what percentage of R&D budgets should be allocated to domain-specific semiconductor design teams. But if what people like Hennessy, Patterson, and Leiserson et al. are saying is even partially true, more research organizations set up like the best ones from the past are needed. The early operating structures of BBN and the CMU Robotics Institute, two underrated giants of the old computing community, are ideal groups to use as models.

As I understand it, Groq’s approach is rather similar to what I’ve described. And it’s phenomenal to see the progress they’ve made! But the vast majority of potential use cases for this org structure might be in market areas that are too small to raise VC rounds or for ideas considered too speculative. These are ideal areas in which to deploy CHIPS Act funding to seed BBN-like organizations and research groups. Whether it be seeding groups do practical work on the most speculative ideas coming out of academic labs or building in areas that are more important to the US than they are to VCs, BBN-model orgs have to chance to connect many pieces of the CHIPS contractor ecosystem that currently do not fit together well — just as DARPA’s contractors on the Autonomous Land Vehicle project did not fit together without CMU.

With the ramp-up in government CHIPS Act funding — endowing groups like the NSTC with up to $11 billion that has not been spent yet — now is the time to ensure groups are built up that can utilize this money as effectively as possible. One straightforward way to do this is for:

• Groups like the NSTC and the DOD Microelectronics Commons to decide and make it known that they are willing to wholly fund Day 0 orgs of 5-20 talented engineers, researchers, and systems engineers organized in BBN-model project teams as contractors.

• Alternative approaches that are similar in spirit but with key logistical differences are also possible. For example:

  • Attaching similar groups to pre-existing government contractors or universities.

  • Having NSTC program managers scout a domain area and assemble these teams to best fill acute needs in their portfolios.

  • Etc.

If any relevant parties are reading this, please reach out and I’d be happy to discuss alternative approaches and bureaucratic details. What is important is that research groups like the ones I’ve described find a way to get funding somehow.

On the university end, it would be great if institutions could quickly find ways to establish homes for research groups like this, with incentivizes and team structures that align with the task at hand. But given the difficulty of making decisions within universities that upset the natural order of things, this can often be slow and requires tough “consensus building.” So it’s maybe not worth holding our breath on that.

In the post-war period, it was common for government R&D funders to be the first — and sometimes only — major funding into new organizations of talented individuals. If they felt the team could do work that was good for the country, they took the risk. This was a common path to starting technical firms for grad students and professors. The market sizes were often modest and the teams often didn’t raise VC money. But the best of MIT, CMU, or the University of Illinois could simply set up shop as a firm and solve whatever problems the government needed them to at a fair market rate. With this recent, strong push to make things happen in the American semiconductor ecosystem, talented groups might have the chance to do something like that again.

Thanks for reading:)

Also, while doing background research for this piece and trying to make sense of the technical and management trends in different eras, I found the writing of Hassan Khan exceptionally helpful. I encourage any FreakTakes readers to check out his blog here.

Subscribe now

Share

If you would like to read more, check out some of the following FreakTakes pieces on related topics:

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

• ILLIAC IV and the Connection Machine

• MOSIS: The 1980s DARPA 'Silicon Broker'

• Managing Lockheed’s Skunk Works

One Last Announcement: Interested readers should check out this policy opportunity FAS sent over. “The Federation of American Scientists is looking for federal policy ideas to tackle critical science and technology issues. Areas of interest include R&D and Innovation, Emerging Technologies, Artificial Intelligence, Energy and Environment, Federal Capacity, Global Security, and more. Learn more and submit an idea here by July 15.”

1

Special thanks to former BBNer Martin Haeberli. He instructed me on how to modify this sentence to be accurate — it was inaccurate in a prior draft. If you see this, give him a follow on Twitter!

2

A note on Hennessy's reasoning for founding his company and the utility of BBN-model orgs:

A major reason Hennessy seems to have founded his company is that he and others felt the technology they were developing at Stanford was obviously good, but industry players seemed many years away from adopting it and building products based on it. Interestingly, this is the reason many real-time computing experts left Lincoln Labs to join BBN — to work on more practical computing projects based on their ideas.  While the VC money allowed Hennessy and co to assemble a “smaller team” that was “as good pound for pound as any team that’s ever been put together,” there were plenty of sales and business model difficulties that proved to be frequent concerns for the small firm. BBN-model orgs might not have the near-term translational upside of VC-funded firms, but they can silo technical teams from existential issues like this and allow them to work on problems that are not quite market-ready or whose upside is more as a public good than one that can be substantially captured by investors.

This piece covers how Kelly Johnson managed Lockheed’s famous Skunk Works. In its early decades, Skunk Works continuously produced novel aircraft that pushed the aviation industry forward. Its three most iconic aircraft were the U-2 “spy plane,” the Sr-71 Blackbird — still considered a cutting-edge aircraft 60 years after it was built — and the partially DARPA-funded F-117 Nighthawk — the first stealth bomber. Johnson believed the simple management playbook he used to operate his Skunk Works was “common sense.” Still, Johnson’s ruthless commitment to maintaining small teams and fiercely anti-bureaucratic processes proved difficult for competitors to replicate. The success of Johnson’s methods and the failures of competitors to replicate them carries key lessons for R&D organizations and funders alike.

Kelly Johnson when he was a young, formidable flight engineer at the young Lockheed. Photo courtesy of More Than My Share of It All.

Subscribe now

What the Skunk Works does is secret. How it does it is not. I have been trying to convince others to use our principles and practices for years. The basic concept as well as specific rules have been provided many times. Very seldom has the formula been followed. — Clarence “Kelly” Johnson

Introduction

This piece will round out a trio of pieces covering exceptional contractors from ARPA’s early decades. The first two pieces covered BBN and CMU’s autonomous vehicle teams. The former was a research firm unusually focused on novelty; the latter was a group of university researchers unusually focused on shipping real technology and incorporating firm-like management structures. Unlike those organizations, Skunk Works appears at first glance more similar to a traditional defense contractor than a research lab. But the novelty of its outputs demonstrates that Skunk Works is an all-time great R&D lab in its own right. Skunk Works got much more funding from the armed forces and CIA than it did ARPA. But it still deserves to be covered in the ARPA playbook because, to many, the name “Skunk Works” has become synonymous with the topic of defense innovation.

Today, many view a big bureaucracy like a defense prime as a terrible place to build ambitiously. This is more true today than it was in the 1950s, but it was still often true back then. Yet, Kelly Johnson was able to build an applied research powerhouse within Lockheed that designed arguably the three most impressive spy planes and stealth bombers in American history. To top it off, many of Skunk Works’ novel planes were built on budget and in only 180 days. This was all made possible by Johnson’s clear vision of what was important when running a lab and the trust Lockheed management and DC project funders had in him.

But before I detail how Kelly built Skunk Works, I must first dive into how Kelly Johnson earned his stripes as a young aircraft engineer. This is key to understanding his management of the Skunk Works. In many ways, his management practices sought to maintain an environment similar to that of the young aviation industry he came up in.

The Education of Kelly Johnson

To many decades-long Skunk Works employees, the essence of the lab was inseparable from its founder and boss: Kelly Johnson. So, to begin to understand the lab and how it operated, we first need to understand Johnson and his background.

As a boy, Kelly came to the decision that he wanted to learn how to build airplanes. He read every book on the only two-decade-old subject he could get his hands on at the local Carnegie Library. When he got to the University of Michigan, he quickly found a job helping a professor operate the school’s wind tunnel. The professors at the University of Michigan, like many engineering schools at the time, did much of their research on industry contracts. This gave Kelly the opportunity to help improve real vehicles for paying customers. In his 1985 autobiography, More Than My Share of It All, Kelly describes several contracts he was working on with a Professor Pawlowski, writing:

He, like some of the other professors, had contracts outside the college. In the wind tunnel, I worked for him on design of the Union Pacific streamlined train, on a smoke-removal project for the city of Chicago, and on one of the very early proposals for generating energy with a wind machine.

In addition to assisting professors with their projects, Kelly asked for permission to take on his own contracts. They agreed and said that he could use the wind tunnel when it was idle. He simply had to pay the university an hourly fee to rent it. The first customer he sought out was the Studebaker Motor Company. Kelly was confident he could help the company. The wind tunnel was a relatively young scientific instrument. Many companies did not have one of their own. He described his attitude going into the contract, writing:

It was obvious that the wind tunnel could be very useful in designing streamlined automobiles. We got an assignment to test the Pierce Silver Arrow, which was to become one of the early “totally-streamlined” cars. We knew all the tricks on how to reduce drag caused by air resistance. We found, for instance, that the big ugly headlamps on Studebaker cars were eating up 16 percent of the power the engine developed at 65 miles an hour. We managed to get them shaped into the fenders. We worked on a lot of other problems and ran many, many tests.

In 1932, Johnson graduated with his bachelor’s degree. It was a horrible job market, but he and a friend borrowed a professor’s car and drove to the West Coast to introduce themselves to aircraft companies to see if they had any work available. One of the companies was a small outfit named Lockheed which had recently been purchased for $40,000 (~$850,000 today). They told Kelly that they did not have a job for him, but that if he came back next year they might have something for him. So, he returned to Michigan and enrolled in a master's program. In this period, his consulting and research assistant projects became even more impressive. The outputs of one of his personal contracting endeavors even made a splash in the local newspaper:

Five of the qualifying cars which will race at Indianapolis Memorial Day have bodies designed by two university graduate students, C.L. Johnson and E.D. Palmer. All of the cars are semi-stock Studebakers and all qualified for the race at speeds ranging between 110 and 116 miles an hour.

Kelly referred to the experience earned on these contracts as “a liberal education for me in the practical application of aerodynamic theory.”

During this year, one of the airplane models sent to Kelly’s professor for wind tunnel testing came from Lockheed. The model, which would become the Lockheed Electra, was in a new business area for the small firm. Board Chairman Robert Gross had decided that the recently re-organized company’s future was no longer in single-engine wooden aircraft, but in twin-engine all-metal designs. These designs could carry more passengers. At the end of testing, Kelly and his professor markedly disagreed on the readiness of Lockheed’s model. Johnson noted:

It developed some very serious problems, I thought, from what I then knew of aerodynamics. It had very bad longitudinal stability and directional-control problems. But most aircraft of that day had similar failings. Professor Stalker, in consultation with Lloyd Stearman, already a recognized top-notch designer at age 33 and first president of the company, decided the figures were acceptable.

The report was sent to Lockheed without Kelly’s opinions included.

After completing his additional year at Michigan, Lockheed hired Johnson to work under Chief Engineer Hall Hibbard — a young man himself, only seven years older than Johnson. Kelly was going to be making far less than he was running his consulting projects at Michigan. Johnson recounted his initial hiring arrangement, writing, “I was to receive $83 a month (~$2,000 today) to start in tool design until they could assign me as an engineer. There were five engineers at the time.” He may have received a small pay bump when he got his title changed to engineer. But this was not about the money for Johnson. This was about doing what he always wanted to do: designing new planes. He reflected on why leaving his lucrative wind tunnel practice and joining Lockheed was an easy decision, writing:

At the university we certainly weren’t going to design aircraft — and that was my goal. But I didn’t make that much money again until ten years later.

Johnson was outspoken from his first moment at the company. He was not one to sit quietly if he thought mistakes were being made. Johnson recalled his first days at the company, writing:

Practically the first thing I told Chappellet and Hibbard was that their plane was unstable and that I did not agree with the university’s wind-tunnel report.

Hibbard was not insulted. He was, instead, quite open-minded and would not forget that Johnson had said this — even if he did not scramble to fix the problem at that moment. Hibbard recounted years later in an interview that he had wanted to get some “new young blood…fresh out of school with newer ideas” in his engineering department. Kelly Johnson proved to be that and more.

But, initially, the young Johnson was assigned to work in the tooling department. There, he received a kind of hands-on education that he felt most airplane designers couldn’t find anywhere in the post-war years. He took quite well to the hands-on nature of the work. He had, after all, spent much of his youth working in construction. In the tooling department, he worked with an old hand named Bill Mylan and learned the airplane production business from the workman’s end of things. While in the tooling department, Kelly learned to actually build planes, designed a furnace to heat-treat a metal that was new to Lockheed’s production, and made beginner’s mistakes passing design instructions to workmen. In no small part, Kelly credited these early experiences with helping him understand how to design airplanes that were buildable.

Johnson’s Growing Reputation

After a few months of tooling work, Hibbard called Kelly into his office to give him a new assignment. Hibbard was going to put the young would-be engineer’s ideas to the test. The assignment would prove that Johnson’s wind tunnel days had left him with a remarkably deep intuition for aerodynamics. Johnson recalls the meeting with Hibbard and getting sent back to his old wind tunnel, writing:

“Kelly, you’ve criticized this wind-tunnel report on the Electra signed by two very knowledgeable people. Why don’t you go back and see if you can do any better with the airplane?”

Hibbard sent me back to the University of Michigan wind tunnel with the Electra model in the back of my car. It took 72 tunnel runs before I found the answer to the problem…It was a process of evolution. On the seventy-second test, I came up with the idea of putting controllable plates on the horizontal tail to increase its effectiveness and get more directional stability. That worked very well, particularly when we removed the wing fillets, or fairings onto the fuselage — put on apparently because they were coming into style and being used successfully on such airplanes as the Douglas DC-1. And we avoided the trouble others had with them when not used properly…We then added a double vertical tail because the single rudder did not provide enough control if one engine went out. That was so effective we removed the main center tail. And there you had the final design of the Electra. The distinctive twin tails on all of the early Lockheed metal airplanes, and the triple tail of the familiar Constellation airliner of the mid-’40s and ’50s, were the result of these tunnel tests.

Hibbard was impressed and sent Johnson a note telling him how great of a job he had done. Johnson very proudly noted that:

When I returned to the plant. I was a full-fledged member of the engineering department. I was number six.

As a member of the small engineering department, he did crucial work on all parts of the Electra. In the process of learning to wear so many different hats for the Electra and other projects, Kelly felt that he became an “honest-to-god aircraft engineer.”

I worked not only on the aerodynamics of the airplane, but on stress analysis, weight and balance, anything and everything they threw at me. And, of course, more wind-tunnel testing. From that, I became the logical choice to be flight test engineer on the airplane when it was ready to fly.

Kelly noted that the Electra, with its iconic twin-tail, became the fastest multi-engine transport in the world at the time, with cruising speeds above 200 miles per hour. Lockheed delivered the first aircraft in mid-1934. At the time, the small company had six engineers and about 200 factory crew.

Over the next few years, Johnson would do all sorts of work on several derivatives of the Electra. For one of these planes, the Model 14, Kelly ran extensive wind tunnel tests to help design a new airplane part called wing flaps. These flaps allowed the wing to remain large during takeoff to aid in control and then become smaller in flight and increase speed. This Lockheed-Fowler Flap earned Johnson the Lawrence Sperry Award — the first of countless aviation awards he would win in his storied career.

But this achievement was nothing compared to Kelly’s 1938 efforts. With World War II ramping up, the British knew they needed an anti-submarine patrol plane. Getting word of this, Lockheed quickly threw together a proposal for an anti-submarine version of their new Model 14 to show the British. The British representatives were so impressed at how quickly the small company incorporated the recommendations from the group’s first meeting — in a single weekend — that they invited Lockheed to meet with their technical staff in England. Lockheed sent a four-person team on the trip. Kelly was the sole representative for the engineering team.

In the first meeting in England, the British requested several adjustments that required an almost complete re-design by the Lockheed team. Upon leaving the meeting, the team acquired a drawing board and drafting equipment. Kelly then proceeded to lock himself in a hotel room over the three-day weekend. Johnson recalled the event, writing:

I had to fit in all this new equipment, re-arrange copilot and radio operator positions, make weight and structural analysis, figure contract pricing, and guarantee that the design would meet certain performance requirements.

It was a three-day holiday weekend — Whit Sunday, Whit Monday. I worked a solid 72 hours on this redesign — not taking time for sleep, just catnapping briefly when absolutely necessary. I was a rumpled figure…When finally I fell into bed for some very sound sleep — in the room I shared with Courtlandt to save on expenses — it was the first time I had removed my clothes in 72 hours.

Johnson’s efforts were well worth it. The team at the Air Ministry was extremely impressed. Amid an ongoing war, Lockheed’s ability to work quickly was reassuring. Lockheed closed the deal in England that week. Before giving Lockheed a contract to build at least 200 these Hudson airplanes, the British only had one point of concern for Lockheed. The impressive designer who re-modeled the plane over the weekend was extremely young — Johnson was 28. In fact, the entire Lockheed team was young. The leader of the attaché, Courtlandt Gross, was only 36. Gross recounted Air Marshal Sir Arthur Varnay expressing his one concern as follows:

Mr. Gross, we like your proposal very much, and we very much like to deal with Lockheed. On the other hand, you must understand that we’re very unused in this country to dealing — particularly on transactions of such magnitude — on the technical say-so of a man as young as Mr. Johnson. And, therefore, I’ll have to have your assurance, and guarantee, in fact, that if we do go forward, the aircraft resulting from the purchase will in every way live up to Mr. Johnson’s specifications.”

Gross assured Varnay that he and his brother, Lockheed’s Chairman, had every confidence in the young Johnson. With this assurance, a deal was struck. It was, according to Kelly, “the largest aircraft production order placed up to that time in the United States.”

Not long after to returning from the trip, Johnson was named Lockheed’s Chief Research Engineer. The young man now had a big title and a fast-growing reputation.

Kelly Johnson testing a Lockheed Electra model in the wind tunnel at the University of Michigan. Phot courtesy of Wikimedia commons.Skunk Works is Born

As the war progressed and Lockheed’s production lines rolled off Hudson’s vital to the war effort, the firm found its way into more and more contracts. Before the US entered the war, Lockheed had proposed building a prototype jet engine-powered plane for the Army Air Corps. The Air Corps was initially cool on the idea, but eventually agreed as they saw the British working on a similar aircraft. Kelly agreed to produce an experimental model of the proposed aircraft in a timeline that would become quite standard for his Skunk Works crew: 180 days.

The Army was quick to get Kelly his contract. Kelly recalled how quickly General Frank Carrol went from deciding this was a good idea to producing a contract, writing:

“We’ll give you a contract for the airplane, Kelly, and for Nate’s engine as well,” said Gen. Frank Carrol, commanding officer of Wright Field. “But you’ll have to use the British engine in the first airplane because we need it — and all the jet fighters you can build — as soon as possible to use against the Me-262. Your new engine couldn’t possibly be ready for service in time.”

Since I had promised to build a jet airplane within 180 days, I asked, “When will we get a contract? When will the time start?” “You will have a Letter of Intent this afternoon by 1:30 p.m.” he replied. “There is a plane leaving Dayton for Burbank at two o’clock. Your time starts then.”

And it did. The date was June 8, 1943. Gen. H. H. “Hap” Arnold, himself, had approved the contract.

Upon returning to Lockheed’s Burbank headquarters with the good news, Robert Gross was quick to tell Kelly that Lockheed had few people, few machines, and almost no room for such a project. They were already up to their eyeballs producing around 28 airplanes per day — P-38s, B-17s, and Hudsons, and others. Gross was also quick to tell Kelly that he thought nothing would come of the contract. But none of that meant Johnson wouldn’t be allowed to pursue the project. Kelly wrote:

But he [Gross] and Hibbard always were open to new ideas and backed me in many critical times…“You brought this on yourself, Kelly,” Gross stated. “Go ahead and do it. But you’ve got to rake up your own engineering department and your own production people and figure out where to put this project.”

This was an opportunity much bigger than just this one contract for Johnson. The Lockheed company had been rapidly expanding throughout his tenure there, and he felt this was making the design of new, interesting planes more difficult. He reflected on these difficulties, writing:

For some time I had been pestering Gross and Hibbard to let me set up an experimental department where the designers and shop artisans could work together closely in development of airplanes without the delays and complications of intermediate departments to handle administration, purchasing, and all the other support functions. I wanted a direct relationship between design engineer and mechanic and manufacturing. I decided to handle this new project just that way.

Upon receiving Gross’ green light, Kelly set up some crates adjacent to the new Lockheed wind tunnel and found a circus tent to put over the top of them. With that, the Skunk Works was born.

To acquire more tools, Kelly got permission to buy out a small local machine shop. He scrounged together 22 other engineers from around the company who he had worked with before and trusted. Johnson had high standards, and he believed they were all very good. He also set the group up with its own purchasing department. To Kelly, it was vital to Skunk Works’ success that “every function…needed to operate independently of the main plant.”

The Air Corps moved almost as fast as Johnson did. Nine days after the project was green-lit, Skunk Works hosted colonel and a major at Skunk Works for a mockup conference that went smoothly.

Many complications arose in completing the project. Kelly emphasized that “it was a complete new world of flying and testing” with these jet-engine aircraft. The plane ran into the same compressibility problem when it approached Mach 1 that Lockheed had encountered with its P-38 development. The team was able to overcome the airplane’s aileron buzzing by installing shock absorbers in just the right fashion. In addition, common problems present in new aircraft designs popped up. Certain parts of the plane proved precarious to weld together. Extensive re-designing and vacuum testing were required to overcome these issues. But whatever problems came up, the small team of particularly talented engineers and workmen worked through them quickly. The small operation seemed to be as effective at producing experimental aircraft as Kelly had hoped.

And the Air Corps was also an excellent project partner. This was vital, in Kelly’s estimation. He noted that:

There were only six people from the military involved and two or three of us from Lockheed. We had approval to proceed that night. Six days later we had our government furnished equipment — guns, radio, wheels and tires, etc. At every stage of the work, we had excellent cooperation from Wright Field and the officers involved with the project. The job could not have been completed on such a tight schedule without it.

Skunk Works delivered the XP-80 plane to the Air Corps on Day 143 of the contract — more than a month ahead of schedule. The XP-80 took its first flight on January 8, 1944. Almost immediately, the Air Corps gave Skunk Works another contract to produce a modified version of the plane with an airframe almost twice as large — to make use of GE’s new engines. The two copies of the aircraft the Air Corps requested were delivered 132 days later. This design, the YP-80, was the precursor to the famous P-80 and two-seat T-33 training vehicle. According to Kelly, there went on to be more than 6,000 built in total.

The plane would not go on to see major use in World War II given the ending of the conflict. But it would later prove itself — first in the North Korean vs. South Korean conflict in 1950 — and also set several records. The most notable of which is probably the F-80 flying the route from Long Beach to LaGuardia in a record 4 hours and 13 minutes, averaging 584 miles per hour.

These first Skunk Works projects were a smashing success. Producing a prototype of a truly novel aircraft will never be completely smooth. But with these projects the team at Skunk Works was beginning to make it clear that they could routinely overcome these obstacles on tight schedules, even if not easily.

Skunk Works’ original XP-80 prototype. Photo courtesy of Wikimedia commons.Kelly Johnson congratulating test pilot Tony LeVier on a successful test flight. Photo courtesy of More Than My Share of It All.Early photo of the F-80. As Johnson noted in the caption of this photo in More Than My Share of It All, “Even by today’s standards the Shooting Star looks sleek and swift in this earlier photo.”The First Iconic Skunk Works Plane: the U-2 “Spy Plane”

In 1953 the group began work on their most ambitious airplane yet. Around this time, defense contractors were made aware of the country’s desperate need for aircraft that could fly recon missions over the USSR to acquire information on Soviet military capabilities. They needed equipment that could do this without being shot down by the Soviet’s advanced anti-aircraft weapons. Always game for a challenge, the Skunk Works put together a proposal for a plane that could fly approximately 70,000 feet, have a range of 4,000 miles, and would be a steady platform for advanced photographic equipment. Without a plane that could reach that height, the CIA and Air Force would have to resort to low-flying, high-casualty tactics to acquire information1

The Air Force initially did not think Skunk Works’ proposal was realistic. Other proposals to achieve this altitude were somewhat primitive technology like weather balloons.2 But the folks in DC decided to let the Skunk Works go ahead with the project anyway. Many in DC had exceptional faith in Kelly Johnson, just as the Gross brothers and Hall Hibbard did back in Burbank. Johnson describes the events that led to Skunk Works winning the contract as follows:

Our first presentation was to the Air Force, where it was turned down as too optimistic. They questioned that any engine even would operate at the altitude we were proposing. They were correct in that there was not proof at that time that this was possible. And the Air Force already had an airplane in development with the Martin company — an airplane with two engines which was preferred to our single-engine design.

But our proposal reached Trevor Gardner, then Assistant Secretary of the Air Force for Research and Development, and a brilliant engineer in his own right. Late in 1953, he invited me to come to Washington to discuss it. He had assembled a committee of scientists and engineers, and for three days they put me through a grilling as I had not had since college exams. They covered every phase of the aircraft design and performance — stability, control, power plants, fuels — everything.

Later I met and lunched with Air Force Secretary Harold Talbott, CIA Director Allen Dulles, and his right-hand man, Larry Houston, among a distinguished group. When I was asked why I thought Lockheed could do what I proposed — build 20 airplanes with spares for roughly $22 million and have the first one flying within eight months, Gen. Donald Putt graciously volunteered, “He has proven it three times already—on the F-80, F-80A, and F-104.”

The CIA took the lead in funding the project and Richard Bissel — Special Assistant to CIA Director Dulles and a fixture of post-war CIA leadership — became Skunk Works’ project liaison.

Kelly selected 24 other engineers to work on the project with him. The total project staff would eventually grow to a headcount of 81. Given the secrecy of the project, Skunk Works was not allowed to fly the plane at its Burbank headquarters. So, Kelly sent a test pilot around to find a secure location on which to build a runway and basic testing facilities. He named the remote strip of land that the pilot found in the Nevada Dessert “Paradise Ranch.” Today it is known as Area 51. Once selected, Lockheed hired contractors to begin constructing some roads, hangars, offices, living accommodations, etc. on the land.3 All of this was done while Skunk Works was working on the novel aircraft.

Skunk Works had to keep an extreme focus on ruthlessly minimizing weight throughout the design and construction of the aircraft. Many staple building materials had to be made thinner or replaced altogether. The entire team was constantly scouring for lighter materials and designs for each part of the aircraft. Each would have to be extensively tested and modified until the engineers were comfortable using it. All of this led to a quite interesting aircraft. Ben Rich explains how strange the aircraft seemed to other pilots and engineers, writing:

We designed and built that airplane for lightness. The wings, for example, weighed only four pounds per square foot, one-third the weight of conventional jet aircraft wings. For taxiing and takeoffs, jettisonable twin-wheeled “pogos” were fitted beneath the enormous fuel-loaded wings and kept them from sagging onto the runway while taking off. The pogos dropped away as the U-2 became airborne.

The fuselage was fifty feet long, built of wafer-thin aluminum. One day on the assembly floor, I saw a worker accidentally bang his toolbox against the airplane and cause a four-inch dent! We looked at each other and shared the same unspoken thought: was this airplane too damned fragile to fly?…Pilots were scared to death flying those big flapping wings into bad weather situations — afraid the wings would snap off…Adding to the sense of the airplane’s fragility was that the razor-thin tail would be attached to the fuselage by just three five-eighth-inch bolts.

(The plane and its enormous, drooping wings that almost sagged onto the runway, proved more resilient than the pilots or designers ever imagined.)

The engine was also a massive obstacle. This was partially a problem for the Skunk Works team but primarily fell to a long-time Skunk Works contractor. Rich described the difficulty in building an engine to operate at these high altitudes, writing:

My principal work was on the engine’s air intake, which had to be designed and constructed with absolute precision to maximize delivery of the thin-altitude air into the compressor face. Up where the U-2 aimed to cruise, just south of the Pearly Gates, the air was so thin that an oxygen molecule was about as precious as a raindrop on the Mojave desert. So the intakes had to be extremely efficient to suck in the maximum amount of oxygen-starved air for compression and burning.

In the end, the engine problem was overcome by the work of Skunk Works and two very impressive contractors. The most important contributor was probably Pratt & Whitney, a long-time Skunk Works partner. The P&W team modified an existing high-altitude engine they made that was considered top-of-the-line. The head of Pratt & Whitney himself put his best people on the job and they found a way to modify the engine so it could function in the extreme conditions. A major contribution was also made by Shell’s research team. Kelly Johnson got Shell’s R&D team to develop “a special low-vapor kerosene for high altitudes” that would not boil or freeze in the extreme conditions in which the U-2 operated. In many Skunk Works projects, key contributions from trusted contractors were vital.

The U-2 was completed and shipped to Paradise Ranch in late July — less than nine months after the contract was first issued. In early August, it accidentally took flight for the first time while conducting a routine taxi test. As test pilot Tony LeVier recalled, “Who’d have guessed an airplane could take off only going 70 knots? That’s how light it was.” The plane took its first official test flight a week later. The high-flying plane was a marvel, “able to glide 250 miles from 70,000 feet.” Within nine months of the first test flight, the CIA U-2s became operational.

On top of the U-2s achieving Kelly’s lofty technical goals and obtaining high-priority intelligence on the USSR, many consider this contract one of the greatest bargains in defense procurement history. Johnson and the Skunk Works came in far under budget. They returned $2 million of the original $22 million budget and were able to construct six extra airplanes from the spare parts the team didn’t need. Rich said they were able to build with these spares simply “because the U-2 functioned so well.”

Kelly Johnson standing with his arm on the thin, dangling wing of an early U-2. He noted on this photo in More Than My Share of It All that the plane was, “considered a technological triumph by any standard.”What did Kelly Johnson think was Skunk Works’ special sauce?

Skunk Works’ success on the U-2 was not an isolated event. The Skunk Works would also go on to produce two more planes considered all-time greats — the Blackbird models and the F-117 stealth bomber — in addition to dozens of other successful planes. This, of course, raises the question: how did Skunk Works staff build so many novel planes on such a routine basis? Did they have a kind of special sauce?

Kelly Johnson believed they did. Kelly famously had a list of 14 rules he used to run the Skunk Works. However, upon reading his biography, it becomes clear that the beliefs and experiences that led to those rules could be distilled down to something like three key principles. The three principles are, roughly:.

1. Reduce the bureaucracy to almost zero. Ideally, one person should have almost complete authority over day-to-day decision-making.

2. Keep the team ruthlessly small.

3. Whenever possible, only take on contracts where there is enough mutual trust with funders and subcontractors to work with them with a minimum amount of bureaucracy. If funder decisions cannot be made swiftly, the project is probably not worth pursuing.

The three beliefs above can be seen at work all over the Skunk Works operation. The following subsections will give the reader a clear idea of how these beliefs manifested themselves in the practical management of the Skunk Works.

Luckily, in More than My Share of It All, Kelly Johnson dedicates an entire chapter to explaining the simplicity of his special sauce. In the chapter, he makes it clear that he can’t believe how few true imitators Skunk Works had — given that so many companies claimed to have their own Skunk Works. Kelly opens the chapter, a bit incredulously, writing:

What the Skunk Works does is secret. How it does it is not. I have been trying to convince others to use our principles and practices for years. The basic concept as well as specific rules have been provided many times. Very seldom has the formula been followed.

As this section will detail, the lack of true Skunk Works imitators will not be shocking to any who are familiar with bureaucracies. The key management principles that helped Skunk Works thrive are simply non-starters in most bureaucracies. This is precisely why so many of Kelly’s rules-of-thumb are obsessed with warding off bureaucratic processes; this is also why Ben Rich lived in such fear of bureaucracies smothering Kelly’s great band of engineers from all sides when he took over the Skunk Works.

Principle #1: Minimize Bureaucracy, Centralize Power

For a variety of reasons, Kelly believed bureaucracies killed progress in designing and building new aircraft. Kelly believed one person should have complete control over day-to-day decision-making. An engineer should be able to come to him with a problem, and he needed to have the authority to tell them “yes” or “no” on the spot.

This type of approach may not have been as rare in his youth, but as his career wore on it became almost extinct. He lived to see the onset of slow decision-making processes begin to expand in the industry. Lamenting this, he writes:

I fear that the way I like to design and build airplanes one day may no longer be possible. It may be impossible even for the Skunk Works to operate according to its proven rules at some point in the future. I see the strong authority that is absolutely essential to this kind of operation slowly being eroded by committee and conference control from within and without. The ability to make immediate decisions and put them into rapid effect is basic to our successful operation.

In the 1950s, even the comparatively small Lockheed had grown bureaucratic enough to stifle ambitious work. Before being recruited to the Skunk Works in the late 1950s, Ben Rich worked in the main portion of the Lockheed company. He said the following about his time there and how the company operated:

Lockheed was very regimented and bureaucratic, and by my fourth year on the payroll I felt stymied and creatively frustrated. I had a wife and a new baby son to support, and my father-in-law, who admired my moxie, was pushing me to take over his bakery-delicatessen, which earned the family a very comfortable living.

The two decades between Kelly Johnson and Ben Rich joining the company had seen Lockheed cease to be the kind of place where an exceptionally driven individual or a genius had the chance to spread their wings. Hall Hibbard used to say that Kelly Johnson could “see air.” The then-Chief Engineer kept assigning the young Johnson more and more responsibility until, by the time he was made Chief Research Engineer at 28, he’d been largely responsible for many key aspects of Lockheed’s designs anyway. Rich — who was eventually Kelly’s hand-picked protegé — on the other hand, was considering leaving Lockheed to run a deli!

Johnson wrote the management chapter in his book hoping that individuals would find it who were brave enough to run a truly ambitious R&D operation. While the aircraft industry did not take his advice, readers of this Substack would be wise to. The same reasons that large organizations resisted properly copying Skunk Works back then are similar to the excuses they would use today — many aspects of bureaucracies and their emergent behavior are evergreen. Johnson outlined the primary reason truly copying Skunk Works was a non-starter in most companies, writing:

Most companies, while desiring the benefits, will not pay the price in revised methods and procedures for setting up a Skunk Works-type of operation. They will not delegate the authority to one individual, as Lockheed did in my case from the very first Skunk Works. It requires management confidence and considerable courage. Without the authority assigned to the Skunk Works by our military customers and the Lockheed corporation, we would not have been able to accomplish many of the things we have done, things about which I felt we could take a risk — and did.

Skunk Works may never have been green-lit in the first place had Hibbard and Gross not had an exceptional level of trust in Kelly Johnson. As Rich recalled, there was a general sentiment at Skunk Works when he joined in the late 1950s: “The open secret in our company was that the chief engineer [Kelly] walked on water in the adoring eyes of CEO Robert Gross.” And Johnson took full advantage of his well-earned long leash.

Kelly set up Skunk Works with a belief in the supreme power of individual visions. He did not believe that committees would design broken planes. But he did believe that they would produce uninspired and unoriginal planes. And there was absolutely no room for processes like that when the goal of his team was to design truly novel, experimental aircraft. He reflected on the impacts of bureaucracy on airplanes themselves, writing:

There is a tendency today, which I hate to see, toward design by committee — reviews and recommendations, conferences and consultations, by those not directly doing the job. Nothing very stupid will result, but nothing brilliant either. And it’s in the brilliant concept that a major advance is achieved.

Development of some of this country’s most spectacular projects — the atom bomb, the Sidewinder missile, the nuclear-powered submarine — all were accomplished by methods other than the conventional way of doing business outside the system.

Johnson’s efforts to minimize bureaucracy and belief in individual visions allowed Skunk Works to both produce ambitious plane designs and build them on short timelines. Kelly recruited some of the best Lockheed had to offer. He implemented a system where a small number of simple drawings were used and things could be changed on the fly. He put all the engineers, designers, and workmen in close quarters. And he did everything he could to enable them to test ideas as fast as they had them.

A genius would feel challenged by the environment and Johnson’s no-bullshit technical grillings, not stifled by bureaucracy. In his book, Ben Rich referred to a 24-year-old aerodynamicist working on a project with him, the late Dave Campbell, as “destined for greatness.” Skunk Works was an environment where those qualities could both show and be taken advantage of. The primary portion of Lockheed was no longer like that. But in Skunk Works, Kelly kept a culture alive similar to the company he first arrived at in the 1930s. In that young Lockheed company, with only five engineers, Kelly was not just an aerodynamicist, but an aerodynamicist, stress analyst, weight/balance engineer, and flight test engineer all in one. He was challenged and largely unencumbered by rules and processes. In that role, his genius could truly be put to work.

When Kelly Johnson was at the helm of the Skunk Works, the individual visions overriding the projects were often his own. Of course, during his reign there was plenty of room for Skunk Works staff to demonstrate their genius on their own components of the plane, helping others problem solve, or via influencing Johnson with their ideas. Johnson even kept quarters on him to dole out if anyone ever proved him wrong. However, the fact of the matter is that in his decades-long run as the lab’s leader, he lost very few quarters. Rich said the following on Johnson’s wide-ranging, freakish understanding of aircraft design:

Nothing got by the boss. Nothing. And that was my sharpest impression of him, one that never changed over the years: I had never known anyone so expert at every aspect of airplane design and building. He was a great structures man, a great designer, a great aerodynamicist, a great weights man. He was so sharp and instinctive that he often took my breath away. I’d say to him, “Kelly, the shock wave coming off this spike will hit the tail.” He would nod. “Yeah, the temperature there will be six hundred degrees.” I’d go back to my desk and spend two hours with a calculator and come up with a figure of 614 degrees. Truly amazing. Or, I’d remark, “Kelly, the structure load here will be…” And he would interrupt and say, “About six point two p.s.i.” And I’d go back and do some complicated drudge work and half an hour later reach a figure of 6.3.

With someone like Kelly around, it makes sense that his vision was the individual vision elevated to the fore. But the lab’s success was not dependent on his vision. Arguably the lab’s greatest military contribution, the F-117 stealth bomber, came in its first project after Johnson retired.

Kelly picked Ben Rich as his successor because he thought Ben was innovative, practical, and an absolute terrier in working through some of Skunk Works’ most difficult problems. Those traits were what was essential. But Rich knew he was no visionary like Kelly Johnson. Rich’s stealth project was extremely unique and gutsy. This was the case because he elevated the ambitious vision of a single individual to the fore of the project. But it was not his vision. He describes the project’s inception, writing:

I wish I could claim to have had a sudden two a.m. revelation that made me bolt upright in bed and shout “Eureka!”…The truth is that an exceptional thirty-six-year-old Skunk Works mathematician and radar specialist named Denys Overholser decided to drop by my office one April afternoon and presented me with the Rosetta Stone breakthrough for stealth technology…

Denys had discovered this nugget deep inside a long, dense technical paper on radar written by one of Russia’s leading experts and published in Moscow nine years earlier…As Denys admitted, the paper was so obtuse and impenetrable that only a nerd’s nerd would have waded through it all…As he explained it, [Pyotr] Ufimtsev had revisited a century-old set of formulas derived by Scottish physicist James Clerk Maxwell and later refined by the German electromagnetics expert Arnold Johannes Sommerfeld. These calculations predicted the manner in which a given geometric configuration would reflect electromagnetic radiation. Ufimtsev had taken this early work a step further. “Ben, this guy has shown us how to accurately calculate radar cross sections across the surface of the wing and at the edge of the wing and put together these two calculations for an accurate total.”

Rich then explains how radar technology works and how existing planes often show up on radar as large as houses. He then continues:

We desperately needed new answers, and Ufimtsev had provided us with an “industrial-strength” theory that now made it possible to accurately calculate the lowest possible radar cross section and achieve levels of stealthiness never before imagined. “Ufimtsev has shown us how to create computer software to accurately calculate the radar cross section of a given configuration, as long as it’s in two dimensions,” Denys told me. “We can break down an airplane into thousands of flat triangular shapes, add up their individual radar signatures, and get a precise total of the radar cross section.”

So, Rich had Overholser and a labmate put together a program to take advantage of this insight and design a model plane out of 2D surfaces that had a minuscule radar cross section — computing power was not strong enough to build the model out of 3D surfaces. The final design’s radar cross section appeared to be smaller than existing planes by several orders of magnitude. Instead of the plane being the size of a home or a barn, the question was now, “Which of these ball bearings in Ben Rich’s pocket would show up bigger than the plane, and which wouldn’t?”

Many Skunk Works veterans thought the design was horrible. It looked like an ugly, crude, origami rendition of a plane. The more a lab member knew about aerodynamics, the uglier the model looked. Many Skunk Works old-timers pushed back. Ben Rich wanted his team — which specialized in building planes that flew exceptionally — to focus on building the least aerodynamically sound plane they’d ever seen. Additionally, many in DC thought Rich must be mistaken about what the true radar cross section would actually be. Kelly Johnson held both of these gripes against Rich. He referred to the radar cross section computations as “theoretical claptrap.” Rich won a rare quarter off Johnson when the wooden model did register a vanishingly small radar cross section. Johnson, still skeptical, flipped him the quarter and said, “Don’t spend it until you see the damned thing fly.”

Luckily, it did. It was a tall task. But with the help of recent developments in computing power, the plane was able to fly using constant computer adjustments to help counteract issues with its chaotic aerodynamics. Kelly was wrong to not believe in the plane. But he was visionary in building a lab that elevated individual design visions and a single, strong boss above the gripes of a retired consultant like him. And he was right to entrust the lab to someone who was a terrier in assessing a technical idea’s merits, even if the results looked very unorthodox.

The plane broke no speed records, but that didn’t matter much since it showed up no bigger than a bird on enemy radar screens. The stealth bomber project is a strong data point in favor of the idea that minimizing bureaucracy and entrusting a bright individual with power was more essential to Skunk Works’s success than any stroke of genius from Kelly Johnson. Kelly Johnson was a genius. But, more importantly, Skunk Works was a place built to facilitate genius ideas into built reality.

Kelly built an operation that left enough leeway for the common sense and practical wisdom of his team to take over. From one project to the next, many of the operational specifics changed. Johnson noted:

The theory of the Skunk Works is to learn how to do things quickly and cheaply and to tailor the systems to the degree of risk. There is no one good way to build all airplanes.

At the start of each project, the Skunk Works leader was free to sit down with his best people and plan out exactly how to build the new plane in the way that made the most sense. From one plane to the next, the process might look extremely different. To make this work, the team needed to be kept small enough that new operating procedures could be implemented without chaos ensuing. And they needed to do this without countless pre-arranged meetings. That wasn’t the Skunk Works way.

Principle #2: Keep the Team Viciously Small

Skunk Works’ small team emphasis allowed team members to seamlessly stay updated on the entirety of the project. It kept things small enough that even the machinists could contribute knowledge to the early design stages of the project, not just have ideas dictated down to them.

Johnson emphasized the high-level importance of maintaining a small team, writing:

Working with a limited number of especially capable and responsible people is another requirement. Reducing reports and other paperwork to a minimum, and including the entire force in the project, stage by stage, for an overall high morale are other basics. With small groups of good people you can work quickly and keep close control over every aspect of the project.

Ben Rich’s book describes precisely how this small team emphasis made a difference in Skunk Works’ processes. He writes:

We had our own unique method for building an airplane. Our organizational chart consisted of an engineering branch, a manufacturing branch, an inspection and quality assurance branch, and a flight testing branch. Engineering designed and developed the Have Blue [stealth prototype] aircraft and turned it over to the shop to build. Our engineers were expected on the shop floor the moment their blueprints were approved. Designers lived with their designs through fabrication, assembly, and testing. Engineers couldn’t just throw their drawings at the shop people on a take-it-or-leave-it basis and walk away.

Our senior shop people were tough, experienced SOBs and not shy about confronting a designer on a particular drawing and letting him know why it wouldn’t work. Our designers spent at least a third of their day right on the shop floor; at the same time, there were usually two or three shop workers up in the design room conferring on a particular problem. That was how we kept everybody involved and integrated on a project. My weights man talked to my structures man, and my structures man talked to my designer, and my designer conferred with my flight test guy, and they all sat two feet apart, conferring and kibitzing every step of the way. We trusted our people and gave them the kind of authority that was unique in aerospace manufacturing. Above all, I didn’t second-guess them.

Even the procurement staff were a part of this. Johnson described why it was vital to keep them in the loop in a similar fashion, writing:

We maintain a very close liaison from me to the designer, to the purchasing agent — so that he understands the urgency in acquiring materials; to the tooling people; and to the people who actually will build any part of the plane.

For those who read the previous FreakTakes piece on ARPAnet contractor BBN, this approach should sound remarkably familiar. In it, I shared the following quote from ARPAnet project leader Frank Heart, reflecting on how he managed his small team of bright engineers:

I think mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

In that piece, I then expanded upon Heart’s point, writing:

Heart was extremely averse to the team growing so large that communication had to be done on paper. He preferred their style of “very, very frequent interaction on problems.” As Walden remembered, “I don’t remember such a thing as we had a weekly progress meeting. We probably were more in tune with the progress than that; we probably did it hourly.” Heart and the entire, small team did their best to stay technically involved with each other’s work. Heart, as a rule, insisted on technically understanding every bit of the project. The individual team members attempted to do the same, which they felt was quite easy to do. The team all had offices and work stations immediately in the vicinity of each other. As things came up, they’d simply go talk with the relevant individual or call an impromptu meeting.

BBN’s Frank Heart and Kelly Johnson shared extremely similar beliefs on how to manage engineering teams building extremely novel equipment.

The Skunk Works project teams were about ten times larger than BBN’s project teams, given the more industrial nature of the operation. But these were still very small teams by aviation standards. Kelly estimated that Skunk Works’ teams were only 10%-25% of the usual size when “compared to the so-called normal systems” of his day. Ben Rich, after some time at the Skunk Works, helped lead a staff of three on the Blackbird project. He was quick to note that “by Skunk Works standards that was almost an empire.”

Even an exceptionally difficult project like the SR-71 Blackbird did not balloon in size. That’s saying something. Nearly every material used and part designed for the airplane needed to be rethought due to the extreme cruising speeds (~2,000 mph) and altitude (~85,000 feet) the plane would reach. Kelly made it clear at the very beginning of the project that the team would need to “start from scratch as if we are building the first airplane, just like the Wright brothers.”

In the face of this overwhelming project, Johnson doubled down on his own people and systems. He did not resort to an inordinate increase in headcount. The following excerpt from Ben Rich confirms the still-modest scale of things on the massive project. Rich writes:

Developing this air-inlet control system was the most exhausting, difficult, and nerve-racking work of my professional life. The design phase took more than a year. I borrowed a few people from the main plant, but my little team and I did most of the work. In fact the entire Skunk Works design group for the Blackbird totaled seventy-five, which was amazing.

Writing his book in 1994, Ben Rich was acutely aware of how much the industry had changed. He closes the above quote with the following quip:

Nowadays, there would be more than twice that number just pushing papers around on any typical aerospace project.

Principle #3: Maintain Similar Expectations of Project Partners

Kelly’s belief in minimizing bureaucracy and keeping team size ruthlessly small did not stop with his own Skunk Works team. He was adamant that, if at all possible, projects should only be undertaken when a project’s funder could promise similar, strong project leadership. This was the second of his famous fourteen rules:

Strong but small project offices must be provided both by the military and industry.

Johnson likely felt there was little use in putting his team into overdrive just for a customer to hold up progress and changes every step of the way. So, he wanted quick “yes” or “no” decisions from funders as well.

Throughout his book, Johnson was quick to lavish praise on those individuals and areas of the armed services that, once they decided to move forward, were excellent project partners. When he had it his way, Johnson’s projects often had something like one project liaison and ~6 people whose opinions mattered in key decisions. Kelly provided many specific cases of what ideal relationships with funders looked like. Sharing one example, he writes:

Operating at its best on our Air Force programs, the Skunk Works could get an almost immediate decision on any problem. I could telephone Wright Field, Dayton, for example, talk to my counterpart who headed the small project office there for the military — and who was allowed to stay with it to conclusion — and get a decision that same morning. Now, that’s just not possible in standard operating procedure. It’s a difficult concept to sell for the first time, though, since it means abandoning the system.

This is what an ideal, standard case looks like. But it should also be noted that Rich and Johnson mention several cases of project liaisons going above and beyond in almost unbelievable ways. In one case, Rich and Johnson noted that an exceptional CIA project partner set up shell companies and a kind of false supply chain to to acquire a large quantity of titanium from the Soviet Union for an aircraft. But, of course, that level of proactivity from the funder was not the usual.

But working with a customer that knew how to make decisions quickly was essential. It was so essential that Ben Rich noted that Kelly had an unwritten fifteenth rule:

Starve before doing business with the damned Navy. They don’t know what in the hell they want and will drive you up a wall before they break either your heart or a more exposed part of your anatomy.

On one occasion, Ben Rich did not heed this warning and sought to build a stealth boat for the Navy. He planned to do it using the same design software that they were using to build the F-117 bomber. Pursuing this project was a decision Rich came to sorely regret.

The project actually broke multiple Kelly Johnson rules. But the first misstep was forced onto Ben Rich due to internal Lockheed politics. Lockheed’s Ocean Division was in financial trouble. Meanwhile business was booming at the Skunk Works even without the stealth ship contract. So, Rich handed off the development contract for the ship to Lockheed’s Ocean Division. But Rich also sent several core team members over to the division to help lead the project. The Skunk Works group group was led by Ugo Coty. Before sending Coty and co., Ben had told the Ocean Division executives that it would be essential to let the mathematically inclined team members have the final say over everything in the ship’s design.

To the executives, this was a seemingly off-the-wall way to build an aircraft or ship. But this is exactly what Rich had decided to do with the F-117 project. It was not at all the normal procedure, but it’s what made the most sense. The radar cross section of the ship could only be as low as the most visible of its parts. Even one periscope being slightly misdesigned could make the ship appear bigger than a barn. And the mathematics of radar cross sections was unintuitive to almost all of the ship engineers.

The Oceans Division proved incapable of following Ben’s advice. And the Navy brought so many cooks into the kitchen that it probably wouldn’t have mattered even if that had not happened. Rich describes the mess caused by the Lockheed and Navy bureaucracies on the project, writing:

“We need Ugo to keep those damned shipbuilders from going off on a tangent,” I told Roy Anderson. “This is one project where the method of shipbuilding is much less important than the stealth technology,” I told Roy. “They’ll want to sacrifice the stealth if it gets in the way of the ship’s performance, but Ugo will force them to stay focused. All Dr. Perry wants to prove out is the stealth. That’s key to this test. If the ship merely floats that’s good enough.” As it happened, my fears about the conflicting agendas between professional shipbuilders and experts on stealth technology, like Dr. Perry, were realized almost from the first day that the Ocean Division took over the project.

Ugo Coty did his best, but he ran into heavy weather. His original six-man operation quickly was shunted aside by eighty-five bureaucrats and paper-pushers running the program for the division. Then the Navy marched in, adding its supervision and bureaucracy into the mix with a fifty-man team of overseers, who stood around or sat around creating reams of unread paperwork. No ship ever went to sea — not even a top-secret prototype — without intensive naval supervision to ensure that all ironclad naval rules and regulations were strictly enforced before the keel was ever laid.

“Where is the paint locker?” a Navy commander demanded of Ugo, rattling the blueprint plans. Since the days of John Paul Jones, every naval ship afloat has a damned paint locker on board. Sea Shadow would definitely not be the only exception since the Revolutionary War.

To hammer home the point about the “Navy way,” Rich goes on to tell a story of Skunk Works modifying its Jetstar for the Navy under Kelly Johnson. The original Jetstar project itself took 55 engineers eight months. The Navy project, which was a little more complicated, took 27 months. Rich noted that “one hint as to the reason why” is that Jetstar’s mockup conference had six visitors on hand. For a similar conference, the Navy sent 300. To him, that was the “Navy way.”

Before moving on, it is important to note that Johnson also had a strong preference for only working with contractors who didn’t require increased bureaucracy to be added to the project. In an ideal case, this would look like the U-2 project in which key technical project problems were offloaded to Pratt & Whitney and Shell researchers to work their way through. In another case, Skunk Works took a crucial tire problem to B.F. Goodrich “which developed a special rubber mixed with aluminum particles that gave our wheels a distinctive silver color and provided radiant cooling.” This was just the solution Skunk Works needed. However, in more standard cases what this means is simply sticking with trustworthy suppliers and extending them the same respect Skunk Works liked to have itself. As Kelly Johnson put it:

Suppliers and others associated with a project must be extended the same kind of rules and permissions that are given us for the entire program. This cuts red tape and costs and allows all participants to concentrate on the product instead of a system. It is so simple.

This was more feasible in some cases than in others. But these were the relationships Johnson strove to maintain.

While all of this sounds quite simple, replicating Skunk Works has proven anything but simple for most would-be copycats.

Why So Many Skunk Works Copycats Failed

In the early 1970s, shortly before Kelly Johnson’s retirement, Northrop made an offer to poach Ben Rich — then one of Johnson’s top lieutenants. Ben Rich recalls Kelly’s frank warning when he confronted Johnson with his offer, writing:

I laid out Northrop’s offer, and he closed his eyes and solemnly shook his head. “Goddam it, Ben, I don’t believe a word that guy said to you. I’ll bet my ranch against Northrop starting its own Skunk Works. Companies give it lip service because we’ve been so successful running ours. The bottom line is that most managements don’t trust the idea of an independent operation, where they hardly know what in hell is going on and are kept in the dark because of security. Don’t kid yourself, a few among our own people resent the hell out of me and our independence. And even those in aerospace who respect our work know damned well that the fewer people working on a project, the less profit from big government contracts and cost overruns. And keeping things small cuts down on raises and promotions. Hell, in the main plant they give raises on the basis of the more people being supervised; I give raises to the guy who supervises least. That means he’s doing more and taking more responsibility. But most executives don’t think like that at all. Northrop’s senior guys are no different from all of the rest in this business: they’re all empire builders, because that’s how they’ve been trained and conditioned. Those guys are all experts at covering their asses by taking votes on what to do next. They’ll never sit still for a secret operation that cuts them out entirely. Control is the name of the game and if a Skunk Works really operates right, control is exactly what they won’t get…Mark my words, you’ll be reporting to a dozen management types and they won’t let you out of their sight for one minute.”

Ben Rich didn’t leave. And the Northrop operation went on to operate essentially how Kelly said it would. This prediction was not Kelly Johnson demonstrating any kind of remarkable prediction abilities. He was simply recounting what he continually saw happen when companies said they wanted a Skunk Works.

The sorts of problems Kelly figured would happen to Northrop and those that had happened in the Naval stealth project arose repeatedly. The following excerpt describes similar problems that afflicted the Army’s Cheyenne rigid-rotor helicopter program. Kelly himself attempted to train Jack Real and six of his top supervisors who would be running the program. Kelly had them shadow his Skunk Works operation for six months. But his efforts were in vain. Kelly writes:

Real and his team began with great enthusiasm to apply our operating methods to meet the Army’s design specifications. But within six months, the satellite Skunk Works had a purchasing department larger than my entire engineering department working on seven projects. They had become buried in the usual paperwork already.

Despite the best of intentions, the Army had at the time ten different test centers and bases involved in the procurement of new weapon systems. And when you have that many representatives involved in design and development, with no single person in charge to represent the customer, the Skunk Works concept cannot work.

It is absolutely imperative that the customer have a small, highly-concentrated project office as a counterpart to the Skunk Works manager and his team. It is not a concept easily adopted after years of working within the system. There has to be an all-out commitment, or the method will not work.

When there was not an “all-out commitment” to work within the basic system, the differences in results were startling.

At the time the Cheyenne contract was cancelled, 145 Army personnel were involved in the program. In contrast, the total at the Skunk Works for both CIA and Air Force representatives in our U-2 and SR-71 programs did not exceed six people.

It should also be noted that Kelly himself was able to come into at least one operation outside the Skunk Works and implement his system successfully. At one point, he was called into the Lockheed Missiles and Space Company to help turn around an ailing project. Kelly came in and rolled out the same playbook I’ve described to you in this piece. He describes his work on the project, writing:

When Skunk Works principles really are applied, they work. An example of their successful application was development of the Agena-D launch vehicle…The satellite that was to become this country’s workhorse in space was in trouble in terms of design and cost but especially in reliability, which stood at an incredibly low 13.6 percent. I was drafted, in effect, to go up to the Lockheed Missiles and Space Company and fix it. We set up a Skunk Works operation with the company’s design project engineer, Fred O’Green, as head…

It proved again our axiom: If you have a good man and let him go, he’ll really perform. In terms of today’s world, that axiom should apply to women as well. When I first reviewed the Agena project, I discovered that 1,206 people were employed in quality control alone, achieving only that 13 percent reliability factor! It should have been the world’s most reliable vehicle just using the inspection department. That was enough people to design and build the thing.

At the Baird Atomic Company, which made the vehicle’s horizon sensor, Lockheed had 40 people inspecting, coordinating, and reporting. Yet Baird had only 35 people building the instrument. We resolved that situation by returning responsibility for the product to the vendor. For example, I telephoned Walter Baird personally since he and I had worked together on a number of other Skunk Works projects. He immediately agreed to pick up his end of the log.

Kelly estimated that the changes saved the government at least $50 million in costs and led to completing in nine months what had been scheduled for 18. 350 drawings had been made instead of the projected 3,900. And quality control personnel had been “slashed” to 69. The operation was now able to output design drawings in a day rather than a month.

Ben Rich had this to say about Kelly’s turnaround of the Agena operation:

The president was already spending a billion dollars in covert funds on the Agena rocket that would boost our first spy satellite into orbit. Bissell was in charge of that program, too, and the first twelve test firings had all been failures. Lockheed’s Missiles and Space Company in Sunnyvale, California, had that contract, and Bissell asked Kelly to evaluate and reorganize their operation. Kelly set up a mini Skunk Works and, coincidentally or not, the thirteenth test shot was a success.

Many claim to have copied Skunk Works in its heyday. But few ever truly did.

A Different World

In many ways, times have changed. Generals can no longer have an aircraft development contract written up and sent out in a few hours. Lowest bidder subcontracts might be required before an outfit like Skunk Works has a new kind of fuel made. Many think the management of public companies is much more short-sighted than it was in the 1960s. The list goes on.

The writing of Ben Rich himself emphatically demonstrates how much these changes haunted him. Throughout his book, he routinely mentions things like nightmares regarding corporate bean counters. In one breath he tells us of the best idea his Skunk Works would ever produce. In the next, he mentions how the Lockheed bureaucracy might threaten the idea being realized. He writes:

I wish I could claim to have had a sudden two a.m. revelation that made me bolt upright in bed and shout “Eureka!” But most of my dreams involved being chased through a maze of blind alleys by a horde of hostile accountants wielding axes and pitchforks.

And these were not small, nagging fears. Ben Rich’s tenure as head of Skunk Works saw the Lockheed and DC bureaucracies creeping in from both sides. At one point, he lobbied Lockheed management to take on a relatively risky production contract for the F-117 sooner rather than later. The idea was only green-lit with much skepticism. Rich writes:

“They’ll want at least one hundred bombers, and we’ll be looking at tens of billions in business. So what’s this risk compared to what we can gain later on? Peanuts.”

It was not a very happy meeting, and the conclusion reached was reluctant and not unanimous. The corporate bean counters insisted we install a fail-safe monitoring and review procedure that would sound the alarm the moment we fell behind or hit any snags. “Above all, no nasty surprises, Ben,” Larry Kitchen warned me.”

The Gross brothers and Hall Hibbard were gone. Kelly Johnson had been a living legend who routinely turned down Lockheed’s CEO position. His freedom and insistence on the same for his Skunk Works was tolerated by subsequent generations of executives throughout his career. But by the end of his career, many in DC and Burbank had little patience for him. They would not be so lenient with his successor.

Ben Rich’s Skunk Works — despite proving itself with its inspired F-117 project design — was being watched. He recounted:

An independent engineering review team, composed entirely of civil servants from Wright Field in Ohio, flew to Burbank to inspect and evaluate our entire program. They had nothing but praise for our effort and progress, but I was extremely put out by their visit. Never before in the entire history of the Skunk Works had we been so closely supervised and directed by the customer. “Why in hell do we have to prove to a government team that we knew what we were doing?” I argued in vain to Jack Twigg, our assigned Air Force program manager. This was an insult to our cherished way of doing things. But all of us sensed that the old Skunk Works valued independence was doomed to become a nostalgic memory of yesteryear, like a dime cup of coffee.”

He was largely right.

Johnson shared an anecdote in his 1985 book, noting just how much the defense procurement world had changed in a few short decades. He noted:

Our Air Corps project officer for the XP-38 was a pilot, a young lieutenant, Benjamin S. Kelsey. He was excellent. In those days (1938) a project officer with that rank had more authority than many four-star generals do today (1985). If we asked Ben for a decision, we got it — on the spot.

Conclusion

Times have surely changed in defense procurement. But the reader’s takeaway should not be that Skunk Works’s management playbook was great, but no longer feasible. It is true that many pools of government capital that used to be risk-tolerant and trusting of individual visions are the opposite nowadays. However, there still exist fantastic funders with a deep belief in bright people with ambitious visions. And many of these funders require only common-sense oversight. With capital from phenomenal funders like these, the Skunk Works management playbook can still prove exceptionally effective. Kelly Johnson’s rules regarding limiting bureaucracy, keeping his team ruthlessly small, and doing whatever possible to allow individual genius to flourish helped create a historically great lab.

The heart of what made the lab exceptional is also strikingly similar to what made BBN and CMU’s autonomous vehicle units so exceptional. All three maintained operating structures that were operated like small companies and were aggressively focused on novelty. The CMU group found a way to make a novelty-focused university group operate in a more firm-like fashion. BBN set up a private firm staffed with MIT professors that could only be rivaled in its novelty focus by the most daring of university labs. In Skunk Works, Kelly Johnson found a way to run a research outfit within a corporate bureaucracy like a small firm. He also insisted that the lab spend close to all of its time building planes that were novel and ambitious. When he stepped down he chose Ben Rich as his successor largely because he knew Ben wouldn’t play it safe with the lab. Without that ambition, the lab would be effective but not ambitious. It wouldn’t be Skunk Works.

Ex-Lockheed CEO Roy Anderson recalls Kelly Johnson saying the following when recommending Rich as successor:

Roy, I raised Ben in my own image. He loves the cutting edge as much as I do, but he knows the value of a buck and he’s as practical as a goddam screwdriver.

To Kelly Johnson, you likely couldn’t run a true Skunk Works without both.

Thanks for reading:)

On the day of Kelly Johnson’s death, Ben Rich and Lockheed ran a full-page ad in the Los Angeles Times. As Ben described the image, “It showed the logo of our Skunk Works skunk with a single tear rolling down his cheek.”

Subscribe now

Share

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Facilitating tool/hardware improvements in a key technology area far from its suspected theoretical frontier.

• Selecting multiple industry contractors for a trial period before choosing the final contractor. (F-117)

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

General Links:

• More Than My Share of It All by Kelly Johnson

• Skunk Works: A Personal Memoir of My Years of Lockheed by Ben Rich and Leo Janos

1

Ben Rich noted that around 100 servicemen may have already been lost on intelligence missions like this to this point.

2

Kelly believed the balloons would have to be about one mile long to achieve this altitude with the present technology

3

This was all done for something like $800,000, which is less than $10 million today.

Today’s piece was put together with the help of several conversations with Answer.AI co-founder Jeremy Howard. It is not a “traditional” FreakTakes piece; the research and advice are much more tailored to a specific group of individuals than usual. However, I was excited to share it on the Substack for those curious to know more about Answer.AI or what the applied side of my FreakTakes work looks like.

Jeremy Howard (former President and Chief Scientist of Kaggle) and Eric Ries (creator of The Lean Startup movement and Long Term Stock Exchange) have teamed up to found a new applied R&D lab: Answer.AI.

When speaking with Jeremy, he made it clear that many details of Answer.AI's structure are still being worked out. Only announced a month ago, the org is still in its early development stages. But the founders have conviction on certain principles. The most prominent of them is one extremely relevant to this Substack: The founders seem to be particularly inspired by Edison’s Menlo Park Lab and the early days of commercial electric research.

Previously, I’ve covered Edison’s Menlo Park Lab, the early GE Research Laboratory, and many other industrial research shops. Since FreakTakes is explicitly written for individuals founding new science orgs, has a particular focus on applied R&D, and has covered the lab that partially inspired Answer.AI, I thought it would be worthwhile to put together a piece specifically for Jeremy and Eric. The goal is to distill into a single piece the takeaways from prior FreakTakes pieces that are most useful to their nascent company. If they take even one nugget of information and use it to run their organization slightly differently, I’ll consider that a win.

In the piece, I’ll briefly examine the (working) plans for the lab and do some historical analysis, detailing:

1. What the earliest electrical R&D labs can teach Answer.AI

2. Useful rules-of-thumb from other historically great applied R&D labs

3. Potential pitfalls to keep in mind as they move forward

You can find more thorough historical evidence in my prior pieces for any of the lab details I mention, listed below:

• Thomas Edison, tinkerer published in Works in Progress

• Tales of Edison's Lab (podcast)

• Irving Langmuir, the General Electric Research Laboratory, and when applications lead to theory

• How did places like Bell Labs know how to ask the right questions?

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

• ILLIAC IV and the Connection Machine

• A Progress Studies History of Early MIT — Part 2: An Industrial Research Powerhouse

• How Karl Compton believed a research department should be run

Each of the orgs listed has lessons to teach Answer.AI. But none are a perfect analog. So, as the piece progresses, I’ll explain which lessons I think most strongly apply to Answer.AI. With that, let’s get into it!

Subscribe now

Share

Edison and “the boys” preparing the First Practical Incandescent Lamp for Testing at Edison’s Menlo Park Lab. Illustration by Harry K. Flemming. Photo Courtesy of the Henry Ford MuseumAnswer.AI in a Nutshell

Jeremy’s blog post announcing Answer.AI makes it clear that the org is, to a large degree, inspired by the field of electricity’s path of progress in the 1800s. He believes the current state of the AI field is similar to the state of the electricity field between the work of Michael Faraday and Edison’s lighting projects. This was an era in which new electrical findings were being pieced together, but few had made any progress in turning the potential of electricity into great applications.

I don’t find this comparison crazy. So far, I don’t believe AI has come close to the level of breakthrough that electricity proved to be. Electricity brought the sunlight indoors for a negligible cost and powers so many of our modern conveniences— refrigeration, TVs, central heating, etc. That’s a high bar. However, given that human ingenuity created the breakthrough that was electricity and each of those applications, it is surely worth considering that AI could grow to be the most impactful field of them all. Whether AI does reach that level of promise, to me, is a question of human ingenuity. So, I have no issue with Jeremy comparing the AI field to the electrical field c. 1830 to 1910.

With that elephant out of the way, let’s briefly examine what sets Answer.AI apart from AI labs like OpenAI and Anthropic. From a funding perspective, Answer.AI seems much, much cheaper. The founders have initially raised $10 million. This stands in stark contrast to the gargantuan initial rounds of OpenAI and Anthropic. Also, Answer.AI's research agenda is more application-centric. The following excerpt from Jeremy’s blog post highlights what he thinks differentiates the lab’s approach:

At Answer.AI we are not working on building AGI. Instead, our interest is in effectively using the models that already exist. Figuring out what practically useful applications can be built on top of the foundation models that already exist is a huge undertaking, and I believe it is receiving insufficient attention.

My view is that the right way to build Answer.AI’s R&D capabilities is by bringing together a very small number of curious, enthusiastic, technically brilliant generalists. Having huge teams of specialists creates an enormous amount of organizational friction and complexity. But with the help of modern AI tools I’ve seen that it’s possible for a single generalist with a strong understanding of the foundations to create effective solutions to challenging problems, using unfamiliar languages, tools, and libraries (indeed I’ve done this myself many times!) I think people will be very surprised to discover what a small team of nimble, creative, open-minded people can accomplish.

At Answer.AI we will be doing genuinely original research into questions such as how to best fine-tune smaller models to make them as practical as possible, and how to reduce the constraints that currently hold back people from using AI more widely. We’re interested in solving things that may be too small for the big labs to care about — but our view is that it’s the collection of these small things matter a great deal in practice.

It would be unfair to say that an application-centric research agenda is necessarily less ambitious than AGI. Those biased toward basic research might say so, but I don’t think that opinion is very historically-informed. Edison himself was application-centric above all else. His deep belief in market signals is fascinating when juxtaposed with the market indifference of many great academic physicists. In the book From Know-How to Nowhere, a history of American learning-by-doing, Elting Morison described the interesting nature of Edison’s motivations:

If the means by which he [Edison] brought off his extraordinary efforts are not wholly clear, neither is the cause for his obsessive labors. No diver into nature's deepest mysteries carrying next to nothing for the advancement of knowledge and even less for the world's goods, he would become absorbed in making something work well enough to make money. The test in the marketplace was for him, apparently, the moment of truth for his experiments.

Edison built his god-like reputation by dreaming in specific applications. He kept market, resource, and manufacturing constraints in mind from the earliest stages of his projects. Edison dreamed practical, realizable dreams. And when the limitations of component technologies stood in the way of his dreams, he often had the talent to invent new components or improve existing materials. Edison’s biggest dream, the light bulb, mandated that  Edison solve a much broader set of problems. The following excerpts from my Works in Progress piece on Edison paint a clear picture of his ambitious but practical dreams:

After Edison’s bulb patent was approved in January 1880, he immediately filed another for a ‘System of Electrical Distribution’. Filing for these so close together was no coincidence. To Edison, it was never just a bulb project. It was a technical business venture on a possibly unprecedented scale. Edison wanted to light up homes all over the world, starting with lower Manhattan.

Bringing the project from dream to mass-market reality would require solving over a hundred technical problems. His was a new bulb that needed to be powered by a generator that did not yet exist at the start of the project, strung up in houses that had no electricity, connected via underground street wiring that was only hypothetical, and hooked up to a power station that had never existed before.

Yet, at the end of two years’ time, Edison would do it. And, just as importantly, the entire venture was profitable by the end of the project’s sixth year.

Edison was clearly doing a different kind of dreaming than those who do basic research. His lighting work embodies what extreme ambition looks like in application-centric research. Answer.AI making this kind of ambitious, applied work their North Star is an extremely interesting goal.

This goal has the potential to give Answer.AI a comparative advantage in the growing space of for-profit AI labs. For example, the most ambitious aspects of OpenAI are considered to be in its research, not its work on applications. Answer.AI’s particular setup can also set it apart from AI startups and academic labs. New AI startups do some research on how to commercialize new AI models in new ways, but they generally have short runways. In this kind of environment, only specific types of research projects can be pursued. Academic labs — for many reasons covered elsewhere on this Substack (such as in the ARPA series) — don't have the right combination of incentives, experience, and staffing to build new technologies in most problem areas. The main incentive of the profession, in a simplified form, is producing many paper studies that get cited many times. Answer.AI has the chance to let its alternative focus lead it to areas under-explored by academics, companies with brief timelines to hit revenue benchmarks, and more AGI-focused R&D labs.

Legally, Answer.AI is a company. But in practice, it might hover somewhere between a lab and a normal “profit-maximizing firm” — as was the case with Edison's lab. The founders seem perfectly content to pursue high-risk projects that might lead to failures or lack of revenue for quite a while. In saying this, I do not mean to imply they are content to light money on fire doing research with no chance of a return. Rather, they hope to fund a body of research projects that ideally have positive ROI in the long term. They are just not overly concerned with short-term revenue creation.

(Making the pursuit of research agendas like this easier is actually one of the founding goals of Ries’ Long Term Stock Exchange — which I address later.)

There is apparently no pressure to produce a product that can hit software VC-style revenue goals within 12-24 months, or anything similar. This is good. Seeking to satisfy these types of metrics does not traditionally permit a company to act like a truly ambitious R&D lab. I’m not saying it can’t happen — DeepMind seems to have made it work in its early years — but it does require pushing against investor pressure quite strongly. The VC money raised for Answer.AI has left the founders with enough voting shares that investors can’t veto founders’ decisions. Additionally, Howard says the company’s investors understand what they are trying to build is, first and foremost, a lab. This is a great step towards building an organization focused on building very useful, very new things rather than the most profitable thing possible — which often comes with bounded technical novelty.

Interestingly, Answer.AI will also keep a small headcount. Jeremy built Fastmail up to one million accounts with only three full-time employees. He hopes to keep the Answer.AI team exceptionally talented and “ruthlessly small” in a similar way; he believes keeping teams small is important to building new, technically complex things.

Now that I've outlined some important pieces of Answer.AI’s vision, I'll dive into the historical analysis. In the first section, I detail lessons that Answer.AI can draw from both Edison’s Menlo Park laboratory and the Early GE Research Laboratory. In the following section, I'll share useful lessons from other historically great industrial R&D labs. Lastly, I’ll highlight the bureaucratic details that explain why the operational models of the great industrial R&D labs have not been replicated often.

Learning from the First Electrical R&D Labs

I find it exciting that Edison’s Menlo Park lab is a North Star for Answer.AI. I covered Edison’s work in several pieces because I think evergreen lessons can be drawn from his work. But I think a more complete way to incorporate lessons from the 1870-1920 electrical space is to draw on the work of both Edison’s Menlo Park Lab and the young GE Research Lab. The latter operated as a more traditional industrial R&D lab. GE Research’s history holds many lessons to help steer Answer.AI’s problem selection and work on its standard projects. However, exceptionally ambitious projects may draw more heavily on the lessons of Edison’s lab.

(As a note, while Edison General Electric was one of the two companies that merged to become GE — along with Thomson-Houston Electric — Edison had essentially nothing to do with the formation of the iconic GE Research Laboratory.)

Different types of projects characterized the work of the two electrical labs. When it came to electrical work, for years, Edison’s lab and mental efforts were focused on doing everything necessary to bring a single, revolutionary product to market. On the other hand, GE Research usually had many separate courses of research underway at once. These projects all sought to improve the science and production of existing lighting systems, but they were otherwise often unrelated to each other. Additionally, GE’s work could be categorized as more traditional “applied research.” The lab was not actively looking to create a field of technology from scratch as Edison did. GE Research's projects were often novel and ambitious, but in a different way than Edison's.

Later, I will explore the types of novelty the GE Research Lab pursued. First, I’ll give the reader a more fine-grained idea of how Edison’s lighting project actually operated.

Lessons from Edison’s Work on Electricity

Edison’s lighting work provides great management lessons for those looking to direct a large chunk of a lab’s efforts toward a single, big idea.

Edison’s major contribution to the field of electricity was not inventing each of the components in his lighting system, but in turning a mass of disparate gadgets, scientific principles, and academic misconceptions into a world-changing system. The burden of doing “night science” — as Francois Jacob refers to it — largely fell on Edison. In the late 1870s, nobody knew much about electricity yet. The existing academic literature had more holes than answers, and many of its so-called “answers” turned out to be wrong or misleading. From this shaky starting point, Edison proceeded. He combined his unique mix of attributes and experience to deliver a world-changing system. These included: knowledge of several adjacent scientific fields, deep knowledge in then-overlooked experimental areas, market knowledge, manufacturing knowledge, and the ability to adequately operate a small research team.

In large part, Edison created his lab as a way to scale himself. As a result, to understand how his lab operated, one needs to know how Edison himself carried out his explorations. Edison was one of the more stubborn experimentalists of all time. He spent most of his waking hours carrying out one experiment or another. While he did pore over scientific literature, for him, nothing was settled until he proved it for himself at the lab bench.

I write in my Works in Progress piece:

Edison respected scientific theory, but he respected experience far more. In Edison’s era of academia as well as today’s, many professors had a certain preference for theory or ‘the literature’ over hands-on improvement. Because of this Edison did not care much for professors. He was even known to go on long diatribes, during which he had assistants open up textbooks, locate scientific statements that he knew to be untrue from experience, and quickly rig up lab demonstrations to disprove them. ‘Professor This or That will controvert [dispute with reasoning] you out of the books, and prove out of the books that it can’t be so, though you have it right in the hollow of your hand and could break his spectacles with it.’

Contained in his head was a database of countless experiments and results that made it seem as if his “intuition” was far beyond his contemporaries. This left him with an unparalleled skillset and body of knowledge. If anyone could feel comfortable pursuing a project that others had previously failed at, it was Edison. Edison’s confidence in his skills was never more on display than when he chose to pursue his lighting work. Many in the scientific establishment knew electric bulb lighting was technically possible, but claimed they had proven that it could never be economical. Edison disagreed.

On top of Edison’s admirable approach to experimentation, he brought a high level of practicality to his process. He knew his inventions needed to make commercial sense in order to make it out of the lab. So, even in early courses of experimentation, he kept factors like manufacturability in mind. He wouldn’t commit much time to something that didn’t make commercial sense. With that being said, Edison wanted to change the world with his technologies more than he wanted to get rich. So, the practical factors he paid aggressive attention to were primarily treated as constraints. He did not optimize for profitability, but he knew his ideas needed to be profitable. Nobody who wanted to optimize for profit would have pursued lighting in the way Edison did. The technical risks were too great.1

Edison was able to imagine an ambitious system that required many technical advances. It was so futuristic that maybe only he was capable of coming up with it. But just as impressively, he was able to do it profitably and on schedule. His dogged commitment to experimentation seems to be largely responsible for this. Edison and “the boys” constantly experimented on every piece of the process to improve and learn more about all the sub-systems in Edison’s grand system. They wanted to know how every piece of every sub-system performed in all conditions. I’ll share just two excerpts from my Works in Progress piece as examples.

The first is from Edmund Morris' biography of Edison. It recounts how thoroughly Edison and his trusted aid, William Batchelor, were in carrying out round after round of filament experiments:

For week after week the two men cut, planed, and carbonized filaments from every fibrous substance they could get — hickory, holly, maple, and rosewood splints; sassafras pith; monkey bast; ginger root; pomegranate peel; fragrant strips of eucalyptus and cinnamon bark; milkweed; palm fronds; spruce; tarred cotton; baywood; cedar; flax; coconut coir; jute boiled in maple syrup; manila hemp twined and papered and soaked in olive oil. Edison rejected more than six thousand specimens of varying integrity, as they all warped or split…

In the dog days, as heat beat down on straw hats and rattan parasols, the idea of bamboo suggested itself to him. Nothing in nature grew straighter and stronger than this pipelike grass, so easy to slice from the culm and to bend, with its silicous epidermis taking the strain of internal compression. It had the additional virtue, ideal for his purpose, of being highly resistant to the voltaic force. When he carbonized a few loops sliced off the outside edge of a fan, they registered 188 ohms cold, and one glowed as bright as 44 candles in vacuo.

This approach went far beyond bulb filaments. The following excerpt describes the work of one of Edison’s lead mechanics in turning the Menlo Park yard into a 1/3 scale model of what they would later install in Lower Manhattan. I write:

[Kruesi, Edison’s mechanic] along with a group of engineers and a team of six diggers, turned the excess land of the lab in Menlo Park, New Jersey…into a one-third-scale model of Edison’s first lighting district in lower Manhattan. This team tested and re-tested the electricity delivery system, digging up Menlo Park’s red clay to lay and re-lay an experimental conduit system. The team carried out countless tests to ensure that they found materials to efficiently carry the electric current while also keeping the delicate materials safe from water and ever-present New York City rats.

The entire process was marked by the classic trial-and-error of the Edisonian process. The first subterranean conducting lines and electrical boxes the group laid were completely ruined by two weeks of rain — despite being coated with coal tar and protected with extra wood. While the diggers dug up the failed attempt so the damage could be examined, Kruesi and a young researcher…studied and tirelessly tested unbelievable numbers of chemical combinations — making full use of the laboratory library and chemical room — until, finally, a blend of ‘refined Trinidad asphaltum boiled in oxidized linseed oil with paraffin and a little beeswax’ was found that protected the electrical current from rain and rats.

Edison built his own style of dogged experimentation into the culture of his lab. Since the lab was meant to scale Edison, this makes perfect sense; he was a man with far more ideas than hands. So, he hired more hands. Edison did not search far and wide to hire the world’s best research minds, and many of those he employed did not even have scientific backgrounds. This didn’t matter much to Edison because most of them were employed to undertake courses of research that he had directed them to pursue. A couple of his Menlo Park employees had advanced scientific degrees, but far more did not. For the most part, the lab and its activities were steered by Edison and his ideas. As a result, the productivity of his lab followed wherever his attention went. After some time working on a project area, Edison would often grow antsy and wish to move on to the next thing — he craved novelty. The lab’s resources and extra hands would move with him. As we’ll see in the next section, this stands in stark contrast to how the GE Research Lab recruited and chose problems.

Menlo Park's electrical activities provide a great management playbook for what it looks like to direct a lab’s efforts toward a single, major system. If Answer.AI does not want to go all-in on one thing, it can still find a way to apply this playbook to a certain focused team of employees while leaving the others to tinker around with exploration-stage ideas. In Edison’s less-focused experimentation periods, his lab served as more of an “invention factory,” doing this sort of fiddling. Additionally, Edison's preference for application and commitment to experimentation over theory in a young area of science can surely provide Answer.AI some inspiration.

Of course, Edison did some things better than others. Edison’s most easily-spottable “deficiency” is that his lab was largely dependent on him. Without him and his big ideas, the lab would have probably ground to a halt. While Edison’s technical vision, practicality, and experimental approach are absolutely worthy of emulation, the lessons of GE Research should probably be added into the mix as well. GE operated as more of a prototypical industrial R&D lab with an approach quite suited to the fact that the science of electricity was beginning to mature in the early 1900s.

Lessons from the Young GE Research Laboratory

The young GE Research lab took a different approach to electricity research than Edison. The lab worked on many unrelated projects at once, recruited more talented researchers, and allowed these talented researchers more freedom to exert the scientific method on commercializable projects. The lab did not undertake projects that were as purposely futuristic as Edison did. Nobody from the lab earned nicknames like “the Wizard of Menlo” or “the Wizard of Recorded Sound.” But early GE Research was still responsible for a Nobel Prize and making the light bulb a much-improved, more cost-effective technology.

Elting Morison wrote the following on the lasting impact of GE Research’s early decades:

There seems little doubt that…much that was done in Schenectady in electrical engineering and some parts of physics was both better done and more interesting than what was being done in those fields in any American university.

In its heyday, even great researchers like Karl Compton hoped to shift their academic departments to operate more like GE Research.

While GE did simultaneously pursue diverse projects, there was a unifying thread holding all of the projects at GE Research together. Each project aimed to improve the quality and profitability of GE’s products and manufacturing. Under that unifying theme, all kinds of projects were encouraged. Much of the research was very applied, particularly in the early years when the lab was still proving itself.

William Coolidge was one of the lab’s most talented applied researchers in its early years. Coolidge joined the lab in 1905, part-time while teaching courses at MIT. Coolidge had the kind of toolkit typical of many MIT professors in that era. He had a far greater grasp of the science of physics and metallurgy than somebody like a blacksmith; he was simultaneously far closer to a blacksmith than one would ever expect a university researcher to be. With this differentiated toolkit, he did science in a way that was not typical of academics. In describing the process that led to his successes at GE, he claimed that he was, “guided in the main by experiment itself rather than by metallurgical knowledge.”

Coolidge expertly applied practical skills in concert with scientific knowledge to pursue the problem. Elting Morison described a small sample of Coolidge’s workflow:

He suspended tungsten powder in an amalgam of bismuth, cadmium, and mercury. He then passed the resulting substance through tiny dies — drawing it — and obtained a silvery pliable wire. At that time, he thought he had reached ductility and the search was over. But when a current was passed through this wire the mercury, cadmium, and bismuth distilled out, leaving, unfortunately, a nonductile tungsten. But it also proved to be tungsten in the purest state he had yet produced.

I continue in my FreakTakes piece, writing:

He eventually iterated his way to a workable process where…the more pure tungsten was put through a specific combination of metal-working processes at a temperature that worked that produced rods of tungsten about 1 mm in diameter. These 1mm rods could then be drawn and re-drawn through rods of decreasing size until you were left with wires of tungsten .01 mm in diameter. When put in the vacuum-sealed bulb, electricity ran through the tungsten filaments and demonstrated an efficiency of 1 watt per candle — extending the life of a bulb up to 27x.

Within 5 years, 85% of all lamps would be made from tungsten. As the project went on, more and more research chemists and technical assistants grew to be involved in the wide-ranging steps and combinations involved in Coolidge’s experiments. But it worked. GE had the factories re-fit and deployed the new bulb. Coolidge moved on to other research.

The success of Coolidge’s hybrid work style, not dissimilar to Edison’s, is surely a useful data point to Answer.AI. But GE Research also did work that went far beyond Coolidge’s technically adept, applied science. The lab was fantastic at making use of talented individuals who were very academic. Irving Langmuir was a prime example. I described his interests in my original piece:

It should be noted…Langmuir did not even care about lightbulbs. Well, I guess that is not technically true. The bulb interested him because 1) he thought a metal like tungsten was cool because it could accept really high temperatures which opened up options to the scientist working with it and 2) these vacuum-sealed bulbs provided a pristine environment for controlled scientific investigations.

To Langmuir, light bulbs were primarily a playground in which to do his science. But Willis Whitney knew how to take an individual like that and direct his energy towards productive ends. The lab deployed a principle that I call extending a “long leash within a narrow fence” to basic researchers like Langmuir.

The way the lab facilitated this was rather simple. On his first day, Langmuir was told to walk around the applied end of the lab and ask people about their projects. Whitney permitted him to undertake any course of investigation of any phenomenon he wanted, but it had to be directly related to an existing problem/limitation/constraint that the applied folks were working through. These applied folks were working on projects that rather directly plugged into GE’s operations, so there was minimal risk of Langmuir’s work not amounting to anything useful if he succeeded and found answers. With that assurance of applicability, Langmuir was given extensive timelines to find answers to open questions.

Langmuir’s first course of research focused on the constant bulb-blackening problem common to bulbs at the time. The problem was generally attributed to a bulb’s imperfect vacuum. Langmuir found this problem to be a great excuse to carry out a course of experimentation he found interesting. Morison described Langmuir’s thought process as follows:

If residual gases — imperfect vacua — produced a bad effect — blackening — here was a fine opportunity to study the effects produced by different gases introduced one by one into the bulb. What he wanted to do, he told Whitney, was simply to plot the interactions of various gases exposed at low pressures to very high temperatures in the filament. Nobody knew very much about this phenomena and he wanted to look into it simply “to satisfy [his] own curiosity.”

Langmuir carried out this course of research over three years. There were many gases and temperatures to test, which took time. But unforeseen results constantly took Langmuir off in different directions. Exploring these unforeseen results often entailed new courses of experiment altogether. With his long leash, Langmuir was able to figure out that imperfect vacua were not what caused bulb blackening at all. Rather, it was that tungsten vapor particles were finding their way onto the wall of the bulb. Temperature was the issue.

He also discovered that different gases markedly changed the rate of evaporation. One extreme example was nitrogen, which reduced the evaporation rate by 100-fold. However, adding nitrogen to the bulbs caused the electrical efficiency of the system to decrease drastically. So, the existing bulb design with nitrogen added was less cost-efficient than the normal bulbs. But Langmuir was undeterred. This was progress.

Existing fundamental research in this area led him to believe that this efficiency issue could be alleviated by increasing the diameter of the filament. Further experimentation proved this to work. He also found that coiling the filament in a certain way could mitigate the heat loss issue. The final result was a novel bulb that used an inert gas instead of a vacuum to reduce bulb blackening. Along with the coiled tungsten filament, this new bulb only required .5 watts per candle and lasted three times longer than any other bulb.

Once he passed the bulb project onto the engineering team at GE Research, Langmuir set his sights on an anomaly he had come across talking with the lab’s more applied staff. The bulbs in the lab had a design that depended on only a few milliamperes of current flowing across the space between one end of the filament and the other. Langmuir noted this anomaly in a letter to Scientific Monthly, writing:

This fact seemed very peculiar to me, for the work of Richardson and others had indicated that at temperatures as high as those used in the tungsten-filament lamp, currents of many amperes should flow across the space. In other words, according to the then-accepted theory of the electron emission from hot filaments, a serious difficulty should have been encountered in the construction of tungsten-filament lamps. The fact that we did not meet any such difficulty therefore seemed to me a peculiar fact that should be investigated.

In the brief course of exploration that followed from Langmuir, he discovered what is now known as the space-charge effect. This work combined with follow-on work from Coolidge to produce an entirely new kind of GE X-ray tube.

Under this “long leash within a narrow fence” guideline, Irving Langmuir would go on to be partially responsible for a handful of new and improved product lines at GE. Additionally, the knowledge he created with his tungsten filament work went far beyond padding GE’s balance sheet. Over the course of his project, he noted that the way tungsten vapor condensed did not gel with existing academic theory. His subsequent exploration of this phenomenon led Langmuir to be credited with founding the field of surface chemistry. Langmuir earned himself a Nobel Prize for his efforts.

There was a symbiosis in the GE lab between Langmuir types and the Coolidge types — the latter skillset being more standard in the lab. I imagine Answer.AI will have no shortage of Coolidge-like individuals: bright, Kaggle Grandmaster-type individuals who understand academic theory but whose specialty is in expertly applying their craft in dirty, practical situations. Someone like Jeremy Howard will likely have great intuition about how to utilize these individuals. The GE playbook — with its “long leash within a narrow fence” principles — can help Answer.AI think through how to deploy basic researchers in its operations

Langmuir’s career at the GE Research Lab provides a clear roadmap for how to optimally leverage a basic researcher’s energies in an applied context. Langmuir getting paid to investigate any anomalies would likely have satisfied his curiosity. However, it was his investigation of the right anomalies that made this a beneficial arrangement for GE Research.

In general, there is a time and place to apply insights from either Edison’s playbook or GE’s. The maturity of a given research field or technology area has a strong hand in dictating which set of principles is more applicable. Edison came first and had to shoulder the burden of developing an extensive technical system to power the “killer app” that was his bulb. GE Research had the benefit of working on an existing technology area with moderately developed science and existing user technology (thanks to Edison), but the technology still needed a lot of work to become reliable and economical.

A lab can simultaneously employ both playbooks. Even most of Edison’s projects were modest in relation to his lighting work. When inventing for existing fields, such as telephony, Edison contained his inventive streak to working within existing technical systems. He knew nobody would rebuild entirely new telephone infrastructure just because the young inventor had rigged up a moderately improved but completely different version. When adding to Bell’s telephone, he simply invented a carbon transmitter that could plug directly into the system. This device made voices come through much clearer. That was it: one gadget that cleanly plugged into the existing system. Technologies like these may not be as earth-shattering as Edison’s lighting system, but they were still enough to make him a world-famous inventor in his own time.

It was about impact. In optimizing impact, I thoroughly suspect Answer.AI to make great use of the playbooks of both of these small industrial research giants.

Learning From Other Historically Great Industrial R&D Labs

I’d now like to highlight applicable lessons from other research operations covered on this Substack. I’ll cover the orgs in no particular order.

Striking the Balance of BBN and CMU’s Autonomous Vehicle Group

FreakTakes recently covered two historically great DARPA contractors who expertly balanced the competing pulls of project novelty and deployable technology. The first was Bolt, Beranek, and Newman (BBN), the contractor primarily responsible for the ARPAnet. The second was Carnegie Mellon’s autonomous vehicle groups.

BBN embodied what it meant to be a “middle ground between academia and the commercial world.” The firm was initially set up by MIT acoustics professors to pursue their contracting work more ambitiously. In its early decades, the firm gradually expanded its contracting efforts into the computing space, initially under the leadership of BBN VP J.C.R. Licklider. BBN soon became a common home for the best researchers in Cambridge, abandoning their academic positions to work for BBN. The firm's growing reputation even earned the monicker the “third university of Cambridge.”

The firm’s revenue was primarily sourced from research contracts given out by orgs like DARPA, research grantmakers, and aerospace firms. BBN’s positioning was somewhat unique; when compared to industry, the firm emphasized novelty and cutting-edge technology work. This insistence on novelty helped the firm recruit individuals who felt a bit too talented to waste away working on derivative projects at Westinghouse. When compared to academia, BBN emphasized working on real technology that people would use in the near term. J.C.R. Licklider is just one prominent example of an individual who left a tenureship at MIT to work on more useful technology down the road at BBN. Leveraging this positioning, the firm was able to recruit the best talent.

BBN also provided its most talented individuals latitude to ply their minds broadly. Many projects at BBN showcased the extreme potential of small teams of talented individuals with broad technical knowledge. Only eight BBNers were primarily responsible for pushing the early ARPAnet into existence. The size of the team was no accident; Frank Heart, the engineering lead of the project, described why he preferred a team of this size in his oral history:

I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

Lockheed Skunk Works legend “Kelly” Johnson also held quite similar beliefs when putting together teams to build experimental aircraft. Particularly in the early stages of novel projects, there is a strong case for keeping things small, with specialists who understand the fields that touch theirs. To me, Jeremy’s belief in small teams seems well-validated by technical history.

BBN demonstrates the ideal case of a research firm that wholly embraces technical novelty. CMU can be thought of as the flipside of that coin: a university that wholly embraced systems-building and used firm-like management practices to do so. The highlight of CMU’s later-1900s systems work was its autonomous vehicle projects. The academic group staffed itself with researchers responsible for technical integration and management-style work to effectively carry out novel technological systems building.

Similar to BBN, CMU’s positioning was differentiated from both industry and academia. This fact became very clear as DARPA’s mid-1980s autonomous vehicle work progressed. CMU was seemingly the only contractor excited about technical novelty and systems integration. Martin Marietta — the defense prime in charge of DARPA’s Autonomous Land Vehicle — obsessed over ways to hit DARPA’s demo benchmarks while using unambitious, dated technologies. Simultaneously, the academic vision research groups cared more about using the camera data to write papers than helping directly contribute to building a functional driving system. CMU was the only contractor involved in the project who truly cared about building a novel, functional system. DARPA eventually recognized this and gave them ample funds to build successive generations of autonomous vehicles. The rest was history.

CMU carried out this work with a management structure that was more firm-like than most academic labs. For example, Chuck Thorpe did project management-style work for the team with firm-like incentives — he was a researcher promoted based on vehicle performance, not his h-index. While this was a firm-like position, the group also had academic-style positions. Its use of graduate students on the projects is one prominent example. Each student on the project had to own a piece of the project that was all their own and could be written up as a thesis.

(I explore how the team mitigated the risks of theses not panning out in my interview with Chuck Thorpe.)

These academic incentives partially enabled the CMU team to continually innovate. Oftentimes these students’ theses perfectly plugged into existing systems, such as a thesis on reducing the processing time of an existing sensor’s data from 15 minutes to 90 seconds. But on the most extreme occasion, in 1988, this incentive structure led a grad student named Dean Pomerleau to successfully train a neural net to steer the vehicle. In that particular case, the requirement to allow each grad student to try something new changed the world.

Answer.AI similarly cares about deep technical novelty and building deployable technology. As such, Answer.AI could benefit from emulating BBN and CMU’s strategies to balance the two. The success of BBN and CMU should hopefully embolden Answer.AI’s founders to trust in the priorities they have set. This balance of goals is uncommon today, but orgs from history have expertly balanced the two to world-changing effect.

With that said, time elapsing without world-changing results might be unnerving. This ambiguity is partially what pushed academia to rely on near-term outcome variables that incentivize the incremental. Fear of wasting time and money is real. The Answer.AI founders would surely like some way to ensure that they are spending theirs on good problems. To deal with that, I think the Bell Labs’ approach to problem selection has a lot to add to the approaches I’ve already covered.

The Bell Labs Approach to Problem Selection

Bell Labs’ management of researchers in its golden era is famous — and it should be. However, Bell researchers were not left to their own devices to pursue whatever they wished, despite what many think. Bell managed their researchers with an approach similar to the “long leash within a narrow fence” approach of GE — which one long-time Bell chemist called “circumscribed freedom.” The most effective tool it used to do this was its corp of excellent systems engineers.

Bell had an expansive product line with massive scale — even more so than GE. Even modest improvements from the research team could have outsized returns. This, of course, is not the case with Answer.AI. However, Bell’s use of systems engineers can still be extremely instructive to Answer.AI, even if Answer.AI may deploy them in different ways.

As I covered extensively on FreakTakes, Bell’s systems engineers often combined several knowledge bases to expose the right researchers to the problems that most needed solving. Within one mind systems engineers often combined STEM backgrounds, knowledge of the nitty-gritty details of Bell’s manufacturing, an understanding of Bell’s implementation problems, detailed knowledge of Ma Bell’s expenses, and familiarity with the researchers at Bell Labs.

Bell knew these systems engineers were a massive part of their secret sauce, ensuring Labs deployed its limited resources on the right kinds of problems with sufficient upside. I wrote in the conclusion of my Bell Labs piece:

Finding a problem in these systems is not so hard for those familiar with the systems. That’s why many researchers and engineers do not feel the need to bring in help. But finding a set of good problems is not finding the best problems. Finding the best problems is a profession in and of itself. A systems engineer is worth it when, under the right scrutiny, it might turn out that the best problem is 10X as financially valuable, does 50X the social good, or is 2X as likely to work as just some run-of-the-mill good problem.

My prior piece delves deeper into the specific problems towards which Bell’s systems engineers led Bell’s researchers. For now, suffice it to say that I think that any new applied science org that can dedicate an (ideally full-time) individual to doing the work of a systems engineer should strongly consider it.

Of course, these systems engineers would need somewhat clear marching orders on what sorts of technologies they should be exploring. Nowadays, many existing roles in industry and academia train individuals to equate revenue or potential citation count with impact. Answer.AI will not be satisfied with these metrics as proxies of impact, and they shouldn’t be. What to direct these systems engineers toward instead should surely be up to Jeremy and Eric.

As a firm that pursued novelty and was not attached to a large industrial operation, BBN might be an interesting source of inspiration. Several of its hallmark projects started with systems-engineer-style contributions. Three examples are DARPA PM Larry Roberts putting out the ARPAnet contract, J.C.R. Licklider’s visionary Libraries of the Future Project, and BBNer Jordan Baruch’s early-1960s pitch to the NIH on a system to build a computer system to facilitate modern hospital operations. All three project initiators had Bell systems-engineer-like exposure to the people and problems of their field — technical backgrounds, regularly spoke with the best academic researchers, knew modern industry’s issues, were able to project the costs and complications of potential projects, etc.

Few have used systems engineers as effectively as Bell Labs, with Bell-style goals, since the great lab was broken up. It would be amazing to see a lab like Answer.AI commit significant staff time to this purpose.

The Cautionary Tale of Thinking Machines Corporation’s Funding

The case of the Thinking Machines Corporation (TMC) is not as directly instructive as the examples above, but TMC made one key mistake that makes it worth mentioning. TMC put itself in the unfortunate position of raising some of its funds from investors whose incentives were not aligned with theirs.

For those who don’t know, TMC was a complete failure as a commercial firm. As a result of its bankruptcy, many write the firm off as a holistic failure. But the firm did accomplish many of the technical goals it set out to achieve. Since the company was conceived with technological goals in mind, rather than a specific market, this was no small feat. The company was founded by Danny Hillis, a PhD from Marvin Minsky’s lab at MIT. Through TMC, he sought to build the machine he conceptualized in his graduate thesis: a truly parallel computer to improve the capacity of all scientists. The young company recruited the best researchers — including scoring Richard Feynman as its “intern” for several summers — and achieved many technical goals that helped pave the way for the modern field of parallel computing. Jeremy, who knows far more about the technical aspects of parallel computing than I do, sang TMC’s technical praises in our first conversation. He emphasized how shocking it was that TMC seemed to be the first to employ so many methods the field still uses today.

However, TMC’s high-powered team and great technical work were not enough to overcome their management follies. The firm spent money as if its financial standing was in line with its technical reputation, which it was not. In retrospect, some of these management decisions — such as a comically expensive long-term lease — could have been avoided without modifying the company’s general approach. However, on a deeper level, there was a dissonance between the company’s technical goals and the funding it raised.

TMC’s two major funders were DARPA’s computing office and private investors. The goals of Hillis and his top-flight technical staff were only aligned with DARPA’s goals. In looking to build the technically most ambitious parallel computer possible, DARPA funding was ideal. The DARPA computing office also felt that TMC’s work was progressing exceptionally well for most of its early years. However, the level of enthusiasm Hillis and the technical staff had for building a machine for science did not bleed into the most profitable areas — like deploying the machine on banking databases or managing logistics for Walmart. As time wore on, it seems that pressures were beginning to mount for TMC to pursue work more in line with those areas. It seems highly likely that TMC would have had to disappoint one of its funders sooner or later, even if it spent funds more wisely in its early years.

Had TMC just raised funds from DARPA and spent them much more modestly, the company might still exist today; it may have even earned a reputation beyond that of NVIDIA.

Copying mid-20th C. Industrial R&D Models is Hard for Incumbents

The middle 20th Century saw both the rise and fall of ambitious American industrial R&D labs. It is not just nostalgia that makes modern researchers look back on these labs with fondness. Not long after these labs were formed, it was becoming clear to many top researchers that the model was special.

In 1927, Karl Compton wrote a prescient letter to Science that praised these labs. At the time of writing the letter, Compton was the head of Princeton’s Physics Department and a part-time GE contractor. His letter asserts that these organizations were doing some of the best science in the country, even though the top universities often had the best men. A portion of the letter, in which Compton praises the labs’ management of scientific projects, reads:

This has been strikingly demonstrated in some of the big industrial research laboratories, from which the output has greatly exceeded the individual capacities of the research workers.

Compton believed there were many lessons that university departments should steal from these exceptional industrial labs. The first was the need to specialize when building a portfolio of researchers and projects. Why should every department attempt to loosely approximate the makeup of researchers and research questions in the field as a whole? No company would ever do such a thing. He believed that, “Such a policy dissipates effort, and if every institution followed it we should have the spectacle of a great many universities all very much alike and all with struggling, mediocre departments.”

On an organizational level, Compton believed the equilibrium of N autonomous professors with N separate budgets and a few grad students in their lab under their control was just not efficient for most projects. It would be silly for all departments to function that way. In the letter, Compton proposes what I’ve taken to calling a “Compton Model” research department. This model is far more structured than a department of mostly autonomous professors doing ad-hoc research with their own separate funds. Compton describes it as follows:

There is another direction in which more effective organization is possible within the universities themselves! Departments of a somewhat more flexible nature than those to which we are accustomed and which could, more than now, be built around one or two outstanding men in the department, could give these men an opportunity for organization and concentration of effort which is now rarely possible.

Another way to describe this model is as a “fund department heads, not projects” model. The model allows one or two individuals to largely shape the research vision, hiring, project selection, capital purchasing, etc. of an entire department at once. In addition, it allows these individuals the latitude to replace salaries spent on additional professors or grad students with full-time engineers or discretionary capital expenses as needed.

The model makes perfect sense. However, it’s remarkably difficult to make happen at a real university. Compton couldn’t succeed in doing so when he took over as MIT President. The existing stakeholders and structures are just too hard to shift in this direction. Absent some special circumstance, most university administrators wouldn’t find the idea even worth considering. However, CMU’s President Cyert was able to build a department that loosely resembled a Compton Model department. And its results were exceptional! But this was the exception. Pivoting an existing department to run like a Compton Model department has proven infeasible in almost all cases.2

The great, old models of managing labs didn’t disappear from industry because they lacked scientific merit. These labs largely began to disappear in the 1970s and 1980s. The 1970s saw a deep recession, which usually hurt R&D budgets. In the 1980s, new corporate management trends surfaced that led to companies being managed more myopically than ever before or since. These, along with other non-scientific trends, were largely responsible for the labs going away when they did. With them, the operational know-how that had slowly been built up throughout the century dissipated. Now, to learn how these orgs operated, one must read oral histories or talk to now-retired engineers.

The first thing these retired engineers will often tell you is that we should bring the old models back! Bell’s long-time researchers had confidence in the Bell model even as a late-1900s court case ended Bell Labs as we know it. In the 1920s, Compton dreamt of copying much of the GE model to use within a university. One should not be afraid to put their time and resources behind bringing these models back. Great engineers largely maintained confidence in these models from their inception until they faded into the background.

Many in academia know it should operate more like BBN, CMU, or GE research. They just can’t change the structures to make it happen. Many R&D leaders know that their company should think more in 10-20-year time horizons when planning research expenditures, but shareholders and shareholder-wary CEOs often do not find this view actionable. Answer.AI can be free from all this. The org can apply the great old models in a new era on new technology.

(Ries being a cofounder of Answer.AI should be comforting. Ries has railed against the types of myopic management trends that make activities like maintaining an expansive industrial R&D lab difficult for firms. He founded the young Long Term Stock Exchange, in part, to help mitigate issues like this.)

Conclusion: Running Your Own Races

It’s up to Answer.AI’s two-headed management team to stick to the organization’s comparative advantages. There is no need to race academic CS researchers or corporate R&D departments in races those two groups feel incentivized to run. Corporate R&D, NSF-funded CS researchers, and AGI-focused labs all have areas in which they clearly will and won’t operate. And those areas don’t come close to covering 100% of the good ideas somebody should obviously be working on. Answer.AI is free to run its own races, uncontested.

Many will feel the org is using an untested model. However, Answer.AI’s founders — like me — believe that this model is proven, but has simply gone away for a while. So, shouldering this “organizational risk” that others are seemingly not willing to do, they have the chance to work on problems without much competition. If their $10 million experiment works, it has the chance to spark a rush of emboldened researchers and engineers to found small research firms, leveraging the models of the once-great dragons of American industrial R&D.

I wish Jeremy and Eric luck in the early stages of their mission. To any researchers and engineers — across all areas — reading this and wishing a BBN, CMU, TMC, or Answer.AI existed in your area, please reach out to me on Twitter. I’d love to see if there’s anything I can do to help.

Thanks for reading:)

Subscribe now

Share

1

Of course, Edison’s lab also worked on more modest projects and often helped make improvements in related manufacturing operations. But the labs lighting work is the reason the lab earned its name in the history books to the extreme degree it did.

2

Even in the case of the ILLIAC IV — where the firm-like ILLIAC project was somewhat separate from the other professors at UIUC and received its own resources — the non-ILLIAC professors felt the project should be moved outside of their department.

The centerpiece of today’s post is an extensive interview with Chuck Thorpe. Thorpe, now President of Clarkson University, spent over two decades at Carnegie Mellon University. These years were largely spent as a student, project manager, and PI working on Carnegie Mellon’s autonomous vehicle vision research. The primary goal of the interview was to better understand how he and others managed systems contracts at CMU — CMU had a strong comparative advantage in this style of work while Thorpe was there. Throughout the interview, he goes on to paint a broader picture of how the CMU computer science department functioned differently than others at the time.

As a preface to the interview, I wrote the following ‘Introduction’ section to highlight why CMU is a singularly important department in autonomous vehicle history. I leave the reader with three options as to how to consume the rest of the piece after reading the introduction. For the bookworms, I have provided an abridged transcript of my meeting with Chuck Thorpe — which picks up after the introduction. For the audiophiles, the full audio of the interview is available — both on Substack and on Spotify. And for the YouTube afficionados, I share a YouTube link of the Zoom interview.

Both images courtesy of a 2001 CMU News article on Thorpe

Subscribe now

Share

Introduction

The first piece in the all-time great DARPA contractors trilogy explored the importance of BBN’s unique structure, embodying the “middle ground between academia and the commercial world.” BBN did so well in this role that it earned itself the monicker “Third Great University of Cambridge.” BBN, as a firm, demonstrated what was possible if a firm staffed itself with the very best engineering researchers and cared deeply about technical novelty more than profit maximizing. The flipside of the coin is CMU and its AI work in the later-1900s. Much of CMU’s AI work in this period answers the question, “What if an academic engineering department made its comparative advantage novel systems contracts?” CMU was an academic department eagerly embracing a style of work more common to the defense primes in the DARPA ecosystem.

In DARPA’s Autonomous Land Vehicle project — covered in the ALV section of the ARPA playbook — CMU stood out as the lone contractor involved who was excited to take on both the burden of novelty and the burden of systems integration work. The academics and the project’s main commercial firm had very different incentives when it came to these issues. As I wrote in the earlier ALV piece:

The contractors who found themselves working on the reasoning system vs. the vision system had very different interests and incentives. Most of the reasoning contractors were commercial firms and had the incentive to fight for a broader scope of work and more tasks getting assigned to the reasoning system because that meant better financial returns to the company. Most of the vision contractors, such as the University of Maryland, were academics and largely content to let that happen…if it meant they got to focus primarily on specific vision sub-tasks that were more in line with the size and scope of projects academics often undertook. For example, the private companies felt that building up much of the systems mapping capabilities, knowledge base, and systems-level work should fall to them.

CMU and one of its project leaders — Chuck Thorpe — seemed to be the strong voice of dissent on many of these matters from the academic community. Ever since CMU had become a major contractor in DARPA’s Image Understanding work, it had taken a rather systems-level approach to attacking vision problems — working with civil and mechanical engineering professors like Red Whittaker to implement their vision work in robotic vehicles — rather than primarily focusing on the component level. The CMU team far preferred this approach to things like paper studies — in spite of the increased difficulty.

As the ALV project wore on, Martin Marietta — the lead systems contractor for the ALV — gradually succumbed to its ‘demo or die’ pressure. As the frequency of demos increased, Martin became much more concerned with technically hitting the benchmarks DARPA specified than pushing forward truly ambitious technological changes. Additionally, many of the academics focusing on vision research concerned themselves more with using their DARPA research funds to improve their performance in somewhat simplified environments that helped them write research papers than to do research that would directly plug into the ALV. With this equilibrium becoming clear, Ronald Ohlander — a former student of Raj Reddy at CMU and the DARPA PM responsible for funding vision research — designated CMU as the systems integrator for his DARPA SCVision portfolio. The Martin Marietta team clearly did not care enough about integrating and testing cutting-edge algorithms. So, he tapped CMU and gave them the funds to properly integrate insights from the research ecosystem.

In the prior piece, I described the situation as follows:

CMU had already, unlike most of the component contractors in the SCVision portfolio, been testing its component technology on its own autonomous vehicle: the Terregator. CMU’s used the Terregator — for often slow and halting runs — to test the algorithms developed for its road-following work. The Terregator proved extremely useful to this research, but its use had also helped the CMU team learn enough to outgrow the machine.

In CMU’s ongoing efforts to implement sensor information from a video camera, a laser scanner, and sonar into one workable vehicle, the CMU researchers had come to realize that they needed funding for a new vehicle. CMU needed a vehicle that could not just accommodate the growing amount of sensor equipment, but also one that could carry out its tests with computers and grad students either on board or able to travel alongside the vehicle. This would allow bugs to be fixed and iterations between ideas to happen much faster. The vehicle that the CMU computer vision and robotics community was casting about trying to get funding for would come to be known as the NavLab. Fortunately, what they were seeking money for seemed to be exactly what Ohlander felt the SCVision portfolio needed. The funding for the NavLab began around early 1986. The funding set aside for the first two vehicles was $1.2 million, and it was estimated that any additional NavLab vehicles would cost around $265,000.

As Martin was somewhat frantically taken up with its demo schedule, CMU, with its longer time windows, was free to focus on untested technology as well as technology that ran quite slowly but seemed promising in terms of building more accurate models of its environment. As the CMU people saw it, if anything proved promising they could upgrade the computing machine used to operate that piece of the system later on…the vision researchers likely figured that a piece of hardware powerful enough would likely exist soon to help push the idea along — even if not in the next year. Lacking the hard metrics of yearly demos with speed requirements loosely matching how fast the vehicle would need to go in the field, the CMU team could afford to think more along the lines of “how do we build a machine that is as accurate as possible, even if it moves really slowly today?”

As CMU took up the mantle of building SCVision’s true test bed vehicles, it also took over the role of SCVision portfolio’s true “integrator.”

Things continued like this for a while. Then, after a successful ALV demo in 1987, Martin’s role as the prime DARPA autonomous vehicle contractor suddenly ended. This abrupt shift happened after a panel of DARPA officials and technology base researchers met to discuss phase two of the program. Roland and Shiman wrote in their book on this era of DARPA computing:

Takeo Kanade of CMU, while lauding Martin’s efforts, criticized the program as “too much demo-driven.” The demonstration requirements were independent of the actual state-of-the-art in the technology base, he argued. “Instead of integrating the technologies developed in the SC tech base, a large portion of Martin Marietta’s effort is spent ‘shopping’ for existing techniques which can be put together just for the sake of a demonstration.” Based on the recommendations of the panel, DARPA quietly abandoned the milestones and ended the ALV’s development program.

The CMU researchers and the NavLab — which were remarkably cheap compared to Martin Marietta — continued to be funded. In this way, CMU took its place as DARPA’s top systems integration contractor for driverless vehicles. And it would stay on top of the heap for several decades.

The CMU driverless vehicles were responsible for many — some might even say most — of the practical driverless vehicle breakthroughs in the field's early years. Much of CMU's earliest AI work on these vehicles revolved around the use of things like radar and laser sensors, path planning, and finding ways to integrate inputs from different models into a single set of instructions. But, in the late 1980s, CMU's work abruptly became quite recognizable to the modern eye. In 1988, then-CMU grad student Dean Pomerleau successfully integrated the first neural net-based steering system into a vehicle — his ALVINN system. In 1995, Dean and fellow grad student Todd Jochem drove the successor to Dean’s ALVINN — the more complex RALPH — across the country on their “No Hands Across America” tour. The RALPH system employed neural nets but also put much more effort into model building and sensor processing. Holding Dean and Todd, the upgraded NavLab was able to autonomously drive 98.5% of the way on its successful cross-country journey. In a sense, much of CMU’s work culminated in the CMU vehicles' performances in the second DARPA Grand Challenge, held in 2005. Many only remember that Stanford's team won this landmark challenge. But CMU's two vehicles finished a close second and third to Stanford’s vehicle.

At the helm of Stanford’s team was Sebastian Thrun. Thrun had, the year before, left his CMU research position — which he had held for almost ten years — before joining Stanford and leading their team. In the early years of the field, an astonishing proportion of the best researchers were either trained at CMU or were long-time professors there. In a 2010 oral history, Thorpe described Sebastian Thrun and many of the early Google driverless car team’s CMU ties, saying:

Now, it’s the Google group, and we’re very grateful to Google, but we think of it as the Carnegie Mellon West group. Sebastian, of course, used to be here, and then went to Stanford and Google. Chris [Urmson] is a Carnegie Mellon person on leave. James Kuffner is a Carnegie Mellon faculty member on leave. If you go down through the list, they – everybody except for one on that group either is a Carnegie Mellon person or has a Carnegie Mellon heritage, so that’s our boys. [laughs] And we’re delighted that they’re off with Google, and Google’s given them the researchers to do really cool stuff.

Carnegie Mellon has its fingerprints all over the breakthroughs and researchers from the early decades of autonomous vehicles. It would be hard to argue that CMU’s efforts didn’t move forward the timelines of this field by some number of years. It also seems clear, looking at the history, that the incentives and management structures CMU used to manage projects like this played a major role in setting the university apart.

The university clearly had a comparative advantage in this style of work. So, I decided to ask Chuck Thorpe some questions about how exactly CMU did what it did.

Dean Pomerleau and Todd Jochem, ready to set off on their “No Hands Across America” tour. Photo courtesy of CMU News article.Chuck Thorpe Interview

In the following transcript, I made some cuts to make reading the text version of the interview more efficient. The audio and video files contain the interview in its entirety — which is about 15% longer.

Spotify podcast link

Finding his way to CMU and the early years of the Robotics Institute

Thorpe opened the interview by detailing how exactly he found his way to CMU as a grad student and how CMU’s approach to computing research set it apart from many other universities at the time.

Thorpe: So, I showed up wanting to work on robots. They gave me a book and the robots were all these robot arms and I thought, “Well, that's not what I want to do.” I want to do mobile robots. The reason I came to Carnegie is I was backpacking with my professors from North Park.

One of them said, “What do you want to do?” I said, “Well, artificial intelligence.” “Oh. You ought to go to Carnegie Mellon. Herb Simon just won the Nobel Prize for inventing AI.” I'll tell you the truth. I had to look up Pittsburgh on a map. I had to go to the library and find out what Carnegie Mellon was. And Carnegie Mellon in the seventies was reinventing itself.

It was a very good Western Pennsylvania engineering school that looked at the demographics of Western Pennsylvania and said, “We're going to be a very small, very good Western Pennsylvania engineering school,” and decided that they needed to go national. Now “national” meant that in 1972, once a year, they would go to Chicago to try to recruit students. And every other year they would go to DC.

So, Dick Cyert, the president, bet his presidency on turning Carnegie into a national brand and really expanded it and grew it. He also, Cyert, was a professor of business and his doctrine was comparative advantage. Find out where you can be one of the top two or three in the world and go for it — and don't try to compete broadly.

And a mid-sized university like Carnegie, that makes a lot of sense. So Raj Reddy was talking with Cyert and convinced Cyert that it was his idea, Cyert's, not Raj's, that we could be top two or three in robotics. And Raj said, “But I'm not interested in playing catch-up small ball. If we're going to do it, let's go big.”

So Cyert asked Raj to run the Institute. He got a million-dollar grant from the Navy and he got a million bucks from Westinghouse. And that was the origins of the Robotics Institute.

Gilliam: And how important was it for him to get a big industry contract and a big public sector contract? Was that specifically the goal? To make sure they had one from each?

Thorpe: Specifically the goal was to do both, really crack into ONR (Office of Naval Research) and then into DARPA because they had a lot of money to do very interesting things. But, also, Carnegie has these deep roots in Western Pennsylvania. And if we could work with Western Pennsylvania industry and really help Westinghouse succeed that was a win all the way around.

This was at a time when the steel industry was visibly dying and trying to figure out what was going to save Western Pennsylvania. They thought it was Westinghouse. It turned out that was not completely the story. Westinghouse had its own financial problems. But that notion of trying to see what we could do locally, what we could do with industry, as well as tapping into the US Government was a big deal.

I, then, interjected to provide some historical context and ask a question.

Gilliam: It's interesting to hear you talk about the comparative advantage, because even Karl Compton, who ran the Princeton physics department, was MIT President for a while…he, at one point, wrote a letter in Science in the 1940s. Essentially, he was trying to be polite as he wrote it, but he said (roughly), “We see the talent going into some of the industrial R& D labs from the physics departments. We feel like, to the man, we have higher average talent, but we are not really touching what they're outputting.”

He thought that the firm structure and the concept of comparative advantage, organizing teams in a way that made sense, is what departments should do. It's what made sense. The concept of 20 different budgets [one for each professor] and 20 different fiefdoms [lab groups] to do small research was silly. It made no sense. It's for pure theoretician-type people. But he later became an MIT president and could not succeed at that.

But it seems like Carnegie in running a lot of this work — it seems like Alan Newell was calling it project style work, the Raj Reddy style of work — it seems like Carnegie succeeded in a lot of that ethos. Would you say that's fair?

Thorpe: So, a lot of this starts with Herb Simon saying, “What can we do really well?”

Herb was a founder of the business school, was a founder of the computer science department, was a founder of cognitive science (the psychology department). Herb then brought in Newell. And the two of them saying, “How do people think and how can we encode that in a computer?”…that sort of set the framework for so much of what has made Carnegie successful.

That whole AI and cognitive psych paradigm…if you look at the growth of the computer science department and then school, there's very good theoretical computer science, but the rest of it is all “AI+”. Yeah, AI + linguistics and you get machine translation. AI + learning and you get computational learning. AI + human interaction and now you get the Human Computer Interaction Center. AI + mechanical engineering and you get robotics.

So, it was the combination of leadership from people like Cyert saying, “I'm going to invest a little bit of money here.” And the intellectual leadership of Simon and Newell saying, “This is going to be a growth area.” Simon and Newell, in turn, hired Raj Reddy, poached him from Stanford. That group of them hired [Takeo] Kanade, poached him from Kyoto. And that sort of set us off and running.

The importance of Raj Reddy, what it means to be ‘project-oriented’, and how CMU utilized its grad students

Gilliam: Could I read you an excerpt from Alan Newell's oral history where he talks about Carnegie Mellon's project-oriented style and Raj Reddy?

Because he uses some terms that I'm sure you're going to hear and you're going to go, “I know exactly what he means!” But to me…I knew what I thought it meant — or maybe it's what I wanted it to mean — so I'd love to run it by you. So I'll read you two remarks and then I'll lead you in with the question.

Alright, so the first remark is:

NORBERG [Interviewer] : You had the money along with Green and Perlis [talking about an extra injection of DARPA funds]. What did you people decide to do with it?

NEWELL: I don't know... support people... just spend it. We didn't decide to do anything with it. One of the features of this environment was that it was decidedly un-entrepreneurial. That seems, in one respect, like a contradiction in terms, but we never took these funds and decided we were going to go out and do big things with these funds. That was an attitude typified the operation around here up until Raj [Reddy] shows up.

And remark two is in reference to the 1970s with DARPA. There were a lot of changing political tides, as you’re aware, and they’re talking about the changing goals of DARPA when DARPA is giving out funds. Newell says:

Newell: ...And so, therefore, we have been pushed towards being applied. A response I have, which is not an attempt to change that, that is exactly what's happened. CMU has gone along with that in spades. We started going along with that in the 1970s. I can remember having conversations with Howard Lackler as we were sitting there under the fixed funds with inflation and still trying to get an extra PDP-10. We were allowing that environment to shape us up so that we were not in fact anything like? And we became project-oriented all in that period. We were not project-oriented in the 1960s at all.

…
Newell:..The corresponding thing is the cats that want to play that game -- the Raj Reddys, the Canatis, and so forth, the Coons -- [find] the game they play is how to do that and do basic science at the same time within the same budget.

So I guess my initial question with that is, as Newell’s talking about ‘project-oriented,’ what does it mean to be project-oriented? How does that differ from business-as-usual AI research on a DARPA grant, the way you might see in a Marvin Minsky lab or something like that?

Thorpe: Oh, so Minsky is a very interesting contrast. Minsky wrote this book, The Society of Mind, where you could read…it was, I don't know, 50 two-page chapters. And you could read it in any order and made just as much sense in any order as in any other order.

That's kind of the MIT notion [presumably Minsky’s MIT AI group] of how ‘mind’ works, and their notion of how research works. It’s that you've got 50 independent projects and, together, something may emerge from that.

Carnegie shows up with people like Raj, who says, “I want to conquer speech. I'm going to put together a big team, and these people are going to be working on the phoneme level, and these people are going to be working on the lexeme level, and these people are going to be working on the syllable level, and these people are going to be working on the word level, and these people are going to be working on the sentence level. And there's going to be an architecture which connects them up and down. And we're going to have to have specialized hardware for all of this to work on.”

And so, Raj says, “Let's build this, this project.” And it was a project that had 16 different dimensions, all of which came together. Now, Raj was also flexible enough to say…when somebody showed up and said, “You know, I've got this idea for a completely different method. Let's not bother with all of the structure. Let's just have a little system where everything sort of learns and it figures out all of those mechanisms.” So that was Hearsay (demo), and that was Harpy (demo), and that was the two different systems.

That competition between big structured AI and machine learning, data-intensive AI has been going on at Carnegie for the last 50 years.

Gilliam: To get very in the weeds here…you talk about putting together a big team. And there's a lot of grad students involved in that. And it seems like on the one hand, ‘project-oriented’…it almost sounds like more of a firm structure than an academic team. A lot of the grad students are the employees. But there seems to be some kind of balance where the grad students are the employees — for example, with the ALV or NavLab where you had “on board grad students” — but also the grad students need a thesis.

With a thesis like yours [working to improve the operating speed of a sensor already in use on a test vehicle]…I could see how your thesis would have folded into a big team with that project's goals quite well. Because you were working on a specific sensor technology, and with Blackboard you could fold all that in. But then there's somebody like Dean Pomerleau [whose 1988 ALVINN system implemented neural net steering on CMU’s vehicle] on the other end of things. It's unclear how his project can mesh with everything.

Can you talk about how those management decisions are made? Because I guess the big goal of what I'm trying to do is to help people understand what the CMU approach is on a management level. So this would be great.

Thorpe: So let me back up and say a couple more things about the big CMU picture, and then I'll talk about Dean and how we organized the NavLab, in particular.

The big CMU picture was fundamentally changed when DARPA sent a telegram —this tells you how long ago it was — saying, “We've got a million bucks to spend. If you can send us a 350-word proposal by the end of the week, we'll send you a million bucks.” (~$5-$10 million today)

And that was the grant in basic research for computer science. And that grant lasted for, I don't know, 15 years or something like that. And even after that grant disappeared, that set the ethos of the computer science department. There's this one big chunk of money that supports all of the graduate students. Therefore, the graduate students are jointly sponsored by the entire department.

Do you know about Black Fridays?

Gilliam: No.

Thorpe: Twice a year, the entire computer science faculty sits down and talks about every graduate student. And the student has written a letter. And they [the faculty] read the letter. And they turn to the advisor and say, “What has your student been doing?” And if the advisor says, “Well, the student is making good progress and they did this and this and this,” and it matches what the letter has said, good. Then the student gets a letter saying satisfactory progress.

If the faculty member says, “You know, I've been trying to work with this guy and he's just so stubborn, he won't do what I tell him to do,” then the faculty write a letter saying, “You must, by next Black Friday, do this and this and this.”

If the student is co-advised and the one advisor says, “Well, I thought that Eric was working on it,” and Eric says, “Oh, I thought Chuck was working on it.” Now the faculty turn on the advisors and say, “You bozos, you have to get your act together and do a better job of advising this guy!” But it prevents one student from being abused by their advisor. If you're not happy with your advisor, because you're sponsored by the department you can change your advisors. But it also prevents one soft-touch advisor from saying, “Aw, poor student, he had a rough semester.” No, the entire department gets together and writes something. And there are these very carefully crafted letters of satisfactory progress, unsatisfactory progress, the dreaded ‘n minus one’ letter, and then being kicked out of the department.

So, the ethos is that we're all in this together; that we've got some joint resources; that the graduate students are joint resources; that we need to properly take care of these students and properly supervise them. So that gives the graduate student a large amount of flexibility. But this responsibility to not just their advisor, but to the entire department.

So that's great because students can find interesting projects. There is the marriage process where you walk in day one and each advisor says, “Here's what I'm up to. Here's how many students I'm looking for.” And then the students get to pick who they would like to work for. And the marriage process matches advisors who have interesting projects with students who are interested in working on them…

That's very different from almost any department that I've seen any place. Almost any place you're an indentured servant to the faculty member who happens to have graduate support. And if you don't get along with your advisor, you have the burden of seeing if there's somebody else who has money who will support you and seeing if you can switch advisors.

So that has set this attitude that we're all in this together. We'll have individual little projects, but we'll also have some of these big projects — like the speech projects — that a lot of people will work on.

Newell himself, at his height, had both his human-computer interaction/user interface stuff, but also his SOAR project — which was a pretty good-sized project with a number of graduate students working on it — trying to figure out how they could all fit together.

Newell said that, “You can do science in lots of different ways. One of the ways you can do it is just by doing something that's an order of magnitude bigger, or an order of magnitude faster. If you can process ten times the number of rules, then you're in a fundamentally different space, and you're doing something different, and the science is different. If you can run your computer ten times as fast, then the nature of the problems that you're dealing with changes in ways that you can't even predict.”

So, doing something 10 times bigger is great. So, he was very supportive of that kind of an attitude.

The Origins of the NavLab

Thorpe: Now, how did we get going on the NavLab? So I was sponsored on the original ONR project to do cool stuff. And one of the cool things that we did was to build Takeo’s direct drive robot arm. Another thing we did was to build this stereo vision for navigating across a room.

Side notion. We were sitting there working one day — graduate students, T-shirts, empty pizza boxes. And we heard the door open and we looked over and there was Raj and President Cyert arguing over who could open the door. And in walks, this Navy admiral in full dress whites with all of the…Nobody had told us the head of ONR was coming to visit…

Larry Matthews, who's now Head of Computer Vision at JPL, Hans Moravec, myself…with robots and pieces. And in walks the head of ONR to see what he's been spending money on. Like, “Guys, if you told us we would have at least cleaned up the lab and probably had a demo ready to run.”

Gilliam: Or put on your good T-shirt.

Thorpe: And put on my good T-shirt! But he was impressed. And many years later I ran into him and he remembered that visit. And so he was happy to see robots.

When I was finishing my thesis, we had a meeting and Takeo asked each of us, “What do you want to do next?” I said, you know, we've been doing a good job indoors. I want to go outdoors and really build driving cars. And when we had gone around and said what we wanted to do next, Takeo came back and said, “Good, because DARPA has this money for Strategic Computing. They're going to be building the world's best computers, and then they need ways of showing it off.”

And, being DARPA, they have to have a way for the Air Force, and a way for the Army, and a way for the Navy… (Thorpe then goes on to explain the Battle Management and Pilot’s Associate systems covered in a prior piece.)…And the Army wants to build a robot scout. “So Chuck, you're in luck because what you want to do next is exactly what DARPA is going to have money for. So let's write a proposal for it.”

We had just hired a guy named Duane Adams. Duane had been Deputy Director of one of the DARPA offices — IPTO. And he came to be the Associate Director of Robotics and to help us write proposals. So, Duane said, “Chuck, start writing.” And we wrote a proposal for what it would take to do road following. And that was the beginning of the NavLab project.

So we submitted the proposal, one for road following and one for the architecture of what the system would have to look like. I went back, finished my thesis, and defended my thesis. I thought I would take a couple weeks off to figure out what I wanted to do next. Raj came up and said, “Congratulations, you passed your thesis. There's a meeting in my office to talk about how we start the NavLab project.” And that was my gap between my thesis and the NavLab project, walking from the defense room into Raj's office to start this meeting, to start planning what we were going to do.

Departmental decisions that made big systems projects feasible

Gilliam: To stop you for a quick second to add to the queue of operational questions. So I have the question of 1) grad students as ‘employees vs. novel thesis generators.’ And now 2) in one of the DARPA year-end reports for the NAVLAB from CMU, I saw them refer to you as a ‘project manager.’ Which is interesting…that's not an academic designation. That sounds very industry. So I would love for you to talk a little about that, too. If I should read into that or what that means to you.

Thorpe: Shouldn't read much into that. But what you should read into is when Newell, Simon, and Reddy really got computer science going — and before that Perlis and Shaw, etc…They said, “If we're going to do big projects,” — and we did big projects like the Andrew Project, that turned into the Andrew File System and things like that. They said, “We need to have more faculty than we have courses to teach. We can support them on soft money, so we need to have research scientists who have full faculty privileges.”

So if you're at MIT and you're not tenure track, you're nobody. Carnegie said, “No, we want to have non-tenure track people, supported on soft money, but who have advising privileges and PI privileges and all of the privileges that faculty have. They just may or may not be teaching courses at this time.” So, unique to computer science, there's this whole research track which is fully equivalent to the tenure track. And that was my underlying title. When I defended my thesis in October, Raj said, “I can't make you a faculty offer. You're going to be a postdoc until July 1 and then I'll make you a research scientist.”

So I came up through the research ranks after having spent that eight months or so as a postdoc. And it's partly that business of trying to figure out what research scientists were that caused computer science to first break out of the Mellon College of Science and be a free-floating department. And then form themselves into a school so they could really have this parallel track.

‘Project manager,’ that was just sort of a title because that's what I was doing on the NavLab project. Raj took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

Because I was a graduate student writing this proposal, I couldn't be a PI. So Takeo was a PI and I think Hans Moravec was a PI. And then as I became a faculty, I became a co-PI and then eventually the PI. But it was pretty clear that Takeo was interested in the project because it was a cool project. He was also interested if he could have a few people working on individual projects. But he didn't want to worry about scheduling stuff, and who got time on the machine, and writing reports. So, he was very happy to turn all of that over to me and let me run it the way that I wanted to run it.

Gilliam: And how was that? Can you describe your approach? Let's say, describe your approach to…somebody who knows how an academic lab works usually, but doesn't do anything like this. They output papers, they play the h-index game a little bit, etc.

Thorpe: Well, you pointed out a very key feature here, that the whole system has to work and that each student has to have something that they can claim as their own and that they can write up as their own thesis topic.

So, we got ready to put the system together and I said, “How are we going to see the road? Jill [Crismann], how do you want to do it?” Oh, well, she wanted to work on electrical engineering, signal processing, color classification scheme. “Great, you go do that. Here's what your interface needs to look like. You need to tell me where the road is in the image. If you tell me that. Someone will take that and will steer the vehicle.”

“Karl [Kluge], do you have any ideas?” “I think that we ought to be able to track edges and track color features. You ought to be able to track yellow lines and track white lines.” “Oh, that sounds like a good idea.” So Karl went off to do that. Great. “If you can tell me where the road is in the image, then I'll have someone figure out how to steer it.”

“Tony [Stentz], what do you want to do?” “Well, I really want to steer the vehicle.” “Good. You can do the path planning for that kind of thing.”

“Martial [Hebert],” he was a postdoc who came over from France, “what do you want to do?” Well, he was a 3D vision guy. “Cool! They're sending us a laser scanner. Do you think you could use that to determine where obstacles are?” “Oh, great. If you can find out where obstacles are, then feed that to me.”

“Jay [Gowdy] what do you want to do? I think you're a good systems guy. If someone tells us where obstacles are and someone tells us where the road is, could you figure out how to put the whole system together and give them useful user interfaces?”

So, really trying to build the system in a modular way, so that each person had something that they could do and be proud of. And if it worked, great! Then we plug it into the system and it runs. And if it doesn't work, well, I've got two other people working on other approaches to that same thing.

Gilliam: And if somebody like Dean [Pomerleau] came in with neural nets — wanting to apply Geoff Hinton's somewhat theoretical methods to this — how do you fold them in? Do they sit off a little further to the side in the lab? What does that look like?

Thorpe: Well, first of all, we said, “Neural nets? Oh, come on, nobody's ever built a neural net that does anything. Haven't they proved that neural nets aren't going to work? Um, you want to try it? Sure, try it. Let's see what happens. What would you like? You want some images? We'll collect you some images. Come on in. We'll see.”

And then talking to our people at DARPA, because there was a different and competing office at DARPA doing neural nets, and saying, “Look, I know you're skeptical about neural nets. We've got this neural net guy, separately funded. I want to give him a shot on this. You don't mind if he uses our robot?” “No, that's fine. We'll disprove neural nets once and for all. So come on in and try it out.”

Dean turns out to be an extraordinarily bright person. And you wouldn't know it to meet him, but he's extraordinarily competitive. And he's willing to get his fingers into everything. So, mostly what he wanted was this recording of images and steering wheel position. And then he went off and started to work on that and built a system with 100 hidden units and it seemed like it could work, but really slowly. And so he cut it down to 50 hidden units and it seemed like it could work as well, and a lot faster. He cut it down to 25 hidden units. And he eventually got down to four hidden units, and now he was working in a believable amount of time and it still worked pretty well. He cut it down to three hidden units and it didn't work anymore. So, okay, four hidden units.

Back to this theme of Strategic Computing and DARPA building supercomputers. One of the supercomputers they built was at Carnegie, and it was a thing called the Warp machine. The Warp machine had ten pipelined units, each of which could do ten million adds and multiplies at once. (The Warp machine and BBN’s butterfly processor were briefly covered in an earlier piece in the ARPA series.)

H. T. Kung was working on systolic computing, where things just sort of chunk through the pipeline. And we said, “Here's this big box. Because we're sponsored by Strategic Computing, we'll put it on the NavLab. Can anybody figure out what to do with it?” “Well, it's a really complicated thing and really hard to process.” But Dean could figure out what to do with it. Partly because Dean is a really, really smart guy.

Partly because the inner structure of a neural net is, “Multiply, add, multiply, add.” You take these inputs, you multiply them by a weight, you add them together, you sum it up, run it through your sigmoid function. So, Dean was, as far as I know, the only person… well, that's maybe a bit of an exaggeration. Some of the speech people were able to use the Warp also, because they were doing similar kinds of things…But Dean got the full hundred megaflops of processing out of the Warp machine! And then he put it onto the NavLab and we got the whole thing to work. So, after that, Jill got her thesis for doing her thing. Carl got his thesis for doing his thing. But Dean's neural nets seemed to work in most situations better than either Jill's or Carl's one. And so that's the one that got all the attention.

Going from ALVINN to RALPH and CMU’s complementary systems projects

Gilliam: What did you consider a bigger practical breakthrough? There's obviously the ALVINN system, which to a modern machine learning engineer’s eye, would give them a little tear. “Wow, it's so minimalistic It can do all this (with mostly just neural nets).” And then RALPH comes around. It’s a little more manual. Did you consider RALPH a much bigger practical step? Or once ALVINN came around, did it seem somewhat clear to you that something like RALPH would be possible and better?

So that was part of it, but the other part of it is..how do you take something like ALVINN…which at its core is just turning steering wheels. It's not doing any representation of the world…how do you take that and merge that with obstacle detection and avoidance? How do you take that and merge that with a map? And so, layering architectures on top of this that can take something which shouldn't really have a geometric representation and extract some sort of a geometric representation so we could do other things with it. That was really the fun part of it.

Gilliam: Is that when it became a proper, all hands on deck, CMU-type project — going from ALVINN to RALPH? It sounds like ALVINN was not a one-man job, but smaller. But then maybe RALPH became a proper CMU systems effort.

Thorpe: So Todd [Jochem] came along and worked with Dean. And part of what Todd did was to say, “How do you get some geometry out of this so that you can train multiple different neural nets and switch between them.” So that's even separate from RALPH. That was still in the ALVINN stuff. How do you train this thing up and have things work together? But if you look at the NavLab '90 videotape where we're going from my house to Keith's house, that was the obstacle detection and stopping. That was landmark recognition. That was the annotated map saying when we're supposed to do what. That was high-accuracy dead reckoning when we were doing a sharp turn and ALVINN couldn't work. Getting all of those system pieces to work together was a lot of fun. And this is on top of an experimental vehicle where you never knew what things were going to break.

Gilliam: I’d like to put something on the record, and you let me know if I'm wrong. So I think if some of my readers read what you said, they would say, “Oh, how fortunate that CMU had this systolic array machine going on and that it happened to be able to — obviously some computer structures were somewhat specialized back then — that it happened to be able to work with the natural language project and the vision project.”

But my reading of the history is that this doesn’t seem like circumstance. Now, this was a period when a lot of people weren't pursuing building machines at universities anymore. Obviously, this is not long after Slotnick [and the ILLIAC IV machine] at U of I. And Slotnick's gone on the record some years before saying “The era when you could build big, good machines at a university is over. Like, “this is for industry now, it's not for universities to do.” It seems like CMU’s systems approach helped prolong the era where you could build a good workable machine at a university.

Also I read that the Warp folks made sure that their machine could work with the other ongoing CMU systems efforts like vision and like natural language. Is that fair? Or is that me giving too much credit to the CMU structure?

Thorpe: So the CMU structure, when I got there, they had just finished building CM*, and before that C.MMP. CM* was a powerful array of 50 PDP-11s. But the difference between C.MMP and CM* was they were playing with…if you have multi-processor systems, do you connect a big crossbar switch with every memory connected to every CPU? Or do you connect them in some sort of a hierarchy? And so this notion of building big systems, big hardware systems, was there.

And of course, computer vision has always been a hog for CPU cycles. So in the early days, they built these systems and then they said, “Computer vision people and speech people, have at it!” Because the computer vision people and the speech people knew that we needed the cycles. And we were willing to put up with a fairly klutzy interface in order to see if we could take advantage of these sorts of things.

So, that ethos was there. The names C.MMP and CM* come out of the Newell and Siewiorek book. So here's Al Newell, a pioneer of AI, writing a textbook on computer architecture because he understood that AI needed multiprocessors in order to work. So that ethos was there.

We had a guy in the computer vision group named Rick Rashid, who was working on how you make the hardware more efficient, and how you make the software more efficient, and how you build microkernels. And he's the guy who went out and started Microsoft Research. So there's this long history of computer vision people having to have the highest possible processing power and being willing to put up with a fair amount of klutzy-ness in order to take a wire-wrapped prototype and really use it for for high performance.

Concluding Thoughts: CMU’s applications-focused culture, its origins, and the legacy of its grad students

Gilliam: This has all been fantastic. Is there anything else you think people should understand about how CMU managed its projects? It was a quite exceptional contractor there for a while…To me, CMU in this era seems like one of the all-time great DARPA contractors. Right up there with BBN in the ARPAnet era and Skunk Works when it built STEALTH. In many ways, CMU seems quite similar to BBN.

(Here, Chuck and I went on a 5-10 minute tangent discussing some of his friends from BBN, DARPA’s Clint Kelly, and others from the DARPA Strategic Computing era. Check that out on the audio if you’re interested. But it’s been skipped in this abridged version.)

Thorpe: …Back to Carnegie. Carnegie had a couple of aspects of the culture. One is that doing big things and doing practical things makes a difference. So if you look at what Carnegie has done…Duane Adams, his first proposal he wrote was for the NavLab. The second proposal he wrote was for the Software Engineering Institute. Running an SEI, that's not a typical academic thing.

But there are places like Berkeley. Berkeley runs the Berkeley labs and the Lawrence labs. So running an FFRDC (federally funded research and development center) is something that a handful of universities do, but not everybody. You don't get a lot of published papers out of that, but what you do is you provide a venue to take the basic research you've done and to turn it into practical things which change society.

So the SEI does big software engineering projects for DOD, but they also do a lot of training. They also run the CERT (Computer Emergency Response Team.) So, when some new virus breaks out, SEI jumps in and tells the world what the virus is and how to fix it. And that's the kind of thing that is a little bit unusual for a university to do. But Carnegie had the culture to say, “Making a real impact in society is important. And this is one way you can do it. Let's do it separately, off campus, so we don't dilute the graduate students and writing the papers, etc. But let's have this pipeline.”

Within robotics, NASA came to us, I don't know, 30 years ago and said, “NASA is spending a lot of money on robotics. We're not getting good PR out of it. Partly because our robots are, they go off to places like Mars. Could you take the technology that NASA is developing and work with people like John Deere?” So we formed the Robotics Engineering Consortium. It sits off campus. It employs more staff and fewer graduate students. It can do confidential projects. And it takes basic research done on campus and turns it into practical projects that Deere can take and turn it into products.

So that whole notion of a halfway house to take basic research and turn it into something bigger, something more practical, something applied…that this is a good thing for a university to do…that's not a universal university ethos.

Gilliam: Two-part question. The first is: why do you think CMU was able to maintain that ethos for so long? And the second part is interesting because, now, you’re a university president. You have the benefit of all sorts of institutional knowledge. Why do you think schools struggle to manage projects this way, if it seems, in many ways, to be a natural default approach for technical subjects?

Well, let me give you two reasons why Carnegie succeeded in doing that. The first one, I think, is really more Newell than Simon. Simon wanted to do great intellectual projects. Newell wanted to build things. So Newell caught on that building big stuff and seeing how it worked in the real world was important. And then Raj pushed that even further, starting out with speech. But then also the hardware projects etc. The other reason why Carnegie did this…if you go back in the history, there was the Carnegie Institute of Technology, and there was the Mellon Institute of Science.

The Mellon Institute of Science was the contract research shop. When the Mellons owned Gulf Oil, an aluminum corporation, and big chunks of various steel companies, they needed a chemistry research shop. And so the Mellon Institute was this chemistry research shop, and it existed to do contract research. If you got to produce some papers, that was good, but it was 50 Ph.D. chemists doing industrial research. In the great merger of ‘67, which formed Carnegie Mellon, we absorbed the Mellon Institute and had to figure out what to do with it. And partly what we did with it was to keep it running as a contract research shop.

So, there was already this notion of an off-campus, sponsored by soft money, more practical research shop that we didn't quite know what to do with. But in an ideal world, you would take basic research from the main campus and then figure out how to apply it, etc. So that was a little bit in the DNA of Carnegie since the merger in ‘67.

Gilliam: It's not dissimilar to MIT circa 1920. It had a program called the Technology Plan. For context, when MIT was young — from 1860 to 1920 — they were perpetually poor and living on shoestring budgets. Back then, more than now, MIT really lived on the ethos of, “We serve industry above all else. That’s what we’re here to do”. So, the Technology Plan came about because, at a certain point, they said, “The most honorable thing an institute of technology can do is provide a great service for a fair price.” So they let their applied researchers loose to work a lot of these contracts to the point where, at a certain point, their chemical engineering/applied chemistry department was 75-80 percent industry funded.

So this is very interesting.

Thorpe: Sure. And MIT has this history of spinning out the Draper Labs and the Lincoln Lab. And again, that's stuff that's too applied to be done in a university lab by graduate students, but that should have some connection to the university. Less connection in the case of Draper Labs because of some firewalls. But MIT has some of that same same ethos. Yeah, Carnegie had this very practical streak to it.

Paul Christiano was the provost for a while, and his dad was a Carnegie Tech graduate in civil engineering back when the civil engineering curriculum had courses like bricklaying. This is really hands-on, get out there and get your hands dirty.

Gilliam: Yeah. Back when they used to be able to teach industry machinists at night and things like that, at places like MIT or Carnegie. Different world we live in now.

Thorpe: So, when I was director of the Robotics Institute, I said, “We have won the battle to be the biggest robotics research group. We're very good at producing smart robots. We're very good at producing smart graduate students. We're not very good at producing smart industry. How do we build up a robotics industry in the city of Pittsburgh?” And we started working with the big foundations there. We started working with the little spinoff companies. There were a bunch of them, but none of them were very big. And we started working with the university management to say, how do we pull all of this together?

And when I say the foundations…like the Hillman Foundation, which is one of the richest foundations in the city…John Denny from L. C. Hillman's office came and sat with us and said, “How are we going to build a robotics industry?”And that led to the Robotics Foundry, and that led to Pittsburgh becoming RoboBurgh, which eventually led to these bigger clusters and some federal manufacturing research centers and so forth.

Gilliam: So in that case, do you take a lot of pride in the following? In your one previous oral history, you referred to the early Google group as “Carnegie Mellon West.” You said this talking about how Sebastian [Thrun], who of course, built his DARPA Grand Challenge vehicle at Stanford, but he came up at CMU. And you pointed out how everybody but one in that early Google autonomous driving group kind of came up or had a deep affiliation with CMU. And it seems like that's held even recently. There are a lot of departments that do pretty good autonomous car work, but Uber came to CMU, specifically, to essentially attempt to buy up a whole swath of the university and its projects wholesale to fold into itself.

Do you take a lot of pride in the projects and people that CMU has been outputting being considered applied enough that they could be a direct input into an industrial process? Rather than what the current applied researcher’s credo seems to be at a lot of universities, namely, “Somebody could pick this up and within a handful of years, it can then be applied.”

You all are kind of directly plugging in.

Thorpe: So, Matt Mason — who was on my thesis committee and was my successor as Director of the Robotics Institute — he's a much more theoretical roboticist. And he says, “You know, if all of the applied people left Carnegie Mellon, we would still be a famous robotics institute for all of the more theoretical stuff that we do.”

But they had to kind of live in the shadow because, face it, it's much easier to do a videotape of a robot flying down the road and to explain to the Pittsburgh Post Gazette what it is that we're up to than it is for the theoretical people to say, “Ooh, and look what I can do with a quaternion, I can represent this thing…and this is a fundamental, advance that people are going to be using for the next century!” And they're right! And we are grateful to them. They just have a harder time getting the publicity out of it.

So the DARPA urban challenge was just a family feud. Sebastian and I have a best paper award together. Sebastian and Red [Whittaker, the engineering lead on the DARPA Grand Challenge teams] have a best paper award together. Chris Urmson, who is Sebastian's number two, was Red's graduate student. So this was just a family feud. Those guys happen to be out at Stanford, but they’re still buddies of mine.

(We then talk for a bit about semi-related topics before I ask him if he’s also proud of the number of NavLab/CMU grad students that have gone on to be technical leads at companies or found companies.)

Thorpe: Yeah, you know, I had expected more of my alumni to go off into academia. If you're an academic yourself, you sort of think, “Shouldn't everybody want to do what I do?” And, no! They've gone off and started their own companies. Jill Crismann was the AI guru for the Department of Defense and now has gone back into private industry. Jenny [Kay] went into academia and still teaches. John Hancock went off into the computer game industry, which is a good thing. Hardware keeps breaking for him. So we thought that actually working on real robots was not as good as working on simulated robots.

Parag [Batavia] started his own company. Raul [Valdes-Perez] went off to Google. So it's been fun watching these guys. They have this attitude that, “We can do it! Here's a cool problem. Let me figure out how to make something work. Oh, cool. That works. Uh, now that it works, let me see if I can figure out how it works and write a paper out of it.”

And that's just a fun attitude and fun to watch them go and succeed.

Gilliam: “And if it works, how do I make it usable? How can it be productized?”

Thorpe: The next thing, and the next thing, and the next thing. Some of them have figured out, “How do I take this and turn it into money?” Others have said, “How do I take this and turn it into robots that run on the surface of Mars?”

I cut off the recording there. Over the course of the interview, I came to admire Carnegie’s applied-minded approach to engineering work even more than I did coming into the interview.

Hopefully, you enjoyed this exploration into how CMU managed its exceptional autonomous vehicle research work from the mid-1980s through the early 2000s. CMU’s strategies are, in practice, quite logical. However, comparing their methods to academic norms, it is clear that CMU continually made decisions to go against the academic norm. Doing this, they maintained their major comparative advantage. The result, in the field of computer science and robotics, was a systems research powerhouse that helped shape the field of autonomous driving as we know it today. In many ways, this period of CMU’s history is the closest university example I have seen to the fabled early years of MIT.

Thanks for reading:) See you next time!

Subscribe now

Share

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Building out an experimental test bed for a research field.

• Putting out contractors for specific “integrators” in addition to prime contractors and basic researchers

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Specific Links:

• Chuck Thorpe’s IEEE oral history (video)(transcript)

• Red Whittaker’s IEEE Oral History (video)(transcript)

• Raj Reddy’s Computer History Museum Oral History (Parts One and Two)

• CMU Strategic Computing Vision Project Report: 1984 to 1985 

• Dean Pomerleau’s 1988 paper, ALVINN: An Autonomous Land Vehicle in a Neural Network

• RALPH paper

General Links:

• Strategic Computing: How DARPA Built the Computer Age

We’ve all heard that “DARPA invented the Internet.” But few have heard of BBN, the contractor that did the most work to bring the ARPAnet into existence. Today’s piece dives into the history of BBN and the firm’s unique structure. A firm like BBN winning the main portion of the ARPAnet project was a pivotal reason the ARPAnet project went so smoothly. BBN embodied the “middle ground between academia and the commercial world.” BBN’s early operating model provides an ideal management framework for anyone looking to deploy researchers on ambitious research projects within the structure of a firm. With BBN’s structure, many difficult projects become possible.

With that, let’s get into it.

Subscribe now

Introduction

DARPA has no lab benches, research facilities, or staff scientists doing science. Instead, it has contractors. DARPA uses contractor ecosystem to execute any ideas the office chooses to fund. Throughout this series I will generally write project histories from the POV of DARPA’s PMs and office directors. This is a natural choice because, depending on the era of the organization, either the project managers or the office directors are usually tasked with driving projects forward.

However, in the coming trilogy of pieces I’ll tell stories from the POV of some of DARPA’s best contractors ever. Projects are selected, funded, and often steered by the DARPA office; but ideas are entirely executed by contractors.

In many cases, which contractor wins a contract might only have a moderate impact on the cost efficiency, speed, and overall results of a project. In those cases, it might be hard for a contractor — whether it be a university research lab, private company, or a non-profit institute like RAND — to prove it is truly top-notch.

One the other hand, there are cases where many in a field do not even bid on a contract because they do not find the project specs feasible. In these cases, a level of excellence can be required to deliver on a proposal. This is the category the now-famous ARPAnet fell into. As most readers know, the ARPAnet was the precursor to the internet. It was as the main contractor on this project that the MIT faculty spin-off research firm Bolt, Beranek, & Newman (BBN) would cement its name in the history books.

The coming trilogy of pieces exploring DARPA contracts from the contractors’ POV provide an essential perspective to this series. Contractors’ work on projects like BBN with the ARPAnet, CMU’s driverless vehicle projects, and Lockheed Skunk Work’s STEALTH project have many management insights in common — despite the obvious differences between those projects and contractors.

Today’s piece dives into the contractor primarily responsible for DARPA’s most famous project: BBN.

What was BBN?

BBN’s stellar results on the ARPAnet project did not shock many in the computing community. In the late 1960s, the firm was known to be special. It was an interesting kind of research firm, of a sort we don’t see much of today. BBN’s staff found themselves natural members of the elite Cambridge, Massachusetts research community despite the firm’s for-profit status. The firm was home to some of the best theoretical researchers in Cambridge — formerly professors at Harvard and MIT — as well as top-tier engineering researchers who often defected from the large computing projects at MIT’s Lincoln Lab.

In general, many researchers defected because they believed BBN was a better place to work on big problems in more exciting ways than at Lincoln. Theoretically inclined researchers, like Robert Kahn, found this a great place to do proper research in an applied context. Great engineers who cared less for theory, like the project’s lead engineer Frank Heart, found the firm an ideal structure to work on implementing real technology that still had an extreme level of novelty.

To be sure, BBN was a company that found ways to make a profit and grow at a considerable rate in its early decades — 26% year over year. However, it was also a firm that went out of its way to ensure that its researchers were happy. It wanted the best problems — as long as it could cover the costs and salaries of the project with some kind of funding — not necessarily the most profitable problems.

Robert Kahn and J.C.R. Licklider are two familiar names that called BBN home for a time. From his view as IPTO chief and minor celebrity in the computing research community in his day, Robert Kahn had a deep understanding of what went on at every notable computer research center in the world. So it should be taken as no small thing when, in an oral history, Kahn had this extreme praise for the firm in this era:

BBN was a kind of hybrid version of Harvard and MIT in the sense that most of the people there were either faculty or former faculty at either Harvard or MIT. If you've ever spent any time at either of those places, you would know what a unique kind of organization BBN was. A lot of the students at those places spent time at BBN. It was kind of like a super hyped-up version of the union of the two, except that you didn't have to worry about classes and teaching. You could just focus on research. It was sort of the cognac of the research business, very distilled. The culture at BBN at the time was to do interesting things and move on to the next interesting thing. There was more incentive to come up with interesting ideas and explore them than to try to capitalize on them once they had been developed.

As we’ll explore, some at the firm would often try to capitalize on big ideas. But this was not the focus of the firm.

From a legal point of view, BBN was legally structured as a partnership for its early decades — kind of like a law firm — before going public in 1966. Teams at BBN were not exactly permanent. Individuals were often recruited simply because BBNers felt they could contribute to some coming project, or simply “raise the average level of competence of the firm.”

The firm’s culture was far more similar to a university lab than an incumbent firm like Honeywell. If a talented researcher found an area of exploration interesting, the firm did what it could to find a payer for that research. For example, this was the case with a young J.C.R. Licklider looking to further explore the world of interactive personal computing on a very expensive machine. Bolt and Beranek did not say no. Rather, they said yes, bought a machine, and found a way to fund the project through a grant from the Ford Foundation. The long research report — later turned into a book — by Licklider and other BBNers titled, “Libraries of the Future” was far more than it seems. The report did more than just carry out a forward-looking analysis by information researchers and engineers assessing what digitized libraries could look like for a library non-profit. The report contained much of the early technical exploration that would grow to become Licklider’s legacy.

Beyond BBN’s whole-hearted embrace of the “research” in “research firm,” its structure as a “firm” allowed it to carry out extended, elaborate implementation projects like the ARPAnet project that were ill-suited to an academic lab. The firm’s expertise in early computing systems — built up by contract research projects as well as the team’s pre-existing experience at places like Lincoln Lab — allowed it to push technology areas forward in a self-funded way. For example, in the 1960s the firm designed, implemented, and maintained one of the first commercial time-sharing systems for non-engineers. The project gave Massachusetts General Hospital a time-sharing system. The system allowed the hospital to handle patient administration in a modern way and automate some of the data analysis in their research. This, of course, was not the first time-sharing system. But projects like this allowed BBN engineers to continue honing expertise with young technology and earn private sector wages in doing so.

The ARPAnet project was another implementation project optimally suited to BBN’s somewhat unique structure and makeup. But, to gain a full understanding of how the firm came to be what it was, let’s take a step back and start at the beginning. BBN was not started as a computing research firm. Bolt and Beranek were not computer scientists. They were acoustics professors who, steeped in the applied research tradition of early MIT, began to become overwhelmed by their contract research work in the 1940s.

BBN’s early years as an acoustics firm

BBN began in a fashion not unlike many early-20th-century MIT spin-off companies. A team of professors and graduate students worked extensively on consulting and contract work. As time went on and word spread, the demand for their contracting services grew. For a while, renting an additional room in their lab building and hiring more graduate students was enough to keep up with demand. Eventually, the young company had to get its own quarters and hire its own full-time employees.

In the case of BBN, the original professors involved were involved in the relatively young area of psycho-acoustics. In the wake of World War II, Richard Bolt was set to become the director of a new acoustics laboratory at the Institute to continue several courses of research that were largely funded by the Navy. Faculty and staff with backgrounds in physics, electrical engineering, architecture, mechanical engineering, aeronautical engineering, and psychology would all contribute to the contracts. In these post-war years, Leo Beranek was recruited to join the lab as its Technical Director from Harvard’s electro-acoustic laboratory. He was also given a professorship.

Since the inception of MIT’s Technology Plan in 1920, the Institute had made a habit of actively facilitating research contracts to its faculty. Professor’s contract salaries covered only four days a week during the academic year and left weekends, holidays, and summers entirely free to the professors for contract work. Professors could wholeheartedly spend their time on contract work with close to zero red tape or permission required from MIT. Unlike places like Harvard — and most universities — the Institute, in this era, encouraged this double life for its professors. Negligible amounts of paperwork and permission-asking were involved. MIT saw no more suitable activity for faculty at a professionally-focused institute of technology than to ply their craft for paying customers. In doing so, professors simultaneously kept up with industry trends, solved industry’s problems, learned how to keep their university research industry-relevant, and brought money into the Institute.

In the heyday of this arrangement, requests from industry regularly came into MIT’s office of the president for help in various problems, such as acoustics. When acoustics queries came in, they were routed to Richard Bolt. In 1946, one of these requests came from an architecture firm responsible for building the UN Headquarters in New York City. Bolt submitted a bid and won the commission. In 1948, the drawings arrived from the firm with the request for Bolt to begin his acoustics consulting work. Given the massive size of the drawings, Bolt realized the project was not a one-man job. He asked Leo Beranek if he’d like to work on the contract as a team. The two had papers drawn up, and the partnership of Bolt & Beranek came into existence.

For this first contract, Bolt and Robert Newman — one of four part-time graduate students hired to work on the project, who was brought into the partnership upon graduation — were responsible for figuring out acoustical treatments for the UN building. Beranek was in charge of the more troublesome task of sound system design and equipment selection for the unorthodox building. Despite the difficulty, the project turned out to be a very public success. Beranek wrote that the firm’s “name was known to architects everywhere, and business boomed.”

To this point, the firm had simply been run out of an extra room in the Acoustics Laboratory. As Beranek described the arrangement:

The firm, Bolt & Beranek, had the blessings of MIT’s new President, James Killian. He offered to help us get started and rented us two rooms in the MIT Acoustics Laboratory for our use, but warned us that we would have to seek space outside of MIT if our needs expanded.

By the end of the business’ first twelve months, the room was already full of equipment the firm had bought to continue to service the growing number of contracts.

In late 1949, the young firm moved to another address in Cambridge. Gradually growing, in 1951 the firm would move again to take up two apartments and a basement in a mostly residential building on Elliot Street. By 1956, the acoustics-focused firm had built up to 50 full-time employees — many of them former Cambridge graduate students — along with an additional office in the architecture hotbed that was LA. This all happened without outside financing beyond the firm’s line of credit with the local bank.

In this period, the firm also added two more of the four founding part-time graduate students to its partnership. These new partners, Jordan Baruch and Samuel Labate, had worked with the firm for years and also completed graduate theses in areas relevant to the firm. Additionally, in 1953, BBN incorporated in order to shield itself from liabilities associated with the growing subsection of its business associated with controlling aircraft noise. The National Advisory Committee on Aeoronautics and companies that manufactured jet engines hired the firm to work on problems such as designing structures to minimize noise during engine testing. Projects like this had many practical uses, but also represented cutting-edge uses of the theory and experiment in acoustics that excited acoustics researchers. (Projects like this also benefited from the insights of psychologists. As it turned out, projects in noise reduction or concert hall design often turned out to be more about “perceived” decibels rather than actual decibel measurements.)

As all of this was happening, Beranek gradually reduced his teaching load at MIT — to 75% in 1951, 50% in 1953, etc. He resigned his tenured professorship in 1958. More and more work — from government contracts, private industry, and research grants — steadily rolled in. As the business grew and contracts rolled in, the MIT and Harvard research communities provided an ample supply of part-time minds and hands. Sometimes individuals were brought in because they were acutely needed. In other cases, the firm simply saw a talented individual and would find some excuse to bring them on.

As the business kept expanding, in size and scope, Beranek set his sights on building up more of a talent base in computing. During the war, Beranek remembered how much of the work of his Harvard Electro-Acoustic Laboratory and its partner lab, the Psycho-Acoustic Laboratory, related to problems dealing with information handling in warships. BBN already combined psychological insights in its acoustics work, working on problems like pilots’ ability to process noise in cockpits. Expanding into the arena of man-machine systems felt natural. To Beranek, this felt like a fascinating area of research with the potential to amplify human labor.

Leo Beranek (left) and Richard Bolt (Right). Photo courtesy of Beranek’s personal collection.BBN enters the computing business

Beranek’s motto in making hires at the firm was, “Each new person hired should raise the average level of competence of the firm.” At its core this was a research firm. And in staffing the research firm, he felt it was vital that they only continue to hire people who they felt were as smart as them. With Beranek’s sights now on computing, he seemingly only had one man in mind to help lead the firm’s computing efforts: J.C.R. Licklider.

Licklider had been a young researcher in the Psycho-Acoustics Laboratory at Harvard during the war. Beranek was so impressed with him that, soon after being brought over to MIT, he pushed for Licklider to be hired there as well. Now, he was going to push Licklider to give up his tenured position at MIT to come to BBN. With the promise of generous stock options, the understanding that the firm did cutting-edge research work, and the title of “vice president in charge of man-machine and information systems,” Licklider came aboard in 1957.

One co-worker said of Licklider’s move that, “he was becoming entranced by computers at this point and felt he could pursue these interests best at BBN.” In the end, Licklider would go on to draw many of his former friends and colleagues from graduate school, the MIT faculty, and elsewhere in his academic travels to come to BBN. After moving to BBN, some simply continued with their pre-existing work under research contracts they’d won before joining BBN that paid for their services. Some were folded into BBN’s regular operations. Others found contracts to do odd things like write textbooks using BBN’s rare equipment.

But before this collecting of colleagues started, it was just Licklider joining the firm from his MIT position. No formal roadmap had been agreed upon with BBN management. Licklider quickly got things rolling, though. Beranek describes Licklider’s early tenure at the firm as follows:

Lick, as he insisted that we call him, was outgoing and always on the verge of a smile; he ended almost every second sentence with a slight chuckle, as though he had just made a humorous statement. He walked with a gentle step, often with a Coca-Cola in hand, and he always found the time to listen to new ideas. Relaxed and self-deprecating, Lick merged easily with the talent already at BBN. He and I worked together especially well: I cannot remember a time when we disagreed.

Licklider had been on staff only a few months when he told me, in the fall of 1957, that he wanted BBN to buy a digital computer for his group. When I pointed out that we already had a punched-card computer in the financial department and several analog computers in the experimental psychology group, he replied that they did not interest him. He wanted a then state-of-the-art digital machine produced by the Royal McBee Co., a subsidiary of Royal Typewriter.

“What will it cost?” I asked.

“Around $30,000.” (~$300,000 today) he replied, rather blandly, and noted that this price tag was a discount he had already negotiated.

I exclaimed, “BBN has never spent anything approaching that amount on a single research apparatus. What are you going to do with it?”

“I don’t know,” Lick responded, “but if BBN is going to be an important company in the future, it must be in computers.”

Although I hesitated at first — $30,000 for a computer with no apparent use seemed just too reckless — I had a great deal of faith in Lick’s convictions and finally agreed that BBN should risk the funds. I presented his request to Labate and Baruch, and with their approval, Lick brought BBN into the digital era. Lick sat at that computer many hours each day, literally hoarding the machine, learning how to do digital programming.

To be sure, this turn of events was probably not surprising to Beranek. Licklider had been deeply involved with computers at MIT through his affiliation with Lincoln Lab computing projects as well as interactions with Wes Clark and other early computing gurus. It was at MIT where Licklider, as a technically focused psychologist, was becoming an evangelist for work we would now categorize as human-computer interaction. But Licklider may have grown expensive more quickly than Beranek anticipated. As contracts and know-how were gradually building up in the computing area of BBN, the firm bought the first PDP-1 from the Digital Equipment Corporation two years after it purchased the first machine.

After describing the Royal McBee purchase in his chapter of “A Culture of Innovation” — an internal history of BBN by BBN alumni — Beranek described the obvious next step:

Then Lick and I took off for Washington D.C. to seek research contracts that would make use of this machine, which carried a price tag of $150,000 (~$1.5 million today). Our visits to the Department of Education, National Institutes of Health, National Science Foundation, NASA, and the Department of Defense proved Lick’s convictions correct, and we soon secured several important contracts.

When it came to talented researchers and academic engineers, BBN’s instincts as researchers often led the way. The organization was a firm and would not willingly light money on fire to pursue its interests. But if a talented group of researchers and engineers felt a problem area was promising, BBN might spend money to pursue it as long as management felt it could probably earn the money back.

In the case of purchasing the PDP-1, this was a more risky leap than the company usually took. But the risk was quickly mitigated, with BBN quickly selling government contracts in areas like man-machine cockpit system displays and computer-based learning. However, the primary grant driving Licklider’s work on computers at the firm was not a government grant. The grant that proved futuristic enough to support Licklider’s grand vision came from a grant-maker the reader would likely not expect.

J.C.R. Licklider (and his trademark smile) with his wife, Louise. Photo courtesy of Leo Beranek’s personal collection.Libraries of the Future

The Council on Library Resources was established by the Ford Foundation to study “libraries of the future.” The Ford Foundation was something like the Gates Foundation of its day and, thus, funded a broad swathe of areas. In particular, the libraries of the future contract arose from the growing “information problem” talked about in the research community in the 1960s. Namely, much more work was being published than any researcher could possibly make use of. Licklider and BBN were tapped to explore whether or not modern computing could do something about this problem.

(For those interested, the Council on Library Resources was advised by Warren Weaver, John Pierce, and Richard Bolt in the early stages of its existence. Weaver and Pierce have been featured in earlier pieces on this Substack.)

Contracts like this two-year contract were not likely where BBN made the lion’s share of its money. However, projects such as this — or projects such as those where NASA paid the company a fee to perform exploration and produce a textbook — allowed BBN to offset the costs of explorations its researchers already wanted to do.

Going above and beyond what the Council on Library Resources likely expected, Licklider’s team used this contract as an excuse to explore seemingly futurist technology. For example, Licklider and the project team dedicated much of the second half of the report to technical explorations under the assumption that future library-goers might interact with the library as a store of knowledge with a question-answering front end, rather than as a means to check out books.

The project seems to have gone a long way in offsetting the costs in a variety of explorations by BBN staff. Below, BBN-alum John Swets describes a question Fisher Black — of Black-Scholes Model fame — pursued:

Fischer Black, a mathematics graduate student at Harvard and part-time BBN employee, produced a series of question-answering systems that involved symbolic logic and computer programming, including one in which a non-human system first solved the “airport problem” posed by McCarthy. Based on some statements about a person’s whereabouts, transportation resources, and local geography, the system answers the question of how to drive to the airport (in part: walk from the desk to my garage, drive my car to the airport.)

(Eventual Turing Award winners John McCarthy and Marvin Minsky were also part-time members of the Libraries of the Future project team.)

Other ideas explored in the report include:

• The importance of improved random access memories for optimal information retrieval on large datasets — like one containing the entirety of scientific papers.

• Black’s question-answering system which successfully used first-order predicate calculus to represent information.1

• Several prototype indexing methods and iterative information retrieval methods for searching academic papers.

• And many qualitative aspects of PC user experience which Licklider is known for.

Many ideas in the report were not novel on their own. However, many of the systems outlined in the report were novel, practical advances that both fulfilled BBN’s contract obligations and the research ambitions of its employees. The report paints a remarkably prescient picture of how personal computers would eventually organize knowledge and interact with users. In reading the report, one can easily recognize many of the ideas for which Licklider would later become famous as the PC revolution got underway.

In this period, BBN was also working on more engineering-focused computing projects. The practical experience and staff the firm acquired in undertaking these projects gave the firm the much-needed experience and talent it would later need to complete the ARPAnet project. The most important of these projects may have been BBN’s work developing small machine time-sharing systems for hospitals.

BBN’s Hospital Time-Sharing Systems

In 1958, Ed Fredkin had only briefly been at Lincoln Lab before deciding to go into business for himself. To get started, he ordered himself a Royal McBee. He did not yet have the money to pay for the computer, so he went seeking out contracts. He consulted with his friends Frank Heart, then at Lincoln Lab, and Tom Marill, who had recently left Lincoln Lab for BBN, to see if they knew of any customers. Marill told Fredkin to see if BBN would be interested in doing some business with his company. So, he set a meeting with BBN’s new computing czar, J.C.R. Licklider. Fredkin’s meeting with the newly-joined Licklider ended with Licklider pitching Fredkin to, “come work at BBN.” Licklider convinced the partners to purchase Fredkin’s Royal McBee as part of his hiring arrangement. With that, Fredkin was added to Licklider’s growing team of computer engineers.

Licklider’s computing team buildup was still in its early stages, as Fredkin remembers it. If you said “computer,” people would say “analog or digital?” to show they at least knew something about computers. In Fredkin’s estimation, even Licklider was very much still learning the ropes. It was clear that Lick’s higher level ideas were fantastic. But as an engineer, Fredkin had this to say about him:

There was nothing I could do to get Lick to be a good programmer. He insisted on being a coder, and had wonderful high-level ideas; but what he always chose to code never made sense to me. I tried to straighten him out a number of times but couldn’t succeed. It’s just that at that time he didn’t have a knack for coding, either in the style of code or in the things he chose to code.

In short, in Fredkin’s estimation, Licklider seemed to be more bird than frog at that time.

But Fredkin may have had high standards. Licklider referred to Fredkin as a “young genius” and said the decision to bring him in to help grow the computing efforts was a no-brainer. Fredkin, who at the time was not the most fit for a corporate environment, found Licklider and BBN very willing to find a way to make the arrangement work. Licklider remembered that “he was having all kinds of psychological problems about getting work done.” So, BBN worked out an arrangement with Fredkin where, at the end of the month, they would figure out what his salary would be for the month. To Lick, this was just one of BBN’s many “fantastically interesting and flexible arrangements” that it worked out with its sharp employees.

Fredkin was instrumental in convincing BBN to buy the first PDP-1 computer and to bring on professors Marvin Minsky and John McCarthy as part-time employees. Soon after this arrangement began, Fredkin surprised McCarthy by saying that he believed some of McCarthy’s time-sharing concepts could actually be implemented on the rather small PDP-1. Fredkin’s combination of engineering design skills and professional connections from Lincoln Lab helped push this work forward. As McCarthy remembers:

Fredkin designed the architecture of an interrupt system and designed a control system for the drum to permit it to be used in a very efficient swapping mode. He convinced Ben Gurley, the chief engineer for D.E.C, to build this equipment.

Fredkin and his colleagues made multiple technical improvements to prove McCarthy’s ideas could be implemented on such a small machine — including inventing the concept of a swapping drum. Fredkin left the firm in 1961, with McCarthy taking over directing some of Fredkin’s time-sharing projects. But Fredkin’s efforts did a lot to jumpstart BBN’s engineering efforts in time-sharing.

This early time-sharing work eventually led to BBN being hand-picked for an extensive course of funded engineering research. The work came about when Jordan Baruch, one of BBN’s partners, was doing acoustics work for the Clinical Center at the NIH. His project involved instrumentation for the cardiac and neurological staff. During his weekly trips to the NIH, Baruch would go to the home of the Clinical Center’s Director, Jack Masur, for gin and jelly beans. On one of these evening visits the topic of computing in healthcare came up because Masur was asked to give a speech on the topic. Baruch seized the opportunity and shared his vision for the possibilities of computers with patient medical records. Masur used many of Baruch’s ideas in his speech. After the speech was a hit, Masur phoned Baruch and told him, “You should go do it.” Masur recommended Baruch apply for a grant from the NIH’s Division of General Medicine to build the system. BBN did. It won the $1 million (~$10 million today), three-year initial grant.

The contracting relationship would continue over the years, growing from a proof-of-concept prototype to an actual system deployed in a hospital. The specific goal of the research project was for BBN to develop a hospital computer system that would automate “the information gathering, processing, storage and retrieval that takes place in the modern hospital.” The work eventually grew into BBN installing terminals around Massachusetts General Hospital (MGH) which were connected to a time-shared computer at BBN. The initial applications were admissions, discharge, medication ordering, test ordering, and test reporting.

As the project wore on, the medical professors at MGH began using the system for certain types of interactive (albeit primitive) data analysis for their medical studies. BBN staff worked with them to develop these capabilities in the system. As former project manager Paul Castelman noted, “At the 1965 RAND/SDC conference on advanced data-management systems, BBN’s Hospital Research System was the only system reported that operated interactively rather than in batch mode.”

Four years after the start of the MGH project, work on the project began slowly winding down. As a technological proof-of-concept, the project was a success. But it seemed that the system’s deployment in a teaching hospital found difficult reception with some of its (apparently many) opinionated faculty. As project member Bill Mann remembered:

The feelings at MGH ranged from “very interesting” to “it may kill my patients, get it out of here,” with a strong bias towards the latter.

In the mid-1960s, the project’s originator, Jordan Baruch, was leaving BBN to spin off a (BBN-sanctioned) joint venture with GE based on the work called Medinet. The MGH system itself was being handed over to the hospital to operate. In this period, Frank Heart joined BBN as a Vice President in charge of managing technical engineering, with a particular interest in BBN’s life sciences efforts. Heart, in his first year and a half at the firm, helped spearhead BBN’s life sciences computation efforts in various areas of biomedical research.

However, not long after Heart’s joining word was swirling around that Larry Roberts had been brought to ARPA to orchestrate a contract that, as an engineering problem, was too fun for the team at BBN to pass up. Heart and several other Lincoln Lab alumni who had joined BBN in this same period were ideally suited to carry out Roberts’ project. They had a toolkit few in the computing research ecosystem had at the time: deep experience in designing and implementing real-time systems.

ARPAnet Context and Beginnings

The computing team that had been amassing at BBN contained several individuals from the small community specializing in real-time computing systems. In his oral history, Frank Heart explained why so few in the computing community had real-time systems experience, saying:

You know, in that period of time, the computer world was really somewhat divided into people who understood real-time systems and those who didn't. I mean, there were people who built computer systems using operating systems, and there were other people who built very finely tuned machine language programs for dealing with phone lines. And those two camps didn't interact a great deal. So the world was not full of people who knew how to make computers run in real-time, and connect to real-time systems. We were not the only group, but it was a somewhat small universe.

Making computers work via communication lines was no easy task. Larry Roberts, the IPTO PM who oversaw the ARPAnet project, had his own difficulties with the technology while at Lincoln Labs. In his last round of research at Lincoln Labs before joining ARPA, Roberts had been attempting to outline the open computer and communication issues that stood in the way of achieving practical inter-computer communications. As Roberts remembered, the results of this work he’d done with Tom Marill showed “that computers could work with each other, and we figured out how to do that — but we couldn’t get the communications to work on it at all.”

This problem, inter-computer networking, is the primary problem Roberts was brought to ARPA by IPTO Director Robert Taylor to work on. Roberts, who initially rejected Taylor’s offer since he so enjoyed his life as a researcher at Lincoln Lab, eventually had his arm twisted by Lincoln Lab staff. He had turned down Robert Taylor’s offer in early 1966. But, after failing to find another candidate as good as Roberts, Taylor went with something of a Plan B. Then-ARPA Director Charles Herzfeld, at Taylor’s bidding, called up Lincoln Lab’s Director to point out that ARPA was 51% of Lincoln’s funding. Herzfeld then re-emphasized just how much they would like to have Roberts’ services. Roberts accepted. Or, as Taylor summarized these events, “I blackmailed Larry Roberts into fame!”

For reasons beyond the diet blackmail, Roberts was coming around to what he viewed as a bit of a research management job. Roberts explained:

I was also coming to the point of view…that this research that we [the computing group at Lincoln] were doing was not getting to the rest of the world; that no matter what we did we could not get it known. It was part of that concept of building the network: how do we build a network to get it distributed so that people could use whatever was done? I was feeling that we were now probably twenty years ahead of what anybody was going to use and still there was no path for them to pick up…So I really was feeling a pull towards getting more into the real world, rather than remain in that sort of ivory tower.

This general sentiment, coupled with BBN’s openness to interesting research problems if the money was right, is partially why the firm was so successful in recruiting Lincoln Lab staff.

Roberts’ willingness to dedicate the next years of his career to the specific problem of orchestrating the building of the ARPAnet is not surprising. He had started heavily thinking about the problem four years prior. The event that pushed Roberts to begin dedicating his research agenda to the idea was a series of conversations he had at a 1962 computing conference with, among others, J.C.R. Licklider. Roberts said this of the ideas that came up talking with the group:

I came to the conclusion that the next thing, really, was making all of this incompatible work compatible with some sort of networking. In other words, we had all of these people doing different things everywhere, and they were all not sharing their research very well. So you could not use anything anybody else did. Everything I did was useless to the rest of the world, because it was on the TX-2 and it was a unique machine. So unless the software was transportable, the only thing it was useful for was written technical papers, which was a very slow process. So, what I concluded was that we had to do something about communications, and that really, the idea of the galactic network that Lick talked about, probably more than anybody, was something that we had to start seriously thinking about. So in a way networking grew out of Lick's talking about that, although Lick himself could not make anything happen because it was too early when he talked about it. But he did convince me it was important.

According to Robert Taylor, the recent development of time-sharing was the reason he felt his tenure as IPTO chief was a particularly good time to make a push for the ARPAnet project. Taylor said:

The thing that struck me about the time-sharing experience was that before there was a time-sharing system, let's say at MIT, then there were a lot of individual people who didn't know each other who were interested in computing in one way or another, and who were doing whatever they could, however they could. As soon as the time-sharing system became usable, these people began to know one another, share a lot of information, and ask of one another, "How do I use this? Where do I find that?" It was really phenomenal to see this computer become a medium that stimulated the formation of a human community.

The ARPAnet, in Taylor’s mind, was meant to make this “interactive notion” work one level higher — across universities rather than just within them.

Prior to Roberts’ arrival, Taylor had already gotten the necessary funds for the early stages of the project approved and set aside by Director Charles Herzfeld. The next step in the project is common to many major ARPA projects like this. Larry Roberts went around talking to anybody in the ARPA computing community who could conceivably be useful. He wanted to have an understanding of all the opinions from all the top people before he put a concrete proposal out into the world.

Many researchers and engineers contributed ideas that made it into the final proposal, such ideas to improve the network’s host-to-host protocols. But one researcher who was consulted in the planning stages of the RFP can be considered of particular importance. That researcher was Wes Clark of Lincoln Labs — who mentored several BBN employees while they were there. Clark, who his contemporaries considered a bit of an inventive genius, is a bit of an unsung hero of this generation of computing for his work in building early small computers — such as the TX-0, TX-2, and LINC. Clark’s thoughts helped convince Roberts that instead of using a large machine in the center of the country to help “run” the ARPAnet, Roberts should have a small computer at each site handle that. Clark’s advice was singularly important in determining the final structure the ARPAnet took — which was remarkably decentralized.

These small computers Clark proposed came to be known as “IMPs.” In the ARPAnet, the IMPs were meant to connect to both the large host computers at the ARPAnet site as well as the corresponding IMPs at other ARPAnet sites. If they could be successfully made, the use of IMPs would substantially ease the burden involved in connecting host sites to communication lines. Instead of each site having to find a way to connect directly to the telephone lines, they’d simply connect to an IMP. It would be the main contractor’s job to make an IMP reliable and easy enough to use to make the ARPAnet work.

Issued in 1968, the RFP was seeking a prime contractor to design, build, and install these IMPs. In essence, whoever won the contract would be principally in charge of the ultimate success or failure of the project. The ARPAnet nodes — largely research sites with a lot of IPTO funding already — would receive their own ARPA contracts to perform some amount of set-up, debugging, and research on their own end. The IMP contractor would be in charge of building IMPs that would be able to interface with most of the large computers at the time — which were not built to be interoperable. This IMP contractor also had to make sure that IMPs could communicate across noisy telephone lines, across thousands of miles, in real-time. This was the problem that had stumped Larry Roberts on a small scale while he was still at Lincoln Lab. This was a difficult research project as well as an intricate kind of engineering implementation problem.

The team at BBN thought this sounded like great fun. Each member of the proposal team — Frank Heart and Severo Ornstein on hardware, Dave Walden and William Crowther on software, and Robert Kahn on theory — make a point of repeatedly emphasizing that the project just seemed to be great “fun” and a fascinating engineering puzzle to solve. One or two felt it was a problem of massive importance. More often, they did not quite grasp how significant the project would become. Regardless, it was clear that this was a contract BBN wanted in on. So, Frank Heart and Robert Kahn — who had been consulted during the RFP process by Larry Roberts — took the lead in BBN’s proposal efforts. Kahn spearheaded the conceptual aspects of the proposal. Heart, who would eventually be put in charge of the project, spearheaded the more practical engineering aspects.

Kahn, who would later become IPTO chief, had also found his way to BBN around the same time Heart had. In 1966, Kahn took leave from MIT and joined BBN on the advice of a fellow engineering professor. Kahn’s research focus on mathematical aspects of communications was very theoretical. Despite Kahn having worked stints at Bell Labs during his Princeton Ph.D., he was still driven mostly by theoretical problems. So, Kahn’s senior colleague felt he would benefit from additional practical experience. Kahn referred to this piece of advice as “the best advice I ever got.” So, he headed to BBN. He initially intended to be there for only a year or two — he ended up being there for around six years. At BBN, he chose to work on networking.

He described his intentions as to “get my hands a little more dirty with some of the practical problems of everyday engineering.” He seemingly felt BBN was the best place to do this. He described BBN as follows:

It was a marvelous little firm at the time, a fraction of the size it is today. It was largely a group of very professional people doing an extension of what Harvard and MIT were doing, except on a full-time basis, without the responsibility of dealing with students and teaching.

With the RFP out, BBN put far more person time into the proposal than was normal for the firm. Five or six full-time people worked “very, very long hours” on the project for approximately three to six months. As Severo Ornstein remembered, “I spent a lot of time, I remember nights till 3 or 4 in the morning, working with Bob Kahn in the back room of my house in Newton on the proposal — designing the system and figuring out how it was all going to work.” Ornstein also noted that, “more dollars were spent preparing that proposal, more man hours charged to it, than I think had ever been done for any [BBN] project.”

As a reference, the firm strove for 70% of an individual’s time to be billable to some contract in some way. This inordinate use of time on a single proposal was a special case. To be sure, the special case was probably partially driven by the understanding that the contract could lead to longer-term contract extensions. But it seems that the primary reason the team spent as much time as they did was because both BBN employees and management believed this to be an exciting technical problem area — one which the firm felt they knew a lot about.

Many researchers on the team had known Larry Roberts at Lincoln Lab, knew him to be exceptionally smart, and figured he would understand how important their real-time computing experience was. Heart’s Lincoln Lab’s work with the Whirlwind computer required the processing of radar data in real time. As he put it, this meant “the computer had to accept the data at phone line rates and deal with each radar scan before the next one came along.” Heart continues, describing what he believed made his BBN engineering team so attractive in his chapter of “A Culture of Innovation,” saying:

This kind of computer use was unusual at the time, but at Lincoln Laboratory the group of people working with me became unusually expert at the real-time use of computers. At Lincoln, in a long series of projects, computers were connected to various radar antenna systems, radio antenna systems, sensors at underground seismic arrays, and sensors at underwater acoustic arrays. Each such project required a detailed understanding of the computer timing relative to the sequence of data arriving from the various sensor systems. This experience at Lincoln — tying computers to phone lines, and constructing hardware and computer programs that involved the timing constraints of such data handling — was a crucial attribute of my group at BBN that years later bid on and won the ARPAnet contract.

BBN’s main competition for the contract was large contractors like Raytheon and Western Union. Companies like this were significantly less focused on cutting-edge research than BBN but had deep experience managing large systems integration and implementation projects. Roberts stated in an oral history that, on paper, he thought that Raytheon’s bid looked quite reasonable. It wasn’t in a completely inferior league to BBN’s. But Roberts, himself having come up at Lincoln Labs with many BBN staff, did not ignore the advantages that the firm’s structure offered. He recalls:

Raytheon had a good proposal that competed equally with BBN, and the only distinguishing thing in the long run for my final decision was that BBN had a tighter team organized in a way that I thought would be more effective than a very steep commercial structure with lots of managers and vertically – so I didn’t believe they’d keep the schedules as well, but they had a good proposal.

To be clear, ARPA often trusted Raytheon with a variety of implementation contracts. So, it is not likely that Roberts meant that Raytheon would not reliably stick to an implementation schedule. Rather, he likely meant that BBN was far more likely to solve its way through all of the novel engineering challenges that stood between these proposals and an actual system being implemented. This was an extremely novel system. That was likely why BBN won the day.

Of course, all engineering projects are at least, in part, research projects; they all have many “unknown unknowns,” even once they are solved on paper or have been implemented before. Those standard challenges obviously existed in this project. However, the ARPAnet system had so many major “known unknowns” that this was as much a research project as an implementation project. The whole system was an experiment. In fact, the research-focused Robert Kahn stayed on with BBN for three more years after BBN won the contract because of this. There were practical, ongoing research problems — such as those in the area of systems design — that were going to require continuous re-working as the project progressed. As Kahn explained:

My notion originally was "Help them get the award; let them go build it and I will go back to doing what I was doing." But I pretty quickly came to realize that that just wasn't a practical notion. There were too many things that had to be thought about.

According to Roberts, companies like IBM even ‘no bid’ the contract because they didn’t think the request, as it was written, was possible. The price for the computing tasks required, they thought, was completely infeasible with the technology that existed. Roberts remembered:

IBM and CDC 'no bid' it because, they told me: "If we used Model 50s at these sites, you're going to go broke, and we think you're crazy." And what else is there, of course, besides Model 50s?

Many corporate researchers in the communications industry felt that the packet switching idea, which the network’s IMP-IMP communications relied upon, was completely infeasible. To players like AT&T, this was an academic idea that head-in-the-clouds researchers came up with; this was all fine and good on paper, but would just not hold up to the practical engineering reality. BBN obviously disagreed. But, as Frank Heart put it, “There was no proof. You couldn’t go to some book and find the answer; it was an experiment.”

The challenge, which some thought impossible at ARPA’s price point and given the noisiness of telephone lines, was set. BBN won the IMP contract. It was now their job to make it all work.

ARPAnet Operations

The initial one-year contract Roberts awarded BBN was to develop the IMP and deliver a four-node network connecting UCLA, the Stanford Research Institute, UC Santa Barbara, and the University of Utah. The contract was for about $1 million in 1969 (~$8 million today). The initial team that worked on the contract that year was made up of the initial five members who drew up the proposal along with about three more full-timers, primarily to work on hardware, and four or so additional part-timers. Several of the hires were Severo Ornstein’s best students from his course at Harvard.

The proposal team had, at least conceptually, worked its way through many problems in the extensive time it spent working on the proposal. So, the team was able to hit the ground running and make a push to have the four-node network operational by the end of the year. In all likelihood, Roberts likely expected the team might not make the deadline. In many cases, ARPA contracts would set extremely ambitious benchmarks that PMs, office directors, and contractors would later re-negotiate. But this did not turn out to be one of those cases.

Other Contractors on the Project

A number of subcontractors were also involved in the ARPAnet project. Each of the node engineering departments was a subcontractor. To a certain extent, these departments were responsible for getting their host computer set up with the IMP. BBN would do what it could to make this as easy as possible for them. But given the state of the technology and the talent present at the nodes, this was deemed to be the optimal approach. Additionally, ARPA used a contractor called DECCO to take charge of the logistically annoying process of purchasing time from AT&T on their telephone communication lines.2 Of course, DECCO purchased what BBN told them to purchase. Only BBN knew what it needed to successfully hook the IMPs up to the phone lines. But one would imagine BBN was still thankful to have DECCO to keep them somewhat insulated from this process.3

Two other contractors of note were the Network Analysis Corporation and the Network Measurement Center. ARPA wanted the Network Analysis Corporation to run simulations to help design optimal network topologies. In theory, this was reasonable. But, in Frank Heart’s estimation, the company did not prove as useful as ARPA expected in determining which sites to add next. In his eyes, practical matters such as whether or not departments were properly staffed or were ready to work on their IMP were mostly responsible for determining the network’s shape as it grew. The Network Measurement Center was run by Leonard Kleinrock at UCLA — the first node of the ARPAnet — and was responsible for taking a variety of measurements to study the network. The Network Measurement Center was, to Heart, somewhat academic in nature and not pivotal in the initial engineering or design of the ARPAnet. But it was useful in taking measurements pertaining to the health and efficiency of the network in its early stages.

Lastly, Honeywell was the subcontractor most pivotal to the contract’s success. BBN had no manufacturing facilities. Thus, they needed to contract with a computer manufacturer to have an IMP of their design actually built. At the time, few computer manufacturers did this. A standard company’s manufacturing process was not built to do wholly custom jobs. The processes were suited to be fiddled with, but not much more than that. But Honeywell had recently acquired a company called the Computer Control Corporation (CCC) that made them an exception. The team from CCC, as Heart put it, “understood a bit about the special systems business.” This was a unique enough skill that at least two contract bidders had used Honeywell as a subcontractor — BBN and Raytheon. Larry Roberts listed this as a reason that Raytheon’s bid was being considered alongside BBN’s.

Some of the Major Technical Problems

A variety of technical problems awaited the BBN team.

One of the major early problems the BBN team spent its time working through was the error control and correction problem. To work through this, the team was able to design special checksumming hardware that would be installed at both the transmitting and receiving end of the circuit. Every packet of information sent would have a 24-bit cyclic checksum appended to it which would be recomputed by the hardware at the receiving end to determine if there were any errors. If there was, the packet would be retransmitted.

How the nodes got there was the most theoretically-involved piece of this engineering experiment. The packet switching technologies we know, love, and hyper-optimize today were much more un-tested back then. This was one of two areas in which Robert Kahn’s presence proved extremely useful to the engineering-focused team. Kahn was able to quickly explain to them various aspects of packet switching algorithms as problems came up. (The other major theoretical area Kahn contributed to was in some mathematical aspects of phone line error problems.) The technical details of the routing algorithms the team initially deployed are described in the project’s completion report to ARPA as follows:

The approach taken was to use a distributed adaptive traffic routing algorithm which estimates on the basis of information from adjacent nodes the globally proper instantaneous path for each message in the face of varying input traffic loads and local line or node failures. Each IMP keeps tables containing an estimate of the output circuit corresponding to the minimum delay path to each other IMP, and a corresponding estimate of the delay. Periodically, each IMP sends its current routing estimates to its neighbors; whenever an IMP receives such a message it updates its internal estimates. Thus, information about changing conditions is regularly propagated throughout the network, and whenever a packet of traffic must be placed on the queue for an output circuit, the IMP uses its latest estimate of the best path.

Another major area of problems came from the technical aspects of the IMP-Host interfaces. The engineers spent substantial time hitting their heads against the wall to find ways to successfully connect IMPs to all different kinds of computers while, also, minimizing the amount of engineering work required by the node contractors to connect their machine to the IMP.

Solutions to all of these problems had to work in close to real-time. Individuals from remote machines were going to use the network to do much more than just send batch messages like emails. They were going to use their computers to remote into another computer as if the whole network was a single, time-shared machine. So, to make this all work at time latencies that made sense, the BBN team had to do things like “write very carefully tailored programs in machine language to optimize the capacity and the low delay of the data path in the switching node.” The team emphasized in its final report that, “Great attention was paid to minimizing the operating time of the inner loops of these programs.” Many areas of minutiae were all-important.

Many problems — from IMP design, to Host-IMP connection software and hardware, to error-correcting software — required this level of care. All of the high-level concepts — computers sending a message across a telephone line, linking two computers together via a communication line in a lab setting, basic message handling across noisy communications lines, etc. — had been demonstrated as feasible before. But BBN had to invent and engineer all that was required to make it go from technically possible to actually usable.

Viewed naively, the implemented packet switching technology was the team’s major “new” contribution. But, as most engineers will tell you, a project like this can be one big pile of unknown unknowns. That, in part, is probably why industry did not consider touching it in their R&D efforts. But, given the BBN staff’s continuous experience in recent years working on real-world time-sharing systems and real-time computer communications, they were probably the team that had the best sense of what the unknowns of the project might look like and how to deal with them as they came up.

The Honeywell Partnership

To get the IMP designed and built, the BBN team heavily leaned on Severo Ornstein. Ornstein had previously worked with Frank Heart and others on various computer projects at Lincoln Lab, worked with Wes Clark on smaller computers at WUSTL, and had since returned to Cambridge to work at BBN and do some lecturing at Harvard. Prior to the networking project, at BBN Ornstein had been working on projects related to the use of computers in education. Ornstein’s interest in the ARPAnet RFP caused him to dedicate more of his time to BBN.

While designing the IMP during the proposal process, Ornstein estimated that he and the team “did 90% of the design then.” But even when BBN had won the proposal, Ornstein still had to put substantial effort into working with Honeywell to get the computers made. Ornstein remembered:

The man from Honeywell who was assigned to build those interfaces from my drawings didn’t understand the drawings well and was not really careful. We ended up having to redo much of his work.

Much of BBN’s interactions with Honeywell may cast some doubt as to whether an industry actor could have carried out this contract nearly as effectively as BBN had. One account by Ornstein regarding the BBN team working their way through a confusing problem will help elucidate why exactly this is. The story both shines some light on what exactly the BBNers were spending their research time on prior to the implementation stages and how Honeywell dealt with problems. Ornstein explains:

They were not used to doing some of the kinds of things we were doing…For example, we discovered some design flaws in the machine. One of the reasons we chose the Honeywell 516 was that we thought it was a mature machine…that was not going to gives us grief. Well, we were wrong. We pushed it very hard…and we uncovered a bug that they could hardly believe, a synchronizer problem…Synchronizer problems are very, very subtle. We had to dig and dig and dig at them, and finally from their back room they produced a really smart enough guy — they do have a few — but, it was very hard to get this guy. We finally got him…when he came and we sat down and I finally had someone I could talk to who would understand what I was talking about and believe me. It was a subtle problem. The program would run fine in the machine for days on end, and at the end of three or four days suddenly the machine would just die, inexplicably. It was a very, very low frequency failure, so infrequent that you could never look at it on a scope; you could only see the effects afterwards. In fact, we had to build special hardware that beat on the trouble spot many, many times faster than normal usage would. Then finally, with all the lights out in the room, we could faintly see on the tracing scope an occasional failure. That was when the Honeywell people finally became convinced that there was a real problem…We showed them the trivial fix they could make to the machine…So, their people weren't absolutely top-notch people; they were okay. They were industrial-strength people, not research-strength people.

Of course, there were industrial operations with plenty of “research-strength” individuals back then — assuming Ornstein meant individuals with the brains and toolkits to consistently find answers to vexing problems like the one mentioned. Bell Labs, Kahn’s old employer, was just one example.

Not assigning this systems engineering-style contract to an academic department makes sense. Most academic departments are just not staffed and structured in a way to make projects like this feasible. But it’s unclear how an outfit like Raytheon would have done working through problems like the one described above. Would they have been able to solve them? If so, how quickly? We see that the Honeywell team was great at reproducing duplicate machines once they had already made one. But there were certain computer engineering principles and skills utilized in the project that some of the builders — from places like Lincoln Lab and Wes Clark’s WUSTL team — had much more experience with than most other groups. That was to BBN’s great advantage.

All of that being said, the formerly-university-employed engineering team still conceptualized this as more of a novel engineering problem than a research problem. Dave Walden, one of the project’s two founding software engineers who later became General Manager of BBN, described how the team viewed the problem as follows:

My view is that mainly what we were doing was we were very pragmatically doing engineering. We had to send these bits down the wire: how do you put a header on the front; how do you put a trailer on the back. There was theory of how you put error-correcting codes on it. Bob Kahn knew that theory and told us what it was. There were some constraints: this is the way that the 303 (or the 301 or whatever the Bell modem is) has to be interfaced to, but after that it was all pretty pragmatic. Not lots of theory of coming from someplace else.

The Team’s Management

Before diving into how the one-year contract’s implementation went, we should first get a better understanding of how the team was managed at BBN. Firstly, it was much smaller than one would expect.

As the first year of work progressed, the team remained small — about eight people. This is how Frank Heart preferred it. As he saw it, that small team of elite talent in the relevant sub-areas was all BBN needed. He described the project’s work style further, saying:

I think mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

Heart was extremely averse to the team growing so large that communication had to be done on paper. He preferred their style of “very, very frequent interaction on problems.” As Walden remembered, “I don’t remember such a thing as we had a weekly progress meeting. We probably were more in tune with the progress than that; we probably did it hourly.” Heart and the entire, small team did their best to stay technically involved with each other’s work. Heart, as a rule, insisted on technically understanding every bit of the project. The individual team members attempted to do the same, which they felt was quite easy to do. The team all had offices and work stations immediately in the vicinity of each other. As things came up, they’d simply go talk with the relevant individual or call an impromptu meeting.

Early IMP Contributors (Left to Right): Truett Thatch, Bill Bartell (Honeywell), Dave Walden, Jim Geisman, Robert Kahn, Frank Heart, Ben Barker, Marty Thorpe, Will Crowther, Severo Ornstein. Not pictured: Bernie Cosell. Photo courtesy of Wikimedia CommonsThe Early Node Installations

With some level of difficulty, BBN worked their way through problem after problem like those in the technical problems and Honeywell sections. With a lot of persistence, the team got the IMPs specified in the proposal built and de-bugged. By September of 1969, the hardware and software for the first IMP was ready for installation.

Unlike many ARPA computing contractors, the UCLA department had a core of individuals who were particularly excited about the ARPAnet project. While many of the professors who owned computers saw the ARPAnet as an opportunity for ARPA to force them to further split their computer time, Leonard Kleinrock and others at UCLA had research interests relating to the network itself. Kleinrock, most notably, had written about the theoretical aspects of packet switching in his prior work. Because of this interest, he formed the Network Measurement Center to monitor certain aspects of the network as a subcontractor for the ARPAnet contract.

In early September 1969, BBN sent its first IMP off to UCLA. When shipping the finished IMPs to the host sites, BBN even went as far as to have staff ride alongside the machines on the plane, just in case. For these early sites, Ornstein and some of the software researchers like Dave Walden traveled to the sites to hook the IMP up to the phone lines and work through any setup issues. In the lead-up to BBN sending the machines, the firm shared a set of specs with the host sites so the sites could do as much work as possible beforehand. On the node end this work was usually led by some kind of hardware lead and software lead.

BBN had apparently done quite an effective job of communicating its technology to UCLA. Also, UCLA had apparently done a good job of internalizing the information and building accordingly. According to Leonard Kleinrock, within a day of initially connecting the IMP and UCLA’s SDS Sigma 7, messages were moving back and forth.

This was a massive success. Still, many questions could not be answered until the second installation was done. Only at that point could BBN see what happened when two nodes tried to communicate with each other. The next month, the BBN team installed its second node at the Stanford Research Institute — which had also been subcontracted by ARPA to do some research on the network itself. Once BBN and SRI had the IMP hooked up, the two node research teams attempted to send the first message over the ARPAnet. As Leonard Kleinrock remembers:

On October 29th, 1969, one of my programmers, Charley Kline, and I were in this room and we decided to log on to this [SRI’s] machine. Now, to log on, one has to type in “LOG” and the remote machine will type in the “IN” So our job simply was to type in “LOG.” Now, up at the other end, there was another programmer waiting to watch all of this. And we had a telephone connection between these two…so they could talk to each other. What happened is Charley typed the “L” and he asked, “You get the ‘L?’” And the answer was, “Got the ‘L.’” He typed the ‘O.’ “You get the ‘O?’” “Got the ‘O.’” He typed the ‘G.’ “You get the ‘G?’” Wacko! The system crashed. This machine [SRI’s] went down.

So the very first message on the internet ever was “LO,” as in, “Lo and behold!” You couldn’t ask for a better, more effective message.

Surely the BBN team considered it enough of a success that any letters were sent at all. The team proceeded to install a new IMP per month for the rest of the year, making four in total. Problems continuously came up through both BBN’s testing and the node researchers’ testing. BBN fixed the problems as they came up. It does not seem like any of these problems proved particularly devastating.

Kleinrock excitedly described the arrangement as follows:

So we had a four-node network running, and we began to test it and find out what some of the problems were. And we could break this network any time we wanted, we found faults, we had BBN test it, etc.

As problems were worked out, there were regular hardware modifications and software releases that had to be rolled out. The software releases were done, initially, by sending Dave Walden around to each of the sites with his paper tapes to get the software working on the machines. In this way, the first batch of nodes became operational and steadily improved. BBN did this on budget and within the one-year timeline of the contract. Larry Roberts and ARPA were, surely, extremely pleased with the progress. By the end of the year, ARPA contracted BBN to expand the network to 19 nodes.

Before discussing how the project scaled, let’s briefly explore Larry Roberts’ working relationship with the BBN team. Roberts was a very active manager in the ARPAnet project, not just its funder.

Frank Heart with one of the first IMPs. Photo courtesy of “A Culture of Innovation.”Working with Larry Roberts

In putting together the RFP for the ARPAnet, Roberts had looked deeply into many technical points and consulted the best members of the computing community. In doing this, he made several key decisions before the contract was ever in BBN’s hands. As Heart put it, “They [DARPA] picked the baud rate of 50 kilobits, they picked the sites, they picked the issues about the checksums. A lot of the work had been done by DARPA in advance.”

As Dave Walden framed things, BBN could surely be considered the ARPAnet’s operator and the primary driver of the engineering work, but ARPA was the high level manager. This entire operation, the ARPAnet itself, was meant to facilitate new capabilities for ARPAnet contractors. So, Roberts providing a constant guiding hand made sense. Also, the ARPA contracting community contained some of the best minds who could make technical suggestions to the BBN team. So, Roberts and ARPA continually facilitated meetings between BBN, node contractors, and others as they saw fit. The now-famous Network Working Group which debated many ARPAnet-relevant ideas, like what later came to be the TCP/IP protocols, was one example of these efforts.

Once the project was underway, Roberts largely left BBN in charge of making day-to-day decisions. They did not need to seek his approval on many things. But he did keep close tabs with the team and would raise objections from time to time. Roberts could technically “pull rank” on BBN as the project’s funder. But the arrangement did not really work like that. If Roberts didn’t agree with the team’s approach to a particular problem, they would discuss it over the phone or in person until they could come to some kind of agreement. In a contentious case, that might take some number of days. But, in most cases, a clear agreement was reached.

The BBN team held extreme respect for Roberts as an engineer. In turn, Roberts seemed to respect the BBN team greatly. Roberts had a direct line of communication not just with Heart, but with individuals in charge of component technologies on the team such as Severo Ornstein. According to Heart, normally a week wouldn’t go by where Roberts didn’t talk with somebody on the team. Ornstein, who was particularly casual, said he simply conversed with Roberts as he would any other colleague.

Beginning to Scale the Network

The initial contract to build the four-node network was a success. So, ARPA opted to give BBN a contract to spearhead the effort to 19 nodes.

The ARPAnet in December 1969

In early 1970, BBN itself became the ARPAnet’s fifth node as the network continued to expand at roughly one node per month.

The ARPAnet in June 1970

As the expansion continued, new engineering problems continually arose. While Frank Heart was known to be an extremely defensive engineer, things were still bound to work imperfectly. This whole project was an experiment and was a first-of-its-kind network, so one could only be a certain level of defensive without halting the project’s progress entirely.

In many areas, the BBN team shipped technology that it initially felt was merely good enough — with the understanding that they could continually improve component technologies on an ongoing basis. One case of this was the network’s protocols. The initial NCP protocols were deemed by the Network Working Group to be quite sub-optimal.4 So, over the years the NCP Protocols protocols were improved until they were entirely replaced by Kahn and Cerf’s now-famous TCP/IP protocols.

As the ARPAnet completion report notes, on occasion, some of the problems present in the operational network were more than small. In 1970, a major problem was demonstrated with IMP flow control and storage allocation. Node hosts, if they didn’t heed the error, could accidentally halt the network’s entire operation. Yet, while BBN and others took the many months required to fix the problem, the ARPAnet continued to provide adequate service to its nodes.

As Dave Walden recounted in his oral history:

Walden: …it almost immediately became obvious to anybody who hadn't believed it before that what we had originally implemented was insufficient. And what we had specified to the host and what they had implemented had a lot of ramifications. They were now done and now it didn't work all that well. So for the first year or two years, quite a long time, there were rules, informal rules in the network that you didn't pump stuff into it from the hosts so fast it swamped the net. Because we all knew the algorithms didn't work, I mean they broke, patently they broke when you pumped stuff at them too hard. And so this interim solution worked in that network — this is the kind of thing that somebody who is working on proving programs correct and stuff can't imagine. It actually was quite useful in those early days of experimentation because everyone agreed not to break it. You know how to break it; let's not break it. Meanwhile, let's go figure out how to fix it…

Interviewer: So it was on the order of a gentlemen's agreement. Just not to go break it.

Walden: Engineers' agreement.

The ARPAnet in September 1971

Given that all the users were well-trained and capable engineers and researchers themselves, BBN could also get away with only gradually scaling up its service capabilities. Dave Walden running around with his tapes to implement software updates was one example of this. As the network expanded and the network began to operate more reliably, BBN was able to dedicate more of its team’s efforts to helping the network operate a little more automatically.

In these middle years of the project, BBN also began to expand its efforts to finding more efficient solutions to manage network issues, answer questions from nodes, etc. In the beginning, BBN’s answer to this problem was to throw Dave Walden at it. As Walden fondly remembers:

Up until September of '70, I had the program listing beside my telephone at home and would get telephone calls at home whenever something stopped. My telephone number was quite literally on the front of the original packet switches and they called my house over here in Allston and I would talk to whoever was there, using my listings. So that's what happened.

With ARPA sponsorship, BBN began running the Network Control Center (NCC) in service of this goal. In building up the NCC, BBN designed and implemented tools and software to regularly ping the network and keep tabs on network problems, monitor the status of hosts, etc. Alexander McKenzie became the “ARPAnet generalist” BBNer put in charge of the customer service aspects of the project as it grew. McKenzie proudly recounted stories of calling telephone companies and telling them a line was about to go down…and then it would. The companies were quite impressed because this was not an ability that they had yet.

Starting in 1972, BBN hired staff to offer de-bugging and other customer services to network nodes full-time — three people worked the day shift, two the late evening shift, and one the midnight shift. Through BBN’s and the NCC’s efforts, nodes were soon able to (in most cases) automatically report their status. Maintenance and debugging were then typically carried out remotely from the monitoring center. As the NCC’s capabilities grew, BBN was able to successfully place IMPs at locations where the staff had less and less computer training.

Additionally, as the network expanded BBN conducted research in an attempt to expand the types of IMP technology as well as the types of communication lines IMPs could be hooked up to. In 1971, BBN finished designing its first terminal interface message processor (TIP). These terminals were able to connect to the ARPAnet and do work on ARPAnet host machines without having to be run on a host computer of their own. On their own, TIPs were more expensive than IMPs. But the idea was extremely exciting to ARPA because the TIPs becoming operational meant one could hook up to the ARPAnet without having a large computer on hand. The first two TIPs, shown below, were installed at MITRE and NASA’s AMES research facility. The early 1970s saw the first TIPs installed, IMPs connecting to the network via satellite technology, the design of standard IMPs ten times more powerful than the initial ones, and the deployment of Ornstein’s Pluribus multi-processor IMP.

The ARPAnet in August 1972

As the ARPAnet hit its 19th node, ARPA had already opted to give BBN a contract to expand the network to over 50 nodes. The first two years of the project had all occurred largely on schedule and on budget. With this next extension, the network even began to include nodes from remote geographic areas like Hawaii and Norway. This required the use of some of the project’s new technological capabilities as well as the project’s first interactions with foreign telephone operators. While these remote nodes did tend to be more buggy, they were largely functional and proved some of the new capabilities to be useful.

The ARPAnet in July 1975. All ARPAnet sketches are courtesy of the ARPAnet Completion Report.

Throughout this period, Larry Roberts’ continual arm-twisting of node sites to do what they needed to do to make the node installations a success proved extremely useful in keeping things on track. As many of the sites’ primary computing funder, he could exercise this power quite easily. By 1975, the ARPAnet project was clearly on stable footing. So, responsibility for the project was transferred to the Defense Communications Agency (DCA). According to DARPA Historian Richard Van Atta, at the time the project was transferred to DCA the total budget outlays throughout the project had been $25 million in total (~$150 million today). That number covers the entire project, not just BBN’s portion. By anybody’s metrics, this should be considered a remarkably cheap success.

Early version of the NCC. Jim Powers standing. Photo courtesy of “A Culture of Innovation.”The First International Conference on Computer Communication

How exactly the outside world — meaning those in the technology community not affiliated with an ARPAnet host site — became aware of the ARPAnet’s success deserves a brief explanation. Even in 1972, many in the communications community doubted this cross-country packet switching network idea would be technically possible anytime soon.

Robert Kahn’s final project as an official member of BBN helped rectify this. In October 1972, he put on a massive demonstration of the ARPAnet at the first-ever International Conference on Computer Communication (ICCC). The conference itself was an extremely useful forcing mechanism for BBN and each of the node contractors. This provided tangible pressure, with a concrete deadline, to thoroughly debug many pieces of technology, sure-up certain protocols, etc. The academics responded much better to the deadlines or threats of their funding being diminished than they did to the mere presence of bugs in the system. Additionally, the existence of the conference provided a push to ensure that as many host sites as possible were functional across as many use cases as possible. To this point, many nodes had only been outfitted with the hardware and software to use some of the ARPAnet’s possible applications.

Dozens of nodes contributed computer hosts to the demonstration. The conference was a sort of DIY showcase where individuals could hook up to the computer hosts via TIPs and work with them the way node users did every day. This could be for things as simple as e-mail, a rapidly growing use case of the users, as well as much more complicated programs. As Roberts recalled:

I think this was the watershed event that made people suddenly realize that packet switching was a real technology. We had thousands of people who went through that — I don't know what the exact number is — at least a thousand who went through that particular exhibit.

Or, as the ARPAnet completion report put it:

The demonstration itself was a spectacular success; with everything working amazingly well, many visitors remarked that the ARPANET technology "really is real."

ARPAnet Results

On a technical level, the ARPAnet experiment was a success in many ways. The project proved that a large network could be built in such a way that node failures were localized and did not crash the rest of the network. It convincingly demonstrated adaptive routing algorithms and packet switching theories. To a large extent, the ARPAnet proved that a decentralized network like this could run itself without a central command center. Finally, from a user perspective, the IMPs and TIPs proved reliable and easy enough to use to facilitate the time-sharing of diverse computers, thousands of miles from one another, for a variety of tasks. The working ARPAnet also succeeded in its role as a test bed for the entire field of computer communications technology research.

But, of course, the success of the ARPAnet project goes beyond the technical successes of the project. As BBN’s ARPAnet completion report stated:

The ARPANET project also proved the feasibility of achieving closely knit communities of technical interest over a widespread geographic area; it is possible that this social feasibility demonstration is as important as the many technical feasibility demonstrations.

Possibly the clearest indicator of this growth is how rapidly internode traffic grew over the network through the mid-1970s.

Courtesy of the ARPAnet Completion Report.

As the usage of the network grew, BBN feared that even the new algorithms being used to handle routing, flow control, and congestion control would not be able to keep up. But, as the team noted, “Luckily, the improvements in the algorithms managed to stay slightly ahead of the growth in network size and traffic.” In BBN’s estimation, the diminishment in growth rate was because at a certain point, the existing host machines being accessed over the network were simply reaching capacity.

Throughout this growth period, acute problems present in the network lessened, BBN sharpened its processes, demand increased, and more research was being done that relied on the network in some way. Several of the BBNers, in their oral history interviews, talked excitedly about how the network gradually became a “utility.” When prompted by the interviewer to say more, Heart explained:

A utility is something people depend upon. Like the electricity, or the phones, or the lights, or the railroads, or the airplanes. Yes, it was a utility. That's the thing that was the amazing surprise. It was started as an experiment to connect four sites, and it became a utility much, much faster than anybody would have guessed. People began to depend upon it. And that was a problem, because that meant when you changed it, or it had problems, they all got mad. So that was a two-edged sword. But it was also very exciting.

Alexander McKenzie noted that, while the ARPAnet was never as reliable as a power company, the team did eventually succeed in keeping the IMPs up 98% or 99% of the time. As this transition to “utility” took place, NCC-style activities took more of an emphasis. Research and operational activities that optimized the reliability of the network slowly became prioritized over network research activities that created knowledge but interfered with service. This was likely disappointing to some BBNers. But it can also be seen as merely a symptom of the project’s success.

The network was becoming so convenient that a substantial percentage of network traffic was intra-node traffic. Computer hosts at one site were communicating with others in the same location. In other words, communication over the ARPAnet had become, for many tasks, preferable to even walking next door to see a colleague.

In 1972, BBN even began attempting to form a sanctioned spin-off to commercialize the technology in the wider market. ARPA was spiritually supportive of this effort even if they did not fund it. In fact, helping get this firm, Telenet, up and running was Larry Roberts’ first job upon leaving ARPA. He had gone to ARPA to make inter-computer networking a reality. Upon leaving, he was heading into a role where he could attempt to get the technology to as many users as possible. Telenet would go on to have a level of success as a growing network provider — but would not be considered a smashing success. The fact that the ARPA-funded technology showed commercial potential as little six years after the project’s initial funding should also be considered a major success.

If one wanted to nitpick, one could point out that ARPA’s major intended application of the network was as a way to reduce computing costs. This never did become the primary use of the network. As computing costs gradually decreased, this particular use of the ARPAnet became relatively unimportant. The network found better uses.

ARPAnet Lessons Learned (and Caveats)

Many small-scale, PM-relevant lessons are present in this report. As one example, the BBN team maintaining extremely close connections with the academic community while implementing sooner rather than later proved to be a key decision. The academically-driven Network Working Group, with its debates on network protocols in the group’s early years, was at a stalemate regarding what protocols were best. The BBN team could have waited far too long to get the perfect set of protocol recommendations from the group. Instead, BBN implemented the NCP protocols and gradually, with much input from the group, upgraded to the much-improved TCP/IP protocols. Another interesting lesson was Roberts’ use of a demonstration as a forcing function to get all the nodes outfitted with the full suite of ARPAnet capabilities.

Minor learnings aside, it is important that the reader fully appreciate just how important having a contractor like BBN was to the success of the project. Not only was the research and prototyping stage of this difficult project done well and on schedule, but BBN seamlessly transitioned it into a utility. Appreciating why they were able to do this is extremely instructive.

The BBN team was staffed with top minds who cared deeply about the research enterprise but operated within the structure of a firm. Dave Walden, explaining why exactly BBN had earned its nickname as the “third university of Cambridge,” emphasized the following:

BBN provided a support structure for self-motivated researchers who were able and willing to find sponsors for the research they wanted to do. The work was not directed in a top-down fashion; within broad limits, senior researchers were free to pursue their own interests, if they could find the necessary financial backing.

To complement this, BBN also had the necessary staff, experience, and structure to manage successful projects as they grew into larger implementation projects. The firm’s hospital time-sharing work is just one example of BBN managing an implementation project with cutting-edge technology. A project at BBN could start in the deeply exploratory research stage and naturally scale into a utility.

The ARPAnet project showcased exactly what BBN was optimized to do. When given the right kind of problem and the right situation, BBN’s model could help facilitate marvelous feats. The ARPAnet itself, just like Edison’s light bulb 90 years prior, proved to be both a field-changing test bed for scientific experiments as well as a world-changing commercial technology. BBN was ideally suited to make a breakthrough of this nature possible. While the mission creep that comes with a profit motive can often steer a commercial firm off of the course of truly ambitious research, BBN was less susceptible to this in these years. If its research somehow became too derivative, the firm would not have been able to recruit staff anywhere near the competence level it sought.

If every scientific/commercial area had a small research firm like BBN, the scope of what would become possible with an ARPA-like PM’s funding would expand greatly. In one’s role as a PM, if the opportunity to help incentivize or facilitate an organization to operate a little more like BBN, there seem to be few downsides — and massive upside — to do so. The more novel a systems contract becomes, the more suitable it is for a firm like BBN.

BBN created such a compelling, applied environment in which to do research that they convinced many professors to give up tenured positions at MIT to come work at the firm. The firm knew exactly how to make use of these bright, applied researchers. Dave Walden put extreme emphasis on how much the “native wit” of the initial technical team he joined — Frank Heart, Severo Ornstein, and Will Crowther — was required to make the project a success, saying:

They are smart guys. There wasn't much theory in how you build a packet switching network. There was a communications theory, but that was all pretty abstract. One just got out there and did it. All the stuff that is now taught in courses in communications about networks and protocols and all of that, I would say we were mainly (as part of the entire community of the host people) inventing it. The academic analysis tended to come later in a lot of cases.

I’ll conclude with Frank Heart’s fond words describing the organization in “A Culture of Innovation:”

First and foremost, BBN was a great place for a technical person to work, and most people really liked working there. It was a middle ground between academia and the commercial world, with the meritocracy and individual freedom of the academic world, along with the potential reward structure and potential impact on the world of commercial ventures. In the case of the ARPAnet and the subsequent explosion of network activity, it was an extremely unusual opportunity for a technical person to “ride a rocket” of change in the world.”

Thanks for reading:)

Share

Subscribe now

If there is demand, I can write a separate piece on how BBN’s culture and management emphasis changed from those described in the piece as the decades wore on. But I considered that exploration out of scope for this particular piece.

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Building out an experimental test bed for a research field.

• Promoting a coordination/service mechanism to reduce material costs and increase research feedback cycles.

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Specific Links:

The following oral histories and related materials were a joy to read through and primarily what the detailed ARPAnet portions of the piece were based on. The ARPAnet Completion Report was simply used to fill in the gaps. These give a far more detailed account of how things actually worked on the project than the completion report ever could.

• Frank Heart’s UMN Oral History

• Larry Roberts’ UMN Oral History

• Larry Roberts’ Computer History Museum Oral History

• Robert Kahn’s 1989 UMN Oral History

• Robert Kahn’s 1990 UMN Oral History

• Dave Walden’s UMN Oral History

• Severo Ornstein’s UMN Oral History

• Alexander Mckenzie’s UMN Oral History

  • Contains many insights on the customer/user service mechanisms BBN utilized as the network grew in the early-to-mid 1970s.

• Robert Taylor’s UMN Oral History

  • Taylor dives into why exactly he felt his tenure as IPTO chief was a good time to attack the network problem and how he went about the early stages of the work.

• Leonard Kleinrock explaining early ARPAnet node installation and testing

• The History of Telenet and the Commercialization of Packet Switching in the U.S.

  • For more on the early history of commercial packet switching, check out this IEEE paper in which Larry Roberts is a co-author

• Licklider et al.’s Libraries of the Future Report

• J.C.R. Licklider Oral History

General Links:

• ARPAnet Completion Report

  • This report was a treasure trove of information related to contracting details, specifics on technical accomplishments, and various graphics summarizing ARPAnet’s growth.

  • The report also highlights how the actual work deviated from the plan, what was added/removed, etc. as the project went on.

  • Lastly, the report lists many of the technical problems the BBN team had to work through at various points.

• A Culture of Innovation: Insider Accounts of Computing and Life at BBN

  • Besides being the source of many quotes and pieces of information in the piece, this is the most thorough account of how BBN actually operated on a day to day basis. Each chapter is written by a different, long-time (and often high-ranking) BBN employee.

  • As one example, the book contains a long section from Frank Heart on how overhead rates impacted project selection and how BBN’s billing system worked. Explanations like this for all sorts of aspects of the operation are included in the book. 

1

The paper is Black 1963.

2

DECCO was a contracting unit of DCA in communication services

3

Even though the Bell representative assigned to the project was apparently proactive and quite excellent.

4

Thanks to Dan Tappan for flagging that it was NCP protocols, not “Telnet protocols.” See his comment as reference