# Source: https://www.answer.ai/posts/2024-01-26-freaktakes-lessons.md

# Lessons from history’s greatest R&D labs
Eric Gilliam
2024-01-26

> **Note from Jeremy**: *There are few things more important to our
> civilization than understanding how to better do R&D. Thankfully, Eric
> Gilliam has dedicated himself to studying this question. As a result,
> he’s become the foremost scholar and historian of 19th and 20th
> century R&D labs. I thought I was fairly well informed when it comes
> to the history of these labs, but after talking to Eric, I quickly
> realised I’m a rank amateur by comparison! Eric’s knowledge of the
> history of modern research and development is unparalled, and I found
> his insights into why some organizations were dramatically more
> effective than others to be utterly compelling. Therefore, I asked him
> for his totally honest assessment of our plans for Answer.AI, based
> both on our written plans and a number of in-depth conversations we
> had together. Today we are publishing a guest post with his analysis,
> cross-posted from
> [FreakTakes](https://www.freaktakes.com/p/lessons-answerai-can-learn-from-historys),
> his blog.*

<audio controls>
  <source src="2024-01-26-freaktakes-lessons.mp3" type="audio/mpeg">
  Your browser does not support the audio tag.
</audio>

- This article is also available in AI-generated spoken audio format –
  play directly in the above player, or click the menu on the right to
  download the mp3 file. Skip to 3m20s to get past the intro section and
  jump to the main article.

Jeremy Howard (former President and Chief Scientist of Kaggle) and Eric
Ries (creator of The Lean Startup movement and Long Term Stock Exchange)
have teamed up to found a new applied R&D
lab: [Answer.AI](https://www.answer.ai). When speaking with Jeremy, he
made it clear that many details of Answer.AI’s structure are still being
worked out. Only announced a month ago, the org is still in its early
development stages. But the founders have conviction on certain
principles. The most prominent of them is one extremely relevant to my
regular readers: The founders seem to be particularly inspired by
Edison’s Menlo Park Lab and the early days of commercial electric
research.

In the piece, I’ll briefly examine the (working) plans for the lab and
do some historical analysis, detailing:

1.  What the earliest electrical R&D labs can teach Answer.AI
2.  Useful rules-of-thumb from other historically great applied R&D labs
3.  Potential pitfalls to keep in mind as they move forward

You can find more thorough historical evidence in my prior pieces for
any of the lab details I mention, listed below:

- [Thomas Edison,
  tinkerer](https://worksinprogress.co/issue/thomas-edison-tinkerer/)
  published in *Works in Progress*
- [Tales of Edison’s
  Lab](https://www.freaktakes.com/p/tales-of-edisons-lab) (podcast)
- [Irving Langmuir, the General Electric Research Laboratory, and when
  applications lead to
  theory](https://www.freaktakes.com/p/irving-langmuir-the-general-electric)
- [How did places like Bell Labs know how to ask the right
  questions?](https://www.freaktakes.com/p/how-did-places-like-bell-labs-know)
- [“The Third University of Cambridge”: BBN and the Development of the
  ARPAnet](https://www.freaktakes.com/p/the-third-university-of-cambridge)
- [An Interview with Chuck Thorpe on CMU: Operating an autonomous
  vehicle research
  powerhouse](https://www.freaktakes.com/p/an-interview-with-chuck-thorpe-on)
- [ILLIAC IV and the Connection
  Machine](https://www.freaktakes.com/p/illiac-iv-and-the-connection-machine)
- [A Progress Studies History of Early MIT — Part 2: An Industrial
  Research
  Powerhouse](https://www.freaktakes.com/p/a-progress-studies-history-of-early-001)
- [How Karl Compton believed a research department should be
  run](https://www.freaktakes.com/p/how-karl-compton-believed-a-research)

Each of the orgs listed has lessons to teach Answer.AI. But none are a
perfect analog. So, as the piece progresses, I’ll explain which lessons
I think most strongly apply to Answer.AI. With that, let’s get into it!

<figure>
<img src="the-boys.jpg" class="preview-image" width="500"
alt="Edison and “the boys” preparing the First Practical Incandescent Lamp for Testing at Edison’s Menlo Park Lab. Illustration by Harry K. Flemming. Photo Courtesy of the Henry Ford Museum" />
<figcaption aria-hidden="true">Edison and “the boys” preparing the First
Practical Incandescent Lamp for Testing at Edison’s Menlo Park Lab.
Illustration by Harry K. Flemming. Photo Courtesy of the Henry Ford
Museum</figcaption>
</figure>

## [Answer.AI](http://Answer.AI) in a Nutshell

Jeremy’s blog post [announcing
Answer.AI](https://www.answer.ai/posts/2023-12-12-launch.html) makes it
clear that the org is, to a large degree, inspired by the field of
electricity’s path of progress in the 1800s. He believes the current
state of the AI field is similar to the state of the electricity field
between the work of Michael Faraday and Edison’s lighting projects. This
was an era in which new electrical findings were being pieced together,
but few had made any progress in turning the potential of electricity
into great applications.

I don’t find this comparison crazy. So far, I don’t believe AI has come
close to the level of breakthrough that electricity proved to be.
Electricity brought the sunlight indoors for a negligible cost *and*
powers so many of our modern conveniences— refrigeration, TVs, central
heating, etc. That’s a high bar. *However*, given that human ingenuity
created the breakthrough that was electricity and each of those
applications, it is surely worth considering that AI *could* grow to be
the most impactful field of them all. Whether AI does reach that level
of promise, to me, is a question of human ingenuity. So, I have no issue
with Jeremy comparing the AI field to the electrical field c. 1830 to
1910.

With that elephant out of the way, let’s briefly examine what sets
[Answer.AI](https://www.answer.ai) apart from AI labs like OpenAI and
Anthropic. From a funding perspective, Answer.AI seems much, much
cheaper. The founders have initially raised USD10 million. This stands
in stark contrast to the gargantuan initial rounds of OpenAI and
Anthropic. Also, Answer.AI’s research agenda is more
application-centric. The following excerpt from Jeremy’s blog post
highlights what he thinks differentiates the lab’s approach:

> At Answer.AI we are not working on building AGI. Instead, our interest
> is in effectively using the models that already exist. Figuring out
> what practically useful applications can be built on top of the
> foundation models that already exist is a huge undertaking, and I
> believe it is receiving insufficient attention.
>
> My view is that the right way to build Answer.AI’s R&D capabilities is
> by bringing together a very small number of curious, enthusiastic,
> technically brilliant generalists. Having huge teams of specialists
> creates an enormous amount of organizational friction and complexity.
> But with the help of modern AI tools I’ve seen that it’s possible for
> a single generalist with a strong understanding of the foundations to
> create effective solutions to challenging problems, using unfamiliar
> languages, tools, and libraries (indeed I’ve done this myself many
> times!) I think people will be very surprised to discover what a small
> team of nimble, creative, open-minded people can accomplish.
>
> At Answer.AI we will be doing genuinely original research into
> questions such as how to best fine-tune smaller models to make them as
> practical as possible, and how to reduce the constraints that
> currently hold back people from using AI more widely. We’re interested
> in solving things that may be too small for the big labs to care about
> — but our view is that it’s the collection of these small things
> matter a great deal in practice.

It would be unfair to say that an application-centric research agenda is
necessarily less ambitious than AGI. Those biased toward basic research
might say so, but I don’t think that opinion is very
historically-informed. Edison himself was application-centric above all
else. His deep belief in market signals is fascinating when juxtaposed
with the market indifference of many great academic physicists. In the
book *[From Know-How to Nowhere](https://amzn.to/3HoUUbW)*, a history of
American learning-by-doing, Elting Morison described the interesting
nature of Edison’s motivations:

> If the means by which he \[Edison\] brought off his extraordinary
> efforts are not wholly clear, neither is the cause for his obsessive
> labors. No diver into nature’s deepest mysteries carrying next to
> nothing for the advancement of knowledge and even less for the world’s
> goods, he would become absorbed in making something work well enough
> to make money. The test in the marketplace was for him, apparently,
> the moment of truth for his experiments.

Edison built his god-like reputation by dreaming in specific
applications. He kept market, resource, and manufacturing constraints in
mind from the earliest stages of his projects. Edison dreamed practical,
realizable dreams. And when the limitations of component technologies
stood in the way of his dreams, he often had the talent to invent new
components or improve existing materials. Edison’s biggest dream, the
light bulb, mandated that  Edison solve a much broader set of problems.
The following excerpts from my [Works in Progress piece on
Edison](https://worksinprogress.co/issue/thomas-edison-tinkerer/) paint
a clear picture of his ambitious but practical dreams:

> After Edison’s bulb patent was approved in January 1880, he
> immediately filed another for a ‘System of Electrical Distribution’.
> Filing for these so close together was no coincidence. To Edison, it
> was never just a bulb project. It was a technical business venture on
> a possibly unprecedented scale. Edison wanted to light up homes all
> over the world, starting with lower Manhattan.
>
> Bringing the project from dream to mass-market reality would require
> solving over a hundred technical problems. His was a new bulb that
> needed to be powered by a generator that did not yet exist at the
> start of the project, strung up in houses that had no electricity,
> connected via underground street wiring that was only hypothetical,
> and hooked up to a power station that had never existed before.
>
> Yet, at the end of two years’ time, Edison would do it. And, just as
> importantly, the entire venture was profitable by the end of the
> project’s sixth year.

Edison was clearly doing a different kind of dreaming than those who do
basic research. His lighting work embodies what extreme ambition looks
like in application-centric research. Answer.AI making this kind of
ambitious, applied work their North Star is an extremely interesting
goal.

This goal has the potential to give Answer.AI a comparative advantage in
the growing space of for-profit AI labs. For example, the most ambitious
aspects of OpenAI are considered to be in its research, not its work on
applications. Answer.AI’s particular setup can also set it apart from AI
startups and academic labs. New AI startups do some research on how to
commercialize new AI models in new ways, but they generally have short
runways. In this kind of environment, only specific types of research
projects can be pursued. Academic labs — for many reasons covered
elsewhere on my Substack (such as in the [ARPA
series](https://www.freaktakes.com/s/arpa-playbook)) — don’t have the
right combination of incentives, experience, and staffing to build new
technologies in most problem areas. The main incentive of the
profession, in a simplified form, is producing many paper studies that
get cited many times. Answer.AI has the chance to let its alternative
focus lead it to areas under-explored by academics, companies with brief
timelines to hit revenue benchmarks, and more AGI-focused R&D labs.

Legally, Answer.AI is a company. But in practice, it might hover
somewhere between a lab and a normal “profit-maximizing firm” — as was
the case with Edison’s lab. The founders seem perfectly content to
pursue high-risk projects that might lead to failures or lack of revenue
for quite a while. In saying this, I do not mean to imply they are
content to light money on fire doing research with no chance of a
return. Rather, they hope to fund a body of research projects that
ideally have positive ROI in the long term. They are just not overly
concerned with short-term revenue creation.

*(Making the pursuit of research agendas like this easier is actually
one of the founding goals of Ries’ Long Term Stock Exchange — which I
address later.)*

There is apparently no pressure to produce a product that can hit
software VC-style revenue goals within 12-24 months, or anything
similar. This is good. Seeking to satisfy these types of metrics does
not traditionally permit a company to act like a truly ambitious R&D
lab. I’m not saying it can’t happen — DeepMind seems to have made it
work in its early years — but it does require pushing against investor
pressure quite strongly. The VC money raised for Answer.AI has left the
founders with enough voting shares that investors can’t veto founders’
decisions. Additionally, Howard says the company’s investors understand
what they are trying to build is, first and foremost, a lab. This is a
great step towards building an organization focused on building very
useful, very new things rather than the most profitable thing possible —
which often comes with bounded technical novelty.

Interestingly, Answer.AI will also keep a small headcount. Jeremy built
Fastmail up to one million accounts with only three full-time employees.
He hopes to keep the Answer.AI team exceptionally talented and
“ruthlessly small” in a similar way; he believes keeping teams small is
important to building new, technically complex things.

Now that I’ve outlined some important pieces of Answer.AI’s vision, I’ll
dive into the historical analysis. In the first section, I detail
lessons that Answer.AI can draw from both Edison’s Menlo Park laboratory
and the Early GE Research Laboratory. In the following section, I’ll
share useful lessons from other historically great industrial R&D labs.
Lastly, I’ll highlight the bureaucratic details that explain why the
operational models of the great industrial R&D labs have not been
replicated often.

## Learning from the First Electrical R&D Labs

I find it exciting that Edison’s Menlo Park lab is a North Star for
Answer.AI. I covered Edison’s work in several pieces because I think
evergreen lessons can be drawn from his work. *But* I think a more
complete way to incorporate lessons from the 1870-1920 electrical space
is to draw on the work of both Edison’s Menlo Park Lab *and* the young
GE Research Lab. The latter operated as a more traditional industrial
R&D lab. GE Research’s history holds many lessons to help steer
Answer.AI’s problem selection and work on its standard projects.
However, *exceptionally ambitious* projects may draw more heavily on the
lessons of Edison’s lab.

*(As a note, while Edison General Electric was one of the two companies
that merged to become GE — along with Thomson-Houston Electric — Edison
had essentially nothing to do with the formation of the iconic GE
Research Laboratory.)*

Different types of projects characterized the work of the two electrical
labs. When it came to electrical work, for years, Edison’s lab and
mental efforts were focused on doing everything necessary to bring a
single, revolutionary product to market. On the other hand, GE Research
usually had many separate courses of research underway at once. These
projects all sought to improve the science and production of existing
lighting systems, but they were otherwise often unrelated to each other.
Additionally, GE’s work could be categorized as more traditional
“applied research.” The lab was not actively looking to create a field
of technology from scratch as Edison did. GE Research’s projects were
often novel and ambitious, but in a different way than Edison’s.

Later, I will explore the types of novelty the GE Research Lab pursued.
First, I’ll give the reader a more fine-grained idea of how Edison’s
lighting project actually operated.

### Lessons from Edison’s Work on Electricity

Edison’s lighting work provides great management lessons for those
looking to direct a large chunk of a lab’s efforts toward a single, big
idea.

Edison’s major contribution to the field of electricity was not
inventing each of the components in his lighting system, but in turning
a mass of disparate gadgets, scientific principles, and academic
misconceptions into a world-changing system. The burden of doing “night
science” — [as Francois Jacob refers to
it](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1800-6)
— largely fell on Edison. In the late 1870s, nobody knew much about
electricity yet. The existing academic literature had more holes than
answers, and many of its so-called “answers” turned out to be wrong or
misleading. From this shaky starting point, Edison proceeded. He
combined his unique mix of attributes and experience to deliver a
world-changing system. These included: knowledge of several adjacent
scientific fields, deep knowledge in then-overlooked experimental areas,
market knowledge, manufacturing knowledge, and the ability to adequately
operate a small research team.

In large part, Edison created his lab as a way to scale himself. As a
result, to understand how his lab operated, one needs to know how Edison
himself carried out his explorations. Edison was one of the more
stubborn experimentalists of all time. He spent most of his waking hours
carrying out one experiment or another. While he did pore over
scientific literature, for him, nothing was settled until he proved it
for himself at the lab bench.

I write in my *Works in Progress* piece:

> Edison respected scientific theory, but he respected experience far
> more. In Edison’s era of academia as well as today’s, many professors
> had a certain preference for theory or ‘the literature’ over hands-on
> improvement. Because of this Edison did not care much for professors.
> He was even known to go on long diatribes, during which he had
> assistants open up textbooks, locate scientific statements that he
> knew to be untrue from experience, and quickly rig up lab
> demonstrations to disprove them. ‘Professor This or That will
> controvert \[dispute with reasoning\] you out of the books, and prove
> out of the books that it can’t be so, though you have it right in the
> hollow of your hand and could break his spectacles with it.’

Contained in his head was a database of countless experiments and
results that made it seem as if his “intuition” was far beyond his
contemporaries. This left him with an unparalleled skillset and body of
knowledge. If anyone could feel comfortable pursuing a project that
others had previously failed at, it was Edison. Edison’s confidence in
his skills was never more on display than when he chose to pursue his
lighting work. Many in the scientific establishment knew electric bulb
lighting was technically possible, but claimed they had proven that it
could never be economical. Edison disagreed.

On top of Edison’s admirable approach to experimentation, he brought a
high level of practicality to his process. He knew his inventions needed
to make commercial sense in order to make it out of the lab. So, even in
early courses of experimentation, he kept factors like manufacturability
in mind. He wouldn’t commit much time to something that didn’t make
commercial sense. With that being said, Edison wanted to change the
world with his technologies more than he wanted to get rich. So, the
practical factors he paid aggressive attention to were primarily treated
as constraints. He did not optimize for profitability, but he knew his
ideas needed to be profitable. Nobody who wanted to optimize for profit
would have pursued lighting in the way Edison did. The technical risks
were too great.

Edison was able to imagine an ambitious system that required many
technical advances. It was so futuristic that maybe only he was capable
of coming up with it. But just as impressively, he was able to do it
profitably and on schedule. His dogged commitment to experimentation
seems to be largely responsible for this. Edison and “the boys”
constantly experimented on every piece of the process to improve and
learn more about all the sub-systems in Edison’s grand system. They
wanted to know how every piece of every sub-system performed in all
conditions. I’ll share just two excerpts from my *Works in Progress*
piece as examples.

The first is from Edmund Morris’ biography of Edison. It recounts how
thoroughly Edison and his trusted aid, William Batchelor, were in
carrying out round after round of filament experiments:

> For week after week the two men cut, planed, and carbonized filaments
> from every fibrous substance they could get — hickory, holly, maple,
> and rosewood splints; sassafras pith; monkey bast; ginger root;
> pomegranate peel; fragrant strips of eucalyptus and cinnamon bark;
> milkweed; palm fronds; spruce; tarred cotton; baywood; cedar; flax;
> coconut coir; jute boiled in maple syrup; manila hemp twined and
> papered and soaked in olive oil. Edison rejected more than six
> thousand specimens of varying integrity, as they all warped or split…
>
> In the dog days, as heat beat down on straw hats and rattan parasols,
> the idea of bamboo suggested itself to him. Nothing in nature grew
> straighter and stronger than this pipelike grass, so easy to slice
> from the culm and to bend, with its silicous epidermis taking the
> strain of internal compression. It had the additional virtue, ideal
> for his purpose, of being highly resistant to the voltaic force. When
> he carbonized a few loops sliced off the outside edge of a fan, they
> registered 188 ohms cold, and one glowed as bright as 44 candles in
> vacuo.

This approach went far beyond bulb filaments. The following excerpt
describes the work of one of Edison’s lead mechanics in turning the
Menlo Park yard into a 1/3 scale model of what they would later install
in Lower Manhattan. I write:

> \[Kruesi, Edison’s mechanic\] along with a group of engineers and a
> team of six diggers, turned the excess land of the lab in Menlo Park,
> New Jersey…into a one-third-scale model of Edison’s first lighting
> district in lower Manhattan. This team tested and re-tested the
> electricity delivery system, digging up Menlo Park’s red clay to lay
> and re-lay an experimental conduit system. The team carried out
> countless tests to ensure that they found materials to efficiently
> carry the electric current while also keeping the delicate materials
> safe from water and ever-present New York City rats.

The entire process was marked by the classic trial-and-error of the
Edisonian process. The first subterranean conducting lines and
electrical boxes the group laid were completely ruined by two weeks of
rain — despite being coated with coal tar and protected with extra wood.
While the diggers dug up the failed attempt so the damage could be
examined, Kruesi and a young researcher…studied and tirelessly tested
unbelievable numbers of chemical combinations — making full use of the
laboratory library and chemical room — until, finally, a blend of
‘refined Trinidad asphaltum boiled in oxidized linseed oil with paraffin
and a little beeswax’ was found that protected the electrical current
from rain and rats. \>

Edison built his own style of dogged experimentation into the culture of
his lab. Since the lab was meant to scale Edison, this makes perfect
sense; he was a man with far more ideas than hands. So, he hired more
hands. Edison did not search far and wide to hire the world’s best
research minds, and many of those he employed did not even have
scientific backgrounds. This didn’t matter much to Edison because most
of them were employed to undertake courses of research that he had
directed them to pursue. A couple of his Menlo Park employees had
advanced scientific degrees, but far more did not. For the most part,
the lab and its activities were steered by Edison and his ideas. As a
result, the productivity of his lab followed wherever his attention
went. After some time working on a project area, Edison would often grow
antsy and wish to move on to the next thing — he craved novelty. The
lab’s resources and extra hands would move with him. As we’ll see in the
next section, this stands in stark contrast to how the GE Research Lab
recruited and chose problems.

Menlo Park’s electrical activities provide a great management playbook
for what it looks like to direct a lab’s efforts toward a single, major
system. If Answer.AI does not want to go all-in on one thing, it can
still find a way to apply this playbook to a certain focused team of
employees while leaving the others to tinker around with
exploration-stage ideas. In Edison’s less-focused experimentation
periods, his lab served as more of an “invention factory,” doing this
sort of fiddling. Additionally, Edison’s preference for application and
commitment to experimentation over theory in a young area of science can
surely provide Answer.AI some inspiration.

Of course, Edison did some things better than others. Edison’s most
easily-spottable “deficiency” is that his lab was largely dependent on
him. Without him and his big ideas, the lab would have probably ground
to a halt. While Edison’s technical vision, practicality, and
experimental approach are absolutely worthy of emulation, the lessons of
GE Research should probably be added into the mix as well. GE operated
as more of a prototypical industrial R&D lab with an approach quite
suited to the fact that the science of electricity was beginning to
mature in the early 1900s.

### Lessons from the Young GE Research Laboratory

The young GE Research lab took a different approach to electricity
research than Edison. The lab worked on many unrelated projects at once,
recruited more talented researchers, and allowed these talented
researchers more freedom to exert the scientific method on
commercializable projects. The lab did not undertake projects that were
as purposely futuristic as Edison did. Nobody from the lab earned
nicknames like “the Wizard of Menlo” or “the Wizard of Recorded Sound.”
But early GE Research was still responsible for a Nobel Prize and making
the light bulb a much-improved, more cost-effective technology.

Elting Morison wrote the following on the lasting impact of GE
Research’s early decades:

> There seems little doubt that…much that was done in Schenectady in
> electrical engineering and some parts of physics was both better done
> and more interesting than what was being done in those fields in any
> American university.

In its heyday, even great researchers like Karl Compton hoped to shift
their academic departments to operate more like GE Research.

While GE did simultaneously pursue diverse projects, there was a
unifying thread holding all of the projects at GE Research together.
Each project aimed to improve the quality and profitability of GE’s
products and manufacturing. Under that unifying theme, all kinds of
projects were encouraged. Much of the research was very applied,
particularly in the early years when the lab was still proving itself.

William Coolidge was one of the lab’s most talented applied researchers
in its early years. Coolidge joined the lab in 1905, part-time while
teaching courses at MIT. Coolidge had the kind of toolkit typical of
many MIT professors in that era. He had a far greater grasp of the
science of physics and metallurgy than somebody like a blacksmith; he
was simultaneously far closer to a blacksmith than one would ever expect
a university researcher to be. With this differentiated toolkit, he did
science in a way that was not typical of academics. In describing the
process that led to his successes at GE, he claimed that he was, “guided
in the main by experiment itself rather than by metallurgical
knowledge.”

Willis Whitney, the founding Director of GE Research and former MIT
professor, recruited Coolidge to build on findings Whitney himself had
made. Whitney’s initial course of research had found an improved
metalized carbonized cellulose filament for bulbs. Whitney’s results
proved very profitable for the lab. It seemed reasonable that an actual
metal filament could perform even better. Whitney thought Coolidge and
his metal-working skills were well-suited to pursue the area further.

Coolidge expertly applied practical skills in concert with scientific
knowledge to pursue the problem. Elting Morison described a small sample
of Coolidge’s workflow:

> He suspended tungsten powder in an amalgam of bismuth, cadmium, and
> mercury. He then passed the resulting substance through tiny dies —
> drawing it — and obtained a silvery pliable wire. At that time, he
> thought he had reached ductility and the search was over. But when a
> current was passed through this wire the mercury, cadmium, and bismuth
> distilled out, leaving, unfortunately, a nonductile tungsten. But it
> also proved to be tungsten in the purest state he had yet produced.

I continue in my FreakTakes piece, writing:

> He eventually iterated his way to a workable process where…the more
> pure tungsten was put through a specific combination of metal-working
> processes at a temperature that worked that produced rods of tungsten
> about 1 mm in diameter. These 1mm rods could then be drawn and
> re-drawn through rods of decreasing size until you were left with
> wires of tungsten .01 mm in diameter. When put in the vacuum-sealed
> bulb, electricity ran through the tungsten filaments and demonstrated
> an efficiency of 1 watt per candle — extending the life of a bulb up
> to 27x.
>
> Within 5 years, 85% of all lamps would be made from tungsten. As the
> project went on, more and more research chemists and technical
> assistants grew to be involved in the wide-ranging steps and
> combinations involved in Coolidge’s experiments. But it worked. GE had
> the factories re-fit and deployed the new bulb. Coolidge moved on to
> other research.

The success of Coolidge’s hybrid work style, not dissimilar to Edison’s,
is surely a useful data point to Answer.AI. But GE Research also did
work that went far beyond Coolidge’s technically adept, applied science.
The lab was fantastic at making use of talented individuals who were
very academic. Irving Langmuir was a prime example. I described his
interests in my original piece:

> It should be noted…Langmuir did not even care about lightbulbs. Well,
> I guess that is not technically true. The bulb interested him
> because 1) he thought a metal like tungsten was cool because it could
> accept really high temperatures which opened up options to the
> scientist working with it and 2) these vacuum-sealed bulbs provided a
> pristine environment for controlled scientific investigations.

To Langmuir, light bulbs were primarily a playground in which to do his
science. But Willis Whitney knew how to take an individual like that and
direct his energy towards productive ends. The lab deployed a principle
that I call extending a “long leash within a narrow fence” to basic
researchers like Langmuir.

The way the lab facilitated this was rather simple. On his first day,
Langmuir was told to walk around the applied end of the lab and ask
people about their projects. Whitney permitted him to undertake any
course of investigation of any phenomenon he wanted, *but* it had to be
directly related to an existing problem/limitation/constraint that the
applied folks were working through. These applied folks were working on
projects that rather directly plugged into GE’s operations, so there was
minimal risk of Langmuir’s work not amounting to anything useful if he
succeeded and found answers. With that assurance of applicability,
Langmuir was given extensive timelines to find answers to open
questions.

Langmuir’s first course of research focused on the constant
bulb-blackening problem common to bulbs at the time. The problem was
generally attributed to a bulb’s imperfect vacuum. Langmuir found this
problem to be a great excuse to carry out a course of experimentation he
found interesting. Morison described Langmuir’s thought process as
follows:

> If residual gases — imperfect vacua — produced a bad effect —
> blackening — here was a fine opportunity to study the effects produced
> by different gases introduced one by one into the bulb. What he wanted
> to do, he told Whitney, was simply to plot the interactions of various
> gases exposed at low pressures to very high temperatures in the
> filament. Nobody knew very much about this phenomena and he wanted to
> look into it simply “to satisfy \[his\] own curiosity.”

Langmuir carried out this course of research over three years. There
were many gases and temperatures to test, which took time. But
unforeseen results constantly took Langmuir off in different directions.
Exploring these unforeseen results often entailed new courses of
experiment altogether. With his long leash, Langmuir was able to figure
out that imperfect vacua were not what caused bulb blackening at all.
Rather, it was that tungsten vapor particles were finding their way onto
the wall of the bulb. Temperature was the issue.

He also discovered that different gases markedly changed the rate of
evaporation. One extreme example was nitrogen, which reduced the
evaporation rate by 100-fold. *However*, adding nitrogen to the bulbs
caused the electrical efficiency of the system to decrease drastically.
So, the existing bulb design with nitrogen added was less cost-efficient
than the normal bulbs. But Langmuir was undeterred. This was progress.

Existing fundamental research in this area led him to believe that this
efficiency issue could be alleviated by increasing the diameter of the
filament. Further experimentation proved this to work. He also found
that coiling the filament in a certain way could mitigate the heat loss
issue. The final result was a novel bulb that used an inert gas instead
of a vacuum to reduce bulb blackening. Along with the coiled tungsten
filament, this new bulb only required .5 watts per candle and lasted
three times longer than any other bulb.

Once he passed the bulb project onto the engineering team at GE
Research, Langmuir set his sights on an anomaly he had come across
talking with the lab’s more applied staff. The bulbs in the lab had a
design that depended on only a few milliamperes of current flowing
across the space between one end of the filament and the other. Langmuir
noted this anomaly in a letter to *Scientific Monthly*, writing:

> This fact seemed very peculiar to me, for the work of Richardson and
> others had indicated that at temperatures as high as those used in the
> tungsten-filament lamp, currents of many amperes should flow across
> the space. In other words, according to the then-accepted theory of
> the electron emission from hot filaments, a serious difficulty should
> have been encountered in the construction of tungsten-filament lamps.
> The fact that we did not meet any such difficulty therefore seemed to
> me a peculiar fact that should be investigated.

In the brief course of exploration that followed from Langmuir, he
discovered what is now known as the space-charge effect. This work
combined with follow-on work from Coolidge to produce an entirely new
kind of GE X-ray tube.

Under this “long leash within a narrow fence” guideline, Irving Langmuir
would go on to be partially responsible for a handful of new and
improved product lines at GE. Additionally, the knowledge he created
with his tungsten filament work went far beyond padding GE’s balance
sheet. Over the course of his project, he noted that the way tungsten
vapor condensed did not gel with existing academic theory. His
subsequent exploration of this phenomenon led Langmuir to be credited
with founding the field of surface chemistry. Langmuir earned himself a
Nobel Prize for his efforts.

There was a symbiosis in the GE lab between Langmuir types and the
Coolidge types — the latter skillset being more standard in the lab. I
imagine Answer.AI will have no shortage of Coolidge-like individuals:
bright, Kaggle Grandmaster-type individuals who understand academic
theory but whose specialty is in expertly applying their craft in dirty,
practical situations. Someone like Jeremy Howard will likely have great
intuition about how to utilize these individuals. The GE playbook — with
its “long leash within a narrow fence” principles — can help Answer.AI
think through how to deploy basic researchers in its operations

Langmuir’s career at the GE Research Lab provides a clear roadmap for
how to optimally leverage a basic researcher’s energies in an applied
context. Langmuir getting paid to investigate *any* anomalies would
likely have satisfied his curiosity. However, it was his investigation
of the *right* anomalies that made this a beneficial arrangement for GE
Research.

In general, there is a time and place to apply insights from either
Edison’s playbook or GE’s. The maturity of a given research field or
technology area has a strong hand in dictating which set of principles
is more applicable. Edison came first and had to shoulder the burden of
developing an extensive technical system to power the “killer app” that
was his bulb. GE Research had the benefit of working on an existing
technology area with moderately developed science and existing user
technology (thanks to Edison), but the technology still needed a lot of
work to become reliable and economical.

A lab can simultaneously employ both playbooks. Even most of Edison’s
projects were modest in relation to his lighting work. When inventing
for existing fields, such as telephony, Edison contained his inventive
streak to working within existing technical systems. He knew nobody
would rebuild entirely new telephone infrastructure just because the
young inventor had rigged up a moderately improved but completely
different version. When adding to Bell’s telephone, he simply invented a
carbon transmitter that could plug directly into the system. This device
made voices come through much clearer. That was it: one gadget that
cleanly plugged into the existing system. Technologies like these may
not be as earth-shattering as Edison’s lighting system, but they were
still enough to make him a world-famous inventor in his own time.

It was about impact. In optimizing impact, I thoroughly suspect
Answer.AI to make great use of the playbooks of both of these small
industrial research giants.

## Learning From Other Historically Great Industrial R&D Labs

I’d now like to highlight applicable lessons from other research
operations covered on my Substack. I’ll cover the orgs in no particular
order.

#### Striking the Balance of BBN and CMU’s Autonomous Vehicle Group

FreakTakes recently covered two historically great DARPA contractors who
expertly balanced the competing pulls of project novelty and deployable
technology. The first was Bolt, Beranek, and Newman (BBN), the
contractor primarily responsible for the ARPAnet. The second was
Carnegie Mellon’s autonomous vehicle groups.

BBN embodied what it meant to be a “middle ground between academia and
the commercial world.” The firm was initially set up by MIT acoustics
professors to pursue their contracting work more ambitiously. In its
early decades, the firm gradually expanded its contracting efforts into
the computing space, initially under the leadership of BBN VP J.C.R.
Licklider. BBN soon became a common home for the best researchers in
Cambridge, abandoning their academic positions to work for BBN. The
firm’s growing reputation even earned the monicker the “third university
of Cambridge.”

The firm’s revenue was primarily sourced from research contracts given
out by orgs like DARPA, research grantmakers, and aerospace firms. BBN’s
positioning was somewhat unique; when compared to industry, the firm
emphasized novelty and cutting-edge technology work. This insistence on
novelty helped the firm recruit individuals who felt a bit too talented
to waste away working on derivative projects at Westinghouse. When
compared to academia, BBN emphasized working on real technology that
people would use in the near term. J.C.R. Licklider is just one
prominent example of an individual who left a tenureship at MIT to work
on more useful technology down the road at BBN. Leveraging this
positioning, the firm was able to recruit the best talent.

BBN also provided its most talented individuals latitude to ply their
minds broadly. Many projects at BBN showcased the extreme potential of
small teams of talented individuals with broad technical knowledge. Only
eight BBNers were primarily responsible for pushing the early ARPAnet
into existence. The size of the team was no accident; Frank Heart, the
engineering lead of the project, described why he preferred a team of
this size in his [oral
history](https://conservancy.umn.edu/handle/11299/107349):

> I tend to believe important things get done by small groups of people
> who all know all about the whole project. That is, in those days all
> the software people knew something about hardware, and all the
> hardware people programmed. It wasn’t a group of unconnected people.
> It was a set of people who all knew a lot about the whole project. I
> consider that pretty important in anything very big. So I suppose if
> you call it a management style, that would be something I’d state. I
> think also that they were a very, very unusually talented group. I
> think things tend to get done best by small groups of very, very good
> people — if you can possibly manage that. You can’t always manage it.
> So if you again want to call it a management style, it is to get the
> very, very best people and in small numbers, so they can all know what
> they’re all doing.

Lockheed Skunk Works legend “Kelly” Johnson also held quite similar
beliefs when putting together teams to build experimental aircraft.
Particularly in the early stages of novel projects, there is a strong
case for keeping things small, with specialists who understand the
fields that touch theirs. To me, Jeremy’s belief in small teams seems
well-validated by technical history.

BBN demonstrates the ideal case of a research firm that wholly embraces
technical novelty. CMU can be thought of as the flipside of that coin: a
university that wholly embraced systems-building and used firm-like
management practices to do so. The highlight of CMU’s later-1900s
systems work was its autonomous vehicle projects. The academic group
staffed itself with researchers responsible for technical integration
and management-style work to effectively carry out novel technological
systems building.

Similar to BBN, CMU’s positioning was differentiated from both industry
and academia. This fact became very clear as DARPA’s mid-1980s
autonomous vehicle work progressed. CMU was seemingly the only
contractor excited about technical novelty *and* systems integration.
Martin Marietta — the defense prime in charge of DARPA’s Autonomous Land
Vehicle — obsessed over ways to hit DARPA’s demo benchmarks while using
unambitious, dated technologies. Simultaneously, the academic vision
research groups cared more about using the camera data to write papers
than helping directly contribute to building a functional driving
system. CMU was the only contractor involved in the project who truly
cared about building a novel, functional system. DARPA eventually
recognized this and gave them ample funds to build successive
generations of autonomous vehicles. The rest was history.

CMU carried out this work with a management structure that was more
firm-like than most academic labs. For example, Chuck Thorpe did project
management-style work for the team with firm-like incentives — he was a
researcher promoted based on vehicle performance, not his h-index. While
this was a firm-like position, the group also had academic-style
positions. Its use of graduate students on the projects is one prominent
example. Each student on the project had to own a piece of the project
that was all their own and could be written up as a thesis.

*(I explore how the team mitigated the risks of theses not panning out
in [my interview with Chuck
Thorpe](https://www.freaktakes.com/p/an-interview-with-chuck-thorpe-on).)*

These academic incentives partially enabled the CMU team to continually
innovate. Oftentimes these students’ theses perfectly plugged into
existing systems, such as a thesis on reducing the processing time of an
existing sensor’s data from 15 minutes to 90 seconds. But on the most
extreme occasion, in 1988, this incentive structure led a grad student
named Dean Pomerleau to successfully train a neural net to steer the
vehicle. In that particular case, the requirement to allow each grad
student to try something new changed the world.

Answer.AI similarly cares about deep technical novelty and building
deployable technology. As such, Answer.AI could benefit from emulating
BBN and CMU’s strategies to balance the two. The success of BBN and CMU
should hopefully embolden Answer.AI’s founders to trust in the
priorities they have set. This balance of goals is uncommon today, but
orgs from history have expertly balanced the two to world-changing
effect.

With that said, time elapsing without world-changing results might be
unnerving. This ambiguity is partially what pushed academia to rely on
near-term outcome variables that incentivize the incremental. Fear of
wasting time and money is real. The Answer.AI founders would surely like
some way to ensure that they are spending theirs on good problems. To
deal with that, I think the Bell Labs’ approach to problem selection has
a lot to add to the approaches I’ve already covered.

#### The Bell Labs Approach to Problem Selection

Bell Labs’ management of researchers in its golden era is famous — and
it should be. However, Bell researchers were not left to their own
devices to pursue whatever they wished, despite what many think. Bell
managed their researchers with an approach similar to the “long leash
within a narrow fence” approach of GE — which one long-time Bell chemist
called “circumscribed freedom.” The most effective tool it used to do
this was its corp of excellent systems engineers.

Bell had an expansive product line with massive scale — even more so
than GE. Even modest improvements from the research team could have
outsized returns. This, of course, is not the case with Answer.AI.
However, Bell’s use of systems engineers can still be extremely
instructive to Answer.AI, even if Answer.AI may deploy them in different
ways.

As I covered extensively on FreakTakes, Bell’s systems engineers often
combined several knowledge bases to expose the *right* researchers to
the problems that *most* needed solving. Within one mind systems
engineers often combined STEM backgrounds, knowledge of the nitty-gritty
details of Bell’s manufacturing, an understanding of Bell’s
implementation problems, detailed knowledge of Ma Bell’s expenses, and
familiarity with the researchers at Bell Labs.

Bell knew these systems engineers were a massive part of their secret
sauce, ensuring Labs deployed its limited resources on the right kinds
of problems with sufficient upside. I wrote in the conclusion of my Bell
Labs piece:

> Finding *a problem* in these systems is not so hard for those familiar
> with the systems. That’s why many researchers and engineers do not
> feel the need to bring in help. But finding a set of *good
> problems* is not finding the *best problems*. Finding the best
> problems is a profession in and of itself. A systems engineer is worth
> it when, under the right scrutiny, it might turn out that the best
> problem is 10X as financially valuable, does 50X the social good, or
> is 2X as likely to work as just some run-of-the-mill good problem.

My prior piece delves deeper into the specific problems towards which
Bell’s systems engineers led Bell’s researchers. For now, suffice it to
say that I think that any new applied science org that can dedicate an
(ideally full-time) individual to doing the work of a systems engineer
should strongly consider it.

Of course, these systems engineers would need somewhat clear marching
orders on what sorts of technologies they should be exploring. Nowadays,
many existing roles in industry and academia train individuals to equate
revenue or potential citation count with impact. Answer.AI will not be
satisfied with these metrics as proxies of impact, and they shouldn’t
be. What to direct these systems engineers toward instead should surely
be up to Jeremy and Eric.

As a firm that pursued novelty and was not attached to a large
industrial operation, BBN might be an interesting source of inspiration.
Several of its hallmark projects started with systems-engineer-style
contributions. Three examples are DARPA PM Larry Roberts putting out the
ARPAnet contract, J.C.R. Licklider’s visionary Libraries of the Future
Project, and BBNer Jordan Baruch’s early-1960s pitch to the NIH on a
system to build a computer system to facilitate modern hospital
operations. All three project initiators had Bell systems-engineer-like
exposure to the people and problems of their field — technical
backgrounds, regularly spoke with the best academic researchers, knew
modern industry’s issues, were able to project the costs and
complications of potential projects, etc.

Few have used systems engineers as effectively as Bell Labs, with
Bell-style goals, since the great lab was broken up. It would be amazing
to see a lab like Answer.AI commit significant staff time to this
purpose.

#### The Cautionary Tale of Thinking Machines Corporation’s Funding

The case of the Thinking Machines Corporation (TMC) is not as directly
instructive as the examples above, but TMC made one key mistake that
makes it worth mentioning. TMC put itself in the unfortunate position of
raising some of its funds from investors whose incentives were not
aligned with theirs.

For those who don’t know, TMC was a complete failure as a commercial
firm. As a result of its bankruptcy, many write the firm off as a
holistic failure. But the firm did accomplish many of the technical
goals it set out to achieve. Since the company was conceived with
technological goals in mind, rather than a specific market, this was no
small feat. The company was founded by Danny Hillis, a PhD from Marvin
Minsky’s lab at MIT. Through TMC, he sought to build the machine he
conceptualized in his graduate thesis: a truly parallel computer to
improve the capacity of all scientists. The young company recruited the
best researchers — including scoring Richard Feynman as its “intern” for
several summers — and achieved many technical goals that helped pave the
way for the modern field of parallel computing. Jeremy, who knows far
more about the technical aspects of parallel computing than I do, sang
TMC’s technical praises in our first conversation. He emphasized how
shocking it was that TMC seemed to be the first to employ so many
methods the field still uses today.

However, TMC’s high-powered team and great technical work were not
enough to overcome their management follies. The firm spent money as if
its financial standing was in line with its technical reputation, which
it was not. In retrospect, some of these management decisions — such as
a comically expensive long-term lease — could have been avoided without
modifying the company’s general approach. However, on a deeper level,
there was a dissonance between the company’s technical goals and the
funding it raised.

TMC’s two major funders were DARPA’s computing office and private
investors. The goals of Hillis and his top-flight technical staff were
only aligned with DARPA’s goals. In looking to build the technically
most ambitious parallel computer possible, DARPA funding was ideal. The
DARPA computing office also felt that TMC’s work was progressing
exceptionally well for most of its early years. However, the level of
enthusiasm Hillis and the technical staff had for building a machine for
science did not bleed into the most profitable areas — like deploying
the machine on banking databases or managing logistics for Walmart. As
time wore on, it seems that pressures were beginning to mount for TMC to
pursue work more in line with those areas. It seems highly likely that
TMC would have had to disappoint one of its funders sooner or later,
even if it spent funds more wisely in its early years.

Had TMC just raised funds from DARPA and spent them much more modestly,
the company might still exist today; it may have even earned a
reputation beyond that of NVIDIA.

## Copying mid-20th C. Industrial R&D Models is Hard for Incumbents

The middle 20th Century saw both the rise and fall of ambitious American
industrial R&D labs. It is not just nostalgia that makes modern
researchers look back on these labs with fondness. Not long after these
labs were formed, it was becoming clear to many top researchers that the
model was special.

In 1927, Karl Compton wrote a prescient letter to *Science* that praised
these labs. At the time of writing the letter, Compton was the head of
Princeton’s Physics Department and a part-time GE contractor. His letter
asserts that these organizations were doing some of the best science in
the country, even though the top universities often had the best men. A
portion of the letter, in which Compton praises the labs’ management of
scientific projects, reads:

> This has been strikingly demonstrated in some of the big industrial
> research laboratories, from which the output has greatly exceeded the
> individual capacities of the research workers

Compton believed there were *many* lessons that university departments
should steal from these exceptional industrial labs. The first was the
need to specialize when building a portfolio of researchers and
projects. Why should every department attempt to loosely approximate the
makeup of researchers and research questions in the field as a whole? No
company would ever do such a thing. He believed that, “Such a policy
dissipates effort, and if every institution followed it we should have
the spectacle of a great many universities all very much alike and all
with struggling, mediocre departments.”

On an organizational level, Compton believed the equilibrium of N
autonomous professors with N separate budgets and a few grad students in
their lab under their control was just not efficient for most projects.
It would be silly for all departments to function that way. In the
letter, Compton proposes what I’ve taken to calling a ‘Compton Model’
research department. This model is far more structured than a department
of mostly autonomous professors doing ad-hoc research with their own
separate funds. Compton describes it as follows:

> There is another direction in which more effective organization is
> possible within the universities themselves! Departments of a somewhat
> more flexible nature than those to which we are accustomed and which
> could, more than now, be built around one or two outstanding men in
> the department, could give these men an opportunity for organization
> and concentration of effort which is now rarely possible.

Another way to describe this model is as a “fund department heads, not
projects” model. The model allows one or two individuals to largely
shape the research vision, hiring, project selection, capital
purchasing, etc. of an entire department at once. In addition, it allows
these individuals the latitude to replace salaries spent on additional
professors or grad students with full-time engineers or discretionary
capital expenses as needed.

The model makes perfect sense. However, it’s remarkably difficult to
make happen at a real university. Compton couldn’t succeed in doing so
when he took over as MIT President. The existing stakeholders and
structures are just too hard to shift in this direction. Absent some
special circumstance, most university administrators wouldn’t find the
idea even worth considering. However, CMU’s President Cyert was able to
build a department that loosely resembled a Compton Model department.
And its results were exceptional! But this was the exception. Pivoting
an existing department to run like a Compton Model department has proven
infeasible in almost all cases.

The great, old models of managing labs didn’t disappear from industry
because they lacked scientific merit. These labs largely began to
disappear in the 1970s and 1980s. The 1970s saw a deep recession, which
usually hurt R&D budgets. In the 1980s, new corporate management trends
surfaced that led to companies being managed more myopically than ever
before or since. These, along with other non-scientific trends, were
largely responsible for the labs going away when they did. With them,
the operational know-how that had slowly been built up throughout the
century dissipated. Now, to learn how these orgs operated, one must read
oral histories or talk to now-retired engineers.

The first thing these retired engineers will often tell you is that we
should bring the old models back! Bell’s long-time researchers had
confidence in the Bell model even as a late-1900s court case ended Bell
Labs as we know it. In the 1920s, Compton dreamt of copying much of the
GE model to use within a university. One should not be afraid to put
their time and resources behind bringing these models back. Great
engineers largely maintained confidence in these models from their
inception until they faded into the background.

Many in academia know it should operate more like BBN, CMU, or GE
research. They just can’t change the structures to make it happen. Many
R&D leaders know that their company should think more in 10-20-year time
horizons when planning research expenditures, but shareholders and
shareholder-wary CEOs often do not find this view actionable.
**Answer.AI can be free from all this. The org can apply the great old
models in a new era on new technology.**

*(Ries being a cofounder of Answer.AI should be comforting. Ries has
railed against the types of myopic management trends that make
activities like maintaining an expansive industrial R&D lab difficult
for firms. He founded the young Long Term Stock Exchange, in part, to
help mitigate issues like this.)*

## Conclusion: Running Your Own Races

It’s up to Answer.AI’s two-headed management team to stick to the
organization’s comparative advantages. There is no need to race academic
CS researchers or corporate R&D departments in races those two groups
feel incentivized to run. Corporate R&D, NSF-funded CS researchers, and
AGI-focused labs all have areas in which they clearly will and won’t
operate. And those areas don’t come close to covering 100% of the good
ideas somebody should obviously be working on.
[Answer.AI](https://www.answer.ai) is free to run its own races,
uncontested.

Many will feel the org is using an untested model. However, Answer.AI’s
founders — like me — believe that this model is proven, but has simply
gone away for a while. So, shouldering this “organizational risk” that
others are seemingly not willing to do, they have the chance to work on
problems without much competition. If their USD10 million experiment
works, it has the chance to spark a rush of emboldened researchers and
engineers to found small research firms, leveraging the models of the
once-great dragons of American industrial R&D.

I wish Jeremy and Eric luck in the early stages of their mission. To any
researchers and engineers — across all areas — reading this and wishing
a BBN, CMU, TMC, or Answer.AI existed in your area, please reach out to
me on [Twitter](https://twitter.com/eric_is_weird). I’d love to see if
there’s anything I can do to help.