# Answer Ai
> Jeremy Howard (founding CEO, previously co-founder of Kaggle and
---
# Source: https://www.answer.ai/posts/2023-12-12-launch.md
# A new old kind of R&D lab
Jeremy Howard
2023-12-12
## tl;dr
Jeremy Howard (founding CEO, previously co-founder of Kaggle and
fast.ai) and Eric Ries (founding director, previously creator of Lean
Startup and the Long-Term Stock Exchange) today launched Answer.AI, a
new kind of AI R&D lab which creates practical end-user products based
on foundational research breakthroughs. The creation of Answer.AI is
supported by an investment of USD10m from Decibel VC. Answer.AI will be
a fully-remote team of deep-tech generalists—the world’s very best,
regardless of where they live, what school they went to, or any other
meaningless surface feature.
## A new R&D lab
In 1831 Michael Faraday showed the world how to harness electricity.
Suddenly there was, quite literally, a new source of power in the world.
He later found the basis of the unification of light and magnetism, and
knew he was onto something big:
> *“I happen to have discovered a direct relation between magnetism and
> light, also electricity and light, and the field it opens is so large
> and I think rich.”* Michael Faraday; letter to Christian Schoenbein
But it wasn’t quite clear how to harness this power. What kinds of
products and services could now be created that couldn’t before? What
could now be made far cheaper, more efficient, and more accessible? One
man set out to understand this, and in 1876 he put together a new kind
of R&D lab, which he called the “Invention Lab”: a lab that would figure
out the fundamental research needed to tame electricity, and the applied
development needed to make it useful in practice.
You might have heard of the man: his name was Thomas Edison. And the
organization he created turned into a company you would know: General
Electric.
Today, we find ourselves in a similar situation. There’s a new source of
power in the world—artificial intelligence. And, like before, it’s not
quite clear how to harness this power. Where are all the AI-powered
products and services that make our lives and work dramatically easier
and more pleasant?
To create these AI-powered products and services, we’ve created a new
R&D lab, called **Answer.AI**. Answer.AI will figure out the fundamental
*research* needed to tame AI, and the *development* path needed to make
it useful in practice.
## An iterative path to harnessing AI
Harnessing AI requires not just low-level computer science and
mathematical research, but also deep thinking about what practical
applications can take advantage of this new power. The “D” in “R&D” is
critical: it’s only by considering the *development* of practical
applications that the correct *research* directions can be targeted.
That’s why Answer.AI is built on the work of experts in both research
*and* development. Co-founders Jeremy Howard (that’s me!) and Eric Ries
have created pioneering ideas in each of these areas. I co-founded
fast.ai, where I have worked for the last 7 years on **research** into
how to best make AI more accessible, particularly through *transfer
learning* and *fine tuning*. I’ve been working with machine learning for
over 30 years, including creating the ULMFiT method of fine-tuning large
language models which is used as the basis of all popular language
models today, including OpenAI’s ChatGPT and Google’s Gemini. I have
developed the longest running online courses on Deep Learning in the
world, in which I show students how to start with simple models and then
iteratively improve them all the way to the state of the art.
I’ve known Eric for years, and there’s no-one I trust or respect more,
which is why I asked him to serve as the founding director of Answer.AI.
Eric has dedicated the last 10 years of his life to improving how
companies operate, serve customers, and are governed. He is the creator
of the Lean Startup movement, which is the basis of how most startups
build products and scale their organizations. His work focuses on
**development**: how can organizations go from an idea to a sustainable,
mission-driven, and profitable product in practice. One of his key
insights was to create and then iteratively improve a *Minimal Viable
Product* (MVP).
I asked Eric for his thoughts on Answer.AI’s unique approach to R&D, and
he summarised better than I ever could, so I’ll just quote his reply
here directly:
> *“People think that the order is research→development, and that
> therefore an R&D lab does “R” and then “D”. That is, the research
> informs the development, and so being practical means having
> researchers and developers. But this is wrong, and leads to a lot of
> bad research, because development should inform research and
> vice-versa. So having development goals is a way to do more effective
> research, if you set that out as your north star.”*
Eric is also an expert on governance and how companies should be led in
order to align profit and increased human flourishing. He created the
[Long-Term Stock Exchange](https://ltse.com/) (LTSE), the first
fundamentally new US Stock Exchange in over 50 years. LTSE mandates that
listed companies and likeminded investors work towards long-term value,
rather than just short-term profit maximization. Eric serves as the
Chairman of LTSE, meaning he is not only up to date on the right
long-term governance frameworks, but on the cutting edge of inventing
new systems.
It will take years for Answer.AI to harness AI’s full potential, which
requires the kind of strategic foresight and long-term tenacity which is
hard to maintain in today’s business environment. Eric has been writing
a book on exactly this topic, and his view is that the key foundation is
to have the right corporate governance in place. He’s helped me ensure
that Answer.AI will always reflect my vision and strategy for harnessing
AI. We’re doing this by by setting up a for-profit organization that
focuses on *long-term* impact. After all, over a long-enough timeframe,
maximizing shareholder value and maximizing societal benefits are
entirely aligned.
Whilst Eric and I bring very different (and complementary) skills and
experiences to the table, we bring the same basic idea of how to solve
really hard problems: solve smaller easier problems in simple ways
first, and create a ladder where each rung is a useful step of itself,
whilst also getting a little closer to the end goal.
## Our research platform
Companies like OpenAI and Anthropic have been working on developing
Artificial General Intelligence (AGI). And they’ve done an astonishing
job of that — we’re now at the point where experts in the field are
claiming that “[Artificial General Intelligence Is Already
Here](https://www.noemamag.com/artificial-general-intelligence-is-already-here/)”.
At Answer.AI we are not working on building AGI. Instead, our interest
is in effectively using the models that already exist. Figuring out what
practically useful applications can be built on top of the foundation
models that already exist is a huge undertaking, and I believe it is
receiving insufficient attention.
My view is that the right way to build Answer.AI’s R&D capabilities is
by bringing together a very small number of curious, enthusiastic,
technically brilliant generalists. Having huge teams of specialists
creates an enormous amount of organizational friction and complexity.
But with the help of modern AI tools I’ve seen that it’s possible for a
single generalist with a strong understanding of the foundations to
create effective solutions to challenging problems, using unfamiliar
languages, tools, and libraries (indeed I’ve done this myself many
times!) I think people will be very surprised to discover what a small
team of nimble, creative, open-minded people can accomplish.
At Answer.AI we will be doing genuinely original research into questions
such as how to best fine-tune smaller models to make them as practical
as possible, and how to reduce the constraints that currently hold back
people from using AI more widely. We’re interested in solving things
that may be too small for the big labs to care about-—but our view is
that it’s the collection of these small things matter a great deal in
practice.
This informs how we think about safety. Whilst AI is becoming more and
more capable, the dangers to society from poor algorithmic decision
making have been with us for years. We believe in learning from these
years of experience, and thinking deeply about how to align the
*applications* of models with the needs of people today. At fast.ai
three years ago we created a pioneering course on [Practical Data
Ethics](https://ethics.fast.ai/), as well as dedicating [a chapter of
our
book](https://github.com/fastai/fastbook/blob/master/03_ethics.ipynb) to
these issues. We are committed to continuing to work towards ethical and
beneficial applications of AI.
## From fast.ai to Answer.AI
Rachel Thomas and I realised over seven years ago that deep learning and
neural networks were on their way to becoming one of the most important
technologies in history, but they were also on their way to being
controlled and understood by a tiny exclusive sliver of society. We were
worried about centralization and control of something so critical, so we
founded fast.ai with the mission of making AI more accessible.
We succeeded beyond our wildest dreams, and today fast.ai’s AI courses
are the longest-running, and perhaps most loved, in the world. We built
the first library to make PyTorch easier to use and more powerful
(fastai), built the fastest image model training system in the world
(according to the Dawnbench competition), and created the 3-step
training methodology now used by all major LLMs (ULMFiT). Everything we
have created for the last 7 years was free—fast.ai was an entirely
altruistic endeavour in which everything we built was gifted to
everybody.
I’m now of the opinion that this is the time for rejuvenation and
renewal of our mission. Indeed, the mission of Answer.AI is the same as
fast.ai: to make AI more accessible. But the *method* is different.
Answer.AI’s method will be to *use* AI to create all kinds of products
and services that are really valuable and useful in practice. We want to
research new ways of building AI products that serve customers that
can’t be served by current approaches.
This will allow us to make money, which we can use to expand into more
and bigger opportunities, and use to drive down costs through better
efficiency, creating a positive feedback loop of more and more value
from AI. We’ll be spending all our time looking at how to make the
market size bigger, rather than how to increase our share of it. There’s
no moat, and we don’t even care! This goes to the heart of our key
premise: creating a long-term profitable company, and making a positive
impact on society overall, can be entirely aligned goals.
## We don’t really know what we’re doing
If you’ve read this far, then I’ll tell you the honest truth: we don’t
actually know what we’re doing. Artificial intelligence is a vast and
complex topic, and I’m very skeptical of anyone that claims they’ve got
it all figured out. Indeed, Faraday felt the same way about
electricity—he wasn’t even sure it was going to be of any import:
> *“I am busy just now again on Electro-Magnetism and think I have got
> hold of a good thing but can’t say; it may be a weed instead of a fish
> that after all my labour I may at last pull up.”* Faraday 1831 letter
> to R. Phillips
But it’s OK to be uncertain. Eric and I believe that the best way to
develop valuable stuff built on top of modern AI models is to try lots
of things, see what works out, and then gradually improve bit by bit
from there.
As Faraday said, “A man who is certain he is right is almost sure to be
wrong.” Answer.AI is an R&D lab for people who aren’t certain they’re
right, but they’ll work damn hard to get it right eventually.
This isn’t really a new kind of R&D lab. Edison did it before, nearly
150 years ago. So I suppose the best we can do is to say it’s a new old
kind of R&D lab. And if we do as well as GE, then I guess that’ll be
pretty good.
---
# Source: https://www.answer.ai/posts/2024-01-26-freaktakes-lessons.md
# Lessons from history’s greatest R&D labs
Eric Gilliam
2024-01-26
> **Note from Jeremy**: *There are few things more important to our
> civilization than understanding how to better do R&D. Thankfully, Eric
> Gilliam has dedicated himself to studying this question. As a result,
> he’s become the foremost scholar and historian of 19th and 20th
> century R&D labs. I thought I was fairly well informed when it comes
> to the history of these labs, but after talking to Eric, I quickly
> realised I’m a rank amateur by comparison! Eric’s knowledge of the
> history of modern research and development is unparalled, and I found
> his insights into why some organizations were dramatically more
> effective than others to be utterly compelling. Therefore, I asked him
> for his totally honest assessment of our plans for Answer.AI, based
> both on our written plans and a number of in-depth conversations we
> had together. Today we are publishing a guest post with his analysis,
> cross-posted from
> [FreakTakes](https://www.freaktakes.com/p/lessons-answerai-can-learn-from-historys),
> his blog.*
- This article is also available in AI-generated spoken audio format –
play directly in the above player, or click the menu on the right to
download the mp3 file. Skip to 3m20s to get past the intro section and
jump to the main article.
Jeremy Howard (former President and Chief Scientist of Kaggle) and Eric
Ries (creator of The Lean Startup movement and Long Term Stock Exchange)
have teamed up to found a new applied R&D
lab: [Answer.AI](https://www.answer.ai). When speaking with Jeremy, he
made it clear that many details of Answer.AI’s structure are still being
worked out. Only announced a month ago, the org is still in its early
development stages. But the founders have conviction on certain
principles. The most prominent of them is one extremely relevant to my
regular readers: The founders seem to be particularly inspired by
Edison’s Menlo Park Lab and the early days of commercial electric
research.
In the piece, I’ll briefly examine the (working) plans for the lab and
do some historical analysis, detailing:
1. What the earliest electrical R&D labs can teach Answer.AI
2. Useful rules-of-thumb from other historically great applied R&D labs
3. Potential pitfalls to keep in mind as they move forward
You can find more thorough historical evidence in my prior pieces for
any of the lab details I mention, listed below:
- [Thomas Edison,
tinkerer](https://worksinprogress.co/issue/thomas-edison-tinkerer/)
published in *Works in Progress*
- [Tales of Edison’s
Lab](https://www.freaktakes.com/p/tales-of-edisons-lab) (podcast)
- [Irving Langmuir, the General Electric Research Laboratory, and when
applications lead to
theory](https://www.freaktakes.com/p/irving-langmuir-the-general-electric)
- [How did places like Bell Labs know how to ask the right
questions?](https://www.freaktakes.com/p/how-did-places-like-bell-labs-know)
- [“The Third University of Cambridge”: BBN and the Development of the
ARPAnet](https://www.freaktakes.com/p/the-third-university-of-cambridge)
- [An Interview with Chuck Thorpe on CMU: Operating an autonomous
vehicle research
powerhouse](https://www.freaktakes.com/p/an-interview-with-chuck-thorpe-on)
- [ILLIAC IV and the Connection
Machine](https://www.freaktakes.com/p/illiac-iv-and-the-connection-machine)
- [A Progress Studies History of Early MIT — Part 2: An Industrial
Research
Powerhouse](https://www.freaktakes.com/p/a-progress-studies-history-of-early-001)
- [How Karl Compton believed a research department should be
run](https://www.freaktakes.com/p/how-karl-compton-believed-a-research)
Each of the orgs listed has lessons to teach Answer.AI. But none are a
perfect analog. So, as the piece progresses, I’ll explain which lessons
I think most strongly apply to Answer.AI. With that, let’s get into it!
Edison and “the boys” preparing the First
Practical Incandescent Lamp for Testing at Edison’s Menlo Park Lab.
Illustration by Harry K. Flemming. Photo Courtesy of the Henry Ford
Museum
## [Answer.AI](http://Answer.AI) in a Nutshell
Jeremy’s blog post [announcing
Answer.AI](https://www.answer.ai/posts/2023-12-12-launch.html) makes it
clear that the org is, to a large degree, inspired by the field of
electricity’s path of progress in the 1800s. He believes the current
state of the AI field is similar to the state of the electricity field
between the work of Michael Faraday and Edison’s lighting projects. This
was an era in which new electrical findings were being pieced together,
but few had made any progress in turning the potential of electricity
into great applications.
I don’t find this comparison crazy. So far, I don’t believe AI has come
close to the level of breakthrough that electricity proved to be.
Electricity brought the sunlight indoors for a negligible cost *and*
powers so many of our modern conveniences— refrigeration, TVs, central
heating, etc. That’s a high bar. *However*, given that human ingenuity
created the breakthrough that was electricity and each of those
applications, it is surely worth considering that AI *could* grow to be
the most impactful field of them all. Whether AI does reach that level
of promise, to me, is a question of human ingenuity. So, I have no issue
with Jeremy comparing the AI field to the electrical field c. 1830 to
1910.
With that elephant out of the way, let’s briefly examine what sets
[Answer.AI](https://www.answer.ai) apart from AI labs like OpenAI and
Anthropic. From a funding perspective, Answer.AI seems much, much
cheaper. The founders have initially raised USD10 million. This stands
in stark contrast to the gargantuan initial rounds of OpenAI and
Anthropic. Also, Answer.AI’s research agenda is more
application-centric. The following excerpt from Jeremy’s blog post
highlights what he thinks differentiates the lab’s approach:
> At Answer.AI we are not working on building AGI. Instead, our interest
> is in effectively using the models that already exist. Figuring out
> what practically useful applications can be built on top of the
> foundation models that already exist is a huge undertaking, and I
> believe it is receiving insufficient attention.
>
> My view is that the right way to build Answer.AI’s R&D capabilities is
> by bringing together a very small number of curious, enthusiastic,
> technically brilliant generalists. Having huge teams of specialists
> creates an enormous amount of organizational friction and complexity.
> But with the help of modern AI tools I’ve seen that it’s possible for
> a single generalist with a strong understanding of the foundations to
> create effective solutions to challenging problems, using unfamiliar
> languages, tools, and libraries (indeed I’ve done this myself many
> times!) I think people will be very surprised to discover what a small
> team of nimble, creative, open-minded people can accomplish.
>
> At Answer.AI we will be doing genuinely original research into
> questions such as how to best fine-tune smaller models to make them as
> practical as possible, and how to reduce the constraints that
> currently hold back people from using AI more widely. We’re interested
> in solving things that may be too small for the big labs to care about
> — but our view is that it’s the collection of these small things
> matter a great deal in practice.
It would be unfair to say that an application-centric research agenda is
necessarily less ambitious than AGI. Those biased toward basic research
might say so, but I don’t think that opinion is very
historically-informed. Edison himself was application-centric above all
else. His deep belief in market signals is fascinating when juxtaposed
with the market indifference of many great academic physicists. In the
book *[From Know-How to Nowhere](https://amzn.to/3HoUUbW)*, a history of
American learning-by-doing, Elting Morison described the interesting
nature of Edison’s motivations:
> If the means by which he \[Edison\] brought off his extraordinary
> efforts are not wholly clear, neither is the cause for his obsessive
> labors. No diver into nature’s deepest mysteries carrying next to
> nothing for the advancement of knowledge and even less for the world’s
> goods, he would become absorbed in making something work well enough
> to make money. The test in the marketplace was for him, apparently,
> the moment of truth for his experiments.
Edison built his god-like reputation by dreaming in specific
applications. He kept market, resource, and manufacturing constraints in
mind from the earliest stages of his projects. Edison dreamed practical,
realizable dreams. And when the limitations of component technologies
stood in the way of his dreams, he often had the talent to invent new
components or improve existing materials. Edison’s biggest dream, the
light bulb, mandated that Edison solve a much broader set of problems.
The following excerpts from my [Works in Progress piece on
Edison](https://worksinprogress.co/issue/thomas-edison-tinkerer/) paint
a clear picture of his ambitious but practical dreams:
> After Edison’s bulb patent was approved in January 1880, he
> immediately filed another for a ‘System of Electrical Distribution’.
> Filing for these so close together was no coincidence. To Edison, it
> was never just a bulb project. It was a technical business venture on
> a possibly unprecedented scale. Edison wanted to light up homes all
> over the world, starting with lower Manhattan.
>
> Bringing the project from dream to mass-market reality would require
> solving over a hundred technical problems. His was a new bulb that
> needed to be powered by a generator that did not yet exist at the
> start of the project, strung up in houses that had no electricity,
> connected via underground street wiring that was only hypothetical,
> and hooked up to a power station that had never existed before.
>
> Yet, at the end of two years’ time, Edison would do it. And, just as
> importantly, the entire venture was profitable by the end of the
> project’s sixth year.
Edison was clearly doing a different kind of dreaming than those who do
basic research. His lighting work embodies what extreme ambition looks
like in application-centric research. Answer.AI making this kind of
ambitious, applied work their North Star is an extremely interesting
goal.
This goal has the potential to give Answer.AI a comparative advantage in
the growing space of for-profit AI labs. For example, the most ambitious
aspects of OpenAI are considered to be in its research, not its work on
applications. Answer.AI’s particular setup can also set it apart from AI
startups and academic labs. New AI startups do some research on how to
commercialize new AI models in new ways, but they generally have short
runways. In this kind of environment, only specific types of research
projects can be pursued. Academic labs — for many reasons covered
elsewhere on my Substack (such as in the [ARPA
series](https://www.freaktakes.com/s/arpa-playbook)) — don’t have the
right combination of incentives, experience, and staffing to build new
technologies in most problem areas. The main incentive of the
profession, in a simplified form, is producing many paper studies that
get cited many times. Answer.AI has the chance to let its alternative
focus lead it to areas under-explored by academics, companies with brief
timelines to hit revenue benchmarks, and more AGI-focused R&D labs.
Legally, Answer.AI is a company. But in practice, it might hover
somewhere between a lab and a normal “profit-maximizing firm” — as was
the case with Edison’s lab. The founders seem perfectly content to
pursue high-risk projects that might lead to failures or lack of revenue
for quite a while. In saying this, I do not mean to imply they are
content to light money on fire doing research with no chance of a
return. Rather, they hope to fund a body of research projects that
ideally have positive ROI in the long term. They are just not overly
concerned with short-term revenue creation.
*(Making the pursuit of research agendas like this easier is actually
one of the founding goals of Ries’ Long Term Stock Exchange — which I
address later.)*
There is apparently no pressure to produce a product that can hit
software VC-style revenue goals within 12-24 months, or anything
similar. This is good. Seeking to satisfy these types of metrics does
not traditionally permit a company to act like a truly ambitious R&D
lab. I’m not saying it can’t happen — DeepMind seems to have made it
work in its early years — but it does require pushing against investor
pressure quite strongly. The VC money raised for Answer.AI has left the
founders with enough voting shares that investors can’t veto founders’
decisions. Additionally, Howard says the company’s investors understand
what they are trying to build is, first and foremost, a lab. This is a
great step towards building an organization focused on building very
useful, very new things rather than the most profitable thing possible —
which often comes with bounded technical novelty.
Interestingly, Answer.AI will also keep a small headcount. Jeremy built
Fastmail up to one million accounts with only three full-time employees.
He hopes to keep the Answer.AI team exceptionally talented and
“ruthlessly small” in a similar way; he believes keeping teams small is
important to building new, technically complex things.
Now that I’ve outlined some important pieces of Answer.AI’s vision, I’ll
dive into the historical analysis. In the first section, I detail
lessons that Answer.AI can draw from both Edison’s Menlo Park laboratory
and the Early GE Research Laboratory. In the following section, I’ll
share useful lessons from other historically great industrial R&D labs.
Lastly, I’ll highlight the bureaucratic details that explain why the
operational models of the great industrial R&D labs have not been
replicated often.
## Learning from the First Electrical R&D Labs
I find it exciting that Edison’s Menlo Park lab is a North Star for
Answer.AI. I covered Edison’s work in several pieces because I think
evergreen lessons can be drawn from his work. *But* I think a more
complete way to incorporate lessons from the 1870-1920 electrical space
is to draw on the work of both Edison’s Menlo Park Lab *and* the young
GE Research Lab. The latter operated as a more traditional industrial
R&D lab. GE Research’s history holds many lessons to help steer
Answer.AI’s problem selection and work on its standard projects.
However, *exceptionally ambitious* projects may draw more heavily on the
lessons of Edison’s lab.
*(As a note, while Edison General Electric was one of the two companies
that merged to become GE — along with Thomson-Houston Electric — Edison
had essentially nothing to do with the formation of the iconic GE
Research Laboratory.)*
Different types of projects characterized the work of the two electrical
labs. When it came to electrical work, for years, Edison’s lab and
mental efforts were focused on doing everything necessary to bring a
single, revolutionary product to market. On the other hand, GE Research
usually had many separate courses of research underway at once. These
projects all sought to improve the science and production of existing
lighting systems, but they were otherwise often unrelated to each other.
Additionally, GE’s work could be categorized as more traditional
“applied research.” The lab was not actively looking to create a field
of technology from scratch as Edison did. GE Research’s projects were
often novel and ambitious, but in a different way than Edison’s.
Later, I will explore the types of novelty the GE Research Lab pursued.
First, I’ll give the reader a more fine-grained idea of how Edison’s
lighting project actually operated.
### Lessons from Edison’s Work on Electricity
Edison’s lighting work provides great management lessons for those
looking to direct a large chunk of a lab’s efforts toward a single, big
idea.
Edison’s major contribution to the field of electricity was not
inventing each of the components in his lighting system, but in turning
a mass of disparate gadgets, scientific principles, and academic
misconceptions into a world-changing system. The burden of doing “night
science” — [as Francois Jacob refers to
it](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1800-6)
— largely fell on Edison. In the late 1870s, nobody knew much about
electricity yet. The existing academic literature had more holes than
answers, and many of its so-called “answers” turned out to be wrong or
misleading. From this shaky starting point, Edison proceeded. He
combined his unique mix of attributes and experience to deliver a
world-changing system. These included: knowledge of several adjacent
scientific fields, deep knowledge in then-overlooked experimental areas,
market knowledge, manufacturing knowledge, and the ability to adequately
operate a small research team.
In large part, Edison created his lab as a way to scale himself. As a
result, to understand how his lab operated, one needs to know how Edison
himself carried out his explorations. Edison was one of the more
stubborn experimentalists of all time. He spent most of his waking hours
carrying out one experiment or another. While he did pore over
scientific literature, for him, nothing was settled until he proved it
for himself at the lab bench.
I write in my *Works in Progress* piece:
> Edison respected scientific theory, but he respected experience far
> more. In Edison’s era of academia as well as today’s, many professors
> had a certain preference for theory or ‘the literature’ over hands-on
> improvement. Because of this Edison did not care much for professors.
> He was even known to go on long diatribes, during which he had
> assistants open up textbooks, locate scientific statements that he
> knew to be untrue from experience, and quickly rig up lab
> demonstrations to disprove them. ‘Professor This or That will
> controvert \[dispute with reasoning\] you out of the books, and prove
> out of the books that it can’t be so, though you have it right in the
> hollow of your hand and could break his spectacles with it.’
Contained in his head was a database of countless experiments and
results that made it seem as if his “intuition” was far beyond his
contemporaries. This left him with an unparalleled skillset and body of
knowledge. If anyone could feel comfortable pursuing a project that
others had previously failed at, it was Edison. Edison’s confidence in
his skills was never more on display than when he chose to pursue his
lighting work. Many in the scientific establishment knew electric bulb
lighting was technically possible, but claimed they had proven that it
could never be economical. Edison disagreed.
On top of Edison’s admirable approach to experimentation, he brought a
high level of practicality to his process. He knew his inventions needed
to make commercial sense in order to make it out of the lab. So, even in
early courses of experimentation, he kept factors like manufacturability
in mind. He wouldn’t commit much time to something that didn’t make
commercial sense. With that being said, Edison wanted to change the
world with his technologies more than he wanted to get rich. So, the
practical factors he paid aggressive attention to were primarily treated
as constraints. He did not optimize for profitability, but he knew his
ideas needed to be profitable. Nobody who wanted to optimize for profit
would have pursued lighting in the way Edison did. The technical risks
were too great.
Edison was able to imagine an ambitious system that required many
technical advances. It was so futuristic that maybe only he was capable
of coming up with it. But just as impressively, he was able to do it
profitably and on schedule. His dogged commitment to experimentation
seems to be largely responsible for this. Edison and “the boys”
constantly experimented on every piece of the process to improve and
learn more about all the sub-systems in Edison’s grand system. They
wanted to know how every piece of every sub-system performed in all
conditions. I’ll share just two excerpts from my *Works in Progress*
piece as examples.
The first is from Edmund Morris’ biography of Edison. It recounts how
thoroughly Edison and his trusted aid, William Batchelor, were in
carrying out round after round of filament experiments:
> For week after week the two men cut, planed, and carbonized filaments
> from every fibrous substance they could get — hickory, holly, maple,
> and rosewood splints; sassafras pith; monkey bast; ginger root;
> pomegranate peel; fragrant strips of eucalyptus and cinnamon bark;
> milkweed; palm fronds; spruce; tarred cotton; baywood; cedar; flax;
> coconut coir; jute boiled in maple syrup; manila hemp twined and
> papered and soaked in olive oil. Edison rejected more than six
> thousand specimens of varying integrity, as they all warped or split…
>
> In the dog days, as heat beat down on straw hats and rattan parasols,
> the idea of bamboo suggested itself to him. Nothing in nature grew
> straighter and stronger than this pipelike grass, so easy to slice
> from the culm and to bend, with its silicous epidermis taking the
> strain of internal compression. It had the additional virtue, ideal
> for his purpose, of being highly resistant to the voltaic force. When
> he carbonized a few loops sliced off the outside edge of a fan, they
> registered 188 ohms cold, and one glowed as bright as 44 candles in
> vacuo.
This approach went far beyond bulb filaments. The following excerpt
describes the work of one of Edison’s lead mechanics in turning the
Menlo Park yard into a 1/3 scale model of what they would later install
in Lower Manhattan. I write:
> \[Kruesi, Edison’s mechanic\] along with a group of engineers and a
> team of six diggers, turned the excess land of the lab in Menlo Park,
> New Jersey…into a one-third-scale model of Edison’s first lighting
> district in lower Manhattan. This team tested and re-tested the
> electricity delivery system, digging up Menlo Park’s red clay to lay
> and re-lay an experimental conduit system. The team carried out
> countless tests to ensure that they found materials to efficiently
> carry the electric current while also keeping the delicate materials
> safe from water and ever-present New York City rats.
The entire process was marked by the classic trial-and-error of the
Edisonian process. The first subterranean conducting lines and
electrical boxes the group laid were completely ruined by two weeks of
rain — despite being coated with coal tar and protected with extra wood.
While the diggers dug up the failed attempt so the damage could be
examined, Kruesi and a young researcher…studied and tirelessly tested
unbelievable numbers of chemical combinations — making full use of the
laboratory library and chemical room — until, finally, a blend of
‘refined Trinidad asphaltum boiled in oxidized linseed oil with paraffin
and a little beeswax’ was found that protected the electrical current
from rain and rats. \>
Edison built his own style of dogged experimentation into the culture of
his lab. Since the lab was meant to scale Edison, this makes perfect
sense; he was a man with far more ideas than hands. So, he hired more
hands. Edison did not search far and wide to hire the world’s best
research minds, and many of those he employed did not even have
scientific backgrounds. This didn’t matter much to Edison because most
of them were employed to undertake courses of research that he had
directed them to pursue. A couple of his Menlo Park employees had
advanced scientific degrees, but far more did not. For the most part,
the lab and its activities were steered by Edison and his ideas. As a
result, the productivity of his lab followed wherever his attention
went. After some time working on a project area, Edison would often grow
antsy and wish to move on to the next thing — he craved novelty. The
lab’s resources and extra hands would move with him. As we’ll see in the
next section, this stands in stark contrast to how the GE Research Lab
recruited and chose problems.
Menlo Park’s electrical activities provide a great management playbook
for what it looks like to direct a lab’s efforts toward a single, major
system. If Answer.AI does not want to go all-in on one thing, it can
still find a way to apply this playbook to a certain focused team of
employees while leaving the others to tinker around with
exploration-stage ideas. In Edison’s less-focused experimentation
periods, his lab served as more of an “invention factory,” doing this
sort of fiddling. Additionally, Edison’s preference for application and
commitment to experimentation over theory in a young area of science can
surely provide Answer.AI some inspiration.
Of course, Edison did some things better than others. Edison’s most
easily-spottable “deficiency” is that his lab was largely dependent on
him. Without him and his big ideas, the lab would have probably ground
to a halt. While Edison’s technical vision, practicality, and
experimental approach are absolutely worthy of emulation, the lessons of
GE Research should probably be added into the mix as well. GE operated
as more of a prototypical industrial R&D lab with an approach quite
suited to the fact that the science of electricity was beginning to
mature in the early 1900s.
### Lessons from the Young GE Research Laboratory
The young GE Research lab took a different approach to electricity
research than Edison. The lab worked on many unrelated projects at once,
recruited more talented researchers, and allowed these talented
researchers more freedom to exert the scientific method on
commercializable projects. The lab did not undertake projects that were
as purposely futuristic as Edison did. Nobody from the lab earned
nicknames like “the Wizard of Menlo” or “the Wizard of Recorded Sound.”
But early GE Research was still responsible for a Nobel Prize and making
the light bulb a much-improved, more cost-effective technology.
Elting Morison wrote the following on the lasting impact of GE
Research’s early decades:
> There seems little doubt that…much that was done in Schenectady in
> electrical engineering and some parts of physics was both better done
> and more interesting than what was being done in those fields in any
> American university.
In its heyday, even great researchers like Karl Compton hoped to shift
their academic departments to operate more like GE Research.
While GE did simultaneously pursue diverse projects, there was a
unifying thread holding all of the projects at GE Research together.
Each project aimed to improve the quality and profitability of GE’s
products and manufacturing. Under that unifying theme, all kinds of
projects were encouraged. Much of the research was very applied,
particularly in the early years when the lab was still proving itself.
William Coolidge was one of the lab’s most talented applied researchers
in its early years. Coolidge joined the lab in 1905, part-time while
teaching courses at MIT. Coolidge had the kind of toolkit typical of
many MIT professors in that era. He had a far greater grasp of the
science of physics and metallurgy than somebody like a blacksmith; he
was simultaneously far closer to a blacksmith than one would ever expect
a university researcher to be. With this differentiated toolkit, he did
science in a way that was not typical of academics. In describing the
process that led to his successes at GE, he claimed that he was, “guided
in the main by experiment itself rather than by metallurgical
knowledge.”
Willis Whitney, the founding Director of GE Research and former MIT
professor, recruited Coolidge to build on findings Whitney himself had
made. Whitney’s initial course of research had found an improved
metalized carbonized cellulose filament for bulbs. Whitney’s results
proved very profitable for the lab. It seemed reasonable that an actual
metal filament could perform even better. Whitney thought Coolidge and
his metal-working skills were well-suited to pursue the area further.
Coolidge expertly applied practical skills in concert with scientific
knowledge to pursue the problem. Elting Morison described a small sample
of Coolidge’s workflow:
> He suspended tungsten powder in an amalgam of bismuth, cadmium, and
> mercury. He then passed the resulting substance through tiny dies —
> drawing it — and obtained a silvery pliable wire. At that time, he
> thought he had reached ductility and the search was over. But when a
> current was passed through this wire the mercury, cadmium, and bismuth
> distilled out, leaving, unfortunately, a nonductile tungsten. But it
> also proved to be tungsten in the purest state he had yet produced.
I continue in my FreakTakes piece, writing:
> He eventually iterated his way to a workable process where…the more
> pure tungsten was put through a specific combination of metal-working
> processes at a temperature that worked that produced rods of tungsten
> about 1 mm in diameter. These 1mm rods could then be drawn and
> re-drawn through rods of decreasing size until you were left with
> wires of tungsten .01 mm in diameter. When put in the vacuum-sealed
> bulb, electricity ran through the tungsten filaments and demonstrated
> an efficiency of 1 watt per candle — extending the life of a bulb up
> to 27x.
>
> Within 5 years, 85% of all lamps would be made from tungsten. As the
> project went on, more and more research chemists and technical
> assistants grew to be involved in the wide-ranging steps and
> combinations involved in Coolidge’s experiments. But it worked. GE had
> the factories re-fit and deployed the new bulb. Coolidge moved on to
> other research.
The success of Coolidge’s hybrid work style, not dissimilar to Edison’s,
is surely a useful data point to Answer.AI. But GE Research also did
work that went far beyond Coolidge’s technically adept, applied science.
The lab was fantastic at making use of talented individuals who were
very academic. Irving Langmuir was a prime example. I described his
interests in my original piece:
> It should be noted…Langmuir did not even care about lightbulbs. Well,
> I guess that is not technically true. The bulb interested him
> because 1) he thought a metal like tungsten was cool because it could
> accept really high temperatures which opened up options to the
> scientist working with it and 2) these vacuum-sealed bulbs provided a
> pristine environment for controlled scientific investigations.
To Langmuir, light bulbs were primarily a playground in which to do his
science. But Willis Whitney knew how to take an individual like that and
direct his energy towards productive ends. The lab deployed a principle
that I call extending a “long leash within a narrow fence” to basic
researchers like Langmuir.
The way the lab facilitated this was rather simple. On his first day,
Langmuir was told to walk around the applied end of the lab and ask
people about their projects. Whitney permitted him to undertake any
course of investigation of any phenomenon he wanted, *but* it had to be
directly related to an existing problem/limitation/constraint that the
applied folks were working through. These applied folks were working on
projects that rather directly plugged into GE’s operations, so there was
minimal risk of Langmuir’s work not amounting to anything useful if he
succeeded and found answers. With that assurance of applicability,
Langmuir was given extensive timelines to find answers to open
questions.
Langmuir’s first course of research focused on the constant
bulb-blackening problem common to bulbs at the time. The problem was
generally attributed to a bulb’s imperfect vacuum. Langmuir found this
problem to be a great excuse to carry out a course of experimentation he
found interesting. Morison described Langmuir’s thought process as
follows:
> If residual gases — imperfect vacua — produced a bad effect —
> blackening — here was a fine opportunity to study the effects produced
> by different gases introduced one by one into the bulb. What he wanted
> to do, he told Whitney, was simply to plot the interactions of various
> gases exposed at low pressures to very high temperatures in the
> filament. Nobody knew very much about this phenomena and he wanted to
> look into it simply “to satisfy \[his\] own curiosity.”
Langmuir carried out this course of research over three years. There
were many gases and temperatures to test, which took time. But
unforeseen results constantly took Langmuir off in different directions.
Exploring these unforeseen results often entailed new courses of
experiment altogether. With his long leash, Langmuir was able to figure
out that imperfect vacua were not what caused bulb blackening at all.
Rather, it was that tungsten vapor particles were finding their way onto
the wall of the bulb. Temperature was the issue.
He also discovered that different gases markedly changed the rate of
evaporation. One extreme example was nitrogen, which reduced the
evaporation rate by 100-fold. *However*, adding nitrogen to the bulbs
caused the electrical efficiency of the system to decrease drastically.
So, the existing bulb design with nitrogen added was less cost-efficient
than the normal bulbs. But Langmuir was undeterred. This was progress.
Existing fundamental research in this area led him to believe that this
efficiency issue could be alleviated by increasing the diameter of the
filament. Further experimentation proved this to work. He also found
that coiling the filament in a certain way could mitigate the heat loss
issue. The final result was a novel bulb that used an inert gas instead
of a vacuum to reduce bulb blackening. Along with the coiled tungsten
filament, this new bulb only required .5 watts per candle and lasted
three times longer than any other bulb.
Once he passed the bulb project onto the engineering team at GE
Research, Langmuir set his sights on an anomaly he had come across
talking with the lab’s more applied staff. The bulbs in the lab had a
design that depended on only a few milliamperes of current flowing
across the space between one end of the filament and the other. Langmuir
noted this anomaly in a letter to *Scientific Monthly*, writing:
> This fact seemed very peculiar to me, for the work of Richardson and
> others had indicated that at temperatures as high as those used in the
> tungsten-filament lamp, currents of many amperes should flow across
> the space. In other words, according to the then-accepted theory of
> the electron emission from hot filaments, a serious difficulty should
> have been encountered in the construction of tungsten-filament lamps.
> The fact that we did not meet any such difficulty therefore seemed to
> me a peculiar fact that should be investigated.
In the brief course of exploration that followed from Langmuir, he
discovered what is now known as the space-charge effect. This work
combined with follow-on work from Coolidge to produce an entirely new
kind of GE X-ray tube.
Under this “long leash within a narrow fence” guideline, Irving Langmuir
would go on to be partially responsible for a handful of new and
improved product lines at GE. Additionally, the knowledge he created
with his tungsten filament work went far beyond padding GE’s balance
sheet. Over the course of his project, he noted that the way tungsten
vapor condensed did not gel with existing academic theory. His
subsequent exploration of this phenomenon led Langmuir to be credited
with founding the field of surface chemistry. Langmuir earned himself a
Nobel Prize for his efforts.
There was a symbiosis in the GE lab between Langmuir types and the
Coolidge types — the latter skillset being more standard in the lab. I
imagine Answer.AI will have no shortage of Coolidge-like individuals:
bright, Kaggle Grandmaster-type individuals who understand academic
theory but whose specialty is in expertly applying their craft in dirty,
practical situations. Someone like Jeremy Howard will likely have great
intuition about how to utilize these individuals. The GE playbook — with
its “long leash within a narrow fence” principles — can help Answer.AI
think through how to deploy basic researchers in its operations
Langmuir’s career at the GE Research Lab provides a clear roadmap for
how to optimally leverage a basic researcher’s energies in an applied
context. Langmuir getting paid to investigate *any* anomalies would
likely have satisfied his curiosity. However, it was his investigation
of the *right* anomalies that made this a beneficial arrangement for GE
Research.
In general, there is a time and place to apply insights from either
Edison’s playbook or GE’s. The maturity of a given research field or
technology area has a strong hand in dictating which set of principles
is more applicable. Edison came first and had to shoulder the burden of
developing an extensive technical system to power the “killer app” that
was his bulb. GE Research had the benefit of working on an existing
technology area with moderately developed science and existing user
technology (thanks to Edison), but the technology still needed a lot of
work to become reliable and economical.
A lab can simultaneously employ both playbooks. Even most of Edison’s
projects were modest in relation to his lighting work. When inventing
for existing fields, such as telephony, Edison contained his inventive
streak to working within existing technical systems. He knew nobody
would rebuild entirely new telephone infrastructure just because the
young inventor had rigged up a moderately improved but completely
different version. When adding to Bell’s telephone, he simply invented a
carbon transmitter that could plug directly into the system. This device
made voices come through much clearer. That was it: one gadget that
cleanly plugged into the existing system. Technologies like these may
not be as earth-shattering as Edison’s lighting system, but they were
still enough to make him a world-famous inventor in his own time.
It was about impact. In optimizing impact, I thoroughly suspect
Answer.AI to make great use of the playbooks of both of these small
industrial research giants.
## Learning From Other Historically Great Industrial R&D Labs
I’d now like to highlight applicable lessons from other research
operations covered on my Substack. I’ll cover the orgs in no particular
order.
#### Striking the Balance of BBN and CMU’s Autonomous Vehicle Group
FreakTakes recently covered two historically great DARPA contractors who
expertly balanced the competing pulls of project novelty and deployable
technology. The first was Bolt, Beranek, and Newman (BBN), the
contractor primarily responsible for the ARPAnet. The second was
Carnegie Mellon’s autonomous vehicle groups.
BBN embodied what it meant to be a “middle ground between academia and
the commercial world.” The firm was initially set up by MIT acoustics
professors to pursue their contracting work more ambitiously. In its
early decades, the firm gradually expanded its contracting efforts into
the computing space, initially under the leadership of BBN VP J.C.R.
Licklider. BBN soon became a common home for the best researchers in
Cambridge, abandoning their academic positions to work for BBN. The
firm’s growing reputation even earned the monicker the “third university
of Cambridge.”
The firm’s revenue was primarily sourced from research contracts given
out by orgs like DARPA, research grantmakers, and aerospace firms. BBN’s
positioning was somewhat unique; when compared to industry, the firm
emphasized novelty and cutting-edge technology work. This insistence on
novelty helped the firm recruit individuals who felt a bit too talented
to waste away working on derivative projects at Westinghouse. When
compared to academia, BBN emphasized working on real technology that
people would use in the near term. J.C.R. Licklider is just one
prominent example of an individual who left a tenureship at MIT to work
on more useful technology down the road at BBN. Leveraging this
positioning, the firm was able to recruit the best talent.
BBN also provided its most talented individuals latitude to ply their
minds broadly. Many projects at BBN showcased the extreme potential of
small teams of talented individuals with broad technical knowledge. Only
eight BBNers were primarily responsible for pushing the early ARPAnet
into existence. The size of the team was no accident; Frank Heart, the
engineering lead of the project, described why he preferred a team of
this size in his [oral
history](https://conservancy.umn.edu/handle/11299/107349):
> I tend to believe important things get done by small groups of people
> who all know all about the whole project. That is, in those days all
> the software people knew something about hardware, and all the
> hardware people programmed. It wasn’t a group of unconnected people.
> It was a set of people who all knew a lot about the whole project. I
> consider that pretty important in anything very big. So I suppose if
> you call it a management style, that would be something I’d state. I
> think also that they were a very, very unusually talented group. I
> think things tend to get done best by small groups of very, very good
> people — if you can possibly manage that. You can’t always manage it.
> So if you again want to call it a management style, it is to get the
> very, very best people and in small numbers, so they can all know what
> they’re all doing.
Lockheed Skunk Works legend “Kelly” Johnson also held quite similar
beliefs when putting together teams to build experimental aircraft.
Particularly in the early stages of novel projects, there is a strong
case for keeping things small, with specialists who understand the
fields that touch theirs. To me, Jeremy’s belief in small teams seems
well-validated by technical history.
BBN demonstrates the ideal case of a research firm that wholly embraces
technical novelty. CMU can be thought of as the flipside of that coin: a
university that wholly embraced systems-building and used firm-like
management practices to do so. The highlight of CMU’s later-1900s
systems work was its autonomous vehicle projects. The academic group
staffed itself with researchers responsible for technical integration
and management-style work to effectively carry out novel technological
systems building.
Similar to BBN, CMU’s positioning was differentiated from both industry
and academia. This fact became very clear as DARPA’s mid-1980s
autonomous vehicle work progressed. CMU was seemingly the only
contractor excited about technical novelty *and* systems integration.
Martin Marietta — the defense prime in charge of DARPA’s Autonomous Land
Vehicle — obsessed over ways to hit DARPA’s demo benchmarks while using
unambitious, dated technologies. Simultaneously, the academic vision
research groups cared more about using the camera data to write papers
than helping directly contribute to building a functional driving
system. CMU was the only contractor involved in the project who truly
cared about building a novel, functional system. DARPA eventually
recognized this and gave them ample funds to build successive
generations of autonomous vehicles. The rest was history.
CMU carried out this work with a management structure that was more
firm-like than most academic labs. For example, Chuck Thorpe did project
management-style work for the team with firm-like incentives — he was a
researcher promoted based on vehicle performance, not his h-index. While
this was a firm-like position, the group also had academic-style
positions. Its use of graduate students on the projects is one prominent
example. Each student on the project had to own a piece of the project
that was all their own and could be written up as a thesis.
*(I explore how the team mitigated the risks of theses not panning out
in [my interview with Chuck
Thorpe](https://www.freaktakes.com/p/an-interview-with-chuck-thorpe-on).)*
These academic incentives partially enabled the CMU team to continually
innovate. Oftentimes these students’ theses perfectly plugged into
existing systems, such as a thesis on reducing the processing time of an
existing sensor’s data from 15 minutes to 90 seconds. But on the most
extreme occasion, in 1988, this incentive structure led a grad student
named Dean Pomerleau to successfully train a neural net to steer the
vehicle. In that particular case, the requirement to allow each grad
student to try something new changed the world.
Answer.AI similarly cares about deep technical novelty and building
deployable technology. As such, Answer.AI could benefit from emulating
BBN and CMU’s strategies to balance the two. The success of BBN and CMU
should hopefully embolden Answer.AI’s founders to trust in the
priorities they have set. This balance of goals is uncommon today, but
orgs from history have expertly balanced the two to world-changing
effect.
With that said, time elapsing without world-changing results might be
unnerving. This ambiguity is partially what pushed academia to rely on
near-term outcome variables that incentivize the incremental. Fear of
wasting time and money is real. The Answer.AI founders would surely like
some way to ensure that they are spending theirs on good problems. To
deal with that, I think the Bell Labs’ approach to problem selection has
a lot to add to the approaches I’ve already covered.
#### The Bell Labs Approach to Problem Selection
Bell Labs’ management of researchers in its golden era is famous — and
it should be. However, Bell researchers were not left to their own
devices to pursue whatever they wished, despite what many think. Bell
managed their researchers with an approach similar to the “long leash
within a narrow fence” approach of GE — which one long-time Bell chemist
called “circumscribed freedom.” The most effective tool it used to do
this was its corp of excellent systems engineers.
Bell had an expansive product line with massive scale — even more so
than GE. Even modest improvements from the research team could have
outsized returns. This, of course, is not the case with Answer.AI.
However, Bell’s use of systems engineers can still be extremely
instructive to Answer.AI, even if Answer.AI may deploy them in different
ways.
As I covered extensively on FreakTakes, Bell’s systems engineers often
combined several knowledge bases to expose the *right* researchers to
the problems that *most* needed solving. Within one mind systems
engineers often combined STEM backgrounds, knowledge of the nitty-gritty
details of Bell’s manufacturing, an understanding of Bell’s
implementation problems, detailed knowledge of Ma Bell’s expenses, and
familiarity with the researchers at Bell Labs.
Bell knew these systems engineers were a massive part of their secret
sauce, ensuring Labs deployed its limited resources on the right kinds
of problems with sufficient upside. I wrote in the conclusion of my Bell
Labs piece:
> Finding *a problem* in these systems is not so hard for those familiar
> with the systems. That’s why many researchers and engineers do not
> feel the need to bring in help. But finding a set of *good
> problems* is not finding the *best problems*. Finding the best
> problems is a profession in and of itself. A systems engineer is worth
> it when, under the right scrutiny, it might turn out that the best
> problem is 10X as financially valuable, does 50X the social good, or
> is 2X as likely to work as just some run-of-the-mill good problem.
My prior piece delves deeper into the specific problems towards which
Bell’s systems engineers led Bell’s researchers. For now, suffice it to
say that I think that any new applied science org that can dedicate an
(ideally full-time) individual to doing the work of a systems engineer
should strongly consider it.
Of course, these systems engineers would need somewhat clear marching
orders on what sorts of technologies they should be exploring. Nowadays,
many existing roles in industry and academia train individuals to equate
revenue or potential citation count with impact. Answer.AI will not be
satisfied with these metrics as proxies of impact, and they shouldn’t
be. What to direct these systems engineers toward instead should surely
be up to Jeremy and Eric.
As a firm that pursued novelty and was not attached to a large
industrial operation, BBN might be an interesting source of inspiration.
Several of its hallmark projects started with systems-engineer-style
contributions. Three examples are DARPA PM Larry Roberts putting out the
ARPAnet contract, J.C.R. Licklider’s visionary Libraries of the Future
Project, and BBNer Jordan Baruch’s early-1960s pitch to the NIH on a
system to build a computer system to facilitate modern hospital
operations. All three project initiators had Bell systems-engineer-like
exposure to the people and problems of their field — technical
backgrounds, regularly spoke with the best academic researchers, knew
modern industry’s issues, were able to project the costs and
complications of potential projects, etc.
Few have used systems engineers as effectively as Bell Labs, with
Bell-style goals, since the great lab was broken up. It would be amazing
to see a lab like Answer.AI commit significant staff time to this
purpose.
#### The Cautionary Tale of Thinking Machines Corporation’s Funding
The case of the Thinking Machines Corporation (TMC) is not as directly
instructive as the examples above, but TMC made one key mistake that
makes it worth mentioning. TMC put itself in the unfortunate position of
raising some of its funds from investors whose incentives were not
aligned with theirs.
For those who don’t know, TMC was a complete failure as a commercial
firm. As a result of its bankruptcy, many write the firm off as a
holistic failure. But the firm did accomplish many of the technical
goals it set out to achieve. Since the company was conceived with
technological goals in mind, rather than a specific market, this was no
small feat. The company was founded by Danny Hillis, a PhD from Marvin
Minsky’s lab at MIT. Through TMC, he sought to build the machine he
conceptualized in his graduate thesis: a truly parallel computer to
improve the capacity of all scientists. The young company recruited the
best researchers — including scoring Richard Feynman as its “intern” for
several summers — and achieved many technical goals that helped pave the
way for the modern field of parallel computing. Jeremy, who knows far
more about the technical aspects of parallel computing than I do, sang
TMC’s technical praises in our first conversation. He emphasized how
shocking it was that TMC seemed to be the first to employ so many
methods the field still uses today.
However, TMC’s high-powered team and great technical work were not
enough to overcome their management follies. The firm spent money as if
its financial standing was in line with its technical reputation, which
it was not. In retrospect, some of these management decisions — such as
a comically expensive long-term lease — could have been avoided without
modifying the company’s general approach. However, on a deeper level,
there was a dissonance between the company’s technical goals and the
funding it raised.
TMC’s two major funders were DARPA’s computing office and private
investors. The goals of Hillis and his top-flight technical staff were
only aligned with DARPA’s goals. In looking to build the technically
most ambitious parallel computer possible, DARPA funding was ideal. The
DARPA computing office also felt that TMC’s work was progressing
exceptionally well for most of its early years. However, the level of
enthusiasm Hillis and the technical staff had for building a machine for
science did not bleed into the most profitable areas — like deploying
the machine on banking databases or managing logistics for Walmart. As
time wore on, it seems that pressures were beginning to mount for TMC to
pursue work more in line with those areas. It seems highly likely that
TMC would have had to disappoint one of its funders sooner or later,
even if it spent funds more wisely in its early years.
Had TMC just raised funds from DARPA and spent them much more modestly,
the company might still exist today; it may have even earned a
reputation beyond that of NVIDIA.
## Copying mid-20th C. Industrial R&D Models is Hard for Incumbents
The middle 20th Century saw both the rise and fall of ambitious American
industrial R&D labs. It is not just nostalgia that makes modern
researchers look back on these labs with fondness. Not long after these
labs were formed, it was becoming clear to many top researchers that the
model was special.
In 1927, Karl Compton wrote a prescient letter to *Science* that praised
these labs. At the time of writing the letter, Compton was the head of
Princeton’s Physics Department and a part-time GE contractor. His letter
asserts that these organizations were doing some of the best science in
the country, even though the top universities often had the best men. A
portion of the letter, in which Compton praises the labs’ management of
scientific projects, reads:
> This has been strikingly demonstrated in some of the big industrial
> research laboratories, from which the output has greatly exceeded the
> individual capacities of the research workers
Compton believed there were *many* lessons that university departments
should steal from these exceptional industrial labs. The first was the
need to specialize when building a portfolio of researchers and
projects. Why should every department attempt to loosely approximate the
makeup of researchers and research questions in the field as a whole? No
company would ever do such a thing. He believed that, “Such a policy
dissipates effort, and if every institution followed it we should have
the spectacle of a great many universities all very much alike and all
with struggling, mediocre departments.”
On an organizational level, Compton believed the equilibrium of N
autonomous professors with N separate budgets and a few grad students in
their lab under their control was just not efficient for most projects.
It would be silly for all departments to function that way. In the
letter, Compton proposes what I’ve taken to calling a ‘Compton Model’
research department. This model is far more structured than a department
of mostly autonomous professors doing ad-hoc research with their own
separate funds. Compton describes it as follows:
> There is another direction in which more effective organization is
> possible within the universities themselves! Departments of a somewhat
> more flexible nature than those to which we are accustomed and which
> could, more than now, be built around one or two outstanding men in
> the department, could give these men an opportunity for organization
> and concentration of effort which is now rarely possible.
Another way to describe this model is as a “fund department heads, not
projects” model. The model allows one or two individuals to largely
shape the research vision, hiring, project selection, capital
purchasing, etc. of an entire department at once. In addition, it allows
these individuals the latitude to replace salaries spent on additional
professors or grad students with full-time engineers or discretionary
capital expenses as needed.
The model makes perfect sense. However, it’s remarkably difficult to
make happen at a real university. Compton couldn’t succeed in doing so
when he took over as MIT President. The existing stakeholders and
structures are just too hard to shift in this direction. Absent some
special circumstance, most university administrators wouldn’t find the
idea even worth considering. However, CMU’s President Cyert was able to
build a department that loosely resembled a Compton Model department.
And its results were exceptional! But this was the exception. Pivoting
an existing department to run like a Compton Model department has proven
infeasible in almost all cases.
The great, old models of managing labs didn’t disappear from industry
because they lacked scientific merit. These labs largely began to
disappear in the 1970s and 1980s. The 1970s saw a deep recession, which
usually hurt R&D budgets. In the 1980s, new corporate management trends
surfaced that led to companies being managed more myopically than ever
before or since. These, along with other non-scientific trends, were
largely responsible for the labs going away when they did. With them,
the operational know-how that had slowly been built up throughout the
century dissipated. Now, to learn how these orgs operated, one must read
oral histories or talk to now-retired engineers.
The first thing these retired engineers will often tell you is that we
should bring the old models back! Bell’s long-time researchers had
confidence in the Bell model even as a late-1900s court case ended Bell
Labs as we know it. In the 1920s, Compton dreamt of copying much of the
GE model to use within a university. One should not be afraid to put
their time and resources behind bringing these models back. Great
engineers largely maintained confidence in these models from their
inception until they faded into the background.
Many in academia know it should operate more like BBN, CMU, or GE
research. They just can’t change the structures to make it happen. Many
R&D leaders know that their company should think more in 10-20-year time
horizons when planning research expenditures, but shareholders and
shareholder-wary CEOs often do not find this view actionable.
**Answer.AI can be free from all this. The org can apply the great old
models in a new era on new technology.**
*(Ries being a cofounder of Answer.AI should be comforting. Ries has
railed against the types of myopic management trends that make
activities like maintaining an expansive industrial R&D lab difficult
for firms. He founded the young Long Term Stock Exchange, in part, to
help mitigate issues like this.)*
## Conclusion: Running Your Own Races
It’s up to Answer.AI’s two-headed management team to stick to the
organization’s comparative advantages. There is no need to race academic
CS researchers or corporate R&D departments in races those two groups
feel incentivized to run. Corporate R&D, NSF-funded CS researchers, and
AGI-focused labs all have areas in which they clearly will and won’t
operate. And those areas don’t come close to covering 100% of the good
ideas somebody should obviously be working on.
[Answer.AI](https://www.answer.ai) is free to run its own races,
uncontested.
Many will feel the org is using an untested model. However, Answer.AI’s
founders — like me — believe that this model is proven, but has simply
gone away for a while. So, shouldering this “organizational risk” that
others are seemingly not willing to do, they have the chance to work on
problems without much competition. If their USD10 million experiment
works, it has the chance to spark a rush of emboldened researchers and
engineers to found small research firms, leveraging the models of the
once-great dragons of American industrial R&D.
I wish Jeremy and Eric luck in the early stages of their mission. To any
researchers and engineers — across all areas — reading this and wishing
a BBN, CMU, TMC, or Answer.AI existed in your area, please reach out to
me on [Twitter](https://twitter.com/eric_is_weird). I’d love to see if
there’s anything I can do to help.
---
# Source: https://www.answer.ai/overview.md
# Answer.AI - Practical AI R&D
Answer.AI is a new kind of AI R&D lab which creates practical end-user
products based on foundational research breakthroughs. To learn more
about the research agenda and background behind our work, see our launch
post and an independent analysis of our lab. See below for a complete
list of published articles.
------------------------------------------------------------------------
### /llms.txt—a proposal to provide information to help LLMs use websites
We propose that those interested in providing LLM-friendly content add a
/llms.txt file to their site. This is a markdown file that provides
brief background information and guidance, along with links to markdown
files providing more detailed information.
Sep 3, 2024
Jeremy Howard
### Small but Mighty: Introducing answerai-colbert-small
Say hello to answerai-colbert-small-v1, a tiny ColBERT model that
punches well above its weight.
Aug 13, 2024
Benjamin Clavié
### How HTMX is Revolutionizing Web Development — Interview with Carson Gross
We recently sat down with Carson Gross, the creator of HTMX, to discuss
how it’s changing the web.
Aug 4, 2024
Elizabeth Creason
### FastHTML: Modern web applications in pure Python
FastHTML is a new way to create modern interactive web apps. It scales
down to a 6-line python file, and scales up to complex production apps.
Aug 3, 2024
Jeremy Howard
### JaColBERTv2.5🇯🇵: Optimising Retrieval Training for Lower-Resources Languages
Introducing JaColBERTv2.5🇯🇵, the new best Japanese retrieval model.
Through this release, we present a thorough analysis to better
understand what helps in training a good multi-vector retrieval model.
Aug 2, 2024
Benjamin Clavié
### Cold-Compress 1.0: A Hackable Toolkit for KV-Cache Compression
Blog
Answer.AI is proud to publicly release **Cold Compress 1.0** , a toolkit
for creating and benchmarking state-of-the-art KV cache compression
methods. Built on top of GPT-Fast, it…\*\*
**\*\*\*\***\*\*\*\***\*\*\*\***\*\*\*\*\_\_\*\*
Aug 1, 2024
Griffin Adams, Faisal Ladhak
### Faith and Fate: Transformers as fuzzy pattern matchers
Are GPT-like models thinking? Unclear. But the *Faith and Fate* paper
(Dziri, 2023) points out they are often “just” pattern matching.
Jul 25, 2024
Alexis Gallagher
### gpu.cpp: portable GPU compute for C++ with WebGPU
We’re thrilled to announce the release of gpu.cpp. gpu.cpp is a
lightweight, open-source library that makes portable GPU compute with
C++ simple.
Jul 13, 2024
Austin Huang
### Politicians can’t change math
It is not possible to provide assurance that a released model can’t be
used to cause harm, because a released model can be changed.
Jun 28, 2024
Jeremy Howard
### A little pooling goes a long way for multi-vector representations
We’re releasing an early version of a simple token pooling trick for
ColBERT. This allows for considerable memory&disk footprint reduction
with very minimal retrieval…
Jun 27, 2024
Benjamin Clavié
### A new kind of literate program – Claudette
Claudette is the first ever ‘literate nbdev’ project. Its source is a
rendered Notebook with callout notes and tips, HTML tables and images,
detailed explanations. It explains how and why the code is written the
way it is, along with how to use the libraries it depends on.
Jun 23, 2024
Jeremy Howard
### Introducing Claudette, a new friend that makes Claude 3.5 Sonnet even nicer
Today, Anthropic launched the most powerful language model available:
Claude 3.5 Sonnet. And today, we are making it ever better, with the
launch of *Claudette*.
Jun 21, 2024
Jeremy Howard
### The definition of ‘Artificial Intelligence’ in SB 1047 was actually meant for systems, not models
SB 1047 defines “AI Model” using a definition designed for “AI Systems”,
and as a result fails to cover anything.
Jun 19, 2024
Jeremy Howard
### What policy makers need to know about AI (and what goes wrong if they don’t)
Policy makers need to understand how AI really works. Otherwise bills
like SB 1047 totally fail to achieve their stated goals.
Jun 17, 2024
Jeremy Howard
### More=Better?
In this post I want to share a few quick experiments to show something
that is both obvious and also perhaps underappreciated: you can get a
sizeable quality boost from a…
May 17, 2024
Jonathan Whitaker
### SB-1047 will stifle open-source AI and decrease safety
By imposing the restrictions on open-source AI, SB-1047 could reduce AI
safety, through reducing transparency, collaboration, diversity, and
resilience.
Apr 29, 2024
Jeremy Howard
### Efficient finetuning of Llama 3 with FSDP QDoRA
We’re releasing FSDP QDoRA, a scalable and memory-efficient method to
close the gap between parameter efficient finetuning and full
finetuning.
Apr 22, 2024
Kerem Turgutlu
### A few tips for working on high-surface-area problems
What do you do when there are too many pieces to fit in your head?
A few tips for working on things that involve a lot of moving pieces,
that have a high “surface area”.
Apr 12, 2024
Johno Whitaker
### Enabling 70B Finetuning on Consumer GPUs
A Technical Deep Dive into FSDP+QLoRA
A detailed guide for adding FSDP and QLoRA support to quantization
libraries and training frameworks.
Mar 14, 2024
Benjamin Warner, Johno Whitaker, Kerem Turgutlu
### You can now train a 70b language model at home
We’re releasing an open source system, based on FSDP and QLoRA, that can
train a 70b model on two 24GB GPUs.
Mar 6, 2024
### Lessons from history’s greatest R&D labs
A historical analysis of what the earliest electrical and great applied
R&D labs can teach Answer.AI, and potential pitfalls
Jan 26, 2024
Eric Gilliam
### A new old kind of R&D lab
Answer.AI will figure out the fundamental research needed to tame AI,
and the development path needed to make it useful in practice
Dec 12, 2023
Jeremy Howard
A new old kind of R&D lab
Answer.AI will figure out the fundamental research needed to tame AI,
and the development path needed to make it useful in practice
