Doug Lenat, 1950-2023 – by Gary Marcus

Doug Lenat was one of the sensible, acerbically humorous folks I’ve ever met. If folks like Marvin Minsky, John McCarthy, and Alan Newell have been among the many first to suppose deeply about how symbolic AI, through which machines manipulate specific verbal-like representations, would possibly work, Doug was the primary to attempt actually exhausting to make it really work. I’ve spent my entire profession arguing for consilience between neural networks and symbolic AI, and on the strictly symbolic aspect of that equation, Lenat was light-years forward of me, not simply extra deeply embedded in these trenches than I, however the architect of a lot of these trenches.

Lenat spent the final 40 years of his life launching and directing a challenge known as Cyc, an intense effort to codify all of widespread sense in machine-interpretable kind. Too few folks desirous about AI at present even know what that challenge is. Many who do, write it off as a failure. Cyc (and the guardian firm, Cycorp, that Lenat fashioned to incubate it) by no means exploded commercially – however hardly anyone ever provides it credit score for the truth that it’s nonetheless in enterprise 40 years later; very only a few AI firms have survived that lengthy.

My very own view is that Cyc has been neither successful nor a failure, however someplace in between: I see it as a ground-breaking, clarion experiment that by no means totally gelled. No, Cyc didn’t set the world on hearth, however sure, it should appear increasingly essential in hindsight, as we ultimately make actual progress in direction of synthetic normal intelligence.

Most younger AI researchers have by no means even heard about it. However each single one in every of them ought to know one thing about Cyc. They don’t want to love it, however they need to perceive what it was, and what it tried to do, and what they could do as a substitute to perform the identical targets.

Not as a result of Cyc will get used out of the field, as some form of drop-in alternative for Massive Language Fashions, however as a result of what Lenat tried to do – to get machines to symbolize and motive about widespread sense — nonetheless should be accomplished. Yejin Choi’s fantastic 2023 TED speak, Why AI is incredibly smart and shockingly stupid, adopted immediately in that custom, explaining why widespread sense continues to be, regardless of their obvious success, missing in present AI techniques. (My 2019 e-book with Ernie Davis, Rebooting AI was very a lot on the identical subject.)

Metaphorically, Lenat tried to discover a path throughout the mountain of widespread sense, the tens of millions of issues we all know in regards to the world however hardly ever articulate. He didn’t totally succeed – we are going to want a unique path – however he picked the vital mountain that we nonetheless should cross.

That’s what Lenat, Choi, Davis, and I’ve all been making an attempt to say, and it’s precisely the place Massive Language Fashions battle, time and again. To take however one in every of a zillion continually altering examples, this very morning somebody despatched me, with Google Bard mixing collectively fact with apparent nonsense, in utterly fluent paragraphs:

Relying on the wording, any given mannequin would possibly or won’t get questions like that proper on any given day; Massive Language Fashions are typically correct in some wordings, inaccurate in others. From them we generally see an phantasm of widespread sense, relying on the vagaries of what the coaching set is and the exact wording of a query, however one thing is clearly nonetheless lacking. And even when this particular instance is patched up, there’ll inevitably be others of an analogous taste. Cyc is an effort to discover a deeper, extra sturdy reply.

Because the AI researcher Ken Forbus of Northwestern College put it to me in an e mail this morning, “The Cyc challenge was the primary demonstration that symbolic representations and reasoning can scale to seize important parts of commonsense. Whereas at present billion-fact information bases are widespread in business, Cyc stays essentially the most superior when it comes to expressiveness, capturing extra of the vary of ideas that people are able to. My group has been utilizing Cyc’s representations in our analysis for many years… Our subject would do properly to be taught extra from the Cyc challenge.” A Google researcher, Muktha Ananda, Director of their Studying Platform, wrote condolences to me this morning, “I’ve at all times been an important admirer of [Lenat’s] imaginative and prescient, perseverance and tenacity. His work on Cyc was an important supply of inspiration for my very own journey into information graphs/webs.”

§

Over the past 12 months, Doug and I attempted to put in writing a protracted, advanced paper that we by no means received to complete. Cyc was each superior in its scope, and unwieldy in its implementation. The most important downside with Cyc from a tutorial perspective is that it’s proprietary. To assist extra folks perceive it, I attempted to convey out of him what classes he discovered from Cyc, for a future technology of researchers to make use of. Why did it work in addition to it did when it did, why did fail when it did, what was exhausting to implement, and what did he want that he had accomplished otherwise? We had almost 40,000 phrases, sprawling, not but totally organized, but full of knowledge. It was half science, half oral historical past. For sure, it takes a very long time to arrange and polish one thing of that size. In between our different commitments, we have been making gradual however regular progress. After which within the new 12 months, I received busy with AI coverage, and he received sick; progress slowed. Nearer to the top, he wrote a shorter, tighter paper, constructing partly on the work we had accomplished collectively. When he realized that he didn’t have a lot time left, we agreed that I’d assist polish the shorter manuscript, a junior accomplice in what we each knew would seemingly be his final paper.

Considered one of his final emails to me, about six weeks in the past, was an entreaty to get the paper out ASAP; on July 31, after a nerve-wracking false-start, it got here out, on arXiv, Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc. The transient article is concurrently a overview of what Cyc tried to do, an encapsulation of what we should always anticipate from real synthetic intelligence, and a name for reconciliation between the deep symbolic custom that he labored in with fashionable Massive Language Fashions.

In his honor, I hope you’ll find time to learn it.

Gary Marcus has centered his profession on integrating neural networks and symbolic approaches; he nonetheless hopes that can occur.