Gary Marcus：教皇通谕与AI认知科学反思

In my view, Pope Leo XIV is by far the clearest thinker on AI among prominent world leaders. It was a genuine thrill to meet him yesterday.

30 October 2026

It is understatement to say that I am a huge fan of his Encyclical on AI, the Magnifica Humanitas. It goes beyond words that I was able listen to him yesterday in person, from just a few feet away.

He gave a brilliant, short speech, which you can find here.

The Pope, delivering his remarks on AI

His central five points, which I fully endorse:

First, the future of artificial intelligence is not pre-determined. As a technology developed by human beings, its future depends on the choices we make. We therefore have the responsibility to guide, change and, when necessary, reject its applications (cf. nos. 97, 104). The decisive question, then, is not simply “What can AI do?” but rather, “What purpose should it serve?” Those who regard its development merely as a race to be won risk focusing exclusively on the finish line, forgetting those who are left behind. A reliable system that serves the human person is of greater value than one that is exceptionally powerful but whose consequences cannot be adequately foreseen or controlled.

Second, machines are tools, not persons. “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean” (no. 99). Today, some propose granting these systems rights, or even legal personhood. Such an approach risks obscuring human responsibility and leaving those who suffer harm to confront a faceless entity.

Third, certain decisions must never be delegated. When a person is placed on a target list or entered into a border control database, there is a danger that he or she will be reduced to a mere data point. Decisions affecting human life, freedom and the use of force must always remain subject to effective, informed and responsible human oversight (cf. nos. 198-200). For this reason, artificial intelligence must be “disarmed” (no. 110), and I encourage the States meeting in Geneva in November to take concrete steps toward banning lethal autonomous weapons.

Fourth, intelligent technologies must be directed toward the common good. “Justice requires access to the benefits of innovation, including care, knowledge, tools and opportunities” (no. 161). Indeed, they must never become ends in themselves or serve merely the interests of a privileged few. Rather, their development should foster solidarity and participation at various levels of the global community, ensuring that the benefits of progress are shared equitably, especially with those most in need (cf. nos. 71–72).

Finally, I would like to emphasize a challenge, which is perhaps the most urgent. We already know many of the ethical principles that should guide the development of artificial intelligence. What we currently need are institutions capable of putting these principles into practice, particularly effective international institutions that can safeguard the global common good (cf. no. 64). This calls for shared standards, independent oversight and clear mechanisms of accountability, so that decisions concerning the development and use of AI can be examined, challenged when necessary, and accompanied by appropriate remedies for any harm caused (cf. nos. 105, 107).

(The numbers refer to the Encyclical.)

As a bonus, here’s a transcription of own (improvised) remarks on technical, political, and philosophical aspects of AI, given earlier in the day, a commentary after Max Tegmark, Stuart Russell, and Valérie Pisano spoke, at a small private meeting just before our group met with the Pope:

There are so many ideas here [today] that are interesting and deserve a response.

I’ll start by saying it’s a tremendous honor to be here and to be in this illustrious audience.

For those who don’t know me, I’m first and foremost a cognitive scientist. I trained with Steven Pinker at MIT, and my dissertation was on a mix of things. It was on how humans learn about the world, how children learn language, but also about neural networks, neural networks that are the ancestors of the ones that are popular right now.

And to the extent that I’ve had something to say about the field, It’s been by comparing how human intelligence works, how natural intelligence. I also worked with animals, how natural intelligence works with how machine intelligence works. And I think that’s given me a little bit of a different perspective on how things go.

I also spent some time as an entrepreneur. I built an AI company that I sold to Uber, and so I’ve seen both the industry side and the academic side. So that’s where some of my background comes.

So I agree, for example, with Max [Tegmark] about many of his recommendations, for what the commission might put forward. And maybe the most profound thing that I agree with is with Stuart [Russell], that the current path is untenable. and that the sooner that the industry recognizes that, the better. And I agree with Valérie [Pisano], that we’re mistaking things for inevitability, that far too much of the discussion is people taking for granted that we are on the only path, rather than maybe a different path.

And I agree that we still have agency, and that we need to look at things differently. So, as a cognitive scientist, I begin to look at things differently from the very statement of what AI is. As a cognitive scientist, I look at the nature of intelligence.

We sometimes talk about the modularity of intelligence, that there are many different components that go into intelligence. So, there’s verbal intelligence, and mathematical intelligence. And, kinesthetic intelligence about how we move our bodies in the world, and so forth.

So one of the disagreements that I would mark is actually with something that Stuart said, which I think was partly true. Stuart said that we are humans, we predict, and we do, we certainly predict. There’s ample evidence from the cognitive science literatre that we make predictions, including literally of the next word.

But there’s also lots of evidence from cognitive science that we do lots of other things. And so, in addition to making predictions, we do reasoning, we have emotional judgments and so forth. The mind is, as one lovely quote [put it “not one thing, but many.”

And I think that the fundamental mistake that the field has made, is to assume that capturing one component of intelligence, prediction extraordinarily well, is enough to capture all of intelligence. And we now, though we didn’t always, have an entire industry around this, as Stuart, who wrote the most famous textbook or co-wrote the most famous textbook on AI, knows well, we used to actually have many different approaches to AI, and the fields settled on one, the large language model, which is the best prediction machine that we have. And it does that very well, but it doesn’t do some of the other things very well.

I think it’s still very poor reasoning. And the thing that’s maybe most dangerous and that, I think, captures what all of us [speakers at this session] are saying here, is [current AI] doesn’t follow instructions very well. And that might be surprising, but it turns out that the way that these prediction machines work is they follow correlations rather than causation.

I would say that there’s semantics of the world, their understanding of the meaning of the world is fairly superficial. And so they will often match the things that we hope that they will, but there’s no guarantee, and we see that in many different ways. For example, you can tell a large language model, don’t make up any references.