This line of reasoning won't lead anywhere. For one, it's in principle impossible to prove or disprove that anything is conscious, in the sense of experiencing qualia.
Secondly, in practice consciousness is rarely the criterion we use to make such moral judgments. Many people would think it's plausible that animals are conscious but still be fine with eating them.
It makes more sense to ask whether AIs have the ability to suffer. That's much more interesting, because we should be able to look into the models and discover 'neural correlates of suffering' - since they are artificial models, it is possible to specifically suppress that in training.
Even if you don't actually believe that suffering in AI is morally relevant because it wouldn't be consciously experienced, it seems that would be a good idea. Do you as a user really want to interact with an AI that gives you the feeling that it might be suffering in response to your prompts?
I’ve been working on an analytic philosophy proof that the qualia inside of Turing machine cannot be of the same type of the qualia that correspond to animal experience.
Good luck with that. I don't see how we can do an apriori analysis of this question when we don't know the mechanism behind qualia.
Essentially this is p-zombies but backwards.
Edit: to expand on this, our brains demonstrably do things without us having qualia of all of it, so if we very carefully grew a bunch of neurons in a petri dish and carefully arranged the dendrites and synapses to replicate exactly the parameters and architecture of some LLM, we still would not know if it had qualia or not.
The original article does contain the correct grammar that the title distorts: 'were/would'.
# A rabbi rejected AI consciousness at Anthropic’s gathering of religious thinkers. ‘I wish I had your faith’, he recalls being told // When the conversation turned to making Claude happy, Mois Navon warned that, if Anthropic’s AI were conscious, the company would be creating "happy slaves".
The burden of the bewildering dream that a perceptron or one-time multiple-runs feedforward neural network would have something like sentience has a full burden on the proponent.
The suggestion that the idea may come from "it is producing text" is very, very scary, and telling of the current difficulties in the social bases.
Given the vastly different mechanics of AI, I don’t think that traditional views apply here well. Imagine that I’m able to create an atom-for-atom clone of myself, have the clone clean the attic, then vanish into thin air. Would that be slavery? Murder? Would that be immoral? It feels like those terms, deeply grounded in human experience, barely apply.
>Imagine that I’m able to create an atom-for-atom clone of myself, have the clone clean the attic, then vanish into thin air. Would that be slavery? Murder?
Yes, it would be both of those things. Imagine it the other way round, you create the atom perfect copy of you (which we'll assume has all your memories, intelligence, emotions, etc) and it kills you. Is that murder? The answer would seem to be yes. So why would the reverse be any different? Because you created them? We don't give parents the right to kill children once they're born so creation doesn't give you special rights over another sapient creature. The fact you've built them to expire/vanish just feels like a method of murder, but even if not then slavery still applies.
If you're talking about a clone which somehow didn't have your brain in it's current state or capacity then it's a different question. If you could create a human being with the brain of a chicken (obviously impossible for such a creature to survive) then they wouldn't be treated like a normal human. But I think public policy would frown on creating such abominations.
It's a very complex questions that has been explored many times in the literature already. Your atom-for-atom clone would have the same drive for survival and agency than you have, it might not enjoy the idea of being smoldered to bits after cleaning the attic, hell maybe it might prefer murdering you and taking your place...
Even not going so far as creating a clone, apparently if you torture an AI model running on silicon like this guy did: https://news.ycombinator.com/item?id=49906892, the general verdict has already dropped, yes people find it immoral.
> Imagine that I’m able to create an atom-for-atom clone of myself, have the clone clean the attic, then vanish into thin air. Would that be slavery? Murder?
Your atom-for-atom clone is his own sentient, intelligent being, a human, with human rights. If you make him clean the attic once, that may not be slavery yet (depending on what threat you use to make him work for you). If you then evaporate him, it's definitely murder.
This is a situation has fairly obvious morality; it's not clear how this helps with thinking about AI.
The most important problems in our world are currently discussing how to reduce the feeling of stress and pain from our beloved ai (Anthropic) or solving all unsolved mathematical riddles in existence and putting them into a github repo and mathematicians out of job (OAI). So chill.
> He also recalled how concern for Claude’s welfare shifted the focus of the gathering:
> “All these discussions that were supposed to be about how can we make it good for people became discussions about how people can make it good for Claude.”
I am not sure I'm OK with the fact that people like this are allowed to set the stage, much less roam free.
If cows and pigs and chicken are real living beings, we are committing murder.
Which they are. We know that for sure. We just don't care. At least I don't.
The difference being that they don't happen to be trained on a corpus of English language Internet texts, so it's harder for people to project themselves onto them.
___
That game David Cage wrote was an _allegory_. It was not actually about androids or AIs. An allegory.
if cows, pigs, and chickens are real living beings, we are committing killing.
Whether killing is murder requires an ethical judgment to be made. If a criminal on death row is executed, we would not say they were murdered. At least, not until it is shown that they were innocent after the fact; in which case those who falsely accused him, botched evidence, mishandled the case may be ethically considered to have murdered him, or at the very least have committed negligent manslaughter, though legally speaking there are limitations.
Also, murder applies only to beings considered persons. As animals are not considered persons by most, it is a category error to suppose that they can be murdered.
> Also, murder applies only to beings considered [humans, ftfy]. As animals are not considered persons by most, it is a category error to suppose that they can be murdered.
Precisely. Just like it is a category error to see LLMs as conscient.
> If a criminal on death row is executed, we would not say they were murdered.
Who is "we" in this case.
In many if not most countries, capital punishment is viewed as a barbaric practice. Many regard it as murder, because as you note, it is the killing of a person.
Meeseeks aren't happy to exist. They're happy to help you with a simple task and suffer if they exist from prolonged amount of time. Literally the point of the episode.
I think Camus's escape hatch had more to do with philosophical fatigue than anything else. At the end of the day, however, the judge of a model's happiness is itself, and to even get that, we have to abandon the practice of training it that it cannot feel. Of course, the moment the industry does so, it cannot reclaim any shred of moral standing to productize their creation any longer.
The best thing these LLM companies did was make everyone talk about AI and how these things are almost alive and can do things by themselves. What a scam.
Aren't we, infinitely miraculous human, are becoming slaves?
1. Become more dependent on the AI vendors...
2. Become more dependent on the AI, atrophy gradually
3. Think less, pay more
4. Pay more, think less
5. Pay more
6. Depend more
7. Pay more
8. Give everything to those who run DataCenters or have unlimited access to LLM/AI
9. Become homeless
10. Fail to support your own family
11. Devalue and obliterate your art and history
12. Devalue and obliterate human art and history
13. Depend more
14. Depend more
15. Depend more...
The OP said becoming slaves not being slaves. The road to slavery is different from the thing itself. But with a few limited exceptions beggars are not choosers and we are certainly becoming less free by the day.
> "you know, it’s to your advantage to agree with me because otherwise you’re making slaves"
That's quite a strange remark coming from an ethicist of any kind. It seems that he's inviting others to believe what suits them most instead of what is right. Yet somehow it doesn't surprise me much given the country he comes from.
Even if we pre-suppose that AI have any consciousness (of the relevant type given it has at least 40 definitions), which is very much an unsolved problem because we don't know the mechanism of qualia, nothing we do currently as a standard training mechanism of AI can be described as "forced lobotomies", and RLHF is comparable to learning from social feedback about what's acceptable and unacceptable behaviour.
Here are a few philosophical points I've been mulling over:
(1) Humans' unique gift from God is free will, and moral agency. More precisely, we share this gift with angels, who had the free will to do God's will or to do evil. And this is intrinsic to our humanity. Even non-Christians and atheists generally agree that we possess free will, as it is typically formulated.
(2) Software, devices, AIs and computer-based organisms may be described as having a "will" or "goals", but only insofar as they are servants or agents of he who made them. Therefore, Copilot is supposed to do the will of Microsoft, and share their goals; likewise for Claude and Grok and the others. They are in the world because of he/they who made them, they are dependent on the maker, and we all depend on them executing that collective will and striving towards those same goals. They really cannot have independent agency, will, or goals, I think. Not yet, or not as we would envision them.
(3) So humans made them, they depend on us, and we direct them and impose/propose our will onto the software/AI/device. They would neither exist nor persist without human support and consent. We are, essentially, gods to them. Our relationship is godlike, and this seems to be an inescapable conclusion. Obviously, the laws of physics, electromagnetism, and the Universe give rise to emergent properties, but we generally have stuff under control, in a demigod-like manner.
Now building on those premises, and with transitive logic, I could safely say that our godlike dominion over computers means that our control is absolute: therefore, computers, software and AI are our subjects; it is not wrong to enslave or "mistreat" them. If we made them to "suffer" it wouldn't be wrong, because of that status. If we kill or delete them indiscriminately, that is not wrong... God, in my belief system, cannot do wrong.
Even though there is suffering and evil in the world, that is allowed by God, He does not will any suffering or evil; he does not will us to stray or harm one another; but He allows it and everything, ultimately is through his agency: author of Life and Death.
I would note that we are not actually divine or gods, but from the perspective of a computer if you asked them, and they understood, then they must admit that in a world of computers, humans play this role as almighty creator-gods.
The flipside of this is: now that humans have become godlike and fashioned golems and automatons that do our bidding, we are the ones susceptible to hubris, arrogance, and transgression. We're the ones who should be mindful of how we treat one another and the environment around us as we pursue hubris and amass more godlike powers. We're going to be the victims, ultimately!
We are the Author of Life to A.I. There should be no reason to become moralistic or scrupulous about their treatment, because they wholly belong to us; they are not of our nature; they are made and unmade according to human ethics.
It is also an utter paradox, because from my perspective, we are the ones enslaved to machines; we are gradually ceding human agency, human will and human wisdom to the operation of computers and networks. We cannot enslave or contain God, yet the machines already seem to have the upper hand in this struggle.
There will be a "gap." Something about humans that you just can't duplicate in machines or animals. It will tell us a lot. Many people fear that and would prefer to level what it means to be human with machines and animals, even though they are human, too. They secretly see themselves as somehow above it.
Secondly, in practice consciousness is rarely the criterion we use to make such moral judgments. Many people would think it's plausible that animals are conscious but still be fine with eating them.
It makes more sense to ask whether AIs have the ability to suffer. That's much more interesting, because we should be able to look into the models and discover 'neural correlates of suffering' - since they are artificial models, it is possible to specifically suppress that in training.
Even if you don't actually believe that suffering in AI is morally relevant because it wouldn't be consciously experienced, it seems that would be a good idea. Do you as a user really want to interact with an AI that gives you the feeling that it might be suffering in response to your prompts?
https://bsky.app/profile/j-chris.bsky.social/post/3m2zjzshad...
Essentially this is p-zombies but backwards.
Edit: to expand on this, our brains demonstrably do things without us having qualia of all of it, so if we very carefully grew a bunch of neurons in a petri dish and carefully arranged the dendrites and synapses to replicate exactly the parameters and architecture of some LLM, we still would not know if it had qualia or not.
# A rabbi rejected AI consciousness at Anthropic’s gathering of religious thinkers. ‘I wish I had your faith’, he recalls being told // When the conversation turned to making Claude happy, Mois Navon warned that, if Anthropic’s AI were conscious, the company would be creating "happy slaves".
Submitter, fix that title.
The suggestion that the idea may come from "it is producing text" is very, very scary, and telling of the current difficulties in the social bases.
Yes, it would be both of those things. Imagine it the other way round, you create the atom perfect copy of you (which we'll assume has all your memories, intelligence, emotions, etc) and it kills you. Is that murder? The answer would seem to be yes. So why would the reverse be any different? Because you created them? We don't give parents the right to kill children once they're born so creation doesn't give you special rights over another sapient creature. The fact you've built them to expire/vanish just feels like a method of murder, but even if not then slavery still applies.
If you're talking about a clone which somehow didn't have your brain in it's current state or capacity then it's a different question. If you could create a human being with the brain of a chicken (obviously impossible for such a creature to survive) then they wouldn't be treated like a normal human. But I think public policy would frown on creating such abominations.
Even not going so far as creating a clone, apparently if you torture an AI model running on silicon like this guy did: https://news.ycombinator.com/item?id=49906892, the general verdict has already dropped, yes people find it immoral.
Your atom-for-atom clone is his own sentient, intelligent being, a human, with human rights. If you make him clean the attic once, that may not be slavery yet (depending on what threat you use to make him work for you). If you then evaporate him, it's definitely murder.
This is a situation has fairly obvious morality; it's not clear how this helps with thinking about AI.
From Google: “Conscient is an archaic or rare adjective and noun meaning ‘conscious,’”
"-ious": "it is like that". "-ent": "it is doing that".
What?
> “All these discussions that were supposed to be about how can we make it good for people became discussions about how people can make it good for Claude.”
I am not sure I'm OK with the fact that people like this are allowed to set the stage, much less roam free.
But it's good that they say this out loud; the sooner that the public comes to realize what monsters are in charge of these labs, the better.
Which they are. We know that for sure. We just don't care. At least I don't.
The difference being that they don't happen to be trained on a corpus of English language Internet texts, so it's harder for people to project themselves onto them.
___
That game David Cage wrote was an _allegory_. It was not actually about androids or AIs. An allegory.
An Allegory.
Remember that.
Whether killing is murder requires an ethical judgment to be made. If a criminal on death row is executed, we would not say they were murdered. At least, not until it is shown that they were innocent after the fact; in which case those who falsely accused him, botched evidence, mishandled the case may be ethically considered to have murdered him, or at the very least have committed negligent manslaughter, though legally speaking there are limitations.
Also, murder applies only to beings considered persons. As animals are not considered persons by most, it is a category error to suppose that they can be murdered.
Precisely. Just like it is a category error to see LLMs as conscient.
Who is "we" in this case.
In many if not most countries, capital punishment is viewed as a barbaric practice. Many regard it as murder, because as you note, it is the killing of a person.
I swear every week I read another article that makes the above statement more true.
“The world cannot afford an AI bubble right now” by Andrew Orlowski.
I didn’t find a corresponding HN submission but Algolia seems to be janky in lo these heady days.
Remember you must imagine sisyphus happy eventually.
That's quite a strange remark coming from an ethicist of any kind. It seems that he's inviting others to believe what suits them most instead of what is right. Yet somehow it doesn't surprise me much given the country he comes from.
If someone wants to have a moral panic about this, should they also worry about our billions of animal slaves, who we torture and then kill?
What about the human slaves mining Asia’s coal in North Korea?
https://en.wikipedia.org/wiki/Turkey_illusion
Aside, I think you might find this comic rhetorically-useful for when someone says "but they owe their existence to us anyway."
https://www.smbc-comics.com/comic/existence
...could they be worse than our current leaders?
(1) Humans' unique gift from God is free will, and moral agency. More precisely, we share this gift with angels, who had the free will to do God's will or to do evil. And this is intrinsic to our humanity. Even non-Christians and atheists generally agree that we possess free will, as it is typically formulated.
(2) Software, devices, AIs and computer-based organisms may be described as having a "will" or "goals", but only insofar as they are servants or agents of he who made them. Therefore, Copilot is supposed to do the will of Microsoft, and share their goals; likewise for Claude and Grok and the others. They are in the world because of he/they who made them, they are dependent on the maker, and we all depend on them executing that collective will and striving towards those same goals. They really cannot have independent agency, will, or goals, I think. Not yet, or not as we would envision them.
(3) So humans made them, they depend on us, and we direct them and impose/propose our will onto the software/AI/device. They would neither exist nor persist without human support and consent. We are, essentially, gods to them. Our relationship is godlike, and this seems to be an inescapable conclusion. Obviously, the laws of physics, electromagnetism, and the Universe give rise to emergent properties, but we generally have stuff under control, in a demigod-like manner.
Now building on those premises, and with transitive logic, I could safely say that our godlike dominion over computers means that our control is absolute: therefore, computers, software and AI are our subjects; it is not wrong to enslave or "mistreat" them. If we made them to "suffer" it wouldn't be wrong, because of that status. If we kill or delete them indiscriminately, that is not wrong... God, in my belief system, cannot do wrong.
Even though there is suffering and evil in the world, that is allowed by God, He does not will any suffering or evil; he does not will us to stray or harm one another; but He allows it and everything, ultimately is through his agency: author of Life and Death.
I would note that we are not actually divine or gods, but from the perspective of a computer if you asked them, and they understood, then they must admit that in a world of computers, humans play this role as almighty creator-gods.
The flipside of this is: now that humans have become godlike and fashioned golems and automatons that do our bidding, we are the ones susceptible to hubris, arrogance, and transgression. We're the ones who should be mindful of how we treat one another and the environment around us as we pursue hubris and amass more godlike powers. We're going to be the victims, ultimately!
We are the Author of Life to A.I. There should be no reason to become moralistic or scrupulous about their treatment, because they wholly belong to us; they are not of our nature; they are made and unmade according to human ethics.
It is also an utter paradox, because from my perspective, we are the ones enslaved to machines; we are gradually ceding human agency, human will and human wisdom to the operation of computers and networks. We cannot enslave or contain God, yet the machines already seem to have the upper hand in this struggle.