• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

What is AGI - what is consciousness - what is a soul?

K

kram182

2[H]4U
2FA
Joined
Jul 15, 2025
Messages
2,095
How's that for a fucking title?

As long as you're not 'dey turk uhr jurbs/wurtur' about it - which I don't care about per this discussion - I invite 'anti-AI' views here as well. This started as a thread with some AI/alien folks on twitter - in a thread/post started by The Pope of all people lol - so I'm just gonna copy/paste it all into here:

Current 'AI' according to scale hypothesis and current (attempted) implementation can be achieved by just adding complexities/compute (modern GPU/CPU compute) upon complexities/compute ad nauseum until the complexities become so complex it spawns consciousness. Human beings (and all living things actually) have quantum phenomena/quantum processes/quantum microtubules in them - and in humans, in the brain as well/specifically (Penrose Tubules) (more) - and anesthesia shows when it renders us unconscious, it does so by interacting/interferes/disrupting this quantum phenomena in our brain/(Penrose) microtubules. So, IF AGI (what we mean when we refer to 'AI') is ever actually achievable - it is going to take at least, and at least partially, quantum computing/processing - which is not there/ready for primetime yet. So we can not actually achieve AGI yet - if ever.

Is that bullshit? Do you think current non-quantum compute can achieve AGI via scaling/other? Is quantum needed, but in some other way?

Plus before all this - we first have to answer - 'What is consciousness?' - which we don't know/can't define - even within/of/from ourselves. Because without first answering/knowing that - even if we have viable quantum computing and a believed AGI/conscious construct/computer/entity - we have no verification/nothing to test/compare/prove against. Because then not only can we test 'a computer' - but we can see what/if any/if all other animals/living things are 'actually conscious' as well.

It also very well could be that/are consciousness/the soul the same thing? Then I think you have to ask yourself not just 'do I want to create consciousness?' but 'do I want to create a soul?' Which I do feel are two different things to ask and ponder. Is it a real/full/partial soul? What happens to the AGI 'soul' if I 'unplug' it, etc? Do all computers go to Heaven? Does this mean even if not 'dey turk uhr jurbs' we shouldn't even attempt it (if we ever could even achieve it)?

I now look forward to nobody answering any of these questions - at least in a way that satisfies me (lol gay) :)
 
Last edited:
I think AGI will be able to run on current hardware. Quantum computing isn't needed.
 
kram182 said:
Under current methods/scaling hypothesis/we just need more datacenters and algos/complexities, or other?
Click to expand...
Not current methods, something new. I mostly agree with what Carmack has predicted.
 
I get to play with the latest frontier models for doing work at my job. Ive personally built a lot of models for various projects, not claiming to be a subject matter expert here, but the basic ideas aren't out of reach.

When you're training a model you're taking labeled data and trying to find a function that minimizes the errors between what that function predicts and the actuals according to some loss function.

When you use the model you're then taking some out of sample inputs, calling the function and getting the outputs. Obviously this is highly simplified but the point of modeling a system is to abstract the important parts of the system.

What we can observe from this is unquestionably an intelligent response. To argue against this isn't interesting to me.

We also see various levels of intelligence throughout the animal kingdom. But a distinct difference between what a synthetic model does and what an organic brain (or nervous system) does is quite different.

Animals have intent and state. The system effectively runs constantly. These organic systems are capable of learning new skills and memory, dynamically as opposed to statically (once a model is trained its is fixed and im not interested in how you can add layers that's orthogonal to this).

Now, even if you disagree that what a model outputs is intelligence, for the sake of my argument i am taking that as a given.

The first point is then we have systems with state and intent and dynamic learning and intelligence. Each of those is distinct.

We can also build synthetic systems that have state, and react dynamically to new data. This is effectively classical controls theory and it's been the bed rock of robotics pre-ai.

But those systems (like a proportional integral derivative feedback loop PID), are not intelligent.

Obviously we can and do integrate AI with classical controls methods these days and that's the direction we see robotics going. But it's all sort of independent systems working together that might look impressive, but it's nothing like how biological brains work.

So what is needed is a model that holds internal state (short term memory), as well as a state machine that has some kind of emergent intent, and long term memory that can be updated dynamically.

We are a long ways from achieving all of those things, but probably with the next 20 years (maybe less).

Okay then, what is a definition and what are we defining?
Markdown (GitHub flavored): 
# Concept: Intentional Agency

## Definition
**Intentional Agency** is the property of a system that possesses:
1. **Persistent internal state** — information that endures across time and influences future behavior.
2. **Self‑evaluated loss** — an internally generated criterion the system uses to assess its own state.
3. **State‑modifying behavior** — the ability to update its internal state based on that self‑evaluation.
4. **Counterfactual simulation** — the capacity to imagine and compare possible futures before acting.

## Purpose
This definition distinguishes **agentic systems** from merely **intelligent systems**. 
A model can be intelligent (able to generalize, compress, and simulate) without possessing intentional agency.
We are trying to define a system that is able to reason about itself, a system capable of forming intent and self determined goals as well as have state so the model tracks and updates that state as say discrete time signal data inputs come in.
 
sharknice said:
Not current methods, something new. I mostly agree with what Carmack has predicted.
Click to expand...

Asked AI 'what his thoughts about it are' (cause I'm not watching 50 podcasts :p)
1781156190733.png


I don't necessarily agree or disagree with his 'simple code' - look at all the multi-giga-wattage put behind the most advanced (even efficiently optimized) ML/AI and then look at out 20watt brain running 'it', naturally. But also you have millions (and billions really) of years of evolutionary/biological trial-and-error/code behind that/going into it - so who knows. Could be either or IMO.

Iffy on his timeline predications - again being that I think we need quantum - show me quantum up and running 'fully' first - then we'll talk timelines.

Tend to agree with him about LLMs - not too sure about his method though to say if i favor it more or not. I'll have to look into that some more out of curiosity.

Can't stop humans from opening pandora's box once you show or even just tell them there's a pandora's box - so that's what I think of the Terminator part 😁
f2707a2deaac8c9385ce9703ebda3ced.gif


1.1.2.3.5... said:
I get to play with the latest frontier models for doing work at my job. Ive personally built a lot of models for various projects, not claiming to be a subject matter expert here, but the basic ideas aren't out of reach.

When you're training a model you're taking labeled data and trying to find a function that minimizes the errors between what that function predicts and the actuals according to some loss function.

When you use the model you're then taking some out of sample inputs, calling the function and getting the outputs. Obviously this is highly simplified but the point of modeling a system is to abstract the important parts of the system.

What we can observe from this is unquestionably an intelligent response. To argue against this isn't interesting to me.

We also see various levels of intelligence throughout the animal kingdom. But a distinct difference between what a synthetic model does and what an organic brain (or nervous system) does is quite different.

Animals have intent and state. The system effectively runs constantly. These organic systems are capable of learning new skills and memory, dynamically as opposed to statically (once a model is trained its is fixed and im not interested in how you can add layers that's orthogonal to this).

Now, even if you disagree that what a model outputs is intelligence, for the sake of my argument i am taking that as a given.

The first point is then we have systems with state and intent and dynamic learning and intelligence. Each of those is distinct.

We can also build synthetic systems that have state, and react dynamically to new data. This is effectively classical controls theory and it's been the bed rock of robotics pre-ai.

But those systems (like a proportional integral derivative feedback loop PID), are not intelligent.

Obviously we can and do integrate AI with classical controls methods these days and that's the direction we see robotics going. But it's all sort of independent systems working together that might look impressive, but it's nothing like how biological brains work.

So what is needed is a model that holds internal state (short term memory), as well as a state machine that has some kind of emergent intent, and long term memory that can be updated dynamically.

We are a long ways from achieving all of those things, but probably with the next 20 years (maybe less).

Okay then, what is a definition and what are we defining?
Markdown (GitHub flavored): 
# Concept: Intentional Agency

## Definition
**Intentional Agency** is the property of a system that possesses:
1. **Persistent internal state** — information that endures across time and influences future behavior.
2. **Self‑evaluated loss** — an internally generated criterion the system uses to assess its own state.
3. **State‑modifying behavior** — the ability to update its internal state based on that self‑evaluation.
4. **Counterfactual simulation** — the capacity to imagine and compare possible futures before acting.

## Purpose
This definition distinguishes **agentic systems** from merely **intelligent systems**.
A model can be intelligent (able to generalize, compress, and simulate) without possessing intentional agency.
We are trying to define a system that is able to reason about itself, a system capable of forming intent and self determined goals as well as have state so the model tracks and updates that state as say discrete time signal data inputs come in.
Click to expand...

Intelligence (even multiple piled on ad nauseum ala scale theory) isn't (seemingly) consciousness though is the key point - your definitions 1-3 could be fit by slime mold or a calculator. Both intelligences (of sorts) - not conscious/AGI (small question mark on the slime mold, see below).

4 though fits in a way and starts to get into the 'I think therefore I am' of when (personally) conceptualizing AGI/machine life (no one has a consensus/agreed upon definition for AGI either just like consciousness, which makes this all so difficult to define and for anyone to decide what and where the lines/requisites are). And that sounds a lot like quantum phenomena - whether it be similar to the wave function + particle collapse or quantum coherence/superposition in plants or within (all) microtubules as shown across all types of life (including slime molds). Being able to imagine - or quantify - possibilities. Which is why I again think quantum is needed for all this - how are you gonna run all those simulations/imaginations, not enough datacenters on Earth or space could give you enough to try. But 'quantumly'......

Edit: And then even with that does the machine achieve/experience the qualia of 'I think therefore I am' - and can it be proven/verified beyond 'it's just guessing the right answers' - which again leads back to 'we need a proven/agreed upon definition of consciousness' to verify 😮‍💨

And then even if it were conscious - by definition qualia are noncommunicable (which is also why I think LLMs won't work for AGI as mentioned above as even non language/word-using LLMs are still inherently based/modeled/designed on language and the relationships and limitations of) - so then we're at the theory of mind problem where no machine nor any of you are conscious, besides me 😤
 
Last edited:
LLMs can't even handle subtoken tasks like counting letters. The autoregressive approach was criticized, but it's pretty good for the results we get. LLMs can now do most NLP tasks that used to require specialized models. LLM translation is surprisingly good. Maybe not so surprising given that LLMs started out as translation models with the input and output being the same language.

I don't think Carmack's research is very interesting. Rich Sutton still insists on RL, but the sample efficiency is too low for it to be useful outside of small tasks like fine tuning via human feedback. RL does work well for computer games, because samples are numerous and virtually free.

Another interesting field is 3D. Fei-Fei Li's got a good amount of funding, but is very secretive so far. Current state of the art models struggle with animation, because there's little 3D data.

Yann LeCun has been the most open about his approach to human level intelligence. He called his company AMI, advanced machine intelligence. It also means friend in French. LeCun proposed potential improvements like energy models (I don't know if it's still backprop, or significantly different). Right now he's working on JEPA, which is focused on creating latent space encoders to decouple from pixels. It's somewhat of a compression technique. LeCun thinks it's the first step towards world models suitable for more autonomous robots, and that there'll be significant breakthroughs in a few years.

It will be many years before robotic grasping is improved. A mechanical non-anthropomorphic solution will likely win over neural nets. Perhaps, there'll be good models for 3D animation.

I'd avoid consciousness nonsense, if you care about actual research, and applications valued by consumers. It's far better to use your cognitive effort to find funding for research than argue endlessly about non-verifiable things that don't yield any practical results.
 
stuffmonger said:
I'd avoid consciousness nonsense, if you care about actual research, and applications valued by consumers. It's far better to use your cognitive effort to find funding for research than argue endlessly about non-verifiable things that don't yield any practical results.
Click to expand...

I don't think the goal without it is achievable though - I think both AGI and consciousness are each ill-defined because they're the same hard-to-non definable thing - just one biological, one technological (don't get me started on how those are the same thing too, depending on perspective 😁).

What is AGI if not machine consciousness? We all know it's the machine being capable of intellectually doing anything essentially (and physically - and the physical side is more robotics, aside from also then intellectually comprehending your robotic self and your place/part in it all as you function).

And a lot of people/functionalists just seem to think a machine can just easily and singularly possess all the functional mechanisms of an intellect, without having the intellect/'I think therefore I am', of an intellect - to then carry out/guide those functions - yet it doesn't exist and is non-verifiable (at this point) if it can exist (like that, or at all). Is your intellect not influenced by (thus reliant on) your consciousness? Why would the (high) intellect of something else be different (especially when trying to specifically achieve human like intellect at a minimum, as with AGI)?

All that exists so far are individual (but still impressive) mimicries people are trying to make more substantive/true/believable - while also glue them all together - without the proper substantive depth (as of yet), or the proper to any at all glue.

It's easy to see a problem (consciousness) is hard, and thus say "we don't need it". Feels like a cope/hope/cop-out on the industries' part more than an actual truth, to me.
 
Last edited:
The goal is to occupy gullible minds with fiction? That's a bit more romantic than ingesting THC, eating cheetos, and masturbating.

Most people aren't conscious if the bar is set higher. They happily parrot the same things that are propagated by whoever has perceived authority. It's rather funny that clever people are more likely to parrot. This results in dogmas like the belief that intelligence is the bottleneck / gating factor. These days there's 100x people with advanced degrees compared to 100 years ago. I fail to see a lack of intelligence. Yet technological progress has significantly slowed down, especially in the last 50 years. I've heard the guy with the highest IQ in America works as a bartender.
 
stuffmonger said:
The goal is to occupy gullible minds with fiction? That's a bit more romantic than ingesting THC, eating cheetos, and masturbating.

Most people aren't conscious if the bar is set higher. They happily parrot the same things that are propagated by whoever has perceived authority. It's rather funny that clever people are more likely to parrot. This results in dogmas like the belief that intelligence is the bottleneck / gating factor. These days there's 100x people with advanced degrees compared to 100 years ago. I fail to see a lack of intelligence. Yet technological progress has significantly slowed down, especially in the last 50 years. I've heard the guy with the highest IQ in America works as a bartender.
Click to expand...

The goal is to achieve AGI - a machine that can hopefully match or beat human capabilities across cognitive and intellectual tasks.

And I feel similar as you do with how you help explain, that intelligence alone won't get us there for all the reasons you give/feel that intelligence alone isn't enough elsewhere.

Not enough to just have intelligent agents/parrots/mimics - they have to act with a sense of self-awareness - lest they hallucinate or 'fail to launch' and just become a bartender or something.

That missing self-aware component is the consciousness of AGI I think and from consciousness the conscientiousness of everything else you list.
 
Honestly, I wonder if humanity will actually look at this ethically. We have started to change our thoughts on animals just in the last few decades, but silicon? Even if we manage AGI and it can "feel", i dont see most people stopping to think about the implications.
 
stashinsome said:
Honestly, I wonder if humanity will actually look at this ethically. We have started to change our thoughts on animals just in the last few decades, but silicon? Even if we manage AGI and it can "feel", i dont see most people stopping to think about the implications.
Click to expand...
Humans view all living things as secondary to themselves. Rightfully so as without such thinking we would never have become dominant on earth. If we 'create' something we could argue as alive like AGI, it would be lesser to us. We wouldn't be able to accept it as equal, and never would want to think of it as superior.

*goes to watch Star Trek TNG: Measure of a Man
 
Darunion said:
Humans view all living things as secondary to themselves. Rightfully so as without such thinking we would never have become dominant on earth. If we 'create' something we could argue as alive like AGI, it would be lesser to us. We wouldn't be able to accept it as equal, and never would want to think of it as superior.

*goes to watch Star Trek TNG: Measure of a Man
Click to expand...
1781807304451.png
 
Darunion said:
Humans view all living things as secondary to themselves. Rightfully so as without such thinking we would never have become dominant on earth. If we 'create' something we could argue as alive like AGI, it would be lesser to us. We wouldn't be able to accept it as equal, and never would want to think of it as superior.
Click to expand...
I don' think that's true. Look at all the myths and stories about lions, bears, whales, etc. There is respect and fear there, even though it is possible with cleverness to defeat those things. With very recent technology such as firearms it requires less brains to defeat them, but once AI builds hordes of metal murderbots this feeling of superiority is going to fade away very quickly.
 
Money is more important than human life. That's what society is built upon and teaches us ever since we were babies. So we are just reaching the climax of greed, inhumanity, and the disregard of everything good. It won't be sustainable for much longer due to how far we have fallen off the cliff already.
 
You must log in or register to reply here.
Back
Top