The Chinese Room, Experience, Knowledge and Communication (#1437)

The Chinese Room, Experience, Knowledge and Communication (#1437)

Re: Why the "Virtual Mind" is Not a Refutation of the Chinese Room

The thought experiment is a very relevant one that touches on some profound issues. But it describes a high level complex situation that involves several principles, which are conceptually entangled and could lead to some confusion. I will explicitly draw out some of them for clarity.

The main points discussed below are:
What is experience and how does a system experience itself?
What is 'participation' in a context?
What is 'knowing' and how does a system 'know' itself?
What is 'communication'?
The Chinese Room

What is experience?
In the low level context of the computational analogy experience relies on two things. Firstly there is direct experience and then there is experience of that experience. Direct experience originates from the phenomenon of state transition. The existential state of a system is defined by state variables that can exhibit particular values (its actually a distribution of probabilities over all state values). These define the actual state of the system. When an input signal is received and interpreted, this then leads to a corresponding state transition or change of internal state. In this way the state of the system is influenced by the incoming signal and thus the system 'experiences' the signal. Direct experience does not imply consciousness of the experience. An example of direct experience is a charged particle experiencing a magnetic field, it's state changes but it is not 'aware' of the experience itself, it is only aware of the magnetic field. A system can only be aware of the incoming signals through its interfaces, otherwise there is no information to produce an experience.

How does a system experience itself?
To be conscious of the experience the system must have internal feedback loops so that it can experience its own state. In such a case when the system's state changes directly, this is then perceived as an internally generated signal that then results in a change in state of the internal monitoring process, hence there is a change of state that registers the direct change of state. With further feedback loops things get more complex and the system can experience the experience of experiencing and so on. What we usually refer to as consciousness is a very particular form of feedback loops that give awareness of our internal state at many levels. Hence when a signal impinges upon us and we directly experience it, we then also experience the experience of experiencing and we are aware that we experienced something.

Control/Monitoring System
The direct process is the primary process that implements the base level functionality of the system and there may also be other processes that monitor this direct process. For example, consider a control system program, it can perceive the state of some environment (via input signals) and interpret this input into an image of the state of the environment. This can then be compared with an image of the desired state of the environment and the difference results in a collection of corrective output signals. This basic control system is aware and responsive to the environment that is available to it through its input and output interfaces, but it is not aware of anything else such as its own operational state or the meaning of its actions or how to decide the properties of the desired state. It has no feedback loops and is simply a direct pipeline for information transformation, hence it is described as the direct process.

If there was another meta-process operating that was a control system that monitored the direct control system, then whilst the direct process is aware of the environment via the interfaces, the meta- system is aware of the direct process via its own interfaces. A system can only be 'aware' via its interfaces. If there is no interface then no information can flow and no computation or "primal awareness" is possible.

These meta-systems can be thought of as forming a stack of successively higher-level feedback loops but it is more realistic to think of a complex network rather than a stack. This gives a system that is similar to a neural network. So consider the situation of a complex network of meta-systems, each of which monitors numerous of the other systems. The total system would not only be aware in a direct manner, it would also be aware of its state of functioning and also aware of its state of awareness of itself.

What is 'knowing' and how does a system 'know' itself?
All this pure awareness does not constitute 'knowing'. That involves a further process of association. Once a system is able to experience its environment and also experience its experience of experiencing, it then becomes aware of itself as an experiential entity in a world, it forms a self image or a concept of 'I'.

From this egoic perspective it can experience itself in relation to its internal state (feelings) and its external state (environment). From this it develops associations between these states or modes of experience. This is the begining of 'meaning'. To recap, the underlying reality has created an experiential context, the system has had experiences of the underlying reality, and it has formed 'ideas' about those experiences. It now forms a complex network of associations between the various experiences, much like an experiential 'idiom'. As the system experiences its experience of experiencing and it also forms associations between these experiences it then comes to know that it knows that it knows.

What is communication?
So far the system only has an internal experiential idiom (ideas). In order to communicate them it requires a further stage of association with external symbols, which are external experiential states to which meanings are associated. The meanings can be arbitrarily assigned, e.g. the word 'cat' bears no intrinsic relation to the reality that is refered to by that word.

Hence an abstract system of symbols arises, which the system may manipulate according to certain basic principles. This symbol manipulation can be done consciously but it is most often sub- conscious, for example, natural language ability in humans. When done subconsciously it is performed by the direct process (or lower level processes) and when done consciously it is performed by the higher level meta-processes. Hence when we do it sub-consciously we are not aware of the details of the process but only of the overall state of the process, i.e. the flow of meaning. However when we do it consciously we are aware of all of the details because the high level processes need to explicitly control the lower level processes.

In regards to this semiotic phenomenon of experiences, ideas and associated symbols, the symbols themselves are in some ways processed as pure symbol regardless of their meaning, but then other meta- processes observe this symbol processing and assign associated meanings to them. The low level symbol processing can be done regardless of whether the higher levels choose to assign meanings or not. For example, the story related by B.M about a "group of linguistic majors at a university [who] decided (for fun) to make up a 'language' where the words had no meaning."

In conclusion, a direct system with many meta-systems can experience its environment and itself, it can assign meanings to experiences and also associate symbols to those meanings. It can manipulate those symbols and be aware of the flow of associated meanings.

The Chinese Room
In the chinese room example, the direct process is the explicit symbol processing algorithm that blindly processes incoming symbols and produces outgoing symbols. This system can only experience things through its interfaces, hence it can only be aware of the incoming signals and it has no knowledge of itself or of any associated meanings for the symbols. Hence that direct process cannot be aware of what it is doing or the significance of its actions.

The human being in the room not only provides the animating drive that implements the symbol processing algorithm, they also provide a complex network of meta-process (sentient awareness of the state of the direct process). Hence the human is the meta-control system that provides high level awareness. Hence the total system is capable of self awareness and it may be asked questions about itself and may give meaningful answers. E.g. if asked "do you know the significance of your actions?" the system may respond that it does not know, it lacks information. It is just a pure symbol processing system, even though it is self aware and fully sentient, there is no network of communicative associations so there can be no ascribed meaning for the symbols.

However once meanings are assigned (the symbols are identified as Chinese) and the system is made aware of these associations (the person learns Chinese), then certain meta-processes would associate these meanings with the underlying symbol manipulation processes and it would come to experience a flow of meaning as well as a flow of symbols.

Hence the Chinese Room thought experiment describes a fully sentient system that has incomplete communication functionality. It does not challenge the issue of whether the system can be aware or self aware and so on, it simply draws out the details of communication. Hence the "virtual mind" proposal is not required. Regardless of there being a room or no room, the total system DOES understand what it is doing, it's just that without being informed of the associated meanings, all it is doing is just processing symbols and it is correctly aware of that fact. If it was then informed of the meanings and the context of those meanings, and it developed a network of associations, it would then begin to process meaning as well as symbols and it would then be correctly aware of the fact that it was processing meanings (actually speaking Chinese).

> Since the room was, in effect,
> running a program, Searle's conclusion was that a computer running a
> program would also have no understanding of what it was doing. His
> conclusion was there was more to consciousness than running a program.

I do not see that the thought experiment challenges the notion of AI. Although it does draw out the limitations of a direct process. Without the feedback loops there can be no experience of the direct experience, hence no high level awareness. And it also draws out the nature of systems of symbols and their role in the conveyance of meaning.