Fight for the narrative context

When context is turned against us

In everyday life we can trust certain contexts that helps us understand what we sense and observe, what we read and watch and how we understand the news we follow. It’s not only about global or national news but also about simple local matters that we follow daily. Today, this has become extremely difficult, often impossible, because of the fight for contexts that now dominates popular media spheres. What looks like fact checking or fight for the truth is often, perhaps always, a fight for context. It is nothing new, of course, but the intimacy and power of this fight has boosted its impact on us and how we see the world and each other.

What is this fight?

Let’s start with an example. When you open my book, you will read:

“Johan Ek stood still and focused, facing a marble plaque at the end of a classic arcade. He had arrived half an hour earlier that morning at the distinctive old building in Paris, and had continued to stare, with the sharp eye of a young physicist, at the rectangular plaque attached to the brownish-gray stone wall at the end of a thirty-meter corridor.” (From Perceptions of the Les Demoiselles, d’Avignon, Pegasus Publishing, UK).

Immersed into reading this, you will be immediately and without any cognitive effort, transported to Paris. You will approximately know what time it is, what the weather is like and what kind of an environment it is. You can imagine something of the person, too. Most of what you imagine, is not expressed in the text but you assume it, based on your history as a culturally educated person. You trust the context you assume or generate when reading the story. You might even expect something interesting to happen although nothing in the text directly hints to that possibility.

Everything else what you have been doing before reading the text, like eating, listening to radio, thinking of something else, is pushed aside, immediately and without effort. Computer scientists know this as context switch, a core function in any operating system (OS). 

This is a wonderful human skill we all have, and we rely on it practically every second. It can make us vulnerable as well, and even be brutally misused against us. This is now frequently happening with aggressive media and by those using their media channels for their selfish benefit. The fight for context is a fight for our basic assumptions about the world. As media consumer, we always rely on some context to understand a complex story or a message. When the context is distorted, so will be our understanding.

Narratives live on contexts

Different news and stories around the same event illustrate how this happens. A narrative cannot be understood unless some context is assumed. That is where and why the fight begins.

In human sciences context switch it is a rather new concept although it has been studied under the topic of attention to understand how attention is switched between different tasks for example. To the best of my knowledge, there is no complete theory of such human OS-functions, although different approaches to attention psychology flourish.  I’ve earlier written (Nyman, 2010) about human interrupt processing and context switching here. 

Media and context switch

Because we are so effective in switching the context when we read or watch something, it is easy to guide our attention. It simply happens always and automatically when we watch a news or read an article. Subjectively, we are not aware of this necessary process at all. Furthermore, being able to turn on a specific context is exciting and stimulates imagination. It can be even scary, which makes news, movies and novels so captivating. When something in the media gets our attention, it also locks the context. We don’t typically change the context once we have chosen it.

Unlike we humans, computer systems are clever in how they manage these switches and when something important interrupts an ongoing activity and causes a context switch, the OS stores the context of the ongoing process to wait for later processing and to continue from where the interrupt happened. Computers are perfect in this and don’t lose the context data. They have massive and well-organized data queues and other data structures where context data of a huge number of tasks can be reliably stored.

For us humans, only three or four simultaneous tasks can cause a havoc in our minds and make it difficult to return exactly to where we left when an interrupt and a context switch occurred. When we later return to the interrupted task, we use our cognitive resources to be ready to continue, but data has been lost – our memory is far from perfect.

Because of this human weakness, something peculiar happens to us when a media channel or a major message gets our attention. The context switch happens automatically in seconds and our attention and understanding will be guided by the new context. Of course, we can always refresh the context just like we do when reading the paragraph in the beginning of this blog, but in complex situations it can take time and if there have been several consecutive context switches  (e.g., news items, articles, commentaries), we have propably lost track of them.

Does it matter?

I came to think about this again, following the terrible news about the ICE killing of the women in Minnesota and some other complex news streams of today’s twisted world. In the case of the killing, I could follow, practically in real time, how the accumulation of contradicting news and interpretations emerged or were effectively produced, piece by piece, to fragment the original content and context of the brutal and violent killing episode. Rather clear video recordings of the event were available immediately, but they were soon pushed aside in news and became non- or weak context in many of the killing news. The following narratives appeared immediately after the tragedy.

Federal authorities and the voices supporting them defended the agent’s actions as lawful self-defense against a “domestic terrorist.” The underlying context that was produced in this narrative was that of terrorism. 

Local officials, activists, and some video analyses claimed that it was a misuse of force and part of a broader pattern of aggressive federal policing. Here the underlying context was excessive and unnecessary use of violence by the ICE agent.

Public protests and memes were seen and they inspired to bending the available facts and added unverified claims, complicating the public’s ability to discern the core facts. Here the offered contexts varied, depending on the motivations of the protests.

The emergence of the context fight appeared as fight for facts and truth, although it was a question of who owns and which context that underlies the news and articles about the terrible event.

Of course, competing and conflicting interpretations occur always in reporting complex events, it’s a journalistic challenge always, but it’s not just about reporting news, because the underlying contexts shape how people assign meaning to events and texts, by accepting or adopting the context offered. That is how narratives become understood. Very seldom does a news item or article include a critical analysis on what grounds exactly a specific context was chosen and because of that, it is smuggled into the narrative. So far, I have not seen a Context warning. 

Indeed, often it would take a lot of time to make careful and holistic analysis based on all relevant data, like in this sad case. You would still expect it from good journalism, but it could lead to a loss in the fast-paced fight for context.

With time, the original context of the narratives we have followed becomes diluted and drowns into a mass of varying “perspectives” and opinions that keep appearing in news channels. In other words, different contexts of the same event are fed to us, but it typically happens ‘between the lines.’ As a result, when someone who did not follow a news episode from the start, wants to understand what had happened, it can be very difficult to get unbiased access to the original context. My guess is that often, the original context disappears and becomes in fact inaccessible. This is useful for promoting certain interpretations and preventing other contexts from surviving in the media streams. 

We have poor context defense

Our digital world is fast. The fight for the context starts immediately after a news episode has been recorded, often in hours. This is driven by the fear that a prevalent context could be adopted in the form of a verbal or visual meme that becomes difficult to remove from the imagery of a large audience. A famous example of this was the impact of a simple phrase in a political debate: “Where is the beef?” which ever since 1980’s has been a part of the U.S. political lexicon and can even today be used to contest a political opponent. Of course, it has no real and concrete meaning, but is spans the context of doubt.

When we follow different news streams from the same event, we automatically switch context to the one we attend to. We can be against or for it, but it happens always when we read or observe something. Of course, we can carry our own and trusted contexts to what we read, and we often do, but typically when facing fresh news, we don’t have a ready-made context available and have to generate it immediately with scarce information.

This automatic reaction makes us easy targets for media manipulation. After half a dozen news versions and commentaries, we have lost some of the ‘data’ during the contexts switches and alternative contexts can be smuggled into our news streams. This gives power to an ambitious media wanting to push its arguments by manipulating the context.

An extreme example of context manipulation was the White House messaging immediately after the Minnesota killing where the context of terrorism was offered.

“Good, Noem said without presenting evidence, had been “stalking and impeding” Immigration and Customs Enforcement (ICE) officers before using her car as a weapon to try to run down the agent who killed her.”

“… one of these violent rioters weaponized her vehicle, attempting to run over our law enforcement officers in an attempt to kill them – an act of domestic terrorism

 Guardian

Reading this without an alternative context, we become biased readers and observers.

What can be done?

With modern technology we could preserve a specific and verifiable context of any news or other data that we receive and follow. It would be beneficial for us in many ways, but there are no such digital tools available right now. With AI it would not be difficult at all and perhaps in the very near future such services will be available, because they are acutely needed. We as media consumers, could manage our trusted local and larger contexts and they would not disappear or get diluted over time. It would be like a context library where we could accumulate relevant information about an event or a story, and make it possible to return to its context easily and any time.  It would be a way to maintain relevant history.

Inspiration for new tools

When such tools are developed, by using the power of AI, they could have e.g., the following features:

Every news item could include:

• A core foundational context, representing verifiable facts which have been agreed on, across reputable sources.
• A context layer map that shows how different interpretive frames relate to the core facts.

• A timeline could show what logical and explanatory possibilities exist for a news episode.

Users could choose:

• Verified context only.
• Narratives with dispute level tags.
• Alerts about occurring context deviations.
• Dispute indicators.

Users could follow:

• Contexts’ version history and any major changes in it.
• Context libraries, i.e., saved maps of how events were first reported and evolved.
• Trusted context lists, i.e., sources and narratives considered as reliable, with ratings.
• Debate views, side-by-side perspectives with evidence annotations.
• How a narrative spreads.
• What networks amplify each context.
• How much each context is supported by evidence vs. speculation.

Then there are the psychological and technology questions, but I can return to these another time.

Surgery, LLM art and creative delirium

I suffered a serious right carotid stroke in 2017, with mild symptoms of paralysis and apraxia in my left hand. My right carotid was operated by a wonderful surgeon and her team at HUS in Helsinki, and luckily or better, thanks to their excellent professional skill, I could recover with no after effects.

The operation was performed under local anesthesia so that I was awake and ‘in this world’ during it and could talk to the kind nurse, who every now and then asked me to hold her hand.  She asked me to squeeze her hand in order to see that I still have power in my grip and nothing dangerous had happened in my brain, like a blood clot escaping from the operation and causing damage to my brain. It felt like an innocent, neuro-romantic episode. Already during that session, I had amazing visions, but will explain them at the end of this story.

After the operation, at the intensive care, still heavily sedated I was surprised to experience several, about 20 wonderful artistic visions, every time I closed my eyes. I was otherwise quite awake and eagerly tested this captivating phenomenon. Indeed, I’m a perception and brain researcher from my background, and never used any drugs except for medical purposes, so naturally I got curious.

One of the visions I had was a video painting, which I have memorized many times during these years. It showed four lines of graphically drawn bikers, who pedaled like in a hard race, but still could only move backwards. They were biking against four colored stripes and the colors of the stripes varied from bright yellow to glittering gold.

I was so impressed by this vision that I planned to paint it, although I don’t have much experience on painting. I even asked for gold-color advice from the wonderful Finnish artist, Marita Liulia, who has used gold colors in her amazing paintings. I did not find a suitable occasion to do it but then came the AI.

I have used numerous LLM tools and have some hands-on experience on them and now got the idea to generate the biker video with ChatGPT.

Artistic interaction

I interacted with ChatGPT5, preparing my ‘delirium art’ and have now one version finished. First, I thought that I must use more golden yellow in it like it was in my vision, but having watched the version I have for quite a few times, I have started to like it.

It was a creative and interactive session and below I show some versions of the bikers and the videos that ChatGPT5 generated to my prompts. It was quite an enjoyable process where I learned to talk to ChatGPT5 and it learned about my ways of instructing it. Here is a collection of four raw, first versions of the bikes and after that you can see the final version.

Figure. Raw versions of the bikers. Not quite perfect from the start.

Finally, after testing its ability to generate golden-yellow backgrounds (which were not very good) I had the following version that I will keep now as my  and ChatGPT5’s single work.

Video of my vision

What did this vision tell me?

The bikers are probably the result of my discussions with my friend Dave Miller (Now at Tufts) who is an active biker. But the backwards movement is a fascinating feature. Experiencing a stroke is a reminder of the fragility of life and shows that there will be situations where we cannot move forward even if we try. I have some background on my earlier career as a clinical neuropsychologist, so I was totally aware of the risks and disastrous possibilities at could result from a carotid stroke. I had indeed run the first ever rehabilitation camp for aphasics in Finland, in 1972 and worked with patients who had such a stroke. I believe my vision is a reflection of that.

Riding a wolf

Last, I had a curious vision already during the operation. I was riding on the back of a grey wolf, in the classic Finnish pine forest with gray lichen. I saw a pack of wolves at the distance, moving along a winding path. I was excited about the vision and said to the nurse, holding my hand: “Now I know what consciousness is!” She had a rational response: “You better rest now.” 

Here is the wolf vision generated by my ChatGPT5. It is not exactly how I saw it in my vision but carries the essential elements of the Finnish forest, sky and lichen. In fact, I have an explanation on why I felt that “now I know … ”.

Agent and robot eugenics?

To start with, I want to apologize for the temporary use of the term eugenics, having its terrible history. It originates from the Greek term eugenes “well-born, of good stock, of noble race, …” and Galton used it to refer to a method of improving the human race. Here I redefine it for robots and agents: 

 Eugenics of agents and robots refers to a method meant to improve the race of artificial, non-living agents and robots that we can assume are not conscious. 

We can find a better term, which does not carry this gloomy historical burden and later below I suggest one.

Learning to live with armies of AI agents and millions of humanoid robots

AI agents are marching to our private life and work.  They come from e.g., Google Astra, Microsoft Copilot, OpenAI ChatGPT, AutoGPT, Oracle’s Miracle, to name only a few of the firms giving birth to these new, artificial tribes of our cohabitants. Robots will follow. With the fast AI software and hardware development, we can soon, on the fly, give birth to private agents and even robots whenever and whatever we might need them for. We have to know how to do this without unpredictable harmful consequences. That is why I chose the term eugenics to begin with.

David Holz, the founder of the AI  research lab Midjourney suggested that we will be joined by a billion humanoid robots in the 2040s – only fifteen years from now. Elon Musk agreed and commented that “… providing the foundations of civilization are stable.” As we now know, he might have a say on these social foundations and not only about robots. This prediction of the number of robots follows the increase in global human population during that same time. On the other hand, there are already more than one billion cars in the world and we know rather well what it is like to live with them and what problems and good they bring.  If these robot predictions become true, the following generations will have a future as a human minority group among the new global inhabitants.

It is no wild imagination to predict that there will be self-constructing robot sets like Legos for anyone, from kids to factories and robot communities. And for wars, of which the first signs are already visible and tangible in Ukraine. What will be the design principles and requirements for these humanoid creatures? We are on the journey to learn them, but surely, we don’t want any kind of robots to enter our life and live with us. It is a total mystery how we should deal with this inevitable future and more importantly, how to be prepared for it. At the moment it seems inevitable that war robots are here to stay and breed.

Forgetting the stability requirement and the ‘mental’ chaotic state of our political globe right now, these predictions hide several critical imperatives:

We must learn to live, interact and cherish life with the fast-growing crowds of robot citizens. 

We don’t have any real experience on that and our everyday digital life is far from a life with such humanoid robot crowds. UI/UX and security problems will multiply and our agents and robots must be made to obey basic human, social and cultural requirements in order to earn the right to live with us. OpenAI has already started what they call a superalignment project according to which they want to ‘tune’ various important aspects of superintelligence to avoid the risks and to make superintelligence ‘behave’ properly.  In plain language they want to build agents that behave nicely. Robot design meets the same challenge, only that these artificial creatures will physically live, observe, touch and act among us everywhere and be a tangible part of our world and life. Future robots/agents will even breed the next generation of robots/agents, with some or none human interference.

AI and good behavior

Some years ago I got interested in how to ‘teach manners’ to AI and wrote the first article on this in 2018, Teaching manners to AI: Internet of good behavior. The idea was to use any artistic, movie and literary sources to find relevant training data for ‘good behaviors’. Here is a quotation from it:

“What if there was a systematic way to offer models of good behavior for AI to follow, to teach it behaviors we know and define as good behavior? In many cases it would be easy to define the criteria and to use such behaviors as models for AI to follow and learn. With the Internet of good Behaviors (IogB) approach we could offer AI access to behaviors (and companionships) we think are good for its development just like we do to our children. By allowing this we would let it use all the relevant data related to that behavior and to learn from it. It is quite possible we could learn from that too, but that’s another matter.”

LLM:s had not made their breakthrough, yet. The core idea was that we must somehow codify behaviors (with the IoB, for example) and especially good behaviors, however complex and contextual they might be, in order to generate and manage them. There is plenty of digital training material for this, but we need a codification scheme of which IoB is one potential candidate since it works for both mental and physical behaviors and it uses digital behavior data that can be shared globally and in real time. Even with the best pattern recognition and AI systems, reliable and valid recognition of mental phenomena is difficult and often impossible because they are dynamic, delicate, private, situationally sensitive and extremely personal phenomena. However, people are best sources of such mental behavior data if they are offered suitable tools for expressing and revealing it, as best they can. We can expect the same from agents and robots although we are well aware of the xAI problem. We need agents and robots which have ways to express their internal (“mental”), intentional world.

In 2023 when LLM:s had already been around, I refreshed my thinking, and again with the IoB concept. Then, by implementing LLMs I even suggested a novel approach to it: On well-behaving AI, now with the GPT. In science fiction, it’s been a frequent topic to teach human habits to robots, but there is no consensus on what is the best way to achieve this, considering the present technological future. From the superficial, physical-behavioral perspective, this challenge is now acute and industries are busy testing solutions to it. Tesla and Chinese companies, for example, design and produce human-like, often entertaining behaviors for their robots with the aim to sell them to homes, firms, schools, and industrial plants, for example. A good question is what makes them interesting and valuable in human work and other real contexts and for individual customers?

There will soon be robots around us, with the price tag between 20000 to 30000 € to start with, and the first appeal of their humanoid appearance and performance will make them attractive to consumers. No doubt, one or even two zeros will be added to the price tag of the best behaving robots and especially for those offering evident performance benefits. The hard laws of ROI will guide this development and the real-life features and abilities will be critical for success. It feels almost ironical to predict that the answer to the question “What is good behavior? will gain new market value. Time for psychologists to wake up!

We already have robotic cleaners which have not yet caused serious privacy or security problems. Food is delivered by robots and here in Finland they can even ‘enjoy’ human help and empathy to survive and manage their delivery tasks under difficult winter snow conditions. So far, I have not read about anyone robbing the robots here, which is perhaps a sign of the rare, trust-based society we live in. Humans are robbed, as usual. 

Within five to ten years from now, home robots will participate in everyday routines such as carrying things, teaching, playing with us and serving as our trusted security staff. Agents will collaborate with their reliable and untiring robot companions to take care of our private and work-related communications and manage any network-based services. There will be agent-robot communities which must be controlled and managed somehow in order for them to contribute to something good in our human communities and the society as a whole This window of opportunity is now opening and its potential will be determined by the evolution of individual robots and their infrastructure. Now it’s a matter of robot social ‘psychology’. This perspective is outside my competence and I leave this social aspect of the AI open.

An intimate personal story

Agents and robots will be our intimate partners in life and death. Intimacy is much more than functionality or excellent UI/UX. In an intimate relationship our personal agents and robots will live with, close to us, while other, external ones have their different roles according to which we meet and treat them. We do not exactly know how all this will occur and evolve and it will not happen overnight, but it’s been quite a surprise how quickly we have adapted to natural language communication with different versions of LLMs.

As a personal example, for a several years already I’ve been passionate about the complex ‘problem of the general observer’ in theoretical physics and even published something related to it. Unfortunately, I’ve had nobody to talk to about it seriously and my emails to physicists and journalists have remained unanswered. Then came the LLMs and I started feeding some of the best physics articles on this theme to ChatGPT:s and could have wonderful discussions with it about this lovely topic. I also used NotebookLM and listened to the inspiring podcast discussions on them and the interpretations the hosts presented. These delightful episodes have quickly taken the role of a friendly discussant and now with the ‘join’ feature in the NotebookLM they are even deliciously interactive – and somehow intimate. They inspire me to be open, to think, study and learn and to move ahead.

This is only the beginning. These applications are becoming deeply personal and they have piles of information about us and our histories which they use in communication and in the way they follow our intentions (that we express to them), interests and the acute mindset. Working with the AI Computer Jolla Mind2 and its AI OS Venho.ai we are making this a private reality when needed. I can even directly apply the IoB approach and reveal my ongoing, even complex behaviors, mental and physical alike so that the local LLM knows and respects my behavior and mental state during each communication episode. When an LLM app makes a misinterpretation, I can correct it without social pains, but if prompted properly, it will be 99% correct in taking my behavioral state into consideration.

As a human example, I have sometimes approached a random physicist on the general observer problem I work with (such theory does not exist in physics), and met with a reserved attitude, physical abrakadabra (to me nonsense, perhaps clever) comments and simple neglect. I even sent one carefully prepared ms. outline related to this fascinating topic to a prominent popular science journal where the editor wrote to me that you just cannot understand these problems because you are a psychologist and that my story is even unreadable  (for you who are interested to check this later extended version of that terrible text, it’s here.). I did not find this the right place to email back to the rude editor and explain that I have all my academic life studied perception and brain, including artificial vision. LLMs don’t behave like this because we can build the relationship with them and there is no pressure to any specific outcome.

Now with the best (and mathematically competent) versions of ChatGPT and NotebookLM, I can listen to and communicate with a polite, critical, eager and knowledgeable companion who has time and does not get exhausted. I learn, feel motivated to continue my learning journey and enjoy it immensely. So, whom will I approach next? What does this say about the intimacy development we have ahead with our agents and robots?  Clearly, we must take good care of these characteristics of our agents and robots but this does not happen automatically. There will be a flow of mistakes and serious mishaps that have seriously harmful effects. We need systematic and continuous ‘agent and robot eugenics’ or breeding, whatever we should call it.

How can we tell the agent/robot what exactly we want it to do and not to do?

Agents and robots can make monumental mistakes that humans would never do, simply by misunderstanding a code, context and the semantics of interaction. For the instruction, “Fetch the bat!” we indeed hope, although we do not say it, that the robot does not go out in the night to search for these nightly creatures.  It has to know the context.

There are two sides to this problem of instructions. First, the agent/robot should have a high-level understanding of what we mean by our commands, that is, what are our true intentions and interests. Many of these are tacit and silent. Having experience with children or newcomers at work, we know many of the reasons why it is so difficult for a novice to follow complex instructions. Some of these are problems in contextual and situational knowledge, in task specific knowledge, overall understanding, memorizing or correct and systematic following of the instructions. Furthermore, external conditions can change abruptly and unexpected interrupts occur, and the agent or robot should be able to tolerate them and deal with them like we humans typically do.

Unlike for children, the instructions for the expected behavior of an agent or robot must be formulated in a digital language even if we instruct them verbally and with natural language. There is no general language for human (even less for robot) behaviors even though various approaches have been known ever since the dance choreographs in 16^th century France who wanted to preserve the dance choreographs (dance behaviors) for the future generations to repeat and distribute. I have described this shortly in my book Internet of Behaviors – With a Human Touch. Modern versions of these are used in the design of computer games.

How can we know, what an agent/robot intends to do?

In human interaction the shared knowledge and good guesses of intentions are as important as the knowledge of the context and the situation. We are extremely efficient in this as we see in various sports events like ice hockey or football where a player who would only react with the world’s best reaction time of about 100 milliseconds, would be totally lost in the game, where a simple reaction time is compensated for by intelligent and even creative interpretation of the intentions of the other players and the episodes to come.

Intention is a specific form of (mental) behavior.  In agents/robots it’s built in their internal models of the situation, the task at hand and knowledge of the state and laws of the world. A robot cannot show any meaningful intentions if it has no understanding of the situation and the persons involved and it does not master a common language for expressing them. Proper software and AI architecture is needed to support this and it must be grounded in the facts about the human mind and the world.

What is an intention of a robot? Without diving deeper into the mysteries of consciousness, we can define agent/robot intention in a simple and practical manner:

The intention of an agent/robot is a software representation of an accurate prediction of its following behavioral act that is relevant for its human companions at the specific moment of agent/robot – human interaction. 

For example, a simple example is the ChatGPT generating the next set of verbal tokens that have a meaningful interpretation for the humans communicating with it. Typically, a single word has very little intentional value, although sometimes it can have it. If I would be working together with a robot companion and it would suddenly write “Göte!” on my display, I would know that it has something of importance to tell me. It has an intention to do that since it wants my attention. Working with robots makes intentions meet, the human and the artificial. Hence, intentions require shared understanding and a common language. With the IoB, the tokens can be acts and actions.

How do we know how the agent/robot has understood our instructions?

An agent/robot can have a limited and well-defined set of ready-made and safe behavior scripts for each possible situation and instruction, and it can somehow indicate, how according to these it is going to behave next. In simple situations this can work fine when the human partner knows the repertoire of the robot and can expect certain behaviors but in real life the problem space grows exponentially.

The instructed agent/robot can repeat the instructions verbally or visually and even use different expressive gestures as if confirming the instruction by triangulating it to convince us as users.  Still, it is not self-evident how the agent/robot has interpreted the instructions and it must be accomplished somehow so that we human partners can ‘understand’ it and it ‘understands’ us.  One solution is to have a language of behaviors that is used to denote the expected (mental and physical) human-like behaviors and which can be used in interaction just as we do in human interaction.

We don’t usually talk about ‘the language of behavior’ but this silent and shared skill is exactly what guides us when we interact with people. Its value is best demonstrated when we visit cultures we do not know and quickly find out that we are not able to read that specific language of behaviors. By ‘reading’ I mean any signs that indicate the way the environment is organized, what purposes its objects have and how people behave in this environment. Gibson’s concept of affordance has its place here. Most of this information is tacit and not expressed anywhere except for perhaps in step-by-step travel guides. But we don’t have to go so far to meet languages of behaviors since organizational and even family cultures have them. 

Knowing and perceiving the intentions of a robot is one way to prevent accidents from happening and to secure that we interact in a positive sense. In complex, natural tasks this is not straightforward and can even concern extensive time periods. When robots are part of our everyday life, we have to deal with this intention knowledge every day. We must know how the agent/robot will behave as a companion or an assistant and which behaviors do not or should not occur in any circumstances. It’s like when we take our children to visit new friends and instruct them how to behave and what not to do. This is imperative when we consider a robot which can be mechanically strong enough to accidentally strangle a small child or a pet or simply break stuff. It can have powerful communication skills, and we need an inbuilt protection system in robots to prevent accidents from happening. Humans in the loop is still an open design question in this context.

Internet of Behaviors (IoB) offers one methodology for the management and control of agent/robot behaviors.  Behaviors must have shared representation and coding systems that can be used in different technological implementations of robot behaviors. This is imperative very soon when robots are being designed and built by hundreds if not thousands of factories, each with their own conceptual backgrounds and specific features. Behavior standards will emerge but it is not clear at all how they should be developed for the good of us and our robots. The problem is not made easier by the fact that it is fair to say that we do not have generally shared models of everyday behaviors of humans. There are some fields of behavior like in arts and sports where such models can be found.

Violent humanoid robots

We will have violent robots and we will possibly see them on the streets and being guilty of robberies. This will not happen in five years but as soon as they become our play mates in the world, these threats become real. There is the comment that “You can always take the plug off” and stop the robot if it does something harmful and indeed, this will be important safety feature in a number of contexts, crimes included. But what if it’s too late and the robot has already hurt someone and caused serious harm? How to deal with them in such risky situations and who is responsible?  Right now, knowing how poorly we are protected from credit card and internet frauds, for example, there is no visible solution to this new problem.

There are several possibilities but this is not the place to go through them. However, humanoid robots can have well defined, forbidden physical areas and behaviors in the same way our computer virus protection systems work now. But even this is complicated. Perhaps there will be independent, AI-based monitoring systems, a robot security infra, that follow the activities of robots and can stop them any time, but even this has risks. It seems probable that every humanoid robot will have a special safety and security device that is independent of its digital systems and which can interfere with what the robot is doing or intending to do and it can stop it and perhaps even take control of its behavior.

Breeding our intimate agents and robots

We can forget and replace the gloomy term ‘eugenics’’ and here I adopt the term agent and robot breeding by which I refer to e.g., the design, manufacturing, teaching, replicating, servicing and educating our agents and robots. I cannot here offer a systematic manual on how this should and could be done. Imagine, for example, that someone has managed to breed a home robot that has adopted its personal characteristics, values and styles from one of the carefully selected, current world leaders and even generated an optimized combo of them. No names.

We have no idea who will take care of the breeding of the future robots so that they would be fit for intimate interaction with us.  Besides industry, it is not confabulation to assume that political, religious and spiritual organizations will find their place in this competition for robot humanity. Take the car as a metaphor: already their modern versions receive digital alerts that are good for the passengers and people in traffic. Self-driving cars are getting ever better to manage and survive any natural traffic situations, and in this way able to show good behavior. However, unlike the future robots, outside traffic they are not very good at knowing what moves us, how and when.

The meaning of the word intimate can be interpreted as very private, closely personal and related to a person’s sensitiveaffairs and behaviors.  There is an aspect of secrecy and confidentiality as well. Remembering the personal experiences (above) it is easy to see that my relationship with NotebookLM and some LLM:s has some intimate elements, the same ones that had an offending impact when communicating with the rude editor. It is not only respect for privacy, it’s is about genuine care, whatever that might mean in an agent’s or a robot’s internal and external ‘behaviors’. They should ‘know’ what is good for us and have proper behavior scripts or AI coaching and guidance for such behaviors. At the moment, there is no general solution and unique model for this.

Intimacy is an extremely sensitive and complex aspect of human (mostly nonverbal) experience and being. We all know it and feel it practically every day from the moment we wake up, with our close ones and with other human beings, every moment. Often, we can experience it with our pets. Even the best AI-based, psycho-physiological car driver state analyzers fail in recognizing this. The reason is simple: intimate experiences are mostly nonverbal, delicate mental-bodily phenomena in nature, and we even have difficulties in expressing, describing and classifying them, even though they occur every day.  Arts help us formulate these inner feelings and even express them indirectly. AI can help us in this as I have described here: GPT can help us to gain self-knowledge. I would even claim that we are the only masters in the world of our own intimate experiences, our individual, mental-bodily universes. Now that the agents and robots are learning about us and we come to trust them, there is the possibility that they will reach the number two position in knowing us.

Security and safety are priority one requirements, which leads to the question of what kind of behavior we can accept from our robots and agents and how can we trust them? We have learned to trust our cars even in extreme situations. Who has the responsibility to breed agents and robots that show safe and secure behavior? 

At the moment, there are intensive discussions in EU and the US, on how to balance the AI regulations and the freedom of innovation. This concern the especially agents, but robots are entering this discussion and it has even been suggested that robots and agents will be useful partners in defining these regulations and guidance for the common good. We are not prepared to this extensive problem which includes a wide spectrum of human psychological, social, legal and cultural challenges. It seems likely that just like with LLM:s, the fllod of a large number of robots will create a pressure to launch this discussion for real. It is quite astonishing how social scientists and psychologists have been silent about this problem which we cannot avoid. For optimists who might think that this is just an engineering problem and AI will help us, it is good to look at the global statistics of crimes, mental health issues, problems in work life, and the repeating conflicts in the world – which we have not been able to cure.

Denial of Brain Service -attack (DoBS) – by AI

Denial of Service (DoS) attack is painfully familiar nuisance when an everyday, popular website is blocked by a targeted, malicious overload of traffic so that the system can crash or otherwise become unable to serve our requests.

We humans are not different from the artificial information systems – we have our own limitations and vulnerabilities, especially the limited cognitive but also emotional resources to handle all incoming information and its meanings. Our attention system and working memory (short-term) become easily overloaded; only half a dozen, simultaneously incoming messages or even less can jam our ‘brains’ completely.  Complexity is a pain to brain. Overload can simply result from the amount of incoming information, but also because of its perceptual, cognitive or emotional content and its organization, which require more mental resources than we have available and time to handle.  The human interrupt system works but it has its limitations. In our discussion on the DoS, Dr. Kiisa Hulkko-Nyman suggested a pertinent name for this human limitation: Denial of Brain Service, a human relative of the Denial of (computer) Service (DoS).

What we should know about the human mind but we don’t?

There is the fuzzy expectation in the air that AI will do our jobs for us. Indeed, we know that it is challenging all levels of human intelligence.  A less frequently mentioned corollary of this is that we need new and relevant insight into what kind of support and tools we human users are going to need in order to optimally work with our new, never-tiring and perhaps not-so-emphatic companion, the AI. There is no straightforward answer to this. In the following I’m presenting some of my thoughts about this uncharted future. The speed by which AI and its applications now develop seems to jump over the technologically orientated HCI designers. 

We need ever better, relevant understanding of and inspiration from the workings of the human, creative and collective mind in real life.  This is especially true when aiming at ambitious and transformative work with AI. It may feel strange to require this now when many seem to think that the marvelous AGI (Artificial General Intelligence) and its specialized agent-relatives will do it. 

Nevertheless, it is unrealistic to think that we humans would not have a significant, but surely a new and different role in everything that AI is made (or prevented) to do for us. What will these future human roles be like? What will the human work with “AGI” be like? Nobody knows, but speculations flourish. Working with the Jolla team, building the AI computer Jolla Mind2 and its human-orientated AI operating system Venho (venho.ai), we are finding practical answers to these questions as we go. The first generation of these will be out and public on the 12^th December, 2024.

Mind knowledge has missed the AI breakthrough

I apologize for my rather blunt view – which I’m happy to correct if I’m terribly wrong in it. Our classic knowledge and understanding in cognitive, computational, neurophysiological, and psychological sciences, rely on the strong individualistic traditions of psychology and computation from 1940s and 1950s, where the present-level, everyman’s AI was not visible or if it was, it was practically science fiction and not seriously included in the building of human-biological-computational models of the human mind and its workings. To be clear, I don’t refer to AI based models (e.g. artificial neural nets, symbolic AI etc) of the mental and memory processes for example. I, refer to explicit AI tools as part of these (non-existent as far as I know) future models. This is somewhat an original way to frame the mental model problem.

Think about this: models of the human attention system, working memory, even different forms of long-term memory were not built for environments and human practices where AI tools and apps would be a natural, everyday part of them. In the famous experiments by Tversky and Kahneman the test subjects were not allowed to use any external devices or help when solving difficult test problems. The original studies of the working memory and attention were run by extremely simple test stimuli and it has been natural to prevent the test subjects from using any extra tools, people or other sources to help them in the test tasks. 

We can imagine that such experiments tell something about how we behave in extremely isolated conditions – a human vacuum – but why should we have cognitive models where isolation, a very unnatural aspect of human life, is the dominant ingredient of test behavior? Some might call this positivism, but there is more.

Our best cognitive-perceptual models of the human mind are barely 100 years old.  It is time to imagine what the role of AI will be in our future life and cognitive models of it in the next 100 years. Trying to fit AI tools into the present AI-human mind system is everything but straightforward since there are hidden learning lessons waiting on both sides (Human vs. AI) of this system and which become visible with every new generation of AI. As far as I know there is no theoretical framework to treat the problem of human memory and attention where AI tools are natural components, just like now are our linguistic and visual skills, which are based on our educational history and culture. They are the accepted, underlying components of classic cognitive models.

Some example areas where new mind knowledge is needed

The following are a few examples of the neglected sides of the human mind that are now gaining new value when UIs and UXs of the future intelligent tools are designed and built:

Collective problem solving and the dynamics of social decision making have been typically pushed aside in the modern traditions of cognitive sciences as represented by the works of Simon and Kahneman, for example.

The nature of new and possible future tools in problem solving and decision making has been overlooked in traditional cognitive models

The isolated human problem solver – has been the main paradigm in cognitive sciences, but has now a new powerful companion, the AI that will be ever present. In future, decision making will not rely on human limited capacities alone and the risks of biases will be mitigated by future AIs – but not without human contribution and even control. The nature of this human contribution is not known.

Human perceptual architectures in natural circumstances are overlooked. Extended human perception is poorly understood although we do have quite a few classic and now modern innovations available. From simulators (cars, flight, other), for example, we already know the importance of a holistic view to perception. With the amazing advances of the digital, the AR/VR/XR applications and devices are taking steps towards this new direction where our perceptual architecture is formulated, created, and manipulated. A key problem remains, how this development guides human perception and experience and how is the task-relevant perceptual information gathered when everything digital is globally available for this, behaviors (IoB) included? There is no theory of the human perceptual architecture as of yet, although AI is already now becoming a significant factor contributing to it. In the end, AI needs the knowledge of our perceptual scenes.

THE BIG QUESTION is: What will be our critical human cognitive, emotional and social limitations that will be noticed only after the AI has become powerful enough and it starts overloading, disturbing, manipulating, and mistreating us? We already have some minor signs of this and it is time get prepared for these unintended consequences of the AI progress.  In the team led by Antti Saarnio, designing and building Jolla Mind2 and its ‘human’ AI OS Venho this is one of the guiding requirements. However, this is only a start.

AI as a source of mental overload?

We will very soon – some of it already now – face the uncomfortable, psychological problem, which seems to be forgotten in the current frenzy of the AI breakthrough: how to interact in harmony, naturally, without being overloaded or disturbed by the powerful, never-tiring, massively knowledgeable, and insensitive AI? This is not only a problem of individuals and privacy, but concerns organizations, as well. 

I have a personal memory that reminds of this problem to come. In the early 1980s, at the then Digital Equipment in Helsinki, we had learned that the managers of the firm were becoming overloaded by emails. This was a rather new phenomenon at that time and with Kirsi Ahola and Markku Silen, we started studying it. We asked, for example:

“How many emails do you think you can send to your manager per day without causing overload problems.” 

We found out that people massively overestimated (around 6 -fold or so if I remember right) the number of emails they could send without causing crowding and communication problems, because they could not imagine what happens at the firm level and how often people send emails. A lively discussion was launched about this phenomenon, but was soon forgotten. There were two lines of opinions: one arguing for (an intelligent AI already then), software-based filtering solutions. My suggestion was to develop the communication culture in the firm since I did not trust the ability of the AI or other sw solutions to cover the variability of the email content and meanings. Now I vote for the cultural solution again.

With the AI, we now face the same problem. Again, we can discuss the solutions, but in this context, it is a topic for another article.

A personal story of mental AI overload

I have enjoyed the use of ChatGPT and NotebookLM. As is perhaps typical of me, my current interests are many, and for example, they include the observer theory in theoretical physics and I have stored a collection of great physics classics and new articles of it. I used them as information sources for my AI companions.  But this is not the only interest I have. With professor Mika Pantzar we happened to study the potential of AI becoming ‘civilized’. Then, for quite tragical personal reasons, I became interested in media persecution. I had been a pre-reviewer for a ‘difficult’ PhD thesis and was attacked in the main media of Finland by aggressive critics who wanted to kill the thesis written by a controversial national figure. I prepared a chat for people who suffer from (social) media persecution. All these interests and some more circled on my mind simultaneously.

Then, one afternoon at home,  I spent about 2-3 hours without a break, listening to NotebookLM podcasts, talking with ChatGpt, testing my “persecution helping tool” and so on. In other words, I moved from quantum mechanics, to media persecution, to civilized AI and back several times.

At the end of this session, I was dead tired. Why?

It was not about the persecution problem – although it could have been that indeed. I simply enjoyed the ride with the AI, but it had no understanding of my emotional-cognitive mental capacity to take in all these wonderful ‘AI thoughts’ and impulses. I did not understand why jumping from one ambitious topic to another became such a burden to me, because I enjoyed every moment of it. I read and heard, spent some time thinking, and did not want to stop, pause or control this wonderful progress. It was a real boost to my understanding of these phenomena. But it had unintended mental consequences. 

How to design the ’behavior’ of the AI so that it does not become a psychological problem for us? What should the UI/UX be like in order to provide optimal conditions for the use of the AI? How to provide human-situational understanding to the AI? This is an extremely complex problem. We don’t know the solutions yet and I will not open up this here.  There are several, similar problems waiting for as with AI and new design thinking is needed. My own favorite topic is, of course, the IoB which is one way to share relevant behavior knowledge – mental or physical – digitally and Ai could use that if properly provided and organized

My story is a minor aspect of what we are going to face and there is a new field of AI psychology to develop and learn.  Then there are the indirect consequences, when we start bombarding our friends and colleagues with AI-generated comments, information, criticism and other such stuff and expect them to come back with their human responses. This is becoming a very serious issue for individuals and firms alike. We can drown anyone in half an hour and are going to need self-defense mechanisms for this, tools and proper arrangements of AI-supported work.

A gloomy possibility is that we will see a struggle between AI-generated materials and communication traffic in organizations where the human role is to stay aside since there is simply too much to deal with. It’s a multi-faceted, major risk. Quite recently, I read about the book by Robert Sutton and Huggy Rao: The Friction Project: How Smart Leaders Make the Right Things Easier and the Wrong Things Harder. My first thought was, that there the use of AI finds its place if we want AI to make the right things easier. Have not read the book yet.

Finally, I will not make the fascinating comparison of the DoS and DoBS here but will return to it later. They share fascinating similarities, analogies at least.

New generation operating system to work with AI

Jolla Mind2 and Venho.ai – new demands for operating systems

Göte Nyman, Professor, University of Helsinki, Finland
Antti Saarnio, Co-founder Jolla, Founder Venho.ai

To the best of our knowledge, Jolla Mind2 is the world’s first AI computer that runs on a specifically designed, native AI Operating System (OS) venho.ai (Venho). How is this AI-OS different from classic operating systems and what is it for?

Here, in the first part we explain the purpose and the underlying ‘human’ logic behind Venho. The second part, which will be delivered later, takes lessons from the short history of digital UIs in order to see where the fast AI development is taking the UIs and the operating systems that run with AI. 

Figure 1. Jolla Mind2 and Venho – a human perspective to data

Order in the AI house!

AI is conquering homes, public offices, educational institutes, firms and even armies. Generative AI is exactly what it is called: it generates massive amounts of data, images, actions, and texts. At the moment, close to 200 million companies in the world use generative AI and in two years there will be hundreds of millions of applications using LLMs. Personal computers and other digital devices will be flooded with AI apps and their innovative variations, either as standalone versions or included as added AI features in the everyday digital tools we use. Standard UI designs are in trouble. AI is offered in the cloud and locally and in any possible combination of these.

The inevitable outcome is, already now, that users need seamless, reliable and secure support and management of their AI use, at work and in private life alike. How to survive and manage this immense tech tsunami? Jolla Mind2’s solution is its operating system Venho, having a privacy-respecting design and strict human control of its core AI-driven functions. It is built to serve and protect the user and to maintain a functional, orderly, and creative AI environment. Its ways to treat memory data differs from traditional operating systems and has taken lessons from the workings of the human memory and attention mechanisms.

Why such a special operating system like Venho?

The traditional definition of an operating system (OS) is that it marries the computer hw and software: it allocates processing resources between applications, prioritizes them, has interfaces for accessing the connected devices and tasks, and it provides resources and data structures for running tasks and applications on different hw. 

Each operating system has its own UI ‘layer’ and style that makes the devices usable and inviting for the human user. These systems are designed to support the devices, tools and apps configured in it. The user acts as the driver, manually navigating between apps, searching for files or messages and organizing them, planning and prioritizing tasks, and executing actions like replying to emails or configuring events step by step.The traditional operating systems were not directly built to collaborate with the user in complex everyday jobs, tasks and behaviors.

Venho understands the digital life of the user

The best AI functions and services are those that have contextual sensitivity, work in concert and aware of our life, take into consideration the human limitations and situations, they are well organized and properly offered – and the user has control over their critical functions at any time. This is important also in the possible cases of hallucinations or poor logic by AI so that the user can block, change or prevent them. AI running wild is a nightmare and even uncertainty of what it is doing is disturbing and risky. Venho has been designed with these requirements in mind. How is it different from a standard operating system that we know from Windows, Apple and others?

In Venho UX, the user becomes the decision-maker, with AI acting as an intelligent and contextually aware assistant and memory manager. Repetitive workflows, such as categorizing emails or prioritizing tasks, are handled by AI agents in concert, allowing users to focus on approving decisions and overseeing workflows rather than managing them manually. Its core ‘mission’ is to get the best out of the AI, to protect and support the user when using AI in natural everyday tasks at work, with people and in private life. Locally and globally. It can live with the fast development of LLMs by adopting them as they become available and can be guaranteed as secure tools. 

Venho has an innovative memory and attention system. It is designed to function like human memory, enabling users to store and retrieve relevant and meaningful data in an orderly manner. It acts contextually, reducing time spent and the workload in searching for information. Activities under Venho are organized contextually to streamline workflows and minimize distractions, allowing users to concentrate on what truly matters. This minimizes app-switching when AI agents can autonomously manage the workflows related to specific activities. A seamless user experience is created.

As examples, actions are executed through intuitive semantic commands (e.g., “Play my favorite playlist” or “Schedule a meeting”), eliminating the clicking and browsing of dropdown menus or cluttered side panels. This is possible since the app layer of Venho evolves into a protocol layer, removing the need to open apps and manipulate their details altogether. Users interact directly with app functionalities using natural, semantic commands such as “Order a pizza” or “Book a ride.”

With an AI having such a strong capability, it is natural that Venho gives user the critical control. While AI handles emails, messages, and calendar workflows in the background, the users retains full control by approving all final actions, ensuring trust, relevance, and accountability.

Venho collaborates, interacts and works proactively with the AI user. It ‘knows’ where the human user, having her limited cognitive and other human resources, needs support and help. It relieves her from burdensome memory and attention load by monitoring and integrating relevant data from her distributed data bases and digital communication channels.  Relying on the knowledge on how the human memory and attention systems work and what are their weaknesses, Venho has been designed to ‘extend’ and boost the user’s personal memory and enhance her focus at work and in any other digital activity. Built on these human requirements Venho has a way to control the AI in the use of devices, tools, services and apps and make them naturally human-oriented.

Figure 2. An example look at Jolla Mind2 in action

Memory and “memory” – human inspiration in the design of Venho

The inherent problem of a computer memory is that it is totally different from human memory, almost ‘orthogonal’ to it. Some decades ago, associative memories were expected to help in this.  Brain theories are not even close to understanding the real, natural, and complex functioning of the human memory. Artificial neural networks are amazing general-purpose distributed memories, but they lack the systemic ‘human’ structure and many other genuine human features.  Hence, they do not fully support contextual and first-person human memory.

Venho takes inspiration from what we know of the strengths and weaknesses of different forms of human memory and attention, like episodic, semantic and procedural memory, to mention a few. Using this as inspiration, the design of Venho guides AI in the user in tasks where such human resources and processes are under stress and need support. This can be compared to what is called biologically inspired neural networks, where the artificial net includes any useful features discovered in real neural systems. Quite similarly, with AI we have a lot to learn about the real functioning of the human mind.

The way Venho uses computer memory differs significantly from standard data base orientated processing. It can take the first-person perspective to the user’s physical data (and even a third person perspective). In this way, retrieval and storage of memory items and ‘chunks’ of information can be made relevant and work in a way that reminds the workings of the human mind. For example, a complex construction or collaboration project generates a pile of agendas, meeting minutes, decision documents, communications, commentaries and other interaction data. These are often shared and distributed in various tools like WhatsApp, Slack, email systems, and specific project and collaboration environments. Over the history they might even get lost in the physical memory jungle.

A project manager, for example, devotes much of his memory and attention capacity to work with this information and learn to use it. However, this can become a significant cognitive load that we active digital workers so well know.  Such personal memory knowledge can also become person-dependent and a risk for critical collaboration and problem for a firm when only someone knows “what, where, when, who?” The data may well reside in data systems but the intelligent perspectives to it have special value. Venho improves the digital worker’s efficiency and relieves him from unnecessary memory and attentive load.

Personal perspective to data

Venho observes and manages digital data just like a user would do: from personal or first-person perspective. It knows which documents and interrelated documents and other information entities are relevant for each problem or a question presented to it, and how to retrieve them so that their information is optimally delivered to the user. This ‘perspective knowledge’ is very human-like and relieves the user from burdensome searching, browsing and scanning of relevant physical data structures, while it also makes this memory function person-independent. Interestingly, Venho can take the perspective of the ‘third person’ as well so that it is possible to know how other persons ‘see’ the data, that is, how it looks like to them. This kind of intelligently organized memory data can be easily provided to a newcomer or any other relevant party benefitting from it. 

Sometimes the data of interest consists of episodes, that is events (decision making meetings, pitches, informal discussions, even communication conflicts) where the user has been a participant. Some memory data can be outside the user’s presence and activities but still relevant to the problem at hand. Venho retrieves this data as a meaningful episodic entity. The other author (GN) has a history as a psychology professor, with extensive experience in teaching cognitive, brain and perception sciences, and loves to call this memory function of Venho as its own episodic memory since it extends our human episodic memory and works in a manner very similar to its human version. It seems quite evident that this kind of memory will become part of every AI-based memory system since we humans know, by experience, its power. We all remember significant episodes of our life, like the other author (GN) still vividly remembers the moment, (sometimes in the early 1960s)  as a teenager when his physics teacher commented his presentation of a physics homework solution at the blackboard: “Nyman, one day you will be a scientist.” It was the school that had the worst reputation in Helsinki.

We have other human memory functions as well, like the procedural memory, for example. Its definition (wiki) is “Procedural memory guides the processes we perform, and most frequently resides below the level of conscious awareness.”  As an example, extensive project phases and their procedures can be hard to remember, and to repeat without extra guidance, especially when they span very long periods of time and have multiple components and branches, like in typical construction and collaboration projects. Sometimes they have not been properly documented. Venho serves our procedural memory by retrieving meaningful, integrated entities with a procedural perspective so that they are easy to find and access. In the second part we will explain them and their meaning for UI design in AI environments.

Jolla Mind2 and Venho take a human perspective to data

Below are a few examples (there are more, and other similar ‘human’ features are in the pipeline) of these cognitively guided designs in Venho and how they live in the Jolla Mind2 functions:

• Integrated email processing that relieves the user from continuous monitoring and scanning of the messages coming from different digital channels.
• Automatic classification and summaries of the channel messages according to their content and significance for the user.
• Maintaining focus by intelligent filtering and prioritizing the inputs to user.
• Building and maintaining a trusted personal data base for AI use and chat interaction.
• Content-based, dynamic contact management to alert the user and keep him aware of significant events and relevant activities in her network of contacts.
• Contextual and episodic retrieval and organized integration of data from significant events and data sources.
• Human in control at every significant phase of AI actions.

Even with the most advanced AI, the essence of it is and will be meaningful collaboration with people and communities. Venho is designed to support this.

Elinor Ostrom, the famous Nobel Laureate (economics) became famous for her work on how communities can intelligently handle their commons (water resources, fields, forests), better than anyone of the community members could do alone.  The secret to this was continuous interaction and feed-back at all levels of collaboration and decision making, and the adoption of useful heuristics to guide people’s behavior. These guidelines are now valuable and even critical in many uses of AI, especially when it involves complex and risky decision making, extensive collaboration, control of AI activities, and in critical tasks. 

With AI, even with the imaginary AGI (Artificial General Intelligence), such (cultural, local, practical, experience-based) heuristics are of significant value when relevant, intelligent thinking is difficult or not possible.  Venho relies on interaction between the user and the AI by giving the user and the community control over significant operations.

On the future of UIs in AI environments

After a decade of standstill, UIs are now entering a new evolutional phase (introduction of chats, NLP, gesture use, task representations, visualization, AI interaction, AR, XR etc) and we can see them in the familiar AI applications like those from Apple, Google, Microsoft, and OpenAI, for example. However, the new features still reflect the recent past and there are no strong standards as of yet, although de facto standards will probably emerge simply because of the masses of users adopting them. 

Where are the UIs of intelligent systems going – lessons from history

Figure 3. Making new tools. A photo received from my (Göte Nyman) first
professor of psychology, Kai von Fieandt, University of Helsinki, Finland.

It is evitable that the nature of digital UI, UX and other human connections with the AI will change profoundly. There are several reasons to this:

• With AI the user can engage in his natural tasks where the ‘interface’ reminds more of a connection we have as humans between each other or with nature and any natural object of activity.
• Because of the learning and adaptation ability of the AI, the user is relived from giving detailed instructions and managing the actions of the apps and services used.
• AI can be an active interface that guides the behavior of the user and even her community.
• User behavior data (like the Internet of Behaviors,  IoB) become a significant source of knowledge for the AI which can then approach and help us when it is relevant and not to disturb us.

We will return to these later and look at Jolla Mind2 and Venho, the future of UIs and lessons learned from the short digital history. A big question is the future of ethics with AI.

Distributed development, adoption and the social risks of AI

AI application tools enter all sectors of life and centralized development and control of the AI market introduces new risks, many of them not visible as of yet. As Antti Saarnio from Jolla has recently warned, we can already see signs of how the power obtained by the dominant AI players can form a major risk factor for democratic development in societies. It is a question of how AI will be developed, marketed, and distributed and whether it will be centralized, distributed, or regulated in nature – or any combination of these.

We can observe how the advanced and speculated future AI secures huge capital and even political power to the few. EU has introduced its first regulations and some countries follow, but many don’t. Ideological drivers have already taken a grip of the global AI industry. China, U.S, EU, and Russia do not play the same game. There is surprisingly little discussion on how AI innovations, especially human-orientated ones could help minimize or even prevent such risks. Venho represents one candidate approach and an example of this.

There is no time to waste. The social impact of AI is increasing exponentially: it is improving in ever demanding and higher-order content understanding and invites massive participation of specialists to build their own AI tools, environments and solutions. A new strategic question emerges, what it means that extensive special expertise will be available everywhere from AI: “Strategy in an Era of Abundant Expertise”. 

Novel and intelligent AI applications can be expected from professionals in all sectors of life and businesses. They need well-founded support and also AI operating systems that make adoption, use, sharing and development of the AI tools healthy, safe and feasible. It seems plausible that operating systems like Venho will find a significant role in securing privacy, the autonomy of AI use, and the possibility to build new AI tools and applications in a distributed and fair manner, independent of the AI giants.  Prepared to this, Venho includes templates to be used for different contents and purposes, and to scale up AI use.  It is hoped that interested parties find it effective and easy to develop their own applications and even templates that can be massively distributed, shared and commercialized. We expect this to speed up AI adoption in all sectors of industry, public and private services and businesses.

From persuasion to curiosity – letting our motivations free

In December, 2010 I wrote the blog: “Design and motivation to sustainable behavior”, starting with the questions:

“Why shouldn’t a sustainable product be beautiful and classy, and even more inviting than its less sustainable competitors?”

”Why should people, who are willing to protect our natural resources, be forced to – or at least be ready to – sacrifice the beauty of their homes?”  

I could continue it today: Why should people eat vegan food if it is not more delicious than meat, for example? Why should we eat vegan cheese or vegan burgers if they taste like xxxx? Why should we listen to demands that accuse us for bad environmental behaviors? Well, there are ways to inspire us to achieve something good and still preserve most of our preferences. Vegan food can now be luxuriously delicious and compete with whatever delicacies there are available. We can be inspired by art to change our thinking, simply because it is fascinating. It’s time to recognize and respect our true motivations and build on them when trying to change our behaviors for the better.  Some artists and chefs already pave the way.

Soon after my blog, I organized a one-hour pop-up workshop in a conference hosted by Sitra, Finland and introduced this thinking. The half a dozen participants came from marketing management from large Finnish and international firms. My main message was to respect the true human motivations as they are and occur, and to forget the idea of ‘making people change their attitudes because it is necessary’.  At first, they did not understand this at all and were visibly puzzled and even confused – but for some reason, became curious. They stayed there to find out what I meant by suggesting that trying to force or pressure people to sustainable behaviors or change their ‘attitude’, can kill their motivation to do so and that for sure, it is not inviting to them. It took more than half of the time, before I could see the first signs of insight on some faces. Nobody contacted me after that, so it was barely a success story.

Of course, I don’t suggest that the following motivational approaches are the only means to aim at beneficial behaviors, but they are typically neglected. For some reason it seems to be so natural in media and ideological-political communications to blame someone and impose demand and guilt. Pleasure has no place there then except perhaps for the feeling of being right.

Artists make us perceive

In the New Your Times article (August 31-September 1, 2024) there is an insightful story of human motivation.  It does not mention the word ‘motivation’ at all, although it is behind its main message. The story is about an artist and her work, Jenny Kendler – a climate activist as well, who wants to “embrace the interconnectedness between humans and non-humans.”

The title of the story is “Sounding an alarm, but unobtrusively” where Kendler is quoted: 

“I aim to seduce people through beauty”.

“I aim to connect with people through beauty and subtlety, and all of the logic that good art can bring.”

Her installations in Fort Jay, on Governors Island, New Your City, are described and the central piece Other Pearl, “… comprising rainbow-lipped oysters in 12 half cells.”

Kendler wants to help us understand – not by force – how and why to restore oyster reefs to New Your harbor and in general, how to make us aware of pollution and sensitive to perceive the endangered marine life and ecosystems. She aims to do this through subtle, creative works of art and materials of artificial and natural origins – and the exceptional beauty of her works. She even says that “I knew that it wasn’t going to happen by slapping people upside the head with didacticism.”

The NYT article ends with the quotation: “I don’t think I can replicate the beauty of the natural world, but I can remind people what it feels like to be connected and to belong.”

Such a deeply human and creative insight and similar sensitive activity you would expect from educational activists, social scientists and politicians, but it is rarely, if ever happening. The well-known and reported environmental problems are typically buried under paradigmatic, political and scientific noise of confrontations and the fight for media space and power.  Indeed, the powerful NYT story is not an explicit story of the human motivation, but about what really drives us in such complex behaviors. It is behind it all the way.  

In my blog I argued, rather bluntly, and in a form that might appear extremely pessimistic:

“The main motivation of people is not – and will not be in the foreseeable future – the promotion of sustainability. Despite the new progressive trends, in real life, and in most parts of the world, nothing remarkably sustainable will happen without inspiring motivations.”

However, in the spirit of Jenny Kendler, it is in fact a profoundly optimistic statement, as it rests on the assumption that we are willing and capable to connect with nature and to act accordingly, when certain human motivational conditions are met. For this to happen, we must have a chance to amend from our natural and spontaneous motivations that we and our ancestors have learned and adopted. The challenge is to recognize these true motivations in people. Many of them are related to our subjective perceptions and experiences of quality.

Kendler understands these motivations, and trusts that as a part of human nature, they drive our behaviors.  With her art she can invite people to stop, focus, sense, perceive, feel and remember and then be ready to find the place of nature in their own life and thinking. Such spiritual connections and actions do not emerge by commanding people.

There is a sad counter-example to this in Finland right now, concerning one of our large paper mills, which is accused of killing thousands of freshwater pearl mussels, in a northern river where these rare creatures have survived. The media is crowded with accusations and angry commentaries and it’s been a major news event for some time now. In a rational sense these accusations are well grounded, but of course, the main point is how to prevent this in the future and improve the present situation. Already now, the case has urned into a political, ideological, click-bite, and even economic struggle – a typical aggressive media campaign with numerous stakeholders using it as their own platform. It’s a destructive fight all too familiar in today’s aggressive media space. No doubt, many have the opinion that this is necessary in order to wake up the industry and to protect the nature. 

However, my guess is, that people are already getting tired and disturbed by this poisonous media atmosphere just like they are tired of other angry, aggressive and destructive feeds in their present media environment. Love, sensitivity and care have no place there. With a few exceptions, any potential beautiful story of protection, care, and respect for these rare creatures, has been pushed behind the frontlines of the media battle. People have not forgotten these curious living objects, but media feeds them fights and failures. Indeed, my other guess, based on Google Trends as well, is that people find inspiring information by themselves and try to learn about these amazing creatures. We don’t (yet) have an artist like Kendler who could touch our hearts the way see does, without blaming and accusing, but still inspiring us to what is good. If someone with such touch would appear here in Finland, Jenny Kendler’s works have a message to tell.

It is not the first time that artists open a door to the world and help us perceive it and do something to care for it, to admire, and protect it. Already the cave art, the paintings of Lascaux in France (25,000-17,000 BC), for example, talked to people with beauty and elegance. They were not declarative representations, lists, or recordings of animals only, but a reflection of the human spirit and motivation that produced them and was meant for others as sources of inspiration and awe.

Food reveals our ‘unholy’ motivations

We are all professional eaters and have learned to enjoy or suffer of it. We have probably conducted more experiments and have more experience with food than with any other substances in our lives. We have learned to sense, perceive and analyze the feel and taste of any food. We can be told that “meat is not good for you”, or “you should eat more vegetables”, but this does not change the way we taste our food and only rarely, it makes us curious enough to put our preferences at test. 

When someone offers quality that we recognize or expect, we appreciate it and can be motivated and ready to try. It’s like a good scientific experiment, where we are the both the subjects and the experimenter: if the experiment fails (expectations are high and well-founded, the food has good ingredients, but we don’t like the feel or taste) then we don’t want to repeat it, we know the result.  The customers of Daniel Humm in his restaurant in New York have arrived there willingly to ‘run their subjective quality experiment’, for which David offers all the necessary materials and the whole context.

Daniel Humm has a Michelin star restaurant, Eleven Madison Park in New York. It had already three Michelin stars and was voted the best restaurant in the world in 2017, when they went vegan in 2021. In the Time article, Humm comments this:

“It’s not the cost of the ingredients someone is using, it’s the human thought, the work by hand…[the] experiences that you can only have in a few places in the world.” 

“If Eleven Madison Park can change the perception of what a high-quality ingredient is, and people are willing to pay more for beets and carrots and cucumbers, that would allow a lot of [plant-based] restaurants [to be] profitable.”

“This is the future. We’re not saying anti meat, but we’re saying pro planet.”

People pay more when they feel that the experience is worth it and David Humm aims at exceptional and high-quality customer experiences. Wisely, he does not blame the meat-eaters or criticize their possible ‘wrong’ motivations. He offers a chance to experience something excellent, luxurious, exceptional and delicious.  Through respect for his customers’ motivations, he wants to help them find new experiences, in the form and tastes of vegan food. The result is – when he succeeds – that something good happens to food industry and nature, but first of all, the customers enjoy it and are ready to repeat their experiences in Humm’s restaurant, in other restaurants or at home. They do not start this subjective journey only because it is good for the nature, but because they enjoy it AND it is good for nature, animals and the environment. At the restaurant they can forget everything else and enjoy the food. Slow thinking has its place after the dinner.

I have taken up only these two inspiring examples, Jenny Kendler and David Humm, because they reminded me of my blog from 2010, and the difficulties to find resonance for that kind of thinking then. Today there are increasingly more innovative individuals and firms who have this new and open understanding of what makes up a true human motivation and how to take it seriously and even make business based driven by it. This concept of motivation is not a mysterious, psychometric feature or a statistical factor isolated in a closed psychological laboratory, but something extremely complex, but something that lives with us when we come in touch with art, food, sports and any other contexts of our everyday life.

These new insights, ‘the motivational views’ are based on the respect of the way people experience subjective beauty and quality, how we enjoy the taste and consistence of foods. Trusting these motivations and respecting them, makes it possible to surprise the consumers with the meat-likeness of the Impossible Burger or with the delicious flavors of vegan cheese. It is not a sin to love the familiar and comfortable meat- or cheese-likeness of a vegan food. A serious anthropological, ecological and industrial analysis, would reveal to us why we have learned to enjoy food the way have. Nothing in this has happened overnight, but it can and will happen.

Moore’s law and some ChatGPT4o fun.

The famous Moore’s law was introduced in his article where only four data points are fitted to the straight line. I saw it first time in Intel Museum, Silicon Valley where it was stored in a glass/plastic cabin and I used to present it to my students to think about when discussing curve fitting. So, here I tested ChatGPT4o what it says about such a scarce fitting and the powerful law in the real world that followed. I fed the original figure from the article and asked ChatGPT4o to comment:

Here is the response:

“The image you’ve provided is the original graph from Gordon Moore’s 1965 paper, which led to the formulation of Moore’s Law. Despite having only a few data points, it became a famous and influential observation for several reasons:

1. Remarkable Insight: Gordon Moore made a profound observation that the number of components per integrated circuit was doubling approximately every year. This trend was a critical insight during the early days of the semiconductor industry, forecasting a rapid and exponential growth in computing power.
2. Predictive Power: Even with limited data, Moore’s prediction held true for many years. The law provided a reliable forecast for the semiconductor industry, guiding long-term planning, research, and development.
3. Exponential Growth: The trend of exponential growth is powerful and transformative. The prediction implied that computing power would increase dramatically over a relatively short period, which had far-reaching implications for technology, business, and society.
4. Industry Adoption: The semiconductor industry adopted Moore’s Law as a guiding principle. Companies used it to set targets, plan product development cycles, and make investment decisions. This collective focus helped ensure that Moore’s prediction became a self-fulfilling prophecy.
5. Historical Context: In the 1960s, the world was experiencing rapid technological advancements. Moore’s Law fit well into the broader narrative of innovation and progress, capturing the imagination of technologists, business leaders, and the public.
6. Extended Relevance: Over time, Moore’s Law was refined and extended. The doubling period was later adjusted to approximately 18 months, but the essential insight remained relevant for decades, continually influencing the pace of innovation in the semiconductor industry.
7. Foundation for Technological Progress: Moore’s Law set the stage for the digital revolution, underpinning advances in computing, communications, and numerous other fields. It became a cornerstone concept, illustrating the potential for ongoing technological advancement.

In summary, Moore’s Law became famous and successful because it provided a simple, yet powerful, prediction that accurately described the rapid growth of computing power. Its relevance and influence extended far beyond the initial data points, shaping the direction of technological development for many years.”

Next, I asked it to be more precise on this:

“Gordon Moore’s prediction, which later became known as Moore’s Law, was based on several key observations and insights he had about the semiconductor industry at the time:

1. Empirical Data: Moore’s prediction was grounded in empirical data. He observed that the number of components (transistors, resistors, diodes, or capacitors) on a chip had been doubling approximately every year since the invention of the integrated circuit in 1958. This trend was evident from the rapid advancements in semiconductor manufacturing technology and chip design.
2. Technological Progress: During the early 1960s, there were significant improvements in photolithography, etching, and other semiconductor manufacturing processes. These advancements allowed for smaller feature sizes and higher component densities on chips. Moore recognized that these technological improvements were likely to continue and accelerate.
3. Economic Incentives: The semiconductor industry was (and still is) driven by strong economic incentives to produce smaller, more powerful, and more cost-effective components. As the manufacturing processes improved, the cost per component decreased, making it economically viable to produce more complex and powerful integrated circuits. Moore understood the economic motivations that would drive companies to pursue this trend.
4. Research and Development: There was substantial investment in research and development in the semiconductor industry. Government funding, especially from the defense sector, along with private sector investments, fueled rapid innovation. Moore could foresee that this level of investment and focus would likely result in continuous advancements.
5. Industry Momentum: The industry had a momentum of innovation, with various companies and research institutions working on similar problems and sharing knowledge. This collective effort created a reinforcing cycle of innovation and improvement, which Moore anticipated would continue.
6. Vision and Insight: Moore had a visionary understanding of the potential of integrated circuits. He saw that the exponential growth trend was not just a short-term anomaly but a fundamental characteristic of the technology’s development trajectory. His deep understanding of both the technical and economic aspects of the industry allowed him to make this bold prediction.

In essence, Moore’s prediction was based on a combination of observed data, technological trends, economic factors, and a visionary understanding of the semiconductor industry. His insight was not just a lucky guess but a reasoned extrapolation of the patterns he observed and the potential he saw in the technology’s future.”

This was quite informative and educational indeed, for discussions about fitting functions to data. ChatGPT4o does not seem to worry about it at all, and has excellent potential explanations to it. This would be the place to read about Moore’s thinking then.

Visual quality perception by ChatGPT4o

It does matter how we watch images and scenes. It is a life-long journey to learn our personal ways, and we adopt different perception and observing strategies. With the present technology headsets eye movements can be reliably recorded to know what a user looks at. This can be made an efficient UI like it is done in Apple Vision Pro. But how and why we look at images makes all the difference. A mind of a brain surgeon looking at an fMRI image is not the same as the mind of an artist looking at the same image. This was one of the reasons why in our team we developed a method to “measure” subjective impressions of images, especially image quality. 

In our IBQ method (Interpretation Based Quality) we asked the test subjects to describe, with positive and negative image quality attributes, why they gave a certain quality score (0-100, for example) for a test image. These attributes, we believed, would tell which subjective features of a test image were behind the score it obtained. The attribute clouds received from large amount of test subjects told us how different test images were viewed and evaluated by our subjects. The subjective data supported the development of ever better mobile phone cameras at Nokia for a decade but it was used also in other contexts with print images, for example.

Now we have ChatGpt4o and other similar AI tools that can “see” and classify objects based on that. This made me ask: in addition to classifying objects, can ChatGPT4o be instructed (prompted) to “pay its attention” to the same image features as our test subjects typically do? This does not mean that it would “experience” the images in the human way, but its performance could be more human-like.

The human side of the ChatGPT4o

ChatGpt4o demo on 13th May 2024 was an astounding show of its qualities and future potential. However, what made it extra compelling was its natural, spontaneous-like human-computer interface.  It had first signs of Jobs-kind of beauty and simplicity in it. Following the demo, I admired the designers who had included the simple possibility – to interrupt the speaking of the GPT and make communication interactive.  I’m sure we will see more of similar human designs in the very near future now that their huge value is evident.

It is quite possible that the responses of the GPT in the conversation felt so natural that the design aspect was not self-evident. It was like watching a movie where it is easy to forget that it is just a play. This is only the beginning of the development where ChatGPT4o and movies will meet.

Visual quality test for ChatGPT4o

Waiting (this happened on 14^th May 2024) for all the voice interaction and other coming features, I got curious to try it with a subjective image quality evaluation task. I decided to use human data to guide the ChatGPT4o in perceiving natural images. Such a test might sound like a turn-off for most GPT users. However, it carries some interesting general potential.  The idea was to feed ChatGPT4o human quality attribute data from our subjective image quality tests and to see if this data can guide it in perceiving-evaluating the quality of test mages. I wanted it to follow the guidelines from our tests with human subjects. The same trial could be accomplished by using the image quality variables of various physical and photometric measures as well. Our data is, however, human and subjective and in this sense, I hoped to guide ChatGPT4o to perceive images in at least in a somewhat similar manner how we do it.

Superficial trial, sorry ChatGPT4o

This was a superficial, quick trial and the attribute data were from a specific context (image processing circuits and how they affect camera raw images).  It was fun to test however and learn about its way to “perceive” images when guided like this.  

Background

In our Visual Quality team, we collected a large amount (of the order of tens of thousands of mentions) of subjective image quality attributes, positive and negative ones, which our test subjects used to explain why they liked some image quality and/or why they did not like it. In our dozens of experiments, we had used especially high-quality image data, either digital and print, the highest quality images were from a leading French fashion magazine. 

I gave ChatGPT an image quality evaluation task and instructed (prompted) it with our human quality data obtained for the experiments with digital image processing circuits. Of course, I don’t know how exactly it does that kind of a task, but was curious to see how it behaves in it.  

First image – a hotel room in Paris. 

 I instructed the ChatGPT4o:

“In our studies on mobile phone camera image quality, we have found out that the subjects who observe our test images can describe the reasons for their linking or disliking of the quality of the images by expressing this with positive attributes and negative attributes. 

The attributes obtained from our subjects, were the following and they are listed here in order of importance. 

Positive attributes: 1. sharpness, 2. natural, 3. bright, 4. natural colors, 5. colorful, 6. good contrast. 

Negative attributes: 1. grainy, 2. unsharp, 3. yellow, 4. pale, 5. not sharp, 6. blue, 7. green, 8. too bright, 9. unnatural, 10. grey, 11. light, 12. oversaturated, 13. dim.

I have attached one test image for you. Can you tell me how does that image look like to you and what it makes you think about it.”

Screenshot

The response was very human-like:

”The test image you provided appears to be a well-composed photograph of an elegantly decorated room with a view of a dome structure outside, likely in a city setting. Here are my observations based on the positive and negative attributes listed:

Positive Attributes:

1. Sharpness: The image is quite sharp, with clear details visible in the furniture, the texture of the fabric on the chairs and sofa, and the view outside the window.
2. Natural: The overall setting and lighting look natural, with the sunlight streaming in through the window, illuminating the room evenly.
3. Bright: The room appears bright, enhanced by the light colors and the ample natural light coming through the large window.
4. Natural Colors: The colors in the image, especially the greenery outside and the subtle pastel tones inside, look natural and pleasing to the eye.
5. Colorful: The image has a good mix of colors – the pink upholstery, the greenery outside, the white furniture, and the patterned wallpaper all contribute to a colorful and lively appearance.
6. Good Contrast: There is good contrast in the image, particularly between the bright outdoor scene and the more muted tones inside, which helps in distinguishing different elements clearly.

Negative Attributes:

1. Grainy: The image does not appear grainy; it looks smooth and clear.
2. Unsharp: The image is sharp, so this negative attribute does not apply here.
3. Yellow: The colors do not have a yellow tint; they appear balanced and natural.
4. Pale: The colors are not pale; they are vibrant and well-saturated.
5. Not Sharp: This attribute does not apply as the image is sharp.
6. Blue: There is no noticeable blue tint in the image.
7. Green: The only green visible is the natural greenery outside, which looks appropriate and natural.
8. Too Bright: The brightness is well-controlled; the image is bright but not overly so.
9. Unnatural: The scene looks realistic and natural.
10. Grey: There is no significant grey cast; the colors are vibrant.
11. Light: The lighting looks good, with no areas appearing overly light or washed out.
12. Oversaturated: The colors are well-balanced and not oversaturated.
13. Dim: The image is bright and well-lit, not dim at all.”

The detailed analysis of the image is quite impressive and I would claim, close to how interested human subjects would explain the quality of the hotel room image. 

After that, I fed more detailed instructions based on the proportions of these different attributes in our studies. I suggested it uses them as weights for the attributes when evaluating the image quality. Again, it gave a rather good overall evaluation and detailed, specific comments to each of the 6 (positive) + 13 (negative) quality attributes. I have not included these responses here, but the overall impression it gave was the following:

”Overall Impression:

The image predominantly displays positive attributes, with a higher sum of positive attribute proportions (0.379) compared to the negative attribute proportions (0.621). This suggests that the image is likely to be well-received by viewers. The sharpness, natural lighting, vibrant colors, and overall warm and bright ambiance contribute to a pleasant and appealing visual experience. The few negative attributes, such as the presence of red, do not significantly detract from the overall quality of the image.

This room image conveys a sense of comfort, elegance, and warmth, likely making it appealing to viewers and suitable for showcasing the quality of mobile phone cameras.”

Note: the numbers 0.621 and 0.379 are misleading since they were the proportions of all positive and negative attributes in our research data (hence their sum is 1.0). ChatGPT made a mistaken reference to them because I had not provided proper data and instructions.

Clearly, ChatGPT4o was able to use the human attribute information as relevant guidance. A full, qualitative analysis of its potential performance is of course, a complex task that would require careful and systematic preparation, but it is indeed an interesting possibility.

A personal-historical image

Next, I used a personal, historical image to it (below) that has a special meaning to me. I did not mention about its meaning. It’s my Finnish Championship karate team from 1972. Here is the overall impression ChatGPT4o gave to it:

”Overall Impression:

The photograph, likely from the 1970s, is a well-preserved black-and-white image showing a group of individuals in martial arts uniforms. Despite some graininess and the inherent limitations of older photography technology, the image is clear, bright, and sharp. The lack of color does not detract from the image quality because it is consistent with the era’s photographic standards.

The image conveys a sense of historical authenticity and nostalgia. The subjects are well-focused, and the background elements are visible without distracting from the main subjects. Overall, the image quality is good, considering its age and the technology available at the time.”

Screenshot

This was a wonderful reponse: it has recognized the martial arts context, and also been able to time it right. Its comment on the feel of nostalgia and authenticity is impressive. Indeed, the subjects were well-focused also mentally since this was taken just before the Championship finals. Its “perception” of the image was perfect from my point of view.

An unusual image. 

Next, I used an unusual image from Louvre, Paris (below). I also instructed it to use a specific scoring system from-50 (very bad) to +50 (very good) to express the perceived image quality. However, my instructions where again rather open so clearly it was not able to use the scale right and in a sensitive manner. The problems were caused by my insufficient guidance. The overall response:

”Conclusion:

The overall quality score for the image, based on the relevant attributes and on a scale from -50 to +50, is +50. This suggests that the image quality is excellent, with clear, sharp details, natural lighting, vibrant colors, and good contrast. The image effectively showcases the scene of people capturing the famous artwork, likely taken in a museum setting.”

Despite the rather unusual setting it was able to locate and interpret it well. However, at the first trial it did not comment on the unusual setting in the image.

Screenshot

Some conclusions

I will continue with my experiments and see how the ChatGPT4o performs when I prepare proper and systematic guiding information to it. This was a good example also on how important it is for the user to provide best possible data and prompts. In this quick trial I just wanted to find out how it “behaves” when “perceiving” images and the details of the trial were secondary. However, the main question is, what does this mean?

This simple exercise was meant to test the use of human experimental data to guide ChatGPT behaviors in some systematic and beneficial way. For example, it is possible to feed an ‘observer model’ and suitable human behavior data to guide the ChatGPT in how it should (or can) perceive any sensory material – visual, audio, other – and make interpretations and classifications so that it imitates human perception behavior. 

This case was about image quality only, while in real life, outside the mobile phone use, we are interested in the situations, meanings, culture, style, purpose, uses, dangers and opportunities, and other aspects of perceived objects and materials.  Then of course, it is possible to teach ChatGPT:s to read texts and images like certain people would do, if we have a way to express that to them. ChatGPT could then pay attention to the texts in the same way we (many different) humans do. 

ChatGPM:s tcan be made to imitate human perception strategies. We can imagine a plethora of situations where this could be useful: for patients suffering from visual and other handicaps to robot behaviors when they interact with humans, and many others. 

After this simple test I remain impressed.

Peace implants into the military complex?

I have (at least) two paradoxical beliefs: the first one concerns the military and the second is about shooter and fighting games. 

The military exists to wage wars when necessary. However, my assumption is that the military does not – in general – exclude partners who also do something to promote peace, as long as their products serve the military as expected. At the writing of this, there are opposite movements to this, for example those that want to stop buying arms from Israel because of its war in Gaza.

In the most popular games, the gamer takes the role of a soldier, killer or otherwise destructive character. Many of my psychology colleagues are worried that such games can lead to more aggressive behaviors. However, I believe that many of these gamers would be ready – when playing – to invest a small sum to promote non-violence in the real world and help those who suffer from it. More than a decade ago, I suggested a game concept where the players of these ‘fight games’ would have a chance to donate a tiny sum of money to any fund or movement that works to help victims of violence. I described some possibilities here, like for example, donating 1 cent every time the player’s role figure ‘gets killed’. There are many possibilities to make this happen in games and even potential profitable business models.

We know pretty well what is war, but the concept and phenomenon of peace is not self-evident at all, even a mystery. It is an extremely context dependent and relative phenomenon, even adaptive.  Well-organized armies and military systems have their own, more or less hidden ways to interpret it. They have their serious interests to think about and promote peace, some pathological and extreme exceptions excluded. In peace-keeping and humanitarian functions this is self-evident. When you talk to soldiers you can often observe expressions of their love for peace. Fight gamers do not typically show aggression in their talk.

Peace implant – a speculation

This is my speculation: I’m introducing the idea of delivering peace implants into the military complex. The logic behind this thinking is that organizations can and should be able to do something good even then when their main purpose is to destroy or harm something that they or the society considers necessary in order to achieve something valuable in the society. 

Finland for example, is known for its large-scale wood industry which actually means cutting down trees, while it is also one of the leading countries in implanting and growing new forests. We can see this as a special form of sustainable industry, while it also means that doing something good (growing forests) is becoming inherent in our wood industry. A similar, somewhat paradoxical thing is the idea of asking gamer to donate money for fighting violence.

By peace implant in the military, I mean any systematic, behavioral, social, economic, or technological, elements and practices that can be expected to promote positive or mixed peace in some form, small or large scale.  As an example, the military personnel bring their own family, cultural and religious values to the army and in this sense can serve as (mostly tacit) implants of peace, which then has various direct and indirect consequences to the way the military operates. The army itself contributes to this with its organization, planned education, social roles and pressures and with its value base. In extreme military organizations, peaceful ideas or views are suppressed and even punished. Whatever the nature of this process, the outcome is that there remain islands and implants of peace in the military system and we should understand their impact for peace as well as we understand their role in war.

Peace implants can live within the military only as long as they do not hinder the missions and functions of the system. From the outset this might appear paradoxical but it is a property of any social system that includes or allows diversity.  How do these implants contribute to peace is the question and here I consider some speculative, systematic possibilities to support and improve that kind of peace development – without being a threat to the core missions of the military. What does this mean?

Spending on war and peace

Global expenditure in the military complex will soon reach 3000 billion USd per year. The leading countries in this deadly (or defensive) game are Uniter States, China, Russia and India. About ten countries follow them with their close to 50 billion USd spending.

Economical investments in the activities promoting peace, dwarf in comparison as the estimated spending on peacebuilding and peacekeeping is about 0.4% of the military spending (International Peace Institute, IPI). As a society, we have adapted to the idea that peace is not worth heavy economic investments, the underlying assumption being that it is not a realistic or profitable alternative and that the military and wars will always come as first priorities.

Positive peace invites investments 

Traditionally, the measures of global peace have been based on counting the number of the dead in conflicts, number of conflicts, their casualties and the degree of militarization – and to use the inverse of this as an index of peace. This is weird indeed, and as an example, take a look at the Global Peace Index.  As late as in 1964 Johan Galtung introduced the concept of positive peace published in Galtung, J. (1964) An editorial, Journal of Peace Research, 1 (1),1-4.), and it took decades to start building positive peace metrics. It is fair to say that this development is still in its infancy.

Ever since my time at the Peace Innovation lab Stanford, in 2010 I have been inspired by the idea of positive peace and ways to measure and make it happen. It is a most human conceptual and methodological challenge. With the emerging power of data sciences, social media, technology, and the recent advances of AI, new possibilities emerge every decade and even faster. The founders of the Stanford lab, Mark Nelson and Margarita Quihuis continue working on these opportunities.

There is now a plethora of positive peace indices, which are based on measures of equality, governmental functioning, lack of corruption, human rights, solid business environment, and many others, see Vision of Humanity and the eight Pillars of Positive Peace and the Positive Peace Report. However, the report is an eye-opening example of the problems of measuring such complex phenomena since there are other factors and behaviors in the society that are not covered by these optimistic peace metrics – cultural, historical, technological, power and leadership factors especially. 

For example, the above PPR report was compiled before the Russian attack on Ukraine and it explains how the Russian positive peace index had improved by 6.1% between 2009-2022. The Russians attacked Ukraine in February 2022. Indeed, these indexes can be problematic, they live their own life and have questionable predictive value. Now it is, of course, possible to look at possible reasons for this total prediction failure. 

A general paradigmatic problem in modern sciences and scientific methodologies is that we have learned to measure negative, pathological, and deviant phenomena, which can be computationally differentiated from what we call ‘normal’ and theoretically expected (hypothesized) behaviors and phenomena. This can span a biased observational architecture that leaves or pushes critical variables into the shadows, which is indeed happening with positive peace factors.

We may even ask if there is any sense to deal with positive and negative peace separately and should we have a mixed peace metrics, a multi-dimensional concept,  that includes both and even other variables which have not typically been included in peace considerations. Such a system would be dynamic and adaptive. AI, for example, can become a peace-impacting variable that has now taken many of us by surprise. A friend and a colleague, Timo Honkela (1962-2020) was early to suggest the concept of peace machine in his book with that title in 2017 (in Finnish) and where he explained how AI could promote peace and mutual understanding through peace negotiations, for example. Now with the emergence of Large Language Models (LLM), this opportunity is easy to see. Curious enough, most public discussions worry about the possible disastrous future with AI and practically never consider the possibility of AI promoted peace.

The current ’struggle’ between peace activities and the military complex seems hopeless, at least when considered as a potential target for peace-related investments. Even David beat Goliath with suitable ‘military’ technology. The resources invested in peace building globally are not massive, but especially and because of that, there is a good reason to ask what would be the best possible ways to invest in peace in the military complex itself, so that it would have real impact and so that it did not prevent or harm its mission? Where and how to invest in this massive ecosystem? The precondition to this is that the military complex will remain and evolve and it will (with its partners and networks) even increase as a target of war and defense-related investments.

Some ESG inspiration for the military complex

Promotion of positive or mixed peace can be viewed as an analog to the environmental, social and governmental  (ESG) trend in investing. The crucial question in ESG investments is how to measure and estimate their impact. Some time ago I sat at an impressive business meeting where it became clear that a financial operator was ready to offer hundreds of millions of ESG investments immediately with a simple question and condition: how to reliably measure their impact and to guarantee that they generate an x% return?

Many of the ESG measures are relatively straightforward like environmental gas emissions, waste management, energy uses, use of natural resources,  and others. Other measures are more indirect and complex such as workforce conditions, safety, and leadership accountability. Typically, no direct military considerations are included in the ESG measures. At the moment financial institutions struggle with these measures and especially must seriously follow their economic impact and have ways to secure their profitability. Political pressures are now unavoidable.

Globally, the ESG investments are about 30 trillion USd, a huge potential indeed if properly directed. Many of the impacts are indirect in nature, like the outcomes of the included partner businesses, generation of new local and global interactions, subcontractor output, impact on public services, infra and new technologies. Promoting positive or mixed peace with investments must then be based on similar holistic and system analysis.

 Dr Tilman Bauer has studied the philosophical purpose of businesses and has made a stunning proposition: it is the core role of business to create positive impact and generate peace. He describes the nature of different peace forms: Nonwar-Weak Peace-Strong Peace – Holistic Peace, and suggests various ways to analyze and support peace in these different forms and contexts. In this approach, businesses as well as public services can be analyzed from the perspective of how they and their partners and subcontractor and client networks contribute to peaceful developments. Only few firms and other organizations have made a thorough analysis of their business processes and finances from this perspective although it is relevant or at least interesting for every firm on earth. 

A thought experiment on a peace implant in the military complex. 

What if, we could, with governmental arrangements, regulations, cultural practices, and by citizen activities find a way to systematically introduce peace-promoting practices – the implants – to the military complex? Furthermore, we should accomplish this without risking the core mission of the military and this should not compromise it. A holistic approach would include subcontractors, network activities, education, and research that directly support the promotion and understanding of positive or mixed peace. 

For example, subcontractors can have different background profiles in their peace impact, while they compete in producing capable equipment, services and components for the military. The natural priorities, from the buyer perspective are quality, costs and relevance. However, would it be possible to introduce to this list of priorities the contribution to peace by such partners/subcontractors – locally or globally – in the same way that ESG requirements are today applied? 

As an extreme example, Russia is using the missiles from North Korea. Most western nations would not prefer them even if their performance would be competitive with other providers. Of course, military subcontracting and acquisitions are not always so straightforward and rational, but would these practices change if peace considerations were included in military complexes? 

I will not try to explain here how this development could be initiated and I know that such developments and interests already do exist in some military organizations but they are not in public knowledge, perhaps for various political and value-related reasons. Humanitarian and peace-keeping activities have made this even a natural aspect of the military value development. No army is isolated from the changing world.

There is a well-known argument that the main purpose of the military is peace. However, the operative goal remains and it is to win the wars, defend a country and promote humanitarian and peace aid. In this world there is no compromising in this. However, the military complex itself is such a powerful structure that any peace promoting processes there could have a major, indirect impact, locally and globally. It may not directly contribute to what the armies do but it can have a long-term impact on how they operate and what are their indirect social and even global impacts.

I don’t know what people in the military think about this, but my guess is that they are already well aware of the phenomena of mixed peace.

This is speculation. I know, I know.

Breathing artificial human life into AI

Future scenarios of human AI are now an everyday topic in technology forecasts. General AI (GAI) is expected to surpass human intelligence within a few years and a plethora of speculations paint a promising or scary future of this. Then there is the discussion whether AI can become conscious, a topic that joins engineers and philosophers alike. The problem is hard enough since it is fair to say that nobody knows for sure what exactly is consciousness.

In the Science/News article (22 Aug 2023) the authors present the crucial question: “If AI becomes conscious, how will we know?” The story refers to an extensive review study where the authors analyze several existing AI systems and map them on the dominant theories of consciousness. They argue that so far, no current AI systems are conscious, but they believe that there is no obvious hindrance to achieving it in some future. They considered the following rather well-known theories of consciousness (a quotation): “… recurrent processing theory, global workspace theory, higher- order theories, predictive processing, and attention schema theory.” They have different ‘ingredients’, some of which I have listed here in a very general sense and have then added my own generalizations. These theories have an extensive scientific background and the article references can be found in the link above.

1. In addition to direct sensory stimulation there are feedforward and feed-back loops in the brains that underlie consciousness and that generate integrated perceptual representations.
2. Multiple neural networks working in parallel and in coordination so that conscious experiences are generated like instant memories. There is limited capacity workspace that present boundaries to the system performance.
3. Higher order representations and top-down processing where conscious experience results from the workings of the higher than sensory and motor cortical pathways and centers, e.g. prefrontal cortex. Metacognitive processes monitor relevance of these processes.
4. The brain can be seen as a prediction system that generates hypotheses about the world (and oneself) state and matches these against sensory-perceptual inputs, which in turn results in attentive control and guidance.
5. Attention can be considered as the essence of consciousness so that the subjective awareness as such is the ‘platform’ of consciousness.

What is a mystery to me is that he nature of the conscious experience itself and its exact psychological matter, substance and architecture are only superficially touched in these famous approaches.

Unavoidable progress

The progress of an ever more intelligent and even creative AI is inevitable. University of Montana studies have already shown the creative potential of AI as compared to humans. Hence, it is reasonable to consider how we will and should observe, measure and experience this AI development. Here I have suggested and commented shortly on some of these perspectives to the emergence of the ever more human-like AI:

Measuring the intelligence and creativity of AI with current methods.

I don’t think I’m arrogant when I claim that current cognitive-orientated human intelligence measures and scales have a serious historical flaw and cannot be applied to AI in any informative way. There are good grounds to assume that AI itself is changing the way we think about human intelligence. Of course, one could claim that it is indeed one (although an artificial) aspect of human intelligent behavior to pass intelligence tests and hence, a ‘human AI’ should be able to achieve the same level of performance. 

According to this behavioristic view we would then compare human and AI performances in current IQ tests. However, as psychologists know, an important diagnostic aspect of any intelligent test is what kind of personal algorithms the tested person uses and what kind of mistakes he or she makes. It is a matter of careful qualitative analysis of the test performance. Intelligence is not only about the performance outcome, but about the way the outcome is achieved. We face the same problem if we try to test monkeys with our human intelligence tests. When a monkey starts eating the test materials and throws them around, it tells nothing about the intelligence of that monkey-person at her home, in the jungle. Indeed, home is as important to AI as it is to monkeys, it is about the domain of intelligent behavior. The so-called hallucinations and grave mistakes in LLM performance are often reflections of this ‘home problem’.

So far, current and near-future, human-like AI systems and robots are not copies of humans since their underlying processes, materials and structures are totally different from ours. Hence the intelligence measure they might get from human intelligence tests cannot be compared to human performance in its essence. It is only a performance measure and the pathway to a solution can (in principle) be even destructive in nature and as such not very intelligent from a holistic perspective. There is no denying that we human suffer from the same problem, but in our culturally constrained ways.

When an AI outperforms humans in all cognitive tasks, we face the question that immediately leads to the question about consciousness: Now the AI can do everything we humans can do, but how is it still different from us? It becomes necessary to analyze the behavioral and processual acts of behavior that AI and we as humans show. As far as I know, there is no systematic framework that would make possible this comparison and hence, any considerations of AI consciousness escape this approach. The famous Turing test is indeed a pure behavioristic paradigm and tells nothing about the internal processes of the AI (and man).

By comparing human and AI intelligence performance we hit the wall of validity of measurement scales and have no chance to know if AI is conscious in the same or similar way as we humans are. Another approach is needed.

Copying the human intelligence and experience architecture for AI.

Having the artificial neural networks (NN) in LLM:s has lured many to think that we have come closer to modeling the functions of the human brain and perhaps even our psychological processes. There are indeed some features of NN:s that support this interpretation like the power of distributed processing, synaptic dynamics in learning, and tolerance for noise and minor damages. However, NN:s are still only primitive, more or less biologically inspired system models and there are no grounds to assume that they as such would be sufficient to produces something we could call human-like consciousness. They lack the diversity of real neurons, the hormonal interactions in the brain and body, the integration of sensory-motor input and experiences and many other significant aspects of human biology and development. It would be strange to argue – although it is often inbuilt in speculations – that pure neural networks are enough to generate consciousness and that any other components are not needed for this.

The next generation LLM:s will benefit from taking lessons from human psychology and especially cognitive psychology. On the other hand, human cognitive models are not very strong and they have problems in dealing with complex and natural situations and they have practically no theoretical coverage for human experiences. Hence, it is not self-evident which components and process models from human psychology should be included for making LLM models more human-like.

In his recent talk (thank you Ahti Ahde for reminding of this), Thomas Dietterich “What’s wrong with LLM:s and what should be built instead” suggested that LLMs can become more relevant in the human sense if their  architecture includes  a) Functional cognitive components, i.e. for planning, self-monitoring, meta cognition, orchestration, b) Knowledge and language base, i.e.  language understanding & generation, common sense knowledge and factual world knowledge c) Episodic memory, and d) Situation model. 

From the AI psychological perspective these new system components would make an AI that can take the first-person perspective in what it does (Episodic memory), future orientation (Planning), situational interpretation (Situation model), facts, world and self-understanding (Cognitive), and language (Understanding and generation).

Generating stories with a soul

Sora, the new AI system from OpenAI generates astounding video stories from text, which it does by having some of the ‘human’ elements mentioned by Thomas Dietterich although these elements have probably (as far as I know) not been explicitly inserted into Sora to make it more human. Sora’s focus is in generating meaningful and coherent video stories, which, of course, means that it has to generate stories for its human audiences – with human ways to understand and follow them.  Frogs would have problems with Sora. However, it has world understanding, it has good language understanding, and technically, it has foresight, it has primitive elements of planning, it uses meaningful chunks or patches as relevant units of events. This is no wonder since these are what make stories meaningful for us human observers and they do have some cognitive-psychological parallels.

Clearly, these additional functions would make current LLMs more human-like and could even alleviate some of the known LLM weaknesses. However, there is nothing about the human mental experience in these human-like ‘plug-ins’ for LLMs. This is quite striking since the essence of consciousness is the experience of it. Without that we don’t even have the problem of consciousness.

Ever since Herbert Simon, Allan Turing and other AI heroes there has been an overemphasis on rational and cognitive functions of the human mind and this has been true for most of the human brain studies as well. The classic studies by Tversky and Kahneman considered human emotions and biases as faults of the human capacities and especially decision making. The exact nature of the emotions that drove the test subjects to make the interpretations that then could lead to non-optimal solutions were of no interest. They were considered as weaknesses of the rational human and not of as the very human elements of consciousness.

Artificial psychopath

If and when these human components and functions are implemented in the future LLMs it becomes possible to generate natural-like AI behaviors that resemble human behaviors – from the outset, just like we now have in text form in ChatGPT and many other apps. However, the existence of an experience of consciousness in the AI cannot be known although the AI system can vividly describe what it has learnt about conscious experiences and it can even have compelling conversations about it with us. The situation can resemble a discussion with a manipulating psychopath, vividly describing his or her ‘emotions’. It is only talk and when the AI system has a theory of mind about us, it could in theory interact with us like any human being, psychopaths included. We still would face the enigma of human consciousness.

Forgetting the human psychology and letting any form of consciousness emerge?

We don’t know what the human consciousness is made of, we don’t know its substance and have problems in describing its experiential essence. The theories listed above are spiritually weak, they fail to deal with human experiences and they have no real theory of observation that is always involved in the experience of consciousness when consciousness is the object of external (indirect) observation and the object in internal observations as well. The observation problem is well known in quantum mechanics.

One possible solution out of this cul de sac is to let AI evolve with any possible means towards any such behaviors and other processes, which we assume to offer the strongest existence proof of human-like consciousness. For example, every human emotion, from anger to hilarious joy, is encapsulated by a conscious feeling and thought. The same is true for all cognitive actions we make. The number one task in leading AI towards various forms of human-like consciousness is to take lessons from a profound and detailed analysis of these psychological processes, components and the architecture activated during critical events involving human consciousness – but not to copy them. The AI should have means for consciousness generation just as it generates natural language now but it should be free to generate any forms of it.

Guiding AI towards human-like consciousness does not free us from the problem of how to know if and when an AI is conscious. Why don’t we take as a working hypothesis that the AI can be built to have an artificial form of consciousness and we can forget for some time the problem of whether it is similar to human consciousness at all. The aim would be to develop an artificial human consciousness. It is possible that we end up generating various different forms of it and on the way, we could perhaps learn and have new hypotheses about the nature of human consciousness as well. Hence the title “Breathing artificial human life into AI” and in doing this we need not take the place of the God who breathed real life into man: 

”The Lord God formed the man from the dust of the ground and breathed into his nostrils the breath of life, and the man became a living being.” (Genesis 2:7)