It’s been some time since I last shared an article on my blog, and today I want to discuss a topic that has significantly influenced my experiences with AI, specifically through the lens of Neuro-Linguistic Programming (NLP), not to be confused with Natural Language Processing—often referred to simply as NLP in technological contexts.

Understanding the Two NLP Concepts

For those who may not be familiar with these concepts, let’s explore them briefly.

Natural Language Processing (NLP) is a branch of artificial intelligence dedicated to enhancing the interaction between computers and humans through natural language. This field encompasses the development of algorithms and models that empower machines to understand, interpret, and generate human language with precision. Essential aspects of NLP include text analysis, speech recognition, machine translation, text generation, and sentiment analysis. Without NLP, modern chatbots would struggle to operate effectively. This technology enables AI systems to comprehend and respond to human speech, making it vital for applications such as virtual assistants, chatbots, and voice-controlled systems.

Neuro-Linguistic Programming (NLP) is a transformative psychological approach created in the 1970s by Richard Bandler and John Grinder. It delves into the intricate connections between neurological processes, language, and behavioral patterns. The fundamental idea behind NLP is that by reshaping our thinking (neuro), communication (linguistic), and behaviors (programming), individuals can significantly enhance their personal development and refine their communication skills. Key concepts include:

• Modeling: The practice of observing and emulating the successful behaviors and thought patterns of others.
• Anchoring: Establishing associations between specific stimuli and emotional states to positively influence feelings or actions.
• Reframing: Modifying the perception of an event or situation to change its meaning and emotional impact.
• Meta-Model: A series of questioning techniques designed to clarify and refine language patterns.

Although Neuro-Linguistic Programming (NLP) isn't directly linked to artificial intelligence, the principles of NLP can be observed in effective prompt engineering practices. I will delve deeper into this connection.

Understanding Our AI as a Conversational Partner

Neuro-Linguistic Programming (NLP) focuses on transforming perception, thought processes, and behaviors. Its main objective is not to change one's personality but to improve communication and foster more effective actions. This approach closely mirrors our use of "persona prompts" and "role-playing" techniques with AI.

Assigning a distinct role or persona to an AI system enables us to:

• Clearly define the context of our conversation.
• Achieve more accurate and pertinent answers.
• Exercise greater control over the nature and tone of the replies.

A prompt such as “You are an experienced science journalist, please explain…” effectively guides the AI in adopting the specific communication style and perspective you seek, resulting in responses that closely match user expectations. In the highly acclaimed book "Co-Intelligence" by Ethan Mollick, the author illustrates various examples of how to implement this prompting method to achieve diverse outcomes, including a range of writing styles tailored to the anchors provided in the LLM input.

Example Screenshots of cursordirectory (https://cursor.directory/) and of other AI Input tools, such as Voila, Triplo.

However, it’s important to note that while this method can enhance response quality and relevance, it does not guarantee accuracy or completeness. Users must remain responsible for critically evaluating and verifying the information received.

In summary, both NLPs and the use of role prompts in AI chats aim to optimize communication and interaction, albeit in very different ways and in completely different contexts.

The Conscious Behavior of LLM-Driven Chatbots

I acknowledge that this statement is a hypothesis. Many people express skepticism about attributing conscious behaviors to AI. We often describe outputs from chatbots as "hallucinations" rather than recognizing their creative potential. This skepticism likely arises from our unease in equating human consciousness with machine functions, as well as our deeply ingrained expectations of computers and software. We anticipate precision, reliability, and a clearly defined set of rules that we can comprehend and control. Something AI (luckily) will not provide us yet. Furthermore, biases stemming from the training data can produce unexpected outcomes, too.

Through NLP methods, we can consciously perceive different thought and behavior patterns and adjust them when necessary. For example, anchoring enables us to establish connections that influence our perceptions. Research suggests that our relationship with chatbots resembles techniques like anchoring bias, where initial information shapes subsequent decisions—similar to how LLMs respond to user input.

NLP primarily revolves around the principles of communication that closely align with the dynamics of chatbot interactions. Successful communication hinges on eliciting the desired responses from our audience. If those responses are absent, it indicates that our messages are not being fully comprehended. Rather than reacting negatively, it is more sensible to modify our communication strategy.

But is it a conscious behavior I am speaking of? Chatbots powered by large language models (LLMs) can essentially be seen as sophisticated autocomplete systems. These models excel at text prediction, demonstrating an impressive ability to anticipate what comes next in a conversation. But isn't this anticipation part of what we would associate with conscious behavior?

At least, if you ask perplexity.ai you will get some approval (see example). The conclusion on perplexity.ai: While the behavior of anticipating conversation or providing words involves both conscious and unconscious processes, it would be correct to describe it as predominantly conscious behavior. The person is actively engaged in the conversation, deliberately processing information, and intentionally contributing to the communication process. This aligns with our understanding of conscious behavior in social interactions and cognitive psychology.

Here's another paper that might be of your interest: Consciousness in Artificial Intelligence: Insights from the Science of Consciousness

The Impact of Chatbots on Our Communication Skills

The second hypothesis is intricate, shaped by my enduring relationship with NLP. My journey commenced while I served as a Creative Director at Publicis, where I facilitated workshops for client teams at Disney. During this time, my supervisor, an NLP trainer, enlightened me about these concepts as I geared up to showcase the Disney method.

I found the Disney method's role-switching aspect particularly beneficial in various situations. For example, preparing for a presentation can enhance both mindset and performance. Acknowledging how we communicate can significantly improve our interactions.

Utilizing chatbots compels us to define our roles and expectations in conversations more clearly. By incorporating NLP-inspired techniques or "persona inputs," we enhance our understanding of our audience and the appropriate language to use. Frequent interaction in these discussions can also encourage others to integrate NLP principles into their behavior.

Conclusion

In conclusion, understanding the link between Neuro-Linguistic Programming (NLP) and AI can greatly improve communication. By learning more about both areas, we can also enhance our interactions with AI systems.

Please note, here are my personal reflections. It’s important to acknowledge that this article carries a bias, as I aimed to find evidence supporting my hypothesis on the connection between NLP and the utilization of LLMs. Additionally, I am not an NLP expert, so my ability to discuss each technique in detail is limited.

Our interaction with AI is often itself related to the technique of "modelling of excellence", it's about analyzing successful behaviors and processes to create effective improvement and training models, we could learn from. Maybe questions like, "Who has faced a similar problem as mine and found a solution?" can lead to more productive discussions and results in dialogue with AI. Maybe at least, you get fun results 😉

In the past year, the impact of generative AI and other AI assistants on the realm of applied arts has been truly remarkable. From generating unique and captivating visual designs to assisting in the creation of innovative videos, AI has proven to be an invaluable tool in pushing the boundaries of artistic expression.

One of the most notable ways generative AI has influenced applied arts is in the realm of digital artworks. With its ability to generate infinite variations of designs and patterns, AI has empowered creators to explore new aesthetic realms and create mesmerizing, one-of-a-kind digital artworks. Creators are now able to harness the power of AI to bring their visions to life, unleashing a whole new level of creativity and imagination.

What does an AI test assistant tell you about the role of AI in applied arts?

If you ask an AI about the impact of AI assistants in the last year, it would sound like (generated with Linguix, a writing assistant I use while writing posts):

AI assistants have streamlined the creative process, saving artists and designers alike significant amounts of time and effort. These assistants are capable of performing tedious tasks, such as image sorting and research, allowing artists to focus more on the actual creation of their art. By taking care of the mundane and repetitive aspects, AI assistants have allowed artists to devote their energy to the aspects that truly matter - crafting innovative and thought-provoking artworks.

The impact of generative AI and other AI assistants on applied arts is not limited to digital mediums alone. In the field of sculpture, for example, AI has played a significant role in helping artists design and produce intricate and complex pieces. By utilizing algorithms and computational methods, AI has offered a fresh perspective on sculptural creation, pushing artists to explore novel forms and experiment with new techniques.

In conclusion, the influence of generative AI and other AI assistants on the realm of applied arts in the past year cannot be overstated. From enabling artists to delve into uncharted aesthetic territories in digital art to revolutionizing the creative process, these technologies have undeniably reshaped the landscape of artistic expression. As we move forward, we can only anticipate further advancements and a continued partnership between artists and AI, opening up even more possibilities for the world of applied arts.

Reading through the answer sounds very compelling, and truly I think this influences the direction of my post; there are many aspects and directions, I didn’t take into consideration before. It also felt a bit like… wow, did I miss so many things?

Let’s ask google’s bard, to provide me some examples to some of these mentioned improvements last year (ChatGPT’s memory ends in January 2022, so it wouldn’t help).

My first question for Bard:

Could you name some artists using AI to design sculptural creations?

As you might expect, I ended up having a smile on my face. At first, Refik Anadol was the top-ranked artist, followed by other artists who are, based on bard connected to the usage of AI.

The true reason for my smile was the double test and the same results; so I tested the question “Could you name artists using ChatGPT or text-to-image generators for their artwork?”.

The result was the same list, with the same errors; Bard didn’t distinguish between artists in general having some relation to computer art or playing with computer algorithms, such as Josef Albers, Maurizio Cattelan mainly known for sculptures and installations, and other artists using algorithms in general.

The most entertaining outcome from the list is the incredible hallucination of Markus Rehm, or at least I hope it is, as I must admit I am not familiar with this artist.

AI has been used in New Media art for quite some time.

Honestly, I don’t know which renowned artists really came up with new pieces of art within our last year that made use of ChatGPT or Text-to-image generators to build the foundation of the artwork itself.

In my understanding, the text being generated by AI shows a misconception about art. Especially new media art is and has always been about the edges of our technology surrounding us.

The popularity of new media art has been consistently rising since around 2014/2015, particularly in the areas of image generators and algorithmic experimentation. Google DeepDream was released in 2015 and many artists played with it. I recommend the read of Memo Akten “#DeepDream is blowing my mind” from 2015.

I know everybody hates #Deepdream, but I really love it. However, it’s not the aesthetics that I love, but the poetry of what’s happening inside the algorithm.

Memo Akten on DeepDream

There’s also a related video to his DeepDream experiment from 2015 on Vimeo.

Besides Akten, also Mario Klingemann started with some DeepDream experiments back in 2015, before he made more use of the emerging Generative adversarial networks (GAN) and created I guess his most famous piece of art in 2018, Memories of Passersby I. By the way, Mario is also one of my heroes as he accompanies my life for a long time; so he was a master in Adobe Flash and so many things that also played a role in my life. Get inspired by his journey and watch this video on YouTube.

GANs and DeepDream played a role for many artists. In 2016, we had great projects, such as “the nightmare Machine” by the MIT Media Lab, and also the beautiful project “The Next Rembrandt” got introduced in 2016. With Edmond de Belamy, a Generative Adversarial Network portrait painting from 2018, being sold for $432,500 at an auction, more and more artists realized that there is a lot of potential in that field.

For a long time GANs were the core to generate images; so before we had DALL-E (in version 2), Imagen, followed by Midjourney, Stable Diffusion and others, we relied on GAN-related image generators to generate pictures.

Back in 2022 I wrote a post on “Speaking computer language – welcome AI Art” and it gave a glimpse into the current status quo (before mass market tools dominated and changed the development). With the introduction of CLIP (Contrast Language-Image Pre-Training) and other similar technologies, we witnessed the emergence of image production tools. However, it is the arrival of the latest text-to-image generators that revolutionized the market, turning it into a mass market phenomenon.

This might sound like I’m joking, but I am not: I believe that one of the hardest problems in artificial intelligence, which still has not been solved satisfactorily, is humor. Once machines actually know how humor works and are not just involuntarily funny sometimes like they are now, they will have achieved true intelligence. And as a free bonus, with that understanding will automatically come the ability to know when to be serious.

Mario Klingemann on the question “What are the hardest problems at the intersection of technology and society that deserve more attention?” in MIT Technology Review

Regarding jokes and humor, at least Sam Altmann had a great reaction after the introduction of Elon Musks Grok.

It’s all about the data – New Media artists don’t care so much about ChatGPT and Text-To-Anything generators.

I already explained the role of data for the art being created by Refik Anadol in my post “Deconstructing AI Art masterpieces: Create Art like Refik Anadol”, in general AI is at its core a set of data technologies to analyze and transform an input.

In the past, artists had to classify the input images for AI models themselves. Now, there is a modern form of slavery where people are paid very low wages, around $2 per hour, to classify data. The problem is that the content they have to classify can be traumatic, see the article “OpenAI Used Kenyan Workers Making $2 an Hour to Filter Traumatic Content from ChatGPT” published in vice magazine.

Getting back to the initial listing, AI proposed artists making use of ChatGPT or text-to-image-generators and the top result of the list, Refik Anadol.

Most of his current artworks still rely on the GANs, so there are different ones, such as DCGAN, PGAN, and StyleGAN algorithms being trained on massive datasets.

Unsupervised — Machine Hallucinations — MoMA, recently being displayed in the MoMA is one of the works of Refik Anadol. The thought of playing with “hallucinations” beautiful reflects machine hallucinations in an expressive style.

Art that is influenced by Text-To-Image generators? Yes, dear Bots, here are some examples.

AI tools are great, no question, no doubt. Creating text, editing text, writing code, editing code, finding inspiration, testing ideas, … there are so many great ways to make use of AI. So let’s finally look at some examples that really relate the usage of mass market tools being available since around a year.

AI fashion examples

AI fashion in general has great examples of how to use text-to-image generators. Though many use the looks to get quick money again (this time in the shape of NFT), I wouldn’t blame them for doing so. Better than generating fake content on the web to gain traffic and generate money with it; see my main criticism in part 2 of the series “who owns the truth – the web”.

“Contemporary” AI Art

I don’t even know if there is something like a contemporary AI Art section, but if so, I would put some work from Violet Forest into this category. Her series “Just an AI girl” is just fun and is playing with contemporary clichés and one of the main issues mainstream text-to-image generators have had the last year; displaying strange hands.

Violet Forest (Violeta Lopéz)

Danielle King

Artificial Childhood Memories is an invitation to reflect on our own memories, the stories we choose to reshape, and the ever-evolving digital canvas upon which we sculpt and re-work the narratives that define us.

Description from Joyn.xyz on Danielle Kings’ work “Artificial Childhood Memories”

In "Artificial Childhood Memories," I delve into the delicate interplay between technology and nostalgia. This exhibition explores the ways in which AI can be harnessed to recreate and reimagine our fading memories, shedding light on the malleability of the stories we tell ourselves.

Danielle King describing her work “Artificial Childhood Memories” on her website.

Art is expressive and should also be related to our society and what moves our zeitgeist. From my understanding of the provided Answer, so far it seems that language models don’t understand differences within applied arts and they also see all "creators" as artists. The field of applied arts is far wider than only arts. It’s the world of different design disciplines and architecture as well.

Looking at my entry point, I had no chance to speak about applied arts in any design discipline so far… only because I was surprised how wrong the Chat output was.

The role of AI tools, such as ChatGPT or Text-to-image, video or anything generators, has on the design practice.

Simple answer: it’s a massive support for finding ideas and especially directly testing ideas. Being able to use language is now a competitive advantage and there’s no need to only rely on pure sketching skills.

But still, I don’t see finishing skills in AI. New Media Artists have proven over the last 10 years, that the manual selection process, or I would call it the classical role of an Art Director, is needed to point AI into the right direction.

Our dear AI buddies help to be far quicker – But still, they are supporting tools

For me being a solopreneur sometimes, sometimes part of a team or as a design-consultant on a project, I had different usages of AI. The best is sketching and selling ideas in a rush. So I had this little superhero being created for a client with some variations within half a day till I got the client's approval. The work to finally get the model created with 3D and some post-production/optimization took much longer.

The image was prepared with leonardo.ai and quickly modified in Photoshop to present to the client.

A main benefit for me is the brainstorming and ideation part. So every time I am stuck with something such as preparing a script for lectures, I could use AI as my muse to get distracted and refocus my thoughts.

The article “AI Reveals the Most Human Parts of Writing” from Wired Magazine illustrates this role of an AI-supported Writing buddy quite well. For some writers, the use of AI writing tools is like a new ghostwriter, but also the effect, that AI is giving you the direction of the text, might emerge. Same in my case for this post. The generated text of AI was a bubble and I wanted to elaborate on it to find out, that also the AI Chatbots had no clue about what they were talking in the end, AI dominated the direction of this text.

In terms of writing, AI assistants are a great aid, also in terms of usability. I assume it is much more pleasing to read an optimized text than my own, plain thoughts.

Text-to-video: Runway ML, Kaiber, Pika Labs and others

For me, there’s no use case yet to say, AI itself is the main tool yet to go somewhere. They massively help to get a feeling of potential looks; so as of November 2023, text-to-video made some impressive advancements, and it’s not Runway ML alone being the only interesting pick. But still it is one of the most impressive.

To get an idea of what Runway ML is capable of, it’s best to look for results on Twitter. @maxescu work is worth a look. He recently produced a video “the outworld” that could also be seen on YouTube, that is 100% AI created (and it is impressive). I would also recommend to filter Twitter results and look for #runwaymlgen2.

Ammaar Reshi is another, highly talented AI Movie Director with great demos.

Other contenders are also there, but all lack the quality of Runway ML at the moment. Maybe Kaiber has the most potential at least for some kind of looks. Other shots are from Pika Labs and from Genmo.

The beauty of these tools is that even when artists or people who try this tool explore it, it helps all creatives broaden their horizons. Creative work is heavily influenced by what we have seen.

UX Design and AI

To give a status quo about UX design and AI, I first start with a personal perspective based on my working process and experience during the last year.

On the visual part of the process, I tried text-to-image generators and their website output; but currently I wouldn’t suggest the use to text-to-image generators for more than a creative direction of a mockup and I wouldn’t always recommend to show it to clients.

The thing is, you easily get an interesting visual appearance. But in terms of structure, user guidance, or in short in terms of usability, my feeling was that the generated screens show one of the biggest pain points that brought me business in 2023.

The best recommendations I could give so far would be Galileo AI, Visily, WireGen for Figma, in terms of tools and 3M VAS to check, to buddy your design.

I had to conduct UX workshops in two agencies that have their strength in classical design or editorial design. Interestingly, both complained about their lack of usability when they incorporated it into their online designs or magazines. Based on my experience, this is a true point I see now in AI-generated websites. They look good based on an arrangement perspective, but are not usable at all.

Jakob Nielsen, part of the world renowned Nielsen Norman Group and maybe the godfather of usability, wrote an article on his own legendary blog about the “Unreliability of AI in Evaluating UX Screenshots”. For a small study, 12 e-commerce screenshots were given ChatGPT for evaluation.

The suggestions that were given were inconsistent and untrustworthy. Nielsen compares the results with an early intern and sees perspective in a Version ChatGPT6 maybe in some years to provide some valuable help in evaluation.

Ok, this article is from one of the strictest masters in usability, but Nielsen isn’t AI in general; he has a lot of great use cases, where AI assistants are helpful, such as ideation.

As another article on “AI-Powered Tools for UX Research: Issues and Limitations” from Norman Nielsen Group said, one of the core problems is that most available tools can’t process visual input yet. This is where ChatGPT's ability to vision comes into play and will show us where the future leads. Similar to the artists, I assume if Jakob Nielsen would “teach” his Bot to classify, it would be able to classify more Usability downsides easier.

ChatGPT Vision – pushing boundaries

I didn’t expect my post to get so long, so I try to get the curve slowly. For me the next major step now is to add more senses to AI. ChatGPT being able to hear, see and speak is massive.

Let’s look at some early visions:

Mckay Wrigley has some great early examples of what ChatGPT Vision is already capable of. Also Pietro Schirano has some interesting examples on the application of ChatGPT Vision.

Summary on the use of AI for applied arts

Yes, you are right. I didn’t look at all developments in applied arts; for me, it is mainly about a glimpse and my personal attitude towards the topic.

When it comes to applied arts, I could write endlessly and share more examples, in music for example, there’s the recent creation of Beatles song and a great short film explaining the process. Or we have “The Project Gutenberg Open Audiobook Collection”, a collection of free audiobooks created with neural text-to-speech technology.

The major issue with all this development is the speed of development and the wide range of tools. It is insane. On the business side, I stopped two clients not to push forward with an AI-supported website chat earlier that year; I told them, it’s not yet the time. 5 months later you found easy tools to train the chat with the website content easily on every corner… but still, I tell them to stay calm; wait for the European Act on the usage of A, and then we see which providers there are with a suitable solution at fair price (of course one could also use an own hosted solution now, but that’s not the point).

Staying calm in times of noise is my personal approach to current AI developments. All investments are pumped into AI, being a front-runner on a business side needs money and willingness to adapt quick; there’s no simple finish yet.

As you might have noticed on my blog, I wrote a lot more about text-to-image generation before we reached the state of mass market. And honestly, it would have been hard for me to write about it, as every few months some new exciting developments would have changed the direction of my thoughts. I always sing up for some early access stuff; so currently I am mostly waiting for access for everArt.ai.

I love AI for applied arts and I love the progress in technology we face. The speed is insane, and I assume the fear of missing out on some of the stuff is big. Especially if you are thinking about business ideas and applications for “quick” money. I also tried some stuff, trained my own models, built some Next.js apps and so on… but finally realized, that’s not my turf.

My role in this game is observing, applying and consulting. And honestly, this is quite relaxing. Why chase for quick money, if you could serve clients better and quicker? Providing happiness is a good sake and more fun to aim for.

As my counter program to the speed in AI developments I recently bought an iDraw H A3 to have some fun “drawing” things. Maybe I also share some post on that one some day.

My true recommendation on AI tools is, integrate them to your workflow, step by step and do yourself a favor to not try each and every tool. Relax, find your influencers and stick to the stuff that works best for you till you have a use case, some elaboration of another tool might be needed. AI Technology changes so rapidly, so don’t waste your time on trying tools.

OpenAI CEO Sam Altman said on social media that a new Moore's Law may appear soon, that is, the number of intelligence in the universe doubles every 18 months. If you imagine this number, you might understand how important it is to stay calm.

The only hope is that downsides of AI, such as bad datasets that infringe intellectual property law or unethical training data, get fixed. Also, the compensation of humans whose work has become training data needs improvement.

In arts and craft there's no truth; there is only exploration.

This is my second post in a row on reflections of our current status quo taking into consideration the developments of AI within the last year.

My first article addressed general issues concerning the training of large language models (LLMs). It delved into the complexities of language and data, and also considered the potential first major application for AI: AI Companions.

In this episode I will focus on the role of AI for the web, and how the web it is changing and what might be implications that follow.

To get this article going, I want to directly start with a video you need to watch, as it is brilliant; it’s from Heydon Pickering, called “What’s the deal with large language models”, being released some days ago (30th October, 2023).

So hopefully you had a chance to watch this brilliant video. If not, there’s one aspect starting at the end of the video, which is also one of my major concerns at the moment regarding AI and the World Wide Web. In the future, AI will increasingly train itself using data it generates.

Heydon describes it as:

The more we generate content and the more it is allowed to generate itself, the further the necrosis will spread. Rapidly, generative media will overwhelm the nominally “real” media from which it was originally trained, forcing it to reproduce with itself. This incestuous feedback loop will obliterate what remains of culture, leaving us gasping for brains, whatever the fuck they are.

Who can be trusted on the internet?

As you may suspect, the question of trust prompted me to share my thoughts on my blog. The root question re-emerges: "who owns the truth?" We all understand that no one owns the truth and that there are many credible sources available.

It has a tendency to just make stuff up out of thin air which is just really bad for Wikipedia — that’s just not OK. We’ve got to be really careful about that.

Jimmy Wales, Founder of Wikipedia on the standard on ChatGPT

Currently, we have our sources to trust. We might have our daily newspaper sources, we might trust Wikipedia and some other official sources as well. The question about the usage of AI in all those familiar publications is one that is already in discussion. There are guidelines for Wikipedia on the usage of AI and as Jimmy Wales explains in an Article (”Will Wikipedia be written by AI? Founder Jimmy Wales is thinking about it”) on the Standard, that the key point is checking if the provided information is true, so some examples were shown on how ChatGPT proposed a truth that is not existing.

In the article Jimmy Wales seemed to be very critical about AI, or especially about ChatGPT, as it is a serious contender to provide knowledge. Especially when more and more capabilities will be packed into AI companions. The truth is, that Wikipedia is also exploring the use of Machine Learning; so there’s ORES (The Objective Revision Evaluation Service), which helps human editors to predict the quality of edits or flags (and the decision-making to keep information or change it). But as you see, this is AI in the role of the wingman or buddy; not in the role of the “source”.

One of the issues with the existing ChatGPT is what they call in the field 'hallucinating' — I call it lying,

Jimmy Wales, Founder of Wikipedia on the standard

The issue with Chatbot hallucinations

The term Chatbot hallucination refers to instances where a chatbot generates incorrect or nonsensical responses, seemingly "imagining" things that aren't based on its training data or the input it has received. The advanced language model generates fictional responses, presenting them as factual information to the user.

Since a large language model is only as good as the data it has to work with, the model will occasionally return an answer that is patently false as the truth, which many have regarded as a serious flaw in AI models, especially if they are relied upon to automate tasks.

There's a good example of how search engines use this hallucinated information and present it as a fact in Wired magazine article “Chatbot Hallucinations Are Poisoning Web Search”.

So what happens if the data AI models are being trained on are messed up by AI itself, as we have seen in Heydon Pickering's video or my entrance issue?

Quick money and spam

Looking at emerging tools around AI, we see many tools being able to write the content for you. Even entire blog posts or websites can be generated with the help of AI tools. What is the specific objective? To attract more website traffic, impact search engine algorithms, and generate a higher number of visitors. That wouldn’t be too bad, if it wasn’t only for the purpose of getting quick money by advertising.

So nowadays you will find tutorials on how to generate faceless YouTube videos almost entirely done by AI, you will find many examples on how to generate books for Amazon KDP also mainly generated by AI and of course a lot of those automated tools for writing blog posts and influence search to get some money from display advertising.

In my point of view, most of this stuff is a new breed of spam. Though the content quality of my mentioned use cases is higher than original spam mails, the problem is much bigger and wider. As vise magazine shared in their article “AI Spam Is Already Flooding the Internet and It Has an Obvious Tell”, many fake user profiles in social media now use AI to write their posts.

“Spam, Spam, Spam, Spam… Lovely Spam! Wonderful Spam!”

The origin of the word spam is related to a Monthy Python sketch being broadcasted in 1970.

On a personal level I never had a big issue with spam, as long as it would be obvious it is spam. Spam is part of internet history. Gary Thuerk, a marketing manager at Digital Equipment Corporation, sent the first spam email to around 300 to 400 ARPANET users, promoting a mainframe computer on the 1st of May in 1978.

Based on the great website “internet artifacts”, Thuerk claimed he sold $13 to $14 million worth of mainframe computers through the campaign.

Today, we categorize spam messages into different types. The obvious one is the classic scam (like a relative in Africa promising an inheritance of 5 million dollars), and the rest we label as "spammy". We don't differentiate between mass mail and a spammy email. We determine which one belongs to which category.

The advantage of classical spam mailings: it’s obvious.

The same applies to good old display advertising: it’s obvious.

While dark parts of content marketing and other advertising sub-formats generated the hidden appearance to influence buying behavior, that is not obvious, most people learned to distinguish or look for little hints to identify if it’s “real” content or advertising.

Now we face contents being generated but not obviously marked as being generated. Truth is fading away. Being able to identify the truth is harder than ever.

And the consequences are not yet obvious. Will the “free” internet in terms of media consumption as we know it finally die? Or will the massive amount of content lead to a final breakdown of the web? Or will the internet in general not be trustworthy anymore?

Too many negative perspectives, as there are ways to gain trust. Gain trust by embracing a culture that shaped the Word Wide Web for long decades: blogging.

The Garden is the web as topology. The web as space. It’s the integrative web, the iterative web, the web as an arrangement and rearrangement of things to one another.

Mike Caulfiled in “The garden and the stream: a technopastoral”

Digital gardening, Indieweb and trust in people

Exploring the current web, one mustn’t only see the downsides there are with the harmful use of AI. Of course, it’s always us users and our use to facilitate or abandon a technology. Being used as a wingman, AI is excellent.

The issue with trust in our world made of quick money thinking is more difficult, and therefore I show some developments that put the trust on a different level. On the level of expertise, attitude of thought, all supported by a clear sender.

Digital gardens, Second brains or Zettelkasten

A digital garden is like a mix of a notebook and a blog. It's a place online where people share ideas that they want to grow over time. Some people may refer to the Digital Garden as a "second brain". It's a method for storing and managing knowledge in a maintainable, scalable, and searchable manner. So you could also call it a digital Zettelkasten.

In blogging, you often write a post and then forget about it. But here, we treat posts like plants. Some may not survive, but others will grow and provide ongoing knowledge for both us and the community.

To get an idea about digital gardening examples, you could explore some sites: Gwern’s site (best check the about section to get an understanding about what his site is about), the wiki knowledge of Nikita Voloboev or beautifully crafted example of Maggie Appleton. When you examine the examples, you will clearly observe that every garden is unique in terms of the content's richness and the quantity being shared.

Here you find a list with more examples on GitHub.

If you want to start with digital gardening, you may face many technical questions. So right now, I wouldn't suggest digital gardening (especially on the web) for those unfamiliar with how to connect tools using APIs or some essential coding skills.

With digital gardening, you’re talking to yourself. You focus on what you want to cultivate over time.

Tom Critchlow,, from MIT Technology Review “Digital gardens let you cultivate your own little bit of the internet”

To begin, you could try Obsidian Publish. Alternatively, you could use a local solution like Roam Research or Notion (search for 'Notion as Zettelkasten').

Currently, there’s no critical mass for digital gardens to go mainstream. But the thought behind it and the transparency of thought or thinking is a great source of trust as the gather process or the thinking process is transparent.

Working with a digital garden seems to be a prototype of how to learn online: one gets creative to connect thoughts, one could collaborate and build upon ideas of others, one gets critical while collecting and connecting the thoughts to build an own representation of the world and finally, especially, if you share your process on the web, it’s a medium of communication and expression of thought that could flourish your seeds.

My recommended reading to dive deeper into digital gardens is from Maggie Appleton: “A Brief History & Ethos of the Digital Garden”.

Blog about it – the IndieWeb, a people-focused alternative to the “corporate web” and the Fediverse

Like digital gardens, the IndieWeb movement helps you take back control of your content from social media. Their principles - your content belongs to you, you should be well-connected, and you should be in control - have inspired me since 2021 to start this blog.

My original blog post at that time was written in German and described the tricky journey of setting up the page.

In addition to certain formal methods used in IndieWeb publishing, like Microformats 2 and Webmention, it's also about how you share your content. This is often done through POSSE (Publish (on your) Own Site, Syndicate Everywhere). But fundamentally, IndieWeb is about a renaissance of blogging.

Everyone should have their own blog. Blogs allow you to express your thoughts and what moves you. And the best is that it’s obvious who is sending the information. There’s no anonymous AI source for the content. And if there’s some false information it’s quite likely that someone will tell you, you are wrong about this or that.

In a POSSE world, everybody owns a domain name, and everybody has a blog.

David Pierce on the Verge “The poster’s guide to the internet of the future”

POSSE is the best way in general to publish content to the web. It shouldn’t matter in which channel you post. You post something, and you are able to distribute the information elsewhere, but the original content belongs to you; not a platform.

The web itself could be seen as the next social network. With the help of tech like ActivityPub, any website can become a social media profile. This is the main concept of the Fediverse. The aim is to make an account on any platform in the Fediverse, and chat with users on all other platforms, without needing a new account.

In October 2023, ActivityPub got integrated into WordPress.com. So websites hosted by WordPress⁣⁣.⁣com can now be followed, reposted, liked, and commented upon by people using Mastodon and other social web apps in just 1 click. For everyone else, if you're using WordPress as a Content Management System, there's a plugin that's been available for a few years.

Unfortunately, similar to digital gardens, there are some barriers which make it harder to get going with IndieWeb, the Fediverse or some other related movement to turn the web back to what it meant to be. Again, the technical site isn’t ready for mainstream yet. With ActivityPub being integrated into WordPress.com one first step towards a broader popularity was done.

Summary on the use of AI for the web

I wouldn’t be able to share all thoughts in an article like this, so first I wanted to share some more recent developments that made my think about the topic itself.

My major concern is the future of the web itself. There’s so much potential, and it would be great if everyone would start to blog and share their thoughts in a free and open space. Not on a platform.

If it comes to AI it’s more or less the use of AI that is frightening. It’s the use of AI for quick money and so a lot of shit contents are being produced on the web. This is not a new development, as all assisting writing tools helped to generate contents faster during the last years and people already started to use it to gain more SEO momentum.

Search engines are hopefully dead anyway sooner or later. The lack of quality was an issue I wrote about last year in “Value, objectivity and the risk of being trapped by search”. The issue of being trapped gets bigger and bigger with bad contents being generated and taken as facts.

Being trained over and over on the same source doesn’t sound like a bright future. Currently, I mainly spoke about content in the shape of text material. But also on the image site (luckily) there are ways now to poison image data.

Nightshade is a method that can be used by content creators to defend their intellectual properties against illegitimate use. Not to mention that it could also be used on purpose to destroy something.

I guess most of us never thought about the manipulation or the poisoning of the data sets before. But especially for future AI companions that do harm, hidden in their agenda, it sounds like a nightmare, especially if we think about the issue of AI consciousness.

This is a much huger topic itself and I could recommend the reading of “An Introduction to the Problems of AI Consciousness” in the Gradient.

The web is at risk of losing trust, and we all need to protect this valuable resource and restore its integrity.

Maybe initiatives such as the “data provenance explorer” help to prevent misuse and to find more transparency in the use of datasets in AI.

I've never been a big fan of social media like Facebook due to my early experiences with online communities. I saw the negative side of human behavior, like deceit and manipulation, and it discouraged me from participating in social media.

What I care about is the quality of information on the web. It's concerning to see the amount of misleading content and poor-quality information. As a lecturer, I've noticed how unreliable information can mislead students. We need to restore trust in online content and the people who create it.

I know I need to contribute more to this effort. I don't post on my blog as often as I should, and I'm considering revamping my site. I'm thinking of sharing more of my notes and discoveries to contribute to the spread of accurate information.

Technology is not making our lives easier, but it accelerates it enormously. To not get trapped by speed, information needs to be trustworthy.

Truth itself is a concept. Science is searching for truth and truth is mostly a status quo. All information transmitted prioritizes and discards certain contexts, which is why it's helpful to constantly expose ourselves to new ideas.

In my next part I will look at the world of images and AI art more closely.

Further Resources

• Who owns the truth? – Part one: AI Companions
• Who owns the truth? – Part three: Applied Arts

The concept of "owning the truth" is complex and often subjective, as truth can be influenced by personal beliefs, societal norms, and cultural context. Some may argue that truth is a collective ownership, shared by all who seek knowledge and understanding. Others might contend that truth is individually owned, shaped by personal experiences and interpretations. Ultimately, the ownership of truth may lie somewhere in between these perspectives, bridging the gap between collective understanding and individual experience.

Speaking about Artificial Intelligence in the context of AI, it’s also quite tricky to speak about the truth and where our current development might lead us. Undoubtable AI is a great wingman, buddy or companion if you want to call it that way. My Notion AI assistant wrote the paragraph about the concept of “owning the truth” for me. For me, not being a native speaker, a handy function and a core concept everyone would agree makes our world better.

Companions or buddies? Language is related to many surrounding influences.

Speaking about AI companions or buddies starts with the separation of both, as it’s a matter of language. And there again we have the issue about our personal perspective to those two words; it’s a matter of personal taste, context and maybe social background.

AI Models are being trained on language and often enough they fail with their use of language. So many stereotypes around gender topics are still present. One more famous issue is that ChatGPT wasn’t able to distinguish between male and female nurses in the English language and thought nurses were female (of course depending on the way the question was asked).

Language itself isn’t the easiest topic, so while some languages have male and female articles, others don’t. Objects with female or male articles are more likely to be associated with different characteristics, based on the language (such as in German a bridge is female, while in roman languages a bridge is male); if you ask people how they like a certain bridge, it is most likely that in German the bridge will be more female, such as nice curves, … and in Roman languages the same bridge might have a solid basement and other attributes that are more related to male clichés.

Luckily this is another topic. But you see, language isn’t easy, so AI can’t do magic on its own.

And there’s also the issue of the data being trained on. There’s a mixed-media installation by Mimi Ọnụọha, being called the library of missing datasets from 2016, that brings the issue to the point.

The artwork describes itself as:

"Missing data sets" are the blank spots that exist in spaces that are otherwise data-saturated. Wherever large amounts of data are collected, there are often empty spaces where no data live. The word "missing" is inherently normative. It implies both a lack and an ought: something does not exist, but it should. That which should be somewhere is not in its expected place; an established system is disrupted by distinct absence. That which we ignore reveals more than what we give our attention to. It’s in these things that we find cultural and colloquial hints of what is deemed important. Spots that we've left blank reveal our hidden social biases and indifferences.

The question is. If AI is trained on data from our society and contents being gathered on the net. It is supposed to have missing data sets as well. Furthermore, it might rely on bad or prejudiced content. In the case of data about Africa, Nigeria, local, alternative movements help to overcome the data gap: archivi.ng

So if we look at AI Companions as our first element of AI Developments… would you like your children to talk to biased bots? Sure everyone would say no… but maybe only because I asked like this.

I don’t own any truth. I am a collector and searcher myself. I don’t have answers. My series is a collection of personal thoughts. Hopefully I get a better answer for myself after expressing my thoughts.

My first part. The world of AI companions.

Introducing the latest tamagotchis: character.ai and others

For me, the Tamagotchi was a symbol for the dot.com boom as I worked on a digital version of it back in 2001/2002, being called studigotchi (so it was meant for students).

The Tamagotchi gained immense popularity in Japan around 1997 and quickly captured the hearts of people worldwide in the subsequent years. Imagine the ingenious concept of having your very own pet, all neatly encapsulated within a compact egg-shaped device. There was a hunger meter and a lot of other ways to grab the users’ attention to care for that tiny little device.

At that time, the internet itself was on the rise and chats and other ways of companions were far, far away. The concepts of dark user patterns, such as the “hook canvas” from Nir Eyal were still not available, but still it was obvious that people like artificial companions. To not get bored, to not feel alone and to feel related to the little artificial device. Of course, it was “sooo cute”.

Now we are facing the push of AI technology and of course also AI Companions emerge. Now armed with the incredible abilities of AI and a deep understanding of user behavior manipulation, we are ready to unveil the next groundbreaking tool designed to generate extraordinary profits.

Et voilà there it is, the first Killer app for AI: AI companions.

While there are different concepts for virtual services to provide a companion, the basic idea is always the same: get a virtual friend.

I assume a great business model and blueprint for most celebrities could be the service of influencer Caryn Marjorie. You can have her as a virtual girlfriend through the caryn.ai service.

Character.ai is the most popular service for now (as far as I know from a western perspective) to find an online AI-buddy. Replika would be another one, also more famous.

character.ai is a platform that uses a neural language model to read and understand text and use this information to chat. You can create any character on this site. So you could create fictional characters, your celebrity, other real people, your dead beloved ones and so on.

Though the concept of a virtual companion is not new, character.ai made it available for the mass market. TechCrunch reported in September 2023, that AI app character.ai is catching up to ChatGPT in the US. Looking at the numbers, you see how impressive the stats are. The amount of users is gigantic and the potential use cases, that are directly covered on the homepage, seem very interesting.

So you could practice something, such as the preparation for an interview or a language. You could brainstorm an idea, plan a trip, write a story or other similar things you could also do with ChatGPT. The difference is… ChatGPT's interface is empty. Getting back to the dot-com world, there also used to be this emptiness when not knowing what to do on the internet.

So giving these little hints is quite useful. I experimented with the language feature shown in the attached screenshot. In this case I thought it sounded much more promising than the actual chatting is. It’s difficult to learn a language when you need to write it all the time… so you see in my responses, I didn’t write long sentences.

What you could also do is chatting to famous people, so I spoke to Albert Einstein, and it feels quite nice to speak to these virtual characters. If my son (6 years), being in love with ancient cultures such as Egypt, Greece or Roman culture, would love such a tool to chat to Ramesses or some other old characters from history.

Luckily there’s a little text marker saying that “everything characters say is made up”… what a pity; a combination of a learned model based on scientific know how about the character and some additional fiction would be much more appealing. But maybe this is just another business model in the context of AI Companions; especially if we take in the fact that ChatGPT's announcement of “ChatGPT can now see, hear, and speak” opens up new possibilities for great companions… so what about a great Bruce Lee Companion analyzing my body movements and giving feedback or some other master of a subject being able to assist me.

I guess even the researched wisdom applied to a character from history could lead to interesting applications, such as a real wingman in research. Why not brainstorm with a character being trained on all the know how about himself to test new hypothesis.

As for most successful concepts, AI Companions aren’t new. And as for more Mass Market successes there is way older groundwork being done in science before. One that comes to my mind is the AI-generated character for Learning and Wellbeing example of the MIT Media Lab.

On the one hand this example is still far more advanced than character.ai, but also shares positive outlooks, of where the journey might lead.

Are AI companions good or harmful?

Currently, we see how much interest AI companions gain in a short time. These virtual companions can be addictive and so character.ai is a rare exception in the spent time on the site. In the following chart you see the so-called Daily Active Users (DAU) to Monthly Active Users (MAU) ratio that measures how sticky the product is.

Despite other AI First Apps, character.ai seems to be very sticky, see the numbers from Sequoia. They share a quite interesting article from an investor's point of view, but the numbers being shown also point to one issue most AI topics have: their usability is… let’s say bad. Still, character.ai seems to create some OK stickiness.

Usability is a big issue in AI. Just imagine employing a prompt engineer to get a good prompt for image generation… not a sign of good usability of the available tools. The focus of the development is currently: quick business use cases and the speed at which money is thrown out of the window reminds me a lot to my time working at the dot-com economy bubble (though I started, when it blew).

The influence of AI Friendship Apps on users’ well-being and addiction

The headline is related to a study being called “One is the loneliest number… Two can be as bad as one. The influence of AI Friendship Apps on users' well-being and addiction”.

In their study they looked at the influence of friendship apps, in this case mainly replika and summarized the reoccurring themes.

• AI friends help people feel less alone as they do not feel judged by it.
• AI friends are always there for me and kind.
• AI friends tell users what they want to hear.

As previously mentioned, these applications are incredibly addictive. However, the advantages for therapeutic purposes are quite evident. Engaging in conversation about certain topics can be extremely beneficial and contribute to one's overall mental well-being.

Woebot – the mental health ally, is one of the apps that is more targeted as a medical support for users with depressions or anxiety and can deliver cognitive-behavioural therapy to reduce those.

So are there any final words to say? Positive or negative on AI?

No. At least I don’t want to judge AI in general. I am none of the people out there signing the techno-optimist manifesto. But I share some thoughts on the role of technology in our society in general.

While the technology itself is not my primary concern, I prioritize the responsible and thoughtful use of it. If I provide a service that allows individuals to improve their mental well-being, I must acknowledge the potential impact of negative experiences with someone they trust. Many people prioritize money and quick ideas instead of recognizing the true potential of technology.

AI Companion has tremendous potential that is worth delving into. However, further improvements are needed in refining the models themselves. Additionally, as previously mentioned, usability remains a pressing concern across all AI solutions.

And as many AI or future thinkers themselves don’t trust AI or want more momentum to secure it, it is worth to look at the issue from different angles.

Further Resources

• One is the loneliest number… Two can be as bad as one. The influence of AI Friendship Apps on users' well-being and addiction
• Generative AI’s Act Two
• ChatGPT can now see, hear, and speak
• Who owns the truth? – Part two: the web

If anyone asked me at the beginning of the year about AI, I would have said, it’s quite useful for text processing and predictive tasks. In the realm of artificial items' creation in graphics, art, or music, it felt like a playground mainly related to the arts.

GPT-3 as an accelerator – GPT-4, entry to the mass market?

The power of GPT-3 unfolded its power, starting in June 2020 and throughout the last years. We saw many upcoming apps using these skills to help you write, structure or code.

Due to their familiarity with the topic, most know or have tried some software for writing text, including jasper, rytr or others. As mentioned in my other article How AI Assistants can help you with text and art, they are useful, but help you more on the surface and for mass-tasks. Maybe GPT-4 will bring massive change and acceleration to the market. GPT-4 is expected to be released in 2023 and will push the market forward again.

Besides known applications, we also find novel approaches to things we are accustomed to, such as "googling", for example. Metaphor is another way to use a search engine, maybe not leading to issues google (and others) are already facing. See my criticism in the article "Value, objectivity and the risk of being trapped by search".

Getting back to GPT-4, most of us don’t know that Microsoft has exclusive rights on GPT-3. When GPT-4 comes out, it may give them an edge. They could add new features to Microsoft Word to make it a writing assistant.

The whole AI market helps them filter the best ideas from current apps and incorporate them into their own product. Having started my career in the first dot com bubble, I currently see similar dangers, as startups pop out from nowhere. Prototypes get released and make incredible money within a few days. From self-portrait apps such as Avatar AI and profilepicture.ai to logo makers and Figma plugins like Magician, there are plenty of great tools to help you create amazing designs.

Applying GPT or Stable Diffusion is now more accessible due to decreased entry barriers. On the other hand, we also see giants firing employees (and maybe shifting their business focus towards more integrated AI applications).

Do we face a paradigm change in applied creativity?

During the days of the Renaissance, humanity itself moved into the center of discovery and human creativity. Do we face a paradigm shift towards machines?

I am sure many people think like that, but not only for creative purposes. We ask for more automation in many aspects of our lives and replace repetitive tasks with automation. In my opinion, this is fine.

It's unsettling how rapidly technology is evolving. Everyone could start generating their own comics, storylines, always supported by AI.

On the one hand we have the text-to-image generators, on the other we see tools writing for oneself and adding other dimensions of ideas. For me, it feels nearly impossible to keep track of all the developments.

With the release of ChatGPT on the 30th of November, another fascinating tool arrived on the scene. If you ask ChatGPT what it is, it answers something like this: ChatGPT is an AI-based chatbot that allows users to interact and ask questions with a virtual assistant in real time. This chatbot is driven by an advanced machine learning algorithm that adapts to the user's conversational style and interests to provide a customized and enjoyable chat experience. This chatbot can be used to write poems, to do some programming for you or write songs. Explore the creativity of users on Twitter and you will see that there’s nearly no limit. The hype cycle of innovation and the peak of inflated expectations is already here.

Various business ideas around ChatGPT see the light of day each day. With the release of GPT4 we might reach another crazy dimension of text to anything tools. Bloomberg titles “ChatGPT Could Be AI’s iPhone Moment". I guess not yet, but we are close. My guess would be GPT4 would be this moment as ChatGPT is a kind of version GPT3.5 and already shows how impressive it is.

The development speed is incredible. Stable Diffusion is currently available in version 2.1, making progress on some details, and accelerating other tools such as the hyped lensa app (available for iOS as well).

In June I thought, Dall-E mini was wow, but since summer we have seen more advanced wows with midjourney, dalle-2 and Stable Diffusion.

As I mentioned in my article Speaking computer language – welcome AI Art, the text prompt gets more important. Expression of thoughts and ideas as a tool to facilitate creativity. In a current course at university called “Interaction in space”, we touch diverse forms of interactions and now face generative programming. During the course, I told the students that generative programming helps with your creative process. The use of generative programming could generate inspiration and provide you with results and directions you might not have considered. The same applies to text-to-anything converters. It’s a supporting tool in the exploration of ideas.

Looking at text-to-image tools, it is crazy how fast one could test visual ideas and modify them into another direction. With ChatGPT you could generate initial code and modify it with the ChatGPT assistant. Maybe learning programming could get easier for students.

The downside of open data

One of the biggest issues I see currently is the problem with scraping data from the web and using this data for training AI models. There's no control and being a creative I could collect great pieces from my favorite designers and feed my own models. You could scrape data from collections like FWA or CSSDesignAwards to train a Website Design model. Maybe such a solution is already available in the market.

My point is... is it fair? Using data someone else collected and categorized for your own models? As in the hyped times of bitcoin, ethical standards are not the focus of business ideas. It's only about making money. Many people who once mined with cryptocurrencies now lend their GPU power to machine learning. Local learning (I tried with a GeForce 3060) takes ages and the footprint of generating all those models is also huge.

I am concerned about the fact that anyone could work with someone else's data without asking for permission. A new Image format including scraper protection might help since everyone should be able to decide if someone else can access the data. Any blog or written word falls into the same category.

noai and noimageai as a solution against bots – as long as they respect them

After some research, I discovered that there's already a solution for the scraping part of it. Deviant Art introduced the noai and noimageai for all their users by default back in November 2022: UPDATE All Deviations Are Opted Out of AI Datasets.

The solution could also be implemented fairly easily on your website – so this is only a bot protection if the bot respects the rules:

Don't allow AI at all:
<meta name="robots" content="noai">

Don't allow AI to use images:
<meta name="robots" content="noimageai">

Make sure none is used
<meta name="robots" content="noai, noimageai">

Still the problem is the ethical agreement itself. The idea of a Hippocratic Oath for developers, ML applicants or anyone using AI is great. Scrapers/bots need to respect rules and those rules must be implemented directly. Similar to early stages of privacy policies, there's no opt-in by default. It's always an opt-out by default before anyone uses another's data.

As artists are mostly affected, it is great to support them by using the protected good of major corporations, see the discussion and examples in this post. Or the way ArtStation reacted, by removing its own creations.

For all creatives I could finally say: trust in ideas and learn to use generative AI.

Block ChatGPT from your website

In addition to the former options to prevent the usage of your images, since August 2023 there are ways to also indicate that ChatGPT should respect your content and not index it. To do so, put the following code into the robots.txt file. Opting out seems the new paradigm.

User-agent: GPTBot
Disallow: /

Living in Europe, I am quite sure, we will have new regulations in the near future; that on the one hand do good at saying no to auto-opt in, but on the other hand add too much overhead, which stops progress.

In the end, it will be interesting what happens if many AI-generated contents flood the internet and AI crawlers work with this mostly generative, bad content. In German there's an expression "the cat bites its tail", which means it's kind of a circulating problem not coming to an end (similar to the expression catch-22).

Related Contents

Microsoft and OpenAI Working on ChatGPT-Powered Bing in Challenge to Google

Updates from January 2023:

2023: The year of the conductor – great article on the current status quo on AI in terms of mainstream technology

A helpful explanation about some parts ChatGPT is struggling with and about how WolframAlpha might be helpful: Wolfram|Alpha as the Way to Bring Computational Knowledge Superpowers to ChatGPT

Great article on the consent issue from the creators of haveibeentrained.com: AI Art and the Problem of Consent.

Have I been trained website

The Great Generative AI Debate: To Use or Abuse?

All online Contents should be crawlable for AI

Updates from October 2023:

Data Poisoning generative Image Models

More Basics from the web

2022 is the year of text to image generators so far. With a crazy hunt for new tools starting in summer. Following some GANs of 2021, such as VQGAN and others, I mentioned in my article “Speaking computer language – welcome AI Art", the hunt for the next generation of text-to-image generation started in April when OpenAI introduced the magic of DALL·E 2.

During the year I shared articles around the topic of AI tools for image generation and if you had asked me in January, I wouldn't have expected such a development. I tested midjourney and DALL·E 2 and shared my thoughts on different ways to utilize AI tools. There's also Google launching more and more new tools on the market, one of the latest is DreamBooth, something that could be added to the workflows of the broadly available text-to-image tools, but that's another story.

While finishing my review on midjourney and DALL·E 2, I noticed the official launch of Stable Diffusion. Then I struggled, as Stable Diffusion is different from all the other tools before: it’s Open Source. Being Open Source, it is a creativity machine on its own, as it is possible to connect it to other tools as well. My core question was… where to start to share my thoughts? Explaining prompt guides as for midjourney or DALL·E 2?

In my following article I wanted to give a broad overview of the current status quo as of September 2022. Ways to use Stable Diffusion and share possibilities of open source software happening currently.

What a good artist understands is that nothing comes from nowhere. All creative work builds on what came before. Nothing is completely original.

from Steal Like an Artist: 10 Things Nobody Told You About Being Creative.

Stable Diffusion – what is it?

Stable Diffusion is a model that can create digital images from text or use text prompts to guide image-to-image translations. It will be released on August 22, 2022.Looking at the research paper you will notice that some names you might have come across in former publications on GANs will show up again, so all people being mentioned in the research paper of VQGAN are mentioned: Patrick Esser, Dominik Lorenz and Robin Rombach.

Understanding the heritage can help comprehend the rapid development of VQGAN in just one year. The progress has been extraordinary.

Currently, all main competitors (DALL·E 2, midjourney and Stable diffusion) rely on the principle of diffusion to create their images, so-called latent diffusion models. If you want to get a deeper understanding, I recommend a good written article from Louis Bouchard or watch his video:

Looking at results with VQGAN and Clip, see my older article and results from one year later, give you an idea about the speed of development. We face many competitors and an impressive research speed. I assume the next challenge is render speed while generating the images and optimizing the diffusion models. In the meantime, the world is allowed to unleash its full potential by connecting more libraries together to Stable Diffusion.

I really recommend trying some of the text-prompts yourself. Generating images on your computer can be difficult. The process involves using language prompts and the speed of image generation tells you how complex it is. On my MacBook (with m1) it’s running ok, but I am not sure on which PC builds you could run it.

Playing with the software, you'll observe cases the current implementations struggle with - one of the harder ones is stigmatizing the world around us. No wonder if the models are trained by common images of the web. The most common pictures on the web are pictures more beautiful than reality.

Watch "Stable Diffusion - What, Why, How?" by Edan Meyer, a one-hour instructional video on creating diffusion videos. It's presented greatly and a great resource to get started with Stable Diffusion.

If you want a shorter version of Stable Diffusion, look at the 2-minute paper.

Stable Diffusion – Quickstart via dreamstudio, collab, huggingface

If you had a look at midjourney or DALL·E 2 before, you experienced simple text-to-image prompts to create your artworks. Being Open Source opens a variety of inputs for Stable Diffusion. I try to give you an overview (with a raise in complexity and flexibility).

Dreamstudio.ai

Dreamstudio is the easy access to Stable Diffusion and the answer to popular midjourney or DALL·E 2. To get started, you need to login. Also, the prompt guide is only available when logged in.

The main difference to midjourney (using discord) and DALL·E 2 (basically relying on the text prompt) is the interface you get provided. Besides a text prompt you also get some rulers to modify the settings. Though the wording being used is still geeky, it is progress to the simple text inputs from the other main rivals. By the way, there also used to be an official discord bot, but they shut it down.

The payment model for dreamstudio feels more complicated than for midjourney or DALL·E 2. You also get free credits to get started but the amount of cost varies with your settings; here’s their explanation:

At default settings, it’s one credit per image. Depending on the image resolution & step count you choose, this can go as low as 0.2 credits per image or as high as 28.2 credits per image. Here is a detailed chart for explanation:

If you need some extra guidance on the photo editor, watch the video from their FAQs: Getting Started With DreamStudio Website Beta, Part Five: Inpainting/Outpainting.

In the FAQs you will find a lot of other helpful sources to get started.

All in all, it’s cool to have a maintained/managed tool showing the potential of Stable Diffusion and easily experiment. If you like what you see, you could continue to pay or switch to Open Source.

As you have seen, the tool also has some limitations at the moment: the size of the images is limited to 1024x1024 (midjourney allows 2048x2048). The size is directly connected to the render speed.

But no worries about the size: one of the biggest sins in photo editing is also a relic of the past. Upscaling pictures is now possible with the help of other AI image upscaling tools. There are so many tools nowadays, maybe you find some inspiration in my other article “How AI Assistants Can Help You with Text and Art”.

You can also look for GIT repositories to size the images. One I could recommend is this txt2imghd. There's also a version on replicate.com.

Stable Diffusion on Huggingface.co

You must have come across hugging face if you got into text to image before. I shared some stuff from hugging face before, such as Dalle-mini, a free Dalle alternative from earlier this year. I really recommend anyone being interested in the topic of AI and machine learning, to visit hugging face every now and then. Under “spaces” you will find apps you could directly test as web-apps. And you notice (sorted by likes) that there’s also an app for Stable Diffusion.

In fact, there are more apps using Stable Diffusion to transform images. The one on the left is the simple text to image prompt and the right one could be used for modifying pictures as well. Feel free to try one of the two here:

Here are some of the outputs being generated in the tool:

And the fun part: It’s for free! There’s no limitation in terms of credits and it’s a perfect playground to get used to how text-to-image prompts work in general. As mentioned before when explaining midjourney or Dalle-2, I recommend reading through some text prompts… or as the magic of open source already unfolds, use sites like lexica.art and look at the eye candy and copy prompts.

As an alternative to lexica.art, you could also try arthub.ai.

Now I almost forgot to quickly explain what diffuse the rest could do for you… it’s kind of the beautifier of your paintings. Draw like in MS Paint and add a text prompt, about what your line drawing might be. In my case presented with an owl (owl got very popular in all this process of text-to-image generation) – the transformation of kid paintings is also quite fun 😉

Stable Diffusion on Google Colab

Google Colab is another resource you need to know when it comes to working with Python in the context of machine learning. Google Colab makes data science, deep learning, neural networks, and machine learning accessible to individual researchers who cannot afford expensive computing infrastructure.

Edan Meyer explains how to use google colab with his own modified colab notebook in his great video “Stable Diffusion – What, Why, How?”. He is explaining the whole process to create images, similar images or even videos. A deeper understanding of how using the code works help to unleash the full creative potential.

In very simple words, you open a colab notebook and follow from top to bottom. On the left you find some little play icons to execute. After executing one after another, there might be some text prompts or variable to modify to lead you to your result.

Local Apps and other Web GUIs for Stable Diffusion

Since we have the first open-source release, it feels the market is exploding. I saw downloadable apps to come and go within 2 weeks, so it’s hard to give you a list of all the options there are. Also, there's no guarantee they are still working. Many people are fascinated by these tools. Some want to make money quickly, others want to share, and everyone wants to keep improving the technology.

So, I start with another Web GUI for Stable Diffusion called peacasso:

And here’s a desktop app to run Stable Diffusion for the mac (m1,m2) called DiffusionBee:

Feel free to share more cool apps, sources in the comments. As of Mid September 2022 there were so many tools, it is hard to keep an overview.

Replicate is another interesting cloud API I wanted to give a try. If you use the tools you will notice, rendering takes time. Using a cloud API for rendering can improve energy efficiency and reduce your carbon footprint, while also decreasing rendering times.

Explore the offerings of recplicate.com, there’s a clean Stable Diffusion install as well as other interesting tools to help you to optimize your images or get a desired effect.

Stable Diffusion – local installation on MAC M1

Finally, some thoughts and links on installing Stable Diffusion locally on your computer. I personally only installed it on an M1 Macbook. With the advantage of the RAM being able to be shared.

There's a tutorial on options to work with Stable Diffusion locally I discovered while finishing the article, but it seems to be the official one (I didn't test it though as I didn't want to break my running version); the tutorial is from AssemblyAI itself: "How to run stable diffusion locally to generate images".

When I tested my local install, I basically followed along with another introduction from replicate.com: "Run Stable Diffusion on your M1 Mac's GPU".

In addition to the instructions, some extra tips, as I also had some trouble while installing. First, check for the correct Python version. You need 3.10, which could be downloaded and installed from python.org.

If you have trouble to switch the python version, there could be many reasons (I couldn't probably name). I used a version control, as options there were ASDF and pyenv. I decided on pyenv and followed the instructions and had no further issues to switch the version.

Another hint I could share with you... watch your folders. Always execute the prompts from the correct folder and be prepared to wait a little for your prompts to be executed.

Here are some renderings on sculptures from my local tests:

Stable Diffusion – local installation on PC

It took me a little while to test the PC installation, but I guess this guide here is quite good and straightforward:

AUTOMATIC1111's SD-WebUI (Windows)

Following the guide you have similar steps as on mac, but it is simpler in the execution as there are more installers to run. Finally you have a webGUI to get going.

Please note, that you need a GPU with at least 6 GB of VRAM. 10 GB or more on a NVIDIA GPU with CUDA is highly recommended to allow larger image sizes. A business card, such as the RTX A series is preferred as the energy consumption compared to the value is much better. Maybe the upcoming NVIDIA generation will bring even more value.

On the tutorial site you also get some other guides (I didn't test):

Stable Diffusion – other installs

• Stable Diffusion in TensorFlow / Keras

Creativity unleashed with Stable Diffusion

To get an idea about the full potential I collected some random examples from the past month. There are so many exciting opportunities. People create comics, movies or just play around.

Kali Yuga models

Kali Yuga provides some cool custom-trained models for Stable Diffusion. Here are some shots taken from Twitter:

You might have guessed it, there are also notebooks on google colab around to work with them. An alternative would be jupyter notebooks, but to keep it simpler, we stay with google colab.

• Notebook for #textilediffusion.
• Notebook for #Pulp-Sci-Fi-Diffusion
• Notebook for #lithography-diffusion
• Notebook for #watercolor-diffusion
• Notebook for #pixel-art-diffusion

Those notebooks are quite easy to handle. I used the #textilediffusion, to generate this sample piece here:

Click the images to jump to the examples. Due to data policy, I don't embed external sources.

Stable Diffusion and Music:

Stable Diffusion and 3D

Here's the GitHub Source for the Stable Diffusion blender texture generator.

Another link, that is connected to Unreal and midjourney is the generation of textures.

Stable Diffusion and Image Tools

Stable Diffusion Photoshop Plugin

There are also options for text-to-image in Canva and GIMP already and Nightcafé and artbreeder seem to have changed their models towards stable diffusion as well (you might remember that earlier this year they both used some GAN+CLIP solution, as mentioned in my article).

Stable Diffusion to generate color palettes from text descriptions

Other sources to mix with Stable Diffusion or to watch in general:

• Get3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images
• OpenAI Whisper
• Google DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (GitHub implementation for Stable Diffusion and Google DreamBooth and another Stable Diffusion Implementation with DreamBooth)

Final thoughts on Stable Diffusion AI – image jockeys on random turntables.

I love it. Going for the open-source path is so great to unleash new instruments of inspiration and with lucky punches, even to create art. Art is expressive and the artworks being created might get expressive. For me it's an enhancement of our daily practices being creative. I also don't think anyone must be afraid of losing jobs. For precise tasks it is still tricky to work with the tools.

The models are only as good as the inputs; handling the inputs is not the easiest practice and as generating the images takes some time… in terms of client work, the way using any generator might not be the fastest option to choose. As always, I assume clients want to negotiate cost as they think everything is going easier now. But currently the image generation at a professional level is still not precise enough. But the tools are already there to help. Google Dreambooth for example could help a lot to bring consistency into image series.

Looking at the development speed I assume the topic about precision will be a matter of the next years, as well as the topic about speed and custom training sets. I am also worried a bit about the footprint being generated from all the image generations (including all the learning upfront). In hugging face there's a note on the Stable Diffusion model card itself: 11250 kg CO2 eq for the learning (calculated with the machine learning emissions calculator).

The development in text-to-image is far from its boarders yet. But it's so impressive to see the speed of the community to adopt these technologies.

Currently the tools help to discover creativity to finally create a mix of outputs we have seen before. In the future I am sure we can use the techniques to do whole mood boards for clients and in some cases even final graphics or artworks for them.

The only art I’ll ever study is stuff that I can steal from.

David Bowie

Like the evolution of web bandwidth, I think we need some more speed in terms of image generation to open new options and create beautiful human-centered-artworks as one day maybe images could be generated in real time based on likings of visitors. I also assume, at the moment we reached a certain grade of automated artworks, more and more people will enjoy discovering real artworks with their hands again and value of the real original artworks will raise.

In general we shouldn't trust media without a critical perspective, especially when we see images. History showed us that people will abuse technology. So I hope for people to find value in real in-person events after Covid pandemic again. Digital could and will be a lot of fake.

The use of some kind of style transfers on Smartphones is very likely by next year. Dynamic Video generation is also on its way (see make-a-video from meta ai), so we have text-to-video generation and music, VR worlds and games are also around the corner, as Midjourney founder David Holz said. Due to open-source development, I won't say it takes 10 years. And also Google Colab and other services to run the software will change their business models to profit from this boom.

Advertising or Branding will also be affected. It will be an exciting journey in the next two or three years. Osmosis pretends to have some of the features that could be interesting for an early stage. Just imagine adding a target group to the prompt, an industry or a segment. For Learning there are many galleries out there, such as dribble, FWA and many more 🙂 it will be fun to see where this journey goes! (Saying this not being in an international network agency anymore).

As mentioned many times, the development speed is incredible. The current hot tool is google dreambooth (connected to Stable diffusion). Every month there might be another unexpected cool tool to pop up. Maybe even photoshop integrated the tools for their next release.

For me, design turned more into managing design over the years but still design is functional, and art is expressive. Currently it feels easier to create automated expressiveness in fantasy worlds, but nonetheless I also use AI tools during the creative process. The results of machine learning give you a condensed view of common thoughts. So, I often used the prompts to check for color ranges of certain clients, compared it, what google says and easily created something one could call the "wisdom of the net" result.

Our traces on the net and our databases of artists created the look we could mix now. We are the image jockeys of generations that are already there. Unfortunately there's still a lot of randomness in the result to utilize it more professionally.

So long, enjoy and go out and play to discover the fun playing with those great tools!

Recommended readings

1. Understand latent diffusion models
2. Get a full introduction to Stable Diffusion

Maybe you remember my article speaking computer language – welcome AI art, speaking about the new role of text prompts and different image AI assistants. I focused on a general overview on GANs, DALL-E and DALL-E Mini, as an access to DALL-E 2 wasn’t available yet for me. Also, Google Imagen being mentioned in the article has no free option yet… but with only one and a half month passing since my last article, Google also introduced another tool, called Parti, a tool being said to produce the best results of any AI art generators so far and Stable Diffusion is also there to launch its service soon.

In this article I want to share my first hands on experience with Midjourney and DALL-E 2 with you. With midjourney moving to open beta and starting their v3 image generation algorithm on the 25th of July, another player pushed to the market… and I guess one or two days later, finally my invite for DALL-E 2 arrived.

At that day I was happy at first, but when I noticed another day later, that another E-Mail I used for signing for DALL-E 2 only a few months ago, despite the first one, being used for application far more than half a year ago, I was a bit disappointed. I thought it’s all connected to their new pricing model and the beta start of midjourney and I got disappointed a bit. The exclusiveness was gone and “mass-market” arrived.

In general the market on AI Art bots seems to push a lot. So for example DALL-E mini, the free tool I mentioned in my other article, is now called craiyon. There are also other tools Worth being mentioned in the context of AI art tools, such as Latitude Voyage (of course in beta) or Apples GAUDI, a prompt tool for indoor 3D scenes.

Luckily I heard the U.S. Copyright office said A.I. Art can't be copyrighted, so I hope no war will heat up this exiting market in the near future. So far I only know that there are patents on algorithms being used. Hopefully the market will show even more diversity in the future.

Midjourney beta – first impressions

Let us first look at our new player. Midjourney. The first impression of midjourney is being geekier and less serious than DALL-E 2. It’s a more playful entrance to a world everyone could test right away. After joining the beta, you will be redirected to Discord, no waiting list, no exclusive clubbing.

Before we investigate discord, a quick explanation of who Midjourney is. Midjourney is an independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.

IN 10 YEARS, YOU’LL BE ABLE TO BUY AN XBOX WITH A GIANT AI PROCESSOR, AND ALL THE GAMES ARE DREAMS.

David Holz in an Interview with the verge

David Holz, the founder of midjourney, did some other geeky, funny things in the past. He used to be co-founder of leap motion, a great exploration of the power of gestures. Midjourney is a small company right now with around 10 people working there.

Exploring the background of midjourney and all their way, how they speak to their audience, in other words their tonality feels great to me. It has the explorative spirit and playfulness I always try to teach students to get started in digital.

Signing up for a free account, you get 25 credits, these can be used to generate images within the discord channel. For anyone being interested, it is even possible to run midjourney on ones own discord server.

After that, you’ll have to pay — either $10 or $30 a month. The pricing is connected to the number of images you want and the privacy you expect. And if you are a huge company with more than 1 billion revenue a year, you get the corporate membership for 600 US dollar a year.

So how does it work?

If you never used discord before, see it as a chat tool, kind of a slack for the masses, with a non-business focus.

So, in the channel you will find all the information there are in general, on announcements, updates and so on. If feels like you as beta tester are part of the team, and even the boss is there, and you could influence the development process. After all, that’s what a beta program is about.

In terms of my professional perspective being more experienced with product and user testing, this is a great way for that kind of product, as the product itself is being used in the same environment.

So here are the steps to create images:

1. Look for a newbies Channel and enter it (in the picture you see newbies-29, newbies-59 and newbies-89)
2. If you have a picture in mind, about what you want to see, start typing “/imagine”, followed by whatever you want to see.
3. Within 60 seconds you should see 4 images.
4. With the 4 images being generated, you could upscale the image or create variations. Therefore, there are U1 to U4 (für upscaling) and V1 to V4 (for version) below the images being generated. Upscaling means that the images scale up to around 1664x1664 Pixels
5. Save the result, click on the generated image (to enlarge) and with the right mouse button being pressed, select save image to your local computer.
6. Finally, you could also ask the bot to send the results to your Direct messages, by reacting with the envelope emoji.

This is enough to create visual beauty.

If you need inspiration, look in midjourney ressource list, there are great links:

The artist visual style encyclopedia – if the style is part of your text prompt it will take effect – such as cyberpunk city, by Franz Marc:

There’s also a prompt builder to guide your inputs and see which variables are available:

And a collection on  “prompt styles and weights”.

All in all, the learning curve, if you are used to discord and used a text prompt before is very flat. Only if you are new to discord and text prompts, you need more orientation.

After all I could say, I loved it. I love the playfulness of the whole app, the results and all the “social” aspects, seeing what all others are building, getting notifications about own results and the whole way it is being setup. It is great fun to try and the documentation is very helpful.

DALL-E 2 beta – first impressions

With the magic of the new and little frustration about the almost open invites, I waited to test my DALL-E 2 invites after I tested midjourney.

As mentioned, entering DALL-E app is a completely different experience. It is you and the text prompt, not the hectic news stream and the dynamics you experience in midjourney. Based on the interface/overall app decision, one could say, that DALL-E 2 wants you to reflect about what you type, while midjourney is a competing social feed, about who creates the coolest results.

The cost system at DALL-E 2 is based on credits. You get 50 credits for the first month for free and 15 credits every month after. With only 15$ you could buy additional credit packs worth 115 credits, which equals around 460 images. OpenAI allows commercial use of DALL-E images. Compared to the image stock market, the costs of the images will be fairly low, but one must be award that you have to get used to the text prompts and the corresponding structure of the input.

That being said let’s try to get a result for our cyberpunk city, by Franz Marc.

In the case of DALL-E, I couldn’t find a quick hands on, official documentation; so, I tried the sentence as following: A cyberpunk city in the style of Franz Marc, as the example being shown by DALL-E on their homepage also used the "in the style of" syntax.

My result on that one was quite… disappointing. I don’t know why DALL-E missed the style of Franz Marc that much:

To investigate further, I needed a prompt guide for DALL-E 2. Googling for some help, you find

1. The DALL-E 2 prompt book – my recommended choice
2. OpenAI DALL-E 2 prompt guide

Going through the prompt options, you notice, that there are more parameters than on midjourney. Lightning, Camera position/angles, genres, usage contexts and many more.

So, I gave it another try, changing the prompt a bit leading to a much better result in terms of the style:

Unfortunately, my tries with real artists didn’t work out so good with DALL-E 2 at the start, so I had another try…

My result from midjourney:

And the same prompt from DALL-E 2:

Getting used to the prompt dialogue on DALL-E increases the results a lot:

Using the right prompts is the essential part to get the best results. If your prompt is done right, it seems DALL-E is more precise, though midjourney is so much fun to use.

For DALL-E there's even a paid service being launched called promptbase to help you with using the right prompts:

If you want to discover more, I recommend searching the web yourself. On twitter I found some comparisons, having a similar impression I do:

Resume

Is it possible to do a final resume on the tools? I guess no. On the one hand I am not experienced enough with both tools, on the other hand I think both tools are great and give us a glimpse into the future. The documentation on midjourney and the entry barrier is lower, so to get started, I would recommend midjourney in any case. The more you are aiming for precise results (as an alternative to stock images), the more it feels DALL-E 2 is the way to go.

Will be exiting what the next two month offer in terms of AI generation tools. 2022 is very exiting so far in terms of AI art generators and it will be exiting to see what's next.

What's next? Update mid August – dreamstudio arrived

As mentioned, the speed new, similar offerings are popping up on the market is impressive. It seems, AI in combination with images is going mainstream. Another tool to check out is dreamstudio. Access is quite easy as well and the results are also impressive. Also the editor looks promising, see my screenshot below. After my family holiday I might try a daily use check of all those great tools.

Recommended readings

1. The AI that creates any picture you want, explained – The process behind DALL-E 2 (and others) explained
2. What AI means for human artists – great collection about views from the artists/designer perspective
3. Making AI Art with Midjourney – great article with examples and explanations
4. The Creativity of Text-based Generative Art – paper on text-based generative art
5. The trouble with DALL-E – Five thoughts on the use of artificial intelligence to accelerate the digital art market
6. My follow up article on Stable Diffusion: Stable Diffusion celebrates new forms of creativity

The world of art is a diverse and beautiful place. The more we explore new forms of art, the more we are introduced to a new way of looking at the world. Art is one of the oldest forms of human expression and creativity and one of the newest forms of art is generative art or AI art or NFT art we could see at a large, popular scale in the shape of NFT galleries. AI art is an art form that uses artificial intelligence and robotics to create art. This form of art is exciting because it is a new and fresh way of seeing the world.

As I am fascinated by AI artists such as Refik Anadol, Mario Klingemann, Sougwen Chung, Memo Akten and many others, I wanted to share some ways to get started with Art in the context of AI. As Art using artificial intelligence is more complex and involves GANs, Machine Learning and other more advanced concepts, my focus on this article is the entry to AI Arts in general. In my point of view generative Art or Design is the best way to start in this area.

I also shared some more articles on the topic.

• If you want to know more about GANs, DALL-E, CLIP and other approaches of deep learning, follow this link.
• If you want to know more about my fascination for Refik Anadol, or if you don't know his work, follow this link.

I love generative worlds surrounding us. While teaching, I like to share some basic concepts about generative design, in terms of randomness or the magic of patterns. It’s so great to influence the idea generation process with the help of the computers. Something I also recommended in the use of AI tools in general. Besides giving a recommendation for the beautiful designed book Generative Coding on the web by Benedikt Gross et all, I want to share some ways and tools to get started in your own creative process.

For this article, I won’t discuss differences in AI art or generative Art as being generative is an underlying pattern for both. If we speak about generative, we speak about autonomous processes that are not under direct control of humans. To get started with AI in art or generative art, you need a basic understanding of accepting your role in the process.

You are not the worker to paint all the time, you are a conductor in paving the way for a desired outcome. So finally, everyone is a director to choose from directions computers generate for them. If you finally reached some degree of mastery, the process of creation will be more straightforward, but in the beginning, it is very explorative.

I also don’t separate between generative design and generative art. Art is expressive, as John Maeda once explained in his Bermuda quadrilateral, whether designs task is to communicate. (There are newer models on this one, too, therefore I recommend looking at Neri Oxmans Krebs Cycle of Creativity).

Generative art saw its first influences at the end of the 19^th century. Playing with geometry or vantage points and the rise of technology and machines paved the way for generative art. Besides visual art, there are many other ways to explore the generative power as well.

Ai art and Literature

Philip M. Parker is maybe one of the most extreme examples on generative literature or AI in literature. He “wrote” or generated more than 1 million books already. He developed some algorithms capable of automatically generating textbooks, poems, crossword puzzles or other books, mostly targeted to the so called longtail, you might have heard in the context of SEO before. So he is targeting specific, niche markets. He’s maybe my proof point of my fear of AI text writing tools, as a lot of crap could be produced and Search engines fail to display the best quality results, see my issue with the myth of the goldfish attention span in my other article.

Ai art and Music

The historic roots in generative music are even older as for generative visual arts. At the end of the 18th century, the “Musikalisches Würfelspiel“ (musical dice game) became a popular way of passing the time. Known names, such as Wolfgang Amadeus Mozart played the game being introduced by Johann Philipp Kirnberger. Back in our time, the most known figure on generative music is Brian Eno. And there are a lot of beautiful tutorials on the web on how to get started with generative music (I have those two still on my list to review):

And here are some generative music tools to explore:

• Alda - a text-based programming language for music composition.
• SuperCollider - a plattform for audio synthesis and algorithmic composition.
• Csound - A sound and music computing system.
• Sonic Pi - The live coding music synth for everyone.

So far, I am not experienced with any of the mentioned, so I can’t recommend any of the mentioned tools, but based on my research all of them a worth having a look at.

Architecture

In the case of architecture, I would prefer to speak about generative design in architecture. In terms of the production process, you could imagine that 3D printing in every scale will lead to new results. Like 3D printing these bots could also be used to brick or other construction related items. To get an idea about the status of technology, visit Autodesk.

For the design process in architecture, we see different degrees of generativeness, we have parametricism and generative design, as described earlier in this article. The most famous designer to apply generative principles is Zaha Hadid Architects.

To find out more, here’s an article on how architecture firms are using generative design.

Look at Eric Mesplé using ferrofluid to create art… so doesn’t it also feel generated? Maybe there’s a generative part to it, but not from computers, but from nature. And sometimes it's good not to fit in any category.

Programming languages – Javascript, Processing

The “classical” way nowadays to get started with generative Art is learning Processing. A programming language specifically designed for creatives. If you investigate the history of the programming language, you will find know names, such as Casey Reas and Ben Fry. They wanted to create a language that makes it easier for creatives to explore programming as a tool for making art.

Processing is available since 2001 and helped many creatives in the world to unfold their creativity by getting easier access to a programming language. Under the processing foundation, there’s also another library available, that got introduced in 2013: p5.js.

P5.js is a library for the programming language Javascript. In P5.js and processing we call a program a sketch. Like our sketching skills on paper, we draw programs with simplified lines of code. I used p5.js to create most of my post pictures and on the homepage and on my writing overview, I have scripts to generate the visuals. Especially on the homepage, each refresh of the page will give you a unique result.

A few lines of javscript code in p5.js could generate the generative look of the two images – the original visual study was done in 1974 by French media artist Vera Molnar.

function setup() 
{
createCanvas(500, 500);
noFill();
rectMode(CENTER);
noLoop();
}
function draw() {
background(220);
for (var y = 50; y < 500; y = y + 100) 
{
for (var x = 50; x < 500; x = x + 100) 
{
var r = 2;
for (var w = 100; w > 0; w = w - r) {
rect(x, y, w, w);
r = random(2, 20);
}
}
}
}

Is any of the two mentioned language difficult to learn? If you read some articles from me, you know I am not a person about absolute feedbacks, and again it is a “depends”. I have some experience teaching generative programming skills for undergraduate designers – the ones being more playful in their way of approaching new items succeed better than others who take it too serious at the start. While the ones being structured, wanting to dig to the core, are the ones who will stay with it on the long run (if they also discover the fun, playfulness part of it).

Even fewer lines of code are needed to generate visual results like the following:

function setup() {
createCanvas(450, 400)
background(255)
}

function draw() {
stroke(0)
line(200, 200, mouseX, mouseY)
ellipse(200,200,mouseY)
}

Programming in general is learning a language, a syntax, some grammar… and I wouldn’t recommend learning programming like a language. Learn it as a process of discovery, of fun playing around. If you got your first confidence, aim for a low-fidelity project you want to realize, don’t aim for big targets or you will get lost directly.

I think p5.js or processing are both great ways to get going. The easy part about both is that there’s an online editor you can use and open sketches from others, right from the start. So looking into my sketch library, you will also find many “copies” of others or modified version from other people.

To get started, I recommend the book Generative Design: Visualize, Program, and Create with JavaScript in p5.js or all the code samples of the book to get started from their website.

3D engines

Mostly I grab students’ attention, when I tell them that everywhere who uses randomness produces generative work. As per definition, as soon as some of our human control gets handed over to the computer, it is generative. And this generativeness is present in almost all the software we have today. You could even use randomness in photoshop.

Speaking about 3D and real-time engines, we speak about the big players out there, Cinema4D, 3DS Max, Maya, Lightwave and Blender, the main known 3D graphics Engines on the market. All of those enable you to be generative and you will find a lot of tutorials there. Those are the main programs being used to create the visual 3D look of designed elements around us.

Also, for specific applications, such as architecture, you will find applications of generative design, so for example the combination of rhino 3D and grasshopper also allows generativeness in architecture.

Overall, there are far more resources on blender specific tutorials. Such as two great articles on medium I could recommend:

• Creative Coding in Blender 2.92: A Primer
• Blender 2.8 Grease Pencil Scripting and Generative Art

Besides the mentioned 3D graphic programs, there are also Real-time engines. Unity and Unreal are the ones, that are quite popular on a wider scale.

There are also tutorials for all of those producing real-time images; here’s one on unity – how Carl Emil Carlsen creates generative art with Unity

As I already mentioned in the article Deconstructing AI Art masterpieces: Create Art like Refik Anadol, in the art scene the tools vvvv, openframeworks, cinder or touchdesigner are quite popular and both could be used to strive for looks and installations we see from contemporary AI Artists.

Besides the mentioned tools, there are many more out there, worth having a look. In the realm of programs, there’s for example Houdini, and if you explore the world of javascript programming language, there are great libraries available as well: three.js has a great user base and awesome tutorials to get started.

Recommendations to get started with AI Art, Generative Art, Generative Design or Creative Coding

If you looked at some of the links, you might have noticed, that mostly code is involved. To be perfectly honest, I don’t believe in sticking with generative design or art, if you won’t touch code. To get started, I recommend starting with p5.js, as it is related to javascript syntax and script syntax is quite useful as an entry point. If you never looked into programming but learned some languages before, we have similarities, also in programming we have different kind of languages that have a similar grammar and syntax. So script languages is a language family cluster, in the case of javascript, it allows you to extend your skillset with a lot of other contemporary libraries and adapt to other script languages easier.

But be aware, learning programming is a non-linear process and more a marathon than a sprint. The threshold concept is a good description for this way up some stairs, discovering new perspectives with every climb.

Everyone in this country should learn how to program because it teaches you how to think.

Steve Jobs

The other thing I would strongly recommend is looking at touchdesigner. There’s a free option for the program and there are so many awesome tutorials out there. Another aspect, why I would recommend it is, it is a node-based system, which means you place modules on a surface and connect them and modify values of the modules.

Getting used to node-based systems is highly recommended as more and more programs introduce node-based options and it allows some playfulness in the early exploration quite easily. Even blender has some node options now.

Another great thing about touchdesigner is the underlying programming language there. It's python. On the road to mastering Art with the help of artificial intelligence, Pyhton is one of the broadest used programming languages in the context of machine learning, so you could connect touchdesigner in a better way to work with underlying data sets or scrape data from the web.

As I spoke about node-based interfaces, I would also recommend cables.gl. To some extent, as it is on my personal list for a more in-depth exploration, and it is also quite accessible for non-coders. I also like that they are still in beta but have a quite good base of users with high quality examples. Another one you might have heard of is nodes.io – your thinking space for exploring ideas with code, also node-based 🙂

And as a final statement, though I really learned to hate that program before… which, in my case means I have a biased opinion since I studied almost 20 years ago and started my experience with 3D with 3DS Max and didn’t like my first-time switch… go for blender if you want to explore the world of 3D. But if you never used 3D software before, set clear goals of what you want to achieve.

There are quite some concepts one must get used to using 3D software. I was never so happy about designing 3D stuff at all. I guess I had most fun modelling some game assets for some student work, a long time ago… but rendering for days to see it was crab, wouldn’t relate to my impatient nature at that time. Now having two kids, I learned to be more patient and maybe I restart in 3D, in blender again.

Also, unity is quite accessible, but more suitable for games, apps and effects. I gave it some tries and quite liked it. Both blender and unity provide a great userbase and a lot of plugins, tutorials, and examples to learn from.

If it’s more about the visual generative part, go with touchdesigner. I was quite impressed by the result my recent course could generate by learning the program themselves. I will add some recommended tutorial links on my recommendations within the next weeks. Enjoy!

Easy entries to programming

1. The coding train – great examples on learning p5.js, machine learning and fun entertainment watching Daniel Shiffman, kind of a Bod Ross in more fun on programming.

Getting started with touchdesigner

1. Introduction to TouchDesigner – by Ben Voigt – a general overview of what the tool could do for you
2. TouchDesigner Beginner Course – by bileam tschepe (elekktronaut) – great tutorials for starters from a self-learner
3. alltd.org – the biggest resource and collection on touchdesigner tutorials
4. Derivative tutorial list
5. Tutorial list from The Interactive & Immersive HQ on Youtube
6. Noto the talking ball – interesting, more advanced tutorial collection
7. Exsstas – interesting, more advanced tutorial collection
8. paketa12 – interesting, more advanced tutorial collection
9. Akenbank – interesting, more advanced tutorial collection
10. The interactive & Immersive HQ – interesting, more advanced tutorial collection
11. plyzitron – some visual inspiring stuff with the option to buy sources via gumroad.
12. Matthew Ragan – great ressource collection on teaching touchdesigner
13. musichackspace – paid courses

Getting started with cables.gl

1. Cables documentation
2. Cables YouTube Channel
3. Introduction to cables.gl – build a data-driven gradient by Kirell Benzi

Books

1. Form+Code in Design , Art, and Architecture (2010) –  by Casey Reas
2. Generative Design: Visualize, Program, and Create with JavaScript in p5.js (2018) – Benedit Gross

Old school resources

1. Visual complexity – old school collection of examples for visual complexity

Have you ever wished you had a personal assistant to help you with your writing or with your artworks?

AI Assistants are becoming more popular and advanced each day. Back in my article on speaking computer language, I gave a short introduction to GANs, DALL·E or Google IMAGEN, text prompts to tell AI what you want to see. In fact, I never thought about AI writer before that article.

Recently I was on Appsumo and realized how many offerings there are on AI writers. There's contentforge, neuron writer, wordhero, rytr, whitecream, creaitor.ai and so many more. There are far more offerings for doing the copywriting part, than to create images. In fact, in terms of images, we mostly have "art" generators, AI graphics are not available on broader scale.

So, for writing this article, I used some of the mentioned tools, to generate parts of my content for me. In general, I must admit, that I optimize my contents for SEO, but I don't do it to drive only traffic. I write my articles because these topics relate with me. I was a bit shocked when I used one generator (whitecream) to write me a complete blog article.

The quality was good. It was one of those articles, that are great for Search engines. It used emotional words, had a good text structure in terms of SEO and all in all, it was a good job from a SEO perspective. From the Content perspective, it was too much on the surface. So I realized the immediate downside of this tools... search is having an even bigger problem, I already mentioned in my article on being trapped by search, that the quality of the content is not checked and why wrong “myths” circle in top results on google and others.

The created contents are good, but the quality of content on the web in general is endangered a lot, if everyone only "writes" to get Search traffic. Therefore, I support the Indieweb and in my point of view, it is more important than ever before to change search algorithms towards an “authority” to people behind the article and their field of competence.

The role of AI writers in the future of copywriting

With AI assistants, copywriters can put their skills to use in a more efficient way. They can make sure that they are not wasting time on skillsets that they don't have and instead focus on what they are best at - creativity, emotions, and research.

We should not think of these AI writers as a replacement for human copywriters. They just aid the content writers by getting rid of writer’s block and generating content ideas at scale.

AI assistants for writing are increasingly getting popular in the workplace. Some companies use them when they need to generate content for a specific topic or niche. While digital agencies use them to generate all kinds of content for their clients.

Believe it, or not, these three paragraphs were also written from an AI writer. You notice the text tends to advertise for AI writers. But this tone of voice, the writer got, was good, even for me, reflecting my thoughts on the AI writers in general. Similar to what I said about Art tools, Copywriting tools could be great addition to streamline your workflow.

Just like any tools, it has its limits. But if we use it in the right way, and with common sense, AI writers could be great writing assistants that help us to get over the writer’s block or come up with ideas at scale.

To give you an idea, about my used tools in this article: I used contentforge as the holder of my article, then I used rytr to write the blog section, mentioned above. In contentforge I used the "generate content" button from time to time, to see what happens.

Some of the generated text is great, and I can see myself using this type of tool in the future for smaller tasks or ideas generation. For longer articles or books, I absolutely believe that a human writer is needed to create a compelling story.

(so, this paragraph before was added "generatively" as an add on to the article I just wrote 🙂 quite funny)

But yes, these tools could be a great support. I am more worried about all the people using them for just pushing their SEO stuff instead of writing contents reflecting their believes, experiences or research.

When it comes to research and real articles, they don't help you to create the real content. But they could be handy to assist some SEO parts, such as a summary, some headlines in between and so on. To do a deep review, I miss the experience.

It is likely that all these tools make use of deep learning and GPT-3, a generative pre-trained transformer. So in my special case, not being a native speaker, tools (also grammarly, jasper or others) could help to improve my writing in terms of grammar or the words being used in general. The other bigger help I see in the area of SEO, if you want your content to be fully optimized for SEO. So it's kind of a polisher for your contents and an ideation source as well.

If you wonder, why I don't speak about chatbots in the context of text,... cause I think they are an own category, there are also many tools around, most act in a conversational way and are trained by machine learning principles. Here's a good entry/overview.

AI assistants in graphic design

I already mentioned some tools to create so called "AI art" in my article "Speaking computer language – welcome AI Art". I guess I also mentioned some logo generators out there already, maybe I also shared a link to Nikolay Ironov, a designer and neural network.

To wrap it up, I clustered some of the tools in some categories here:

AI generated images

For generating images at a general scale, we could include all the tools being mentioned in my other article on AI art before. DALL-E (1,2, mini) or Imagen and all the Ganbreeders such as artbreeder, nightcafe, wombo.art are some of the AI assistants for graphic design, to name a few.

But there are also tools to create people that operate with AI, like the incredible Metahuman builder from the new unreal 5 engine, there are AI driven ways to create people images. Generated photos is such as platform. There’s also a face generator available to create an image. Of course, you could get quick results, that look like fake quickly, but there are also images, that look quite promising;

Similar to generated photos, you could create AI actors with notaperson.ai. In my point of view the characters sill fell a bit static, but the powers of creating random people is there. If you don’t believe, watch the random results on thispersondoesnotexist.com. With the help of StyleGAN2 a lot of images are generated in a GAN way. The world of fakes is real, and besides images, also speech could get modified easily.

If you don’t like real pictures, look at thiswaifudoesnotexist.net. (A waifu is a fictive female Japanese manga/anime character).

AI generated logos

There are also a bunch of logo generators out there. Nikolay Ironov offers one and on logoai.com, you could quickly generate your own logos with some guided dialogue. A similar builder could be experienced here as a service from shopify or at logomaster.ai.

You might think, how scarry this is? No, it’s not. Honestly, I included AI tools in my exploration on the branding of a client already. Of course, branding is more than a logo, as it’s about the identity of the brand. Each personality has different characteristics that need application to the branding guidelines.

But now let’s assume you type in a DALLE Mini interface the attributes of a brand or a short description and look at the result; now compare it to some other text-input based generator. Won’t you think you get some kind of color scheme on that?

At least for my last branding project, I included AI Tools in my research. I got some straightforward color schemes I could extract and that were helpful. I didn’t use one of the mentioned logo generators though, but they could also be helpful in the exploration process as they help to narrow down the chosen path we want to follow.

In my case the brand itself was connected to AI and Edge Computing, so there was a brand fit with the tools being used. At that moment it felt right to ask the question, how an AI might see the company, reminding me of a nice piece of art by Christian Mio Loclair called Narciss, a a robot, built to analyze its own physical embodiment. Looking at itself and reflecting, what it sees; so why not use such tools, if they match to the brand?

AI drawing

I am not too experienced with AI drawing tools and so I only share the one I discovered some years ago. Autodraw. A Tool to help you create icons. It gives you proposals based on the shape you were drawing. So, in my case it recognized my fish shape and gives me proposals about other fish related shapes at the top.

Another fun application in a similar direction is https://quickdraw.withgoogle.com/. Challenge your Mouse drawing skills with form recognition.

Other AI tools for images

Finally, another tool you could give a test is an deep learning based image upscaler.

Resume

Is there a final wrap-up on my mentioned AI tools at all? Yes, there is... we are getting more and more tools that could distract and being applied wrong lead to bad results. Whether it be a world full of fakes (in terms of visual, text, voice) leading to less trust in digital environments or a tendency we often see when tools streamline processes. A lot of trash gets produced.

The danger of expecting magic with the tools is also there. These are tools that could assist, but they don’t define the plan, what you are aiming for. So, define the strategy upfront and evaluate afterwards. A sentence that sounds so true since 20 years and more being in digital for almost anything, whether it be creating websites, apps, designing things or any business objectives. Tools can’t replace goals and a strategy. AI assistants or tools are in this case not different from any other form of tools.

The perfect AI tool must integrate to your own process. In my case I see most potential in the exploration stage. As most creative processes are similar and alike, I refer to an easy system, such as human-centered design. We speak about inspiration, ideation, and Implementation. During these stages of the creative process,

AI could help you exploring your thoughts, flipping the problem and many more things that help you to get going. All the mentioned “methods” would be placed within the stage of inspiration. Also, in terms of a reframing of the topic, AI could help, as it is another perspective on the problem – often from a “wisdom of the crowds” perspective, as the trained algorithms are trained by the masses.

During the Implementation I see AI as an add-on to evaluate own results, in terms of writing to check the grammar, in terms of design to check for uniqueness or testing.

So finally, I could say, I believe in a lot of the automations being possible with AI, but like UX-Design, that got streamlined over the years with bet applied patterns (and corresponding dark patterns), AI-tools will lead to the same effect, some rebound effects: Imagine a lot of sites being written by AI and google ranks being strongly influenced by AI. The overall quality of the content will drop, and a search won’t be user friendly anymore. Hopefully AI results will get recognized and filtered.

Or on the graphic side, customers think of using AI builders for logos and more website design from finished templates. This will lead to less identity, uniqueness on the web and meaningless branding as all look the same. Uniqueness will be the struggle of the future as it is the struggle for all streamlined processes.

I recommend anyone being interested, to have a look at the tools yourself. There are enough tools with a free version to play around. Have fun and use the tools for good!

The command prompt: A black, empty screen and a blinking line, waiting for my input. Could you imagine that this prompt is the access to art or AI art? The command line shows a visual result of your terms, not ASCII art or related items. Nowadays, those prompts have different shapes, such as a form field or the use of voice interfaces.

Your imagination of thought, translated to words in your natural language (mostly in English), gives you access to new creativity. In the words of advertising, you need to express a good core idea, you want to get visualized. Similar to creatives being used to Image search engines, the need to find the correct words gets more and more important.

To explore my thoughts I look at the evolution of the way we tell machines, what to do and how CLIP and other neural models transform the way we could create what we are looking for. Whether it be AI art or maybe any picture we want to see, in general or what I would call AI graphics.

Computer language

A computer language is a language we use to talk to machines. It has been in use for about 70 years.In terms of human history, a very short time.

The first, more popular programming languages found their roots between 1960 and 1970. The first “high-level programming language” was Plankalkül from around 1945. Even earlier, the roots of programming go back to Ada Lovelace, yes, a woman, in the year 1883.

Nowadays, we even have low code or no-code ways to speak with machines. Mostly those use visual environments to drag together code components or nodes and are called visual programming languages.

So, in the past, we needed to write code in a language, following grammar. Now, we could also drag together many nodes and hope for an effect, while playing with parameters. At some point of customization those visual environments also allow to add code manually. But it's not essentially necessary to write the code yourself anymore.

DALL·E and other neural networks allow access to Artificial intelligence as a source of processing your thoughts via a text prompt. If you want, you could say that you tell the machine, what it should program with your natural language. Your chosen words define the quality of the output you are aiming for.

Natural language and natural interfaces

The natural language is the way we as humans speak with each other. To some extent even our voice-interfaces around us are now able to communicate with us in a way that feels almost natural. In this case our command prompt mostly starts with “Alexa”, “Siri” or “Hey Google” to initialize the dialogue. Natural language processing takes the task of converting speech into words in the background to take the right commands.

In 2013, there was a trend to use natural language forms in interfaces to make form usage feel more natural. Forms were integrated into sentences rather than presented as a separate option. Stats are more usable in natural language, but confusing numbers make it difficult to understand the impact of improvements.

The tendency to make interfaces more natural or more human, so we could use our natural senses to interact, is present. But the evolution of the interfaces also depends on technological developments.

There are many interesting developments, especially if you investigate the MIT Media lab and the tangible media group. They strive for seamlessly coupling the worlds of bits and atoms by giving dynamic physical form to digital information and computation. Research on venous materials and human-flora interaction provides insights into our future.

One of the easiest ways to interact now is our language, expressed by voice interfaces. When using Neuralink or similar chips inside our bodies, it's likely that we'll need to use interfaces with a well-structured language initially.

If you are looking for more scientific terms on how the language is being used for generating visual results, look for text-to-image synthesis.

GANs produce visual effects resembling the WebGL example on alteredqualia.com that utilizes DeepWarp, a method commonly employed for gaze movement.

CLIP, GANs or AI driven Art and approaches of deep learning.

Natural language processing is also an important tool when it comes to playing with so called CLIPs. CLIP is “Contrast Language-Image Pre-Training”, a neural network trained on a variety of text-image pairs. CLIP got introduced around January 2021.

Before getting into CLIP, a short excursion on learning. Deep learning is part of machine learning and helps machines to learn. The machine is supposed to increase its own skills without the help of humans. On contrast to machine learning, where humans interfere in the learning process, in deep learning humans take care of providing information for learning. To do so, we could also use our input prompt. The process of analysis is up to the machine itself.

An example for machine learning would be connecting words and images by labelling the images with the appropriate word. A typical combination for this cause would be ImageNet and WordNet.

CLIP is a neural network model that is already trained on 400,000,000 (image, text) pairs. So, this means that this is the data that is used in training the CLIP Model. You can input an image to the model and get the most probable caption or summary of it. Alternatively, you can use natural language as an input. The true magic comes when you see examples. Travis Hoppe shares some interesting applications in his Twitter account.

To get a full “beginners” understanding of the CLIP model, I recommend this article: “A beginner’s guide to the CLIP Model”. You will find many interesting articles on kdnuggets.com around artificial intelligence, computer vision, data science, machine learning, natural language processing or programming (Python).

To try CLIP yourself, you could use VQGAN developed by the University of Heidelberg. It’s a GAN (a generative adversarial network), those are special types of neural networks consisting of “Generator” and “discriminator” networks. There is a GAN that generates images. The generator is like an artist and the discriminator is like a critic during training. When the generator can produce an image that the discriminator can't distinguish from a human-made one, we can use it for creating art.

There are huge libraries of available GANs for very different purposes. Some of the lists I could recommend:

The GAN Zoo

GANS awesome applications

To play with the technology to learn, use artbreeder and generate new images; Artbreeder is also based on GAN based image generation.

Exploring Applications of CLIP

How to get started? I would always recommend playing with the technology. Most technology is too complex for an early deep dive, so start by playing. If you found your motivation to dig deeper, do it with all your might as you need a lot of patience to get a good understanding.

So, let’s look at VQGAN + CLIP you could use directly yourself online. This is a collab notebook, a small development environment running Python programming language. Your task is to input some values... To get started, I would recommend to only change the “natural language prompt” of what you want to see. In my case I tested some results of different apps with the sentence “Children playing in the forest” (because my kids are attending a kindergarten in the wood while writing these lines).

These are my first results using the CLIP linked above (to try yourself, open the link, basically go through the page and always execute the play button to the left and wait till there’s a check icon next to it, to signal it’s loaded), only for chapter “Parameters” enter a text. In the end you could also create a video. The results here are early iterations. The program would continue till the “free” volume is used up.

To compare my result, I checked for other applications out there and their result. So there’s for example an application that is also available as an app; In wombo.art you could test directly. The result from wombo art had similar patterns as my test with YQGAN and looked like the picture on the at the top left.

Then I used another popular web-service to compare: Nightcafe you can see on the right.

You see, when using a webservice, you end up getting one result. Using a google colab, you get many iterations and it's not as fast as the webservice or app. (Google Colab or Colaboratory is a way to execute Pyhton code in the browser).

There are also other CLIPS on google colab you could use. The following picture is the (an early iterative) result of the combination VQGAN+CLIP (z+quantize method):

You could continue playing with many more version out there. Here you will find an overview of newer applications: https://softologyblog.wordpress.com/

You notice that the basic style follows one pattern and it’s obvious nightcafe also uses some way of VQGAN combination. To create magic, or what I would call magic based on a text prompt, there’s still some way to go.

The magic I aim for: I tell the machine what I want to see, and they give me iterations of the thing I type in in. It mustn't be art or AI art.

To create this magic, we look at DALL·E. OpenAI released DALL·E 2 in April 2022. They show the real magic of the future. By the way, I wouldn't call the results art. Mostly we generate graphics.

A short introduction to DALL·E, DALL·E 2 and DALL·E mini

“DALL·E 2 is an artificial intelligence model that takes a text prompt and/or existing image as an input and generates a new image as an output. DALL·E 2 was developed by researchers at OpenAI to understand the capabilities and broader implications of multimodal generative models. To help us and others better understand how image generation models can be used and misused.”
(From their own description)

To make it shot, look at their website and get a feeling of the beauty that could be generated with DALL·E 2 or watch the short video on youtube:

Unfortunately, DALL·E 2 is not easily available to test yet. But you could sign up on their waiting list.

According to OpenAI DALL·E 2 isn't trained on faces, to avoid the risk of deep fakes. The danger is already there. Finally it the time to doubt any images being shown to us!

On twitter try a search for #dalle or #dalle2 and you find many inspiring pictures. Going through the magical results, it feels as if image searches could get a relic of the past soon or will only be needed for special causes. Image stock market could be AI generated in my point of view. Here are some randomly selected shots from some twitter members (all linked):

Yes, it’s a pity, we all can’t directly play with this fascinating technology!

Luckily I found DALL·E mini. And that's your friend to play! By the way, I also created my post image with Dall-e mini. I asked for "AI Art blue and yellow". One of the results could be found on the overview site of my writings.

The description of DALL·E mini:
“OpenAI had the first impressive model for generating images through DALL·E. DALL·E mini is an attempt at reproducing those results with an open-source model.”

They also have DALL·E mega, a large version of the mini, that is being trained. To see the fascinating world of training, look into the training space.

And finally, here’s your playground to play with the DALL-E mini: https://huggingface.co/spaces/dalle-mini/dalle-mini

That’s my result on "playing children in the forest"; quite an improvement in a very short time. If you play more with DALL·E mini you will notice, that faces still are really not trained much. But I already love it!

To feel the new power of words and to discover some magic along using words, go out and play for yourself!

By the way, there's another tool on the web, giving you similar results. It's neuralblender. Here's my kids results from them:

If you want to get inspired by other AI Artists, look at the work of Refik Anadol, his process is the application of a lot of machine learning to a real world of art. As I said, in my point of view, the term of AI Art is mostly wrong, looking at the output of DALL·E or other CLIP applications, as the outputs are not expressive. In my point of view, we speak about AI graphics.

Maybe you could guess, what my prompt in this case was (created with DALL·E Mini):

Have fun playing!

And guess what... My challenge is to build websites via a text prompt. Currently I am looking for available sources to have a good segmentation (Structural attributes, Style attributes). Let's see when I will share my results or findings.

Some more tipps on playing with DALL·E mini... use objects. To avoid deep fakes, faces won't work. Play around with the text prompt. You could try with the attribute, in the style of... or try the DrawBench prompts being used for Google Imagen first.

Google Imagen

While publishing this article on Mai 24th, Google came up with a competitor to OpenAI DALL·E 2. The market is on fire! And Google-Imagen impresses the same way, DALL·E 2 did.

Be excited to check out the first tweets on the topic #imagen.

And we will all be happy to play with the first, real demos!

Recommended further readings

1. Infinite Images and the latent camera
2. Hierarchical Text-Conditional Image Generation with CLIP Latents
3. DALL·E 2
4. Imagen by Google
5. StyleGan 3
6. Explore my newer articles around the topics:
   How AI Assistants Can Help You with Text and Art
   Generative Art – how to get started with AI Art
   Midjourney and DALL-E 2 – first hands on impressions
   Stable Diffusion celebrates new forms of creativity

FAQs

What if you were asked to create art like Refik Anadol? Or to produce some "AI art"?

In case you haven't heard of him, go directly to the vimeo channel of Refik Anadol and enjoy the moment of discovery!

When I first saw his pieces (only virtually so far), I was flashed. I really love that look of organic movement and abstract shapes. And one question that came to my mind was… how on earth was it done? As I did some research about him a little while ago, but recently rediscovered my love for his work, I wanted to share my findings about him and what could be called AI art.

For me, art reflects humanity’s capacity for imagination. And if I push my compass to the edge of imagination, I find myself well connected with the machines, with the archives, with knowledge, and the collective memories of humanity.

Refik Anadol in an interview with MoMA in 2021

To get an idea who Refik Anadol is and what AI in the context of art is, I recommend watching the ted talk with him.

Painting with data

Melting Memories or Machine hallucinations are examples of his stunning artworks collection. Many of his artworks take the perspective of the computer: what does it look like if a computer is dreaming?

In a world surrounded by data, Refik Anadol uses this data and presents it in a visually stunning representation. These data sets could include many terabytes.

To work with the data he uses machine learning (ML) to produce interactive AI projects. If you are up for Quantum AI by google, there’s also a project you could check out: Quantum memories. And here’s some insight from the data perspective on youtube.

In the past five years, we’ve trained more than 100 AI models, and used close to five petabytes of raw data.

Refik Anadol in an interview with MoMA from 2021

As I mentioned Refik Anadol works a lot with data. For the piece “Melting Memories”, he used EEG data to paint. In the description of the artwork you could find more details on how the masterpiece was realized. Refik describes the use of EEG data and some challenges converting it to vvvv. A live programming environment, everyone, who ever looked at interactive environments might have heard of. IIf you are searching for software that is native to mac, you could give touchdesigner a try. If you are more of a programmer, you could also look at openframeworks.

Besides data, it seems Refik Anadol likes giant canvasses for his art. So he also likes facades to transform architectural spaces. The illumination of Frank Gehry’s Walt Disney Concert Hall is an iconic example. Titled WDCH Dreams, he makes use of projection mapping to illuminate the building.
Google Arts & Culture has an interesting page about Refik Anadol, explaining more facets of the piece.

To get the projection right, you could see that he had a model of the building in his office.

In my practice, I’m not just creating a bunch of shiny pixels. I’m doing my very best to bring depth to surface by understanding exactly why technology may be purposeful and impactful for my imagination. Not because it’s the hype, or because it’s new, or because it is popular.

Refik Anadol in an interview in Harpers Bazaar Arabia

What are the takeaways from AI Art?

It is truly fascinating to see how Refiks Anadols' work circles around our current digital, AI-driven and real landscape. Of course, the work does not address every aspect that is relevant at the moment. This includes environmental aspects, as I mentioned in my article Improving the world as a designer. However, his art makes future aspects of AI tangible in a visually appealing way. Everything I expect from art.

There’s an interesting discussion in MoMA magazine on “Modern Dream: How Refik Anadol Is Using Machine Learning and NFTs to Interpret MoMA’s Collection”. The article describes Refik's Journey to get into his machine hallucination series. It is an excellent source of inspiration, as well as an overview of the status quo of AI-art and its beautiful perspectives. In terms of the ever emerging NFT landscape maybe even an option to get into AI-art yourself. If your alarm signals are ringing, when i mention NFT, be assured, there are eco-friendly NFTs, too.

By the way, if you want to dive deeper into equipment, I guess one piece I forgot about is the projector he is using. And if you want to get a comprehensive overview of his most famous pieces and the money he earned, you can find more on artnsketch. Unfortunately I didn't find details on how he creates the depth in his works. However, I am pretty sure it's the combination of architecture and the illusion of the depth, that creates the magic you might know from playing with generative art.

If you want to dive into the world of generative art, I would recommend getting started with p5JS or processing, but also touchdesigner offers awesome tutorials to get started. Another way to learn is following Daniel Shiffman, aka the coding train on youtube. No matter if generative programming or getting started with Machine Learning (ML). Watching him is already a classic. There's also an impressive collection of AI artists, tools and resources at aiartists.org.

Those principles being applied within AI-artworks are, in my opinion, relevant to design disciplines as well. Interacting in 3D space and mapping visually engaging worlds in a metaverse or other VR experience is challenging and requires more thought, as the world of AI is already showing. Maybe Refik offers such a world for us in one of his latest projects: dataland.

For now my final summary is, thank you, Refik Anadol, for inspiring creative visual discoveries and an inspiring combination of contemporary technologies to provide beauty.

❤️ @refikanadol

Other articles and sources:

1. AI Artists
2. Some resources I collected in my article "Generative Art – how to get started with AI Art"
3. My other writings often focus on how AI is being used in Design, Art, and society in general and I often share some thoughts.
4. AI Art: How artists are using and confronting machine learning | HOW TO SEE LIKE A MACHINE – great video from MoMA with some more description of Refik's work
5. Ted Talk from Refik Anadol: "How AI art could enhance humanity's collective memory"

Recommended tutorials to get started:

1. Refik Anadol Particles in TouchDesigner, Part One - Tutorial
2. Refik Anadol Particles in TouchDesigner, Part Two - Tutorial
3. Some more links to tutorials and how to get started with AI Art you will find in my article Generative Art – how to get started with AI Art