Motion from Stillness

The main concept linking the two practices is that of an emergent whole—that is, a whole with properties that do not exist in its component parts. A movie is really a population of photographs linked in a series, none of which possesses the property of motion. But in the assemblage formed by a projector, a screen, and a viewer, motion emerges as if by magic, and indeed, magicians were involved in the medium early on. I deal in my books with emergent properties—the properties of communities and organizations that cannot be reduced to the people that compose them—but there, I cannot manipulate them. In film, I can intervene (I used to paint special effects directly on the film itself) and generate new emergent effects at will.

Because I was dealing with emergent effects from the start, my focus as an artist has never changed. Although around 1983 I stopped making art movies for a while, I worked as a 3D computer artist for almost twenty years. And because I knew from personal experience how hard it is to create 3D models of people and endow them with articulated motion, when I saw Midjourney’s capability to generate scenes without any 3D models, and then Runway’s capacity to animate them without any skeletons or inverse kinematics, I was really impressed. I couldn’t wait to see what I could make emerge from these programs.

The innovations are totally real. The conflict (for funds and recruits) between symbolic AI and connectionist AI is an old one. Advocates of symbolic AI, like Marvin Minsky and Seymour Papert, actually tried to destroy their rival when neural nets had only two layers: input and output. In the 1980s, there was a resurgence of connectionism with the introduction of three-layered neural nets and a way to train them (back-propagation). The third layer gave neural nets the capacity to be more than just pattern associators. Today’s neural nets have dozens, if not hundreds, of these intermediate layers, which give them even more powerful emergent abilities. These abilities are real, and to the extent that they do not depend on the manipulation of symbols using logic, they are entirely novel. They capture the difference between “knowing that” (verbal knowledge transmitted by talks and books) and “knowing how” (embodied knowledge taught by example and learned by doing). In this way, they can influence philosophy itself.

Well, there is something to this thought, to the extent that AI image generators can produce photorealistic images that could pass for real photos without the images indexically referring to anything real. This is going to be a big problem for the legal system and for the concept of photographic evidence, but not so much for artists. The robot historian would probably see the artistic history as a progression of ways to generate emergent effects: apparent motion, analog special effects, digital special effects, and effects that are entirely synthetic, as with AI. Since the very intelligence of this imaginary robot would most likely be connectionist (not symbolic), it would probably think about the newest developments as marking the point that its own intellectual capabilities, and its own visual field, emerged as a form of machinic know-how.

It’s an old obsession of mine. In 1980 I made a film with Hollywood production values about a private eye who gets into trouble. The film was not a big success—it went against the minimalism dominant at the time—but it did influence the following generation of filmmakers in New York. The movie was called Raw Nerves: A Lacanian Thriller. Pirated copies can still be found online.

When I arrived in New York in 1975, there was an old cinema that showed two noir movies every Tuesday—titles like Kiss Me Deadly and Murder, My Sweet. The private eye was always an antihero with personal problems, but who for the same reason was always insignificant to the bad guys, and they never saw him coming. I just fell in love with the expressionistic lighting and the twisted plots.

Well, the biases introduced by the training data show up when you give Midjourney a prompt that it cannot understand. Then it gives you an image in its default genre, which tends to be sci-fi, always with a beautiful woman. (I can see how nerd programmers would introduce those two biases). But other than that, the training set is basically the whole internet, so the biases are more subtle.

There are always hallucinations, the six-fingered hand being the most famous. These errors are interesting, making the image more surreal, but it’s hard to take advantage of them in a narrative video. On the other hand, there were several images that I did not request explicitly, but were so good that when I saw them, I had to change the plot to accommodate them. For me, the most important things are the resources the program gives you to control the continuity of a character’s appearance, the scene’s lighting and mood, and the kinds of interactions among the characters, like fights or hugs. Midjourney is good at that.

The neural networks we use today are incapable of uniformity or homogeneity; they are built to generate continuous variations. It is possible that in the future they will add modules that can constrain them to imitate, but for now they can only create. From an industry point of view, this is not a good thing. An assembly line has to have exact repetition (or, at most, customized repetition), and neural nets are useless under that paradigm. But even in commercial art studios or in architecture, there is always a client who sees a project and wants to keep some parts identical but change some others. That cannot be done today.

My presentation at that event was about genetic algorithms, which are simulations of biological evolution. They are closely related to neural nets and can be used to breed neural nets. But I did not enjoy the rest of the conference. The way I remember it, many other presentations were delivered in the postmodern style (which I’ve always hated), so I can’t remember any details. Sadie was good, as usual—clear and articulate—but there were so many others just offering the forgettable McDonald’s version of French philosophy.