The Secret Language of DALLE-2
How Giannis Daras and Alexandros G. Dimakis from University of Texas at Austin discovered that DALLE-2 has a secret language.
So DALLE-2 and Imagen made a lot of noise as they are able to produce high-quality images from the text description. However, at least in the case of DALLE-2, there are problems with text generation. If you want to create a sign with the word “Deep Learning”
, these are the results:
For more complicated stuff like “Two whales talking about food, with subtitles
” we will get complete gibberish as a text of subtitles: “Wa ch zod ahaakes rea
”. However, if you take this gibberish and use it as input to DALLE-2 it will generate the images of sea food.
Guys wrote a small paper about it. You can find more examples there.
I am just wondering, if it is a bug or a feature. ¯\_(ツ)_/¯
Let’s see if we are going to decipher this somehow.