Season Of Mists And Mellow Fruitfulness, AI Poet XiaoIce Is No Less

William Wordsworth’s poem titled Daffodils paints a beautiful picture with words. The countryside, the scenery and the colours come alive as he writes, “I wandered lonely as a cloud That floats on high o’er vales and hills, When all at once I saw a crowd, A host, of golden daffodils; Beside the lake, beneath the trees, Fluttering and dancing in the breeze.” His words are powerful, and they tease and test the human imagination to go further than mere text — they force us to visualise. Researchers from across the globe have discovered that vision is a common source of inspiration for poetry. Now, in a new paper, researchers from Microsoft, the National Taiwan University, and the University of Montreal, have built a prototype that writes poems based on images submitted by users. “The objects and the sentimental imprints that one perceives from an image may lead to various feelings depending on the reader,” say the researchers in the paper. Now sample this piece of poetry: “Cities are too ashamed to face the countryside, Let us compare beautiful hearts. And concentrate a scenery, Like swaying with the wind” Believe it or not this is a poem generated by an artificial intelligence system called XiaoIce after it was given the image of tall skyscrapers. The image below also shows other poems that the algorithm came up with. [caption id="attachment_27460" align="aligncenter" width="991"] AI poetry[/caption]   XiaoIce works in the following steps:

1. See the image
2. The algorithm extracts a few keywords that are in the image
3. Then those keywords are expanded by matching them to other words that are strongly related
4. The association is established with the help of poems already written by humans
5. The algorithms also perceive the “sentiment” from the image
6. The verses are generated by a recurrent neural network which is trained on many human poems

In fact, one of the special things about XiaoIce is that the approach is evaluated by human testers and is also compared to many baselines. Interestingly, the researchers claim that their method can create much more “artistic” poems compared to other methods. But not many efforts have been taken by researchers to generate poems from imagery and this makes research effort unique.  The researchers say that, “ XiaoIce has already generated more than 12 million poems for users since its release in July 2017. A book of its poems has been published by Cheers Publishing, which claimed that the book is the first-ever poetry collection written by an AI in human history”

Images And Poetry

The research paper notes that compared to traditional Chinese poetry, modern Chinese poetry is easier to read and critique. They opine that modern poetry involves great amount of imagination and creative uses of language. The whole idea of the researchers that come out as a theme is that an image is worth much more than a thousand words, and hence more information than keywords. The other theme is that there are many ways to interpret images. The researchers went as far as to publish some collections of the poetry written by the AI agent. Over 12 million poems have been generated using this AI agent and the research team says, “A book of 139 generated poems called Sunshine Misses Windows (2017, Cheers Publishing) was published last year, which was said to be the first ever poetry collection written and published by an AI in human history.” The researchers compared the method built by them against an established baseline and a competitive method known as CTRIP. It is seen XiaoIce and CTRIP both have simply outperformed the baseline system. One of the poems that has also become popular written by XiaoIce: “Wings hold rocks and water tightly In the loneliness Stroll the empty The land becomes soft"

Experiment Setup And Evaluation

The researchers’ test crawled and downloaded 100 public domain images from Bing image service from Microsoft. They also started searching 60 randomly chosen nouns and adjectives in the predefined categories. Till this date since the researchers have made the system public from July 2017 the public have submitted about 1.2 million images and have used the system to create poems. The researchers created their own metric since other metric such as BLEU and METEOR do not take emotions and feelings into account. Here, the assessors give a rating from 1 (dislike) to 5 (like) after they compare all the poems after reading. The design of the whole testing is close A-B test widely used in search evaluation. The researchers have worked with five engineering background college students and two literature background students to judge the poetry. [caption id="attachment_27462" align="aligncenter" width="1019"] Comparison between AI poets.[/caption] The researchers said, “Human evaluation results of our method and two other baselines. While the “related” part is dominated by Image2caption and CTRIP is stronger in the “fluent” part, our method significantly outperforms both baselines in other aspects.” They added that a large scale user study that their generated modern poems earn much more acclaim than the generated captions. “The reasons,” explained the researchers, “Are that our generated poems are more imaginative, touching and impressive. Although the generated traditional poems are more fluent than our modern poems, the method we proposed significantly outperform them in all other three aspects.”