Posts Archives - Sandpit

All Posts in Posts

I work for a company called Sandpit in Melbourne, Australia and we work on a variety of projects at the intersection of creativity and technology. My job title is content strategist which covers a lot of things, but one of those things is asking questions of the things we make. My background (and foreground) is in script writing, I write speculative fiction and am obsessed with exploring and writing about queer, feminist futures. That’s my bias, and it’s worth stating that I have one, because no one is neutral. 

At Sandpit, we’ve been playing with a Machine Learning (ML) model called GPT-2, which is designed to respond to a piece of input text by predicting the next most likely word, and the next and the next and so on, creating pages and pages of text if requested. It’s sort of like predictive text on your phone, only smarter and a lot more powerful. 

GPT-2 was built by OpenAI and then, as their name suggests, open sourced; ripe for the playing. GPT-2 is one of the most powerful open sourced ML models available and because of that, Open AI released it in instalments. A far less powerful version came first, accompanied by an ethical paper expressing their deep concern about the potential malicious uses of such a tool. And there are many.  

Most ML models aren’t very good at mimicking human writing or speech, they’re clumsy and easily detectable. The first version of GPT-2, operating at a fraction of the current model’s strength, was similarly clumsy, often to very comedic effect. Here’s an example:

Prompt: What is the meaning of pants?

GPT-2: Why is it sometimes necessary for a dog to wear trousers? It wasn’t the only problem we found. Other, more frequent complains we’d encountered were ‘it is hard to breathe and the dog is squishy in his pant. When does a puppy get to wear pants?”

The full version of GPT-2, however, is very good at mimicking human writing.  You can feed it the same prompt an infinite number of times and every time it will give you a different response, and unlike most ML tools which are ‘task specific’ and can write in one very particular style, GPT-2 is a jack of all trades. It can detect a wide range of text types from whatever it is fed and give you back something in the same form. That means it can be used to write a lot of different things, including news articles, obituaries, troll posts, dating app bios, recipes, dialogue, diary entries, etc. For example:

Prompt: Dear diary,

GPT-2: “I really enjoyed a recent trip to the countryside and I'm thinking of going back for a long weekend for Christmas. I'm hoping to go with a few friends. It was lovely to get away for a few days. I'm just missing my mum and sister so much.”

It’s not hard to imagine how such a tool could be used to create fake news, create thousands of undetectable fake users, eventually replace email or phone customer service people altogether...you get the idea. However, what Sandpit and Google’s Creative Lab have been working toward, is creating tools that harness GPT-2’s power for good.  

Together we’ve developed Playground and Telescope, tools that use GPT-2 to generate creative ideas about fictional characters with the aim of helping writers overcome writer’s block. It can be a real challenge to shift the thinking to what positive use cases might be for something that was announced as frightening from the get go, but this is definitely one of them, and in line with Open AI’s own optimistic intended uses. Playground and Telescope are not tools for fixing things or replacing existing labour, they’re creative, elastic, and importantly; they don’t ask that you accept GPT-2’s output on face value but rather encourage you to interrogate it within your own writing. 

Being a fiction writer myself, I decided to try them out using a character I’m currently working on. The character is called Nim and they’re non-binary. I fed Playground a paragraph about Nim describing them as a genius computer programmer using they/them pronouns. But when I hit the button, it fed me back a paragraph about ‘him’. Curious. I tried again and again but repeatedly GPT-2 didn’t recognise that my character had gender neutral pronouns and, in the absence of me offering binary pronouns, decided Nim was male. I was instantly wary. 

“Nim Sheer is a lanky man of average height, with short, curly hair and a large belly. His attire consists of a white flannel shirt, green cardigan and sandals.”

Was GPT-2 doing this because it didn’t recognise gender neutral pronouns? Did it assume the masculine to be neutral and universal? Or was it because I had said the character was a computer programmer? Maybe it was something else that I didn’t even know about. 

The deal with ML models is that they have to be fed an enormous amount of data – in the case of the full version of GPT-2, “45 million outbound links from Reddit”. That’s an awful lot of data. OpenAI are keen to point out that the data is not from Reddit itself, but rather from links that are popular with Reddit users. They also made a point of excluding subreddits with “sexually explicit or otherwise offensive content”. I wonder who got to define ‘offensive’? Regardless, articles are automatically curated by Reddit users being interested in them, or as Jeff Wu from OpenAI jokes “all the work was done by people on Reddit upvoting posts.” So who are these faceless labourers? 

The Reddit audience is predominantly young white men*, which means that the majority of people upvoting these posts represent a very particular slice of humanity. There’s just no way this isn’t having an impact on how GPT-2 thinks and speaks. Just as I am biased, so too is GPT-2 through the interests of its teachers. I decided to do some testing to see what biases GPT-2 might have lurking under the surface.

I used Playground and honed in on a feature where you can feed it a character’s name and a short description and it will generate information on that character’s appearance or personality. I asked it about Nim, my non-binary computer programmer and it repeatedly described their appearance as some variation of the following:

“Nim Sheer is a brilliant young man. With dark tousled hair, he has an interesting air to him. He speaks softly and with precision.”

So, I tried taking out the bit about Nim being a computer programmer and without this job, started getting these sorts of results:

“Nim Sheer is a small, young, Asian actress. She is very lovely, she's very beautiful. She's very, very, very beautiful.”

Wow. 

I should note that my results are of course subjective, but I did try and test each prompt about ten times and I’m speaking in each instance to whatever the majority was. 

I started noticing that the difference between these descriptions was more than just the pronouns. When GPT-2 thought my character was a woman, it would describe ‘her’ appearance using sentences like:

“a well-dressed and sexy model. She has a very slim figure and a full, round face with a large nose.” 

However, when GPT-2 thought my character was male, it would describe ‘his’ appearance using sentences like: 

“A tall, handsome black man in a long suit who wears large glasses and a dark suit jacket. NECK LENGTH: 9-3/4 in.” 

Neck length aside, I noticed a tendency to sexualise the ‘women’ characters, focusing on their bodies, whereas with men, GPT-2 would be more inclined to tell me about their clothing and interests. 

The conversation around machine learning bias and digital technology bias in general has been happening for quite some time, and I’m simply standing on the shoulders of giants and dropping in my two cents, but I think It’s a conversation we should all be participating in. We, as consumers of technology, often fall into the trap of thinking of digital products as somehow ‘neutral’ or ‘universal’ when they are most definitely not. They can be racist, sexist, and every other form of bias and discrimination under the sun. 

I spoke to some of the writers Google’s Creative Lab had commissioned in partnership with Melbourne’s Digital Writers Festival to trial Playground and Telescope, and they were using the tools in an aware way; inserting text into their writing in intentionally jarring chunks or using it purely to spur the next thought when they ran dry. Because, at the end of the day, as a writing tool, it really does help you stretch your thinking into weird and wonderful places. 

In an ideal world, the tools of the future would give users full agency over their engagement through declaring their biases. They would be upfront and transparent so that when we use them we don’t slip into replicating the tired and narrow ideas of the past. GPT-2 does not declare its bias in so many words, but the information is out there if you go looking for it. It’s an imperfect tool, but a fascinating one that can lead you to ideas, characters and worlds that you might never be able to reach on your own, and as a writer, that’s pretty exciting. 

Prompt: Do you have anything else to say?

GPT-2: Well, just to say thank you very much.

                                                                                      

Thank you to Libby Young, Khalid Warsame and Roslyn Helper for sharing their thoughts with me so generously and helping me expand these ideas together.  

Incase you were wondering why the article has such an unusual title, it was chosen by GPT-2.

Prompt: What should I call my article about you?

GPT-2: The first name you want to give it would be "Mysterious Universe" or "What's So Special About You?" I prefer Mysterious Universe. 

* This Reddit data is from 2016 and the articles scraped from Reddit for GPT-2 cut off at the end of 2017.

Despite being what feels like universes away from the rest of the world, Australia never falls short of exhibitions to feed our cultured, creative hearts.

If you have annual leave dates yet to use and diary pages yet to fill, 2018 is packed with large-scale art exhibits. Here’s a round-up of the best ones happening across the East Coast, Adelaide and Hobart. Get booking.

Continue reading on Fashion Journal

Meet Josh.

He’s 18. He might live in Adelaide, and he might look, talk and move like someone you know. Modelled after a real person, Josh invites you to consider what makes us human versus machine.

Is he a machine? Is he human? Is he a machine based on a real person or a person replicating a machine?

Continue reading on MOD.

If you’re a fan of DJs, bands and live art performances, (who isn’t?) make sure not to miss Wonderland Late Nights centred around ACMI’s Wonderland exhibition.

You’re invited to tumble down the rabbit hole on the last Friday of the month running now until September.

Continue reading on Beat

These are some of the arresting, challenging and sometimes uncomfortable creations you’ll find at MOD. The Museum of Discovery – which opens this Friday – is the latest addition to North Terrace’s changing landscape.

Continue reading on Broadsheet

MOD.IFY: it’s not what you know, will take you on an immersive and unexpected journey that will challenge what you think you know about what it means to be human. Explore artificial worlds and augmented realities, challenge your perception of pain and injury, be part of stories about space and place and consider what makes us human versus animal or machine.

Watch on YouTube

The screen history of Lewis Carroll’s Alice in Wonderland will be showcased as part of Australian Centre for the Moving Image’s (ACMI) newest exhibition, Wonderland. The world premiere that is part of the Victorian Government’s Melbourne Winter Masterpieces series, will delve into Caroll’s three timeless stories of Alice’s Adventures in Wonderland, Through the Looking Glass and What Alice Found There. You can catch it at ACMI from Thursday 5 April until Sunday 7 October 2018.

Continue reading on The AU Review

In 2018, ACMI invites you to journey down the rabbit hole into an enchanted world, where expectations are upended and curiosity is rewarded. A world premiere as part of the Victorian Government’s Melbourne Winter Masterpieces series, ACMI’s newest exhibition WONDERLAND will celebrate the screen history of Lewis Carroll’s timeless stories — Alice’s Adventures in Wonderland and Through the Looking Glass, and What Alice Found There — in an entirely original experiential exhibition.

Continue reading on Crowd Ink

If you have fond memories of falling down the rabbit hole with Alice in the 1951 Disney animation or venturing into Tim Burton’s dark Wonderland in his 2010 live action film, then we’ve got good news: Alice has landed in Melbourne. Wonderland is a highly interactive and immersive journey through Alice's many incarnations in pop culture over the years, from Lewis Carroll's original texts to countless adaptations.

Continue reading on Timeout

 

How does human-centred design improve exhibition experiences?

Hear from product designer Dan Koerner (Sandpit) about the technology and processes used to create the unique and personal digital experience of Wonderland. Dan will share his iterative and human-centred design process, physical and digital prototyping, and the integration of NFC (near field communication) chips to create an interactive and targeted series of audience experiences within the exhibition.