Posts Archives - Sandpit

All Posts in Posts

“I never know what to call him… umm, it,” says Penny Stevens.

Penny is a project manager at Sandpit – an Adelaide-based design and technology firm – and is used to working on curious briefs. But the one she is describing eclipses even her normal standard of weird.

The strangely realistic eyeballs are connected to the…

…off-putting, fleshy tongue and teeth (via several face plates)

Skin tones (mixed by Marshall) are connected to the…

…face flesh (that feels creepily realistic)

“I think because it is so closely based on a human, you begin to humanise it with your actions and language,” she continues.

“It” is a robot head known as Josh, which has been designed and built in collaboration between Sandpit and rapidly-rising freelance prop and animatronic expert Marshall Tearle.

The idea for Josh, though, comes from the soon-to-open Museum of Discovery (MOD) at UniSA – an institution billed as examining ideas at the intersection of science, art, and innovation.

MOD’s digital exhibitions manager, Simon Loffler, dreamed up the concept of constructing and installing an animatronic head while considering the best way to explore the increasingly blurry line that sits between humans and robots.

“We’ve killed off dozens of ideas so it’s nice this one has stayed strong,” says Simon.

“This concept was to build an animatronic head that you can kind of have conversations with, and give people a feel for what that would be like.”

Simon took the decision to make the robot as realistic as possible – placing it firmly alongside robots that induce unease with the depth of their familiarity.

“We wanted to make it as generic as possible… so it’s more about the ideas the installation is exploring than who it is.”

Simon recruited Marshall to work on the project, and together they set about finding a generic human on which to model the prototype.

Adelaide teenager Yazeed Daher was chosen, and within weeks he found himself in Marshall’s studio, his head encased in silicone.

“It’s a pretty full on process,” says Marshall. “He’s got reliefs on him and his hair is all covered. We then cover him in a skin-safe silicone which is, in turn, covered in a plaster bandage.”

Marshall’s role in the process is enormous – he produces the entirety of the physical head, from its life-like skin, lips, eyes, tongue and teeth (that he builds and paints) to the mechanical innards that are used to animate the head.

Marshall says the greatest challenges have been finding room in the head to fit all the required mechanisms and working out how to position the motors around its mouth to create realistic movement as it speaks.

Even the eyeballs took hours upon hours to manufacture.

“The core of the eye is a separate part to the lens, so it roughly replicates how our eyes actually are,” says Marshall.

“They get air brushed and hand painted and also if you look closely, there’s little fibres in there that replicate the blood vessels. It’s kind of a daunting process – the last process of doing the lens, you can mess it up. I actually made five eyes to get those two.”

If Marshall is Josh’s body, then Sandpit is its brain. The firm is writing the code that powers Josh’s behavior – programming in things like a waking response that will be triggered by someone walking into the room in which the head is housed.

In deciding how Josh’s face will move in response to those around it, Sandpit are building a kind of personality into the robot – which will speak a limited range of phrases based on recordings they’ve made of Yazeed’s voice.

The process, says Penny, has involved far more subtlety than they initially imagined.

“It’s amazing when you realise how much a small movement of your face can communicate,” she says.

When MOD opens in May, Josh will be on display – so it’s only a few months until anyone who walks through the door can experience exactly what a small movement of a hyper-realistic robot’s face can conjure for them.


Data to Decisions CRC is excited to announce the creation of the Data Science Competency Framework (DSCF), as part of its objective to build a sustainable big data workforce. This is an Australian first competency framework for data scientists. The framework defines a series of competencies specifically focused on describing the abilities, knowledge and experience required by data scientists, data engineers and data analysts.

By describing competencies, the DSCF underpins workforce development at the individual, team and organisational level.  It facilitates the identification of development needs, enhances role clarity, supports recruitment and the establishment of career pathways, which in turn enables teams working in the big data space to better achieve their outcomes.

To further support the development of the data workforce, D2D CRC is utilising the framework as a basis for the Data Science Development Planning Tool. The tool is an online self-assessment system that enables people to identify gaps in their competencies. In turn the tool provides development options for data scientists, data engineers and data analysts of any level.

D2D CRC has teamed with award-winning digital studio Sandpit, a fellow Base64 inhabitant, to develop the system. Sandpit create captivating, meaningful and highly personalised experiences in the online and physical world for clients Australia-wide.

“Both the framework and the tool will facilitate development, recognition and help managers bring the right mix of skills together for the team” said Education and Training Manager, Megan Prideaux.

“We are excited to be working with Sandpit, a local tech organisation, who are building the tool. Thanks to Sandpit’s impressive capabilities, the tool will be able to take people through the self-assessment in a considered, effective and easy-to-use way.”

The Data Science Development Planning Tool is expected to be completed later this year, with a small group of partner organisations already signed up to pilot it before making it more widely available.

A multiracial couple fall in love in a video game. A lamp explains art. And a science-fiction film asks what it takes to colonise other planets. Melbourne designers are exploring the social and ethical dimensions of life in the 21st century, on Earth and off.

"Design for the Common Good," curator Keinton Butler calls it. The senior curator of design and architecture at the Museum of Applied Arts and Sciences in Sydney selected Melbourne designers Ken Wong, Dan Koerner, Paul Marcus Fuog and Lucy McRae for this year's Sydney Design Festival.

Lucy McRae  McRae has designed a fictional research and training ground, The Institute of Isolation.

Lucy McRae McRae has designed a fictional research and training ground, The Institute of Isolation.

"The [Common Good] exhibition explores the work and practices of a new generation of designers who are boldly taking action to effect positive change and influence long-term sustainability in our region," says Butler, who will interview the internationally recognised designers in a public program at Melbourne's MPavilion, in the lead-up to the March festival.

While cutting-edge technologies feature, it's old-fashioned storytelling that drives their designs.

In Sandpit's The Story of Lamp, audiences read the book and follow clues to lamps around the Arts Centre. Technology embedded in the lamps and the book talks to the reader, identifying artworks around them.

In Sandpit's The Story of Lamp, audiences read the book and follow clues to lamps around the Arts Centre. Technology embedded in the lamps and the book talks to the reader, identifying artworks around them.

"Games can be more than just entertainment and a time filler," says game designer Ken Wong. "Games can teach us things – teach us empathy, help us deal with experiences, and understand the world. I'd like to believe we've achieved a bit of that in Florence."

After designing the successful puzzle game Monument Valley – multimillion sales, two BAFTA awards and Apple's 2014 iPad Game of the Year – Wong launched his own studio, Mountains. Florence is its first project, scheduled for release this year. It tells the story of Florence, a Malaysian-Australian like Wong himself, and her Indian-Australian love interest, Krish.

"I don't know if the world knows that that's what Australians look like," says Wong. "I think it's important to tell these types of stories and have that kind of representation."

Dispelling the default position that games are designed for a male market requiring male heroes, games are increasingly populated with women and other under-represented groups. They are also shorter, less-time-consuming stories.

Florence i tells the story of Florence, a Malaysian-Australian like Wong himself, and her Indian-Australian love interest, Krish.

Florence i tells the story of Florence, a Malaysian-Australian like Wong himself, and her Indian-Australian love interest, Krish.

"Hugely successful games like Edith Finch and the gay dad simulator Dream Daddy are about realistic people [and] their relationships," says Wong. "Games have grown up in the last couple of years and there's all sorts of stories we can tell. Love is a big part of our lives and I wanted to tell a story about a relationship."

It's a screen thing

Where Wong finds mobile devices offer "intimacy" between the player and the game, Dan Koerner is less enamoured.

Florence is the first project of Ken Wong's studio Mountains.

Florence is the first project of Ken Wong's studio Mountains.

"It just results in a whole bunch of disconnected people staring at their screens," says the creative director of Sandpit. In interactive designs for clients such as Arts Centre Melbourne, Melbourne Zoo and Penguin Books, Sandpit combines analogue and digital technologies. "We're working increasingly without screens," says Koerner. "What does the internet look like without a screen on it?"

To encourage families to explore the wealth of art around the Arts Centre theatres, Sandpit designed an immersive experience called The Story of Lamp, illustrated by Nick Lewis. Audiences read the book and follow clues to lamps around the Arts Centre. Technology embedded in the lamps and the book talks to the reader, identifying artworks around them.

"Sound has become a very important part of our work," says Koerner, who has a background in film and theatre. "We work a lot with composers, sound designers and music."

Injecting humour and personality are key to its storytelling. One lamp hums to itself 24/7. "There were people going into opera shows and walking past this strange lamp that was just humming to itself," he says.

"We talk about the dramaturgy of the experience from beginning to end," says Koerner, who is working on ACMI's forthcoming blockbuster Wonderland.

For Lucy McRae, designing speculative worlds is a way to provoke people to consider what it takes to live beyond our planet, travelling vast distances in isolation and microgravity.

"The body is not designed to live beyond Earth," she says. "To live for prolonged periods of time in space, fundamental aspects of human biology will need to change.

"What is the future of procreation in space?" she asks. "How do we train the mind? What's the psychology behind sending someone into a vacuum, which is very different to the way human biology behaves on Earth. We probably have more chance of living under water than we would in space because it's similar. Can we design new programs, new ways to condition the body that will adapt [our] cognitive capacity?"

With the help of experts in astrobiology, endocrinology and engineering at Harvard and MIT, McRae designed a fictional research and training ground, The Institute of Isolation. In the film, which will screen in Sydney, she also performs the role of an astronaut preparing for life off Earth.

"Could we design isolation?" she asks. "Could isolation be treated like a building or a vaccine, and we create it from the point of view of an architect or physician? These fictional locations explore whether the design of isolation into buildings could play a role in advancing human biology on an evolutionary scale."

For McRae, as with the other designers, storytelling is a powerful way to do "good".

"If we can connect science with imagination more closely, we can begin to understand and steer science in a more ethical, democratised manner. Because things are undefined, fiction can drive politics."

It's up to us to make it a happy ending.

In November I turned to Facebook asking…

Could use some help… I’m working on a list of immersive shops, creative technologists and folks who are making amazing things with story and code. This is intended to be the start of an open resource. The work can be diverse — docs, brands, installations, product, experimental narrative, art and/or culture jamming. The hope is that this can be focused on the teams and/or individuals, not just the work. You can list yourself if you like too.

All suggestions welcome. I’ll compile the results and combine them with the research that we’re doing at the Columbia University Digital Storytelling Lab. If possible can you please help to spread this? Thanks so much in advance.

The following list was curated from various responses to the post and direct messages I received. Big thanks to everyone who participated in the discussion! Please let me know if I’ve missed anything. Thanks!

View List

Best Learning App—Tablet
La Francomobile (TV5 Québec Canada/Manito Média/Tobo)
Little Lunch (Australian Children’s Television Foundation)
Nancy Drew: Codes & Clues (HeR Interactive/Brandissimo!/Picture Fish Studios)

View Article

ACMI's latest virtual-reality commission, Ghosts, Toast and the Things Unsaid, is an immersive, intimate virtual-reality experience that explores the thoughts we don't share, the desires we keep covered and how they affect our relationships.

The 15-minute interactive work, which places you in a room as a ghost – overhearing the private conversations of a man called Steve and a woman called Maude as they fall in love, then grow apart – is collaboration between Google’s Creative Lab, Grumpy Sailor and ACMI X resident, Sandpit.

Entry is strictly limited to 24 tickets per day. Bookings are recommended and limited to two tickets per booking. Online bookings are available one week prior to the performance. Please arrive 10 minutes before the booked time or your spot may be given away.

Read Article

Dan Koerner is a Creative Director of Sandpit and has a background in live performance, film and digital as a director and creative collaborator.

Sandpit blends design and technology to create captivating experiences in the digital and physical world and has designed multiplatform extensions for feature films 52 Tuesdays, The Boy Castaways and has directed an interactive audio tour experience I, Animal for Melbourne Zoo. Sandpit recently created an interactive phone booth project Dial-A-Story with Penguin Books, and Windmill Theatre on their performance, and feature film, Girl Asleep.

Dan and the Sandpit team have spoken at Transmedia Hollywood in Los Angeles, and the Arts Participation Incubator’s ‘Technologies for Participation’ Seminar at MONA. Sandpit was a participant of Screen Australia’s Multi-Platform Clinic, and Screen Australia and the Australia Council for the Arts’ Hive Lab during the 2012 Melbourne Festival. They are currently artistic directors of the digital theatre initiative - one of two triennially funded programs by the Australia Council. Sandpit is a founding tenant of The Australian Centre for the Moving Image’s ACMI X co-working space.

This year Sandpit created Ghosts, Toast and the Things Unsaid, a collaboration with Google’s Creative Lab. Unsurprisingly for Sandpit this involved an interactive, digitally augmented ghost sheet.

Seb Chan describes his career as ‘happy accidents’ seizing opportunities and working with really smart people.

It’s taken him from Powerhouse Museum in Sydney, to Cooper Hewitt Smithsonian Design Museum in New York and now chief experience officer at Australian Centre for the Moving Image (ACMI) in Melbourne where they recently opened ACMI X studio for the creative industries.

In Wellington for the National Digital Forum, Seb tells us about holistic visitor-centric design strategy and how more than ever communities need places to convene, feel safe and be challenged by ideas, creativity and art.

“Museums need to step up their game in this regard – and as keepers of humanity’s communal past and with a unique ability to present the connections between the present and the past - we need to be a beacon and not retreat from a globally connected world.”

What aspect of your creative work gives you the biggest thrill?

Seeing things that I’ve been involved in making happen being used, played with, and experienced. I like to put an emphasis on ‘launching’ so that moment of people actually getting hands-on with what you’re made comes as early in the process as possible – then iterating and evolving in response to all the inevitable challenges that emerge.

What's your number one business tip for surviving (and thriving) in the creative sector?

Staying creative. Its really important to remember that you’re in the creative sector because of the ‘creative’ part – and its equally easy to lose sight of that and get caught in the treadmill of business operations and logistics that are no different to working in other sectors. And right now in these suddenly more uncertain times, its really important to watch your own mental health. Take care of yourself, and take care of those around you.

What’s your connection with NZ?

I was born in NZ and travelled on a NZ passport right up until 2011! My mum is from Wellington and my dad was a Chinese migrant to Auckland – so even though I grew up in Australia I’ve always had a relationship with NZ. I love the way NZ just gets on with things and manages to use its distance and smaller scale to its advantage and incubates some of the most interesting ideas.

Tell us a bit about your creative and digital background.

Happy accidents mostly. The usual “Generation X kid interested in computers and music discovers the Internet and WWW in 1994” story. After that it has been about seizing opportunities and working with really smart people – and timing. The work I was doing at the Powerhouse Museum around the early use of the social web and open access content in museums started small in 2005 and then grew from there to a point that by 2008 I was advising many different institutions around the world. I first spoke at NDF in 2006 about the work we were doing with ‘social tagging’. Since then I lived in New York for four years helping redesign and rebuild the Cooper Hewitt Design Museum for the Smithsonian.

Tell us about your role as Chief Experience Officer at the Australian Centre for the Moving Image.

I’ve just ticked over the first year in the role and we are collectively trying to negotiate the borders of the role. It really touches almost all the areas of the museum and in the last year I’ve been involved in such a wide range of things from wayfinding and building navigation to audio guides; exhibition design to getting video games back on a more equal footing with film inside the institution. At a strategic level I’ve also been helping the institution rethink what it is, who it is for, and what it does – which is the sort of work that Executive teams always need far more time to do. You’re going to see ACMI change a lot over the next 4 years and really seize the mantle of being the ‘National Museum of Film, TV, Video Games, Digital Culture and Art'.

What is holistic visitor-centric design strategy and why is it important?

A design strategy is about how we deliver the mission of the museum in a way that is coherent and legible to its visitors, collaborators and stakeholders. Design in museums used to be relgated to the marketing department or, if there was an internal team for exhibition design, share with them. A ‘holistic visitor-centric design strategy’ is about elevating above the operational needs of ‘visual design outputs’ and thinking about how visitors ‘use’ the museum from its galleries and café and shop, to its online and educational activities; undertaking user research and observation, and then designing better systems to deliver the needs of visitors in new, more impactful ways. Obviously these days, this means digital transformation but its also much broader than that – which is what keeps it exciting.

Tell us about ACMI X, studio for the creative industries.

ACMI X is the brainchild of our CEO, Katrina Sedgwick. Katrina wanted to develop closer ties between the museum and the industries that it exhibits and the opportunity of an office relocation provided the means to create a 60 seat co-working space inside the museum’s new offices. This means that we have a number of VR startups, artists, online video providers, editors, film makers, and interactive agencies embedded amongst the museum staff. This has already led to some great collaborations and commissions – Sandpit’s Ghosts, Toasts and the Things Unsaid is a unique VR piece that transformed an immersive theatre production into a VR experience in our galleries. We’ve also got postgrad students and researchers from RMIT and University of Melbourne working out of our space as well as a regular program of industry events and talks that now have a venue outside of the public museum space to happen in. Melbourne is quite the hotbed of creative technology in Australia at the moment and its an exciting time to bring producers and makers into the centre of the museum.
What are you talking about at the National Digital Forum?

I’m coming back to NDF after being away for six years and I’ll be talking with Dowse Art Museum Director Courtney Johnston about everything from innovation, change, leadership and ethics. Courtney knows me well and will we will be able to have a very frank and open discussion about the realities facing the cultural sector.

What are some of the current and future challenges and opportunities in your sector?

We’re in a terrible period of turmoil and uncertainty now. And more than ever our communities are going to need places to convene, feel safe, but also be challenged by ideas, creativity and art. Museums need to step up their game in this regard – and as keepers of humanity’s communal past and with a unique ability to present the connections between the present and the past - we need to be a beacon and not retreat from a globally connected world.

What’s your big idea for 2017?

I’m a little bereft of big new ideas in the wake of recent events. Small ideas, widely connected seem to be more sensible proposition right now. That and an ongoing commitment to social justice.

About Seb Chan

Seb Chan is the Chief Experience Officer (CXO) at ACMI (Australian Centre for the Moving Image) where he is responsible for creating and implementing a holistic, multi-channel, visitor-centred design strategy. Until August 2015, he was Director of Digital & Emerging Media, at Cooper Hewitt Smithsonian Design Museum in New York. There he led the museum’s digital renewal and its transformation into an interactive and playful new museum, which reopened after a three-year rebuilding and reimagining project. His team’s work won awards from the American Association of Museums, Museums and the Web, One Club and D&AD. Their work also featured in Slate, The Verge, and Fast Company. Seb introduced new thinking to the Smithsonian Design Museum in the area of acquiring digital media and software, notably acquiring the iOS App Planetary – the first App to enter the
Smithsonian’s collection.

Prior to relocating to New York, Seb was Head of Digital, Social & Emerging Technologies at the Powerhouse Museum in Sydney, where he led teams responsible for the Powerhouse’s pioneering work throughout the ’00s. This work encompassed open access, mass collaboration and digital engagement, as well as large scale Australian cross-agency projects. He has also worked as cultural sector consultant with organisations across the world and was the co-author of Culture24′s influential Lets Get Real action research project in 2011 and 2013, which helped evolve data-informed digital decision making in the UK performing arts and heritage sectors. Seb serves on several non-profit boards, and is a regular speaker at digital and cultural sector conferences and events. He has also led a parallel life in electronic art and music – organising and curating both festivals and international touring. He was also the founding editor-in-chief for a long running music magazine. In his spare time, Seb enjoys overly sweet dessert wines, and high grade chocolate.

It's Halloween - the spookiest time of the year! To celebrate, we partnered with Sandpit to install an all consuming Virtual Reality experience, Ghosts, Toast and the Things Unsaid.

In this ACMI Lowdown Podcast, Shelley Matulick, Content Producer, speaks to the work's creator and director, Dan Koerner.