Automation brings massive advances to laboratory processes, but what happens when the machine makes an error?
In the case of an echo machine, which is used in drug discovery to precisely transfer small volumes of liquid from one container to another using acoustic technology, if the robotic grippers catch too early and the plates are slightly misaligned, the liquid will miss the holes in the wells. The machine, however, is not aware of the error – and human scientists aren’t either, until they collect the data a week later and discover that 20-30% of the wells are unviable.
That’s a big loss – both in terms of data and cost of materials.
During this year’s Hack Week at Recursion, research scientist and software engineer Matthew Viglione decided to apply a technology he had a particular interest in – robotic vision – to catch these machine errors and rerun the tasks without human intervention.
With two lab contractors – Moroni Chalidi and Lindsey Beecher - and two automation engineers – Viglione and Tyler Browning – they got to work installing a simple camera with masking tape that could capture what was happening in the lab in real time and writing code that could teach the machine how to autonomously recover from missteps.
They focused on solving a couple potential issues – identifying when caps were improperly off or on vials, and detecting when cell plates were misaligned. Either one of these errors could lead to significant damage and costs, Viglione says.
They started by running a series of tests using the cameras. First, vials with all white caps, and some caps missing, to ensure that the algorithm could detect the missing caps. Then they added additional cap colors – “lab techs will often draw on the vials with Sharpies – there are a lot of things that could throw off the vision algorithm,” Viglione said. They continued to train the algorithm – from correctly identifying that caps were missing to determining which specific caps were missing.
To solve the problem of plate misalignment, they used the attached camera to take extensive photos and videos of echo plate orientation and used those to train the algorithm. “When that alignment is off by even a millimeter, the compound will miss the wells and the cells will die or not receive enough compound,” Viglione says. They included plates with all wells full, with some wells full, with live cells, and with chemicals.
Although misalignment happens just 5% of the time, Viglione says, it has serious consequences. “When it happens, we have to throw away one third of that data because of one little bug. There are 1,500 experiments in a plate – that’s a lot of data.”
Using the camera and algorithm, the machine can be prompted to instead detect the misalignment, reflip the plate, and autonomously recover.
This is the first time computer vision has been applied to improve efficiency in Recursion's automated wet lab -- it’s exciting, says Viglione.
“Hack Week appeals to my entrepreneurial and research side,” says Viglione. “Robotic vision is something I did in grad school – it requires specialized expertise, but once you have it there’s so much you can do.”
The team has already received approval to integrate their capping solution into the Recursion operating system. They are manufacturing camera cases and mounts, and installing them into the part of the RecursionOS that runs the robotics so it can autonomously detect when an error has happened and correct it – no human interaction required.
Author: Brita Belli, Senior Communications Manager at Recursion.
