Digital designers point your eyes at this neat Kickstarter project which is utilising computer vision tech to speed up the early stages of the design process. The basic idea is to give designers a way to quickly transform the sketches in their (paper) notepad into a functioning UI prototype to test work flow and get feedback on early-stage design ideas.
How does it work? The plan is to make an app, called AppSeed, which lets designers use their phone to snap a photo of their design. AppSeed’s computer vision algorithms will then isolate and identify the different UI elements in the sketches, and make them work as intended — so, for instance, a quick sketch of a text box becomes a field where text can be inputted.
AppSeed’s creators are using the open source computer vision library, OpenCV, plus their own algorithms to isolate and identify UI elements. The software will either guess what an element is, based on analysing things like its shape and or relative location. Or prompt the user to specify its function from a list of UI elements if that portion of the design isn’t immediately obvious to its artificial eye.
“The app uses the OpenCV library to identify enclosed shapes and then decides what the UI is based off its location, size and shape (but only when obvious). So an arrow shape in the top left is likely a back button while a series of three short horizontal lines in the same spot is likely a menu button. An empty rectangle that is about 50px tall and very wide can be assumed to be an input text,” explains AppSeed’s Greg Goralski.
“The app makes things into buttons by default and of course the designer can set the UI directly (which is the main way things get set). It uses two techniques, find contour and template matching, to identify shapes and specific elements.”
The focus for AppSeed is squarely on the early stages of UI design — the “initial brainstorming and iterating stage” as Goralski puts it. “The more ideas you have at the start of a project, the better the final product is. We want to make that part easier, faster and more intuitive.”
It’s interesting that an app for digital designers is convinced that paper will continue to play a key role in the process — despite there being no shortage of electronic alternatives for sketching directly into a computer (tablets, smart pens, design software and so on). But for brainstorming early design work, Goralski reckons old school ink is hard to beat.
“We can definitely have digital files brought in directly, but feel that the sketching process with pen and paper will be with us for a long time,” he says. “Pen and paper (along with whiteboard and marker) are so quick and intuitive that they are a natural place to start a project. Digital tools are great, and have been around for a long time now, but pen and paper feels right, is easily collaborated on and is almost always with you.”
The design that’s been transferred from paper to AppSeed will run as a UI prototype within the app but can also be shared as an HTML5 prototype so others can test its work flow without also having to have AppSeed.
There’s currently no full list of UI elements that will be added to the finished app (assuming it gets funded), as the team plan to take feedback from backers to find out what they want, but Goralski says likely additions are: Maps, Streetview, slider, popups, side menus, switches, lists, back button, social media connects, open camera (and other buttons), as well as gestures such as swipe, pinch and tap.
After the AppSeed prototype stage is done and dusted, designs can then be handed off for the next stage of the design process by being sent to Photoshop as a layered file (i.e. with each button or element separated onto its own layer for easier manipulation).
Goralski says the main competitors to the AppSeed concept are POP app and Protosketch but he argues that by bringing computer vision tech into the mix AppSeed is going one better.
“Both [rival apps] take pictures of a sketch to bring it onto the device as we do. The difference is that we then isolate the elements with computer vision. It is this step that gives us the ability to actually make the draw shape into the UI element. This is a powerful difference,” he tells TechCrunch.
At the moment, the Canada-based AppSeed team have a functioning prototype of their app but are looking for a funding bump to turn that into a fully fledged iOS app (with an Android version planned as a stretch goal for the crowdfunding campaign).
They’re seeking CAD$30,000 to build the initial iOS app — and have taken less than half that amount in pledges, with around a third of their 30-day campaign left to go. Pledges for a Kickstarter edition of the app start at $30 $8, ($30 gets you the app plus a sketchbook). They’re aiming to ship the app in January.