To create a functional and attractive user interface, developers need to try to sketch out their ideas on a whiteboard — what a white canvas is to a painter, a whiteboard is that for a UI developer. The entire team of developers strives to create the most feasible, attractive and straightforward user interface for the users. Now, when the developers are done sketching all types of designs and finally stumble upon their champion on the whiteboard, the final sketch is then captured as a photograph and manually translated into working HTML wireframe to operate within a web browser. No matter how fun this process sounds, it takes lots of efforts and of course, delays in the design process. And, that’s when Sketch2Code enters.

The possibility of directly translating the final sketch onto web browser comes like a cool breeze on a hot day to developers, especially during the manual translation of the design into the working HTML wireframe. Sketch2Code is that fresh breeze.

Sketch2Code To The Rescue

Sketch2Code is a web-based AI solution to transform a hand-drawn user interface design from picture to a sound HTML markup code.

The Process

• First, the user needs to create the design/wireframe or a hand-drawn mockup, which is then uploaded on the website.
• Then the next step would be the custom vision algorithm predicting the HTML elements and their placements on the image.
• Spatial information of all bounding boxes of the elements predicted is used by a layout algorithm to generate a grid structure that houses all of these.
• The HTML generation makes use of all the pieces of information it is given and generates an HTML markup code reflecting the result.
• It takes time depending upon the simplicity of the design and then once it is done, the user can download the HTML code.

The Working

Sketch2Code first identifies the design patterns which have been uploaded by the user. It understands the handwritten text, and the structures are drawn, and next, the engine builds the HTML based on the best match. This technology can realise what a UI/UX team have brought on a whiteboard and can translate its understanding to an HTML code.

Now it becomes possible to generate HTML wireframes directly from a hand-drawn image.

Building On the Previous Ink To Code.

Sketch2Code feels like the improved version of Ink to Code which also held the promise of giving you a way to create a prototype for UI even if the user had the limited knowledge of coding. Sketch2Code is more advanced and flexible than Ink to Code. For example —

Ink to code required users to sketch their designs through an exclusive Windows 10 app, then draw symbols to represent text fields and other elements. But, Sketch2Code only needs the handwritten image by the developer to upload it to the website, and the AI takes care of the rest. 

If you want to try out Sketch2Code, its code is available on GitHub.

Not the Novel Attempt at Low/No Code Tools

Microsoft, for a long time, dedicated its research to generate no-code and low-code tools to help people build apps and websites with limited coding knowledge. Microsoft had introduced PowerApps which was a low-coding building platform allowing local coders to construct apps by merely dragging elements.

Microsoft has been the only one, among the tech giants, who has introduced this kind of platforms, however, several startups and Google App Makers also started to offer a low-code building platform for custom business apps.

Outlook

The overall outlook on the situation comes from varied perspectives. For an employee, such a situation would make workings easy but might also replace you in your job, so there is continuous inbuilt fear of AI among the employees. On the other hand, for someone who has been using Microsoft services, Sketch2Code might have a neutral reaction overall — like these Disqus users:

Dinesh Dialani, who thought the app needed more refinement. He said, “Though it seems to be quite interesting; however, when I tried with my design, I didn’t get what I wanted to achieve. And that’s why I believe it might require more refinement.”

Or figuerres (user name) — another Disqus user — remarked: It’s funny, how things go. For instance, Microsoft Expression Blend had a XAML based version of this idea, but then Microsoft killed Expression and Silverlight along with the Expression Web tools; however, some of that stuff should have been kept by Microsoft, as it was perfect.