All About TensorFlow.JS Architecture Powered-DermAssist

DermAssist performs on-device image quality inspections using TensorFlow.js.


TensorFlow.js is a JavaScript library for constructing and training machine learning models and delivering them in the browser or on Node.js. In general, it is a free and open-source JavaScript library for training and deploying machine learning models. In addition, the DermAssist online application was established to help people understand skin-related problems better. The programme has been created with ease of use in mind.

AI-powered DermAssist tool

The artificial intelligence-powered dermatologist assist tool is a web-based programme that assists you in determining what may be wrong with your skin. Once the programme is launched, use your phone’s camera to capture three photographs from different skin, hair, or nail concern angles. The AI model evaluates this data and uses its knowledge of 288 criteria to generate a list of possible matching circumstances that you may further investigate.


Sign up for your weekly dose of what's up in emerging technology.

The tool will display dermatologist-reviewed material and frequently asked questions for each matching condition, as well as similar matching photographs from the web. The tool is not designed to provide a diagnosis or replace professional medical advice. Many illnesses require clinician assessment, in-person examination, or additional testing such as a biopsy. Rather than that, it provides access to authoritative information that enables a more educated choice regarding the next move.

Architecture of TensorFlow.js

As the name implies, TensorFlow.js is built on TensorFlow, with a few exceptions for the JS environment. This library includes two distinct sets of APIs:

  • The Ops API makes it possible to do lower-level linear algebra operations like matrix multiplication and tensor addition.
  • Like the Keras API, the Layers API provides developers with high-level model building blocks and best practices for neural networks.

Source: Architecture of TensorFlow.js

Backends for TensorFlow.js

TensorFlow.js uses the concept of backends to support device-specific kernel implementations. 

It now supports three backends: 

  • browser-based, 
  • WebGL-based, and 
  • Node. js-based. 

TensorFlow.js will also support the two emerging web technologies WebAssembly and WebGPU, in the future as a backend. In addition, TensorFlow.js leverages WebGL, a cross-platform web standard that provides low-level 3D graphics APIs, to accelerate parallelized computations. The WebGL backend is the most sophisticated of the three TensorFlow.js backends. The introduction of Node.js and event-driven programming has increased the use of JS in server-side applications over time. Server-side JS has complete access to the filesystem, the native operating system kernel, and all C and C++ libraries. 

To support machine learning’s server-side use cases in JavaScript, TensorFlow.js includes a Node.js backend that connects to the official TensorFlow C API via the N-API. Moreover, TensorFlow.js includes a slower CPU implementation in plain JS as a backup. This fallback can be used in any execution environment and is automatically invoked without WebGL or the TensorFlow binary.

TensorFlow.js Integration with Observables and the Image Capture API

To begin, the researcher wrapped the model ImageQualityPredictor in a wrapper class. This Typescript class exposed just two methods:

  • A static method “createImageQualityPredictor” delivers a promise for an ImageQualityPredictor when provided the model’s URL.
  • A method called “makePrediction” accepts ImageData and returns an array of quality predictions that exceed a specified threshold.


TensorFlow.js has been used in many domains since its inception. Here are some of the paper’s interesting examples:

  1. Gestural Interfaces

TensorFlow.js is used in applications that take webcam inputs. For example, this toolkit is used to build applications that translate sign language to voice, control a web browser with your face, and do real-time facial recognition and pose detection.

  1. Research dissemination

The library has helped ML researchers share their algorithms with others. For example, the Magenta.js package enables in-browser access to generative music models. In addition, using TensorFlow.js on the web has enhanced their work’s visibility among musicians.

  1. Desktop and production applications

JavaScript has also been used to create desktop and production applications. For example, an open-source performance measurement tool, Node Clinic, recently included a TensorFlow.js model to distinguish between user-caused and internal Node.js spikes.

More Great AIM Stories

Dr. Nivash Jeevanandam
Nivash holds a doctorate in information technology and has been a research associate at a university and a development engineer in the IT industry. Data science and machine learning excite him.

Our Upcoming Events

Conference, in-person (Bangalore)
MachineCon 2022
24th Jun

Conference, Virtual
Deep Learning DevCon 2022
30th Jul

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM