Skip to main content

Tensorflow.js React App

Mong Kok, Hongkong

TensorFlow.js is a library for machine learning in JavaScript. Develop ML models in JavaScript, and use ML directly in the browser or in Node.js.

A Tutorial based on the [Tensorflow Object Detection Tutorial] and the Tensorflow.js Code Example by @nicknochnack.

Steps​

ls -la public/model

   4096 Jan  8 22:11 .
   4096 Jan  8 22:11 ..
4194304 Jan  8 20:54 group1-shard1of3.bin
4194304 Jan  8 20:54 group1-shard2of3.bin
3356824 Jan  8 20:54 group1-shard3of3.bin
 467951 Jan  8 20:54 model.json
  • Step 5. Link the model into your src/App.js:
const net = await tf.loadGraphModel('http://localhost:3000/model/model.json')

Note this only works when you access the App via http://localhost:3000/. You can replace localhost with the IP address of your machine - but you will get an CORS Error if you try to access it. The clean solution would be to stash your model on a server where you can adjust the webserver to allow resources from different IPs (this path is described in the Youtube video linked above). But this way is good enough for testing.

  • Step 6. Edit src/utilities.js to add your labels - in my case I only have one labeled object in my Tensorflow model. Add all of yours inside the Label Map:
const labelMap = {
    1:{name:'INSTAR', color:'red'}
}
  • Step 7. Fix the bounding box - see Github Issue
  • Step 8. Run the App npm start
  • Step 9. The Tensorflow detection returns an Object that contains a lot of different data. Find the correct one by incrementing console.log(await obj[0].array()) in src/App.js and check the output:

For me the position was 1 for confidence scores, 2 for object classes and 4 for the corner position of the boundary box - adjust these values accordingly in src/App.js

const boxes = await obj[3].array()
const classes = await obj[1].array()
const scores = await obj[0].array()

Scores

console.log(await obj[0].array())

Tensorflow.js Object Detection React Application

Classes

console.log(await obj[1].array())

Tensorflow.js Object Detection React Application

Boxes

console.log(await obj[3].array())

Tensorflow.js Object Detection React Application

Workbox Issue​

The Workbox service worker that ships with Create React App overrides the consol.log() output. Simply unregister the SW in your browser and reload the page to get rid of it:

Console -> Application tab -> Service workers -> sw.js unregister

Test your Object Detection​

Tensorflow.js Object Detection React Application