33 Commits

Author SHA1 Message Date
71b7b16bdd Add router debugging keys
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-16 09:16:19 -07:00
1936d90550 Fix structure click selection error
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-12 20:22:41 -07:00
8bf74e51ea Cleanup additional tensor in detection worker
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-12 15:25:53 -07:00
21e46713a7 Fix infolink error on chip auto deselection
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-12 15:03:32 -07:00
726d56131c Clean up variable declarations
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-11 20:20:00 -07:00
874901086d Add Structure Class (#203)
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
This PR adds a class for detected structures and two additional classes to handle the coordinates of the bounding boxes for those structures.  The classes do a better job handling conversions between image pixels, canvas pixels, and screen pixels.  This means that they take the place of several chunks of the previous code such as the box2cvs function.

Reviewed-on: #203
2024-10-12 02:28:22 +00:00
a98577e206 Optimize reactive vue data variables
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-05 16:30:33 -07:00
9e90823858 Fix meta tag warning
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-04 18:52:00 -07:00
269e62b5fb Fix scrolling on non-detect pages
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-04 18:51:29 -07:00
1c62f2783c Update help page with clipboard info
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-04 18:31:44 -07:00
9ba3580056 Add clipboard image selection method (#201)
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 1m2s
Closes #200

This pr adds a clipboard button to the image selection popover if the connection secure (clipboard is blocked by most browsers over insecure connection).

Signed-off-by: Justin Georgi <justin.georgi@gmail.com>

Reviewed-on: #201
2024-10-04 16:30:37 +00:00
9415fa3783 Improve infolink visuals
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-10-03 08:50:46 -07:00
966782d8b9 Loop through structure clicks
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-29 12:01:29 -07:00
ce76528958 Change highlight alpha from filter to global alpha
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 36s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-27 12:00:25 -07:00
c3420dbcdf Change structure highlight to css filter (#198)
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 36s
Closes #196

Instead of drawing a bounding box, structures are highlighted when selected by using css fitlers to render non-structure regions of the image in half-opaque grayscale.

Signed-off-by: Justin Georgi <justin.georgi@gmail.com>

Reviewed-on: #198
2024-09-27 17:33:51 +00:00
f32f107078 Disable pinch events on safari main document
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-27 10:31:54 -07:00
43ccf20561 Fix pinch to zoom
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-25 20:04:59 -07:00
8c2a135afb Update help page
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 36s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-25 08:27:03 -07:00
ab6af04e5b Use canvas for image rendering (#197)
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Closes #194, Closes #189

Instead of making the detection image the background of the canvas element the image is now drawn as part of the canvas.  This enables pan and zoom of the image as well.

Reviewed-on: #197
2024-09-25 15:05:00 +00:00
8ba930ed2e Remove run attempt from build number
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 36s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-12 19:55:30 -07:00
d2ee45c61a Add build info to specs page
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-12 19:47:09 -07:00
e4a3d1ab46 Remove safari detection
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-12 18:56:03 -07:00
862773d622 Upgrade tensorflowjs
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-11 16:32:55 -07:00
47ec235cfa Clean up new worker configuration
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 39s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-11 11:46:37 -07:00
dcdde0289b Set worker use as configuration setting
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-11 10:55:26 -07:00
390faf0a29 Replace url parse with new url for safari
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-09-10 17:41:20 -07:00
523b50ec65 Cleanup for 0.5.0 alpha release
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 36s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-08-21 15:48:55 -07:00
f35b28a7fb Parse model urls for full generalization
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 42s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-08-21 14:44:45 -07:00
94995a7a74 Enable vite preview script
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-08-21 11:55:55 -07:00
daf17bcdff Remove model root in favor of relative urls
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-08-20 17:34:46 -07:00
56a6d85f75 Get better model root using import.meta
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 37s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-08-20 17:06:47 -07:00
46b5ba7d6e Fix root of model urls
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 36s
Signed-off-by: Justin Georgi <justin.georgi@gmail.com>
2024-08-15 16:48:50 -07:00
401e5831c7 Fallback to non worker tfjs when on Safari (#193)
All checks were successful
Build Dev PWA / Build-PWA (push) Successful in 38s
Safari's worker limitations mean that detection threads in the worker barely function. Until Apple quits being whiny jerks about PWAs, this workaround is required to bypass the message calls to the workers and use the old single threaded system when Safari is detected.

Reviewed-on: #193
2024-08-15 22:43:19 +00:00
25 changed files with 919 additions and 339 deletions

View File

@@ -13,6 +13,8 @@ jobs:
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Install node modules - name: Install node modules
run: npm install run: npm install
- name: Add build number
run: sed -i 's/####/#${{ github.run_number }}/' ./src/js/store.js
- name: Build pwa - name: Build pwa
run: npm run build run: npm run build
- name: Replace previous dev pwa - name: Replace previous dev pwa

5
.gitignore vendored
View File

@@ -40,7 +40,8 @@ cordova/platforms/
cordova/plugins/ cordova/plugins/
cordova/www/ cordova/www/
# Production build # Production build
www/ www/
# VSCode settings
.vscode/settings.json

View File

@@ -47,9 +47,10 @@ The following site settings are avaible:
| name | description | values | default | | name | description | values | default |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `agreeExpire` | number of months before users are shown the site agreement dialog again<br />set to 0 to display dialog on every reload | integer >= 0 | 3 | | `agreeExpire` | number of months before users are shown the site agreement dialog again<br />set to 0 to display dialog on every reload | integer >= 0 | 3 |
| `demo` | set to **true** to enable demo mode by default | boolean | false | `demo` | set to **true** to enable demo mode by default | boolean | false |
| `regions` | array of regions names to enable | thorax, abdomen, limbs, head | [thorax, abdomen, limbs, head] | | `regions` | array of regions names to enable | thorax, abdomen, limbs, head | [thorax, abdomen, limbs, head] |
| `useExternal` | detemines the ability to use an external detection server:<br />**none** - external server cannot be configured<br />**optional** - external server can be configured in the app's settings page<br />**list** - external server can be selected in the app's settings page but only the configured server(s) may be selected<br />**required** - external server settings from conf file will be used by default and disable server options in the settings page | none, optional, list, required | **optional** | | `useExternal` | detemines the ability to use an external detection server:<br />**none** - external server cannot be configured<br />**optional** - external server can be configured in the app's settings page<br />**list** - external server can be selected in the app's settings page but only the configured server(s) may be selected<br />**required** - external server settings from conf file will be used by default and disable server options in the settings page | none, optional, list, required | **optional** |
| `disableWorkers` | force app to use a single thread for detection computations instead of multi threading web workers | boolean | **optional** |
| `external` | properties of the external server(s) ALVINN may connect to<br />This setting must be a single element array if **useExternal** is set to **required**.<br />This setting must be an array of one or more elements if **useExternal** is set to **list** | external server settings array | []| | `external` | properties of the external server(s) ALVINN may connect to<br />This setting must be a single element array if **useExternal** is set to **required**.<br />This setting must be an array of one or more elements if **useExternal** is set to **list** | external server settings array | []|
| `infoUrl` | root url for links to information about identified structures<br />Structure labels with spaces replaced by underscores will be appended to this value for full information links (*e.g.,* Abdominal_diapragm) | string | info link not shown | | `infoUrl` | root url for links to information about identified structures<br />Structure labels with spaces replaced by underscores will be appended to this value for full information links (*e.g.,* Abdominal_diapragm) | string | info link not shown |

View File

@@ -1,5 +1,5 @@
<?xml version='1.0' encoding='utf-8'?> <?xml version='1.0' encoding='utf-8'?>
<widget id="edu.midwestern.alvinn" version="0.5.0-rc" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0" xmlns:android="http://schemas.android.com/apk/res/android"> <widget id="edu.midwestern.alvinn" version="0.5.0-alpha" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0" xmlns:android="http://schemas.android.com/apk/res/android">
<name>ALVINN</name> <name>ALVINN</name>
<description>Anatomy Lab Visual Identification Neural Network.</description> <description>Anatomy Lab Visual Identification Neural Network.</description>
<author email="jgeorg@midwestern.edu" href="https://midwestern.edu"> <author email="jgeorg@midwestern.edu" href="https://midwestern.edu">

View File

@@ -1,7 +1,7 @@
{ {
"name": "edu.midwestern.alvinn", "name": "edu.midwestern.alvinn",
"displayName": "ALVINN", "displayName": "ALVINN",
"version": "0.5.0-rc", "version": "0.5.0-alpha",
"description": "Anatomy Lab Visual Identification Neural Network.", "description": "Anatomy Lab Visual Identification Neural Network.",
"main": "index.js", "main": "index.js",
"scripts": { "scripts": {

173
package-lock.json generated
View File

@@ -1,16 +1,16 @@
{ {
"name": "alvinn", "name": "alvinn",
"version": "0.5.0-rc", "version": "0.5.0-alpha",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "alvinn", "name": "alvinn",
"version": "0.5.0-rc", "version": "0.5.0-alpha",
"hasInstallScript": true, "hasInstallScript": true,
"license": "UNLICENSED", "license": "UNLICENSED",
"dependencies": { "dependencies": {
"@tensorflow/tfjs": "^4.17.0", "@tensorflow/tfjs": "^4.21.0",
"dom7": "^4.0.6", "dom7": "^4.0.6",
"framework7": "^8.3.0", "framework7": "^8.3.0",
"framework7-icons": "^5.0.5", "framework7-icons": "^5.0.5",
@@ -3354,16 +3354,17 @@
} }
}, },
"node_modules/@tensorflow/tfjs": { "node_modules/@tensorflow/tfjs": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs/-/tfjs-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs/-/tfjs-4.21.0.tgz",
"integrity": "sha512-yXRBhpM3frlNA/YaPp6HNk9EfIi8han5RYeQA3R8OCa0Od+AfoG1PUmlxV8fE2wCorlGVyHsgpiJ6M9YZPB56w==", "integrity": "sha512-7D/+H150ptvt+POMbsME3WlIvLiuBR2rCC2Z0hOKKb/5Ygkj7xsp/K2HzOvUj0g0yjk+utkU45QEYhnhjnbHRA==",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@tensorflow/tfjs-backend-cpu": "4.17.0", "@tensorflow/tfjs-backend-cpu": "4.21.0",
"@tensorflow/tfjs-backend-webgl": "4.17.0", "@tensorflow/tfjs-backend-webgl": "4.21.0",
"@tensorflow/tfjs-converter": "4.17.0", "@tensorflow/tfjs-converter": "4.21.0",
"@tensorflow/tfjs-core": "4.17.0", "@tensorflow/tfjs-core": "4.21.0",
"@tensorflow/tfjs-data": "4.17.0", "@tensorflow/tfjs-data": "4.21.0",
"@tensorflow/tfjs-layers": "4.17.0", "@tensorflow/tfjs-layers": "4.21.0",
"argparse": "^1.0.10", "argparse": "^1.0.10",
"chalk": "^4.1.0", "chalk": "^4.1.0",
"core-js": "3.29.1", "core-js": "3.29.1",
@@ -3375,9 +3376,10 @@
} }
}, },
"node_modules/@tensorflow/tfjs-backend-cpu": { "node_modules/@tensorflow/tfjs-backend-cpu": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-cpu/-/tfjs-backend-cpu-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-cpu/-/tfjs-backend-cpu-4.21.0.tgz",
"integrity": "sha512-2VSCHnX9qhYTjw9HiVwTBSnRVlntKXeBlK7aSVsmZfHGwWE2faErTtO7bWmqNqw0U7gyznJbVAjlow/p+0RNGw==", "integrity": "sha512-yS9Oisg4L48N7ML6677ilv1eP5Jt59S74skSU1cCsM4yBAtH4DAn9b89/JtqBISh6JadanfX26b4HCWQvMvqFg==",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@types/seedrandom": "^2.4.28", "@types/seedrandom": "^2.4.28",
"seedrandom": "^3.0.5" "seedrandom": "^3.0.5"
@@ -3386,15 +3388,16 @@
"yarn": ">= 1.3.2" "yarn": ">= 1.3.2"
}, },
"peerDependencies": { "peerDependencies": {
"@tensorflow/tfjs-core": "4.17.0" "@tensorflow/tfjs-core": "4.21.0"
} }
}, },
"node_modules/@tensorflow/tfjs-backend-webgl": { "node_modules/@tensorflow/tfjs-backend-webgl": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-webgl/-/tfjs-backend-webgl-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-webgl/-/tfjs-backend-webgl-4.21.0.tgz",
"integrity": "sha512-CC5GsGECCd7eYAUaKq0XJ48FjEZdgXZWPxgUYx4djvfUx5fQPp35hCSP9w/k463jllBMbjl2tKRg8u7Ia/LYzg==", "integrity": "sha512-7k6mb7dd0uF9jI51iunF3rhEXjvR/a613kjWZ0Rj3o1COFrneyku2C7cRMZERWPhbgXZ+dF+j9MdpGIpgtShIQ==",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@tensorflow/tfjs-backend-cpu": "4.17.0", "@tensorflow/tfjs-backend-cpu": "4.21.0",
"@types/offscreencanvas": "~2019.3.0", "@types/offscreencanvas": "~2019.3.0",
"@types/seedrandom": "^2.4.28", "@types/seedrandom": "^2.4.28",
"seedrandom": "^3.0.5" "seedrandom": "^3.0.5"
@@ -3403,21 +3406,23 @@
"yarn": ">= 1.3.2" "yarn": ">= 1.3.2"
}, },
"peerDependencies": { "peerDependencies": {
"@tensorflow/tfjs-core": "4.17.0" "@tensorflow/tfjs-core": "4.21.0"
} }
}, },
"node_modules/@tensorflow/tfjs-converter": { "node_modules/@tensorflow/tfjs-converter": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-converter/-/tfjs-converter-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-converter/-/tfjs-converter-4.21.0.tgz",
"integrity": "sha512-qFxIjPfomCuTrYxsFjtKbi3QfdmTTCWo+RvqD64oCMS0sjp7sUDNhJyKDoLx6LZhXlwXpHIVDJctLMRMwet0Zw==", "integrity": "sha512-cUhU+F1lGx2qnKk/gRy8odBh0PZlFz0Dl71TG8LVnj0/g352DqiNrKXlKO/po9aWzP8x0KUGC3gNMSMJW+T0DA==",
"license": "Apache-2.0",
"peerDependencies": { "peerDependencies": {
"@tensorflow/tfjs-core": "4.17.0" "@tensorflow/tfjs-core": "4.21.0"
} }
}, },
"node_modules/@tensorflow/tfjs-core": { "node_modules/@tensorflow/tfjs-core": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-core/-/tfjs-core-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-core/-/tfjs-core-4.21.0.tgz",
"integrity": "sha512-v9Q5430EnRpyhWNd9LVgXadciKvxLiq+sTrLKRowh26BHyAsams4tZIgX3lFKjB7b90p+FYifVMcqLTTHgjGpQ==", "integrity": "sha512-ZbECwXps5wb9XXcGq4ZXvZDVjr5okc3I0+i/vU6bpQ+nVApyIrMiyEudP8f6vracVTvNmnlN62vUXoEsQb2F8g==",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@types/long": "^4.0.1", "@types/long": "^4.0.1",
"@types/offscreencanvas": "~2019.7.0", "@types/offscreencanvas": "~2019.7.0",
@@ -3434,28 +3439,31 @@
"node_modules/@tensorflow/tfjs-core/node_modules/@types/offscreencanvas": { "node_modules/@tensorflow/tfjs-core/node_modules/@types/offscreencanvas": {
"version": "2019.7.3", "version": "2019.7.3",
"resolved": "https://registry.npmjs.org/@types/offscreencanvas/-/offscreencanvas-2019.7.3.tgz", "resolved": "https://registry.npmjs.org/@types/offscreencanvas/-/offscreencanvas-2019.7.3.tgz",
"integrity": "sha512-ieXiYmgSRXUDeOntE1InxjWyvEelZGP63M+cGuquuRLuIKKT1osnkXjxev9B7d1nXSug5vpunx+gNlbVxMlC9A==" "integrity": "sha512-ieXiYmgSRXUDeOntE1InxjWyvEelZGP63M+cGuquuRLuIKKT1osnkXjxev9B7d1nXSug5vpunx+gNlbVxMlC9A==",
"license": "MIT"
}, },
"node_modules/@tensorflow/tfjs-data": { "node_modules/@tensorflow/tfjs-data": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-data/-/tfjs-data-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-data/-/tfjs-data-4.21.0.tgz",
"integrity": "sha512-aPKrDFip+gXicWOFALeNT7KKQjRXFkHd/hNe/zs4mCFcIN00hy1PkZ6xkYsgrsdLDQMBSGeS4B4ZM0k5Cs88QA==", "integrity": "sha512-LpJ/vyQMwYHkcVCqIRg7IVVw13VBY7rNAiuhmKP9S5NP/2ye4KA8BJ4XwDIDgjCVQM7glK9L8bMav++xCDf7xA==",
"license": "Apache-2.0",
"dependencies": { "dependencies": {
"@types/node-fetch": "^2.1.2", "@types/node-fetch": "^2.1.2",
"node-fetch": "~2.6.1", "node-fetch": "~2.6.1",
"string_decoder": "^1.3.0" "string_decoder": "^1.3.0"
}, },
"peerDependencies": { "peerDependencies": {
"@tensorflow/tfjs-core": "4.17.0", "@tensorflow/tfjs-core": "4.21.0",
"seedrandom": "^3.0.5" "seedrandom": "^3.0.5"
} }
}, },
"node_modules/@tensorflow/tfjs-layers": { "node_modules/@tensorflow/tfjs-layers": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-layers/-/tfjs-layers-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-layers/-/tfjs-layers-4.21.0.tgz",
"integrity": "sha512-DEE0zRKvf3LJ0EcvG5XouJYOgFGWYAneZ0K1d23969z7LfSyqVmBdLC6BTwdLKuJk3ouUJIKXU1TcpFmjDuh7g==", "integrity": "sha512-a8KaMYlY3+llvE9079nvASKpaaf8xpCMdOjbgn+eGhdOGOcY7QuFUkd/2odvnXDG8fK/jffE1LoNOlfYoBHC4w==",
"license": "Apache-2.0 AND MIT",
"peerDependencies": { "peerDependencies": {
"@tensorflow/tfjs-core": "4.17.0" "@tensorflow/tfjs-core": "4.21.0"
} }
}, },
"node_modules/@tensorflow/tfjs/node_modules/regenerator-runtime": { "node_modules/@tensorflow/tfjs/node_modules/regenerator-runtime": {
@@ -3472,7 +3480,8 @@
"node_modules/@types/long": { "node_modules/@types/long": {
"version": "4.0.2", "version": "4.0.2",
"resolved": "https://registry.npmjs.org/@types/long/-/long-4.0.2.tgz", "resolved": "https://registry.npmjs.org/@types/long/-/long-4.0.2.tgz",
"integrity": "sha512-MqTGEo5bj5t157U6fA/BiDynNkn0YknVdh48CMPkTSpFTVmvao5UQmm7uEF6xBEo7qIMAlY/JSleYaE6VOdpaA==" "integrity": "sha512-MqTGEo5bj5t157U6fA/BiDynNkn0YknVdh48CMPkTSpFTVmvao5UQmm7uEF6xBEo7qIMAlY/JSleYaE6VOdpaA==",
"license": "MIT"
}, },
"node_modules/@types/minimist": { "node_modules/@types/minimist": {
"version": "1.2.5", "version": "1.2.5",
@@ -3492,6 +3501,7 @@
"version": "2.6.11", "version": "2.6.11",
"resolved": "https://registry.npmjs.org/@types/node-fetch/-/node-fetch-2.6.11.tgz", "resolved": "https://registry.npmjs.org/@types/node-fetch/-/node-fetch-2.6.11.tgz",
"integrity": "sha512-24xFj9R5+rfQJLRyM56qh+wnVSYhyXC2tkoBndtY0U+vubqNsYXGjufB2nn8Q6gt0LrARwL6UBtMCSVCwl4B1g==", "integrity": "sha512-24xFj9R5+rfQJLRyM56qh+wnVSYhyXC2tkoBndtY0U+vubqNsYXGjufB2nn8Q6gt0LrARwL6UBtMCSVCwl4B1g==",
"license": "MIT",
"dependencies": { "dependencies": {
"@types/node": "*", "@types/node": "*",
"form-data": "^4.0.0" "form-data": "^4.0.0"
@@ -3506,7 +3516,8 @@
"node_modules/@types/offscreencanvas": { "node_modules/@types/offscreencanvas": {
"version": "2019.3.0", "version": "2019.3.0",
"resolved": "https://registry.npmjs.org/@types/offscreencanvas/-/offscreencanvas-2019.3.0.tgz", "resolved": "https://registry.npmjs.org/@types/offscreencanvas/-/offscreencanvas-2019.3.0.tgz",
"integrity": "sha512-esIJx9bQg+QYF0ra8GnvfianIY8qWB0GBx54PK5Eps6m+xTj86KLavHv6qDhzKcu5UUOgNfJ2pWaIIV7TRUd9Q==" "integrity": "sha512-esIJx9bQg+QYF0ra8GnvfianIY8qWB0GBx54PK5Eps6m+xTj86KLavHv6qDhzKcu5UUOgNfJ2pWaIIV7TRUd9Q==",
"license": "MIT"
}, },
"node_modules/@types/resolve": { "node_modules/@types/resolve": {
"version": "1.17.1", "version": "1.17.1",
@@ -3520,7 +3531,8 @@
"node_modules/@types/seedrandom": { "node_modules/@types/seedrandom": {
"version": "2.4.34", "version": "2.4.34",
"resolved": "https://registry.npmjs.org/@types/seedrandom/-/seedrandom-2.4.34.tgz", "resolved": "https://registry.npmjs.org/@types/seedrandom/-/seedrandom-2.4.34.tgz",
"integrity": "sha512-ytDiArvrn/3Xk6/vtylys5tlY6eo7Ane0hvcx++TKo6RxQXuVfW0AF/oeWqAj9dN29SyhtawuXstgmPlwNcv/A==" "integrity": "sha512-ytDiArvrn/3Xk6/vtylys5tlY6eo7Ane0hvcx++TKo6RxQXuVfW0AF/oeWqAj9dN29SyhtawuXstgmPlwNcv/A==",
"license": "MIT"
}, },
"node_modules/@types/trusted-types": { "node_modules/@types/trusted-types": {
"version": "2.0.6", "version": "2.0.6",
@@ -3646,7 +3658,8 @@
"node_modules/@webgpu/types": { "node_modules/@webgpu/types": {
"version": "0.1.38", "version": "0.1.38",
"resolved": "https://registry.npmjs.org/@webgpu/types/-/types-0.1.38.tgz", "resolved": "https://registry.npmjs.org/@webgpu/types/-/types-0.1.38.tgz",
"integrity": "sha512-7LrhVKz2PRh+DD7+S+PVaFd5HxaWQvoMqBbsV9fNJO1pjUs1P8bM2vQVNfk+3URTqbuTI7gkXi0rfsN0IadoBA==" "integrity": "sha512-7LrhVKz2PRh+DD7+S+PVaFd5HxaWQvoMqBbsV9fNJO1pjUs1P8bM2vQVNfk+3URTqbuTI7gkXi0rfsN0IadoBA==",
"license": "BSD-3-Clause"
}, },
"node_modules/acorn": { "node_modules/acorn": {
"version": "8.11.2", "version": "8.11.2",
@@ -3814,7 +3827,8 @@
"node_modules/asynckit": { "node_modules/asynckit": {
"version": "0.4.0", "version": "0.4.0",
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==" "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
"license": "MIT"
}, },
"node_modules/at-least-node": { "node_modules/at-least-node": {
"version": "1.0.0", "version": "1.0.0",
@@ -4412,6 +4426,7 @@
"version": "1.0.8", "version": "1.0.8",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
"license": "MIT",
"dependencies": { "dependencies": {
"delayed-stream": "~1.0.0" "delayed-stream": "~1.0.0"
}, },
@@ -4872,6 +4887,7 @@
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==", "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
"license": "MIT",
"engines": { "engines": {
"node": ">=0.4.0" "node": ">=0.4.0"
} }
@@ -5357,6 +5373,7 @@
"version": "4.0.0", "version": "4.0.0",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz", "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz",
"integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==", "integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==",
"license": "MIT",
"dependencies": { "dependencies": {
"asynckit": "^0.4.0", "asynckit": "^0.4.0",
"combined-stream": "^1.0.8", "combined-stream": "^1.0.8",
@@ -6577,7 +6594,8 @@
"node_modules/long": { "node_modules/long": {
"version": "4.0.0", "version": "4.0.0",
"resolved": "https://registry.npmjs.org/long/-/long-4.0.0.tgz", "resolved": "https://registry.npmjs.org/long/-/long-4.0.0.tgz",
"integrity": "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA==" "integrity": "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA==",
"license": "Apache-2.0"
}, },
"node_modules/lower-case": { "node_modules/lower-case": {
"version": "2.0.2", "version": "2.0.2",
@@ -6688,6 +6706,7 @@
"version": "1.52.0", "version": "1.52.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
"license": "MIT",
"engines": { "engines": {
"node": ">= 0.6" "node": ">= 0.6"
} }
@@ -6696,6 +6715,7 @@
"version": "2.1.35", "version": "2.1.35",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
"license": "MIT",
"dependencies": { "dependencies": {
"mime-db": "1.52.0" "mime-db": "1.52.0"
}, },
@@ -6823,6 +6843,7 @@
"version": "2.6.13", "version": "2.6.13",
"resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.13.tgz", "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.13.tgz",
"integrity": "sha512-StxNAxh15zr77QvvkmveSQ8uCQ4+v5FkvNTj0OESmiHu+VRi/gXArXtkWMElOsOUNLtUEvI4yS+rdtOHZTwlQA==", "integrity": "sha512-StxNAxh15zr77QvvkmveSQ8uCQ4+v5FkvNTj0OESmiHu+VRi/gXArXtkWMElOsOUNLtUEvI4yS+rdtOHZTwlQA==",
"license": "MIT",
"dependencies": { "dependencies": {
"whatwg-url": "^5.0.0" "whatwg-url": "^5.0.0"
}, },
@@ -6841,17 +6862,20 @@
"node_modules/node-fetch/node_modules/tr46": { "node_modules/node-fetch/node_modules/tr46": {
"version": "0.0.3", "version": "0.0.3",
"resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
"integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==" "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==",
"license": "MIT"
}, },
"node_modules/node-fetch/node_modules/webidl-conversions": { "node_modules/node-fetch/node_modules/webidl-conversions": {
"version": "3.0.1", "version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==" "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==",
"license": "BSD-2-Clause"
}, },
"node_modules/node-fetch/node_modules/whatwg-url": { "node_modules/node-fetch/node_modules/whatwg-url": {
"version": "5.0.0", "version": "5.0.0",
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
"integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==", "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==",
"license": "MIT",
"dependencies": { "dependencies": {
"tr46": "~0.0.3", "tr46": "~0.0.3",
"webidl-conversions": "^3.0.0" "webidl-conversions": "^3.0.0"
@@ -8481,7 +8505,8 @@
"node_modules/seedrandom": { "node_modules/seedrandom": {
"version": "3.0.5", "version": "3.0.5",
"resolved": "https://registry.npmjs.org/seedrandom/-/seedrandom-3.0.5.tgz", "resolved": "https://registry.npmjs.org/seedrandom/-/seedrandom-3.0.5.tgz",
"integrity": "sha512-8OwmbklUNzwezjGInmZ+2clQmExQPvomqjL7LFqOYqtmuxRgQYqOD3mHaU+MvZn5FLUeVxVfQjwLZW/n/JFuqg==" "integrity": "sha512-8OwmbklUNzwezjGInmZ+2clQmExQPvomqjL7LFqOYqtmuxRgQYqOD3mHaU+MvZn5FLUeVxVfQjwLZW/n/JFuqg==",
"license": "MIT"
}, },
"node_modules/semver": { "node_modules/semver": {
"version": "6.3.1", "version": "6.3.1",
@@ -11858,16 +11883,16 @@
} }
}, },
"@tensorflow/tfjs": { "@tensorflow/tfjs": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs/-/tfjs-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs/-/tfjs-4.21.0.tgz",
"integrity": "sha512-yXRBhpM3frlNA/YaPp6HNk9EfIi8han5RYeQA3R8OCa0Od+AfoG1PUmlxV8fE2wCorlGVyHsgpiJ6M9YZPB56w==", "integrity": "sha512-7D/+H150ptvt+POMbsME3WlIvLiuBR2rCC2Z0hOKKb/5Ygkj7xsp/K2HzOvUj0g0yjk+utkU45QEYhnhjnbHRA==",
"requires": { "requires": {
"@tensorflow/tfjs-backend-cpu": "4.17.0", "@tensorflow/tfjs-backend-cpu": "4.21.0",
"@tensorflow/tfjs-backend-webgl": "4.17.0", "@tensorflow/tfjs-backend-webgl": "4.21.0",
"@tensorflow/tfjs-converter": "4.17.0", "@tensorflow/tfjs-converter": "4.21.0",
"@tensorflow/tfjs-core": "4.17.0", "@tensorflow/tfjs-core": "4.21.0",
"@tensorflow/tfjs-data": "4.17.0", "@tensorflow/tfjs-data": "4.21.0",
"@tensorflow/tfjs-layers": "4.17.0", "@tensorflow/tfjs-layers": "4.21.0",
"argparse": "^1.0.10", "argparse": "^1.0.10",
"chalk": "^4.1.0", "chalk": "^4.1.0",
"core-js": "3.29.1", "core-js": "3.29.1",
@@ -11883,35 +11908,35 @@
} }
}, },
"@tensorflow/tfjs-backend-cpu": { "@tensorflow/tfjs-backend-cpu": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-cpu/-/tfjs-backend-cpu-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-cpu/-/tfjs-backend-cpu-4.21.0.tgz",
"integrity": "sha512-2VSCHnX9qhYTjw9HiVwTBSnRVlntKXeBlK7aSVsmZfHGwWE2faErTtO7bWmqNqw0U7gyznJbVAjlow/p+0RNGw==", "integrity": "sha512-yS9Oisg4L48N7ML6677ilv1eP5Jt59S74skSU1cCsM4yBAtH4DAn9b89/JtqBISh6JadanfX26b4HCWQvMvqFg==",
"requires": { "requires": {
"@types/seedrandom": "^2.4.28", "@types/seedrandom": "^2.4.28",
"seedrandom": "^3.0.5" "seedrandom": "^3.0.5"
} }
}, },
"@tensorflow/tfjs-backend-webgl": { "@tensorflow/tfjs-backend-webgl": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-webgl/-/tfjs-backend-webgl-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-webgl/-/tfjs-backend-webgl-4.21.0.tgz",
"integrity": "sha512-CC5GsGECCd7eYAUaKq0XJ48FjEZdgXZWPxgUYx4djvfUx5fQPp35hCSP9w/k463jllBMbjl2tKRg8u7Ia/LYzg==", "integrity": "sha512-7k6mb7dd0uF9jI51iunF3rhEXjvR/a613kjWZ0Rj3o1COFrneyku2C7cRMZERWPhbgXZ+dF+j9MdpGIpgtShIQ==",
"requires": { "requires": {
"@tensorflow/tfjs-backend-cpu": "4.17.0", "@tensorflow/tfjs-backend-cpu": "4.21.0",
"@types/offscreencanvas": "~2019.3.0", "@types/offscreencanvas": "~2019.3.0",
"@types/seedrandom": "^2.4.28", "@types/seedrandom": "^2.4.28",
"seedrandom": "^3.0.5" "seedrandom": "^3.0.5"
} }
}, },
"@tensorflow/tfjs-converter": { "@tensorflow/tfjs-converter": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-converter/-/tfjs-converter-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-converter/-/tfjs-converter-4.21.0.tgz",
"integrity": "sha512-qFxIjPfomCuTrYxsFjtKbi3QfdmTTCWo+RvqD64oCMS0sjp7sUDNhJyKDoLx6LZhXlwXpHIVDJctLMRMwet0Zw==", "integrity": "sha512-cUhU+F1lGx2qnKk/gRy8odBh0PZlFz0Dl71TG8LVnj0/g352DqiNrKXlKO/po9aWzP8x0KUGC3gNMSMJW+T0DA==",
"requires": {} "requires": {}
}, },
"@tensorflow/tfjs-core": { "@tensorflow/tfjs-core": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-core/-/tfjs-core-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-core/-/tfjs-core-4.21.0.tgz",
"integrity": "sha512-v9Q5430EnRpyhWNd9LVgXadciKvxLiq+sTrLKRowh26BHyAsams4tZIgX3lFKjB7b90p+FYifVMcqLTTHgjGpQ==", "integrity": "sha512-ZbECwXps5wb9XXcGq4ZXvZDVjr5okc3I0+i/vU6bpQ+nVApyIrMiyEudP8f6vracVTvNmnlN62vUXoEsQb2F8g==",
"requires": { "requires": {
"@types/long": "^4.0.1", "@types/long": "^4.0.1",
"@types/offscreencanvas": "~2019.7.0", "@types/offscreencanvas": "~2019.7.0",
@@ -11930,9 +11955,9 @@
} }
}, },
"@tensorflow/tfjs-data": { "@tensorflow/tfjs-data": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-data/-/tfjs-data-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-data/-/tfjs-data-4.21.0.tgz",
"integrity": "sha512-aPKrDFip+gXicWOFALeNT7KKQjRXFkHd/hNe/zs4mCFcIN00hy1PkZ6xkYsgrsdLDQMBSGeS4B4ZM0k5Cs88QA==", "integrity": "sha512-LpJ/vyQMwYHkcVCqIRg7IVVw13VBY7rNAiuhmKP9S5NP/2ye4KA8BJ4XwDIDgjCVQM7glK9L8bMav++xCDf7xA==",
"requires": { "requires": {
"@types/node-fetch": "^2.1.2", "@types/node-fetch": "^2.1.2",
"node-fetch": "~2.6.1", "node-fetch": "~2.6.1",
@@ -11940,9 +11965,9 @@
} }
}, },
"@tensorflow/tfjs-layers": { "@tensorflow/tfjs-layers": {
"version": "4.17.0", "version": "4.21.0",
"resolved": "https://registry.npmjs.org/@tensorflow/tfjs-layers/-/tfjs-layers-4.17.0.tgz", "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-layers/-/tfjs-layers-4.21.0.tgz",
"integrity": "sha512-DEE0zRKvf3LJ0EcvG5XouJYOgFGWYAneZ0K1d23969z7LfSyqVmBdLC6BTwdLKuJk3ouUJIKXU1TcpFmjDuh7g==", "integrity": "sha512-a8KaMYlY3+llvE9079nvASKpaaf8xpCMdOjbgn+eGhdOGOcY7QuFUkd/2odvnXDG8fK/jffE1LoNOlfYoBHC4w==",
"requires": {} "requires": {}
}, },
"@types/estree": { "@types/estree": {

View File

@@ -1,7 +1,7 @@
{ {
"name": "alvinn", "name": "alvinn",
"private": true, "private": true,
"version": "0.5.0-rc", "version": "0.5.0-alpha",
"description": "ALVINN", "description": "ALVINN",
"repository": "", "repository": "",
"license": "UNLICENSED", "license": "UNLICENSED",
@@ -14,7 +14,8 @@
"cordova-ios": "cross-env TARGET=cordova cross-env NODE_ENV=production vite build && node ./build/build-cordova.js && cd cordova && cordova run ios", "cordova-ios": "cross-env TARGET=cordova cross-env NODE_ENV=production vite build && node ./build/build-cordova.js && cd cordova && cordova run ios",
"build-cordova-android": "cross-env TARGET=cordova cross-env NODE_ENV=production vite build && node ./build/build-cordova.js && cd cordova && cordova build android", "build-cordova-android": "cross-env TARGET=cordova cross-env NODE_ENV=production vite build && node ./build/build-cordova.js && cd cordova && cordova build android",
"cordova-android": "cross-env TARGET=cordova cross-env NODE_ENV=production vite build && node ./build/build-cordova.js && cd cordova && cordova run android", "cordova-android": "cross-env TARGET=cordova cross-env NODE_ENV=production vite build && node ./build/build-cordova.js && cd cordova && cordova run android",
"postinstall": "cpy --flat ./node_modules/framework7-icons/fonts/*.* ./src/fonts/" "postinstall": "cpy --flat ./node_modules/framework7-icons/fonts/*.* ./src/fonts/",
"preview": "vite preview"
}, },
"browserslist": [ "browserslist": [
"IOS >= 15", "IOS >= 15",
@@ -23,7 +24,7 @@
"last 5 Firefox versions" "last 5 Firefox versions"
], ],
"dependencies": { "dependencies": {
"@tensorflow/tfjs": "^4.17.0", "@tensorflow/tfjs": "^4.21.0",
"dom7": "^4.0.6", "dom7": "^4.0.6",
"framework7": "^8.3.0", "framework7": "^8.3.0",
"framework7-icons": "^5.0.5", "framework7-icons": "^5.0.5",

View File

@@ -6,6 +6,7 @@ regions:
- limbs - limbs
- head - head
useExternal: none useExternal: none
disableWorkers: false
external: external:
- name: Mserver - name: Mserver
address: "192.169.1.105" address: "192.169.1.105"

View File

@@ -5,7 +5,7 @@ let model = null
onmessage = function (e) { onmessage = function (e) {
switch (e.data.call) { switch (e.data.call) {
case 'loadModel': case 'loadModel':
loadModel('.' + e.data.weights,e.data.preload).then(() => { loadModel(e.data.weights,e.data.preload).then(() => {
postMessage({success: 'model'}) postMessage({success: 'model'})
}).catch((err) => { }).catch((err) => {
postMessage({error: true, message: err.message}) postMessage({error: true, message: err.message})
@@ -57,7 +57,7 @@ async function loadModel(weights, preload) {
} }
async function localDetect(imageData) { async function localDetect(imageData) {
console.time('pre-process') console.time('sw: pre-process')
const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3) const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3)
let gTense = null let gTense = null
const input = tf.tidy(() => { const input = tf.tidy(() => {
@@ -65,24 +65,28 @@ async function localDetect(imageData) {
return tf.concat([gTense,gTense,gTense],3) return tf.concat([gTense,gTense,gTense],3)
}) })
tf.dispose(gTense) tf.dispose(gTense)
console.timeEnd('pre-process') console.timeEnd('sw: pre-process')
console.time('run prediction') console.time('sw: run prediction')
const res = model.predict(input) const res = model.predict(input)
const tRes = tf.transpose(res,[0,2,1]) const tRes = tf.transpose(res,[0,2,1])
const rawRes = tRes.arraySync()[0] const rawRes = tRes.arraySync()[0]
console.timeEnd('run prediction') console.timeEnd('sw: run prediction')
console.time('post-process') console.time('sw: post-process')
const outputSize = res.shape[1] const outputSize = res.shape[1]
const output = {
detections: []
}
let rawBoxes = [] let rawBoxes = []
let rawScores = [] let rawScores = []
let getScores, getBox, boxCalc
for (var i = 0; i < rawRes.length; i++) { for (let i = 0; i < rawRes.length; i++) {
var getScores = rawRes[i].slice(4) getScores = rawRes[i].slice(4)
if (getScores.every( s => s < .05)) { continue } if (getScores.every( s => s < .05)) { continue }
var getBox = rawRes[i].slice(0,4) getBox = rawRes[i].slice(0,4)
var boxCalc = [ boxCalc = [
(getBox[0] - (getBox[2] / 2)) / modelWidth, (getBox[0] - (getBox[2] / 2)) / modelWidth,
(getBox[1] - (getBox[3] / 2)) / modelHeight, (getBox[1] - (getBox[3] / 2)) / modelHeight,
(getBox[0] + (getBox[2] / 2)) / modelWidth, (getBox[0] + (getBox[2] / 2)) / modelWidth,
@@ -101,7 +105,7 @@ async function localDetect(imageData) {
let boxes_data = [] let boxes_data = []
let scores_data = [] let scores_data = []
let classes_data = [] let classes_data = []
for (var c = 0; c < outputSize - 4; c++) { for (let c = 0; c < outputSize - 4; c++) {
structureScores = rawScores.map(x => x[c]) structureScores = rawScores.map(x => x[c])
tScores = tf.tensor1d(structureScores) tScores = tf.tensor1d(structureScores)
resBoxes = await tf.image.nonMaxSuppressionAsync(tBoxes,tScores,10,0.5,.05) resBoxes = await tf.image.nonMaxSuppressionAsync(tBoxes,tScores,10,0.5,.05)
@@ -109,7 +113,7 @@ async function localDetect(imageData) {
tf.dispose(resBoxes) tf.dispose(resBoxes)
if (validBoxes) { if (validBoxes) {
boxes_data.push(...rawBoxes.filter( (_, idx) => validBoxes.includes(idx))) boxes_data.push(...rawBoxes.filter( (_, idx) => validBoxes.includes(idx)))
var outputScores = structureScores.filter( (_, idx) => validBoxes.includes(idx)) let outputScores = structureScores.filter( (_, idx) => validBoxes.includes(idx))
scores_data.push(...outputScores) scores_data.push(...outputScores)
classes_data.push(...outputScores.fill(c)) classes_data.push(...outputScores.fill(c))
} }
@@ -119,18 +123,15 @@ async function localDetect(imageData) {
tf.dispose(tBoxes) tf.dispose(tBoxes)
tf.dispose(tScores) tf.dispose(tScores)
tf.dispose(tRes) tf.dispose(tRes)
tf.dispose(resBoxes)
const valid_detections_data = classes_data.length const valid_detections_data = classes_data.length
var output = { for (let i =0; i < valid_detections_data; i++) {
detections: [] let [dLeft, dTop, dRight, dBottom] = boxes_data[i]
}
for (var i =0; i < valid_detections_data; i++) {
var [dLeft, dTop, dRight, dBottom] = boxes_data[i]
output.detections.push({ output.detections.push({
"top": dTop, "top": dTop,
"left": dLeft, "left": dLeft,
"bottom": dBottom, "bottom": dBottom,
"right": dRight, "right": dRight,
// "label": this.detectorLabels[classes_data[i]].name,
"label": classes_data[i], "label": classes_data[i],
"confidence": scores_data[i] * 100 "confidence": scores_data[i] * 100
}) })
@@ -138,14 +139,14 @@ async function localDetect(imageData) {
} }
tf.dispose(res) tf.dispose(res)
tf.dispose(input) tf.dispose(input)
console.timeEnd('post-process') console.timeEnd('sw: post-process')
return output || { detections: [] } return output || { detections: [] }
} }
async function videoFrame (vidData) { async function videoFrame (vidData) {
const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3) const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3)
console.time('frame-process') console.time('sw: frame-process')
let rawCoords = [] let rawCoords = []
try { try {
const input = tf.tidy(() => { const input = tf.tidy(() => {
@@ -155,7 +156,7 @@ async function videoFrame (vidData) {
const rawRes = tf.transpose(res,[0,2,1]).arraySync()[0] const rawRes = tf.transpose(res,[0,2,1]).arraySync()[0]
if (rawRes) { if (rawRes) {
for (var i = 0; i < rawRes.length; i++) { for (let i = 0; i < rawRes.length; i++) {
let getScores = rawRes[i].slice(4) let getScores = rawRes[i].slice(4)
if (getScores.some( s => s > .5)) { if (getScores.some( s => s > .5)) {
let foundTarget = rawRes[i].slice(0,2) let foundTarget = rawRes[i].slice(0,2)
@@ -171,6 +172,6 @@ async function videoFrame (vidData) {
} catch (e) { } catch (e) {
console.log(e) console.log(e)
} }
console.timeEnd('frame-process') console.timeEnd('sw: frame-process')
return {cds: rawCoords, mW: modelWidth, mH: modelHeight} return {cds: rawCoords, mW: modelWidth, mH: modelHeight}
} }

View File

@@ -74,6 +74,14 @@
} }
}, },
async created () { async created () {
document.addEventListener('keydown', e => {
if (e.code == 'KeyR') {
console.log(f7.views.main.router.history)
}
if (e.code == 'KeyB') {
f7.views.main.router.back()
}
})
if (!window.cordova) { if (!window.cordova) {
const confText = await fetch('./conf/conf.yaml') const confText = await fetch('./conf/conf.yaml')
.then((mod) => { return mod.text() }) .then((mod) => { return mod.text() })
@@ -98,6 +106,9 @@
store().set('siteDemo',this.siteConf?.demo) store().set('siteDemo',this.siteConf?.demo)
store().set('infoUrl',this.siteConf?.infoUrl) store().set('infoUrl',this.siteConf?.infoUrl)
const loadServerSettings = localStorage.getItem('serverSettings') const loadServerSettings = localStorage.getItem('serverSettings')
if (this.siteConf.disableWorkers) {
store().disableWorkers()
}
if (this.siteConf?.useExternal) { if (this.siteConf?.useExternal) {
if (!['none','list','optional','required'].includes(this.siteConf.useExternal)) { if (!['none','list','optional','required'].includes(this.siteConf.useExternal)) {
console.warn(`'${this.siteConf.useExternal}' is not a valid value for useExternal configuration: using 'optional'`) console.warn(`'${this.siteConf.useExternal}' is not a valid value for useExternal configuration: using 'optional'`)

View File

@@ -17,6 +17,9 @@
<path v-else-if="icon == 'head'" d="M194-80v-395h80v315h280v-193l105-105q29-29 45-65t16-77q0-40-16.5-76T659-741l-25-26-127 127H347l-43 43-57-56 67-67h160l160-160 82 82q40 40 62 90.5T800-600q0 57-22 107.5T716-402l-82 82v240H194Zm197-187L183-475q-11-11-17-26t-6-31q0-16 6-30.5t17-25.5l84-85 124 123q28 28 43.5 64.5T450-409q0 40-15 76.5T391-267Z"/> <path v-else-if="icon == 'head'" d="M194-80v-395h80v315h280v-193l105-105q29-29 45-65t16-77q0-40-16.5-76T659-741l-25-26-127 127H347l-43 43-57-56 67-67h160l160-160 82 82q40 40 62 90.5T800-600q0 57-22 107.5T716-402l-82 82v240H194Zm197-187L183-475q-11-11-17-26t-6-31q0-16 6-30.5t17-25.5l84-85 124 123q28 28 43.5 64.5T450-409q0 40-15 76.5T391-267Z"/>
<path v-else-if="icon == 'photo_sample'" d="M240-80q-33 0-56.5-23.5T160-160v-640q0-33 23.5-56.5T240-880h480q33 0 56.5 23.5T800-800v640q0 33-23.5 56.5T720-80H240Zm0-80h480v-640h-80v280l-100-60-100 60v-280H240v640Zm40-80h400L545-420 440-280l-65-87-95 127Zm-40 80v-640 640Zm200-360 100-60 100 60-100-60-100 60Z"/> <path v-else-if="icon == 'photo_sample'" d="M240-80q-33 0-56.5-23.5T160-160v-640q0-33 23.5-56.5T240-880h480q33 0 56.5 23.5T800-800v640q0 33-23.5 56.5T720-80H240Zm0-80h480v-640h-80v280l-100-60-100 60v-280H240v640Zm40-80h400L545-420 440-280l-65-87-95 127Zm-40 80v-640 640Zm200-360 100-60 100 60-100-60-100 60Z"/>
<path v-else-if="icon == 'reset_slide'" d="M520-330v-60h160v60H520Zm60 210v-50h-60v-60h60v-50h60v160h-60Zm100-50v-60h160v60H680Zm40-110v-160h60v50h60v60h-60v50h-60Zm111-280h-83q-26-88-99-144t-169-56q-117 0-198.5 81.5T200-480q0 72 32.5 132t87.5 98v-110h80v240H160v-80h94q-62-50-98-122.5T120-480q0-75 28.5-140.5t77-114q48.5-48.5 114-77T480-840q129 0 226.5 79.5T831-560Z"/> <path v-else-if="icon == 'reset_slide'" d="M520-330v-60h160v60H520Zm60 210v-50h-60v-60h60v-50h60v160h-60Zm100-50v-60h160v60H680Zm40-110v-160h60v50h60v60h-60v50h-60Zm111-280h-83q-26-88-99-144t-169-56q-117 0-198.5 81.5T200-480q0 72 32.5 132t87.5 98v-110h80v240H160v-80h94q-62-50-98-122.5T120-480q0-75 28.5-140.5t77-114q48.5-48.5 114-77T480-840q129 0 226.5 79.5T831-560Z"/>
<path v-else-if="icon == 'zoom_to'" d="M440-40v-167l-44 43-56-56 140-140 140 140-56 56-44-43v167h-80ZM220-340l-56-56 43-44H40v-80h167l-43-44 56-56 140 140-140 140Zm520 0L600-480l140-140 56 56-43 44h167v80H753l43 44-56 56Zm-260-80q-25 0-42.5-17.5T420-480q0-25 17.5-42.5T480-540q25 0 42.5 17.5T540-480q0 25-17.5 42.5T480-420Zm0-180L340-740l56-56 44 43v-167h80v167l44-43 56 56-140 140Z"/>
<path v-else-if="icon == 'reset_zoom'" d="M480-320v-100q0-25 17.5-42.5T540-480h100v60H540v100h-60Zm60 240q-25 0-42.5-17.5T480-140v-100h60v100h100v60H540Zm280-240v-100H720v-60h100q25 0 42.5 17.5T880-420v100h-60ZM720-80v-60h100v-100h60v100q0 25-17.5 42.5T820-80H720Zm111-480h-83q-26-88-99-144t-169-56q-117 0-198.5 81.5T200-480q0 72 32.5 132t87.5 98v-110h80v240H160v-80h94q-62-50-98-122.5T120-480q0-75 28.5-140.5t77-114q48.5-48.5 114-77T480-840q129 0 226.5 79.5T831-560Z"/>
<path v-else-if="icon == 'clipboard'" d="M200-120q-33 0-56.5-23.5T120-200v-560q0-33 23.5-56.5T200-840h167q11-35 43-57.5t70-22.5q40 0 71.5 22.5T594-840h166q33 0 56.5 23.5T840-760v560q0 33-23.5 56.5T760-120H200Zm0-80h560v-560h-80v120H280v-120h-80v560Zm280-560q17 0 28.5-11.5T520-800q0-17-11.5-28.5T480-840q-17 0-28.5 11.5T440-800q0 17 11.5 28.5T480-760Z"/>
</svg> </svg>
</template> </template>
@@ -44,7 +47,10 @@
'limbs', 'limbs',
'head', 'head',
'photo_sample', 'photo_sample',
'reset_slide' 'reset_slide',
'zoom_to',
'reset_zoom',
'clipboard'
] ]
return iconList.includes(value) return iconList.includes(value)
} }

View File

@@ -150,8 +150,7 @@
.structure-info { .structure-info {
position: absolute; position: absolute;
z-index: 3; z-index: 3;
color: rgb(15, 32, 108); color: #0f206c;
background: yellow;
border-radius: 100%; border-radius: 100%;
} }

View File

@@ -18,7 +18,7 @@
<meta name="msapplication-tap-highlight" content="no"> <meta name="msapplication-tap-highlight" content="no">
<title>ALVINN</title> <title>ALVINN</title>
<% if (TARGET === 'web') { %> <% if (TARGET === 'web') { %>
<meta name="apple-mobile-web-app-capable" content="yes"> <meta name="mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent"> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<link rel="apple-touch-icon" href="icons/apple-touch-icon.png"> <link rel="apple-touch-icon" href="icons/apple-touch-icon.png">
<link rel="icon" href="icons/favicon.png"> <link rel="icon" href="icons/favicon.png">

View File

@@ -4,9 +4,11 @@ const state = reactive({
disclaimerAgreement: false, disclaimerAgreement: false,
enabledRegions: ['thorax','abdomen','limbs','head'], enabledRegions: ['thorax','abdomen','limbs','head'],
regionIconSet: Math.floor(Math.random() * 3) + 1, regionIconSet: Math.floor(Math.random() * 3) + 1,
version: '0.5.0-rc', version: '0.5.0-alpha',
build: '####',
fullscreen: false, fullscreen: false,
useExternal: 'optional', useExternal: 'optional',
workersEnabled: 'true',
siteDemo: false, siteDemo: false,
externalServerList: [], externalServerList: [],
infoUrl: false infoUrl: false
@@ -21,6 +23,10 @@ const agree = () => {
state.disclaimerAgreement = true state.disclaimerAgreement = true
} }
const disableWorkers = () => {
state.workersEnabled = false
}
const getServerList = () => { const getServerList = () => {
if (state.useExternal == 'required') { if (state.useExternal == 'required') {
return state.externalServerList[0] return state.externalServerList[0]
@@ -46,12 +52,15 @@ export default () => ({
isFullscreen: computed(() => state.fullscreen), isFullscreen: computed(() => state.fullscreen),
demoMode: computed(() => state.siteDemo), demoMode: computed(() => state.siteDemo),
externalType: computed(() => state.useExternal), externalType: computed(() => state.useExternal),
useWorkers: computed(() => state.workersEnabled),
getRegions: computed(() => state.enabledRegions), getRegions: computed(() => state.enabledRegions),
getVersion: computed(() => state.version), getVersion: computed(() => state.version),
getBuild: computed(() => state.build),
getIconSet: computed(() => state.regionIconSet), getIconSet: computed(() => state.regionIconSet),
getInfoUrl: computed(() => state.infoUrl), getInfoUrl: computed(() => state.infoUrl),
set, set,
agree, agree,
disableWorkers,
getServerList, getServerList,
toggleFullscreen toggleFullscreen
}) })

157
src/js/structures.js Normal file
View File

@@ -0,0 +1,157 @@
class Coordinate {
constructor(x, y) {
this.x = x
this.y = y
}
toRefFrame(...frameArgs) {
if (frameArgs.length == 0) {
return {x: this.x, y: this.y}
}
let outFrames = []
//Get Coordinates in Image Reference Frame
if (frameArgs[0].tagName == 'IMG' && frameArgs[0].width && frameArgs[0].height) {
outFrames.push({
x: this.x * frameArgs[0].width,
y: this.y * frameArgs[0].height
})
} else {
throw new Error('Coordinate: invalid reference frame for frameType: Image')
}
//Get Coordinates in Canvas Reference Frame
if (frameArgs[1]) {
if (frameArgs[1].tagName == 'CANVAS' && frameArgs[1].width && frameArgs[1].height) {
let imgWidth
let imgHeight
const imgAspect = frameArgs[0].width / frameArgs[0].height
const rendAspect = frameArgs[1].width / frameArgs[1].height
if (imgAspect >= rendAspect) {
imgWidth = frameArgs[1].width
imgHeight = frameArgs[1].width / imgAspect
} else {
imgWidth = frameArgs[1].height * imgAspect
imgHeight = frameArgs[1].height
}
outFrames.push({
x: (frameArgs[1].width - imgWidth) / 2 + this.x * imgWidth,
y: (frameArgs[1].height - imgHeight) / 2 + this.y * imgHeight
})
} else {
throw new Error('Coordinate: invalid reference frame for frameType: Canvas')
}
}
//Get Coordinates in Screen Reference Frame
if (frameArgs[2]) {
if (frameArgs[2].zoom && frameArgs[2].offset && frameArgs[2].offset.x !== undefined && frameArgs[2].offset.y !== undefined) {
outFrames.push({
x: outFrames[1].x * frameArgs[2].zoom + frameArgs[2].offset.x,
y: outFrames[1].y * frameArgs[2].zoom + frameArgs[2].offset.y
})
} else {
throw new Error('Coordinate: invalid reference frame for frameType: Screen')
}
}
return outFrames
}
toString() {
return `(x: ${this.x}, y: ${this.y})`
}
}
export class StructureBox {
constructor(top, left, bottom, right) {
this.topLeft = new Coordinate(left, top)
this.bottomRight = new Coordinate(right, bottom)
}
getBoxes(boxType, ...frameArgs) {
let lowerH, lowerV, calcSide
switch (boxType) {
case 'point':
lowerH = 'right'
lowerV = 'bottom'
break
case 'side':
lowerH = 'width'
lowerV = 'height'
calcSide = true
break
default:
throw new Error(`StructureBox: invalid boxType - ${boxType}`)
}
if (frameArgs.length == 0) {
return {
left: this.topLeft.x,
top: this.topLeft.y,
[lowerH]: this.bottomRight.x - ((calcSide) ? this.topLeft.x : 0),
[lowerV]: this.bottomRight.y - ((calcSide) ? this.topLeft.y : 0)
}
}
const tL = this.topLeft.toRefFrame(...frameArgs)
const bR = this.bottomRight.toRefFrame(...frameArgs)
let outBoxes = []
tL.forEach((cd, i) => {
outBoxes.push({
left: cd.x,
top: cd.y,
[lowerH]: bR[i].x - ((calcSide) ? cd.x : 0),
[lowerV]: bR[i].y - ((calcSide) ? cd.y : 0)
})
})
return outBoxes
}
}
export class Structure {
constructor(structResult) {
this.label = structResult.label
this.confidence = structResult.confidence
this.box = new StructureBox(
structResult.top,
structResult.left,
structResult.bottom,
structResult.right
)
this.deleted = false
this.index = -1
this.passThreshold = true
this.searched = false
}
get resultIndex() {
return this.index
}
set resultIndex(newIdx) {
this.index = newIdx
}
get isDeleted() {
return this.deleted
}
set isDeleted(del) {
this.deleted = !!del
}
get isSearched() {
return this.searched
}
set isSearched(ser) {
this.searched = !!ser
}
get aboveThreshold() {
return this.passThreshold
}
setThreshold(level) {
if (typeof level != 'number') {
throw new Error(`Structure: invalid threshold level ${level}`)
}
this.passThreshold = this.confidence >= level
}
}

View File

@@ -3,11 +3,11 @@ import { f7 } from 'framework7-vue'
export default { export default {
methods: { methods: {
async openCamera(imContain) { async openCamera(imContain) {
var cameraLoaded = false let cameraLoaded = false
const devicesList = await navigator.mediaDevices.enumerateDevices() const devicesList = await navigator.mediaDevices.enumerateDevices()
this.videoDeviceAvailable = devicesList.some( d => d.kind == "videoinput") let videoDeviceAvailable = devicesList.some( d => d.kind == "videoinput")
if (this.videoDeviceAvailable) { if (videoDeviceAvailable) {
var vidConstraint = { let vidConstraint = {
video: { video: {
width: { width: {
ideal: imContain.offsetWidth ideal: imContain.offsetWidth
@@ -41,24 +41,24 @@ export default {
tempCtx.drawImage(vidViewer, 0, 0) tempCtx.drawImage(vidViewer, 0, 0)
this.getImage(tempCVS.toDataURL()) this.getImage(tempCVS.toDataURL())
}, },
async videoFrameDetect (vidData) { async videoFrameDetectWorker (vidData, vidWorker) {
const startDetection = () => { const startDetection = () => {
createImageBitmap(vidData).then(imVideoFrame => { createImageBitmap(vidData).then(imVideoFrame => {
this.vidWorker.postMessage({call: 'videoFrame', image: imVideoFrame}, [imVideoFrame]) vidWorker.postMessage({call: 'videoFrame', image: imVideoFrame}, [imVideoFrame])
}) })
} }
vidData.addEventListener('resize',startDetection,{once: true}) vidData.addEventListener('resize',startDetection,{once: true})
this.vidWorker.onmessage = (eVid) => { vidWorker.onmessage = (eVid) => {
if (eVid.data.error) { if (eVid.data.error) {
console.log(eVid.data.message) console.log(eVid.data.message)
f7.dialog.alert(`ALVINN AI model error: ${eVid.data.message}`) f7.dialog.alert(`ALVINN AI model error: ${eVid.data.message}`)
} else if (this.videoAvailable) { } else if (this.videoAvailable) {
createImageBitmap(vidData).then(imVideoFrame => { createImageBitmap(vidData).then(imVideoFrame => {
this.vidWorker.postMessage({call: 'videoFrame', image: imVideoFrame}, [imVideoFrame]) vidWorker.postMessage({call: 'videoFrame', image: imVideoFrame}, [imVideoFrame])
}) })
if (eVid.data.coords) { if (eVid.data.coords) {
imageCtx.clearRect(0,0,imCanvas.width,imCanvas.height) imageCtx.clearRect(0,0,imCanvas.width,imCanvas.height)
for (var coord of eVid.data.coords) { for (let coord of eVid.data.coords) {
let pointX = (imCanvas.width - imgWidth) / 2 + (coord[0] / eVid.data.modelWidth) * imgWidth - 10 let pointX = (imCanvas.width - imgWidth) / 2 + (coord[0] / eVid.data.modelWidth) * imgWidth - 10
let pointY = (imCanvas.height - imgHeight) / 2 + (coord[1] / eVid.data.modelHeight) * imgHeight - 10 let pointY = (imCanvas.height - imgHeight) / 2 + (coord[1] / eVid.data.modelHeight) * imgHeight - 10
console.debug(`cx: ${pointX}, cy: ${pointY}`) console.debug(`cx: ${pointX}, cy: ${pointY}`)
@@ -72,8 +72,7 @@ export default {
const imCanvas = this.$refs.image_cvs const imCanvas = this.$refs.image_cvs
const imageCtx = imCanvas.getContext("2d") const imageCtx = imCanvas.getContext("2d")
const target = this.$refs.target_image const target = this.$refs.target_image
var imgWidth let imgWidth, imgHeight
var imgHeight
f7.utils.nextFrame(() => { f7.utils.nextFrame(() => {
imCanvas.width = imCanvas.clientWidth imCanvas.width = imCanvas.clientWidth
imCanvas.height = imCanvas.clientHeight imCanvas.height = imCanvas.clientHeight

View File

@@ -56,7 +56,7 @@
}, },
computed: { computed: {
commentText () { commentText () {
var text = f7.textEditor.get('.comment-editor').getValue() let text = f7.textEditor.get('.comment-editor').getValue()
if (this.userEmail) { if (this.userEmail) {
text += `\\n\\nSubmitted by: ${this.userEmail}` text += `\\n\\nSubmitted by: ${this.userEmail}`
} }
@@ -65,9 +65,9 @@
}, },
methods: { methods: {
sendFeedback () { sendFeedback () {
var self = this let self = this
var issueURL = `https://gitea.azgeorgis.net/api/v1/repos/Georgi_Lab/ALVINN_f7/issues?access_token=9af8ae15b1ee5a98afcb3083bb488e4cf3c683af` const issueURL = `https://gitea.azgeorgis.net/api/v1/repos/Georgi_Lab/ALVINN_f7/issues?access_token=9af8ae15b1ee5a98afcb3083bb488e4cf3c683af`
var xhr = new XMLHttpRequest() let xhr = new XMLHttpRequest()
xhr.open("POST", issueURL) xhr.open("POST", issueURL)
xhr.setRequestHeader('Content-Type', 'application/json') xhr.setRequestHeader('Content-Type', 'application/json')
xhr.setRequestHeader('accept', 'application/json') xhr.setRequestHeader('accept', 'application/json')

View File

@@ -1,23 +1,36 @@
<template> <template>
<f7-page name="detect" :id="detectorName + '-detect-page'"> <f7-page name="detect" :id="detectorName + '-detect-page'" @wheel="(e = $event) => e.preventDefault()" @touchmove="(e = $event) => e.preventDefault()">
<!-- Top Navbar --> <!-- Top Navbar -->
<f7-navbar :sliding="false" :back-link="true" back-link-url="/" back-link-force> <f7-navbar :sliding="false" :back-link="true" back-link-url="/" back-link-force>
<f7-nav-title sliding>{{ regions[activeRegion] }}</f7-nav-title> <f7-nav-title sliding>{{ regionTitle }}</f7-nav-title>
<f7-nav-right> <f7-nav-right>
<f7-link v-if="!isCordova" :icon-only="true" tooltip="Fullscreen" :icon-f7="isFullscreen ? 'viewfinder_circle_fill' : 'viewfinder'" @click="toggleFullscreen"></f7-link> <f7-link v-if="!isCordova" :icon-only="true" tooltip="Fullscreen" :icon-f7="isFullscreen ? 'viewfinder_circle_fill' : 'viewfinder'" @click="toggleFullscreen"></f7-link>
<f7-link :icon-only="true" tooltip="ALVINN help" icon-f7="question_circle_fill" href="/help/"></f7-link> <f7-link :icon-only="true" tooltip="ALVINN help" icon-f7="question_circle_fill" href="/help/"></f7-link>
</f7-nav-right> </f7-nav-right>
</f7-navbar> </f7-navbar>
<f7-block class="detect-grid"> <f7-block class="detect-grid">
<!--<div style="position: absolute;">{{ debugInfo ? JSON.stringify(debugInfo) : "No Info Available" }}</div>-->
<div class="image-container" ref="image_container"> <div class="image-container" ref="image_container">
<SvgIcon v-if="!imageView.src && !videoAvailable" :icon="f7route.params.region" fill-color="var(--avn-theme-color)"/> <SvgIcon v-if="!imageView.src && !videoAvailable" :icon="f7route.params.region" fill-color="var(--avn-theme-color)"/>
<div class="vid-container" :style="`display: ${videoAvailable ? 'block' : 'none'}; position: absolute; width: 100%; height: 100%;`"> <div class="vid-container" :style="`display: ${videoAvailable ? 'block' : 'none'}; position: absolute; width: 100%; height: 100%;`">
<video id="vid-view" ref="vid_viewer" :srcObject="cameraStream" :autoPlay="true" style="width: 100%; height: 100%"></video> <video id="vid-view" ref="vid_viewer" :srcObject="cameraStream" :autoPlay="true" style="width: 100%; height: 100%"></video>
<f7-button @click="captureVidFrame()" style="position: absolute; bottom: 32px; left: 50%; transform: translateX(-50%); z-index: 3;" fill large>Capture</f7-button> <f7-button @click="captureVidFrame()" style="position: absolute; bottom: 32px; left: 50%; transform: translateX(-50%); z-index: 3;" fill large>Capture</f7-button>
</div> </div>
<canvas id="im-draw" ref="image_cvs" @click="structureClick" :style="`display: ${(imageLoaded || videoAvailable) ? 'block' : 'none'}; flex: 1 1 0%; max-width: 100%; max-height: 100%; min-width: 0; min-height: 0; background-size: contain; background-position: center; background-repeat: no-repeat; z-index: 2;`" /> <canvas
<f7-link v-if="getInfoUrl && (selectedChip > -1)" id="im-draw"
:style="`left: ${infoLinkPos.x}px; top: ${infoLinkPos.y}px; transform: translate(calc(-50% - ${infoLinkPos.adj}px),calc(-50% - ${infoLinkPos.adj}px));`" ref="image_cvs"
@wheel="spinWheel($event)"
@mousedown.middle="startMove($event)"
@mousemove="makeMove($event)"
@mouseup.middle="endMove($event)"
@touchstart="startTouch($event)"
@touchend="endTouch($event)"
@touchmove="moveTouch($event)"
@click="structureClick"
:style="`display: ${(imageLoaded || videoAvailable) ? 'block' : 'none'}; flex: 1 1 0%; max-width: 100%; max-height: 100%; min-width: 0; min-height: 0; background-size: contain; background-position: center; background-repeat: no-repeat; z-index: 2;`"
></canvas>
<f7-link v-if="getInfoUrl && (selectedChip > -1) && showResults[selectedChip]"
:style="`left: ${infoLinkPos.x}px; top: ${infoLinkPos.y}px; transform: translate(-50%,-50%); background: hsla(${showResults[selectedChip].confidence / 100 * 120}deg, 100%, 50%, .5)`"
class="structure-info" class="structure-info"
:icon-only="true" :icon-only="true"
icon-f7="info" icon-f7="info"
@@ -61,16 +74,19 @@
</f7-button> </f7-button>
</div> </div>
<f7-segmented class="image-menu" raised> <f7-segmented class="image-menu" raised>
<f7-button popover-open="#region-popover">
<RegionIcon :region="activeRegion" :iconSet="getIconSet" />
</f7-button>
<f7-button v-if="!videoAvailable" :class="(!modelLoading) ? '' : 'disabled'" popover-open="#capture-popover"> <f7-button v-if="!videoAvailable" :class="(!modelLoading) ? '' : 'disabled'" popover-open="#capture-popover">
<SvgIcon icon="camera_add"/> <SvgIcon icon="camera_add"/>
</f7-button> </f7-button>
<f7-button v-if="videoAvailable" @click="closeCamera()"> <f7-button v-if="videoAvailable" @click="closeCamera()">
<SvgIcon icon="no_photography"/> <SvgIcon icon="no_photography"/>
</f7-button> </f7-button>
<f7-button @click="() => showDetectSettings = !showDetectSettings" :class="(imageLoaded) ? '' : 'disabled'"> <f7-button v-if="!structureZoomed && selectedChip >= 0" style="height: auto; width: auto;" popover-close="#image-popover" @click="zoomToSelected()">
<SvgIcon icon="zoom_to" />
</f7-button>
<f7-button v-else :class="(canvasZoom != 1) ? '' : 'disabled'" style="height: auto; width: auto;" popover-close="#image-popover" @click="resetZoom()">
<SvgIcon icon="reset_zoom" />
</f7-button>
<f7-button @click="toggleSettings()" :class="(imageLoaded) ? '' : 'disabled'">
<SvgIcon icon="visibility"/> <SvgIcon icon="visibility"/>
<f7-badge v-if="numResults && (showResults.length != numResults)" color="red" style="position: absolute; right: 15%; top: 15%;">{{ showResults.length - numResults }}</f7-badge> <f7-badge v-if="numResults && (showResults.length != numResults)" color="red" style="position: absolute; right: 15%; top: 15%;">{{ showResults.length - numResults }}</f7-badge>
</f7-button> </f7-button>
@@ -93,23 +109,6 @@
</f7-page> </f7-page>
</f7-panel> </f7-panel>
<f7-popover id="region-popover" class="popover-button-menu">
<f7-segmented raised class="segment-button-menu">
<f7-button :class="(getRegions.includes('thorax')) ? '' : ' disabled'" style="height: auto; width: auto;" href="/detect/thorax/" popover-close="#region-popover">
<RegionIcon :region="0" :iconSet="getIconSet" />
</f7-button>
<f7-button :class="(getRegions.includes('abdomen')) ? '' : ' disabled'" style="height: auto; width: auto;" href="/detect/abdomen/" popover-close="#region-popover">
<RegionIcon :region="1" :iconSet="getIconSet" />
</f7-button>
<f7-button :class="(getRegions.includes('limbs')) ? '' : ' disabled'" style="height: auto; width: auto;" href="/detect/limbs/" popover-close="#region-popover">
<RegionIcon :region="2" :iconSet="getIconSet" />
</f7-button>
<f7-button :class="(getRegions.includes('head')) ? '' : ' disabled'" style="height: auto; width: auto;" href="/detect/head/" popover-close="#region-popover">
<RegionIcon :region="3" :iconSet="getIconSet" />
</f7-button>
</f7-segmented>
</f7-popover>
<f7-popover id="capture-popover" class="popover-button-menu"> <f7-popover id="capture-popover" class="popover-button-menu">
<f7-segmented raised class="segment-button-menu"> <f7-segmented raised class="segment-button-menu">
<f7-button style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('camera')"> <f7-button style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('camera')">
@@ -118,6 +117,9 @@
<f7-button style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('file')"> <f7-button style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('file')">
<SvgIcon icon="photo_library" /> <SvgIcon icon="photo_library" />
</f7-button> </f7-button>
<f7-button v-if="secureProtocol" style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('clipboard')">
<SvgIcon icon="clipboard" />
</f7-button>
<f7-button v-if="demoEnabled" style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('sample')"> <f7-button v-if="demoEnabled" style="height: auto; width: auto;" popover-close="#capture-popover" @click="selectImage('sample')">
<SvgIcon icon="photo_sample"/> <SvgIcon icon="photo_sample"/>
</f7-button> </f7-button>
@@ -139,11 +141,27 @@
import submitMixin from './submit-mixin' import submitMixin from './submit-mixin'
import detectionMixin from './detection-mixin' import detectionMixin from './detection-mixin'
import cameraMixin from './camera-mixin' import cameraMixin from './camera-mixin'
import touchMixin from './touch-mixin'
import detectionWorker from '../assets/detect-worker.js?worker&inline' import detectionWorker from '@/assets/detect-worker.js?worker&inline'
import { Structure, StructureBox } from '../js/structures'
const regions = ['Thorax','Abdomen/Pelvis','Limbs','Head and Neck']
let activeRegion = 4
let classesList = []
let imageLoadMode = "environment"
let serverSettings = {}
let otherSettings = {}
let modelLocation = ''
let miniLocation = ''
let reloadModel = false
let detectWorker = null
let vidWorker = null
let canvasMoving = false
let imageLocation = new StructureBox(0, 0, 1, 1)
export default { export default {
mixins: [submitMixin, detectionMixin, cameraMixin], mixins: [submitMixin, detectionMixin, cameraMixin, touchMixin],
props: { props: {
f7route: Object, f7route: Object,
}, },
@@ -153,35 +171,28 @@
}, },
data () { data () {
return { return {
regions: ['Thorax','Abdomen/Pelvis','Limbs','Head and Neck'],
resultData: {}, resultData: {},
selectedChip: -1, selectedChip: -1,
activeRegion: 4,
classesList: [],
imageLoaded: false, imageLoaded: false,
imageView: new Image(), imageView: new Image(),
imageLoadMode: "environment",
detecting: false, detecting: false,
detectPanel: false, detectPanel: false,
showDetectSettings: false, showDetectSettings: false,
detectorName: '', detectorName: '',
detectorLevel: 50, detectorLevel: 50,
detectorLabels: [], detectorLabels: [],
serverSettings: {},
otherSettings: {},
isCordova: !!window.cordova, isCordova: !!window.cordova,
secureProtocol: location.protocol == 'https:',
uploadUid: null, uploadUid: null,
uploadDirty: false, uploadDirty: false,
modelLocation: '',
miniLocation: '',
modelLoading: true, modelLoading: true,
reloadModel: false,
videoDeviceAvailable: false,
videoAvailable: false, videoAvailable: false,
cameraStream: null, cameraStream: null,
infoLinkPos: {}, infoLinkPos: {},
detectWorker: null, canvasOffset: {x: 0, y: 0},
vidWorker: null canvasZoom: 1,
structureZoomed: false,
debugInfo: null
} }
}, },
setup() { setup() {
@@ -189,64 +200,78 @@
}, },
created () { created () {
let loadOtherSettings = localStorage.getItem('otherSettings') let loadOtherSettings = localStorage.getItem('otherSettings')
if (loadOtherSettings) this.otherSettings = JSON.parse(loadOtherSettings) if (loadOtherSettings) otherSettings = JSON.parse(loadOtherSettings)
let modelRoot = this.isCordova ? 'https://localhost' : '.'
this.detectorName = this.f7route.params.region this.detectorName = this.f7route.params.region
switch (this.detectorName) { switch (this.detectorName) {
case 'thorax': case 'thorax':
this.activeRegion = 0 activeRegion = 0
break; break;
case 'abdomen': case 'abdomen':
this.activeRegion = 1 activeRegion = 1
break; break;
case 'limbs': case 'limbs':
this.activeRegion = 2 activeRegion = 2
break; break;
case 'head': case 'head':
this.activeRegion = 3 activeRegion = 3
break; break;
} }
this.modelLocation = `${modelRoot}/models/${this.detectorName}${this.otherSettings.mini ? '-mini' : ''}/model.json` let modelJ = `../models/${this.detectorName}${otherSettings.mini ? '-mini' : ''}/model.json`
this.miniLocation = `${modelRoot}/models/${this.detectorName}-mini/model.json` let miniJ = `../models/${this.detectorName}-mini/model.json`
fetch(`${modelRoot}/models/${this.detectorName}/classes.json`) modelLocation = new URL(modelJ,import.meta.url).href
miniLocation = new URL(miniJ,import.meta.url).href
let classesJ = `../models/${this.detectorName}/classes.json`
fetch(new URL(classesJ,import.meta.url).href)
.then((mod) => { return mod.json() }) .then((mod) => { return mod.json() })
.then((classes) => { .then((classes) => {
this.classesList = classes classesList = classes
this.detectorLabels = this.classesList.map( l => { return {'name': l, 'detect': true} } ) this.detectorLabels = classesList.map( l => { return {'name': l, 'detect': true} } )
}) })
var loadServerSettings = localStorage.getItem('serverSettings') const loadServerSettings = localStorage.getItem('serverSettings')
if (loadServerSettings) this.serverSettings = JSON.parse(loadServerSettings) if (loadServerSettings) serverSettings = JSON.parse(loadServerSettings)
}, },
mounted () { mounted () {
this.detectWorker = new detectionWorker() if (serverSettings && serverSettings.use) {
this.detectWorker.onmessage = (eMount) => {
self = this
if (eMount.data.error) {
console.log(eMount.data.message)
f7.dialog.alert(`ALVINN AI model error: ${eMount.data.message}`)
}
self.modelLoading = false
}
this.vidWorker = new detectionWorker()
this.vidWorker.onmessage = (eMount) => {
self = this
if (eMount.data.error) {
console.log(eMount.data.message)
f7.dialog.alert(`ALVINN AI nano model error: ${eMount.data.message}`)
}
}
if (this.serverSettings && this.serverSettings.use) {
this.getRemoteLabels() this.getRemoteLabels()
this.modelLoading = false this.modelLoading = false
} else { } else {
this.modelLoading = true this.modelLoading = true
this.detectWorker.postMessage({call: 'loadModel', weights: this.modelLocation, preload: true}) if (!this.useWorkers) {
this.vidWorker.postMessage({call: 'loadModel', weights: this.miniLocation, preload: true}) this.loadModel(modelLocation, true).then(() => {
this.modelLoading = false
}).catch((e) => {
console.log(e.message)
f7.dialog.alert(`ALVINN AI model error: ${e.message}`)
this.modelLoading = false
})
} else {
detectWorker = new detectionWorker()
detectWorker.onmessage = (eMount) => {
self = this
if (eMount.data.error) {
console.log(eMount.data.message)
f7.dialog.alert(`ALVINN AI model error: ${eMount.data.message}`)
}
self.modelLoading = false
}
vidWorker = new detectionWorker()
vidWorker.onmessage = (eMount) => {
self = this
if (eMount.data.error) {
console.log(eMount.data.message)
f7.dialog.alert(`ALVINN AI nano model error: ${eMount.data.message}`)
}
}
detectWorker.postMessage({call: 'loadModel', weights: modelLocation, preload: true})
vidWorker.postMessage({call: 'loadModel', weights: miniLocation, preload: true})
}
} }
window.onresize = (e) => { if (this.$refs.image_cvs) this.selectChip('redraw') } window.onresize = (e) => { if (this.$refs.image_cvs) this.selectChip('redraw') }
}, },
computed: { computed: {
regionTitle () {
return regions[activeRegion]
},
message () { message () {
if (this.modelLoading) { if (this.modelLoading) {
return "Preparing ALVINN..." return "Preparing ALVINN..."
@@ -259,17 +284,17 @@
} }
}, },
showResults () { showResults () {
var filteredResults = this.resultData.detections let filteredResults = this.resultData.detections
if (!filteredResults) return [] if (!filteredResults) return []
var allSelect = this.detectorLabels.every( s => { return s.detect } ) const allSelect = this.detectorLabels.every( s => { return s.detect } )
var selectedLabels = this.detectorLabels const selectedLabels = this.detectorLabels
.filter( l => { return l.detect }) .filter( l => { return l.detect })
.map( l => { return l.name }) .map( l => { return l.name })
filteredResults.forEach( (d, i) => { filteredResults.forEach( (d, i) => {
filteredResults[i].resultIndex = i d.resultIndex = i
filteredResults[i].aboveThreshold = d.confidence >= this.detectorLevel d.setThreshold(this.detectorLevel)
filteredResults[i].isSearched = allSelect || selectedLabels.includes(d.label) d.isSearched = allSelect || selectedLabels.includes(d.label)
}) })
if (!filteredResults.some( s => s.resultIndex == this.selectedChip && s.aboveThreshold && s.isSearched && !s.isDeleted)) { if (!filteredResults.some( s => s.resultIndex == this.selectedChip && s.aboveThreshold && s.isSearched && !s.isDeleted)) {
@@ -290,13 +315,13 @@
} }
}, },
demoEnabled () { demoEnabled () {
return this.otherSettings.demo || this.demoMode return otherSettings.demo || this.demoMode
}, },
infoLinkTarget () { infoLinkTarget () {
if (!this.getInfoUrl) return '' if (!this.getInfoUrl) return ''
let structure = this.showResults.find( r => r.resultIndex == this.selectedChip) let structure = this.showResults.find( r => r.resultIndex == this.selectedChip)
return structure ? this.getInfoUrl + structure.label.replaceAll(' ','_') : '' return structure ? this.getInfoUrl + structure.label.replaceAll(' ','_') : ''
} },
}, },
methods: { methods: {
chipGradient (confVal) { chipGradient (confVal) {
@@ -304,46 +329,74 @@
return `--chip-media-gradient: conic-gradient(from ${270 - (confFactor * 360 / 2)}deg, hsl(${confFactor * 120}deg, 100%, 50%) ${confFactor}turn, hsl(${confFactor * 120}deg, 50%, 66%) ${confFactor}turn)` return `--chip-media-gradient: conic-gradient(from ${270 - (confFactor * 360 / 2)}deg, hsl(${confFactor * 120}deg, 100%, 50%) ${confFactor}turn, hsl(${confFactor * 120}deg, 50%, 66%) ${confFactor}turn)`
}, },
async setData () { async setData () {
//const detectWorker = new detectionWorker() if (detectWorker) {
this.detectWorker.onmessage = (eDetect) => { detectWorker.onmessage = (eDetect) => {
self = this self = this
if (eDetect.data.error) { if (eDetect.data.error) {
self.detecting = false self.detecting = false
self.resultData = {} self.resultData = {}
loadFailure() loadFailure()
f7.dialog.alert(`ALVINN structure finding error: ${eDetect.data.message}`) f7.dialog.alert(`ALVINN structure finding error: ${eDetect.data.message}`)
} else if (eDetect.data.success == 'detection') { } else if (eDetect.data.success == 'detection') {
self.detecting = false self.detecting = false
self.resultData = eDetect.data.detections self.resultData = {detections: []}
if (self.resultData) { eDetect.data.detections.detections.forEach((d) => {
self.resultData.detections.map(d => {d.label = self.detectorLabels[d.label].name}) d.label = self.detectorLabels[d.label].name
let detectedStructure = new Structure(d)
self.resultData.detections.push(detectedStructure)
})
self.uploadDirty = true
} else if (eDetect.data.success == 'model') {
reloadModel = false
loadSuccess()
} }
self.uploadDirty = true f7.utils.nextFrame(() => {
} else if (eDetect.data.success == 'model') { this.selectChip("redraw")
self.reloadModel = false })
loadSuccess()
} }
} }
let loadSuccess = null let loadSuccess = null
let loadFailure = null let loadFailure = null
let modelReloading = new Promise((res, rej) => { let modelReloading = null
loadSuccess = res if (!this.useWorkers && reloadModel) {
loadFailure = rej await this.loadModel(modelLocation)
if (this.reloadModel) { reloadModel = false
this.detectWorker.postMessage({call: 'loadModel', weights: this.modelLocation})
} else {
loadSuccess()
}
})
if (this.serverSettings && this.serverSettings.use) {
this.remoteDetect()
} else { } else {
Promise.all([modelReloading,createImageBitmap(this.imageView)]).then(res => { modelReloading = new Promise((res, rej) => {
this.detectWorker.postMessage({call: 'localDetect', image: res[1]}, [res[1]]) loadSuccess = res
loadFailure = rej
if (reloadModel) {
detectWorker.postMessage({call: 'loadModel', weights: modelLocation})
} else {
loadSuccess()
}
}) })
} }
if (serverSettings && serverSettings.use) {
this.remoteDetect()
} else if (this.useWorkers) {
Promise.all([modelReloading,createImageBitmap(this.imageView)]).then(res => {
detectWorker.postMessage({call: 'localDetect', image: res[1]}, [res[1]])
})
} else {
createImageBitmap(this.imageView).then(res => {
return this.localDetect(res)
}).then(dets => {
this.detecting = false
this.resultData = dets
this.uploadDirty = true
}).catch((e) => {
console.log(e.message)
this.detecting = false
this.resultData = {}
f7.dialog.alert(`ALVINN structure finding error: ${e.message}`)
})
}
f7.utils.nextFrame(() => {
this.selectChip("redraw")
})
}, },
selectAll (ev) { selectAll (ev) {
if (ev.target.checked) { if (ev.target.checked) {
@@ -353,12 +406,12 @@
} }
}, },
async selectImage (mode) { async selectImage (mode) {
this.imageLoadMode = mode imageLoadMode = mode
if (this.isCordova && mode == "camera") { if (this.isCordova && mode == "camera") {
navigator.camera.getPicture(this.getImage, this.onFail, { quality: 50, destinationType: Camera.DestinationType.DATA_URL, correctOrientation: true }); navigator.camera.getPicture(this.getImage, this.onFail, { quality: 50, destinationType: Camera.DestinationType.DATA_URL, correctOrientation: true });
return return
} }
if (mode == "camera") { if (mode == "camera" && !otherSettings.disableVideo) {
this.videoAvailable = await this.openCamera(this.$refs.image_container) this.videoAvailable = await this.openCamera(this.$refs.image_container)
if (this.videoAvailable) { if (this.videoAvailable) {
this.selectedChip = -1 this.selectedChip = -1
@@ -366,12 +419,14 @@
this.imageView.src = null this.imageView.src = null
this.$refs.image_cvs.style['background-image'] = 'none' this.$refs.image_cvs.style['background-image'] = 'none'
this.resultData = {} this.resultData = {}
var trackDetails = this.cameraStream.getVideoTracks()[0].getSettings() const trackDetails = this.cameraStream.getVideoTracks()[0].getSettings()
var vidElement = this.$refs.vid_viewer let vidElement = this.$refs.vid_viewer
vidElement.width = trackDetails.width vidElement.width = trackDetails.width
vidElement.height = trackDetails.height vidElement.height = trackDetails.height
if (!this.otherSettings.disableVideo) { if (!this.useWorkers) {
this.videoFrameDetect(vidElement) this.videoFrameDetect(vidElement, miniLocation)
} else {
this.videoFrameDetectWorker(vidElement, vidWorker)
} }
return return
} }
@@ -389,36 +444,62 @@
}).open() }).open()
return return
} }
if (mode == 'clipboard') {
navigator.clipboard.read().then(clip => {
if (!clip[0].types.includes("image/png")) {
throw new Error("Clipboard does not contain valid image data.");
}
return clip[0].getType("image/png");
}).then(blob => {
let clipImage = URL.createObjectURL(blob);
this.getImage(clipImage)
}).catch(e => {
console.log(e)
f7.dialog.alert(`Error pasting image: ${e.message}`)
})
return
}
this.$refs.image_chooser.click() this.$refs.image_chooser.click()
}, },
onFail (message) { onFail (message) {
alert(`Camera fail: ${message}`) alert(`Camera fail: ${message}`)
}, },
selectChip ( iChip ) { selectChip ( iChip ) {
const [imCanvas, imageCtx] = this.resetView()
if (this.selectedChip == iChip) { if (this.selectedChip == iChip) {
this.selectedChip = -1 this.selectedChip = -1
this.resetView()
return return
} }
if (iChip == 'redraw') { if (iChip == 'redraw') {
if (this.selectedChip == -1) return if (this.selectedChip == -1) {
this.resetView()
return
}
iChip = this.selectedChip iChip = this.selectedChip
} }
const [imCanvas, imageCtx] = this.resetView(true)
let structBox, cvsBox, screenBox
[structBox, cvsBox, screenBox] = this.resultData.detections[iChip].box.getBoxes('side', this.imageView, imCanvas, {zoom: this.canvasZoom, offset: {...this.canvasOffset}})
const boxCoords = this.box2cvs(this.resultData.detections[iChip])[0] this.infoLinkPos.x = Math.min(Math.max(screenBox.left, 0),imCanvas.width)
this.infoLinkPos.y = Math.min(Math.max(screenBox.top, 0), imCanvas.height)
let boxLeft = boxCoords.cvsLeft const imageScale = Math.max(this.imageView.width / imCanvas.width, this.imageView.height / imCanvas.height)
let boxTop = boxCoords.cvsTop imageCtx.drawImage(this.imageView, structBox.left, structBox.top, structBox.width, structBox.height, cvsBox.left, cvsBox.top, cvsBox.width, cvsBox.height)
let boxWidth = boxCoords.cvsRight - boxCoords.cvsLeft imageCtx.save()
let boxHeight = boxCoords.cvsBottom - boxCoords.cvsTop imageCtx.arc(cvsBox.left, cvsBox.top, 14 / this.canvasZoom, 0, 2 * Math.PI)
this.infoLinkPos.x = boxCoords.cvsLeft imageCtx.closePath()
this.infoLinkPos.y = boxCoords.cvsTop imageCtx.clip()
let boxMin = Math.min(boxHeight, boxWidth) imageCtx.drawImage(this.imageView,
this.infoLinkPos.adj = (boxMin >= 50) ? 0 : Math.min(10, 50 - boxMin) structBox.left - (14 / this.canvasZoom * imageScale),
structBox.top - (14 / this.canvasZoom * imageScale),
imageCtx.strokeRect(boxLeft, boxTop, boxWidth, boxHeight) (28 / this.canvasZoom * imageScale),
(28 / this.canvasZoom * imageScale),
cvsBox.left - (14 / this.canvasZoom),
cvsBox.top - (14 / this.canvasZoom),
(28 / this.canvasZoom), (28 / this.canvasZoom))
imageCtx.restore()
this.selectedChip = iChip this.selectedChip = iChip
this.resultData.detections[iChip].beenViewed = true this.resultData.detections[iChip].beenViewed = true
@@ -434,15 +515,24 @@
this.uploadDirty = true this.uploadDirty = true
}); });
}, },
resetView () { resetView (drawChip) {
const imCanvas = this.$refs.image_cvs const imCanvas = this.$refs.image_cvs
const imageCtx = imCanvas.getContext("2d") const imageCtx = imCanvas.getContext("2d")
imCanvas.width = imCanvas.clientWidth imCanvas.width = imCanvas.clientWidth
imCanvas.height = imCanvas.clientHeight imCanvas.height = imCanvas.clientHeight
imageCtx.clearRect(0,0,imCanvas.width,imCanvas.height) imageCtx.clearRect(0,0,imCanvas.width,imCanvas.height)
imageCtx.translate(this.canvasOffset.x,this.canvasOffset.y)
imageCtx.scale(this.canvasZoom,this.canvasZoom)
imageCtx.globalAlpha = 1 imageCtx.globalAlpha = 1
imageCtx.strokeStyle = 'yellow' imageCtx.strokeStyle = 'yellow'
imageCtx.lineWidth = 3 imageCtx.lineWidth = 3 / this.canvasZoom
if (this.imageLoaded) {
const imageLoc = imageLocation.getBoxes('side', this.imageView, imCanvas)
if (drawChip) {imageCtx.globalAlpha = .5}
imageCtx.drawImage(this.imageView, 0, 0, this.imageView.width, this.imageView.height, imageLoc[1].left, imageLoc[1].top, imageLoc[1].width, imageLoc[1].height)
if (drawChip) {imageCtx.globalAlpha = 1}
}
this.structureZoomed = false
return [imCanvas, imageCtx] return [imCanvas, imageCtx]
}, },
getImage (searchImage) { getImage (searchImage) {
@@ -450,18 +540,22 @@
if (this.videoAvailable) { if (this.videoAvailable) {
this.closeCamera() this.closeCamera()
this.detecting = true this.detecting = true
this.reloadModel = true reloadModel = true
resolve(searchImage) resolve(searchImage)
} else if (this.isCordova && this.imageLoadMode == "camera") { } else if (this.isCordova && imageLoadMode == "camera") {
this.detecting = true this.detecting = true
resolve('data:image/jpg;base64,' + searchImage) resolve('data:image/jpg;base64,' + searchImage)
} }
if (imageLoadMode == 'clipboard') {
this.detecting = true
resolve(searchImage)
}
const reader = new FileReader() const reader = new FileReader()
reader.addEventListener("load", () => { reader.addEventListener("load", () => {
this.detecting = true this.detecting = true
resolve(reader.result) resolve(reader.result)
},{once: true}) },{once: true})
if (this.imageLoadMode == 'sample') { if (imageLoadMode == 'sample') {
fetch(`${this.isCordova ? 'https://localhost' : '.'}/samples/${this.detectorName}-${searchImage}.jpeg`).then( resp => { fetch(`${this.isCordova ? 'https://localhost' : '.'}/samples/${this.detectorName}-${searchImage}.jpeg`).then( resp => {
return resp.blob() return resp.blob()
}).then(respBlob => { }).then(respBlob => {
@@ -483,22 +577,24 @@
this.imageView.src = imgData this.imageView.src = imgData
return(this.imageView.decode()) return(this.imageView.decode())
}).then( () => { }).then( () => {
const [imCanvas, _] = this.resetView() this.canvasOffset = {x: 0, y: 0}
imCanvas.style['background-image'] = `url(${this.imageView.src})` this.canvasZoom = 1
/****** const imCanvas = this.$refs.image_cvs
* setTimeout is not a good solution, but it's the only way imCanvas.width = imCanvas.clientWidth
* I can find to not cut off drawing of the canvas background imCanvas.height = imCanvas.clientHeight
******/ const imageCtx = imCanvas.getContext("2d")
// setTimeout(() => { const imageLoc = imageLocation.getBoxes('side', this.imageView, imCanvas)
imageCtx.drawImage(this.imageView, 0, 0, this.imageView.width, this.imageView.height, imageLoc[1].left, imageLoc[1].top, imageLoc[1].width, imageLoc[1].height)
f7.utils.nextFrame(() => {
this.setData() this.setData()
// }, 1) })
}).catch((e) => { }).catch((e) => {
console.log(e.message) console.log(e.message)
f7.dialog.alert(`Error loading image: ${e.message}`) f7.dialog.alert(`Error loading image: ${e.message}`)
}) })
}, },
async submitData () { async submitData () {
var uploadData = this.showResults let uploadData = this.showResults
.filter( d => { return d.aboveThreshold && d.isSearched && !d.isDeleted }) .filter( d => { return d.aboveThreshold && d.isSearched && !d.isDeleted })
.map( r => { return {"top": r.top, "left": r.left, "bottom": r.bottom, "right": r.right, "label": r.label}}) .map( r => { return {"top": r.top, "left": r.left, "bottom": r.bottom, "right": r.right, "label": r.label}})
this.uploadUid = await this.uploadData(this.imageView.src.split(',')[1],uploadData,this.uploadUid) this.uploadUid = await this.uploadData(this.imageView.src.split(',')[1],uploadData,this.uploadUid)
@@ -508,42 +604,85 @@
this.detectorLevel = value this.detectorLevel = value
}, },
structureClick(e) { structureClick(e) {
const boxCoords = this.box2cvs(this.showResults) let self = this
var findBox = boxCoords.findIndex( (r, i) => { return r.cvsLeft <= e.offsetX && function loopIndex(i) {
r.cvsRight >= e.offsetX && if (self.selectedChip == -1) return i
r.cvsTop <= e.offsetY && let li = i + self.selectedChip
r.cvsBottom >= e.offsetY && if (li >= numBoxes) li -= numBoxes
this.resultData.detections[i].resultIndex > this.selectedChip && return li
this.resultData.detections[i].aboveThreshold &&
this.resultData.detections[i].isSearched &&
!this.resultData.detections[i].isDeleted
})
this.selectChip(findBox >= 0 ? this.resultData.detections[findBox].resultIndex : this.selectedChip)
},
box2cvs(boxInput) {
if (!boxInput || boxInput.length == 0) return []
const boxList = boxInput.length ? boxInput : [boxInput]
const [imCanvas, imageCtx] = this.resetView()
var imgWidth
var imgHeight
const imgAspect = this.imageView.width / this.imageView.height
const rendAspect = imCanvas.width / imCanvas.height
if (imgAspect >= rendAspect) {
imgWidth = imCanvas.width
imgHeight = imCanvas.width / imgAspect
} else {
imgWidth = imCanvas.height * imgAspect
imgHeight = imCanvas.height
} }
const cvsCoords = boxList.map( (d, i) => { let boxCoords = []
return { this.resultData.detections.forEach(d => {
"cvsLeft": (imCanvas.width - imgWidth) / 2 + d.left * imgWidth, let cvsBox = d.box.getBoxes('point',this.imageView,this.$refs.image_cvs)[1]
"cvsRight": (imCanvas.width - imgWidth) / 2 + d.right * imgWidth, cvsBox.clickable = d.aboveThreshold && d.isSearched && !d.isDeleted
"cvsTop": (imCanvas.height - imgHeight) / 2 + d.top * imgHeight, boxCoords.push(cvsBox)
"cvsBottom": (imCanvas.height - imgHeight) / 2 + d.bottom * imgHeight
}
}) })
return cvsCoords const numBoxes = boxCoords.length
let clickX = (e.offsetX - this.canvasOffset.x) / this.canvasZoom
let clickY = (e.offsetY - this.canvasOffset.y) / this.canvasZoom
let boxEnd = boxCoords.splice(0, this.selectedChip)
boxCoords = boxCoords.concat(boxEnd)
const findBox = boxCoords.findIndex( (r, i) => {
let di = loopIndex(i)
if (di == this.selectedChip ) return false
return r.clickable &&
r.left <= clickX &&
r.right >= clickX &&
r.top <= clickY &&
r.bottom >= clickY
})
this.selectChip(findBox >= 0 ? this.resultData.detections[loopIndex(findBox)].resultIndex : this.selectedChip)
},
toggleSettings() {
this.showDetectSettings = !this.showDetectSettings
f7.utils.nextFrame(() => {
this.selectChip("redraw")
})
},
startMove() {
canvasMoving = true
},
endMove() {
canvasMoving = false
},
makeMove(event) {
if (canvasMoving) {
this.canvasOffset.x += event.movementX
this.canvasOffset.y += event.movementY
this.selectChip("redraw")
}
},
spinWheel(event) {
let zoomFactor
if (event.wheelDelta > 0) {
zoomFactor = 1.05
} else if (event.wheelDelta < 0) {
zoomFactor = 1 / 1.05
}
this.canvasZoom *= zoomFactor
this.canvasOffset.x = event.offsetX * (1 - zoomFactor) + this.canvasOffset.x * zoomFactor
this.canvasOffset.y = event.offsetY * (1 - zoomFactor) + this.canvasOffset.y * zoomFactor
this.selectChip("redraw")
},
resetZoom() {
this.canvasZoom = 1
this.canvasOffset.x = 0
this.canvasOffset.y = 0
this.selectChip("redraw")
},
zoomToSelected() {
const imCanvas = this.$refs.image_cvs
const boxCoords = this.resultData.detections[this.selectedChip].box.getBoxes('point', this.imageView, imCanvas)
const boxWidth = boxCoords[1].right - boxCoords[1].left
const boxHeight = boxCoords[1].bottom - boxCoords[1].top
const boxMidX = (boxCoords[1].right + boxCoords[1].left ) / 2
const boxMidY = (boxCoords[1].bottom + boxCoords[1].top ) / 2
const zoomFactor = Math.min(imCanvas.width / boxWidth * .9, imCanvas.height / boxHeight * .9, 8)
this.canvasZoom = zoomFactor
this.canvasOffset.x = -(boxMidX * zoomFactor) + imCanvas.width / 2
this.canvasOffset.y = -(boxMidY * zoomFactor) + imCanvas.height / 2
this.selectChip("redraw")
this.structureZoomed = true
} }
} }
} }

View File

@@ -1,11 +1,118 @@
import * as tf from '@tensorflow/tfjs'
import { f7 } from 'framework7-vue' import { f7 } from 'framework7-vue'
let model = null
export default { export default {
methods: { methods: {
async loadModel(weights, preload) {
if (model && model.modelURL == weights) {
return model
} else if (model) {
tf.dispose(model)
}
model = await tf.loadGraphModel(weights)
const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3)
/*****************
* If preloading then run model
* once on fake data to preload
* weights for a faster response
*****************/
if (preload) {
const dummyT = tf.ones([1,modelWidth,modelHeight,3])
model.predict(dummyT)
}
return model
},
async localDetect(imageData) {
console.time('mx: pre-process')
const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3)
let gTense = null
const input = tf.tidy(() => {
gTense = tf.image.rgbToGrayscale(tf.image.resizeBilinear(tf.browser.fromPixels(imageData), [modelWidth, modelHeight])).div(255.0).expandDims(0)
return tf.concat([gTense,gTense,gTense],3)
})
tf.dispose(gTense)
console.timeEnd('mx: pre-process')
console.time('mx: run prediction')
const res = model.predict(input)
const tRes = tf.transpose(res,[0,2,1])
const rawRes = tRes.arraySync()[0]
console.timeEnd('mx: run prediction')
console.time('mx: post-process')
const outputSize = res.shape[1]
let rawBoxes = []
let rawScores = []
for (let i = 0; i < rawRes.length; i++) {
const getScores = rawRes[i].slice(4)
if (getScores.every( s => s < .05)) { continue }
const getBox = rawRes[i].slice(0,4)
const boxCalc = [
(getBox[0] - (getBox[2] / 2)) / modelWidth,
(getBox[1] - (getBox[3] / 2)) / modelHeight,
(getBox[0] + (getBox[2] / 2)) / modelWidth,
(getBox[1] + (getBox[3] / 2)) / modelHeight,
]
rawBoxes.push(boxCalc)
rawScores.push(getScores)
}
if (rawBoxes.length > 0) {
const tBoxes = tf.tensor2d(rawBoxes)
let tScores = null
let resBoxes = null
let validBoxes = []
let structureScores = null
let boxes_data = []
let scores_data = []
let classes_data = []
for (let c = 0; c < outputSize - 4; c++) {
structureScores = rawScores.map(x => x[c])
tScores = tf.tensor1d(structureScores)
resBoxes = await tf.image.nonMaxSuppressionAsync(tBoxes,tScores,10,0.5,.05)
validBoxes = resBoxes.dataSync()
tf.dispose(resBoxes)
if (validBoxes) {
boxes_data.push(...rawBoxes.filter( (_, idx) => validBoxes.includes(idx)))
let outputScores = structureScores.filter( (_, idx) => validBoxes.includes(idx))
scores_data.push(...outputScores)
classes_data.push(...outputScores.fill(c))
}
}
validBoxes = []
tf.dispose(tBoxes)
tf.dispose(tScores)
tf.dispose(tRes)
const valid_detections_data = classes_data.length
const output = {
detections: []
}
for (let i =0; i < valid_detections_data; i++) {
const [dLeft, dTop, dRight, dBottom] = boxes_data[i]
output.detections.push({
"top": dTop,
"left": dLeft,
"bottom": dBottom,
"right": dRight,
"label": this.detectorLabels[classes_data[i]].name,
"confidence": scores_data[i] * 100
})
}
}
tf.dispose(res)
tf.dispose(input)
console.timeEnd('mx: post-process')
return output || { detections: [] }
},
getRemoteLabels() { getRemoteLabels() {
var self = this let self = this
var modelURL = `http://${this.serverSettings.address}:${this.serverSettings.port}/detectors` const modelURL = `http://${this.serverSettings.address}:${this.serverSettings.port}/detectors`
var xhr = new XMLHttpRequest() let xhr = new XMLHttpRequest()
xhr.open("GET", modelURL) xhr.open("GET", modelURL)
xhr.setRequestHeader('Content-Type', 'application/json') xhr.setRequestHeader('Content-Type', 'application/json')
xhr.timeout = 10000 xhr.timeout = 10000
@@ -17,8 +124,8 @@ export default {
f7.dialog.alert(`ALVINN has encountered an error: ${errorResponse.error}`) f7.dialog.alert(`ALVINN has encountered an error: ${errorResponse.error}`)
return return
} }
var detectors = JSON.parse(xhr.response).detectors const detectors = JSON.parse(xhr.response).detectors
var findLabel = detectors let findLabel = detectors
.find( d => { return d.name == self.detectorName } )?.labels .find( d => { return d.name == self.detectorName } )?.labels
.filter( l => { return l != "" } ).sort() .filter( l => { return l != "" } ).sort()
.map( l => { return {'name': l, 'detect': true} } ) .map( l => { return {'name': l, 'detect': true} } )
@@ -32,9 +139,9 @@ export default {
xhr.send() xhr.send()
}, },
remoteDetect() { remoteDetect() {
var self = this let self = this
var modelURL = `http://${this.serverSettings.address}:${this.serverSettings.port}/detect` const modelURL = `http://${this.serverSettings.address}:${this.serverSettings.port}/detect`
var xhr = new XMLHttpRequest() let xhr = new XMLHttpRequest()
xhr.open("POST", modelURL) xhr.open("POST", modelURL)
xhr.timeout = 10000 xhr.timeout = 10000
xhr.ontimeout = this.remoteTimeout xhr.ontimeout = this.remoteTimeout
@@ -51,7 +158,7 @@ export default {
self.uploadDirty = true self.uploadDirty = true
} }
var doodsData = { const doodsData = {
"detector_name": this.detectorName, "detector_name": this.detectorName,
"detect": { "detect": {
"*": 1 "*": 1
@@ -65,5 +172,64 @@ export default {
this.detecting = false this.detecting = false
f7.dialog.alert('No connection to remote ALVINN instance. Please check app settings.') f7.dialog.alert('No connection to remote ALVINN instance. Please check app settings.')
}, },
async videoFrameDetect (vidData, miniModel) {
await this.loadModel(miniModel)
const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3)
const imCanvas = this.$refs.image_cvs
const imageCtx = imCanvas.getContext("2d")
const target = this.$refs.target_image
await tf.nextFrame();
imCanvas.width = imCanvas.clientWidth
imCanvas.height = imCanvas.clientHeight
imageCtx.clearRect(0,0,imCanvas.width,imCanvas.height)
let imgWidth, imgHeight
const imgAspect = vidData.width / vidData.height
const rendAspect = imCanvas.width / imCanvas.height
if (imgAspect >= rendAspect) {
imgWidth = imCanvas.width
imgHeight = imCanvas.width / imgAspect
} else {
imgWidth = imCanvas.height * imgAspect
imgHeight = imCanvas.height
}
while (this.videoAvailable) {
console.time('mx: frame-process')
try {
const input = tf.tidy(() => {
return tf.image.resizeBilinear(tf.browser.fromPixels(vidData), [modelWidth, modelHeight]).div(255.0).expandDims(0)
})
const res = model.predict(input)
const rawRes = tf.transpose(res,[0,2,1]).arraySync()[0]
let rawCoords = []
if (rawRes) {
for (let i = 0; i < rawRes.length; i++) {
let getScores = rawRes[i].slice(4)
if (getScores.some( s => s > .5)) {
let foundTarget = rawRes[i].slice(0,2)
foundTarget.push(Math.max(...getScores))
rawCoords.push(foundTarget)
}
}
imageCtx.clearRect(0,0,imCanvas.width,imCanvas.height)
for (let coord of rawCoords) {
console.log(`x: ${coord[0]}, y: ${coord[1]}`)
let pointX = (imCanvas.width - imgWidth) / 2 + (coord[0] / modelWidth) * imgWidth -5
let pointY = (imCanvas.height - imgHeight) / 2 + (coord[1] / modelHeight) * imgHeight -5
imageCtx.globalAlpha = coord[2]
imageCtx.drawImage(target, pointX, pointY, 20, 20)
}
}
tf.dispose(input)
tf.dispose(res)
tf.dispose(rawRes)
} catch (e) {
console.log(e)
}
console.timeEnd('mx: frame-process')
await tf.nextFrame();
}
}
} }
} }

View File

@@ -21,6 +21,7 @@
</ul> </ul>
</li> </li>
<li>Click on the image file icon <SvgIcon icon="photo_library" class="list-svg"/> to load a picture from the device storage.</li> <li>Click on the image file icon <SvgIcon icon="photo_library" class="list-svg"/> to load a picture from the device storage.</li>
<li>If the clipboard is available on the system, then there will be a paste icon <SvgIcon icon="clipboard" class="list-svg"/> to paste image data directly into the app.</li>
<li>If demo mode is turned on, you can click on the marked image icon <SvgIcon icon="photo_sample" class="list-svg"/> to load an ALVINN sample image.</li> <li>If demo mode is turned on, you can click on the marked image icon <SvgIcon icon="photo_sample" class="list-svg"/> to load an ALVINN sample image.</li>
</ul> </ul>
</li> </li>
@@ -30,8 +31,11 @@
<li>Click on each tag to see the structure highlighted in the image or click on the image to see the tag for that structure (additional clicks to the same area will select overlapping structres).</li> <li>Click on each tag to see the structure highlighted in the image or click on the image to see the tag for that structure (additional clicks to the same area will select overlapping structres).</li>
<li>Tag color and proportion filled indicate ALVINN's level of confidence in the identification.</li> <li>Tag color and proportion filled indicate ALVINN's level of confidence in the identification.</li>
<li>An incorrect tag can be deleted by clicking on the tag's <f7-icon icon="chip-delete" style="margin-right: 1px;"></f7-icon> button.</li> <li>An incorrect tag can be deleted by clicking on the tag's <f7-icon icon="chip-delete" style="margin-right: 1px;"></f7-icon> button.</li>
<li>Click on the zoom to structure button <SvgIcon icon="zoom_to" class="list-svg"/> to magnify the view of the selected structure</li>
</ul> </ul>
</li> </li>
<li>Pan (middle click or touch and drag) and zoom (mouse wheel or pinch) to manually select detailed views in the image.</li>
<li>The reset zoom button <SvgIcon icon="reset_zoom" class="list-svg"/> will return the image to its initial position and magnification.</li>
</ol> </ol>
<h2>Advanced Features</h2> <h2>Advanced Features</h2>
<h3>Detection Parameters</h3> <h3>Detection Parameters</h3>

View File

@@ -97,8 +97,7 @@
</style> </style>
<script> <script>
import { touchstart } from 'dom7' import RegionIcon from '../components/region-icon.vue'
import RegionIcon from '../components/region-icon.vue'
import store from '../js/store' import store from '../js/store'
import { f7 } from 'framework7-vue' import { f7 } from 'framework7-vue'
@@ -113,6 +112,11 @@ import RegionIcon from '../components/region-icon.vue'
} }
}, },
setup() { setup() {
//URL TESTING CODE
//let testUrl = URL.parse(`../models/thorax/model.json`,import.meta.url).href
//console.log(testUrl)
//let testUrl2 = new URL(`../models/thorax/model.json`,import.meta.url)
//console.log(testUrl2)
return store() return store()
}, },
methods: { methods: {

View File

@@ -91,7 +91,7 @@
computed: { computed: {
otherIp () { otherIp () {
let filteredIps = {} let filteredIps = {}
for (var oldIp in this.serverSettings.previous) { for (let oldIp in this.serverSettings.previous) {
if (oldIp != this.serverSettings.address) { if (oldIp != this.serverSettings.address) {
filteredIps[oldIp] = this.serverSettings.previous[oldIp] filteredIps[oldIp] = this.serverSettings.previous[oldIp]
} }
@@ -109,12 +109,12 @@
} }
}, },
created () { created () {
var loadServerSettings = localStorage.getItem('serverSettings') const loadServerSettings = localStorage.getItem('serverSettings')
if (loadServerSettings) this.serverSettings = JSON.parse(loadServerSettings) if (loadServerSettings) this.serverSettings = JSON.parse(loadServerSettings)
if (!this.serverSettings.previous) this.serverSettings.previous = {} if (!this.serverSettings.previous) this.serverSettings.previous = {}
var loadThemeSettings = localStorage.getItem('themeSettings') const loadThemeSettings = localStorage.getItem('themeSettings')
if (loadThemeSettings) this.themeSettings = JSON.parse(loadThemeSettings) if (loadThemeSettings) this.themeSettings = JSON.parse(loadThemeSettings)
var loadOtherSettings = localStorage.getItem('otherSettings') const loadOtherSettings = localStorage.getItem('otherSettings')
if (loadOtherSettings) this.otherSettings = JSON.parse(loadOtherSettings) if (loadOtherSettings) this.otherSettings = JSON.parse(loadOtherSettings)
}, },
methods: { methods: {
@@ -136,7 +136,7 @@
) )
saveSetting.then( saveSetting.then(
() => { () => {
var toast = f7.toast.create({ const toast = f7.toast.create({
text: 'Settings saved', text: 'Settings saved',
closeTimeout: 2000 closeTimeout: 2000
}) })
@@ -144,7 +144,7 @@
this.isDirty = false; this.isDirty = false;
}, },
() => { () => {
var toast = f7.toast.create({ const toast = f7.toast.create({
text: 'ERROR: No settings saved', text: 'ERROR: No settings saved',
closeTimeout: 2000 closeTimeout: 2000
}) })

View File

@@ -8,6 +8,8 @@
<f7-block-title medium>Details</f7-block-title> <f7-block-title medium>Details</f7-block-title>
<f7-list> <f7-list>
<f7-list-item title="Version" :after="alvinnVersion"></f7-list-item> <f7-list-item title="Version" :after="alvinnVersion"></f7-list-item>
<f7-list-item title="Build" :after="alvinnBuild"></f7-list-item>
<f7-list-item title="Workers" :after="useWorkers ? 'Enabled' : 'Disabled'"></f7-list-item>
</f7-list> </f7-list>
<f7-block-title medium>Models</f7-block-title> <f7-block-title medium>Models</f7-block-title>
<f7-list style="width: 100%;"> <f7-list style="width: 100%;">
@@ -51,7 +53,9 @@
headneckDetails: {}, headneckDetails: {},
miniHeadneckDetails: {}, miniHeadneckDetails: {},
alvinnVersion: store().getVersion, alvinnVersion: store().getVersion,
alvinnBuild: store().getBuild,
isCordova: !!window.cordova, isCordova: !!window.cordova,
useWorkers: store().useWorkers,
otherSettings: {} otherSettings: {}
} }
}, },
@@ -59,7 +63,7 @@
return store() return store()
}, },
created () { created () {
var loadOtherSettings = localStorage.getItem('otherSettings') const loadOtherSettings = localStorage.getItem('otherSettings')
if (loadOtherSettings) this.otherSettings = JSON.parse(loadOtherSettings) if (loadOtherSettings) this.otherSettings = JSON.parse(loadOtherSettings)
fetch(`${this.isCordova ? 'https://localhost' : '.'}/models/thorax/descript.json`) fetch(`${this.isCordova ? 'https://localhost' : '.'}/models/thorax/descript.json`)
.then((mod) => { return mod.json() }) .then((mod) => { return mod.json() })

View File

@@ -5,8 +5,8 @@ export default {
newUid (length) { newUid (length) {
const uidLength = length || 16 const uidLength = length || 16
const uidChars = 'abcdefghijklmnopqrstuvwxyz0123456789' const uidChars = 'abcdefghijklmnopqrstuvwxyz0123456789'
var uid = [] let uid = []
for (var i = 0; i < uidLength; i++) { for (let i = 0; i < uidLength; i++) {
uid.push(uidChars.charAt(Math.floor(Math.random() * ((i < 4) ? 26 : 36)))) uid.push(uidChars.charAt(Math.floor(Math.random() * ((i < 4) ? 26 : 36))))
} }
return uid.join('') return uid.join('')
@@ -14,24 +14,23 @@ export default {
uploadData (imagePayload, classPayload, prevUid) { uploadData (imagePayload, classPayload, prevUid) {
let uploadImage = new Promise (resolve => { let uploadImage = new Promise (resolve => {
const dataUid = prevUid || this.newUid(16) const dataUid = prevUid || this.newUid(16)
var byteChars = window.atob(imagePayload) let byteChars = window.atob(imagePayload)
var byteArrays = [] let byteArrays = []
var len = byteChars.length
for (var offset = 0; offset < len; offset += 1024) { for (let offset = 0; offset < byteChars.length; offset += 1024) {
var slice = byteChars.slice(offset, offset + 1024) let slice = byteChars.slice(offset, offset + 1024)
var byteNumbers = new Array(slice.length) let byteNumbers = new Array(slice.length)
for (var i = 0; i < slice.length; i++) { for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i) byteNumbers[i] = slice.charCodeAt(i)
} }
var byteArray = new Uint8Array(byteNumbers) let byteArray = new Uint8Array(byteNumbers)
byteArrays.push(byteArray) byteArrays.push(byteArray)
} }
var imageBlob = new Blob(byteArrays, {type: 'image/jpeg'}) const imageBlob = new Blob(byteArrays, {type: 'image/jpeg'})
var xhrJpg = new XMLHttpRequest() let xhrJpg = new XMLHttpRequest()
var uploadUrl = `https://nextcloud.azgeorgis.net/public.php/webdav/${dataUid}.jpeg` let uploadUrl = `https://nextcloud.azgeorgis.net/public.php/webdav/${dataUid}.jpeg`
xhrJpg.open("PUT", uploadUrl) xhrJpg.open("PUT", uploadUrl)
xhrJpg.setRequestHeader('Content-Type', 'image/jpeg') xhrJpg.setRequestHeader('Content-Type', 'image/jpeg')
xhrJpg.setRequestHeader('X-Method-Override', 'PUT') xhrJpg.setRequestHeader('X-Method-Override', 'PUT')
@@ -39,8 +38,8 @@ export default {
xhrJpg.setRequestHeader("Authorization", "Basic " + btoa("LKBm3H6JdSaywyg:")) xhrJpg.setRequestHeader("Authorization", "Basic " + btoa("LKBm3H6JdSaywyg:"))
xhrJpg.send(imageBlob) xhrJpg.send(imageBlob)
var xhrTxt = new XMLHttpRequest() let xhrTxt = new XMLHttpRequest()
var uploadUrl = `https://nextcloud.azgeorgis.net/public.php/webdav/${dataUid}.txt` uploadUrl = `https://nextcloud.azgeorgis.net/public.php/webdav/${dataUid}.txt`
xhrTxt.open("PUT", uploadUrl) xhrTxt.open("PUT", uploadUrl)
xhrTxt.setRequestHeader('Content-Type', 'text/plain') xhrTxt.setRequestHeader('Content-Type', 'text/plain')
xhrTxt.setRequestHeader('X-Method-Override', 'PUT') xhrTxt.setRequestHeader('X-Method-Override', 'PUT')
@@ -51,7 +50,7 @@ export default {
resolve(dataUid) resolve(dataUid)
}) })
return uploadImage.then((newUid) => { return uploadImage.then((newUid) => {
var toast = f7.toast.create({ const toast = f7.toast.create({
text: 'Detections Uploaded: thank you.', text: 'Detections Uploaded: thank you.',
closeTimeout: 2000 closeTimeout: 2000
}) })

51
src/pages/touch-mixin.js Normal file
View File

@@ -0,0 +1,51 @@
export default {
data () {
return {
touchPrevious: {}
}
},
methods: {
startTouch(event) {
if (event.touches.length == 1) {
this.touchPrevious = {x: event.touches[0].clientX, y: event.touches[0].clientY}
}
if (event.touches.length == 2) {
let midX = (event.touches.item(0).clientX + event.touches.item(1).clientX) / 2
let midY = (event.touches.item(0).clientY + event.touches.item(1).clientY) / 2
this.touchPrevious = {distance: this.touchDistance(event.touches), x: midX, y: midY}
}
},
endTouch(event) {
if (event.touches.length == 1) {
this.touchPrevious = {x: event.touches[0].clientX, y: event.touches[0].clientY}
} else {
//this.debugInfo = null
}
},
moveTouch(event) {
switch (event.touches.length) {
case 1:
this.canvasOffset.x += event.touches[0].clientX - this.touchPrevious.x
this.canvasOffset.y += event.touches[0].clientY - this.touchPrevious.y
this.touchPrevious = {x: event.touches[0].clientX, y: event.touches[0].clientY}
break;
case 2:
let newDistance = this.touchDistance(event.touches)
let midX = (event.touches.item(0).clientX + event.touches.item(1).clientX) / 2
let midY = (event.touches.item(0).clientY + event.touches.item(1).clientY) / 2
let zoomFactor = newDistance / this.touchPrevious.distance
this.canvasZoom *= zoomFactor
this.canvasOffset.x = (midX - 16) * (1 - zoomFactor) + this.canvasOffset.x * zoomFactor + (midX - this.touchPrevious.x)
this.canvasOffset.y = (midY - 96) * (1 - zoomFactor) + this.canvasOffset.y * zoomFactor + (midY - this.touchPrevious.y)
this.touchPrevious = {distance: newDistance, x: midX, y: midY}
break;
}
this.selectChip("redraw")
},
touchDistance(touches) {
let touch1 = touches.item(0)
let touch2 = touches.item(1)
return Math.sqrt((touch1.clientX - touch2.clientX) ** 2 + (touch1.clientY - touch2.clientY) ** 2)
}
}
}