Add camera stream #30

Closed
opened 2023-12-03 22:42:43 +00:00 by jgeorgi · 4 comments
Owner

Don't know if the doods connections are generally going to be good enough to run on a stream, but this is an end goal eventually.

Don't know if the doods connections are generally going to be good enough to run on a stream, but this is an end goal eventually.
jgeorgi added the
enhancement
label 2023-12-03 22:42:43 +00:00
jgeorgi self-assigned this 2023-12-03 22:42:43 +00:00
jgeorgi added this to the ALVINN project 2023-12-03 22:42:44 +00:00
Author
Owner

Not sure this is even feasible with cordova. Pytorch example already has the built-it camera demo, but that's local video, not streaming to an external server.

Not sure this is even feasible with cordova. Pytorch example already has the built-it camera demo, but that's local video, not streaming to an external server.
Author
Owner

Perhaps something like this can be used to at least simulate what I'm looking for (especially when combined with a websocket connection).

https://github.com/cordova-plugin-camera-preview/cordova-plugin-camera-preview

At the very least, this can probably be used to show the image capture right in the detect screen instead of a new view.

This is a bust: it doesn't seem to work at all on the browsers. If this is going to happen I may need separate browser and app methods.

Perhaps something like this can be used to at least simulate what I'm looking for (especially when combined with a websocket connection). https://github.com/cordova-plugin-camera-preview/cordova-plugin-camera-preview At the very least, this can probably be used to show the image capture right in the detect screen instead of a new view. This is a bust: it doesn't seem to work at all on the browsers. If this is going to happen I may need separate browser and app methods.
Author
Owner

#116 Now makes this significantly more likely (at least in the PWA deployment). The biggest hurdle now, I think, is the speed of the local detection.

#116 Now makes this significantly more likely (at least in the PWA deployment). The biggest hurdle now, I think, is the speed of the local detection.
Author
Owner

I think this is no longer a separate feature. I've been trying to resolve how to make this work without making a really ugly, too cluttered video frame. The answer is that it is integrated into the camera image capture (again, on the PWA, I don't know if this can be done with cordova or not, given how that camera access works). When the camera is running and ready to capture, the video feed can be processed and with little post-processing, any structure over a certain threshold (80% ?) will just be indicated by a dot at the structures center point. No, recalculating bounding boxes, no NMS, just one quick reshape and filter.

The hard part might even be done already depending on this:
https://js.tensorflow.org/api/latest/#data.webcam

I think this is no longer a separate feature. I've been trying to resolve how to make this work without making a really ugly, too cluttered video frame. The answer is that it is integrated into the camera image capture (again, on the PWA, I don't know if this can be done with cordova or not, given how that camera access works). When the camera is running and ready to capture, the video feed can be processed and with little post-processing, any structure over a certain threshold (80% ?) will just be indicated by a dot at the structures center point. No, recalculating bounding boxes, no NMS, just one quick reshape and filter. The hard part might even be done already depending on this: https://js.tensorflow.org/api/latest/#data.webcam
jgeorgi added this to the 0.5.0 milestone 2024-03-08 02:44:23 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ALVINN/ALVINN_f7#30
No description provided.