2025 could be a breakout year for Computer Vision.

Hyperpunk & techni-optimist. Co-founder, Scout Game.
https://scoutgame.xyz

How far do you think we are from realtime depth mapping with just rgb smartphone cameras? That feels like it’ll be the real turning point for deployment of next gen CV.

Rated 💯 by multiple networks. Fluent in /unglish. Unbothered by /whatday of the week it is.

Allegedly a designer.

I'm not really an expert in computer vision, so its hard to tell on that timing but agree it would be a next level unlock. Its also one of those things that's a bummer that we only really have 2 phone options and Apple is unlikely to add special hardware for this until they believe in AI.

creating and destroying; built flying cars & autonomous vehicles, now building roc.camera, writings: july.rocks

tagging in @july to share some of his thoughts

My feeling is that it kinda has been for the past few years, and just doesn't seem to stop -- increasingly there is an on-going convergence of CV with General AI/ML - you saw it initially when CNNs blew every conventional computer vision methods (feature extraction, feature matching) with CNNs like AlexNet, Yolo, etc. Now with ViT vision transformers, and VLMs and the like. SAM (segment anything) is pretty amazing

That being said, edge perception stuff in the context of robotics, devices, AR/VR, and devices in general -- CV continues to be relevant. TinyML, perception models that are energy efficient and work on the edge GPU that it's meant to run on - prob a lot more work to do here, but if you look at companies like ST and TI, all they are making are MCUs that can run ML on it, so...

Also I am excited about 3D Gaussian splatting in the same way that I was excited aobut NeRFs, but 3D Gaussian splatting is a lot more efficient than NeRF - it can handle a lot more, order of magnitude more FPS. So that's cool

ensdata.net ✦ pugson.net/rfe ✦ metadata.vision ✦ 𝙿𝚁𝙴𝚅: @rainbow 🌈 ✦ 𝙽𝙾𝚆: 𝚆𝙾𝚁𝙺𝙸𝙽𝙶 𝙾𝙽 ???

some of my thoughts:

https://warpcast.com/july/0x555b7274

Robotics

I know I know everyone on the TL is excited about Gaussian Splatting and NeRF (neural radiance fields) is old news at this point but this is a great intro to NeRF:

https://youtu.be/Z96YMktT_T4

For fun, I've been thinking a lot about 3D Gaussian Splatting & NeRFs - mostly out of my own curiosity. Specifically on how (in the future) on could take a series of photos or videso to create SfM data (structure from motion) -- turn it into point cloud or use COLMAP, and then use 3D Gaussian Splatting to make it super easy to view (on some sort of web view?) Perhaps in the long run we can have Roc Camera go not only take verifiably real photos but also capture verifiably real worlds. 

gsplat seems like a pretty cool tool - here's a real-time large scale rendering of over 30 millions Gaussians in total using gsplat

also this is not a feature in our roadmap, just thinking out loud how it would be cool

https://docs.gsplat.studio/main/examples/large_scale.html

In the longer run, I can see devices / robots doing something like this - start with SLAM (simultaneous localization and mapping) - which many vehicles or robots constantly do already to localize themselves in their reference frame - SLAM provides the poses, you feed image frames / point cloud / camera video into and use something like 3D Gaussian splatting to update the 3D representation pretty fast (much faster than NeRF) - and there's papers out there doing dynamic 3d gaussian splatting (instead of a still image, think dynamically changing video) - then one can move through or see that 3d representation that is dyanmically changing in real-time - so you can do cool things like put virtual objects in there (AR/VR) or get reference frames / better navigation, odometry etc (robots / vehicles) or just real-time visualization / interaction

Have you seen realtime Gaussian splatting (or anything else producing high detail low latency depth maps) on commodity smartphones using only rgb cameras?