Interesting: what if your OS could "see" UI elements and react to them?

This video is a bit academic, but it’s an interesting idea. UI elements like buttons, scroll bars, and so on, are hard coded into windows and things, of course. But they’re also visually distinctive. What if your computer could see these interactive elements and adjust your input (say, slowing down your mouse when you hit a button) based only on that?

It’s nice to know that there are people working on crazy high-level UI stuff like this. When you think about how little windows and icons have changed since the early days, it’s kind of depressing. Maybe this kind of thing isn’t super exciting right now, but you take a few of these concepts and integrate them into an OS and it may provide a whole new experience.