Deep Science: Introspective, detail-oriented and disaster-chasing AIs


Image Credits: Kehan Chen (opens in a new window) / Getty Images

Research papers come out far too frequently for anyone to read them all. That’s especially true in the field of machine learning, which now affects (and produces papers in) practically every industry and company. This column aims to collect some of the most relevant recent discoveries and papers — particularly in, but not limited to, artificial intelligence — and explain why they matter.

It takes an emotionally mature AI to admit its own mistakes, and that’s exactly what this project from the Technical University of Munich aims to create. Maybe not the emotion, exactly, but recognizing and learning from mistakes, specifically in self-driving cars. The researchers propose a system in which the car would look at all the times in the past when it has had to relinquish control to a human driver and thereby learn its own limitations — what they call “introspective failure prediction.”

For instance, if there are a lot of cars ahead, the autonomous vehicle’s brain could use its sensors and logic to make a decision de novo about whether an approach would work or whether none will. But the TUM team says that by simply comparing new situations to old ones, it can reach a decision much faster on whether it will need to disengage. Saving six or seven seconds here could make all the difference for a safe handover.

It’s important for robots and autonomous vehicles of all types to be able to make decisions without phoning home, especially in combat, where decisive and concise movements are necessary. The Army Research Lab is looking into ways in which ground and air vehicles can interact autonomously, allowing, for instance, a mobile landing pad that drones can land on without needing to coordinate, ask permission or rely on precise GPS signals.

Their solution, at least for the purposes of testing, is actually rather low tech. The ground vehicle has a landing area on top painted with an enormous QR code, which the drone can see from a fairly long way off. The drone can track the exact location of the pad totally independently. In the future, the QR code could be done away with and the drone could identify the shape of the vehicle instead, presumably using some best-guess logic to determine whether it’s the one it wants.

Illustration showing how an AI tracks cells through a microscope.
Image Credits: Nagoya City University

In the medical world, AI is being put to work not on tasks that are not much difficult but are rather tedious for people to do. A good example of this is tracking the activity of individual cells in microscopy images. It’s not a superhuman task to look at a few hundred frames spanning several depths of a petri dish and track the movements of cells, but that doesn’t mean grad students like doing it.

This software from researchers at Nagoya City University in Japan does it automatically using image analysis and the capability (much improved in recent years) of understanding objects over a period of time rather than just in individual frames. Read the paper here, and check out the extremely cute illustration showing off the tech at right … more research organizations should hire professional artists.

This process is similar to that of tracking moles and other skin features on people at risk for melanoma. While they might see a dermatologist every year or so to find out whether a given spot seems sketchy, the rest of the time they must track their own moles and freckles in other ways. That’s hard when they’re in places like one’s back.

MIT researchers have created a machine learning model that can identify and classify potentially dangerous skin pigmentation, flagging the riskiest ones for further inspection. It was trained on more than 20,000 images from 133 patients, and performed very well in a lab setting, catching 90% of the “suspicious pigmented lesions” experts located.

Animated image showing how an algorithm sorts moles into different categories.
The colors correspond to different predicted risk levels. Image Credits: MIT

The question of how to get a good shot of your own back isn’t clear yet, but a partner or tripod could help out there, and of course this could be done at a doctor’s office as well before calling in the dermatologist. You can read the paper in Science Translational Medicine.

Interestingly, here’s one situation where researchers opted not to use computer vision — spotting cows (and elk) from space to track their movements. Ten undergrads manually highlighted 31,000 cows … you can see why some labs opt to train a cow-spotting (or sea cow-spotting) AI instead.

A similar task loathed by photographers is watching for “hot pixels,” where a single pixel in a camera’s sensor or image processing pipeline shows up as far brighter than the rest. This can be fixed easily in post-processing, of course, but it would be better if it could be eliminated entirely. Samsung published a paper that proposes exactly that, and on a scale that looks like a photographer’s worst nightmare.

Close-up of pixels in an image, along with which hot pixels were missed.
Left to right: The image with added hot pixels, false positives and misses. Image Credits: Samsung

Spotting these hot pixels isn’t a particularly challenging task — they’re very distinctive — but that doesn’t mean it’s easy to build a machine learning process like this into the existing pipeline. However, once an approach is proven to be effective, work can progress to efficient implementation on hardware. The model created by Samsung missed only a fraction of a percent of artificial hot pixels, but also had a pretty high false positive rate in high-contrast regions, so there’s still work to be done.

Not all patterns can afford to be picked up after the fact. Seismic activity, for instance, may be scrutinized for the possibility that it might lead to landslides or debris flow — the latter of which is the idea behind this system from ETH Zurich.

An area in Switzerland subject to dangerous debris flows (like a chunky, extra-dangerous mudslide) has been monitored for more than 10 years, but researchers were unhappy with the short time frame of the warnings it produced — in cases of natural disaster, every second counts. They installed seismometers, but had to build a machine learning model that could distinguish the characteristic vibrations of debris flow from the ordinary seismic motion produced in a mountainous area.

Trained on signals from previous flows, the system detected every one of 13 debris flows that occurred in the summer of 2020, and provided more than 20 minutes’ improvement in warning for those downstream. Now they’re looking at whether this model can be generalized to detect flows in other high risk areas. You can read the paper here.

Black and white satellite image of a city annotated with color flooding data.
Image Credits: Tohoku University

After a disaster, there is almost always a secondary crisis of information, since communications networks are frequently down or spiking in usage. Social media, in particular, tends to be flooded with messages and media, from “Whoa, did anyone else feel that quake?” to “The power lines are down right outside my building!” Now which of these messages would you want to know about, as an emergency responder?

At Virginia Tech, researchers are looking into quick, accurate assessment of social media content in the context of a disaster with the aim of locating people and places that should be prioritized. The machine learning model they created was trained with bespoke data, human-annotated tweets that let a computer make a preliminary assessment of things that should be looked at.

And at Tohoku University, they have the interesting idea of mining news footage for imagery of areas affected by floods, extrapolating from the footage and other data which buildings will have serious damage. It’s a short jump from there to Instagram and TikTok videos showing the same, but those also present privacy issues.


More TechCrunch

After Apple loosened its App Store guidelines to permit game emulators, the retro game emulator Delta — an app 10 years in the making — hit the top of the…

Adobe comes after indie game emulator Delta for copying its logo

Meta is once again taking on its competitors by developing a feature that borrows concepts from others — in this case, BeReal and Snapchat. The company is developing a feature…

Meta’s latest experiment borrows from BeReal’s and Snapchat’s core ideas

Welcome to Startups Weekly! We’ve been drowning in AI news this week, with Google’s I/O setting the pace. And Elon Musk rages against the machine.

Startups Weekly: It’s the dawning of the age of AI — plus,  Musk is raging against the machine

IndieBio’s Bay Area incubator is about to debut its 15th cohort of biotech startups. We took special note of a few, which were making some major, bordering on ludicrous, claims…

IndieBio’s SF incubator lineup is making some wild biotech promises

YouTube TV has announced that its multiview feature for watching four streams at once is now available on Android phones and tablets. The Android launch comes two months after YouTube…

YouTube TV’s ‘multiview’ feature is now available on Android phones and tablets

Featured Article

Two Santa Cruz students uncover security bug that could let millions do their laundry for free

CSC ServiceWorks provides laundry machines to thousands of residential homes and universities, but the company ignored requests to fix a security bug.

9 hours ago
Two Santa Cruz students uncover security bug that could let millions do their laundry for free

OpenAI’s Superalignment team, responsible for developing ways to govern and steer “superintelligent” AI systems, was promised 20% of the company’s compute resources, according to a person from that team. But…

OpenAI created a team to control ‘superintelligent’ AI — then let it wither, source says

TechCrunch Disrupt 2024 is just around the corner, and the buzz is palpable. But what if we told you there’s a chance for you to not just attend, but also…

Harness the TechCrunch Effect: Host a Side Event at Disrupt 2024

Decks are all about telling a compelling story and Goodcarbon does a good job on that front. But there’s important information missing too.

Pitch Deck Teardown: Goodcarbon’s $5.5M seed deck

Slack is making it difficult for its customers if they want the company to stop using its data for model training.

Slack under attack over sneaky AI training policy

A Texas-based company that provides health insurance and benefit plans disclosed a data breach affecting almost 2.5 million people, some of whom had their Social Security number stolen. WebTPA said…

Healthcare company WebTPA discloses breach affecting 2.5 million people

Featured Article

Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Microsoft won’t be facing antitrust scrutiny in the U.K. over its recent investment into French AI startup Mistral AI.

11 hours ago
Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Ember has partnered with HSBC in the U.K. so that the bank’s business customers can access Ember’s services from their online accounts.

Embedded finance is still trendy as accounting automation startup Ember partners with HSBC UK

Kudos uses AI to figure out consumer spending habits so it can then provide more personalized financial advice, like maximizing rewards and utilizing credit effectively.

Kudos lands $10M for an AI smart wallet that picks the best credit card for purchases

The EU’s warning comes after Microsoft failed to respond to a legally binding request for information that focused on its generative AI tools.

EU warns Microsoft it could be fined billions over missing GenAI risk info

The prospects for troubled banking-as-a-service startup Synapse have gone from bad to worse this week after a United States Trustee filed an emergency motion on Wednesday.  The trustee is asking…

A US Trustee wants troubled fintech Synapse to be liquidated via Chapter 7 bankruptcy, cites ‘gross mismanagement’

U.K.-based Seraphim Space is spinning up its 13th accelerator program, with nine participating companies working on a range of tech from propulsion to in-space manufacturing and space situational awareness. The…

Seraphim’s latest space accelerator welcomes nine companies

OpenAI has reached a deal with Reddit to use the social news site’s data for training AI models. In a blog post on OpenAI’s press relations site, the company said…

OpenAI inks deal to train AI on Reddit data

X users will now be able to discover posts from new Communities that are trending directly from an Explore tab within the section.

X pushes more users to Communities

For Mark Zuckerberg’s 40th birthday, his wife got him a photoshoot. Zuckerberg gives the camera a sly smile as he sits amid a carefully crafted re-creation of his childhood bedroom.…

Mark Zuckerberg’s makeover: Midlife crisis or carefully crafted rebrand?

Strava announced a slew of features, including AI to weed out leaderboard cheats, a new ‘family’ subscription plan, dark mode and more.

Strava taps AI to weed out leaderboard cheats, unveils ‘family’ plan, dark mode and more

We all fall down sometimes. Astronauts are no exception. You need to be in peak physical condition for space travel, but bulky space suits and lower gravity levels can be…

Astronauts fall over. Robotic limbs can help them back up.

Microsoft will launch its custom Cobalt 100 chips to customers as a public preview at its Build conference next week, TechCrunch has learned. In an analyst briefing ahead of Build,…

Microsoft’s custom Cobalt chips will come to Azure next week

What a wild week for transportation news! It was a smorgasbord of news that seemed to touch every sector and theme in transportation.

Tesla keeps cutting jobs and the feds probe Waymo

Sony Music Group has sent letters to more than 700 tech companies and music streaming services to warn them not to use its music to train AI without explicit permission.…

Sony Music warns tech companies over ‘unauthorized’ use of its content to train AI

Winston Chi, Butter’s founder and CEO, told TechCrunch that “most parties, including our investors and us, are making money” from the exit.

GrubMarket buys Butter to give its food distribution tech an AI boost

The investor lawsuit is related to Bolt securing a $30 million personal loan to Ryan Breslow, which was later defaulted on.

Bolt founder Ryan Breslow wants to settle an investor lawsuit by returning $37 million worth of shares

Meta, the parent company of Facebook, launched an enterprise version of the prominent social network in 2015. It always seemed like a stretch for a company built on a consumer…

With the end of Workplace, it’s fair to wonder if Meta was ever serious about the enterprise

X, formerly Twitter, turned TweetDeck into X Pro and pushed it behind a paywall. But there is a new column-based social media tool in town, and it’s from Instagram Threads.…

Meta Threads is testing pinned columns on the web, similar to the old TweetDeck

As part of 2024’s Accessibility Awareness Day, Google is showing off some updates to Android that should be useful to folks with mobility or vision impairments. Project Gameface allows gamers…

Google expands hands-free and eyes-free interfaces on Android