Machine Learning Versus Machine Discovery

Machine learning is hot. Where it applies, it heatedly enables data-rich and knowledge-lean automation of valuable tasks of perception, classification and numeric prediction. Its sibling, machine discovery, deals with uncovering new knowledge that enlightens or guides human beings. Let’s consider where learning or discovery best applies — and why this matters for business.

Years ago I was a machine-discovery researcher. Scholarly articles were published in the journal Machine Learning, and presentations were made at Machine Learning conferences, since it seemed that learning and discovery were similar human activities. As a (veteran) entrepreneur, I’m often asked whether a learning approach makes sense for automating some task, which motivated me to pen this piece. First, let’s call in some foundations.

A key idea in artificial intelligence (AI) is that intellectual work can be seen as heuristic search within a “problem space” that leads to solutions.

Consider the familiar task of a TV homicide detective who arrives to a dead body on the floor. A lousy detective gets the phone book and interviews people starting on page 1. A really lousy detective also considers space invaders and escaped baboons, and inquires with NASA and the local zoo to pursue those leads. They use bad heuristics.

A good detective uses good heuristics, starting with proven questions, such as: What was the cause of death? Who last saw the victim alive? Any enemies? Secret romance? Owed money? The good detective proceeds to more-effectively search the large space of possible culprits using insights from the answers. Great detectives might even come up with novel heuristics.

The key idea of machine discovery is that discovery is like other intellectual tasks. Thus, the key AI idea of heuristic search in problem spaces applies also to discovery tasks.

On the other hand, the key idea of machine learning is that, given enough data with associated outcomes, together with notions of what data features are relevant to predicting those outcomes, software can be trained to make those associations in future cases. Classic examples involve using historical data to learn how to classify loan applicants into credit-risk categories, or to predict when customers will churn.

Which hammer — learning or discovery — best pounds which nails?

Armed with these key ideas, let’s consider which is the better design — discovery or learning — for a proposed app: A guest-introducer for large parties or events. A good party host identifies areas of common interest among guests and endeavors to introduce them, explaining what they have in common in order to stimulate conversation. It’s a hard task and hosts are busy. Given an attendance list, could making good introductions be automated?

An AI or discovery approach proceeds like this: Study, or figure out, what makes a good introduction. What determines quality? Is there scope for innovative introductions that serve the core purpose? What data sources enable these automated inferences (e.g., LinkedIn profiles or other biographical sources)?

Then, one could generate automated introductions, like this: “The three of you graduated from the same college around the same time,” or “Both of you mentioned having served in the Peace Corps in Africa.” Or even, “You two are the only people here who know about machine learning.”

Bad heuristics could lead to, “Both of you have been divorced four times or more” (embarrassing), or “All of you are from the Midwest” (too unfocused) or “Your birthdays are in winter” (irrelevant).

We’ve discussed the key ideas of machine learning and discovery and how to approach a specific new application. Let’s generalize: Which hammer — learning or discovery — best pounds which nails?

Discovery requires studying the task logic (i.e., the space of possible solutions), the knowledge that prioritizes good paths within that space and algorithm design to make it all practical. There is scope for innovation in the space being searched and the heuristics used. But the most innovations may come from novel, creative outputs on specific inputs, because automation enables exploring a much larger space of possibilities than people can practically consider.

Let’s consider three examples of machine-discovery engines, each using programmed heuristics to search for and report human-readable knowledge from within a large space of possibilities:

  • Search engines — commercialized in the 1990s — search many information documents, using heuristics such as page rank and the proximity of the query words within each document or its title, to report a list of citations, each with excerpts dynamically customized as a function of the query.
  • Clustering engines — commercialized around 2000 — group hundreds of search results into topic folders, using heuristics such as the linguistic quality of the extracted topics, how many search results each topic covers and how well the topics partition the results into non-overlapping groups, to describe the main themes present within the returned search results.
  • Benchmarking engines — commercialized in 2015 — discover outlier performance within large, noteworthy peer groups, using heuristics such as brevity, sensible combinations of attributes and sentence types that pair well with outlier types, to output English paragraphs that convey benchmarking insights for the targeted entity.

Here are symptoms that point to a discovery approach: Task outputs aren’t just classifications or numerical predictions. People write books or articles about the task in order to teach novices. There isn’t abundant data on input/correct-output pairs. It’s common to persuade others why a particular output is consistent with the input data and task knowledge. The task knowledge is circumscribed, so that general common sense is not needed to perform the task.

What does this mean for the technology business? Machine learning semi-automates the task of task automation, which reduces cost. Learning applies to many data-rich tasks. Machine discovery will address specific tasks that require knowledge and training when done humanly. Discovery tends to be hand-crafted, more elaborate and rarer.

You’ll need considerable in-house or vendor AI expertise. Vendors will be fewer and focused on specific intellectual tasks with deep impact to make the venture economically viable. Vendors will tend not to call themselves machine-discovery companies. Market differentiation using discovery will be easier because there are fewer suppliers, unlike machine learning, which has many.

Machine learning and discovery will remain close siblings, but productively living apart as they mature.