Latent Space Is Not Understanding

A variational autoencoder trained on molecules learns to compress any molecular structure into a vector of, say, 56 floating-point numbers, and then decompress that vector back into a molecular structure. If you do this well enough, the compressed representation is smooth: molecules that are structurally similar end up near each other in the 56-dimensional space, and interpolating between two points in that space produces a sequence of chemically coherent intermediates. You can travel from aspirin to ibuprofen through a path of plausible molecules, none of which are aspirin or ibuprofen.

This is genuinely impressive. It took real engineering to make it work. The question I want to press on is what it actually means, and whether the word "understanding" applies to any part of it.

What the model can do

A well-trained molecular VAE can do several useful things. It can generate novel molecules by sampling from the latent space. It can optimize molecules by moving through the latent space in the direction of a predicted property gradient. It can identify structurally related molecules by finding nearest neighbors in latent space. For each of these tasks, the latent representation is genuinely useful, and systems built on this approach have produced real results in drug discovery and materials design.

The latent vector is a compressed description of a molecule. It encodes structural information in a form the model can manipulate. This is compression, and it is powerful.

What the model cannot do

Ask the model why ibuprofen is an anti-inflammatory. The model has no answer. It can tell you that ibuprofen maps to a particular region of latent space. It can generate molecules nearby. It cannot explain the mechanism by which ibuprofen inhibits COX-2 enzymes, because that mechanism involves specific geometric interactions between the drug and the enzyme active site, hydrogen bonds at particular distances, the displacement of a water molecule, a conformational change in the protein. None of this is encoded in the latent vector. The latent vector is a summary, not an explanation.

This distinction matters because in science, understanding is not the same as accurate description. Understanding implies the ability to predict counterfactuals: if you change this part of the molecule, what happens to that property, and why. A latent space model can predict the property change, approximately, by interpolating. It cannot explain why, because it does not represent the causal mechanism. The "why" lives in the physics: quantum mechanics, orbital overlap, steric repulsion, solvation entropy. None of that is in the 56 numbers.

The benchmark problem

The machine learning community has a specific way of evaluating molecular generative models: validity, uniqueness, novelty, and property statistics on benchmarks like GuacaMol and MOSES. These metrics are reasonable proxies for specific capabilities. They do not measure understanding.

A model can score well on all four while having learned nothing more than surface patterns in SMILES strings. SMILES, the text representation of molecular structure most commonly used in ML datasets, has regularities that a model can exploit without ever building an internal representation of three-dimensional geometry or electronic structure. Models trained on SMILES often generate molecules that satisfy SMILES syntax and pass RDKit validity checks but have strained geometries, unrealistic bond angles, or unusual valences that would immediately disqualify them in any serious chemical context.

When these models are evaluated on scaffold hopping, meaning generating active molecules with genuinely different core structures, performance drops sharply. The learned representations do not generalize across chemical families in the way that actual understanding of molecular properties would allow. The model has memorized a distribution, not learned a theory.

What understanding would require

I have thought about this while building the generative molecule designer and the neural network potential trainer as parts of the same research program. The NNP trainer learns to predict atomic forces from molecular geometry. It does not understand chemistry either, but it comes closer: it has learned a representation that respects E(3) symmetry, that generalizes across similar bonding environments, and that can be interrogated by computing its gradients with respect to atomic positions to recover something like a force field. That is more structure than a latent space.

Genuine machine understanding of a molecule would require, at minimum, the ability to predict how its properties change when you make specific structural modifications, to identify which features of the structure are causally responsible for which properties, and to generalize these predictions to structural classes not present in the training data. This is what a chemist does, drawing on a framework of quantum mechanics, thermodynamics, and reaction mechanisms that took a century to assemble.

The latent space is a useful map. Maps are tools, not understanding. A map of a city tells you how to get from one place to another. It does not tell you anything about the social history, the architectural logic, or the economic forces that produced the streets you are navigating. A molecular latent space tells you which molecules are near each other in some learned sense of similarity. It does not tell you why they are similar or what that similarity implies for their behavior in contexts the model has not seen.

This does not mean latent space models are not useful. They are useful precisely because they are fast, differentiable, and trainable on the data we have. But the word "understanding" should be reserved for something harder to build and more valuable when achieved. Conflating compression with comprehension is not a harmless metaphor. It sets expectations that the tools cannot meet, and it directs research energy toward benchmarks that measure the wrong thing.