Last Updated on August 20, 2026 by Rob Cashman
Over the weekend, a new article appeared on the BBC announcing that biotechnology researchers have used artificial intelligence (AI) to design complete viral genomes that can infect harmful bacteria. I just finished writing the article about misaligned AI models being released into the wild, so the reference to the AI breakthrough caught my eye. My conclusion in this article is that there is an inherent danger when humans intervene in complex systems, especially when the variables affecting those systems are not fully considered.
The claim in the article was that “the viruses infect bacteria, not humans,” (red flag #1), and that the AI training data “excluded viruses that could infect complex organisms,” (red flag #2). The implication that introducing an AI-trained virus strain into a complex system to influence the propagation of one form of bacteria over another was ‘safe’ immediately gives me pause.
Why Intervening in Nature Is More Complicated Than “Good” and “Bad”
Perhaps these viruses are safe. Perhaps every one of these safety claims is true. We still need to ask, “what happens when we intervene in a complex system whose relationships or variables we don’t fully understand?”
In no way am I suggesting the scientists here made a mistake. I have always been in favor of the advancement of technology and medicine, and if humanity can use AI to accomplish that end just as Pasteur taught surgeons to wash their hands because he noticed that bacteria and germs killed patients on the operating table, even better.
So at first glance, using AI “to design brand new viruses” that can get rid of disease and improve our health and longevity seems to be a wonderful use of AI. But the issue isn’t AI; the issue is that they are potentially manipulating the balance of nature and enforcing selection upon a natural system which already does so itself.
This act of “placing one’s finger on the scale” without considering all the variables (and purposefully depriving the AI models of training data of viruses which could infect complex organisms) can lead to the AIs being unaware of the effects of those viruses and the emergence of new and significantly more deadly diseases and unforeseen circumstances. So this article really isn’t about AI at all, it’s more about assumptions we make when interfering in complex systems.
I don’t question the usefulness in looking for biological applications that improve our longevity. The danger occurs when a researcher decides that one part of a complex system is “good” while another part is “bad.”
Nature Doesn’t Believe in Villains
The definitions distinguishing “bad bacteria” and “good bacteria” are common in our language, and frequently used (most commonly, in the context of the ecosystem in the ‘gut’).
We understand what is meant by it, but the terms quietly smuggle in an assumption that nature itself doesn’t make. Whether something is labeled ‘good’ or ‘bad’ in the context of bacteria is understood only as it relates to us and our health, not to the ecosystem into which they are looking to interfere.
Bacteria are not good or bad. Their existence on this planet predates humanity, and their numbers vastly outweighs ours, but to the best of our limited understanding, bacteria are not moral agents.
They do not try to help humanity, just as they are they not trying to hurt us. Bacteria merely pursue the objective function that nature has ‘programmed into their code’: survive, reproduce, and compete for limited resources.
The fact that certain kinds of bacteria cause diseases within humans has no bearing on whether classes of them should be eliminated or not. A type of bacteria that kills us in the most horrible way is no more “evil” than a lion hunting a gazelle. On the same note, bacteria that helps us digest our food isn’t being altruistic, nor is it a justification for them being ‘selected’ to propagate and increase in quantity.
Both are simply ‘executing their code’ and acting according to their nature, according to their own ecological niches. How they affect us is something for us to understand and, where necessary, counteract. There is nothing wrong with treating disease; the error is assuming that because an organism produces an undesirable effect within one part of a system, eliminating that organism must necessarily improve the system as a whole.
Let’s take this argument one level deeper…
While arguing that it is an error to call something “good” or “bad,” this is not because the act of assigning value to whether certain bacteria types help us or not is merely assigning a semantic title or descriptor on something that has none. Rather, the error is the assumption that something in nature can tolerate human intervention and manipulation because the researchers conclude (or even because an AI concludes) it knows better. My biggest concern is that of unintended consequences.
Most life-saving innovation and new technology occurs through experimentation and observing what results occur when a particular variable is modified. The creation of modified virus that can hunt harmful bacteria is merely one example of this. But stopping there without considering the unintended consequences undermines the effort because it misses the balancing variables which are already in play in that ecosystem.
Missing Variable #1: Balance Within a Complex System.
Suppose biotech researches succeeded in eliminating many if not all harmful bacterium on Earth. Would humanity become healthier, safer, would we live longer? Not necessarily.
Every ecological system contains feedback loops. Predators limit the number of prey in the ecosystem. Competition between companies prevents monopolies. Scarcity of certain resources keeps populations from exploding. Remove one competitor, and another expands into the newly available niche.
We see this in forests. We see this in oceans. We see it in classic mafia movies. We even see it inside our human gut.
Nobody in an ecosystem needs to sit around a conference table and decide that there should be three competing criminal organizations rather than one. If one organization becomes too powerful, the incentives of everyone else in the system change. Alliances form. Competitors emerge. The balance shifts. Bosses are disappeared.
Organisms (whether bacteria, viruses, bugs, birds, or predators) — even those that we classify as “harmful” to us often perform an invisible function simply by existing. Their existence limits the presence of something else. Competition itself becomes part of the stabilizing mechanism. This is why ecosystems are so difficult to engineer, and why every act of injecting chemicals into the air, or introducing species of mosquitos into an unnatural environment are all interesting ideas, but they are innovations that need to be taken with extreme caution and the utmost care.
“Bad bacteria” are not collections of independent species that we could eliminate without affecting anything else. They are a necessary part in the balance of dynamical systems that rely on them (and their harmful effects) to serve their function.
The Doughnut Problem
Imagine lawmakers concluded that doughnuts make people happy. What if their solution was to have us eat an unlimited number of doughnuts each day?
The problem, of course, is not that doughnuts are evil. The problem is assuming that more of something desirable is always better. Almost every biological system eventually reaches the opposite conclusion.
Water is essential to our survival; too much of it kills you. Oxygen is essential to our being able to breathe air; too much of it becomes toxic to us. Exercise is healthy; too much of it destroys your health. Even vitamins become poisonous at sufficient concentrations.
The universe seems strangely consistent about one principle: Very few things are inherently good. And, almost everything becomes harmful when removed from the feedback mechanisms which normally regulate it.
The question therefore isn’t simply whether something is good or bad. The question is under what conditions, in what quantity, and in relationship to what else does it become good or bad?
Whether we are dealing with harmful bacteria, or any other variable within a system — a variable is not merely what something is; it is what it is doing within the system at that moment in relation to other parts of that system.
What if we made the error in analyzing the harmful effects of one or more individual actors within the system (the harmful bacteria we are seeking to eliminate with new-technology viruses), but we did not understand what function or objective it was serving within that system?
Evolution Doesn’t Optimize Happiness
I want to shift the conversation from the dangers that emerge when focusing on harmful actors within a system to the dangers that emerge when applying poorly formed objectives to a complex system. Complex systems and ecosystems can exhibit remarkably stable behavior without having a system-level objective at all. There may be no ‘entity’ within the system deciding that equilibrium should be maintained. Instead, what looks like an objective can emerge from the countless interactions between the participants within the system.
Evolution does not impose survival. Survival is what makes evolution visible.
Evolution and Natural Selection: Processes, Not Actors
Biologists often infer that evolution or natural selection acted upon a system to affect the outcome. The philosophies of evolution and natural selects speaks as if these were active agents that selected ‘the fittest’ of those organisms that survived. But neither evolution nor natural selection needs to act as a selector to describe with hindsight what happened.
Evolution and natural selection do not require that either acts with intention. These need not be an external ‘force,’ ‘agent,’ or an ‘actor’ exerting its influence on an ecosystem attempting to make organisms healthier, stronger, more fit, and more likely to survive. They do not need to be ‘conscious selectors’ that choose winners and losers on a complex ecosystem, as they are currently seen.
Rather, ‘Evolution’ and ‘Natural Selection’ might merely be the names we give to processes through which survival and reproduction produce observable changes in populations over time.
Even if Evolution was a ‘system-level’ entity, it would still have no opinions, no preferences, no long-term vision, no desires, or free will of its own, as I’ve described here (and paraphrased, below):
“…System-level entities appear to have substance or ‘existence’ of their own, but really, their ‘existence’ emerges from the combined actions, decisions, beliefs, and interactions of agent-level actors within that system. They do not have a body or a physical form, nor do they possess independent volition. Rather, their ‘behavior’ emerges from the interaction of the individual actors within the system.”
I would say the same thing about ‘Natural Selection’:
Even if “Natural Selection” too was a ‘system-level’ entity, it wouldn’t act on its own to choose one organism to be the winner that propagates, and the other the loser [it would be the individual actors within that system which make those choices or adaptations which result in their survival or their demise]. Just as evolution does not need to have a will of its own to explain what it teaches, natural selection too does not need to have a will of its own to explain what we can learn from it.
‘Natural selection’ as explained by evolution simply preserves strategies and features of the agent-level organisms that continue reproducing under current environmental constraints. Just as ‘Evolution’ is a backwards-looking observation of what changed over time, ‘Natural Selection’ too is a backward-looking observing of which organisms survived and which did not. Neither asserts or imposes objectives on the participants of the complex ecosystem.
Applying evolution to ecosystems and organisms within them, we can observe that “healthy” ecosystems are not those containing only beneficial organisms. Healthy ecosystems are those in which competing forces have historically prevented any single strategy from overwhelming the rest. This is not achieved because each organism cooperates; it is achieved because no organism type or feature is allowed to win completely.
An ecosystem under the effects of evolution and natural selection does not have an objective function, but it nevertheless behaves as though it has one.
There is no system-level entity sitting above the ecosystem deciding that stability is preferable to instability. There is no “will of the ecosystem.” Stability and balance emerge from countless local interactions between organisms pursuing their own survival, reproduction, competition, cooperation, and adaptation.
If we think one step further, we could observe that Evolution and Natural Selection might not be ‘forces,’ or even ‘agents’ at all, but rather they might simply be system-level processes.
System-Level Processes; a better description of Evolution and Natural Selection
A better explanation for Evolution and Natural Selection is that these are system-level processes. Like system-level agents, entities, or forces, these do not have volition or ‘will’ of their own, and they do not exert an influence on those within their system. But there is an interesting observation that we can learn by calling them processes:
A process can produce consequences without being an actor.
With this description, we could better understand concepts which have perplexed scientists for generations.
Gravity does not need to be an actor. Natural selection does not need to be an actor. The stock market does not need to be an actor. Relevant to our topic, an ecosystem does not need to be an actor.
Corporations (as system-level entities) teach that one system-level entity can affect other system-level entities (and the individuals within them).
A corporation, or any ‘fictitious entity’ recognized by the laws of a particular state is still a system-level entity. It can make contracts, sue or be sued, acquire another company, hire or fire employees, produce and sell products, make investments, etc. It does this through those that work within its structure. And even though it has legal status of being a ‘person,’ it has no independent consciousness or volition.
It can affect its employees, competitors, customers, markets, governments, and even other corporations. In other words, from it we can notice:
one system-level entity can have an effect on other system-level entities without that entity ever being an actor.
This is applicable to the question of whether ‘kill only these harmful bacteria types’ ecosystem interference can have unintended outcomes affective even those outside that ecosystem.
The danger of increased optimization through new AI-level intelligence.
Artificial intelligence is becoming extraordinarily good at optimizing toward explicit objectives. “Destroy this bacterium.” “Increase this crop yield.” “Reduce this disease.” “Make money online.” “Produce paperclips.”
While these objectives might satisfy the needs of the individuals seeking them, they do not align with the balance a complex system appears to require. Complex systems rarely optimize around isolated goals, and they don’t elevate the objectives of one participant within the ecosystem over another. They optimize around the combined interactions of and between all involved in that system.
If one objective function (e.g., propagate this custom-trained virus to kill these kinds of bacterium) omits the interactions these so-called harmful bacterium with other actors in the system, this optimization can accidentally dismantle the very mechanisms this ecosystem already relies upon to maintain stability.
Artificial Intelligence merely increases the capabilities of the researchers; it follows instructions and magnifies objectives inputted into it. It does not verify that the objectives are complete.
Taking this one step further, the problem with relying on AI to achieve an objective is not that the AI might refuse to obey us. It might be an AI that obeys us perfectly.
So the issue is not necessarily that the AI is misaligned with the objectives of the researcher, but that the objectives of the researcher might be misaligned with the system it wishes to affect. AI (a sufficiently capable optimizer) faithfully pursuing an incomplete objective can produce consequences that are far outside what the researcher ever intended.
Whether we are discussing harmful objects (e.g., harmful bacteria), or harmful objectives (prioritizing the needs of one participant within an ecosystem over another), we are still missing one more variable: the relationships that each agent within the system shares with each of the other agents also in that system.
Missing Variable #2: Relationships Within a Complex System
Most people instinctively believe that a complex system consists of things, when really a system is better understood as relationships among them.
Change an object and you have modified one participant. Change a relationship and you have modified the system itself.
We naturally think of bacteria as individual organisms that function independently within a system. But ecosystems don’t “see” bacteria any more than a market “sees” one company or stock, or linguistically, how a conversation “sees” individual words rather than how they relate to each other to form sentences, and eventually, the communication of ideas. What determines the behavior of a system is not merely the identity of its individual actors, but the network of relationships among them.
An ecosystem does not seek balance, but balance is something that can emerge from the interactions between the participants within that ecosystem.
Ecosystems do have within them a balance between the organisms in that system. But to achieve that balance, the relationships and interactions between those participants are important. Every bacterium occupies a web of competitive, cooperative, parasitic, and symbiotic interactions. Remove one organism and you’ve altered hundreds of relationships simultaneously. Animals within an ecosystem need oxygen to breath, and they expel carbon dioxide as waste; but plants require carbon dioxide to breathe, and they expel oxygen as their waste.
Disturb the balance between how the participants in an ecosystem relate to one another, the complex ecosystem as a whole is put into jeopardy, in turn endangering the lives of all participants within that system. Or, we could just as easily state, “remove one node from a network and you don’t merely remove the node. You remove every edge connected to it.”
I would venture to suggest that a most accurate representation of a complex system is not the actors within that system, but the relationships between those actors within that system.
Ecology studies relationships between organisms and their environments. Economics studies exchanges between participants in markets. Neuroscience studies networks of neurons and the pathways connecting them. Information theory studies relationships between signals and information. Control theory studies feedback between a system and its environment.
In each case, understanding the individual components is not enough. To understand a system, we need to understand what happens between the components.
Undesired Effects Serve a Function Within Complex Systems
It must be noted too that in complex systems, ‘undesired’ effects or states themselves often serve an important function.
In economic systems, inflation discourages reckless borrowing. In psychology, stress stimulates adaptation. In business, competition drives innovation. Even conflict between the actors in one system can expose weaknesses in that system that it needs to grow and persevere that comfort between those actors might have left hidden.
The world is full of forces that appear destructive or undesirable when viewed from one lens, yet stabilizing when viewed systemically.
Perhaps lastly, because even this analysis could be missing variables, there is one last variable relevant to the ‘inject viruses into the ecosystem’ debate, and that is the misunderstanding of the boundaries in which they are thinking to interfere within.
Missing Variable #3: Boundaries of the System Itself
Perhaps the most dangerous decision we make before optimizing a complex system is deciding what the system actually is.
Because if the researchers are defining the boundary incorrectly, everything downstream can be perfectly logical and still be wrong.
By isolating one class of bacteria as ‘harmful’ and looking for a biotechnology solution to eliminate that class, researchers might make a mistake in considering what the actual boundaries of the system are that they are interfering with.
If the system is a closed system (e.g., a system in a ‘vacuum’) that comprises only the harmful bacteria and the humans which are adversely affected by them (bacteria + humans), then of course, such a solution of ‘eliminate the harmful bacteria’ would be a no-brainer. The optimization AI proposes would be a perfect solution to the problem.
However, if the researchers define the system too narrowly, then this can be how the unintended consequences might occur. For example, instead of the seeing the boundaries of the system as merely, “bacteria + human,” if they were really, “bacteria + human + AI-engineered virus + other viruses already in the system + immune system + food + competing bacteria + environmental conditions + apparent evolutionary pressures,” dipping their AI-engineered virus into this system without considering the proper boundaries of the system they are affecting could result is harm or death to the system.
The Balanced Approach: Focus on the Relationships within the System and the Boundaries of the System Itself; Not the Object or the Harmful Actor
I’m not particularly worried about AI creating viruses. Scientists have demonstrated remarkable care in restricting what they train, what they generate, and how they validate their work. Even my awareness of the possible concerns resulting from having misalignment between the objectives of the AI models versus the objectives of the humans instructing them also does not impact my trust in the scientific method when approached with restraint.
I noted some ‘red flags’ at the beginning of the article, e.g., the assumption that the modded viruses would only infect bacteria, not humans, or that the viruses they are using AI to design pose no threat to complex organisms, or that we should rely on an AI’s assurance that these viruses are ‘safe’. In each of these ‘red flags,’ the danger is that these statements might be incorrect — each one can easily be true.
Philosophically, even if each of these statements are true, can the introduction of foreign elements into a balanced ecosystem throw that system into a state of imbalance, and can this cause unintended consequences, the damage of that ecosystem itself, and as a result, the exact harm we were seeking to avoid by attempting to interfere with it?
Before we ask how to optimize a system, perhaps we should ask whether we have correctly identified the system. And then, before we eliminate an undesirable actor within that system, perhaps we should ask what relationships will disappear with it.
AI is a wonderful tool, and the idea of training modded viruses to attack only certain types of harmful bacteria is ingenious. But before we give an AI such an objective, perhaps we should ask what assumptions about the system are hidden inside that objective.
The problem with complex systems isn’t that they contain too many variables for us to understand. We often just don’t realize which variables we’ve left out.
Sometimes the most dangerous thing we can do to a complex system is successfully optimize it for the wrong purpose.