Project VENOM: The Air Force is adding AI pilots to 6 F-16s

The U.S. Air Force has announced plans to equip a small group of F-16 Fighting Falcons with artificial intelligence (AI) pilots, as part of the Project Viper Experimentation and Next-Gen Operations Mode, or VENOM. The news came just about a month after the branch announced an AI-piloted and heavily modified F-16 known as the X-62A successfully flew a variety of air-combat missions over the span of two weeks this past December.

Project VENOM is the latest development in the Air Force Research Laboratory’s Skyborg autonomous aircraft teaming architecture, which aims to pair crewed fighters with AI-enabled drone wingmen for the 21st-century fight. The Air Force is investing heavily into this concept, with a reported $50 million allocated in its 2024 budget proposal to the VENOM effort alone.

A great deal of new information was revealed about Project VENOM in an online webinar hosted by the Mitchell Institute for Aerospace Studies that saw substantial reporting from Emma Helfrich at The Warzone.

This effort will not pull pilots out of the cockpit. Instead, it will equip six F-16s out of Eglin Air Force Base with AI agents that will ride along with their human counterparts. The goal is to use these aircraft in a variety of test operations to gather useful data on manned-unmanned-teaming concepts while simultaneously helping to foster higher levels of trust between pilots and AI “agents.” This focus on trust-building has been a significant part of both the Air Force and the Defense Advanced Research Projects Agency’s (DARPA) efforts to incorporate AI into American airpower.

“Project VENOM right now is bringing in F-16s that will put autonomy on to help us get after the autonomy that we need to get that into kind of daily operation,” Maj. Gen. Evan C. Dertien, commander, Air Force Test Center said. “So, I see lots of challenges as far as building this NGAD family of systems, but I also see lots of opportunities to go deliver something new and innovative that will help out the warfighter.”

This manned-unmanned-teaming concept is not only vital to America’s forthcoming stealth fighter in active development, the Next Generation Air Dominance Fighter (NGAD), but it’s also expected to link forthcoming Block 4 F-35s with drone wingmen as well.

Related: AI wins flawless victory against human fighter pilot in virtual dogfights

The nuts and bolts of Project VENOM

Project VENOM is one of three ongoing Air Force programs aimed at equipping fighters with their own constellations of support drones to extend sensor reach and increase combat capability. Alongside Project VENOM, the Air Force is continuing similar work in the Collaborative Combat Aircraft (CCA) program and within the Experimental Operations Unit (EOU) team.

These efforts all have a great deal of overlap by design. The CCA program has been described as the “brainchild” of Air Force Secretary Frank Kendall and could be seen as the broad initiative to couple crewed and uncrewed aircraft for future combat operations. The EOU team is tasked with developing the real tactics and procedures required to accomplish this goal, and Project VENOM, as Air Force’s Chief Scientist, Dr. Victoria Coleman puts it, is the bridge between the two.

“With Venom, we will take six F-16s and we’ll kit them out with an autonomous safety box, where we’ll be able to drop in autonomy code and we’ll have a human pilot experiment with it and fly with it to make sure that A – it works, and B – that it has the benefits that we expect the autonomy to deliver operationally,” Coleman explained.

Project VENOM will build directly on the Air Force’s recently disclosed success using AI to fly their X-62A VISTA (Variable Stability In-flight Simulator Test Aircraft) in a series of air-combat exercises last December. But while the X-62A is a heavily modified Block 30 F-16D, an iteration of the famed Viper that entered service in the late 1980s, VENOM will place artificial intelligence in the cockpit of more contemporary fighters equipped with far more robust sensor suites.

The X-62A, which was previously known as the NF-16D, is an experimental testbed equipped with a multi-axis thrust vectoring (MATV) engine nozzle similar to those employed by super-maneuverable Russian fighters like the Su-35. The aircraft couples this capability with a unique fly-by-wire system meant to allow the aircraft to replicate the in-flight behavior of any aircraft in service, allowing pilots inside the X-62A to feel like they’re flying anything from a C-130 to an F-22 Raptor.

This unique control system made the X-62A well-suited to have an AI agent incorporated into its onboard systems, making it a logical first choice for the effort. But despite the incredible aerobatic capabilities of the X-62A, it’s not a combat aircraft, and as such, doesn’t carry the modern systems American F-16s carry into the fight.

“The VISTA aircraft is great for the control development and we kind of have the safety wrapper to develop autonomy, but what we don’t have on that aircraft is a lot of sensors,” Dertien said. “So by getting [the autonomy engine] on the VENOM aircraft you now have an [active electronically scanned array] radar, you have electronic warning, you have all those things that can expand your autonomy algorithm to react to the inputs that it’s getting to make decisions for yourself.”

But while the clear intent of Project VENOM is to develop an AI agent that’s capable of managing the complexity of air combat operations, this effort is still maturing. These AI-enabled F-16s won’t be diving straight into advanced fighter maneuvers. Just like a new pilot needs time to develop a mastery of their aircraft, Project VENOM’s AI still has a lot to learn. But while class is in session for the AI, its human counterparts will also be studying how best to couple the skillsets offered by the AI with human operators.

“You know, self-driving cars didn’t go from fully manual to fully automated, right? The Tesla’s and all the other electric vehicles, they’ve traveled millions or billions of miles where they learned and figured out how to interface with a human operator and to do so safely and securely. We don’t get to skip that part in the Air Force,” Coleman explained. “So Venom is precisely that capability that allows us to bridge the gulf from one state to the other. So we’re very excited about it.” 

And of course, the technical challenges inherent to creating a system that can fly a nimble F-16 in combat are significantly larger than those of a self-driving car. Not only do aircraft add an additional axis of movement (vertical, as well as horizontal) and an immense increase in speed, but they’re also regularly under fire from a variety of increasingly advanced ground and air-based threats. Before Project Venom’s F-16s can take on the rigors of such a mission, they first have to nail down the basics.

“What the F-16 allows us to focus on, since it’s a crewed fighter, we can work on getting the aircraft to and from the airspace to develop the autonomy there,” Dertien said. “And also in the airspace to work on the manned teaming and the uncrewed operation. So, I think it’s a natural evolution from everything you’ve seen before.”

Related: How US special operators use Artificial Intelligence to get an edge

Not replacing pilots, but assisting them

Even with AI-piloted aircraft quickly becoming a reality, the intent behind efforts like Project VENOM is not to replace human pilots but rather to supplement them with artificial intelligence agents, not just in wingman roles, but inside the cockpit as well.

The United States currently has two publicly disclosed 6th generation stealth fighter programs in active development, both of which are expected to operate as “families of systems,” with the crewed fighter flying alongside AI-enabled drones tasked with a variety of combat operations, ranging from intelligence, surveillance, and reconnaissance (ISR) to electronic warfare to kinetic strike operations.

The concept of using a constellation of drones, each taking their cues from a human pilot at the center of an airborne network, promises to turn each pilot into a formation unto themselves, dramatically increasing combat capability and capacity. But even if the Pentagon can work out all of the circuitry and programming involved, they still face one more hurdle: The cognitive load that all of this added capability will place on pilots. The human brain may have a great deal of computational horsepower of its own, but at a certain point, even it can be overwhelmed.

The cockpit of today’s F-35 already represents a massive leap forward in managing the mental workload of flying a high-performance fighter behind enemy lines, streamlining the dozens of gauges and read-outs that normally litter a fighter cockpit into a combination of helmet-mounted display and large, interactive touch screens. But even with all of the F-35’s sensor-fusing power, pilots are still tasked with managing the complex functions of a jet-powered aircraft flying at hundreds of miles per hour while simultaneously keeping track of targets, friendlies, potential threats, and much more.

As U.S. Air Force F-35 pilot Hasard Lee explained to Sandboxx News, it’s so much to manage that it’s not uncommon for young, inexperienced pilots to get so caught up in the mission at hand that they make a serious mistake flying the aircraft.

“It’s easy to get sucked into the moment — the soda straw, as we call it,” Lee, who flew 82 combat missions in the F-16 before transitioning to the F-35, explained. “This happens to new wingmen all the time. They hear there’s a troops-in-contact situation and they just get so locked into it that they fly into a mountain or aren’t able to assess, ‘should we drop this danger-close munition or should we do a show of force?'”

Related: Will AI weapons bring the downfall of humanity?

Building trust with a machine

This real limit to human ability has led to a fighter pilot culture that places a significant emphasis on managing cognitive load by using a combination of mental training and active focus to allow for rapid, but sound, decision-making. This represents a constant challenge in today’s advanced and highly automated F-35s, but the challenge will only grow as pilots are increasingly tasked with issuing orders to AI-enabled wingmen waiting to take their cues from their human operators flying nearby.

It’s because of this issue of focus that artificial intelligence won’t be relegated only to the drone wingmen. It will be flying alongside human pilots right there in the cockpit as well.

Once these AI agents have a thorough understanding of how to manage routine flight operations, gained through efforts like Project VENOM, they will increasingly be able to take over standard flight duties from their human operators, freeing up the pilot’s time and focus for more pressing matters, like combat. AI co-pilots could fly the aircraft along its intended route while pilots survey the battlespace using the full breadth of their onboard sensor suite. The AI could react to inbound surface-to-air missiles faster than any human could, and could even employ countermeasures when the aircraft is in jeopardy.

But before artificial intelligence can empower human pilots to focus on the battle rather than flying their planes, America’s pilots have to learn to trust them — and that’s a big part of what Project VENOM, and previous AI efforts, have been all about. The only way to create that trust is to consistently demonstrate the value of these AI systems to real pilots and wait for the culture to catch up.

“It goes back foundationally to that trust in autonomy, right? This is where it starts now,” said Brig. Gen. Dale R. White, Air Force program executive officer for Fighters and Advanced Aircraft. “We’re going to integrate into the F-16. We’re going to start practicing with our algorithms … leveraging all of those things and the previous work that’s been done understanding that foundationally we’ll have that autonomous engine and then we’ll have to continue algorithm development.”

Nearly three years ago now, DARPA held their AlphaDogfight trials, which was a series of AI versus AI dogfights that culminated in the most capable AI agent, an entrant from Heron Systems, being pitted against a human F-16 pilot in a guns-only dogfight held entirely within the digital environment. The AI’s overwhelming victory over the human pilot was largely touted as a step toward putting such pilots out of the job, but even that effort, according to DARPA, was more about proving to human pilots that AI can be a benefit than anything.

Related: US Air Force AI pilot flies real F-16 in simulated dogfights

Why use AI instead of remote-controlled aircraft?

The U.S. Air Force already operates a vast fleet of uncrewed aircraft, many of which are considered RPAs, or Remotely Piloted Vehicles. Aircraft like the MQ-9 Reaper, for instance, may be able to loiter over a target area for more than 24 hours, but they aren’t automated. Reapers are controlled by two-person teams made up of a licensed pilot and an enlisted sensor operator that switch out every eight or so hours. This approach has proven consistently effective in ISR and even attack operations in permissive environments, in which MQ-9s engage targets on the ground with munitions like the AGM-114 Hellfire missile, or even its highly specialized, sword-packing variant, the R9X.

So if this remotely piloted model works for air-to-ground operations, what’s stopping the Air Force from simply incorporating this capability into more nimble fighter-support drones? While there are a number of challenges inherent to such an effort, the biggest one AI can help overcome… is lag.

According to a U.S. Air Force MQ-9 pilot I interviewed in 2021 (who asked to be identified only as Captain Dennis), it takes about 1.2 seconds for the video and sensor feed from the MQ-9 to reach him at his control station. Once he sees something on his screen that warrants a response, he inputs the commands and it takes another 1.2 seconds for those commands to relay all the way back to the airborne Reaper on the other side of the globe.

As a result, there’s usually a 2-3 second delay (or more) between when the MQ-9 spots a threat and when it can actually react to it. In a slow-moving prop-driven Reaper flying in uncontested airspace, that three seconds isn’t much of an issue, but in air combat, life or death decisions are made in fractions of a second.

Concerns about lag and signal integrity are all but eliminated when equipping an uncrewed aircraft with AI that allows it to make rapid decisions on its own. Of course, it’s important to remember that this decision-making ability is not consciousness as it’s often depicted in science fiction, but rather the rapid ability to analyze the situation, compare it to hundreds of thousands of hours of previous flight data, and quickly determine the right course of action for the circumstances, based on the data it has to pull from.

AI agents in control of aircraft are, to some extent, just sorting through a massive list of “if, then” statements, choosing the right ones for the situation, and then executing. If a surface-to-air missile is launched from behind the aircraft, then deploy countermeasures and execute maneuvers, and so forth. The more experiential data these systems have to pull from, the more robust their ability to execute complex tasks becomes.

Related: Navy SEALs aim to incorporate AI into future ops

It’s all about the data

Project VENOM is just one aspect of a much broader effort to incorporate artificial intelligence into the modern battlefield, but while this effort isn’t necessarily in its infancy, it’s still got a ways to go before we could call it mature. Getting to that point will require logging lots of hours in aircraft like VENOM’s F-16s and the X-62A, as well as future efforts in a variety of other crewed and uncrewed aircraft. But the timetable is rapidly accelerating.

“The neat thing about this is you can do it really quickly. It usually takes a very long time to certify software to fly in airplanes. If you were to wait for that, maybe you would do one drop a year,” Coleman said. “VENOM enables us to do tens of thousands of cold drops per week. So it’s a really fundamental kind of capability that bridges and accelerates that transition from fully manned to this mixed mode. Human-machine teaming.”

This AI-teaming capability is expected to come as part of the Air Force’s Next Generation Air Dominance fighter slated to enter service by the mid-2030s, and that means programs like Project VENOM will need to operate with that kind of velocity to keep pace.

Feature image created by Alex Hollings using DoD images and graphics.

Read more from Sandboxx News

• 6 crazy DoD programs that sound like science fiction
• AI wins flawless victory against human fighter pilot in virtual dogfights
• Project Horizon: Nukes and shotguns on the moon
• Navy SEALs aim to incorporate AI into future ops
• How US special operators use Artificial Intelligence to get an edge