AI TEAMWORK|DRONE SWARMS|SIMULATION

The CompositionAxis

Controller-Guided, Composition-Agnostic Multi-Agent RL for Drone Swarms in a Physically Grounded Digital Twin

Teaching one AI pilot to fly a team of drones of any size — on simulated drones that behave like real ones, in games where cheating doesn't pay.

Zahid Hussain

Texas A&M University

now flying5v5same AI brain

17team line-ups, all practised

5games, one simulator

↓ SCROLL

Start here

Everything you need to follow this page, in five minutes

No background in drones or AI needed. These six ideas are enough to read every section that follows. Any underlined word, anywhere on the page, can be hovered (or tapped) for a plain explanation.

the project in one breath

Train one AI pilot that can fly a team of drones of any size, split between teams in any way — on simulated drones that behave like real ones, in games where cheating doesn't pay.

Learning by trial and error

Reinforcement learning is how the AI learns. It tries an action, sees what happened, and gets a score — a reward. Over millions of tries it does more of what scored well. Nobody tells it the right move; it has to discover it.

The AI's brain: a policy

The thing being trained is a policy: a program that takes in what one drone senses and outputs what that drone should do. It is a neural network — a huge adjustable formula. Every drone runs its own copy of the same trained brain.

How a drone actually moves

A quadrotor has four rotors that only push along its own “up”. To go sideways it must lean, and leaning takes time. Air resistance, wind and a draining battery all get in the way. Many AI studies ignore this and simulate idealised dots instead.

Teams, and games between teams

Here many drones act at once — multi-agent learning. In cooperative tasks every drone is on one side (fly in formation, keep watch over points). In competitive tasks two teams play against each other (chase, capture-the-flag, hold zones).

Practising in a simulator

Real drones crash and cost money, so all practice happens in a digital twin — a detailed computer model of the arena and the drones. The AI plays millions of rounds (episodes) there before anything flies for real.

A hand-written pilot to learn from

For every task, engineers also wrote a rule-based pilot — the scripted controller. It's decent but not brilliant. The AI first learns from it, then has to beat it. Beating it is how success is measured.

the page, as a map

1The problem and the idea— why today's AI drone teams fall short 2The world— realistic drones, a safety guard, a changing arena 3The game from a drone's eyes— what each drone knows, and what it can do 4How the AI is built and trained— one brain for any team, practice schedules, the teacher 5The tasks and how results are judged— five games, strict rules, the scores to beat

how to read it

Every section opens in plain words. The deeper detail — formulas, exact settings — comes after, and each formula has an “in words” line beneath it.
Underlined words are explained on hover, and all of them are collected in the glossary at the end.
Figures are live. Drag the sliders and drones — the numbers are computed, not drawn.
Numbers come from the paper. Anything illustrative is labelled as such where it appears.

01 — The problem

Why AI drone teams aren't ready for the real world yet

Researchers already know about each of these three shortcuts on its own. What hasn't been done before is removing all three at once, across both team-versus-team and all-on-one-side games. Click a card to see what this paper does about it.

The paper's central idea is the composition axis: treat “how many drones, and how they're split between the teams” as something to practise across, not something fixed in advance. With 6 to 10 drones and each team holding between 30% and 70% of them, there are exactly seventeen possible line-ups — few enough to practise every one of them and keep score on each.

What the paper contributes

Five pieces that fit together

The AI's basic network design isn't new, and the paper says so. What's new is practising across every team line-up, using the hand-written pilot as a teacher rather than only as an opponent, and building a simulator and scoring rules strict enough that the results can be trusted.

(i)Simulator

A realistic, checkable drone simulator

Simulated drones lean, lag, feel drag, gusts, other drones' downdraft, battery drain and signal delay — all varied from round to round. They only know about each other through 'tracks' built from time-stamped, noisy sightings that obstacles can block, shared over a team radio — and they learn of game events only from a referee's announcements. The hand-written pilots work from exactly the same information. A safety guard stops every pilot, human-written or AI, from flying into things. Obstacles move every round. And the game rules are written so the obvious cheats are impossible or pointless.

(ii)CAST + PCR

One brain for any team, and a smart practice schedule

An AI design (CAST) that reads a list of what it can see — however long — instead of fixed seats, so it works for any team size. A practice scheduler (PCR) picks which team line-up to play next, and a league of past opponents is organised by line-up too.

(iii)CASH

One brain for different kinds of drone

Each drone tells the AI its own top speed and acceleration, and the AI adjusts its output to suit — so fast chasers and nimble dodgers can share the same brain.

(iv)CGPO

Learning from a hand-written pilot

Each task has a rule-based pilot written by engineers. The AI learns from it in four ways, in two versions: one that flies alone, and one that adds small corrections on top of the pilot. An optional 'trust gate' decides moment by moment how much to follow the pilot.

(v)Honest scoring

Scores that mean what they say

Crashing never pays. Winning always clearly out-earns losing, and there are no points to farm. Hard tasks start easy. The AI is judged by its weakest side, not its average. And every saved AI is checked for cheating before its score counts.

02 — The vehicle

The simulated drones fly like real ones

The AI never touches the motors. It just asks: 'accelerate this way, this hard' — the same kind of instruction a real drone's autopilot accepts. The simulator then works out what a real drone would actually do with that request, including every way it falls short.

the physics, in one line

\dot v = T z_b - g e_3 - k_1 v_{\mathrm{air}} - k_2 \lVert v_{\mathrm{air}}\rVert v_{\mathrm{air}} - a_{\mathrm{dw}} e_3, \qquad v_{\mathrm{air}} = v - w

in words

How the drone's speed changes ( $\dot v$ ) = the rotors' push ( $T$ ) in whatever direction the drone is leaning ( $z_b$ ) − gravity − two kinds of air resistance, measured against the moving air rather than the ground ( $v - w$ , where w is the wind) − the downdraft from drones above.

Before any of that, the request is checked against what the airframe can do: the drone may lean at most 40°, and its rotors can push between 0.1 and 2.2 times gravity (g). And the drone doesn't snap to the new lean and push — it catches up over 0.12 s and 0.04 s, and hears the command a moment late (latency).

(a) how fast it speeds up — real drone vs. idealised dot

requested acceleration10.0 m/s²

Both are told “full speed ahead” and both feel the same air resistance. The idealised dot obeys instantly. The realistic drone has to lean first (a delay), and it can only lean 40° — so above about 8.2 m/s² of requested acceleration it simply can't deliver. Toggle each effect off to see what it costs. After 3 s the drone is at 13.07 m/s versus the dot's 15.40 m/s — the drone is 15% slower.

how far below1.0 m

Rotors blow air downward. A drone flying underneath gets pushed down and has to work harder just to stay level. The push is strongest right below (up to 0.6 g, i.e. 60% of gravity) and fades to nothing 4 m down, spreading wider as it falls. At 1.0 m below, the push is 0.450 g. The strength and reach are the paper's; the exact sideways spread is our reconstruction.

(b) which moves are physically possible

box size6.0 m/s²

Drag the circle to request a move: sideways (left–right) and up/down. The dashed box is everything the AI is allowed to ask for; the green fan is what a real drone can actually do, given its 40° lean limit and the fact that rotors can only push, never pull. Ask for “hard sideways and hard downward” at once and it can't — it would have to lean past 40° with almost no push. This request is cut back: you asked for (5.0 sideways, -5.0 up) and the drone delivers (4.0, -5.0) m/s².

the model, piece by piece

What the AI commands: u ∈ [−1,1]³
Maximum lean: 40°
Rotor push range: 0.1 g – 2.2 g
Delay to lean: 0.12 s
Delay to change push: 0.04 s
Signal delay: one tiny step
Rotor drag: 0.25 per second
Body drag: 0.02 per metre
Wind: up to 2 m/s + 1 m/s gusts
Downdraft: 0.6 g → 0 over 4 m
Battery drain: −20% push
Decision rate: every 0.1 s

varied for every drone, every round

Thrust-to-weight ±10%Attitude and motor lags ±30%Both drag coefficients ±30%

Varying the drones each round means the AI can't rely on one exact airframe — a standard trick for making simulator-trained skills survive on real hardware. The extra lift drones get very close to the ground is left out; drones never fly below 1 m, where it would matter.

03 — The shield

A safety guard that stops crashes before they happen

Penalising collisions isn't enough: an AI will happily accept a small penalty if it wins more points elsewhere. So a safety guard sits between every pilot — AI or hand-written — and the drone, and strips out any part of a command that would fly it into something it couldn't stop in time.

the safety bubble, seen from above

approach speed6.0 m/sbraking power6.0 m/s²

A drone flies toward an obstacle. Outside the orange ring it's free; inside, the guard removes any push toward the obstacle and steers it around. Speed it up and watch the ring grow: the faster you're going, the earlier you need to start braking. The bar underneath shows the three parts of the safety distance. The AI trains with the guard switched on, so it learns to work with it (CBF-RL).

the safety distance

d^\star = r + \frac{v_{\mathrm{in}}^2}{2 a_{\max}} + \tau\,\lvert v_{\mathrm{in}}\rvert, \qquad v_{\mathrm{in}} = \min(v\cdot n,\, 0)

in words

Safety distance ( $d^\star$ ) = a fixed 4 m buffer + the distance needed to brake from your current approach speed ( $v_{\mathrm{in}}$ ) + extra room for the drone's 0.4 s reaction delay. Only the part of your speed heading toward the obstacle counts — flying past it doesn't shrink the bubble.

Around obstacles

The guard keeps a safety bubble around each obstacle. The bubble grows with speed: 4 m of margin, plus the distance needed to brake, plus extra for the drone's reaction delay. Inside it, any push toward the obstacle is removed and the drone is slowed; a gentle push away and sideways steers it around.

Walls, floor and ceiling

The arena's six faces get the same treatment — a 2 m margin, 3 m above the floor — so no pilot can fly into a wall it couldn't stop in time.

Teammates

Two drones on the same team are kept at least 2 m apart, plus whatever distance they'd need to stop if closing on each other.

Opponents — deliberately not protected

Chasing, tagging and blocking have to stay possible, so the guard never separates drones on opposite teams.

A subtle but important detail: the guard changes what the drone does, not what the AI asked for. The AI is still scored on its own request, so the learning maths stays correct — and the AI has to discover for itself which requests get overruled.

04 — The arena

A new obstacle course every round

If the obstacles never moved, the AI could just memorise the map. So they're rearranged at the start of every round, and each drone has to 'see' them with its sensors, every time.

obstacle layout — a fresh one every round

Between 3 and 8 pillars and boxes are dropped at random inside the central 40 × 40 m (dashed square). Any placement that lands too close to another obstacle (less than 6 m of clear space) or on top of an important spot — the formation's path, the bases, the landing pads, the zones (blue circles) — is thrown away and tried again. This round got 4.

what a drone can actually sense

Spotting other drones: 15 m, line of sight
Keeping track: tracks, dropped after 3 s
Obstacle sensors: 10 beams, 12 m
Team radio: 20 m, 5% dropout
The referee: announcements, 1 per second

random, but repeatable

Everything random — the layout, the wind, each drone's build, the sensor noise — comes from a starting number, the seed. Use the same seed and you get exactly the same round again, which makes every result replayable. The same simulator code is used for training, testing, recordings and a live 3-D viewer in the browser.

The arena is 60 × 60 × 30 m — about half a football pitch, three storeys high. The AI makes a decision every 0.1 s, while the physics is calculated five times in between (every 20 ms) so the motion stays smooth and accurate.

05 — The decision problem

The game, from the AI's point of view

So far we've seen the world: realistic drones, a safety guard, a changing arena. Now: how that world is turned into a game the AI can learn. Who's playing, what's really going on, what each drone can know, what it can do, and how it's scored.

the game, in seven parts

Researchers write this as a “partially observable Markov game”. The symbols are just labels; the plain meaning is beside each.

\mathcal{G} = \big\langle\, \mathcal{N},\ \mathcal{S},\ \{\mathcal{O}_i\},\ \{\mathcal{A}_i\},\ P,\ \{R_i\},\ \gamma \,\big\rangle

\mathcal{N}

Who plays

every drone in this round's line-up

There is always room for 10. Unused slots are 'parked': invisible, can't collide, earn nothing, teach nothing.

\mathcal{S}

What's really going on

the full truth: every drone, the wind, the obstacles, the score

Only the simulator knows all of it. No drone ever does.

\mathcal{O}_i

What one drone perceives

a list: about itself, about other drones, about objects

Never the other drones themselves — only its own tracks of them: best guesses from time-stamped, slightly noisy sightings, blocked by obstacles and limited to 15 m, pooled with teammates over a 20 m radio — plus the referee's announcements.

\mathcal{A}_i

What one drone can do

three numbers, each between −1 and 1

How hard to accelerate along the arena's east–west, north–south and up–down axes (fixed directions, not the drone's heading) — scaled by that drone's own maximum acceleration.

P

How the world responds

safety guard → realistic drone physics, every 0.1 s

Wind gusts, sensor noise and radio dropouts add randomness — but the same starting number (seed) replays a round exactly.

R_i

How it's scored

points for the task, small hints, penalties for bumps, crashes and walls

Tuned so winning clearly pays more than losing, and no points can be farmed.

\gamma

How much the future counts

0.99

A point one step from now is worth 99% of a point now. And a drone's own crash ends its future — nothing is counted after it.

The full truth: what the simulator keeps track of

To work out what happens next, the simulator needs all of this, every moment. Together it's called the state. No single drone ever sees it all — which is exactly what makes the game hard.

Each drone

where it is and how fast it's moving
how far it's currently leaning, and how hard its rotors push — both still catching up to the last command
the command that's on its way (the tiny signal delay)
how much battery is left, which limits its push
its own randomised build: how strong, how quick to respond, how draggy
its top speed and acceleration, its team, and whether it's playing or parked
its tracks: a best guess of where every other drone is, and how old each guess is

The world

this round's steady wind (up to 2 m/s, random direction)
the current gust
where the 3–8 obstacles are
the 60 × 60 × 30 m arena walls
which radio links dropped out this moment

The game

the objects: formation spots, points to watch, pads, flags, zones
their status: how stale each point is, who carries a flag, who holds each zone
timers: respawns, the round clock
the score
how hard the task is set, and who's on which team

Why a drone's view isn't the full truth. It never sees other drones directly — only its own best guesses (tracks), built from sightings that are 0.1 s late, a little off, limited to 15 m and blocked by obstacles, plus what teammates radio in. A drone that ducks behind a pillar simply fades from its knowledge, and game events arrive only through the referee's announcements. Obstacles are only known through ten sensor beams. The wind and other drones' builds aren't among its inputs. And because the drone is always catching up to its last command, the same instruction can play out differently depending on what it was just doing. It's like driving in fog with a slightly sluggish car.

This list is what the paper's physics needs to run; the paper describes the model, not the simulator's internal bookkeeping.

one round, as the simulator runs it

Every 0.1 s

1sightings are made and shared; each drone updates its tracks and builds its view
2the same AI brain decides a move for every drone
3the safety guard removes anything dangerous
4physics runs five times (every 20 ms)
5points, bumps, tags and crashes are counted

Formation

40 s = 400 decisions per drone = 2000 physics updates

Capture-the-flag

60 s = 600 decisions per drone = 3000 physics updates

or first to three captures

who sees what

During the game, each drone decides on its own, using only what it can sense and what teammates radio in (the dotted circles). No drone sees the whole field. This part — the actor — is what actually flies.

how each drone is scored, every moment

r^i_t = r^{\text{task}}_t + \underbrace{\gamma\,\Phi(s_{t+1}) - \Phi(s_t)}_{\text{shaping}} - c^{\text{contact}}_t - c^{\text{crash}}_t - c^{\text{wall}}_t

in words

A drone's points this moment ( $r^i_t$ ) = points for the task + small hint points for progress (built so hints can never change what the best strategy is — potential-based shaping) − penalties for bumping into things, crashing, and flying close to walls.

A grouping of the terms the paper lists; the hint term is written in its standard form.

Task points

What the game is actually about — holding zones, watching points, landing, capturing. Its weight is set by measurement so it always dominates.

Hint points

Small nudges for progress, e.g. getting closer to your spot. Built with a mathematical recipe that guarantees hints can't change what the best strategy is.

Bump penalty

Touching an obstacle, a teammate or an opponent costs a little every moment it lasts, plus a one-off cost the first time.

Crash penalty

Crashing costs points and ends that drone's count — so dying never looks like a free average score.

Wall penalty

Flying close to the arena walls costs a little, on top of the safety guard that already stops the drone hitting them.

Task points, game by game — zone control: 0.3 × (zones held − zones lost) each step, plus a win bonus. Coverage: pay for stale points, earn back exactly what a visit clears. Pursuit: a landing bonus plus whatever the runner would have earned by staying. Capture-the-flag: flag bonuses that are zero-sum between teams.

Before learning uses them, points and inputs are rescaled to steady ranges, so no single number overwhelms the rest. Parked slots contribute nothing at any stage.

06 — What a drone sees

What each drone knows — and what it doesn't

A drone never sees the other drones directly. It spots them — only within 15 m, only if no obstacle is in the way, a split second late and a little off — and its teammates share what they spot over a 20 m radio. From those sightings it keeps a running best guess of where each drone is, called a track. Drag the ringed drone and watch what it can and can't know.

┅ 15 m sensing┈ 20 m radio— sensor beam hits an obstacle— I can see it┄ a teammate sees it┄ blocked● team A · ● team B · ○ parked

Drag the ringed drone · click any drone to see through its eyes · click a blue radio link to cut it

what the AI actually receives

itself1 item10 + 10 + 6 + 4 + 2 + 3 + 8 = 43 numbers

drones9 itemswhere, how moving, teammate, inactive, task flag, mask, age — 11 each

objects8 itemswhat, where, detail, whose, on/off

Each item is called a token. The counts match the observation layout in the environment code (222 numbers in all). The layout is identical for every drone, every team size and every game — only the numbers and the on/off switches change.

what never reaches the input

· opponents that no one in its radio group can see — too far, or behind something
· any obstacle that none of the ten beams happens to hit
· the wind, and how other drones are built
· any name or number for a drone — there is no “drone #3” anywhere

I see it

known by radio

switched off

About itself — A1

1 item

10 obstacle sensors · how far until each beam hits something (max 12 m)

5.3

12.0

4.9

12.0

7.1

12.0

distance to the 4 walls, floor and ceiling (m)

west

24.0

east

36.0

north

30.0

south

30.0

floor

12.0

ceiling

18.0

Floor and ceiling shown for an assumed 12 m altitude.

10 about its own motion4 team and clock2 its top speed and acceleration3 the hand-written pilot's suggestion8 task-specific numbers

The pilot's suggestion is only filled in for the “adds corrections” version, so that AI can see what it's correcting. The paper doesn't list the ten motion numbers individually.

About other drones

always 9 slots, full or not

slot	known how?	where, from me (m)	moving	mate	inactive	age	mask
B1	I see it · 11.7 m	+6.2, +10.1	-2.2, -0.2	0	0	0.00	1
A2	I see it · 12.2 m	+10.0, -7.2	-0.6, 1.3	1	0	0.00	1
A3	I see it · 13.0 m	-6.8, +11.1	-1.7, 0.7	1	0	0.00	1
B2	A2 sees it · radio	+13.1, +1.9	-1.6, 1.2	0	0	0.00	1
B4	A4 sees it · radio	+27.0, +20.0	-1.5, -0.5	0	0	0.00	1
A4	reports itself · radio	+19.0, +8.1	-1.0, -0.7	1	0	0.00	1
B3	behind an obstacle	0, 0	0, 0	0	0	—	0
A5	out of reach	0, 0	0, 0	1	0	—	0
B5	parked	0, 0	0, 0	0	1	—	0

These are the drone's tracks — best guesses, not the truth: each sighting is judged as of 0.1 s ago, projected forward to now, and is slightly off (more so further away), so the numbers differ a little from where the dots are drawn. A drone behind an obstacle can't be seen, unless a teammate in the radio group sees it. Rows that are switched off are zeros the AI ignores (a mask). “Age” is how long since the guess was last refreshed, as a share of the 3 s limit — all fresh here; the figure below shows it growing. Shown flat for readability; the real numbers are 3-D. The drone names are for you: the AI's slots carry no identity, and unfilled ones are simply zeros with the mask off.

About objects

up to 8

zone Z1

pos -14.0, +0.0

relation +1 own

zone Z2

pos +9.0, -19.0

relation 0 neutral

zone Z3

pos +30.0, +6.0

relation −1 enemy

Each also carries one extra detail and an on/off switch. “Own” and “enemy” are from this drone's side — click an orange drone and they swap.

the state table — one drone's complete input

The scenario: a 5-v-4 round of zone control (one of the 17 line-ups; the standard game is 5 v 5). Three zones, each with a 6 m radius: Z1 held by team A, Z2 held by nobody, Z3 held by team B. One of team B's slots is parked. Every drone flies at 12 m. The table below is what drone A1 (team A) feeds into the AI right now — drag it, or click another drone, and the examples update.

#	Field	What it means	Format	Numbers	Example — as the network receives it
About itselfone token · a dead or inactive drone gets all zeros · 1 × 43 numbers
0–2	Position	where I am: x and y from the arena centre, z from the floor	3-D vector: x ÷ 30, y ÷ 30 (−1…1), z ÷ 15 (0 at the floor, 2 at the ceiling)	3	(−6.0, +0.0, 12.0) m → (−0.20, +0.00, +0.80)	from code
3–5	Velocity	how fast I'm moving; 1.0 always means my own top speed	3-D vector ÷ my v_max	3	(+1.2, −0.4, +0.0) m/s → (+0.12, −0.04, +0.00) if v_max = 10 m/s	from code
6–8	Acceleration	the acceleration the airframe actually achieved (after lag, drag and wind; gravity excluded) — not the command	3-D vector ÷ my a_max	3	(+0.00, +0.00, +0.00) — cruising at constant velocity in this still scene	from code
9	Battery	charge left, including the voltage sag	number, 0 to 1	1	0.94 — e.g. early in the round	illustrative
10–19	Obstacle rays	how far each of 10 beams gets before hitting an obstacle, ÷ 12 m — 1.00 means nothing within 12 m	list of 10 numbers, 0 to 1	10	[0.44, 1.00, 1.00, 1.00, 1.00, 1.00, 0.41, 1.00, 0.59, 1.00]	from code
20–25	Wall distances	distance to the west, east, north and south walls (÷ 30) and the floor and ceiling (÷ 15)	list of 6 numbers	6	west 0.80 · east 1.20 · north 1.00 · south 1.00 · floor 0.80 · ceiling 1.20	from code
26	Which team	team index ÷ (number of teams − 1): 0 = red / chasers, 1 = blue / runners; always 0 in the one-team games	number, 0 or 1	1	0.00 — team A	from code
27	Teammates alive	alive, active teammates (not counting me) ÷ the roster size of 10 — counts every drone in the arena, seen or not	number, 0 to 1	1	0.40 — 4 teammates	from code
28	Opponents alive	alive, active opponents ÷ 10 — also an exact count, including ones I can't see	number, 0 to 1	1	0.40 — 4 opponents	from code
29	Time left	1 − step ÷ max steps, falling from 1 to 0	number, 0 to 1	1	0.80 — e.g. 12 s into a 60 s round	illustrative
30–31	Speed and agility limits	my top speed and top acceleration, each divided by a reference value (12 m/s and 8 m/s² in the code)	2 numbers: v_max ÷ 12, a_max ÷ 8	2	the chase game's chaser (10 m/s, 5 m/s²) would read (0.83, 0.63)	likely
32–34	Pilot's suggestion	the hand-written pilot's command at this moment	3-D vector, each −1 to 1	3	(0, 0, 0) — filled in only for the “adds corrections” version	from code
35–37	Task: nearest zone	the nearest zone by ground distance, as target minus me (x ÷ 30, y ÷ 30, z ÷ 15)	3-D vector	3	Z1: (−0.47, +0.00, −0.80)	from code
38	Task: my team owns it	1 if my team holds that zone	0 or 1	1	1	from code
39	Task: nobody owns it	1 if the zone is neutral or tied	0 or 1	1	0	from code
40	Task: zones we hold	fraction of all zones my team holds	number, 0 to 1	1	0.33	from code
41	Task: zones they hold	fraction of all zones the other team holds	number, 0 to 1	1	0.33	from code
42	Task: spare	unused in zone control	always 0	1	0	from code
About other dronesone token per slot, filled from tracks nearest first · 9 × 11 numbers
0–2	Where it is (guess)	the track's estimate of that drone's position, relative to me, divided by the 15 m sensing range	3-D vector (x, y, z), metres ÷ 15	3	(+6.2, +10.1, +0.0) m → fed in as (+0.42, +0.67, +0.00) — B1	from code
3–5	How it's moving (guess)	the track's estimate of its velocity relative to mine, divided by twice my top speed	3-D vector, m/s ÷ (2 × v_max)	3	(−2.2, −0.2, +0.0) m/s — B1	from code
6	Is teammate	1 if that drone is on my team	0 or 1	1	0 — B1 is an opponent	from code
7	Is inactive	1 if that drone is out of play	0 or 1	1	0 — B1 is flying	from code
8	Task flag	one task-specific flag per slot: tagged or landed in pursuit, carrying the flag in capture-the-flag	0 or 1	1	0 — probably unused in zone control	likely
9	Mask (slot filled)	1 if the slot holds a live track; unfilled slots are zeros the network ignores	0 or 1	1	1 — 6 of the 9 slots are filled right now	from code
10	Track age	time since the track was last updated, divided by the 3 s timeout, clipped to 0–1	number, 0 to 1	1	0.00 — seen this step; a track 2.3 s old reads 0.77	from code
About objectsone token per slot · 8 × 10 numbers
0–3	Type	flag, point of interest, zone or goal — the four entity kinds	one-hot over 4 categories (a single 1)	4	zone → (0, 0, 1, 0), if the slots follow that order	from code
4–6	Where it is	relative to me, divided by half the arena (30 m); objects are fixed or carry a beacon, so this is exact	3-D vector (x, y, z), metres ÷ 30	3	(−14.0, +0.0, −12.0) m → (−0.47, +0.00, −0.40) — Z1	from code
7	Attribute	one value per object: staleness for a point of interest, flag-at-home for a flag, presence count for a zone	1 number	1	0 — drones inside Z1 right now	likely
8	Owner	mine, nobody's, or the other team's — from this drone's side	+1, 0 or −1	1	+1 — Z1 is held by my team	from code
9	Mask (slot filled)	1 if the slot holds an object	0 or 1	1	1 for the 3 zones, 0 for the 5 unused slots	from code
	Total	43 + 9 × 11 + 8 × 10 — the same layout for every drone, every line-up and every task		222	matches the observation layout in the environment code

the 8 task numbers (35–42), game by game

Positions are “target minus me”, x and y ÷ 30, z ÷ 15. Zones, bases, flags, pads and points of interest are shared game state, so every drone is meant to receive them.

Formation	3my slot's position3my slot's velocity ÷ v_max1distance to my slot ÷ 301stage ÷ 3 (static 0, translate 0.33, orbit 0.67, morph 1)	8 numbers
Coverage	3the point I'm heading for1its staleness ÷ max staleness1fraction of points covered now2zeros “Heading for” = the lowest ground distance ÷ (1 + staleness / 10), so nearby stale points win. One of the eight is not documented yet.	7 of 8 listed
Pursuit — chaser	3the nearest runner still free1is it within 15 m?1fraction of runners still free1am I tagged or landed?1zero The nearest runner's position is taken from the true state whenever it is within 15 m — not from the drone's tracks. One of the eight is not documented yet.	7 of 8 listed
Pursuit — runner	3the nearest landing pad, centred at half its height1pad radius ÷ 30 (= 0.1)1role (= 0)1fraction of runners still free1am I tagged or landed?1zero	8 numbers
Capture-the-flag	3my own base3the enemy flag's live position (it follows its carrier)1am I carrying it?1am I stunned?	8 numbers
Zone control	3the nearest zone by ground distance1my team owns it1nobody owns it (neutral or tied)1fraction of zones we hold1fraction they hold1zero The scenario above. The order of these eight is reconstructed and not yet confirmed.	8 numbers

No text ever reaches the network. Everything is a number: categories like “zone” become a row of 0s and a 1, yes/no questions become 0 or 1, and “whose” becomes +1, 0 or −1. The examples are shown after each field's fixed scaling (÷ 30, ÷ 15, ÷ 12 …). On top of that, every number except the presence flags (masks) goes through a running per-number rescaling before the network sees it (normalisation). The drone's own state is exact — no noise.

Legend. live — defined by the paper or the code; the value is computed from the scene.from code — confirmed from the observation layout in the environment code (base_env.py), not stated in the paper.paper — value fixed by the paper for this scenario.likely — the code suggests this, but it has not been confirmed.illustrative — the paper names this field but not a value for this case; the example is ours.not listed — named in the paper and the code layout, but its contents are not documented. The coach (critic) receives the same layout for every drone, built from true positions with age 0.

how a track is kept — and lost

A chaser (C) follows a runner that slips behind a pillar. The hollow circle is the chaser's best guess; the orange dot is the truth.

seen now by

nobody

guess is off by

5.3 m

age of the guess

2.3 s of 3 s

time4.0 s

While the runner is visible, each sighting nudges the guess halfway toward it and adjusts its estimated speed (an α-β tracker, α = 0.5, β = 0.25). Once it's hidden, the guess keeps sliding along at the last known speed — but the runner has turned, so the guess drifts off. After 3 s with no sighting it's dropped. A sighting that comes after more than two steps of drifting, or lands outside the gate, restarts the track rather than dragging it. With a teammate watching from another angle, the guess stays fresh throughout.

Computed with the paper's tracker settings, time-stamped sightings, gate and noise model; the scene itself is illustrative. This is why obstacles become tactical, and why the radio has to share enemy sightings, not just teammates' positions. The same holds for fates: a crash in view ends a track, but a crash nobody saw leaves it drifting until it times out, still looking alive.

what the referee tells everyone

Game events don't reach the drones by magic either. Like a real match, a referee announces what happened, and that is all anyone knows. The hand-written pilots work from exactly the same information; only the coach, during practice, sees the truth.

Fixed objects

known exactly

formation slots, pads, bases, zones and points of interest

A flag on the ground

where the last announcement put it

pickups, drops, returns and captures are announced

A carried flag

beacon, once a second, ±2 m

so a carrier's exact path is not public

Zone ownership

broadcast once a second

who holds each zone — but never how many drones are inside

Who is still playing

the scoreboard's count

tags, stuns and landings are announced; a crash nobody saw is not, so the opponents-alive number can be wrong

Why a list, not fixed seats

Fixed seats — 'drone #1 goes here, #2 there' — break the moment the team size changes. A list of items with on/off switches has room for any team up to 10, in any order. That's what lets one AI brain handle all 17 line-ups.

Why the on/off switch matters

Empty and unknown slots are still there, but switched off, so the AI ignores them completely and learns nothing from them. The shape of the input never changes — only which slots are on.

Why the radio matters

It passes on teammates' positions and — just as important — what they can see: an opponent hidden from you may be in plain view of a teammate. Cut a radio link and watch drones vanish from the input. Only teammates relay; opponents never do — and a stunned or landed drone, though still visible to others, shares nothing.

07 — What a drone does

What each drone does: three numbers, ten times a second

Every 0.1 s, the AI gives each drone three numbers: how hard to accelerate along the arena's three fixed directions — east–west, north–south and up–down. That's the whole decision. Between that request and the drone actually moving sit the safety guard and the limits of the drone itself.

the choice

u \in [-1, 1]^3

Three numbers, each anywhere from −1 to 1 (“u” is just the name). One set per drone, every 0.1 s.

the meaning

a_{\mathrm{des}} = u\, a_{\max}

Multiplied by that drone's maximum acceleration — so “1” means “flat out”, whatever the drone.

the whole team

\mathbf{a}_t = (u^1_t, \dots, u^{n_0+n_1}_t)

All drones decide at the same moment, each from its own view. Parked slots do nothing.

from the AI's decision to actual movement

Set a command with the sliders. Each box shows what's left of it after one more step, seen from the side (x = east–west, z = up–down).

1 · the AI's three numbers

(0.9, 0.0, -0.8)

each between −1 and 1

→

2 · × this drone's max

(7.2, 0.0, -6.4) m/s²

the acceleration requested

→

3 · after the safety guard

(7.2, 0.0, -6.4) m/s²

nothing dangerous — unchanged

→

4 · what the drone can do

(2.9, 0.0, -6.4) m/s²

cut back: can't lean that far

x · east–west0.90y · north–south0.00z · up–down-0.80

drone's max8.0 m/s²

Then the drone still has to lean and spin up — taking 0.12 s and 0.04 s — so the movement arrives a moment later, as in figure (a) of the vehicle section.

the action table — one drone's output, and what becomes of it

Same scenario as the state table: one drone in the 5-v-4 zone-control round. The examples follow the sliders above. Directions are the arena's fixed axes, not the drone's own heading — the paper's action is a world-frame acceleration.

Field	What it means	Format	Numbers	Produced by	Example (this scenario)
Best guess (mean)	the centre of the AI's choice on each axis	3-D vector, each about −1 to 1	3	AI	(+0.86, +0.03, −0.76)	illustrative
Spread (randomness)	how far the AI may stray from its guess; log σ is kept in [−2.5, 0] ([−2.5, −1] for the corrections version)	3 numbers (one per axis)	3	AI	log σ = (−1.2, −1.2, −1.2) → σ ≈ 0.30	illustrative
The action u	the pick actually used: how hard to accelerate along the arena's x (east–west), y (north–south) and z (up–down), as a share of the maximum	3-D vector, each −1 to 1, no units	3	AI	(+0.90, +0.00, −0.80)	live
Requested acceleration	a_des = u × a_max, with this drone's a_max = 8.0 m/s²	3-D vector, m/s²	3	rule	(+7.2, +0.0, −6.4)	live
After the safety guard	any push toward an obstacle it couldn't stop before is removed	3-D vector, m/s²	—	simulator	(+7.2, +0.0, −6.4)	live
What the drone delivers	cut back to the 40° lean and the 0.1–2.2 g push; arrives after the lean and motor delays	3-D vector, m/s²	—	simulator	(+2.9, +0.0, −6.4)	live
Corrections version	a = clip(pilot + 0.25 × u); shown for a pilot command of (+0.60, 0, 0)	3-D vector, each −1 to 1	3	AI + pilot	(+0.82, +0.00, −0.20)	live
How often	one action per active drone, every 0.1 s; parked slots send nothing	—	—	—	5-v-4 round: 9 drones → 90 actions (270 numbers) per second	live

The AI itself outputs only the first two rows; the pick u is drawn from them. Everything after u is done by fixed rules and the simulator — the AI is judged on u, not on what the guard or the airframe changed. The mean, spread and pilot command are example values; the formulas and limits are the paper's.

how the AI picks a number: best guess + a little randomness

best guess0.55randomness-0.80

spread σ

0.449

allowed randomness

[-2.5, 0]

picks outside range

15.9%

For each of the three numbers, the AI outputs a best guess and then picks randomly around it, following a bell curve (Gaussian). The randomness is how it keeps exploring. Its spread σ is kept between about 0.08 and 1 (the slider is on a log scale: −2.5 to 0), and a small bonus rewards staying a bit random so it doesn't settle too early.

The paper gives the allowed range but not how a pick outside it is brought back in.

why “accelerate”, not “spin motor 3 faster”

Real drones already have an autopilot that turns “accelerate forward” into motor speeds and keeps the drone stable. Speaking that language means a trained AI could sit on top of a real one. It also means the AI and the hand-written pilot give orders in the same form — so the AI can copy the pilot, be compared with it, or add corrections to it.

what the AI isn't told

When the safety guard or the drone's limits change a command, the AI isn't told. It's judged on what it asked for and what happened next — so it has to learn, from experience, which requests actually work.

Two limits worth remembering. Leaning at most 40°, a drone holding its height can accelerate sideways at no more than about 8.2 m/s². And since its rotors can only ease off, never pull down, it can speed up downward by at most 8.8 m/s² — a little less than falling (g = 9.81 m/s²).

08 — The composition axis

Practising every team line-up — but mostly the ones worth practising

There are only seventeen possible line-ups, so the AI can practise all of them and keep a score for each. The trick is choosing which one to play next. Like a good coach, the scheduler avoids drills the AI has mastered and drills that are hopeless for now, and focuses on the ones it wins about half the time.

the practice scheduler (PCR)

Each box is a line-up (e.g. 3v5 = 3 drones against 5). The bar is how often the AI currently wins it; the darker the box, the more often it gets practised. Press play to watch training unfold.

N = 6

N = 7

N = 8

N = 9

N = 10

Hover a line-up for its numbers. Line-ups the AI always wins, or always loses, fade; the ones in between stay dark.

how the next line-up is picked

P(c) = (1-\rho)\,P_{\mathrm{score}}(c) + \rho\,P_{\mathrm{stale}}(c)

P_{\mathrm{score}}(c) \propto \mathrm{rank}(s_c)^{-1/\beta}, \quad s_c = p_c(1-p_c)

in words

Each line-up c has a recent win rate p. Its “worth practising” score is p × (1 − p): zero if the AI always wins (p = 1) or always loses (p = 0), highest at 50/50. Line-ups are ranked by that score and picked mostly by rank — but 30% of the time (ρ = 0.3) by how long since they were last played, so nothing is forgotten.

the line-ups

Drones in total: 6 – 10
Each team's share: 30% – 70%
In the chase game: chasers hold 50–70%, never outnumbered
Team-vs-team line-ups: 17
All-one-team sizes: 5
Room always kept for: 10 drones

Unused slots are parked — invisible, unable to collide, and ignored by learning. So the AI's input always has the same shape, whatever the line-up.

Other methods practise small teams first and build up to one target size, then stop. This scheduler never stops covering every line-up, so the final AI is good at all of them — not just the one it ended on.

09 — CAST

One brain that reads a list, however long

The AI's network (called CAST) reads each drone's view as a list of items — itself, other drones, objects — and uses a transformer, the same kind of network behind modern chatbots, to decide which items matter. Nothing in it says 'drone #3', so the same brain works for any team size.

the pilot (actor) — one drone's view

What a transformer does

It looks at every item in the list and, for each one, decides how much the others matter to it — a chaser nearby matters more than a teammate far away. This weighing is called attention.

Why no drone numbers

Most multi-drone AIs label inputs 'drone #1', '#2'… and break when that count changes. CAST has no labels, so a 3-drone team and a 10-drone team look the same to it: just shorter or longer lists.

Pilot and coach

The pilot (actor) flies, using one drone's list of best guesses. The coach (critic) only exists during practice: it reads every drone at once, built from their true positions rather than guesses, to judge how well the game is going.

About itself

10 numbers about its own motion
10 obstacle sensors (like a parking sensor)
6 distances to the walls, floor and ceiling
4 numbers about its team and the clock
2 numbers: its own top speed and acceleration
what the hand-written pilot would do right now
8 numbers specific to the task

The pilot's suggestion is only filled in for the 'corrections' version, so that AI can see what it is correcting.

About other drones

9 slots

where I think it is, relative to me (÷ 15 m)
how I think it's moving (÷ twice my top speed)
is it a teammate?
is it out of play? (inactive)
a task-specific flag
is this slot filled? (mask)
how old my guess is (÷ 3 s)

Built from this drone's tracks — its best guesses — nearest first, never from the true positions. The age tells the AI how stale each guess is, since it has no memory of its own.

About objects

up to 8

what it is: flag, point to watch, zone or goal (one-hot)
where it is, relative to me (÷ 30 m)
one attribute (e.g. how many drones are in a zone)
mine, neutral, or the enemy's? (+1 / 0 / −1)
is this slot filled? (mask)

Game objects are fixed in place or carry a beacon, so unlike drones they are known exactly. The same three kinds of item describe every task.

10 — League and capability

Practising against its own past selves — on every line-up

In team-vs-team games, the AI practises against saved copies of itself from earlier in training, plus the hand-written pilot. Each practice match is picked as a pair: which opponent, and which line-up. The pairs the AI still loses get picked most — so practice always targets its weak spots.

how often each match-up gets picked

	3v3	4v4	5v5	4v6	3v7
scripted π_b	0.28 4.7%	0.28 4.7%	0.28 4.7%	0.16 6.3%	0.05 8.0%
ckpt 50	0.68 1.1%	0.68 1.1%	0.68 1.1%	0.56 1.9%	0.44 2.9%
ckpt 100	0.54 2.0%	0.54 2.0%	0.54 2.0%	0.42 3.1%	0.30 4.4%
ckpt 150	0.40 3.3%	0.40 3.3%	0.40 3.3%	0.28 4.7%	0.16 6.3%
ckpt 200	0.26 4.9%	0.26 4.9%	0.26 4.9%	0.14 6.6%	0.05 8.0%

Rows are opponents (the hand-written pilot, and saved copies of the AI from training steps 50, 100, …); columns are line-ups. Each box shows how often the AI beats that opponent on that line-up, and — in small print — how often that match gets picked. Press play a few times: the picked match improves, gets picked less, and practice moves on to the next weak spot. The hand-written pilot currently gets 28% of matches.

how the opponent pool works

Every 50 training updates, a copy of the AI is saved into a pool of 8 opponents.
In half of all practice games (probability 0.5), one team is played by a pool member instead of the current AI.
Each opponent gets an Elo rating (as in chess) — overall and separately for every line-up.
The hand-written pilot never leaves the pool: it's picked a lot while the AI still loses to it, and fades once the AI wins.
A match-up is kept for at least three rounds' length, and only rounds played entirely under it count toward its win rate and Elo. The frozen team teaches nothing — its moves never enter the learning.

CASH — one brain, different drones

c = \left(\tfrac{v_{\max}}{V_{\mathrm{ref}}},\, \tfrac{a_{\max}}{A_{\mathrm{ref}}}\right) \;\xrightarrow{\;\text{hypernet}\;}\; W_{\mathrm{last}}, b_{\mathrm{last}}

in words

Each drone passes in two numbers — its top speed and top acceleration, compared with a reference drone. A small helper network (a hypernetwork) uses them to adjust the AI's final decision step to suit that drone.

So fast chasers and nimble runners share one brain instead of needing one each. And because speed and acceleration are just numbers, the AI can even fly a drone with limits it never practised with.

11 — CGPO

Learning from the hand-written pilot — then surpassing it

Starting from scratch wastes the hand-written pilot twice over: the AI has to rediscover basic flying on its own, and practising only against copies of itself makes it good at beating itself rather than the pilot it will be judged against. So the pilot joins training in four ways.

way 1

Copy first

Before any trial and error, the AI copies the pilot.

The hand-written pilot flies for 200,000 steps while the AI watches, and the AI is trained to reproduce its moves (20 passes over the data). Practice starts from 'decent' instead of 'random'.

way 2

A gentle pull back

Straying far from the pilot costs a little.

During training the AI pays a penalty for moving too far from what the pilot would do. The penalty fades over the first 30% of training down to 5% of its strength — and stays there, like a light tether that's never fully cut.

way 3

Let the pilot fly the start

Sometimes the pilot drives the first part of a round.

In half of the rounds at first, the pilot flies the opening (up to 100 steps) and the AI takes over from there. This happens less and less over training. The AI isn't scored on moves it didn't make.

way 4

Keep it as a sparring partner

The pilot never leaves the pool of opponents.

The hand-written pilot is always one of the opponents the AI can be matched against. It gets picked often while the AI still loses to it, and fades away once the AI wins.

how the pilot's help fades during training

Drag through training, from start (0%) to finish (100%).

training12%

pull toward pilot

62%

rounds pilot starts

30%

pilot flies first

60 steps

The pull toward the pilot (the anchor) never disappears — it settles at 5% for the rest of training, like a light tether. The same trick keeps chatbots close to their original behaviour while they're fine-tuned. The “pilot flies the start” curve is drawn reaching zero at 30%, our simplest reading of “fades out”; with the trust gate on, it keeps a small permanent share.

two versions — every task is trained both ways

a = \pi_\theta(o)

The AI learns with the pilot's help, but when it flies, it flies on its own. The formula just says: the action is whatever the AI (π_θ) decides from what it observes (o).

The only version that can do better than the pilot where the pilot is weak.

12 — The trust gate

Follow the pilot where it's right — and only there

The gentle pull toward the pilot is the same strength everywhere. But the pilot isn't equally good everywhere: the formation pilot is excellent at holding still and clumsy when things move fast; the chase pilot may be strong on one side and weak on the other. The optional trust gate decides, moment by moment, how much to follow it.

the trust gate

w_i(s) = \sigma\!\left(\kappa\,\frac{Q_i(s, a_T) - Q_i(s, a)}{\hat\sigma_\Delta}\right)

\beta_i(s,t) = \beta_0\big[f(t) + (1 - f(t))\,w_i(s)\big]

in words

Trust (w) is a smooth 0-to-1 switch (σ) on how much better the pilot's move scores than the AI's. The pull toward the pilot (β) is then full strength where trust is 1 and zero where trust is 0 — blended with a fixed schedule (f) early on, while the coach is still learning.

The coach learns one more skill: scoring a specific move in a specific moment ( $Q(s, a)$ — “how good is move a in situation s?”). It practises this on the AI's moves and on the pilot's moves from the rounds the pilot flew, so it knows both.

Then, at every moment, it compares: the pilot's move ( $a_T$ ) versus the AI's actual move ( $a$ ).

one moment — drag how much better the pilot's move looks

pilot's edge0.60coach still learning0.25

trust in pilot

0.769

pull strength

0.826

The coach isn't sure — the pull sits somewhere in between.

Early on, the coach hasn't learned to score moves yet, so a fixed schedule does the pulling (the second slider). As the coach improves, the slider falls to 0 and the gate takes over completely.

every moment at once — how the coach sorts them

training updates10

AI trusted

18%

average trust

0.49

pilot trusted

22%

In a quick test run, within ten training updates the coach was already confident in about one moment in five each way — trusting the pilot in some, the AI in others. These three numbers are exactly what the paper will report when it tests the gate on the formation and chase games.

The bars are illustrative: tuned so that ten updates matches the paper's one-in-five figure. The paper doesn't publish the full distribution.

For the corrections version

The same trust decides how much the AI may correct the pilot in each moment: small corrections where the pilot is trusted, large ones where it isn't.

Keeping the coach honest

The pilot keeps flying the start of a few rounds forever, so the coach never forgets what the pilot's moves are worth.

What it costs

One extra calculation by the coach per batch of experience — cheap.

13 — The full iteration

The whole training loop, one step at a time

Here's how all the pieces fit into one repeating cycle — the paper's 'Algorithm 1'. Three things worth noticing: the hand-written pilot helps in several places but is gone when the AI flies; two separate practice schedules run at once (who plays, and how hard); and the AI is always judged on the real, full-difficulty task, on each side of the game.

what's active in this step

step 0 · Once, at the start

Copy the pilot: the hand-written pilot flies 200,000 steps, and the AI learns to reproduce its moves. It also learns the typical range of every input, so numbers arrive on a sensible scale.

the loop — click any step (the grey text is the paper's shorthand)

steps 1–7 repeat, thousands of times

the training settings

Training methodMAPPO

the team version of PPO, a standard trial-and-error learner

Practice per update64 × 128 steps

64 games played side by side, 128 steps each

Study passes5 × 32

each batch of experience is studied 5 times, in 32 chunks

Future vs. nowγ = 0.99

how much future points count (λ = 0.95 smooths the judging of each move)

Randomness bonus0.005

a small reward for keeping choices varied; zero for the corrections version

Learning speed3 × 10⁻⁴

how big each adjustment is; shrinks to 10% of that by the end

Safety brakesclip 10 · stop at 0.02

caps on how much one update can change the AI

RescalingPopArt-style

inputs, points and predictions kept on steady scales

Randomness limitslog σ ∈ [−2.5, 0]

how random actions may be; [−2.5, −1] for the corrections version

Practice per run3 × 10⁷ steps

30 million drone decisions — parked slots don't count

Repeats3 seeds

every setting trained three times from different random starts

Brain size128 wide · 4 heads · 2 layers

the size of the transformer inside CAST

Correction sizeα = 0.25

corrections are scaled down to a quarter

Every training run uses the realistic drones, the safety guard, fresh obstacles and changing line-ups — nothing is switched off to make learning easier. A small planner checks how much computer is free and decides how many runs to launch at once. Because a roster slot can change team between rounds, every sample is sent to its team's model, input rescaling and coach for that round, and each parallel game draws from its own random seed.

14 — An honest signal

Three rules that stop the AI from cheating

Given any loophole, an AI will find it — it's only trying to maximise points, not to do what you meant. These three rules close the three most common loopholes: dying for profit, farming bonus points, and hiding a weak side behind an average.

Rule 1

Leaving the game must never look like a good deal

When a drone crashes, is tagged or lands but the round carries on, the standard maths keeps estimating that drone's future — from a state the coach (critic) was never trained on: the blank placeholder of an empty slot, or a runner sitting on its pad. That value drifts toward 'average', so leaving early looks like a free average score. AIs then learn to crash.

\delta^i_t = r^i_t + \gamma\, V(s^i_{t+1})\,(1-d_t)\,e^i_{t+1} - V(s^i_t)

in words

How much better a moment went than expected (δ) = the points just earned + the coach's guess of the future ( $V$ , shrunk slightly by γ) − what the coach expected. The fix is the $e^i_{t+1}$ term: a switch that is 1 while the drone is still in its round and 0 once it has crashed, been tagged or landed — so after that, no imaginary future is added. (This is how the AI's advantage is computed.)

A drone that is stunned in capture-the-flag hasn't left: it can't act, but its round goes on. Its stunned moments stay in the count with zero points and teach the coach, but the AI is not trained on actions it couldn't take.

a drone crashes at moment 3 — the round carries on

guessed future0.66

worth of crashing

0.66

worth of staying alive

0.51

Crashing looks better than living. Training will push the AI toward crashing.

The slider stands for a meaningless guess; its starting value and the 0.51 are illustrative — the paper describes the problem, not a measured pair of numbers.

Rule 2

Winning must clearly pay more than losing — and nothing can be farmed

Small hint points that each look sensible can add up to more than the actual goal. You can't see this by reading the scoring rules; you have to measure it.

Play a strategy that wins and one that deliberately loses, and make sure the winner earns clearly more — by more than the normal round-to-round noise.

Hints are only given for progress toward targets that stay put, and switch off when a target jumps (a respawn, a switch of target, a change of formation shape); bonuses one team could trade back and forth are zero-sum; a drone that leaves early is paid what it gives up; and a crash is charged the rest of the round's points — so neither leaving nor crashing ever beats staying.

test the scoring by measuring it

hand-written pilot (wins)13.7

camp on one zone (loses)2.7

points per zone held0.30

winner earns 11.0 more; normal round-to-round wobble is ±3.2 — clear enough — winning obviously pays.

This is how the paper arrived at 0.3 points per zone — exactly how fast the real score moves. The bar values here are illustrative; the 0.3 and the method are the paper's.

Rule 3

Judge the AI by its weakest side

In a two-sided game, averaging the win rates of both sides gives the same score to an AI that's good on both sides and to one that's perfect on one side and hopeless on the other.

\mathrm{score} = \min_{\text{role}} \mathrm{win}_{\text{role}} + \frac{0.9}{n}\,\tanh(2\bar m)

in words

Take the lower of the two sides' win rates, then add a tiny bonus for winning by bigger margins: the margin m̄ is the average score difference as a share of the total (−1 to 1), and the bonus is at most 0.9 ÷ n for n test games — less than one game's worth, so it only separates AIs whose weaker side ties.

Checked every 40 training updates, on 16 repeatable games at full difficulty and a standard team line-up. The best saved AI is the one with the best average over three such checks.

win rate on each side, against the hand-written pilot

side A0.00side B1.00points margin0.00

simple average

0.50

looks fine — is hopeless on one side

the paper's score

0.00

weaker side + a tie-breaker under 1/16

An AI winning 87.5% on one side and 12.5% on the other, and one winning 0% and 100%, both average 0.50. The margin term is worth less than one test game, so it only orders AIs with the same weaker-side win rate — it can't rescue a side that never wins.

15 — The second curriculum

Start the chase easy, then make it harder

A brand-new AI chaser, flailing around a 60 m arena trying to get within 1.5 m of a runner, basically never succeeds — and with no successes, it has nothing to learn from. So the chase game has a difficulty dial. It starts easy and turns up only as the chasers improve, like training wheels coming off.

turn the difficulty dial

0 is easiest, 2 is the real game. Tests always use 2, so every reported score is on the real game.

difficulty1.30

d ∈ [0, 1]

The runners go from half speed to full speed

A slow runner can actually be caught, so the chasers get some wins to learn from at all.

d ∈ [1, 2]

The tagging distance shrinks from 4 m to 1.5 m

Speed is raised first: a generous tagging distance keeps wins possible while the chasers are still learning to close in.

what advances it

The dial goes up by 0.1 whenever the chasers have won more than 70% of their last 40 rounds. It never goes back down.

▬ chasers' win rate — this decides┅ overall win rate — always about ½▬ difficulty (scaled)

difficulty

2.0

chasers win

0.67

steps up

20 / 20

Why not just use the overall win rate? When the AI plays against itself, one copy always wins and one loses, so the overall rate sits at about 50% no matter what. It says nothing about whether the chasers are getting better. So the dial watches the struggling side — the chasers — directly.

Illustrative curve: the rule (step 0.1, 70%, last 40 rounds, never down) is the paper's; the shape of the learning is made up to show it.

16 — The suite

The five games

Two cooperative games (all drones on one side) and three competitive ones (team against team). All five use the same simulator, the same sensors and the same kinds of scoring, work with any team line-up, and report a simple win/lose at the end of each round for the practice scheduler. In the team games, the hand-written pilots see only what their radio group's tracks and the referee's broadcasts show — exactly like the AI. Pick a game to watch it.

A simplified animation of the game's layout and rules.

Formation

standard game: 6 drones, 40 s

Fly as a shape and hold it. Each drone is assigned a spot in the shape, and practice goes in four stages: hold still, move the whole shape, spin it, and change it into a different shape. Earlier stages keep being practised in 40% of rounds. Spots are re-assigned at every change of shape, and a spot that would fall inside an obstacle is moved to just outside it (1.5 m).

how it's scored — and why it can't be cheated

Scored by the average distance from each drone to its spot over the second half of the round. A crashed drone counts as at least 10 m off, and a crash also costs the rest of the round's points — so a drone that's badly out of place is never better off crashing.

the hand-written pilot for this game

A classic steering rule (PID) that flies each drone to its spot and away from obstacles.

17 — Evaluation protocol

How results will be judged — decided before any results exist

The paper fixes, in advance, which questions it's asking, how the best AI is chosen, how scores are reported, and how cheating is checked for. Deciding this up front means no one can pick flattering numbers after seeing them.

Q1Even across teams?

Does one AI trained with the practice scheduler do equally well on every team line-up — including ones it rarely practised?

Q2Beyond practice?

Trained with 4 to 8 drones, does it still work with 10?

Q3Different drones?

For fast chasers against nimble runners, is 'one brain told each drone's limits' as good as separate brains per team?

Q4Which version wins where?

Where does each version — flying alone, or adding corrections — beat the hand-written pilot, and on which side of each game?

Q5Does the trust gate help?

Does it keep the pilot's precision where the pilot is good, without holding the AI back where it has found something better?

Q6Does easy-first work?

Does starting the chase easy make the chaser role learnable at all?

Q7What does each piece add?

What happens without the pilot, without the practice scheduler, without the speed/agility input, without randomness?

choosing the best saved AI

During training, a snapshot of the AI (a checkpoint) is tested every 40 updates on 16 repeatable games, at full difficulty and the standard line-up. The one kept (“best.pt”) has the best average over three tests in a row, so one lucky test can't win it.

Formationsmallest average distance from the spots

Stages it hasn't reached yet count as 10 m off — so an AI that has learned all four stages always beats one that's slightly more precise on fewer.

Coveragemost points kept watched

Team vs. teamwin rate on its weaker side + (0.9 ÷ n)·tanh(2m̄)

Against the hand-written pilot. m̄ is the average score margin as a share of the total, n the number of test games — so the tie-breaker is worth less than one game and can't hide a side that loses.

how final scores are reported

The kept AI plays 20 fresh games — with random settings it has never seen in training or testing.
Everything is repeated for three separate training runs (seeds) and reported as average ± spread (mean ± std).
Team-vs-team results are reported for each side separately, never just averaged.
Every training run is logged in detail — how learning progressed, how the drones behaved each round, the state of the practice schedules — so anyone can inspect it.

the cheating check

Before any score counts, the AI is checked for loophole-exploiting behaviour — the tell-tale signs below. The same signs are tracked during training too, so cheating shows up while it's emerging, not after.

Hugging the ceiling or floor

a classic place to hide from rules

Bumping into things

walls, obstacles, teammates, opponents

Crashes

how often, per round

Maxed-out commands

how often the AI slams a control to its limit

Just copying the pilot?

how closely the AI's output tracks the hand-written pilot's

Points vs. real progress

do high-scoring rounds actually do the task? If points and task performance drift apart, the AI is earning from hints, not the goal

Pursuit

how high runners are when they land; loitering late in the round

Capture-the-flag

how many pickups become captures; how high pickups happen

Zone control

how high drones sit in zones; camping on one zone

Coverage

how long between visits to each point

Formation

how many drones died

The sharpest sign: do high-scoring rounds actually do the task well? If the AI is earning points from loopholes or hints rather than the real goal, the points and the real performance stop moving together.

the implementation audit

A behaviour check can't catch a learner that is wrong in ways that still train. So before the reported runs, the observation pipeline, the learner and the simulator were reviewed line by line against their specification. Every confirmed defect was fixed together with a test that reproduces it, and the test suite now runs on every change. Among the defects found:

The drone slots were hidden

a presence mask was read from the wrong feature, so nearly every visible drone was hidden from the network

The coach saw two different pictures

it was evaluated on different inputs while playing and while learning

Samples went to the wrong team

under changing line-ups, some experience was routed to the other team's model

A chaser input ignored obstacles

one task input saw runners through pillars

Tracks could explode

velocity estimates blew up after a runner was re-acquired

The reference values in the next section were measured after this audit, with every hand-written pilot acting on its own group's perception and through the safety guard.

planned “remove one piece” experiments (ablations) — three runs each

No hand-written pilot

train without any of the pilot's four kinds of help — on formation and capture-the-flag

No speed/agility input

one shared brain that isn't told each drone's limits — on the chase game

No practice scheduler

pick line-ups at random instead — on capture-the-flag

No randomness

formation without random action noise or obstacle jitter

To test whether one AI really works across line-ups, each team game gets a grid: trained on line-up X, tested on line-up Y — for the scheduler-trained AI and for one trained on a single fixed line-up. A separate test trains capture-the-flag with 4–8 drones and plays 5 v 5, 4 v 6 and 3 v 7. The trust gate is compared against the fixed pull on formation and the chase game.

18 — Reference values

The scores the AI has to beat

The AI training runs — five games, both versions, three repeats each, 30 million decisions per run — are still going. So in this draft, every AI score is marked 'pending'. What's final are the scores of the hand-written pilot and of random flying, measured in exactly the same simulator the AI trains and is tested in. Those are the bars to clear.

Status. Every number on this page is one the paper reports. Where the paper says “pending”, so does this page — no AI results are shown because none have been reported yet.

hand-written pilot: how far off its formation spots, by stage

Lower is better. This is the bar the AI has to beat.

In the orange stages the spots move. The pilot steers toward where each spot is now rather than where it's going, so it lags behind — exactly where a learning AI has room to do better. The thin lines show the typical spread over 20 rounds.

the pilot against itself

Two games are close to even. The chase isn't: working from imperfect tracks, the chasers lose most games.

Pursuit

chasers 0.35

runners 0.65

Capture-the-flag

red 0.55

blue 0.45

Zone control

red 0.50

blue 0.50

20 rounds each at the standard line-ups, draws counting half, with every pilot acting on its own group's perception. In the chase, 62% of runners reach a pad: a runner that ducks behind a pillar is lost unless another chaser sees it, and a chaser that sees nobody can only guard the pads. To beat the pilot, an AI has to win more than half its games on both sides.

Reading the tables: “Random” is a drone pressing random controls — the floor. “Scripted” is the hand-written pilot — the bar to beat. “CGPO guided / residual” are the two AI versions (flies alone / adds corrections), and “+ trust gate” is with the optional trust gate. ↓ means lower is better, ↑ higher is better; “0.50±0.14” means about 0.50, typically within 0.14 either way.

Table 1 — The two one-team games

Formation: average distance from each drone to its spot, in metres — lower is better — for each practice stage, with 6 drones. Coverage: share of points kept watched (higher is better) and how stale points get, in seconds (lower is better), with 8 drones. Reference rows are averaged over 20 rounds on test seeds kept apart from training, measured after the implementation audit; AI rows will be averaged over three training runs.

Method	static (m) ↓	translate (m) ↓	orbit (m) ↓	morph (m) ↓	watched ↑	stale (s) ↓
Random	32.1±3.9	34.9±5.1	34.0±4.8	33.0±4.4	0.22±0.06	13.9±2.8
Scripted	0.65±0.18	1.47±0.62	1.05±0.31	1.64±0.48	0.90±0.04	1.3±0.6
CGPO guided	pending	pending	pending	pending	pending	pending
CGPO residual	pending	pending	pending	pending	pending	pending
CGPO + trust gate	pending	pending	pending	pending	—	—

Table 2 — The three team-vs-team games, per side

Win rate on each side of each game against the hand-written pilot, at the standard line-ups: pursuit 3 v 3, capture-the-flag and zone control 5 v 5. The 'scripted vs. scripted' row is the pilot playing itself, with draws counting half.

	Pursuit		Capture-the-flag		Zone control
Method	pursuers	evaders	red	blue	red	blue
Scripted vs. scripted	0.35	0.65	0.55	0.45	0.50	0.50
CGPO guided	pending	pending	pending	pending	pending	pending
CGPO residual	pending	pending	pending	pending	pending	pending
CGPO + trust gate	pending	pending	—	—	—	—

Where the pilot is strong

Even with gusting wind, the safety guard and new obstacles every round, the hand-written pilot holds a still six-drone shape to within 0.65 m, and a spinning one to 1.05 m.

Where it is weak

When the shape moves (1.47 m) or changes (1.64 m), it falls behind. A rule that chases where the spot is now, without anticipating where it's going, always lags a moving target.

Imperfect sight hits the chase hardest

Working from tracks, the hand-written chasers win only 0.35 of games and 62% of runners reach a pad. A runner that slips behind a pillar is lost unless another chaser sees it, and a chaser that sees no runner can only guard the pads.

The other two games are close to fair

Capture-the-flag (red wins 0.55) and zone control (0.50) are near even. So an AI that wins more than half its games on both sides has genuinely beaten the pilot, not exploited a lopsided game.

The safety guard holds

No hand-written pilot ever crashed or touched a wall.

What the paper itself says are the limits

Size

Everything runs on one computer with one graphics card. The AI is tested on the team sizes it practised plus one step bigger; much bigger swarms would need a bigger design.

The drone model is simplified

The drone's lean is modelled simply rather than as a full spinning body, and the extra lift near the ground is left out. Moving to real hardware would mean re-measuring the delays, drag and downdraft, and re-tuning the safety guard.

Perception is simplified too

Drones sense in every direction at once (they don't choose where to look, and a 360° ring of sensors would justify that), and sightings are never missed or false. The tracking system is designed to absorb those errors if they're added. The referee's announcements are assumed to be reliable.

The corrections version inherits the pilot's flaws

Adding corrections on top of the pilot keeps the pilot's weaknesses as faithfully as its strengths.

The benchmark is fixed, not perfect

The hand-written pilots are strong but not the best possible, so beating them shows progress against a fixed bar — not the best play there could be.

The cheating check only finds what it looks for

It can only rule out the kinds of cheating it knows how to spot.

AI results aren't in yet

The trained-AI results are still pending in this draft. The hand-written and random reference numbers are final.

Glossary

Every term on this page, in plain words

The same definitions you get by hovering an underlined word, all in one place.

82 terms

Ablation

Games & evaluation

An experiment that removes one piece of the system to measure how much that piece mattered.

Action

AI & learning

What a drone decides to do at a moment. Here: which direction to accelerate and how hard — three numbers.

Actor

AI & learning

The policy network — the one that actually picks actions. It's what gets deployed on the drones.

Advantage (GAE)

AI & learning

How much better an action turned out than the critic expected. Positive advantage → do that more; negative → do it less. GAE is a standard way to compute it.

Agent

AI & learning

One decision-maker. Here, each drone is an agent.

Anchor

This project

A penalty for straying too far from what the hand-written pilot would do. It keeps early learning sensible.

Battery sag

Drones & physics

As the battery drains, its voltage drops and the motors can push less hard. Here the drone loses up to 20% of its thrust over a flight.

Bootstrapping

AI & learning

Instead of waiting until the end of the round to see how things went, the AI uses the critic's prediction of the future as a stand-in. Fast — but only as good as the prediction.

CASH

This project

Lets one brain fly different kinds of drone. Each drone tells the network its top speed and acceleration, and the network adjusts its output to suit.

CAST

This project

The paper's AI design: one network that reads a list of 'things it can see', however long, with no fixed seat for drone #1, #2, … — so it works for any team size.

CBF-RL

Drones & physics

Prior research showing that training with a safety filter switched on (rather than adding it only afterwards) lets the AI learn to work with it.

Centralised training, decentralised execution (CTDE)

AI & learning

During practice, a coach (the critic) can see the whole field. During the real game, each drone decides using only its own view.

CGPO

This project

The paper's way of training with the hand-written pilot's help: copy it first, get pulled gently toward it, let it fly the start of some rounds, and keep it as a sparring partner.

Checkpoint

AI & learning

A saved snapshot of the AI at some moment during training — like saving a game. 'best.pt' is the snapshot the paper keeps as its best.

Composition / roster

This project

How many drones are on each team, e.g. '4 v 6'. The paper's central idea is to train on every composition, not just one.

Critic (value function)

AI & learning

A second network used only during training. It predicts 'how well is this going to go from here?' so each action can be judged as better or worse than expected.

Curriculum

Games & evaluation

An order for practice, like a school syllabus: what to practise and when, so learning is efficient.

Difficulty curriculum

This project

Starting a task easy and making it harder as the AI improves — like training wheels that come off gradually.

Digital twin / simulator

This project

A detailed computer model of the real world, where the AI can practise millions of times safely and cheaply before going near real hardware.

Discount factor (γ)

AI & learning

How much future rewards count compared to immediate ones. γ = 0.99 means a reward one step later is worth 99% of the same reward now — the AI cares about the future, but slightly prefers sooner.

Downwash

Drones & physics

The column of air a drone's rotors blow downward. Fly underneath another drone and you get pushed down, so you have to work harder to stay up.

Drag

Drones & physics

Air resistance. The faster you fly, the harder the air pushes back.

Elo rating

Games & evaluation

The rating system from chess: win against strong opponents and your rating rises a lot, lose to weak ones and it falls.

Embedding

AI & learning

Turning something (like 'this is a flag' or 'this is team A') into a list of numbers the network can work with.

Entropy

AI & learning

How random the AI's choices are. An 'entropy bonus' rewards keeping some randomness so the AI keeps exploring instead of settling too early.

Episode

AI & learning

One round of the game, from the starting whistle to the end. Training plays millions of these.

Exploration

AI & learning

Trying actions the AI isn't sure about, to discover better ones. Without it, the AI gets stuck doing the first thing that sort of worked.

Flight stack / autopilot

Drones & physics

The low-level software on a real drone that keeps it stable and turns simple commands ('accelerate forward') into motor speeds.

Drones & physics

The acceleration of gravity, about 9.81 m/s². '2.2 g' of thrust means the rotors can push 2.2 times harder than needed to hover.

Gaussian (bell curve)

AI & learning

The familiar bell-shaped curve. The policy picks each action as a best guess plus a little bell-curve randomness, so it keeps trying slight variations.

Hungarian assignment

Games & evaluation

A classic algorithm for pairing things up optimally — e.g. which drone should fill which spot in a formation so total travel is smallest.

Hypernetwork

AI & learning

A small network whose job is to produce the settings of another network. Here it adjusts the policy's final step depending on how fast and nimble a particular drone is.

Imitation / behaviour cloning

AI & learning

Learning by copying an expert's actions — here, copying the hand-written pilot — before or while learning from rewards.

Lag

Drones & physics

The delay between a command and the drone actually responding — it takes time to tilt and to spin rotors up or down.

Latency

Drones & physics

A fixed communication delay: a new command reaches the drone's internal controller one tiny step late.

League

Games & evaluation

A pool of opponents to practise against: saved past versions of the AI plus the hand-written pilot.

Line of sight

Drones & physics

A clear straight line between two drones. If an obstacle is in the way, they can't see each other.

Markov

AI & learning

A situation is 'Markov' if knowing the current state is enough to predict what happens next — you don't need the history. The full simulator state is Markov; one drone's view is not.

Mask

AI & learning

An on/off switch on each slot in the list. Empty or unseen slots are switched off so the network ignores them completely.

Mean ± std

Games & evaluation

The average result, plus how much it varies. '0.50 ± 0.14' means about 0.50 on average, typically within 0.14 either side.

Multi-agent RL (MARL)

AI & learning

Reinforcement learning with several AIs acting at once, where each one's choices change what the others experience — like a team sport instead of a solo game.

Neural network

AI & learning

A very large adjustable formula made of millions of numbers. Training tunes those numbers until the formula turns inputs (what a drone senses) into good outputs (what to do).

Normalisation

AI & learning

Rescaling numbers to a standard range (say, around −1 to 1) so no single input dominates and learning stays stable.

Observation

AI & learning

What one drone can perceive at a single moment — its own sensors plus what teammates radio in. Never the whole picture.

Parked slot

This project

The system always has room for 10 drones. When fewer are playing, the unused ones are 'parked' — invisible and ignored.

Partially observable

AI & learning

Each player sees only part of the game. Like playing cards: you know your hand, not everyone's.

PCR (Prioritized Composition Replay)

This project

The practice scheduler. It picks which team line-up to play next, favouring the ones the AI wins about half the time — where there's most to learn.

PFSP

Games & evaluation

'Prioritised fictitious self-play' — pick opponents you usually lose to more often, so practice focuses on weaknesses.

PID controller

Drones & physics

A classic, hand-tuned steering rule: correct in proportion to how far off you are, how long you've been off, and how fast the error is changing. Found in thermostats, cruise control and drones.

Point mass

Drones & physics

An idealised 'dot' that moves exactly as commanded — no tilting, no delay, no limits. Much easier to simulate, and unrealistic.

Policy

AI & learning

The AI's decision-making program: it takes in what a drone senses and outputs what the drone should do. Training means improving the policy.

PPO / MAPPO

AI & learning

A widely used training method. After each batch of play, it nudges the policy toward the actions that turned out better than expected — in small, careful steps so learning doesn't go off the rails. MAPPO is the version for teams.

Privileged critic

AI & learning

A coach that, during practice only, is allowed to see information the players can't — here, every drone's true position. It never flies, so the extra knowledge costs nothing at deployment.

Quadrotor / multirotor

Drones & physics

A drone with four (or more) spinning rotors. It can't push sideways directly — it tilts, and part of the rotors' lift then points sideways.

Referee

This project

The game's official announcer. Drones learn of game events — tags, landings, flag pickups, who holds each zone — only from its announcements and once-a-second broadcasts, never by magic.

Reinforcement learning (RL)

AI & learning

Learning by trial and error. The AI tries something, gets a score, and gradually does more of what scored well — the way you'd train a dog with treats.

Residual (bounded)

This project

Instead of flying alone, the AI adds small corrections on top of the hand-written pilot's commands — like a driving instructor nudging the wheel.

Return

AI & learning

The total reward collected over time, from now until the end of the round.

Reward

AI & learning

The score the AI receives after each moment — positive for good things (holding a zone), negative for bad (bumping into something). The AI's only goal is to collect as much as possible.

Reward shaping (potential-based)

AI & learning

Extra hint points that guide learning — e.g. a little reward for getting closer to a goal. The 'potential-based' form is a mathematical recipe that guarantees hints can't change what the best strategy is.

Role

Games & evaluation

Which side you play in a lopsided game — pursuer or evader, red or blue. A good policy must be good in both roles.

Safety shield

Drones & physics

A guard between the AI and the drone that removes any part of a command that would fly it into something it couldn't stop in time.

Scripted controller

This project

A hand-written pilot made of rules engineers wrote — no learning involved. It's decent at each task, and serves as teacher, sparring partner and the benchmark to beat.

Seed

Games & evaluation

A starting number that fixes all the randomness in a run. Same seed → exactly the same run, every time. Using three seeds checks results aren't a fluke.

Self-play

Games & evaluation

The AI practises by playing against copies of itself. It's how AlphaGo got good at Go.

Specification gaming (reward hacking)

This project

When an AI finds a loophole that scores points without doing what you actually wanted. A classic example: a boat-racing AI that circles forever collecting bonus items instead of finishing the race.

Specification-gaming audit

This project

A check, run before any result is reported, that looks for signs the AI is cheating — exploiting a loophole rather than doing the task.

Split

This project

The share of drones on the first team. 3 v 7 has a split of 0.3; 5 v 5 has 0.5.

State

AI & learning

The complete truth about the world at one moment — where every drone is, the wind, the obstacles, the score. Only the simulator knows it all.

Thrust

Drones & physics

The push from the rotors, along the drone's 'up' direction. Tilt the drone and the push tilts with it.

Tilt / attitude

Drones & physics

How far the drone leans. Leaning more turns more of the lift into sideways push — up to a safe limit (40° here).

Token

AI & learning

One item in the list the network reads — a small bundle of numbers describing one thing: the drone itself, another drone, or an object like a flag.

Track

This project

A drone's running best guess of where another drone is and how it's moving, built from sightings. Refreshed when the other drone is seen; if it isn't seen for 3 s, the guess is dropped.

Track gate

Drones & physics

A sanity check on each new sighting: if it lands too far from where the track expected (more than 3 noise widths plus twice the target's top speed times the time since), the track is restarted instead of being dragged across the arena.

Training

AI & learning

The long practice phase where the AI plays the game over and over and its policy is adjusted after each batch of experience.

Transformer / attention

AI & learning

A kind of neural network that reads a list of items of any length and learns which items to pay attention to. It's the same basic design behind modern chatbots.

Trust gate

This project

A switch, set moment by moment, that decides how strongly to pull the AI toward the hand-written pilot — strongly where the pilot is good, not at all where the AI has found something better.

Weights / parameters

AI & learning

The numbers inside a neural network. 'One parameter set' means one single trained brain — the same numbers used for every drone, every team size, every game.

Win rate

Games & evaluation

The fraction of games won. 0.55 means 55 wins out of 100.

Wind gusts (Dryden)

Drones & physics

Random changes in wind. 'Dryden' is a standard engineering model of realistic turbulence.

Zero-sum

Games & evaluation

Whatever one team gains, the other loses — so the two can't both profit by trading points back and forth.

α-β tracker

Drones & physics

A simple, classic way to keep a track. Each step it predicts where the target has moved; when a sighting arrives it moves the guess part of the way toward it (α) and corrects its speed estimate a little (β).