The Role of the Security Expert During the Cognitive Revolution

Adapted from my BSides Prague keynote, April 2026.

It is arguable whether we're living through a "cognitive revolution," but the seismic shifts are real. I still remember the feeling I got the first time I opened ChatGPT three years ago and talked with a machine — mind-blowing. Now it feels completely normal. The unsettling part is that these moments keep coming. Over the last six months multiple developments have re-drawn the map of what a security expert is supposed to be good at.

Code is cheap now. What does that mean for the security industry?

In this article, I'll discuss three enduring skills for the next era of cybersecurity practice — specifically, eval-driven development and context engineering.

Knowledge has a short shelf life

Do you know what nmap -p- -T5 does? Or ssh -R 8080:127.1:80? Congratulations. That knowledge no longer gives you an edge. A single English sentence to an agent ("scan all ports as fast as possible," "tunnel my local webserver, I have an SSH key") produces the same result, and the agent gets the flags right more often than I do at 2 a.m.

Skills that survive

Given how quickly AI is improving, what do we hire humans for? Three things keep coming up in interviews and onboarding.

Skill #1: Tech-lead instincts

Karpathy's recent autoresearch experiment, in which an agent ran for days against a verifiable objective, produced architectural improvements he says he would not have come up with himself. As he puts it, "You are the bottleneck." The skill that maps onto this regime is the one good tech leads always had:

• Trust and delegate to humans and to agents. Do not micromanage the implementation; verify the result.
• Broad knowledge, deep enough to think fluently. You do not need every ssh flag in working memory, but you need to know ssh tunneling exists, what shape of problem it solves, and where its limits are. Otherwise it is not in your action space.
• Knowing when to go deep and when to stay high-level. Most of the time, point estimates and three-minute summaries are fine. When something looks wrong or matters strategically, you should be able to drop down to technical nuance.

This is why fundamentals still matter even though surface knowledge is ephemeral. Nmap role. SSH can tunnel. File uploads can lead to RCE. ASLR-disabled stack overwrite is a thing. Npm preinstall scripts run on install. We do not type any of this, but without these primitives, we cannot think fluently, and we cannot recognize when an agent is bullshitting us.

Skill #2: Taste

Taste is the answer to "what?" rather than "how?". When implementations are cheap, leverage moves upstream to questions about which problems are worth solving and which approaches will actually work. People with taste pick problems that matter and approaches that succeed. People without it spend months on dead ends, or on technically impressive work nobody needs.

A good-taste example I will claim. In October 2025 I gave a talk at BSides NYC arguing that code automation + language automation = supply-chain risk, and that this attack surface was about to expand fast.

Since then we have seen the rise of Shai-Hulud (twice), team PCP campaigns, and attacks on packages with hundreds of millions of monthly downloads. At AISLE, on the back of that bet, we built and are now rolling out a supply-chain module that watches dependency posture for our customers in real time.

A bad-taste example to balance the ledger. Back in 2023 I bet hard on small specialized language models for security tasks. I even wrote a paper (Nebula, transformer-based dynamic malware analysis), trained the models, and released the code. The argument was the same one Clem Delangue made publicly, that 2024 would be the year specialized small models take over 99% of use cases.

Three years later it has not happened. The big-model paradigm is still eating the world. The small language model bet might still be right on a longer timeline, but as a prediction about 2024-2026 it was wrong, and being honest and reflecting on that is part of how taste is trained.

Skill #3: Agency

Briefly, because it is the most obvious of the three, and yet the single biggest differentiator I see in hires, as we've grown to 40 technical staff at AISLE. Agency is the ability to make it happen despite three other projects, an inbox of pings, doctor's appointments, and a noisy life. It's a blend of time management, prioritization, and intrinsic motivation. This is not an experience-level thing. I see fresh grads with more agency than ten-year veterans, and it shows up in output within weeks.

How this maps to cybersecurity

Eval-driven development

Test-driven development was vital in the 2010s. Eval-driven development is the update for AI-heavy systems, and security work is full of it.

Take a concrete example: zero-day discovery via fuzzing. The classical pipeline is: target codebase → human security researcher writes a fuzz harness, picks seeds, sets up the build → libFuzzer or AFL++ runs for days → bugs (hopefully). Plug an agent into the harness-writing step and you have an immediately useful application. But: how do we know whether prompt v1.1 or v1.2 actually produces a better harness? The whole craft of AI engineering hinges on this question, and the answer is the same everywhere: build a benchmark.

The basic structure iterates:

1. Define inputs and outputs for every AI-native component in the pipeline.
2. Build a benchmark that scores it.
3. Use the benchmark to move quickly across many cases at once and dive deeply on the ones that matter.

We do not need a thousand cases. Sometimes ten or fifteen verifiable ones are enough to bootstrap. What we cannot do is judge a system on three anecdotes and a vibe.

Everything is a context problem

A thought experiment: The president of some small country decides whether to attack a neighbor. Intelligence walks in and dumps a stack of reports on the desk: economic indicators, troop movements, diplomatic cables, and satellite imagery. With perfect information (Laplace's demon) there is no decision to make; the answer falls out. With incomplete information, no amount of cleverness rescues you.

Agents are exactly the same. The "jagged frontier" of capability, which is sometimes superhuman and sometimes embarrassingly dumb, is real, but a surprising fraction of the dumbness disappears once we give the model the right context.

For security teams in 2026, this means context engineering is the new detection engineering. Where we used to tune Sigma rules, we now tune what gets pulled into the agent's working set, what gets summarized, what is retrieved on demand, and what is dropped on the floor. It's the same discipline of deciding what matters in this moment, but applied to different layers of the stack.

What will matter next year?

Honestly, I do not know. The pace is faster than my ability to absorb it. That being said, I think these two data points are worth sitting with.

METR's task-length metric, which measures the duration of human work AI can reliably complete, has been doubling roughly every seven months for several years. And the AI Security Institute's (AISI's) recent evaluations of frontier models show top systems now executing a significant portion of multi-step offensive tasks (lateral movement, credential extraction, web-app exploitation, browser credential theft, even rudimentary C2 reverse-engineering) end-to-end. (AISI says cyber capability has been doubling every 4.7 months since late 2024.) The "AI 2027" forecast from Kokotajlo et al., published a year ago, predicted "reliable agent" capability around April–May 2026. We are roughly there.

I will not pretend to forecast 2027–2028. I will say: the bet that the curve flattens has been a losing bet for the last five years, and I do not see the mechanism that flattens it now.

Closing

We are in a similar shape as a London mill worker in 1850 asked about the Industrial Revolution. From the inside, it does not feel like a revolution. It feels like the work my dad did stopped mattering, and I am trying to feed my family. That period also had Luddites who broke into factories to smash the machines, since machines took human jobs. They were wrong, but the impulse is understandable.

The deeper bet I would make is on the Jevons paradox: when something becomes cheaper, we use much more of it, not less. Cars got more fuel-efficient, and we drove more miles. Knowledge work is getting cheaper per unit, but the world does not have less knowledge work to do. It has staggeringly more. I am no longer running two projects at once. I am running five, and none of them are dropping.

Expert input will be as valuable as ever, just expressed at a different level of the stack — closer to taste, judgment, system design, and context curation, and further from typing the right nmap flag. The fundamentals still matter. The agency still matters. Reading and talking to people still matter. It's the output that looks different.

That is my take. Techno-optimistic. I think the coming years will be amazing, yet weirder than any of us can plan for.