← Back to Autonomy
Study Notes · AI Security Series · By Majid Mazouchi

How Hackers Weaponize Artificial Intelligence

Six ways attackers use AI today — explained in simple words, with real cases, defenses, ATT&CK mappings, and a checklist you can run with your team.

Based on: “AI ATTACKS! How Hackers Weaponize Artificial Intelligence”
IBM Technology · presented by Jeff Crume · ~19 min
Contents

AI is not inventing brand-new attacks. It makes old attacks faster, cheaper, more believable, and easier for low-skill attackers.

01

AI-Powered Login Attacks

In simple words: a robot that finds your login page and keeps guessing passwords — intelligently.

Guessing passwords (brute force) and trying leaked passwords on many sites (credential stuffing) used to require scripts, tuning, and some skill. Now an AI agent can look at a webpage the way a human does: find the username and password fields, understand how the form works, and run guessing campaigns on its own. Tools in this category (the talk references the BruteForceAI concept) use LLM reasoning to plan the attack instead of a human writing custom code for every site.

Picture it: an attacker doesn't study your portal at all. They point an agent at it and say “get in.” The agent reads the page, identifies the form, and starts working — patiently, around the clock.
Real-world case

Credential stuffing at scale. Attacks like the 2022–2023 wave against major retailers and streaming services reused billions of leaked username/password pairs from earlier breaches. Automation — not clever hacking — was the whole game: bots tried known passwords against millions of accounts, and the ones reused across sites fell. AI agents make the same approach cheaper to run and harder to rate-limit.

What AI adds

  • Finds and understands login pages automatically
  • Lowers the skill bar — no scripting knowledge needed
  • Runs at machine scale, 24/7, across many targets

How to defend

  • MFA everywhere; move toward passwordless / passkeys
  • Rate limiting, lockout policies, bot detection
  • Monitor for credential-stuffing patterns and anomalies
ATT&CK T1110 Brute Force T1110.004 Credential Stuffing T1078 Valid Accounts
Basic
MFA on email and admin/VPN accounts; account lockout after failed attempts.
Intermediate
MFA everywhere, bot detection / CAPTCHA on login, alerting on impossible-travel and credential-stuffing patterns.
Advanced
Phishing-resistant passkeys / FIDO2, continuous risk-based authentication, automated session revocation on anomaly.
Practical note

The cheapest big win is MFA on every external-facing system, especially admin portals and VPNs. If a password alone can get someone in, assume an agent will eventually guess or find that password.

If you suspect it — first 3 actions
  1. Force-reset affected credentials and revoke active sessions/tokens.
  2. Enable or enforce MFA on the targeted accounts immediately.
  3. Pull auth logs for source IPs, success-after-failure patterns, and lateral logins.
02

AI-Based Ransomware

In simple words: ransomware that thinks — choosing what to lock, how, and what to charge.

Classic ransomware was fixed code: encrypt files, drop a note. AI-assisted ransomware (the talk references the PromptLock proof-of-concept) can inspect your files, decide which ones are valuable, generate the encryption logic on the fly, and even write a personalized ransom note. Because AI can generate endless variations, the malware becomes polymorphic — slightly different every time, so signature-based antivirus has less to match against.

Picture it: instead of encrypting everything blindly, the malware reads enough to know your finance folder matters more than your wallpaper collection — and prices the ransom accordingly.
Real-world case

Colonial Pipeline (2021). A single compromised VPN password with no MFA let the DarkSide group in, forcing a shutdown of the largest U.S. fuel pipeline and a ~$4.4M ransom payment (partly later recovered). It shows how ransomware combines with attack #1 — and why backups plus identity hardening matter more than any single tool. AI lowers the bar to launch campaigns like this.

What AI adds

  • Smart targeting — classifies which files are sensitive
  • Generated, ever-changing code defeats signatures
  • Customized ransom demands and messages

How to defend

  • Immutable, tested, offline backups
  • EDR that watches behavior (mass file changes), not just signatures
  • Least privilege, network segmentation, allowlisting
ATT&CK T1486 Data Encrypted for Impact T1490 Inhibit System Recovery T1027 Obfuscated/Polymorphic
Basic
Regular backups stored separately; antivirus/EDR deployed on endpoints.
Intermediate
Immutable + offline backups with tested restores, network segmentation, least-privilege accounts, behavior-based EDR.
Advanced
Application allowlisting, automated isolation on mass-encryption detection, ransomware tabletop drills, honeypot file canaries.
Practical note

Backups only count if they're immutable and actually restorable. Run a restore drill: if you've never tested recovery, you don't have backups — you have hope.

If you suspect it — first 3 actions
  1. Isolate affected hosts from the network (don't power off — you may lose forensic state).
  2. Protect backups — verify they're offline/immutable and untouched before restoring.
  3. Engage IR + legal; do not pay or negotiate before assessing scope and obligations.
03

AI-Powered Phishing

In simple words: the badly written scam email is gone. Now it reads like your colleague wrote it.

For years, training said: look for spelling mistakes and awkward grammar. LLMs erase that signal completely. They write fluent, professional, polite emails in any language, in seconds. Worse, they can pull public information — LinkedIn profiles, company pages, press releases, social media — and personalize the message to you: your role, your projects, your boss's name. The old red flags are no longer reliable.

Picture it: an email referencing the real project you presented last week, in flawless language, signed with your manager's real name and title. Nothing about the writing looks wrong — because nothing about the writing is wrong.
Real-world case

Measured impact. Security researchers and vendors have repeatedly shown that AI-generated phishing matches or beats human-written lures on click-through, while taking a fraction of the time to produce. The practical effect: campaigns that once needed a skilled English-speaking writer can now be spun up in any language, instantly, by anyone.

What AI adds

  • Perfect grammar in any language — no more telltale errors
  • Personalization at scale from scraped public data
  • Thousands of unique, convincing emails per hour

How to defend

  • Train on behavior, not grammar: unusual requests, urgency, pressure
  • Verify out-of-band before sending money, credentials, or files
  • Email authentication (DMARC/DKIM/SPF) + secure email gateway
ATT&CK T1566 Phishing T1566.002 Spearphishing Link T1598 Phishing for Information
Basic
Spam filter + SPF/DKIM; periodic awareness reminders.
Intermediate
Enforced DMARC, secure email gateway, link rewriting/sandboxing, behavior-focused training and easy report-phish button.
Advanced
Continuous simulated phishing, AI-assisted anomaly detection on email/intent, out-of-band verification policy for sensitive requests.
Practical note

Update your mental checklist: don't ask “does this email look fake?” — ask “is this request normal, and did it arrive through the normal channel?” A perfect-looking email asking for something unusual is exactly the modern attack.

If you clicked / replied — first 3 actions
  1. Disconnect and report to security; don't try to clean it yourself.
  2. Change passwords for any credentials entered, from a clean device, and revoke sessions.
  3. Warn colleagues — the same lure is almost certainly hitting others.
04

Deepfake Fraud

In simple words: a fake voice or video of someone you trust, asking you to do something costly.

Generative AI can clone a voice from a short audio sample and produce convincing fake video. Attackers use this to impersonate executives, vendors, or even family members. The attack is psychological: our instinct says “I heard them, I saw them, so it's real.” That instinct is now unsafe.

Picture it: Friday, 4:55 PM. A video call from “the CFO”: urgent acquisition, confidential, wire the funds now. The face is right. The voice is right. Everything is wrong.
Real-world case

Arup, Hong Kong (2024). A finance employee joined a video call with what looked like the company CFO and several colleagues — all of them AI deepfakes — and was convinced to make transfers totaling roughly $25 million. It remains the clearest example of deepfake video defeating human trust at scale, and why money movement must never rely on voice or video alone.

What AI adds

  • Voice cloning from seconds of public audio
  • Real-time fake video calls of real people
  • Exploits human trust in seeing and hearing

How to defend

  • Multi-person approval for wire transfers — no exceptions
  • Callback verification on a known number, never the one provided
  • Code words / verification questions for sensitive requests
ATT&CK T1656 Impersonation T1585 Establish Accounts + FBI BEC typologies
Basic
Written policy: verify any payment/credential request by a second channel.
Intermediate
Dual approval + callback-on-known-number for all wire transfers; staff trained on deepfake tactics.
Advanced
Shared verification code words for executives, strict thresholds requiring in-person/PKI sign-off, deepfake-awareness drills.
Practical note

Make it policy, in writing: no financial action is ever taken based on voice or video alone. Urgency + secrecy + money is the deepfake signature. The defense is procedure, not technology.

If a transfer was made — first 3 actions
  1. Call your bank immediately to attempt a recall/freeze — minutes matter.
  2. Report to law enforcement (e.g., FBI IC3) — fast reporting improves recovery odds.
  3. Preserve evidence: the call invite, recordings, chat, and transfer records.
05

AI-Powered Exploit Generation

In simple words: AI shrinks the time between “a bug is announced” and “the bug is attacked.”

When a vulnerability (a CVE) is published, it describes a weakness. Turning that description into a working attack used to take real expertise and time — days or weeks. Now attackers can feed the public advisory, affected code, and documentation into AI systems and get help producing attack logic much faster (the talk references research like CVE Genie on automating this). AI doesn't magically invent exploits, but it compresses the timeline — and your patch window shrinks with it.

Picture it: a critical CVE drops Monday morning. By Monday afternoon, automated attempts are already hitting unpatched, internet-facing systems. “We'll patch next sprint” is no longer a plan.
Real-world case

MOVEit Transfer (2023). The Cl0p group exploited a single vulnerability in a widely used file-transfer tool to breach 2,000+ organizations and affect tens of millions of people — exploitation began almost as fast as the flaw became known. AI-accelerated exploit development pushes more vulnerabilities toward this “patch in days, not weeks” reality.

What AI adds

  • Reads and explains vulnerability reports instantly
  • Speeds up proof-of-concept attack development
  • Less expertise needed to weaponize public CVEs

How to defend

  • Patch internet-facing systems fast — within days, not weeks
  • Virtual patching / WAF rules to buy time
  • Prioritize by real-world exploitability (e.g., CISA KEV), not CVSS alone
ATT&CK T1190 Exploit Public-Facing App T1203 Exploitation for Client Execution T1588.006 Vulnerabilities
Basic
Regular patch cycle; know which systems face the internet.
Intermediate
Asset inventory tracked against CISA KEV, emergency patch process for exposed assets, WAF in front of public apps.
Advanced
Continuous attack-surface management, virtual patching automation, exploit-prediction (EPSS) driven prioritization, threat-intel feeds.
Practical note

Keep an inventory of what's exposed to the internet and track it against CISA's Known Exploited Vulnerabilities (KEV) catalog. A medium-scored bug that's actively exploited matters more than a critical-scored one that isn't.

When a critical CVE drops — first 3 actions
  1. Check exposure: is the affected product internet-facing in your environment?
  2. Mitigate now: patch, or apply a WAF/virtual-patch rule, or take it offline.
  3. Hunt for prior exploitation — assume it may already have been hit.
06

Full Kill-Chain Automation

In simple words: not one AI tool — an AI agent running the entire attack, end to end.

This is the most serious idea in the talk. Everything above uses AI for one step. The real shift is an agent orchestrating all the steps: scan for targets, pick a strategy, craft the phishing, generate the exploit, move through the network, analyze stolen data, decide what's valuable, write the extortion demand, and adapt when blocked — with minimal human input. Cybercrime starts to look like an automated business process, running at machine speed against human-speed defenders.

Picture it: the attacker isn't typing commands at 2 AM anymore. They started an agent on Monday and check the results on Friday — like reviewing a report from an employee.
Real-world case

The direction of travel. Security vendors and AI labs have begun reporting incidents where attackers used AI agents to automate large portions of intrusion campaigns — reconnaissance, code generation, and triage of stolen data — with humans only setting goals and approving key steps. Fully autonomous end-to-end attacks remain emerging rather than routine, but the trajectory is clear and is exactly what this section warns about.

What AI adds

  • End-to-end automation: recon → exploit → extortion
  • Adapts in real time when one path is blocked
  • Machine-speed attacks that outpace human-only SOCs

How to defend

  • AI-assisted SOC: automated detection and response
  • Zero trust and identity-first architecture
  • Strong logging across endpoint, cloud, identity, network + continuous red teaming
ATT&CK spans the full matrix — Recon Initial Access Lateral Movement Exfiltration Impact
Basic
Centralized logging; documented incident-response plan.
Intermediate
SIEM with correlation, EDR + identity monitoring, network segmentation, on-call SOC coverage.
Advanced
Zero-trust architecture, AI-assisted detection/response (SOAR), continuous red teaming, automated containment with human approval gates.
Practical note

If attacks run at machine speed, a purely manual SOC is structurally too slow. Defensive agents are needed — but treat them like junior staff with sharp tools: evidence, permission gates, audit logs, rollback, and human approval for high-impact actions. Observe → reason → verify → act → audit.

If you see fast multi-stage activity — first 3 actions
  1. Activate IR and assume the scope is broader than the first alert.
  2. Contain by identity and segment — disable compromised accounts, isolate affected zones.
  3. Centralize timeline across endpoint/identity/cloud/network logs to outpace the agent's adaptation.

One-Slide Summary

AttackWhat AI addsMain defense
Login attacksAutomated page discovery + credential attacksMFA, rate limits, anomaly detection
RansomwareAdaptive targeting, generated polymorphic codeImmutable backups, EDR, segmentation
PhishingPerfect grammar, scraped personalizationVerification habits, email authentication
Deepfake fraudCloned voice/video of trusted peopleCallback verification, dual approval
Exploit generationFaster CVE-to-exploit pipelineRapid patching, KEV-driven prioritization
Full kill chainAutonomous multi-stage planning + executionAI-assisted SOC, zero trust, deep logging

Mapping to MITRE ATT&CK

The same six attacks, expressed as the standard tactics and techniques used across the security industry. Use these IDs to connect this awareness material to your detection rules, threat models, and SOC playbooks.

AttackPrimary technique(s)ATT&CK ID
Login attacksBrute Force · Credential Stuffing · Valid AccountsT1110 · T1110.004 · T1078
RansomwareData Encrypted for Impact · Inhibit Recovery · ObfuscationT1486 · T1490 · T1027
PhishingPhishing · Spearphishing Link · Phishing for InfoT1566 · T1566.002 · T1598
Deepfake fraudImpersonation · Establish AccountsT1656 · T1585
Exploit generationExploit Public-Facing App · Client Execution · Acquire VulnsT1190 · T1203 · T1588.006
Full kill chainSpans Recon → Impact across the matrixTA0043 … TA0040
Note

For AI-specific threats — attacks on AI systems rather than with them — see MITRE ATLAS, the companion knowledge base covering adversarial machine learning. The LLM-risk section below is a plain-words version of that landscape.

Defenders Have AI Too

The talk focuses on offense, but the same capabilities power defense. The arms race only makes sense seen from both sides: every advantage AI gives attackers has a defensive mirror.

Attacker uses AI to…

Move at machine speed — automate recon, exploitation, and triage faster than humans can react.

Scale and personalize — generate thousands of tailored lures and payloads cheaply.

Lower the skill bar — let low-skill actors run sophisticated campaigns.

Defender uses AI to…

Detect anomalies — flag unusual login, file, and network behavior that rules miss, and correlate weak signals across sources.

Accelerate the SOC — auto-triage alerts, summarize incidents, draft response steps, and cut mean-time-to-respond.

Hunt and simulate — AI-assisted threat hunting, log analysis, and continuous red-team simulation to find gaps first.

Guardrail

Defensive AI must not be blindly trusted either. Give agents evidence requirements, permission gates, audit logs, rollback, and human approval for high-impact actions. Industry data consistently shows organizations using security AI and automation contain breaches faster and at lower cost — but only when the automation is governed, not left to act unchecked.

When Your Own AI Is the Attack Surface

The six attacks use AI against you. But if you build with LLMs and agents — copilots, RAG systems, internal assistants — those systems become targets too. These are the headline risks from the OWASP Top 10 for LLM Applications, in plain words.

LLM01

Prompt Injection

Hidden instructions in a webpage, document, or email trick the model into ignoring its rules or running unintended actions.

LLM02

Sensitive Disclosure

The model leaks secrets, internal data, or other users' information it shouldn't have surfaced.

LLM04

Data & Model Poisoning

Tainted training or RAG data quietly biases or backdoors the system's behavior.

LLM05

Improper Output Handling

Trusting model output blindly — passing it to code, SQL, or shells — turns a reply into an exploit.

LLM06

Excessive Agency

An agent with too many permissions can take damaging actions when manipulated. Scope tightly.

LLM08

Vector / RAG Weaknesses

Poisoned or over-shared embeddings let attackers plant content or pull data across tenant boundaries.

Practical note

If you deploy internal copilots or agents, treat untrusted input as hostile, validate every model output before acting on it, scope tool permissions to the minimum, and keep a human in the loop for anything irreversible. The same observe → reason → verify → act → audit loop applies to your own agents.

Readiness Checklist

Walk through this with your team. Every unchecked box is a place an AI-accelerated attack has an easier path. These boxes are for working through on screen — they don't save.

Identity & access

Ransomware resilience

Phishing & fraud

Vulnerabilities & operations

If you build with AI

Key Terms

MFA
Multi-factor authentication — needing more than a password (a code, app prompt, or key) to log in.
Passkey / FIDO2
A phishing-resistant login that replaces passwords with a cryptographic key tied to your device.
Credential stuffing
Trying username/password pairs leaked from one breach against many other sites.
EDR
Endpoint Detection & Response — software that watches devices for malicious behavior and can respond.
Polymorphic
Malware that changes its own form each time so signature-based scanners can't match it.
Immutable backup
A backup that cannot be altered or deleted once written — so ransomware can't encrypt it.
CVE
Common Vulnerabilities and Exposures — a public ID for a specific known software flaw.
CVSS
A 0–10 severity score for a vulnerability. Severity is not the same as how likely it is to be exploited.
CISA KEV
A government catalog of vulnerabilities known to be actively exploited — a strong patch-priority list.
WAF
Web Application Firewall — filters malicious web traffic; can “virtually patch” a flaw while you fix it.
SOC
Security Operations Center — the team/function that monitors and responds to security alerts.
SIEM / SOAR
Tools that collect and correlate logs (SIEM) and automate response workflows (SOAR).
Zero trust
Never trust by default — verify every user, device, and request, even inside the network.
Kill chain
The full sequence of an attack, from reconnaissance through to impact.
BEC
Business Email Compromise — fraud that impersonates a trusted person to redirect money or data.
RAG
Retrieval-Augmented Generation — feeding an LLM your documents at query time so it answers from them.
Prompt injection
Hiding instructions in content the model reads, to hijack what it does.
MITRE ATT&CK
An industry-standard catalog of attacker tactics and techniques, used to model and detect threats.

Quick Self-Quiz

Tap a question to reveal the answer. Six questions — one per attack.

Why is “check for bad grammar” no longer good phishing advice?
Because LLMs write fluent, professional text in any language. The reliable signal now is the request itself — is it unusual, urgent, or pressuring you to move money/credentials — and whether it came through the normal channel.
What single control most reduces AI-powered login attacks?
MFA on every external-facing account, ideally phishing-resistant passkeys. If a password alone grants access, an automated agent will eventually get in.
Why do AI-assisted ransomware variants weaken traditional antivirus?
Because AI can generate endless polymorphic variations, so each sample looks different and there's no fixed signature to match. Behavior-based EDR and immutable backups matter more than signatures.
What's the one rule that defeats deepfake CFO fraud?
Never move money or secrets on voice or video alone. Require dual approval and call back on a known number. The defense is procedure, not detection technology.
Why should you prioritize patching by CISA KEV, not just CVSS?
CVSS measures severity; KEV tracks what's actually being exploited. A medium-severity bug under active attack is more urgent than a critical one nobody is using — and AI shrinks the time to exploitation.
What makes “full kill-chain automation” the most serious threat?
It's not one AI task but an agent running every stage — recon to extortion — at machine speed, adapting when blocked. Human-only defense can't keep pace, so governed defensive automation and zero trust become necessary.
Core Takeaway

AI gives attackers scale, speed, personalization, and adaptability. Defenders need verification, identity hardening, automation, and AI of their own.

This is not just “hackers using a chatbot.” It's a shift toward agentic cyber operations — AI systems that plan and execute multi-step attacks. The attacker is becoming an orchestrator of agents, not a writer of scripts. Defense has to adopt the same agentic structure, with strict boundaries:

OBSERVE REASON VERIFY ACT AUDIT

References & Further Reading

Primary source first; the rest are standard, freely available references for going deeper on each topic.

  1. IBM Technology — “AI ATTACKS! How Hackers Weaponize Artificial Intelligence” (Jeff Crume). The ~19-minute talk these notes are based on. YouTube · search: IBM Technology AI Attacks Jeff Crume
  2. IBM X-Force Threat Intelligence Index — annual real-world attack trends, including identity attacks and AI-related threats. ibm.com/reports/threat-intelligence
  3. IBM Cost of a Data Breach Report — breach costs and the measured effect of security AI/automation on response time. ibm.com/reports/data-breach
  4. MITRE ATT&CK — the standard knowledge base of attacker tactics and techniques referenced in the mapping table. attack.mitre.org
  5. MITRE ATLAS — ATT&CK's companion focused on adversarial threats to and from AI systems. atlas.mitre.org
  6. OWASP Top 10 for LLM Applications — the key risks behind the “AI as a target” section (prompt injection, excessive agency, etc.). genai.owasp.org
  7. CISA Known Exploited Vulnerabilities (KEV) Catalog — the patch-prioritization list from the exploit-generation section. cisa.gov/known-exploited-vulnerabilities-catalog
  8. NIST AI Risk Management Framework (AI RMF 1.0) — governing AI systems, relevant to deploying defensive agents with controls. nist.gov/itl/ai-risk-management-framework
  9. NIST SP 800-63B — Digital Identity Guidelines — authentication and MFA guidance behind the login-attack defenses. pages.nist.gov/800-63-3
  10. FBI IC3 — public advisories and reporting for Business Email Compromise and deepfake-enabled fraud (the deepfake case data). ic3.gov
  11. CISA #StopRansomware Guide — consolidated, practical ransomware prevention and response guidance. cisa.gov/stopransomware