Over the past 18 months, the CISO playbook for generative AI has been relatively simple: Control the browser.
Security teams hardened cloud access security broker (CASB) policies, blocking or monitoring traffic to known AI endpoints, and routing usage through authorized gateways. The operating model was clear: If sensitive data leaves the network for an external API call, we can observe, intercept, and stop it. But this model is starting to break down.
A quiet hardware change pushes the use of the large language model (LLM) to the extreme, taking it off the grid. Call it Shadow AI 2.0, or the “bring your own model” (BYOM) era: workers running competent models natively on laptops, offline, without API calls and obvious network signatures. Governance talk is still framed as “moving data to the cloud,” but more immediate enterprise risk is “unchecked output within the device.”"
Traditional data loss prevention (DLP) doesn’t see the interaction when the result happens locally. And when security can’t see it, it can’t handle it.
Why the local result suddenly becomes practical
Two years ago, running a useful LLM on a business laptop was a niche stunt. Today, this is common for technical teams.
Three things combined:
-
Consumer-grade accelerators got serious: A MacBook Pro with 64GB of internal storage can often run 70B-class models at usable speeds (with practical limits on context length). What once required multiple GPU servers is now possible on a high-end laptop for many real-world workflows.
-
Quantization has gone mainstream: It’s now very easy to compress models into smaller, faster formats that fit in laptop memory.
-
Distribution is frictionless: Open weight models are a command away and the tools ecosystem makes “download → run → chat” irrelevant.
Result: An engineer can pull down multi-GB model artifacts, turn off Wi-Fi and run sensitive workflows locally, review source code, summarize documents, compile customer communications, and even perform exploratory analysis on regulated datasets. No outgoing packets, no proxy logs, no cloud audit trails.
From a network security perspectivethis activity cannot be distinguished by “nothing happened”.
The risk is not only that data is separated from the company
If the data doesn’t leave the laptop, why should the CISO care?
Because the dominant risks are shifting from exfiltration to integrity, provenance and compliance. In practice, local output creates three types of blind spots that most businesses don’t operate on.
1. Code and decision contamination (integrity risk)
Local models are often adopted because they are fast, private and “no approval required." The downside is that they are often screened for enterprise environments.
A common scenario: A high-level developer downloads a community-tuned coding model because it compares well. They stick to internal auth logic, payment flows, or infrastructure scripts to “clean up”." The model returns a result that looks competent, compiles, and passes unit tests, but subtly compromises security (weak input checking, dangerous defaults, fragile concurrency changes, internally disallowed dependency options). The engineer undertakes the change.
If that interaction happened offline, you may have no record of the AI affecting the code path at all. And when you later respond to an incident, you’ll be investigating the symptom (the vulnerability) without discovering the root cause (uncontrolled model usage).
2. Licensing and IP exposure (compliance risk)
Many high-performance models come with included licenses restrictions on commercial useresponsibilities that are inconsistent with attribute requirements, usage area limits, or proprietary product development. When employees run the models locally, that use can bypass the organization’s normal procurement and legal review process.
If a team uses a non-commercial model to create production code, documentation, or product behavior, the company may inherit the risk of later exposure during an M&A investigation, customer security investigations, or litigation. The difficult part is not only the licensing conditions, but also the lack of inventory and tracking. Without a managed model center or usage log, you won’t be able to prove what’s being used where.
3. Model supply chain exposure (source risk)
The local result also modifies the software chain problem. Endpoints are starting to collect large model artifacts and toolchains around them: their own loaders, converters, runtimes, plugins, UI shells, and Python packages.
Here’s an important technical nuance: The file format matters. Like newer formats though Protective means designed to prevent arbitrary code execution, old Acid based PyTorch files it can execute malicious payloads simply when loaded. If your developers are grabbing unchecked checkpoints from Hugging Face or other repositories, they’re not just downloading data, they may have downloaded an exploit.
Security teams have spent decades learning how to treat unknown executables as adversaries. BYOM requires extending this mindset to model artifacts and the surrounding runtime stack. The biggest organizational gap today is that most companies don’t have the equivalent of a materials software for models: Source, hashes, authorized sources, browsing and lifecycle management.
Mitigating BYOM: treat model weights as software artifacts
You cannot resolve the local result by blocking URLs. You need endpoint-aware controls and a developer experience that makes the safe way the easy way.
Here are three practical ways:
1. Move control to the endpoint
Network DLP and CASB are still important for cloud use, but not sufficient for BYOM. Start considering local model usage as an endpoint management problem by looking for specific signals:
-
Inventory and detection: Scan .gguf files larger than 2GB, such as processes with high resolution call.cpp or Ollama and general domestic listeners default port 11434.
-
Process and implementation awareness: Monitor repeated high GPU/NPU (neural processing unit) usage from unverified runtimes or unknown local result servers.
-
Device policy: use it mobile device management (MDM) and endpoint detection and response (EDR) policies to control the installation of unapproved working hours and to enforce basic hardening on engineering devices. The point is not to punish experiments. It is to restore eyesight.
2. Provide paved road: Internal, selected model center
Shadow AI often the result of friction. Validated tools are too restrictive, too general, or too slow to validate. A better approach is to offer an internal directory that includes:
-
Validated models for general tasks (coding, generalization, classification)
-
Approved licenses and user manual
-
Hash-hardened versions (safer formats like Safetensors are preferred)
-
Clean up files for safe local use, including where sensitive data is not allowed. If you want developers to stop trashing, give them something better.
3. Update policy language: “Cloud services” are no longer enough
Most acceptable use policies talk about SaaS and cloud tools. BYOM requires a policy that clearly covers:
-
Load and run model artifacts on corporate endpoints
-
Accepted sources
-
Licensing eligibility requirements
-
Rules for using models with sensitive data
-
Storage and access expectations for local output tools This doesn’t have to be too strict. It is necessary to be unambiguous.
The perimeter switches back to the device
Over a decade, we’ve moved security controls “up” to the cloud. Local inference pulls a meaningful part of the AI activity “down” to the endpoint.
5 signal shadow AI moved to endpoints:
-
Large model artifacts: Memory consumption not explained by .gguf or .pt files.
-
Local result servers: Processes listening on ports such as 11434 (Ollama).
-
Examples of GPU usage: Increases in GPU usage when offline or disconnected from a VPN.
-
Lack of model inventory: Code outputs cannot be adapted to specific model versions.
-
License Ambiguity: existence "non-commercial" model weights in production structures.
Shadow AI 2.0 isn’t a hypothetical future, it’s a predictable result of fast hardware, easy distribution, and developer demand. CISOs who only focus on network control will miss what’s happening in the silicon sitting on employees’ desks.
The next phase of AI governance is less about blocking websites and more about controlling artifacts, provenance, and politics at the endpoint without killing productivity.
Jayachander Reddy is a Senior MLOps Engineer with Kandakatla.
