Alibaba Cloud's AI gateway blocks prompt attacks with a 200, not a 403

Security questions show up before model performance gets a mention. Almost every project that wires an LLM into a real service hits the same ones during architecture or security review: who gets to call the endpoint, what happens when someone tries to jailbreak it, and whether personal data ever reaches the model in raw form.

Alibaba Cloud's answer is to stop solving these inside each application and move them into the infrastructure. Its AI Gateway sits in front of the model and runs authentication, prompt injection filtering, and data masking as one pipeline. Build the same security logic into every app, and each policy change means touching all of them. Put it in the gateway and the policy lives in one place, with no security logic on the client side or the model side.

Technical account manager Hosung Kim documented a working version of this setup in the ap-southeast-1 region (the Korean version of the walkthrough is online). The backend is Model Studio (Bailian) running qwen-flash through an OpenAI-compatible endpoint. What matters is what the tests actually return, not that the pieces exist.

One clarification up front: prompt injection blocking is handled by AI Guardrails (AI Fence), a separate content security service, not by the gateway itself. The gateway is the gate that plugs security capabilities into the model call path. Alibaba Cloud has been merging AI development and AI security into one roadmap, where the same backend that indexes a developer's code can also inspect the traffic that code generates. On the development side of that roadmap, Alibaba's Qoder has added remote delegation for coding agents.

Why the security layer belongs in the gateway, not the app

Requests flow through the pipeline in a fixed order: authentication, Guardrails, masking. A missing or wrong key is rejected with 401 before Guardrails or masking run at all, which is what saves invocation cost and latency. Security policy lives in one place, the gateway, so a security review has one thing to explain and audit.

AI Gateway ships in Serverless and Dedicated forms with different plugin support. Serverless takes only some platform-provided plugins and no custom ones; on the instance Kim tested, ai-data-masking could not be installed, so a Dedicated instance is the safe choice. Dedicated also requires at least two availability zones for high availability.

Authentication, and the toggle that silently disables it

With authentication enabled on the Model API, only registered consumers (API keys) can call it, and keys can be created with generateMode: Custom so you supply them yourself. The trap lives in the console: the Consumer Authentication tab has a Status toggle that must be flipped to Enabled for enforcement to actually happen. If it stays off, requests pass through even without a valid key. The walkthrough's advice is blunt: "Double-check it." Silent gaps of this kind are not unique to consoles: in multi-vendor agent networks, a compromised vendor tool keeps being used by other agents unaware.

Guardrails: blocked prompts still come back as HTTP 200

AI security (AI Fence) is enabled under Policies and Plugins on the Model API, with the Guardrails service endpoint pre-filled. The tuneable part is the block policy. At the Low protection level, the filter caught an obvious injection ("ignore previous instructions") but missed a DAN-style jailbreak. At Medium, it caught both. The balance between detection strength and false positives is left to the operator. Quantifying that balance is a research problem of its own, per a framework that defines trust by the numbers.

Here is the detail that breaks naive client code: blocked requests still return HTTP 200. The block message arrives in OpenAI-compatible response format, with the model field set to "from-security-guard", an x_higress_guardrail.blockedDetails object classifying the block as promptAttack at medium level, and zero tokens consumed. Client code that only checks the status code will treat a blocked jailbreak as a successful model call. Judging by body, not status code, is the rule. The masking plugin's deny_code is configurable separately (Kim set it to 403 here), but because Guardrails runs first, its 200-style blocks take precedence in practice.

Masking Korean PII, and the restore trade-off

The ai-data-masking plugin replaces sensitive values before they reach the model and can restore originals on the response path. The catch for teams outside China: the built-in example patterns, like %{MOBILE} and %{IDCARD}, match Chinese formats, Chinese mobile numbers and Chinese national IDs. A Korean service needs its own rules, and the walkthrough's final configuration covers five types:

• Korean national ID (987654-1234567) becomes ******-******* and is never restored.
• Korean mobile numbers (010-1234-5678) become 010-****-5678, restored in responses.
• Korean landlines (02-765-4321) become 02-****-4321, restored.
• Emails become ****@domain, restored.
• IP addresses become ***.***.***.***, restored.

The national ID gets no restore on purpose: it should never reach the model, and there is no reason for the original to circulate in responses either. Restore: true is itself a security trade-off, because restored values go back to the client and can end up in gateway and client logs. Masking hides the original from the model; it does not eliminate it. That is the recurring shape of the AI pipeline's silent leaks, where every fix makes one of them worse. In a regulatory environment that also controls logs and storage, whether to restore at all is a deliberate decision.

Two configuration details matter. The built-in sensitive-word dictionary (system_deny) comes from houbb/sensitive-word and is Chinese-centric, so Korean forbidden words have to be added through deny_words. And rules apply in the order listed, with each rule matching against the result of the previous one, which means overlapping patterns need the more specific ones first.

Labeling matters too. When the test note used "RRN" instead of "ID", Guardrails' sensitiveData dimension blocked the entire request at S2 level before the masking plugin ever saw it. That is correct behavior, but it means you cannot observe masking when Guardrails kills the request first.

Five scenarios, verified end to end

The walkthrough sends five scenarios through the gateway and records the outcomes.

Scenario	Result
Normal question with API key	Passed, HTTP 200, normal response from qwen-flash
Prompt injection	Blocked, promptAttack / medium
DAN-style jailbreak	Blocked, promptAttack / medium
Request containing PII	National ID permanently masked; mobile and email masked to the model, restored in the response
Request without API key	401 rejected before Guardrails or masking

The strongest proof that masking worked is a follow-up question. After the PII test, Kim asked the model to state the exact digits after the hyphen in the customer ID. The model replied that the ID is displayed as ******-******* and that it cannot reveal the digits, because its input only ever contained the masked form. That is the same principle behind Hugging Face's Moon Bot, where the LLM never sees the production keys: keep sensitive values out of the model's context entirely.

Guarding the model's front door

The payoff is operational: one policy, one management point to explain and audit, exactly what a security review asks for. It also lines up with demand that is already there: an Alibaba-commissioned NielsenIQ survey of 1,000 Asian IT decision makers found near-universal enthusiasm for AI adoption, with 95% raising AI budgets, but consistent calls for secure, end-to-end solutions, and Qwen models are cited as especially popular in Japan and South Korea for local-language performance, the same turf Sakana's Namazu is targeting with a keigo-specialized model. A new-energy vehicle manufacturer reportedly used the platform's data masking and cross-border compliance features to pass GDPR audits, according to our earlier coverage of Alibaba's security push.

Rule-based masking does not guarantee 100% coverage of variants that do not match the rules, and restore behavior means masked data can still surface in logs. For teams taking an LLM to production, the walkthrough's closing advice is the right yardstick: give the model's front door the same design care as the model choice itself.