nCERT issued an advisory urging organisations to use Generative Artificial Intelligence (GenAI) tools safely. The advisory specifically covers public AI chatbots, coding assistants, browser extensions, AI-enabled applications, and third-party AI services.
Uncontrolled use of public AI services could expose sensitive information and create cybersecurity risks. Unauthorised use of these tools, termed “Shadow AI,” risks exposing intellectual property, source code, credentials, and other organisational data.
Risks associated with GenAI tools include prompt injection attacks, insecure AI-generated code, malicious integrations, inaccurate outputs, and compromised third-party AI models. Organisations must establish and enforce Generative AI Acceptable Use Policies.
The advisory recommends that organisations prohibit employees from submitting classified, confidential, personal, proprietary, or credential-related information to unapproved or public AI platforms. Furthermore, organisations are advised to maintain centrally vetted lists detailing approved AI tools, models, plugins, APIs, and platforms.
The advisory calls for qualified human review of any AI-generated code and other critical AI outputs before they are used in decision-making or deployed. nCERT recommends extending Data Loss Prevention, access monitoring, and endpoint security controls to cover AI interfaces.
These controls should be capable of detecting unauthorised AI usage, suspicious API activity, sensitive-data submissions, and Shadow AI. Organisations should align their AI governance practices with the NIST AI Risk Management Framework and the OWASP Top 10 for LLM Applications.
The guidelines also recommend regular staff training alongside maintaining audit trails for all AI tool usage. Organisations must continuously monitor for violations of internal policies, including unreviewed AI-generated code, unauthorised AI plugins and APIs, and suspicious AI access.
Should an AI-related security incident occur, organisations must contain unauthorised access, preserve logs and evidence, revoke compromised credentials or API keys, and take corrective measures. Incidents involving GenAI policies violations, prompt injection, AI supply-chain compromise, or sensitive-data exposure must be reported to National CERT Pakistan.
