Questionnaire for AI/ML/GenAI Engineering Teams

Questionnaire for devs used by sec engineers during walkthroughs of GenAI applications.

1. Model Architecture and Controls

Model Security Layers

• Has intent filtering been implemented? Describe the mechanism

• Is there a jailbreak/prompt injection detection layer?

• Are there controls to detect and handle adversarial attacks (input perturbations)?

• For RAG systems: Is output grounding implemented?

• Is the environment sandboxed?

External Access

• Does the model have access to external APIs?

• Does the model have file system or network access?

• What are the controls around external resource access?

Model Training

• Is this a fine-tuned model? If yes:

  • Is the training data available for review?

  • Will the model be trained on user conversations?

  • What quality checks exist for training data?

• Are there controls to prevent bias in model outputs?

2. Data Handling and Privacy

Data Processing

• Is PII or confidential information being processed?

• Has Data Privacy/Protection approval been obtained?

• Is data being anonymized before processing?

• Are conversations being stored? If yes:

  • What database security controls are implemented?

  • What is the data retention policy?

Data Sources

• What are the input data sources?

• How is data integrity verified?

• Are there quality checks for untrusted user inputs?

Third-Party Data Sharing

• Is data shared with third parties?

• Have third-party services been security vetted?

• Is the vendor Infosec-GRC onboarded?

• Has legal approval been granted for data sharing?

3. Input/Output Controls

Input Management

• What input validation and sanitization is implemented?

• Are there size/format restrictions on inputs?

• How are file uploads handled and validated?

• Is there manual review for any inputs?

Output Controls

• What output moderation systems are in place?

• Is output encoding implemented?

• How are inappropriate/malicious outputs filtered?

• Are model outputs logged and monitored?

4. Access Control and Rate Limiting

Authentication & Authorization

• What authentication methods are implemented?

• How are user roles and permissions managed?

• Is MFA required for sensitive operations?

Rate Limiting

• Is rate limiting implemented per user?

• Are there token consumption limits?

• How are API quotas enforced?

5. Monitoring and Logging

Activity Monitoring

• Are user inputs logged?

• Are model outputs logged?

• How are logs protected and retained?

• Is there automated alerting for suspicious patterns?

Security Monitoring

• How is system health monitored?

• Are there alerts for unusual model behavior?

• How are security incidents detected and handled?

6. Application Security

Security Testing

• Has VAPT been performed on the web application?

• Is SAST implemented in the CI/CD pipeline?

• Is Software Composition Analysis (SCA) performed on the codebase?

Error Handling

• How are application errors handled?

• Is there a fallback mechanism for model failures?

• How are failed requests logged?

7. Compliance and Governance

Legal and Compliance

• Are there regulatory requirements for the use case?

• Has legal review been completed?

• Are there data sovereignty requirements?

Security Standards

• What data security standards are being followed?

• Are there industry-specific compliance requirements?

• How is compliance monitored and maintained?