AI Test Engineer

About the job

Must have:

• Experience of testing AI applications
• Experience of test automations and tools, e.g. Tosca, Python, Selenium
• Strong knowledge & use of QA methodologies e.g. Agile, Waterfall, DevOps, BDD for functional test activities.
• Proven experience as a Quality Assurance Engineer, with a significant focus on testing Artificial Intelligence and Machine Learning products.
• Agentic AI Agent testing and validation.
• Strong knowledge of Responsible AI principles and practices, including experience in testing for fairness, bias detection, explainability, privacy, and security in AI systems.
• Familiarity with Agentic AI concepts and architectures.
• Experience in working with teams in DevSecOps
• In-depth knowledge of test toolsets and development languages including C#, Java
• Solid understanding and practical experience in testing Generative AI (GenAI) Solutions, including evaluating model outputs, prompt engineering, and understanding the nuances of generative models.
• Understanding of DevOps pipeline i.e. change, release & deployment processes
• Experience in conducting UAT, defect reporting & triage and preparing QA metrics
• Understanding across multiple disciplines of test, e.g. functional, integration, performance, compatibility
• ISTQB Foundation Level certificate (or equivalent experience)
• Test management tool expertise (such as Atlassian Jira, Azure DevOps etc.)
• Strong written, verbal and presentation skills
• Professional and confident with a strong drive & enthusiasm to multi-task.
• Acute attention to detail.

Desirable experience:

• A proven track record in a top tier consultancy or a large enterprise / organisation
• .Direct experience in testing Agentic AI Solutions, including evaluating agent behaviour, goal achievement, and interaction patterns
• .Experience with performance testing and load testing of AI models and applications
• .Knowledge of data governance and data quality best practices in an AI context
• .ISTQB Certified Agile Tester qualificatio
• nISTQB Certified AI Tester qualificatio

n

Requirements added by the job poster

• 1+ years of work experience with Generative AI Tools

• 2+ years of work experience with Application Testing

• 3+ years of work experience with Selenium Testing