AI Automation Engineer - 3+ Years Experience

Job Summary:
We are looking for a QA professional with experience in both manual and automation testing, who is also proficient in using AI tools to enhance testing efficiency. The role involves validating AI-driven features and ensuring high-quality software delivery through intelligent test practices.

Key Responsibilities:

• LLM Application Testing: Design and execute comprehensive test strategies for applications
• powered by Large Language Models, focusing on functionality, user experience, and response
accuracy.
• Conversational Flow Testing: Test chat interfaces, dialogue systems, and conversational AI
applications to ensure proper flow, context retention, and appropriate responses.
• Input/Output Validation: Validate LLM application responses for different user inputs, edge
cases, and ensure consistent application behavior across various scenarios.
• Prompt and Response Testing: Test various user prompts and validate that the application,
handles different types of queries appropriately and returns expected functionality.
• End-to-End Application Testing: Design, develop, and execute detailed test cases for AI application functionality, including user interfaces, APIs, and integration points.
• Edge Case and Boundary Testing: Create and execute tests for edge cases, unusual user inputs, and boundary conditions to ensure the LLM application handles unexpected scenarios gracefully.
• Integration Testing: Test the integration between LLM services and application components, including APIs, databases, and third-party services.
• User Experience Testing: Ensure LLM-powered features provide intuitive and helpful user experiences, testing response times and application usability.

• Cross-functional Collaboration: Work closely with Backend Developers, Frontend Developers Product teams, and stakeholders to understand LLM application requirements and ensure comprehensive test coverage.
• Documentation and Reporting: Document test cases for LLM application feature.

Required Qualifications:

• Education: Bachelor's degree in Computer Science, Engineering, Data Science, or related technical field.
• Experience: 3–5 years of hands-on experience in QA testing with experience or strong interest applications that integrate with AI/LLM services.
• LLM Application Knowledge: Basic understanding of how Large Language Models work within applications, API integrations, and conversational AI user interfaces.
• Testing Expertise: Proven experience in both Manual and Automation Testing with deep understanding of SDLC, STLC, and AI-specific testing methodologies.

Technical Skills:

• Proficiency in test automation tools and frameworks
• Strong API testing experience (Postman, REST Assured, etc.)
• Understanding of web technologies (HTML, CSS, JavaScript basics)
• Knowledge of databases and SQL for data validation
• Experience with cloud platforms and basic understanding of microservices architecture.