Job Role: Senior AI QE Engineer
Location : West Palm Beach, Florida (Onsite)
Full Time
Job Description:
Role Summary
We are looking for a Senior AI Quality Engineer with strong automation expertise and hands-on experience validating LLMs, GenAI workflows, and AI-driven applications. This role involves building automated test suites, validating AI outputs, ensuring system reliability, and contributing to the quality strategy for next-generation AI features.
Core Experience
- 6–10 years as a Software Engineer, SDET, or Automation Engineer.
- Strong coding skills in Python, TypeScript, or Java.
- Hands-on experience developing automation scripts, tools, or frameworks.
- Practical experience using LLMs, prompt engineering, and evaluating AI-generated outputs.
- Familiarity with agentic AI systems and exposure to tools like LangGraph, AutoGen, or CrewAI.
- Basic understanding of Model Context Protocol (MCP) or context-aware workflow automation (nice to have).
AI / ML Technologies integration
- Practical experience with AI/ML frameworks:
- LangChain, Hugging Face, GPT models, vector databases, RAG pipelines.
- Experience with ML/DL libraries such as:
- Scikit-learn, PyTorch, TensorFlow, Keras, Transformers, OpenCV.
- Ability to work with embeddings, similarity search, and content evaluation metrics.
GenAI & AI Agent integrations
- Ability to integrate or build GenAI components, including RAG pipelines or agent-based workflows.
- Support model evaluation tasks such as:
- Output quality checks
- Hallucination detection
- Prompt validation
- Regression checks for model updates
Automation & Quality Engineering
- Experience building automation using Python, PyTest, Selenium, Playwright, or API testing libraries.
- Ability to design and execute automated tests for:
- Functional
- Integration
- API
- Basic performance & reliability testing
- Hands-on experience testing RESTful APIs and building automated API suites.
Cloud, DevOps & CI/CD
- Exposure to deploying AI or automation solutions on AWS.
- Working knowledge of CI/CD pipelines, including:
- Automated testing
- Model validation steps
- Versioning & artifact management
SDLC & Collaboration
- Strong understanding of the software development lifecycle, including requirements, development, testing, and defect analysis.
- Ability to collaborate with Developers, Data Scientist, and QE teams, clearly communicating progress, risks, and results.
- Skilled in documenting and tracking defects and participating in defect triage.