• Job Description

  Req#: 24-16785
  Job Description:
  
  Responsibilities:
  • Evaluate and test Generative AI POC models built using Open AI and Vertex LLM Models.
  • Design, develop, and execute detailed test plans to validate the model performance with real customer data.
  • Perform data validation, ensuring accuracy and completeness of the input/output data flow through AI models.
  • Collaborate with data scientists and engineers to ensure the model's output is aligned with the expected results.
  • Create test cases to assess model accuracy, bias, performance, and edge cases.
  • Identify model weaknesses, inaccuracies, and areas for optimization.
  • Report bugs, issues, and improvement areas, providing detailed feedback to development teams.
  • Use automated testing tools and frameworks for model testing.
  • Maintain comprehensive documentation of the QA process and results.
  Requirements:
  • Proven experience in testing Machine Learning/AI models.
  • Familiarity with Generative AI models like Open AI GPT, Vertex AI models.
  • Strong knowledge of data validation, model performance, and quality assurance practices.
  • Experience with model evaluation metrics such as accuracy, precision, recall, F1 score, and bias analysis.
  • Proficiency in Python or other relevant programming languages.
  • Familiarity with testing frameworks and automated testing tools for AI models.
  • Strong analytical and problem-solving skills.
  Preferred:
  • Experience in testing AI models with real-world datasets.
  • Knowledge of model versioning, deployment, and monitoring.

• About the company

  We are a team of technical visionaries who joined forces to form a company that would gather the best and brightest talent in the world. The company uses this talent to help other organizations fulfill their technological dreams and goals and make them a reality. Welcome to Cynet Systems Inc. – a premier staffing and recruiting company – a nationally and locally certified diversity firm.

Notice