Design and curate training data, evaluation benchmarks, and behavior guidelines for NeuralOS AI agents. You'll ensure our agents are helpful, accurate, and safe across diverse enterprise use cases.
What You'll Do
- Design evaluation frameworks for measuring agent quality and safety
- Create and curate training datasets for domain-specific agent behaviors
- Develop behavior guidelines and safety guardrails for production agents
- Test and red-team agent responses across edge cases and adversarial inputs
- Collaborate with ML engineers on fine-tuning and RLHF pipelines
- Document best practices for agent prompt engineering and capability design
What We're Looking For
- 2+ years of experience in AI/ML evaluation, data annotation, or prompt engineering
- Strong understanding of LLM capabilities, limitations, and failure modes
- Experience designing evaluation rubrics and quality metrics
- Excellent analytical and critical thinking skills
- Familiarity with AI safety and responsible AI frameworks
- Strong written communication skills
Nice to Have
- Background in linguistics, cognitive science, or human-computer interaction
- Experience with RLHF or constitutional AI methods
- Familiarity with enterprise software domains (CRM, finance, HR)