Senior Software Engineer

Responsibilities - Design and ship end-to-end features spanning data models, APIs, background workers, and UI. - Build scalable systems for running, scoring, aggregating, and comparing agent and model evaluation trials. - Improve benchmark task quality using verifiers, rubric-based grading, and LLM-as-a-judge evaluation. - Integrate new models, agents, and benchmarks through generalizable systems rather than one-off implementations. - Maintain performance and reliability through efficient queries, safe migrations, and load-resistant concurrency. - Review pull requests and help maintain consistent engineering standards, tests, and error handling. Requirements : - 5+ years of experience building and operating production backend systems, including ownership of at least one service from design through on-call. - Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field. - Strong Python, Django or a similar framework, PostgreSQL, and task-queue experience such as Celery. - Hands-on experience with LLM evaluation, including benchmark construction, agent-trial scoring, and result aggregation and comparison. - Ability to identify ambiguous instructions, weak or gameable verifiers, leaked answers, and invalid checks. - Comfort working with TypeScript and React when frontend changes are needed. - Experience with sandboxed or containerized agent environments such as Docker, Kubernetes, Modal, or Daytona is preferred. - Experience with LLM-as-a-judge or rubric-based grading, coding agents, MCP, distributed scheduling, concurrency limits, database locking, or open-source evaluation and infrastructure projects is preferred. Benefits : - Work at the frontier of AI with leading AI labs and contribute to datasets, reinforcement-learning environments, and benchmarks. - Opportunities to contribute to AI research and showcase work at conferences such as ICLR, ICML, and NeurIPS. - Collaborate with colleagues with experience at Google, Meta, Amazon, and other leading technology companies. - Startup-paced environment with significant ownership and exposure to enterprise AI applications. - The posting states base salary plus equity.