QA Engineer, AI
Posted 7 days 20 hours ago by TORTUS
You are the last line of defence between our code and patient care.
TORTUS runs in hospitals, GP surgeries, ambulances, and across dozens of clinical environments - on iPads, in Epic, Cerner, Surgery Connect, EMIS, and multiple browsers. You'll own quality end-to-end: building the automated test infrastructure, running AI output evaluations, managing releases, and ensuring nothing ships that could break in any of those places. You'll be managing AI coding agents, building simulation suites for clinical AI outputs, and pioneering what quality engineering looks like when development is AI-accelerated.
We're looking for a technically strong QA engineer who is obsessed with AI tooling. You'll be building and maintaining automated test suites, but you'll also be running multiple coding agent instances, designing Playwright harnesses at scale, and creating entirely new kinds of tests - like evaluating whether our AI-generated clinical notes are correct. You'll also need to be comfortable rolling up your sleeves for hands on manual and exploratory testing - complex clinical workflows across multiple deployment surfaces demand human judgement, not just automation. If you're the kind of person who sees AI agents as force multipliers rather than novelties, we want to talk to you.
What Else Would Be Great- 2-5 years of experience in QA engineering, software testing, or SDET roles
- Hands on experience with automated testing frameworks (Playwright, Jest, Vitest, or similar)
- Genuine enthusiasm for AI assisted development - you're already using Claude Code, Cursor, Codex, or similar tools to ship faster
- Familiarity with CI/CD pipelines and integrating quality gates into automated workflows
- Understanding of requirements traceability and how to maintain requirement to test case mapping
- Experience writing test plans, protocols, and verification reports
- Comfort testing across multiple platforms and environments (web, mobile, iPad, embedded in third party systems)
- Exposure to compliance standards (IEC 62304, ISO 13485, ISO 14971) or Quality Management Systems is a strong plus - but not required
- Strong written communication - you'll be producing documentation that goes into our regulatory technical file
- Own and expand the end to end test suite using Playwright, Jest, and Vitest - covering unit, integration, system, and E2E testing
- Build automated test harnesses for all deployment surfaces: web app, iPad, Epic, Cerner, Surgery Connect, EMIS, and multiple browsers
- Design test cases covering functional requirements, edge cases, and failure modes
- Integrate tests into CI pipelines to gate every PR with passing tests
- Use AI coding agents to accelerate test creation and maintenance
- Build end to end simulation suites for our clinical AI pipeline: upload known audio, verify transcripts, clinical codes, and guidelines are correct
- Create smoke tests for AI outputs
- Collaborate with the ML team on AI evaluations - both deterministic tests and non deterministic model output evaluations
- Detect quality regressions when prompts, models, or pipelines change
- Own the QA process for versioned releases
- Run regression and manual exploratory testing across all supported environments before promoting to production
- Manage the release checklist - ensure nothing ships without verification on all target platforms
- Maintain the requirements traceability matrix, linking software requirements to test cases and results
- Ensure verification coverage with documented evidence for our regulatory technical file
- Support ISO 13485 audits and Class IIa certification with testing evidence
- Automate compliance workflows where possible
- Log, triage, and track defects with clear severity classification
- Work with developers to reproduce issues, verify fixes, and close the loop
- Ensure no high severity defects ship without resolution and re testing
After 12-18 months, you've transformed how TORTUS ships software. The automated test suite covers all major deployment surfaces, and developers trust it to catch issues before they reach staging. You've built a simulation suite that catches AI regressions before they reach clinicians - we know within minutes if a release has broken clinical coding or note generation. The traceability matrix is immaculate, and auditors breeze through our verification evidence. Releases are versioned, documented, and predictable. You've reduced manual regression time by 70%+ through automation and AI assisted test generation. The engineering team sees you as the person who makes it safe to move fast.
Where Does This Role Take You?This is a foundational role in a fast growing, safety critical AI company. High performers will have opportunities to:
- Lead the QA function as the team scales, potentially growing into a QA Lead or Engineering Manager role
- Specialise in AI evaluation and clinical safety validation - a genuinely novel discipline
- Shape the intersection of AI accelerated development and medical device compliance
- Build out a quality engineering team as TORTUS grows internationally
- Google Cloud Platform (GCP), Google Kubernetes Engine (GKE)
- Terraform, GitOps & ArgoCD
- PostgreSQL + Redis
- TypeScript across our stack (React, NestJS), Python/FastAPI for ML workloads
- Playwright for E2E testing, Jest/Vitest for unit and integration tests
- Self hosted ML models, MLflow for experiment tracking
- Competitive salary + equity
- 9 day fortnight (every other Friday off)
- 25 days holiday + bank holidays
- Mon & Fri WFH optional - office first culture
- Latest MacBook + the right equipment to make you at your best
- The chance to genuinely transform healthcare
TORTUS was founded to address one of the most fundamental and persistent problems in healthcare: human error driven by cognitive overload and administrative burden. Modern clinicians are overwhelmed by documentation, compliance, and fragmented digital systems, leaving less time and attention for patient care.
Our mission is to eliminate avoidable human error in medicine by augmenting clinicians with real time, agentic AI. An AI co pilot for every clinician.
The core product is a real time AI system that operates inside live patient consultations. It transcribes and structures conversations, surfaces relevant clinical context and guidelines, and executes downstream actions such as documentation, prescribing workflows, and follow ups. Doctors and AI collaborate in real time, with the clinician always in control.
Traction- 500,000+ paid consultations processed
- 10x year on year growth
- 60 NHS hospitals deployed
- 60-80% daily adoption when rolled out
- 25% time savings per clinician
- Regulated medical device (Class I today, progressing to Class IIa)
We are an office first company based in the historic Holborn Town Hall. Where possible, we aim to be in the office at least 3 days a week (typically Tuesday, Wednesday and Thursday).