Toptal
ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)
Remote, US
Role snapshot
Toptal is hiring a ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM) on a full-time basis. This is a remote role open to candidates in US. The listing highlights Artificial Intelligence, Docker, Git, Graphics Processing Unit and LLM.
- Company
- Toptal
- Location
- Remote, US
- Work mode
- Remote
- Job type
- Full-time
- Experience
- 3+ years
- Posted
- Nov 5, 2026
Who this role suits
Practitioners with a few years behind them who can work independently, make trade-offs and help newer teammates.
How to prepare your application
- Check your résumé against the listed skills (Artificial Intelligence, Docker and Git) and give each one a concrete example: what you built or achieved, not just that you used it.
- Prepare examples of decisions you drove, how you measured the result, and how you worked with other teams.
- Be ready to discuss trade-offs and scope, not only the tools you have used.
- Read what Toptal publishes about its product or team, so you can say clearly why this role and this company.
- For remote roles, show that you communicate clearly in writing and can work without close supervision.
Apply only on Toptal's own site. A genuine employer never asks candidates for money, deposits or bank details to apply, interview or “secure” a job. If something looks wrong, use Report a problem with this listing and we will review it. Read how we check listings.
About the role
Overview
We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct.
***THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.***
Responsibilities
⢠Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.
⢠Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.
⢠Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.
⢠Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.
⢠Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.
⢠Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable.
Skills/Experience
⢠3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield
⢠Did this work in the last 3 years and can name the systems, tools, data and rules used
⢠Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls
⢠Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM
⢠Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them
⢠Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases
⢠Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each
Nice-to-have
⢠AI lab or LLM infrastructure experience
⢠Contributions to PyTorch, vLLM or SGLang
⢠GPU profiling experience
⢠Docker and Git proficiency
⢠Experience writing automated tests
Process
⢠Task vetting instead of an interview: complete a vetting task so the client can assess your task design
⢠Paid per task; each task takes about 60 to 75 minutes
⢠Work is delivered task by task, with reviewer feedback on submissions
⢠Expected start in early November 2026
Please mention the word **ASTONISHED** and tag RMi4yOC43Ni4xMjE= when applying to show you read the job post completely (#RMi4yOC43Ni4xMjE=). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they're human.