How to hire an AI engineer: what to check, what to ask and what it costs
Hiring for AI is harder than it looks, because a good demo and a good engineer are not the same thing. This guide covers what AI engineers build, the skills to test, questions to ask, what moves the price and how to plan the first months. If your project is mostly about data, also look at our data strategy consultants and Python developers.
What an AI engineer builds
Most AI engineers today work with large language models that someone else trained, such as the models from OpenAI, Anthropic or Google, or open-source models like Llama. The job is to turn a general model into a feature that works for your users: a help assistant that answers from your own articles, a tool that pulls fields out of contracts, or an agent that updates records in your CRM. Some AI developers also train smaller machine learning models for forecasting or scoring, where a large model would be slow and expensive.
Interest keeps growing. In the Stack Overflow Developer Survey 2025, 84 percent of respondents said they use or plan to use AI tools in their development work, up from 76 percent the year before. The same survey found that more developers distrust the accuracy of AI tools (46 percent) than trust it (33 percent). That gap is why testing matters so much in AI work.
Skills to check
- Python and software basics: clean code, tests, Git and simple, documented APIs.
- Model APIs: streaming, structured output, timeouts, retries and keeping costs under control.
- Retrieval: splitting documents, embeddings, vector search and showing sources with each answer.
- Evaluation: a test set built from real questions, with answers scored before every release.
- Safety: limits on what an agent can do, filtering of inputs and outputs, and care with personal data in prompts.
- Judgment: knowing when a simple rule or a small model beats a large language model.
Five interview questions
- Our assistant sometimes makes up answers. How would you find out how often, and how would you reduce it?
- Walk me through a retrieval pipeline you built. How did you choose the chunk size and how many results to pass to the model?
- How would you compare two models for our use case, beyond trying a few prompts by hand?
- Our monthly model bill doubled. Where do you look first?
- Tell me about an AI feature that did not work out. What did you learn from it?
Strong answers are specific. Good candidates talk about test sets, the numbers they tracked and the trade-offs between quality, speed and cost. Be careful with anyone who only talks about the newest model.
Sample job description
Keep it short and concrete. For example: “We are hiring an AI engineer to build the assistant inside our customer portal. You will build retrieval over our help center and order data in Python, work with model APIs from OpenAI and Anthropic, set up automated tests for answer quality and ship through our CI pipeline. You will work with our back-end team and product manager. Must have: Python in production, LLM features used by real customers, vector search. Nice to have: fine-tuning, classic machine learning and AWS.” For a full template you can adapt, see our software engineer job description template.
What drives the cost
Seniority matters most. An engineer who can design the whole system, choose models and own quality costs more than one who builds to a clear spec. Location comes next: pay in the US and Western Europe is higher than in Latin America, Eastern Europe, South Africa or India. Then come the hours you need and any specialist skills, such as training models on your own data or working with sensitive records. Plan for running costs too. Model providers bill by the amount of text processed, so a careful engineer can lower your bill every month by caching, choosing smaller models and trimming prompts.
Start with a small, paid task
Before you make an offer, give your last one or two candidates a short paid task based on a real problem. For example, answer ten real customer questions from a sample of your help articles, with sources, and report how many answers were correct. Keep it to a few hours. How they measured quality will tell you more than the answers themselves.
Freelance or dedicated
A freelancer is a reasonable choice for a short experiment with a clear end. If the feature will reach customers, choose someone who stays. AI features need ongoing care: new models come out, your content changes and users ask questions nobody expected. A dedicated engineer keeps the test set current and the answers accurate.
Your first three months
In the first week, share access to the code, the data sources and the model accounts, and agree on one small use case with a clear measure of success. In the first month, ship a first version to a small group of users with a test set and logging in place. By the third month, your engineer should report quality and cost trends, suggest the next use case and review AI code from the rest of the team.
Mistakes to avoid
- Judging an AI engineer on demos instead of on how they test and measure.
- Sending customer or personal data to a model provider without checking your agreement with that provider.
- Letting an agent change records or send messages without limits and logs.
- Building on one model provider with no way to switch when prices or quality change.
For a wider view of how AI is changing outsourcing, read our article on artificial intelligence in outsourcing.