So you want to hire an AI Engineer?

Hiring AI Engineers for Startups and Building AI Teams – what we’ve learnt at OQ.
Over the past 12 months we’ve seen a wave of new and old clients reach out about hiring their first AI Engineer, it seems like it’s the role that everybody wants to bring into the team right now as there’s ever-increasing pressure to leverage the latest developments in LLMs and Generative AI.

If you’re looking at building products that leverage anything related to LLMs, Generative AI or AI Agents, it’s highly likely you’ll need an AI Engineer to do this.

Unfortunately as this is a relatively new job title, the best way to hire it isn’t looking at the pre-existing talent pool of AI Engineers but to look at who has the skills that go into the role.

As the work of an AI Engineer is spread across a mixture of software engineering, data engineering and ML engineering – these are typically the backgrounds and skillsets we’re focussing on. Because AI Engineers are almost always focussing on products, we typically exclude candidates coming from a data science background unless they have clearly worked on production use cases and have some software engineering skills.

How do you define which background to focus on? Well as always, it’s best to work backwards from your use case. What will this team actually build?

Start with The Business Problem
With any decision to bring someone into the team, as a hiring manager you should be thinking about what exactly it is you’re going to build and what stage of this process you’re in, and then work out who would be the right kind of person to build it. Sounds simple!

If you’re at the start of the process and your main challenges are ensuring that your models are fed by accurate data, and your use case is more search and RAG oriented, hiring a data engineer with some AI knowledge should be enough for now.

If your main challenge is getting a product to market quickly that is mostly based on connecting APIs with out of the box tools and models, then a software engineer with AI knowledge will work.

Conversely, if you’re solving a novel problem or you have considerations around security, reliability or cost that mean you need to fine-tune and train models yourself, then you should definitely look at bringing an ML Engineer into the team.

The search is usually more nuanced than this suggests, but the reality is that you’re unlikely to find all of these skills in one person and the best approach is likely to be to hire candidates with a blend of these skillsets, with a some overlap and complimentary profiles that fill in the gaps.

Should Data Engineers get more credit?

Time and time again when speaking to leaders in Data and AI, they stress the importance of data and data quality on the success of AI projects. If you’re building an AI team, I’d stress that at least one of your team has to have a solid data engineering skillset.
Ultimately:
– Bad data in = bad results out
– No data fed back into the system = you’re not able to improve outputs
– Badly designed databases = slow responses

When everyone has access to the same tools and models, ultimately your data and how you are able to tap into it becomes on of your assets.

Should You Train or Fine-tune models?
Here’s a big question many companies face: do you just use existing models, or do you train/fine-tune your own? This question defines what level of ML experience you need within your team.
Here are some key questions to ask when deciding?
Do you have specific language, data, or processes that general models don’t understand?
For example, a construction AI product might need fine-tuning to understand specific construction documentation.
Do you have unique proprietary data, strong in-house expertise, and do you have budget and time?
Do you have strict considerations related to privacy?
Is intellectual property a concern for your business?
Are your customers comfortable with your use of third party models? Some clients in less forward thinking industries can be resistant to this.
If the answer is yes to these question then, it’s likely you’ll need to make sure there is some solid ML Engineering skills within your AI team.

Strengths + Risks of Each Engineering Archetype

Data Engineer → AI Engineer:
Strengths: Can handle pipelines, data quality, databases, orchestration; great for RAG setups, observability, and feedback loops.
Best Fit: If your AI relies on internal data for feeding LLMs, agents, knowledge graphs, MCP servers etc. You need to recycle user feedback into models. If your proprietary data is your key asset within the product.
Risk: May lack ML background and would need support on model training/evaluation, if coming from Analytics/BI they could lack software eng fundamentals.

Software Engineer → AI Engineer
Strengths: Strong at integration with out of the box tools, building APIs, making systems scalable and especially great for productionizing AI systems and keeping them reliable.
Best Fit: If your AI products must integrate with customer-facing apps, need uptime, and need to ship fast without breaking things.
Risks:May treat models as black boxes and might struggle to solve problems if the outputs are not as expected

ML Engineer → AI Engineer
Strengths: Knows how to fine-tune/train models, handle experiments, optimize accuracy, and can understand what’s happening under the hood if something breaks or outputs aren’t as expected.
Best Fit: If your business requires custom models or fine-tuning.
Risks: May not think about scaling, reliability, or costs. Might work slower than your software engineering colleagues.

_______________________

Building your AI team is likely to take a heavy dose of pragmatism and flexibility. Given the maturity of the role and the different technical skillsets that go into building AI products you’ll likely need to hire a team of complimentary engineers who can cover all of the above bases.

ML Engineers moving into the role will bring depth into the role, allow you to fine-tune models and be able to tell you what’s happening when things don’t turn out the way you expected. Software engineers moving into the role will help you deliver with speed and make sure things are reliable and work in production. Your data engineers will enable you to keep the right data coming into your LLMs and Agents and make sure your customer’s search queries happen quickly and accurately.

If you only have the budget for one hire, don’t look for a unicorn and ideally look for someone who can cover two of these archetypes, which two will be defined by your roadmap and exactly where you are in the AI product development process.