From starting off as an undergraduate research assistant to a senior machine learning engineer, James Kunstle has worked at the intersection of research and engineering. Today, James leads the AI systems and training team in Red Hat Enterprise Linux AI (RHEL AI). It’s an extremely fast-paced field where the speed of technological progress requires a lot of continuous learning—but according to James, that’s what makes working in the field so dynamic and exciting. We chatted with James to learn more about his career journey so far and get the inside scoop on working as a machine learning (ML) engineer.
James’s interest in computers began at a relatively young age. “When I was around 14, I convinced my parents to get me a powerful computer for a CAD course at school. Technically, I didn’t have to have one, but I really wanted one because I was super into the custom PC thing, and all my friends were too. I picked out all the parts, watched guides, built it myself and had a lot of fun playing games on it.”
“But the unexpected hobby that came from it was getting really into benchmarking and overclocking the PC to see what kind of performance numbers I could get. At the time, it was all utilities—I had no idea how to actually program and wouldn’t learn until six years later,” says James.
A college class taught by a Red Hatter
James’s first real introduction to ML came through an undergraduate data science class at Boston University taught by a fellow Red Hatter, Lance Galleti. Through the Red Hat Collaboratory At Boston University, James began working with Lance as a research assistant during his undergraduate studies, exploring how to use adversarial ML to make models intentionally misclassify image data and, in turn, understand and omit the vulnerabilities of artificial intelligence (AI) systems. This was James’ first foray into the ML space, but he wouldn’t start working in the field full-time until a few years later.
As James progressed through his Master in Computer Science coursework, focused on computer systems and AI, he joined Red Hat as an intern, working on packaging utilities for Fedora CentOS. This was followed by two years as a software engineer in the Open Source Program Office (OSPO), where he worked with his colleague, Cali, to build Project Aspen—a community metric visualization project with the aim of better understanding how open source communities grow and interact. These projects helped James develop a solid foundation in data science and engineering, and he learned a lot, and fast, from his engineering leads along the way.
A chance encounter in the Boston cafeteria
James’s transition into the ML engineer role began with a chance encounter in the office cafeteria. “I was in the Boston office one day when two engineering leaders casually mentioned a project happening downstairs that needed extra hands,” says James, reminiscing about the early days of InstructLab. This chance encounter led James to InstructLab, where he worked on expanding its knowledge taxonomy.
That quickly evolved into a deeper technical role. “I was totally in the weeds for weeks. I was on a little three-person team that got InstructLab working on a MacBook for the first time, which was really cool. I even got a nice note from Matt Hicks at the end. I had no idea what I was doing at the time because I was so out of practice and hadn’t done that kind of work in a year. But we made it work, and from there, I kind of got swept up in the current—we were more or less informally asked to keep working on the project.”
Before James knew it, he was fully immersed in InstructLab and ended up at a career-defining crossroads where he had to decide between continuing his work in the OSPO or diving deeper into AI.
Ultimately, the opportunity to work on the leading edge of ML won out. “I was really wrenched in both directions because I cared so much about what I was working on in OSPO. It came down to the size of the opportunity. I really wanted to flex different muscles in my head and do things that were kind of my dream as an undergrad and a graduate student.”
Going all-in on machine learning
James’ current team focuses on model post-training, including supervised fine-tuning and hardware-influenced optimization. The MLengineer role itself is still relatively new, but James shared with us what the day-to-day of the role actually entails.
Firstly, according to James, a portion of their time is spent in meetings coordinating across teams. The model production, synthetic data generation (SDG) and core InstructLab teams are all tightly interwoven, so collaboration is key.
As James’s speciality is more focused on AI computing, he spends a lot of his time on new applications and developments within his particular niche. “It’s a lot of working in the upstream, reading code, learning how things are actually working under the hood and some of these more unorthodox implementations and optimizations to make things faster—reading the loss curve of the model as it trains, making sure that it is correct and it’s not going to break everything. A lot of debugging goes into that too.” The team also collaborates closely with their colleagues in the AI innovation team on the best ways to approach problems.
When asked about the ups and downs of the role, James mentions that there have been more highlights than challenges so far.
“One of the big moments was open sourcing InstructLab—we all got together in a conference room, and I bought champagne for everybody. As the corks popped, one of the engineers hit a giant foam button that open sourced the repo. Watching people start to trickle in was a huge moment.”
The team dynamic has also been a highlight. “I’ve had really great leadership along the way who have given me, and other engineers, the space to learn what we need to. There was a period of time where I was working a ton because I was so hungry for knowledge—moving to InstructLab felt like a once-in-a-career pivot, and you don’t usually get these opportunities to join right at the beginning of a project.”
One of the biggest challenges in the field, however, is simply keeping up. As AI continues to evolve faster than a person can absorb, continuous learning is key.
“I can learn something in a lot of depth and when I revisit that a couple months later, I’m way out of date. It’s a massive continuous learning undertaking. I think I’ve read more in the last year than I did during my entire undergraduate degree. The only tractable solution for handling that information fire hose is to specialize in something that your brain naturally likes.”
Solving tough problems—in the open
So what are those areas of specialization that keep James motivated to keep learning? James says that he’s primarily interested in compute—how to achieve faster inference and training, how to scale systems effectively and how hardware choices influence design decisions. “Really, I’m a technologist who loves using new open source tools to do things faster. Compute-heavy machine learning projects are fragile, layered and complex.”
For James, these are the most rewarding problems: experimental, difficult to fully grasp and often lacking clear documentation or established solutions. They require a willingness to navigate ambiguity, troubleshoot issues without obvious causes and learn continuously in a rapidly evolving field.
Ultimately, working in an industry in flux is also what James feels makes his role unique. “To not know where your field will be in two or three months is a pretty unique experience for an engineer,” says James. And working in the open source AI space at Red Hat allows him to be on the leading edge of these technologies.
“The most interesting projects out there are open source, and because I’m a Red Hatter, I’m immediately a peer in those communities. The default at Red Hat when there’s something interesting upstream is to start throwing our engineering weight behind it—making contributions, updating docs and looking at tests. There’s little resistance to trying new stuff and moving fast and getting involved. Red Hat has a unique position in the field and an upstream-first mentality.”
Looking to the future—but enjoying the journey along the way
James sees his career evolving in two phases. “Right now, I’m in the part where I have to get really strong technically. So I have a lot of fun as an engineer and especially as a machine learning engineer. I have a lot of great distinguished engineer mentors and people around who I learn from,” says James.
“Then, at some stage, I would love to focus more on strategic and technical leadership. I’ve had to work really hard to be good at engineering—my natural gifts are a lot more people-oriented and strategy-oriented, so I’d like to lean into that more.”
The AI industry can be all-consuming, so James leans on a handful of hobbies to decompress. “I love cycling. I also do a lot of urban hiking —I like to climb Capitol Hill and the neighbouring parks or Magnolia and Queen Ann where I like to walk around all the steep areas and hang out. I’m also a big baseball fan. I love the New York Mets.”
Advice for aspiring machine learning engineers
James believes that the fundamental difference between a machine learning engineer and a software engineer role is slim, but ultimately comes down to the former having a strong mathematical background.
“A lot of the work in our team can be done by strong software engineers. The only gap is when you have to make a mathematically informed decision about a specific niche, and for that, it helps to have done the coursework and be comfortable reading research papers,” explains James. “Can you read something like the DeepSeek paper, which blew everyone away, and gut-check what the paper says? If a model is exhibiting weird behaviour, do you know how to figure out what the problem is?”
A lot of this is teachable though, says James, and there are many engineers across Red Hat that have moved laterally into the AI field. “There’s so much space for everybody. You don’t have to know how it all works off the bat to be involved.”
More broadly, he believes that a knowledge of the AI space is essential for any engineer or technologist. “You can start by getting invested in any AI-focused community. Helping with tutorials and improving the docs is the easiest way to get involved—you just need to take the time to figure out how to do it and then make it easier for the next person.” (Like getting InstructLab to run on a MacBook perhaps?)
Our AI Engineering team is growing, and we’re looking for passionate technologists to join James in making AI technology available and accessible to all. Learn more about the team and explore our open roles here.
