Tech Talk Interviews

Q: How do you move from AI demo to real-world impact?

To be able to move from a cool AI demo to an actual real-world impactful application that lasts long term, you need to think about AI not as a model but as a product. Think about everything that you need to define properly. Start by defining your business problem correctly and then think about what are the required AI tasks associated with it to achieve that business problem. Don't start from the model, start from the business problem.

To make it scalable, you need to think about your data readiness. You need to have consistent, complete, and reliable data. Once you have reliable datasets, you need to think about a modular, expandable model architecture that allows you to evolve your system as the requirements evolve over time.

Once you have a reliable system, you need to think about measuring business KPIs and model metrics. Think about how you can tie those together. Define very clear business KPIs that would demonstrate the success of your product while measuring model metrics that can be tied together.

Along with this, you need to make sure continuous model improvement happens along with clear adoption plans and clear feedback mechanisms set in place. When you have a very high performing model, it does not guarantee that the model will be adopted. You need to ensure seamless integration of these enterprise level scale model architectures into the existing business use cases, whether they're internal or external use cases that you're trying to integrate into.

Q: How did you scale your AI-powered matching system?

I can talk about a product matching system that I recently built at Staples. It's been in production for more than two years now. It's a high scale enterprise level product that is capable of performing product matching for coding tools as well as several other search systems that we have across our enterprise.

The main capabilities that we developed that made sure this product was productionized and has been successfully adopted 100% were having a clear data design. Once you have clear reliable data, have automated data pipelines that refresh that data so that your application and your data and your model outputs are not stale.

What we did was take a product query that a customer requests and we need to find the closest Staples catalog products that are mapped to that requested product. To achieve that, in terms of the technical solution, we first worked on query understanding. A user might request a query that is not standardized. It might be from a vocabulary that your model is not trained from. So you need to standardize that, you need to extract and make it meaningful.

Once you have a standardized meaningful query, we tried using a hybrid search methodology. Hybrid search is nothing but utilizing a keyword based mechanism, BM25, combining that with a dense retrieval embedding model something like MPNET. We fine-tuned that model on our high quality ground truth data that we have from our previously labeled human label data.

Once we had our fine-tuned domain specific embedding model along with this keyword based model, this hybrid approach gave us really high quality outputs. It ensured high recall but it was still prone to false positives and that's where we introduced an LLM based validation method. Once we have our top K candidates that we need to return in product matching use case, we look for any alignment between the query, the intent in the query, and the relevance of the results with the query to ensure only top K high quality candidates are returned.

We made sure to design this in a user centered approach. We created a first version, onboarded a set of users, created a pilot team and kept evolving the system, kept making them run actual A/B tests and try different features over time. That feedback helped us build something that is actually sustainable and provides a long-term solution. It also made sure all the system constraints are satisfied because every time you build an AI application you have to serve that, you need to be integrated into the actual business workflows. So we were testing it right from the UI. It also helped us evaluate the operational efficiency of the system.

Q: What hurdles did you face when scaling AI?

Generally speaking in product matching problems, the query can be ambiguous. The query can be too short. It might not be meaningful enough by default. Methodologies like query understanding really helped us. We try to extract the meaning out of those tokens and the words that you have from the query and expand that into a more meaningful vocabulary that aligns with our trained model's catalog.

Another bigger problem that we faced was that queries generally have very nuanced domain specific terms and these pre-trained models that are off the shelf available are generally not trained on that. They do not capture the nuanced domain specific vocabulary. For that we ended up fine-tuning our models.

In addition to this, one of the biggest problems is always to balance between latency and model accuracy. As you go towards advanced models, they become less and less scalable because they require longer time to run. Having a hybrid approach and a multi-stage retrieval ranking approach actually helped us solve this. Do the heavy lifting with faster models, but right before you serve the high quality results, build a nuanced ranker that can rank your top end candidates instead of millions of SKUs at the same time to give you the final top 10 that you probably need.

Q: How does AI transform customer experiences?

Scalable AI systems transform the customer experience by making systems less inconsistent than a typical system. It also helps you make faster decisions. With faster decisions, it also helps you get better quality results. Typically talking about the cross referencing product matching use case that we were discussing, it helps you get more precise matches higher up in your results and that naturally translates into saving you time and making faster decisions.

Businesses need to focus on certain things to be able to make the most out of these scalable AI systems. Think about having reliable data pipelines. Store high quality data which is consistent across systems. Enterprises should also think about clear business KPIs that matter to them. You need to define the business problem and the business KPIs that would allow us to measure over time the actual contribution of these scalable AI models.

Once you're able to measure the KPIs, you should also be able to think about building evaluation pipelines. Constant feedback being captured and measured through continuously trackable dashboards helps you get quantifiable results on what your true performance is and that increases the trust and reliability into the system.

Continuous model improvement is the key as users are involved in the process of searching and selecting through these different search systems. The important part is to provide and utilize this human label data. Providing these signals, capturing the feedback and utilizing these signals as future model improvement strategies is super important.

Models are likely to experience model drift over time as the queries and the domain expands. So it's important to track model drift and as it's identified you have set up alarm systems that can track it and keep retraining your model incrementally to be able to come as close to the domains.

I think businesses should focus on building small pilots that show the impact that a model or a system is likely to generate before they actually go for the kill. They should think about production grade design right from the beginning when they're developing a pilot. Building a pilot in the beginning does not mean you can build something that's lying locally in your computer. Think about how you can build it at scale. Think about model versioning. Think about stable data pipelines right from the beginning. Think about it as what business problem are we solving rather than the best model that we can build.

Going from the business problem with measurable KPIs to clear AI models that can solve that problem, a combination of models that can actually solve that problem is the key to success in long term.

Q: What's your advice for scaling AI projects?

I think balancing innovation with operational efficiency is super important. You could build a high performing model that does not meet the latency requirements of a practical system where the business problem is actually served through. So you need to ensure that operational constraints are met while you're innovating the system and think about the best possible approach that is likely to meet those operational constraints.

Change management is another piece that you should always take care about. Adoption, clear adoption strategies need to be set right from the pilot launch to the product launch to continuous feedback meetings that you would set up. So that model is firstly adopted and then it's continuously leveraged to be able to generate the business impact that it was intended.

Conclusion

Scaling AI from demo to long-term impact requires a fundamental shift in thinking. You need to treat AI as a product that solves business problems, not as a model that showcases technical capability. Build on foundations of reliable data, modular architecture, and clear business metrics. Design for production from day one, balance innovation with operational reality, and prioritize adoption as much as accuracy. 

The organizations that master these principles will move beyond impressive demos to AI systems that deliver sustained competitive advantage. As my experience demonstrates, the path from experiment to enterprise impact is challenging but achievable when you focus on solving real problems with scalable, user-centered solutions.