Informa helps businesses and professionals in hundreds of ways.

Our international portfolio of live events, world-leading research publications, and innovative digital services provide specialists with the knowledge and connections they need to thrive.

The Big AI Debate: Episode 2

Can Innovation Exist Without Governance?

Panel:

  • Host: Sejal Patel, Senior Conference Producer, The AI Summit Series
  • Nilesh Kikle, Director Enterprise AI & Automation Delivery and Engineering Services Lead, Pfizer
  • Alex Golbin, Transformation Executive - Technology, Data, GRC & Resilience, Northern Trust

November 4, 2026. 10 min read

Is regulation the thing that holds innovation back, or the thing that keeps it going? In this episode of The Big AI Debate, host Sejal Patel sits down with Nilesh Kikle and Alex Golbin to test how far AI can move without rules, who should set them, and what is lost or gained either way.

Expect real examples, sharp questions and a few disagreements, plus a lightning round where the guests have to commit to a quick answer.

Can Innovation Exist Without Governance?

Governance isn't a brake on AI innovation, it's the steering wheel that keeps it on the road. That's the central argument explored in the latest episode of The Big AI Debate, hosted by The AI Summit Series, where AI practitioners Nilesh Kikle and Alex Golbin go head-to-head on whether fast-moving AI teams can survive without formal oversight.

Hosted by Sejal Patel, the episode opens with a lightning round designed to expose gut instincts before either guest has time to hedge. Ship a model that scores 95% in testing, or hold it for another month? Kikle argues for more testing, "accuracy doesn't mean you got everything right in terms of guardrails." Golbin takes a more nuanced stance, tying the decision to risk classification: critical use cases warrant caution, low-risk ones can ship and iterate. The round continues through a string of sharp calls, a 2% error rate on customer-facing answers, who holds final veto on launch, whether real customer data belongs in a pilot before review is finished, and on almost every point, the two converge: risk level decides the answer, not speed alone.

Is governance a brake or a steering wheel?

Kikle reframes the debate early: "I consider it not as a break in the systems, I consider it as a steering. It allows us to give direction." Golbin, author of Governing AI Risk, agrees that governance "cannot be a bad word, an office of no" but insists it must be enforced in code, not buried in policy documents nobody reads. His River Charter framework, resolve convergence, integrate hybrid cognition, validate risk debt, elevate data capital, reinforce resilience, underpins his view that governance has to evolve for the AI era rather than copy legacy models wholesale. Both agree that governance done badly absolutely slows teams down; the real distinction is between genuine governance engineering and what Golbin calls "governance theatre," endless committees and signoffs that exist to be seen rather than to work.

What happens when teams skip the review process?

The conversation turns practical with a hypothetical: a five-person team ships an AI tool with no review board, locked in for three months with only two safeguards allowed from a list of five. Kikle chooses identity and access restrictions, defining the system's "blast radius" plus full decision logging. Golbin picks human approval for high-impact actions plus the same access controls. The split reveals a deeper agreement: both want traceability baked in before deployment, not bolted on after something breaks. Pushed further on a realistic scenario, an HR chatbot built in an afternoon with no sign-off, both guests flag the same failure modes: model drift, unclear data provenance, and a lack of clear accountability when something goes wrong.

Team A vs Team B: speed or scrutiny?

A sharper scenario follows two teams at the same company build the same AI agent to summarize documents and draft replies. Team A ships in ten days with minimal review, hits 70% adoption within a year, suffers one data leak, but fixes it within two weeks. Team B spends five months on compliance, documentation and sign-off, achieves zero incidents, but only 22% adoption, and leadership is questioning whether the investment was worth it. Both guests initially back Team A for low-risk use cases. But Kikle flips his vote when the stakes change: "What if their use case... is to summarize incoming regulations, and one of them goes to the wrong person?" At that point, both agree the calculus shifts entirely to risk severity, not just speed of delivery.

Where do the two experts actually disagree?

Despite near-total alignment on principles, their real tension lies in framing. Golbin resists calling AI "technology" at all, arguing the industry's blind spot is over-indexing on the machine side of the equation and under-indexing on the human side. He prefers to see it as a "cognitive handshake" between human and AI. Kikle counters with his own term, a "human-machine symbiotic relationship," insisting the technology is a vehicle, not the destination. Both land on the same closing point regardless of terminology: sustainable AI, human judgement, and clear accountability will matter more than raw velocity as autonomous agents scale and token costs compound.

The takeaway

The episode makes a clear, quotable case: governance done right doesn't slow innovation — it compounds confidence over time, provided it's built as engineering practice rather than paperwork. As Kikle puts it, the discipline you build on your first use case doesn't have to be repeated from scratch on the next one; the rules, stakeholders and expectations are already established, so the process accelerates rather than drags. For teams scaling AI in regulated environments, the real question isn't whether to govern, but how fast governance can be made to move with you. The episode's key takeaways:

  • Governance compounds rather than repeats: early investment in rules and stakeholder buy-in makes each subsequent AI use case faster to clear, not slower.
  • Engineering beats paperwork: governance embedded in code and workflow outperforms policies that exist only in documents nobody reads.
  • Speed and oversight aren't opposites: the real trade-off is between genuine governance and "governance theatre," not between innovation and control.
  • Regulated teams aren't slower, they're more honest: accepting the true cost of failure up front is what makes long-term velocity possible.

[00:00] Welcome to The Big AI Debate, where leading minds tackle AI's biggest question. I'm Sejal Patel, this episode's host, and today we're asking, can innovation exist without governance? AI teams are being told to build faster, ship sooner, and figure out the guardrails along the way. But governance has a reputation for doing exactly the opposite of

[00:30] Slowing people down, adding approvals, and killing momentum. So here's the question Is governance the handbrake on AI innovation or the reason innovation can survive at scale? Joining me today are two well known AI practitioners Milesh Kikle and Alex Golbin. Disclaimer The views and opinions expressed by

[00:59] during this conversation are for discussion purposes only and do not necessarily represent the views or positions of the companies, organizations, or institutions the participants are affiliated with. Let's start with a quick lightning round. I'll give you two options. You'll pick a side and defend it. Ten words or less. A model scores 95% in testing.

[01:28] Ship it this week or hold it for another month of testing. Nilesh, you go first. I would say test it more before you release it, because accuracy doesn't mean that you got everything right in terms of guardrails.

[01:47] Alex? I'd say ship it based on the type of use case. If the use case is considered critical based on the risk rating, then I would side with Nilesh. If it's a low risk, go ship it iterate and learn from it. Governance that lives in a policy document or governance that is enforced in code.

[02:15] enforced in code. It's all about governance engineering. AI is different. It's not what we're used to. But that's the only way for us to have a sustainable future. Yeah. I would say the same thing is again not on a different side this time because having it documented doesn't mean that it exists. It has to live by itself and people should see it. It's working. Okay. Next question. I

[02:43] Two percent error rate on customer facing answers. Acceptable or not? Again, it would depend on the use case risk. If it is a very serious healthcare related application, that risk or that decision could be catastrophic. That's my take. Some application needs high accuracy rates. I agree. Two percent sound generally pretty high for any type of high volume use case.

[03:12] By based on criticality, I could change my answer. Final veto on launch. The engineer who built it or an independent risk owner? I think it would be the risk owner because engineers' focus is different. Again, everyone works as a part of the cross-environment teams, but still there should be one accountable person, which I would say the risk owner should take the call.

[03:40] Risk owner, agree with Nalash.

[03:44] Okay. Real customer data in a pilot before the review is finished. Yes or no? I would say yes. The review needs to be finished, looking at the data.

[03:59] Based on the type of use case could vary, but generally yes, I agree with Nalesh.

[04:05] A decision that AI can't explain. Block it or allow it and monitor it. I would say block it because explainability is super important, especially for creating the trust with end users. End users needs to understand how you have reached to that conclusion.

[04:25] Yeah, I'm not supportive of any type of black box. Explainability is so essential, so blocket. A six month review process. Ever justified or always a sign of something is broken?

[04:42] It's a proactive way to make sure that things are working. They are in check. So it's always a good thing to proactively and monitor the stuff because every feedback is a gift and it's a signal to make sure that direction is aligned with the governance, the whole linears of the things that we are doing. So definitely it's always a good thing to do it, irrespective of the outcome. We are designed to basically follow that outcome and adjust our pace, speed, and everything else around the governance.

[05:13] Proactive monitoring, control testing, observability, traceability. Six months, it's fine. If, Sejel, you are saying that before you go live, take six months to do a review. Definitely not. something that would be very, very wrong with the governance process. You gotta iterate quickly, learn from it, and then do recursive learning from the business outcome perspective. Okay, very interesting.

[05:41] There were some disagreements and some agreements, so that was I think a very interesting lightning round. So now let's start with the word itself. When you hear governance, what comes to mind? Because for a lot of people it means more meetings, more approvals, more paperwork, basically a slower way of getting anything done.

[06:11] Is that a fair criticism? Nilish, you go first? My take Sejal on the governance when I hear it, I considered it not as a break in the systems, I considered it as a steering. It allows us to give us a direction. It tells us what are the rules of the road, so you can maneuver that particular journey very well and reach to the destination. And again, it's not the destination that you want to reach. It's a continuous journey, as we earlier said. It's a proactive monitoring. So to me

[06:41] In one sentence, governance is a steering. It's not a break. I named my latest book, Governing AI Risk. So I had spent a lot of time thinking about the word governance and what that means and what it means for the future of enterprises. I think that governance needs to really be different in the age of AI for us to have a sustainable enterprise success and business outcome.

[07:09] Governance cannot be a bad word, an office of no. Governance has to drive adoption, it has to be enabler, but yet still has to have enough stature where you could be empowered to say no. So governance will be a strategic business differentiation for most of the enterprises to adopt AI. Okay, interesting. Nilish, you're the one actually building

[07:39] and shipping things. What does governance feel like from your side of the table? So for me, I totally agree that innovation is possible without governance, but at the same time, we really don't want to build the products that is like POC based or lab-based. We need to be in the production. We need to scale it. So governance that I take it is a way of building the things as Alex rightly said, with AI,

[08:08] Things are slightly different than our legacy ways of working. Because this technology is a very different technology. It's a very novel technology, very powerful. So the guardrails and every whole nine yards of ethical values, responsible AI comes into this picture. So I would say building and shipping has to have this foundation, solid foundation of governance. And as I said earlier, governance gives you this immense power for innovate because.

[08:37] You know the rules, you know how you want to adapt to those rules. So that is why the acceleration comes. So my take is governance is definitely required. And it's paradoxically sounds like a break, but I think that it gives you a very clear picture, the clarity, how you want to build a product which will be audited through internal audit and it will be accepted through the regulatory bodies. And that is what creates the trust and the value.

[09:07] Okay. Alex, is that reputation deserved?

[09:12] Traditionally, governance has a bad name. People think of going to governance as yet another thing that they have to think about. I think our opportunity as leaders is to rethink what governance is and what is part of that scope. I introduced a framework called River, River Charter. It has five pillars: resolve convergence, integrate hybrid cognition, validate risk debt, elevate data capital, and reinforce resilience.

[09:40] So governance needs to think about those five dimensions in a age of AI and really be very proactive, integrated, and I'm all for governance engineering and not just pieces of paper that nobody reads. Alex, what's the first thing that breaks when teams move fast without the right controls? And who notices it too late? So the story in my book takes place in twenty thirty eight.

[10:10] Traced decisions in 2026. So the question is how does the risk compound hidden vulnerabilities? And when do you determine that if governance is not fully in place because of this drive to acceleration? And that is why Sejil is just so important to put the right guardrails in place up front. Really look at across all of the dimensions.

[10:38] Make sure that there is traceability, observability, making sure that the right people are making decisions, that they're escalated, and it's not just governance theater, it's real governance engineering. No, I agree with you, but I feel that governance makes things slower. Do you agree or has governance ever made either of you faster?

[11:08] yeah, there is there are a million cases where governance makes things slower. That's that's by design. Please go to this governance committee before you go to this committee. Please go to the subcommittee. Please go to this working group. Please submit five pieces of paper. Please have it reviewed and socialized with everyone at your company. And the question is again, are you participating in the governance theater? Or is this real governance that's effective and decision oriented?

[11:37] So if governance is not done right, it would absolutely make things slower. Hundred percent. How about you, Nilish? What are you? I would say when you go to governance, definitely there is initial discussions happening at a l greater scale, but it's a compounding effect, as Alex said. You acquire the experience, the rules get clear, very methodical. So next time you acquire that acceleration, because you don't need to start from scratch and you know what people expect. And

[12:07] When you build governance with multiple stakeholders, their thought process, their inputs are taken into consideration. So overall, whatever you are building, it's democratic. Every stakeholder has a say, and this process has a compounding effect. So X amount of time that you take for the first use case may not be the same for the next use case. Again, everything is risk-based. So overall, I would say going through this process, which is adapted for AI.

[12:35] It doesn't slow down, rather it gives you a very confident, clear path of building the things towards the destination, which is passing the internal audits and getting them approval from regulatory bodies based on the domain in which you're releasing your use case. Let's use a more realistic scenario. A team uses AI to build a working tool in a single afternoon. Let's say it's a chatbot that answers HR policy questions. No review.

[13:05] No sign off, no documentation. It works. People love it. Six months later, what's broken? I'll go first. So does it have traceability, observability? What about the right level of controls? How do we know it doesn't drift, doesn't hallucinate, and is it even aligned with our objectives? So those controls are all really, really important.

[13:34] And not just day one, but it can all drift and change over time. I think that I would actually encourage teams building and going and and showing to people in the afternoon does the first thing. Because you gotta change hearts, not just minds. And people need to understand it, experience, and then you iterate, iterate, iterate. But doing that without controls, doing it without governance, doing it without

[14:04] a way that's sustainable, it's a recipe for disaster. Alex, one more caveat question to that. From a builder's perspective, what is the first thing that breaks when speed wins every argument? Drifting is pro with AI is probably one of the common scenario because you don't know what data was used to train the model. You don't know what is the hardness around it. You don't know the identity and permissions.

[14:33] So a lot of things can go wrong, but the most obvious one is just drifting over time. Okay. Malesh, you what are your thoughts? I would add more to what Alex said. I'm just basically doubling down on it. I think the accountability, the use case that you mentioned, it's the HR data policies that are coming to users and how they are absorbing it. If there is problem, who is accountable? Is it the builder? Is it the HR who approved it? So it is very important because

[15:01] Some of the decisions are financial decisions, some of the decisions are personal decisions. So a lot of different policies are included in the HR chatbot. And again, chatbot is a chatbot. Like it is not going out or doing a number of different things and giving you the output. It is mainly a grounded information. But again, as Alex said, data is super important. You gotta make sure that everything is grounded. Plus, someone is accountable if there is a conflicting message and drift.

[15:30] Over a period of time, things would change because the pattern in which people are coming could change, the way in which model is providing the answer could change, and number of things are there. So overall, yes, I would say if you release anything, there has to be all these pillars of governance and accountability, especially in case of HR use cases, super important. And Alex, when does a a quick fix quietly turn into a real liability?

[16:01] You're literally reading a chapter from my book where there was a demo that a temporary guardrails were turned off. And what happens? And then the liability only shows up years and years later once it discovered and really causes this fictional agricultural disaster in chapter one from a startup that I called fictionally Bull in the China Shop AI. Okay.

[16:31] Well, I haven't read your book yet. I might have to now. So to both of you, who notices when the prototype becomes production? I think stakeholders are the one if you are following the norms, but typically when digital builds the prototypes and they are so keen to put it into the market or user front facing, so they are the first one to put it. But typically I would say

[17:00] Every stakeholder has to be play a role in terms of decision when things goes to production. When I say stakeholders, they are people from business side, product owners, compliance, legal, as well as architects and engineers, because every person has a contribution to that system and together they can take a call, whether it is ready for production or no. Otherwise, it should stay as a part of POC or more of a demo more. And this is not new.

[17:29] So if you think about it, in financial services, we've been dealing with shadow IT, citizen developer, low code, RPA solutions, Excel spreadsheets, end user computing, user-defined technologies for years and years and years. And what's interesting is that what is the life cycle of that? So, hey, a smart developer in operations develops a spreadsheet with a bunch of pivot tables and macros, gives it to

[17:58] His or her team members, everyone starts using it, and then before you know it, it compounds, it's being relied on, it now becomes an operational system quietly. And what is that liability over time? So who would notice that? Well, you have to have governance processes for software development lifecycle, for and user computing lifecycle, and clearly for the AI governance. So things like this don't happen.

[18:26] And they get hidden behind the overall infrastructure. Seems like you're both in agreement in a lot of things when it comes to governance at the moment. So both of you work in industries where governance is not optional. Places where a mistake does not mean a bad quarter, it can mean real harm to real people. Does that kill innovation?

[18:54] Or does it force a different kind of innovation?

[19:00] So I'll start off with you, Nilish. What does it actually take to move fast when everything has to be validated, documented, and defensible? What I would say, Sejal, is that when you say that the solution that you're building, the risk case is high, you are definitely gonna work very methodical to make sure that whatever you're building is compliant and it is

[19:29] Gonna get approved by the regulatory bodies. And it doesn't basically make me slow down. It allows me to make sure that all the decisions should be adhered to the governance protocols and follow them. So I would say that it gives me an extra way of making sure that whatever we are building is gonna pass through all the internal audits as well as regulatory bodies. So I do not say that it slows me down, rather, it gives me some sort of a

[19:59] A way to make sure that yes, where it is gonna land eventually and what efforts that we need to build along all the guardrails and every possible dimension in which any governance is required for AI systems. So that's my take on high-risk use cases. Alex, do you agree with that? I would agree with Nilash with the part that you gotta do it right. So you do have that infrastructure that would minimize

[20:27] Having you being slowed down. But let's be honest. If I am in a financial services regulated institution, if I'm in a healthcare company that really is supposed to be compliant, versus I'm a fintech that just rolling out code and it has subject to very different set of regulations. Yeah, I can go a lot faster. I can go a lot faster because maybe I don't need to make an investment in.

[20:55] getting synthetic data. I can use live production data because nobody will stop me. And that will allow me to get quicker to iterate, innovate, and come up with a better product. Now, what is that hidden liability? That's a different conversation we explored earlier. But the fact is if you are in a regulated institution versus a startup that is just doing stuff, yeah, it is a different game and requires a different equation.

[21:25] So is a regulated industry slower or is it just more honest about what failure costs? I would say Sigil is that regulated industry exists, it's a part of the governance. It's like it's a subfield of governance. So it has to be there depending upon the risk case that you have for the your use case. So I would say it's always there. It should be there because if you're building any product which is like patient centric, you really have to make sure that it follows the guardrails and every

[21:55] norms of the product that is supposed to be handled by patients? Yeah, it's being honest about what it costs and it doesn't, and it really it's a forcing mechanism to make sure that you do not take shortcuts for sustainable success. It's just the reality of where we are. If a startup tried to operate like industries such as pharma or finance, would they survive or would they get

[22:25] out innovated before they shipped anything? Probably out innovated if I were to be honest. However, and I do quite a bit of work with startups helping them think about how to be AI ready, enterprise ready. And the way to think about it is to anticipate those challenges up front. Not after the fact when you were trying to do a sale, but it's really part of your overall design and that

[22:53] can make you successful, that can make you unbreak you. Because one of the things that startups need to understand and they need to learn that when they sell to enterprises, especially the ones that are regulated, it's a different way of thinking. And if you embed it as part of your product methodology, that's not just an administrative challenge that will actually be a big success and accelerator because you will make it

[23:21] Easier for enterprises to consume your innovation. I would agree with Alex and I'll just double down on it that any startups who want to basically sell something to enterprises, big enterprises, AI transformation within themselves is super important so that they can sell that particular transformation to an enterprise company. So it is very must that the startup companies focus on the way of doing things. They are not traditional way of building the software. When you build

[23:51] things or systems with AI, you have a very different mindset, culture, the people who participate in building the products and your understanding of the product actually, because this is a very different technology that we are talking right now. And it is a evolving technology. It is evolving at a rapid pace. I think everyone knows that any changes happening three months down the line, that change is obsolete. So

[24:17] Ev it is a constant journey, constant learning. So this is super important for startup to understand that they have to innovate, but but at the same time follow this trend that understand this technology and provide regulated industries needs that particular governance or that kind of discipline when they want to sell the technology to the big enterprises. I hope Alex you would agree. I would definitely agree if you don't mind, say Joel. I would like to just add on one one thing. And this is I think

[24:47] A big blind spot in the industry. And the Nelash you and U answer use the word technology like three or four times. So I would actually argue this is not technology. I don't think the way we should be thinking about it, even though a lot of us grew up as engineers, this is just a smarter technology. So it's about human and AI working together in a very collaborative way.

[25:13] And often we are thinking about the one side of the problem, which is how do you make your AI more reliable? How do you make sure that it has the right data integrity, the right data? But who is checking up on the human part of the equation? Because this is not a technology. This is an ecosystem. I call it cognitive handshake in my framework where human and AI collaborate together in a way that unlocks Dynx productivity, Dynx critic.

[25:42] Creativity, not just empowerment, but inspiration. So those who really look at it from that mindset, I think will really differentiate themselves in the marketplace. Yeah. And and Sejal, if you allow me, I call this as a human machine symbiotic relationship. And I do agree, when I say technology, I don't mean that it's the destination. It's a vehicle. That's what I call in my write ups that I do. It's a vehicle.

[26:10] That allows you to reach your goal, which is humans and machines working in a symbiotic collaborative environment. You are going to augment your intelligence. It's not going to take your intelligence, it's gonna augment your intelligence. So that's how this technology is gonna help you as a part of your journey. It's a vehicle, I would say. It's not a destination. Okay. Well, now we have a little bit of a heated debate going on. So

[26:38] Let's look at a case hypothetical, but built from real patterns. Two teams, same company, same task. Build an AI agent that summarizes documents and drafts replies. Team A ships in ten days, basically logging, no formal review. Team B spends five months on testing, documentation, and sign off.

[27:08] Fast forward twelve months. Now team A have seventy percent adoption across staff, four incidents, one leak, confidential text to the wrong person, but they responded fast, added controls, fixed the root cause in two weeks. Tool is now stable and embedded in daily work. Team B have twenty two percent of adoption.

[27:37] Zero incidents, but a vendor tool launched nine months ago, and leadership is questioning if the investment was worth it. Which team would you rather be on? Nilish, you make your case first. I would say, Sejil, based on the risk factor involved in this particular conversational use case, again, the team A, if

[28:06] It's a digital assistant talking to data and giving it back to the users. And there is no risk involved in terms of critical decisions. It is not affecting the brand or reputation or any financial losses. So I would say I'll take the stand for team A because it's a low risk thing and you are able to ship it faster. But at the same time, you are saying that I'm gonna learn, assuming that use case is low risk as things are in the lifecycle and I'm gonna fix it rather than

[28:36] Putting all the nine yards of things for a low r low risk use case. So that's what I would take the stand for team, knowing that it's a low risk. Okay. Alex? I think Team A, I think one of my frameworks is called Current, River Current. it's all about the speed of iteration. And you can have a very safe and sound company or a product that's not adapted. And guess what? It's just not very useful.

[29:06] So it's all about adaptation. And the way to get to adaptation is to iterate quicker. But to iterate quicker, you need to think about those controls up front. You need to think about how will people believe you that they'll actually want to do it. So integrating that in, you can be the smartest person in the world, and you can try to predict user behavior. And guess what? I can be a mediocre person and I can just do quicker iterations and I'll learn from actual experience.

[29:35] And it will get better adaption. So as long as you're taking a risk-based approach, as long as you are conscientious about controls and risks that you are taking, I would certainly vote for team A. You both picked Team A. Now, Nilesh, make a strongest case that you can for the other team. Steal man it. So team B, assuming that if they say that this is the use case that has got

[30:05] risk element that is higher than the low risk, say minimal, medium risk or high risk. I think they can definitely defend their initial early wins could be on a slower pace, but eventually I would say team B is going to follow all the governance norm. It's a high risk case. They should be able to have the stakeholder statement showing that every party agrees that it is slow to the market or adoption is lesser. But in the long run, the compounding effect would be there.

[30:34] If the technology changes or the new rules or regulations are coming, I would say they team B would have the advantage of compounding effect. And again, everything is determined by the risk of the use case that they're trying to build for Team B. Okay. So this is to both of you. What would have to change for you to switch sides? The severity of the incident, the speed of the fix, or the adoption gap?

[31:05] Those are three good factors that would definitely drive the equation. So, say Julie, in the example that you gave us, where you kind of flip the switch and said, Well, what if I can tell you something that would make you vote for team B? And what if team B is working in regulatory change? And what if their use case in summarizing documents is to summarize incoming regulations? And what if

[31:32] One of the regulations that the summarize they send to a wrong person, and ultimately the company is no longer in compliance with the global regulation and now get now gets fined billions of dollars and also has huge reputational damage. If it's anything like that, I vote for team B. And that would mean you really look at the severity of the use cases from a risk-based approach. And then of course.

[32:01] the adaptation variable as well, because if whatever you build between team A and Team B just becomes so cumbersome that nobody will use it, well, that will be a big equation. But it doesn't sound like that's the way you portray Team B. It sounds like it's more about the risk associated with failure and also severity of the incident. Like illustratively, that's how I just described it.

[32:31] The AI Summit Series brings together people shaping the future of artificial intelligence. The next AI Summit takes place in New York on December ninth and tenth. Click the link in description to find out more and grab your seat. Let's make this practical. Imagine a team of five building an AI tool. Small team, no risk department, no review board. They launch on Monday.

[33:00] But here's the constraint. They have time to put exactly two safeguards in place before they go live. After that, they're locked in for three months. No changes, no additions. So the question is, which two of the following do you choose? Here are your options. Log every decision the AI makes.

[33:26] A human approval step for high impact actions, a test set that is rerun before every change, strict limits on what the AI can cause the AI can access and touch. One click rollback. Nilish, you go first. Which two do you pick and why? I would pick two.

[33:53] The la second last where we control what is the territory in which AI system is acting. basically the identity, it's a guardrail or the radius, blast radius. And the second thing is logging. Because if despite that anything goes wrong, you still have the dump of how you have reached to the decision. So these are the two important things if I have to pick up out of five. Alex, your turn. Which two and defend them. Yeah. So I would take the

[34:22] High risk human decisions that were made. That's that's really critical. And then I would put I would pick the one from the last that Nilesh picked as well for identity access management. What can AI access? Okay. So there's a slight split here. Nilesh, you didn't pick the one that Alex chose on high risk. Why not? The reason why I picked the logging is

[34:50] Even let's say humans are going to take the decisions. Certain times the backup information is the logging as well, how humans got into the loop at what time, what happened prior to that. So that's why I picked up logging is it's complementary, logging plus human in the decision. So I would say when Alex, you would agree with me that when humans are taking the decision, but by the time what has happened when humans got into the loop, there needs to be some visibility.

[35:19] So that logging is also important along with human. So I picked up the foundation, the starting point first before reaching to the human. Okay. Alex, same question to you. You left out login as List chose. What breaks first without it? Yeah, login is really important. The reason why I went with the human is because you said all. So the word all it was the qualifier.

[35:49] Because I think the best practice doesn't have to be to like really log everything, everything is to log the right events. So given if you would have given me three, I would have picked that one too. Or if you would not have said all and you would have said fit for purpose logging, then I probably would have gone with that one. Yeah. And Sejan, originally I thought you were gonna ask me pick up only one, but you said two.

[36:14] So that's what made a little bit challenging because what would be the second one to pick up when I picked up the logging? Well, I like to make it challenging here as best as I can. Okay, so we're coming to our final question, and this is in two parts now. So I'm going to ask the part one of this to both of you, and then we'll move on to part two. So

[36:38] Name one thing that will be standard practice in AI teams in five years that almost nobody does today. Alex, I'll start with you here.

[36:51] my god, you you always have such a good and challenging questions because you're asking us to only pick one. I would come back to the blind side that I mentioned earlier. I believe right now a blind spot is too much emphasis on the AI part of the handshake and not the human part of the handshake, where we have this whole methodology of human in a loop, human over the loop, please make sure that there is a human somewhere. And is it a human with a calculator?

[37:19] Is it a human with access to ChatGPT? The whole model right now does not work. Human needs to play his or her part of the equation. Who is checking the biases? Who is really making sure that there is the right level of judgment and qualification? The symbiotic relationship that Nalesh and I mentioned earlier. So I think that future AI teams that will be successful.

[37:45] will really have as many conversations about the human part of the equation as much of a technical AI part. And that's what's gonna be the future. Okay. Vilish? My take on the future, Sejal, is the hybrid implementation. Automation is wherever possible, because mechanical intelligence is not needed going forward. Whatever you can automate with the risk-based approach, automate it.

[38:15] And provide the opportunity for autonomy. Again, when I say autonomy, follow the guardrails of creating the autonomous agent, human in the loop, the most important part. Humans' judgment is super important going forward. So I would say it's a symbiotic relationship with humans and machines together, with the hybrid implementation of allowing automation where possible, allowing human judgment wherever it's needed, and follow the complete guardrails of autonomous systems.

[38:45] Where as I said, AI is a vehicle. It's gonna take us to the new destination, which we all are eager and we are taking careful steps. So these is two motto that I would say it's a vehicle and it's symbiotic relationship with human judgment allowed wherever possible. And automation also. Okay. Well, we have covered a lot today. Tell me one thing the two of you still disagree about.

[39:15] And whether this disagreement is healthy or dangerous. There were several things that we agreed, slightly disagreed. So overall, my take is that both Alex is a well-known person in terms of the compliance, the audit, regulations, and I really look forward for reading his book. But overall I would say is that this particular era of building the systems using AI.

[39:44] Is going to be a lot different. Every person needs to unlearn and relearn the things because they are not the classic way or legacy way of building the things. So I would say going forward, what is gonna change is your mindsets, the way you look at this technology, and it's kind of how you build it, making sure that it always has the internal audit pass, acceptance from regulators, and the customer adoption.

[40:15] Alex? I really enjoyed the conversation and I agreed with a lot of things that Nilesh said. If you are truly, truly, truly, truly forcing me to pick one thing that maybe there is still slight residual disagreement, I think just words like technology and I'm actually more around like, well, what is the terminology that we're using? Because that makes that puts puts your mind on a different mindset.

[40:43] Artificial intelligence, agents, technology. Those are all terms that I think need to start changing as we're gonna start driving AI adoption of the future. And it may not be called AI, it may not be called agents, it will be communities of agents that will be powered with a whole financial system and regulations. And we have to think about those entities in a very different way, and that will drive enterprise adaptation.

[41:13] So it's probably less about what Nalesh and I did not agree. I would actually say there were some topics in this 40 minutes that we did not fully cover that would probably be important, the different way of thinking about vendors when we talk about this. What about the impact of data quality? What about convergence of all the different disciplines that really need to be looked at, like cybersecurity and data risk, resilience?

[41:43] And what would that mean for the enterprises. Okay. And Sajal, I just will double down on this. Going forward, we all are working in a very different era of putting machines and humans together. And one of the points, Alex, you did allude to that, sustainable AI. When we build agents, a task that used to take X amount of tokens is now gonna take ten X tokens or three X tokens. Again, computers, people are trying to put

[42:13] systems compute in such a way that the cost of inference is lesser. But overall, when autonomy with agents increase, how many tokens we're gonna burn? So sustainable AI is one of the very passionate and dear thing to me, saying that progress, innovation is all good, but at the same time, humanity, human values, and sustainable AI is greater good for the planet Earth. So this is what one of the things that everyone needs to put to their heart that sustainable AI is one of the

[42:43] component of governance and everyone should be aware because in the end we are doing this for humanity, the better earth. So that is super important point I would like to address or add here. Then am I safe to say or ask that is this a real disagreement or just a difference in what you're each protecting I would say Sejil is that there is always a healthy debate, healthy argument. Everyone has a stance and

[43:12] Together I would say we address each other's gap concerns and make the system better, which can be happy camper on both the sides. Every point Alex said is super important and it's something that needs to be part of the future design or future ways of doing or building the systems. Alex rightly said that future will be storm of agents and with autonomy and everything coming around. So these are all the things that are gonna be super important that

[43:42] Governance is going to be super important going forward. And governance by mean is all ethical values, responsible AI values, which has got sustainability built in. So those are the things I would say are super important going forward. Alex? Yeah, I agree. I think we agreed on the principles, and I think it just we are looking at things from slightly different lenses, converging to the same outcome. How does this

[44:08] Become successful, how do we reimagine governance in the age of AI that will give us sustainability? Nalesha's point is better future of humanity. My thesis is more sustainability of the enterprises and the industry and the whole ecosystem. Because without the right level of industry level governance, and I'm not just talking about regulations, there is not gonna be a good place that we would enjoy.

[44:36] Okay, well, disclaimer. The views and opinions expressed during the conversation are for discussion purposes only and do not necessarily represent the views or positions of the companies, organizations or institutes the participants are affiliated with. Nilesh, Alex, thank you for joining me on The Big AI Debate. And to everyone listening.

[45:05] Thank you for being a part of the conversation. Here's what we're left with Is governance what really holds innovation back or what lets it last? The future of AI won't be shaped by speed alone. It will be shaped by what we build, what we choose to check, and what we're willing to be accountable for. And that's exactly why we need to have

[45:34] Keep having this debate. Before you go, we want to hear from you. Can innovation exist without governance? Vote in the poll, let us know where you stand, because this debate doesn't end here. It's happening in your teams, your companies, and in your code right now. Until next time, stay curious, stay inspired.

[46:02] And keep exploring what's next in AI. I'm Sejl Patel, thanks for listening.