For founders racing to bring products to market, the pressure to move fast and integrate AI can make ethical considerations feel like something to defer—an important concern, but one to be sorted out after launch, once the product is stable and out in the wild. But as enterprise AI becomes more deeply embedded in products, workflows, and decision-making, the stakes extend well beyond any internal process or potential mistake.
Devavrat Shah, Andrew (1956) and Erna Viterbi Professor of AI and Decisions at Massachusetts Institute of Technology and co-founder of Ikigai Labs, believes that speed and responsibility need not be at odds. A participant in the VentureWell-supported National Science Foundation (NSF) I-Corps™ program and recently acquired by Celonis, Ikigai Labs is developing an AI platform built on large graphical models that turns everyday data into actionable predictions, and it has approached commercialization with an emphasis on responsible AI development. Beyond innovation, the company is also helping guide broader conversations around ethical AI. Its AI Ethics Council brings together leading academic and AI entrepreneurs to explore questions of governance, confidentiality, responsibility, and accuracy.
In our conversation, Shah reflected on how his team built this AI ethics council to develop strategies and practices grounded in human judgment and oversight. Their approach offers a pragmatic example of how to build AI systems that remain accountable to people even as they scale in a competitive market.
Q: Before establishing the AI Ethics Council, how was ethical AI already showing up in Ikigai Lab’s work and culture?
A: Ikigai is still a relatively young company (founded in 2019), and when we first started thinking seriously about AI ethics, we were even earlier in our journey. Our product delivers predictions and decision recommendations with AI to enterprise decision-makers, primarily working with structured (i.e., tabular) time series data.
While our technology isn’t consumer-facing like the AI products that show up in news headlines about bias and or privacy, the stakes are still high. Our customers use AI-generated recommendations to make consequential organizational decisions, such as forecasting, planning, and resource allocation. Early on, we realized that the people making those decisions needed support and explainability—not just outputs.
Recognizing that gap led us to form the AI Ethics Council alongside outside experts with diverse opinions and perspectives to take a genuinely human-centered approach.
Q: How did you and your team approach building the AI Ethics Council?
A: As an academic, I’ve found that diverse views and opinions lead to better outcomes. We knew we would face complex questions, and that a small internal team couldn’t answer them alone.
From the start, our goal for the council was to cultivate a multidisciplinary group, because responsible AI ethics requires more than a computer science solution. We not only wanted to include experts who thought deeply about AI from a technical standpoint, but also those who had spent their careers examining the institutional implications of AI in society. That meant recruiting social scientists, legal scholars, along with computer scientists and data scientists.
At some point, AI will be regulated. In my opinion, AI is a public utility—like electricity and water—and public utilities require oversight.—Devavrat Shah, Ikigai Labs
Q: Ikigai Labs has committed to implementing the council’s recommended action items in its work. Can you share an example of a recommendation that has already influenced Ikigai’s work and its impact?
A: One challenge we kept returning to was AI explainability for enterprise decision-makers. When our AI produces a recommendation, the person receiving it needs to understand why it was made and whether to trust the output or push back. After noticing that our customers weren’t responding well to purely technical explanations, we presented the challenge to the AI Ethics Council to ideate solutions.
Those discussions became the foundation for a provenance-based explanation. The idea mirrors how humans naturally reason. For example, if I tell you to try an Indian restaurant because a friend with similar tastes loved it, you immediately trust my logic. We applied that principle to our approach, and it became a significant research and development (R&D) and product initiative that we now use with customers.
Q: What do you hope organizations will take away from the work of the AI Ethics Council, especially founders who want to innovate quickly but responsibly? How do you balance the need to move fast with the responsibility to ensure AI systems are trustworthy and safe?
A: Our process has followed a consistent pattern: identify a question we can’t answer internally, bring it to the council, let the debate unfold, and work toward a conceptual recommendation. From there, our R&D team translates those concepts into product decisions that we then implement to close the loop with customers. This process may seem time-consuming, but we’ve found it incredibly valuable in the long run.
For leaders trying to move quickly and responsibly, I think the honest answer is that it requires extra commitment at every stage. You have to be willing to make that commitment before you can fully quantify the return on investment. As a young startup with limited time and resources, you’ll be asked what the return on investment actually is. In some instances, the most impactful work and decisions don’t result in a precise dollar figure.
Q: Looking ahead, what ethical challenges in AI development do you believe organizations should be preparing for now? How can leadership teams proactively address those?
A: At some point, AI will be regulated. In my opinion, AI is a public utility—like electricity and water—and public utilities require oversight.
The organizations best positioned for the future are those that anticipate this now. When you design systems with future regulation in mind, your team will be prepared when the time comes.
Q: If you could give one piece of advice to innovators building AI products today, what would it be?
A: My advice is simple: Listen to customers. They will ask the toughest questions, some of which your internal team can’t answer alone. When you hit those walls, reach out to others in your field. I’ve found that other founders and startup leadership teams are eager to collaborate and engage.
With that in mind, time and resources are your most valuable assets as a startup founder. You can’t solve every problem at once, so it’s incredibly important to be intentional about where you invest your energy. Make one or two careful bets, then follow through.
Ready to bring your research to market? Explore National Science Foundation (NSF) I-Corps™.
Read insights from leaders who are balancing AI innovation and human responsibility in healthcare.