IBM

Since the rise of generative AI, there's been a growing concern about the proliferation of models and what that means for security and compliance teams. As one of its New Product Introductions, IBM set out to build an AI Security product, as part of its Guardium data security portfolio. The company needed to understand what the new era of generative AI looked like in the world of security. I set out to get answers.

Goal:

• Understand the landscape of the generative AI market
• Understand how security teams are securing AI
• Understand how governance and compliance is affecting AI security
• Guage the viability of IBM's proposed AI security offering

Method: User interviews.

Participants: Eight customers, with job responsibilites that involve securing generative AI models.

The complimentary strategy enables current IBM customers who use Guardium AI Security and Watsonx.governance, to maximize the offerings and achieve robust a AI security and AI governance solution. This strategy was validated, as participants indicated that the synergy achieved would provide maximum value.

These insights are related to the general landscape of AI security and governance.

• Broad Recognition of Threats:
• The broad recognition of the susceptibility of AI interactions to specific cyber threats suggests a genuine understanding of the vulnerabilities inherent in these technologies.
• Cross-Industry Issue:
• The lack of visibility into AI models usage is a prevalent issue across various industries, indicating a widespread need for solutions in this area.
• Security and Compliance at Risk:
• Without adequate visibility, organizations face heightened security vulnerabilities and compliance risks, underscoring the need for effective management and monitoring solutions.
• Widespread Data Concern:
• The protection of sensitive training data is a major concern across participants, indicating the universal importance of data security in AI model training.
• Access Control Challenge:
• Inadequate control over AI model access is a widely recognized challenge, impacting all organizations. This highlights the need for universally applicable solutions.

“The AI security problem is huge. It affects everyone. Nobody can escape it.”

These insights are classified under the theme of Security Environment

Challenges in effective threat management

The difficulties expressed in detecting and mitigating threats highlight a gap in current capabilities, suggesting a need for advanced tools and strategies tailored to the unique characteristics of AI interactions.

"It's like we're practically defenseless with this enemy."

Need for comprehensive governance gains steam

The mention of AI-specific governance frameworks indicates a move towards more structured approaches to managing AI model access. This reflects an understanding that traditional security measures may need to be adapted or expanded to effectively manage the unique challenges presented by AI technologies.

"The GRC team is talking to us about the EU AI Act. That's great - but now we've got to figure out the details on the ground."

Proactive security measures exist for some

The implementation of AI security training and continuous monitoring points to a proactive security posture among some organizations. However, the varied levels of implementation suggest room for broader adoption and enhancement of these practices.

"It's gonna take us awhile to get there, but we're trying."

Data protection is at the core

The emphasis on encryption and anonymization showcases the need for robust data protection measures that can secure sensitive information against unauthorized access and exposure.

"We rely on strong encryption to help us with this problem."

Regulatory Environment

These insights are classified under the theme of Regulatroy Environment.

Widespread compliance concerns abound

The unanimous concern over governance and compliance challenges reflects the critical importance of this issue in the deployment and operation of AI technologies.

"We've got a lot of mountains to climb when it comes to compliance."

Need for dynamic governance frameworks is real

The emphasis on adopting comprehensive and dynamic governance frameworks suggests that static approaches are insufficient to address the evolving nature of AI regulations.

"We've got a lot of people working on how to govern our data. There's just so much to consider when it comes to these frameworks."

Global compliance complexity persists

The challenges associated with global compliance underscore the need for strategies that can accommodate the nuances of operating across different regulatory jurisdictions.

"This is all moving at different speeds across the globe."

Collaboration for compliance isn't optional

The reported engagement with legal and compliance teams underlines the necessity of interdisciplinary collaboration in navigating the complex regulatory environment surrounding AI.

"This is becoming more and more of a team sport."

Product Capabilities

These insights are classified under the theme of Product Capabilities.

Complexity demands simplification

The complexity of securing training data, combined with the necessity to comply with stringent regulations, underscores the need for sophisticated data security solutions and compliance tools that can simplify these processes for organizations.

"This is all way too much. We're about to have AI throughout the entire environment. We need simple and streamlined tools if we're ever gonna have a fighting chance to actually be successful."

Governance and tooling required to fight shadow AI

Strengthening governance and establishing clear policies are essential steps in mitigating the risks associated with invisible AI deployments. This approach must be coupled with tools for enforcement and compliance verification.

"Shadow AI is a big concern. We need set boundaries with some teeth in order to combat that."

Prompt injections drive fear

The specific mention of prompt injections as a concern by a majority of participants, indicates a need for specialized protective measures that can address this and similar threats.

"The havok that a prompt injection can wreak is dizzying."

Need for role-based access

The adoption of RBAC by several participants underscores its importance as a strategy for mitigating risks associated with inadequate access control. It suggests that more organizations could benefit from implementing or enhancing RBAC systems.

"RBAC isn't gonna solve all our problems, but it's an essential part of the foundation."

Content can drive understanding

Technical accuracy in terminologies and labels is appreciated, but there's a gap in how accessible these terms are to a broader audience. The feedback suggests a balancing act is needed between technical precision and user-friendliness, ensuring that users from various backgrounds can fully engage with the information.

"We've got a lot of different teams working on this. And everyone doesn't operate at the same technical level. Just give it to us straight."

These insights are classified under the theme of Product Viability.

Guidance is essential for customer success

While the effort to visualize AI security risks is appreciated, there is a consistent theme across screens for the need for clearer navigation and the ability to easily translate information into action. Participants indicate that while the foundational structure is sound, enhancements in guiding user interaction would significantly improve the experience.

"Pretty graphics are great, but I need actionable info first - that's table stakes for me."

A single pane of glass is needed

There's a clear demand for tools and platforms that offer comprehensive visibility into all AI models in use, including the capability to detect Shadow AI projects.

"I need a solution that does it all. I can't be adding another ten tools just for AI."

Visibility isn't enough

Across all screens, there's acknowledgment of the solution's potential to address significant security concerns around AI models. However, the call for more actionable insights and specific recommendations highlights an area for development, indicating that users are looking for not just awareness but also tools and strategies for immediate application.

"This is nice, but I need more. I need help actually fixing things."

High perceived value shows promise

The majority of participants believe the solution could effectively address key problems, highlighting its strengths in risk assessment and visibility. This indicates a strong market need for such solutions and a good alignment of the solution's features with basic user priorities.

"This is definitely something we need."

Vaired purchase intent underscores room for improvement

There are varying degrees of certainty about purchase intent, underscoring the importance of addressing user concerns, enhancing clarity and usability, and perhaps offering trials to solidify interest and intent.

"I mean, I like it. But I can't say that that I'm definitely sold at this point."

With our new understanding of the landscape, along with the user mental model of AI security and their unique needs, we built an entperprise-grade solution that allows security and governance teams together to secure any and all AI.

AI Security Dashboard

The AI Security Dashboard provides full visibility into the AI environment. Whether it's generative AI, shadow AI or agentic AI - discover it all and manage it under a single pane of glass.

AI SPM

The AI Security Posture Management tool discovers vulnerabilities and misconfigurations in AI - providing insight in what to remediate next.

AI Gateway

The AI Gateway tool enables the creation of global policies for prompt protection.

Integrations

We created integrations that provide synergies for security and governance teams - from Watsonx.governance to OpenPages - security and GRC teams are finally unified.

The AI Security endeavor was an eye-opening experience. While generative AI is indeed a new and fast-moving space, the level of confusion and frustration amongst participants was more than expected.

At the same time, it was seeing that extreme level of frustration that drove the team to create something great for our customers. It's a prime example of how empathy can drive innovation.