Problem & Constraints
Conversational AI and AI agents now span enterprise platforms, each with different approaches to accessing organizational knowledge. Making those experiences useful can require enterprise information to be exposed or replicated across multiple environments, creating fragmented governance, duplicated knowledge, and inconsistent approaches to identity, access, and lifecycle management.
At the same time, the employee experience should not be organized around where information happens to live or which system provides it. A restaurant employee might ask about training, a recipe, an operating procedure, or a problem that crosses multiple knowledge domains. The challenge remains supporting both focused assistants and broader agents capable of reasoning across those domains without requiring the user or conversational client to know where the answer should come from.
This is the same underlying enterprise problem explored in the original early-2025 architecture.
- Enterprise data control
- Enterprise knowledge needed to remain within company-controlled platforms rather than being replicated into each assistant provider's environment.
- Identity and authorization
- Internal and externally delivered experiences needed to use Microsoft Entra ID company credentials and RBAC-controlled access.
- Multiple enterprise domains
- Training, recipes, operational knowledge, and other enterprise information originated from different sources and needed to remain independently manageable.
- Flexible assistant scope
- The architecture needed to support both narrowly focused assistants and broader agentic experiences without requiring every use case to adopt the same reasoning or orchestration model.
- Cross-domain reasoning
- Broader experiences needed to reason across multiple enterprise domains and determine which knowledge domains were required to satisfy a request.
- Experience independence
- Enterprise knowledge and retrieval capabilities could not be tightly coupled to a particular assistant, vendor platform, or conversational interface.
Architecture Overview
The architecture organizes enterprise knowledge into domain-specific Foundry IQ knowledge bases, with lightweight Skills providing bounded REST API abstractions over each knowledge domain. Azure API Management fronts every Skill, establishing a consistent and secure access point while insulating consumers from the underlying knowledge and retrieval implementation. This allows Training, Recipes, Operational Knowledge, and future domains to evolve independently while presenting a common enterprise pattern for accessing governed knowledge.
Above the Skills layer, enterprise-controlled agents compose one or more Skills into Competencies that own skill selection, retrieval orchestration, reasoning, and response synthesis. Azure API Management provides a common entry point for conversational clients and allows each Competency to be exposed through REST, MCP, or both, depending on the capabilities of the consuming experience. Keeping reasoning within the Competency rather than delegating it to individual clients helps provide a more consistent interpretation of enterprise knowledge across different assistants, while the separation between clients, competencies, skills, and knowledge bases allows each layer to evolve independently.
Key Architecture Decisions
Abstract knowledge behind bounded skills
Use lightweight REST APIs to represent domain-specific knowledge capabilities, keeping consumers independent of Foundry IQ knowledge bases and the retrieval mechanisms behind them.
Separate retrieval from reasoning
Let Foundry IQ knowledge bases handle retrieval while enterprise agents own skill selection, cross-domain reasoning, and response synthesis, giving each layer a clear architectural responsibility.
Compose skills into competencies
Configure enterprise agents with only the Skills required for a particular domain or use case, allowing focused and multi-domain experiences to share the same underlying knowledge capabilities.
Centralize reasoning within the competency
Keep retrieval orchestration and response generation within the enterprise-controlled agent rather than delegating them to consuming assistants, reducing variation in how enterprise knowledge is interpreted and presented across experiences.
Standardize access through API Management
Front both Skills and Competencies with Azure API Management to establish consistent, governed access boundaries rather than requiring individual services or clients to implement their own exposure model.
Support REST and MCP at the consumption boundary
Expose Competencies through REST, MCP, or both so clients can use the integration model they support without changing the underlying agent, Skills, or enterprise knowledge architecture.
Keep clients independent of knowledge architecture
Use Competencies as the default interface for completed responses, keeping retrieval and orchestration complexity behind the enterprise boundary. Explicitly entitled clients may also call bounded Skills through API Management when they need a focused capability; knowledge bases remain behind those APIs.
Preserve enterprise identity and authorization end to end
Use Microsoft Entra ID and role-based access control across clients, Competencies, Skills, and knowledge access so enterprise identity remains authoritative regardless of the conversational experience or integration protocol.
Engineering Approach
The architecture combines managed enterprise knowledge, lightweight API abstractions, enterprise agents, and governed integration boundaries into a composable retrieval platform. Each layer has a distinct responsibility, allowing knowledge bases, Skills, Competencies, and conversational experiences to evolve independently while supporting both REST and MCP-based consumption through common enterprise security and governance.
- Application & Services
- Foundry Agent Service · Azure API Management · RESTful APIs · MCP Servers
- Data & Messaging
- Microsoft Foundry IQ · Knowledge Bases · Unstructured Data · Semi-Structured Data · Structured Data
- Patterns & Principles
- Retrieval-Augmented Generation (RAG) · Skills & Competencies · Capability Abstraction · Composable Capabilities · Separation of Concerns · Loose Coupling
- Release & Reliability
- Independent Skill Evolution · Independent Competency Evolution · Backend Isolation · Protocol Independence · Extensible Knowledge Domains
- Security
- Microsoft Entra ID · Role-Based Access Control (RBAC) · Azure API Management · Token-Based Authentication · Governed API Access
Results & Evolution
The current-day design preserves the original goal of making enterprise knowledge reusable across conversational experiences while reducing the amount of custom architecture required to support it. Foundry IQ provides a managed knowledge layer beneath bounded Skills, while enterprise-controlled agents compose those Skills into Competencies that own retrieval orchestration, reasoning, and response synthesis. REST and MCP exposure through Azure API Management allow those Competencies to serve different conversational clients without changing the underlying knowledge architecture.
Compared with the early-2025 design, the architectural principles remain largely intact while the platform capabilities have evolved. Knowledge remains abstracted from the experience consuming it, Skills remain independently manageable, and Competencies continue to compose bounded capabilities around a business domain. What changes is the implementation: Foundry IQ assumes more of the knowledge-retrieval responsibility, agent services provide a native home for competency-level reasoning, and MCP introduces a standardized agent-facing interface alongside REST. The result is a simpler and more interoperable architecture without requiring enterprise knowledge or reasoning to be delegated to each consuming assistant.
Architectural Perspective
Good architecture should preserve useful abstractions even as the technologies behind them change. Foundry IQ, agent services, and MCP simplify how this architecture can be implemented, but the underlying principles remain: clear boundaries, separation of concerns, and independently evolvable capabilities. New technology should serve those principles rather than dictate the architecture.