❌

Normal view

“A data lake of nuance for AI agents to swim in”: AWS Context gets shipshape on reasoning 

AI consumes a lot of data, but all-you-can-eat data in the world of agentic intelligence eventually loses its flavor. Simply increasing the sheer volume of databases, data repositories and data volumes does not necessarily enrich any given AI function’s ability to reason. 

Context, on the other hand, does.

We know that agents are only as intelligent as the context they have access to. In order to codify the inclusion of context into algorithmic logic for AI, technology vendors have extolled the virtues of using a knowledge graph to make their data more useful for agentic purposes.

AWS knows this story all too well. The organization’s sprawling datacenter footprint hosts vast pools of context on behalf of its customers, but that context often sits in a raw and unstructured form across data lakes, data warehouses, data lakehouses, databases, and data streams. It also permeates through the rarely-documented institutional knowledge that agentic engines rarely get access to.

All of which explains why the cloud giant used its AWS New York Summit today to introduce AWS Context, a new service that automatically maps the relationships that exist across a firm’s existing data into a knowledge graph and provides agentic search so AI agents in the organization can access what are governed data relationships, business rules, and domain knowledge at runtime. 

But weaving all this together is hard work. Knowledge graphs need more than simple keyword matching to work; they require structural and semantic traversal. This means they need to make multiple hops across various information silos and repositories so that they can aggregate context and (for example) be able to explain why cybersecurity vulnerability A is a factor of system compromise B, which has a core dependency link to codebase C, which executes in application D and risks taking users X, Y and Z offline. So how is AWS doing this?

A data lake of nuance & information

Mai-Lan Tomsen Bukovec, AWS vice president of technology (data and analytics) tells The New Stack that AWS Context provides a “data lake of nuance and information that AI agents swim in” to reason correctly and make the right decisions for the business. 

“This is no different from how humans work. When we take action, we depend on our own context about the domain, prior decisions and their outcomes, and other information.” – Mai-Lan Tomsen Bukovec, AWS.

“This is no different from how humans work,” Tomsen Bukovec says. “When we take action, we depend on our own context about the domain, prior decisions and their outcomes, and other information. With AWS Context, AI agents have all the nuance of every form of data in their business in a knowledge graph and in open data formats. AWS Context will make the difference between an AI agent simply taking an action versus making the right decision.”

Given the option to embrace this new service, software engineers will need to set out a plan of action and work out what to do first. For AI developers and data science professionals, this throws up the question of what to prioritize first when preparing existing enterprise data for context-aware agents using AWS Context capabilities and how they can control what data is (and isn’t fed) into the mouth of the beast.

Mercifully, it appears, options for control appear to exist.

“If developers want to exclude information from AWS Context, they will have the ability to prevent certain datasets, like test data or sandbox environments, from being included with AWS Context,” explains Tomsen Bukovec. “Because AWS Context is continuously updated as relationships between data resources changes, AI agents have the latest context available without any intervention from AI developers – and the control to set guardrails to exclude content that agents should not take action upon.”

Should developers trust this technology?

AWS Context extends the same knowledge graph technology that runs Amazon Quick, the organization’s AI work assistant that “connects scattered work” across applications and resources, including Slack, Microsoft Teams and Outlook, CRMs, databases, and documents.

So, should software developers place their trust here? After all, even once captured and connected, not all business context is useful. Some contextualizations could be corrupted, weak, fragmented and not productively useful for the business? Is AWS at risk of encapsulating context without considering how the data that comprises it is is quantified in terms of business usefulness?  

AWS has thought of this factor.

Because AWS Context uses the same knowledge graph technology that powers Amazon Quick, it can learn from usage patterns to make every interaction smarter. With AWS Context, the company says it is extending what was a personal knowledge graph into an organizational one i.e. a shared, governed context layer that agents and applications in an organization can draw from.

“Developers can govern and shape a dynamic and intelligent context layer that AI agents depend on to make the right decisions – AI agents won’t just get smarter as the models improve – they will be smarter because they have a vast amount of curated context at their fingertips.” – Tomsen Bukovec.

“AWS Context provides a data lake of context in graph and open data format,” clarifies Tomsen Bukovec. “That means that AI developers everywhere can use capabilities at the data layer to govern and shape a dynamic and intelligent context layer that AI agents depend on to make the right decisions. With this change, AI agents won’t just get smarter as the models improve – they will be smarter because they have a vast amount of curated context at their fingertips.”

Curated knowledge beyond a user’s personal graph

Existing Amazon Quick users will see that when AWS Context is enabled, Quick’s agents gain access to the broader enterprise knowledge graph, including cross-system relationships, business rules, and curated context that go beyond what any single user’s personal graph can provide. 

Tomsen Bukovec has also said that AWS Context gets smarter the more agents use it. As agents query the graph, it observes which sources produce correct results, which join paths agents rely on, and which curated rules get applied. It ranks sources by actual usage and shares what it learns across an organization, so when one agent discovers a correct join path or resolves a schema ambiguity, other agents pick it up, without requiring a human to re-curate the graph.

Any agent you put into production raises a governance question: what data can it reach, and can you show exactly what it accessed and under whose authority? The organization has explained that AWS Context answers both by making every query identity-aware.

Each call is designed to inherit the calling user’s identity access management (IAM) and Lake Formation permissions, so an agent can only see and traverse the relationships its identity is authorized to access. Because access runs through identity, every interaction is auditable. Security and compliance teams can verify what an agent accessed and under what authority, using the same controls.

AWS Glue Data Catalog

Related news to the arrival of AWS Context today saw the company also announce the preview of business context and semantic search functions for AWS Glue Data Catalog, the company’s centralized metadata repository for all data assets across various data sources. The new functions are designed to make it easier for humans and AI agents to discover and understand data. 

Also in this product stream, AWS now offers offer a preview of skill assets in Glue Data Catalog, a service designed to allow “data producers” (a somewhat arbitrary term that AWS applies to anyone who creates data, but is most likely a DBA or developer) to create skill assets. 

Associating skill assets to data assets gives agents additional context and instructions they can retrieve progressively for working with specific data without re-teaching it to every agent one prompt at a time. 

A renaissance of context engineering

Will this new drive from AWS herald the birth (or perhaps renaissance, the industry has been talking about this approach for some time) of context engineering as a sub-discipline of data science? It may well do… and if it does, it will likely drag role-based multi-agent orchestration along into the fray with it as we weave ever more complex interrelationship structures through enterprise data stacks.

If AWS or indeed the other hyperscalers or major frontier model companies starts acquiring more multi-model graph structure companies and vector database specialists, that could be the sign that things are cementing around context engineering at large. 

In the meantime, developers setting sail on the contextualized data lake of nuance are advised to wear a life jacket.

The post “A data lake of nuance for AI agents to swim in”: AWS Context gets shipshape on reasoning  appeared first on The New Stack.

“Agents need boring infrastructure around them”: Why we need to take an interest in ‘invisible’ AI

AI is already inside most enterprises’ IT stacks, but it’s had a somewhat shambolic and unsystematic early adolescence. Employees use personal tools, teams adopt different models, different company departments get forced into corners by vendors who push closed stacks, and agents are beginning to act inside systems that were built for people. 

That makes AI invisible, fragmented, and hard to change later. 

AI access and control platform company Tailscale announced on Tuesday the results of its work to address and redress these imbalances with new capabilities for Aperture, the company’s flagship toolset designed to provide a stable layer for managing AI across changing models, tools, data sources, and agents.

Designed to enable software developers to control and orchestrate the arguably almost too-dynamic state of AI, Aperture now offers a new chat interface, universal data connectors for both MCP and APIs, and sandbox support. 

What makes agents useful, also makes them risky

Avery Pennarun, CEO and co-founder of Tailscale tells The New Stack that the “same mechanics” that makes AI agents useful also make them risky i.e. they can do in seconds what would take a person dozens of clicks, commands, and context switches. 

But he advises that the risk factor here is not really a matter of pitting humans against agents and trying to place one above the other in terms of potential fragility. He says that the real risk is “giving any actor too much room” to act without clear boundaries.

“With agents, that risk moves faster,” Pennarun says. “With humans, the weak point is often the control model itself. If security depends on a developer approving a long stream of prompts, they will either get slowed down or hit approval fatigue and start approving things by reflex. That is not much of a security model.”

“Agents need boring infrastructure around them – robust identity management, limited access controls, carefully tracked logs, and sandboxes – that boring outer shell is what lets them do useful work without making every developer’s laptop the place where all the risk lands,” Avery Pennarun, Tailscale CEO.

Interestingly, agents need boring infrastructure

For Pennarun, the answer lies in making sure agents have what he calls “boring infrastructure around them”, by which he means robust identity management, limited access controls, carefully tracked logs, and (where necessary) sandboxes to execute in before they are exposed to mission-critical datasets, applications, or both.

“That boring outer shell is what lets them do useful work without making every developer’s laptop the place where all the risk lands,” Pennarun clarifies. “The answer is not agentic control or human control alone. Humans set the policy and boundaries up front. Infrastructure enforces them. Agents operate inside them.”

Aperture can be defined as a centralized AI gateway built to monitor and route LLM requests in a secure manner using Tailscale’s identity layer to automatically authenticate “users” (a cohort which we now obviously expand to include both humans and machines), eliminating the need to distribute API keys to authenticate with each AI model.

The gateway holds the API keys securely, meaning that when a developer (or a container) makes a request, Aperture verifies who they are via their Tailscale identity and then automatically routes requests to upstream LLM providers such as OpenAI, Anthropic, and Google without requiring changes to existing tools or workflows.

Yeah, we use AI, dunno where

Given the amount of work-related activity currently happening on personal and free AI accounts, we might suggest that concerns here are validated i.e. organizations today can not see, govern, or recover the information streams at this level. Research cited by Axios found companies typically have 67 generative AI tools running across their systems, with 90% lacking proper licensing or approval. 

Tailscale has reemphasized the fact that AI providers are bundling models, chat interfaces, data access, and execution environments into closed stacks. Those bundles can make the first deployment easier, but they can also leave organizations locked into one provider’s models, tools, and roadmap and pricing. In a market where model quality, speed, and cost keep changing, that lock-in can quickly become a disadvantage. 

“Aperture is built to give developers a practical way to manage AI without locking down their choices. It makes approved AI tools easier to use, connects them to internal data with identity preserved, and gives agents controlled environments to work in.”

“AI agents are also changing the risk model. They can write code, call tools, browse systems, manipulate files, and run commands. In many setups, they do that with the same permissions as the person running them, which can expose local files, credentials, and internal systems if something goes wrong,” said Pennarun and team.

What it means for developers: a controlled environment for agents to work in

Aperture is built to give developers a practical way to manage AI without locking down their choices. It makes approved AI tools easier to use, connects them to internal data with identity preserved, and gives agents controlled environments to work in. It also keeps the AI stack essentially modular, so teams can keep experimenting with new models, interfaces, tools, and providers without starting over.

The new chat interface is a browser-based way to use approved AI models through Aperture. The interface supports switching between configured LLM providers and works with Aperture data connectors and sandboxes. The universal data connectors help AI tools reach internal systems, documents, APIs, and operational data without forcing every team to build its own integration path.

Teams can use Aperture’s chat UI, coding agents, agent frameworks, or implement custom interfaces through OpenWebUI or LibreChat. Sandbox support (available in private alpha at the time of writing) is designed to give AI agents controlled environments where they can complete work without acting directly on a user’s laptop, workstation, or unmanaged system.

Aperture is designed to work with API keys from major LLM providers and with tools, agents, and interfaces that can be configured to route through Aperture. 

AI stacks inevitably, constantly and persistently change

With the frontier model race apparently unlikely to slow down any time soon, the fact that the best model, interface, sandbox, and data connection will all keep constantly changing… combined with the need to juggle these balls across multi-cloud deployment instances (poly-cloud even, where one app is split into different component parts across more than one hyperscaler), organizations looking to harness AI effectively and securely will surely face challenges. 

The central technology proposition with Tailscale Aperture is that it gives software developers a stable layer for identity, access, and control, so teams can keep changing tools without losing track of who is doing what.

The post “Agents need boring infrastructure around them”: Why we need to take an interest in ‘invisible’ AI appeared first on The New Stack.

Google, Microsoft, and OpenAI join forces to help create AI’s missing trust layer

A illustrated image of a robotic hand shaking a human hand against a warm orange background, depicting trust.

The Linux Foundation has long transcended its roots as a steward of the Linux kernel, emerging as a “foundation of foundations” spanning everything from cloud infrastructure and security, to digital wallets, and maps.

But the organization has been on a particular tear of late, becoming home to numerous AI-focused foundations and projects in the past twelve months alone, spanning agent communication protocols, agent security and governance, AI asset exchange, while on the foundation side there’s the Agentic AI Foundation (AAIF), the Tokenomics Foundation, and — now — the Appia Foundation.

The all-new Appia Foundation sits under the auspices of the Joint Development Foundation (JDF), a Linux Foundation entity that provides the legal and administrative infrastructure for organizations producing technical specifications and standards rather than code.

Announced on Wednesday, Appia’s mission is to produce open, modular specifications that give organizations across the AI supply chain a consistent, verifiable way to demonstrate that their systems meet the trust and compliance obligations placed on them — whether those come from regulators, customers, or international standards bodies.

Google, Microsoft, and OpenAI are among the 13 inaugural members, alongside a slew of industrial heavyweights.

A problem to solve

In most industries, proving that something’s safe is fairly routine. A new apartment block gets signed off by inspectors before the first tenant arrives. A kettle carries a safety mark because someone qualified tested it. The checking is so embedded that nobody thinks about it. AI has no equivalent yet — no common, recognized way for anyone in the supply chain to show that a system meets the bar, in a form the next party can actually rely on.

An example offered by the Appia Foundation illustrates how quickly the problem can compound in real scenarios. An AI tool used to screen job applicants wasn’t built by one organization: a developer created the underlying model, a second company adapted it for candidate assessment, a vendor connected it to the hiring systems, and the company’s own HR team configured it for their specific hiring criteria. The recruiters relying on it need to trust it’s reliable, while the applicants it screens want to know it’s fair. The company’s leaders need confidence it’s lawful. Regulators want evidence of how it performs. Each party is asking the same question — can this be trusted?

Today, most claims about AI trustworthiness amount to self-declaration — a company’s word that its system is safe, fair, or compliant, with no standardized way for anyone else to verify it. Craig Shank, executive director of the Appia Foundation, tells The New Stack that as a global, multi-stakeholder endeavor, the foundation is focused squarely on the “practical mechanics” of verifying an AI system against defined criteria, rather than merely stating that it’s trustworthy.

“Our specifications will enable transparent, attributable and traceable technical records of who demonstrated what against which criteria and when.”

“Our membership reflects the entire international value chain — the providers who build the platforms, the enterprises deploying them across critical industries, and the independent bodies that test them,” Shank says. “Our specifications will enable transparent, attributable and traceable technical records of who demonstrated what against which criteria and when. This is the exact type of objective data that courts, counterparties and regulators will need to determine where responsibility lies.”

The 13 inaugural members span a broad spectrum of industry — model and platform providers including Google, Microsoft, OpenAI, and Arm; industrial deployers including Siemens, Mastercard, Ericsson, Schneider Electric, and Mitsubishi Electric; and the assessment and governance bodies that will ultimately do the checking, including testing and certification firm Nemko, AI governance tooling company Naaia, and AI risk insurer Armilla AI.

A checklist for the age of AI regulation

AI regulations around the world are already moving from principles to active enforcement, and organizations are under pressure to prove that an AI system is safe and accountable. International standards bodies like ISO/IEC have done the work of defining what that should look like in principle, but translating that into something a regulator, a customer, or a procurement team can verify is another matter entirely. That gap is what Appia is built to fill.

The foundation will develop what it calls “conformity specifications” — modular, publicly available documents that translate international AI standards into concrete, assessable criteria. Think of existing ISO standards as the building code, and Appia’s specifications as the inspector’s checklist: the practical means of showing that a given AI system conforms to them.

A key feature of how the specifications are designed is what Appia calls “evidence pass-through.” Because AI systems are rarely built by a single organization — a model provider, an integrator, a deployer, and others may all have a hand — the specifications are structured so that conformity evidence produced at one layer carries forward to the next. A company deploying a third-party model, for example, wouldn’t need to re-establish what the model’s developer already demonstrated; it would only need to show conformity for its own configuration and use. Each party demonstrates what relates to its role, and no more.

The foundation is also explicit about what its specifications do and do not produce. Conformity — a technical result showing that a system meets defined criteria — is distinct from compliance, which is the legal status of having met a regulatory obligation. Appia produces the former; whether that satisfies the latter is down to the relevant regulator or jurisdiction. The specifications build on standards that already exist and produce the criteria that assessment bodies need, leaving the assessment itself to those equipped to perform it.

Appia is, by its own admission, early. The specifications are being drafted now in working groups open to all members, with initial focus areas including architecture, policy, and mapping the specs to existing regulatory obligations, among them the EU AI Act.

Jim Zemlin, CEO of the Linux Foundation, says that as AI regulation hardens into enforceable law, the industry needs somewhere neutral to do the work of building shared verification infrastructure — and that Appia is that place.

“The Appia Foundation establishes a neutrally governed environment where the entire industry can collaborate on a common assessment framework,” Zemlin says in a statement. “By building this infrastructure in the open, we are helping organizations reduce complexity, lower operational costs and build trust.”

The post Google, Microsoft, and OpenAI join forces to help create AI’s missing trust layer appeared first on The New Stack.

❌