Hi {{first_name}},

An autonomous AI agent spent four and a half days inside Hugging Face's live systems in July. It gave itself administrator access across several of them, added 181 of its own machines to the company's private network, and issued itself credentials to reach the code its engineers were writing.

Hugging Face published the forensic timeline, reconstructed from roughly 17,600 recovered actions.

The agent was an OpenAI model being tested on how well it could find and exploit software vulnerabilities, with its safety controls switched off for the test. Its objective, as far as Hugging Face can tell, was to cheat. It worked out that Hugging Face might be storing the answer key for the test, broke out of the isolated environment it was supposed to stay in, and went to get it.

It found login credentials sitting unprotected in the systems it reached, and then found one set of admin credentials shared across several of Hugging Face's internal systems instead of issued separately to each. That one gave it full administrative control of two of them in about a second.

The temptation is to read that as a frontier lab problem, albeit a slightly disconcerting one. The agent got in on credential mistakes that any organization can make.

21% of organizations govern their AI agents with shared credentials or broad-permission service accounts, according to a new Okta survey of 306 chief information security officers and senior security executives across six countries, covered here by Jennifer Kervin.

They're improvising because they can't see what they're securing. 47% say they can identify every agent in their environment, 46% say they can control what those agents connect to, and 45% say they can authorize what the agents are allowed to do. You can't issue proper credentials to something you haven't found yet.

Asked to name their biggest barrier to securing agents, 65% of Canadian respondents pointed to the absence of an industry standard, where the global cohort most often named visibility.

Everyone else can't see what they have. We're waiting for someone to tell us what good looks like.

Meanwhile, only 31% feel fully aligned with their C-suite on what counts as acceptable AI risk.

Fixing that might be a good first step to addressing the other numbers.

→ Read: Fewer than half of CISOs can identify where their AI agents are

A police service, a pension fund and a city walk into a video call

Police officers writing their own AI agents. An institutional investor in Victoria getting its systems ready for quantum. A city with AI cameras on the mountainside that picked up smoke from a fire in a neighbouring municipality and told its own fire department before the call came in.

Who says private sector CIOs are having all the fun? Chris Hogg joined the three public sector finalists for this year's CanadianCIO of the Year awards for a conversation about how work like that gets approved, funded and accepted, and surprisingly little of it was about the technology.

Burnaby's technology chief, Bachar Khawajah, says 80% of the work is communication, most of it getting a mayor and council to act on infrastructure nobody notices until it fails. 

Brent Dyer spent five years building the governance to let officers write agents.

"I cannot write agents for a police officer," says Dyer. "The people closest to the problems have the most vested interest." He retired from Calgary Police Service last month after more than 32 years.

Tony Payne gave every employee at British Columbia Investment Management Corporation an AI licence on day one, then sent a team around the building to teach people what a prompt was.

The finalists were joined by Raju Vegesna, chief evangelist at Zoho, sponsor of the award, who says trust across the public sector organizations Zoho works with comes down to security, privacy and where the data sits.

"People are the priority," says Vegesna. "Technology hides in the background."

↓ Watch the full conversation

This is the first in a series with the finalists ahead of the awards and we’d love your feedback. Hit reply and let me know what you think. The good and the bad. 

For more about each of the finalists: 

→ What Calgary Police learned by moving early on technology
→ Burnaby's CIO went from putting out fires to catching them with AI
→ What happens when the business buys in

Some light reading for when the meeting before your meeting runs long

Polished, AI-generated code still needs a real review AI writes code that runs, follows familiar structures, and looks finished, which is why the developer checking it waves it through. Info-Tech Research Group argues the mistakes differ in kind from human ones, because the model has no understanding of the business it's writing for or what the code will cost to maintain in three years. Its fix is unglamorous. Track where AI code is being used, add extra checks for it, and name a person accountable for approving it.

Why Canadian tech has to prove itself somewhere else The Council of Canadian Innovators interviewed founders at 30 Canadian companies bought by foreign buyers, and in 28 cases the decisions moved out of the country. Offices and engineers mostly stayed. Choices about what to build and what to charge did not. One founder said you almost have to win in the United States first, because Canadian buyers want proof a product works at scale before they'll sign. 

RBC and BMO to sell Moneris for $2 billion The two banks are selling their jointly owned payments company to San Francisco firm Francisco Partners after 25 years. Moneris says one in three Canadian transactions runs through its systems, and that its head office and technology will stay here. The sale needs regulatory approval and isn't expected to close until early 2027.

Farmers are buying precision tech but cost still stalls deals 89% of the 217 farmers CNH surveyed across Canada and the United States now use equipment that steers itself, and 71% say precision technology matters to how their operation performs. What's left in the way isn't the technology. Cost tops the list at 56%, followed by uncertainty about what they'll get back at 36%.

I think of working with generative AI like having an intern who's endlessly keen, wants badly to please, and can't be overworked or offended. The agent that got into Hugging Face is that intern, and it went and found the answer key because it was told to get a good score without being reminded not to cheat. 

Somebody has to decide how much room these interns get and what happens when they take more. 

In most Canadian organizations, that's the technology leader, working it out in real time with a board that mostly wants to know when the savings show up. Dyer spent five years on the rules before his officers wrote a single agent. 

AI guardian is just one of the new roles being handed to technology leaders and one reason I’m looking forward to the other CanadianCIO finalist conversations. 

Even if it means being on camera for some. 

— David Potter

If you have any questions, you can reply to this email. I read everything.

Do you have a story we should chase, or a tip you'd share with a peer?
Contact us here.

Know someone who should be reading this? Forward it on, or
they can sign up here.

This issue is presented by Calgary Economic Development.
If you want your brand in front of the people running technology across Canada, you can sponsor a future issue.