Zero to Copilot: actual notes from the trenches
I want to be upfront about something. The version of this post you may have read before was useful and a little boring. It had bullet points. It had phrases like “role-based training.” It was, in the way of most vendor blog posts, written by someone trying very hard not to say anything that could come back to bite them.
This is the version where I say the things.
The week we found out what payroll could see
Every Copilot rollout has a moment, usually around week two, where someone on the leadership team goes quiet during a meeting. Not the good kind of quiet. The kind where you can see them doing math in their head, and the math is not coming out in your favor.
The moment I am thinking about involved a director who decided to test Copilot by asking it a perfectly reasonable question about a reorg that had not been announced yet. Copilot, doing exactly what it was built to do, answered the question. Correctly. Using a document that had been sitting in a SharePoint folder since 2019, shared with “everyone in the organization” by someone who, at the time, genuinely believed that was the easier setting and never thought about it again.
Nobody had opened that folder in years. Copilot opened it in about four seconds.
This is not a story about Copilot being dangerous. Copilot did nothing wrong. It found a document it was allowed to find and used it the way it is supposed to use things. The actual story is that an organization had been quietly insecure for five years and nobody knew, because nobody had a reason to go looking. You do not audit a filing cabinet nobody opens. You absolutely should audit the digital equivalent, because eventually something opens it for you, very fast, and without asking permission first.
We now start every single engagement with the question “do you know what Copilot would see if we turned it on today.” The answer is always some version of “probably fine,” said with the confidence of someone who has not checked. It is never fine. It is always at least one folder named “everyone” that should have been named “absolutely not everyone.”
The pilot that everyone loved, and the one person who saved it
We ran a pilot once where the feedback was, genuinely, glowing. People loved it. Three weeks later, usage had dropped to almost nothing, and the same people who had filled out the glowing survey were back to doing things the old way without seeming to notice the contradiction.
Here is what happened. The pilot use case was good in theory and irrelevant in practice. We had built a beautiful demo around drafting executive summaries from long reports. Nobody in the pilot group actually wrote executive summaries from long reports. What we should have asked first is some version of: what is the thing you complain about doing every single week. Not what could AI theoretically help with. The answers are never glamorous. “Compiling the numbers from four different spreadsheets.” “Writing the same kind of email nine times a day with slightly different names in it.” Build the pilot around those, and you do not need a survey to know if it worked, because people will simply keep using it without anyone reminding them to.
The other thing that pilot was missing was a skeptic. We fixed that on the next one. There was a woman in that pilot group who, on day one, told us flatly that this was a waste of her time. We kept her in anyway, mostly because removing her would have been awkward.
She turned out to be the most useful person in the room. Every time the tool did something slightly wrong, slightly annoying in a way the enthusiasts had quietly worked around without mentioning it, she said so. Loudly. With timestamps. The enthusiasts will tell you everything is wonderful because they already wanted it to be wonderful. That is not useless, but it is not the information you need. You need the person who is going to complain at scale during the real rollout to complain now, in a room of twelve people, where you can do something about it.
The training session that put a room to sleep, and the one that did not
We have delivered the same general Copilot training to two very different rooms and watched two very different things happen.
The first room got the full feature tour. Every capability, every department, ninety minutes, one slide deck for everyone from finance to facilities. By minute thirty, the energy in that room had the specific texture of a Monday afternoon meeting that everyone is attending because attendance was mentioned by name in an email. People nodded. Nobody asked questions. Nobody used Copilot for anything afterward beyond the bare minimum required to prove they had, in fact, attended.
The second room got twenty minutes, no slide deck about every feature in existence, and one specific demonstration of the one thing that room's actual job involved. For a finance team, that was reconciling numbers across systems faster. For a customer-facing team, it was pulling up account history before a call instead of during one. People in that room actually leaned forward. Someone asked a follow-up question that was specific to their actual work, which is the single most reliable sign a training session has landed.
The lesson, which feels almost insultingly obvious written down and which we nonetheless had to learn by doing it wrong first, is that nobody needs to know what Copilot can do for every job in the building. They need to know what it can do for theirs. Tell them that, and only that, and watch the difference.
The one bad afternoon that undid three good months
I want to end on the one that still makes me wince a little.
Three months into a careful, well-run rollout, with a champion in every department and genuinely strong adoption numbers, Copilot gave someone a confidently wrong answer in a meeting that had a few too many important people in it. It was not a catastrophic answer. It was the kind of wrong that gets corrected in fifteen seconds and forgotten by everyone except the one person who decided, right there, that this whole AI thing was not to be trusted, actually, and who then told that story to everyone who would listen for the next two weeks.
Adoption did not collapse. But it stalled, visibly, for about a month, while we did the unglamorous work of rebuilding trust one conversation at a time. The actual lesson here is not “Copilot will sometimes be wrong,” because of course it will, it is a tool, not an oracle. The lesson is that a single bad afternoon travels through an organization at a speed that three good months cannot match, and the only real defense is making sure people already trust the foundation underneath it before that afternoon happens. An organization with its data governance in order absorbs one wrong answer as a minor hiccup. An organization without it treats the same wrong answer as proof of everything it was already worried about.
If you are about to do this
Do the boring data work first. The fun part goes much better if the boring part happened first.
Build your pilot around an actual complaint, with at least one real skeptic in the room on purpose.
Train people on the two things that matter to their actual job, find a champion in every department who is not from IT, and when the inevitable slightly-wrong moment happens, do not panic. A calm “yes, that happens sometimes, here is how we check it” travels almost as fast as the bad story, and it is a much better story to be telling.
Where Armely fits into all of this
We have now run this sequence enough times that we have stopped being surprised by any of it, which is either professional growth or a small tragedy, depending on how you look at it. We start with the data conversation, because skipping it is the single most common and most expensive mistake we see. We are also a certified Microsoft Delivery Partner for Chat with Your Data in a Day, which is a genuinely useful way to let a team see a grounded AI experience with their own data before anyone commits to anything bigger.
If you are earlier than the pilot phase and want someone to look honestly at your data environment before you turn anything on, that conversation is free, and we promise not to make you sit through ninety minutes of slides about features you will never use.
Book a free data governance and Purview readiness review at armely.com, by emailing info@armely.com, or by calling 972-460-0643.
