CoPilot Gains and Realities

Let's go over the current corporate standard for AI, Microsoft CoPIlot.

Anyone working in a modern office will have already used Microsoft CoPilot by now, and how much or how little you use generally depends on how much the technology has been promoted and how well the AI has been trained to suit your day to day tasks.
CoPilot is primarily access through Teams on the left hand side menu. Clicking on it opens up CoPilot in he main window and you get a whole prompt window that looks like the below. Here's a little overview of what you see.
  • Work/Web - You can flip between allowing the prompt to only draw from work (intranet/internal sources) data and from Web data that is from the Internet. I won't lie, I have never had a any decent results from the Web option that I couldn't pull from something public like Gemini or ChatGPT, and get better results.

Let's begin

  • Prompt Bar - This is where you type your request and all the details that you need CoPilot to carry out. Clicking on the "+" allows you to add images, documents or media to help the AI give you a better result. Clicking on the microphone allows you to convert speech to words which works very well.

  • Auto - This dropdown allow you to determine how deeply you want the AI to "think" about their response. The richer and more thought-out your want you response, the longer it will take.

  • Learn Find Summarize Suggest etc - These help target specific searches by helping the user with keywords that will actually work and perform what is needed. However, there is nothing stopping your from just typing in "Learn..." or "Find...". It's useful but not absolutely necessary.
I put in a typical prompt asking the AI to summarise and create an executive statement on a document slide pack, which is something typically I would do and would need to sit and read the whole thing through. I upload my document and type in the following prompt.

Where the prompt does well

The results are pretty good and written to a fairly high standard even though I'm using Auto mode and not utilising the deep thinking modes that would give me more professional language and better insights. It even cites the sources (we had one!) and is fine.
I can easily export this or email this to myself.
I plugged in another very typical prompt where I want CoPilot to organise a meeting with my co-worker who I have full access to his diary. There is literally no barrier to the AI making this booking so I put it to the test.

Where the prompt fails

The results are a lacklustre. Though it can find a) my co-worker's profile and also b) that he has the proposed time slot free in his diary, it isn't able to actually make the booking for me.
Instead it prompt me back and gives me instructions on how to create a booking in Teams.

Not quite the result I was after given that Teams has both CoPilot and my Outlook calendar both under the same application.
I gave CoPilot the benefit of the doubt and fed it a spreadsheet with data on countries and financial data. I asked it straightforward question to count the items in the spreadsheet and group them by countries. Should return a standard grid with count figures, nothing more.

Another fail

Though I understand the rationale to pinpoint the exact data, the spreadsheet only has one workbook/tab so it shouldn't have been difficult.
CoPilot really shines when it comes to the more menial tasks like note taking and transcripts for meetings. It does well in storing and recognising voices in meetings and remembering which person said what. but these feel more like automation rather than "artificial intelligence".

What the AI does do well

Even though it looks like a failure of CoPilot, there are many redeeming qualities to it in other features that are either still being refined or features that are to come. Including:
  • Research and Analyst - These are two new modes of CoPilot that do deeper functions providing you feed it the right data and can write a killer prompt that explains exactly what you want.

  • SharePoint Agent - It's a CoPilot AI bot that you can place right next to your document library. Putting this AI in close proximity to your files allows the AI to do deeper crawls and more targeted searches of your documents and returns answers that are more spot-on and accurate. I'll do a separate article to delve into this.

  • Knowledge Agent - A promising AI that would live on site collections and allow you to do AI based file classifications and to generate knowledge wiki pages semi-automatically. Besides it's temporary appearance (pulled back in January 2026, but is returning soon), there were also holes. The permanent nature of the agent meant that non-admin users could possible wreck havoc on metadata tags accidentally. So hopefully it returns soon with massive improvements.

Silver lining