I recently saw a post (maybe on Reddit) where someone was asking for a script that would download all the files in a SharePoint site. I found myself with a few spare minutes and an open ChatGPT window, so I took a swing at it. I think the results turned out pretty well. I do want to give a shout out to the Salaudeen Rajack at SharePoint Diary. We did get some help from this script of his.
I tried to write it as well as I could, so I wrote it as a function inside of a module file. Because of that you have to save the script as a PSM1 file (as opposed to a regular old PS1 file). Then use Import-Module to import that PSM1 file, which adds my function, Download-SharePointFiles to your PowerShell host. Then run Download-SharePointFiles with the correct parameters. It looks like this:
That gives you output that looks like this:
I hope you find it helpful. Let me know in the comments if there’s any functionality you’d like me to add.
The PnP team recently announced they were making changes to how the PnP.PowerShell and M365 CLI authenticate. The short story is that if you want to use the PnP.PowerShell module or the M365 CLI after September 9th, 2024, you’ll need to create your own Application Registration, also known as an App Reg. I know, I know, that sounds confusing and scary. Fortunately there’s nothing to it. I’ve got all the steps for you to follow along. You’ll be ready to go in no time.
If you’re reading this, I assume you already have the PnP.PowerShell module installed. Good, because you’ll need that. The steps I’m going to show you will probably work with any version 2.0.0 or later, but I recommend updating to at least 2.9.0, which is the latest version as of this blog post. It’s got some extra sauce in it to make this go more smoothly.
After you’ve got the PnP.PowerShell module installed, open up PowerShell. This module has supported using custom app regs for a while, so all of the plumbing is already there. We need to run the Register-PnPAzureADApp cmdlet (also aliases as Register-EntraIDApp, they’re the same thing) to create our own App Reg. Example #1 from the help is what I use. Here’s what it looks like:
The name of the App Reg we’re creating is “PnP.PowerShell.” My tenant is 1kgvf, but of course you’ll use your own. It’s going to create a certificate for us (we won’t need it) and store it in the CurrentUser Certificate Store. And finally, since I’m using MFA, like every good user does, I use the –Interactive switch to do browser based authentication. You’ll have to log in as a Global Admin, or a user the Global Admin has given permission to create App Regs to.
After I log in Azure gets to work doing what it does. Since I didn’t pass any scopes to Register-PnPAzureADApp it uses its default set. You’ll get prompted to authenticate a second time then asked to consent to them.
We’ll talk more about that in a bit.
Once it’s done, you’ll get a screen like this:
Here is where you’ll get the Client ID (also called AppID and AzureAppID, it’s all the same) you’ll need when you connect. In my example that’s 001ed5a0-be10-4bc3-a40c-a1cad0c987b7. You can also get that ID number by going into the Azure Portal and looking at the Enterprise applications.
Find your App Reg and click it.
Then you can copy it to your clipboard.
Now that you have your ClientID, let’s use that to connect to Microsoft 365. In my case I would use this connect statement:
Of course you’ll use your own tenant name and the ClientID that you created. The one I created isn’t visible to your tenant.
You’ll get prompted for a username and password, and hopefully some MFA. Then you’ll be connected to M365. Here’s how it all looks:
As you can see, I’m connected and a quick function check looks like everything is working fine. Success!!! Well, sort of…
We talked before about scopes, and how the Register-AzureADApp cmdlet used its default scopes since I didn’t specify any. Those scopes covered SharePoint, and Users, but not much else. For instance, if I try to get a list of the Teams in my tenant, I’m met with an authentication screen. After I authenticate (with a Global Admin) I get a page wanting more permission:
Now it wants access to Read all groups, and more User permissions. I clicked Consent and then Accept. It returned my Teams to me.
This adds some complication. If the user you normally use PnP.PowerShell with is a GA, then you’re golden. Every time you stumble onto something you can’t do, it’ll prompt you and you’ll consent. If you have a separate GA user, or someone else is doing the GA stuff for you, you’ll have to go into the Azure Portal and add the additional Scopes in there. That sounds like something that would fit really well in another blog post.
There’s another gotcha. By default, when Register-PnPAzureADApp creates the App Reg, it only gives the user that created it permission to use it. If anyone else needs to use it you’ll need to go into the Azure Portal, open the App Reg, go to the Users and Groups blade, and add the additional users. That won’t give them any additional permissions anywhere in M365, it simply gives them permission to use this App Reg when using PnP.PowerShell.
I mentioned at the top that version 2.9.0 had some extra sauce to help with this. It can get a bit tedious adding the –ClientID SomeUglyGUID part every time you connect. This is particularly painful if you’ve got scripts and the like with connect statements that don’t have the -ClientID parameter. In 2.9.0 and later Connect-PnPOnline supports an environment variable ENTRAID_APP_ID. If no –ClientID is specified, and that variable has a value, Connect-PnPOnline will use that. Here’s what it looks like:
That variable value will go away when you close PowerShell. If you put the $env:ENTRAID_APP_ID = "001ed5a0-be10-4bc3-a40c-a1cad0c987b7" line in your PowerShell profile it will get populated every time you open PowerShell. If you’re a local admin on your machine you can also set a persistent environment variable in an admin prompt with this line:
If you’ve been around the block with SharePoint or Microsoft 365 administration, you know that handling client credentials can sometimes feel like juggling with fire. When I start a new engagement with a client I generally get credentials to access their tenant. Of course they immediately go into our password management tool. I also do a lot of PowerShell scripting for my clients, so I save them to the Windows Credential store too, so that I can connect with Connect-PnPOnline without having to enter them each time. And while that’s not a lot of work, I thought I could streamline it. That’s why I’ve put together a PowerShell command, Add-ClientCredential, that makes it a little easier.
What Does Add-ClientCredential do?
In a nutshell, this PowerShell command is designed to streamline the process of adding client credentials in a SharePoint or Microsoft 365 environment. It stores your credential for https://tenant.sharepoint.com and https://tenant-admin.sharepoint.com. If you don’t specify any credentials when you connect with Connect-PnPOnline it will look for them in the Windows Credential Store. If you have one assigned for the root of the tenant, https://tenant.sharepoint.com, it will also use that for other sites in the tenant, like https://tenant.sharepoint.com/sites/ToddisTheBest, if a credential is not saved for that specific site.
Keep in mind that will show up in plain text in PowerShell’s Get-History, or the Transcript file, if you have that running.
I built in some smarts so that if there is already a credential stored for “Contoso” it will let you know and ask you if you want to overwrite it.
If you want to get super fancy you can add the –TestCredential switch which will test the credentials you gave it by logging in with them. All of this is available if you run help Add-ClientCredential –Examples.
In the background this function usesAdd-PnPStoredCredential to store the credentials for you. It adds them for the root of the tenant, and the –admin URL.
How to Get Started
To get your hands on this little beauty, head over to my GitHub repository. You can download addclientcreds.psm1 itself, or clone the whole repo. Use Import-Module to import it into your PowerShell host and you’re ready to go.
Wrapping Up
addclientcreds.psm1 is my attempt to put a little more simplicity and sanity into the world of SharePoint and Microsoft 365 administration. I hope you find it as useful as I do. As always, I welcome your feedback and questions. Drop a comment below or shoot me a message on Twitter @ToddKlindt.
Today, I want to share a handy PowerShell function I developed recently to enhance readability of ShareGate log files.
If you are using ShareGate for SharePoint migrations or management tasks, you might be familiar with the extensive Excel logs produced by ShareGate. While these logs are rich in details, they can sometimes be too rich in details. They often require some formatting for better readability or to highlight the necessary details. When doing a lot of migrations I found myself doing the same steps over and over again to these logs. Being lazy, I thought, “Someone should automate this!”. I sat down with my buddy PowerShell and this is what we came up with.
This function, Format-ShareGateLogFile, tackles this by opening a ShareGate log file in Excel format and applying a few changes. It does this by using Doug Finke’sexcellent ImportExcel PowerShell Module. It adds a table to the first worksheet, formats the first column as “Date-Time”, calculates the duration of the log file, and formats the duration as “[h]:mm:ss” in the last row. Finally, it saves the changes and closes the Excel file.
To use this function, you’ll have to download Logfiles.psm1 from my Github Repo. Then use Import-Module to import it into your PowerShell session.
After that’s done you can run Format-ShareGateLogFile. You will need to provide the path to the Excel file to format as an argument to the Path parameter, which is mandatory. For instance,
This will format the Excel file located at “C:\path\to\ShareGateLogFile.xlsx” for readability.
The function also accepts two optional switches: Open and HideColumns. If you use the Open switch, the function will open the formatted Excel file automatically after it has finished formatting. The HideColumns switch will hide specified columns (E-U, W-AR, AT-BA) in the Excel file. For instance,
This will format the Excel file, hide the specified columns, and open the Excel file automatically after it has been formatted.
One neat feature of the function is that it can accept pipeline input for the `Path` parameter. This means you can pipe in a series of file paths to the function and it will format each file in turn. For instance,
This will get all the Excel (.xlsx) files in the specified folder, and for each one, it will be formatted for readability, with specified columns hidden, and the Excel file opened automatically after it has been formatted.
This function has saved me heaps of time while working with ShareGate log files, and I hope it does the same for you. Happy scripting!
Have you ever updated some properties for a user in SharePoint Online, waited what seemed like a reasonable length of time, then did a search for that user? Have you ever done that and found your old values still there? Gah! Me too! Did you then use Mikael Svenson’s brilliant script to fix that? Yeah, me too. A few times.
I’m also a tinkerer, and I love me some PowerShell. So I cracked open VS Code, put on some aggressive music, loaded up Mikael’s code, and looked around a bit. I found a couple of things that I’d do differently. Not that there’s anything wrong with his code, it’s good stuff. But I wanted to so some stuff like turn his code into a function, so I could add it to an existing module I have. I wanted to move the JSON file from the root of the web to the Shared Documents library, innocent little changes like that. And his code had been written several years ago. PowerShell has matured since then, as coders we’ve matured, so I cleaned a few other little things up while I was in there. It was fun, and it kept me off the streets for a while.
After I got done I reached out to Mikael to see if he was okay with me sharing it with all of you. Mikael, being the stand up guy that he is, was totally good with it. Then I reached out to Paul Bullock about getting it published to the PnP Script Samples. I’ve been meaning to start submitting some stuff to that, but I didn’t have anything I thought was worthy. Paul graciously accepted my script, so now you can all enjoy it too.
While you’re there, check out all the other great script samples. There’s some gold in those hills.
As an M365 Admin it’s not tough to see that the future is hidden somewhere in the Microsoft Graph. And this particular M365 Admin would like to get there using his favorite tool, PowerShell. Microsoft released the Graph SDK, but it didn’t really scratch the itch for me. I found it confusing and really tough to use. But, as much as I ignored it, the Graph didn’t go away. With increased frequency there would be some little tidbit of information I’d want and the answer would be, “It’s in the Graph.” Grrrrr.
It uses PnP.PowerShell, so you’ll need that module installed, and you’ll need to connect with Connect-PnPOnline. From there you can pass it any Graph endpoint, v1.0 or beta. You can find out what the endpoints are at the Graph Explorer. Right now this only supports Gets. You’ll also have to manually put in any parameters or filters. I’ve added help and examples. I hope to expand them both in the future.
The output is a stand PSCustomObject, so you can send its output down the pipeline however you’d like. Select-Object, Where-Object, Sort-Object, the world is your oyster.
Let me know if this is useful and what you’re doing with it.
I’ve done a few articles about the new PnP.PowerShell module. One of the biggest changes from its ancestor, SharePointPnPPowerShellOnline, is that it requires the registration of an Azure Application before you can connect with it. In this blog post I’m going to explain how to get that Azure App registered if you’re not a Tenant Admin in your tenant.
You don’t need to be a Tenant Admin to use the PnP.PowerShell cmdlets. You don’t even need to be a SharePoint Admin or a site collection admin. There are plenty of cmdlets you can run, like Add-PnPFile if you’re only a Member of the site. However, before you can run the most import PnP cmdlet of all, Connect-PnPOnline, the PnP Azure Application has to be registered in your tenant by a tenant admin. If it’s not, you’ll get a sad message that looks like this:
Here’s the text:
Connect-PnPOnline: AADSTS65001: The user or administrator has not consented to use the application with ID ‘31359c7f-bd7e-475c-86db-fdb8c937548e’ named ‘PnP Management Shell’. Send an interactive authorization request for this user and resource.
In most cases the person introducing the PnP.PowerShell module is a tenant admin, so it’s not an issue. They run Register-PnPManagementShellAccess and Bob’s your uncle. But it’s not uncommon for an organization to be large enough that the SharePoint or Microsoft 365 Admin team is not a tenant admin. In that case the Tenant Admin, who likely doesn’t know what a PnP.PowerShell is, has to register the Azure App before the SharePoint Admin can enjoy the bliss that is PnP.PowerShell. Fortunately, there’s an easy enough solution, the Consent URL.
The Consent URL is the URL to a web page your Tenant Admin can go to to consent the PnP.PowerShell Azure App without needing to install anything, or really know anything about the PnP.PowerShell. There are a few ways to get the Consent URL. It doesn’t matter how you do it, they all get you to the same place.
The easiest way to remember is to run Register-PnPManagementShellAccess –ShowConsentUrl after installing the PnP.PowerShell. You’ll be asked to log in, but you don’t need to be an sort of admin. It’s only logging in so it knows when tenant you’re in. Then it will give you the Consent URL. It looks like this:
The part in the red box is your tenant’s ID.You had to log so the cmdlet could get that number. The Client_id refers to the PnP.PowerShell, so it’s the same everywhere.
You can also specify your tenant’s name instead of its ID. This works as well:
Whether you get the URL from running Register-PnPManagementShellAccess –ShowConsentUrl or by copying it out of this blog post and putting your tenant’s information, send that URL to your Tenant Admin. When they browse to the page it will look like this:
All they need to do is click Accept and you’re ready to go.
That’s a pretty long, scary list of permissions, and it might spook some admins. Accepting this does not give everyone in your tenant all of those permissions. The PnP.PowerShell Azure App uses Delegation, which means any user using it to access objects in Microsoft 365 has to have permission to access it. The PnP.PowerShell does not allow anyone access to anything they don’t otherwise have access to. If they don’t believe you, have them try. Have someone that cannot open up a SharePoint site in the browser try to connect to it with Connect-PnPOnline. They won’t be able to.
If they want to check out what the Azure App has permission to, or heaven forbid, remove it, you can browse to the Azure AD Portal and find it in the Enterprise Applications.
The Permissions blade will show you all delegated permissions the app has. Feel free to poke around, but resist the urge to change any, even if you’re positive you’ll never use them. I promise it’ll only hurt you in the future.
After your tenant admin has done all of that you should be able to get back to all that PowerShell and PnP goodness.
Throughout my IT career I have had to create tens (or hundreds, or thousands) of objects to test something. It could be a bunch of Windows Users, a bunch of folders, files, etc. It seems like every time that happens I end up starting from scratch on the process. To stop that silly cycle I decided to make the process official by blogging it. Let’s stop this madness!
This time it started with my friend Michal Pisarek posting this tweet:
Orchestry needs to test their lifecycle features and he wanted to stress test it real good!
As is often the case, I see a tweet like that and my first thought is “Challenge Accepted!” The mechanics of how to create Teams with PowerShell is pretty simple but where this really gets tricky, at least for me, is the names. Especially if you’re looking at creating 20,000 like Michal is. In the past the way I’ve handled that is the old tried and true “Adjective Noun Number” formula. To get near 20,000 I wanted a long list of nouns and adjectives to pull from. I scoured the Internet and pulled together two files, nouns.txt and adjectives.txt. You can find them in this GitHub repo. Then I tack a random two digit number at the end to reduce the chance of collision. I put those files in the same directory as this PowerShell script and let ‘er rip!
You can find the file CreateLotsofTeams.ps1 in that same GitHub repo.
You can alter the nouns and adjectives files as you see fit. Set the the $NumberofTeams variable to how many Teams you want and you’re set. This script uses the venerable PnP.PowerShell module. You’ll need that installed and its Azure Application registered before you can run this. Be sure to change the Connect-PnPOnline line to reflect your tenant’s name, unless you actually work for Contoso.
Because of some weird timing, the current version of the PnP.PowerShell, 1.6.0, won’t work with this script as there is a bug in New-PnPTeamsTeam that prevents it from actually creating a Team. Ironic, I know. I put notes in the CreateLotsofTeams.ps1 file on how to handle that. But if you’re running it and it looks successful but no Teams are being created, look there first.
Also, for whatever reason, when you look at the Groups Sites in SharePoint they don’t show up as being Teams enabled, but they really are.
<broken image>
You can see in this crudely mocked up screenshot that the Teams are in the Teams client even though SharePoint Admin Center swears they don’t exist.
And while this script’s purpose in life is to create lots and lots of Teams, it could be easily modified to create lots and lots of anything. If you just need Groups, swap out New-PnPTeamsTeam with New-PnPMicrosoft365Group. If you just need SharePoint sites, use New-PnPTenantSite. Folders? Add-PnPFolder. I think you see where I’m going with this.
If you’re like Michal and you’re going to create 20,000 Teams, or anything, I hope you have a comfortable chair. It’s going to take a while. Michal is seeing about 1 Team a minute. It’s going to take him a couple of weeks at that pace. Almost certainly PowerShell is the bottleneck in this situation. If you’re looking at a similar situation, my advice is to open up another PowerShell window and run another instance of CreateLotsofTeams.ps1 there. And maybe run a few instances on another machine entirely. In the past that has helped me speed this things up considerably.
My fellow Sympraxian Julie Turner recently published an outstanding blog post on what a Microsoft 365 Dev Tenant is and why you should get one. I’ve been a long time user of Dev or Demo tenants and I find them to be an invaluable resource both for people trying to gain skills in Microsoft 365, but also grizzled old veterans like myself. Like the old saying goes, “The question isn’t whether you have a Test Site, it’s whether you have a Production Site.” If you don’t have a Dev or Demo Tenant to test things in then you’ve just demoted your Production Tenant to a Test Tenant. Don’t be that person. I did a session on the PnP PowerShell today for Petri.com. One of the people there was hesitant to try out PowerShell on their tenant. I recommended they get one of these Dev tenants and hone their skills there.
Microsoft will soon be releasing a new version of the much maligned Edge Browser. The new version will be based on the Chromium engine, and along with that it inherits some great functionality. One of those features is Profiles. This means all of the techniques you use in Chrome to manage your Office 365 users works exactly the same. If you are already running the Chromium Edge (lovingly referred to as “ChrEdge”) beta, you can go to this page, edge://settings/profiles, to manage the Profiles, or click the Profile icon in the upper right corner of the browser.
An added benefit of Microsoft’s added touch to the Chromium Profile engine is that you can sync settings with an MSA (Microsoft Account) account like your old school @hotmail.com account. You can also sign in with an Azure AD account, which we all have courtesy of Office 365. I’ll blog more about that later as it’s a feature I’m really excited about.
