Skip Ribbon Commands
Skip to main content
 

 Follow Me

 
 
 
 
 

 SharePoint Blog List

 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Todd Klindt's home page > Todd Klindt's Office 365 Admin Blog
What's going on with TK.
December 11
Bulk Undelete Files in Office 365 and SharePoint Online with PnP PowerShell

Quite often as a consultant, you get to work on truly fun and funny situations. The customer email that prompted this blog post is one of them.

My contact at the customer site emailed with a problem. A few weeks earlier one of their users deleted some pictures from one of their document libraries. Okay, not a few pictures, nearly 100,000 of them. And not only had that user deleted nearly 100,000 items, they hadn’t told anyone for 3 weeks, while they tried to upload the pictures they still had locally. I know what you’re saying, this is a job for the Recycle Bin! And you’re right, it is, but the other factors made it complicated. They couldn’t just restore all of the files in the Recycle Bin as there were also 3 weeks’ worth of legitimately deleted documents in there, including documents from that user. Also, since the user had re-uploaded a bunch of the deleted documents there would be “overwrite” prompts all over that would slow down the process. And let’s not forget that even if it all went smoothly, someone would have to manually restore nearly 100,000 files. No easy feat.

The customer talked to me because they wanted to share the story (it was a big funny after the fact) but also because they knew I always said that PowerShell could do anything. They were hoping PowerShell could bale them out of this mess too. I don’t mean to spoil the end of this story, but they were right, it could.

My tool of choice when it comes to making magic in Office 365 is the PnP PowerShell. I had never done this before, so I had to do a bit of spelunking. I ran Get-Command *recycle* -Module SharePointPnPPowerShellOnline to see what PowerShell cmdlets availed themselves to me.  Looky, looky, there it is, Restore-PnPRecycleBinItem. Did you just hear angels sing? I know I did.

Now we needed to weave a little PowerShell magic. We couldn’t restore all of the deleted files, since other folks had legitimately deleted files in the meantime. We also only wanted to restore .JPG files that this user had deleted. Here’s command that got the files we needed:

Get-PnPRecycleBinItem | Where-Object -Property Leafname -Like -Value "*.jpg"  | Where-Object -Property Dirname -Like -Value "Shared Documents/*"  | Where-Object -Property DeletedByEmail -EQ –Value shane@tkdemo.com

That returned all the files, now what to do with them? Restore them, of course. This bit of code grabs all the files, counts them as it restores them, then spits out the time and date when it’s done.

$bin = Get-PnPRecycleBinItem | Where-Object -Property Leafname -Like -Value "*.jpg"  | Where-Object -Property Dirname -Like -Value "Shared Documents/*"  | Where-Object -Property DeletedByEmail -EQ –Value shane@tkdemo.com

$bin | foreach  -begin { $a = 0} -Process  {Write-Host "$a - $($_.LeafName)" ; $_ | Restore-PnPRecycleBinItem -Force ; $a++ } -End { Get-Date }

Normally, that would have worked, but it was going to take some time to restore the files. A loooong time. Several days in fact. With some testing I surmised that the client PowerShell was the bottleneck. I used a little PowerShell trick to break the entire 100,000 item collection into chunks of 10,000 and run it multiple PowerShell windows and on multiple machines. I changed the second line to look something like this:

($bin[20001..30000]) | foreach  -begin { $a = 0} -Process  {Write-Host "$a - $($_.LeafName)" ; $_ | Restore-PnPRecycleBinItem -Force ; $a++ } -End { Get-Date }

The $bin[20001..30000] only sends items 20001 to 30000 of the collection down the pipeline. I changed that on each client. $bin[0..10000], $bin[10001..20000], and so on. After they finished I ran through it all one more time to make sure I didn’t miss any. The entire script looked like this:

# Make sure necessary modules are installed
# PnP PowerShell to get access to Office 365
Install-Module SharePointPnPPowerShellOnline

# Module to securely store passwords
Install-Module CredentialManager

# Saved credentials
New-StoredCredential -Target "ImportantSite" -UserName
madowner@tkdemo.com -Password 'Password goes here' -Persist LocalMachine

# Now the actual meat
# Connect to the site collection where the files were deleted
Connect-PnPOnline -Url
https://tkdemo.sharepoint.com/ -Credentials ImportantSite

# Filter the recycle bin for only the files we want to restore, JPGs, from the Shared document library, deleted by Shane
$bin = Get-PnPRecycleBinItem | Where-Object -Property Leafname -Like -Value "*.jpg"  | Where-Object -Property Dirname -Like -Value "Shared Documents/*"  | Where-Object -Property DeletedByEmail -EQ -Value
shane@tkdemo.com

# See how many pictures Shane deleted.
$bin.count

# Walk through the collection and restore each document. Spit out the time at the end so you know how long it took. Also keep a counter to see how it’s going as it churns through the collection
$bin | foreach  -begin { $a = 0} -Process  {Write-Host "$a - $($_.LeafName)" ; $_ | Restore-PnPRecycleBinItem -Force ; $a++ } -End { Get-Date }

# Since the OM is the bottleneck you can run this on multiple machines to speed things up. Here’s how to only restore a subset of the collection
($bin[20001..30000]) | foreach  -begin { $a = 0} -Process  {Write-Host "$a - $($_.LeafName)" ; $_ | Restore-PnPRecycleBinItem -Force ; $a++ } -End { Get-Date }

I hope you never need to use this. Smile

tk

ShortURL: https://www.toddklindt.com/PoshRestoreSPOFiles

December 10
Podcast 412 - Preconditioned Fail

Todd has left the podcast for the day and Shane brings in a guest. Troy from AppRiver. They discuss security and hacking. Troy talks about what kinds of things the bad guys are doing and what organizations and individuals can do to help protect against those things. Things even the technically challenged could do. Shane then tells us about community and upcoming events.

Audio File

Video File

Podcast 412 - Preconditioned Fail (Time 0_11_50;25)

YouTube (Subscribe)

Subscribe in iTunes

Running Time: 39:56

Links:

SysKit Security Manager
Information architecture in a flat SharePoint world
Creating PDFs from SharePoint Data
Online Free Flow Conference
Thrive Conference 2018
SP Fest Chicago
SPTechCon Austin 2019
North American Collab Summit
SharePoint Conference NA

ShortURL: https://www.toddklindt.com/Podcast412

December 05
Podcast 411 - You're Not Going to Like This

Shane and Todd spend time this week talking about some tech toys Shane has bought over the years including a Ring doorbell, new Wi-Fi APs, and a really big monitor. They also discuss some fun PowerShell topics, and what they wear when they work from home.

Audio File

Video File

Podcast 411 - You're Not Going to Like This (Time 0_03_49;18)

YouTube (Subscribe)

Subscribe in iTunes

Running Time: 50:48

Links:

SysKit Security Manager
What’s the Recommended Editor for PowerShell Scripts?
PowerShell support for PowerApps (preview)EXPLORING THE POWERSHELL ALTERNATIVE TO NETSTAT
What to Wear to Work—When You Work at Home
Online Free Flow Conference
Thrive Conference 2018
SP Fest Chicago
SPS Coimbatore
SPTechCon Austin 2019
North American Collab Summit
SharePoint Conference NA

ShortURL: https://www.toddklindt.com/Podcast411

November 16
Podcast 410 - Literally No One Wants That

Marc Anderson joins Todd this week while Shane is teaching a class. They talk about a problem that has crept into the PnP PowerShell and how to fix it. Marc talks about some developery topics like the new SPFX release and the Office 365 CLI. They also discuss how Microsoft is improving the experience of having multiple accounts, and how AutoSave is great, but you have to keep your eye on it.

Audio File

Video File

Podcast 410 - Literally No One Wants That (Time 0_02_59;11)

YouTube (Subscribe)

Subscribe in iTunes

Running Time: 59:02

Links:

Getting a “Method not found” Error when using PnP PowerShell
Office 365 CLI
Azure portal and the new account manager!
How Broken Are Office 365 SharePoint Permissions?
SPFx 1.7
SharePoint Framework v1.7.0 - What's in the latest update of SPFx?
AutoSave in Office 365: Search Implications
PowerApps and Flow Training
Thrive Conference 2018
SP Fest Chicago
SPS Coimbatore
SPTechCon Austin 2019
North American Collab Summit
SharePoint Conference NA

ShortURL: https://www.toddklindt.com/Podcast410

November 13
Getting a “Method not found” Error when using PnP PowerShell

This question has come up a few times so I thought I’d blog it so I had an easy place to point people. You can also go to the PnP PowerShell GitHub Comments for the same information.

If you’ve watched any of my podcasts, or seen me speak at any technical events you know I’m a big fan of the Patterns and Practices (PnP) PowerShell cmdlets. They make working with Office 365 tolerable, even enjoyable. The PnP PowerShell is a community led, Microsoft supervised, project on GitHub. If you’re not already using it, you should check it out.

Along with scads of other software running on our machines, it gets updated once a month or so. In the last couple of months a couple of clashing updates have caused PnP users some stress. Specifically when they use the Connect-PnPOnline cmdlet to connect to SharePoint Online they get a “Method not found” error message. The PnP team was on it. They published this fix:

  1. Delete all the SharePoint Components from the Control Panel (Add/Remove)
  2. Open the GAC location (C:\windows\Microsoft.NET\assembly\GAC_MSIL), search for the term SharePoint and delete all the folders. (This is required since the reference is still old which is used by the PnP commands)
  3. Re-Start the machine
  4. Install the latest version of the SharePoint PnP

That should do it. From the comments I’ve seen, and my own experience, this has a 100% success rate. Again, you can read all about it on GitHub.

Thanks to the PnP Team for all you do, and for getting this fix out so quickly.

tk

ShortURL: https://www.toddklindt.com/PoshPnPMethodNotFound

November 13
Podcast 409 - Hold Your Tongue Sideways
November 06
Connecting to Office 365 with Multiple Accounts in PowerShell without Losing Your Damned Mind

The last (and first) installment in my ever popular “Without Losing Your Damned Mind” blog series, How to Connect to with Multiple Office 365 Accounts in your Browser without Losing Your Damned Mind, was a mild success, and the series was picked up. We decided it was time for a sequel, and here it is.

As an IT Pro/Administrator in the Microsoft space you have to love PowerShell. You don’t have to be an expert in it (though that really helps, I envy those guys) but you have to be confident enough with PowerShell that you can see some PowerShell on the web and be able to understand what it’s doing before you run it yourself. The way you do that is slowly, after months and years, forcing yourself to do tasks in PowerShell no matter how much it sucks. In the context of Office 365 that can get cumbersome because you have to authenticate against cloud services all the time, with many different accounts, and that requires that cursed act of typing your password. Over and over and over again. I hate typing passwords so very much. It gets worse when sometimes I’m connecting as me, sometimes as my own tenant global admin, sometimes as a customer’s tenant admin, sometimes as a different customer’s admin, it never ends.

A while back I started using Windows’ Credential Manager to manage my PowerShell logins, and that’s what this blog post is going to be about. I will assume you already know how to connect to Office 365 and Azure with PowerShell and that you have all of the necessary modules, executables, and the like installed.

The secret sauce that makes this all bearable is a PowerShell module called PowerShell Credential Manager. This little beauty uses native PowerShell to access the Credential Manager in Windows. There are other modules that do similar things, and you can actually do all of this yourself in PowerShell without a module. I’ve found this module to be the best that works for me, but this process is the same regardless of what you use.

You can install it with this line:

Install-Module -Name CredentialManager

It installs the following cmdlets:

Get-StoredCredential
Get-StrongPassword
New-StoredCredential
Remove-StoredCredential

You can use New-StoredCredential to put a new entry in the Windows Credential Manager. You can also use the Credential Manager UI to do it. It’s all the same. To create a credential with New-StoredCredential use something like this:

New-StoredCredential –Target O365Admin –username admin@tenant1.onmicrosoft.com -password pass@word1 –Persist LocalMachine

New-StoredCredential –Target Jimmy –username Jimmy@tenant1.onmicrosoft.com -password pass@word1 –Persist LocalMachine

New-StoredCredential –Target OtherAdmin –username admin@tenant2.onmicrosoft.com -password pass@word1 –Persist LocalMachine

And so on. A couple of things of note. First, all of the passwords are in plain text. If you’re running a transcript make sure you clear those out. Also, the –Persist switch is important. If you don’t add that, the Stored Credential will vanish into the ether when you close this PowerShell window. You won’t know why it doesn’t work the next and you’ll be very confused. Maybe that’s just me. You can also add a comment for the credential with the –Comment parameter.

Running the command looks like this:

image

The entries look like this in Windows:

SNAGHTML67af9a89

You can get to this screen in Control Panel, or by typing “cred” in the Start Menu. The credentials you create in PowerShell will show up under “Generic Credentials” and you can use anything that’s already there. You can also create credentials with the cleverly named, “Add a generic credential” link at the top. It doesn’t matter how they get there, it just matters that they’re there. You can create as many credentials as you want. There’s no charge. Smile

Now that we’ve loaded the Credential Store up with some yummy Office 365 Credentials, how do we use? The magic word is Get-StoredCredential.

We can use Get-StoredCredential to securely pull credentials from the Credential Manager and pass them to a PowerShell cmdlet. You can save them in a variable, or just call it directly when you connect. Here is how you connect to Azure AD:

Connect-MsolService -Credential (Get-StoredCredential -Target O365Admin)

It looks like this:

image

No passwords were typed (Hooray!) and that means you can securely automate it. The process is the same for other Connect style cmdlets. Here are a couple of examples:

Connect-AzureAD -Credential (Get-StoredCredential -Target O365Admin)

Connect-SPOService -Url https://blogpost-admin.sharepoint.com -Credential (Get-StoredCredential -Target O365Admin)

If I wanted to sign in as the Jimmy account, I would pass that to the –Target parameter:

Connect-SPOService -Url https://blogpost-admin.sharepoint.com -Credential (Get-StoredCredential -Target Jimmy )

I’m a frequent user of the PnP PowerShell. I’d give up sliced bread before I gave them up. While you can use this same approach with them, you have another option that’s even less typing:

Connect-PnPOnline -Url https://blogpost-admin.sharepoint.com -Credential O365Admin

The Connect-PnPOnline cmdlet (and maybe others) natively know to check the Credential Manager if it’s passed a string.

That’s all there is to it. Once you create a credential it’s yours to use however you’d like, as often as you’d like. This isn’t restricted to Office 365, either. Any PowerShell cmdlets that take credentials can take advantage of this.

Enjoy.

tk

ShortURL: https://www.toddklindt.com/PoshMultipleAccounts

November 05
Podcast 408 - Never Free the Trolls (Just Be Nice)

Todd got a new job and they kick the podcast off talking about that. After they're finished gushing about that they talk about some topics Todd recently covered in classes. They also give their talk in the recent IBM purchase of Red Hat and someone that made Shane and Todd's day.

Thanks to our sponsor and the inventors of bear hugs, AppRiver.

Audio File

Video File

Podcast 408 - Never Free the Trolls (Just Be Nice) (Time 0_02_08;08)

YouTube (Subscribe)

Subscribe in iTunes

Running Time: 47:26

Links:

SysKit Insights 2.0
AppRiver
SharePoint Calculated Columns and Validation Formulas
PowerApps Save Data
North American Collab Summit

ShortURL: https://www.toddklindt.com/Podcast408

November 02
Behold! I Joined Sympraxis Consulting!

Over the few months I’ve been doing a lot of consulting projects, and some of my favorites have been with my long time friends Marc Anderson (blog|twitter) and Julie Turner (blog|twitter). I’ve known Marc since roughly the beginning of time, and met Julie a few years ago when she joined him at Sympraxis. We’ve all been old friends ever since. They’re both developery types and over the last few months they’ve brought me in to help with some projects that needed an admin’s touch. At SPTechCon a couple of months ago we were sitting around catching up and Marc suggested that we put a ring on it and make it official.

What a magnificent idea!

Yesterday I officially started at Sympraxis. I’ll be doing all the same things I’ve been doing. Blogging, podcasting, teaching, and loving on SharePoint and Office 365 every day. Now though, Julie and Marc are legally required to chat with me on Teams during the day when I’m bored. Smile 

I will also continue to be the chief Evangelist for Syskit. I’m going to be staying very busy.

If you’d like to bring Sympraxis into a project, drop us a note on our Contact Page and we’ll get a hold of you.

Thanks to Marc and Julie for bringing me onboard.

tk

ShortURL: https://www.toddklindt.com/JoinSympraxis

October 31
Podcast 407 - You Can Only Laugh

Todd kicks off the show by telling Shane the drama of his weekend. After he's done whining they get down to business and talk about fun tech stuff like SharePoint 2019, a new version of SPDocKit, improvements to the OneDrive Sync Client, and how Shane triumphed over Sling.

Thanks to our sponsor and the inventors of bear hugs, AppRiver.

Audio File

Video File

Podcast 407 - You Can Only Laugh (Time 0_17_21;11)

YouTube (Subscribe)

Subscribe in iTunes

Running Time: 55:37

Links:

SharePoint Server 2019 is RTM
SPDocKit 8.0 is also out
OneDrive will now remove offline files if they aren’t accessed for a certain period
AppRiver
Apple’s TV subscription service starts in 2019 to compete with Netflix, Amazon
PowerApps and Flow Training
Thrive Conference 2018
North American Collab Summit

ShortURL: https://www.toddklindt.com/Podcast407

1 - 10Next
 

 Subscribe to my Netcast

 

You can watch Shane and I's Cloudy Podcast live every Wednesday Morning at 10:00 am Central US time at http://www.toddklindt.com/Netcast

You can subscribe to the Podcast in the following ways:

MP3 Audio

Windows WMV video

YouTube Channel

 

 SPDocKit

 
June 2018
 

 Todd's Upcoming Events

 
  
  
There are no items to show in this view of the "Events" list.