Yearly Archives: 2013

ShareGate Can’t open a website – IN-017-006 The requested list or document library cannot be found. It may have been deleted.

I’ve run into this error a few times in ShareGate-

You try to open a Site Collection or web, and see this:

IN-017-006

SharePoint Designer is not much help either:

SPD

The solution, after much trial and error, was actually pretty simple!

But before I get to the solution, lets look at some things I tried that led me to the solution…

I searched the term “there are no items to show in this view” for SharePoint Designer on the internet.  This turned up a few possibilities – one being a corrupt /_vti_bin/Listdata.svc.

That wasn’t the case for me- some webs from the same site collection worked just fine – it was just one specific web that did not.

The other thing that turned up was the possibility of a corrupt list.  Often due to a missing feature needed by the list.

I turned to PowerShell and ran the following commands:

$web = get-spweb http://my.url.com/sites/mynoncooperatingweb
$web.lists

This started to return results, then errored out –

I was definitely on to something – there was something wrong with one of my lists.

Next I needed to figure out which one.

There are a few ways to do this – the easiest would likely have been to go to the website and choose “Site Actions”->”View all site content” and then just click on each one.

I didn’t do that.

Instead I mucked with PowerShell some more, I was able to tell from the error when I ran my above code which list was causing the problem.

My next step was to see if there was any data in the list.

Based on “Site Actions”->”View all site content” , it had an item count of zero, but I wasn’t sure I could trust this given the list was corrupt.

So I turned to SQL – I knew which Database it was in. (Get-SPContentDatabase -site $url will tell you this)

Select Top 1000 *
  From AllDocs
  Where 
     DirName like 'url/to/my/corrupt/document/library/%'
     Order By Dirname

This listed a few results,  but the results looked like what is usually found in the Forms folder – I didn’t for example, see anything at all that looked like end user data  or documentation.

Armed with this re-assurance that the list was not needed, it seemed the easiest way forward was just to delete the list – this would be easy – or so I thought….

I didn’t have any hopes of deleting the list through the UI, (though to be clear, it did show up under “Site Actions”->”View all site content” and clicking on that threw an error)

So I again turned to PowerShell:

I saw a delete method, so I called it without the ( ) to see what it was looking for, It needed a GUID.

I ran the command shown below to get a list of ID’s from my site – in my case, the second list was the corrupt one.

Then I put that guid in the delete function as shown: and got the error as shown:

powershell_delete_list

“List does not exists”

So that didn’t work.

Next I searched the internet for more information –

I found a post that mentioned the recycle bin. Ah Ha!

I looked at “Site Actions”->”View all site content” and then the recycle bin, which of course was empty.

So I went to Site Actions -> Site Settings -> then went to “Go To Top Level Site Settings” -> then  Site Collection Administration(heading) ->Recycle Bin.

After sorting by URL, I found the corrupt list there.

First I tried restoring the list, it threw an error.

Then I tried to delete the list – that worked, and put the item in the Second stage recycle bin.

Next I went to the 2nd Stage “Deleted from end user Recycle Bin” area and deleted it from there.

Back in Site actions -> “View all site content” the list no longer showed up.

I relaunched SharePoint Designer and I am again able to bring up the list of lists and libraries.

I again tried a migration in ShareGate and it’s purring along like a kitten.

So to make my long story short, the error in ShareGate was caused by a corrupt/broken list – this same error affected SharePoint Designer (they both presumably use the same web service interface to get information from SharePoint) and to a degree, it even affected looking at the lists in PowerShell.

I suspect there is a bug somewhere, but I suppose it’s possible the user hit delete at the exact instance the application pool restarted or something like that.

At the time this happened we were running  SP2010 SP1 + June 2011 CU – if you run across a similar situation, please leave a comment with your version of SharePoint, hopefully this is addressed in SP2!

– Jack

I’m Presenting at SharePoint Fest Chicago on October 7th

On October 7th, I’m teaming up with Michael Blumenthal to give a half day workshop on using PowerShell with SharePoint.  Michael’s got some great material and between the two of us you should walk away with a firm introduction as well as a walk through on a few of the more useful scripts from this site.

The Workshops are optional and are the day before the official conference begins (Workshops are on Monday, Conference is Tuesday and Wednesday)  Although there is an extra cost for the workshops, I think you get more for your money on workshop day as the sessions benefit in 2 big ways:

  • Workshops are Longer – basically a half day vs 1 hour sessions during the conference.
  • Workshops are more intimate – where there might be 100 people in a 1 hour session, expected attendance for workshop sessions is typically half that.

I hope to see you there!

http://www.sharepointfest.com/Chicago/

UPDATE: the presentation, scripts and handout PDF are available for download here: SPFestHandout.zip

 

Simple PowerShell Script Logging

Here’s a very simple logging mechanism I’ve used before.

Disclaimer: I’m sure someone, somewhere has written a better one – if so please leave a link the comments.

Overview:  I wanted a simple logging function that would both display to the screen, and also write to disk.  I also wanted to be able to set the display color on output.

The logging function is simple and writes to disk and to the screen.

#PowerShell Logging Script
#SharePointJack.com

#Tip, if viewing on my blog, click the full screen icon in the toolbar above

# "Global" variables:
# the filename is scoped here
# this creates a log file with a date and time stamp
$logfile = "C:\YourLogFileNameGoesHere_$(get-date -format `"yyyyMMdd_hhmmsstt`").txt"

#this is our logging function, it must appear above the code where you are trying to use it.
#note there is a technique to get around needing this at the top, read the blog post to find out more...
function log($string, $color)
{
   if ($Color -eq $null) {$color = "white"}
   write-host $string -foregroundcolor $color
   $string | out-file -Filepath $logfile -append
}

# examples:
# log something
log "this is a simple output string, it will appear white"

# log with color on screen:
log "This string will appear yellow on screen" yellow
log "This will appear red" red

# powershell shortcuts useful for building strings:
$myvariable = "hello"
log "$myvariable world"

# include double quotes in your string:
log "`"this was quoted`""  #NOTE: This character is the tick (top left of a US keyboard) - it doesn't look like it comes across in this blog.

# use more than simple variables in a string:
$cmds = get-command
log "there are $($cmds.count) commands available"

 

Note: To make the above example easy and simple, I put the log function at the top.
In Powershell, functions must always come before the code that calls them.
An easy way to get around this is to wrap your code in a function and call that function at the very bottom of the ps1 file
Like this:

#Example of how to put functions below your own code.
#SharePointJack.com

$logfile = "C:\YourLogFileNameGoesHere_$(get-date -format `"yyyymmdd_hhmmtt`").txt"

Function Main()
{
   log "In Main Function" green
   #do stuff that calls a function here.
   $commands = MySpecialFunction
   log "$($commands.count) commands found!"
}

Function MySpecialFunction()
{
   log "I'm in a function" yellow
   $commands = get-command  
   return $commands
}

function log($string, $color)
{
   if ($Color -eq $null) {$color = "white"}
   write-host $string -foregroundcolor $color
   $string | out-file -Filepath $logfile -append
}

Main  #the last line of your script should call your Main function up top.

Note that the $logFile line was intentionally left on top, since it needs to be available globally.

I have a video that covers the above scripts, as well as Start-Transcript and Stop-Transcript

SharePoint 2010 SP2 installation trouble

“let no good deed go unpunished”

I’m not sure who said that, but it certainly seems true with SharePoint Patching – It’s Sunday night and I thought I would “be nice” and upgrade our development environment over the weekend so the developers don’t loose a day during the week.

What was I thinking…

My install:

2 SharePoint boxes plus one FAST box.

I had downloaded 4 updates:

– SharePoint Server SP2

– Language Packs for office products SP2

– Office Web Apps SP2

– Fast Server SP2

Progress:

The FAST server SP2 installed on the fast server pretty quickly – it asked for a reboot no trouble there – I didn’t see any indication that there were any post setup tasks to run, but then again I don’t know that I looked very hard.

The Problem Children: my two SharePoint boxes.

I ran setup on the 3 remaining SP2 installers in this order: Server, Language, OWA. The installs went pretty quick thanks to a tip I had heard on Todd Klindt’s netcast last week about stopping a bunch of services prior to launching setup.  Luckily, I had also run a script I had leftover from SP1 that backed up key files and took an inventory of running services, content databases, etc…

With the 3 binaries installed, It was time to run the Dreaded “SharePoint Products Configuration Wizard”.  I say dreaded, because looking back, I can’t think of a single time it’s ever worked the first time after an upgrade. Something has always gone wrong and this Do-gooder’s Sunday night was no exception.

At stage 9 of 10, it threw an error saying the SharePoint Admin Service (SPADMINV4)  wasn’t started.
Piece of cake, I’ll just start it. Wait, no I can’t – it won’t start.

A few searches on the internet and I found this article:

http://support.microsoft.com/kb/2756815

It mentioned the Claims to Windows token service not starting either – hmm, that seemed to be the case too.

I checked a different environment that hadn’t been patched with SP2 and sure enough the claims to windows token service was running on that one, but it wasn’t running on my dev environment. Hmmm.

Do you remember above where I mentioned I had an inventory of what services were running before the upgrade??? Guess what – Claims to windows token service (C2WTS) was not running BEFORE my Upgrade, yet SP 2010  PRE SP2 seemed to be working just fine.

So perhaps there is something in SP 2010 SP2’s admin service that requires the C2WTS service.  Or maybe that came with SP1, and I thought I had SP1 but was running RTM. Who knows.

Anyhow back to the fix, I used the second solution in KB 2756815 – My servers are behind a proxy server, and can’t get out to the internet so the C2WTS service gets mad when it can’t check the internet for the latest and greatest gossip about what certificates have fallen out of favor. In fact C2WTS gets so mad, it refuses to start – which is where KB2756815 comes in – I was able to change a local policy setting and after I did that, C2WTS started right up and after that, The SharePoint Admin Service (SPAdminV4) started as well.

As I write this, the Farm is in upgrade task step 9 of 10 and its 41.17% done- Hopefully it will finish up, then I can run it on my second box, and start upgrading content databases and be on my merry way.

But of course not…

Somewhere after about 44% I stepped away only to return and be greeted by the “Configuration Failed” screen – again.

2 problems seen in the logs:

    • The user profile sync service would not start – I actually saw this flash across the screen during the operation of the wizard – it nicely said that I could/should start that after the install was done.
    • The second issue: A timeout, Looks to be related to the user profile service.

The error mentioned setting the USP Sync Service Instance to Offline, so maybe I’ll get lucky and it will work the second time I run the wizard.

For Kicks, I’m running the wizard once on the second machine to see if it fairs any better. The second machine isn’t running User Profile Sync, which I don’t think should matter as much of the upgrade work is done via timer jobs and should work across all nodes in the farm, but just in case, it’s worth trying – I could use a success.

And, I got one! The second node finished ok, so now back to the first node.

Success again! Re-running the Products config wizard on the first node a second time also worked! Now all I need to do is start that user profile service… I tried it from windows services, but it didn’t start – not a big deal since I know that quite a bit goes on when provisioning User Profile Sync and remember above, SharePoint mentioned that it put UPS offline.

Next I try to Central Admin->”Services on Server” I find the service, click start and am asked for the farm account password – on the password screen in nice red letters, it says that after I provision the service I need to do an IIS reset. I think to myself “I haven’t done anything yet” and I enter the needed password and click OK.
I am immediately greeted by one of those SharePoint error screens with a correlation ID on it. Off to look at the ULS logs.

System.Data.SqlClient.SqlException: A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 0 – An existing connection was forcibly closed by the remote host.)

Hmm, makes me wonder if something else is wrong.

I checked a few other SP Service Apps and they seemed fine.

Is it worth trying again? I go back to services on server and click start – I see the Red IISreset message and figure it can’t hurt to do it BEFORE – so I give that a shot, and guess what, I am able to click OK and not get an error – the service now shows as “Starting”

This doesn’t bother me – I’ve done User profile sync in 2010 a few times and I know it takes something like 10 minutes to finish setting up FIM in the background, but it’s taking a long time and I’m getting nervous…

I check the windows services – FIM Sync is started, FIM itself is still disabled. Hmm – I can’t say I’ve ever watched it so I don’t know which one gets put up first, so I wait a while longer, then I try a reboot, still no luck. Time for some powershell

I look at the user profile sync services in powershell with this command:

Get-SPServiceInstance | where { $_.typename -like ‘User Profile Synchronization *’ }

I get a line back that shows it is “Provisioning” – I’ve seen this before so it’s time to Kill that and try again.

I did a Stop-SPServiceInstance followed by the ID of the “Stuck” Sync Service.

Now back to Central admin for another try at starting it.

Still no luck.

Spencer Harbar is pretty much the expert on this and his blog says that if we’re in the situation we’re in, that we should just go ahead and rebuild the user profile service from scratch.  There is only one problem here – we use Newsgator, a 3rd party add in, that relies heavily on User Profiles – So I’ve got a support ticket open with them to find out if UPS gets reset, if it will break a bunch of associations.

I’ll hopefully have an update tomorrow.

Update:

The issue turned out to be related to Duplicate Certificates for ForeFront.

Here are a few steps I took to get from Problem to Resolution.

  • Enabled verbose logging for User Profile Service in Central Admin
  • Looked at the Windows Event logs
  • Reset the SharePoint Timer Cache
  • Started the User Profile Sync Service on a different Node.

I noticed a few additional things:

In the Event Log, Event ID 234 from “ILM Web Service Configuration”

ILM Certificate could not be created: Cert step 2 could not be created: C:\Program Files\Microsoft Office Servers\14.0\Tools\MakeCert.exe -pe -sr LocalMachine -ss My -a sha1 -n CN=”ForefrontIdentityManager” -sky exchange -pe -in “ForefrontIdentityManager” -ir localmachine -is root

ILM Certificate could not be created: Cert could not be added: C:\Program Files\Microsoft Office Servers\14.0\Tools\CertMgr.exe -add -r LocalMachine -s My -c -n “ForefrontIdentityManager” -r LocalMachine -s TrustedPeople

I also noticed that the UPS Sync worked just fine on the other node, so that was good news.

MS had suggested that the FIM certs might have been messed up so I looked at certificates on both the working and non-working systems

What I found was that the Non-working system had more than one certificate.

MS said it was safe to delete all the forefront certs so that’s what I did, but it still didn’t work.

As it turned out, there was more than one place the certificates

The FIM certificates were found in two locations:

  • Certificates (Local Computer) -> Personal -> Certificates -> ForefrontIdentityManager
  • Certificates (Local Computer) -> Trusted Root Authority -> Certificates -> ForeFrontIdentityManager

I had deleted the ones from Personal, but not the Trusted Root Authority

http://www.cleverworkarounds.com/2010/08/15/more-user-profile-sync-in-sp2010-certificate-provisioning-issues/  Does a great job explaining the certificates so no need to rehash it here.

So in summary, Extra ForeFront certificates in the certificate store were the reason I couldn’t start the User Profile Sync Service.

 

 

 

 

Powershell to find all the Access Database Sites on your Web Application in SharePoint

I needed to track down all the access DB’s and wasn’t sure how to do it.

One of our developers came up with a script which I modified a bit.

Original credit goes to Ravi Konaparthi for the script.

This script will look for Access Services DB’s in a SharePoint 2010 Farm.
(This version only looks at a given Web application, but you can easily modify it to search everything – have a look at the comments)

$snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
if ($snapin -eq $null) 
{
	Write-Host "Loading SharePoint Powershell Snapin"
    Add-PSSnapin "Microsoft.SharePoint.Powershell"
}

#Foreach loop to loop through all webs, and compare the web template and write findings to .csv file.

function Get-AccessDB()
{
    log "In Get-AccessDB"
    $url = getURL
    #NOTE - if you want to look at ALL Sites and not just a specific web application, just remove -webapplication $url from the line below.
    $sites=Get-SPSite -webapplication $url  -limit all
    foreach($Site in $sites)
	{
		log "Inspecting Site $($site.url)"
		foreach($web in $site.AllWebs)
		{
			log "inspecting web $($web.url)"
			if ($web.WebTemplate -like "*ACCSRV*")
			{
				log "Found Access DB in $($web.url)" "Green"
				$hash = @{"[URL]"=$web.Url;}
				New-Object PSObject -Property $hash | Sort-Object
			}
		}
	$site.Dispose()
	}
}

function log($txt, $color)
{
  if ($color -eq $null) { $color = "White" }
  write-host $txt -foregroundcolor $color
}

function getURL()
{
    switch(hostname)
	{
		"SPDEVPC" {$machineURL = "http://yourDevURL.yoursite.com"}
		"SPStagePC" {$machineURL = "https://yourStgURL.yoursite.com"}
		"SPProdSrv" {$machineURL = "https://yourProdURL.yoursite.com"}
		default {write-host "This should be run from either SPDEVPC, SPSTAGEPC or SPPRodSRV - Press any key to exit this script"; read-host; exit}
	}
	return $machineURL
}

Get-AccessDB | Export-csv D:\accessdb.csv

 

Send an email from PowerShell

I’ve used similar code in a few scripts, I have here – I’m putting this post here as a fast reference for those times when I want to copy/paste some email code…

$EmailFrom = "youremail@yourdomain.com"
$EmailTo = "systemreplyemail@yourdomain.com"
$EmailSubject = "Email Subject"
$EmailBody = "Email Body"
$SMTPServer = "yoursmtpserver.yourdomain.com"
$logfile = $filenameToAttach

Send-MailMessage -From $EmailFrom -To $EmailTo -Subject $EmailSubject -body $EmailBody -attachments $Logfile -SmtpServer $SMTPServer

Note this is just a basic example, as always get-help send-mailmessage is the best way to get some additional info on attachments, multiple recipients, credentials etc…

 

SharePoint 2010 Service Pack 2 (SP2) download links

SharePoint 2010 Service Pack 2 was released this month.

I’ve seen a few posts on blogs, but none so far were complete.

Here’s a list, I gathered this from: http://support.microsoft.com/kb/2687524
Note: This page took forever to load – in fact I almost closed it since it didn’t seem to have any relative information, then data showed up – they must build the page with an AJAX callback – it has quite a lot of detail!

I’ve organized the list below by product configurations so a few items are repeated (ie you’ll see the language packs in both the SP standard and Project server section)

Foundation / Free:

SharePoint 2010 Foundation SP2: wsssp2010-kb2687464-fullfile-x64-en-us.exe KB article: 2687464

SharePoint 2010 Foundation Language Pack SP2: wsslpksp2010-kb2687466-fullfile-x64-en-us.exe KB article: 2687466

Standard and Higher:

SharePoint 2010 Server (includes Foundation) SP2: oserversp2010-kb2687453-fullfile-x64-en-us.exe  KB article: 2687453

MS 2010 series Server products Language Pack SP2: oslpksp2010-kb2687462-fullfile-x64-en-us.exe  KB 2687462

Office Web Apps SP2: wacsp2010-kb2687470-fullfile-x64-en-us.exe  KB article: 2687470

FAST search for SP2010 SP2: fsserversp2010-kb2687446-fullfile-x64-en-us.exe  KB article: 2687446 (Fast search was a separate product offering, higher end than what came bundled with SharePoint 2010)

Project Server:

SharePoint/Project Server 2010 SP2: svrprjsp2010-kb2687452-fullfile-x64-en-us.exe KB article: 2687452

MS 2010 series Server products Language Pack SP2: oslpksp2010-kb2687462-fullfile-x64-en-us.exe  KB 2687462

“Other” servers/connectors etc:

FAST search for SP2010 SP2: fsserversp2010-kb2687446-fullfile-x64-en-us.exe  KB article: 2687446 (Fast search was a separate product offering, higher end than what came bundled with SharePoint 2010)

Microsoft Search Server 2010 SP2: sserversp2010-kb2687461-fullfile-x64-en-us.exe KB article: 2687461
(Note: Search Server 2010 SP2 was a separate offering by MS to allow search without needing sharepoint – if you are using the search features of SharePoint, you’ll just use either the foundation or server versions of SP2 above)

Microsoft Groove Server 2010 SP2: grssp2010-kb2687448-fullfile-x64-en-us.exe KB: 2687448

SharePoint 2010 Indexing Connector for Documentum SP2: dctmsp2010-kb2687459-fullfile-x64-en-us.exe KB: 2687459

MS Duet Ent. for SP2010 and SAP SP2: duetsrvsp2010-kb2687445-fullfile-x64-en-us.exe KB: 2687445

Installation tips: 

Much has been written about this, and I’ve seen some conflicting advice. I don’t have a “right answer” here – but I will say it’s a good idea to be prepared:

    • Practice on a test environment first.
    • take a backup of your IIS directory, especially the web.config files on your web front ends
    • Take a backup of your “hive” (c:\program files\common files\microsoft shared\web server extensions\14\*) there may be customization here.
    • Take backups of every DB, especially the content DB’s
    • Don’t forget about 3rd party DB’s like newsgator, K2, etc.

When possible:

    • If your SharePoint infrastructure is Virtualized, you can get a fool proof backup by doing this:
      • Shut down all your VM’s and then take a snapshot in the powered off state.
      • Take SQL backups while the VM’s are off.

The above steps might be a bit extreme, and maybe aren’t necessary, but they do give the best option of pain free recovery if you have to roll back. The logic being you can restore it to the exact same state (that of a shutdown system) – Every other backup method involves backing up (and subsequently restoring) a moving target…

Update:

This might be a good technique to follow when patching – basically it proposes shutting down the IISAdmin and SPTimerServces, and stopping or pausing the Search Services prior to patching then starting them back up afterwards.

http://blogs.msdn.com/b/russmax/archive/2013/04/01/why-sharepoint-2013-cumulative-update-takes-5-hours-to-install.aspx

– Jack

 

 

 

 

 

 

 

PowerShell to send an email when something happens in the Event Log.

Most larger companies have SCOM or some other monitoring tool in place to watch over SharePoint servers and let the right people know when things go wrong.

That said, there are times when you just need a quick and dirty way to get notified when something shows up in the event log – It might be something temporary, or something you are asked to watch – but to keep under the radar – Or maybe the SCOM team has a backlog and you need to know you’ll be alerted before you go home for the day.

Here is an extremely simple script that can be used in conjunction with Event Viewer to send an email.

Here’s how it works:

1) In Event viewer, find the event log you want to watch. Find the event you are looking for and right click it and choose “Attach a Task To this Event…”

2) Follow the wizard and when you get to the part about what to do, you can choose the default email option, or if you want to add some additional logic, choose to “Run a program”

3) Copy the contents of the script below to a file with a .ps1 extension and alter the script to fit your specific use case.

4) Specify c:\windows\system32\WindowsPowerShell\v1.0\powershell.exe as the name of the executable (do this even if you are using a later version of powershell)

5) Specify the path and name of the script file you created above as the parameter.

$event = get-eventlog -LogName Application -source "Put your source here" -newest 1
#get-help get-eventlog will show there are a handful of other options available for selecting the log entry you want.
if ($event.EntryType -eq "Error")
{
    $PCName = $env:COMPUTERNAME
    $EmailBody = $event.Message
    $EmailFrom = "Your Return Email Address <$PCName@yourdomain.com>"
    $EmailTo = "youremail@yourdomain.com" 
    $EmailSubject = "Your Event Log event was found!"
    $SMTPServer = "mailserver.yourdomain.com"
    Write-host "Sending Email"
    Send-MailMessage -From $EmailFrom -To $EmailTo -Subject $EmailSubject -body $EmailBody -SmtpServer $SMTPServer
}
else
{
    write-host "No error found"
    write-host "Here is the log entry that was inspected:"
    $event
}

Now, anytime the event you specified is found in the log, your script will be triggered.

The first line of the script fetches the most recent log item that matches some core criteria, it’s important to refine this as much as possible, so that you’re not emailing yourself details of event you want and not an event that showed up a split second later.

Final Thoughts:

This is a simple solution, for simple scenarios. I can attest that it’s worked great for me, for my limited scope scenario.  My company also uses SCOM to alert us on the more normal SharePoint issues – I’m not advocating this as a replacement for proper monitoring, but used in the right scenarios, it can be an extra tool available to you.

– Jack

Automating SharePoint Deployments in Windows Azure using PowerShell

There’s a nice article on MSDN titled “Automating SharePoint Deployments in Windows Azure using PowerShell

The article talks about building up a sharepoint environment in Azure. The main difference between this and setting up an office 365 instance that I wrote about in my last post, is that in Azure, you’re in complete control over all the VM’s that make up your environment. If you have MSDN you get a monthly spending allowance on Azure, so you should be able to play around with this stuff without incurring any costs (So long as you don’t leave everything running 24×7) – Seems perfect for test environments. At the bottom of the article are links to a few articles that discuss using PowerShell with Azure.