After 18 months of persistence, we’re finally able to migrate to SPO.
Up until yesterday, it wasn’t possible to migrate user data from SharePoint if the user was missing from Active Directory. It is now if you’re migrating to SharePoint online and using the new migration API!
Early on we ran into what was a pretty glaring problem for us, and I suspect for anyone else trying to migrate using a client side migration tool.
The Problem:
One of the data types in SharePoint is the “user” data type.
This is most commonly used/seen in document libraries – it shows you who last edited/updated a document.
It’s also commonly used as a field type in a SharePoint list.
For example, you might have a sharepoint list named “legal cases”
The problem is, using the client side API used for most migration tasks involving SharePoint Online, you can’t insert a name of a person if that name can’t be found in Azure Active Directory.
The 18 month Path to Resolution:
While it didn’t take Microsoft 18 months of actual work to fix this, it was about 18 months from start to finish.
I won’t bore you with the details, but the take away here is to never give up. We were told ‘no’ or ‘it can’t be done’ or ‘that’s by design’ multiple times. This was an important issue so we pressed on, reaching out to every contact we knew, from multiple levels within our organization to multiple levels within Microsoft. All that persistence paid of!
Sidenote
One conversation along the way that was particularly interesting was a special presentation Microsoft had arranged where we got to talk to the person who led Microsoft’s internal migrations from on premise to office 365. I was eager to ask them how they had moved list data with abandoned user data – I was certain they had some internal tool, or did some back door load that wasn’t available to outsiders, or maybe had some script that identified every instance of the missing users and recreated them in AD so the migration could complete. When I asked, they skirted the issue. I pressed on. After they told me, I understood why. They didn’t do it. When pressed they said they had to leave some data on premise because there was no way to move this data to SPO. On that day, I felt both validated and let down at the same time!
The Solution:
At ignite 2015, Microsoft announced a new Migration API for SharePoint Online.
On the same day, ShareGate announced that it would support this new API with it’s new ‘insane mode’. I spoke with a few people, some thought this new API would resolve the issue, while others said it would not – it didn’t at that time. 🙁
Shortly after our case must have reached one of the senior escalation engineers at Microsoft – I remember being told that the new API resolved the issue, then going back with evidence that it didn’t and I think that’s when traction really picked up. We supplied a Business impact statement and Microsoft added the fix to the list of things they were working on. The feedback I got down the road was that this ended up being a huge undertaking for them. It wasn’t nearly as easy as one would think, due to how SharePoint is structured internally. There were multiple setbacks, but we received excellent communication and updates. The time line didn’t bother me – I was thrilled to know it was being worked on.
Fast forward to fairly recently and we received word that the fix was approved and would be moved into our tenant. This was great news! Work didn’t end there however.
Microsoft went into depth about the work done to fix this issue.
While I had expected some new API, or an option when sending data like “override if missing” no such changes were needed. Microsoft updated the migration API to handle all of the needed back end stuff seamlessly. They did not update the CSOM. This meant that for this to work, the new API had to be used.
We were already using ShareGate coupled with insane mode which uses the API. I remember from past conversation with them that ShareGate uses a combination of insane mode and CSOM – even when insane mode is being used – I figured this would mean ShareGate would need to be aware of the changes to the migration API and would need to handle things differently. For example, in the past, ShareGate could replace a missing user with a user of our choice, this would no longer be necessary.
ShareGate was great to work with – they had long been aware of the user data migration issue and understood what I was talking about almost immediately. Once the API had been updated, the three of us worked together to ensure that Sharegate’s test tenant also had the new Mirgration API updates so they could code up a solution.
I’m soo pleased to say that Sharegate turned this around in about a month, and we received a beta last Friday. Even more impressive is that the very first beta from ShareGate with support for this worked like a charm!
Even more impressive is that the very first beta from ShareGate with support for this worked like a charm!
Sharegate has released version 5.8 with this functionality Today (2/29/2016).
Confirmed: Microsoft has already rolled this out to all O365 tenants. (as of 2/29/2016)
Special kudos and thanks to Brent Vezzoso and the unnamed hero’s within Microsoft who worked so tirelessly to make this happen. I can’t tell you enough how much this means to me, my company, the SharePoint community, and to Microsoft. You’ve done a great thing here!
