If you read my last post, I was sent into a troubleshooting death spiral as a result of applying cumulative updates to SharePoint, the Workflow Manager, and Service Bus. Well the reason I went down that rabbit hole is because I was having workflow issues to begin with. Workflows were winding up in a ‘Suspended’ state with HTTP 401 and HTTP 404 errors. Unfortunately those errors told me nothing about the source of the problem – however the 401 and 404 errors indicated it was some sort of permission issue. The Event Log told me very little so I fired up the ULS log viewer on both the APP and WFE servers, started them in real time mode, fired off a workflow by creating a new item, and then stopped the ULS logs. Here is an error that finally led me to the problem…
Exception occured in scope Microsoft.SharePoint.SPListItem.UpdateWithFieldValues. Exception=System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
at Microsoft.SharePoint.SPGlobal.HandleUnauthorizedAccessException(UnauthorizedAccessException ex)
at Microsoft.SharePoint.Library.SPRequest.AddOrUpdateItem(String bstrUrl, String bstrListName, Boolean bAdd, Boolean bSystemUpdate, Boolean bPreserveItemVersion, Boolean bPreserveItemUIVersion, Boolean bUpdateNoVersion, Int32& plID, String& pbstrGuid, Guid pbstrNewDocId, Boolean bHasNewDocId, String bstrVersion, Object& pvarAttachmentNames, Object& pvarAttachmentContents, Object& pvarProperties, Boolean bCheckOut, Boolean bCheckin, Boolean bUnRestrictedUpdateInProgress, Boolean bMigration, Boolean bPublish, String bstrFileName, ISP2DSafeArrayWriter pListDataValidationCallback, ISP2DSafeArrayWriter pRestrictInsertCallback, ISP2DSafeArrayWriter pUniqueFieldCallback)
at Microsoft.SharePoint.SPListItem.AddOrUpdateItem(Boolean bAdd, Boolean bSystem, Boolean bPreserveItemVersion, Boolean bNoVersion, Boolean bMigration, Boolean bPublish, Boolean bCheckOut, Boolean bCheckin, Guid newGuidOnAdd, Int32& ulID, Object& objAttachmentNames, Object& objAttachmentContents, Boolean suppressAfterEvents, String filename, Boolean bPreserveItemUIVersion)
at Microsoft.SharePoint.SPListItem.UpdateInternal(Boolean bSystem, Boolean bPreserveItemVersion, Guid newGuidOnAdd, Boolean bMigration, Boolean bPublish, Boolean bNoVersion, Boolean bCheckOut, Boolean bCheckin, Boolean suppressAfterEvents, String filename, Boolean bPreserveItemUIVersion)
at Microsoft.SharePoint.ServerStub.SPListItemServerStub.InvokeMethod(Object target, String methodName, ClientValueCollection xmlargs, ProxyContext proxyContext, Boolean& isVoid)
at Microsoft.SharePoint.Client.ServerStub.InvokeMethodWithMonitoredScope(Object target, String methodName, ClientValueCollection args, ProxyContext proxyContext, Boolean& isVoid)
Boring, but Googling “sharepoint workflow Exception=System.UnauthorizedAccessException” led me to THIS link where a user mentioned that adding NT Authority\Authenticated Users resolved the problem for him. The problem seems to originate when access to the site is doled out to users using AD groups. While it works for users’ access to a site/list/library, it presents cross-lookup stuff within SharePoint workflows.
So short story long, I added NT Authority\Authenticated Users to the members group of the site and/or list and all was resolved.
I’m still not certain why my workflows were previously working without this. The fix is an easy one, but a difficult one to narrow down. Here are some other things I tried en route to this simple fix. It’s hard to say whether any of these things needed to be completed in addition to the above…
Refreshing timer job “refresh trusted security token services metadata feed” <deep breath> HERE.