U4-8249 - Azure Website not initializing Examine Index after scaling

Created by Giordano 29 Mar 2016, 11:45:11 Updated by Cameron W 07 Sep 2016, 05:55:00

We have only one instance enabled on Azure, and this problem happens sometimes, when Azure decides to change machine, terminating the current web application on the current server, and bringing a new instance on a new machine online.

From the log file, I see that the cold boot is started, but at the end no Examine indexes are created. And in the meanwhile SaveLastSynced(lastId) is executed, without actually having synced. This put the application in an invalid state.

I cannot understand why Examine indexes aren't created.. I attach the log when the application starts on a new machine, and my ExamineSettings.config.

Log:

2016-03-11 22:07:23,624 [P3412/D2/T1] INFO Umbraco.Core.CoreBootManager - Umbraco 7.3.7 application starting on RD000D3A22DBDF

2016-03-11 22:07:23,702 [P3412/D2/T1] INFO Umbraco.Core.PluginManager - Determining hash of code files on disk

2016-03-11 22:07:23,780 [P3412/D2/T1] INFO Umbraco.Core.PluginManager - Hash determined (took 83ms)

2016-03-11 22:07:28,888 [P3412/D2/T1] INFO Umbraco.Core.DatabaseContext - CanConnect = True

2016-03-11 22:07:28,916 [P3412/D2/T1] WARN Umbraco.Core.Sync.DatabaseServerMessenger - No last synced Id found, this generally means this is a new server/install. The server will build its caches and indexes, and then adjust its last synced Id to the latest found in the database and maintain cache updates based on that Id.

2016-03-11 22:07:28,963 [P3412/D2/T1] INFO umbraco.content - Load Xml from file...

2016-03-11 22:07:28,963 [P3412/D2/T1] ERROR umbraco.content - Failed to load Xml from file. System.IO.DirectoryNotFoundException: Could not find a part of the path 'D:\local\Temporary ASP.NET Files\root\25b6bfe3\4b69c977\UmbracoData\umbraco.config'. at System.IO._Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITYATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) at umbraco.content.LoadXmlFromFile()

2016-03-11 22:07:28,979 [P3412/D2/T1] INFO umbraco.content - Loading content from database...

2016-03-11 22:07:30,588 [P3412/D2/T1] INFO umbraco.content - Loading content from database...

2016-03-11 22:07:31,808 [P3412/D2/T1] INFO Umbraco.Web.Cache.CacheRefresherEventHandler - Initializing Umbraco internal event handlers for cache refreshing

2016-03-11 22:07:31,855 [P3412/D2/T1] INFO Umbraco.Web.Search.ExamineEvents - Initializing Examine and binding to business logic events

2016-03-11 22:07:31,855 [P3412/D2/T1] INFO Umbraco.Web.Search.ExamineEvents - Adding examine event handlers for index providers: 4

2016-03-11 22:07:31,871 [P3412/D2/T1] INFO MyProject.EventHandlers.MyProjectApplicationEventHandler - OnApplicationStarted: add OnGatheringNodeData to ExternalIndexer

2016-03-11 22:07:33,982 [P3412/D2/T1] INFO Umbraco.Core.CoreBootManager - Umbraco application startup complete (took 11087ms)

2016-03-11 22:07:34,636 [P3412/D2/T13] INFO umbraco.content - Save Xml to file...

2016-03-11 22:07:34,636 [P3412/D2/T6] INFO umbraco.content - Saved Xml to file.

2016-03-11 22:34:41,331 [P3412/D2/T35] INFO Umbraco.Core.Sync.ApplicationUrlHelper - ApplicationUrl: https://MyProject.azurewebsites.net/umbraco (using scheduledTasks/@baseUrl)

ExamineSettings.config

Comments

Giordano 29 Mar 2016, 11:46:12

Here you can find more information https://our.umbraco.org/forum/umbraco-7/using-umbraco-7/75289-azure-website-not-initializing-examine-index-after-scaling#comment-242825


Cameron W 07 Sep 2016, 05:54:37

We are seeing the same issue as well. Every time Azure migrates our web app to a new server - our Examine indexes are empty and do not rebuild despite the logs indicating that the indexes are being built:

2016-09-06 21:02:15,350 [P10180/D2/T1] WARN Umbraco.Core.Sync.DatabaseServerMessenger - No last synced Id found, this generally means this is a new server/install. The server will build its caches and indexes, and then adjust its last synced Id to the latest found in the database and maintain cache updates based on that Id.

2016-09-06 21:02:15,413 [P10180/D2/T1] WARN umbraco.content - Failed to load Xml, file does not exist.

2016-09-06 21:03:10,021 [P4692/D2/T11] WARN Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache - Could not retrieve media 4920 from Examine index, reverting to looking up media via legacy library.GetMedia method

2016-09-06 21:03:14,412 [P4692/D2/T11] WARN Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache - Could not retrieve media 4925 from Examine index, reverting to looking up media via legacy library.GetMedia method

We're running Umbraco version 7.4.3

We are using a 'staging' deployment slot in our Azure WebApp for deployments, which is warmed up before be swapped with the production slot. Not sure if it makes any difference or is similar to your setup Giordano.

Below is our ExamineSettings.config. We have set useTempStorage="true" for all indexers and searchers as recommended for flexible load balancing.

  <add name="InternalMemberIndexer" 
       type="UmbracoExamine.UmbracoMemberIndexer, UmbracoExamine"
       supportUnpublished="true"
       supportProtected="true"
       useTempStorage="Sync"
       tempStorageDirectory="UmbracoExamine.LocalStorage.AzureLocalStorageDirectory, UmbracoExamine"
       analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"/>

    <!-- default external indexer, which excludes protected and unpublished pages-->
    <add name="ExternalIndexer" 
         useTempStorage="Sync"
         tempStorageDirectory="UmbracoExamine.LocalStorage.AzureLocalStorageDirectory, UmbracoExamine"
         type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"/>

    <add name="PublicSiteIndexer" 
          type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"
          supportUnpublished="false"
          supportProtected="true"
          useTempStorage="Sync"
          tempStorageDirectory="UmbracoExamine.LocalStorage.AzureLocalStorageDirectory, UmbracoExamine"
          interval="10"
          analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
          indexSet="PublicSite"/>
</providers>
  <add name="PublicSiteSearcher" 
       type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
       useTempStorage="Sync"
       tempStorageDirectory="UmbracoExamine.LocalStorage.AzureLocalStorageDirectory, UmbracoExamine"
       analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" 
       indexSet="PublicSite" 
       enableLeadingWildcards="true"/>
</providers>
{code}


Priority: Normal

Type: Bug

State: Submitted

Assignee:

Difficulty: Normal

Category: Packages

Backwards Compatible: True

Fix Submitted:

Affected versions: 7.4.3

Due in version:

Sprint:

Story Points:

Cycle: