U4-1750 - Sorting is Slow

Maybe it's related to http://issues.umbraco.org/issue/U4-1714

Sorting only 20 nodes in Umbraco v6.0.3 took 12 minutes

I can't reproduce that, sorting 20 nodes took about 10 seconds here. Is it part of a site with many nodes maybe where it needs read/write a lot (especially descendants) to the umbraco.config? Tried it with a site with about 50 nodes in total (30 of them under the 20 nodes that I sorted).

I'm experiencing lengthy execution times when sorting 206 nodes on an otherwise very small brochure site. They're not nested (all at level 3 in a single parent folder) and the nodes have 1 text property and 2 media picker properties, so they're tiny! So far the sorting is more than 10 minutes in, and still running. Umbraco 6.0.3, clean build, SQL Express 2008 R2.

I'm getting this error: ERROR umbraco.presentation.webservices.nodeSorter - [Thread 36] An error occurred

Trying to sort 100+ of nodes I get this error in my log almost every time.

It's also probably connected with this one: http://our.umbraco.org/forum/core/general/40361-v603-sorting-very-slow-%28275-children%29

In attach I've pasted whole error from a log.

Similar issue in 6.0.4: 2013-04-27 10:07:13,929 [16] ERROR umbraco.presentation.webservices.nodeSorter - [Thread 96] Could not update sort order System.Threading.ThreadAbortException: Thread was being aborted. at System.Runtime.CompilerServices.RuntimeHelpers._CompileMethod(IRuntimeMethodInfo method) at System.Reflection.Emit.DynamicMethod.CreateDelegate(Type delegateType) at Umbraco.Core.Persistence.Database.PocoData.GetFactory(String sql, String connString, Boolean ForceDateTimesToUtc, Int32 firstColumn, Int32 countColumns, IDataReader r) at Umbraco.Core.Persistence.Database.FindSplitPoint(Type typeThis, Type typeNext, String sql, IDataReader r, Int32& pos) at Umbraco.Core.Persistence.Database.CreateMultiPocoFactory[TRet](Type[] types, String sql, IDataReader r) at Umbraco.Core.Persistence.Database.GetMultiPocoFactory[TRet](Type[] types, String sql, IDataReader r) at Umbraco.Core.Persistence.Database.d__1c1.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at umbraco.cms.businesslogic.datatype.DefaultData.LoadValueFromDatabase() at umbraco.cms.businesslogic.datatype.DefaultData.get_Value() at umbraco.cms.businesslogic.datatype.DefaultData.ToXMl(XmlDocument data) at Umbraco.Core.Models.PropertyExtensions.ToXml(Property property) at Umbraco.Core.Models.ContentExtensions.ToXml(IContentBase contentBase, String nodeName) at Umbraco.Core.Models.ContentExtensions.ToXml(IContent content) at Umbraco.Core.Services.ContentService.Save(IContent content, Boolean changeState, Int32 userId, Boolean raiseEvents) at Umbraco.Core.Services.ContentService.Save(IContent content, Int32 userId, Boolean raiseEvents) at umbraco.cms.businesslogic.web.Document.Save() at umbraco.presentation.webservices.nodeSorter.UpdateSortOrder(Int32 ParentId, String SortOrder)

It's reallyyyy slow for me too since Umbraco 6.xx. (we are at 6.0.3)

The editors are doing a simple sort by moving the position of one node to the top. It takes 4 minutes (it used to take less than 30 sec in 4.8.1).

We do have a lot of nodes. But not directly under the same parent

I have for example:

ERROR umbraco.presentation.webservices.nodeSorter - [Thread 24] An error occurred System.Threading.ThreadAbortException: Thread was being aborted. at System.Runtime.CompilerServices.RuntimeHelpers._CompileMethod(IRuntimeMethodInfo method) at System.Reflection.Emit.DynamicMethod.CreateDelegate(Type delegateType) at Umbraco.Core.Persistence.Database.PocoData.GetFactory(String sql, String connString, Boolean ForceDateTimesToUtc, Int32 firstColumn, Int32 countColumns, IDataReader r) at Umbraco.Core.Persistence.Database.FindSplitPoint(Type typeThis, Type typeNext, String sql, IDataReader r, Int32& pos) at Umbraco.Core.Persistence.Database.CreateMultiPocoFactory[TRet](Type[] types, String sql, IDataReader r) at Umbraco.Core.Persistence.Database.GetMultiPocoFactory[TRet](Type[] types, String sql, IDataReader r) at Umbraco.Core.Persistence.Database.d__1c1.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at umbraco.cms.businesslogic.datatype.DefaultData.LoadValueFromDatabase() at umbraco.cms.businesslogic.datatype.DefaultData.get_Value() at umbraco.cms.businesslogic.web.Document.Save() at umbraco.presentation.webservices.nodeSorter.UpdateSortOrder(Int32 ParentId, String SortOrder)

UPDATE : The sorting works for some nodes, but it doesn't work for others. I didn't find out what is different yet...

UPDATE : On dev or staging environment, the sorting works normal. On the LIVE site however (=>meaning lot's of users and editors), when I sort some nodes, the dialog is showing "loading" or "sorting" (I don't remember) but it never ends. However if I close the dialog after 1 min and Reloads the node, then the sorting did work. Also, on some other nodes, the dialog does close...

It seems that the "Sort" does : "save" or "save and publish" on all sorted nodes and it times out and/or give an error on one of the Save.

This is still an issue in 6.0.5. The nodes get sorted, but the dialog doesn't return with a success message. Same error trace:

2013-05-21 15:10:09,253 [10] ERROR umbraco.presentation.webservices.nodeSorter - [Thread 118] Could not update sort order System.Threading.ThreadAbortException: Thread was being aborted. at System.Runtime.CompilerServices.RuntimeHelpers._CompileMethod(IRuntimeMethodInfo method) at System.Reflection.Emit.DynamicMethod.CreateDelegate(Type delegateType) at Umbraco.Core.Persistence.Database.PocoData.GetFactory(String sql, String connString, Boolean ForceDateTimesToUtc, Int32 firstColumn, Int32 countColumns, IDataReader r) at Umbraco.Core.Persistence.Database.FindSplitPoint(Type typeThis, Type typeNext, String sql, IDataReader r, Int32& pos) at Umbraco.Core.Persistence.Database.CreateMultiPocoFactory[TRet](Type[] types, String sql, IDataReader r) at Umbraco.Core.Persistence.Database.GetMultiPocoFactory[TRet](Type[] types, String sql, IDataReader r) at Umbraco.Core.Persistence.Database.d__1c1.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at umbraco.cms.businesslogic.datatype.DefaultData.LoadValueFromDatabase() at umbraco.cms.businesslogic.datatype.DefaultData.get_Value() at umbraco.editorControls.CsvToXmlData.ToXMl(XmlDocument data) at Umbraco.Core.Models.PropertyExtensions.ToXml(Property property) at Umbraco.Core.Models.ContentExtensions.ToXml(IContentBase contentBase, String nodeName) at Umbraco.Core.Models.ContentExtensions.ToXml(IContent content) at Umbraco.Core.Services.ContentService.SaveAndPublishDo(IContent content, Boolean omitCacheRefresh, Int32 userId, Boolean raiseEvents) at umbraco.cms.businesslogic.web.Document.SaveAndPublish(User u) at umbraco.cms.businesslogic.web.Document.Publish(User u)

Fixed in changeset e301945b2504

Additional update in changeset 953c466faf8f
Now we only save and/or publish items that have actually changed, previously we republished each item being sorted, but generally most items do not change their sort order so it's much faster to skip publishing for those items.

This is listed as a breaking change because we're using the new ContentService API that doesn't fire the Save and Publish events when items get sorted. So if you're listening to those specifically because you want to handle those after sorting items then you'll need to update your code to listen to ContentService.Saving/Saved and ContentService.Publishing/Published

I just tried to test this out on the latest 6.1.0 changeset, but there looks to be some problems. When I sort, neither the tree nor the XML cache updates. Though if I open the sort dialog again, it shows in the newly sorted order.

Here's a screencast: http://www.screenr.com/Fq7H

Is it just me? :)

@Tom Hmmm, works on my machine.. :-S is this an upgrade?

Fresh install - built that changeset, copied Umbraco.Web.UI to a new IIS site, installed a starter kit. You doing anything different? I can try to dig deeper.

@Tom Whoops, was behind in hg, can repro now, know what it is! Thanks for the test! :-)

Fixed in 6659e8a582bb for realz .. it was my bad, so have been sitting in the corner to figure out what I did wrong. Sorry!

Haha, no worries. Confirmed - works again :) Seems speedy on a starter kit site, not tested on a real site yet.