U4-2474 - Remove caching for user permissions

Created by Shannon Deminick 08 Jul 2013, 07:36:24 Updated by Shannon Deminick 01 Aug 2017, 07:53:10

Tags: Unscheduled

Relates to: U4-10075

Subtask of: U4-8632

Previous to v4.7 we never used to replicate permissions to individual nodes, after this version we did which was due to performance issues with looking up inherited permissions among a large set up nodes.

However one thing was overlooked, the fact that we cache permission sets for all users. Now that permissions are replicated this means that there can be an enormous amount of data in the permissions table and we seem to be caching all of it!!!

Need to remove this cache since it was only ever there to try to speed up the permissions lookup when they were not replicated.

Comments

Shannon Deminick 08 Jul 2013, 23:53:41

As a test I setup an environment with 3 users and lots of nodes and assigned permissions to each one generating 70,000 permissions records. The memory usage after loading this in was approximately 6 MB. The total number of umbracoNode's in the db to get this many permissions records is 1447. Therefore, if someone had 100,000 umbracoNode's with permissions assigned, then this figure could be quite large if permissions are assigned to top level nodes (and therefore replicate to all descendants).


Shannon Deminick 09 Jul 2013, 01:52:10

In the meantime, I've optimized how the current permissions caching is structured. Before we were putting the permissions in the same cache as the user cache, I have now separated this. The permissions cache is now tagged as BelowNormal priority and is cached for 20 minutes after which the system will go re-look them up for a given user. This should keep cache levels low when permissions in the back office is not needed.


Shannon Deminick 09 Jul 2013, 01:52:38

The changes mentioned above were implemented in 868edee5e647f6210e0a2507e615a93a2671f251


Sebastiaan Janssen 18 Jul 2013, 16:13:22

Shouldn't this issue be set to "fixed" and due in 6.1.3?


Shannon Deminick 19 Jul 2013, 01:24:06

Not really, I've just made the current caching structure work a bit better but as per above this can still cause giant memory consumption if there are a lot of nodes with permissions assigned. The correct way to do this is to do this:

Finalize the implementation of the ApplicationCache to handle request based cache. Then we we only store the permissions results for the current user in memory for the current request and don't have to go to the db for every child node. For the current request we will store the permissions of any requested node and it's direct children. Then if other child permissions need to be looked up, we will get the permissions for that node and it's direct children and add to the current request cache. Generally we will only need to lookup one node's child permissions but if we need to look up more, then it's only one SQL call for each child list.

By doing that we will save on all of that memory which could potentially be huge on big installs, it just means there might be an extra couple of (fast) SQL calls when rendering tree children but with the new optimized tree this should be no problem.

I guess we could also look at this another way and make all of these changes part of the permissions overhaul that we discussed at the retreat... probably wouldn't happen for some time though!


Sebastiaan Janssen 19 Jul 2013, 09:33:13

Gotcha, thanks!


Shannon Deminick 14 Jul 2017, 00:57:57

This is resolved with this issue: http://issues.umbraco.org/issue/U4-10075


Stephan 17 Jul 2017, 13:55:02

Indeed.


Priority: Normal

Type: Bug

State: Fixed

Assignee:

Difficulty: Normal

Category:

Backwards Compatible: True

Fix Submitted:

Affected versions: 4.8.0, 4.9.0, 4.10.0, 4.11.0, 6.0.0, 6.1.0, 4.9.1, 4.11.1, 4.11.2, 4.11.3, 4.11.4, 6.0.1, 4.11.5, 6.0.2, 4.11.6, 6.0.3, 6.0.4, 4.11.7, 6.1.1, 4.11.9, 6.0.5, 4.11.8, 6.0.7, 6.1.2, 4.8.1, 4.10.1, 4.11.10

Due in version: 7.7.0

Sprint: Sprint 63

Story Points:

Cycle: