U4-7048 - Umbraco 7.3 RC. Templates not read from disk.

Created by Peter Gregory 01 Sep 2015, 01:57:51 Updated by Stephan 15 Sep 2015, 08:02:46

Created template in back office. Then overwrote template on disk. New template not picked up in back office. old template still displayed. Coming from DB rather than from disk?

Only seems to happen to templates that have a master set.

Comments

Peter Gregory 01 Sep 2015, 02:00:42

If you update and save any of the templates all templates update.


Stephan 04 Sep 2015, 08:02:31

Cannot repro for a standalone MVC view, nor a standalone WebForms master. Could repro for a with-a-master WebForms master. Not sure about with-a-master MVC view. So, confirming that something's wrong, and looking into it.


Stephan 04 Sep 2015, 09:19:08

Edit: can repro for any type of template. Edit in Settings, save something, then edit the file on disk, then go back to Settings and the edits are not visible. Probable cause is the in-memory caching of templates, that fails to detect that the actual disk file has changed.


Stephan 08 Sep 2015, 09:33:47

More details...

Root cause

Templates (as well as scripts, partial views and stylesheets) are IFile entities, which are memory-cached in their corresponding repositories. Which means that once the entity has been loaded once, it will not change until the cache is cleared, which does not happen if the disk file only is modified.

This is similar to what happens when some content is changed by editing it directly in the database. So, one answer could be that the behaviour is expected and normal. And to make sure there is a way to notify Umbraco that some files have changed, so that the cache can be cleared. However, files are different from database, as it is quite usual to edit files using a tool other than Umbraco's backend.

Therefore, we must make sure that we do not cache the content of templates, scripts, etc. in the runtime cache. This, in addition, makes sense as it is of no use to have all the texts of all templates, scripts, etc. in-memory where they would be essentially never used.

Fix

When an entity is stored into the runtime cache, it is deep-cloned. And when it is retrieved from that cache, it is deep-cloned too. We make sure that content is not cloned. Therefore, the cached entity has no content, and retrieved entities have no content. Getting the content of an entity triggers a lazy loading of that content from disk.

Loading the content from the entity, means that the entity needs to have access to the IFileService (or, at least, to its repository, or the underlying filesystems). However, it is valid to write code such as the following, and we do not want to obsolete it:

var script = new Script("script.js");

That created entity has no way of reaching the service/repository/filesystem. Therefore, it all works that way:

  • the File object has a (internal) GetFileContent property which is a func that can lazy-load content
  • when a File is created by the repository, it is initialized with a func that can lazy-load content from disk
  • when a File is created using its ctor (as above), the func is null and it manages its content in memory
  • when a File is saved, the repository ensures it has a proper func so that further clones can lazy-load content from disk

This applies to anything that is a File, ie a Stylesheet, Script, PartialView or Template.

Caveat

When the file content is loaded, we should ''in theory'' also refresh the UpdateDate of the entity. However, there is no way we can do it properly without conflicting with the dirty properties tracking mechanism. So, that date is never going to change.


Stephan 08 Sep 2015, 09:38:31

Pushed e577648efdcf0d61ac9e994926dce12f4aa84ae4.

Warning: the code also refactors and clears the PhysicalFileSystem, addressing some security issues. Which means we are much more strict wrt the filesystem. All tests are passing, though.


Shannon Deminick 08 Sep 2015, 09:39:15

We have a way to disable dirty property tracking and re-enabling it... not sure if that would allow us to make this work or not ? (we do this in the deep cloning logic of an entity)


Stephan 08 Sep 2015, 09:43:06

Damn, I ''know'' and yet I overlooked that one. Assuming I already have (or maybe not) some dirty properties... does it "just" suspend the tracking, eg would the following code transparently update the date without tracking?

file.DisableChangeTracking(); file.UpdateDate = DateTime.Now; file.EnableChangeTracking();


Shannon Deminick 08 Sep 2015, 11:01:25

IIRC :) might be worth adding an assert to a test or something


Stephan 08 Sep 2015, 15:58:44

Pushed 6b5057b8708eb82f2af278225e3e0075dbb7f2b4 doing more fixes. The update date trick above is still commented out as I'd need to test that it works - will remain TODO for the time being.


Stephan 15 Sep 2015, 08:02:42

Closing in its current state for 7.3.


Priority: Major

Type: Bug

State: Fixed

Assignee: Stephan

Difficulty: Normal

Category:

Backwards Compatible: True

Fix Submitted:

Affected versions: 7.3.0

Due in version: 7.3.0

Sprint:

Story Points:

Cycle: