U4-2203 - MVC as default rendering on shared host results in encoding error

Created by Thomas Höhler 08 May 2013, 09:18:36 Updated by Shannon Deminick 26 Jun 2017, 06:40:03

Hi all

I am doing a site for a non profit project which will be hosted at 1&1 on a shared host with medium trust.

If I am using WebForms as rendering engine all is fine. If I am using Mvc as rendering engine I get encoding errors. Analyzed with Fiddler I can reproduce it to the usage of the "accept-encode" header. If I use "gzip, deflate" I get a compression error which means that the magic number is incorrect. The same project on my virtual server with full trust gives no error. If I put the the the web.config I get no erros on my virtual server. On the shared host I cannot set this element due to the config inheritance restrictions.

So it occures only with MVC rendering on the shared host.

Attached is a pdf with screenshots from fiddler and FF (text is in german). The problem can be reproduced on http://coelle.org (atm)

Cheer, Thomas

1 Attachments

Download 2013-05-07 - Fehlerbeschreibung 1&1.pdf

Comments

Thomas Höhler 08 May 2013, 09:25:19

btw: The hoster says that they are not using dynamic compression

Can anyone reproduce it on another shared hoster?


Thomas Höhler 11 May 2013, 20:07:56

Seems that the hoster has a problem right now, we will wait until they come back with a solution.


Thomas Höhler 05 Jun 2013, 10:05:13

Sorry for not coming back with actual infos:

It seems that this is an issue related to the hoster as they are gzipping the responses where the errors occure. This seems not to be an umbraco problem or medium trust related.


Sebastiaan Janssen 05 Jun 2013, 10:59:38

Thanks for getting back to this. So this is not something we can fix? Can this issue be closed then?


Thomas Höhler 05 Jun 2013, 11:34:52

It's not definitely solved by the hoster, but I am 95% sure that this is a problem the hoster has.

I made tests on my server with medium trust with and without compression, all workes fine.

The weird thing is: this problem occures only with mvc rendering of the content. The backend is fine, also when you use web forms rendering. But also using a standalone mvc app is working fine. That's the fact where I can't say that this issue is definitely solved

I would hold this issue back till the hoster comes back with more informations


Sebastiaan Janssen 05 Jun 2013, 11:50:29

Aha, I forgot about the attachment you added! Yes, it would definitely be good to know what they say. They'll probably blame Umbraco and that could be fair. The MVC implementation is still pretty new and not tested on that many hosters yet. Will keep it open and hope for more detailed info. :)


Thomas Höhler 05 Jun 2013, 12:03:22

yep, I will come back if I have more infos


Dean Lynn 22 Jul 2013, 12:33:17

I have also encountered this problem. I have raised the issue with 1&1 and will feedback any information they provide.


Dean Lynn 22 Jul 2013, 17:03:48

I have had a response from 1&1:

"We do not support gzip as of the moment under the shared hosting account however, you can use zlib compression for your site."

Their response is interesting, is it possible that Umbraco's core CMS code or one of the prepackaged components is attempting to compress some content using gzip which is failing due to the server's lack of support?

I'm not sure if such a situation would result in a corrupt response, I would be surprised if the process made it that far before crashing if the compression was not supported on the server.


Thomas Höhler 22 Jul 2013, 19:12:07

I got the same response in germany, but THEY ARE COMPRESSING.

Test 1 I have a virtual server where I made tests together with the 1&1 hosting via fiddler: On my Server: Using a clean Umbraco instance with MVC rendering:

  1. request with "accept-encoding: " => result ok, NO COMPRESSION
  2. request with "accept-encoding: gzip, deflate" => result ok, NO COMPRESSION

Test 2 Now I turned on dynamic compression on my servers IIS: On my Server: Using a clean Umbraco instance with MVC rendering:

  1. request with "accept-encoding: " => result ok, NO COMPRESSION
  2. request with "accept-encoding: gzip, deflate" => result ok, WITH GZIP COMPRESSION

Test 3 Now I added a new subdomain to the 1&1 hosting (http://test.coelle.org) with a clean mvc 4 webapp:

  1. request with "accept-encoding: " => result ok, NO COMPRESSION
  2. request with "accept-encoding: gzip, deflate" => result ok, WITH GZIP COMPRESSION

Test 4 Now I added another new subdomain to the 1&1 hosting (http://test2.coelle.org) with a clean Umbraco instance using mvc rendering:

  1. request with "accept-encoding: " => result ok, NO COMPRESSION
  2. request with "accept-encoding: gzip, deflate" => result WITH GZIP COMPRESSION but gives ERROR (encoding error: Magic Number of gzip missing)

Test 5 Now I added another new subdomain to the 1&1 hosting (http://test3.coelle.org) with a clean Umbraco instance using WebForms (Masterpages) rendering:

  1. request with "accept-encoding: " => result ok, NO COMPRESSION
  2. request with "accept-encoding: gzip, deflate" => result ok, WITH GZIP COMPRESSION

So: 1&1 is doing compression but why the hell is it only occuring on mvc rendering via umbraco?


Dean Lynn 22 Jul 2013, 20:11:43

I have found the cause of the problem. I commented out all references to ClientDependency in the Web.config and now my site runs as expected. It seems as though that module is attempting to compress content but failing. This forum post helped locate the issue: http://our.umbraco.org/forum/ourumb-dev-forum/bugs/13525-Client-Dependency-Module-playing-havoc-with-gzip-content-compression


Thomas Höhler 22 Jul 2013, 20:27:02

I can confirm that if I delete all CD ref in web.config the site runs as expected.

Thus it's weired as I've deactivated CD via Debug Mode and config. And why is this only for mvc rendering?

I think we can close this issue but should open another one with the detailed informations for CD

Good job Dean


Sebastiaan Janssen 23 Jul 2013, 14:31:45

Ah, well that explains why I couldn't find references to gzip in the core, if it's in the ClientDependency core. Good fine! will assign to Shannon so he can have a look at making gzip optional.


Thomas Höhler 23 Jul 2013, 23:11:42

@Shannon: If you need the 1&1 environment to reproduce the behaviour just ping me


Thomas Höhler 28 Jul 2013, 11:10:36

One problem left: if you kill all references to CD in the web.config the umbraco backend breaks.

So just downloaded the CD code, changed in ClientDependency.Core.Mvc.MvcFilter the function IsCompressibleContentType to return always false and readded the references to CD in the web.config. Now the backend and the frontend are rendering correct.


Shannon Deminick 29 Jul 2013, 00:58:31

@Thomas, if you change that in the code of CDF that will completely disable CDF for use in MVC.

The problem is not that CDF is trying to compress the output of the page, it is that the content is already compressed when CDF is trying to read the response output stream (I'm assuming). By default, CDF will attempt to compress anything that is matched in the mimeTypeCompression section in the config. By default there is nothing listed there so therefore it will not try to compress anything.

The reason this is probably failing in MVC is because in MVC there is not 'post processing' so we need to parse the tags of the response to get the CDF script/link tags re-rendered properly. In webforms, we have the PreRender post processing methods but this doesn't exist in MVC. The alternate approach is if you are using CDF on your front-end, then you'd have to use the Cdf view engines to do this parsing: https://github.com/Shandem/ClientDependency/wiki/Mvc otherwise it is up to the MvcFilter to do the parsing.

So, somehow I need to figure out where this error is being thrown or how to replicate this locally... Do you have a full stack trace by any chance or does it even give you one with regards to this error?

In the meantime, if you just remove the CDF module from the web.config the back office should continue to work properly, what will no longer work is: CDF with MVC (if you are not using the Cdf view engines), CDF's mime type compression, CDF's rogue file replacement.

I'll see what I can come up with today though.


Shannon Deminick 29 Jul 2013, 02:30:13

Hey guys,

Ok I think I've figured out what the problem is and I can replicate it (sort of)... What is happening is that we apply a response filter on the HttpResponse object. But somewhere along the lines this hosting provider is probably also applying an http response filter (most likely to compress the output). They are probably doing this via an HttpModule that they have registered in their machine.config.

There's a strange quirk when applying compression using a response filter in that the compression filter must be assigned before any other filters are assigned that might change the response output. If a compression filter is applied after any filter that modifies the response, then we will get encoding errors.

So I can only assume what is happening here is that the MVC filter has modified the output (even it if not by string, but even a change by one byte will cause this error) and then the hosting provider's filter is be attached after which then causes this issue.

There's a couple of ways to potentially fix this:

  • Find out from the hosting provider if it is in fact a module of some sort that is doing the compression then see if you can modify you web.config to ensure that their module executes before the CDF module. This might not be possible though due to medium trust or whatever other limitations they've imposed.
  • Remove the CDF module from config, but if you want CDF+MVC support then you'll have to follow this: https://github.com/Shandem/ClientDependency/wiki/Mvc and it will only work for the razor view engine.


Shannon Deminick 29 Jul 2013, 02:44:09

You could also do this on app startup:

ClientDependencySettings.Instance.ConfigSection.Filters.Remove("MvcFilter");

However, that is more or less the same as just removing the cdf module from config.


Thomas Höhler 05 Aug 2013, 15:30:18

I got an email from 1&1 on thursday where they apologize for the late responses as they have a lot of work with the exchange setups. But they asked now if we want to enable or disable the compression. I answered that we want to disable the compression and they did in 10 minutes. They disbaled the compression on domain level.

After that I took the original CD and Umbraco is working as expected now. But we have another Domain which is pointing to this Umbraco instance and this is still getting the Encoding Error.

Will ask them where they are doing the compression.


Thomas Höhler 12 Aug 2013, 09:26:53

Just got the information that 1&1 is doing the compression on the IIS Level. So if someone has thís problem just ask 1&1 to deactivate the compression for this url.


Matthias Bier 30 Oct 2014, 11:16:21

our hosting provider uses a module called helicon ape which seems to cause the same problem. I worked with them and they now disabled it completely. But the problems remains.

Seems as if the only solution would be to completely uninstall the module ([http://issues.umbraco.org/issue/U4-4509#comment=67-13538]) - not sure if there are going too...

Is there nothng one can do from the umbraco side? I already tried to disable CDF but this broke the whole site...


Shannon Deminick 30 Oct 2014, 11:40:21

when you say 'disable CDF' what do you mean? Did you read the above comments?

http://issues.umbraco.org/issue/U4-2203#comment=67-8715 http://issues.umbraco.org/issue/U4-2203#comment=67-8716

You also didn't post the actual error you are getting...


Matthias Bier 30 Oct 2014, 12:42:24

I tried disabling CDF with this codesnippet in the App_Code folder: using ClientDependency.Core; using ClientDependency.Core.Config; using Umbraco.Core; using umbraco.BusinessLogic; using umbraco.cms.businesslogic; using umbraco.cms.businesslogic.web;

namespace Umbraco.Extensions.EventHandlers { public class RegisterEvents : ApplicationEventHandler { protected override void ApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext) { ClientDependencySettings.Instance.ConfigSection.Filters.Remove("MvcFilter"); }
} }

What happens is that all styles and design in the backend are gone away and the page looks very white. Perhaps I missed something here.

The error on the console is the same as before (helicon ape installed but deactivated): "The user object is invalid, the remainingAuthSeconds is required."

With helicon ape activated, I get the encoding error...


Shannon Deminick 26 Jun 2017, 06:40:03

Closing issue due to inactivity - see blog post for details https://umbraco.com/blog/issue-tracker-cleanup/


Priority: Normal

Type: Bug

State: Closed

Assignee: Shannon Deminick

Difficulty: Normal

Category:

Backwards Compatible: True

Fix Submitted:

Affected versions:

Due in version:

Sprint:

Story Points:

Cycle: