U4-9792 - Update Umbraco to be FIPS compliant

Created by Shannon Deminick 20 Apr 2017, 00:13:15 Updated by Jason Prothero 25 Jul 2017, 17:41:38

Tags: Community Contrib Gold partner

Microsoft no longer recommends FIPS compliance

https://blogs.technet.microsoft.com/secguide/2014/04/07/why-were-not-recommending-fips-mode-anymore/

Implementation

This would require removing all usages of ToMd5 or any other custom Md5 operations in c# and changing it to use the ToSHA1 instead.

Alternatively - and probably better to avoid any potentially breaking changes, we change all ToMd5 operations to be ToSafeHash which will check if Fips compliancy is enabled in web.config and if so will delegate to ToMd5 or ToSHA1 accordingly.

Further to this, it would require that all other Umbraco products become updated:

  • Courier
  • Forms
  • Deploy

Caveats

However, doing this will not leave the whole CMS as FIPS compliant. Any libraries that Umbraco utilizes are not guaranteed to be FIPS compliant and this would require some testing. In many cases if a library is not FIPS compliant, Umbraco will not be able to become FIPS compliant unless those libraries are able to be updated and integrated which is out of the control of Umbraco HQ. And of course if any Umbraco 3rd party packages are not FIPS compliant there is nothing we can do about that.

Some things would also be disabled by enabling FIPS compliance that requires Md5 hashing such as using a Gravatar avatar in the back office since their APIs require an Md5 hash which cannot be created with FIPS enabled.

Enabling/Disabling FIPS compliancy can be toggled in web.config https://blogs.msdn.microsoft.com/shawnfa/2008/03/14/disabling-the-fips-algorithm-check/

Comments

Jason Prothero 20 Apr 2017, 16:42:30

FYI, the web.config method only works on .NET 3.5 or earlier. I just tested it with 7.5.13 on .NET 4.5 and it doesn't do anything (errors still persist).


Jason Prothero 20 Apr 2017, 17:27:37

I see 8 references to ToMD5() in the Umbraco CMS repo.

https://github.com/umbraco/Umbraco-CMS/search?utf8=%E2%9C%93&q=ToMd5&type=

15 total references to 'Md5': https://github.com/umbraco/Umbraco-CMS/search?utf8=%E2%9C%93&q=Md5&type=

Key areas:

  • Gravatar (would need to be disabled. A few locations Md5 hashing the email for Gravatar)
  • TinyMce3 (Could we replace with TinyMce 4 or something? Or remove the compression for FIPS)
  • RazorMacroEngine (legacy, correct?)
  • UmbracoExamine.LocalStorage
  • src/umbraco.cms/businesslogic/macro/Macro.cs (obsolete)
  • src/Umbraco.Web/umbraco.presentation/umbraco/developer/Packages/SubmitPackage.aspx.cs (still used?)
  • src/Umbraco.Core/StringExtensions.cs (This may be an issue depending on how widespread its used - perhaps allow configuration of default hashing mechanism? Used in UmbracoHelper)
  • src/Umbraco.Core/StringExtensions.cs (obsolete)
  • src/Umbraco.Web/umbraco.presentation/umbraco/developer/Packages/installer.aspx.cs (obsolete)
  • src/Umbraco.Core/Security/BackOfficeUserStore.cs

It appears that the impact isn't too wide-spread in the Core and may not cause too much backwards compatibility issues. It would be great if the Hashing method was a configuration. ProWorks would be happy to work with the Umbraco Core team to help make this happen. While we would like Forms and Courier to be compliant, its likely that we can remove that need from this issue. Simply getting the Core compliant would be good enough.

-Jason


Shannon Deminick 20 Apr 2017, 23:28:37

Hi, replies inline:

FYI, the web.config method only works on .NET 3.5 or earlier. I just tested it with 7.5.13 on .NET 4.5 and it doesn't do anything (errors still persist).

What do you mean "errors still persist" ... there shouldn't be any errors if the <enforceFIPSPolicy enabled="false"/> or if that section doesn't exist. Of course if this is enabled, you'll get errors because the code is not FIPS compliant

It would be great if the Hashing method was a configuration

Yes, this is what I mentioned in the main notes, if you have the enforceFIPSPolicy enabled="false" then we can adjust the hashing based on using ToSafeHash as a replacement to ToMd5

But yes, adjusting the core won't be hugely difficult, but as i said, you'll need to update all dependencies, I definitely know of several that are not compliant.


Jason Prothero 20 Apr 2017, 23:39:09

That web.config value isn't valid in .NET 4.5. It was only for 3.5 because it was impossible to support FIPS on 2.0 or 3.5. I did try that setting and I'm getting the same security exceptions as without that web.config setting in it.

I'll post the initial error I get from the Umbraco logs next message.

Which dependencies aren't compliant?

Thanks for looking into this!

-Jason


Jason Prothero 20 Apr 2017, 23:41:23

I setup Umbraco 7.3.13 (the latest release) on a test VM.

It was running fine on the server before attempting to setup FIPS mode.

When I turned on FIPS using the Registry Key method identified here: https://www.howtogeek.com/245859/why-you-shouldnt-enable-fips-compliant-encryption-on-windows/ it no longer rendered the front-end of the site.

This error log below implies the MD5 encryption used in some Lucene plugin is the cause. This is the type of error we have seen in the past when FIPS is enabled and MD5 encryption is detected.

Full Startup Logs after server restart and website load:

2017-04-20 09:27:21,123 [P5076/D2/T1] INFO Umbraco.Core.CoreBootManager - Umbraco 7.5.13 application starting on FIPS 2017-04-20 09:27:21,213 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Determining hash of code files on disk 2017-04-20 09:27:21,235 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Hash determined (took 21ms) 2017-04-20 09:27:21,362 [P5076/D2/T1] INFO Umbraco.Core.MainDom - Acquiring MainDom... 2017-04-20 09:27:21,383 [P5076/D2/T1] INFO Umbraco.Core.MainDom - Acquired MainDom. 2017-04-20 09:27:21,391 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of umbraco.interfaces.IApplicationStartupHandler 2017-04-20 09:27:21,529 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of umbraco.interfaces.IApplicationStartupHandler, found 0 (took 139ms) 2017-04-20 09:27:21,686 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Core.PropertyEditors.IPropertyEditorValueConverter 2017-04-20 09:27:21,687 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Core.PropertyEditors.IPropertyEditorValueConverter, found 0 (took 1ms) 2017-04-20 09:27:21,687 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Core.PropertyEditors.IPropertyValueConverter 2017-04-20 09:27:21,688 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Core.PropertyEditors.IPropertyValueConverter, found 0 (took 1ms) 2017-04-20 09:27:21,754 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Web.Mvc.SurfaceController 2017-04-20 09:27:21,755 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Web.Mvc.SurfaceController, found 0 (took 0ms) 2017-04-20 09:27:21,755 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Web.WebApi.UmbracoApiController 2017-04-20 09:27:21,787 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Web.WebApi.UmbracoApiController, found 0 (took 31ms) 2017-04-20 09:27:22,280 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Core.Media.IThumbnailProvider 2017-04-20 09:27:22,281 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Core.Media.IThumbnailProvider, found 0 (took 0ms) 2017-04-20 09:27:22,281 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Core.Media.IImageUrlProvider 2017-04-20 09:27:22,281 [P5076/D2/T1] INFO Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Core.Media.IImageUrlProvider, found 0 (took 0ms) 2017-04-20 09:27:25,538 [P5076/D2/T1] INFO Umbraco.Core.DatabaseContext - CanConnect = True 2017-04-20 09:27:26,706 [P5076/D2/T1] INFO Umbraco.Web.Cache.CacheRefresherEventHandler - Initializing Umbraco internal event handlers for cache refreshing 2017-04-20 09:27:26,794 [P5076/D2/T1] INFO Umbraco.Web.Search.ExamineEvents - Initializing Examine and binding to business logic events

2017-04-20 09:27:27,119 [P5076/D2/T1] ERROR Umbraco.Core.CoreBootManager - An error occurred running OnApplicationStarted for handler Umbraco.Web.Search.ExamineEvents System.Configuration.ConfigurationErrorsException: The type initializer for 'Lucene.Net.Store.FSDirectory' threw an exception. (C:\inetpub\wwwroot\UmbracoCms.7.5.13\config\ExamineSettings.config line 12) ---> System.TypeInitializationException: The type initializer for 'Lucene.Net.Store.FSDirectory' threw an exception. ---> System.SystemException: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms. at System.Security.Cryptography.MD5CryptoServiceProvider..ctor() --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Security.Cryptography.CryptoConfig.CreateFromName(String name, Object[] args) at System.Security.Cryptography.MD5.Create() at Lucene.Net.Store.FSDirectory..cctor() ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms. at System.Security.Cryptography.MD5CryptoServiceProvider..ctor() --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Security.Cryptography.CryptoConfig.CreateFromName(String name, Object[] args) at System.Security.Cryptography.MD5.Create() at Lucene.Net.Store.FSDirectory..cctor() --- End of inner exception stack trace --- at Lucene.Net.Store.FSDirectory..cctor() --- End of inner exception stack trace --- at Lucene.Net.Store.FSDirectory..ctor(DirectoryInfo path, LockFactory lockFactory) at Examine.LuceneEngine.DirectoryTracker.<>c__DisplayClass5_0.b__0(String s) at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at Examine.LuceneEngine.DirectoryTracker.GetDirectory(DirectoryInfo dir, Boolean throwIfEmpty) at Examine.LuceneEngine.Providers.LuceneIndexer.InitializeDirectory() at Examine.LuceneEngine.Providers.LuceneIndexer.Initialize(String name, NameValueCollection config) at UmbracoExamine.BaseUmbracoIndexer.Initialize(String name, NameValueCollection config) at UmbracoExamine.UmbracoContentIndexer.Initialize(String name, NameValueCollection config) at System.Web.Configuration.ProvidersHelper.InstantiateProvider(ProviderSettings providerSettings, Type providerType) --- End of inner exception stack trace --- at System.Web.Configuration.ProvidersHelper.InstantiateProvider(ProviderSettings providerSettings, Type providerType) at System.Web.Configuration.ProvidersHelper.InstantiateProviders(ProviderSettingsCollection configProviders, ProviderCollection providers, Type providerType) at Examine.ExamineManager.EnsureProviders() at Umbraco.Web.Search.ExamineEvents.ApplicationStarted(UmbracoApplicationBase httpApplication, ApplicationContext applicationContext) at Umbraco.Core.ApplicationEventHandler.OnApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext) at Umbraco.Core.CoreBootManager.b__a(IApplicationEventHandler x)

2017-04-20 09:27:27,218 [P5076/D2/T1] ERROR Umbraco.Core.UmbracoApplicationBase - An unhandled exception occurred System.Configuration.ConfigurationErrorsException: The type initializer for 'Lucene.Net.Store.FSDirectory' threw an exception. (C:\inetpub\wwwroot\UmbracoCms.7.5.13\config\ExamineSettings.config line 12) ---> System.TypeInitializationException: The type initializer for 'Lucene.Net.Store.FSDirectory' threw an exception. ---> System.SystemException: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms. at System.Security.Cryptography.MD5CryptoServiceProvider..ctor() --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Security.Cryptography.CryptoConfig.CreateFromName(String name, Object[] args) at System.Security.Cryptography.MD5.Create() at Lucene.Net.Store.FSDirectory..cctor() ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms. at System.Security.Cryptography.MD5CryptoServiceProvider..ctor() --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Security.Cryptography.CryptoConfig.CreateFromName(String name, Object[] args) at System.Security.Cryptography.MD5.Create() at Lucene.Net.Store.FSDirectory..cctor() --- End of inner exception stack trace --- at Lucene.Net.Store.FSDirectory..cctor() --- End of inner exception stack trace --- at Lucene.Net.Store.FSDirectory..ctor(DirectoryInfo path, LockFactory lockFactory) at Examine.LuceneEngine.DirectoryTracker.<>c__DisplayClass5_0.b__0(String s) at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at Examine.LuceneEngine.DirectoryTracker.GetDirectory(DirectoryInfo dir, Boolean throwIfEmpty) at Examine.LuceneEngine.Providers.LuceneIndexer.InitializeDirectory() at Examine.LuceneEngine.Providers.LuceneIndexer.Initialize(String name, NameValueCollection config) at UmbracoExamine.BaseUmbracoIndexer.Initialize(String name, NameValueCollection config) at UmbracoExamine.UmbracoContentIndexer.Initialize(String name, NameValueCollection config) at System.Web.Configuration.ProvidersHelper.InstantiateProvider(ProviderSettings providerSettings, Type providerType) --- End of inner exception stack trace --- at System.Web.Configuration.ProvidersHelper.InstantiateProvider(ProviderSettings providerSettings, Type providerType) at System.Web.Configuration.ProvidersHelper.InstantiateProviders(ProviderSettingsCollection configProviders, ProviderCollection providers, Type providerType) at Examine.ExamineManager.EnsureProviders() at Umbraco.Web.Search.ExamineEvents.ApplicationStarted(UmbracoApplicationBase httpApplication, ApplicationContext applicationContext) at Umbraco.Core.ApplicationEventHandler.OnApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext) at Umbraco.Core.CoreBootManager.b__a(IApplicationEventHandler x) at Umbraco.Core.EnumerableExtensions.ForEach[TItem](IEnumerable1 items, Action1 action) at Umbraco.Core.CoreBootManager.Complete(Action1 afterComplete) at Umbraco.Web.WebBootManager.Complete(Action1 afterComplete) at Umbraco.Core.UmbracoApplicationBase.StartApplication(Object sender, EventArgs e) 2017-04-20 09:27:37,341 [P5076/D2/T8] INFO Umbraco.Core.MainDom - Stopping... 2017-04-20 09:27:37,341 [P5076/D2/T8] INFO Umbraco.Core.MainDom - Released MainDom.


Shannon Deminick 21 Apr 2017, 00:53:11

Yes this is my point exactly, you cannot enable FIPS whether it's via registry or via the web.config (if it supports that), it simply won't work because Umbraco is not FIPS compliant and neither are many of the libraries it uses.

The error you are getting above is because Examine uses Md5 algorithms. I know CDF does too but CDF does have FIPS compliancy support but that's only enabled via the config section currently, and there's probably several others. In some cases if a library uses Md5 that we rely on and cannot upgrade it, we'll be at a dead end.

Md5 is very common to use - and the point is that FIPS compliancy doesn't make things more secure, Md5 can be used as a hash that is not use for security ... which is pretty much all these things use it for.


Jason Prothero 28 Apr 2017, 21:50:23

Shannon,

I've had a chat with a colleague and apparently they solve it by using a Managed implementation for the MD5 hashing. Since the application is no longer using the Windows implementation, it doesn't know that its using MD5. Apparently, a managed implementation is actually faster: http://stackoverflow.com/questions/14850674/how-much-faster-is-the-native-implementation-of-the-native-cryptographic-hashes/14850676#14850676

Perhaps we could use this sort of strategy to get FIPS compliance and buy time to allow for some configuration of hashing providers in the future (v8?). This would have the benefit of not breaking existing installs and being completely backwards compatible.

Thoughts?

Thanks, Jason


Shannon Deminick 30 Apr 2017, 06:51:42

@Prothero This just basically goes against the whole concept of FIPS. The point of FIPS is for added security (which it fails at anyways, which is why Microsoft doesn't recommend using this setting anymore anyways ... see the first link on this issue). If you are going to go through all of the effort to bypass what FIPS has been setup for than what is the point of enabling FIPS in the first place?

In any case, creating a separate MD5 hashing algorithm isn't the problem. We can just switch to SHA1 as I've said above and we really don't have any breaking changes but It's not even the breaking changes I'm worried about. We have no control over the libraries we use which is the bigger concern. If any library that we use uses md5 for hashing of any sort you'll still get a YSOD if FIPS is enabled.

It would be possible to load all DLLs that is in the BIN of umbraco into JetBrains dotPeek and then do a search on usages of the 5d5 crypto class which should show you which libs are using it. We can tweak Examine and CDF because I own those libraries but if anything else is using it than that is out of our control.


Jason Prothero 01 May 2017, 15:56:56

@Shandem I fully understand that. I was trying to suggest something that would make it easier to move forward. Ultimately, I just need to be able to install Umbraco in a U.S. data center in two months. They won't likely be listening to any arguments about the validity of FIPS.


Shannon Deminick 02 May 2017, 00:02:57

@Prothero Ah, i wasn't aware of the underlying reason why you are so adamant about FIPS.

This is how this should work:;

  • Create a new string extension method in Umbraco Core, and use basically exactly how I have CDF doing it (this was already in place in CDF but I've just updated to use a simpler SHA1 FIPS compliant hash): https://github.com/Shazwazza/ClientDependency/blob/master/ClientDependency.Core/StringExtensions.cs#L90
  • Replace all ToMd5 usages in Umbraco Core with GenerateHash but keep in mind we'll need to treat some things like the gravatar differently because that currently requires an MD5 so we can just disable it for now (in 7.7 you can have a custom avatar)

We'll need to also search the Umbraco Core for any rogue usages of MD5CryptoServiceProvider and replace accordingly

If you are able to make a PR for that we can review it? ... but as I said the bigger problem will be dependencies.

I can update Examine since that is not FIPS compliant in the same way that CDF is done but I don't have control over other libraries.


Jason Prothero 02 May 2017, 16:05:48

Thanks! I'll get this on the next sprint (starting next week) and see how far I can get. I think if you can just help me when I have questions I should be able to take it from there. At least we will find out if there are any other external dependencies or roadblocks that will need to be overcome.


Shannon Deminick 04 May 2017, 01:38:23

yup of course, happy to assist just let me know if you have any questions


Jason Prothero 10 May 2017, 20:06:39

@Shandem I've been working through this and had a couple of questions about legacy libraries and how I should approach them.

RazorMacroEngine:

https://github.com/umbraco/Umbraco-CMS/blob/dev-v7/src/umbraco.MacroEngines/RazorCore/RazorMacroEngine.cs#L29

It is public, but I couldn't find any other references.

My plan is to rename it to GetHash(), then call GenerateHash() from there. Is feels like I shouldn't just call GenerateHash in a *Md5() method as that's a bit confusing because it could really be returning Sha1 or something else in the future.

Thoughts?

umbraco.presentation.library:

https://github.com/umbraco/Umbraco-CMS/blob/86cc3f69aacccc8e3451aa77f5183489bde16ea2/src/Umbraco.Web/umbraco.presentation/library.cs#L664

I'm guessing that changing this to SHA1 may break something. However, I didn't do a deep dive into what webservice and the class is obsolete so perhaps this doesn't matter? If this call fails, it just doesn't get a member, which is perhaps OK?

In this case since its legacy, does it make sense to rename this or just create a new hash() method? Or just swap out the ToMD5() to GenerateHash() and call it good?


Jason Prothero 10 May 2017, 20:11:08

Also, UmbracoHelper has a CreateMd5Hash() public method. I didn't see any references, but thought that would be a breaking change to remove it. Instead, I just added a new method called CreateHash() to allow people to use that if they want to be FIPS compliant.

Thoughts? Is that a good approach, or do you want me to avoid changing the API of UmbracoHelper?


Jason Prothero 10 May 2017, 22:42:54

Shannon,

Based on the Lucene.Net issue tracker, it appears that they may have addressed the FIPS issue, but that it may need a re-compile. https://issues.apache.org/jira/browse/LUCENENET-175

I pulled down the source, set the FIPSCompliant = true in the SupportClass.cs and got past the Lucene error, but now Examine is throwing:

Could not load all types from "UmbracoExamine, Version=0.7.0.24345, Culture=neutral, PublicKeyToken=null" due to LoaderExceptions, skipping: . System.IO.FileLoadException: Could not load file or assembly 'Lucene.Net, Version=2.9.4.1, Culture=neutral, PublicKeyToken=85089178b9ac3181' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

Will I need to compile a custom version of Examine to load in the DLL that I'm building?

I wish they would have just checked the CryptoConfig setting instead of making their own setting hard coded in the source...

-Jason


Shannon Deminick 11 May 2017, 04:19:03

Hi Jason,

  • The GetMd5 on razor macro engine should be marked obsolete - do not use and put an [EditorBrowsable(Never)] on it, then change all of it's usages to the new (if you've created it) StringExtensions.GenerateHash() ** If people have used this than there is nothing we can do about it, just like if packages or people's code use the ToMd5 string extensions method or have their own Md5 code usage, there is nothing we can do about that either ** You can't rename it since it's a breaking change
  • Same process goes for the library method, obsolete it, make sure nothing uses it and where things do use it change it for the new StringExtensions.GenerateHash() ** You can't rename it or remove it since it's a breaking change ** Can you point me to where this web service thing uses it?
  • For the UmbracoHelper.CreateMd5Hash - also obsolete and use [EditorBrowsable(Never)] then create a new method like you described called CreateHash which wraps the StringExtensions.GenerateHash()
  • As for Lucene ... we will not ship with a non-official Lucene package, therefore if you wish to try to use Umbraco with FIPS compliancy you will need to build this DLL yourself and replace the one that Umbraco is shipped with. We can of course add docs on our about FIPS compliancy with a link to download this patched Lucene version but shipping with a non-official Nuget dependency is not something we want to do (we've been down that road and it's not fun)
  • Examine itself will need to be updated since it uses MD5, Examine will not be required to be rebuilt against the patched version of your Lucene assembly


Jason Prothero 11 May 2017, 16:13:28

Thanks Shannon!

The webservice calls I was referencing are here:


Jason Prothero 11 May 2017, 16:29:54

Shannon,

Here's my first round of changes (on my Fork). Could you review and let me know if there's anything I need to change?

https://github.com/protherj/Umbraco-CMS/commits/U4-9792-fips-support

Thanks, Jason


Jason Prothero 11 May 2017, 18:24:58

OK, I think I finally got the correct version of Lucene.Net (tag Lucene.Net_2_9_4_RC3) and modified this line:

https://github.com/apache/lucenenet/blob/Lucene.Net_2_9_4_RC3/src/core/SupportClass.cs#L1421

to be:

static public bool FIPSCompliant = CryptoConfig.AllowOnlyFipsAlgorithms;

I also added the using to the top:

using System.Security.Crypography;

After copying that over Examine / Umbraco was happy.

Now I just need a new version of Examine for UmbracoExamine to reference and then I think I'll be past all the Lucene/Examine issues.

When do you plan on releasing an update of Examine to Nuget?

Thanks, Jason


Shannon Deminick 15 May 2017, 03:05:41

If you can create a PR for Umbraco core that would be the easiest way to follow along with what you are doing and to add review comments instead of having access to your fork. Once you create a PR any changes you push to your branch will show up in the PR.

I don't have a whole lot of time right now to do any Examine releases but feel free to send a PR for that too if you have time.


Jason Prothero 15 May 2017, 17:39:37

@Shandem

Here's the Pull Request for the Umbraco side of things:

https://github.com/umbraco/Umbraco-CMS/pull/1945

I'll look at Examine next.

-Jason


Jason Prothero 15 May 2017, 18:56:15

@Shandem

I got the Fanoe 7.6.1 site working with FIPS enabled in the Registry (with Lucene.Net recompiled & deployed, your Examine changes deployed, and my PR change deployed)!

How can I help you release version and release 0.1.82 of Examine to Nuget? A pull request of the version change? Can I actually push anything to Nuget?

Any chance my Pull Request could make it into Umbraco 7.6.2?

Thanks, Jason


Jason Prothero 15 May 2017, 20:10:01

I tested Forms and it works fine. TinyMCE works. So far so good.


Shannon Deminick 16 May 2017, 04:12:03

Nice work so far, i've pushed a new version of Examine to Nuget for you to use. I can't guarantee any timeframe for getting this PR merged in, we have internal priorities that need to be completed first but I'll see what i can do, i'm sure we can get it in for 7.6.2 or 7.6.3


Jason Prothero 16 May 2017, 19:28:06

Thanks!

I updated my Pull Request to use the 0.1.83 version of Examine (Nugetted). I also downloaded the artifacts for AppVeyeor build on my last push to the PR and setup Umbraco on my FIPS test server with that version. (https://ci.appveyor.com/project/Umbraco/umbraco-cms-hs8dx/build/7497/artifacts)

I got an error with Lucene.Net (expected), then copied the re-compiled Lucene.Net DLL with the FIPS support turned on, and everything fired up and it ran both on the front-end and back-office. That's what I was hoping for.

Let me know if there's anything else I can test to help with the release process.


One final note: I did have an issue installing Umbraco when FIPS was enabled. So I turned off FIPS, installed, then turned it back on. I don't believe this is an issue because installation will likely happen on a non-FIPS box then copied to a live server with FIPS enabled. That said, I could spend some time digging into the error that I got if it would hold up a release.

Thanks, Jason


Jason Prothero 24 May 2017, 14:52:16

@Shandem How are things going? Any thoughts or revisions you need me to make to my pull request? When do you think it might get merged in?


Jason Prothero 12 Jun 2017, 21:16:13

I've updated the Pull Request to fix ToGuid() to be RFC 4122 compliant with MD5 and SHA1 hashes.

https://github.com/umbraco/Umbraco-CMS/pull/1945

This is ready for review.

People are getting a little ansy to start testing the release. Please let me know if there is anything that remains an issue and I'll turnaround a change as fast as I can.

Thanks, Jason


Shannon Deminick 13 Jun 2017, 13:16:55

This has been pulled in now, thanks! I'm just running some last tests, etc... but all looks good


Shannon Deminick 13 Jun 2017, 13:17:45

@Prothero Next up, it would be super fantastic to update our docs to describe how to get this working ;)


Jason Prothero 13 Jun 2017, 15:48:28

Yay!

Also, happy to help with the docs! Can you point me to where the docs repo is?

Thanks, Jason


Jason Prothero 13 Jun 2017, 16:00:08

Also, when are the potential next dates it could get released?

-J


Shannon Deminick 13 Jun 2017, 16:16:10

All docs live here: https://github.com/umbraco/UmbracoDocs/ and then are published on Our

Not sure next release date, i'll keep you posted


Jason Prothero 13 Jun 2017, 16:49:19

@sebastiaan Where do you think is the appropriate place to add Documentation about FIPS?


Jason Prothero 19 Jun 2017, 22:58:04

I have a tutorial page created and a Pull Request to the UmbracoDocs repo here:

https://github.com/umbraco/UmbracoDocs/pull/487

Thanks, Jason


Jason Prothero 22 Jun 2017, 16:22:42

Any chance 7.6.4 could get released next week? I'm getting a lot of pressure from my client because they have to go through a long-ish testing process and need to start that as soon as possible.


Jason Prothero 22 Jun 2017, 16:23:12

Also, thanks for all your help and responsiveness on this so far!


Shannon Deminick 23 Jun 2017, 00:31:57

@Prothero no 7.6.4 won't be out next week. That said there is nothing stopping you from using a 7.6.4 build yourself and we would encourage you to do so instead of having to wait on any official release. You can just clone down the repo and get the latest dev-v7. Update the /build/UmbracoVersion.txt to have a pre-release name like:

# Usage: on line 2 put the release version, on line 3 put the version comment (example: beta)
7.6.0
alpha001

Then run Build.bat, you'll get a fully built 7.6.4-alpha001 pre-release made, you can even use the output via Nuget by doing:

Update-Package UmbracoCms -Pre -Source X:\PathToTheBuildFolder

Where PathToTheBuildFolder is the same folder as the UmbracoVersion.txt

Then when 7.6.4 final is out, you can just do Update-Package UmbracoCms directly from Nuget


Jason Prothero 23 Jun 2017, 15:14:19

I'll try to see if I can get the ball moving on that for them. They haven't been super open to testing pre-release versions.

Any idea when 7.6.4 may drop?

Thanks, Jason


Shannon Deminick 26 Jun 2017, 06:16:03

Yes but i assume you aren't going live on this server right now? even if you are on a "pre-release" it shouldn't make much difference. What is in the dev-v7 branch will become 7.6.4 and it will be very easy for you to upgrade to 7.6.4 final when it's out. I suspect it will be 2 weeks if all goes according to plan.


Jason Prothero 26 Jun 2017, 16:27:47

Yep, we have to test in a staging environment to get the approval to go live. Apparently, its a bit of a long process.


Jason Prothero 26 Jun 2017, 18:24:58

Should I checkout and build from dev-v7 or use the Nightlies here: http://nightly.umbraco.org/?container=umbraco-750?


Shannon Deminick 27 Jun 2017, 00:15:21

Either, it doesn't really matter but what will matter is the build number of the Nuget package. This most likely will make no difference for you but for example, last night's nightly was: UmbracoCms.Core.7.6.4-build7971.nupkg and if we were to make a beta release it would look like UmbracoCms.Core.7.6.4-beta.nupkg and as it would turn out the build version would be higher than the beta so that would be confusing. In the future we plan to have our nightly versions listed as 'alpha000xx'


Jason Prothero 27 Jun 2017, 22:42:43

I upgraded to the 7.6.4 version from yesterday and it all worked great on my FIPS VM. Looking good!


Shannon Deminick 20 Jul 2017, 13:18:40

Hey @Prothero I've just noticed that in Lucene you can specify this on startup (even in the version we are using):

SupportClass.Cryptography.FIPSCompliant = false;

Can you check if that works without having to replace the Lucene assembly?


Jason Prothero 20 Jul 2017, 15:51:01

Well, that would be much easier. I'll test this out today. I'm going to attempt to just use Umbraco startup events to set it and hope that works.

Thanks, Jason


Jason Prothero 21 Jul 2017, 22:11:40

Unfortunately, in my testing it appears that even ApplicationInitialized is too late to set this property in order for it to not throw an error when FIPS is enabled.

I'll poke around Examine and see if there is a place in there that makes sense that could work.

Are there any other places you think that might make sense to try this?


Jason Prothero 21 Jul 2017, 22:41:15

What would be the earliest place I could set this in Examine? (or Umbraco)


Shannon Deminick 24 Jul 2017, 05:18:35

You can override the WebBootManager with your own and then create your own Global.asax and override GetBootManager. Here's an example of overriding global.asax: https://our.umbraco.org/Documentation/Reference/using-ioc#example then in your custom boot manager, override Initialize, this is pretty much the first thing that runs during bootup.

Actually, an easier option is to just use a PreApplicationStartMethod attribute like we do here: https://github.com/umbraco/Umbraco-CMS/blob/dev-v7/src/Umbraco.Core/BindingRedirects.cs#L7 then this code will run before anything actually happens.


Jason Prothero 24 Jul 2017, 21:43:22

Wow. The second approach worked!

That's a pretty simple solution. I basically added a class to my project:

using System.Security.Cryptography; using System.Web; using MyProject.Events;

[assembly: PreApplicationStartMethod(typeof(PreAppStartup), "Initialize")]

namespace MyProject.Events { public sealed class PreAppStartup { public static void Initialize() } }

and it worked!

Any thoughts on this approach? Seems like a better option to me.

-Jason


Shannon Deminick 25 Jul 2017, 04:51:55

Yes that's fine, might want to name the class something more relevant. Now you'll just need to update the FIPS docs with this :)


Jason Prothero 25 Jul 2017, 17:14:30

Sounds good, I'll update the FIPS docs today and submit a PR


Jason Prothero 25 Jul 2017, 17:41:38

OK, PR submitted:

https://github.com/umbraco/UmbracoDocs/pull/507

-J


Priority: Up for grabs

Type: Feature (request)

State: Fixed

Assignee:

Difficulty: Normal

Category:

Backwards Compatible: True

Fix Submitted:

Affected versions:

Due in version: 7.6.4

Sprint:

Story Points:

Cycle: 2