U4-11520 - Automate Umbraco installation

Created by Sébastien Sougnez 16 Jul 2018, 12:17:01 Updated by Sébastien Sougnez 09 Aug 2018, 17:56:23

Tags: Up For Grabs

It's currently possible to automate a lot of steps of a new Umbraco installation, however, it's not possible to automate the part where the administrator credentials and the database information are asked. In a world where everything is automated (via Puppet and other stuff like that), I think it would be very useful to have some kind of way to automate that part as well.

I checked a bit and I noticed that there is this "InstallService" where a ticket needs to be asked, then used to process the install. Maybe it would be cool to create some kind of route where it is possible to send default configuration such as:

{ username: "xx", password: "yy", dbType: 0, starterKit: "0000...000" }

That would trigger a simple installation.

What do you guys think?

Comments

Sébastien Sougnez 16 Jul 2018, 20:13:46

I've been working a bit on this and I made this commit: https://github.com/ssougnez/Umbraco-CMS/commit/568cb5d1315cb778a67b2e9a69fc4d0cd6bd3ab5

I didn't create a PR because I don't know if you guys are interested in this. Basically, what I did in this commit:

  • I created a new model to receive fresh install data.
  • I created a new Web API route (FreshInstall) that triggers the installation based on the input data.

The main idea is to do at the server side what the AngularJs installer service does client side. I call the "GetStatus" method to get a ticket for the installation, then I call "PostPerformInstall" to execute each step of the installation process. Note that I'm taking advantage of the overlapping recycling (https://msdn.microsoft.com/en-us/library/ms525803(v=vs.90).aspx#Anchor_1) to ensure that the worker process remains alive even when an application pool recycle occurs. The worker process dies once the installation is over.

Moreover, I'm calling "PostPerformInstall" via "127.0.0.1" to ensure that the installation continues on the same server than the one it started on. I also set the "Host" header to ensure that the HTTP call can reaches Umbraco.

I tested by calling this route via PostMan with the following data:

{ user: { name: "S├ębastien Sougnez", email: "s.sougnez@areaprog.com", password: "Umbraco2018", subscribeToNewsLetter: false }, database: { dbType: 0 }, configureMachineKey: true, starterKit: "00000000-0000-0000-0000-000000000000" }

And it works pretty well. I didn't tested it with SQL Server or MySql because I wanted to have your opinion first.


Sébastien Sougnez 17 Jul 2018, 12:13:42

Hi, it's me again ^^

I continued my work on that one and I think I have something pretty usable. I tested the commit described in the previous comment to automate the installation process on a SQL Server. I triggered the installation with the following payload:

{ user: { name: "S├ębastien Sougnez", email: "s.sougnez@areaprog.com", password: "Umbraco2018", subscribeToNewsLetter: false }, database: { dbType: 1, server: ".\\SQLExpress", databaseName: "UmbracoRocks", integratedAuth: true }, configureMachineKey: true, starterKit: "00000000-0000-0000-0000-000000000000" }

And everything worked as expected.

Moreover, I also added an "Upgrade" route that can be used to automate Umbraco upgrade. This route must be called with a payload such as:

{ username: "s.sougnez@areaprog.com", password: "Umbraco2018" }

The HttpInstallAuthorize attribute does not cause any issue as the upgrade must be called when the web.config version differs from the assembly version, so it can be called without being logged. However, some steps in the upgrade process requires authentication, which is why credential needs to be passed. These are used by the AutomaticInstallClientHandler to login in the back office before running the upgrade.

The commit related to that change is here: https://github.com/ssougnez/Umbraco-CMS/commit/1d15e4119c43714600ff3ae8c3258662f16cb1ee

I won't be working further on this until someone reviews it. I think this should be examined closely because automation is really something to take care of and this could be a valid solution.

Cheers


Sébastien Sougnez 19 Jul 2018, 07:42:37

As no one seemed to argue with this, I raised this PR: https://github.com/umbraco/Umbraco-CMS/pull/2789

Hope I'll get a feedback :-D


Ronald Barendse 19 Jul 2018, 09:43:10

@ssougnez Automated upgrades should indeed be built-in! Having a website show the Umbraco login page after deploying an update is not a great user/visitor experience...

I do understand that upgrades can go wrong and should be monitored. The upgrade process must also run only once (e.g. not concurrently on every request until it's completed): maybe check if this is already handled correctly?

One thing to consider: a better approach could be to add a marker file, just like [Umbraco Deploy|https://our.umbraco.com/documentation/Umbraco-Cloud/Set-Up/Power-Tools/generating-uda-files/#generate-uda-files-manually] does with the data/deploy and data/deploy-export files. Just adding a marker file like data/upgrade to the deployment would then automatically upgrade Umbraco on the first request (and change the file name to upgrade-progress, upgrade-complete or upgrade-failed?).

Regarding the initial install: have you used https://github.com/aaronpowell/Chauffeur?


Sébastien Sougnez 19 Jul 2018, 10:01:08

@rgmbarendse Concerning the concurrent request, what do you mean exactly?

If you meant that it must be impossible to trigger two upgrades one after the other, it's already ensured as at that moment, the application is configured and the "HttpInstallerOnly" attribute would block the call.

If you meant "calling the Install/Upgrade route twice at the same time", it would be indeed interesting to put some guard for this.

About the marker files, I don't really know them. However, I have the impression that the problem will still be the same. Imagine you do that, you launch the back office, then the upgrade starts. It could be that the web app pool gets recycled by a step and in this case, I'm not sure that the overlapping recycling will apply (to confirm). Moreover, the installation process is currently made in a way that it cannot run in a one shot, it needs some recycling. So doing this would induce a entire rework of the install process?

About Chauffeur, I'm usually reluctant on using third party tool. You always face the possibility that the tool is not upgraded at the same time as the main product, then you get tied to it, and as installing/upgrading "Umbraco" seems to be a core functionality, it's weird to me to delegate it to another tool (but I might be wrong).

Concretely, what does it mean? Do you reject my PR ou do you ask me to improve it with some safe guards ? :-)


Claus Jensen 20 Jul 2018, 07:45:39

@ssougnez As mentioned on the PR we will need to discuss this change with the team - we will get back to you when we have an answer - Thanks!


Sébastien Sougnez 09 Aug 2018, 17:56:23

I see that this PR is up for grabs ? I'm not sure what that means but if it means that it could be taken by someone, maybe you could give us the summary of what has been decided? What's the chosen implementation for this?


Priority: Normal

Type: Feature (request)

State: Open

Assignee:

Difficulty: Normal

Category: Installation

Backwards Compatible: True

Fix Submitted:

Affected versions: 7.12.0

Due in version:

Sprint:

Story Points:

Cycle: