U4-8528 - Publish Umbraco.Tests on nuget

Created by Lars-Erik Aabech 27 May 2016, 08:07:07 Updated by Jeavon Leopold 17 Mar 2017, 21:11:34

Would be really nice to be able to have a nuget package of the tests accompanying the current version of Umbraco. With a dependency on the right NUnit version too, creating a test project for Umbraco sites is just a matter of install-package Umbraco.Tests. :)


Warren Buckley 27 May 2016, 08:17:04

@lars-erik I presumed you meant this for Umbraco Core as opposed to Forms? I Have moved this to Core for you.

Lars-Erik Aabech 27 May 2016, 08:20:33

Thanks. :)

Jason Prothero 12 Jan 2017, 20:54:55

Yes, this would be nice, even its its only on MyGet or a TeamCity feed or something. It would make it much less scary to include this dll as its not "one more edge-case" to remember when upgrading.

Jason Prothero 16 Feb 2017, 16:12:26

@Shandem @sebastiaan Thoughts on this? Alternatively, would it be OK if the community put the relevant bits on Myget in an "experimental" state?

Thanks, Jason

Sebastiaan Janssen 16 Feb 2017, 16:17:29

Just don't call it Umbraco.Test and don't use the Umbraco logo. Could name it something like Our.Community.Umbraco.Tests or something.

Lars-Erik Aabech 16 Feb 2017, 16:20:16

And regarding whether HQ will do this? It'll just be until V8 has taken hold anyway...

Lee Kelleher 16 Feb 2017, 16:21:44

@Prothero re: MyGet... just in note, there's a Community MyGet feed here: https://www.myget.org/gallery/umbraco-packages - open to all. I'm not worried if you use it or not, just letting you know :-)

Sebastiaan Janssen 16 Feb 2017, 16:22:47

It has very very low priority for us. Sorry. :)

Lars-Erik Aabech 16 Feb 2017, 16:25:29

If I PR the nuget build step to the core so it's just lying there ready for a nuget push? :D

Sebastiaan Janssen 16 Feb 2017, 16:32:41

If only releasing software was that simple @lars-erik - as soon as we release something like this we're also immediately responsible for maintaining it and dealing with bug reports, testing that it all still works for each release, etc. :-)

Anyway, I would happily accept that PR but I am curious how you want to deal with breaking changes? We WILL break a lot of things in this library all the time.

Lars-Erik Aabech 16 Feb 2017, 16:45:09

Breaking would be great. If Umbraco.Tests changes, then something should probably break in tests using it too. It would give us a heads up that something might have to be fixed in our own code as well. It's exactly what I'd expect. You could even upgrade Umbraco only for the test project and if that still passes 100%, you'll happily upgrade your site.

Lars-Erik Aabech 16 Feb 2017, 16:52:43

And besides, it's already released. It's available on github in unbuilt state. And it's under the MIT license, so you've got no responsibilities what-so-ever for it's use in my software.

Sebastiaan Janssen 16 Feb 2017, 17:01:22

Okay, this is true, we should make new Umbraco releases like that too: sorry, we accept no responsibility :P

/joke mode off now

Jason Prothero 16 Feb 2017, 17:02:42

I hadn't seen that @leekelleher, great idea!

So if we name the Nuget package Our.Umbraco.Community.Tests and then just have the Umbraco.Tests.dll and a few supporting test configs, etc. Will that work? We could have the version be perpetually alpha. Like release-7.5.9-alpha or something. That may set expectations at the correct level.

That's all ok?

I'm happy to set this up, but I'm going to wait just a bit to see if Lars somehow convinces the powers that be to setup a Nuget feed for it under the Umbraco umbrella.

Lars-Erik Aabech 16 Feb 2017, 17:07:55

I assume @Sebastiaan has a point in that not everyone views publicly available open source as I do, and that HQ will get stupid support requests / demands if they release it. But the second coolest thing then would be if we could have a hook on github and automate builds and publishes of Umbraco.tests on https://www.myget.org/gallery/umbraco-packages. I don't mind the structure, as long as I don't have to build it manually all the time, and it has the same version number as the corresponding release. Maybe even someone at HQ could do the publish as long as it isn't HQ as such. ><

Jason Prothero 16 Feb 2017, 17:12:08

@lars-erik I like that idea even better.

Lars-Erik Aabech 16 Feb 2017, 17:14:10

Help us, @crumpled_jeavon. You're our only hope!

// @Prothero

Lars-Erik Aabech 16 Feb 2017, 17:14:46

(If only someone knowledgeable in the ways of build automation would like this too ;) )

Paul Gower 16 Feb 2017, 17:22:22

@lars-erik I'd like to help in some way with this. I'm actually speaking at uWestFest on Unit Testing in Umbraco and was going to get the audience to vote on this issue because it would be a great benefit if we had an easy way to get the Umbraco core unit tests/mocks.

@sebastiaan I understand the concern about support but I agree with @lars-erik on letting the community help support it and I would not expect there to not be breaking changes (e.g. the core team should not worry about breaking changes when they need to change/fix something in the unit test project)

Christian Thillemann 17 Feb 2017, 12:37:52

I creted a powershell script that automatically create a NuGet package using appveyor It checks for a new version once a day

https://www.nuget.org/packages/kedde.umbraco.testdlls/ https://ci.appveyor.com/project/kedde/umbracobuild https://github.com/kedde/UmbracoBuild

Jason Prothero 17 Feb 2017, 16:03:52


Jason Prothero 17 Feb 2017, 18:00:26


Any chance you could work with @leekelleher to get this nuget into the Community feeds in MyGet mentioned above? (https://www.myget.org/gallery/umbraco-packages)

That would allow it to live on past your involvement and could allow others to help.


Christian Thillemann 17 Feb 2017, 18:48:06

@Prothero sure. @leekelleher which steps do you think we need to take? Is there any documentation I can read? A public repository? A build server?

Lars-Erik Aabech 18 Feb 2017, 10:26:59

@Kedde #h5yr You just need to be able to do nuget push. It should be possible to do with AppVeyor? Full command would be something like "nuget push Umbraco.Tests..nupkg https://www.myget.org/nuget " Dunno if there's any credentials or key required for the MyGet feed.

Lee Kelleher 18 Feb 2017, 14:05:08

The "Umbraco Community Packages" MyGet Feed is set-up as a community feed, so anyone can push a package up to it. The details are on the gallery page: https://www.myget.org/gallery/umbraco-packages - click on the "Connect to feed" button for the details.

If you could prefix the package name with " Our.Umbraco." that'd be ace... but it's not a requirement, just keeps things tidy.

@Kedde I wasn't aware that you could schedule AppVeyor builds, (I've since googled and found about their [crontab support|https://www.appveyor.com/docs/build-configuration/#scheduled-builds])... nice, thanks for the prompt!

Christian Thillemann 20 Feb 2017, 17:38:43

I will name the package Our.Umbraco.Community.Tests or Our.Umbraco.Community.Test(singular) what would you suggest?

@lars-erik thanks, pretty much the same as regular nuget package.

@leekelleher is there a "Umbraco Community Packages"-Build-server? in case that any settings need to be changed and I'm not around?

Lee Kelleher 20 Feb 2017, 18:45:56

@Kedde Nope, all the developers of the packages have their own CI/build set-up (usually an AppVeyor associated with their own code repo).

Just to say, it's not a requirement to use that community feed... I was just trying to be helpful by suggesting it. :-)

Christian Thillemann 20 Feb 2017, 21:35:12

Made some changes to the source repository and the build server and it now builds.

nuget and myget https://www.myget.org/feed/umbraco-packages/package/nuget/Our.Umbraco.Community.Tests https://www.nuget.org/packages/Our.Umbraco.Community.Tests/

source https://github.com/kedde/Our.Umbraco.Community.Tests

build server: https://ci.appveyor.com/project/kedde/umbracobuild

Lars-Erik Aabech 26 Feb 2017, 15:52:46

I just used the nuget.org one in my first project. Effin' brilliant! #h5yr @Kedde! Looks like it even published a new version. :) I guess we can close this one.

Jeavon Leopold 17 Mar 2017, 21:11:34

@lars-erik sorry I totally missed your cry for help on this one! Great job @Kedde I will be removing all of our Umbraco.Tests.dlls and replacing them with this!

Priority: Normal

Type: Feature (request)

State: Closed


Difficulty: Very Easy


Backwards Compatible: True

Fix Submitted:

Affected versions:

Due in version:


Story Points: