U4-10794 - Logging of consent

Created by Claus Jensen 02 Jan 2018, 08:06:20 Updated by Shannon Deminick 26 Feb 2018, 13:25:54

Subtask of: U4-10796

This feature is about registering that a user has given consent of a given action. Initially, this feature should solely be a logging feature and thus “just” an API. This feature should not focus on how consent is given, but later on it could provide the logic needed for double opt-in consents (ie. check a box, get an email, click the link in the email).

For v1 it could be as simple as a table storing email or a member-id, a timestamp, a key (UDI?) and an “app”. The app might be ambiguous given the nature of UDIs, but could ease querying later on? The comment field might also be ambiguous, but could ease reporting and more complex scenarios.

For instance:

Email: nh@umbraco.com
MemberId: null
Timestamp: 2017-06-01T13:45:30
Key: umb://forms/4fed18d8c5e34d5e88cfff3a5b457bf2
App: Forms
Comment: Consent for storing name and email about the user
Status: The status of the consent item, this could be used for different types of consent (i.e. Completed by API, Completed by Double opt-in, Pending Double opt-in, etc..)  


Stephan 24 Jan 2018, 15:07:19


So it's essentially a new database table, keeping track of:

  • memberId (is that for members only? also for users?)
  • timestamp
  • memberEmail (at the time of consent) (would we need more infos?)
  • application (a string?)
  • comment (free text) (what about memberEmail?)
  • status (seems to be an enum?)

About status... seems to be three main categories: pending, granted, revoked - and then some sub-categories explaining ''how'' consent was granted or revoked?

And we want to implement some kind of API (at Service level, or a WebAPI?) to manage consents.


Shannon Deminick 24 Jan 2018, 16:29:17

See: https://docs.google.com/document/d/1897XGDGOdnC6eZSIUEUoSi5wOwwovqvHww5DzpECIjQ/edit?usp=sharing for further details

Stephan 25 Jan 2018, 10:40:03

If a user gives consent and then later on revokes his consent... do we want to track this? The way I understand it, with regards to "granular logging" in the GDPR document, is that the consent feature only maintains the current consent state - so if revoked, it says 'revoked', no history - and history would be part of the logging. Making sense?

Stephan 25 Jan 2018, 11:09:35

Not wanting to overthink but... just documenting. So far, this tracks that a ''source'' (user, member, ...) has given consent for an ''action''. We are not tracking situations where a source would give consent for an action, on a specific ''target''.

Stephan 26 Jan 2018, 10:14:43

Have pushed code to branch https://github.com/umbraco/Umbraco-CMS/tree/temp-u4-10794

So far: a consent is define by IConsent and has:

  • a Source (Udi) which is whoever is consenting to, eg umb://user/1234
  • an Action (Udi) which is being consented, eg umb://app-actions/share-my-data
  • a State (enum) which can be Pending, Granted, Revoked...
  • a Comment (string) which can be pretty much anything
  • an UpdateDate (DateTime) which is the date the consent was last updated

Consents are managed by an IConsentService that does basic CRUD and can retrieve consents by source, action, action type... and will enforce a unique constraint on (Source, Action).

There is no "application" there because the idea would be that the Action entity type represents the application, so the Action could eg be umb://ucommerce-gdpr/receive-spam.

For Source, in case of members we should use the Member unique ID eg umb://member/ and for users... not sure, we don't seem to have a unique ID for users?

I have implemented the Comment field which could contain the email of whoever is consenting, but I am not entirely happy with this free-form field which is going to be abused, and wondering whether it should be an AdditionalData form meant to contain JSON?

Pushing this to review for discussion.

''also: As soon as we want to use Udis to identify, say, things, with Udis looking like umb://thing/1234, we need to register the "thing" entity type. Currently, the only way to register an entity type is by providing a Deploy IServiceConnector marked with ad-hoc attributes (see the tests for an example). I haven't implemented it for users nor members but we probably want to. Also, ppl will need to do it for their own entity types, eg ucommerce-actions - and then this is where it's awkward - having to implement a deploy connector for this. Discuss?''

Stephan 26 Jan 2018, 14:52:00

Quick note: the branch currently breaks 500+ unit tests - trying to figure out why

Shannon Deminick 26 Jan 2018, 22:22:20

Do we really need to make this all adhere to UDIs? As you've mentioned this is going to require further overhead for both us and developers and it doesn't really add any value.

Stephan 27 Jan 2018, 11:12:46

Well the idea of uniquely identifying source and action looked nice but due to complexity... think I'm going to move back to "a string" and let ppl manage those strings, indeed

Claus Jensen 29 Jan 2018, 07:38:17

I vote for the simple string version as well. I never really figured out what the benefit of doing a UDI'ish thing would be and never got around to asking (busy week).

Stephan 29 Jan 2018, 15:29:15

have simplified, now ready for review (and tests don't fail anymore)

Sebastiaan Janssen 31 Jan 2018, 10:43:35

Had a bit of a chat about this and it seems like it would be best to actually log the history and not just the current state.

Still reviewing the rest, but that's one thing that should be changed for this to be completed.

Sebastiaan Janssen 31 Jan 2018, 11:11:27

PR https://github.com/umbraco/Umbraco-CMS/pull/2424

Stephan 31 Jan 2018, 14:20:50

refactored as per our discussion - now keeping history - see ConsentServiceTests for usage

Alan Mac Kenna 23 Feb 2018, 10:42:16

Something to think about down the line - I'm not sure this should be called a "Consent" logging service. Consent is only one of the lawful bases to process information under the GDPR. You may need to record a user's contractual agreement to processing, for example, so IConsent and the recording of 'consent' could be a bit misleading.

Potentially the service should be working on the more abstract ILawfulBasis/IProcessingBasis of which Consent could be one type of user agreement you would want to record. This opens up the possibility of recording other lawful bases (for example - maybe you want to process based on Legitimate Interest, you should at least be informing the user of your processing based on legitimate interest and with the ILawfulBasis service you could record the fact that you displayed a notice to the user.

The lawful basis that you may wish to record, consent being just one of them: https://gdpr-info.eu/art-6-gdpr/

Alan Mac Kenna 26 Feb 2018, 10:25:43

In case it's of any use here's a few words I wrote on GDPR UX and forms needing to capture & record agreement of more than one type of lawful basis.


Shannon Deminick 26 Feb 2018, 11:00:12

Thanks @alan.mackenna , that's a very indepth article, nice work! I've updated the methods on the IConsentService to be more explicit so that there's room for more flexibility in the IConsentService if required in the future. This API allows you to register consent for pretty much any type you want to specify and also retrieve the data in a variety of ways.

Alan Mac Kenna 26 Feb 2018, 11:10:40

@Shandem sounds good, cheers!

Priority: Normal

Type: Task

State: Fixed




Backwards Compatible: True

Fix Submitted:

Affected versions:

Due in version: 7.9.0

Sprint: Sprint 77

Story Points: 3

Cycle: 7