CON-1483 - Option of not storing the submitted data on the form in the Forms data store

Created by Claus Jensen 02 Jan 2018, 08:16:54 Updated by Alan Mac Kenna 20 Feb 2018, 15:21:28

Subtask of: U4-10796

For Forms we need to add the option of not storing the submitted data on a form internally in Forms. The default state of this should also be configurable in a config file (ie. StoreDataByDefault: false). If the editor choose not to store the form data internally, it’s solely up to the workflows in the form to handle the data, store it elsewhere if needed and it won’t be possible to use the built-in reporting features of Forms.

For data housekeeping in Forms, it’s already possible to search for a given date frame and bulk delete entries (thus in line with the good manners of not keeping data forever).

Comments

Niels Hartvig 25 Jan 2018, 08:07:20

When StoreDataByDefault is true, we should make it default to include a consent field for new forms. It would be in the format of a mandatory radiobutton list with no default option selected (we'll work on the default wording and it should be configurable (using the build-in UI language translation):

Consent for storing submitted data

  • It'll be stored safely and in according to our privacy policy ( ) Yes, I give permission to store my data ( ) No, I'd like my data to be removed

(the reason to use radio buttons with yes and no options is that it gives the best opt-in rate and that it's a part of GDPR that people should actively give consent).


Warren Buckley 26 Jan 2018, 11:42:56

PR for this is here - https://github.com/umbraco/Forms/pull/178/files

Notes

*New UI option on each form under settings - to toggle Store Records in DB - we had this already but no UI *Entries child tree node is always visible now & does not hide if the Store Records in Db is set to false *Entries viewer - will display a warning message to remind users that form is not storing any records in the DB when setting is disabled *New forms created, blank or via templates we ship (contact form) or even customer templates will add a dataConsent field automatically as the last field of the form *The new consent field if site visitor answers no then Workflows still execute but we do not store the record in the DB - even if the form setting is setup to do so *We only store the answer in the DB if the site visitor answers yes to the dataConsent field *Some countries ie non-EU will not need this GDPR rule, so they can remove the field from the form scaffold *New field added to form uses 3 dictionary keys - so customers can tweak the copy to their liking **We have a migration for 6.0.7 that adds in the 3 dictionary keys required with default copy/values *The logic for this is that we check that dataConsent field is present on the form and if so we check what value the site visitor selected, by comparing the copy from the dictionary item. So for this to work customers need to use the dictionary keys. So we will need to explain/document this in case people remove the dictionary keys from the field with static copy

Test Notes

  • Verify the new Form migration for 6.0.7 fires & you get three dictionary keys with populated copy

  • Create a new blank form or one from a template & confirm the dataConsent field is added in

  • Goto settings of Form designer - verify form is storing records in DB

  • Render the form on a page & confirm the dictionary labels render out on the page

  • Fill out the form and answer YES to the dataConsent question ** Verify the answer stored in the DB/can be seen in the entries viewer & confirm workflows executed

  • Fill out the form and answer NO to the dataConsent question ** Verify the answer is NOT in the entries viewer but still confirm workflows executed/email sent for example

  • Goto the form designer and settings & disable/uncheck the store records in the db option

  • Goto the entries viewer and confirm you see the new warning message

  • Create a new form that allows stuff to be stored in the DB

  • Remove the dataConsent field that we automatically add

  • Add form to page and ensure all submissions are always stored in the DB

  • Toggle store in DB setting on the form & confirm that new entries are no longer stored but workflows still execute


Rick Mason 26 Jan 2018, 17:20:57

It's worth noting that consent is not always required within the EU either. There are other legal bases for processing data which may apply depending on the form. Also if a form is optional then simply choosing to fill it in can be taken as consent to processing the data (according to our GDPR expert), so long as that is explained in the privacy notice.


Alan Mac Kenna 28 Jan 2018, 19:09:30

If I'm reading this right it implies that storing the data in form entries will only be done if consent is given. My understanding is that it is not necessarily 'storing' the data in form entries that is the problem without consent - it is processing the data in the first place. If the workflow executes regardless and the data is sent in an email, for example, it is 'processed' and ends up being stored somewhere. This is what you need consent for - if indeed consent is the lawful basis that you are choosing to collect and process personal information under. Apologies if I've misunderstood the intentions here!


Alan Mac Kenna 28 Jan 2018, 19:18:21

Regarding the use of radio buttons and a link to the privacy policy. GDPR requires unbundled consent. It suggests you present the required information in a layered manner. Linking to a general privacy policy will not meet the test.

Ideally, we would be able to create a form field that we can use to present layers of info to the user. A UI item that could work for this is a layered modal that allows the user to click for an explanation before giving consent. They can drill down a hierarchy of relevant info (using links in the modal text where relevant) that the GDPR requires to be presented to the user. They could click 'close' within the modal at any time or 'back' to go back to the previous 'layer' of info.

It doesn't have to be GDPR specific, the functionality would enable what GDPR suggests as a way to transparently inform and unbundle processing purposes and related info from your privacy policy. It might even be useful for other purposes!


Niels Hartvig 29 Jan 2018, 09:01:48

@caterwomtious I guess experts interpret the GDPR in different ways. The people we've consulted, said that you definitely need unbundled consent.

@alan.mackenna the form field with modal option is a great idea and something we'll consider for a future iteration #h5yr


Alan Mac Kenna 29 Jan 2018, 12:12:40

@hartvig Sounds good re: form field modal. Thanks.

It's important not to equate unbundled consent with providing a radio button and saying "It'll be stored safely and in accordance with our privacy policy"

This does not achieve unbundled consent. For unbundled consent to be achieved, required info - e.g. who is the data controller, purposes of processing etc. needs to be unbundled from the privacy policy and displayed in context to the user. Just want to make sure that's clear.

Cheers.


Alan Mac Kenna 20 Feb 2018, 15:21:28

@hartvig fyi I wrote a piece on GDPR and UX with an example of unbundling and layered in-context delivery of processing & privacy info.

https://www.serveit.com/gdpr-user-experience/


Priority: Normal

Type: Task

State: Fixed

Assignee:

Difficulty:

Category:

Backwards Compatible: True

Fix Submitted:

Affected versions: 6.0.6

Due in version: 7.0.0

Sprint: Sprint 77

Story Points: 3

Cycle: 7