U4-8197 - Remove translations trees and affiliated files

Created by Shannon Deminick 17 Mar 2016, 12:35:17 Updated by Ronald Barendse 10 Jul 2018, 13:21:18

Tags: Up For Grabs

Subtask of: U4-5419

umbraco.loadOpenTasks & umbraco.loadYourTasks still uses BaseTree/ITree, needs to be removed along with all associated files, config and settings.

The translation system is old and out dated and needs to be replaced with an xliff based system

Comments

Kevin Jump 25 Jun 2018, 12:47:05

Hi few thoughts on translations for v8.

The current translation workflow with the creation of tasks from target nodes is broken in at least --two-- three ways.

=== 1. Translated Content Creation ===

With no link between the nodes in umbraco you can only send the current content to translation, so if you are say in a french site you can only send the french text to translate - this is wrong because that text may already be translated - you need to send the English text from another node to be translated and then imported over the french node ''(this means a relationship between the nodes needs to exist before the option should be presented)''

this may not matter as much with v8 variants - but it still will be an issue for people building sites with different language nodes (a mix of 1-1 and is different nodes is common in multilingual builds)

=== 2. XML Format===

The xml format is non-standard and it contains all the meta-data (e.g. JSON from the grid)

=== 3. Task flow===

The Tasks don't contain state, so if create a task, but then update content and create a new task it overwrites the existing in-progress task - when you do get a file back it will be out of sync / miss content.

=Proposal : Don't have a translation section=

Instead have option(s) on a node to 'create/download' and 'import/upload' translation file(s) directly to the content.

This way you bypass the task bit (that doesn't work anyway) - and the user just exports and imports files they send of to be translated. (and you don't need the section).

''this option should only appear on multivariat nodes or nodes with a relationship to another language node in the site''

=== XML / Xliff ===

The format of the xml is an issue, this could be updated to XLIFF quite quickly, but you would still have to address the JSON in content issues.

Content from metadata

The grid and others have lots of json - it needs filtering to remove content for the file, and then the content needs putting back into the right place when the translation comes back - the Translation Manager (and the EU solution i believe) both have code to do the mapping of data from content to and from

in the case of Translation Manager - there is a IValueMapper interface for this, these mappers are similar to propertyEdtiorConverters - but have to work both ways for just the content bits (so no ids etc). They are more like deploy Value Connectors but for content not ids,

I am not sure of the willingness to create Yet-Another-Value-Mapper in the core - but that is what would be needed to put the clean XLIFF of all properties into the process.

just mapping the existing content to XLIFF would be a start.

or you could leave this functionality out as translation manager does handle the translation (and workflow) of the content anyway ;)


Ronald Barendse 09 Jul 2018, 20:59:24

I agree with @KevinJump: instead of rewriting the existing (outdated) code to the new structure (API/AngularJS), I would put that effort into allowing better workflows to be built (especially for the new vary by culture in V8).

One of the main problems comes from the extensibility Umbraco offers: all property editors can define their own data format and therefore require different mappers to extract IDs/UDIs for deployment (Umbraco Deploy, uSync, etc.) and text for export/import and translations (CMSImport, Translation Manager, etc.). You can't expect every package author to write all of these mappers, so it would be nice to have something in the core that returns this meta-data.

Having a unified way to indicate what data is stored in an editor (or actually the property value) would make most of these mappers obsolete. The meta-data should at least indicate these values using a key/index:

  • Text: including info whether it's translatable (and if so, in what language it's written)
  • HTML: also whether it's translatable and be parsed for links
  • UDI: content, media, members or even other custom types (like the ones generated by Fluidity)

The current Umbraco.Core.Deploy.IValueConnector could be extended to return this meta-data or maybe even the Umbraco.Core.PropertyEditors.IPropertyValueConverter. Because almost every editor uses a PropertyValueConverter (and otherwise should create one!), maybe some attributes on the returned object could provide the required meta-data (just like data annotations used by ORMs or validation).


Shannon Deminick 10 Jul 2018, 01:18:32

Hi all! I've changed this title and description to "Remove..." we completely agree that this is an old/out dated system. In fact i was under the impression that we removed this functionality some time ago, but maybe it's just been hidden at some stage.

@simonech has an xliff system that they built for their very large multi-lingual website and I know he's been working on getting something that could either be part of core for v8 or work as a plugin for v8. @simonech I can't recall if there was a task/issue setup for this already? If not maybe we should open one to move the above discussion there?


Ronald Barendse 10 Jul 2018, 13:21:18

Nice, this should make V8 even more clean... Now lets start the discussion what preparations (especially breaking changes) would be needed to allow new translation and export/import functionality to be added (in a later V8 release or package).

I see @simonech has created an XLIFF library (https://github.com/simonech/XliffLib) and talked about the implementation for the website of the Council of the European Union at CG18 (http://codeclimber.net.nz/archive/2018/05/22/codegarden18-git-xliff2-ci-cd/). Would be great to combine all knowledge and come up with a great OOTB experience in Umbraco!


Priority: Up for grabs

Type: Task

State: Open

Assignee:

Difficulty: Normal

Category:

Backwards Compatible: False

Fix Submitted:

Affected versions:

Due in version: 8.0.0

Sprint:

Story Points:

Cycle: