U4-9068 - Truncate string containing emoji fails

Created by Søren Kottal 13 Oct 2016, 11:15:24 Updated by Sebastiaan Janssen 03 Nov 2016, 16:54:11

Tags: PR

When truncating a string containing emoji using the UmbracoHelper, a YSOD is returned containing the following stacktrace:

Exception type: EncoderFallbackException Exception message: Unable to translate Unicode character \uD83E at index 99 to specified code page. at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknown, Int32 index) at System.Text.EncoderFallbackBuffer.InternalFallback(Char ch, Char& chars) at System.Text.UTF8Encoding.GetBytes(Char chars, Int32 charCount, Byte* bytes, Int32 byteCount, EncoderNLS baseEncoder) at System.Text.EncoderNLS.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush) at System.Text.EncoderNLS.GetBytes(Char[] chars, Int32 charIndex, Int32 charCount, Byte[] bytes, Int32 byteIndex, Boolean flush) at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) at System.IO.StreamWriter.Dispose(Boolean disposing) at System.IO.TextWriter.Dispose() at Umbraco.Web.HtmlStringUtilities.Truncate(String html, Int32 length, Boolean addElipsis, Boolean treatTagsAsContent) at Umbraco.Web.UmbracoHelper.Truncate(String html, Int32 length, Boolean addElipsis) at ASP._Page_Views_Partials_Modules_NewsItem_cshtml.Execute() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance) at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer) at Umbraco.Core.Profiling.ProfilingView.Render(ViewContext viewContext, TextWriter writer) at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection) at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData) at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model) at ASP._Page_Views_Partials_Modules_NewslistWithFilter_cshtml.Execute() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance) at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer) at Umbraco.Core.Profiling.ProfilingView.Render(ViewContext viewContext, TextWriter writer) at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection) at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData) at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName) at ASP._Page_Views_Homepage_cshtml.b__1() at System.Web.WebPages.WebPageBase.<>c__DisplayClass7.b__5(TextWriter tw) at System.Web.WebPages.HelperResult.WriteTo(TextWriter writer) at System.Web.WebPages.WebPageBase.Write(HelperResult result) at ASP._Page_Views_Master_cshtml.Execute() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy() at System.Web.Mvc.WebViewPage.ExecutePageHierarchy() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.WebPages.WebPageBase.<>c__DisplayClass3.b__2(TextWriter writer) at System.Web.WebPages.HelperResult.WriteTo(TextWriter writer) at System.Web.WebPages.WebPageBase.Write(HelperResult result) at System.Web.WebPages.WebPageBase.RenderSurrounding(String partialViewName, Action1 body) at System.Web.WebPages.WebPageBase.PopContext() at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance) at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer) at Umbraco.Core.Profiling.ProfilingView.Render(ViewContext viewContext, TextWriter writer) at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResult(ControllerContext controllerContext, ActionResult actionResult) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult) at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList1 filters, ActionResult actionResult) at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<>c__DisplayClass2b.<BeginInvokeAction>b__1c() at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<BeginInvokeAction>b__1e(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase1.End() at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult) at System.Web.Mvc.Controller.<BeginExecuteCore>b__1d(IAsyncResult asyncResult, ExecuteCoreState innerState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase1.End() at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult) at System.Web.Mvc.Controller.<BeginExecute>b__15(IAsyncResult asyncResult, Controller controller) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase1.End() at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult) at System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute(IAsyncResult asyncResult) at System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__5(IAsyncResult asyncResult, ProcessRequestState innerState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

The emojis used are "🚙💨😉! Go Hard Or Go Home 🇯🇵🤘🏽! "

Comments

Lotte Pitcher 03 Nov 2016, 14:48:19

At the Hackathon we were able to replicate this YSOD behaviour if the truncate was cutting off an emoji half way through (i.e. halfway through the two character unicode pair). A PR is coming that tests the final character and, if it is the first half of a two character code pair, also brings the second half.


Lotte Pitcher 03 Nov 2016, 16:12:27

PR: https://github.com/umbraco/Umbraco-CMS/pull/1586


Priority: Normal

Type: Bug

State: Fixed

Assignee:

Difficulty: Normal

Category:

Backwards Compatible: True

Fix Submitted: Pull request

Affected versions: 7.3.1

Due in version: 7.5.5

Sprint:

Story Points:

Cycle: