Email encoding issues

dstj · August 17, 2009, 4:37pm

Hi,

Upgrading from 4.6.0.1 to the latest 4.7.2.4 seems to have created issues with email encoding. This results in some of my emails scoring “very high” on some spam filters.

Check the images attached to this post to see the results from my tests with French accentuated sentences. The only thing that changed is the Aspose.Network version. As you can see, the FROM and subject field encoding “broke” with the new version. UTF-8 seems to be the new default, but I don’t see any mention of that in the release notes. The problem seems to have been introduced in v4.7.1.0.

Aside from the viewing problem, the biggest problem for me is that the SPAM score in Barracuda jumped because of this.

pts rule name description

2.87 BAD_ENC_HEADER Message has bad MIME encoding in the header

For the meantime, I reverted back to 4.6.0.1. I find it bizarre that there is no mention of this important change in the release notes though.

Can you provide any assistance?

Thanks for your help,

dstj.

Here is the code used to send the test emails:

Aspose.Network.License license = new Aspose.Network.License();
license.SetLicense(@“Aspose.Total.lic”);

const string toEmail = “me@mydomain.com”;
const string fromEmail = “me@mydomain.com”;
const string senderEmail = “noreply@mydomain.ca”;

const string smtpServer = “smtp.myisp.com”;

Aspose.Network.Mail.MailMessage email;
Aspose.Network.Mail.SmtpClient client;

//ISO-8859-1
email = new Aspose.Network.Mail.MailMessage();
email.Sender = new Aspose.Network.Mail.MailAddress(senderEmail);

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, “TEST Montréal”);
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, “TEST Montréal”));

email.SubjectEncoding = System.Text.Encoding.GetEncoding(28591); //ISO-8859-1 (Western ISO)
email.Subject = “ISO-8859-1: Voix ambiguë d’un coeur qui au zéphyr préfère les jattes de kiwis.”;

email.BodyEncoding = System.Text.Encoding.GetEncoding(28591); //ISO-8859-1 (Western ISO)
email.TextBody = “ISO-8859-1: Voix ambiguë d’un coeur qui au zéphyr préfère les jattes de kiwis.”;

client = new Aspose.Network.Mail.SmtpClient(smtpServer);
client.AuthenticationMethod = Aspose.Network.Mail.SmtpAuthentication.None;
client.Send(email);

// UTF 8 version
email = new Aspose.Network.Mail.MailMessage();
email.Sender = new Aspose.Network.Mail.MailAddress(senderEmail);

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, “TEST Montréal”);
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, “TEST Montréal”));

email.SubjectEncoding = Encoding.UTF8;
email.Subject = “UTF-8: Voix ambiguë d’un coeur qui au zéphyr préfère les jattes de kiwis.”;

email.BodyEncoding = System.Text.Encoding.GetEncoding(28591);
email.TextBody = “UTF-8: Voix ambiguë d’un coeur qui au zéphyr préfère les jattes de kiwis.”;

client = new Aspose.Network.Mail.SmtpClient(smtpServer);
client.AuthenticationMethod = Aspose.Network.Mail.SmtpAuthentication.None;
client.Send(email);

// Default version
email = new Aspose.Network.Mail.MailMessage();
email.Sender = new Aspose.Network.Mail.MailAddress(senderEmail);

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, “TEST Montréal”);
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, “TEST Montréal”));

[//email.SubjectEncoding](https://email.subjectencoding/) = System.Text.Encoding.GetEncoding(28591);
email.Subject = “Default: Voix ambiguë d’un coeur qui au zéphyr préfère les jattes de kiwis.”;

[//email.BodyEncoding](https://email.bodyencoding/) = System.Text.Encoding.GetEncoding(28591);
email.TextBody = “Default: Voix ambiguë d’un coeur qui au zéphyr préfère les jattes de kiwis.”;

client = new Aspose.Network.Mail.SmtpClient(smtpServer);
client.AuthenticationMethod = Aspose.Network.Mail.SmtpAuthentication.None;
client.Send(email);

iret · August 18, 2009, 10:01am

Hi dstj,

Thanks for your post.

Could you please save the mailmessage files to disk for our debugging? You can just call the MailMessage.Save(filename)

We will get back to you ASAP.

dstj · August 18, 2009, 11:44am

Hi,

Here are the saved eml files, for v4.6.0.1, v4.7.1.0 and v4.7.2.4. But with the code I provided, you could have probably reproduced the same files for your tests.

Let me know what you find out.

Dominic.

iret · August 19, 2009, 10:11am

Hello,

You can specify the encoding used for an email address, for example:

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, "TEST Montréal", System.Text.Encoding.GetEncoding(28591));
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, "TEST Montréal",System.Text.Encoding.GetEncoding(28591)));

Could you add it to your code, and see if it helps?

Thanks,

dstj · August 19, 2009, 1:45pm

Hi,

It works for the From, To and Sender, but the Subject in Thunderbird still displays :
=?iso-8859-1?Q?ISO-8859-1: Voix ambigu=EB d’un coeur qui au z=E9phyr pr=E9f=E8re les jattes de kiwis.?= for the ISO-8859-1 email.

Secondly, I find it very confusing to specify encoding everywhere. Is there a way to specify a DefaultEncoding for all email fields (subject, textbody, htmlbody, from, to, sender, headers, etc.) ? This would reduce the maintenance effort when changes are made by Aspose to the component.

Lastly, why wasn’t the UTF-8 change mentionned in the release notes? This bad encoding problem is very bad for producing high quality emails. Plus, it ranks high in spam filters. This is for me considered a breaking change in the API.

Thanks

Dominic.

iret · August 22, 2009, 11:01am

Hi Dominic,

Thanks for your reply.

For the subject issue on Thunderbird, we will fix it ASAP.

For a default encoding fo an email message, we will add some feature to support that.

For the default encoding to UTF-8, we will think about to change it.

I will get back to you shortly.

dstj · October 27, 2009, 1:16pm

Hi,

I upgraded to version 4.8.0.0 and noticed that a problem reported in this thread is not fixed.

In Thunderbird, the subject field is improperly decoded when using ISO-8859-1 all around.

Attached is a archive with two .eml files. In the “fix” version, I only edited the Subject field to make it load properly in Thunderbird v2.0.0.23 (20090812).

Subject: =?iso-8859-1?Q?ISO-8859-1: Voix ambigu=EB d’un coeur qui au z=E9phyr
pr=E9f=E8re les jattes de kiwis.?=

becomes

Subject: =?iso-8859-1?Q?ISO-8859-1=3A_Voix_ambigu=EB_d’un_coeur_qui_au_z=E9?=
=?iso-8859-1?Q?phyr_pr=E9f=E8re_les_jattes_de_kiwis=2E?=

as it were in v4.6.0.1. Can you confirm the problem?

Also, can you provide an update on a “DefaultEncoding” property?

Thanks,

Dominic.

iret · October 28, 2009, 11:41am

Hi Dominic,

We are still working on this issue. I will keep you posted as soon as it is ready.

Thanks,

iret · November 5, 2009, 7:35am

Hi,

Could you please try the attached dll? To set the preferred text encoding setting, you can use The MailMessage.PreferredTextEncoding property.

//ISO-8859-1

var email = new Aspose.Network.Mail.MailMessage();

email.PreferredTextEncoding = System.Text.Encoding.GetEncoding(28591);

email.Sender = new Aspose.Network.Mail.MailAddress("test@mail.com");

email.From = new Aspose.Network.Mail.MailAddress("test@mail.com", "TEST Montral");

email.To.Add(new Aspose.Network.Mail.MailAddress("test@mail.com", "TEST Montral"));

email.Subject = "ISO-8859-1: Voix ambigu d'un coeur qui au zphyr prfre les jattes de kiwis.";

email.TextBody = "ISO-8859-1: Voix ambigu d'un coeur qui au zphyr prfre les jattes de kiwis.";

Thanks,

dstj · November 5, 2009, 5:27pm

Hi, thanks for the follow up. Problems remain or have appeared.

Using
email.PreferredTextEncoding = System.Text.Encoding.GetEncoding(28591); //ISO-8859-1 (Western ISO)

The FROM field is still in UTF-8:

From: "=?utf-8?Q?TEST Montr=C3=A9al?=" me@mydomain.com

See the three eml files attached.

The Thunderbird subject field now presents problem with the “Default” encoding. See the attached image.

Also, why do you now “base64 encode” your utf-8 text contain?

As it is, this hotfix will not resolve the problems I have with Aspose.Network.

Thanks,

Dominic.

By the way, you’re missing all the accentuated characters from your test sentence. So your tests won’t really work. Here’s another test sentence containing french diatrics that doesn’t
really mean anything : “Déjà ambiguë, à Montréal, je préfère haïr ce
garçon.”

aspose.notifier · November 9, 2009, 5:11pm

The issues you have found earlier (filed as 11387) have been fixed in [this update ](http://www.aspose.com/community/files/51/.net-components/aspose.network-for-.net/entry206641.aspx).

This message was posted using Notification2Forum from Downloads module by aspose.notifier.

dstj · November 9, 2009, 6:04pm

You have published the hotfix, but some problems that I have reported still remain. For example, when using ISO-8859-1 as the preferred encoding, the From and To fields are still encoded as UTF-8.

See the ISO-8859-1 .eml headers :

MIME-Version: 1.0
From: “=?utf-8?Q?FROM Montr=C3=A9al?=” me@mydomain.ca
Sender: “noreply@mydomain.ca” noreply@mydomain.ca
To: “=?utf-8?Q?TO Montr=C3=A9al?=” you@mydomain.ca
Date: 9 Nov 2009 18:36:58 -0500
Subject: =?iso-8859-1?B?SVNPLTg4NTktMTogROlq4CBhbWJpZ3XrLCDgIE1vbnRy6WFsLCBqZSBwculm?=
=?iso-8859-1?B?6HJlIGhh73IgY2UgZ2Fy529uLg==?=
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

ISO-8859-1: D=E9j=E0 ambigu=EB, =E0 Montr=E9al, je pr=E9f=E8re ha=EFr ce ga=
r=E7on.

Here is the code used for my tests. If my test is faulty, please explain where is the mistake.

// ref. Email encoding issues
Aspose.Network.License license = new Aspose.Network.License();
license.SetLicense(@“Aspose.Total.lic”);

const string toEmail = “you@mydomain.ca”;
const string fromEmail = “me@mydomain.ca”;
const string senderEmail = “noreply@mydomain.ca”;

const string smtpServer = “smtp.myisp.com”;

const string fromString = “FROM Montréal”;
const string toString = “TO Montréal”;
const string testString = “Déjà ambiguë, à Montréal, je préfère haïr ce garçon.”;

Aspose.Network.Mail.MailMessage email;
Aspose.Network.Mail.SmtpClient client;

//ISO-8859-1
email = new Aspose.Network.Mail.MailMessage();
email.PreferredTextEncoding = System.Text.Encoding.GetEncoding(28591); //ISO-8859-1 (Western ISO)

email.Sender = new Aspose.Network.Mail.MailAddress(senderEmail, senderEmail);

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, fromString);
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, toString));

email.Subject = "ISO-8859-1: " + testString;

email.TextBody = "ISO-8859-1: " + testString;

client = new Aspose.Network.Mail.SmtpClient(smtpServer);
client.AuthenticationMethod = Aspose.Network.Mail.SmtpAuthentication.None;
client.Send(email);
email.Save(@“C:\temp\asposeNetworkTest02-iso-8859-1.eml”);

// UTF 8 version
email = new Aspose.Network.Mail.MailMessage();
email.PreferredTextEncoding = Encoding.UTF8;

email.Sender = new Aspose.Network.Mail.MailAddress(senderEmail);

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, fromString);
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, toString));

email.Subject = "UTF-8: " + testString;

email.TextBody = "UTF-8: " + testString;

client = new Aspose.Network.Mail.SmtpClient(smtpServer);
client.AuthenticationMethod = Aspose.Network.Mail.SmtpAuthentication.None;
client.Send(email);
email.Save(@“C:\temp\asposeNetworkTest02-utf-8.eml”);

// Default version
email = new Aspose.Network.Mail.MailMessage();
email.Sender = new Aspose.Network.Mail.MailAddress(senderEmail, senderEmail);

email.From = new Aspose.Network.Mail.MailAddress(fromEmail, fromString);
email.To.Add(new Aspose.Network.Mail.MailAddress(toEmail, toString));

email.Subject = "Default: " + testString;

email.TextBody = "Default: " + testString;

client = new Aspose.Network.Mail.SmtpClient(smtpServer);
client.AuthenticationMethod = Aspose.Network.Mail.SmtpAuthentication.None;
client.Send(email);
email.Save(@“C:\temp\asposeNetworkTest02-default.eml”);

iret · November 9, 2009, 11:37pm

Hi,

I have logged this issue. We cannot resolve the From/To default encofing setting based on current design. We need to change some design to make it fixed. I will keep you posted.

But you can specify a specified encoding when create email address as a workaround at this time. For example:


MailAddress CreateMailAddressDefaultEncoding(string address, string displayname)
{
return new MailAddress(address, displayname, Encoding.GetEncoding(28591));
}

MailMessage messge = new MailMessage();
message.Form = CreateMailAddressDefaultEncoding(user@domain.com, "displayname");
message.To.Add(CreateMailAddressDefaultEncoding(user2@domain.com, "displayname2"));

dstj · November 10, 2009, 9:24am

Hi,

Sure, I can do that, but it kind of misses the point of having a new property called “PreferredTextEncoding”, doesn’t it?

Until the issues affecting the quality of the emails sent via Aspose.Network are fully resolved, I’ll keep on using v4.6.0.1. I cannot risk using a defective component.

I do appreciate your efforts, but I have to say that the last few releases of Aspose.Network were just not up par. Outbound email are crucial to my organization and my confidence in Aspose.Network as been somewhat diminished by the breaking changes, such as the passage to UTF-8 as default, that were not even mentioned in the release notes!

Please keep me posted on the progress of these issues.

Dominic.

dstj · January 19, 2010, 12:04pm

Hi,

Can you provide an update on the issues reported in this thread? Is the PreferredTextEncoding property now working with the “From” and “To” fields?

Can you also provide an estimate on when the next release will be out so I can test it?

Thank you for your answers,

Dominic.

saqib.razzaq · January 21, 2010, 12:03am

Hi Dominic,

We are still working on these issues. I will update you about the estimated time of the next release shortly. We are sorry for the inconvenience.

dstj · February 11, 2010, 9:58pm

Hi,

Using Aspose.Network 4.9.0.0, the encoding issues with the From/To fields still do not appear to be corrected.

But I have come up with another one. Even when using the code you suggested for To and From fields, the encoding does not respect RFC 2047 when the displayName
contains white spaces. The spaces are not properly encoded. This results in the
SpamAssassin flag BAD_ENC_HEADER which raises the spam score…

email.To.Add(new Aspose.Network.Mail.MailAddress(toAddress, displayName, System.Text.Encoding.GetEncoding(28591)));

For example :
displayName = “This is a demo”

will get encoded as :

From: “=?iso-8859-1?Q?This is a demo?=” myemail@mydomain.com

This violates the RFC. Here the related excerpt:

IMPORTANT: 'encoded-word’s are designed to be recognized as 'atom’s
   by an RFC 822 parser.  As a consequence, unencoded white space
   characters (such as SPACE and HTAB) are FORBIDDEN within an
   ‘encoded-word’.  For example, the character sequence

      =?iso-8859-1?q?this is some text?=

   would be parsed as four 'atom’s, rather than as a single ‘atom’ (by
   an RFC 822 parser) or ‘encoded-word’ (by a parser which understands
   ‘encoded-words’).  The correct way to encode the string “this is some
   text” is to encode the SPACE characters as well, e.g.

      =?iso-8859-1?q?this=20is=20some=20text?=

   The characters which may appear in ‘encoded-text’ are further
   restricted by the rules in section 5.

Can you please provide a quick fix for this issue as it can significantly impact email delivery.

Thanks a lot,

Dominic.

dstj · February 12, 2010, 9:18am

Hi again,

To be fair to Aspose, the RFC issue I reported seems to be a defect in .NET itself. See this bug reported on Microsoft Connect: https://connect.microsoft.com/VisualStudio/feedback/details/367014/system-net-mail-mailaddress-encodes-mail-addresses-incorrectly

Is there anything that can be done, this is truly a big pain…

Thanks,

Dominic.

saqib.razzaq · February 12, 2010, 2:59pm

Hi Dominic,

We are still working on it. We will notify you as soon as we make some progress to fix this issue.

aspose.notifier · February 27, 2010, 8:24am

The issues you have found earlier (filed as 13781) have been fixed in [this update ](http://www.aspose.com/community/files/51/.net-components/aspose.network-for-.net/entry224476.aspx).

This message was posted using Notification2Forum from Downloads module by aspose.notifier.