HTML-mails content not stored in description of ticket

[hju@…]

We encountered a new problem with html-mails. The mail-content is not transfered in the ticket description. (It is a new installed version of email2trac under windows...)

No problems till now with html-mails under the old installation (0.3)

Example: Looks like ticket #135

Any idea?

[bas]

Just for clarity is the HMTL version not added as attachment to the ticket? If it is this the normail behaviour.

[hju@…]

Yes, the HTML-Version is attached as untiteled-part.2.html

So the different behaviour was a "bug" in our old version? (Got HTML-Attachment plus text!)

If it is this the normail behaviour.

OK, but this is not very friendly to use!

Instead of reading the description you must:

• click on the attachment
• click on download...
• click on "OK" in the Dialog Box

Is there any way to get a Text-Version of the Mail instead of the html-attachment? (as an option: CONVERT_HTML_TO_TEXT: 1)

[bas]

Your right it is not friendly. Maybe we have some utility that converts the html mail. Just a note:

• html messages an contain images and other strange tags
• people can sent multipart messages that also can contain html messages. So we have to parse all of them.

Or do you have a solution for html messages

[hju@…]

Or do you have a solution for html messages

Not yet... (and I'm no mail-specialist)

But I'll have a look at our "old" version (0.3)

# html messages an contain images and other strange tags

Thats right. But in version 0.3 of email2trac, the text and images of a html-mail were (accidently?) converted into Description and Attachments (HTML-Text and Images)

[hju@…]

Hy Bas,

I've looked around a little. Maybe we got the description because the sending mailer includes both, text/plain and text/html?

There is a HTML-Parser in Python (since 2.2) which can hopefully be used to translate the html-body auf these mails.

• ​http://www.python.org/doc/2.4.4/lib/module-HTMLParser.html
• ​http://www.python.org/doc/2.4.4/lib/module-htmllib.html

Also I found some examples for converting HTML to Text

• ​http://cis.poly.edu/cs912/parsing.txt
• ​http://pypi.python.org/pypi/htmltotext/0.6

I think the best way should be: (if CONVERT_HTML_TO_TEXT is true)

• .walk through the email as you do
• if there is a text/plain part, use it for description (some mailers add both, text/plain and text/html)
• if no text/plain, convert text/html to text and build description
• inline grafics should be attached
• links should be added as a list(or: converted to WikiFormatting in text)

What do you think? It's just a snap for you, isn't it?

[basv@…]

That is the situation in most cases text and html. Thanks for the
pointers for HTML conversions.
When i message is send in text and html is marked as alternative. In
1.0.0 there is an option to drop the html version. So it is not
attached to the ticket.

Your steps are logical and some are already implemented. I have to
plan some time for it. Currently i am a  bit swamped in other
projects. So it will take some time to implement.


--
Bas van der Vlies
basv@sara.nl

[olle@…]

+1 on this. All users that have Outlook (99% in our organisation) send (html) mails that ends up as an attachment. It creates alot of extra work for us that deals with Trac (and confusion for the user that is coming back to look at his/her ticket). The description/message appears as an attachment named "None". We use Outlook 2010. Trac runs on Ubuntu.

Our users appreciate the fact that they don't have to use the Trac-interface, so I'd hate to remove this feature for them.

Email that comes from Apple Mail (1%) works like a charm!

[bas]

Apple mail sent both in plain text and html, Outlook only html that is the root cause of the problem. I am alwys open for patches ;-)

[anonymous]

I would if I could :) I'm not a developer, but I'd be happy to pay $50 for this feature if that makes a difference!

[hju@…]

Replying to bas:

Apple mail sent both in plain text and html, Outlook only html that is the root cause of the problem.

I would if I could :) I'm not a developer,

Well, it seems that Outlook may be configured to send html AND plain text too. I sometimes receive mails from an Microsoft Exchange V6.5 Server which contain both. I'm sorry but I don't know ho to configure it, but perhaps it is worth to have a look at it.

[bas]

That is true Outlook can send mail in html and text. The most people only have html format configured. And email2trtac an not handle html email, only as attachment. Maybe i gonna use the command line 'html2text' to convert it. I do not know if it is available on Windows.

[bas]

I just tied the html Trac wikiproccessor. But that does not work. I have to strip a lot of html tags before i get some text in the ticket

[anonymous]

Would html2text do this for you?

​http://www.linuxquestions.org/questions/linux-general-1/convert-html-emails-to-plain-text-emails-150252/

[bas]

the output is a bit weird. But it could work. There are several other utilities that accomplish the same result. So maybe we have to use external program or internal that only supports a subset

[anonymous]

Hmm. I've been looking at the code of those emails being sent and it seems like Outlook 2010 is actually including a text-version separate to the html version. Hmm.

Both 
Content-Type: text/plain; charset="iso-8859-1"
and
Content-Type: text/html; charset="iso-8859-1"
exist...

[bas]

Can you attach an raw exampe. If you set debug level it will save the raw email. So i can easily test it.

[anonymous]

I've set debug=1, but the attachment (filename: None) looks the same without debug. I did reload apache before testing.

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Verdana;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";
	mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.E-postmall17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";
	mso-fareast-language:EN-US;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=SV link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Does this text show?<o:p></o:p></p><p class=MsoNormal><o:p>&nbsp;</o:p></p><p class=MsoNormal><b><span style='font-size:9.0pt;font-family:"Verdana","sans-serif";color:#C90019;mso-fareast-language:SV'>Eva L</span></b><b><span style='font-size:9.0pt;font-family:"Verdana","sans-serif";mso-fareast-language:SV'><o:p></o:p></span></b></p><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Verdana","sans-serif";color:black;mso-fareast-language:SV'>Event<br><b><br>Kvadrat Stockholm AB</b><br><b>Sveavägen 90, 7tr</b> Besöksadress<br><b>Box 45002, 104 30 Stockholm</b> Postadress<br><br><b>08-419</b> Telefon<br><b>08-62</b> Telefax<br><b>070-51</b> Mobil</span><span style='mso-fareast-language:SV'><o:p></o:p></span></p><p class=MsoNormal><o:p>&nbsp;</o:p></p></div></body></html>

[bas]

what version do you use and on which os? I must have the raw email. It is sometginh like:

• tmp*,email2trac

Tis file is just the email message that is being parsed by enail2trac

[anonymous]

Outlook 2010, Windows 7. I just sent you an email with the raw email attached. I'm using Apple Mail myself.

[bas]

To whom did you send it? It is not in this ticket.

[anonymous]

basv at sara dot nl Didn't want to share the raw content of that email with everyone.

[bas]

ok. That i have to wait. Did not receive it yet. or it is blocked. You can also upload it to the ftp.sara.nl and put in the pub/incoming directory.

[anonymous]

Done

[howmuch@…]

I'm having a similar set of problems and getting asked by management to fix it. Is there any progress on this? If not, I'd be happy to work on it myself.

[bas]

I have some ideas but not had the time to do it and its not that easuas it sounds. I am always open for patches and code contributions ;-)

[olle@…]

I'd be happy to contribute with testing and feedback! I've noticed it's quite hard to get users to remember/select "send as plain text" in Outlook :)

[anonymous]

Any news, Bas?

[bas]

Not from my side. I am a bit busy with migration projects

[bas]

Just a question on which os is trac installed. For Linux we could use:

• html2text

I just did a raw implementations and it seems to work.

[bas]

(In [498]) added support for html to text conversion, in email2trac.conf:

• html2text_cmd: /usr/bin/html2text -nobs

see #152

[anonymous]

I am now testing the HTMLParser python modules. amd i think i will switch to use this module instead of html2text which is an external program

[olle@…]

Super! Let me know if you need any testing done.

[bas]

The code to convert html to text is already in the trunk version. But that uses the html2text command. So you could test that.