Summary
=======
I've made some changes to the way email going through the UKTC is
handled. Hopefully the impact of this will be mainly positive, but some
issues are possible. Feedback regarding any problems is invited, off
list to uktc-admin@xxxxxx.tree-care.info
Changes
=======
The script responsible for adding headers and footers to messages, and
for removing or converting HTML parts has been completely rewritten.
Desired impact
--------------
Sponsors' messages and other headers and footers should now display
correctly irrespective of the character set that the original email was
written in.
Headers added to messages with attachments should no longer appear to
Outlook users as the entire message, with the original message
appearing as a text attachment. This is a bug in Outlook, rather than
the UKTC systems, but as Outlook is so widely used it seems sensible
to work around it.
Where it is necessary to convert HTML parts to plain text the method of
doing this has been improved.
Known issues
------------
The main issue identified affects those subscribers receiving plain
text digests. For the time being plain text digests are likely to
contain a warning above each message about the message originally
using the character set UTF-8, and all 'non-ascii' characters will
be missing or replaced with question marks. This includes all accented
characters and the '£' character.
Potential issues
----------------
Users with older email software, and possibly some users of web mail
systems, may experience unusual rendering of 'non-ascii' characters, eg
accented characters, the £ (pound) and € (Euro) symbols, © (copyright)
symbol, ° (degree) symbol, fractions such as ½ (half) etc.
I AM PARTICULARLY SEEKING FEEDBACK ON HOW WIDESPREAD THIS ISSUE IS.
If the example characters above appear as junk to you, please let me
know off list by emailing uktc-admin@xxxxxx.tree-care.info, indicating
which email programme and operating system (eg Windows XP, Mac OS X
etc) you are using. If you are receiving the plain text digest there is
no need to contact me - I am aware of the issues with this.
If you experience any other issues following this change, please
advise me on uktc-admin@xxxxxx.tree-care.info, again indicating what
email programme and operating system you use, what the problem is and
which message(s) you have seen it in.
Background
==========
All messages to the UKTC are passed through a filter which removes or
converts any HTML parts and adds the sponsors' messages and other
footers. In order to keep down the amount of memory required by this,
when the filter was written a number of assumptions were made. In
particular, it was assumed that an incoming email would use one of a
small number of character sets.
At the time, this was a reasonable assumption. Almost all English
language email used one of these character sets. However, this is no
longer the case. In particular, the use of unicode based character sets
such as UTF-8 is becoming increasingly common. Unicode is, in effect, a
superset of existing character sets which contains pretty much every
character in common use throughout the globe (including all the Chinese
ones), and a good many that aren't in common use.
Additionally, the way in which the filter added text to messages that
contained attachments, although perfectly valid, and in many ways the
most sensible way to go, caused problems for Outlook.
The filter has now been rewritten from the ground up. In order to
ensure there is no conflict between character sets used for headers
and footers and those used for the main message, all messages are being
converted to UTF-8. The way of dealing with messages with attachments
has been changed to improve the way Outlook displays such messages.
Furthermore, improvements have been made to the routines for converting
HTML to plain text.
--
Chris Hastie
Administrator
The UK Tree Care Mailing List
http://uk.tree-care.info/
--
The UK Tree Care mailing list
To unsubscribe send mailto:uktc-unsubscribe@xxxxxx.tree-care.info
The UKTC is supported by The Arbor Centre
http://www.arborcentre.co.uk/
