Re: working progress for UTF-8

[Cyrille Chepelov <cyrille chepelov org>]

Akira,

 here are the details. The "audit" file mentions all .[ch] files as of the
beginning of summer 2001. Files with no tags have not been audited for UTF-8
correctness. Those with just "UTF8" are okay, some are tagged "!UTF8
(done)", meaning they had non-compliances, but that it's now sorted out. 
(forget the "LAZY" tags, these are for the now removed support for
"lazyprop"-style of handling properties). 

I would highly suggest you begin with files tagged "!UTF8 (done)" to see how
it's done; the main complexity here is that the program must still work even
if not HAVE_UNICODE (we will drop non-unicode support one day, but not yet),
and it must also work when there are situations of GTK_CHARSET_MISMATCH
(such as GTK_DOESNT_TALK_UTF8_WE_DO, which was my target for gtk1.4/GNOME1,
or GTK_TALKS_UTF8_WE_DONT which is the case currently on Win32, and of
course the final target is GTK_TALKS_UTF8 and (!GTK_CHARSET_MISMATCH) (this
is the GNOME2 /gtk2 case).

I'll be happy to give more detailed explanations on more specific cases...
lib/prop_text.[ch] is certainly a good place to look at for the interaction
between dia and gtk in the various states of UTF-8 awareness.

> BTW I have another question. it's about gnome-print support.
> Dia can print it out without gnome-print. but it's just
> complex that support the printing for CJK. so Dia also can't
> handle it. if the printer has no fonts for CJK, current
> implementation will not be print out even if possible.
> so it needs to implement like gnome-print for printing with
> all of environment, I think. I mean the fonts should be
> embedded on PS.
> for example, this problem should occur on Windows too. but I
> don't try to run it on Windows. doesn't anyone fall on this
> problem?

The solution currently implemented is the following: there is a module in
lib/ps-utf8, called the "PS Unicoder", which follows the approach taken by
Microsoft in its Postscript drivers; basically, character maps are built on
the fly, using only the characters actually used by the text. It of course
works great in 8859-1, I haven't got reliable and consistent reports 0for
non-8859-1 (let's say, it "sorta works"). Now, I'm not running 8859-1
anymore, but I don't have the time to test (and for only two different
characters...). Of course, managing the encoding tables into something the
printer understands is only 50% of the job...

...Currently, dia depends on having the printer (or the Ghostscript) to 
include a very strict set of fonts. This is the main reason there is not a 
lot of freedom in the choice of fonts. Downloading more fonts means being able to
locate the font files or glyph outlines on the local system or the
potentially remote font server, and turning them into things the printer can
swallow and display nicely. This is not my area of expertise; Lars Clausen
had some work in progress there, you'd probably better ask him (he'll
probably comment if he reads this <grin/>). 

Happy hacking !

	-- Cyrille

-- 
Grumpf.

acconfig.h
app/app_procs.c
app/app_procs.h
app/color_area.c
app/color_area.h
app/commands.c
app/commands.h
app/connectionpoint_ops.c
app/connectionpoint_ops.h
app/create_object.c
app/create_object.h
app/cursor.c
app/cursor.h
app/cut_n_paste.c
app/cut_n_paste.h
app/defaults.c
app/defaults.h
app/dia_embedd.c
app/diagram.c
app/diagram.h
app/diapagelayout.c
app/diapagelayout.h
app/dia-props.c
app/dia-props.h
app/diaunitspinner.c
app/diaunitspinner.h
app/disp_callbacks.c
app/disp_callbacks.h
app/display.c
app/display.h
app/export_png.c
app/filedlg.c
app/filedlg.h
app/grid.c
app/grid.h
app/group.c
app/group.h
app/gtkhwrapbox.c
app/gtkhwrapbox.h
app/gtkvwrapbox.c
app/gtkvwrapbox.h
app/gtkwrapbox.c
app/gtkwrapbox.h
app/handle_ops.c
app/handle_ops.h
app/interface.c
app/interface.h
app/layer_dialog.c
app/layer_dialog.h
app/lineprops_area.c
app/lineprops_area.h
app/linewidth_area.c
app/linewidth_area.h
app/load_save.c
app/load_save.h
app/magnify.c
app/magnify.h
app/main.c
app/menus.c
app/menus.h
app/modify_tool.c
app/modify_tool.h
app/object_ops.c
app/object_ops.h
app/pagesetup.c
app/pagesetup.h
app/paginate_gnomeprint.c
app/paginate_gnomeprint.h
app/paginate_psprint.c
app/paginate_psprint.h
app/pixmaps.h
app/plugin-manager.c
app/plugin-manager.h
app/preferences.c
app/preferences.h
app/properties.c
app/properties.h
app/recent_files.c
app/recent_files.h
app/render_eps.c
app/render_eps.h
app/render_gdk.c
app/render_gdk.h
app/render_gnomeprint.c
app/render_gnomeprint.h
app/render_libart.c
app/render_libart.h
app/scroll_tool.c
app/scroll_tool.h
app/select.c
app/select.h
app/splash.c
app/tool.c
app/tool.h
app/undo.c
app/undo.h
app/win32print.c
app/win32print.h
app/winmain.c
config.h
intl/bindtextdom.c
intl/dcgettext.c
intl/dcigettext.c
intl/dcngettext.c
intl/dgettext.c
intl/dngettext.c
intl/explodename.c
intl/finddomain.c
intl/gettext.c
intl/gettext.h
intl/gettextP.h
intl/hash-string.h
intl/intl-compat.c
intl/l10nflist.c
intl/libgettext.h
intl/libgnuintl.h
intl/loadinfo.h
intl/loadmsgcat.c
intl/localcharset.c
intl/localealias.c
intl/ngettext.c
intl/textdomain.c
lib/arrows.c                UTF8
lib/arrows.h                UTF8
lib/attributes.c            UTF8
lib/attributes.h            UTF8
lib/bezier_conn.c           UTF8
lib/bezier_conn.h           UTF8
lib/beziershape.c           UTF8
lib/beziershape.h           UTF8
lib/boundingbox.c           UTF8
lib/boundingbox.h           UTF8
lib/charconv.c              UTF8(by nature)
lib/charconv.h              UTF8(by nature)
lib/color.c                 UTF8
lib/color.h                 UTF8
lib/connection.c            UTF8
lib/connection.h            UTF8
lib/connectionpoint.h       UTF8
lib/connpoint_line.c        UTF8
lib/connpoint_line.h        UTF8
lib/dia_dirs.c              UTF8
lib/dia_dirs.h              UTF8
lib/diagramdata.c           UTF8
lib/diagramdata.h           UTF8
lib/dia_image.c             UTF8
lib/dia_image.h             UTF8
lib/diamenu.h               UTF8
lib/diavar.h                UTF8
lib/dia_xml.c                   UTF8
lib/dia_xml.h                   UTF8
lib/dummy_dep.h                 UTF8
lib/element.c                   UTF8
lib/element.h                   UTF8
lib/filter.c                    UTF8 (?)
lib/filter.h                    UTF8
lib/focus.c                     UTF8
lib/focus.h                     UTF8
lib/font.c                      ???? (should be OK)                          
lib/font.h                      ???? (should be OK)                          
lib/geometry.c                  UTF8
lib/geometry.h                  UTF8
lib/handle.h                    UTF8
lib/intl.c                      ???? should be OK       
lib/intl.h                      UTF8
lib/lazyprops.c                 (dead)
lib/lazyprops.h                 (dead)
lib/message.c                   +UTF8
lib/message.h                   UTF8
lib/neworth_conn.c              UTF8
lib/neworth_conn.h              UTF8
lib/objchange.c                 UTF8
lib/objchange.h                 UTF8
lib/object.c                    UTF8
lib/object.h                    UTF8
lib/orth_conn.c                 UTF8
lib/orth_conn.h                 UTF8
lib/paper.c                     UTF8
lib/paper.h                     UTF8
lib/plug-ins.c                  wait
lib/plug-ins.h                  wait
lib/poly_conn.c                 UTF8
lib/poly_conn.h                 UTF8
lib/polyshape.c                 UTF8
lib/polyshape.h                 UTF8
lib/properties.c                +UTF8
lib/properties.h                +UTF8
lib/ps-utf8.c                   UTF8(by nature)
lib/ps-utf8.h                   UTF8(by nature)
lib/render.h                    UTF8
lib/render_object.c             (dead)
lib/render_object.h             (dead)
lib/render_store.c              (dead)
lib/render_store.h              (dead)
lib/sheet.c                     UTF8?
lib/sheet.h                     UTF8
lib/string_prerenderer.c        UTF8
lib/string_prerenderer.h        UTF8
lib/text.c                      UTF8 (held off)
lib/text.h                      UTF8
lib/utils.c                     UTF8
lib/utils.h                     UTF8
lib/widgets.c                   UTF8?
lib/widgets.h                   UTF8
objects/chronogram/chronogram.c         UTF8
objects/chronogram/chronogram.h         UTF8
objects/chronogram/chronoline.c         UTF8, LAZY(DONE)
objects/chronogram/chronoline_event.c   NOT_UTF8!(DONE)
objects/chronogram/chronoline_event.h   UTF8
objects/chronogram/chronoref.c           UTF8,LAZY(DONE)
objects/custom/custom.c
objects/custom/custom_object.c
objects/custom/custom_util.c
objects/custom/custom_util.h
objects/custom/shape_info.c
objects/custom/shape_info.h
objects/EML/dbox.c
objects/EML/dbox.h
objects/EML/eml.c
objects/EML/eml.h
objects/EML/instantiation.c
objects/EML/interaction.c
objects/EML/interaction.h
objects/EML/interaction-ortho.c
objects/EML/listfun.c
objects/EML/listfun.h
objects/EML/nlist.c
objects/EML/nlist.h
objects/EML/process.c
objects/EML/process_dialog.c
objects/EML/process.h                     LAZY(taken some care of)
objects/ER/attribute.c
objects/ER/entity.c
objects/ER/er.c
objects/ER/participation.c
objects/ER/relationship.c
objects/flowchart/box.c
objects/flowchart/diamond.c
objects/flowchart/ellipse.c
objects/flowchart/flowchart.c
objects/flowchart/parallelogram.c
objects/FS/flow.c
objects/FS/flow-ortho.c
objects/FS/flow-poly.c
objects/FS/fs.c
objects/FS/function.c
objects/GRAFCET/action.c                  UTF8, LAZY(DONE)
objects/GRAFCET/action_text_draw.c        UTF8
objects/GRAFCET/action_text_draw.h        UTF8
objects/GRAFCET/boolequation.c            UTF8
objects/GRAFCET/boolequation.h            UTF8, LAZY(DONE)
objects/GRAFCET/condition.c               UTF8
objects/GRAFCET/grafcet.c                 UTF8
objects/GRAFCET/grafcet.h                 UTF8
objects/GRAFCET/step.c                    UTF8, LAZY(DONE)
objects/GRAFCET/transition.c              UTF8
objects/GRAFCET/vector.c                  UTF8, LAZY(DONE)
objects/GRAFCET/vergent.c                 UTF8, LAZY(DONE)
objects/network/antenna.c                 (dead)
objects/network/bus.c                     (dead)
objects/network/computer.c                (dead)
objects/network/disc.c                    (dead)
objects/network/flash.c                   (dead)
objects/network/hub.c                     (dead)
objects/network/modem.c                   (dead)
objects/network/modularswitch.c           (dead)
objects/network/monitor.c                 (dead)
objects/network/network.c                 UTF8
objects/network/network.h                 UTF8
objects/network/printer.c                 (dead)
objects/network/rj45plug.c                (dead)
objects/network/scead-plug.c              (dead)
objects/network/wanlink.c                 UTF8
objects/SADT/annotation.c                 UTF8, LAZY(DONE)
objects/SADT/arrow.c                      UTF8, LAZY(DONE)
objects/SADT/box.c                        UTF8, LAZY(DONE)
objects/SADT/sadt.c                       UTF8
objects/SADT/sadt.h                       UTF8
objects/standard/arc.c
objects/standard/bezier.c
objects/standard/beziergon.c
objects/standard/box.c
objects/standard/ellipse.c
objects/standard/image.c
objects/standard/line.c
objects/standard/polygon.c
objects/standard/polyline.c
objects/standard/standard.c
objects/standard/textobj.c
objects/standard/zigzagline.c
objects/sybase/client.c                 (dead)
objects/sybase/dataserver.c             (dead)
objects/sybase/ltm.c                    (dead)
objects/sybase/repserver.c              (dead)
objects/sybase/rsm.c                    (dead)
objects/sybase/stableq.c                (dead)
objects/sybase/sybase.c                 (dead)
objects/sybase/sybase.h                 (dead)
objects/UML/actor.c                     UTF8 + StdProp(DONE)
objects/UML/association.c               !UTF8 (will go with more StdProp)
objects/UML/branch.c                    UTF8
objects/UML/class.c
objects/UML/class_dialog.c
objects/UML/class.h
objects/UML/classicon.c                 UTF8
objects/UML/component.c                 UTF8
objects/UML/constraint.c                UTF8
objects/UML/dependency.c                UTF8
objects/UML/generalization.c            UTF8
objects/UML/implements.c                UTF8
objects/UML/large_package.c             UTF8
objects/UML/lifeline.c                  UTF8
objects/UML/message.c                   !UTF8(DONE)
objects/UML/node.c                      UTF8    
objects/UML/note.c                      UTF8
objects/UML/object.c                    UTF8 + StdProp(DONE)
objects/UML/realizes.c                  UTF8
objects/UML/small_package.c             UTF8
objects/UML/state.c                     UTF8
objects/UML/stereotype.c                !UTF8 (DONE)
objects/UML/stereotype.h                !UTF8 (DONE)
objects/UML/uml.c                       UTF8
objects/UML/uml.h                       UTF8
objects/UML/usecase.c                   UTF8
plug-ins/cgm/cgm.c
plug-ins/dxf/dxf.c
plug-ins/dxf/dxf-export.c
plug-ins/dxf/dxf-import.c
plug-ins/hpgl/hpgl.c
plug-ins/pstricks/pstricks.c
plug-ins/pstricks/render_pstricks.c
plug-ins/pstricks/render_pstricks.h
plug-ins/python/diamodule.c
plug-ins/python/pydia-color.c
plug-ins/python/pydia-color.h
plug-ins/python/pydia-cpoint.c
plug-ins/python/pydia-cpoint.h
plug-ins/python/pydia-diagram.c
plug-ins/python/pydia-diagramdata.c
plug-ins/python/pydia-diagramdata.h
plug-ins/python/pydia-diagram.h
plug-ins/python/pydia-display.c
plug-ins/python/pydia-display.h
plug-ins/python/pydia-error.c
plug-ins/python/pydia-error.h
plug-ins/python/pydia-export.c
plug-ins/python/pydia-export.h
plug-ins/python/pydia-font.c
plug-ins/python/pydia-font.h
plug-ins/python/pydia-geometry.c
plug-ins/python/pydia-geometry.h
plug-ins/python/pydia-handle.c
plug-ins/python/pydia-handle.h
plug-ins/python/pydia-image.c
plug-ins/python/pydia-image.h
plug-ins/python/pydia-layer.c
plug-ins/python/pydia-layer.h
plug-ins/python/pydia-object.c
plug-ins/python/pydia-object.h
plug-ins/python/pydia-properties.c
plug-ins/python/pydia-properties.h
plug-ins/python/pydia-property.c
plug-ins/python/pydia-render.c
plug-ins/python/python.c
plug-ins/shape/shape.c
plug-ins/shape/shape-export.c
plug-ins/svg/render_svg.c
plug-ins/svg/svg.c
plug-ins/wmf/wmf_gdi.h
plug-ins/wpg/wpg.c
plug-ins/wpg/wpg_defs.h
plug-ins/xfig/xfig.c
plug-ins/xfig/xfig-common.c
plug-ins/xfig/xfig-export.c
plug-ins/xfig/xfig.h
plug-ins/xfig/xfig-import.c