Randall Randall wrote:
> Yep, they are. However, when I was first learning CL, the argument
> that these sorts of libraries didn't exist publicly because they were
> so *easy* seemed bogus. So while those of you who've been using CL
> for more than a few months will undoubtedly find little of use in
> things like my gettext package and unicode package, I want those
> searching for these in the future to find them, so it's clear that
> such things do exist.
For me, lisp turns out to be a great description language, since
English is surprisingly bad; authors make too many assumptions which
not everyone share. (Like say, why XML is so great. You read their
glowing introductions, then wonder when it is you're supposed to
realize XML's greatness.)
So for me at least, the best thing people can do is come along and
describe some domain they have experience with, in lisp. Like text
handling. Dave Roberts posted a nice little state machine once, and
I've used it in a couple exploratory programs as kind of a motif to
build on. So, thanks!
MfG,
Tayssir
Tayssir John Gabbour wrote:
> For me, lisp turns out to be a great description language, since
> English is surprisingly bad; authors make too many assumptions which
> not everyone share. (Like say, why XML is so great. You read their
> glowing introductions, then wonder when it is you're supposed to
> realize XML's greatness.)
>
Heh, funny you should comment on this right now. Currently, at my work,
there are quite a few programmers trying to push XML as our data
storage format, and a handful of us that are opposed (of which I am
one). Initially, there was no real reason for opposition, just a gut
feeling that it was a bad choice.
My main opposition was that it was crazy to use XML in place of a
system that could just as easily be handled with s-expr or even a
simple line based format like:
begin <node>
begin <child>
key1 = value
key2 = value
end
end
However, the ayes had it, and it was initially introduced. Well, it
turns out that the gut feeling a few of us had was right. For anyone
else currently going through this, here are a few reasons to *not* use
XML as a data storage format:
1. Whitespace is a killer -- especially when you require it
2. Data usage is through the roof (this is being done on a console,
where it is really bad to create an XML tree in memory, instead of just
referring back to the original text).
3. We have yet to find a merge utility that handles XML (or any other
non-line based format) well. Because of this, the XML data that should
be accessible by anyone is now checkout-exclusive. We're currently
using Perforce.
4. Extremely verbose. Vs. line based formats, XML tends to be ~ 4x as
large. Unacceptable on a console.
5. Error checking is a real pain (ie close tags in the wrong order). I
can't count how many malformed XML files broke game code.
There are a couple more small things (like the fact that we only
support a subset of XML, and when a new hire is brought in, they
instantly try and use what we don't support, etc). And I am sure there
are others. I'm currently revisiting the problem and trying to convince
everyone to switch to another format -- anything else! ;)
Before I go in on Tuesday with my list of problems and why a different
format will solve most (if not all) of these, anyone else have any
other glowing examples of baggage that XML brings with it?
Jeff
Stefan Ram wrote:
> "Jeff" <···@nospam.insightbb.com> writes:
> > 1. Whitespace is a killer -- especially when you require it
>
> I do not understand, what you refer to here.
For example, given the following:
<format> Compressed DXT5 </format>
What is the format: " Compressed DXT5 " or "Compressed DXT5"? XML
(like HTML) will strip prefixed and tailing whitespace, as well as
trimming extra whitespace. It isn't possible to insert \n or \t
anywhere in a text string where it may be required. It would just be
more clear to have a format that wrapped required text in quotes.
Jeff
"Jeff" <···@nospam.insightbb.com> writes:
> 5. Error checking is a real pain (ie close tags in the wrong order). I
> can't count how many malformed XML files broke game code.
Not surprising. A certain amount of redundancy is necessary for error
correction, but it is not sufficient. Just adding redundancy strictly
*increases* the liklihood of error. You need an additional mechanism
to recover from errors.
> Heh, funny you should comment on this right now. Currently, at my work,
> there are quite a few programmers trying to push XML as our data
> storage format, and a handful of us that are opposed (of which I am
> one). Initially, there was no real reason for opposition, just a gut
> feeling that it was a bad choice.
I was recently ordered to use XML for a big project's input file
format. I was initially opposed, but now I'm okay with it. I still
would prefer sexps, but I make concessions.
So here's what I've discovered. I hope it helps you make the best of
what you've got going on.
There's a lot of technologies layered on top of XML. Use them;
there's tools written to use them.
For validity testing (which XML refers to as using "schemas"), RELAX
NG is very good. There is another one called XML Schema, which is
pretty rotten IMHO, but part of that had to do with my particular
needs.
There's a transformation language called XSLT that's good. In the
project I'm working on now, I use XSLT to extract a subset of the
information in the XML document and form a set of sexps that my Lisp
code can handle better. Presently, I then use Lisp to transform it in
odd ways into a Prolog expression. I'd use XSLT to go the whole way
if it weren't that I need an optimizer, and while XSLT is
Turing-complete, LIsp is better.
Emacs's built-in XML mode sucks. Get nXML mode. You can hand it a
RELAX NG schema, and it will then handle completion, realtime validity
testing, etc. (If your schema isn't in RELAX NG, you can use a tool
called Trang to convert it.) Other editors, such as oXygen/, are
around too.
Find a good library. Ideally, it should support the technologies
you're using. You mentioned that your library doesn't preserve
whitespace; that means that it's broken. All my XML handling today
is done in Perl, and I use XML::LibXML. As I mentioned, I use XSLT to
make sexps before I hand it off to Lisp. C has libxml2. If you use
Java, then there's lots of XML libraries, such as Xerces.
I don't know what the state of the art in Lispland is regarding XML.
I think that Craig wrote some XML stuff, cursing all the while. You
can kluge together a minimal XML-handling readtable fairly quickly, if
you really want to.
> 2. Data usage is through the roof (this is being done on a console,
> where it is really bad to create an XML tree in memory, instead of just
> referring back to the original text).
How about using XML for source, but a binary structure (tokenized,
linked, etc.) on the distributed disk? You could (for instance) use
XSLT to transform it to a sexp representation, and then compile-file
that, and put that on the disk.
> 3. We have yet to find a merge utility that handles XML (or any other
> non-line based format) well. Because of this, the XML data that should
> be accessible by anyone is now checkout-exclusive. We're currently
> using Perforce.
Remember the other technologies I mentioned? One of them is XInclude.
> 4. Extremely verbose. Vs. line based formats, XML tends to be ~ 4x as
> large. Unacceptable on a console.
Again, binary structure on disk, XML in the source file.
> 5. Error checking is a real pain (ie close tags in the wrong order). I
> can't count how many malformed XML files broke game code.
If you compile to a binary structure, then it will be tested for
well-formedness (that is, tags nest properly) when you do that
compile. Your editor should also help you maintain well-formedness.
If you want, you can also use a schema technology such as RELAX NG to
test for validity (that is, everything's where it should be). Some
editors (including nxml-mode and oXygen/) can test validity while
you're writing.
These days, I have a sort of love-hate relationship with XML. I'd
rather be using sexps, but that'd be unacceptable to my lusers, who
are scared of anything that you can't parse in commodity libraries.
So while I think that XML is a piece of crap-- mostly overly verbose--
I've learned to deal with it, and use XML's strengths (mostly,
widespread industry adoption) to my advantage. Not to sound
saccharine, it's the lemons/lemonade thing.
When I first started using XML, I was constantly cursing it. There
was a tangle of intertwined technologies, with different levels of
support for them in different libraries. But after I sorted through
some of the mishmash, I got okay with it.
This is the section of my browser bookmarks that I have XML references
on. I hope it helps.
<DT><H3>XML</H3>
<DL><p>
<DT><A HREF="http://www.axkit.org/wiki/view/AxKit/XMLReferenceCard">Reference card</A>
<HR>
<DT><H3>XML</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/REC-xml">XML 1.0 W3C Recommendation (Second Edition), 2002-10-06</A>
<HR>
<DT><A HREF="http://www.w3schools.com/xml/xml_whatis.asp">XML Tutorial at w3schools.com</A>
</DL><p>
<DT><H3>XML Base</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xmlbase/">XML Base W3C Recommendation, 2001-06-27</A>
</DL><p>
<DT><H3>XInclude</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xinclude/">XInclude 1.0 W3C Candidate Reommendation, 2002-10-17</A>
</DL><p>
<DT><H3>XHTML</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xhtml1/">XHTML 1.0 W3C Recommendation (Second Edition), 2002-08-01</A>
<DT><A HREF="http://www.w3.org/TR/html4/">HTML 4.01 W3C Recommendation 1999-12-24</A>
<HR>
<DT><A HREF="http://www.w3schools.com/xhtml/xhtml_intro.asp">XHTML Tutorial at w3schools.com</A>
<DT><A HREF="http://www.zvon.org/xxl/xhtmlReference/Output/index.html">XHTML Reference at zvon.org</A>
<DT><A HREF="http://www.zvon.org/xxl/CSS2Reference/Output/index.html">CSS2 Reference at zvon.org</A>
</DL><p>
<DT><H3>XSLT</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xslt">XSLT 1.0 W3C Recommendation, 1999-11-16</A>
<HR>
<DT><A HREF="http://www.w3schools.com/xsl/xsl_intro.asp">XSLT Tutorial at w3schools.com</A>
<DT><A HREF="http://www.zvon.org/xxl/XSLTReference/Output/index.html">XSLT Reference at zvon.org</A>
<DT><A HREF="http://www.zvon.org:9001/saxon/cgi-bin/XLab/XML/xlabIndex.html?stylesheetFile=XSLT/xlabIndex.xslt">X Lab at zvon.org</A>
<DT><H3>EXSLT</H3>
<DL><p>
<DT><A HREF="http://www.exslt.org">EXSLT Website</A>
</DL><p>
</DL><p>
<DT><H3>XPath</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xpath">XPath 1.0 W3C Recommendation 1999-11-16</A>
<HR>
<DT><A HREF="http://www.zvon.org/xxl/XPathTutorial/Output/index.html">XPath Tutorial at zvon.org</A>
<DT><A HREF="http://www.zvon.org/xxl/XSLTreference/Output/index.html">XSLT Reference at zvon.org</A>
<DT><A HREF="http://www.zvon.org:9001/saxon/cgi-bin/XLab/XML/xlabIndex.html?stylesheetFile=XSLT/xlabIndex.xslt">X Lab at zvon.org</A>
</DL><p>
<DT><H3>XLink</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xlink/">XLink W3C Recommendation, 2001-06-27</A>
<HR>
<DT><A HREF="http://www.zvon.org/xxl/xlink/Output/xlink_refs.html">XLink Reference at zvon.org</A>
</DL><p>
<DT><H3>XPointer</H3>
<DL><p>
<DT><A HREF="http://www.w3.org/TR/xptr-framework/">XPointer Framework W3C Proposed Recommendation, 2002-11-13</A>
<DT><A HREF="http://www.w3.org/TR/xptr-element/">XPointer element() Scheme W3C Proposed Recommendation, 2002-11-13</A>
<DT><A HREF="http://www.w3.org/TR/xptr-xmlns/">XPointer xmlns() Schene W3C Proposed Recommendation, 2002-11-13</A>
<DT><A HREF="http://www.w3.org/TR/xptr-xpointer/">XPointer xpointer() Schene W3C Proposed Recommendation, 2002-11-13</A>
<HR>
<DT><A HREF="http://www.zvon.org/xxl/xpointer/refs/Output/xpointer_refs.html">XPointer Reference at zvon.org</A>
</DL><p>
<DT><H3>RELAX NG</H3>
<DL><p>
<DT><A HREF="http://www.relaxng.org/">Home page</A>
<DT><A HREF="http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=relax-ng">OASIS TC</A>
<HR>
<DT><A HREF="http://www.relaxng.org/spec.html">Specification</A>
<DT><A HREF="http://www.relaxng.org/compact.html">Compact Syntax</A>
<DT><A HREF="http://www.relaxng.org/compatibility.html">DTD Compatibility</A>
<DT><A HREF="http://www.relaxng.org/xsd.html">XML Schema Datatypes</A>
<HR>
<DT><A HREF="http://www.relaxng.org/tutorial.html">RELAX NG Tutorial</A>
<DT><A HREF="http://www.relaxng.org/compact-tutorial.html">RELAX NG Compact Syntax Tutorial</A>
<DT><A HREF="http://zvon.org/xxl/RelaxNG/Output/index.html">RELAX NG Reference at zvon.org</A>
<HR>
<DT><A HREF="http://books.xmlschemata.org/relaxng/ch14s01.html">Extensible schemas</A>
</DL><p>
<HR>
<DT><H3>libxml2</H3>
<DL><p>
<DT><A HREF="file:///usr/local/share/doc/libxml2/xml.html">Website (local copy)</A>
<DT><A HREF="file:///usr/local/share/doc/libxml2/html/index.html">Reference Manual</A>
<DT><A HREF="file:///usr/local/share/doc/libxml2/FAQ.html">FAQ</A>
<DT><A HREF="file:///usr/local/share/doc/libxml2/tutorial/index.html">Tutorial</A>
<DT><A HREF="file:///usr/local/share/doc/libxml2/encoding.html">Encodings support</A>
</DL><p>
<DT><H3>libxslt</H3>
<DL><p>
<DT><A HREF="file:///usr/local/share/doc/libxslt/html/xslt.html">Website (local copy)</A>
<DT><A HREF="file:///usr/local/share/doc/libxslt/html/index.html">Reference Manual</A>
<DT><A HREF="file:///usr/local/share/doc/libxslt/html/FAQ.html">FAQ</A>
<DT><A HREF="file:///usr/local/share/doc/libxslt/html/tutorial/libxslttutorial.html">Tutorial</A>
<DT><A HREF="file:///usr/local/share/doc/libxslt/html/xsltproc.html">xsltproc command-line tool</A>
</DL><p>
<DT><A HREF="http://zvon.org/ZvonSW/ZvonGraphotron/index.html">Graphotoron - index</A>
<DT><A HREF="http://zvon.org/index.php?nav_id=home&mime=xml_xslt">ZVON.org</A>
<DT><A HREF="http://www.w3.org/XML/">Extensible Markup Language (XML)</A>
<DT><A HREF="http://wwws.sun.com/software/xml/developers/index.html">XML - Developer Connection</A>
</DL>
Cheers,
joelh