Of interest:
• The Complexity And Tedium of Software Engineering
http://xahlee.org/UnixResource_dir/writ/programer_frustration.html
in particular, there are 2 issues with emacs that might be interesting
here.
plain text version follows
--------------------------------------------------
The Complexity And Tedium of Software Engineering
Xah Lee, 2009-06-02
This page is a blog of a experience of few examples that illustrates
some seemingly trivial task can become quite tedius and complicated in
the software industry.
A Complexity with Emacs
Discovered a emacs problem.
Summary:
• Seems that emacs 23 will have a problem loading a css-mode written
by Stefan Monnier
• The css-mode.el file does not contain any sort of version number,
which makes the problem worse.
Detail: I have a Mac running emacs 22 with OS X, and i have PC running
emacs 23 and Windows Vista. When i use the emacs 23 to load css mode,
it gives this error: “if: Wrong type argument: integerp, (0 . 8)”.
The problem seems simple in retrospect, but wasn't simple at all when
you trying to get things done and things don't work as expected.
Here's the story.
Emacs 22 does not have a css mode, emacs 23 does. There's one css mode
written by Stefan. I've been using it on the Mac for the past couple
of years. The same package is now bundled into emacs 23, which i'm
using on PC. However, the code in the 2 files are not identical. I
have my emacs setup to load css mode. Since i am migrating to PC,
alone with my whole emacs system, and am setting up a Mac and PC and
network that'd let me work with either machine harmoniously. On the
PC, when i start css mode, it gives error, but not always. You don't
know what's wrong. It could be a fuckup in the emacs distro i'm using
on PC (which is emacsW32), or it could be emacs 23 problem (emacs 23
is still in beta), or it could be something in my emacs customization
that works perfectly well on the Mac but not on the PC with whole new
environment. Eventually, i realized that's because sometimes i started
plain emacs 23 without loading my own setup, it was using the bundled
css mode, so no error, but when i loaded my own emacs setup, it gives
error. This seems still simple in retrospect, but wasn't then. I added
a version check to my emacs init file(s), so that if emacs is 23,
don't load my css mode. The next day, same symptom occurs. Eventually
I realized that's because the load path is set so that my own version
of css mode comes before the bundled css mode, so that, when i load
css, again my old version is loaded. Eventually, the solution is to
check if css mode bundled with emacsW32 runs ok on emacs 22 on my Mac;
if so, simply use that as the single source for my Mac and PC. When
doing this work on checking what versions are the files, i realized
that the 2 css mode's files don't contain version number at all. All
this takes 5 minutes to read, but was one of the gazillion seemingly
trivial issues and problems when setting my Mac/PC networking
environment with cygwin and all. This took me one week, and haven't
gotten to wholly converting my Mac files to PC. Added the time to
google for for the answer, possibly write a report to the emacsers,
etc, all in all, i'd say this problem caused me 4 hours.
Here's the emacs version i'm using. On the Mac: “GNU Emacs 22.2.1
(powerpc-apple-darwin8.11.0, Carbon Version 1.6.0) of 2008-04-05 on
g5.tokyo.stp.isas.jaxa.jp”. On Windows “GNU Emacs 23.0.94.1 (i386-
mingw-nt6.0.6001) of 2009-05-28 on LENNART-69DE564 (patched)”
--------------------------------------------------
URL Encoding
Discovered a subtle issue with automating url encoding. In url, if you
have the ampersand “&” char, and if this url is to be inside a html
doc as a link, then, there's no automated procedure that determines
correctly whether the char should be encoded as “%26” or “&”. If
the char is used as part of the file name or path, then it should be
encoded as “&”, but if it is used as a separator for CGI
parameters, then it should be encoded as “%26”.
The rule for Percent encoding the char is that the ampersand is a
reserved char, therefore it must be percent encoded if it is used for
normal file path names. So, when it is NOT used as part of path names,
but used as CGI parameter separaters, with GET method of HTTP request,
then it must be left as it is. Now, in html, the ampersand char must
be encoded as html entities “&” when adjacent chars are not space
(basically). So, it becomes “&”.
In summary, if the url in html link contains “&”, and the char is a
CGI param separator, then encode it to “&”, else, encode it as
“%26”, and my discovery is that the purpose of the char used in url
cannot be syntactically determined with 100% accuracy.
Of course, in practice, all this matters shit. Just use “&” plainly
and it all works in 99.999% of situations.
--------------------------------------------------
Unicode File Names
Unison and Unicode File Names
Summary: Unison does not support unicode chars.
When using Unison version 2.27, to sync files from Mac to PC, the file
name on the mac is “赵薇_flag.jpg”, it is copied to PC using Unison
2.27, and the file name became “èµµè-╪_flag.jpg” when viewed under
Windows, but the file name shows up correctly when viewed in EmacsW32.
Mac version: 10.4.11, Windows Vista SP1.
This may indicate that Unison encode file names in utf8, or just
ascii. Indeed, it is said on Wikipedia that unicode have problems with
non-ascii file names.
When the file is copied from Mac to Windows or Windows to Mac,
operating either on Windows or Mac as the local machine, using either
OS's file manager, the file name is copied correctly.
Setting up Unison itself is not so trivial. It is trivial in concept,
but actually took hours. I have Unison on my Mac installed, and use it
few times a year, since about 2006, so i'm familiar with it. On PC,
first you have to install cygwin. I know there are Unison binaries for
Windows but since i use cygwin, so my first choice is staying with
cygwin, since it contains the whole unix environment. Installing
cygwin is another story, but once you installed Unison in cygwin, and
tried to test sync my Mac and PC, you run into the problem that sshd
must be turned on in one of the machines. Namely, sshd should run on
the “remote” machine. (setting up a local network among Win and Mac is
another complex and tedious story) Then, there's the issue of deciding
which machine you want sshd to run or both. On the Mac, i can turn on
sshd in a minute. On Windows, i'm not sure. I'm not sure if Windows
Vista home edition provide ssh server, and am not sure how to turn it
on if so. As far as i know, Windows Vista Home does not come with a
ssh client. In the process, also realize that firewall must be turned
off for ssh port. So, you spend 30 min or possibly hours (here and
there) reading or probing with Windows Firewall control panel and
whatnot other admin tools. After a while, i decided it's easier just
to turn on sshd on the Mac then Unison from the Windows side to the
Mac. At least, have this direction work first, and when that works, i
can play with the other direction. After all this done, i tried to
Unison, but Unison reports that the Unison version on my Mac and PC is
not the same, so it doesn't work. Geeze. The one on my Mac turns out
to be Unison 2.13.x, and the one i have in Cygwin is 2.31.x. Now, i
figured that with each release of Unison, it probably obsolete some
older versions. So, back to digging Unison docs and the web. The
simplest solution comes to mind is to update my Unison on my Mac to
latest, of which, the unison on fink haven't been updated for a couple
of years. I use Fink, and am fairly familiar with using Fink. However,
after “fink selfupdate” etc, then “fink desc Unison”, the version
there reported seems to be 2.13. Then, searching web on fink home page
indicates they have unisone-2.27.57-1007, for my OS 10.4 PowerPC. So,
why doesn't it show up in my fink? I remember i had a case last year
where fink had obsolete mirror databases, a fault entirely on their
end. After spending maybe some more 30 min, i decided to install
Unison from a website, binary download. After that done, i got Unison
2.27.x on the Mac. I tried to sync again, still no go. So, it seems
like that the Unison version number must be the same or very close.
Checking on Unison website, it looks like the current stable release
is 2.27.x, so, my cygwin's 2.31.x is actually a beta. Damn. So, now
back to cygwin. Luckily, it appears there are several version of
Unison there to be installed, and i easily installed 2.27. Then,
finally, test sync is successful. Now, i go back to get my files ready
in a state to be synced. (long story there. See: Perl Script for
Removing Mac Resource Fork and Switching from Mac/Unix To PC/Windows )
When finally i'm ready to Unison, then there's the Chinese character
problem!
--------------------------------------------------
Emacs on Windows and Unicode File Names
When using emacsW32, dired has problem dealing with files with chinese
char on remote machine. Consequently, when trying to copy, delete,
etc, the behavior may be dangerous.
e.g. i have this file 林志玲.jpg on a Mac. I use emacsW32 on Windows to
view it thru network. (e.g. the path is //169.254.145.104/xah/
web/ ... ) the file name shows up as “_viu0y~a.jpg” in dired.
“GNU Emacs 23.0.94.1 (i386-mingw-nt6.0.6001) of 2009-05-28 on
LENNART-69DE564 (patched)”, Mac version: 10.4.11, Windows Vista SP1.
Xah
∑ http://xahlee.org/
☄
On Jun 3, 11:50 pm, Xah Lee <······@gmail.com> wrote:
> Of interest:
> • The Complexity And Tedium of Software Engineering
> http://xahlee.org/UnixResource_dir/writ/programer_frustration.html
Addendum:
The point in these short examples is not about software bugs or
problems. It illustrates, how seemingly trivial problems, such as
networking, transferring files, running a app on Mac or Windwos,
upgrading a app, often involves a lot subtle complexities. For mom and
pop users, it simply stop them dead. For a senior industrial
programer, it means some conceptually 10-minutes task often ends up in
hours of tedium.
In some “theoretical” sense, all these problems are non-problems. But
in practice, these are real, non-trivial problems. These are
complexities that forms a major, multi-discipline, almost unexplored
area of software research. I'm trying to think of a name that
categorize this issue. I think it is a mix of software interface,
version control, release control, formal software specification,
automated upgrade system, etc. The ultimate scenario is that, if one
needs to transfer files from one machine to another, one really should
just press a button and expect everything to work. Software upgrade
should be all automatic behind the scenes, to the degree that users
really don't need fucking to know what so-called “version” of software
he is using.
Today, with so-called “exponential” scientific progress, and software
has progress tremendously too. In our context, that means there are a
huge proliferation of protocols and standards. For example, unicode,
gazillion networking related protocols, version control systems,
automatic update technologies, all comes into play here. However, in
terms of the above visionary ideal, these are only the beginning.
There needs to be more protocols, standards, specifications, and more
strict ones, and unified ones, for the ideal scenario to take place.
Xah
∑ http://xahlee.org/
☄
On Jun 3, 11:50 pm, Xah Lee <······@gmail.com> wrote:
> Of interest:
> • The Complexity And Tedium of Software Engineering
> http://xahlee.org/UnixResource_dir/writ/programer_frustration.html
>
> in particular, there are 2 issues with emacs that might be interesting
> here.
>
> plain text version follows
> --------------------------------------------------
>
> The Complexity And Tedium of Software Engineering
>
> Xah Lee, 2009-06-02
>
> This page is a blog of a experience of few examples that illustrates
> some seemingly trivial task can become quite tedius and complicated in
> the software industry.
>
> A Complexity with Emacs
>
> Discovered a emacs problem.
>
> Summary:
>
> • Seems that emacs 23 will have a problem loading a css-mode written
> by Stefan Monnier
>
> • The css-mode.el file does not contain any sort of version number,
> which makes the problem worse.
>
> Detail: I have a Mac running emacs 22 with OS X, and i have PC running
> emacs 23 and Windows Vista. When i use the emacs 23 to load css mode,
> it gives this error: “if: Wrong type argument: integerp, (0 . 8)”.
>
> The problem seems simple in retrospect, but wasn't simple at all when
> you trying to get things done and things don't work as expected.
> Here's the story.
>
> Emacs 22 does not have a css mode, emacs 23 does. There's one css mode
> written by Stefan. I've been using it on the Mac for the past couple
> of years. The same package is now bundled into emacs 23, which i'm
> using on PC. However, the code in the 2 files are not identical. I
> have my emacs setup to load css mode. Since i am migrating to PC,
> alone with my whole emacs system, and am setting up a Mac and PC and
> network that'd let me work with either machine harmoniously. On the
> PC, when i start css mode, it gives error, but not always. You don't
> know what's wrong. It could be a fuckup in the emacs distro i'm using
> on PC (which is emacsW32), or it could be emacs 23 problem (emacs 23
> is still in beta), or it could be something in my emacs customization
> that works perfectly well on the Mac but not on the PC with whole new
> environment. Eventually, i realized that's because sometimes i started
> plain emacs 23 without loading my own setup, it was using the bundled
> css mode, so no error, but when i loaded my own emacs setup, it gives
> error. This seems still simple in retrospect, but wasn't then. I added
> a version check to my emacs init file(s), so that if emacs is 23,
> don't load my css mode. The next day, same symptom occurs. Eventually
> I realized that's because the load path is set so that my own version
> of css mode comes before the bundled css mode, so that, when i load
> css, again my old version is loaded. Eventually, the solution is to
> check if css mode bundled with emacsW32 runs ok on emacs 22 on my Mac;
> if so, simply use that as the single source for my Mac and PC. When
> doing this work on checking what versions are the files, i realized
> that the 2 css mode's files don't contain version number at all. All
> this takes 5 minutes to read, but was one of the gazillion seemingly
> trivial issues and problems when setting my Mac/PC networking
> environment with cygwin and all. This took me one week, and haven't
> gotten to wholly converting my Mac files to PC. Added the time to
> google for for the answer, possibly write a report to the emacsers,
> etc, all in all, i'd say this problem caused me 4 hours.
>
> Here's the emacs version i'm using. On the Mac: “GNU Emacs 22.2.1
> (powerpc-apple-darwin8.11.0, Carbon Version 1.6.0) of 2008-04-05 on
> g5.tokyo.stp.isas.jaxa.jp”. On Windows “GNU Emacs 23.0.94.1 (i386-
> mingw-nt6.0.6001) of 2009-05-28 on LENNART-69DE564 (patched)”
>
> --------------------------------------------------
> URL Encoding
>
> Discovered a subtle issue with automating url encoding. In url, if you
> have the ampersand “&” char, and if this url is to be inside a html
> doc as a link, then, there's no automated procedure that determines
> correctly whether the char should be encoded as “%26” or “&”. If
> the char is used as part of the file name or path, then it should be
> encoded as “&”, but if it is used as a separator for CGI
> parameters, then it should be encoded as “%26”.
>
> The rule for Percent encoding the char is that the ampersand is a
> reserved char, therefore it must be percent encoded if it is used for
> normal file path names. So, when it is NOT used as part of path names,
> but used as CGI parameter separaters, with GET method of HTTP request,
> then it must be left as it is. Now, in html, the ampersand char must
> be encoded as html entities “&” when adjacent chars are not space
> (basically). So, it becomes “&”.
>
> In summary, if the url in html link contains “&”, and the char is a
> CGI param separator, then encode it to “&”, else, encode it as
> “%26”, and my discovery is that the purpose of the char used in url
> cannot be syntactically determined with 100% accuracy.
>
> Of course, in practice, all this matters shit. Just use “&” plainly
> and it all works in 99.999% of situations.
>
> --------------------------------------------------
> Unicode File Names
>
> Unison and Unicode File Names
>
> Summary: Unison does not support unicode chars.
>
> When using Unison version 2.27, to sync files from Mac to PC, the file
> name on the mac is “赵薇_flag.jpg”, it is copied to PC using Unison
> 2.27, and the file name became “èµµè-╪_flag.jpg” when viewed under
> Windows, but the file name shows up correctly when viewed in EmacsW32.
>
> Mac version: 10.4.11, Windows Vista SP1.
>
> This may indicate that Unison encode file names in utf8, or just
> ascii. Indeed, it is said on Wikipedia that unicode have problems with
> non-ascii file names.
>
> When the file is copied from Mac to Windows or Windows to Mac,
> operating either on Windows or Mac as the local machine, using either
> OS's file manager, the file name is copied correctly.
>
> Setting up Unison itself is not so trivial. It is trivial in concept,
> but actually took hours. I have Unison on my Mac installed, and use it
> few times a year, since about 2006, so i'm familiar with it. On PC,
> first you have to install cygwin. I know there are Unison binaries for
> Windows but since i use cygwin, so my first choice is staying with
> cygwin, since it contains the whole unix environment. Installing
> cygwin is another story, but once you installed Unison in cygwin, and
> tried to test sync my Mac and PC, you run into the problem that sshd
> must be turned on in one of the machines. Namely, sshd should run on
> the “remote” machine. (setting up a local network among Win and Mac is
> another complex and tedious story) Then, there's the issue of deciding
> which machine you want sshd to run or both. On the Mac, i can turn on
> sshd in a minute. On Windows, i'm not sure. I'm not sure if Windows
> Vista home edition provide ssh server, and am not sure how to turn it
> on if so. As far as i know, Windows Vista Home does not come with a
> ssh client. In the process, also realize that firewall must be turned
> off for ssh port. So, you spend 30 min or possibly hours (here and
> there) reading or probing with Windows Firewall control panel and
> whatnot other admin tools. After a while, i decided it's easier just
> to turn on sshd on the Mac then Unison from the Windows side to the
> Mac. At least, have this direction work first, and when that works, i
> can play with the other direction. After all this done, i tried to
> Unison, but Unison reports that the Unison version on my Mac and PC is
> not the same, so it doesn't work. Geeze. The one on my Mac turns out
> to be Unison 2.13.x, and the one i have in Cygwin is 2.31.x. Now, i
> figured that with each release of Unison, it probably obsolete some
> older versions. So, back to digging Unison docs and the web. The
> simplest solution comes to mind is to update my Unison on my Mac to
> latest, of which, the unison on fink haven't been updated for a couple
> of years. I use Fink, and am fairly familiar with using Fink. However,
> after “fink selfupdate” etc, then “fink desc Unison”, the version
> there reported seems to be 2.13. Then, searching web on fink home page
> indicates they have unisone-2.27.57-1007, for my OS 10.4 PowerPC. So,
> why doesn't it show up in my fink? I remember i had a case last year
> where fink had obsolete mirror databases, a fault entirely on their
> end. After spending maybe some more 30 min, i decided to install
> Unison from a website, binary download. After that done, i got Unison
> 2.27.x on the Mac. I tried to sync again, still no go. So, it seems
> like that the Unison version number must be the same or very close.
> Checking on Unison website, it looks like the current stable release
> is 2.27.x, so, my cygwin's 2.31.x is actually a beta. Damn. So, now
> back to cygwin. Luckily, it appears there are several version of
> Unison there to be installed, and i easily installed 2.27. Then,
> finally, test sync is successful. Now, i go back to get my files ready
> in a state to be synced. (long story there. See: Perl Script for
> Removing Mac Resource Fork and Switching from Mac/Unix To PC/Windows )
> When finally i'm ready to Unison, then there's the Chinese character
> problem!
>
> --------------------------------------------------
> Emacs on Windows and Unicode File Names
>
> When using emacsW32, dired has problem dealing with files with chinese
> char on remote machine. Consequently, when trying to copy, delete,
> etc, the behavior may be dangerous.
>
> e.g. i have this file 林志玲.jpg on a Mac. I use emacsW32 on Windows to
> view it thru network. (e.g. the path is //169.254.145.104/xah/
> web/ ... ) the file name shows up as “_viu0y~a.jpg” in dired.
>
> “GNU Emacs 23.0.94.1 (i386-mingw-nt6.0.6001) of 2009-05-28 on
> LENNART-69DE564 (patched)”, Mac version: 10.4.11, Windows Vista SP1.
>
> Xah
> ∑http://xahlee.org/
>
> ☄
From: John Thingstad
Subject: Re: The Complexity And Tedium of Software Engineering
Date:
Message-ID: <op.uu1i7rsout4oq5@pandora>
På Fri, 05 Jun 2009 08:07:39 +0200, skrev Xah Lee <······@gmail.com>:
> On Jun 3, 11:50 pm, Xah Lee <······@gmail.com> wrote:
> The point in these short examples is not about software bugs or
> problems. It illustrates, how seemingly trivial problems, such as
> networking, transferring files, running a app on Mac or Windwos,
> upgrading a app, often involves a lot subtle complexities. For mom and
> pop users, it simply stop them dead. For a senior industrial
> programer, it means some conceptually 10-minutes task often ends up in
> hours of tedium.
What on earth gave you the idea that this is a trivial problem?
Networks have been researched and improved for the last 40 years!
It is a marvel of modern engineering that they work as well as they do.
>
> In some “theoretical” sense, all these problems are non-problems. But
> in practice, these are real, non-trivial problems. These are
> complexities that forms a major, multi-discipline, almost unexplored
> area of software research.
Again, it is it not a trivial problem theoretically.
Unexplored? What world are you on?
> I'm trying to think of a name that
> categorize this issue. I think it is a mix of software interface,
> version control, release control, formal software specification,
> automated upgrade system, etc. The ultimate scenario is that, if one
> needs to transfer files from one machine to another, one really should
> just press a button and expect everything to work. Software upgrade
> should be all automatic behind the scenes, to the degree that users
> really don't need fucking to know what so-called “version” of software
> he is using.
>
Actually they mostly are. At least on my machine. (I use Windows XP and
Ubuntu Linux.)
> Today, with so-called “exponential” scientific progress, and software
> has progress tremendously too. In our context, that means there are a
> huge proliferation of protocols and standards. For example, unicode,
> gazillion networking related protocols, version control systems,
> automatic update technologies, all comes into play here. However, in
> terms of the above visionary ideal, these are only the beginning.
> There needs to be more protocols, standards, specifications, and more
> strict ones, and unified ones, for the ideal scenario to take place.
>
No, there are already to many protocols and the ideas of how a network
infrastructure should be built are mostly in place. I think we would
benefit from "cleaning up" the existing interface. That is by removing
redundancy.
What does need further research is distributed processing. Again this is a
highly complex problem and a lot of work has been put into trying to make
simpler and more manageable interfaces and protocol's. See for example the
languages Erlang and Oz to get an idea.
---------------------
John Thingstad
From: Kenneth Tilton
Subject: Re: The Complexity And Tedium of Software Engineering
Date:
Message-ID: <4a297a08$0$31258$607ed4bc@cv.net>
Xah Lee wrote:
> On Jun 3, 11:50 pm, Xah Lee <······@gmail.com> wrote:
>> Of interest:
>> • The Complexity And Tedium of Software Engineering
>> http://xahlee.org/UnixResource_dir/writ/programer_frustration.html
>
> Addendum:
>
> The point in these short examples is not about software bugs or
> problems. It illustrates, how seemingly trivial problems, such as
> networking, transferring files, running a app on Mac or Windwos,
> upgrading a app, often involves a lot subtle complexities. For mom and
> pop users, it simply stop them dead. For a senior industrial
> programer, it means some conceptually 10-minutes task often ends up in
> hours of tedium.
Quibble: those are not /tedious/. Those are as fascinating as an episode
of House, trying not only to get new information but how to get it and
how to figure out when some information already in hand is actually
misinformation, a classic solution to hard problems. Also figuring out
coincidences mistaken for cause and effect.
But that is just a quibble, ie, I think you need a different word, and
it is OK if it still conveys some form of unleasantness.
Hair-pulling? Head-banging?
>
> In some “theoretical” sense, all these problems are non-problems. But
> in practice, these are real, non-trivial problems. These are
> complexities that forms a major, multi-discipline, almost unexplored
> area of software research. I'm trying to think of a name that
> categorize this issue. I think it is a mix of software interface,
> version control, release control, formal software specification,
> automated upgrade system, etc. The ultimate scenario is that, if one
> needs to transfer files from one machine to another, one really should
> just press a button and expect everything to work. Software upgrade
> should be all automatic behind the scenes, to the degree that users
> really don't need fucking to know what so-called “version” of software
> he is using.
I think you are looking for an immaculate road system on a volcanic
island still growing ten feet a day.
>
> Today, with so-called “exponential” scientific progress, and software
> has progress tremendously too. In our context, that means there are a
> huge proliferation of protocols and standards. For example, unicode,
> gazillion networking related protocols, version control systems,
> automatic update technologies, all comes into play here. However, in
> terms of the above visionary ideal, these are only the beginning.
> There needs to be more protocols, standards, specifications, and more
> strict ones, and unified ones, for the ideal scenario to take place.
But when would we write the software? Even with all the head-banging,
look what we have been able to do with computers, leaving aside for the
moment the flight control algorithms of the Airbus?
When progress stops we will have time to polish our systems, not before.
But then you will be able to use the word "tedium".
kt
On 2009-06-05 21:03:33 +0100, Kenneth Tilton <·········@gmail.com> said:
> When progress stops we will have time to polish our systems, not before.
Is that an endorsement of mediocrity?
--
JFB
From: Kenneth Tilton
Subject: Re: The Complexity And Tedium of Software Engineering
Date:
Message-ID: <4a2a6321$0$31269$607ed4bc@cv.net>
verec wrote:
> On 2009-06-05 21:03:33 +0100, Kenneth Tilton <·········@gmail.com> said:
>
>> When progress stops we will have time to polish our systems, not before.
>
> Is that an endorsement of mediocrity?
No, of General Patton.
hth, kt