From: billc
Subject: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <04cbc1d0-2418-47fe-b2c2-929b2fb01562@1g2000hsl.googlegroups.com>
Hi all,

After a bit of a break over the Christmas/New Year period, we will be
holding our first lispvan meeting of 2008. It promises to be an
interesting one too - Brad Beveridge has a strong background in
embedded systems programming and has recently started using Lisp in
some of the work that he has been doing. His presentation will cover a
few of the things that he's been up to.

Unfortunately, our normal venue (the Think! Cafe) has closed down, so
I had to find an alternative. I think Calhoun's Coffee House and
Restaurant is a pretty good option. It has a good selection of food,
coffee, beer, free wifi, is relatively quiet, is relatively
"large" (big enough for our group as well as regular patrons), and
caters to laptop users and students. Unfortunately, there won't be a
projector, so we'll have to VNC a couple of laptops so that we aren't
all crowded around Brad's laptop. We've done that in the past, so this
shouldn't be a problem (so long as some people remember to bring
laptops! ;-) ).

Here's the "official" meeting notice:

Topic: Doing Evil Things with CL
Presenter: Brad Beveridge
Date: Thursday, February 28, 2008
Time: 7pm - 10pm (or whenever)
Venue: Calhoun's, 3035 West Broadway, Vancouver
Summary: Brad will be talking about various low-level bits and pieces,
including:

- Loading code that is normally a Linux driver into a Lisp image
- Creating CFFI bindings to foreign code
- Writing Lisp code to load binary files using the binary-types
library
- Parsing Mach-O (OS X) executable files

Brad indicated that he probably doesn't have enough material or code
to present for very long, so he would like to treat the presentation
as a way to start discussion.

Join us for a beer/coffee and a chance to see/discuss how CL can be
used for embedded systems work.

Additional links and any updates will be posted on my blog:
http://bc.tech.coop/blog/080205.html

--
Bill Clementson

From: viper-2
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <70f81ce6-f72c-4082-a3bc-0ef37eac24ef@l32g2000hse.googlegroups.com>
On Feb 5, 2:53 pm, billc <········@gmail.com> wrote:
> Hi all,
>

> Restaurant is a pretty good option. It has a good selection of food,
> coffee, beer, free wifi, is relatively quiet...
>
> Join us for a beer/coffee ...

I prefer coffee in the mornings. Do they serve Red Stripe? If so, I
might be tempted to drop in.

When you've had enough of embedded systems I could talk to you about
some of my adventures with Lisp 3rd edition. ;-)

agt
From: billc
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <5f8f29b2-0857-422e-94f5-d7bcc3761ab7@l1g2000hsa.googlegroups.com>
On Feb 5, 3:05 pm, viper-2 <········@mail.infochan.com> wrote:
> On Feb 5, 2:53 pm, billc <········@gmail.com> wrote:
> > Restaurant is a pretty good option. It has a good selection of food,
> > coffee, beer, free wifi, is relatively quiet...
>
> > Join us for a beer/coffee ...
>
> I prefer coffee in the mornings. Do they serve Red Stripe? If so, I
> might be tempted to drop in.

Unfortunately, they don't have Red Stripe, but if you want to fly over
to attend the meeting, I'll bring along some Red Stripe for you. ;-)

> When you've had enough of embedded systems I could talk to you about
> some of my adventures with Lisp 3rd edition. ;-)

If you're referring to the 3rd edition of the ANSII CL spec, then,
yes, that would make a fascinating topic!

- Bill
From: viper-2
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <3b8115c5-a00d-4855-9558-dee623a5c46f@s8g2000prg.googlegroups.com>
On Feb 6, 11:33 am, billc <········@gmail.com> wrote:


> Unfortunately, they don't have Red Stripe, but if you want to fly over
> to attend the meeting, I'll bring along some Red Stripe for you. ;-)

Just make sure the beer is red and not black.

http://bc.tech.coop/blog/071002.html

:-)


> > When you've had enough of embedded systems I could talk to you about
> > some of my adventures with Lisp 3rd edition. ;-)
>
> If you're referring to the 3rd edition of the ANSII CL spec, then,
> yes, that would make a fascinating topic!

I'm surprised that Lisp3rd edition doesn't grab you ;-)


But talking about the ANSII CL spec doesn't grab me.

I tell you what: I'll come up with a Lispy topic that will fascinate
your Lispvan passengers and if the invitation is still open I'll fly
up to another meeting later on.

agt
From: viper-2
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <4f5dd16b-15c1-455c-bec0-114670c19c92@e10g2000prf.googlegroups.com>
On Feb 6, 2:53 pm, viper-2 <········@mail.infochan.com> wrote:
> On Feb 6, 11:33 am, billc <········@gmail.com> wrote:

> > If you're referring to the 3rd edition of the ANSII CL spec, then,
> > yes, that would make a fascinating topic!
>
> I'm surprised that Lisp3rd edition doesn't grab you ;-)
>
> But talking about the ANSII CL spec doesn't grab me.

OK, OK, I get it. Although we (the 4400)  were taken by the future,
and have no memory of what transpired during the abduction, some of us
do experience flashes of memory that become evident only when we make
assumptions about the present based on those recollections.

I had assumed, for instance, that the ANSII CL spec was beyond
revision 2. :-))

Obviously you'd be fascinated by the details of the  future ANSII CL
revision 3, but I just can't recall.

cheers
agt
From: billc
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <de7f8cc6-213e-48af-9be4-19e19e0820d4@j20g2000hsi.googlegroups.com>
On Feb 6, 12:37 pm, viper-2 <········@mail.infochan.com> wrote:
> On Feb 6, 2:53 pm, viper-2 <········@mail.infochan.com> wrote:
>
> > On Feb 6, 11:33 am, billc <········@gmail.com> wrote:
> > > If you're referring to the 3rd edition of the ANSII CL spec, then,
> > > yes, that would make a fascinating topic!
>
> > I'm surprised that Lisp3rd edition doesn't grab you ;-)
>
> > But talking about the ANSII CL spec doesn't grab me.
>
> OK, OK, I get it. Although we (the 4400)  were taken by the future,
> and have no memory of what transpired during the abduction, some of us
> do experience flashes of memory that become evident only when we make
> assumptions about the present based on those recollections.
>
> I had assumed, for instance, that the ANSII CL spec was beyond
> revision 2. :-))
>
> Obviously you'd be fascinated by the details of the  future ANSII CL
> revision 3, but I just can't recall.

There have been efforts to reverse the effect of the Promicin-
inhibitor; so, hopefully, details of the future ANSII CL spec will
come back to you. ;-)

- Bill
From: viper-2
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <127e2ea2-1647-4aa8-8c10-7f3e4d016447@q77g2000hsh.googlegroups.com>
On Feb 6, 4:05 pm, billc <········@gmail.com> wrote:

> There have been efforts to reverse the effect of the Promicin-
> inhibitor; so, hopefully, details of the future ANSII CL spec will
> come back to you. ;-)

No inhibitor for me, and that is not how the whole business works
(sigh). I'll explain when I get to LispVan.

agt
From: Dimiter "malkia" Stanev
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil    Things with CL
Date: 
Message-ID: <60sjpeF1s6k8dU2@mid.individual.net>
billc wrote:

> Topic: Doing Evil Things with CL
> Presenter: Brad Beveridge
> Date: Thursday, February 28, 2008
> Time: 7pm - 10pm (or whenever)
> Venue: Calhoun's, 3035 West Broadway, Vancouver
> Summary: Brad will be talking about various low-level bits and pieces,
> including:
> 
> - Loading code that is normally a Linux driver into a Lisp image
> - Creating CFFI bindings to foreign code
> - Writing Lisp code to load binary files using the binary-types
> library
> - Parsing Mach-O (OS X) executable files
> 
> Brad indicated that he probably doesn't have enough material or code
> to present for very long, so he would like to treat the presentation
> as a way to start discussion.
> 
> Join us for a beer/coffee and a chance to see/discuss how CL can be
> used for embedded systems work.
> 
> Additional links and any updates will be posted on my blog:
> http://bc.tech.coop/blog/080205.html
> 
> --
> Bill Clementson
> 

Sounds like fun - Any chance anyone doing this here - in LA, Santa Monica?

Bill, is anyone going to record in on video? The low-level things & 
pieces of Common-Lisp is what interrest me most :) at the moment.

Thanks,
Dimiter "malkia" Stanev.
From: billc
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <30a9caa1-6daf-4b41-a704-1fde47ecb52f@l16g2000hsh.googlegroups.com>
On Feb 5, 6:19 pm, "Dimiter \"malkia\" Stanev" <······@gmail.com>
wrote:
> billc wrote:
> > Topic: Doing Evil Things with CL
> > Presenter: Brad Beveridge
> > Date: Thursday, February 28, 2008
> > Time: 7pm - 10pm (or whenever)
> > Venue: Calhoun's, 3035 West Broadway, Vancouver
> > Summary: Brad will be talking about various low-level bits and pieces,
> > including:
>
> > - Loading code that is normally a Linux driver into a Lisp image
> > - Creating CFFI bindings to foreign code
> > - Writing Lisp code to load binary files using the binary-types
> > library
> > - Parsing Mach-O (OS X) executable files
>
> > Brad indicated that he probably doesn't have enough material or code
> > to present for very long, so he would like to treat the presentation
> > as a way to start discussion.
>
> > Join us for a beer/coffee and a chance to see/discuss how CL can be
> > used for embedded systems work.
>
> > Additional links and any updates will be posted on my blog:
> >http://bc.tech.coop/blog/080205.html
>
> > --
> > Bill Clementson
>
> Sounds like fun - Any chance anyone doing this here - in LA, Santa Monica?

I'm sure that Brad could be persuaded to fly down to give his
presentation in LA if someone was to present him with tickets and
accommodation.;-)

> Bill, is anyone going to record in on video? The low-level things &
> pieces of Common-Lisp is what interrest me most :) at the moment.

I usually try to record the meetings; however, sometimes it isn't
possible. Best to fly out and attend in person! ;-)

- Bill
From: billc
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <77f966ca-a184-425a-9e92-05714c6a277e@s12g2000prg.googlegroups.com>
On Feb 6, 8:38 am, billc <········@gmail.com> wrote:
> On Feb 5, 6:19 pm, "Dimiter \"malkia\" Stanev" <······@gmail.com>
> > Sounds like fun - Any chance anyone doing this here - in LA, Santa Monica?
>
> I'm sure that Brad could be persuaded to fly down to give his
> presentation in LA if someone was to present him with tickets and
> accommodation.;-)
>
> > Bill, is anyone going to record in on video? The low-level things &
> > pieces of Common-Lisp is what interrest me most :) at the moment.
>
> I usually try to record the meetings; however, sometimes it isn't
> possible. Best to fly out and attend in person! ;-)

A link to the presentation and video is on my blog:
http://bc.tech.coop/blog/080229.html

- Bill
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c857c2$0$29464$607ed4bc@cv.net>
billc wrote:
> On Feb 6, 8:38 am, billc <········@gmail.com> wrote:
> 
>>On Feb 5, 6:19 pm, "Dimiter \"malkia\" Stanev" <······@gmail.com>
>>
>>>Sounds like fun - Any chance anyone doing this here - in LA, Santa Monica?
>>
>>I'm sure that Brad could be persuaded to fly down to give his
>>presentation in LA if someone was to present him with tickets and
>>accommodation.;-)
>>
>>
>>>Bill, is anyone going to record in on video? The low-level things &
>>>pieces of Common-Lisp is what interrest me most :) at the moment.
>>
>>I usually try to record the meetings; however, sometimes it isn't
>>possible. Best to fly out and attend in person! ;-)
> 
> 
> A link to the presentation and video is on my blog:
> http://bc.tech.coop/blog/080229.html

Sounds like Brad re-invented Verrazano rather than dust it off and 
reinvigorate it. He is now a true Lisper! :)

And Lisp still does not have a binding generator. :(

Well, it started with an SoC, maybe SoC 2008 can finish it. Anyone know 
how CFFI-Grovel is faring (and why they did not start from Vzn either?)

<sigh>

There is so little hope for You People.

kenny

-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <e0956381-4622-4b95-8696-e45fa0fcf6ca@s12g2000prg.googlegroups.com>
>
> Sounds like Brad re-invented Verrazano rather than dust it off and
> reinvigorate it. He is now a true Lisper! :)
>
> And Lisp still does not have a binding generator. :(
>
> Well, it started with an SoC, maybe SoC 2008 can finish it. Anyone know
> how CFFI-Grovel is faring (and why they did not start from Vzn either?)
>
> <sigh>
>
> There is so little hope for You People.
Well, if when you say "re-invented Verranzano" you mean "invented a
binding generator with fundamentally different properties" then yes.
The way I generate bindings is very different from the way Verrazano
does.
I looked at VZN awhile ago, and when it was being worked on for SoC it
had hope, but if I remember correctly the author got stuck on C++
(which may happen to me too).

Verrazano is fundamentally a C/C++ parser, it just happens to use GCC-
XML to parse the C++ code & then generate bindings.  Systems like VZN
and SWIG suffer from a number of problems:
1) They usually have corner cases where the parsing is too hard, and
they give up
2) It is hard to get the compiler parser and the binding generator
parser looking at the same set of code.  #defines and makefiles
complicate this
3) From parse information alone you do not get implementation specific
details, such as object layout.

#3 is the killer.

My basic idea is to parse out the debug information from a finished
binary, which IMHO has a number of advantages.
1) Only one file to work with, so trivial to turn the key on
2) You get exact data layout for objects
3) You could generate bindings for any other language that happens to
generate debug information & has an ABI that is supported by your
Lisp.  The D language springs to mind.

I didn't just ignore Verranzo, I looked at it and found it lacking.
I'm hoping that another few months of part time work will get me to
the point where I can work usefully with C++ code.

Hope that helps
Brad
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c87fee$0$25046$607ed4bc@cv.net>
bradb wrote:
>>Sounds like Brad re-invented Verrazano rather than dust it off and
>>reinvigorate it. He is now a true Lisper! :)
>>
>>And Lisp still does not have a binding generator. :(
>>
>>Well, it started with an SoC, maybe SoC 2008 can finish it. Anyone know
>>how CFFI-Grovel is faring (and why they did not start from Vzn either?)
>>
>><sigh>
>>
>>There is so little hope for You People.
> 
> Well, if when you say "re-invented Verranzano" you mean "invented a
> binding generator with fundamentally different properties" then yes.

Awesome. I listened to your voice-over of the background music up to 
where I heard you say about Vzn only that it was not maintained.

Next time I will parse that to mean "cannot handle 
implementation-specific data structure layout".

> The way I generate bindings is very different from the way Verrazano
> does.
> I looked at VZN awhile ago, and when it was being worked on for SoC it
> had hope, but if I remember correctly the author got stuck on C++

Really? I thought he delivered FTGL for the Gnu compiler (which I heard 
also covers OS X sharables), I was whining at him to also do VC++ but 
that he never got round to.

> (which may happen to me too).
> 
> Verrazano is fundamentally a C/C++ parser, it just happens to use GCC-
> XML to parse the C++ code & then generate bindings.  Systems like VZN
> and SWIG suffer from a number of problems:
> 1) They usually have corner cases where the parsing is too hard, and
> they give up
> 2) It is hard to get the compiler parser and the binding generator
> parser looking at the same set of code.  #defines and makefiles
> complicate this
> 3) From parse information alone you do not get implementation specific
> details, such as object layout.
> 
> #3 is the killer.
> 
> My basic idea is to parse out the debug information from a finished
> binary, which IMHO has a number of advantages.
> 1) Only one file to work with, so trivial to turn the key on
> 2) You get exact data layout for objects
> 3) You could generate bindings for any other language that happens to
> generate debug information & has an ABI that is supported by your
> Lisp.  The D language springs to mind.

When I did my C->Lisp converter using Antlr... wow, it has been a while, 
but I think that also works off gcc-xml. anyway... it occurred to me 
that a peek at the source would give away when a char* was expected to 
be a null-terminated C string, automating one more aspect of binding 
generation. Maybe both approaches could be applied.

> 
> I didn't just ignore Verranzo, I looked at it and found it lacking.

OK, so maybe next time you are doing Lisp karaoke you should sing "Vzn 
is unmaintained *and* it sucks".

Just checked the slides, all I see is "no maintained Lisp generators I 
know of".

Not sure why I am getting yelled at for your buggy talk. :)

> I'm hoping that another few months of part time work will get me to
> the point where I can work usefully with C++ code.

Why not let Google pay someone to do it? Summer of Code 2008 is on the 
way (proposals from students due in three weeks) and LispNYC is 
soliciting proposals now for prospective candidates to consider (search 
half-way down for a link to a form you can use):

   http://www.lispnyc.org/home.clp

You would be the mentor, and if you have a student in mind you cannot 
designate them to Google but it certainly would give them a boost to 
have your backing.

kenny


-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <722ff4d1-246f-47ef-8c41-7fef77ce8562@i7g2000prf.googlegroups.com>
On Feb 29, 1:58 pm, Ken Tilton <···········@optonline.net> wrote:
> bradb wrote:
> >>Sounds like Brad re-invented Verrazano rather than dust it off and
> >>reinvigorate it. He is now a true Lisper! :)
>
> >>And Lisp still does not have a binding generator. :(
>
> >>Well, it started with an SoC, maybe SoC 2008 can finish it. Anyone know
> >>how CFFI-Grovel is faring (and why they did not start from Vzn either?)
>
> >><sigh>
>
> >>There is so little hope for You People.
>
> > Well, if when you say "re-invented Verranzano" you mean "invented a
> > binding generator with fundamentally different properties" then yes.
>
> Awesome. I listened to your voice-over of the background music up to
> where I heard you say about Vzn only that it was not maintained.
>
> Next time I will parse that to mean "cannot handle
> implementation-specific data structure layout".
Oh Kenny, you're breaking my heart - please listen to the whole thing.
Also, what was wrong with saying Verranzano is not maintained?  Who is
the maintainer, and when did he last post (hint, last devel post was
December last year).  When was the last code commit?  When was the
last time Verranzano did anything for me?  And have you seen the list
of packages it depends on?  Eight!  I depend on BINARY-TYPES and
ASDF.  My code is 500 lines long, and I'd wager that it does basically
all that Verranzano does.

Sigh, I'm arguing with people on the internet - Kenny you're too damn
good at trolling.  And I don't even think you're trying that hard.


>
> > The way I generate bindings is very different from the way Verrazano
> > does.
> > I looked at VZN awhile ago, and when it was being worked on for SoC it
> > had hope, but if I remember correctly the author got stuck on C++
>
> Really? I thought he delivered FTGL for the Gnu compiler (which I heard
> also covers OS X sharables), I was whining at him to also do VC++ but
> that he never got round to.

Sure, and the symbols look like _ZOUfoo_eet.  Which is worthless.  But
lets just agree that my system is a better idea, and leave Verrazano
in the past shall we?  I promise that I'll make my stuff an open
project in the future - unless I can make a buck off it & sell it to
somebody ;)  I know you'll appreciate that sentiment, damn open source
hippies.

> > (which may happen to me too).
>
> > Verrazano is fundamentally a C/C++ parser, it just happens to use GCC-
> > XML to parse the C++ code & then generate bindings.  Systems like VZN
> > and SWIG suffer from a number of problems:
> > 1) They usually have corner cases where the parsing is too hard, and
> > they give up
> > 2) It is hard to get the compiler parser and the binding generator
> > parser looking at the same set of code.  #defines and makefiles
> > complicate this
> > 3) From parse information alone you do not get implementation specific
> > details, such as object layout.
>
> > #3 is the killer.
>
> > My basic idea is to parse out the debug information from a finished
> > binary, which IMHO has a number of advantages.
> > 1) Only one file to work with, so trivial to turn the key on
> > 2) You get exact data layout for objects
> > 3) You could generate bindings for any other language that happens to
> > generate debug information & has an ABI that is supported by your
> > Lisp.  The D language springs to mind.
>
> When I did my C->Lisp converter using Antlr... wow, it has been a while,
> but I think that also works off gcc-xml. anyway... it occurred to me
> that a peek at the source would give away when a char* was expected to
> be a null-terminated C string, automating one more aspect of binding
> generation. Maybe both approaches could be applied.

Right now I treat all char*'s as strings.  That's probably a mistake,
but neither system solves this problem.

>
>
>
> > I didn't just ignore Verranzo, I looked at it and found it lacking.
>
> OK, so maybe next time you are doing Lisp karaoke you should sing "Vzn
> is unmaintained *and* it sucks".
>
> Just checked the slides, all I see is "no maintained Lisp generators I
> know of".

I don't sing, but if I did, I'd sing: "no recent posts; no recent
code; no active maintainer - I've got the unmaintained package
blues".  And I'd dedicate the song to Verrazano.

>
> Not sure why I am getting yelled at for your buggy talk. :)
I'm not yelling, and I accept that my talk probably has valid bugs.

> > I'm hoping that another few months of part time work will get me to
> > the point where I can work usefully with C++ code.
>
> Why not let Google pay someone to do it? Summer of Code 2008 is on the
> way (proposals from students due in three weeks) and LispNYC is
> soliciting proposals now for prospective candidates to consider (search
> half-way down for a link to a form you can use):
>
>    http://www.lispnyc.org/home.clp
>
> You would be the mentor, and if you have a student in mind you cannot
> designate them to Google but it certainly would give them a boost to
> have your backing.

>
Too good; you spend two posts trolling at me & now you're asking me to
do you a favour?  :)  In all seriousness, I don't have the time or
inclination to mentor anybody for SoC - guess I'm not a good enough
open source fairy.
My idea is fairly simple though, and a month's worth of part-time (<
10 hours/week) work has me a long way there.  Anybody else could do
the same.

Cheers
Brad

PS - Kenny, I suspect in real life you & I would get on like the
proverbial house on fire.  On the internet it's a bit harder to read
tone - this post should be consumed with a filter of light-hearted
tone.
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c89d65$0$5610$607ed4bc@cv.net>
bradb wrote:
> On Feb 29, 1:58 pm, Ken Tilton <···········@optonline.net> wrote:
> 
>>bradb wrote:
>>
>>>>Sounds like Brad re-invented Verrazano rather than dust it off and
>>>>reinvigorate it. He is now a true Lisper! :)
>>
>>>>And Lisp still does not have a binding generator. :(
>>
>>>>Well, it started with an SoC, maybe SoC 2008 can finish it. Anyone know
>>>>how CFFI-Grovel is faring (and why they did not start from Vzn either?)
>>
>>>><sigh>
>>
>>>>There is so little hope for You People.
>>
>>>Well, if when you say "re-invented Verranzano" you mean "invented a
>>>binding generator with fundamentally different properties" then yes.
>>
>>Awesome. I listened to your voice-over of the background music up to
>>where I heard you say about Vzn only that it was not maintained.
>>
>>Next time I will parse that to mean "cannot handle
>>implementation-specific data structure layout".
> 
> Oh Kenny, you're breaking my heart - please listen to the whole thing.

Oh, you came back to Verrazano? It was a long tape and I have trolling 
to do, I need the Cliffs notes.

> Also, what was wrong with saying Verranzano is not maintained?

<sigh> Where did I say it was maintained? You Brits are always calling 
us Americans dense, looks like reading ocmprehension is no better across 
the pond. All I said was that that was all you said.

>  Who is
> the maintainer, and when did he last post (hint, last devel post was
> December last year).  When was the last code commit?  When was the
> last time Verranzano did anything for me?  And have you seen the list
> of packages it depends on?  Eight!  I depend on BINARY-TYPES and
> ASDF.  My code is 500 lines long, and I'd wager that it does basically
> all that Verranzano does.

Nice.

> 
> Sigh, I'm arguing with people on the internet - Kenny you're too damn
> good at trolling.  And I don't even think you're trying that hard.

Right, I was commending your work and simply pointing out that you 
needn't be yelling at me since I was going by your information.

>>>The way I generate bindings is very different from the way Verrazano
>>>does.
>>>I looked at VZN awhile ago, and when it was being worked on for SoC it
>>>had hope, but if I remember correctly the author got stuck on C++
>>
>>Really? I thought he delivered FTGL for the Gnu compiler (which I heard
>>also covers OS X sharables), I was whining at him to also do VC++ but
>>that he never got round to.
> 
> 
> Sure, and the symbols look like _ZOUfoo_eet.  Which is worthless.  But
> lets just agree that my system is a better idea, and leave Verrazano
> in the past shall we?  I promise that I'll make my stuff an open
> project in the future - unless I can make a buck off it & sell it to
> somebody ;)  I know you'll appreciate that sentiment, damn open source
> hippies.

OK, so your unavailable currently unopen version is better? Right, well 
then look at this perpetual motion machine I have here, does not even 
need the cold fusion device or room-temp superconductors I have.

> Right now I treat all char*'s as strings.  That's probably a mistake,
> but neither system solves this problem.

You need to look around in the Antlr/GCC-XML/I-forget-what for pointers 
being passed to str*.

> I don't sing, but if I did, I'd sing: "no recent posts; no recent
> code; no active maintainer - I've got the unmaintained package
> blues".  And I'd dedicate the song to Verrazano.

Brad: "That dog should be destroyed because it is not on a leash."
Kenny: "You want to kill the dog because it's owner screwed up?"
Brad: "That dog has killed three people so far!"
Kenny: "Oh. You only said it was not on a leash."
Brad: "Hunh? What leash? I don't see a leash!"

>>Why not let Google pay someone to do it? Summer of Code 2008 is on the
>>way (proposals from students due in three weeks) and LispNYC is
>>soliciting proposals now for prospective candidates to consider (search
>>half-way down for a link to a form you can use):
>>
>>   http://www.lispnyc.org/home.clp
>>
>>You would be the mentor, and if you have a student in mind you cannot
>>designate them to Google but it certainly would give them a boost to
>>have your backing.
> 
> 
> Too good; you spend two posts trolling at me & now you're asking me to
> do you a favour?  :)  In all seriousness, I don't have the time or
> inclination to mentor anybody for SoC - guess I'm not a good enough
> open source fairy.

Oh, please. Mentoring is no burden, you make them send you weekly 
progress reports which you do not read and when they say they are done 
you install it and see if it works.

Go here: http://www.lispnyc.org/home.clp

By the way, congratulations for taking Lisp community uselessness to a 
new height at which you cannot be bothered to let someone else pay 
someone else to do the work. Nice!

> PS - Kenny, I suspect in real life you & I would get on like the
> proverbial house on fire.

Oh. I thought we were getting along great. You might want to compare 
notes with anyone named Pascal to find out what constitutes not getting 
along with His Kennyness.

Himself

-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <3233ac6f-f852-425e-955f-b307708db6bc@i12g2000prf.googlegroups.com>
On Feb 29, 4:03 pm, Ken Tilton <···········@optonline.net> wrote:
> bradb wrote:
> > On Feb 29, 1:58 pm, Ken Tilton <···········@optonline.net> wrote:
>
> >>bradb wrote:
>
> >>>>Sounds like Brad re-invented Verrazano rather than dust it off and
> >>>>reinvigorate it. He is now a true Lisper! :)
>
> >>>>And Lisp still does not have a binding generator. :(
>
> >>>>Well, it started with an SoC, maybe SoC 2008 can finish it. Anyone know
> >>>>how CFFI-Grovel is faring (and why they did not start from Vzn either?)
>
> >>>><sigh>
>
> >>>>There is so little hope for You People.
>
> >>>Well, if when you say "re-invented Verranzano" you mean "invented a
> >>>binding generator with fundamentally different properties" then yes.
>
> >>Awesome. I listened to your voice-over of the background music up to
> >>where I heard you say about Vzn only that it was not maintained.
>
> >>Next time I will parse that to mean "cannot handle
> >>implementation-specific data structure layout".
>
> > Oh Kenny, you're breaking my heart - please listen to the whole thing.
>
> Oh, you came back to Verrazano? It was a long tape and I have trolling
> to do, I need the Cliffs notes.
The Cliffs notes is: I'm doing something interesting & you'll have to
wait for a bit until I make it public, so there! :)

>
> > Also, what was wrong with saying Verranzano is not maintained?
>
> <sigh> Where did I say it was maintained? You Brits are always calling
> us Americans dense, looks like reading ocmprehension is no better across
> the pond. All I said was that that was all you said.
Ha, you're better at this that me - anybody ever tell you that you
should have been a lawyer?  I assumed that you'd stopped listening
because of my horribly inaccurate statement that Vzn was abandonware.
I'm a Kiwi, BTW.

>
> >  Who is
> > the maintainer, and when did he last post (hint, last devel post was
> > December last year).  When was the last code commit?  When was the
> > last time Verranzano did anything for me?  And have you seen the list
> > of packages it depends on?  Eight!  I depend on BINARY-TYPES and
> > ASDF.  My code is 500 lines long, and I'd wager that it does basically
> > all that Verranzano does.
>
> Nice.
>
>
>
> > Sigh, I'm arguing with people on the internet - Kenny you're too damn
> > good at trolling.  And I don't even think you're trying that hard.
>
> Right, I was commending your work and simply pointing out that you
> needn't be yelling at me since I was going by your information.
>
>
>
> >>>The way I generate bindings is very different from the way Verrazano
> >>>does.
> >>>I looked at VZN awhile ago, and when it was being worked on for SoC it
> >>>had hope, but if I remember correctly the author got stuck on C++
>
> >>Really? I thought he delivered FTGL for the Gnu compiler (which I heard
> >>also covers OS X sharables), I was whining at him to also do VC++ but
> >>that he never got round to.
>
> > Sure, and the symbols look like _ZOUfoo_eet.  Which is worthless.  But
> > lets just agree that my system is a better idea, and leave Verrazano
> > in the past shall we?  I promise that I'll make my stuff an open
> > project in the future - unless I can make a buck off it & sell it to
> > somebody ;)  I know you'll appreciate that sentiment, damn open source
> > hippies.
>
> OK, so your unavailable currently unopen version is better? Right, well
> then look at this perpetual motion machine I have here, does not even
> need the cold fusion device or room-temp superconductors I have.
I'll swap you, my code for your cold fusion device.  Always wanted to
be an evil genius.

>
> > Right now I treat all char*'s as strings.  That's probably a mistake,
> > but neither system solves this problem.
>
> You need to look around in the Antlr/GCC-XML/I-forget-what for pointers
> being passed to str*.
<serious mode>
Can that really work?  I guess you can heuristically trace what data
winds up going to standard string functions, but that sounds hokey at
best.  I really don't see how you can extract the semantics from C
code that a char* is really a NUL terminated string.  I think I'm
going to have to have tuning code during the parse where you can
specify "always treat char* and const char* as string", "never treat
as string", "for these functions do/don't expect a string"
Either that or write Lisp code that emulates C's broken concept of a
string (shudder).

>
> > I don't sing, but if I did, I'd sing: "no recent posts; no recent
> > code; no active maintainer - I've got the unmaintained package
> > blues".  And I'd dedicate the song to Verrazano.
>
> Brad: "That dog should be destroyed because it is not on a leash."
> Kenny: "You want to kill the dog because it's owner screwed up?"
> Brad: "That dog has killed three people so far!"
> Kenny: "Oh. You only said it was not on a leash."
> Brad: "Hunh? What leash? I don't see a leash!"
>
Yup, I lose at the internet.

>
>
> >>Why not let Google pay someone to do it? Summer of Code 2008 is on the
> >>way (proposals from students due in three weeks) and LispNYC is
> >>soliciting proposals now for prospective candidates to consider (search
> >>half-way down for a link to a form you can use):
>
> >>  http://www.lispnyc.org/home.clp
>
> >>You would be the mentor, and if you have a student in mind you cannot
> >>designate them to Google but it certainly would give them a boost to
> >>have your backing.
>
> > Too good; you spend two posts trolling at me & now you're asking me to
> > do you a favour?  :)  In all seriousness, I don't have the time or
> > inclination to mentor anybody for SoC - guess I'm not a good enough
> > open source fairy.
>
> Oh, please. Mentoring is no burden, you make them send you weekly
> progress reports which you do not read and when they say they are done
> you install it and see if it works.
>
> Go here:http://www.lispnyc.org/home.clp
>
> By the way, congratulations for taking Lisp community uselessness to a
> new height at which you cannot be bothered to let someone else pay
> someone else to do the work. Nice!
Always aiming to please.  Still not volunteering.

Brad

>
> > PS - Kenny, I suspect in real life you & I would get on like the
> > proverbial house on fire.
>
> Oh. I thought we were getting along great. You might want to compare
> notes with anyone named Pascal to find out what constitutes not getting
> along with His Kennyness.
>
> Himself
>
> --http://smuglispweeny.blogspot.com/http://www.theoryyalgebra.com/
>
> "In the morning, hear the Way;
>   in the evening, die content!"
>                      -- Confucius
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c8a834$0$25045$607ed4bc@cv.net>
bradb wrote:
> I'm a Kiwi, BTW.

They gave up New Zealand?! Well, they still have India...

kenny


-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c8b327$0$5609$607ed4bc@cv.net>
bradb wrote:
>>Oh, please. Mentoring is no burden, you make them send you weekly
>>progress reports which you do not read and when they say they are done
>>you install it and see if it works.
>>
>>Go here:http://www.lispnyc.org/home.clp
>>
>>By the way, congratulations for taking Lisp community uselessness to a
>>new height at which you cannot be bothered to let someone else pay
>>someone else to do the work. Nice!
> 
> Always aiming to please.  Still not volunteering.

<pulling>Is the code available?</teeth>

CFFI was originated by James Bielman, extended to its current state by 
Luis Oliveira. Working with JB, tho.

kenny

-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <fc857e65-e21f-4302-8ccf-4718bd06f558@i12g2000prf.googlegroups.com>
On Feb 29, 5:36 pm, Ken Tilton <···········@optonline.net> wrote:
> bradb wrote:
> >>Oh, please. Mentoring is no burden, you make them send you weekly
> >>progress reports which you do not read and when they say they are done
> >>you install it and see if it works.
>
> >>Go here:http://www.lispnyc.org/home.clp
>
> >>By the way, congratulations for taking Lisp community uselessness to a
> >>new height at which you cannot be bothered to let someone else pay
> >>someone else to do the work. Nice!
>
> > Always aiming to please.  Still not volunteering.
>
> <pulling>Is the code available?</teeth>
No.  But if you want it, flick me an email.  At some unspecified
future time I'll make a common-lisp.net project.

> CFFI was originated by James Bielman, extended to its current state by
> Luis Oliveira. Working with JB, tho.

I like CFFI very much.  Thanks to the authors, etc.
If you get somebody with a serious affliction for wanting to extend my
stuff (possibly for SoC), they can have my code & I'll happily work in
that branch and help out on a no-responsibility basis.  But I'm not
going to be the cheerleader yet - hell, my stuff doesn't even have a
True Name yet :)

Brad
From: Pascal Bourguignon
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <87ejau6696.fsf@thalassa.informatimago.com>
bradb <··············@gmail.com> writes:

> Right now I treat all char*'s as strings.  That's probably a mistake,
> but neither system solves this problem.

We could use some kind of type inference to determine whether a
pointer points to a single element or a vector (and whether the vector
is "null" terminated or not).  

void f(void* p){
    char* s=malloc(strlen(p));
    strcpy(s,p);
    /* ... */
}

==> p must be a pointer to a null terminated vector of char.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

READ THIS BEFORE OPENING PACKAGE: According to certain suggested
versions of the Grand Unified Theory, the primary particles
constituting this product may decay to nothingness within the next
four hundred million years.
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <738c0b55-c1be-44cb-9a7e-3a069255ad83@i12g2000prf.googlegroups.com>
On Mar 1, 3:19 am, Pascal Bourguignon <····@informatimago.com> wrote:
> bradb <··············@gmail.com> writes:
> > Right now I treat all char*'s as strings.  That's probably a mistake,
> > but neither system solves this problem.
>
> We could use some kind of type inference to determine whether a
> pointer points to a single element or a vector (and whether the vector
> is "null" terminated or not).
>
> void f(void* p){
>     char* s=malloc(strlen(p));
>     strcpy(s,p);
>     /* ... */
>
> }
>
> ==> p must be a pointer to a null terminated vector of char.
Right, but that involves actual code analysis.  That kind of system
would probably work somewhat well for binding generators that parse
code.  But I'm still doubtful - what if I've wrapped all standard C
functions with my own?
I think there's no real solution that's not mostly manual.  One simple
solution would be for the C code to have actually typedefed char* as
"string", and then to use that new type instead of char*.  The Lisp
side can then distinguish based on type.
I suspect that the solution I'll probably work with is this:
 - treat (const) char* is string if that is their exact type
 - any other types that boil down to that (typedef s8 char) as non-
strings
 - allow the user to give hints about what the semantics really are,
such as
(treat-as-string '(u8))
(treat-all-char*-as-string t)
(functions-use-strings "^str.*")
(function-arg-list "main" '(t nil))

Thoughts?

Cheers,
Brad
From: D Herring
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <kKmdnbopCe4eAFTanZ2dnUVZ_sGvnZ2d@comcast.com>
bradb wrote:
> On Mar 1, 3:19 am, Pascal Bourguignon <····@informatimago.com> wrote:
>> bradb <··············@gmail.com> writes:
>>> Right now I treat all char*'s as strings.  That's probably a mistake,
>>> but neither system solves this problem.
>> We could use some kind of type inference to determine whether a
>> pointer points to a single element or a vector (and whether the vector
>> is "null" terminated or not).
>>
>> void f(void* p){
>>     char* s=malloc(strlen(p));
>>     strcpy(s,p);
>>     /* ... */
>>
>> }
>>
>> ==> p must be a pointer to a null terminated vector of char.
> Right, but that involves actual code analysis.  That kind of system
> would probably work somewhat well for binding generators that parse
> code.  But I'm still doubtful - what if I've wrapped all standard C
> functions with my own?
> I think there's no real solution that's not mostly manual.  One simple
> solution would be for the C code to have actually typedefed char* as
> "string", and then to use that new type instead of char*.  The Lisp
> side can then distinguish based on type.
> I suspect that the solution I'll probably work with is this:
>  - treat (const) char* is string if that is their exact type
>  - any other types that boil down to that (typedef s8 char) as non-
> strings
>  - allow the user to give hints about what the semantics really are,
> such as
> (treat-as-string '(u8))
> (treat-all-char*-as-string t)
> (functions-use-strings "^str.*")
> (function-arg-list "main" '(t nil))
> 
> Thoughts?

The "char * =?= string" issue is an ambiguity even in the C world, 
there it is handled socially though API documentation...  There are 
other ambiguities arising from a conversion -- how to change 
names/handle C++ name mangling, which functions to make available, 
whether to resolve typedefs to primitive types, (in C++) how to 
compile new libs that instantiate inline and template functions, how 
to map class functions and namespaces (e.g. use generic functions? use 
long package names?), enum conversions...

At the end of the day, people don't want a raw C to Lisp binding; they 
want a usable, idiomatic one.  Thus, I think there's a need to 
identify the common binding choices and define an API that guides the 
conversion process.  A DSL that says "here I would make these 
decisions for those ambiguities".  The code generator would then zip 
your DSL with the debug info to create the binding.

- Daniel

BTW, I wrote some (often crude) code that uses libelf+libdwarf to 
create Lisp trees that capture all the debug information (the dwarf 
format contains much better information than stabs).  If you're 
interested, I could push it to repo.or.cz...

P.S.  The biggest hit against Verrazano is its use of gccxml.  XML 
verbosity rivals that of debug information, and gccxml only works with 
gcc...
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <bbc27884-a87c-44ba-abf6-5a653379848c@i29g2000prf.googlegroups.com>
On Mar 1, 10:16 am, D Herring <········@at.tentpost.dot.com> wrote:
> bradb wrote:
> > On Mar 1, 3:19 am, Pascal Bourguignon <····@informatimago.com> wrote:
> >> bradb <··············@gmail.com> writes:
> >>> Right now I treat all char*'s as strings.  That's probably a mistake,
> >>> but neither system solves this problem.
> >> We could use some kind of type inference to determine whether a
> >> pointer points to a single element or a vector (and whether the vector
> >> is "null" terminated or not).
>
> >> void f(void* p){
> >>     char* s=malloc(strlen(p));
> >>     strcpy(s,p);
> >>     /* ... */
>
> >> }
>
> >> ==> p must be a pointer to a null terminated vector of char.
> > Right, but that involves actual code analysis.  That kind of system
> > would probably work somewhat well for binding generators that parse
> > code.  But I'm still doubtful - what if I've wrapped all standard C
> > functions with my own?
> > I think there's no real solution that's not mostly manual.  One simple
> > solution would be for the C code to have actually typedefed char* as
> > "string", and then to use that new type instead of char*.  The Lisp
> > side can then distinguish based on type.
> > I suspect that the solution I'll probably work with is this:
> >  - treat (const) char* is string if that is their exact type
> >  - any other types that boil down to that (typedef s8 char) as non-
> > strings
> >  - allow the user to give hints about what the semantics really are,
> > such as
> > (treat-as-string '(u8))
> > (treat-all-char*-as-string t)
> > (functions-use-strings "^str.*")
> > (function-arg-list "main" '(t nil))
>
> > Thoughts?
>
> The "char * =?= string" issue is an ambiguity even in the C world,
> there it is handled socially though API documentation...  There are
> other ambiguities arising from a conversion -- how to change
> names/handle C++ name mangling, which functions to make available,
> whether to resolve typedefs to primitive types, (in C++) how to
> compile new libs that instantiate inline and template functions, how
> to map class functions and namespaces (e.g. use generic functions? use
> long package names?), enum conversions...
>
> At the end of the day, people don't want a raw C to Lisp binding; they
> want a usable, idiomatic one.  Thus, I think there's a need to
> identify the common binding choices and define an API that guides the
> conversion process.  A DSL that says "here I would make these
> decisions for those ambiguities".  The code generator would then zip
> your DSL with the debug info to create the binding.
>
> - Daniel
>
> BTW, I wrote some (often crude) code that uses libelf+libdwarf to
> create Lisp trees that capture all the debug information (the dwarf
> format contains much better information than stabs).  If you're
> interested, I could push it to repo.or.cz...
>
> P.S.  The biggest hit against Verrazano is its use of gccxml.  XML
> verbosity rivals that of debug information, and gccxml only works with
> gcc...
Dwarf is on my todo list and what was I was originally targeting - but
I couldn't convince Apple's GCC to output dwarf.  I think I'd
personally prefer not to use libdwarf.  I don't expect it is too hard
to parse dwarf info, and having everything done in Lisp reduces
dependencies.

Cheers,
Brad
From: D Herring
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <8N2dneKT_5YId1TanZ2dnUVZ_tCrnZ2d@comcast.com>
bradb wrote:
> On Mar 1, 10:16 am, D Herring <········@at.tentpost.dot.com> wrote:
>> BTW, I wrote some (often crude) code that uses libelf+libdwarf to
>> create Lisp trees that capture all the debug information (the dwarf
>> format contains much better information than stabs).  If you're
>> interested, I could push it to repo.or.cz...
>>
> Dwarf is on my todo list and what was I was originally targeting - but
> I couldn't convince Apple's GCC to output dwarf.  I think I'd
> personally prefer not to use libdwarf.  I don't expect it is too hard
> to parse dwarf info, and having everything done in Lisp reduces
> dependencies.

Fair enough.  I found it to be difficult enough that libdwarf was 
worthwhile; but you may be better with "mostly standard" binary 
formats.  libdwarf does bring its own problems.

I've heard that xcode doesn't support dwarf on gcc 3, only 4.

- Daniel
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c9b2b7$0$5618$607ed4bc@cv.net>
bradb wrote:
> On Mar 1, 3:19 am, Pascal Bourguignon <····@informatimago.com> wrote:
> 
>>bradb <··············@gmail.com> writes:
>>
>>>Right now I treat all char*'s as strings.  That's probably a mistake,
>>>but neither system solves this problem.
>>
>>We could use some kind of type inference to determine whether a
>>pointer points to a single element or a vector (and whether the vector
>>is "null" terminated or not).
>>
>>void f(void* p){
>>    char* s=malloc(strlen(p));
>>    strcpy(s,p);
>>    /* ... */
>>
>>}
>>
>>==> p must be a pointer to a null terminated vector of char.
> 
> Right, but that involves actual code analysis.  That kind of system
> would probably work somewhat well for binding generators that parse
> code.  But I'm still doubtful - what if I've wrapped all standard C
> functions with my own?

Verrazano built a graph as it went. If one gets to a call to a str* 
function, one propagates the news back up the graph. Mind you this was 
just a random thought as I was eyeballing Antlr output, I did not think 
very hard about how much practical help it would be. But it is one 
aggravating thing about talking to C.

kenny

-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: Pascal Bourguignon
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <87zlti46mw.fsf@thalassa.informatimago.com>
bradb <··············@gmail.com> writes:

> On Mar 1, 3:19 am, Pascal Bourguignon <····@informatimago.com> wrote:
>> bradb <··············@gmail.com> writes:
>> > Right now I treat all char*'s as strings.  That's probably a mistake,
>> > but neither system solves this problem.
>>
>> We could use some kind of type inference to determine whether a
>> pointer points to a single element or a vector (and whether the vector
>> is "null" terminated or not).
>>
>> void f(void* p){
>>     char* s=malloc(strlen(p));
>>     strcpy(s,p);
>>     /* ... */
>>
>> }
>>
>> ==> p must be a pointer to a null terminated vector of char.
> Right, but that involves actual code analysis.  That kind of system
> would probably work somewhat well for binding generators that parse
> code.  But I'm still doubtful - what if I've wrapped all standard C
> functions with my own?

Of course, usual functions can be analyzed to infer their signatures,
even 'wrappers'.  Once we've determined that the above f is
(declaim (ftype (string) (values)) f)
we can determine the type of g:
char* g(void* p){f(p);return(p);}
to be
(declaim (ftype (string) string) g)
and not 
(declaim (ftype ((pointer char)) (pointer char)) g)

Of course, if what we have is:

void* h(void* p){
  if(0==random(2)){
     static char a='a';
     return(&a);
  }else{
     f(p);
     return(p);
  }
}

then we can only infer:

(declaim (ftype ((or string (pointer char))) (or string (pointer char))) h)

but this is rather nasty to write such a function in C...


> I think there's no real solution that's not mostly manual.  One simple
> solution would be for the C code to have actually typedefed char* as
> "string", and then to use that new type instead of char*.  The Lisp
> side can then distinguish based on type.
> I suspect that the solution I'll probably work with is this:
>  - treat (const) char* is string if that is their exact type
>  - any other types that boil down to that (typedef s8 char) as non-
> strings
>  - allow the user to give hints about what the semantics really are,
> such as
> (treat-as-string '(u8))
> (treat-all-char*-as-string t)
> (functions-use-strings "^str.*")
> (function-arg-list "main" '(t nil))
>
> Thoughts?

For the C compiler, typedef char* string; or typedef char*
pointer_to_one_char; are identical.  What you're saying is that you
need a C program with type declarations. (Like in CL, we'd want to
work on sources with (declare (type ...)) declarations).


It's probably easier to add separate 'hints' than to add typedefs and
edit the sources.  And  are these typedefed names kept in the
debugging infos?  I wouldn't be surprized if at the debugging level
only char* was left.  This may depend on the source language too.


80% of the time, assuming char* = string should be good enough.  If
the debugging info don't say if char* points to a character or a
string (or a vector of chars), then with additionnal type inference
you reach 96% of the case.

For the 4% left, let the user give us his hints.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

The world will now reboot.  don't bother saving your artefacts.
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <2c8e96ff-c6e8-4f44-9bc6-a543021ce5a1@e23g2000prf.googlegroups.com>
On Mar 1, 10:54 am, Pascal Bourguignon <····@informatimago.com> wrote:
> bradb <··············@gmail.com> writes:
> > On Mar 1, 3:19 am, Pascal Bourguignon <····@informatimago.com> wrote:
> >> bradb <··············@gmail.com> writes:
> >> > Right now I treat all char*'s as strings.  That's probably a mistake,
> >> > but neither system solves this problem.
>
> >> We could use some kind of type inference to determine whether a
> >> pointer points to a single element or a vector (and whether the vector
> >> is "null" terminated or not).
>
> >> void f(void* p){
> >>     char* s=malloc(strlen(p));
> >>     strcpy(s,p);
> >>     /* ... */
>
> >> }
>
> >> ==> p must be a pointer to a null terminated vector of char.
> > Right, but that involves actual code analysis.  That kind of system
> > would probably work somewhat well for binding generators that parse
> > code.  But I'm still doubtful - what if I've wrapped all standard C
> > functions with my own?
>
> Of course, usual functions can be analyzed to infer their signatures,
> even 'wrappers'.  Once we've determined that the above f is
> (declaim (ftype (string) (values)) f)
> we can determine the type of g:
> char* g(void* p){f(p);return(p);}
> to be
> (declaim (ftype (string) string) g)
> and not
> (declaim (ftype ((pointer char)) (pointer char)) g)
>
> Of course, if what we have is:
>
> void* h(void* p){
>   if(0==random(2)){
>      static char a='a';
>      return(&a);
>   }else{
>      f(p);
>      return(p);
>   }
>
> }
>
> then we can only infer:
>
> (declaim (ftype ((or string (pointer char))) (or string (pointer char))) h)
>
> but this is rather nasty to write such a function in C...
So from a base set of known functions you can back propagate the known
information to the functions that call those known string signatures.
Which means you need to do call analysis.  Hmm, actually call analysis
might not be as hard as I thought.  You could scan the binary code of
the function body looking for branch statements and record where the
branches go, and there is your call graph.  Future work perhaps :)

>
>
>
> > I think there's no real solution that's not mostly manual.  One simple
> > solution would be for the C code to have actually typedefed char* as
> > "string", and then to use that new type instead of char*.  The Lisp
> > side can then distinguish based on type.
> > I suspect that the solution I'll probably work with is this:
> >  - treat (const) char* is string if that is their exact type
> >  - any other types that boil down to that (typedef s8 char) as non-
> > strings
> >  - allow the user to give hints about what the semantics really are,
> > such as
> > (treat-as-string '(u8))
> > (treat-all-char*-as-string t)
> > (functions-use-strings "^str.*")
> > (function-arg-list "main" '(t nil))
>
> > Thoughts?
>
> For the C compiler, typedef char* string; or typedef char*
> pointer_to_one_char; are identical.  What you're saying is that you
> need a C program with type declarations. (Like in CL, we'd want to
> work on sources with (declare (type ...)) declarations).
>
> It's probably easier to add separate 'hints' than to add typedefs and
> edit the sources.  And  are these typedefed names kept in the
> debugging infos?  I wouldn't be surprized if at the debugging level
> only char* was left.  This may depend on the source language too.
The debug stabs I've been working with give type information for every
typedef where that type is actually used.  So if the C app happens to
follow a convention of using 'typedef char* cstring', then the
following CFFI types will be generated.
(defctype char* :string)
(defctype cstring char*)

Cheers
Brad

>
> 80% of the time, assuming char* = string should be good enough.  If
> the debugging info don't say if char* points to a character or a
> string (or a vector of chars), then with additionnal type inference
> you reach 96% of the case.
>
> For the 4% left, let the user give us his hints.
>
> --
> __Pascal Bourguignon__                    http://www.informatimago.com/
>
> The world will now reboot.  don't bother saving your artefacts.
From: Ken Tilton
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <47c9b356$0$5633$607ed4bc@cv.net>
bradb wrote:
> So from a base set of known functions you can back propagate the known
> information to the functions that call those known string signatures.
> Which means you need to do call analysis.  Hmm, actually call analysis
> might not be as hard as I thought.  You could scan the binary code of
> the function body looking for branch statements and record where the
> branches go, and there is your call graph.  Future work perhaps :)

oops, i shoulda read all the messages before posting.

kenny


-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: Barry Fishman
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <m34pbp59k9.fsf@barry_fishman.acm.org>
Pascal Bourguignon <···@informatimago.com> writes:
> We could use some kind of type inference to determine whether a
> pointer points to a single element or a vector (and whether the vector
> is "null" terminated or not).  
>
> void f(void* p){
>     char* s=malloc(strlen(p));
>     strcpy(s,p);
>     /* ... */
> }
>
> ==> p must be a pointer to a null terminated vector of char.

Ouch!  One can do all the type analysis you like, but simple human
inspection shows that this is a function that should fixed before
calling under any circumstance.

For valid code, I think it is worthwhile hand generating a s-expression
based protocol descriptions rather than relying on what can be guessed
from the C header file, or even debug information.  There is too much
type play.

extern void glDrawElements(GLenum mode, GLsizei count, GLenum type,
                           const GLvoid *indices)

leaves the meaning of the runtime value for "type" important in building
a lisp interface.  Are such things in the debuggers static information?
And OpenGL is relatively manageable protocol.

Run time type errors in Lisp are usually easy to find and fix.
Allocations errors are better left to C/C++ programs.

-- 
Barry Fishman
From: bradb
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <4e17ed17-58b0-4255-98b1-12b253c90041@s12g2000prg.googlegroups.com>
On Mar 2, 9:18 am, Barry Fishman <·············@acm.org> wrote:
> Pascal Bourguignon <····@informatimago.com> writes:
> > We could use some kind of type inference to determine whether a
> > pointer points to a single element or a vector (and whether the vector
> > is "null" terminated or not).
>
> > void f(void* p){
> >     char* s=malloc(strlen(p));
> >     strcpy(s,p);
> >     /* ... */
> > }
>
> > ==> p must be a pointer to a null terminated vector of char.
>
> Ouch!  One can do all the type analysis you like, but simple human
> inspection shows that this is a function that should fixed before
> calling under any circumstance.
>
> For valid code, I think it is worthwhile hand generating a s-expression
> based protocol descriptions rather than relying on what can be guessed
> from the C header file, or even debug information.  There is too much
> type play.
>
> extern void glDrawElements(GLenum mode, GLsizei count, GLenum type,
>                            const GLvoid *indices)
>
> leaves the meaning of the runtime value for "type" important in building
> a lisp interface.  Are such things in the debuggers static information?
> And OpenGL is relatively manageable protocol.
>
> Run time type errors in Lisp are usually easy to find and fix.
> Allocations errors are better left to C/C++ programs.

I agree.  I don't think that any automated tool can actually properly
capture the usage conventions and semantics of a C program.  The best
a binding program can really do is take some of the tedium away by
generating the raw connection layer.  Determining the semantics of
what data actually means is really up to the programmer.  For a C
library to live seamlessly in the Lisp world you probably need to
layer more Lisp-like semantics over the raw C bindings.

Brad
From: Pascal Bourguignon
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <87ejat3st0.fsf@thalassa.informatimago.com>
Barry Fishman <·············@acm.org> writes:

> Pascal Bourguignon <···@informatimago.com> writes:
>> We could use some kind of type inference to determine whether a
>> pointer points to a single element or a vector (and whether the vector
>> is "null" terminated or not).  
>>
>> void f(void* p){
>>     char* s=malloc(strlen(p));
>>     strcpy(s,p);
>>     /* ... */
>> }
>>
>> ==> p must be a pointer to a null terminated vector of char.
>
> Ouch!  One can do all the type analysis you like, but simple human
> inspection shows that this is a function that should fixed before
> calling under any circumstance.

Oops, right!  It's plain wrong.


> For valid code, I think it is worthwhile hand generating a s-expression
> based protocol descriptions rather than relying on what can be guessed
> from the C header file, or even debug information.  There is too much
> type play.
>
> extern void glDrawElements(GLenum mode, GLsizei count, GLenum type,
>                            const GLvoid *indices)
>
> leaves the meaning of the runtime value for "type" important in building
> a lisp interface.  Are such things in the debuggers static information?
> And OpenGL is relatively manageable protocol.
>
> Run time type errors in Lisp are usually easy to find and fix.
> Allocations errors are better left to C/C++ programs.
>
> -- 
> Barry Fishman

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

WARNING: This product attracts every other piece of matter in the
universe, including the products of other manufacturers, with a
force proportional to the product of the masses and inversely
proportional to the distance between them.
From: Ken Tilton
Subject: How Google SoC works [was Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL]
Date: 
Message-ID: <47c88349$0$25041$607ed4bc@cv.net>
Ken Tilton wrote:
> 
> 
> bradb wrote:
> 
>> I'm hoping that another few months of part time work will get me to
>> the point where I can work usefully with C++ code.
> 
> 
> Why not let Google pay someone to do it? Summer of Code 2008 is on the 
> way (proposals from students due in three weeks) and LispNYC is 
> soliciting proposals now for prospective candidates to consider (search 
> half-way down for a link to a form you can use):
> 
>   http://www.lispnyc.org/home.clp
> 
> You would be the mentor, and if you have a student in mind you cannot 
> designate them to Google but it certainly would give them a boost to 
> have your backing.

I should clarify:

Google approves (a) a student to do (b) a project (c) under the guidance 
of a /group/ such as LispNYC. LispNYC proposals are a brainstorm list to 
give students ideas on the frontend. Google takes proposals directly 
from students, including proposals not suggested by LispNYC which at 
this stage is irrelevant except ISTR a place for the student to indicate 
they are already in touch with someone at the group. Google forwards 
project/students (unfiltered, I think, except for completeness of 
submission) to LispNYC, who responds with a ranking. Google then makes a 
final determination up or down.

kenny

-- 
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
  in the evening, die content!"
                     -- Confucius
From: Stanisław Halik
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil  Things with CL
Date: 
Message-ID: <fqa23p$m7s$1@news2.task.gda.pl>
thus spoke Ken Tilton <···········@optonline.net>:

> Anyone know how CFFI-Grovel is faring [...]

Got forked into iolib-alien-grovel which gets all the development
nowadays.

-- 
Nawet świnka wejdzie na drzewo kiedy ją chwalą.
From: Rob Warnock
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <UYWdnbiF8dBgtjTanZ2dnUVZ_jqdnZ2d@speakeasy.net>
billc  <········@gmail.com> wrote:
+---------------
| Topic: Doing Evil Things with CL
| Presenter: Brad Beveridge
...
| Summary: Brad will be talking about various low-level bits and pieces,
| including:
| 
| - Loading code that is normally a Linux driver into a Lisp image
+---------------

Hey, that's *my* trick!!  ;-}

Well, no, actually. My signature hack is writing the driver in a
Lisp image in the first place, and, only after it's functionally
complete *and* tested by mmap()'ing the hardware into the Lisp
image and using the Lisp-based driver to poke at the hardware
including DMA'ing into pinned user-mode pages, then hand-compiling[1]
it to C and dropping it into the Unix/Linux kernel...


-Rob

[1] Well, I do tend to automatically generate the C header files,
    mainly the hardware register bit definitions (structs & manifest
    constants) from within the Lisp image. And also some simple
    C macros. And initialized arrays of magic constants. And also
    sometimes function prototypes. But the body of the executable
    function code is still hand-compiled to C.

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: billc
Subject: Re: Vancouver Lisp Users Group meeting for February 2008 - Doing Evil 	Things with CL
Date: 
Message-ID: <ce1ee52a-eb14-4530-b590-e092ff5078b0@v4g2000hsf.googlegroups.com>
On Feb 5, 7:39 pm, ····@rpw3.org (Rob Warnock) wrote:
> billc  <········@gmail.com> wrote:
>
> +---------------
> | Topic: Doing Evil Things with CL
> | Presenter: Brad Beveridge
> ...
> | Summary: Brad will be talking about various low-level bits and pieces,
> | including:
> |
> | - Loading code that is normally a Linux driver into a Lisp image
> +---------------
>
> Hey, that's *my* trick!!  ;-}

Actually, you've got quite a few neat tricks! If you ever make a trip
up to Vancouver, please let me know as we would love to have you talk
at one of our lispvan meetings.

- Bill