From: Karol Skocik
Subject: stream question
Date: 
Message-ID: <1126535880.929729.270850@z14g2000cwz.googlegroups.com>
Hi,
  answer for this my question will be trivial for you, but I cant find
nice solution for one basic stream functionality.
  What I want to do, is to employ one C/C++ code indenter for one C++
lib I want to use with Lisp, but the lib's source is terribly formated,
so I want to make it readable. Since I hope to get some comments on my
newbie code, I am giving the source as it is running on CMUCL 19b/Linux
:

(defun is-file-p (file)
  (let ((filename (if (pathname file) (namestring file) file)))
    (if (and (probe-file filename)
	     (char/= #\/ (elt filename (1- (length filename)))))
	t nil)))

(defun has-suffix-p (file suffixes)
  (let* ((filename (if (pathname file) (namestring file) file))
	 (position-of-dot (position #\. filename :from-end t :test #'char=))
	 (suffix (if position-of-dot (subseq filename (1+ position-of-dot))
"")))
    (if (member suffix suffixes :test #'string=) t nil)))

(defun files-with-suffix (root-dir suffixes)
  (labels ((rec (dirs acc)
	     (cond ((null dirs) acc)
		   ((is-file-p (car dirs))
		    (if (has-suffix-p (car dirs) suffixes)
			(cons (car dirs) (rec (cdr dirs) acc))
			(rec (cdr dirs) acc)))
		   (t (rec (directory (car dirs))
			   (rec (cdr dirs) acc))))))
    (rec (directory root-dir) nil)))

;; -------------- here it is :

(mapc (lambda (file)
	(with-output-to-string (s)
	  (run-program "/usr/local/bin/bcpp" (list (namestring file)) :output
s)
	  (with-open-file (f file :direction :output :if-exists :overwrite)
	    ;; *HERE* I want to write the content of stream s to file,
	    ;; for example converting that to string but HOW?
            )))
      (files-with-suffix "/home/neptun/vflib/" '("h" "cc")))

I am sure that there must be something like this. I was searching for a
while, but streams made me confused a little bit, I mean I don't know
Gray streams now, or simple-streams, this looks like very basic
functionality.

Thanks for any ideas,
  Karol
From: Marco Antoniotti
Subject: Re: stream question
Date: 
Message-ID: <evjVe.5$pa3.1426@typhoon.nyu.edu>
Hi

Karol Skocik wrote:
> Hi,
>   answer for this my question will be trivial for you, but I cant find
> nice solution for one basic stream functionality.
>   What I want to do, is to employ one C/C++ code indenter for one C++
> lib I want to use with Lisp, but the lib's source is terribly formated,
> so I want to make it readable. Since I hope to get some comments on my
> newbie code, I am giving the source as it is running on CMUCL 19b/Linux
> :
> 
> (defun is-file-p (file)
>   (let ((filename (if (pathname file) (namestring file) file)))
>     (if (and (probe-file filename)
> 	     (char/= #\/ (elt filename (1- (length filename)))))
> 	t nil)))

If there is one thing you do not want to do is to convert a pathname to 
a namestring.  See the chapter in Practical Common Lisp about handling 
this sort of things.

However, the function above can be defined as

(defun is-file-p (file)
   (let* ((p (pathname file))
          (pt (pathname-type p))
          (pn (pathname-name p))
     (and pn
          (not (eq :unspecific pn))
          (not (eq :unspecific pt))
          (probe-file p))))

Note that there may be other vagaries with the above function, but it is 
a good start.


> 
> (defun has-suffix-p (file suffixes)
>   (let* ((filename (if (pathname file) (namestring file) file))
> 	 (position-of-dot (position #\. filename :from-end t :test #'char=))
> 	 (suffix (if position-of-dot (subseq filename (1+ position-of-dot))
> "")))
>     (if (member suffix suffixes :test #'string=) t nil)))

TRT to do here is the following

(defun has-suffix-p (file suffixes)
   (member (pathname-type (pathname file))
           suffixes
           :test #'string-equal))

> 
> (defun files-with-suffix (root-dir suffixes)
>   (labels ((rec (dirs acc)
> 	     (cond ((null dirs) acc)
> 		   ((is-file-p (car dirs))
> 		    (if (has-suffix-p (car dirs) suffixes)
> 			(cons (car dirs) (rec (cdr dirs) acc))
> 			(rec (cdr dirs) acc)))
> 		   (t (rec (directory (car dirs))
> 			   (rec (cdr dirs) acc))))))
>     (rec (directory root-dir) nil)))


;;; On a UNIX system this has good chances of working.
(defun files-with-suffix (root-dir suffixes)
    (let ((df (merge-pathnames
                 (make-pathname :directory '(:relative :wild-inferiors)
                                :name :wild
                                :type :wild)
                 root-dir)))
       (directory df)))




> 
> ;; -------------- here it is :
> 
> (mapc (lambda (file)
> 	(with-output-to-string (s)
> 	  (run-program "/usr/local/bin/bcpp" (list (namestring file)) :output
> s)
> 	  (with-open-file (f file :direction :output :if-exists :overwrite)
> 	    ;; *HERE* I want to write the content of stream s to file,
> 	    ;; for example converting that to string but HOW?
>             )))
>       (files-with-suffix "/home/neptun/vflib/" '("h" "cc")))
> 
> I am sure that there must be something like this. I was searching for a
> while, but streams made me confused a little bit, I mean I don't know
> Gray streams now, or simple-streams, this looks like very basic
> functionality.
> 
> Thanks for any ideas,

In general you do not want to look at the namestrings of a pathname to 
do these sort of things.  Despite all the implementation dependencies 
PATHNAMEs are usually pretty good at this sort of things.


Cheers
--
Marco
From: Pascal Bourguignon
Subject: Re: stream question
Date: 
Message-ID: <87br2ygm47.fsf@thalassa.informatimago.com>
Marco Antoniotti <·······@cs.nyu.edu> writes:
> ;;; On a UNIX system this has good chances of working.
> (defun files-with-suffix (root-dir suffixes)
>     (let ((df (merge-pathnames
>                  (make-pathname :directory '(:relative :wild-inferiors)
>                                 :name :wild
>                                 :type :wild)
>                  root-dir)))
>        (directory df)))

Wouldn't it be better to pass root-dir as defaults to make-pathname,
to have it inhering portably  host & device?

 (defun files-with-suffix (root-dir suffixes)
     (let ((df (merge-pathnames
                  (make-pathname :directory '(:relative :wild-inferiors)
                                 :name :wild
                                 :type :wild
                                 :defaults root-dir) ; add this
                  root-dir)))
        (directory df)))


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Cats meow out of angst
"Thumbs! If only we had thumbs!
We could break so much!"
From: Karol Skocik
Subject: Re: stream question
Date: 
Message-ID: <1126552397.739391.317500@g43g2000cwa.googlegroups.com>
Hmm, this goes into subdirs :

 (defun files-with-suffix (root-dir suffixes)
     (let ((df (merge-pathnames
                  (make-pathname :directory '(:relative
:wild-inferiors)
                                 :name :wild
                                 :type :wild
                                 :defaults root-dir) ; add this
                  root-dir)))
        (directory df)))

 but it does not choose the files having specific suffix.
The :type in make-pathname expects this type :

(OR BASE-STRING LISP::PATTERN (MEMBER NIL :UNSPECIFIC :WILD))

so it looks like that LISP::PATTERN is the candidate for being selector
of files based on suffix, but I don't have a clue what that might be.
If it's not possible to make LISP::PATTERN from something like this :
'("h" "cc") to choose
just c++ sources, then probably only iterating through suffixes list
will do.
What is LISP::PATTERN? 

Karol
From: Marco Antoniotti
Subject: Re: stream question
Date: 
Message-ID: <3%kVe.7$pa3.289@typhoon.nyu.edu>
Karol Skocik wrote:
> Hmm, this goes into subdirs :
> 
>  (defun files-with-suffix (root-dir suffixes)
>      (let ((df (merge-pathnames
>                   (make-pathname :directory '(:relative
> :wild-inferiors)
>                                  :name :wild
>                                  :type :wild
>                                  :defaults root-dir) ; add this
>                   root-dir)))
>         (directory df)))
> 
>  but it does not choose the files having specific suffix.
> The :type in make-pathname expects this type :
> 
> (OR BASE-STRING LISP::PATTERN (MEMBER NIL :UNSPECIFIC :WILD))
> 
> so it looks like that LISP::PATTERN is the candidate for being selector
> of files based on suffix, but I don't have a clue what that might be.
> If it's not possible to make LISP::PATTERN from something like this :
> '("h" "cc") to choose
> just c++ sources, then probably only iterating through suffixes list
> will do.
> What is LISP::PATTERN?

An implementation dependent thing you should not touch with a 20-meters 
pole. :)

What you want is much simpler.

   (defun files-with-suffix (root-dir suffixes)
      (let ((df (merge-pathnames
                   (make-pathname :directory '(:relative :wild-inferiors)
                                  :name :wild
                                  :type :wild
                                  :defaults root-dir) ; add this
                   root-dir)))
         (delete-if (lambda (p) (has-suffix-p p suffixes))
                    (directory df))))

Cheers
--
Marco
From: Karol Skocik
Subject: Re: stream question
Date: 
Message-ID: <1126556036.738211.323130@g43g2000cwa.googlegroups.com>
Ok, I will stay away of LISP::PATTERN. But it has such attractive name
:)
Anyway, this does not work as my original version :

(defun files-with-suffix (root-dir suffixes)
      (let ((df (merge-pathnames
                   (make-pathname :directory '(:relative
:wild-inferiors)
                                  :name :wild
                                  :type :wild
                                  :defaults root-dir) ; add this
                   root-dir)))
         (delete-if (lambda (p) (has-suffix-p p suffixes))
                    (directory df))))

CL-USER> (files-with-suffix "/home/neptun/vflib/" '("h" "cc"))
(#P"/home/neptun/vflib/Makefile" #P"/home/neptun/vflib/README"
 #P"/home/neptun/vflib/doc/" #P"/home/neptun/vflib/doc/CHANGES"
 #P"/home/neptun/vflib/doc/Makefile"
#P"/home/neptun/vflib/doc/vflib-1.html"
 #P"/home/neptun/vflib/doc/vflib-10.html"
 #P"/home/neptun/vflib/doc/vflib-11.html"
 #P"/home/neptun/vflib/doc/vflib-12.html"
 #P"/home/neptun/vflib/doc/vflib-13.html"
 #P"/home/neptun/vflib/doc/vflib-14.html"
 #P"/home/neptun/vflib/doc/vflib-15.html"
 #P"/home/neptun/vflib/doc/vflib-16.html"
 #P"/home/neptun/vflib/doc/vflib-17.html"
 #P"/home/neptun/vflib/doc/vflib-18.html"
 #P"/home/neptun/vflib/doc/vflib-19.html"
 #P"/home/neptun/vflib/doc/vflib-2.html"
 #P"/home/neptun/vflib/doc/vflib-20.html"
 #P"/home/neptun/vflib/doc/vflib-21.html"
 #P"/home/neptun/vflib/doc/vflib-22.html"
 #P"/home/neptun/vflib/doc/vflib-23.html"
 #P"/home/neptun/vflib/doc/vflib-24.html"
 #P"/home/neptun/vflib/doc/vflib-25.html"
 #P"/home/neptun/vflib/doc/vflib-26.html"
 #P"/home/neptun/vflib/doc/vflib-27.html"
 #P"/home/neptun/vflib/doc/vflib-28.html"
 #P"/home/neptun/vflib/doc/vflib-29.html"
 #P"/home/neptun/vflib/doc/vflib-3.html"
 #P"/home/neptun/vflib/doc/vflib-30.html"
 #P"/home/neptun/vflib/doc/vflib-4.html"
 #P"/home/neptun/vflib/doc/vflib-5.html"
 #P"/home/neptun/vflib/doc/vflib-6.html"
 #P"/home/neptun/vflib/doc/vflib-7.html"
 #P"/home/neptun/vflib/doc/vflib-8.html"
 #P"/home/neptun/vflib/doc/vflib-9.html"
#P"/home/neptun/vflib/doc/vflib.html"
 #P"/home/neptun/vflib/doc/vflib.pdf"
#P"/home/neptun/vflib/doc/vflib.sgml"
 #P"/home/neptun/vflib/include/" #P"/home/neptun/vflib/lib/"
 #P"/home/neptun/vflib/lib/dummy" #P"/home/neptun/vflib/src/")

What I wanted to achieve was something like combination of flatten and
filter. So that it recurses the directory structure and collects only
those files having suffix in suffixes like '("h" "cc").
But it's OK, because when I fix testing suffix as string and test it
using make-pathname, I think it will do what I want and also it should
be portable as well.

Thanks for ideas,
  Karol
From: Marco Antoniotti
Subject: Re: stream question
Date: 
Message-ID: <PNlVe.9$pa3.1600@typhoon.nyu.edu>
Karol Skocik wrote:
> Ok, I will stay away of LISP::PATTERN. But it has such attractive name
> :)

Good boy :)


> Anyway, this does not work as my original version :
> 
> (defun files-with-suffix (root-dir suffixes)
>       (let ((df (merge-pathnames
>                    (make-pathname :directory '(:relative
> :wild-inferiors)
>                                   :name :wild
>                                   :type :wild
>                                   :defaults root-dir) ; add this
>                    root-dir)))
>          (delete-if (lambda (p) (has-suffix-p p suffixes))
>                     (directory df))))

Ooops.  Of course.  I forgot a NOT.  The LAMBDA should be

	(lambda (p) (not (has-suffix-p p suffixes)))

Cheers
--
Marco








> 
> CL-USER> (files-with-suffix "/home/neptun/vflib/" '("h" "cc"))
> (#P"/home/neptun/vflib/Makefile" #P"/home/neptun/vflib/README"
>  #P"/home/neptun/vflib/doc/" #P"/home/neptun/vflib/doc/CHANGES"
>  #P"/home/neptun/vflib/doc/Makefile"
> #P"/home/neptun/vflib/doc/vflib-1.html"
>  #P"/home/neptun/vflib/doc/vflib-10.html"
>  #P"/home/neptun/vflib/doc/vflib-11.html"
>  #P"/home/neptun/vflib/doc/vflib-12.html"
>  #P"/home/neptun/vflib/doc/vflib-13.html"
>  #P"/home/neptun/vflib/doc/vflib-14.html"
>  #P"/home/neptun/vflib/doc/vflib-15.html"
>  #P"/home/neptun/vflib/doc/vflib-16.html"
>  #P"/home/neptun/vflib/doc/vflib-17.html"
>  #P"/home/neptun/vflib/doc/vflib-18.html"
>  #P"/home/neptun/vflib/doc/vflib-19.html"
>  #P"/home/neptun/vflib/doc/vflib-2.html"
>  #P"/home/neptun/vflib/doc/vflib-20.html"
>  #P"/home/neptun/vflib/doc/vflib-21.html"
>  #P"/home/neptun/vflib/doc/vflib-22.html"
>  #P"/home/neptun/vflib/doc/vflib-23.html"
>  #P"/home/neptun/vflib/doc/vflib-24.html"
>  #P"/home/neptun/vflib/doc/vflib-25.html"
>  #P"/home/neptun/vflib/doc/vflib-26.html"
>  #P"/home/neptun/vflib/doc/vflib-27.html"
>  #P"/home/neptun/vflib/doc/vflib-28.html"
>  #P"/home/neptun/vflib/doc/vflib-29.html"
>  #P"/home/neptun/vflib/doc/vflib-3.html"
>  #P"/home/neptun/vflib/doc/vflib-30.html"
>  #P"/home/neptun/vflib/doc/vflib-4.html"
>  #P"/home/neptun/vflib/doc/vflib-5.html"
>  #P"/home/neptun/vflib/doc/vflib-6.html"
>  #P"/home/neptun/vflib/doc/vflib-7.html"
>  #P"/home/neptun/vflib/doc/vflib-8.html"
>  #P"/home/neptun/vflib/doc/vflib-9.html"
> #P"/home/neptun/vflib/doc/vflib.html"
>  #P"/home/neptun/vflib/doc/vflib.pdf"
> #P"/home/neptun/vflib/doc/vflib.sgml"
>  #P"/home/neptun/vflib/include/" #P"/home/neptun/vflib/lib/"
>  #P"/home/neptun/vflib/lib/dummy" #P"/home/neptun/vflib/src/")
> 
> What I wanted to achieve was something like combination of flatten and
> filter. So that it recurses the directory structure and collects only
> those files having suffix in suffixes like '("h" "cc").
> But it's OK, because when I fix testing suffix as string and test it
> using make-pathname, I think it will do what I want and also it should
> be portable as well.
> 
> Thanks for ideas,
>   Karol
> 
From: Marco Antoniotti
Subject: Re: stream question
Date: 
Message-ID: <SWkVe.6$pa3.1132@typhoon.nyu.edu>
Pascal Bourguignon wrote:
> Marco Antoniotti <·······@cs.nyu.edu> writes:
> 
>>;;; On a UNIX system this has good chances of working.
>>(defun files-with-suffix (root-dir suffixes)
>>    (let ((df (merge-pathnames
>>                 (make-pathname :directory '(:relative :wild-inferiors)
>>                                :name :wild
>>                                :type :wild)
>>                 root-dir)))
>>       (directory df)))
> 
> 
> Wouldn't it be better to pass root-dir as defaults to make-pathname,
> to have it inhering portably  host & device?
> 
>  (defun files-with-suffix (root-dir suffixes)
>      (let ((df (merge-pathnames
>                   (make-pathname :directory '(:relative :wild-inferiors)
>                                  :name :wild
>                                  :type :wild
>                                  :defaults root-dir) ; add this
>                   root-dir)))
>         (directory df)))
> 

Well, yes.

Cheers
--
marco
From: Pascal Bourguignon
Subject: Re: stream question
Date: 
Message-ID: <873boai6ok.fsf@thalassa.informatimago.com>
"Karol Skocik" <············@gmail.com> writes:

> Hi,
>   answer for this my question will be trivial for you, but I cant find
> nice solution for one basic stream functionality.
>   What I want to do, is to employ one C/C++ code indenter for one C++
> lib I want to use with Lisp, but the lib's source is terribly formated,
> so I want to make it readable. Since I hope to get some comments on my
> newbie code, I am giving the source as it is running on CMUCL 19b/Linux:
> [...]

(defun files-with-suffixes (dirpath suffixes)
  (loop for ftype in suffixes
     nconc (directory (make-pathname :defaults dirpath
                                     :name :wild
                                     :type ftype))))

> ;; -------------- here it is :
>
> (mapc (lambda (file)
> 	(with-output-to-string (s)
> 	  (run-program "/usr/local/bin/bcpp" (list (namestring file)) :output
> s)
> 	  (with-open-file (f file :direction :output :if-exists :overwrite)
> 	    ;; *HERE* I want to write the content of stream s to file,
> 	    ;; for example converting that to string but HOW?
>             )))
>       (files-with-suffix "/home/neptun/vflib/" '("h" "cc")))


(mapc (lambda (file)
        ;; we need to load contents before opening file for overwriting...
        (let ((contents (with-output-to-string (s)
                            (run-program "/usr/local/bin/bcpp"
                              (list (namestring file)) :output  s))))
          (with-open-file (f file :direction :output :if-exists :overwrite)
            (write-sequence contents f))))
      (files-with-suffixes "/home/neptun/vflib/" '("h" "cc")))

Or, use dolist insteaad of mapc:

(dolist (file (files-with-suffixes "/home/neptun/vflib/" '("h" "cc")))
  ;; we need to load contents before opening file for overwriting...
  (let ((contents (with-output-to-string (s)
                    (run-program "/usr/local/bin/bcpp"
                      (list (namestring file)) :output  s))))
    (with-open-file (f file :direction :output :if-exists :overwrite)
      (write-sequence contents f))))


> I am sure that there must be something like this. I was searching for a
> while, but streams made me confused a little bit, I mean I don't know
> Gray streams now, or simple-streams, this looks like very basic
> functionality.

The result of with-output-to-string is a string containing the data
written to the stream.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Small brave carnivores
Kill pine cones and mosquitoes
Fear vacuum cleaner
From: Karol Skocik
Subject: Re: stream question
Date: 
Message-ID: <1126544499.553792.40740@g49g2000cwa.googlegroups.com>
Yes! Thats it, thanks. I thought that I need to explicitly manipulate
the stream, and I could not find it in docs...
Your files-with-suffixes works fine, but does not go into subdirs which
is something I need. But I have at least found out what make-pathname
does :)

Have a nice day,
  Karol
From: Pascal Bourguignon
Subject: Re: stream question
Date: 
Message-ID: <87fysagqbo.fsf@thalassa.informatimago.com>
"Karol Skocik" <············@gmail.com> writes:

> Yes! Thats it, thanks. I thought that I need to explicitly manipulate
> the stream, and I could not find it in docs...
> Your files-with-suffixes works fine, but does not go into subdirs which
> is something I need. But I have at least found out what make-pathname
> does :)

Use it as: (files-with-suffixes "/some/dir/**/" suffixes)
or modify it to do it systematically:

(defun files-with-suffixes (dirpath suffixes)
  (loop for ftype in suffixes
     nconc (directory (make-pathname
                       :defaults dirpath
                       :directory (append (pathname-directory dirpath)
                                          '(:wild-inferiors))
                       :name :wild
                       :type ftype))))



-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
comp.lang.lisp / 2005 / sep