From: Bata
Subject: Reference other array objects from within loop?
Date: 
Message-ID: <h3u0k2$gvf$1@news.albasani.net>
Hello all,
 I am in the process of making a program that I hope will help me
simulate trading programs in CL.  Helps me to learn the language if I am
making something that I would have some actual use for.  I have already
(with guidance from this newsgroup) written the code that reads the raw
data from a csv-file and puts the relevant data into an array of
objects.  What I need to figure out now is how best to analyze the data.
 What I need is a way to run a particular (arbitrary) piece of code over
each and every object in the array.  This is the easy part since I can
just put this code within a 'loop' command (or the closest most useful
mash-up of 'do' or something).  The conceptual difficulty that I am
facing right now is because of the nature of the code within the loop.
 I need this code to be able to gather data from "this" object ("this"
being any particular iteration of the loop in the array) as well as (for
example) the last 15 objects in that same array that is being traversed.
 For example, I would like to know how to do the following in the code
that is within the loop:
   Is the current bar's high above or below the moving average (of
high's (inconsequential to the main question asked here)) of the _last_
15 bars?
 I feel that I should define a macro that will evaluate the above code
(well, the code that answers the above question actually) over each and
every bar (naturally, there will be a check to make sure that I don't
get kooky results for the first 15 objects (bars) in the array).  How do
I get the code within the loop to be able to call the slots in the past
15 objects of the array?

Thank you,
Bata

From: Mirko
Subject: Re: Reference other array objects from within loop?
Date: 
Message-ID: <38bb4995-3af6-4308-b3e1-ca6f115eb807@r2g2000yqm.googlegroups.com>
On Jul 18, 10:33 pm, Bata <·········@yahoo.ca> wrote:
> Hello all,
>  I am in the process of making a program that I hope will help me
> simulate trading programs in CL.  Helps me to learn the language if I am
> making something that I would have some actual use for.  I have already
> (with guidance from this newsgroup) written the code that reads the raw
> data from a csv-file and puts the relevant data into an array of
> objects.  What I need to figure out now is how best to analyze the data.
>  What I need is a way to run a particular (arbitrary) piece of code over
> each and every object in the array.  This is the easy part since I can
> just put this code within a 'loop' command (or the closest most useful
> mash-up of 'do' or something).  The conceptual difficulty that I am
> facing right now is because of the nature of the code within the loop.
>  I need this code to be able to gather data from "this" object ("this"
> being any particular iteration of the loop in the array) as well as (for
> example) the last 15 objects in that same array that is being traversed.
>  For example, I would like to know how to do the following in the code
> that is within the loop:
>    Is the current bar's high above or below the moving average (of
> high's (inconsequential to the main question asked here)) of the _last_
> 15 bars?
>  I feel that I should define a macro that will evaluate the above code
> (well, the code that answers the above question actually) over each and
> every bar (naturally, there will be a check to make sure that I don't
> get kooky results for the first 15 objects (bars) in the array).  How do
> I get the code within the loop to be able to call the slots in the past
> 15 objects of the array?
>
> Thank you,
> Bata

First, for looping over a vector, you may use (map 'vector ...)

As for your problem, a two-pass solution would work.  First generate a
moving average vector from your data (the first fifteen values would
be nil since you do not have enough numbers)

Then you can loop over the two vectors like so:
(map 'vector #'(lambda (price moving-average) ...) price-vector moving-
average-vector)

I did skip the hairy details on generating the moving average vector.
One, I am really tired to think of something clean.  And two, I guess
that you will run your comparison routine on the n-15 points, since
you do not have the moving average for the first 15 points.  So you
will have to decide how to deal with that issue as well.

I don't see a role for the macro here yet.

hth,

Mirko
From: Paul Donnelly
Subject: Re: Reference other array objects from within loop?
Date: 
Message-ID: <87tz1973ql.fsf@plap.localdomain>
Bata <·········@yahoo.ca> writes:

>  I need this code to be able to gather data from "this" object ("this"
> being any particular iteration of the loop in the array) as well as (for
> example) the last 15 objects in that same array that is being traversed.

I don't see any reason to make it more complicated than a dotimes loop
and subtracting from the array index. If part of your code needs the
n-2th item, just pass the array and an index and let it get it on its
own. Seems like what you've got is a more complicated access pattern
than map or something supports.

Now, you *could* abstract it as a function or a macro if you think that
will be cleaner. But on the other hand, maybe abstracting from just this
one occasion will lead to an abstraction whose use and function isn't
obvious without the context it would have as concrete code. If I'm not
sure how I want an abstraction to act, I write it as un-abstract
code. If I have to write very similar code a bunch of times, that's when
I decide to abstract. And at that point, the behavior of my abstraction
function (which may likely have a macro as a frontend) should be clearer
to me.
From: D Herring
Subject: Re: Reference other array objects from within loop?
Date: 
Message-ID: <4a62a22b$0$10253$6e1ede2f@read.cnntp.org>
Bata wrote:
> Hello all,
>  I am in the process of making a program that I hope will help me
> simulate trading programs in CL.  Helps me to learn the language if I am
> making something that I would have some actual use for.  I have already
> (with guidance from this newsgroup) written the code that reads the raw
> data from a csv-file and puts the relevant data into an array of
> objects.  What I need to figure out now is how best to analyze the data.
>  What I need is a way to run a particular (arbitrary) piece of code over
> each and every object in the array.  This is the easy part since I can
> just put this code within a 'loop' command (or the closest most useful
> mash-up of 'do' or something).  The conceptual difficulty that I am
> facing right now is because of the nature of the code within the loop.
>  I need this code to be able to gather data from "this" object ("this"
> being any particular iteration of the loop in the array) as well as (for
> example) the last 15 objects in that same array that is being traversed.
>  For example, I would like to know how to do the following in the code
> that is within the loop:
>    Is the current bar's high above or below the moving average (of
> high's (inconsequential to the main question asked here)) of the _last_
> 15 bars?
>  I feel that I should define a macro that will evaluate the above code
> (well, the code that answers the above question actually) over each and
> every bar (naturally, there will be a check to make sure that I don't
> get kooky results for the first 15 objects (bars) in the array).  How do
> I get the code within the loop to be able to call the slots in the past
> 15 objects of the array?

Do this (more or less) the same way you would in any other language. 
Change the inner function definition to take two arguments: the 
current value and also any shared state it may need.  Or bind a 
dynamic variable (see defvar and special) to use as a hidden parameter.

For kicks, a circular list makes an easy ring buffer.  Here's a simple 
helper function.

(defun make-circle (length)
   (let ((list nil))
     (dotimes (n length)
       (push nil list))
     (setf (cdr (last list)) list)
     list))

Be careful with circular lists; don't pass them to LENGTH, and make 
sure to set *print-circle* to t.

- Daniel
From: Pascal J. Bourguignon
Subject: Re: Reference other array objects from within loop?
Date: 
Message-ID: <87eisdc627.fsf@galatea.local>
Bata <·········@yahoo.ca> writes:

> Hello all,
>  I am in the process of making a program that I hope will help me
> simulate trading programs in CL.  Helps me to learn the language if I am
> making something that I would have some actual use for.  I have already
> (with guidance from this newsgroup) written the code that reads the raw
> data from a csv-file and puts the relevant data into an array of
> objects.  What I need to figure out now is how best to analyze the data.
>  What I need is a way to run a particular (arbitrary) piece of code over
> each and every object in the array.  This is the easy part since I can
> just put this code within a 'loop' command (or the closest most useful
> mash-up of 'do' or something).  The conceptual difficulty that I am
> facing right now is because of the nature of the code within the loop.
>  I need this code to be able to gather data from "this" object ("this"
> being any particular iteration of the loop in the array) as well as (for
> example) the last 15 objects in that same array that is being traversed.
>  For example, I would like to know how to do the following in the code
> that is within the loop:
>    Is the current bar's high above or below the moving average (of
> high's (inconsequential to the main question asked here)) of the _last_
> 15 bars?
>  I feel that I should define a macro that will evaluate the above code
> (well, the code that answers the above question actually) over each and
> every bar (naturally, there will be a check to make sure that I don't
> get kooky results for the first 15 objects (bars) in the array).  How do
> I get the code within the loop to be able to call the slots in the past
> 15 objects of the array?


Note that this has nothing to do with Lisp specifically.  This is a
general programming question.  You should learn how to program.


GENTLE = Common Lisp: A Gentle Introduction to Symbolic Computation
         http://www-cgi.cs.cmu.edu/afs/cs.cmu.edu/user/dst/www/LispBook/index.html
         http://www.cs.cmu.edu/~dst/LispBook/
    
HTDP   = How to Design Programs -- An Introduction to Computing and Programming
         http://www.htdp.org/2003-09-26/Book/  

SICP   = Structure and Interpretation of Computer Programs
         http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html
         http://swiss.csail.mit.edu/classes/6.001/abelson-sussman-lectures/
         http://www.codepoetics.com/wiki/index.php?title=Topics:SICP_in_other_languages
         http://eli.thegreenplace.net/category/programming/lisp/sicp/
         http://www.neilvandyke.org/sicp-plt/


IDAA  = "Introduction to the Design and Analysis of Algorithms", Anany V. Levitin 
        http://www.amazon.com/Introduction-Design-Analysis-Algorithms-2nd/dp/0321358287/ref=sr_1_1?ie=UTF8&s=books&qid=1247998140&sr=8-1




I'm not sure it's a good idea to store your CSV data in an array.  It
would probably be easier to store it in a vector, since I assume the
different columns are not of the same type (you must have columns for
the ticker symbol, columns for the quotes, columns for the date,
columns for the volumes, etc).  So it would be better to store each
row in a structure or an object, and keep them in a vector of such
objects.  This is something you would understand having learned how to
program.

And having learned how to program, you will be able to build a loop to
scan the rows taking into account the P previous items. 


Something like:

  (loop
     :with p = 15
     :for i :from p :to (length data)
     :do (flet ((process-rows (data start end)
                   (format t "moving average: ~F~%"
                         (/ (loop :for i :from start :below end
                                  :sum (price (aref data i)))
                            (- end start)))))
             (process-rows data (- i p) i)))




Of course, for moving averages, there's a way to avoid O(n�) sums, by
keeping the moving sum, and subtracting an old data point and adding a
new one at each iteration.  You could build a closure or an object to
represent such a moving average:


(defclass moving-average ()
   ((width :reader width :initarg :width :type (integer 1))
    (sum   :initform 0)
    (data  :initarg :data :reader data :type vector)
    (key   :initarg :key :reader key :type function
           :documentation "Projector to get the value to be averaged from the data slot.")
    (index :initform 0 :reader index :initarg :from)))

(defmethod initialize-instance :after ((self moving-average) &key &allow-other-keys)
   (with-slots ((sum sum) (key key) (data data) (width width)) self
      (setf sum (loop :for i :from 0 :below (min (length data) width) 
                      :sum (funcall key (aref data i)))))
   self)

(defmethod average-value ((self moving-average))
   (/ (slot-value self 'sum) (slot-value self 'width)))

(defmethod advance ((self moving-average))
   (with-slots ((sum sum) (key key) (data data) (index index) (width width)) self
      (when (< (+ index width) (length data))
         (setf sum (+ (- sum (funcall key (aref data index)))
                      (funcall key (aref data (+ index width)))))
         (incf index))))

(defmethod donep ((self moving-average))
   (with-slots ((index index) (width width) (data data)) self
       (>= (+ index width) (length data))))


;; So you could write:

(defstruct quote date ticker volume open close)
(let* ((data (coerce (loop :for i :from 0 :to 100
                          :collect (make-quote :date (+ 2009000 i)
                                               :open (random (abs (1+ (* 10 (sin i)))))))
                    'vector))
       (ma (make-instance 'moving-average  :data data :key (function quote-open) :width 15)))
   (loop :for i :from 15 
         :until (donep ma)
         :do (advance ma)
             (format t "~3D: ~8,2F  ~8,2F~%" 
                       (quote-date (aref data i))
                       (quote-open (aref data i))
                       (average-value ma))))

                              
-- 
__Pascal Bourguignon__