What is the right way to create a new array (1D) from an existing
array, where the new array will have the specialized type of the type
of the existing array's elements (reguardless of the existing array's
type: ie could be t but all elements are double-float, or could be
double-float)?
;; this works if a is of type t, but not if it is specialized (say,
double-float)
(defun create-specialized-1d-array (a)
(let ((nelms (array-total-size a)))
(make-array nelms
:initial-contents (make-array nelms :displaced-to a)
:element-type (type-of (row-major-aref a 0)))))
(defparameter array-t #2a((1d0 2d0) (3d0 4d0)))
(type-of array-t) -> (SIMPLE-ARRAY T (2 2))
(defparameter array-d (make-array '(2 2) :element-type 'double-float
:initial-contents '((1d0 2d0) (3d0
4d0))))
(type-of array-d) -> (SIMPLE-ARRAY DOUBLE-FLOAT (2 2))
(defparameter vect-t #(1d0 2d0))
(type-of vect-t) -> (SIMPLE-VECTOR 2)
(defparameter vect-d (make-array 2 :element-type 'double-float
:initial-contents '(1d0 2d0)))
(type-of vect-d) -> (SIMPLE-ARRAY DOUBLE-FLOAT (2))
(defparameter array-t-spec (create-specialized-1d-array array-t))
-> want #(1d0 2d0 3d0 4d0) having type double-float
; following doesn't work:
; Array element type of :DISPLACED-TO array does not match specified
element type
(defparameter array-d-spec (create-specialized-1d-array array-d))
-> want #(1d0 2d0 3d0 4d0) having type double-float
(defparameter vect-t-spec (create-specialized-1d-array vect-t))
-> want #(1d0 2d0) having type double-float
; following doesn't work:
; Array element type of :DISPLACED-TO array does not match specified
element type
(defparameter vect-d-spec (create-specialized-1d-array vect-d))
-> want #(1d0 2d0) having type double-float
I thought I might be able to discriminate between t and specialized
arrays and then handle the cases seperatley. The problem I ran into
with this approach is that the element type is not always returned by
type-of:
(type-of vect-t)
-> (SIMPLE-VECTOR 2) ; no t, no double-float??
(type-of vect-d)
-> (SIMPLE-ARRAY DOUBLE-FLOAT (2)) ; I can get double-float
On Fri, 24 Oct 2008 12:03:35 -0700, jra wrote:
> What is the right way to create a new array (1D) from an existing array,
> where the new array will have the specialized type of the type of the
> existing array's elements (reguardless of the existing array's type: ie
> could be t but all elements are double-float, or could be double-float)?
In case all elements of an array are subtype of some particular type that
you want to specialize to, you will have to recognize this yourself and
create the new array with an explicit element-type.
Think about it: what should (vector 1 2 3) specialize to? Fixnum?
(unsigned-byte 2)? (signed-byte 17)? You get the idea.
Tamas
Tamas K Papp <······@gmail.com> wrote:
+---------------
| jra wrote:
| > What is the right way to create a new array (1D) from an existing array,
| > where the new array will have the specialized type of the type of the
| > existing array's elements (reguardless of the existing array's type: ie
| > could be t but all elements are double-float, or could be double-float)?
|
| In case all elements of an array are subtype of some particular type that
| you want to specialize to, you will have to recognize this yourself and
| create the new array with an explicit element-type.
|
| Think about it: what should (vector 1 2 3) specialize to? Fixnum?
| (unsigned-byte 2)? (signed-byte 17)? You get the idea.
+---------------
While this is true, the OP might want to look at some of the posts
in the latter half of this thread, "Subject: copying arrays":
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/e6e10cfa55a74fe5/3cecb0fbdd9f7289
There are a couple of suggested implementations of a COPY-ARRAY
which preserves the original array's properties.
Also see <http://www.cliki.net/SHALLOW-COPY-ARRAY>.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
On Oct 24, 1:02 pm, Tamas K Papp <······@gmail.com> wrote:
> On Fri, 24 Oct 2008 12:03:35 -0700, jra wrote:
> > What is the right way to create a new array (1D) from an existing array,
> > where the new array will have the specialized type of the type of the
> > existing array's elements (reguardless of the existing array's type: ie
> > could be t but all elements are double-float, or could be double-float)?
>
> In case all elements of an array are subtype of some particular type that
> you want to specialize to, you will have to recognize this yourself and
> create the new array with an explicit element-type.
>
> Think about it: what should (vector 1 2 3) specialize to? Fixnum?
> (unsigned-byte 2)? (signed-byte 17)? You get the idea.
>
> Tamas
Hi Tamas,
My immediate response to your question was going to be that I wanted
to specialize (vector 1 2 3) to (type-of (row-major-aref (vector 1 2
3) 0)). But then I tried it:
CL-USER> (type-of (row-major-aref (vector 1 2 3) 0))
BIT
CL-USER> (type-of (row-major-aref (vector 1 2 3) 1))
(INTEGER 0 536870911)
CL-USER> (type-of (row-major-aref (vector 1 2 3) 2))
(INTEGER 0 536870911)
I wasn't expecting the type of 1 to come back as BIT! How would one go
about determining the most specific subtype of all of the elements of
a general array? Is this the right way?
(defun best-type-of-elements (arr)
(reduce (lambda (type1 type2)
(cond
((subtypep type2 type1) type1)
((subtypep type1 type2) type2)
(t t)))
arr
:key #'type-of))
So, my new resoponse is that, suppose that either I use something like
best-type-of-elements or I can assume that (type-of (row-major-aref my-
array 0)) will give me either SINGLE-FLOAT or DOUBLE-FLOAT that I can
apply to all of the elments, how can I get create-specialized-1d-array
from the OP to work? How about this?
(defun create-specialized-1d-array (arr)
(let ((nelms (array-total-size arr))
(arr-type (type-of arr))
(elt-type (type-of (row-major-aref arr 0))))
; type-of leaves out t for an vector of t, like (type-of #(1 2)),
; so we say if length of type is < 3 then the elements are of t
(if (or (< (length arr-type) 3)
(eq (second arr-type) t))
; arr's elms of t (general)
(make-array nelms
:element-type elt-type
:initial-contents (make-array nelms :displaced-
to arr))
; arr's elms of something else (specialized), like doulbe-
float
(make-array nelms
:element-type elt-type
:displaced-to arr))))
This seems like a hack, but I'll use it!
CL-USER> (type-of (create-specialized-1d-array array-t))
(SIMPLE-ARRAY DOUBLE-FLOAT (4))
CL-USER> (type-of (create-specialized-1d-array array-d))
(VECTOR DOUBLE-FLOAT 4)
CL-USER> (type-of (create-specialized-1d-array vect-t))
(SIMPLE-ARRAY DOUBLE-FLOAT (2))
CL-USER> (type-of (create-specialized-1d-array vect-d))
(VECTOR DOUBLE-FLOAT 2)
Also Tamas, thanks for array-operations. I used it on a recent
project.
-Jason
On Sat, 25 Oct 2008 10:56:27 -0700, jra wrote:
> On Oct 24, 1:02 pm, Tamas K Papp <······@gmail.com> wrote:
>> On Fri, 24 Oct 2008 12:03:35 -0700, jra wrote:
>> > What is the right way to create a new array (1D) from an existing
>> > array, where the new array will have the specialized type of the type
>> > of the existing array's elements (reguardless of the existing array's
>> > type: ie could be t but all elements are double-float, or could be
>> > double-float)?
>>
>> In case all elements of an array are subtype of some particular type
>> that you want to specialize to, you will have to recognize this
>> yourself and create the new array with an explicit element-type.
>>
>> Think about it: what should (vector 1 2 3) specialize to? Fixnum?
>> (unsigned-byte 2)? (signed-byte 17)? You get the idea.
>>
>> Tamas
>
> Hi Tamas,
>
> My immediate response to your question was going to be that I wanted to
> specialize (vector 1 2 3) to (type-of (row-major-aref (vector 1 2 3)
> 0)). But then I tried it:
>
> CL-USER> (type-of (row-major-aref (vector 1 2 3) 0)) BIT
> CL-USER> (type-of (row-major-aref (vector 1 2 3) 1)) (INTEGER 0
> 536870911)
> CL-USER> (type-of (row-major-aref (vector 1 2 3) 2)) (INTEGER 0
> 536870911)
>
> I wasn't expecting the type of 1 to come back as BIT! How would one go
> about determining the most specific subtype of all of the elements of a
> general array? Is this the right way?
>
> (defun best-type-of-elements (arr)
> (reduce (lambda (type1 type2)
> (cond
> ((subtypep type2 type1) type1)
> ((subtypep type1 type2) type2)
> (t t)))
> arr
> :key #'type-of))
Something like this. But there is no guarantee that this will do what
you want, even though on most implementations it might (but I don't
really know what you want :-)) See type-of in the Hyperspec, in
particular, it will return a recognizable subtype [1] of a built-in type
[2]. The problem is that AFAIK the concept of recognizable-subtype is
implementation dependent.
[1] http://www.lispworks.com/documentation/HyperSpec/
Body/26_glo_r.htm#recognizable_subtype
[2] http://www.lispworks.com/documentation/HyperSpec/
Body/04_bc.htm#standardizedatomictypespecs
> So, my new resoponse is that, suppose that either I use something like
> best-type-of-elements or I can assume that (type-of (row-major-aref my-
> array 0)) will give me either SINGLE-FLOAT or DOUBLE-FLOAT that I can
> apply to all of the elments, how can I get create-specialized-1d-array
> from the OP to work? How about this?
> Also Tamas, thanks for array-operations. I used it on a recent project.
I struggled with your problem in implementing array-operations, and I
realized that the best thing I could do in general is have the user
specify the type explicitly. In most cases, there is no "smart" way to
guess types. And there is no guarantee that the first element of an
array will be representative either. That's why most functions in array-
operations have an explicit element-type for the resulting array, IMO
this is the cleanest approach.
Maybe we could help more if you told us what your goal is. Are you
trying to find the most narrow supertype because of neatness, or speed,
or storage efficiency? Or something else? What problem are you trying
to solve?
HTH,
Tamas
On Oct 25, 8:34 am, Tamas K Papp <······@gmail.com> wrote:
> Maybe we could help more if you told us what your goal is. Are you
> trying to find the most narrow supertype because of neatness, or speed,
> or storage efficiency? Or something else? What problem are you trying
> to solve?
I was working with cl-octave. It is cmucl centric and required some
modification for use with sbcl. Luckily, I found a diff in the cl-
octave mailing list archives that did the trick. While using the
package I found that I was able to send general arrays, but
specialized arrays caused problems. When I "fixed" it, I caused the
reverse problem: specialized arrays worked, general arrays did not. I
needed a way of handling both. Reflecting on it now, I wonder if I
should have upgraded everything to general arrays before encountering
the troublesome code.
Here is the original:
;;; An array is send via the file 'cl2o.dat'. The file is writtten
;;; using a raw call to unix fread with cmucl specific code. The file
;;; is read in octave using fread.
(defmethod set/octave (name (a array))
(start-octave)
(let* ((elt-type (type-of (row-major-aref a 0)))
(flat-a (make-array (array-total-size a)
:displaced-to a
:element-type elt-type))
(dims (array-dimensions a)))
...
Here is what I'm using now:
; making the dangerous assumption that all elements
; are of the same type as the first
(defun create-specialized-1d-array (arr)
(let ((nelms (array-total-size arr))
(arr-type (type-of arr))
(elt-type (type-of (row-major-aref arr 0))))
; type-of leaves out t for an vector of t, like (type-of #(1 2)),
; so we say if length of type is < 3 then the elements are of t
(if (or (< (length arr-type) 3)
(eq (second arr-type) t))
; arr's elms of t (general)
(make-array nelms
:element-type elt-type
:initial-contents (make-array nelms :displaced-
to arr))
; arr's elms of something else (specialized), like doulbe-
float
(make-array nelms
:element-type elt-type
:displaced-to arr))))
;;; An array is send via the file 'cl2o.dat'. The file is writtten
;;; using a raw call to unix fread with cmucl specific code. The file
;;; is read in octave using fread.
(defmethod set/octave (name (a array))
(start-octave)
(let* ((elt-type (type-of (row-major-aref a 0)))
(flat-a (create-specialized-1d-array a))
(dims (array-dimensions a)))
...
The function set/octave goes on to write the array to a file, from
which it is later read into octave. The sbcl case:
...
(sb-unix:unix-write (sb-sys:fd-stream-fd f)
(sb-sys:vector-sap (coerce flat-a `(simple-array ,elt-type
(*))))
0
(* lisp-nb-bytes (length flat-a)))
...
jra <·········@gmail.com> writes:
> My immediate response to your question was going to be that I wanted
> to specialize (vector 1 2 3) to (type-of (row-major-aref (vector 1 2
> 3) 0)). But then I tried it:
>
> CL-USER> (type-of (row-major-aref (vector 1 2 3) 0))
> BIT
> CL-USER> (type-of (row-major-aref (vector 1 2 3) 1))
> (INTEGER 0 536870911)
> CL-USER> (type-of (row-major-aref (vector 1 2 3) 2))
> (INTEGER 0 536870911)
>
> I wasn't expecting the type of 1 to come back as BIT! How would one go
> about determining the most specific subtype of all of the elements of
> a general array? Is this the right way?
>
> (defun best-type-of-elements (arr)
> (reduce (lambda (type1 type2)
> (cond
> ((subtypep type2 type1) type1)
> ((subtypep type1 type2) type2)
> (t t)))
> arr
> :key #'type-of))
I would think that a simpler and more useful method would be decide what
sorts of specialized arrays you want to make use of and then program
something specific to that set using TYPECASE. So, for example:
(defun my-element-type (x)
(typecase x
(FIXNUM 'FIXNUM)
(DOUBLE-FLOAT 'DOUBLE-FLOAT)
(SINGLE-FLOAT 'SINGLE-FLOAT)
(otherwise T)))
and just pass it the first element of the array. Of course, that does
assume homogeneous arrays. And it does eliminate the problem of having
to infer this based on what TYPE-OF returns. This is especially true
for subranges of integer, where the subrange can end up being to
specific for any changes to your new array. In other words, if you want
to be able to make assignments to the array elements, you don't
necessarily want a type that is too narrow.
Plus, you run the risk of having disjoint subranges of integer force you
to use T. For example, if you have the types
(INTEGER 0 100)
(INTEGER -100 0)
neither one is a subtype of the other, so you would have to fall back on
T, rather than a more useful FIXNUM.
--
Thomas A. Russ, USC/Information Sciences Institute
On Sun, 26 Oct 2008 11:15:40 -0700, jra wrote:
> On Oct 24, 1:02 pm, Tamas K Papp <······@gmail.com> wrote:
>> On Fri, 24 Oct 2008 12:03:35 -0700, jra wrote:
>> > What is the right way to create a new array (1D) from an existing
>> > array, where the new array will have the specialized type of the type
>> > of the existing array's elements (reguardless of the existing array's
>> > type: ie could be t but all elements are double-float, or could be
>> > double-float)?
>>
>> In case all elements of an array are subtype of some particular type
>> that you want to specialize to, you will have to recognize this
>> yourself and create the new array with an explicit element-type.
>>
>> Think about it: what should (vector 1 2 3) specialize to? Fixnum?
>> (unsigned-byte 2)? (signed-byte 17)? You get the idea.
>>
>> Tamas
>
> PreScript: I've tried to post this reply twice before and it never seems
> to make it into the discussion (I'm accessing the group using Google
> groups). Apologies if the ends up being posted multiple times. If it
> doesn't make it this time, I might just create a new discussion.
I am seeing it for the third time :-)
Use a decent news server.
Tamas
On Oct 24, 9:03 pm, jra <·········@gmail.com> wrote:
> What is the right way to create a new array (1D) from an existing
> array, where the new array will have the specialized type of the type
> of the existing array's elements (reguardless of the existing array's
> type: ie could be t but all elements are double-float, or could be
> double-float)?
It's pretty easy, actually. You could use the function upgraded-array-
element-type to find what the type of the array would be. Or you can
let make-array do it for you:
(defun make-specialized-vector (vector)
(let ((types (map 'list 'type-of vector)))
(make-array (length vector)
:element-type (cons 'or types)
:initial-contents vector)))
It's going to involve a significant amount of type calculation,
though, so don't expect it to be especially efficient.
On Sat, 25 Oct 2008 11:48:04 -0700, Thomas F. Burdick wrote:
> On Oct 24, 9:03 pm, jra <·········@gmail.com> wrote:
>> What is the right way to create a new array (1D) from an existing
>> array, where the new array will have the specialized type of the type
>> of the existing array's elements (reguardless of the existing array's
>> type: ie could be t but all elements are double-float, or could be
>> double-float)?
>
> It's pretty easy, actually. You could use the function upgraded-array-
> element-type to find what the type of the array would be. Or you can let
> make-array do it for you:
>
> (defun make-specialized-vector (vector)
> (let ((types (map 'list 'type-of vector)))
> (make-array (length vector)
> :element-type (cons 'or types)
> :initial-contents vector)))
>
> It's going to involve a significant amount of type calculation, though,
> so don't expect it to be especially efficient.
And here as a version with less storage:
(defun make-specialized-vector2 (vector)
(let (types)
(map nil (lambda (x)
(pushnew (type-of x) types :test #'equal))
vector)
(make-array (length vector)
:element-type (cons 'or types)
:initial-contents vector)))
Sadly, neither version does what Jason really wants. In SBCL:
(array-element-type (make-specialized-vector #(1 2 3)))
; => (UNSIGNED-BYTE 29)
So far, so good. Then
(array-element-type (make-specialized-vector (vector 0.1 0.2 pi)))
; => T
Hey, what's going on? Hint:
(upgraded-array-element-type '(or double-float single-float)) ; => T
When I started working with CL, I considered this weird, but now I think
I understand why: the result above is certainly correct, and so would
double-float be. It is not up to the implementation to decide. The
programmer has to make this choice.
So Jason's best bet is to specify the type manually.
HTH,
Tamas
On Sat, 25 Oct 2008 19:08:47 +0000, Tamas K Papp wrote:
> (upgraded-array-element-type '(or double-float single-float)) ; => T
>
> When I started working with CL, I considered this weird, but now I think
> I understand why: the result above is certainly correct, and so would
> double-float be. It is not up to the implementation to decide. The
Sorry, I meant float (the supertype of single-float and double-float),
not double-float.
Tamas
On Oct 24, 8:03 pm, jra <·········@gmail.com> wrote:
> What is the right way to create a new array (1D) from an existing
> array, where the new array will have the specialized type of the type
> of the existing array's elements (reguardless of the existing array's
> type: ie could be t but all elements are double-float, or could be
> double-float)?
A couple of assumptions: you want the most-specialized array type for
the given elements; your implementation has a reasonable
implementation of type-of and/or you're only interested in floating
point types; you're not concerned with making the conversion super-
efficient.
In addition to type-of, Common Lisp has a function upgraded-array-
element-type which, given a type, will tell you the most-specific
element-type you can use to hold objects of that type. You don't
actually need to call it yourself, though -- you can let make-array do
the heavy lifting for you.
(defun specialized-vector (vec)
(let ((types (map 'list 'type-of vec)))
(make-array (length vec)
:element-type (cons 'or types)
:initial-contents vec)))
That function is portable and should do what you want. However, type-
of is annoyingly missing a constraint in its definition: the types it
returns are reaquired to be subtypes of the object's class, but not
necessarily subtypes of the most specific array element-type for that
object. So a conforming implementation could give you a fixnum vector
rather than an (unsigned-byte 8) vector, or an unspecialized one
instead of an (unsigned-byte 32) one, for example.
Since the specialized types you'll be missing out on are almost
certainly of the (signed-byte x) and (unsigned-byte y) genre, you can
handle integer ranges yourself:
(defun specialized-vector (vec)
(labels ((%type-of (x)
(if (integerp x)
(list 'integer (min x 0) (max x 0))
(type-of x))))
(let ((types (map 'list #'%type-of vec)))
(make-array (length vec)
:element-type (cons 'or types)
:initial-contents vec))))
Still pretty simple, and now it Does The Right Thing portably.
On 26 oct, 14:03, "Thomas F. Burdick" <········@gmail.com> wrote:
> (defun specialized-vector (vec)
> (labels ((%type-of (x)
> (if (integerp x)
> (list 'integer (min x 0) (max x 0))
> (type-of x))))
> (let ((types (map 'list #'%type-of vec)))
> (make-array (length vec)
> :element-type (cons 'or types)
> :initial-contents vec))))
>
> Still pretty simple, and now it Does The Right Thing portably.
As Christophe Rhodes pointed out to me off-list, this would be a lot
easier using eql types:
(defun specialized-vector (vec)
(let ((types (loop for elt across vec collect `(eql ,elt))))
(make-array (length vec)
:element-type (cons 'or types)
:initial-contents vec)))
This won't magically do something your implementation doesn't support
(like giving you an array specialized on (or single-float double-
float). What it will give you is the most specific vector you can
have, given the contents.