···········@netscape.net (Werner) writes:
> I try a Lisp OS to develop , and need still a good Idea for a File
> system !
> Has somebody a good idea for a Lisp File system ?
I'd say that it rather depends on the other features of your system.
You'd not get the same kind of file system if your system is 100%
capability based of if it's distributed OS, or if it's a unix system.
You could also do entirely without a file system if you have
persistent storage of all the OS objects like in EROS
(http://www.eros-os.org): the RAM is then just considered as a big
cache of the whole disks memories, and each application could define
its own "file system objects". At the user interface, I'd prefer
something like unix file system (with perhaps adding versions).
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
Werner wrote:
> I try a Lisp OS to develop , and need still a good Idea for a File system !
> Has somebody a good idea for a Lisp File system ?
Hi!
POSIX-compliant?
What special forms do you think you will need beyond ANSI CL? What Lisp would
you use?
What processor platform? Simulated or real?
Oliver Korpilla
Oliver Korpilla schrieb:
> Werner wrote:
>> I try a Lisp OS to develop , and need still a good Idea for a File
>> system !
>> Has somebody a good idea for a Lisp File system ?
> POSIX-compliant?
Come on, POSIX-compliancy is not lisp style. It has to be something
better :)
So either faster or more other features - quota, snapshot, large file
support, ACL, versioning, journalling = undo in case of corruption, or a
real undo as in the old Lisp Machine, ... - on a single machine,
or take the distributed approach (AFS, Coda, ...).
Christopher Browne has a nice overview of the newer ones:
http://cbbrowne.com/info/fs.html
And maybe this short overview will help:
http://www.osdata.com/holistic/connect/filesys.htm
http://www.faqs.org/docs/Linux-HOWTO/Filesystems-HOWTO.html is quite ok.
ARLA as free AFS variant (distributed) looks fine.
Search for OS/MVS => http://www.osdata.com/oses/mvs.htm
and http://www.mvsbook.fsnet.co.uk/chap03a.htm#Section2
This might relate somehow to LispOS, and other esoteric problems
(getting data from punch cards, sorry magnetic tapes, reasonable fast :)
--
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/
Reini Urban wrote:
>> POSIX-compliant?
>
> Come on, POSIX-compliancy is not lisp style. It has to be something
> better :)
>
> So either faster or more other features - quota, snapshot, large file
> support, ACL, versioning, journalling = undo in case of corruption, or a
> real undo as in the old Lisp Machine, ... - on a single machine,
> or take the distributed approach (AFS, Coda, ...).
Uh, excuse me, instead of reimplementing POSIX you're arguing for reimplementing
Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
With kind regards,
Oliver
Oliver Korpilla <·······@fh-landshut.de> writes:
> Reini Urban wrote:
> >> POSIX-compliant?
> > Come on, POSIX-compliancy is not lisp style. It has to be something
> > better :)
> > So either faster or more other features - quota, snapshot, large
> > file support, ACL, versioning, journalling = undo in case of
> > corruption, or a real undo as in the old Lisp Machine, ... - on a
> > single machine,
> > or take the distributed approach (AFS, Coda, ...).
>
> Uh, excuse me, instead of reimplementing POSIX you're arguing for
> reimplementing Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
No. It would be better to implement a capability based OS like eros
(http://www.eros-os.org).
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
Pascal Bourguignon <····@mouse-potato.com> wrote:
> Oliver Korpilla <·······@fh-landshut.de> writes:
>>
>> Uh, excuse me, instead of reimplementing POSIX you're arguing for
>> reimplementing Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
>
> No. It would be better to implement a capability based OS like eros
> (http://www.eros-os.org).
As a Linux system administrator, I greatly agree.
Today's Linux (and BSD) systems do have vast security advantages over
older Unix systems. Network services such as Web and FTP servers are
not usually run as "root" these days. You have (if you care to use
them) fine-grained restrictions on network access, user privileges, and
system resources.
However, all these features have basically been kludged on to a design
which originally assumed that:
* Each process belongs to one user, and should be able to do
anything that user can do.
* Categories of privilege (groups) are declared by the root user
only. Users cannot create new ones: if James wants to let
Emily and Robert read a file, but not Erica, then James must
petition the admin to create a group.
* Users cannot express subsets of their own privilege level. I
cannot, e.g., start an "sbcl" process with the proviso that it
is not allowed to write to disk.
* Administrative tasks frequently cannot be delegated except by
handing over a setuid binary, which has the full power of root
available to it. There is no way to grant users access to a
program that does -only one- rootly thing (like, say, "ping"!)
without trusting that program with -full- root privilege. A
security bug in the ping program lets every user be root.
* A handful of bits are sufficient to express file permissions.
* Limitations on the use of system resources (ulimit) are
strictly optional.
* Even the administrator cannot control delegation of privilege
by other users -- there is no mandatory access control. There
is no way to prevent a user from setting his own files world
readable.
* Everyone's temporary files and IPC sockets should be dumped in
a world-writable /tmp directory, so if anyone can guess what
someone else will use for a filename there, they can do a
symlink exploit. If you can guess what /tmp file a root
process will create, you can trick root into wiping out the OS
kernel, the system logs, or the password file.
* Any process can talk to the network ... but in order to run a
"well-known service" (port < 1024) your process must run as
the superuser, with all the superuser's privilege. Running a
network service -not- as root takes extra work ("dropping
privilege") rather than being the default condition.
A declarative, capabilities-based system would be a vast improvement
over the current state of Unix/Linux security. One whose systems
programming language did not invite stupidities such as buffer overflow
would be even better.
And -- while I'm wishing -- if it comes with a Web server and a fast
relational database (not SQL, if you please!) then people might actually
use it. :)
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
"Karl A. Krueger" <········@example.edu> writes:
> A declarative, capabilities-based system would be a vast improvement
> over the current state of Unix/Linux security. One whose systems
> programming language did not invite stupidities such as buffer overflow
> would be even better.
And Lisp would be a great asset in a capability system. One usability
problem of capability systems is that capabilities are quite fine
grained, but the administrators and users want to define capability at
a higher level. You would need an expert system to compile a high
level capability policy into detailed setting.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
Pascal Bourguignon <····@mouse-potato.com> writes:
> [...]
> One usability
> problem of capability systems is that capabilities are quite fine
> grained, but the administrators and users want to define capability at
> a higher level. You would need an expert system to compile a high
> level capability policy into detailed setting.
I suppose that resembles the case of DEC developing an expert system
for configuring VAXes (I've only read about it, haven't seen it).
---Vassil.
--
Vassil Nikolov <········@poboxes.com>
Hollerith's Law of Docstrings: Everything can be summarized in 72 bytes.
Pascal Bourguignon <····@mouse-potato.com> writes:
> [...]
> One usability
> problem of capability systems is that capabilities are quite fine
> grained, but the administrators and users want to define capability at
> a higher level. You would need an expert system to compile a high
> level capability policy into detailed setting.
I suppose that resembles the case of DEC developing an expert system
for configuring VAXes (I've only read about it, haven't seen it).
---Vassil.
--
Vassil Nikolov <········@poboxes.com>
Hollerith's Law of Docstrings: Everything can be summarized in 72 bytes.
I agree with you on this matter, that current capability support is immature,
hard to administer, and breaks a lot of stuff.
I guess that's no wonder, since this stuff is modelled after an incomplete
standard - weren't the POSIX security extensions pulled out of lack of unanimity?
So, the current model sucks, alright. That's a lack in concept, and not in
implementation. The problem itself doesn't outright cry Lisp allover, doesn't it?
If it were easy to make a POSIX-compliant OS kernel matching Linux/FreeBSD in
capability and performance, more people would do. One person can design a
simple, well-designed kernel. But it takes a lot of people, efforts and brains
to make it "usable" (in comparison to mature alternatives).
With kind regards,
Oliver
Oliver Korpilla schrieb:
> Reini Urban wrote:
>>> POSIX-compliant?
>>
>>
>> Come on, POSIX-compliancy is not lisp style. It has to be something
>> better :)
>>
>> So either faster or more other features - quota, snapshot, large file
>> support, ACL, versioning, journalling = undo in case of corruption, or
>> a real undo as in the old Lisp Machine, ... - on a single machine,
>> or take the distributed approach (AFS, Coda, ...).
>
> Uh, excuse me, instead of reimplementing POSIX you're arguing for
> reimplementing Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
For sure not.
Linux FS e.g. cannot do snapshots and versioning.
Also it is not very good with distributed fs and distributed computing.
It is also not very good in dynamically avoiding problems. Very bad in fact.
Together with a proper sexpr syntax in the metadata, logs and protocol
it's a completely different beast than any linux fs.
--
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/
Reini Urban wrote:
> Oliver Korpilla schrieb:
>> Uh, excuse me, instead of reimplementing POSIX you're arguing for
>> reimplementing Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
>
>
> For sure not.
> Linux FS e.g. cannot do snapshots and versioning.
> Also it is not very good with distributed fs and distributed computing.
> It is also not very good in dynamically avoiding problems. Very bad in
> fact.
>
> Together with a proper sexpr syntax in the metadata, logs and protocol
> it's a completely different beast than any linux fs.
It's simple to deride the working solution. It's easy to imagine pipe dreams.
Where is the code to support your claims?
Linux may not be perfect, but it exists, works, improves and is free.
That's not to diss Lisp, I love it!
With kind regards,
Oliver Korpilla
>> Together with a proper sexpr syntax in the metadata, logs and protocol
>> it's a completely different beast than any linux fs.
> It's simple to deride the working solution. It's easy to imagine pipe
> dreams. Where is the code to support your claims?
>
> Linux may not be perfect, but it exists, works, improves and is free.
To comment on some of the preposters: the beauty of Linux (and there is
some interesting resemblance to Lisp :) is not that it is "perfect" as
it is, but it is, due to its unix nature, very modular. So there is no
such thing as a "Linux FS", but rater a list of filesystems shipped
with the kernel sources. But everyone is free to add to this list.
For this concrete case, there is a project to implement linux filesystems
in python, and one (extreme) case of its usage a Python module implementing
GoogleFS, that is a filesystem which performs its storage on your gmail
account. If you can write such weird cases in Python using the Linux
interfaces, then it should not require wizardry to do the same in Lisp,
adding any feature you want.
So in my eyes, the most pragmatic solution is, google for the Python
module, and write a port that allows generic filesystems be written
in Common Lisp.
Peter
--
pet project: http://dawn.netcologne.de
homepage: http://www.peter-herth.de
lisp stuff: http://www.peter-herth.de/lisp.html
get Ltk here: http://www.peter-herth.de/ltk/
Peter Herth <·····@netcologne.de> writes:
> >> Together with a proper sexpr syntax in the metadata, logs and protocol
> >> it's a completely different beast than any linux fs.
>
>
> > It's simple to deride the working solution. It's easy to imagine pipe
> > dreams. Where is the code to support your claims?
> >
> > Linux may not be perfect, but it exists, works, improves and is free.
>
> To comment on some of the preposters: the beauty of Linux (and there is
> some interesting resemblance to Lisp :) is not that it is "perfect" as
> it is, but it is, due to its unix nature, very modular. So there is no
> such thing as a "Linux FS", but rater a list of filesystems shipped
> with the kernel sources. But everyone is free to add to this list.
> For this concrete case, there is a project to implement linux filesystems
> in python, and one (extreme) case of its usage a Python module implementing
> GoogleFS, that is a filesystem which performs its storage on your gmail
> account. If you can write such weird cases in Python using the Linux
> interfaces, then it should not require wizardry to do the same in Lisp,
> adding any feature you want.
> So in my eyes, the most pragmatic solution is, google for the Python
> module, and write a port that allows generic filesystems be written
> in Common Lisp.
IIRC, there's a lisp in the kernel somewhere.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
Centuries ago, Nostradamus foresaw when Pascal Bourguignon <····@mouse-potato.com> would write:
> IIRC, there's a lisp in the kernel somewhere.
There can be... The module is called Schemix
<http://savannah.nongnu.org/projects/schemix/>
--
(reverse (concatenate 'string "gro.gultn" ·@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/oses.html
Why is it that when you're driving and looking for an address, you
turn down the volume on the radio?
Peter Herth <·····@netcologne.de> wrote:
>
>
>>> Together with a proper sexpr syntax in the metadata, logs and protocol
>>> it's a completely different beast than any linux fs.
>
>
>> It's simple to deride the working solution. It's easy to imagine pipe
>> dreams. Where is the code to support your claims?
>>
>> Linux may not be perfect, but it exists, works, improves and is free.
>
> To comment on some of the preposters: the beauty of Linux (and there is
> some interesting resemblance to Lisp :) is not that it is "perfect" as
> it is, but it is, due to its unix nature, very modular. So there is no
> such thing as a "Linux FS", but rater a list of filesystems shipped
> with the kernel sources. But everyone is free to add to this list.
And even though the list is _very_ long already, even if we only count
filesystems shipped with the standard kernel, people are still adding to
it, simply _because_ of the extensability of the Linux VFS framework.
> For this concrete case, there is a project to implement linux filesystems
> in python, and one (extreme) case of its usage a Python module implementing
> GoogleFS, that is a filesystem which performs its storage on your gmail
> account.
Not entirely correct. That would be GMailFS - the GoogleFS is something
_completely_ different (failure resilient distributed BLOB storage as
used by Google, see their paper on the topic).
> If you can write such weird cases in Python using the Linux
> interfaces, then it should not require wizardry to do the same in Lisp,
> adding any feature you want.
> So in my eyes, the most pragmatic solution is, google for the Python
> module, and write a port that allows generic filesystems be written
> in Common Lisp.
Now all we need is something stepping forward with a Linux Userland
Filesystem implementation written in Common Lisp ;-)
Regards,
Alex.
--
"Opportunity is missed by most people because it is dressed in overalls and
looks like work." -- Thomas A. Edison
Alexander Schreiber wrote:
> Not entirely correct. That would be GMailFS - the GoogleFS is something
> _completely_ different (failure resilient distributed BLOB storage as
> used by Google, see their paper on the topic).
Yes, thank you for the correction, I mixed the names up.
Peter
--
pet project: http://dawn.netcologne.de
homepage: http://www.peter-herth.de
lisp stuff: http://www.peter-herth.de/lisp.html
get Ltk here: http://www.peter-herth.de/ltk/
On Fri, 24 Sep 2004 18:47:51 +0200, Alexander Schreiber
<···@usenet.thangorodrim.de> wrote:
>Not entirely correct. That would be GMailFS - the GoogleFS is something
>_completely_ different (failure resilient distributed BLOB storage as
>used by Google, see their paper on the topic).
What paper? O:-)
Fernando <···@NOSPAMeasyjob.net> wrote:
> On Fri, 24 Sep 2004 18:47:51 +0200, Alexander Schreiber
><···@usenet.thangorodrim.de> wrote:
>
>
>>Not entirely correct. That would be GMailFS - the GoogleFS is something
>>_completely_ different (failure resilient distributed BLOB storage as
>>used by Google, see their paper on the topic).
>
> What paper? O:-)
http://www.cs.rochester.edu/sosp2003/papers/p125-ghemawat.pdf
Regards,
Alex.
--
"Opportunity is missed by most people because it is dressed in overalls and
looks like work." -- Thomas A. Edison
Oliver Korpilla <·······@fh-landshut.de> wrote:
> Reini Urban wrote:
>> Oliver Korpilla schrieb:
>>
>> Together with a proper sexpr syntax in the metadata, logs and protocol
>> it's a completely different beast than any linux fs.
>
> It's simple to deride the working solution. It's easy to imagine pipe dreams.
> Where is the code to support your claims?
>
> Linux may not be perfect, but it exists, works, improves and is free.
But Linux already exists. Linux is 100% successful at being Linux. If
I want Linux, I know where to find it.
If someone wants to build a new OS on new foundations (especially with
such a big difference built-in, as to be in Lisp and not in C) then they
would do well to aim at a different goal: not to be Linux, but to be
something (at least) as worthwhile as Linux but different from it.
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
After a long battle with technology, Oliver Korpilla <·······@fh-landshut.de>, an earthling, wrote:
> Reini Urban wrote:
>> Oliver Korpilla schrieb:
>
>>> Uh, excuse me, instead of reimplementing POSIX you're arguing for
>>> reimplementing Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
>> For sure not.
>> Linux FS e.g. cannot do snapshots and versioning.
>> Also it is not very good with distributed fs and distributed computing.
>> It is also not very good in dynamically avoiding problems. Very bad
>> in fact.
>> Together with a proper sexpr syntax in the metadata, logs and
>> protocol
>> it's a completely different beast than any linux fs.
>
> It's simple to deride the working solution. It's easy to imagine pipe
> dreams. Where is the code to support your claims?
>
> Linux may not be perfect, but it exists, works, improves and is free.
>
> That's not to diss Lisp, I love it!
The further challenge is that it takes rather a lot of work to write
device drivers for each new disk or video controller that comes along.
It so happens that there are already rather large communities working
to support such things for Linux and for (mapcar #'(lambda (x)
(string-concat x "BSD")) '("Free" "Open" "Net" "Dragonfly")).
Conspicuously, they have already worked through the learning curves so
that adding the _next_ SATA controller or the _next_ card from nVidia
is considerably easier than the last.
Most "little OS projects" get hung up because the authors only wrote
device drivers for some limited set of devices, and interest naturally
died when vendor conglomerations meant that no more S3-like video
cards or Symbios SCSI adaptors were getting released.
Plan 9 from Bell Labs languished for a long time because they decided
only to support Symbios SCSI hardware, and so people with Adaptec or
IDE hardware couldn't use it unless they were quite stunningly
committed. Adding in IDE support opened it back up, I believe, but
this is not an atypical sort of roadblock...
--
"cbbrowne",·@","ntlug.org"
http://linuxfinances.info/info/oses.html
Black holes are where God divided by zero.
Reini Urban <······@x-ray.at> wrote:
> Oliver Korpilla schrieb:
>> Reini Urban wrote:
>>>> POSIX-compliant?
>>>
>>>
>>> Come on, POSIX-compliancy is not lisp style. It has to be something
>>> better :)
>>>
>>> So either faster or more other features - quota, snapshot, large file
>>> support, ACL, versioning, journalling = undo in case of corruption, or
>>> a real undo as in the old Lisp Machine, ... - on a single machine,
>>> or take the distributed approach (AFS, Coda, ...).
>>
>> Uh, excuse me, instead of reimplementing POSIX you're arguing for
>> reimplementing Linux? Sounds not much lisp-ier to me! ;) Lisp-ux.
>
> For sure not.
> Linux FS e.g. cannot do snapshots and versioning.
Snapshots are implemented at the LVM (block device) layer. Versioning
filesystems seem to be in very short supply regardless of operating
system (not counting version control systems with a FS interface), the
most current implementation I know of is the VMS filesystem.
> Also it is not very good with distributed fs and distributed computing.
Distributed Filesystems exist: AFS, Lustre, GFS (the last two being
cluster filesystems) and for distributed computing you have things like
PVM, MPI and Mosix. And for both areas there are also a metric ton of
other solutions.
> It is also not very good in dynamically avoiding problems. Very bad in fact.
Hint: failure resilient computing is _hard_ and _expensive_. Look at all
the effort the Tandem Himalaya HA systems went to supply a disaster
tolerant (as in: blowing up part of the system) computing environment.
> Together with a proper sexpr syntax in the metadata, logs and protocol
> it's a completely different beast than any linux fs.
Things like network and disk bandwith (especially the later) are neither
free nor unlimited. Using s-expressions for filesystem data, either on
disk or over the wire seriously increases your overhead for moving data
around. Doing this in a performance sensitive subsystem like the
filesystem is not going to help you in any way. And its not like Lisp
were unable to deal with binary data structures, it is perfectly capable
of doing so.
Regards,
Alex.
--
"Opportunity is missed by most people because it is dressed in overalls and
looks like work." -- Thomas A. Edison
Alexander Schreiber <···@usenet.thangorodrim.de> writes:
>> It is also not very good in dynamically avoiding problems. Very bad in fact.
>
> Hint: failure resilient computing is _hard_ and _expensive_. Look at all
> the effort the Tandem Himalaya HA systems went to supply a disaster
> tolerant (as in: blowing up part of the system) computing environment.
Any links? I only found some UNIX compatibility stuff.
Regards,
--
____________________________
Julian Stecklina / _________________________/
________________/ /
\_________________/ LISP - truly beautiful
Julian Stecklina <··········@web.de> wrote:
> Alexander Schreiber <···@usenet.thangorodrim.de> writes:
>
>>> It is also not very good in dynamically avoiding problems. Very bad in fact.
>>
>> Hint: failure resilient computing is _hard_ and _expensive_. Look at all
>> the effort the Tandem Himalaya HA systems went to supply a disaster
>> tolerant (as in: blowing up part of the system) computing environment.
>
> Any links? I only found some UNIX compatibility stuff.
The systems family is called Nonstop Himalaya, made by Tandem. Using, of
course, specially developed hardware and their own operating system.
http://www.stanford.edu/~candea/teaching/lectures/redundancy/plain/index.htm
http://www.fact-index.com/t/ta/tandem_computers.html
http://www.verizonit.com/pdf/himalaya_s_series.pdf
http://www.stanford.edu/~jzhuo/soco/how-tandem.html
http://www.hpl.hp.com/techreports/tandem/
which contains a lot of papers from Tandem
An interesting bit of statistics is that Tandem claimed 42% of the
actual failures of their Nonstop systems being due to operator error.
If you want to keep on digging:
http://www.google.com/search?q=Tandem+Himalaya+Nonstop
Regards,
Alex.
--
"Opportunity is missed by most people because it is dressed in overalls and
looks like work." -- Thomas A. Edison
Alexander Schreiber <···@usenet.thangorodrim.de> writes:
> The systems family is called Nonstop Himalaya, made by Tandem. Using, of
> course, specially developed hardware and their own operating system.
[Lots of interesting links]
Thanks very much!
Regards,
--
____________________________
Julian Stecklina / _________________________/
________________/ /
\_________________/ LISP - truly beautiful
···········@netscape.net (Werner) writes:
> Thank you for the hints .
> A completely untrodden path to go , is not advisable .
> I will begin with a Unix Kernel and the File System ext2fs .
Trodding the unix path with lisp is like driving a F1 into a mountain track.
--
__Pascal Bourguignon__
Pascal Bourguignon wrote:
> ···········@netscape.net (Werner) writes:
>
>> Thank you for the hints .
>> A completely untrodden path to go , is not advisable .
>> I will begin with a Unix Kernel and the File System ext2fs .
>
> Trodding the unix path with lisp is like driving a F1 into a mountain
> track.
lol
Especially if there's a nice highspeed-highway built around the mountain
that seems unused since quite a couple of years.
-ts